Add support for https://luxxle.com
Localization is not yet supported because it doesn't seem to work on their
website either, no matter which language I select, it only returns English web
results
RawWeb is a search engine for personal websites / blog posts.
It has its own index and the personal websites were selected
by hand. Results are quite good for what it is imo. [^1]
[^1]: https://github.com/0x2E/RawWeb.org
- adds support for https://reloado.com (german)
- as it has its own index, the results are hit or miss and mostly German,
but still worth integrating imo
Add support for https://startsiden.abcnyheter.no, a netherlandish search engine
that probably uses Google or Bing? idk it also returns English results, but
e.g. ``test`` returns mostly results from netherlands.
S1Search provides various different search services, which all seem
to be somewhat based on Google and Yahoo. The site looks kinda suspicious,
but the results are fine.
You can find a list of their engines by using a subdomain finder like
https://web-toolbox.dev/en/tools/subdomain-lookup and search for `s1search.co`.
Chatnoir is an open source search engine developed by universities, based on
CommonCrawl (and others). It's uncommented by default - we don't want to
overload the universities with bot traffic that targets SearXNG (sad truth why
we can't have nice things anymore)
* [mod] template images.html: reformatted for readability (no func change)
In preparation for upcoming changes, the template is being reformatted for
better readability; no functional changes are being made.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* [mod] image results: add list of alternative formats
To test alternatives formats apply patch from below, query ``!flaticon bmw`` and
open the detail view for the image.
diff --git a/searx/engines/flaticon.py b/searx/engines/flaticon.py
index 06b6a8e25..d88388705 100644
--- a/searx/engines/flaticon.py
+++ b/searx/engines/flaticon.py
@@ -8,7 +8,7 @@ from urllib.parse import urlencode
import typing as t
-from searx.result_types import EngineResults
+from searx.result_types import EngineResults, ImageRef
if t.TYPE_CHECKING:
from searx.extended_types import SXNG_Response
@@ -61,6 +61,14 @@ def response(resp: "SXNG_Response"):
thumbnail_src=_fix_url(result["png"]),
img_src=_fix_url(result["png512"]),
author=result["team_name"],
+ formats=[
+ ImageRef(label="PNG 100x100", url="https://example.org/test.png", subtype="png"),
+ ImageRef(label="SVG", url="https://example.org/test.svg", subtype="svg+xml"),
+ ImageRef(url="https://example.org/test.jpg", subtype="jpeg"),
+ ImageRef(url="https://example.org/test.bmp", subtype="bmp"),
+ ImageRef(url="https://example.org/test.ico", subtype="x-icon"),
+ ImageRef(url="https://example.org/test.tif", subtype="tiff"),
+ ],
)
)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
---------
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
e.g. when searching for "!tiger pottering github", it crashes.
not really sure why - the problem is that the HTML doesn't
really uses descriptive classes or ids, only Tailwind,
so it's very hard to select only the results HTML.
The default Helix configuration for Python is different,
so the pylint warnings aren't shown and the formatter
re-formats files by accident when you edit an existing file.
Therefore, this commit adds `python` language configuration
to ease developing SearXNG with Helix Editor [^1].
[^1]: https://helix-editor.com
html.duckduckgo.com captchas all my IPs very fast. I figured out that using
duckduckgo.com works even if html.duckduckgo.com is captcha-ed, hence adding
support for duckduckgo.com's general web search here.
This implementation fetches the link to the first API page
(i.e. ``links.duckduckgo.com/d.js?...``) from duckduckgo.com and uses the ``n``
parameter of the API to fetch all subsequent pages.
This also means that it's not possible to immediately search for the third
page - the first and the second page would need to be loaded first.
The reason why we can't just normally use the `vqd` value is that the API URLs
require an additional parameter `dp` which seems generated at server-side, so we
can't build it ourselves and must scrape it from the HTML pages.
The initialization of the DB schema ("base schema") has so far been done on
demand, which causes race conditions with competing threads and processes.
The DDL statements for creating the "base schema" are now executed as part of
the initialization of the app.
Further improvements were made to harden the database applications:
- Wikidata & Radio-Browser engine perform their initialization only once (so far
the initialization was carried out in each thread/process).
- If multiple processes try to set DB's WAL mode when opening the DB at the same
time, this usually leads to another race condition, which is now also caught.
Related:
- https://github.com/searxng/searxng/issues/6181#issuecomment-4586705Closes: #6181
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Add support for https://tiger.ch (general, news)
It is disabled and inactive by default because it's just a metasearch engine
like SearXNG is, so it's mostly useful for bypassing rate-limits on other
engines: (it has its own German index, but it's not that great) in theory it
supports different locales, but I was too lazy to implement that (I only need
German and English results anyways, which are returned by default...)
Add support for https://seek.ninja (general)
It's very slow because the engine uses Server-side events, that incrementally
send data in their HTTP response [1].
I.e. we wait for the end of the response (7+ seconds), even though the results
data arrives within a few seconds -> it's very slow, because SearXNG wants to
get the full response body before it calls the `response(resp)` method
We could use httpx-sse [2], but I'm not sure how to integrate this into SearXNG
and if it's worth it
[1] https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/
[2] https://github.com/florimondmanca/httpx-sse
The engine is using very aggressive Cloudflare blocking for
a while now, no matter if using a normal browser like Firefox
or not.
Closes: https://github.com/searxng/searxng/issues/5976
On the first page of the WEB search, there are, among other things, sections for
videos and news. The video results from these sections should not be used as
results in the WEB search of SearXNG.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Bnyro
