Add support for https://seek.ninja (general)
It's very slow because the engine uses Server-side events, that incrementally
send data in their HTTP response [1].
I.e. we wait for the end of the response (7+ seconds), even though the results
data arrives within a few seconds -> it's very slow, because SearXNG wants to
get the full response body before it calls the `response(resp)` method
We could use httpx-sse [2], but I'm not sure how to integrate this into SearXNG
and if it's worth it
[1] https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/
[2] https://github.com/florimondmanca/httpx-sse
The engine is using very aggressive Cloudflare blocking for
a while now, no matter if using a normal browser like Firefox
or not.
Closes: https://github.com/searxng/searxng/issues/5976
PodcastIndex.org started using a Proof-of-Work JavaScript
challenge whose results are sent as `X-Pow-*` request headers.
Although it is technically possible to re-implement the
PoW challenge in Python, it's likely impossible to maintain
because
- the actual Proof of Concept logic might change very often
- the whole idea of the Proof of Work challenge is to use
a "big" amount of resources (about 1s on my PC); so executing the challenge
would almost block all other work on the SearXNG instance
At first glance, the challenge looks very similar to what
Anubis does, because it also uses SHA256 hashes.
- https://lite.qwant.com seems to be dead.
- The request parameters were changed to match the ones from the Qwant website.
- Qwant is now set to inactive by default due to its strict rate-limits
[mod] normalize variable name for the max number of results per request
In the past, we have used different names for the variable that specifies the
maximum number of hits in the outgoing request.
- ``page_size``
- ``number_of_results``
- ``nb_per_page``
Since *page_size* is the most accurate term and is also used in the XPath
engines, all other engines are adjusted accordingly within this
patch .. documentation adjusted accordingly.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Notes:
- Safesearch doesn't seem to work properly?
- In theory multiple languages are supported, but even in the web UI, they don't work properly
- Possibly, we could cache the request hashes (h query parameter), I'm not sure if it ever changes
The AOL engines deliver no results or too many incorrect results; in #5972 it is
reported that the AOL-images do not work, and on AOL-web the language selection
seems to depend more on the IP than on the selected language / Locally, the
language selection works for me, on the public server only English results are
displayed, which significantly worsens the result list.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
URLs, name and colors are automatically rendered into manifest.json.
Furthermore user preference of theme (light, dark, black) and theme colors are
respected. Theme colors can be set in settings.yml
Submitting an issue on GitHub isn't a end user task .. most issue reports coming
from this form are not filled out, since end users are not familiar with the
needs of a developer community.
Closes: https://github.com/searxng/searxng/issues/5820
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Since about a month, the website just says "temporarily unavailable", so it's safe to assume that it's just no longer working
Related:
- https://github.com/searxng/searxng/pull/3798
The default settings for the suspend times were previously 24 hours and 3 hours,
respectively. Based on my experience, these defaults are too high; most engines
handle suspend times of 3 minutes or 1 hour (captcha) without any problems.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Bnyro