Clemens/searxng
1
0
Fork 0
mirror of https://github.com/searxng/searxng.git synced 2026-06-11 20:37:51 +02:00
Code Issues Packages Projects Releases Wiki Activity
9,484 Commits 4 Branches 0 Tags
a1490676e3c04239b3a314cb1fbb3144bc894b14
Commit Graph

9484 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Bnyro a1490676e3 [mod] fireball: small fixup from code review (#6240) 
Co-authored-by: Markus Heiser <markus.heiser@darmarIT.de>
 2026-06-11 12:09:57 +02:00
Bnyro 3a382cb3f3 [chore] helix config: enable pyling and use black via pylsp 2026-06-11 11:03:38 +02:00
Ivan Gabaldon 9d9d605b15 [fix] ci: use install buildhost script (#6105) 2026-06-11 08:23:37 +02:00
Bnyro de03f4eb11 [feat] engines: add fireball engine (general, news, videos) 2026-06-10 21:00:49 +02:00
Markus Heiser 00f7c68a6f [chore] drop emacs' obsolete .dir-locals template (#6236) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2026-06-10 17:38:19 +02:00
Bnyro 41c98b3b41 [chore] devops: add languages config for helix editor 
The default Helix configuration for Python is different,
so the pylint warnings aren't shown and the formatter
re-formats files by accident when you edit an existing file.

Therefore, this commit adds `python` language configuration
to ease developing SearXNG with Helix Editor [^1].

[^1]: https://helix-editor.com
 2026-06-10 17:38:01 +02:00
Bnyro f4c63c8eb0 [feat] engines: add duckduckgo web engine as alternative to html.duckduckgo.com 
html.duckduckgo.com captchas all my IPs very fast. I figured out that using
duckduckgo.com works even if html.duckduckgo.com is captcha-ed, hence adding
support for duckduckgo.com's general web search here.

This implementation fetches the link to the first API page
(i.e. ``links.duckduckgo.com/d.js?...``) from duckduckgo.com and uses the ``n``
parameter of the API to fetch all subsequent pages.

This also means that it's not possible to immediately search for the third
page - the first and the second page would need to be loaded first.

The reason why we can't just normally use the `vqd` value is that the API URLs
require an additional parameter `dp` which seems generated at server-side, so we
can't build it ourselves and must scrape it from the HTML pages.
 2026-06-10 16:49:56 +02:00
Markus Heiser 26801e92af [fix] sqlitedb: create DB Schema (DDL) during app initialization (hardening) (#6187) 
The initialization of the DB schema ("base schema") has so far been done on
demand, which causes race conditions with competing threads and processes.

The DDL statements for creating the "base schema" are now executed as part of
the initialization of the app.

Further improvements were made to harden the database applications:

- Wikidata & Radio-Browser engine perform their initialization only once (so far
  the initialization was carried out in each thread/process).

- If multiple processes try to set DB's WAL mode when opening the DB at the same
  time, this usually leads to another race condition, which is now also caught.

Related:

- https://github.com/searxng/searxng/issues/6181#issuecomment-4586705

Closes: #6181

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2026-06-10 15:48:49 +02:00
Bnyro f3fab143be [feat] engines: add tiger.ch engine 
Add support for https://tiger.ch (general, news)

It is disabled and inactive by default because it's just a metasearch engine
like SearXNG is, so it's mostly useful for bypassing rate-limits on other
engines: (it has its own German index, but it's not that great) in theory it
supports different locales, but I was too lazy to implement that (I only need
German and English results anyways, which are returned by default...)
 2026-06-08 13:35:13 +02:00
Bnyro 72a827ae93 [fix] yep: send Sec-Fetch headers to bypass "access denied" (#6223) 
Avoids yep's botblocking by sending Sec-Fetch-* headers (as the browser does).
 2026-06-08 10:55:17 +02:00
Bnyro 6ca9d3784c [feat] engines: add seek-ninja general engine (#6217) 
Add support for https://seek.ninja (general)

It's very slow because the engine uses Server-side events, that incrementally
send data in their HTTP response [1].

I.e. we wait for the end of the response (7+ seconds), even though the results
data arrives within a few seconds -> it's very slow, because SearXNG wants to
get the full response body before it calls the `response(resp)` method

We could use httpx-sse [2], but I'm not sure how to integrate this into SearXNG
and if it's worth it

[1] https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/
[2] https://github.com/florimondmanca/httpx-sse
 2026-06-08 07:09:06 +02:00
Bnyro 63f264220b [feat] engines: add heexy engine (general, images) (#6218) 2026-06-08 05:54:35 +02:00
Austin-Olacsi 41fcf0be4b [fix] aol engine uses wikidata id for C++ (#6221) 2026-06-08 05:32:26 +02:00
Bnyro 86903a2c66 [fix] flaticon: crash if result tag has no name (#6219) 2026-06-07 14:16:44 +02:00
Markus Heiser 70de3cc561 Revert "[fix] no such table during engine init (#6185)" (#6215) 
This reverts commit 9d49a9f344.
 2026-06-07 09:23:35 +02:00
Bnyro 51b6fd4f23 [del] karmasearch: remove engine (cloudflared) (#6213) 
The engine is using very aggressive Cloudflare blocking for
a while now, no matter if using a normal browser like Firefox
or not.

Closes: https://github.com/searxng/searxng/issues/5976
 2026-06-07 06:49:09 +02:00
Brock Vojkovic 9d49a9f344 [fix] no such table during engine init (#6185) 2026-06-07 06:04:12 +02:00
Bnyro e260a732c8 [fix] online engine processor: accept language headers doesn't get sent for 'all' language 2026-06-06 18:24:16 +02:00
Markus Heiser 0429198415 [mod] swisscows WEB: ignore video results from the first page 
On the first page of the WEB search, there are, among other things, sections for
videos and news.  The video results from these sections should not be used as
results in the WEB search of SearXNG.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2026-06-06 18:04:19 +02:00
Markus Heiser e7cf57e9ae [mod] swisscows engines: add language / region support 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2026-06-06 18:04:19 +02:00
Bnyro ed369ac0ec [feat] engines: add support for swisscows general 2026-06-06 18:04:19 +02:00
Bnyro 94bdbb5c63 [feat] engines: add support for swisscows videos 2026-06-06 18:04:19 +02:00
Bnyro 465b5229c6 [feat] engines: add swisscows news engine 2026-06-06 18:04:19 +02:00
Bnyro cbf97fd262 [feat] engines: add swisscows images engine 
The implementation is basically a 1:1 port of the reverse engineered
swisscows JavaScript code. (it's been obfuscated, so I've restructured it
and made the variable names idiomatic instead of obfuscated var names like "a", "o", "i")

```js
/*
e: "/v5/images/search"
t: {
	itemsCount: "50"
	locale: "de-DE"
	offset: "50"
	query: "test"
	spellcheck: "true"
}
*/
// HASH library used: https://github.com/h2non/jshashes
function generateNonceAndSignature(queryParams, urlPath) {
  // urlPath = "/v5/images/search"
  // sort keys alphabetically and join to query string
  let queryStringSorted = '?' + U().stringify(queryParams, {
    arrayFormat: 'repeat',
    allowDots: !0
  }).split('&').map(e => {
    let[key, value] = e.split('=');
    return [key, decodeURIComponent(value)]
  }).sort((e, t) => e[0].localeCompare(t[0])).map(e => e.join('=')).join('&');

  function caesarShift(str, offset = 13) {
      const alphabet = 'abcdefghijklmnopqrstuvwxyz';
      let result = [];
      for (let a = 0; a < str.length; a++) {
        let c = str[a],
        alphabetIndex = alphabet.indexOf(c.toLowerCase());
        if ( - 1 !== alphabetIndex) {
          alphabetIndex += offset;
          while (alphabetIndex >= alphabet.length) alphabetIndex -= alphabet.length;
          c = c === c.toUpperCase() ? alphabet[alphabetIndex] : alphabet[alphabetIndex].toUpperCase()
        }
        result.push(c)
      }
      return result.join('')
    }
  const r = new (sha256Instance()).SHA256;
  const random = randomString(32);
  const randomShifted = caesarShift(random);
  let to_hash = [urlPath, queryStringSorted, randomShifted].join('');
  let signature = r.b64(to_hash);
  signature = signature.replace(/=/g, '').replace(/\+/g, '-').replace(/\//g, '_');
  return {
    nonce: random,
    signature: signature
  }
}

function randomString(length) {
  let t = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~',
  n = '';
  for (let r = 0; r < length; r++) n += t.charAt(Math.floor(Math.random() * t.length));
  return n
}
```
 2026-06-06 18:04:19 +02:00
dependabot[bot] 37187dc2d8 [upd] web-client (simple): Bump the minor group across 1 directory with 5 updates (#6169) 
Bumps the minor group with 5 updates in the /client/simple directory:

| Package | From | To |
| --- | --- | --- |
| [@biomejs/biome](https://github.com/biomejs/biome/tree/HEAD/packages/@biomejs/biome) | `2.4.15` | `2.4.16` |
| [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) | `25.8.0` | `25.9.1` |
| [edge.js](https://github.com/edge-js/edge) | `6.5.0` | `6.5.1` |
| [stylelint](https://github.com/stylelint/stylelint) | `17.11.1` | `17.12.0` |
| [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) | `8.0.13` | `8.0.16` |

Updates `@biomejs/biome` from 2.4.15 to 2.4.16
- [Release notes](https://github.com/biomejs/biome/releases)
- [Changelog](https://github.com/biomejs/biome/blob/main/packages/@biomejs/biome/CHANGELOG.md)
- [Commits](https://github.com/biomejs/biome/commits/@biomejs/biome@2.4.16/packages/@biomejs/biome)

Updates `@types/node` from 25.8.0 to 25.9.1
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

Updates `edge.js` from 6.5.0 to 6.5.1
- [Release notes](https://github.com/edge-js/edge/releases)
- [Changelog](https://github.com/edge-js/edge/blob/6.x/CHANGELOG.md)
- [Commits](https://github.com/edge-js/edge/compare/v6.5.0...v6.5.1)

Updates `stylelint` from 17.11.1 to 17.12.0
- [Release notes](https://github.com/stylelint/stylelint/releases)
- [Changelog](https://github.com/stylelint/stylelint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/stylelint/stylelint/compare/17.11.1...17.12.0)

Updates `vite` from 8.0.13 to 8.0.16
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.0.16/packages/vite)

---
updated-dependencies:
- dependency-name: "@biomejs/biome"
  dependency-version: 2.4.16
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
- dependency-name: "@types/node"
  dependency-version: 25.9.1
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: edge.js
  dependency-version: 6.5.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
- dependency-name: stylelint
  dependency-version: 17.12.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: vite
  dependency-version: 8.0.14
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-05 16:26:27 +02:00
dependabot[bot] 2f049cb037 [upd] github-actions: Bump actions/checkout from 6.0.2 to 6.0.3 (#6204) 
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.2 to 6.0.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/de0fac2e4500dabe0009e67214ff5f5447ce83dd...df4cb1c069e1874edd31b4311f1884172cec0e10)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 6.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-05 16:17:01 +02:00
dependabot[bot] eb39bc0dc1 [upd] github-actions: Bump github/codeql-action from 4.36.0 to 4.36.2 (#6203) 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.36.0 to 4.36.2.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/7211b7c8077ea37d8641b6271f6a365a22a5fbfa...8aad20d150bbac5944a9f9d289da16a4b0d87c1e)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.36.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-05 16:16:35 +02:00
dependabot[bot] 007a4e2155 [upd] pypi: Bump typer from 0.26.3 to 0.26.7 in the minor group (#6205) 
Bumps the minor group with 1 update: [typer](https://github.com/fastapi/typer).


Updates `typer` from 0.26.3 to 0.26.7
- [Release notes](https://github.com/fastapi/typer/releases)
- [Changelog](https://github.com/fastapi/typer/blob/master/docs/release-notes.md)
- [Commits](https://github.com/fastapi/typer/compare/0.26.3...0.26.7)
 2026-06-05 11:54:28 +02:00
github-actions[bot] 13ce187e64 [l10n] update translations from Weblate (#6206) 
19b2047a9 - 2026-05-30 - daemul72 <daemul72@noreply.codeberg.org>
 2026-06-05 11:52:35 +02:00
Bnyro 26fa181b84 [feat] gmx: detect captchas 2026-06-05 08:07:30 +02:00
Bnyro 0f35ef7cd6 [feat] json engine: add option to not send page num on first page 2026-06-05 08:04:49 +02:00
Bnyro b1ae576b2d [fix] xpath engine: add missing send_page_num_on_first_page docstring 2026-06-05 08:04:49 +02:00
Bnyro e6559c9ad6 [fix] gabanza: result URLs are invalid 2026-06-04 08:55:19 +02:00
Bnyro 5bae05514b [feat] engines: add zapmeta general search engine 2026-06-03 22:38:59 +02:00
Bnyro 00ca5776f2 [feat] engines: add gabanza general engine 2026-06-03 22:38:23 +02:00
Bnyro 577f5f2f30 [fix] online engines: send_accept_language_header is sent even if disabled 2026-06-03 22:37:13 +02:00
Bnyro 253dc86c10 [fix] duckduckgo: image requests get blocked 2026-06-03 22:37:13 +02:00
Bnyro 3066bc19eb [fix] public domain image archive: fails to extract API url 2026-06-03 22:35:21 +02:00
Austin-Olacsi e964708c00 [fix] bilibili engine: fix Referer and add Accept HTTP header (#6189) 2026-06-02 06:06:31 +02:00
Bnyro 7159b8aed3 [feat] marginalia: add support for pagination 2026-05-31 12:54:53 +02:00
Bnyro 246f5a5499 [mod] svgrepo: remove engine 
- SVGRepo uses Cloudflare for every session, no matter
if you're opening it in a browser or not
 2026-05-31 12:54:32 +02:00
vojkovic 300695de5c [fix] crash when lock is omitted 2026-05-31 01:37:37 +08:00
Markus Heiser bd863f16b1 [build] /static 2026-05-30 22:43:50 +08:00
Markus Heiser 4ac822fd7f [mod] typification of the preference settings 
no functional change / except the missing online doc which is now available::

    $ make docs.live
    $ xdg-open "http://127.0.0.1:8000/admin/settings/settings_preferences.html"

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2026-05-30 22:43:50 +08:00
vojkovic e1d25c5078 [mod] enable image proxy for public instances 2026-05-30 22:43:50 +08:00
dependabot[bot] 01159b82fe [upd] pypi: Bump the minor group with 3 updates (#6164) 
Bumps the minor group with 3 updates: [granian](https://github.com/emmett-framework/granian), [basedpyright](https://github.com/detachhead/basedpyright) and [typer](https://github.com/fastapi/typer).


Updates `granian` from 2.7.4 to 2.7.5
- [Release notes](https://github.com/emmett-framework/granian/releases)
- [Commits](https://github.com/emmett-framework/granian/compare/v2.7.4...v2.7.5)

Updates `basedpyright` from 1.39.5 to 1.39.6
- [Release notes](https://github.com/detachhead/basedpyright/releases)
- [Commits](https://github.com/detachhead/basedpyright/compare/v1.39.5...v1.39.6)

Updates `typer` from 0.25.1 to 0.26.3
- [Release notes](https://github.com/fastapi/typer/releases)
- [Changelog](https://github.com/fastapi/typer/blob/master/docs/release-notes.md)
- [Commits](https://github.com/fastapi/typer/compare/0.25.1...0.26.3)
 2026-05-30 11:33:10 +02:00
Bnyro 780ee32564 [fix] pexels: fix engine crashes with SearxEngineAccessDeniedException 2026-05-29 22:03:22 +02:00
github-actions[bot] 217c9a1597 [l10n] update translations from Weblate (#6170) 
207f98ecc - 2026-05-26 - mustafa-phd <mustafa-phd@noreply.codeberg.org>
3b51fbca7 - 2026-05-25 - Amirkhandrend-Nicest-XII <amirkhandrend-nicest-xii@noreply.codeberg.org>

Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
 2026-05-29 14:47:43 +02:00
dependabot[bot] 70e810bd7b [upd] github-actions: Bump docker/setup-qemu-action from 4.0.0 to 4.1.0 (#6166) 
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/ce360397dd3f832beb865e1373c09c0e9f86d70a...06116385d9baf250c9f4dcb4858b16962ea869c3)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: 4.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-29 14:42:48 +02:00
dependabot[bot] baab1c160a [upd] github-actions: Bump github/codeql-action from 4.35.5 to 4.36.0 (#6167) 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.35.5 to 4.36.0.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/9e0d7b8d25671d64c341c19c0152d693099fb5ba...7211b7c8077ea37d8641b6271f6a365a22a5fbfa)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.36.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-29 14:42:28 +02:00