Skip to content
Back to top
View this page
Edit this page

# Favicons (source)

Implementations for providing the favicons in SearXNG.

There is a command line for developer purposes and for deeper analysis. Here is an example in which the command line is called in the development environment:

    $ ./manage pyenv.cmd bash --norc --noprofile
    (py3) python -m searx.favicons --help

searx.favicons.favicon_url(authority: str ) → str

  Function to generate the image URL used for favicons in SearXNG’s result lists. The `authority` argument (aka netloc / RFC 3986) is usually a (sub-) domain name. This function is used in the HTML (jinja) templates.

      <div class="favicon">
         <img src="{{ favicon_url(result.parsed_url.netloc) }}">
      </div>

  The returned URL is a route to `favicon_proxy` REST API.

  If the favicon is already in the cache, the returned URL is a data URL (something like `data:image/png;base64,...`). By generating a data url from the `cache.FaviconCache`, additional HTTP roundtripps via the `favicon_proxy` are saved. However, it must also be borne in mind that data urls are not cached in the client (web browser).

searx.favicons.favicon_proxy()

  REST API of SearXNG’s favicon proxy service

      /favicon_proxy?authority=<...>&h=<...>

  `authority`:

    Domain name RFC 3986 / see `favicon_url`

  `h`:

    HMAC RFC 2104, build up from the server.secret_key setting.

## Favicons Config

searx.favicons.config.CONFIG_SCHEMA: int = 1

  Version of the configuration schema.

searx.favicons.config.TOML_CACHE_CFG: dict [ str , FaviconConfig ] = {}

  Cache config objects by TOML’s filename.

class searx.favicons.config.FaviconConfig(cfg_schema: int, cache: FaviconCacheConfig = <factory>, proxy: FaviconProxyConfig = <factory>)

  The class aggregates configurations of the favicon tools

  cfg_schema: int

    Config’s schema version. The specification of the version of the schema is mandatory, currently only version `CONFIG_SCHEMA` is supported. By specifying a version, it is possible to ensure downward compatibility in the event of future changes to the configuration schema

  cache: FaviconCacheConfig

    Setup of the `cache.FaviconCacheConfig`.

  proxy: FaviconProxyConfig

    Setup of the `proxy.FaviconProxyConfig`.

  classmethod from_toml_file(cfg_file: Path, use_cache: bool ) → FaviconConfig

    Create a config object from a TOML file, the `use_cache` argument specifies whether a cache should be used.

## Favicons Proxy

Implementations for a favicon proxy

class searx.favicons.proxy.FaviconProxyConfig(max_age: int = 604800, secret_key: str = '', resolver_timeout: int = 3.0, resolver_map: dict[str, str]=<factory>, favicon_path: str = '/home/runner/work/searxng/searxng/searx/static/themes/{theme}/img/empty_favicon.svg', favicon_mime_type: str = 'image/svg+xml')

  Configuration of the favicon proxy.

  max_age: int

    HTTP header Cache-Control `max-age`

  secret_key: str

    By default, the value from server.secret_key setting is used.

  resolver_timeout: int

    Timeout which the resolvers should not exceed, is usually passed to the outgoing request of the resolver. By default, the value from outgoing.request_timeout setting is used.

  resolver_map: dict [ str , str ]

    The resolver_map is a key / value dictionary where the key is the name of the resolver and the value is the fully qualifying name (fqn) of resolver’s function (the callable). The resolvers from the python module `searx.favicons.resolver` are available by default.

  get_resolver(name: str ) → Callable | None

    Returns the callable object (function) of the resolver with the `name`. If no resolver is registered for the `name`, `None` is returned.

  favicon(**replacements)

    Returns pathname and mimetype of the default favicon.

  favicon_data_url(**replacements)

    Returns data image URL of the default favicon.

searx.favicons.proxy.favicon_proxy()

  REST API of SearXNG’s favicon proxy service

      /favicon_proxy?authority=<...>&h=<...>

  `authority`:

    Domain name RFC 3986 / see `favicon_url`

  `h`:

    HMAC RFC 2104, build up from the server.secret_key setting.

searx.favicons.proxy.search_favicon(resolver: str, authority: str ) → tuple [ None | bytes , None | str ]

  Sends the request to the favicon resolver and returns a tuple for the favicon. The tuple consists of `(data, mime)`, if the resolver has not determined a favicon, both values are `None`.

  `data`:

    Binary data of the favicon.

  `mime`:

    Mime type of the favicon.

searx.favicons.proxy.favicon_url(authority: str ) → str

  Function to generate the image URL used for favicons in SearXNG’s result lists. The `authority` argument (aka netloc / RFC 3986) is usually a (sub-) domain name. This function is used in the HTML (jinja) templates.

      <div class="favicon">
         <img src="{{ favicon_url(result.parsed_url.netloc) }}">
      </div>

  The returned URL is a route to `favicon_proxy` REST API.

  If the favicon is already in the cache, the returned URL is a data URL (something like `data:image/png;base64,...`). By generating a data url from the `cache.FaviconCache`, additional HTTP roundtripps via the `favicon_proxy` are saved. However, it must also be borne in mind that data urls are not cached in the client (web browser).

## Favicons Resolver

Implementations of the favicon resolvers that are available in the favicon proxy by default. A resolver is a function that obtains the favicon from an external source. The resolver function receives two arguments (`domain, timeout`) and returns a tuple `(data, mime)`.

searx.favicons.resolvers.allesedv(domain: str, timeout: int ) → tuple [ None | bytes , None | str ]

  Favicon Resolver from allesedv.com / https://favicon.allesedv.com/

searx.favicons.resolvers.duckduckgo(domain: str, timeout: int ) → tuple [ None | bytes , None | str ]

  Favicon Resolver from duckduckgo.com / https://blog.jim-nielsen.com/2021/displaying-favicons-for-any-domain/

searx.favicons.resolvers.google(domain: str, timeout: int ) → tuple [ None | bytes , None | str ]

  Favicon Resolver from google.com

searx.favicons.resolvers.yandex(domain: str, timeout: int ) → tuple [ None | bytes , None | str ]

  Favicon Resolver from yandex.com

## Favicons Cache

Implementations for caching favicons.

`FaviconCacheConfig`:

  Configuration of the favicon cache

`FaviconCache`:

  Abstract base class for the implementation of a favicon cache.

`FaviconCacheSQLite`:

  Favicon cache that manages the favicon BLOBs in a SQLite DB.

`FaviconCacheNull`:

  Fallback solution if the configured cache cannot be used for system reasons.

searx.favicons.cache.state()

  show state of the cache

searx.favicons.cache.maintenance(force: bool = True, debug: bool = False)

  perform maintenance of the cache

searx.favicons.cache.init(cfg: FaviconCacheConfig )

  Initialization of a global `CACHE`

final class searx.favicons.cache.FaviconCacheConfig(db_type: Literal ['sqlite', 'mem'] = 'sqlite', db_url: str = '/tmp/faviconcache.db', HOLD_TIME: int = 2592000, LIMIT_TOTAL_BYTES: int = 52428800, BLOB_MAX_BYTES: int = 20480, MAINTENANCE_PERIOD: int = 3600, MAINTENANCE_MODE: Literal ['auto', 'off'] = 'auto')

  Configuration of the favicon cache.

  db_type: Literal ['sqlite', 'mem']

    Type of the database:

    `sqlite`:

      `cache.FaviconCacheSQLite`

    `mem`:

      `cache.FaviconCacheMEM` (not recommended)

  db_url: str

    URL of the SQLite DB, the path to the database file.

  HOLD_TIME: int

    Hold time (default in sec.), after which a BLOB is removed from the cache.

  LIMIT_TOTAL_BYTES: int

    Maximum of bytes (default) stored in the cache of all blobs. Note: The limit is only reached at each maintenance interval after which the oldest BLOBs are deleted; the limit is exceeded during the maintenance period. If the maintenance period is too long or maintenance is switched off completely, the cache grows uncontrollably.

  BLOB_MAX_BYTES: int

    The maximum BLOB size in bytes that a favicon may have so that it can be saved in the cache. If the favicon is larger, it is not saved in the cache and must be requested by the client via the proxy.

  MAINTENANCE_PERIOD: int

    Maintenance period in seconds / when `MAINTENANCE_MODE` is set to `auto`.

  MAINTENANCE_MODE: Literal ['auto', 'off']

    Type of maintenance mode

    `auto`:

      Maintenance is carried out automatically as part of the maintenance intervals (`MAINTENANCE_PERIOD`); no external process is required.

    `off`:

      Maintenance is switched off and must be carried out by an external process if required.

class searx.favicons.cache.FaviconCacheStats(favicons: int | None = None, bytes: int | None = None, domains: int | None = None, resolvers: int | None = None, field_descr: tuple[tuple[str, str, ~typing.Callable[[int, int], str] | type], ...] = (('favicons', 'number of favicons in cache', <function humanize_number>), ('bytes', 'total size (approx. bytes) of cache', <function humanize_bytes>), ('domains', 'total number of domains in cache', <function humanize_number>), ('resolvers', 'number of resolvers', <class 'str'>)))

  Dataclass which provides information on the status of the cache.

class searx.favicons.cache.FaviconCache(cfg: FaviconCacheConfig )

  Abstract base class for the implementation of a favicon cache.

  abstractmethod set(resolver: str, authority: str, mime: str | None, data: bytes | None ) → bool

    Set data and mime-type in the cache. If data is None, the `FALLBACK_ICON` is registered. in the cache.

  abstractmethod state() → FaviconCacheStats

    Returns a `FaviconCacheStats` (key/values) with information on the state of the cache.

  abstractmethod maintenance(force: bool = False)

    Performs maintenance on the cache

final class searx.favicons.cache.FaviconCacheNull(cfg: FaviconCacheConfig )

  A dummy favicon cache that caches nothing / a fallback solution. The NullCache is used when more efficient caches such as the `FaviconCacheSQLite` cannot be used because, for example, the SQLite library is only available in an old version and does not meet the requirements.

  set(resolver: str, authority: str, mime: str | None, data: bytes | None ) → bool

    Set data and mime-type in the cache. If data is None, the `FALLBACK_ICON` is registered. in the cache.

  state()

    Returns a `FaviconCacheStats` (key/values) with information on the state of the cache.

  maintenance(force: bool = False)

    Performs maintenance on the cache

final class searx.favicons.cache.FaviconCacheSQLite(cfg: FaviconCacheConfig )

  Favicon cache that manages the favicon BLOBs in a SQLite DB. The DB model in the SQLite DB is implemented using the abstract class `sqlitedb.SQLiteAppl`.

  For introspection of the DB, jump into developer environment and run command to show cache state:

      $ ./manage pyenv.cmd bash --norc --noprofile
      (py3) python -m searx.favicons cache state

  The following configurations are required / supported:

  - `FaviconCacheConfig.db_url`

  - `FaviconCacheConfig.HOLD_TIME`

  - `FaviconCacheConfig.LIMIT_TOTAL_BYTES`

  - `FaviconCacheConfig.BLOB_MAX_BYTES`

  - `MAINTENANCE_PERIOD`

  - `MAINTENANCE_MODE`

  DB_SCHEMA: int = 1

    As soon as changes are made to the DB schema, the version number must be increased. Changes to the version number require the DB to be recreated (or migrated / if an migration path exists and is implemented).

  DDL_BLOBS = 'CREATE TABLE IF NOT EXISTS blobs (\n sha256 TEXT,\n bytes_c INTEGER,\n mime TEXT NOT NULL,\n data BLOB NOT NULL,\n PRIMARY KEY (sha256))'

    Table to store BLOB objects by their sha256 hash values.

  DDL_BLOB_MAP = "CREATE TABLE IF NOT EXISTS blob_map (\n m_time INTEGER DEFAULT (strftime('%s', 'now')), -- last modified (unix epoch) time in sec.\n sha256 TEXT,\n resolver TEXT,\n authority TEXT,\n PRIMARY KEY (resolver, authority))"

    Table to map from (resolver, authority) to sha256 hash values.

  SQL_DROP_LEFTOVER_BLOBS = 'DELETE FROM blobs WHERE sha256 IN ( SELECT b.sha256 FROM blobs b LEFT JOIN blob_map bm ON b.sha256 = bm.sha256 WHERE bm.sha256 IS NULL)'

    Delete blobs.sha256 (BLOBs) no longer in blob_map.sha256.

  set(resolver: str, authority: str, mime: str | None, data: bytes | None ) → bool

    Set data and mime-type in the cache. If data is None, the `FALLBACK_ICON` is registered. in the cache.

  property next_maintenance_time: int

    Returns (unix epoch) time of the next maintenance.

  maintenance(force: bool = False)

    Performs maintenance on the cache

  state() → FaviconCacheStats

    Returns a `FaviconCacheStats` (key/values) with information on the state of the cache.

final class searx.favicons.cache.FaviconCacheMEM(cfg: FaviconCacheConfig )

  Favicon cache in process’ memory. Its just a POC that stores the favicons in the memory of the process.

  Attention

  Don’t use it in production, it will blow up your memory!!

  set(resolver: str, authority: str, mime: str | None, data: bytes | None ) → bool

    Set data and mime-type in the cache. If data is None, the `FALLBACK_ICON` is registered. in the cache.

  state()

    Returns a `FaviconCacheStats` (key/values) with information on the state of the cache.

  maintenance(force: bool = False)

    Performs maintenance on the cache