Skip to content
Back to top
View this page
Edit this page

# Caches

Implementation of caching solutions.

- `searx.cache.ExpireCache` and its `searx.cache.ExpireCacheCfg`

class searx.cache.ExpireCacheCfg(name: str, db_url: str = '', MAX_VALUE_LEN: int = 10240, MAXHOLD_TIME: int = 604800, MAINTENANCE_PERIOD: int = 3600, MAINTENANCE_MODE: Literal['auto', 'off'] = 'auto', password: bytes = b'ultrasecretkey')

  Configuration of a `ExpireCache` cache.

  name: str

    Name of the cache.

  db_url: str

    URL of the SQLite DB, the path to the database file. If unset a default DB will be created in /tmp/sxng_cache_{self.name}.db

  MAX_VALUE_LEN: int

    Max length of a serialized value.

  MAXHOLD_TIME: int

    Hold time (default in sec.), after which a value is removed from the cache.

  MAINTENANCE_PERIOD: int

    Maintenance period in seconds / when `MAINTENANCE_MODE` is set to `auto`.

  MAINTENANCE_MODE: Literal['auto', 'off']

    Type of maintenance mode

    `auto`:

      Maintenance is carried out automatically as part of the maintenance intervals (`MAINTENANCE_PERIOD`); no external process is required.

    `off`:

      Maintenance is switched off and must be carried out by an external process if required.

  password: bytes

    Password used by `ExpireCache.secret_hash`.

    The default password is taken from secret_key. When the password is changed, the hashed keys in the cache can no longer be used, which is why all values in the cache are deleted when the password is changed.

class searx.cache.ExpireCacheStats(cached_items: dict[str, list[tuple[str, Any, int | None]]])

  Dataclass which provides information on the status of the cache.

  cached_items: dict[str, list[tuple[str, Any, int | None]]]

    Values in the cache mapped by context name.

class searx.cache.ExpireCache

  Abstract base class for the implementation of a key/value cache with expire date.

  abstractmethod set(key: str, value: Any, expire: int | None, ctx: str | None = None) → bool

    Set key to value. To set a timeout on key use argument `expire` (in sec.). If expire is unset the default is taken from `ExpireCacheCfg.MAXHOLD_TIME`. After the timeout has expired, the key will automatically be deleted.

    The `ctx` argument specifies the context of the `key`. A key is only unique in its context.

    The concrete implementations of this abstraction determine how the context is mapped in the connected database. In SQL databases, for example, the context is a DB table or in a Key/Value DB it could be a prefix for the key.

    If the context is not specified (the default is `None`) then a default context should be used, e.g. a default table for SQL databases or a default prefix in a Key/Value DB.

  abstractmethod get(key: str, default: Any = None, ctx: str | None = None) → Any

    Return value of key. If key is unset, `None` is returned.

  abstractmethod maintenance(force: bool = False, truncate: bool = False) → bool

    Performs maintenance on the cache.

    `force`:

      Maintenance should be carried out even if the maintenance interval has not yet been reached.

    `truncate`:

      Truncate the entire cache, which is necessary, for example, if the password has changed.

  abstractmethod state() → ExpireCacheStats

    Returns a `ExpireCacheStats`, which provides information about the status of the cache.

  static build_cache(cfg: ExpireCacheCfg) → ExpireCacheSQLite

    Factory to build a caching instance.

    Note

    Currently, only the SQLite adapter is available, but other database types could be implemented in the future, e.g. a Valkey (Redis) adapter.

  static normalize_name(name: str) → str

    Returns a normalized name that can be used as a file name or as a SQL table name (is used, for example, to normalize the context name).

  secret_hash(name: str | bytes) → str

    Creates a hash of the argument `name`. The hash value is formed from the `name` combined with the `password`. Can be used, for example, to make the `key` stored in the DB unreadable for third parties.

class searx.cache.ExpireCacheSQLite(cfg: ExpireCacheCfg)

  Cache that manages key/value pairs in a SQLite DB. The DB model in the SQLite DB is implemented in abstract class `SQLiteAppl`.

  The following configurations are required / supported:

  - `ExpireCacheCfg.db_url`

  - `ExpireCacheCfg.MAXHOLD_TIME`

  - `ExpireCacheCfg.MAINTENANCE_PERIOD`

  - `ExpireCacheCfg.MAINTENANCE_MODE`

  DB_SCHEMA: int = 1

    As soon as changes are made to the DB schema, the version number must be increased. Changes to the version number require the DB to be recreated (or migrated / if an migration path exists and is implemented).

  init(conn: Connection) → bool

    Initializes the DB schema and properties, is only executed once even if called several times.

    If the initialization has not yet taken place, it is carried out and a True is returned to the caller at the end. If the initialization has already been carried out in the past, False is returned.

  maintenance(force: bool = False, truncate: bool = False) → bool

    Performs maintenance on the cache.

    `force`:

      Maintenance should be carried out even if the maintenance interval has not yet been reached.

    `truncate`:

      Truncate the entire cache, which is necessary, for example, if the password has changed.

  create_table(table: str) → bool

    Create DB `table` if it has not yet been created, no recreates are initiated if the table already exists.

  property table_names: list[str]

    List of key/value tables already created in the DB.

  property next_maintenance_time: int

    Returns (unix epoch) time of the next maintenance.

  set(key: str, value: Any, expire: int | None, ctx: str | None = None) → bool

    Set key/value in DB table given by argument `ctx`. If expire is unset the default is taken from `ExpireCacheCfg.MAXHOLD_TIME`. If `ctx` argument is `None` (the default), a table name is generated from the `ExpireCacheCfg.name`. If DB table does not exists, it will be created (on demand) by `self.create_table`.

  setmany(opt_list: list[tuple[str, Any, int | None]], ctx: str | None = None) → int

    Efficient bootload of the cache from a list of options. The list contains tuples with the arguments described in `ExpireCacheSQLite.set`.

  get(key: str, default: Any = None, ctx: str | None = None) → Any

    Get value of `key` from table given by argument `ctx`. If `ctx` argument is `None` (the default), a table name is generated from the `ExpireCacheCfg.name`. If `key` not exists in the table or the table not exists, the `default` value is returned.

  pairs(ctx: str) → Iterator[tuple[str, Any]]

    Iterate over key/value pairs from table given by argument `ctx`. If `ctx` argument is `None` (the default), a table name is generated from the `ExpireCacheCfg.name`.

  state() → ExpireCacheStats

    Returns a `ExpireCacheStats`, which provides information about the status of the cache.