Skip to content
Back to top
View this page
Edit this page

# Google Engines

## Google API

SearXNG’s implementation of the Google API is mainly done in `get_google_info`.

For detailed description of the REST-full API see: Query Parameter Definitions. The linked API documentation can sometimes be helpful during reverse engineering. However, we cannot use it in the freely accessible WEB services; not all parameters can be applied and some engines are more special than other (e.g. Google News).

## Google WEB

This is the implementation of the Google WEB engine. Some of this implementations (manly the `get_google_info`) are shared by other engines:

- Google Images

- Google News

- Google Videos

- Google Scholar

- Google Autocomplete

searx.engines.google.max_page = 50

  Google max 50 pages

searx.engines.google.ui_async(start: int ) → str

  Format of the response from UI’s async request.

  - `arc_id:<...>,use_ac:true,_fmt:prog`

  The arc_id is random generated every hour.

searx.engines.google.get_google_info(params: OnlineParams, eng_traits: EngineTraits ) → dict [ str , Any ]

  Composing various (language) properties for the google engines (Google API).

  This function is called by the various google engines (Google WEB, Google Images, Google News and Google Videos).

  Parameters:

    - param (dict) – Request parameters of the engine. At least a `searxng_locale` key should be in the dictionary.

    - eng_traits – Engine’s traits fetched from google preferences (`searx.enginelib.traits.EngineTraits`)

  Return type:

    dict

  Returns:

    Py-Dictionary with the key/value pairs:

    language:

      The language code that is used by google (e.g. `lang_en` or `lang_zh-TW`)

    country:

      The country code that is used by google (e.g. `US` or `TW`)

    locale:

      A instance of `babel.core.Locale` build from the `searxng_locale` value.

    subdomain:

      Google subdomain `google_domains` that fits to the country code.

    params:

      Py-Dictionary with additional request arguments (can be passed to `urllib.parse.urlencode()`).

      - `hl` parameter: specifies the interface language of user interface.

      - `lr` parameter: restricts search results to documents written in a particular language.

      - `cr` parameter: restricts search results to documents originating in a particular country.

      - `ie` parameter: sets the character encoding scheme that should be used to interpret the query string (‘utf8’).

      - `oe` parameter: sets the character encoding scheme that should be used to decode the XML result (‘utf8’).

    headers:

      Py-Dictionary with additional HTTP headers (can be passed to request’s headers)

      - `Accept: '*/*`

searx.engines.google.detect_google_sorry(resp: SXNG_Response )

  Detect Google’s bot-protection responses (CAPTCHA / sorry pages).

  Google may block requests in several ways:

  1. Redirect to sorry.google.com (standard CAPTCHA).

  2. HTTP 302 redirect to `/sorry/index?...` on the same host – when the HTTP client doesn’t follow the redirect, the response body is a short HTML stub with a link to the sorry page.

  3. Short HTML response (<2000 bytes) containing “/sorry/” – a meta-refresh or JS redirect variant.

searx.engines.google.request(query: str, params: OnlineParams ) → None

  Google search request

searx.engines.google.response(resp: SXNG_Response )

  Get response from google’s search request

searx.engines.google.fetch_traits(engine_traits: EngineTraits, add_domains: bool = True)

  Fetch languages from Google.

## Google Autocomplete

searx.autocomplete.google_complete(query: str, sxng_locale: str ) → list [ str ]

  Autocomplete from Google. Supports Google’s languages and subdomains (`searx.engines.google.get_google_info`) by using the async REST API:

      https://{subdomain}/complete/search?{args}

## Google Images

This is the implementation of the Google Images engine using the internal Google API used by the Google Go Android app.

This internal API offer results in

- JSON (`_fmt:json`)

- Protobuf (`_fmt:pb`)

- Protobuf compressed? (`_fmt:pc`)

- HTML (`_fmt:html`)

- Protobuf encoded in JSON (`_fmt:jspb`).

searx.engines.google_images.max_page = 50

  Google max 50 pages

searx.engines.google_images.request(query, params)

  Google-Image search request

searx.engines.google_images.response(resp)

  Get response from google’s search request

## Google Videos

This is the implementation of the Google Videos engine.

Content-Security-Policy (CSP)

This engine needs to allow images from the data URLs (prefixed with the `data:` scheme):

    Header set Content-Security-Policy "img-src 'self' data: ;"

searx.engines.google_videos.request(query, params)

  Google-Video search request

searx.engines.google_videos.response(resp)

  Get response from google’s search request

## Google News

This is the implementation of the Google News engine.

Google News has a different region handling compared to Google WEB.

- the `ceid` argument has to be set (`ceid_list`)

- the hl argument has to be set correctly (and different to Google WEB)

- the gl argument is mandatory

If one of this argument is not set correctly, the request is redirected to CONSENT dialog:

    https://consent.google.com/m?continue=

The google news API ignores some parameters from the common Google API:

- num : the number of search results is ignored / there is no paging all results for a query term are in the first response.

- save : is ignored / Google-News results are always SafeSearch

searx.engines.google_news.request(query: str, params: OnlineParams ) → None

  Google-News search request

searx.engines.google_news.response(resp: SXNG_Response ) → EngineResults

  Get response from google’s search request

searx.engines.google_news.ceid_list = ['AE:ar', 'AR:es-419', 'AT:de', 'AU:en', 'BD:bn', 'BE:fr', 'BE:nl', 'BG:bg', 'BR:pt-419', 'BW:en', 'CA:en', 'CA:fr', 'CH:de', 'CH:fr', 'CL:es-419', 'CN:zh-Hans', 'CO:es-419', 'CU:es-419', 'CZ:cs', 'DE:de', 'EE:et', 'EG:ar', 'ES:ca', 'ES:es', 'ET:en', 'FI:fi', 'FR:fr', 'GB:en', 'GH:en', 'GR:el', 'HK:zh-Hant', 'HU:hu', 'ID:en', 'ID:id', 'IE:en', 'IL:en', 'IL:he', 'IN:bn', 'IN:en', 'IN:gu', 'IN:hi', 'IN:ml', 'IN:mr', 'IN:pa', 'IN:ta', 'IN:te', 'IT:it', 'JP:ja', 'KE:en', 'KR:ko', 'LB:ar', 'LT:lt', 'LV:en', 'LV:lv', 'MA:fr', 'MY:en', 'MY:ms', 'NA:en', 'NG:en', 'NL:nl', 'NO:no', 'NZ:en', 'PH:en', 'PK:en', 'PL:pl', 'RO:ro', 'RS:sr', 'RU:ru', 'SA:ar', 'SE:sv', 'SG:en', 'SI:sl', 'SK:sk', 'SN:fr', 'TH:th', 'TR:tr', 'TZ:en', 'UA:ru', 'UA:uk', 'UG:en', 'US:en', 'VN:vi', 'ZA:en', 'ZW:en']

  List of region/language combinations supported by Google News. Values of the `ceid` argument of the Google News REST API.

## Google Scholar

Google Scholar is a freely accessible web search engine that indexes the full text or metadata of scholarly literature across an array of publishing formats and disciplines.

Compared to other Google services the Scholar engine has a simple GET REST-API and there does not exists `async` API. Even though the API slightly vintage we can make use of the Google API to assemble the arguments of the GET request.

### Configuration

    - name: google scholar
      engine: google_scholar
      shortcut: gos

### Implementations

searx.engines.google_scholar.max_page = 50

  Google max 50 pages

searx.engines.google_scholar.request(query: str, params: OnlineParams ) → None

  Google-Scholar search request

searx.engines.google_scholar.response(resp: SXNG_Response ) → EngineResults

  Parse response from Google Scholar

searx.engines.google_scholar.time_range_args(params: OnlineParams ) → dict [ str , int ]

  Returns a dictionary with a time range arguments based on `params["time_range"]`.

  Google Scholar supports a detailed search by year. Searching by last month or last week (as offered by SearXNG) is uncommon for scientific publications and is not supported by Google Scholar.

  To limit the result list when the users selects a range, all the SearXNG ranges (day, week, month, year) are mapped to year. If no range is set an empty dictionary of arguments is returned.

  Example; when user selects a time range and we find ourselves in the year 2025 (current year minus one):

      { "as_ylo" : 2024 }

searx.engines.google_scholar.detect_google_captcha(dom: ElementBase | _Element )

  In case of CAPTCHA Google Scholar open its own not a Robot dialog and is not redirected to `sorry.google.com`.

searx.engines.google_scholar.parse_gs_a(text: str | None ) → tuple [ list [ str ], str , str , datetime | None ]

  Parse the text written in green.

  Possible formats: * “{authors} - {journal}, {year} - {publisher}” * “{authors} - {year} - {publisher}” * “{authors} - {publisher}”