Browse Source

[fix] engine archlinux: avoid Anubis challenge by User-Agent "SearXNG" (#4779)

Of the archlinux wikis only wiki.archlinux.org has a has Anubis challenge.

About Anubis[1]:

> Anubis decides to present a challenge using this logic:
>
> - User-Agent contains "Mozilla"
> ...
> This should ensure that git clients, RSS readers, and other low-harm clients
> can get through without issue ..

[1] https://github.com/TecharoHQ/anubis/blob/6c0ff3f4d5e7a66ea85e5873e6ee4c28692091d7/docs/docs/design/how-anubis-works.mdx#challenge-presentation


Suggested-by: @unixfox https://github.com/searxng/searxng/issues/4646#issuecomment-2855322406
Closes: https://github.com/searxng/searxng/issues/4646

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Markus Heiser 3 days ago
parent
commit
9006866019
1 changed files with 3 additions and 0 deletions
  1. 3 0
      searx/engines/archlinux.py

+ 3 - 0
searx/engines/archlinux.py

@@ -51,6 +51,9 @@ def request(query, params):
     if netloc == main_wiki:
         eng_lang: str = traits.get_language(sxng_lang, 'English')  # type: ignore
         query += ' (' + eng_lang + ')'
+        # wiki.archlinux.org is protected by anubis
+        # - https://github.com/searxng/searxng/issues/4646#issuecomment-2817848019
+        params['headers']['User-Agent'] = "SearXNG"
     elif netloc == 'wiki.archlinuxcn.org':
         base_url = 'https://' + netloc + '/wzh/index.php?'