Browse Source

[doc] move dosc of botdetection from developer to admin section

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Markus Heiser 1 year ago
parent
commit
597c68b4aa
3 changed files with 22 additions and 0 deletions
  1. 1 0
      docs/admin/index.rst
  2. 4 0
      docs/admin/searx.botdetection.rst
  3. 17 0
      searx/botdetection/__init__.py

+ 1 - 0
docs/admin/index.rst

@@ -15,6 +15,7 @@ Administrator documentation
    installation-apache
    update-searxng
    answer-captcha
+   searx.botdetection
    api
    architecture
    plugins

+ 4 - 0
docs/src/searx.botdetection.rst → docs/admin/searx.botdetection.rst

@@ -19,6 +19,8 @@ Bot Detection
   :members:
 
 
+.. _botdetection rate limit:
+
 Rate limit
 ==========
 
@@ -29,6 +31,8 @@ Rate limit
   :members:
 
 
+.. _botdetection probe headers:
+
 Probe HTTP headers
 ==================
 

+ 17 - 0
searx/botdetection/__init__.py

@@ -2,6 +2,23 @@
 # lint: pylint
 """.. _botdetection src:
 
+The :ref:`limiter <limiter src>` implements several methods to block bots:
+
+a. Analysis of the HTTP header in the request / can be easily bypassed.
+
+b. Block and pass lists in which IPs are listed / difficult to maintain, since
+   the IPs of bots are not all known and change over the time.
+
+c. Detection of bots based on the behavior of the requests and blocking and, if
+   necessary, unblocking of the IPs via a dynamically changeable IP block list.
+
+For dynamically changeable IP lists a Redis database is needed and for any kind
+of IP list the determination of the IP of the client is essential.  The IP of
+the client is determined via the X-Forwarded-For_ HTTP header
+
+.. _X-Forwarded-For:
+   https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For
+
 X-Forwarded-For
 ===============