Browse Source

[doc] update & fix documentation of the "SearXNG LXC suite"

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Markus Heiser 1 year ago
parent
commit
c9833ded9f
6 changed files with 371 additions and 216 deletions
  1. 187 152
      docs/dev/lxcdev.rst
  2. 2 0
      docs/utils/index.rst
  3. 155 56
      docs/utils/lxc.sh.rst
  4. 10 4
      docs/utils/searxng.sh.rst
  5. 5 4
      utils/lxc-searxng.env
  6. 12 0
      utils/searxng.sh

+ 187 - 152
docs/dev/lxcdev.rst

@@ -16,6 +16,12 @@ In this article we will show, how you can make use of Linux Containers (LXC_) in
    should have a serious meaning about the terms: *distributed*, *merge* and
    should have a serious meaning about the terms: *distributed*, *merge* and
    *linux container*.
    *linux container*.
 
 
+   **hint**
+
+   If you have issues with the internet connectivity of your containers read
+   section :ref:`internet connectivity docker`.
+
+
 .. contents:: Contents
 .. contents:: Contents
    :depth: 2
    :depth: 2
    :local:
    :local:
@@ -25,38 +31,38 @@ In this article we will show, how you can make use of Linux Containers (LXC_) in
 Motivation
 Motivation
 ==========
 ==========
 
 
-Usually in our development cycle, we edit the sources and run some test and/or
-builds by using ``make`` :ref:`[ref] <makefile>` before we commit.  This cycle
-is simple and perfect but might fail in some aspects we should not overlook.
+Most often in our development cycle, we edit the sources and run some test
+and/or builds by using ``make`` :ref:`[ref] <makefile>` before we commit.  This
+cycle is simple and perfect but might fail in some aspects we should not
+overlook.
 
 
   **The environment in which we run all our development processes matters!**
   **The environment in which we run all our development processes matters!**
 
 
 The :ref:`makefile` and the :ref:`make install` encapsulate a lot for us, but
 The :ref:`makefile` and the :ref:`make install` encapsulate a lot for us, but
-they do not have access to all prerequisites.  For example, there may have
-dependencies on packages that are installed on the developer's desktop, but
+these tools do not have access to all prerequisites.  For example, there may
+have dependencies on packages that are installed on developer's desktop, but
 usually are not preinstalled on a server or client system.  Another example is;
 usually are not preinstalled on a server or client system.  Another example is;
 settings have been made to the software on developer's desktop that would never
 settings have been made to the software on developer's desktop that would never
 be set on a *production* system.
 be set on a *production* system.
 
 
-  **Linux Containers are isolate environments and not to mix up all the
-  prerequisites from various projects on developer's desktop is always a good
-  choice.**
+  **Linux Containers are isolate environments**, we use them to not mix up all
+  the prerequisites from various projects on developer's desktop.
 
 
 The scripts from :ref:`searx_utils` can divide in those to install and maintain
 The scripts from :ref:`searx_utils` can divide in those to install and maintain
-software:
+software
 
 
 - :ref:`searxng.sh`
 - :ref:`searxng.sh`
 
 
-and the script :ref:`lxc.sh`, with we can scale our installation, maintenance or
-even development tasks over a stack of isolated containers / what we call the:
+and the script
 
 
-  **SearXNG LXC suite**
+- :ref:`lxc.sh`
 
 
-.. hint::
+with we can scale our installation, maintenance or even development tasks over a
+stack of isolated containers / what we call the:
 
 
-   If you see any problems with the internet connectivity of your
-   containers read section :ref:`internet connectivity docker`.
+- :ref:`searxng lxc suite`
 
 
+.. _lxcdev install searxng:
 
 
 Gentlemen, start your engines!
 Gentlemen, start your engines!
 ==============================
 ==============================
@@ -69,7 +75,7 @@ once:
 
 
 .. tabs::
 .. tabs::
 
 
-  .. group-tab:: desktop
+  .. group-tab:: desktop (HOST)
 
 
      .. code:: bash
      .. code:: bash
 
 
@@ -81,7 +87,7 @@ fork:
 
 
 .. tabs::
 .. tabs::
 
 
-  .. group-tab:: desktop
+  .. group-tab:: desktop (HOST)
 
 
      .. code:: bash
      .. code:: bash
 
 
@@ -89,76 +95,61 @@ fork:
         $ git clone https://github.com/searxng/searxng.git searxng
         $ git clone https://github.com/searxng/searxng.git searxng
         $ cd searxng
         $ cd searxng
 
 
-The :ref:`lxc-searxng.env` consists of several images, see ``export
-LXC_SUITE=(...`` near by :origin:`utils/lxc-searxng.env#L19`.  For this blog post
-we exercise on a archlinux_ image.  The container of this image is named
-``searxng-archlinux``.  Lets build the container, but be sure that this container
-does not already exists, so first lets remove possible old one:
-
-.. tabs::
-
-  .. group-tab:: desktop
-
-     .. code:: bash
-
-        $ sudo -H ./utils/lxc.sh remove searxng-archlinux
-        $ sudo -H ./utils/lxc.sh build searxng-archlinux
-
 .. sidebar:: The ``searxng-archlinux`` container
 .. sidebar:: The ``searxng-archlinux`` container
 
 
    is the base of all our exercises here.
    is the base of all our exercises here.
 
 
-In this container we install all services :ref:`including searx, morty & filtron
-<lxc.sh install suite>` in once:
+The :ref:`lxc-searxng.env` consists of several images, see ``export
+LXC_SUITE=(...`` near by :origin:`utils/lxc-searxng.env#L19`.
+For this blog post we exercise on a archlinux_ image.  The container of this
+image is named ``searxng-archlinux``.
+
+Lets build the container, but be sure that this container does not already
+exists, so first lets remove possible old one:
 
 
 .. tabs::
 .. tabs::
 
 
-  .. group-tab:: desktop
+  .. group-tab:: desktop (HOST)
 
 
      .. code:: bash
      .. code:: bash
 
 
-        $ sudo -H ./utils/lxc.sh install suite searxng-archlinux
+        $ sudo -H ./utils/lxc.sh remove searxng-archlinux
+        $ sudo -H ./utils/lxc.sh build searxng-archlinux
 
 
-To proxy HTTP from filtron and morty in the container to the outside of the
-container, install nginx into the container.  Once for the bot blocker filtron:
 
 
-.. tabs::
+.. sidebar::  further read
 
 
-  .. group-tab:: desktop
+   - :ref:`lxc.sh install suite`
+   - :ref:`installation nginx`
 
 
-     .. code:: bash
-
-        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
-          ./utils/filtron.sh nginx install
-        ...
-        INFO:  got 429 from http://10.174.184.156/searx
-
-and once for the content sanitizer (content proxy morty):
+To install the complete :ref:`SearXNG suite <searxng lxc suite>` and the HTTP
+proxy :ref:`installation nginx` into the archlinux container run:
 
 
 .. tabs::
 .. tabs::
 
 
-  .. group-tab:: desktop
+  .. group-tab:: desktop (HOST)
 
 
      .. code:: bash
      .. code:: bash
 
 
-        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
-          ./utils/morty.sh nginx install
+        $ sudo -H ./utils/lxc.sh install suite searxng-archlinux
+        $ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
+        $ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL
         ...
         ...
-        INFO:  got 200 from http://10.174.184.156/morty/
+        [searxng-archlinux]    SEARXNG_URL          : http://n.n.n.140/searxng
 
 
 .. sidebar:: Fully functional SearXNG suite
 .. sidebar:: Fully functional SearXNG suite
 
 
-   From here on you have a fully functional SearXNG suite running with bot
-   blocker (filtron) and WEB content sanitizer (content proxy morty), both are
-   needed for a *privacy protecting* search engine.
+   From here on you have a fully functional SearXNG suite (including a
+   :ref:`redis db`).
 
 
-On your system, the IP of your ``searxng-archlinux`` container differs from
-http://10.174.184.156/searx, just open the URL reported in your installation
-protocol in your WEB browser from the desktop to test the instance from outside
-of the container.
+In such a SearXNG suite admins can maintain and access the debug log of the
+services quite easy.
 
 
-In such a earXNG suite admins can maintain and access the debug log of the
-different services quite easy.
+In the example above the SearXNG instance in the container is wrapped to
+``http://n.n.n.140/searxng`` to the HOST system.  Note, on your HOST system, the
+IP of your ``searxng-archlinux`` container is different to this example.  To
+test the instance in the conatiner from outside of the container, in your WEB
+browser on your desktop just open the URL reported in your installation
 
 
 .. _working in containers:
 .. _working in containers:
 
 
@@ -166,77 +157,76 @@ In containers, work as usual
 ============================
 ============================
 
 
 Usually you open a root-bash using ``sudo -H bash``.  In case of LXC containers
 Usually you open a root-bash using ``sudo -H bash``.  In case of LXC containers
-open the root-bash in the container using ``./utils/lxc.sh cmd
-searxng-archlinux``:
+open the root-bash in the container is done by the ``./utils/lxc.sh cmd
+searxng-archlinux`` command:
 
 
 .. tabs::
 .. tabs::
 
 
-  .. group-tab:: desktop
+  .. group-tab:: desktop (HOST)
 
 
      .. code:: bash
      .. code:: bash
 
 
         $ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
         $ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
         INFO:  [searxng-archlinux] bash
         INFO:  [searxng-archlinux] bash
-        [root@searxng-archlinux searx]# pwd
-        /share/searxng
+        [root@searxng-archlinux SearXNG]$
 
 
-The prompt ``[root@searxng-archlinux ...]`` signals, that you are the root user in
-the searxng-container.  To debug the running SearXNG instance use:
+The prompt ``[root@searxng-archlinux ...]`` signals, that you are the root user
+in the container (GUEST).  To debug the running SearXNG instance use:
 
 
 .. tabs::
 .. tabs::
 
 
-  .. group-tab:: root@searxng-archlinux
+  .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
 
 
      .. code:: bash
      .. code:: bash
 
 
-        $ ./utils/searx.sh inspect service
+        $ ./utils/searxng.sh instance inspect
         ...
         ...
         use [CTRL-C] to stop monitoring the log
         use [CTRL-C] to stop monitoring the log
         ...
         ...
 
 
-Back in the browser on your desktop open the service http://10.174.184.156/searx
+  .. group-tab:: desktop (HOST)
+
+     .. code:: bash
+
+        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance inspect
+        ...
+        use [CTRL-C] to stop monitoring the log
+        ...
+
+
+Back in the browser on your desktop open the service http://n.n.n.140/searxng
 and run your application tests while the debug log is shown in the terminal from
 and run your application tests while the debug log is shown in the terminal from
 above.  You can stop monitoring using ``CTRL-C``, this also disables the *"debug
 above.  You can stop monitoring using ``CTRL-C``, this also disables the *"debug
 option"* in SearXNG's settings file and restarts the SearXNG uwsgi application.
 option"* in SearXNG's settings file and restarts the SearXNG uwsgi application.
-To debug services from filtron and morty analogous use:
 
 
-Another point we have to notice is that the service (:ref:`SearXNG <searxng.sh>`
+Another point we have to notice is that the service :ref:`SearXNG <searxng.sh>`
 runs under dedicated system user account with the same name (compare
 runs under dedicated system user account with the same name (compare
-:ref:`create searxng user`).  To get a shell from these accounts, simply call:
+:ref:`create searxng user`).  To get a login shell from these accounts, simply
+call:
 
 
 .. tabs::
 .. tabs::
 
 
-  .. group-tab:: root@searxng-archlinux
+  .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
 
 
      .. code:: bash
      .. code:: bash
 
 
-        $ ./utils/searxng.sh instance cmd bash
-
-To get in touch, open a shell from the service user (searxng@searxng-archlinux):
-
-.. tabs::
+        $ ./utils/searxng.sh instance cmd bash -l
+        (searx-pyenv) [searxng@searxng-archlinux ~]$ pwd
+        /usr/local/searxng
 
 
-  .. group-tab:: desktop
+  .. group-tab:: desktop (HOST)
 
 
      .. code:: bash
      .. code:: bash
 
 
-        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance cmd bash
-        INFO:  [searxng-archlinux] ./utils/searxng.sh instance cmd bash
-        [searxng@searxng-archlinux ~]$
+        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance cmd bash -l
+        INFO:  [searxng-archlinux] ./utils/searxng.sh instance cmd bash -l
+        (searx-pyenv) [searxng@searxng-archlinux ~]$ pwd
+        /usr/local/searxng
 
 
 The prompt ``[searxng@searxng-archlinux]`` signals that you are logged in as system
 The prompt ``[searxng@searxng-archlinux]`` signals that you are logged in as system
-user ``searx`` in the ``searxng-archlinux`` container and the python *virtualenv*
+user ``searxng`` in the ``searxng-archlinux`` container and the python *virtualenv*
 ``(searxng-pyenv)`` environment is activated.
 ``(searxng-pyenv)`` environment is activated.
 
 
-.. tabs::
-
-  .. group-tab:: searxng@searxng-archlinux
-
-     .. code:: bash
-
-        (searxng-pyenv) [searxng@searxng-archlinux ~]$ pwd
-        /usr/local/searxng
-
 
 
 Wrap production into developer suite
 Wrap production into developer suite
 ====================================
 ====================================
@@ -256,12 +246,11 @@ With the use of the :ref:`searxng.sh` the SearXNG service was installed as
 
 
 .. tabs::
 .. tabs::
 
 
-  .. group-tab:: desktop
+  .. group-tab:: uwsgi@searxng
 
 
      .. code:: bash
      .. code:: bash
 
 
-        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
-          systemctl stop uwsgi@searxng
+        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl stop uwsgi@searxng
 
 
 With the command above, we stopped the SearXNG uWSGI-App in the archlinux
 With the command above, we stopped the SearXNG uWSGI-App in the archlinux
 container.
 container.
@@ -278,17 +267,25 @@ least you should attend the settings of ``uid``, ``chdir``, ``env`` and
   virtualenv = /usr/local/searxng/searxng-pyenv
   virtualenv = /usr/local/searxng/searxng-pyenv
   pythonpath = /usr/local/searxng/searxng-src
   pythonpath = /usr/local/searxng/searxng-src
 
 
-If you have read the :ref:`"Good to know section" <lxc.sh>` you remember, that
-each container shares the root folder of the repository and the command
-``utils/lxc.sh cmd`` handles relative path names **transparent**.  To wrap the
-SearXNG installation into a developer one, we simple have to create a smylink to
-the **transparent** reposetory from the desktop.  Now lets replace the
-repository at ``searxng-src`` in the container with the working tree from outside
-of the container:
+If you have read the :ref:`Good to know` you remember, that each container
+shares the root folder of the repository and the command ``utils/lxc.sh cmd``
+handles relative path names **transparent**.
+
+To wrap the SearXNG installation in the container into a developer one, we
+simple have to create a smylink to the **transparent** reposetory from the
+desktop.  Now lets replace the repository at ``searxng-src`` in the container
+with the working tree from outside of the container:
 
 
 .. tabs::
 .. tabs::
 
 
-  .. group-tab:: container becomes a developer suite
+  .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
+
+     .. code:: bash
+
+        $ mv /usr/local/searxng/searxng-src  /usr/local/searxng/searxng-src.old
+        $ ln -s /share/SearXNG/ /usr/local/searxng/searxng-src
+
+  .. group-tab:: desktop (HOST)
 
 
      .. code:: bash
      .. code:: bash
 
 
@@ -296,7 +293,7 @@ of the container:
           mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
           mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
 
 
         $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
         $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
-          ln -s /share/searx/ /usr/local/searxng/searxng-src
+          ln -s /share/SearXNG/ /usr/local/searxng/searxng-src
 
 
 Now we can develop as usual in the working tree of our desktop system.  Every
 Now we can develop as usual in the working tree of our desktop system.  Every
 time the software was changed, you have to restart the SearXNG service (in the
 time the software was changed, you have to restart the SearXNG service (in the
@@ -304,48 +301,83 @@ container):
 
 
 .. tabs::
 .. tabs::
 
 
-  .. group-tab:: desktop
+  .. group-tab:: uwsgi@searxng
 
 
      .. code:: bash
      .. code:: bash
 
 
-        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
-          systemctl restart uwsgi@searx
+        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl restart uwsgi@searxng
 
 
 
 
 Remember: :ref:`working in containers` .. here are just some examples from my
 Remember: :ref:`working in containers` .. here are just some examples from my
 daily usage:
 daily usage:
 
 
+To *inspect* the SearXNG instance (already described above):
+
 .. tabs::
 .. tabs::
 
 
-  .. group-tab:: desktop
+  .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
 
 
-     To *inspect* the SearXNG instance (already described above):
+     .. code:: bash
+
+        $ ./utils/searx.sh inspect service
+
+  .. group-tab:: desktop (HOST)
 
 
      .. code:: bash
      .. code:: bash
 
 
-        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
-          ./utils/searx.sh inspect service
+        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searx.sh inspect service
 
 
-     Run :ref:`makefile`, e.g. to test inside the container:
+Run :ref:`makefile`, e.g. to test inside the container:
+
+.. tabs::
+
+  .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
 
 
      .. code:: bash
      .. code:: bash
 
 
-        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
-          make test
+        $ make test
 
 
-     To install all prerequisites needed for a :ref:`buildhosts`:
+  .. group-tab:: desktop (HOST)
 
 
      .. code:: bash
      .. code:: bash
 
 
-        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
-          ./utils/searxng.sh install buildhost
+        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux  make test
+
 
 
-     To build the docs on a buildhost :ref:`buildhosts`:
+
+To install all prerequisites needed for a :ref:`buildhosts`:
+
+.. tabs::
+
+  .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
 
 
      .. code:: bash
      .. code:: bash
 
 
-        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
-          make docs.html
+        $ ./utils/searxng.sh install buildhost
+
+  .. group-tab:: desktop (HOST)
+
+     .. code:: bash
+
+        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh install buildhost
+
+
+To build the docs on a buildhost :ref:`buildhosts`:
+
+.. tabs::
+
+  .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
+
+     .. code:: bash
+
+        $ make docs.html
+
+  .. group-tab:: desktop (HOST)
+
+     .. code:: bash
+
+        $ sudo -H ./utils/lxc.sh cmd searxng-archlinux make docs.html
+
 
 
 .. _lxcdev summary:
 .. _lxcdev summary:
 
 
@@ -356,48 +388,51 @@ We build up a fully functional SearXNG suite in a archlinux container:
 
 
 .. code:: bash
 .. code:: bash
 
 
+   $ sudo -H ./utils/lxc.sh build searxng-archlinux
    $ sudo -H ./utils/lxc.sh install suite searxng-archlinux
    $ sudo -H ./utils/lxc.sh install suite searxng-archlinux
+   ...
+   Developer install? (wraps source from HOST into the running instance) [YES/no]
 
 
-To access HTTP from the desktop we installed nginx for the services inside the
-container:
+To wrap the suite into a developer one answer ``YES`` (or press Enter).
 
 
-.. tabs::
+.. code:: text
 
 
-  .. group-tab:: [root@searxng-archlinux]
+   link SearXNG's sources to: /share/SearXNG
+   =========================================
 
 
-     .. code:: bash
+   mv -f "/usr/local/searxng/searxng-src" "/usr/local/searxng/searxng-src.backup"
+   ln -s "/share/SearXNG" "/usr/local/searxng/searxng-src"
+   ls -ld /usr/local/searxng/searxng-src
+     |searxng| lrwxrwxrwx 1 searxng searxng ... /usr/local/searxng/searxng-src -> /share/SearXNG
 
 
-        $ ./utils/filtron.sh nginx install
-        $ ./utils/morty.sh nginx install
+On code modification the instance has to be restarted (see :ref:`uWSGI
+maintenance`):
 
 
-To wrap the suite into a developer one, we created a symbolic link to the
-repository which is shared **transparent** from the desktop's file system into
-the container :
+.. code:: bash
 
 
-.. tabs::
+   $ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl restart uwsgi@searxng
 
 
-  .. group-tab:: [root@searxng-archlinux]
+To access HTTP from the desktop we installed nginx for the services inside the
+container:
 
 
-     .. code:: bash
+.. code:: bash
 
 
-	$ mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
-	$ ln -s /share/searx/ /usr/local/searxng/searxng-src
-	$ systemctl restart uwsgi@searx
+   $ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
 
 
-To get information about the searxNG suite in the archlinux container we can
+To get information about the SearxNG suite in the archlinux container we can
 use:
 use:
 
 
-.. tabs::
-
-  .. group-tab:: desktop
-
-     .. code:: bash
-
-        $ sudo -H ./utils/lxc.sh show suite searxng-archlinux
-        ...
-        [searxng-archlinux]  INFO:  (eth0) filtron:    http://10.174.184.156:4004/ http://10.174.184.156/searx
-        [searxng-archlinux]  INFO:  (eth0) morty:      http://10.174.184.156:3000/
-        [searxng-archlinux]  INFO:  (eth0) docs.live:  http://10.174.184.156:8080/
-        [searxng-archlinux]  INFO:  (eth0) IPv6:       http://[fd42:573b:e0b3:e97e:216:3eff:fea5:9b65]
-        ...
+.. code:: text
+
+   $ sudo -H ./utils/lxc.sh show suite searxng-archlinux
+   [searxng-archlinux]  INFO:  (eth0) docs-live:  http:///n.n.n.140:8080/
+   [searxng-archlinux]  INFO:  (eth0) IPv6:       http://[fd42:555b:2af9:e121:216:3eff:fe5b:1744]
+   [searxng-archlinux]  uWSGI:
+   [searxng-archlinux]    SEARXNG_UWSGI_SOCKET : /usr/local/searxng/run/socket
+   [searxng-archlinux]  environment /usr/local/searxng/searxng-src/utils/brand.env:
+   [searxng-archlinux]    GIT_URL              : https://github.com/searxng/searxng
+   [searxng-archlinux]    GIT_BRANCH           : master
+   [searxng-archlinux]    SEARXNG_URL          : http:///n.n.n.140/searxng
+   [searxng-archlinux]    SEARXNG_PORT         : 8888
+   [searxng-archlinux]    SEARXNG_BIND_ADDRESS : 127.0.0.1
 
 

+ 2 - 0
docs/utils/index.rst

@@ -20,6 +20,8 @@ Common command environments
 
 
 The scripts in our tooling box often dispose of common environments:
 The scripts in our tooling box often dispose of common environments:
 
 
+.. _FORCE_TIMEOUT:
+
 ``FORCE_TIMEOUT`` : environment
 ``FORCE_TIMEOUT`` : environment
   Sets timeout for interactive prompts. If you want to run a script in batch
   Sets timeout for interactive prompts. If you want to run a script in batch
   job, with defaults choices, set ``FORCE_TIMEOUT=0``.  By example; to install a
   job, with defaults choices, set ``FORCE_TIMEOUT=0``.  By example; to install a

+ 155 - 56
docs/utils/lxc.sh.rst

@@ -14,6 +14,11 @@
 ``utils/lxc.sh``
 ``utils/lxc.sh``
 ================
 ================
 
 
+With the use of *Linux Containers* (LXC_) we can scale our tasks over a stack of
+containers, what we call the: *lxc suite*.  The :ref:`lxc-searxng.env` is
+loaded by default, every time you start the ``lxc.sh`` script (*you do not need
+to care about*).
+
 .. sidebar:: further reading
 .. sidebar:: further reading
 
 
    - snap_, `snapcraft LXD`_
    - snap_, `snapcraft LXD`_
@@ -21,10 +26,16 @@
    - `LXC/LXD Image Server`_
    - `LXC/LXD Image Server`_
    - `LXD@github`_
    - `LXD@github`_
 
 
-With the use of *Linux Containers* (LXC_) we can scale our tasks over a stack of
-containers, what we call the: *lxc suite*.  The *SearXNG suite*
-(:origin:`lxc-searxng.env <utils/lxc-searxng.env>`) is loaded by default, every time
-you start the ``lxc.sh`` script (*you do not need to care about*).
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+
+.. _lxd install:
+
+Install LXD
+===========
 
 
 Before you can start with containers, you need to install and initiate LXD_
 Before you can start with containers, you need to install and initiate LXD_
 once::
 once::
@@ -38,44 +49,19 @@ take some time**::
 
 
   $ sudo -H ./utils/lxc.sh build
   $ sudo -H ./utils/lxc.sh build
 
 
+.. sidebar:: hint
+
+   If you have issues with the internet connectivity of your containers read
+   section :ref:`internet connectivity docker`.
+
 A cup of coffee later, your LXC suite is build up and you can run whatever task
 A cup of coffee later, your LXC suite is build up and you can run whatever task
 you want / in a selected or even in all :ref:`LXC suite containers <lxc.sh
 you want / in a selected or even in all :ref:`LXC suite containers <lxc.sh
 help>`.
 help>`.
 
 
-.. hint::
-
-   If you see any problems with the internet connectivity of your
-   containers read section :ref:`internet connectivity docker`.
-
-If you do not want to build all containers, **you can build just one**::
-
-  $ sudo -H ./utils/lxc.sh build searxng-archlinux
-
-*Good to know ...*
-
-Each container shares the root folder of the repository and the command
-``utils/lxc.sh cmd`` **handles relative path names transparent**, compare output
-of::
-
-  $ sudo -H ./utils/lxc.sh cmd -- ls -la Makefile
-  ...
-
-In the containers, you can run what ever you want, e.g. to start a bash use::
-
-  $ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
-  INFO:  [searxng-archlinux] bash
-  [root@searxng-archlinux SearXNG]#
-
-If there comes the time you want to **get rid off all** the containers and
-**clean up local images** just type::
-
-  $ sudo -H ./utils/lxc.sh remove
-  $ sudo -H ./utils/lxc.sh remove images
-
 .. _internet connectivity docker:
 .. _internet connectivity docker:
 
 
 Internet Connectivity & Docker
 Internet Connectivity & Docker
-==============================
+------------------------------
 
 
 .. sidebar::  further read
 .. sidebar::  further read
 
 
@@ -115,21 +101,132 @@ Reboot your system and check the iptables rules::
   :FORWARD ACCEPT [7048:7851230]
   :FORWARD ACCEPT [7048:7851230]
 
 
 
 
+.. _searxng lxc suite:
+
+SearXNG LXC suite
+=================
+
+The intention of the *SearXNG LXC suite* is to build up a suite of containers
+for development tasks or :ref:`buildhosts <Setup SearXNG buildhost>` with a very
+small set of simple commands.  At the end of the ``--help`` output the SearXNG
+suite from the :ref:`lxc-searxng.env` is introduced::
+
+   $ sudo -H ./utils/lxc.sh --help
+   ...
+   LXC suite: searxng
+     Suite includes installation of SearXNG
+     images:     ubu2004 ubu2204 fedora35 archlinux
+     containers: searxng-ubu2004 searxng-ubu2204 searxng-fedora35 searxng-archlinux
+
+As shown above there are images and containers build up on this images.  To show
+more info about the containers in the *SearXNG LXC suite* call ``show suite``.
+If this is the first time you make use of the SearXNG LXC suite, no containers
+are installed and the output is::
+
+  $ sudo -H ./utils/lxc.sh show suite
+
+  LXC suite (searxng-*)
+  =====================
+
+  +------+-------+------+------+------+-----------+
+  | NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+  +------+-------+------+------+------+-----------+
+
+  WARN:  container searxng-ubu2004 does not yet exists
+  WARN:  container searxng-ubu2204 does not yet exists
+  WARN:  container searxng-fedora35 does not yet exists
+  WARN:  container searxng-archlinux does not yet exists
+
+If you do not want to run a command or a build in all containers, **you can
+build just one**. Here by example in the container that is build upon the
+*archlinux* image::
+
+  $ sudo -H ./utils/lxc.sh build searxng-archlinux
+  $ sudo -H ./utils/lxc.sh cmd searxng-archlinux pwd
+
+Otherwise, to apply a command to all containers you can use::
+
+  $ sudo -H ./utils/lxc.sh build
+  $ sudo -H ./utils/lxc.sh cmd -- ls -la .
+
+Running commands
+----------------
+
+**Inside containers, you can run scripts** from the :ref:`toolboxing` or run
+what ever command you need.  By example, to start a bash use::
+
+  $ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
+  INFO:  [searxng-archlinux] bash
+  [root@searxng-archlinux SearXNG]#
+
+.. _Good to know:
+
+Good to know
+------------
+
+Each container shares the root folder of the repository and the command
+``utils/lxc.sh cmd`` **handle relative path names transparent**::
+
+ $ pwd
+ /share/SearXNG
+
+ $ sudo -H ./utils/lxc.sh cmd searxng-archlinux pwd
+ INFO:  [searxng-archlinux] pwd
+ /share/SearXNG
+
+The path ``/share/SearXNG`` will be different on your HOST system.  The commands
+in the conatiner are executed by the ``root`` inside of the container.  Compare
+output of::
+
+  $ ls -li Makefile
+  47712402 -rw-rw-r-- 1 markus markus 2923 Apr 19 13:52 Makefile
+
+  $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ls -li Makefile
+  INFO:  [searxng-archlinux] ls -li Makefile
+  47712402 -rw-rw-r-- 1 root root 2923 Apr 19 11:52 Makefile
+  ...
+
+Since the path ``/share/SearXNG`` of the HOST system is wrapped into the
+container under the same name, the shown ``Makefile`` (inode ``47712402``) in
+the ouput is always the identical ``/share/SearXNG/Makefile`` from the HOST
+system.  In the example shown above the owner of the path in the container is
+the ``root`` user of the conatiner (and the timezone in the container is
+different to HOST system).
+
+
 .. _lxc.sh install suite:
 .. _lxc.sh install suite:
 
 
 Install suite
 Install suite
-=============
+-------------
+
+.. sidebar::  further read
 
 
-To install the complete :ref:`SearXNG suite (includes searx, morty & filtron)
-<lxc-searxng.env>` into all LXC_ use::
+   - :ref:`working in containers`
+   - :ref:`FORCE_TIMEOUT <FORCE_TIMEOUT>`
 
 
+To install the complete :ref:`SearXNG suite <lxc-searxng.env>` into **all** LXC_
+containers leave the container argument empty and run::
+
+  $ sudo -H ./utils/lxc.sh build
   $ sudo -H ./utils/lxc.sh install suite
   $ sudo -H ./utils/lxc.sh install suite
 
 
+To *build & install* suite only in one container you can use by example::
+
+  $ sudo -H ./utils/lxc.sh build searxng-archlinux
+  $ sudo -H ./utils/lxc.sh install suite searxng-archlinux
+
 The command above installs a SearXNG suite (see :ref:`installation scripts`).
 The command above installs a SearXNG suite (see :ref:`installation scripts`).
 To :ref:`install a nginx <installation nginx>` reverse proxy (or alternatively
 To :ref:`install a nginx <installation nginx>` reverse proxy (or alternatively
 use :ref:`apache <installation apache>`)::
 use :ref:`apache <installation apache>`)::
 
 
-    sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
+  $ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
+
+Same operation just in one container of the suite::
+
+  $ sudo -H ./utils/lxc.sh cmd searxng-archlinux FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
+
+The :ref:`FORCE_TIMEOUT <FORCE_TIMEOUT>` environment is set to zero to run the
+script without user interaction.
 
 
 To get the IP (URL) of the SearXNG service in the containers use ``show suite``
 To get the IP (URL) of the SearXNG service in the containers use ``show suite``
 command.  To test instances from containers just open the URLs in your
 command.  To test instances from containers just open the URLs in your
@@ -137,22 +234,22 @@ WEB-Browser::
 
 
   $ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL
   $ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL
 
 
-  [searxng-ubu2110]      SEARXNG_URL          : http://n.n.n.147/searxng
-  [searxng-ubu2004]      SEARXNG_URL          : http://n.n.n.246/searxng
-  [searxnggfedora35]     SEARXNG_URL          : http://n.n.n.140/searxng
-  [searxng-archlinux]    SEARXNG_URL          : http://n.n.n.165/searxng
+  [searxng-ubu2110]      SEARXNG_URL          : http://n.n.n.170/searxng
+  [searxng-ubu2004]      SEARXNG_URL          : http://n.n.n.160/searxng
+  [searxnggfedora35]     SEARXNG_URL          : http://n.n.n.150/searxng
+  [searxng-archlinux]    SEARXNG_URL          : http://n.n.n.140/searxng
 
 
+Clean up
+--------
 
 
-Running commands
-================
+If there comes the time you want to **get rid off all** the containers and
+**clean up local images** just type::
 
 
-**Inside containers, you can use make or run scripts** from the
-:ref:`toolboxing`.  By example: to setup a :ref:`buildhosts` and run the
-Makefile target ``test`` in the archlinux_ container::
+  $ sudo -H ./utils/lxc.sh remove
+  $ sudo -H ./utils/lxc.sh remove images
 
 
-  sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh install buildhost
-  sudo -H ./utils/lxc.sh cmd searxng-archlinux make test
 
 
+.. _Setup SearXNG buildhost:
 
 
 Setup SearXNG buildhost
 Setup SearXNG buildhost
 =======================
 =======================
@@ -160,7 +257,7 @@ Setup SearXNG buildhost
 You can **install the SearXNG buildhost environment** into one or all containers.
 You can **install the SearXNG buildhost environment** into one or all containers.
 The installation procedure to set up a :ref:`build host<buildhosts>` takes its
 The installation procedure to set up a :ref:`build host<buildhosts>` takes its
 time.  Installation in all containers will take more time (time for another cup
 time.  Installation in all containers will take more time (time for another cup
-of coffee).::
+of coffee). ::
 
 
   sudo -H ./utils/lxc.sh cmd -- ./utils/searxng.sh install buildhost
   sudo -H ./utils/lxc.sh cmd -- ./utils/searxng.sh install buildhost
 
 
@@ -174,13 +271,13 @@ To get IP of the container and the port number *live docs* is listening::
 
 
   $ sudo ./utils/lxc.sh show suite | grep docs.live
   $ sudo ./utils/lxc.sh show suite | grep docs.live
   ...
   ...
-  [searxng-archlinux]  INFO:  (eth0) docs.live:  http://n.n.n.12:8080/
+  [searxng-archlinux]  INFO:  (eth0) docs.live:  http://n.n.n.140:8080/
 
 
 
 
 .. _lxc.sh help:
 .. _lxc.sh help:
 
 
-Overview
-========
+Command Help
+============
 
 
 The ``--help`` output of the script is largely self-explanatory:
 The ``--help`` output of the script is largely self-explanatory:
 
 
@@ -189,8 +286,10 @@ The ``--help`` output of the script is largely self-explanatory:
 
 
 .. _lxc-searxng.env:
 .. _lxc-searxng.env:
 
 
-SearXNG suite
-=============
+SearXNG suite config
+====================
+
+The SearXNG suite is defined in the file :origin:`utils/lxc-searxng.env`:
 
 
 .. literalinclude:: ../../utils/lxc-searxng.env
 .. literalinclude:: ../../utils/lxc-searxng.env
    :language: bash
    :language: bash

+ 10 - 4
docs/utils/searxng.sh.rst

@@ -5,6 +5,9 @@
 ``utils/searxng.sh``
 ``utils/searxng.sh``
 ====================
 ====================
 
 
+To simplify the installation and maintenance of a SearXNG instance you can use the
+script :origin:`utils/searxng.sh`.
+
 .. sidebar:: further reading
 .. sidebar:: further reading
 
 
    - :ref:`architecture`
    - :ref:`architecture`
@@ -12,8 +15,11 @@
    - :ref:`installation nginx`
    - :ref:`installation nginx`
    - :ref:`installation apache`
    - :ref:`installation apache`
 
 
-To simplify the installation and maintenance of a SearXNG instance you can use the
-script :origin:`utils/searxng.sh`.
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
 
 
 Install
 Install
 =======
 =======
@@ -28,8 +34,8 @@ The installation is described in chapter :ref:`installation basic`.
 
 
 .. _searxng.sh overview:
 .. _searxng.sh overview:
 
 
-Overview
-========
+Command Help
+============
 
 
 The ``--help`` output of the script is largely self-explanatory:
 The ``--help`` output of the script is largely self-explanatory:
 
 

+ 5 - 4
utils/lxc-searxng.env

@@ -42,10 +42,11 @@ EOF
 lxc_suite_install() {
 lxc_suite_install() {
     (
     (
         lxc_set_suite_env
         lxc_set_suite_env
-        FORCE_TIMEOUT=0
-        export FORCE_TIMEOUT
-        "${LXC_REPO_ROOT}/utils/searxng.sh" install all
-        rst_title "suite installation finished ($(hostname))" part
+        FORCE_TIMEOUT=0 "${LXC_REPO_ROOT}/utils/searxng.sh" install all
+        rst_title "Suite installation finished ($(hostname))" part
+        if ask_yn "Developer install? (wraps source from HOST into the running instance)" Yn; then
+            "${LXC_REPO_ROOT}/utils/searxng.sh" searxng.install.link_src "$(pwd)"
+        fi
         lxc_suite_info
         lxc_suite_info
         echo
         echo
     )
     )

+ 12 - 0
utils/searxng.sh

@@ -467,6 +467,18 @@ EOF
     popd > /dev/null
     popd > /dev/null
 }
 }
 
 
+searxng.install.link_src() {
+    rst_title "link SearXNG's sources to: $2" chapter
+    echo
+    tee_stderr 0.1 <<EOF | sudo -H -u "${SERVICE_USER}" -i 2>&1 | prefix_stdout "$_service_prefix"
+mv -f "${SEARXNG_SRC}" "${SEARXNG_SRC}.backup"
+ln -s "${2}" "${SEARXNG_SRC}"
+ls -ld /usr/local/searxng/searxng-src
+EOF
+    echo
+    uWSGI_restart "$SEARXNG_UWSGI_APP"
+}
+
 searxng.install.pyenv() {
 searxng.install.pyenv() {
     rst_title "Create virtualenv (python)" section
     rst_title "Create virtualenv (python)" section
     echo
     echo