docs: use make and uitls/searx.sh in containers

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

docs/admin/buildhosts.rst

@@ -4,9 +4,6 @@
 Buildhosts
 ==========
 
-To get best results from build, its recommend to install additional packages
-on build hosts.
-
 .. sidebar:: This article needs some work
 
    If you have any contribution send us your :pull:`PR <../pulls>`, see
@@ -17,6 +14,23 @@ on build hosts.
    :local:
    :backlinks: entry
 
+To get best results from build, its recommend to install additional packages
+on build hosts (see :ref:`searx.sh`).::
+
+  sudo -H ./utils/searx.sh buildhost
+
+This will install packages needed by searx:
+
+.. include:: ../../build/docs/includes/searx.rst
+   :start-after: START distro-packages
+   :end-before: END distro-packages
+
+and packages needed to build docuemtation and run tests:
+
+.. include:: ../../build/docs/includes/searx.rst
+   :start-after: START build-packages
+   :end-before: END build-packages
+
 .. _docs build:
 
 Build docs

docs/build-templates/searx.rst

@@ -28,6 +28,32 @@ ${fedora}
 
 .. END distro-packages
 
+.. START build-packages
+
+.. tabs::
+
+  .. group-tab:: Ubuntu / debian
+
+    .. code-block:: sh
+
+      $ sudo -H apt-get install -y \\
+${debian_build}
+
+  .. group-tab:: Arch Linux
+
+    .. code-block:: sh
+
+      $ sudo -H pacman -S --noconfirm \\
+${arch_build}
+
+  .. group-tab::  Fedora / RHEL
+
+    .. code-block:: sh
+
+      $ sudo -H dnf install -y \\
+${fedora_build}
+
+.. END build-packages
 
 .. START create user
 

docs/utils/lxc.sh.rst

@@ -1,5 +1,4 @@
 
-
 .. _snap: https://snapcraft.io
 .. _snapcraft LXD: https://snapcraft.io/lxd
 .. _LXC/LXD Image Server: https://uk.images.linuxcontainers.org/
@@ -7,6 +6,8 @@
 .. _LXD: https://linuxcontainers.org/lxd/introduction/
 .. _`LXD@github`: https://github.com/lxc/lxd
 
+.. _archlinux: https://www.archlinux.org/
+
 .. _lxc.sh:
 
 ================
@@ -21,24 +22,31 @@
    - `LXD@github`_
 
 With the use of *Linux Containers* (LXC_) we can scale our tasks over a stack of
-containers, what we call the: *lxc suite*.  Before you can start with
-containers, you need to install and initiate LXD_ once::
+containers, what we call the: *lxc suite*.  The *searx suite*
+(:origin:`lxc-searx.env <utils/lxc-searx.env>`) is loaded by default, every time
+you start the ``lxc.sh`` script (*you do not need to care about*).
+
+Before you can start with containers, you need to install and initiate LXD_
+once::
 
   $ snap install lxd
   $ lxd init --auto
 
-The *searx suite* (:origin:`lxc-searx.env <utils/lxc-searx.env>`) is loaded by
-default, every time you start the ``lxc.sh`` script (you do not need to care
-about).  To make use of the containers from the *searx suite*, you have to build
-the :ref:`LXC suite containers <lxc.sh --help>` first.  But be warned, this
-might take some time::
+To make use of the containers from the *searx suite*, you have to build the
+:ref:`LXC suite containers <lxc.sh help>` initial.  But be warned, this might
+take some time::
 
   $ sudo -H ./utils/lxc.sh build
 
 A cup of coffee later, your LXC suite is build up and you can run whatever task
 you want / in a selected or even in all :ref:`LXC suite containers <lxc.sh
---help>`.  Each container shares the root folder of the repository and the
-command ``utils/lxc.sh cmd`` handles relative path names *transparent*::
+help>`.
+
+*Good to know ...*
+
+Eeach container shares the root folder of the repository and the
+command ``utils/lxc.sh cmd`` **handles relative path names transparent**,
+compare output of::
 
   $ sudo -H ./utils/lxc.sh cmd -- ls -la Makefile
   ...
@@ -46,29 +54,70 @@ command ``utils/lxc.sh cmd`` handles relative path names *transparent*::
   [searx-fedora31]  -rw-r--r-- 1 root root 7603 Mar 30 11:54 Makefile
   [searx-archlinux] -rw-r--r-- 1 root root 7603 Mar 30 11:54 Makefile
 
-With this in mind, you can run :ref:`searx.sh` and install packages, needed by
-searx::
+If there comes the time you want to **get rid off all** the containers and
+**clean up local images** just type::
 
-  $ sudo -H ./utils/lxc.sh cmd -- ./utils/searx.sh install packages
+  $ sudo -H ./utils/lxc.sh remove
+  $ sudo -H ./utils/lxc.sh remove images
 
-And run one of the :origin:`Makefile` targets::
 
-  $ sudo -H ./utils/lxc.sh cmd -- make test.sh
+Install suite
+=============
 
-You can install a *buildhost environment* into the containers (time for another
-cup of coffee)::
+To install the complete :ref:`searx suite (includes searx, morty & filtron)
+<lxc-searx.env>` into all LXC_ use::
 
-  $ sudo -H ./utils/lxc.sh install buildhost
+  $ sudo -H ./utils/lxc.sh install suite
 
-If you want to get rid off all the containers, just type::
+The command above installs a searx suite (see :ref:`installation scripts`).  To
+get the IP (URL) of the filtron service in the containers use ``show suite``
+command.  To test instances from containers just open the URLs in your
+WEB-Browser::
 
-  $ sudo -H ./utils/lxc.sh remove
+  $ sudo ./utils/lxc.sh show suite | grep filtron
+  [searx-ubu1604]  INFO:  (eth0) filtron:    http://n.n.n.135:4004/
+  [searx-ubu1804]  INFO:  (eth0) filtron:    http://n.n.n.141:4004/
+  [searx-ubu1910]  INFO:  (eth0) filtron:    http://n.n.n.137:4004/
+  [searx-ubu2004]  INFO:  (eth0) filtron:    http://n.n.n.127:4004/
+  [searx-fedora31]  INFO:  (eth0) filtron:    http://n.n.n.18:4004/
+  [searx-archlinux]  INFO:  (eth0) filtron:    http://n.n.n.12:4004/
 
-To clean up your local images use::
+  
+Running commands
+================
 
-  $ sudo -H ./utils/lxc.sh remove images
+**Inside containers, you can use make or run scripts** from the
+:ref:`toolboxing`.  By example: to setup a :ref:`buildhosts` and run the
+Makefile target ``test`` in the archlinux_ container::
+
+  sudo -H ./utils/lxc.sh cmd searx-archlinux ./utils/searx.sh buildhost
+  sudo -H ./utils/lxc.sh cmd searx-archlinux make test
+
+
+Setup searx buildhost
+=====================
 
-.. _lxc.sh --help:
+You can **install the searx buildhost environment** into one or all containers.
+The installation procedure to set up a :ref:`build host<buildhosts>` takes its
+time.  Installation in all containers will take more time (time for another cup
+of coffee).::
+
+  sudo -H ./utils/lxc.sh cmd -- ./utils/searx.sh buildhost
+
+To build (live) documentation inside a archlinux_ container::
+
+  sudo -H ./utils/lxc.sh cmd searx-archlinux make docs-clean docs-live
+  ...
+  [I 200331 15:00:42 server:296] Serving on http://0.0.0.0:8080
+
+To get IP of the container and the port number *live docs* is listening::
+
+  $ sudo ./utils/lxc.sh show suite | grep docs-live
+  ...
+  [searx-archlinux]  INFO:  (eth0) docs-live:  http://n.n.n.12:8080/
+
+
+.. _lxc.sh help:
 
 Overview
 ========
@@ -77,3 +126,11 @@ The ``--help`` output of the script is largely self-explanatory:
 
 .. program-output:: ../utils/lxc.sh --help
 
+
+.. _lxc-searx.env:
+
+searx suite
+===========
+
+.. literalinclude:: ../../utils/lxc-searx.env
+   :language: bash

utils/lxc-searx.env

@@ -49,18 +49,6 @@ lxc_suite_install() {
     )
 }
 
-lxc_suite_prepare_buildhost() {
-    (
-        lxc_set_suite_env
-        FORCE_TIMEOUT=0
-        export FORCE_TIMEOUT
-        # "${LXC_REPO_ROOT}/utils/searx.sh* install packages
-        "${LXC_REPO_ROOT}/utils/searx.sh"   install buildhost
-        rst_title "buildhost installation finished ($(hostname))" part
-        echo
-    )
-}
-
 lxc_suite_info() {
     (
         lxc_set_suite_env

utils/lxc.sh

@@ -82,7 +82,7 @@ usage::
   $_cmd [start|stop] [containers|<name>]
   $_cmd show         [info|config|suite|images]
   $_cmd cmd          [--|<name>] '...'
-  $_cmd install      [suite|base|buildhost]
+  $_cmd install      [suite|base]
 
 build
   :containers:   build, launch and 'install basic' packages on 'containers'
@@ -106,7 +106,6 @@ cmd
 install
   :suite:        install LXC suite; ${LXC_SUITE_INSTALL_INFO}
   :base:         prepare LXC; install basic packages
-  :buildhost:    prepare LXC; install buildhost packages
 
 EOF
     usage_images
@@ -150,7 +149,9 @@ main() {
 
     # don't check prerequisite when in recursion
     if [[ ! $1 == __* ]]; then
-        ! required_commands lxc && lxd_info && exit 42
+        if ! in_container; then
+            ! required_commands lxc && lxd_info && exit 42
+        fi
         [[ -z $LXC_SUITE ]] && err_msg "missing LXC_SUITE" && exit 42 
     fi
 
@@ -240,7 +241,7 @@ main() {
         install)
             sudo_or_exit
             case $2 in
-                suite|base|buildhost)
+                suite|base)
                     lxc_exec "${LXC_REPO_ROOT}/utils/lxc.sh" __install "$2"
                     ;;
                 *) usage "$_usage"; exit 42 ;;
@@ -252,7 +253,6 @@ main() {
             case $2 in
                 suite) lxc_suite_install ;;
                 base) FORCE_TIMEOUT=0 lxc_install_base_packages ;;
-                buildhost) lxc_suite_prepare_buildhost ;;
             esac
             ;;
         doc)

utils/searx.sh

@@ -48,7 +48,7 @@ git build-essential libxslt-dev zlib1g-dev libffi-dev libssl-dev
 shellcheck"
 
 BUILD_PACKAGES_debian="\
-graphviz imagemagick texlive-xetex librsvg2-bin
+firefox graphviz imagemagick texlive-xetex librsvg2-bin
 texlive-latex-recommended texlive-extra-utils ttf-dejavu"
 
 # pacman packages
@@ -59,7 +59,7 @@ git base-devel libxml2
 shellcheck"
 
 BUILD_PACKAGES_arch="\
-graphviz imagemagick texlive-bin extra/librsvg
+firefox graphviz imagemagick texlive-bin extra/librsvg
 texlive-core texlive-latexextra ttf-dejavu"
 
 # dnf packages
@@ -70,7 +70,7 @@ git @development-tools libxml2
 ShellCheck"
 
 BUILD_PACKAGES_fedora="\
-graphviz graphviz-gd ImageMagick librsvg2-tools
+firefox graphviz graphviz-gd ImageMagick librsvg2-tools
 texlive-xetex-bin texlive-collection-fontsrecommended
 texlive-collection-latex dejavu-sans-fonts dejavu-serif-fonts
 dejavu-sans-mono-fonts"
@@ -116,7 +116,7 @@ usage() {
 usage::
 
   $(basename "$0") shell
-  $(basename "$0") install    [all|user|searx-src|pyenv|uwsgi|apache|packages]
+  $(basename "$0") install    [all|user|searx-src|pyenv|uwsgi|apache|packages|buildhost]
   $(basename "$0") update     [searx]
   $(basename "$0") remove     [all|user|pyenv|searx-src]
   $(basename "$0") activate   [service]
@@ -135,7 +135,7 @@ install / remove
   :uwsgi:      install searx uWSGI application
   :settings:   reinstall settings from ${REPO_ROOT}/searx/settings.yml
   :packages:   install needed packages from OS package manager
-  :buildhost:  install packages from OS package manager needed by buildhost
+  :buildhost:  install packages from OS package manager needed by buildhosts
 update searx
   Update searx installation ($SERVICE_HOME)
 activate service
@@ -688,9 +688,15 @@ rst-doc() {
     local debian="${SEARX_PACKAGES_debian}"
     local arch="${SEARX_PACKAGES_arch}"
     local fedora="${SEARX_PACKAGES_fedora}"
+    local debian_build="${BUILD_PACKAGES_debian}"
+    local arch_build="${BUILD_PACKAGES_arch}"
+    local fedora_build="${BUILD_PACKAGES_fedora}"
     debian="$(echo "${debian}" | sed 's/.*/          & \\/' | sed '$ s/.$//')"
     arch="$(echo "${arch}"     | sed 's/.*/          & \\/' | sed '$ s/.$//')"
     fedora="$(echo "${fedora}" | sed 's/.*/          & \\/' | sed '$ s/.$//')"
+    debian_build="$(echo "${debian_build}" | sed 's/.*/          & \\/' | sed '$ s/.$//')"
+    arch_build="$(echo "${arch_build}"     | sed 's/.*/          & \\/' | sed '$ s/.$//')"
+    fedora_build="$(echo "${fedora_build}" | sed 's/.*/          & \\/' | sed '$ s/.$//')"
 
     eval "echo \"$(< "${REPO_ROOT}/docs/build-templates/searx.rst")\""