chore: rewrite node installation for new universal installer

maintenance/index.rst

@@ -25,9 +25,11 @@ You can check elasticsearch status and index distribution on any of the nodes:
 
 .. code-block:: console
 
-    nextron@es-node1:~$ curl http://127.0.0.1:9200/_cat/health
-    nextron@es-node1:~$ curl http://127.0.0.1:9200/_cat/nodes
-    nextron@es-node1:~$ curl http://127.0.0.1:9200/_cat/shards
+    nextron@cockpit4:~$ sudo su -
+    [sudo] password for nextron:
+    root@cockpit4:~# curl -u elastic:$(cat /etc/asgard-analysis-cockpit/elastic.password) http://127.0.0.1:9200/_cat/health
+    root@cockpit4:~# curl -u elastic:$(cat /etc/asgard-analysis-cockpit/elastic.password) http://127.0.0.1:9200/_cat/nodes
+    root@cockpit4:~# curl -u elastic:$(cat /etc/asgard-analysis-cockpit/elastic.password) http://127.0.0.1:9200/_cat/shards
 
 Removing Elasticsearch nodes
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -40,14 +42,17 @@ value of "node_to_remove" to the actual node name):
 
 .. code-block:: console
 
-    nextron@es-node1:~$ curl -X PUT "http://127.0.0.1:9200/_cluster/settings" \
+    nextron@cockpit4:~$ curl -X PUT "http://127.0.0.1:9200/_cluster/settings" \
+      -u elastic:$(cat /etc/asgard-analysis-cockpit/elastic.password) \
+      -H "Content-Type: application/json" \
       -d '{"transient": {"cluster.routing.allocation.exclude._name": "node_to_remove"} }'
+      
 
 Then wait until the node has no shards left:
 
 .. code-block:: console
 
-    nextron@es-node1:~$ curl http://127.0.0.1:9200/_cat/shards
+    nextron@cockpit4:~$ curl http://127.0.0.1:9200/_cat/shards
 
 Once no shards are assigned to the node, it is safe to shut it down. When you have
 replicas of each index (number_of_replicas >= 1), the cluster should automatically

nodes/index.rst

@@ -1,94 +1,162 @@
-.. index:: Nodes
+.. index:: Cluster Node
 
-Cluster Node setup
+Cluster Node Setup
 ==================
 
-Prerequisites
-~~~~~~~~~~~~~
+.. warning::
+    The Cluster Installation is currently not possible due to
+    the update of version 4 of the Analysis Cockpit. We will soon
+    release a new ISO which will fix this.
 
-The following prerequisites have to be given:
+Nextron Universal Installer
+---------------------------
+
+The Nextron Universal Installer is a web based installer
+which will guide you through the installation of our
+ASGARD products. The Nextron Universal Installer will install
+**one** of the following products on your server (this manual
+focuses on the ``Elasticsearch Cluster Node for ASGARD Analysis Cockpit``):
 
-* Server must be suitable for the Nextron base image.
+- ASGARD Management Center; alternatively if your license permits:
+
+  * ASGARD Broker
+  * ASGARD Gatekeeper
+  * ASGARD Lobby
+
+- ASGARD Analysis Cockpit; alternatively:
+
+  * Elasticsearch Cluster Node for ASGARD Analysis Cockpit
+
+- ASGARD Security Center, in the following variants:
+
+  * ASGARD Security Center (Backend Only)
+  * ASGARD Security Center (Frontend Only)
+  * ASGARD Security Center (All-in-one, unrecommended)
 
-* All nodes must be able to reach each other by resolving the fully qualified host name.
+.. note::
+   You can only install one product on one server, since the
+   products are not designed to coexist on the same server.
+   The exception being the ASGARD Security Center (All-in-one).
+
+The installation takes roughly between 5-15 minutes, depending
+on your internet connection and the server you are installing
+the product on.
+
+If you encounter problems during your installation, please see
+:ref:`nodes/index:diagnostic pack` for further instructions.
 
-* TCP port 9300 must be open between all nodes (Note: API port 9200 is only used locally).
+Requirements
+------------
 
-Elasticsearch node installation
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The installation of the ``Elasticsearch Cluster Node for ASGARD
+Analysis Cockpit`` requires the following:
 
-Install the server from the Nextron ISO base image as you normally would
-when installing the Analysis Cockpit itself, but **DO NOT** run the Nextron Installer.
-
-Instead, copy ``/usr/share/asgard-analysis-cockpit/scripts/es-node-install.sh``
-to the new node and run it:
-
-.. code-block:: console
-
-    nextron@es-node1:~$ chmod +x es-node-install.sh
-    nextron@es-node1:~$ sudo ./es-node-install.sh
-
-The script will automatically install Elasticsearch and configure the node to
-join the cluster with the Analysis Cockpit host as its master.
-
-Resulting Elasticsearch configuration
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The Elasticsearch configuration can be found in ``/etc/elasticsearch/elasticsearch.yml``.
-It will look like the following:
-
-.. code-block:: yaml
-    :linenos:
-
-    cluster.name: elasticsearch
-    cluster.routing.allocation.exclude._name: elastic-test-01.nextron
-    path.data: /var/lib/elasticsearch
-    path.logs: /var/log/elasticsearch
-    node.roles: [ data, ingest ]
-    http.host: "_local:ipv4_"
-    transport.host: "_site:ipv4_"
-    discovery.seed_hosts: [ elastic-test-01.nextron ]
-    search.default_allow_partial_results: false
-    xpack.security.http.ssl.enabled: false
-    xpack.security.enrollment.enabled: false
-    xpack.security.transport.ssl:
-        enabled: true
-        verification_mode: certificate
-        client_authentication: required
-        keystore.path: elastic-certificates.p12
-        truststore.path: elastic-certificates.p12
-
-Enabling the node
-~~~~~~~~~~~~~~~~~
-
-After the installation, restart elasticsearch:
-
-.. code-block:: console
-
-    nextron@es-node1:~$ sudo systemctl restart elasticsearch.service
-
-The node should automatically join the cluster. To check if the node has
-joined the cluster, run the following command (``number_of_nodes`` should
-be 1+X, where X is the number of nodes you have added):
-
-.. code-block:: console
-    :emphasize-lines: 6
-
-    nextron@cockpit4:~$ curl -s http://127.0.0.1:9200/_cluster/health | jq
-    {
-      "cluster_name": "elasticsearch",
-      "status": "green",
-      "timed_out": false,
-      "number_of_nodes": 4,
-      "number_of_data_nodes": 4,
-      "active_primary_shards": 10,
-      "active_shards": 20,
-      "relocating_shards": 0,
-      "initializing_shards": 0,
-      "unassigned_shards": 8,
-      "delayed_unassigned_shards": 0,
-      "number_of_pending_tasks": 0,
-      "number_of_in_flight_fetch": 0,
-      "task_max_waiting_in_queue_millis": 0,
-      "active_shards_percent_as_number": 71.42857142857143
-    }
+- A valid license file for the ASGARD Analysis Cockpit
+- A configured FQDN with working DNS resolution
+- Internet access during installation (see :ref:`nodes/index:connectivity check`)
+- All nodes must be able to reach each other by resolving the fully qualified host name.
+- TCP port 9300 must be open between all nodes.
+
+Installation
+------------
+
+Install the server from the Nextron ISO base image as
+you normally would when installing the Analysis Cockpit
+itself.
+
+After the ISO installer is finished with the setup,
+you will be greeted at the console login prompt with
+the following message:
+
+.. figure:: ../images/setup_nextronInstaller.png
+   :alt: Login prompt ASGARD Server
+
+Follow the instructions and navigate to the webpage
+displayed on your console. You will most likely get
+a browser warning when you connect the first time to
+the page. This is due to the page using a self signed
+certificate, since it will only be used to install the
+Elasticsearch Cluster Node. You can safely ignore this
+warning and proceed to the page.
+
+You will be greeted with a small introduction as to what
+the Nextron Universal Installer is and what it does. After
+you click ``Next``, you will be presented with the landing
+page of the Nextron Universal Installer.
+
+.. figure:: ../images/setup_nextronInstaller-landing.png
+   :alt: landing page of the Universal Installer
+
+Enter the Installation Code from the terminal and click
+``Next``. The Installer will now guide you through the
+installation.
+
+You will be prompted at one point to upload your cluster
+configuration file. This file is generated by the Analysis
+Cockpit and contains all the necessary information for
+the Elasticsearch Cluster Node to join the cluster.
+
+Please see :ref:`setup/index:resulting elasticsearch configuration`
+for further information on how to generate the cluster configuration
+file.
+
+Once the installation is finished, your Cluster Node is
+a part of the Analysis Cockpit cluster and will start
+receiving data.
+
+Connectivity Check
+------------------
+
+The Nextron Universal Installer will try to connect to our
+update server in order to download all the necessary packages
+once the installation starts. Make sure you can reach the
+update servers (TCP/443 on update-301.nextron-systems.com).
+
+Please configure your proxy settings if you are behind a
+proxy (see :ref:`nodes/index:proxy and ntp settings`).
+
+Valid FQDN
+----------
+
+The Nextron Universal Installer will prompt you to verify the
+FQDN which you configured during the installation of the base
+system. This is needed in order for your server to communicate
+via a HTTPs connection with the Analysis Cockpit and other
+Cluster Nodes. 
+
+If the displayed FQDN is not correct, you can change it by
+clicking on the ``View FQDN Change Instructions`` button.
+This will open a dialog with instructions on how to change
+the FQDN of your server. Once you have changed the FQDN,
+you can continue with the installation.
+
+.. figure:: ../images/setup_nextronInstaller-fqdn.png
+   :alt: FQDN Verification of the Universal Installer
+
+Proxy and NTP Settings
+----------------------
+
+If you need to configure a proxy or change the NTP settings
+of your system, you can do so by clicking on the ``Settings``
+button in the left menu of the Nextron Universal Installer.
+
+.. figure:: ../images/setup_nextronInstaller-settings.png
+   :alt: Settings of the Universal Installer
+
+If you configured a proxy during the ISO installation, those
+settings will be carried over into the Universal Installer.
+The settings will also be carried over into your ASGARD
+Management Center. The same goes for NTP.
+
+Diagnostic Pack
+---------------
+
+In case of errors or problems during the installation, you can
+download a diagnostic pack by navigating to the ``Diagnostics``
+tab in the left menu of the Nextron Universal Installer. Click
+on the ``Download Diagnostic Pack`` button to download the
+diagnostic pack. You can then send the diagnostic pack to our
+support team for further analysis.
+
+.. figure:: ../images/setup_nextronInstaller-diagnostics.png
+   :alt: Diagnostics of the Universal Installer

setup/index.rst

@@ -1,37 +1,38 @@
-.. index:: Setup
+.. index:: Analysis Cockpit Setup
 
 Analysis Cockpit Setup
 ======================
 
+.. warning::
+    The Cluster Installation is currently not possible due to
+    the update of version 4 of the Analysis Cockpit. We will soon
+    release a new ISO which will fix this.
+
 Prerequisites
 ~~~~~~~~~~~~~
 
 Elasticsearch Cluster setup requires:
 
-* A fully functional installation of Analysis Cockpit version 4.x
-
-* At least two additional nodes with a similar high-end spec
-
-* High-performance low-latency networking between all nodes
+- A fully functional installation of Analysis Cockpit version 4.x
+- At least two additional nodes with a similar high-end spec
+- High-performance low-latency networking between all nodes
 
 Analysis Cockpit preparation
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 After installation, the Analysis Cockpit runs with a single
 local Elasticsearch instance as usual. To prepare it for use with
-a cluster, run ``es-cluster-install.sh``:
+a cluster, run ``es-cluster-setup.sh``:
 
 .. code-block:: console
 
     nextron@cockpit4:~$ sudo /usr/share/asgard-analysis-cockpit/scripts/es-cluster-setup.sh
 
 The script will configure Elasticsearch in the following way:
 
-* The Analysis Cockpit node continues to be the master node but data is automatically moved away from it once possible.
-
-* SSL certificates are used for authentication of nodes.
-
-* Any number of data nodes can be added with exactly the same configuration and certificate (as long as they are reachable).
+- The Analysis Cockpit node continues to be the master node but data is automatically moved away from it once possible.
+- SSL certificates are used for authentication of nodes.
+- Any number of data nodes can be added with exactly the same configuration and certificate (as long as they are reachable).
 
 Resulting Elasticsearch configuration
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -64,19 +65,20 @@ It will look like the following:
 
 The configuration:
 
-* designates the Analysis Cockpit node as the (only) cluster master.
+- Designates the Analysis Cockpit node as the (only) cluster master.
+- Automatically moves existing data away from the Analysis Cockpit node, and distributes it across the other nodes.
+- TLS security is enabled so that nodes authenticate by certificate.
 
-* automatically moves existing data away from the Analysis Cockpit node, and distributes it across the other nodes.
-
-* TLS security is enabled so that nodes authenticate by certificate.
-
-
-Cluster Node configuration script
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Cluster Node configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 In addition to reconfiguring the Analysis Cockpit, ``es-cluster-setup.sh`` will
-create a script es-node-install.sh that contains the required configuration for
-additional nodes to join the cluster.
+create a configuration file ``clusternode.conf`` which contains the required
+configuration for additional nodes to join the cluster. The file can be found
+on your Analysis Cockpit in ``/usr/share/asgard-analysis-cockpit/scripts/clusternode.conf``.
+
+Download this configuration file for further usage in our Nextron
+Universal Installer
 
 Restarting Elasticsearch
 ~~~~~~~~~~~~~~~~~~~~~~~~
@@ -87,3 +89,4 @@ Finally, restart elasticsearch so that it picks up the new configuration:
 
     nextron@cockpit3:~$ sudo systemctl restart elasticsearch
 
+Your Analysis Cockpit is now ready to be used in a cluster setup.