docs: Edge Processor

docs/edge_processor.md

@@ -0,0 +1,110 @@
+# Edge Processor integration guide
+
+## Intro
+
+`Edge Processor` can be used on that usecases:
+
+* Enrich log message extra data (for example add some field or override index) using `SPL2`
+* Filter log message using `SPL2`
+* Send log messages to alternative destanations (like `AWS S3`, `Apache Kafka`, etc.)
+
+## How it's working
+
+```mermaid
+stateDiagram
+    direction LR
+
+    SC4S: SC4S
+    EP: Edge Processor
+    Dest: Another destination
+    Device: Your device
+    S3: AWS S3
+    Instance: Instance
+    Pipeline: Pipeline with SPL2
+
+    Device --> SC4S: Syslog protocol
+    SC4S --> EP: HEC
+    state EP {
+      direction LR
+      Instance --> Pipeline
+    }
+    EP --> Splunk
+    EP --> S3
+    EP --> Dest
+```
+
+## Basic Setup
+
+### Docker / Podman
+
+Setup on your `env_file` HEC URL as IP of EP worker EC2 instance.
+Token you can find in EP "global settings" page.
+
+```
+SC4S_DEST_SPLUNK_HEC_{EP1}_URL=http://x.x.x.x:8088
+SC4S_DEST_SPLUNK_HEC_{EP1}_TOKEN=secret
+```
+
+### Kubernetes
+
+Setup on your `values.yaml` HEC URL as IP of EP worker EC2 instance.
+Token you can find in EP "global settings"page.
+
+```
+splunk:
+  hec_url: "https://x.x.x.x:8088"
+  hec_token: "secret"
+```
+
+## mTLS encryption
+
+### Preparing certs
+
+Before setup you need to [generate mTLS certificates](https://docs.splunk.com/Documentation/SplunkCloud/9.1.2308/EdgeProcessor/SecureForwarders). Server mTLS certificates should be uploaded to `Edge Processor` and client certifcates should be used with `SC4S`.
+
+Please rename your files, we expcting such filenames for client mTLS cerificates:
+
+  * `key.pem` - client certificate key
+  * `cert.pem` - client certificate
+  * `ca_cert.pem` - certificate authority
+
+### Docker / Podman
+
+  1. Use HTTPS in HEC url: `SC4S_DEST_SPLUNK_HEC_DEFAULT_URL=https://x.x.x.x:8088`
+  2. Move your clients mTLS certificates to `/opt/sc4s/tls/hec`
+  3. Mount `/opt/sc4s/tls/hec` to `/etc/syslog-ng/tls/hec` using docker/podman volumes.
+  4. Define mounting mTLS point for HEC: `SC4S_DEST_SPLUNK_HEC_DEFAULT_TLS_MOUNT=/etc/syslog-ng/tls/hec`
+  5. Start/Restart SC4S
+
+### Kubernetes
+
+  1. Add secret name of mTLS certs at `values.yaml`:
+
+```
+splunk:
+  hec_url: "https://x.x.x.x:8088"
+  hec_token: "secret"
+  hec_tls: "secret-name"
+```
+
+  2. Add your mtls certs at `secrets.yaml`:
+
+```
+hec_tls:
+  key: |
+    -----BEGIN PRIVATE KEY-----
+    Exmaple key
+    -----END PRIVATE KEY-----
+  cert: |
+    -----BEGIN CERTIFICATE-----
+    Exmaple cert
+    -----END CERTIFICATE-----
+  ca: |
+    -----BEGIN CERTIFICATE-----
+    Example ca
+    -----END CERTIFICATE-----
+```
+
+  3. Encrypt your `secrets.yaml` using `ansible-vault`.
+  4. Add IP of cluster nodes to inventory file `ansible/inventory/inventory_microk8s_ha.yaml`
+  5. Deploy ansible playbook `ansible/playbooks/microk8s_ha.yml`

docs/gettingstarted/ansible-docker-podman.md

@@ -1,29 +1,33 @@
-## Notice
-SC4S installation can now be automated with Ansible. All you need to do now is provide a host on which you want to run SC4S and basic configuration (Splunk endpoint, HEC token, TLS configuration, etc.).
+## About automating SC4S installation with Ansible
+You can use Ansible to automate SC4S installation. To do this you provide the host on which you want to run SC4S and some basic configuration, including:
+* The Splunk endpoint
+* The HEC token
+* TLS configuration
 
 # Initial Configuration
 
-All you need to do before running sc4s with Ansible is providing `env_file`. In the env file provide at least proper Splunk endpoint and HEC token.
-Create a file in `ansible/resources` catalog or edit [example file](/ansible/resources/env_file).
+Before you run SC4S with Ansible provide the `env_file` with the Splunk endpoint and HEC token.
+
+1. Create a file in `ansible/resources` catalog or edit [example file](/ansible/resources/env_file).
 
 ``` dotenv
 --8<---- "ansible/resources/env_file"
 ```
-Next provide a host on which you want to run Docker Swarm cluster and host application in inventory file:
+2. Provide a host on which you want to run Docker Swarm cluster and host application in the inventory file:
 ``` yaml
 --8<---- "ansible/inventory/inventory.yaml"
 ```
 ## Deploy SC4S
-Now you can run ansible playbook to deploy the application if you have ansible installed on your host
-or use docker ansible image provided in the package:
+1. If you have ansible installed on your host, run Ansible playbook to deploy the application. Otherwise you 
+can use the Docker Ansible image provided in the following package:
 ```bash
 # From repository root
 docker-compose -f ansible/docker-compose.yml build
 docker-compose -f ansible/docker-compose.yml up -d
 docker exec -it ansible_sc4s /bin/bash
 ```
-Once you are in containers remote shell you can run Docker Swam ansible playbook.
-If you are authenticating via username/password:
+2. Once you are in the container's remote shell, run the Docker Swam Ansible playbook.
+To authenticate using username/password:
 ``` bash 
 ansible-playbook -i path/to/inventory.yaml -u <username> --ask-pass path/to/playbooks/docker.yml
 or
@@ -39,9 +43,9 @@ ansible-playbook -i path/to/inventory.yaml -u <username> --key-file <key_file> p
 
 # Verify Proper Operation
 
-SC4S has a number of "preflight" checks to ensure that the container starts properly and that the syntax of the underlying syslog-ng
-configuration is correct.  After this step completes, to verify SC4S is properly communicating with Splunk,
-execute the following search in Splunk:
+SC4S performs a number of "preflight" checks to ensure that the container starts properly and that the syntax of the underlying syslog-ng configuration is correct.  
+
+After the checks are complete, execute the following search in Splunk to verify that SC4S is properly communicating with Splunk:
 
 ```ini
 index=* sourcetype=sc4s:events "starting up"
@@ -52,19 +56,18 @@ This should yield an event similar to the following:
 ```ini
 syslog-ng starting up; version='3.28.1'
 ```
-You can verify if all services in swarm cluster are working by checking ```sc4s_container``` field in splunk- each service should be recognized by different container id. All other fields should be the same.
+To verify that all services in a Swarm cluster are working, check the ```sc4s_container``` field in Splunk. Each service should be recognized by a different container ID. All other fields should be the same.
 
-When the startup process proceeds normally (without syntax errors). If you do not see this,
-follow the steps below before proceeding to deeper-level troubleshooting:
+The startup process then proceeds normally without syntax errors. If it does not, try the following:
 
 * Check to see that the URL, token, and TLS/SSL settings are correct, and that the appropriate firewall ports are open (8088 or 443).
-* Check to see that the proper indexes are created in Splunk, and that the token has access to them.
+* Check to see that the proper indexes are created in Splunk, and that the token has access to the indexes.
 * Ensure the proper operation of the load balancer if used.
-* Lastly, execute the following command to check the sc4s startup process running in the container (on the node that is hosting sc4s service).
+* Execute the following command to check the SC4S startup process running in the container on the node that is hosting the SC4S service:
 ```bash
 sudo docker ps
 ```
-You will get an ID and <image name>, next: 
+You will get an ID and <image name>. Next, execute: 
 
 ```bash
 docker logs <ID | image name> 
@@ -82,4 +85,4 @@ syslog-ng checking config
 sc4s version=v1.36.0
 starting goss
 starting syslog-ng
-```
+```

mkdocs.yml

@@ -14,6 +14,11 @@ markdown_extensions:
   - sane_lists
   - codehilite
   - pymdownx.snippets
+  - pymdownx.superfences:
+      custom_fences:
+        - name: mermaid
+          class: mermaid
+          format: !!python/name:pymdownx.superfences.fence_code_format
 
 
 theme:
@@ -57,6 +62,7 @@ nav:
   - SC4S Lite (Experimental):
       - Intro: "lite.md"
       - Pluggable modules: "pluggable_modules.md"
+  - Edge Processor: "edge_processor.md"
   - Troubleshooting:
       - SC4S Startup and Validation: "troubleshooting/troubleshoot_SC4S_server.md"
       - SC4S Logging and Troubleshooting Resources: "troubleshooting/troubleshoot_resources.md"