This composition configures fluent-bit to read out memory and CPU metrics, transform them and send them to Splunk via the HTTP Event Collector (HEC). The main Splunk instance contains an example dashboard displaying the incoming metrics.
End-to-end data pipeline visibility is provided by exposing metrics in Prometheus format for fluent-bit, envoy and the Splunk forwarders.
- Backpressure-sensitive heartbeat
- Transport to remote Splunk platform over HTTP
- Prometheus metrics for fluent-bit, envoy and splunkforwarder
- Grafana dashboards
- v1.2.0 - Replace nginx with envoy
- v1.1.0 - Replace Splunk UF with nginx, transport events over HTTP to HEC
- v1.0.0 - Ship HEC-ready events to Splunk UF, transport events via Splunk2Splunk TCP
Bring up the containers by running,
docker-compose up
To bring down and clean up the containers run,
docker-compose down
| URL | Username | Password | |
|---|---|---|---|
| Splunk UI | http://localhost:8000/ | admin | admin |
| Prometheus | http://localhost:9090/ | admin | admin |
| Grafana | http://localhost:3000/ | admin | admin |
| Envoy | http://localhost:8001/ | N/A | N/A |
| mtail | http://localhost:3903/ | N/A | N/A |
| mtail | http://localhost:3904/ | N/A | N/A |
This docker-compose image uses,
- A public, official splunk enterprise image
- A public, official splunk enterprise universalforwarder image
- A public, official fluentbit image
- A public, official prometheus image
- A public, official grafana image
- A public, official envoy image
- A public, unofficial mtail image
The Splunk HF, IDX and SHC components are all run by the main splunk image.
Main fluent-bit.conf
It uses the fluent-bit http output plugin with the plugin's Basic
Authentication. See Format events for HTTP Event Collector
for the Splunk HEC documentation.
For a shell on the containers, run the commands below.
./script/shell-splunk.sh
./script/shell-splunkforwarder01.sh
./script/shell-splunkforwarder02.sh
./script/shell-fluentbit.sh
./script/shell-prometheus.sh
./script/shell-grafana.sh
./script/shell-mtail01.sh
./script/shell-mtail02.sh
To test the HEC, expose a forwarder port 8088 on localhost and run the cURL command,
curl -u 'x:3e6ffd12-0f69-46bb-ad0d-71cffb661a0d' -X POST -d'
{
"event": {
"event_key1": "value1",
"event_key1": "value1"
},
"fields": {
"field_key1": "value1",
"field_key1": "value1"
}
}' http://localhost:8088/services/collector/event
If the interface is up and running, The expected response is,
{"text":"Success","code":0}