This repo is used for building and running Cribl demos. All demos contained within utilize Docker. To get started, all you need to do is:
git clone https://https://github.com/criblio/cribl-demo.git
cd cribl-demo
docker-compose up -d
This will launch a Cribl demo environment with a number of sources and destinations. Several of these systems expose their own interfaces, like Cribl, Splunk and Elasticsearch (Kibana):
| System | URL | Username | Password |
|---|---|---|---|
| Cribl | http://localhost:9000/login?username=admin&password=cribldemo | admin | cribldemo |
| Splunk | https://localhost:8000/en-US/account/insecurelogin?loginType=splunk&username=admin&password=cribldemo- | admin | cribldemo |
| Elasticsearch (Kibana) | http://localhost:9200 | ||
| Grafana | http://localhost:8200 | admin | cribldemo |
| Graphite | http://localhost:8100 |
The Cribl UI shows you all the sources, routes and pipelines and will give you a good overview of the types of data flowing in real time.
The Splunk environment contains a number of dashboards which shows off use cases for Cribl. It's a nice overview of the capabilities, and it contains easy links to the pipelines which are reshaping the data.
Data for this demo comes from two sources: Gogen and Filebeat. Gogen generates data through a number of different protocols, like HTTP, Splunk Universal Forwarder, TCP JSON, Syslog, and onto a Kafka bus. Cribl is configured to receive or pull from all of those particular protocols. Gogen is configured to generate fake data like Weblogs, Transaction logs, etc. It will backfill one hour's worth of data on startup, which you will see as a spike in the graphs. Secondly, Filebeat is configured to grab logs from Docker.
Gogen
`- HTTP -> cribl:10001
`- Splunk Universal Forwarder -> cribl:9999
`- TCP -> cribl:10001
`- Syslog -> cribl:10003
`- Kafka - topic cribl
`- Dogstatsd -> cribl:8125
`- Cribl
`- S2S -> splunk:9997
`- Elastic Bulk Ingestion -> elastic:9200
`- S3 -> minio:80
`- Graphite -> graphite:2003
On the output side, Cribl is outputting to Splunk, Elasticsearch, Graphite, and S3 (Minio). Data can be found in the following locations:
| System | Data Location |
|---|---|
| Splunk | index=cribl |
| Splunk | index=cribl-modified |
| Elasticsearch | filebeat-* |
| Elasticsearch | bigjson |
| Elasticsearch | bigjson-trimmed |
| Minio | ./data |
Stop the demo through docker-compose:
docker-compose down
We use Elastic Filebeat to pick up logs from the docker container. This may require you to run docker as root in order to access /var/run/docker.sock. In that case you may need to run sudo docker-compose up -d to run the demo.
If you have docker in a non-standard location, we may need to find a different root directory. If Filebeat still isn't picking up logs, you can try running: DOCKER_LIB_CONTAINERS=$(docker info -f '{{.DockerRootDir}}')/containers && sudo DOCKER_LIB_CONTAINERS=${DOCKER_LIB_CONTAINERS} docker-compose up -d.