Using nDPId with Filebeat and Elasticsearch (no Logstash required)

[GhostNaix]

Using the following configs in filebeat allows information from nDPId to be stored in searchable form in elasticsearch

filebeat.inputs:
- type: unix
  id: "NDPId-logs" # replace this as well to your liking
  max_message_size: 100MiB
  index: "index-name" # Replace this with your desired index name in Elasticsearch
  enabled: true
  path: "/var/run/nDPId.sock" # point nDPId to this Unix Socket
  processors:
    - script: # execute javascript to remove the first 5-digit-number and also the Newline at the end
        lang: javascript
        id: trim
        source: >
          function process(event) {
            event.Put("message", event.Get("message").trim().slice(5)); 
          }
    - decode_json_fields: # Decode the Json output
        fields: ["message"]
        process_array: true
        max_depth: 10
        target: ""
        overwrite_keys: true
        add_error_key: false
     - drop_fields: # Deletes the Message field, which is the undecoded json (You may comment this out if you need the original message)
           fields: ["message"] 
     - rename:
           fields:
             - from: "source" # Prevents a conflict in Elasticsearch and renames the field
                to: "Source_Interface" 

With this config you can now send information from nDPId to Elasticsearch and thus draw graphs with kibana as well as explore the data.

That's all from me.

[utoni]

Do you think it makes sense to include the config file in the repository?
If so, would you like to create a PR?

[GhostNaix]

I believe so as it was pretty difficult finding information to get this to work with Elasticsearch, I can include a fully commented filebeat config if you so wish in the examples directory of the repo in the PR or do you just want me to include this part in the examples directory?.

[utoni]

Thanks! A PR would be awesome!
I think it makes sense to include this pretty neat config below the examples directory, so others may use this easily.

[GhostNaix]

PR request has been sent.