Clarification of fowarding logs for ELK Lab 2.3

[grizzlycode]

Description:

I recently revisited the FOR608 course material and wanted to implement the method for forwarding logs to SOF-ELK using Winlogbeat. However, I encountered a discrepancy between the course's provided configuration and the version of Winlogbeat (8.8.1) I'm using for SOF-ELK.

The course specifies the following path for the Winlogbeat configuration file:

file: ${path.home}/module/security/config/winlogbeat-security.js

But in my current Winlogbeat installation, that filepath and file doesn't exist. However, I do see this file located at:

/module/security/ingest/security.yml

A quick comparison suggests that both files serve a similar purpose?

Questions:

Is security.yml simply a replacement for the older winlogbeat-security.js file?
If so, are there any necessary syntax changes for the configuration file to ensure compatibility with the new YAML format?

Additional Information:

Winlogbeat version: 8.8.1
SOF-ELK version: 8.8.1
FOR608 course material reference: Lab 2.3
Expected Behavior:

Clarification on the configuration file path and any required syntax updates for seamless integration with the latest Winlogbeat versions.

[mpilking]

Hi @grizzlycode ,

Good question about the latest version of Winlogbeat and how their config may have changed. I haven't used the latest version. We were sticking with 7.10.2 mainly because it's the last Apache 2.0 version and still the recommended version for OpenSearch too. But I should look into trying it with the latest. Give me a day or so to test it out and report back.

FYI, this general topic came up in a recent SOF-ELK issue/discussion here: philhagen/sof-elk#332. In that issue, I showed an example of forwarding event logs with Winlogbeat 7.10.2 into the latest version of SOF-ELK.

Thanks,
Mike

[mpilking]

Hi @grizzlycode ,

To answer your question...

Is security.yml simply a replacement for the older winlogbeat-security.js file?
If so, are there any necessary syntax changes for the configuration file to ensure compatibility with the new YAML format?

Yes, it is a replacement. Elastic changed the module configuration files from JavaScript to YAML in version 7.12.0 for consistency purposes. I tested it out and there doesn't seem to be a need to specify the module file specifically within the configuration file. It works just fine without it. I sent Security event logs to the latest SOF-ELK (Elasticsearch version 8.8.1) and the events show up parsed out as expected. So, you should be able to run the command to setup the dashboards and then the command to ingest the logs just as it shows in the 608 lab if you use this modified version of the winlogbeat-seurity-archive-evtx.yml config file:

# This is a basic configuration file to forward Windows security EVTX files
# directly to Elasticsearch. It uses Winlogbeat's included parsing processor
# script to extract out additional useful fields from the security event logs. 

# This config is based on documentation from Elastic for reading in EVTX files:
# https://www.elastic.co/guide/en/beats/winlogbeat/master/reading-from-evtx.html

# Review Winlogbeat's included "winlogbeat-reference.yml" for very detailed
# description of these configuration options and MANY more.

# To run this against multiple EVTX files, use a FOR loop such as the following.
# This will go recursively through G:\Elastic\Winlogbeat\logs\ looking for any 
# .evtx files and feed them into the command starting with ".\winlogbeat.exe..."

# for /r G:\Elastic\Winlogbeat\logs\ %f in (*.evtx) do .\winlogbeat.exe -e -c .\winlogbeat-security-archive-evtx.yml -E EVTX_FILE=%f

winlogbeat.event_logs:
  - name: ${EVTX_FILE}
    no_more_events: stop 

winlogbeat.shutdown_timeout: 30s 

winlogbeat.registry_file: archive-security-evtx-registry.yml

output.elasticsearch.hosts: ['127.0.0.1:9200']

[grizzlycode]

Thank you for investigating this issue. While the provided configuration appears to be functional, I've encountered an anomaly where the process doesn't stop after log ingestion and begins sending Windows metrics. I'll investigate this separately. Thanks again for your support!