Skip to content

[crowdstrike,m365_defender,microsoft_defender_endpoint,trend_micro_vision_one] map source-specific fields to process.* ECS fields #13984

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 27, 2025

Conversation

mohitjha-elastic
Copy link
Contributor

@mohitjha-elastic mohitjha-elastic commented May 23, 2025

Proposed Commit Message

crowdstrike, m365_defender, microsoft_defender_endpoint, trend_micro_vision_one: map
source-specific fields to `process.*` ECS fields.

* crowdstrike: add process.entity_id and process.parent.entity_id ECS
  mappings in alert and falcon data streams.
* m365_defender: add process.entity_id and process.parent.entity_id ECS
  mappings in all data streams.
* microsoft_defender_endpoint: add process.entity_id and
  process.parent.entity_id ECS mappings in log data stream.
* trend_micro_vision_one: add process.entity_id ECS mapping in detection
  data stream.

These mappings align the data streams with ECS standards and ensure
consistency across ingested data, supporting reliable analyzer
development.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

To test integration

Clone integrations repo.
Install the elastic package locally.
Start the elastic stack using the elastic package.
Move to integrations/packages/<integration_name> directory.
Run the following command to run tests.
elastic-package test -v

NOTE: To test individual integrations, replace <integration_name> with one of the following: crowdstrike, m365_defender, microsoft_defender_endpoint, or trend_micro_vision_one.

Related issues

@mohitjha-elastic mohitjha-elastic self-assigned this May 23, 2025
@mohitjha-elastic mohitjha-elastic requested a review from a team as a code owner May 23, 2025 11:39
@mohitjha-elastic mohitjha-elastic added enhancement New feature or request Integration:crowdstrike CrowdStrike Integration:microsoft_defender_endpoint Microsoft Defender for Endpoint Integration:m365_defender Microsoft M365 Defender Integration:trend_micro_vision_one Trend Micro Vision One Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] Team:Sit-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors] labels May 23, 2025
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

…add_ecs_for_data_analyzer

Conflicts:
	packages/crowdstrike/changelog.yml
	packages/crowdstrike/data_stream/alert/sample_event.json
	packages/crowdstrike/data_stream/falcon/sample_event.json
	packages/crowdstrike/docs/README.md
	packages/m365_defender/changelog.yml
	packages/m365_defender/data_stream/alert/sample_event.json
	packages/m365_defender/data_stream/incident/sample_event.json
	packages/m365_defender/docs/README.md
	packages/microsoft_defender_endpoint/changelog.yml
	packages/microsoft_defender_endpoint/data_stream/log/sample_event.json
	packages/microsoft_defender_endpoint/docs/README.md
@elastic-vault-github-plugin-prod

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@efd6 efd6 changed the title [crowdstrike/m365_defender/microsoft_defender_endpoint/trend_micro_vision_one] Map Process.* ECS mappings to Ensure Consistency for Data Analyzer [crowdstrike,m365_defender,microsoft_defender_endpoint,trend_micro_vision_one] Map Process.* ECS mappings to Ensure Consistency for Data Analyzer May 25, 2025
Comment on lines 298 to 309
- convert:
field: process.pid
tag: convert_process_pid_to_string
target_field: process.entity_id
type: string
ignore_missing: true
- convert:
field: process.parent.pid
tag: convert_process_parent_pid_to_string
target_field: process.parent.entity_id
type: string
ignore_missing: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave as numbers and note numeric keyword in the test definition.

Comment on lines 110 to 121
- convert:
field: process.pid
tag: convert_process_pid_to_string
target_field: process.entity_id
type: string
ignore_missing: true
- convert:
field: process.parent.pid
tag: convert_process_parent_pid_to_string
target_field: process.parent.entity_id
type: string
ignore_missing: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave as numbers.

@@ -2237,6 +2237,46 @@ processors:
if (!processParentPid.isEmpty()) {
ctx.process.parent.pid = convertToOrderedArray(processParentPid);
}
- foreach:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave as numbers.

Comment on lines 237 to 248
- convert:
field: process.pid
tag: convert_process_pid_to_string
target_field: process.entity_id
type: string
ignore_missing: true
- convert:
field: process.parent.pid
tag: convert_process_parent_pid_to_string
target_field: process.parent.entity_id
type: string
ignore_missing: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave as numbers.

@@ -295,6 +295,11 @@ processors:
field: process.pid
copy_from: trend_micro_vision_one.detection.process.pid
ignore_failure: true
- convert:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave as a number.

@mohitjha-elastic mohitjha-elastic requested a review from efd6 May 26, 2025 12:44
Copy link

@elasticmachine
Copy link

💚 Build Succeeded

History

cc @mohitjha-elastic

@efd6 efd6 changed the title [crowdstrike,m365_defender,microsoft_defender_endpoint,trend_micro_vision_one] Map Process.* ECS mappings to Ensure Consistency for Data Analyzer [crowdstrike,m365_defender,microsoft_defender_endpoint,trend_micro_vision_one] map source-specific fields to process.* ECS fields May 26, 2025
Copy link
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest commit message body:

* crowdstrike: add process.entity_id and process.parent.entity_id ECS
  mappings in alert and falcon data streams.
* m365_defender: add process.entity_id and process.parent.entity_id ECS
  mappings in all data streams.
* microsoft_defender_endpoint: add process.entity_id and
  process.parent.entity_id ECS mappings in log data stream.
* trend_micro_vision_one: add process.entity_id ECS mapping in detection
  data stream.

These mappings align the data streams with ECS standards and ensure
consistency across ingested data, supporting reliable analyzer
development.

@efd6 efd6 merged commit 4ecefbb into elastic:main May 27, 2025
8 checks passed
@elastic-vault-github-plugin-prod

Package crowdstrike - 1.71.0 containing this change is available at https://epr.elastic.co/package/crowdstrike/1.71.0/

@elastic-vault-github-plugin-prod

Package m365_defender - 3.6.0 containing this change is available at https://epr.elastic.co/package/m365_defender/3.6.0/

@elastic-vault-github-plugin-prod

Package microsoft_defender_endpoint - 2.36.0 containing this change is available at https://epr.elastic.co/package/microsoft_defender_endpoint/2.36.0/

@elastic-vault-github-plugin-prod

Package trend_micro_vision_one - 2.1.0 containing this change is available at https://epr.elastic.co/package/trend_micro_vision_one/2.1.0/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Integration:crowdstrike CrowdStrike Integration:m365_defender Microsoft M365 Defender Integration:microsoft_defender_endpoint Microsoft Defender for Endpoint Integration:trend_micro_vision_one Trend Micro Vision One Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] Team:Sit-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ensure Consistency Across Ingested Data for Analyzer Development
3 participants