Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[8.x] [Entity Analytics] [Entity Store] [API] Changes to support event.ingested as a configurable timestamp field for init and enable endpoints (#208201) #208866

Merged
merged 1 commit into from
Jan 30, 2025
Merged

[8.x] [Entity Analytics] [Entity Store] [API] Changes to support event.ingested as a configurable timestamp field for init and enable endpoints (#208201) #208866

merged 1 commit into from
Jan 30, 2025

Conversation

jaredburgettelastic
Copy link
Contributor

@abhishekbhatia1710
[Entity Analytics] [Entity Store] [API] Changes to support event.inge…
fe37164 
…sted as a configurable timestamp field for init and enable endpoints (elastic#208201)

## Summary

This PR introduces support for configuring the `timestamp` field for
entity store enablement.

By default, the `timestamp` field is set to `@timestamp`, but users can
opt to use `event.ingested` or another preferred value based on their
requirements.

### Entity Store API changes

#### Entity Store enable

```
POST kbn:/api/entity_store/enable
{
  "timestampField": "event.ingested"
}
```

#### Result

```
{
  "engines": [
    {
      "status": "started",
      "type": "user",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "event.ingested"
    },
    {
      "status": "started",
      "type": "host",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "event.ingested"
    },
    {
      "status": "started",
      "type": "universal",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "event.ingested"
    }
  ],
  "status": "running"
}
```

```
POST kbn:/api/entity_store/enable
{
}
```
#### Result

```
{
  "engines": [
    {
      "status": "started",
      "type": "host",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "@timestamp"
    },
    {
      "status": "started",
      "type": "user",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "@timestamp"
    },
    {
      "status": "started",
      "type": "universal",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "@timestamp"
    }
  ],
  "status": "running"
}
```

Different entity types can have distinct `timestampField` values, as
described below:

For instance, the `host` entity uses `event.ingested` as its
`timestampField`, while other entities default to the `@timestamp`
field.

```
POST kbn:/api/entity_store/engines/host/init
{
  "timestampField": "event.ingested"
}

POST kbn:/api/entity_store/engines/user/init
{
}
```

#### Result

```
{
  "engines": [
    {
      "status": "started",
      "type": "universal",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "@timestamp"
    },
    {
      "status": "started",
      "type": "host",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "event.ingested"
    },
    {
      "status": "started",
      "type": "user",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "@timestamp"
    }
  ],
  "status": "running"
}
```

### Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

### Testing steps

- Check out this PR branch.
- Start Kibana.
- Avoid enabling the Entity Store from the UI initially. Instead, enable
it via the API by following the steps outlined below.

```
DELETE kbn:/api/entity_store/engines/user
DELETE kbn:/api/entity_store/engines/host

GET kbn:/api/entity_store/status

 All engines with the same timestamp Field
POST kbn:/api/entity_store/enable
{
  "timestampField": "event.ingested"
}

# All engines with the same timestamp Field
POST kbn:/api/entity_store/enable
{
}

# Different entity with different timestamp Field
POST kbn:/api/entity_store/engines/host/init
{
  "timestampField": "event.ingested"
}

POST kbn:/api/entity_store/engines/user/init
{
}
```

4. Afterward, you can test the functionality through the UI. First,
clear the entity data from the UI, then start the Entity Store. Note
that the UI currently does not offer a way to configure this parameter.

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
Co-authored-by: Mark Hopkin <[email protected]>
(cherry picked from commit a060bae)
@jaredburgettelastic jaredburgettelastic enabled auto-merge (squash) January 30, 2025 00:11
@jaredburgettelastic jaredburgettelastic mentioned this pull request Jan 30, 2025
Merged
3 tasks
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 18.6MB 18.6MB +366.0B

@jaredburgettelastic jaredburgettelastic merged commit ad15e7d into elastic:8.x Jan 30, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@abhishekbhatia1710 abhishekbhatia1710 abhishekbhatia1710 approved these changes

@kibanamachine kibanamachine Awaiting requested review from kibanamachine kibanamachine is a code owner

Assignees
No one assigned
Labels
backport
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jaredburgettelastic @elasticmachine @abhishekbhatia1710