[Security Solution] Fix old siem feature override #207333

semd · 2025-01-21T11:42:13Z

Summary

Adds the feature override for the old siem feature as well, we changed that to the new one here

https://github.com/elastic/kibana/pull/201780/files#diff-5aba630e58630c087c90368aa97296afb736f62579a23285cef901dc1c3921edR27

Related failure: #207285

The problem happened because MKI tests are using the outdated roles definition with the old feature_siem which was lacking the feature override in the serverless.security.yml

elasticmachine · 2025-01-21T11:42:16Z

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

janmonschke · 2025-01-21T11:53:01Z

I'm trying to make sense of this file and the failing tests. From my understanding, all granted privileges should be defined in their BaseKibanaFeatureConfig. For security/siem that would be in https://github.com/elastic/kibana/blob/8b37d12233910593e3bc5edadc7ee413aa754199/x-pack/solutions/security/packages/features/src/security/v1_features/kibana_features.ts . However, there's no mentioning of discover in those definitions. So these privilege overrides cannot be handled by the replacedBy privilege feature since there's no knowledge of those.

I wonder what was the reasoning of adding these feature overrides in the first place?

semd · 2025-01-21T12:06:04Z

@elasticmachine merge upstream

…urity

semd · 2025-01-21T12:31:40Z

I wonder what was the reasoning of adding these feature overrides in the first place?

@janmonschke In Security serverless projects, the platform features (aka Analytics) are "shared", they exist in all the different types of projects (solutions).
These privileges are registered, but they are not granted explicitly by the solution-specific roles, you can see that on the custom role creation page, we only have Security and Management:

These platform features are granted implicitly when a role has a solution-specific feature, this is defined by these overrides. Essentially, this config is telling Kibana that when a role has siemV2 privilege, it should also be granted the discover, dashboards... privileges automatically.

The problem is that MKI is still executing the tests with the siem privilege, not siemV2, and we changed the feature override in Kibana from siem to siemV2. So the problem is solved just by bringing back the old override (duplicating).

If you want to test the bug locally you can change the local roles definition x-pack/test_serverless/shared/lib/security/kibana_roles/project_controller_security_roles.yml and use feature_siem instead of feature_siemV2 and start serverless locally.

The only remaining question I have is that these predefined roles do have explicit platform privileges granted like feature_discover and feature_dashboard, but they seem to be ignored.

cc @azasypkin

azasypkin · 2025-01-21T12:42:07Z

The only remaining question I have is that these predefined roles do have explicit platform privileges granted like feature_discover and feature_dashboard, but they seem to be ignored.

Sorry, I’m slightly out of context, but this does sound weird. Is it “just” the test that’s failing (there are too many variables in tests to pinpoint the culprit), or is there some real Discover/Dashboard functionality that isn’t available to users with the aforementioned predefined roles?

semd · 2025-01-21T12:51:45Z

The only remaining question I have is that these predefined roles do have explicit platform privileges granted like feature_discover and feature_dashboard, but they seem to be ignored.

Sorry, I’m slightly out of context, but this does sound weird. Is it “just” the test that’s failing (there are too many variables in tests to pinpoint the culprit), or is there some real Discover/Dashboard functionality that isn’t available to users with the aforementioned predefined roles?

The latter, features can not be accessed:

It happens for all the analytics features that we override (discover, dashboard, visualize, maps).

To reproduce it:

pull main to include the last role migration
Edit your Kibana security roles definition here and replace all siemV2 by siem.
Start Kibana locally in serverless=security mode, and go to /app/discover

azasypkin · 2025-01-21T12:55:55Z

Start Kibana locally in serverless=security mode, and go to /app/discover

@semd how do you run Kibana and ES, and what role do you use to log in? I assume this file (https://github.com/elastic/kibana/blob/main/x-pack/test_serverless/shared/lib/security/kibana_roles/project_controller_security_roles.yml) is only used when you run FTR locally, for usual dev server Kibana is supposed to use https://github.com/elastic/kibana/blob/main/packages/kbn-es/src/serverless_resources/project_roles/security/roles.yml

janmonschke · 2025-01-21T13:04:52Z

config/serverless.security.yml

@@ -50,6 +50,33 @@ xpack.features.overrides:
        - feature: "maps"
          privileges: [ "read" ]

+  ### Security's feature privileges are fine-tuned to grant access to Discover, Dashboard, Maps, and Visualize apps.


Suggested change

### Security's feature privileges are fine-tuned to grant access to Discover, Dashboard, Maps, and Visualize apps.

### We're keeping around a copy of these overrides for the deprecated `siem` feature as well to make sure that they're applied all roles properly

semd · 2025-01-21T13:06:29Z

Start Kibana locally in serverless=security mode, and go to /app/discover

@semd how do you run Kibana and ES, and what role do you use to log in? I assume this file (https://github.com/elastic/kibana/blob/main/x-pack/test_serverless/shared/lib/security/kibana_roles/project_controller_security_roles.yml) is only used when you run FTR locally, for usual dev server Kibana is supposed to use https://github.com/elastic/kibana/blob/main/packages/kbn-es/src/serverless_resources/project_roles/security/roles.yml

@azasypkin

ES -> yarn es serverless --projectType security
kibana -> yarn serverless-security --no-base-path

About the roles.yml duplication, I am not sure what is the purpose of each one, it would be great to centralize this definition. But for now changing project_controller_security_roles.yml should be enough to reproduce the issue in the local Kibana.

The test failure is MKI, which executes UI tests to real QA deployments, so the test uses the production definition at https://github.com/elastic/elasticsearch-controller/blob/main/helm/values.yaml , which still has not been updated with the new features. I am simulating this config locally to reproduce it

elasticmachine · 2025-01-21T13:14:47Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 6d5fe54
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-207333-6d5fe54eae70

Failed CI Steps

Jest Tests #20

Test Failures

[job] [logs] Jest Tests #20 / Case View Page observables tab should render the utility bar for the observables table
[job] [logs] Jest Tests #20 / HoverableAvatar renders the tooltip when hovering

Metrics [docs]

✅ unchanged

History

cc @semd

azasypkin · 2025-01-21T13:17:11Z

But for now changing project_controller_security_roles.yml should be enough to reproduce the issue in the local Kibana.
yarn es serverless --projectType security

I’m pretty sure that project_controller_security_roles.yml isn’t used at all when you run ES that way, but let me double-check.

semd · 2025-01-21T13:30:24Z

But for now changing project_controller_security_roles.yml should be enough to reproduce the issue in the local Kibana.
yarn es serverless --projectType security

I’m pretty sure that project_controller_security_roles.yml isn’t used at all when you run ES that way, but let me double-check.

@azasypkin Yes, you are right, I double checked and the one being used is https://github.com/elastic/kibana/blob/main/packages/kbn-es/src/serverless_resources/project_roles/security/roles.yml, sorry for the confusion, I always do the changes to both places at the same time to make sure they are applied.
Have you been able to reproduce it?

elena-shostak

Confirmed, reproduced the issue and the fix locally ✅

kibanamachine · 2025-01-21T14:51:14Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12889522989

## Summary Adds the feature override for the old `siem` feature as well, we changed that to the new one here https://github.com/elastic/kibana/pull/201780/files#diff-5aba630e58630c087c90368aa97296afb736f62579a23285cef901dc1c3921edR27 Related failure: elastic#207285 The problem happened because MKI tests are using the outdated roles definition with the old `feature_siem` which was lacking the feature override in the serverless.security.yml Co-authored-by: Elastic Machine <[email protected]> (cherry picked from commit 9077414)

kibanamachine · 2025-01-21T14:56:51Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

azasypkin · 2025-01-21T15:47:35Z

Have you been able to reproduce it?

@semd, yes, thanks! And here’s what’s happening. This is what we had in main before this PR:

serverless.security.yml
---
## Fine-tune the security solution feature privileges. Also, refer to `serverless.yml` for the project-agnostic overrides.
xpack.features.overrides:
  ### Discover feature is hidden in Role management since it's automatically granted by SIEM feature.
  discover.hidden: true
...  
### Security's feature privileges are fine-tuned to grant access to Discover, Dashboard, Maps, and Visualize apps.
  siemV2:
    privileges:
      ### Security's `All` feature privilege should implicitly grant `All` access to Discover features.
      all.composedOf:
        - feature: "discover"
          privileges: [ "all" ]

And the excerpt of the role definition:

roles.yaml
---
viewer:
  cluster: []
  indices:
    - names: ...
      privileges:
        - read
  applications:
    - application: 'kibana-.kibana'
      privileges:
        - feature_siemV2.read
        - feature_siemV2.read_alerts
        - feature_siemV2.endpoint_list_read
        - feature_discover.all
        - ...
      resources: '*'

xpack.features.overrides.discover.hidden: true in serverless.security.yml means that Kibana doesn’t register the feature_discover.* privileges at all. Instead, all its actions are added to the actions of feature_siemV2.read. This is why a role that references both feature_discover.all and feature_siem.read privileges still doesn’t grant any Discover-related access - the feature_discover.all privilege doesn’t exist, and there are no overrides for the feature_siem.read privilege to include additional Discover-related actions.

semd · 2025-01-21T15:54:28Z

@azasypkin Got it! Thanks for the explanation. So the xpack.features.overrides.discover.hidden: true is the one causing this (expected) behavior. I knew I was missing something.

This means that all the feature_discover.*, feature_dashboard.*, features_maps.* and feature_visualize.* privileges that we have in our predefined security roles are basically useless, is that correct? If so, do you think it is a good idea to remove them, to prevent confusion?

## Summary Adds the feature override for the old `siem` feature as well, we changed that to the new one here https://github.com/elastic/kibana/pull/201780/files#diff-5aba630e58630c087c90368aa97296afb736f62579a23285cef901dc1c3921edR27 Related failure: elastic#207285 The problem happened because MKI tests are using the outdated roles definition with the old `feature_siem` which was lacking the feature override in the serverless.security.yml Co-authored-by: Elastic Machine <[email protected]>

logeekal · 2025-01-21T16:28:51Z

@semd , I am also trying to read through and understand the issue here.

The problem is that MKI is still executing the tests with the siem privilege, not siemV2, and we changed the feature override in Kibana from siem to siemV2. So the problem is solved just by bringing back the old override (duplicating).

Do you mean that this problem and this solution are temporary till MKI is using updated siemV2 privilege?

…07373) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Fix old siem feature override (#207333)](#207333)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Sergi Massaneda <[email protected]>

azasypkin · 2025-01-22T14:42:37Z

This means that all the feature_discover., feature_dashboard., features_maps.* and feature_visualize.* privileges that we have in our predefined security roles are basically useless, is that correct? If so, do you think it is a good idea to remove them, to prevent confusion?

@semd That’s correct, and yeah, I believe it’s a good idea to remove everything we believe is unused to make it easier to reason about these roles when needed (like in this case).

semd · 2025-01-22T14:51:39Z

@semd , I am also trying to read through and understand the issue here.

The problem is that MKI is still executing the tests with the siem privilege, not siemV2, and we changed the feature override in Kibana from siem to siemV2. So the problem is solved just by bringing back the old override (duplicating).

Do you mean that this problem and this solution are temporary till MKI is using updated siemV2 privilege?

Yes, that's what I tested

duplicate siem feature override

9a48aa3

semd added v9.0.0 Team:Threat Hunting Security Solution Threat Hunting Team backport:version Backport to applied version labels v8.18.0 labels Jan 21, 2025

semd self-assigned this Jan 21, 2025

semd requested review from a team as code owners January 21, 2025 11:42

semd added the release_note:skip Skip the PR/issue when compiling release notes label Jan 21, 2025

janmonschke added the ci:build-serverless-image label Jan 21, 2025

Merge branch 'main' into fix/old_siem_feature_override_serverless_sec…

6d5fe54

…urity

janmonschke reviewed Jan 21, 2025

View reviewed changes

afharo approved these changes Jan 21, 2025

View reviewed changes

elena-shostak approved these changes Jan 21, 2025

View reviewed changes

semd merged commit 9077414 into elastic:main Jan 21, 2025
8 checks passed

kibanamachine mentioned this pull request Jan 21, 2025

[8.x] [Security Solution] Fix old siem feature override (#207333) #207373

Merged

wayneseymour mentioned this pull request Jan 21, 2025

[SKIP ON MKI] x-pack/test_serverless/../view_case.ts #207386

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution] Fix old siem feature override #207333

[Security Solution] Fix old siem feature override #207333

semd commented Jan 21, 2025 •

edited by kibanamachine

Loading

elasticmachine commented Jan 21, 2025

janmonschke commented Jan 21, 2025

semd commented Jan 21, 2025

semd commented Jan 21, 2025 •

edited

Loading

azasypkin commented Jan 21, 2025

semd commented Jan 21, 2025 •

edited

Loading

azasypkin commented Jan 21, 2025

janmonschke Jan 21, 2025

semd commented Jan 21, 2025 •

edited

Loading

elasticmachine commented Jan 21, 2025 •

edited

Loading

azasypkin commented Jan 21, 2025

semd commented Jan 21, 2025 •

edited

Loading

elena-shostak left a comment •

edited

Loading

kibanamachine commented Jan 21, 2025

kibanamachine commented Jan 21, 2025

azasypkin commented Jan 21, 2025

semd commented Jan 21, 2025

logeekal commented Jan 21, 2025

azasypkin commented Jan 22, 2025

semd commented Jan 22, 2025

	### Security's feature privileges are fine-tuned to grant access to Discover, Dashboard, Maps, and Visualize apps.
	### We're keeping around a copy of these overrides for the deprecated `siem` feature as well to make sure that they're applied all roles properly

[Security Solution] Fix old siem feature override #207333

[Security Solution] Fix old siem feature override #207333

Conversation

semd commented Jan 21, 2025 • edited by kibanamachine Loading

Summary

elasticmachine commented Jan 21, 2025

janmonschke commented Jan 21, 2025

semd commented Jan 21, 2025

semd commented Jan 21, 2025 • edited Loading

azasypkin commented Jan 21, 2025

semd commented Jan 21, 2025 • edited Loading

azasypkin commented Jan 21, 2025

janmonschke Jan 21, 2025

Choose a reason for hiding this comment

semd commented Jan 21, 2025 • edited Loading

elasticmachine commented Jan 21, 2025 • edited Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

History

azasypkin commented Jan 21, 2025

semd commented Jan 21, 2025 • edited Loading

elena-shostak left a comment • edited Loading

Choose a reason for hiding this comment

kibanamachine commented Jan 21, 2025

kibanamachine commented Jan 21, 2025

💚 All backports created successfully

Questions ?

azasypkin commented Jan 21, 2025

semd commented Jan 21, 2025

logeekal commented Jan 21, 2025

azasypkin commented Jan 22, 2025

semd commented Jan 22, 2025

semd commented Jan 21, 2025 •

edited by kibanamachine

Loading

semd commented Jan 21, 2025 •

edited

Loading

semd commented Jan 21, 2025 •

edited

Loading

semd commented Jan 21, 2025 •

edited

Loading

elasticmachine commented Jan 21, 2025 •

edited

Loading

semd commented Jan 21, 2025 •

edited

Loading

elena-shostak left a comment •

edited

Loading