[8.x] [Sessions] Reindexing the .kibana_security_session_1 index to the 8.x format. (#204097) #207784
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… format. (elastic#204097) Closes elastic#200603 ## Summary Reindexes the Kibana Security session system index to the 8.x format to support 9.0 readiness. ### Release note Creates Kibana Security session index to only if the `kibana_security_session_1` index or the reindexed version do not exist. ### Notes ### How to test For this test, you'll need at least 3 copies of Kibana cloned locally. One each on 7.17, 8.x and main - ensuring you've run `yarn kbn bootstrap` on each of them. Step 0. Verify on the PR branch ----- - Start ES as `yarn es snapshot --license=trial` - Start kibana `yarn start --no-base-path` - Login to kibana in a private browsing window - Navigate to dev tools and run ``` GET .kibana_security_session/_alias ``` - You should see ``` { ".kibana_security_session_1": { "aliases": { ".kibana_security_session": { "is_write_index": true, "is_hidden": true } } } } ``` This indicates that there were no aliases/index present and the new index was created. Step 1. On 7.17 ----- - Run ES with `yarn es snapshot --license=trial -E path.data=/tmp/esdata` - Run kibana - Login with the `elastic` user - Navigate to dev tools and run the following query ``` GET .kibana_security_session_1/_search { "query": { "match_all": {} } } ``` - You should see your current session being returned as the result for this query - You can now shut down ES and kibana. Step 2. On 8.x ----- - Run ES with `yarn es snapshot --license=trial -E path.data=/tmp/esdata` <--- point to the same folder as the previous run - Run kibana, open a private browser window and login. - Navigate to Kibana upgrade assistant and Migrate system indices and wait for it to run. - Now in Dev tools, run the same query. You should see two sessions. - One with the idleSessionTimeout returned as null and the other one containing a value - indicating one was created on 7.x and the other in 8.x - Make a backup of the data folder `cp -r /tmp/esdata /tmp/esdatabkp` Step 3(OPTIONAL). On main (without the changes in this PR) ----- - Run ES with `yarn es snapshot --license=trial -E path.data=/tmp/esdata` - This should throw an error Step 4. On 8.x ----- - First use the backup for the path `cp -r /tmp/esdatabkp /tmp/esdata2` - Start ES only (do not run Kibana yet) by pointing to the copy: `yarn es snapshot --license=trial -E path.data=/tmp/esdata2` - ES should start up and you need to delete 1 index and 2 datastreams using the ES APIs and any method you prefer. For your convenience, you can use the same script as mine: ```ts import axios from 'axios'; const clearIndexAndDatastream = async () => { { const res = await axios.delete( "http://localhost:9200/.kibana-event-log-7.17.28-000001", { headers: { Authorization: "Basic ZWxhc3RpYzpjaGFuZ2VtZQ==", accept: "*/*", "Content-Type": "application/json", "Kbn-Xsrf": "true", }, } ); console.log("deleted index:", JSON.stringify(res.data)); } { const res = await axios.delete( "http://localhost:9200/_data_stream/ilm-history-5", { headers: { Authorization: "Basic ZWxhc3RpYzpjaGFuZ2VtZQ==", accept: "*/*", "Content-Type": "application/json", "Kbn-Xsrf": "true", }, } ); console.log("deleted ds1:", JSON.stringify(res.data)); } { const res = await axios.delete( "http://localhost:9200/_data_stream/.logs-deprecation.elasticsearch-default", { headers: { Authorization: "Basic ZWxhc3RpYzpjaGFuZ2VtZQ==", accept: "*/*", "Content-Type": "application/json", "Kbn-Xsrf": "true", }, } ); console.log("deleted ds2:", JSON.stringify(res.data)); } }; clearIndexAndDatastream(); ``` You should see the result as: ``` deleted index: {"acknowledged":true} deleted ds1: {"acknowledged":true} deleted ds2: {"acknowledged":true} ``` - Now login to Kibana in a private browsing window and navigate to Upgrade assistant and run the migration. - Navigating to devtools and running the same query as above will show you three results. One with no idleTimeout and 2 with timeouts (One on 7.x and two on 8.x format respectively) ``` GET .kibana_security_session_1/_search { "query": { "match_all": {} } } ``` - You can now shut ES and kibana at this point. Step 5. On the branch of this PR ----- - Run ES with `yarn es snapshot --license=trial -E path.data=/tmp/esdata2` - Run Kibana and login using a private window. - Navigating to dev tools and run: ``` GET .kibana_security_session/_alias ``` To show a result as: ``` { ".kibana_security_session_1-reindexed-for-9": { "aliases": { ".kibana_security_session": { "is_hidden": true }, ".kibana_security_session_1": { "is_hidden": true } } } } ``` This indicates that no new index was created and we are using the reindexed version from 8.x. - You should also run the query to check for sessions: ``` GET .kibana_security_session_1/_search { "query": { "match_all": {} } } ``` - This should return 4 sessions in the results This confirms that the session was re-indexed correctly using the right aliases. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: Elastic Machine <[email protected]> (cherry picked from commit 26350ff)
5 tasks
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
💚 Build Succeeded
Metrics [docs]
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto8.x:Questions ?
Please refer to the Backport tool documentation