From b9a0fa0fa9cfe762c3d81337ba5e5f4f857f908e Mon Sep 17 00:00:00 2001 From: talktovikas Date: Fri, 27 Dec 2024 14:49:02 +0530 Subject: [PATCH 1/4] adding knife troubleshooting section. Signed-off-by: talktovikas --- .../content/automate/troubleshooting.md | 36 +++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/components/docs-chef-io/content/automate/troubleshooting.md b/components/docs-chef-io/content/automate/troubleshooting.md index c2a557520d9..c30531a0703 100644 --- a/components/docs-chef-io/content/automate/troubleshooting.md +++ b/components/docs-chef-io/content/automate/troubleshooting.md @@ -180,3 +180,39 @@ max_shards_per_node = 1000 ``` Once done, run the chef-automate config patch `` to deploy your change. + +## Issue: Knife Search Limits at 10000 Records + +### Details + +The knife node list and knife node search commands are inconsistent in terms of the number of records they return. Specifically, knife search is limited to a maximum of **10000** records by default. + +### Fixes + +This happens because OpenSearch,by default, limits the maximum number of records(or document) returned in a single query to **10000**. This is a safeguard to prevent large queries from overloading the system. If you are trying to retrive more than **10000** records, this approach will do that. + +Change the max_result_window to accomodate more than **10000** records. + +```bash +curl -XPUT "http://127.0.0.1:10144/chef/_settings" \ + -d '{ + "index": { + "max_result_window": 50000 + } + }' \ + -H "Content-Type: application/json" +``` +Changes can be verified by doing: + +```bash +curl http://127.0.0.1:10144/_settings?pretty +``` + +To set the value of tracking total hits in OpenSearch, patch the following configuration in the `.toml` file. + +```bash +[erchef.v1.sys.index] + track_total_hits = true +``` + +Once done, run the chef-automate config patch `` to deploy your change. \ No newline at end of file From 54143524ac380b0544443dab49b86aef2cfb2a2c Mon Sep 17 00:00:00 2001 From: talktovikas Date: Fri, 27 Dec 2024 17:47:21 +0530 Subject: [PATCH 2/4] fixing Punctuations Signed-off-by: talktovikas --- components/docs-chef-io/content/automate/troubleshooting.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/components/docs-chef-io/content/automate/troubleshooting.md b/components/docs-chef-io/content/automate/troubleshooting.md index c30531a0703..40bf2903415 100644 --- a/components/docs-chef-io/content/automate/troubleshooting.md +++ b/components/docs-chef-io/content/automate/troubleshooting.md @@ -189,9 +189,9 @@ The knife node list and knife node search commands are inconsistent in terms of ### Fixes -This happens because OpenSearch,by default, limits the maximum number of records(or document) returned in a single query to **10000**. This is a safeguard to prevent large queries from overloading the system. If you are trying to retrive more than **10000** records, this approach will do that. +This happens because OpenSearch, by default, limits the maximum number of records (or documents) returned in a single query to **10000**. This is a safeguard to prevent large queries from overloading the system. If you are trying to retrieve more than **10000** records, this approach will do that. -Change the max_result_window to accomodate more than **10000** records. +Change the max_result_window to accommodate more than **10000** records. ```bash curl -XPUT "http://127.0.0.1:10144/chef/_settings" \ From a3a0c321d207befaeb4fb7eac654beade55245c6 Mon Sep 17 00:00:00 2001 From: talktovikas Date: Mon, 6 Jan 2025 15:44:48 +0530 Subject: [PATCH 3/4] rewording docs. Signed-off-by: talktovikas --- .../docs-chef-io/content/automate/troubleshooting.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/components/docs-chef-io/content/automate/troubleshooting.md b/components/docs-chef-io/content/automate/troubleshooting.md index 40bf2903415..83b4e185c50 100644 --- a/components/docs-chef-io/content/automate/troubleshooting.md +++ b/components/docs-chef-io/content/automate/troubleshooting.md @@ -191,7 +191,9 @@ The knife node list and knife node search commands are inconsistent in terms of This happens because OpenSearch, by default, limits the maximum number of records (or documents) returned in a single query to **10000**. This is a safeguard to prevent large queries from overloading the system. If you are trying to retrieve more than **10000** records, this approach will do that. -Change the max_result_window to accommodate more than **10000** records. +#### Step 1: Increase the max_result_window to retrieve more than **10000** records. + +In case of embedded opensearch: ```bash curl -XPUT "http://127.0.0.1:10144/chef/_settings" \ @@ -208,6 +210,10 @@ Changes can be verified by doing: curl http://127.0.0.1:10144/_settings?pretty ``` +For external OpenSearch, ensure the max_result_window is also increased accordingly. + +#### Step 2: Patch the config in Automate + To set the value of tracking total hits in OpenSearch, patch the following configuration in the `.toml` file. ```bash From 8101605bd27d47e6af9be8be25eb68f86aa200f6 Mon Sep 17 00:00:00 2001 From: Dishank Tiwari Date: Tue, 7 Jan 2025 12:54:31 +0530 Subject: [PATCH 4/4] reviewed and fixed Signed-off-by: Dishank Tiwari --- .../content/automate/troubleshooting.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/components/docs-chef-io/content/automate/troubleshooting.md b/components/docs-chef-io/content/automate/troubleshooting.md index 83b4e185c50..8d8b70797e9 100644 --- a/components/docs-chef-io/content/automate/troubleshooting.md +++ b/components/docs-chef-io/content/automate/troubleshooting.md @@ -181,19 +181,19 @@ max_shards_per_node = 1000 Once done, run the chef-automate config patch `` to deploy your change. -## Issue: Knife Search Limits at 10000 Records +## Issue: Knife search limits at 10000 records ### Details -The knife node list and knife node search commands are inconsistent in terms of the number of records they return. Specifically, knife search is limited to a maximum of **10000** records by default. +The knife node list and knife node search commands are inconsistent in the number of records they return. By default, knife search returns a maximum of 10,000 records. ### Fixes -This happens because OpenSearch, by default, limits the maximum number of records (or documents) returned in a single query to **10000**. This is a safeguard to prevent large queries from overloading the system. If you are trying to retrieve more than **10000** records, this approach will do that. +This issue occurs because OpenSearch, by default, limits the maximum number of records (or documents) returned in a single query to 10,000. This safeguard prevents large queries from overloading the system. This approach will do that if you try to retrieve more than 10,000 records. -#### Step 1: Increase the max_result_window to retrieve more than **10000** records. +#### Step 1: Increase the max_result_window to retrieve more than **10000** records. -In case of embedded opensearch: +In the case of embedded opensearch: ```bash curl -XPUT "http://127.0.0.1:10144/chef/_settings" \ @@ -204,13 +204,14 @@ curl -XPUT "http://127.0.0.1:10144/chef/_settings" \ }' \ -H "Content-Type: application/json" ``` -Changes can be verified by doing: + +Changes can be verified by doing the following: ```bash curl http://127.0.0.1:10144/_settings?pretty ``` -For external OpenSearch, ensure the max_result_window is also increased accordingly. +For external OpenSearch, ensure the `max_result_window` is also increased accordingly. #### Step 2: Patch the config in Automate @@ -221,4 +222,4 @@ To set the value of tracking total hits in OpenSearch, patch the following confi track_total_hits = true ``` -Once done, run the chef-automate config patch `` to deploy your change. \ No newline at end of file +Once done, run the chef-automate config patch `` to deploy your change.