From a2b37f762e59629a05a4ee39633f98ea78cb1994 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Thu, 13 Feb 2025 10:45:10 -0500 Subject: [PATCH 1/3] Adds the "Enable threat intelligence integrations" page (#405) * first draft * More changes * Removes refs from raw migrated toc * Update solutions/security/get-started/enable-threat-intelligence-integrations.md * Update solutions/security/get-started/enable-threat-intelligence-integrations.md * Update solutions/security/get-started/enable-threat-intelligence-integrations.md * Update solutions/security/get-started/enable-threat-intelligence-integrations.md * Update solutions/security/get-started/enable-threat-intelligence-integrations.md * Update solutions/security/get-started/enable-threat-intelligence-integrations.md * Update solutions/security/get-started/enable-threat-intelligence-integrations.md * Update solutions/security/get-started/enable-threat-intelligence-integrations.md * Removes comments about anchors --- ...ting-started-threat-intelligence-view.png} | Bin .../serverless/security-overview-dashboard.md | 2 +- .../security-threat-intelligence.md | 72 ----------------- .../security/es-threat-intel-integrations.md | 68 ---------------- raw-migrated-files/toc.yml | 2 - ...enable-threat-intelligence-integrations.md | 76 +++++++++++++++--- 6 files changed, 68 insertions(+), 152 deletions(-) rename images/{serverless--getting-started-threat-intelligence-view.png => getting-started-threat-intelligence-view.png} (100%) delete mode 100644 raw-migrated-files/docs-content/serverless/security-threat-intelligence.md delete mode 100644 raw-migrated-files/security-docs/security/es-threat-intel-integrations.md diff --git a/images/serverless--getting-started-threat-intelligence-view.png b/images/getting-started-threat-intelligence-view.png similarity index 100% rename from images/serverless--getting-started-threat-intelligence-view.png rename to images/getting-started-threat-intelligence-view.png diff --git a/raw-migrated-files/docs-content/serverless/security-overview-dashboard.md b/raw-migrated-files/docs-content/serverless/security-overview-dashboard.md index d470eb02e..82e471098 100644 --- a/raw-migrated-files/docs-content/serverless/security-overview-dashboard.md +++ b/raw-migrated-files/docs-content/serverless/security-overview-dashboard.md @@ -58,7 +58,7 @@ For more information about connecting to threat intelligence sources, visit [Ena :::: -:::{image} ../../../images/serverless--getting-started-threat-intelligence-view.png +:::{image} ../../../images/getting-started-threat-intelligence-view.png :alt: Threat Intelligence view on the Overview dashboard :class: screenshot ::: diff --git a/raw-migrated-files/docs-content/serverless/security-threat-intelligence.md b/raw-migrated-files/docs-content/serverless/security-threat-intelligence.md deleted file mode 100644 index d7ffffc88..000000000 --- a/raw-migrated-files/docs-content/serverless/security-threat-intelligence.md +++ /dev/null @@ -1,72 +0,0 @@ -# Enable threat intelligence integrations [security-threat-intelligence] - -The Threat Intelligence view provides a streamlined way to collect threat intelligence data that you can use for threat detection and matching. Threat intelligence data consists of [threat indicators](../../../troubleshoot/security/indicators-of-compromise.md#ti-indicators) ingested from third-party threat intelligence sources. - -Threat indicators describe potential threats, unusual behavior, or malicious activity on a network or in an environment. They are commonly used in indicator match rules to detect and match known threats. When an indicator match rule generates an alert, it includes information about the matched threat indicator. - -::::{note} -To learn more about alerts with threat intelligence, visit [View alert details](../../../solutions/security/detect-and-alert/view-detection-alert-details.md). - -:::: - - -Refer to the following sections to learn how to connect to threat intelligence sources using an [{{agent}} integration](../../../solutions/security/get-started/enable-threat-intelligence-integrations.md#agent-ti-integration), the [Threat Intel module](../../../solutions/security/get-started/enable-threat-intelligence-integrations.md#ti-mod-integration), or a [custom integration](../../../solutions/security/get-started/enable-threat-intelligence-integrations.md#custom-ti-integration). - -:::{image} ../../../images/serverless--getting-started-threat-intelligence-view.png -:alt: The Threat Intelligence view on the Overview dashboard -:class: screenshot -::: - -There are a few scenarios when data won’t display in the Threat Intelligence view: - -* If you’ve chosen a time range that doesn’t contain threat indicator event data, you are prompted to choose a different range. Use the date and time picker in the {{security-app}} to select a new range to analyze. -* If the {{agent}} or {{filebeat}} agent hasn’t ingested Threat Intel module data yet, the threat indicator event counts won’t load. You can wait for data to be ingested or reach out to your administrator for help resolving this. - - -## Add an {{agent}} integration [agent-ti-integration] - -1. Install a [{{fleet}}-managed {{agent}}](https://www.elastic.co/guide/en/fleet/current/install-fleet-managed-elastic-agent.html) on the hosts you want to monitor. -2. In the Threat Intelligence view, click **Enable sources** to view the Integrations page. Scroll down and select **Elastic Agent only** to filter by {{agent}} integrations. - - ::::{tip} - If you know the name of {{agent}} integration you want to install, you can search for it directly. Alternatively, choose the **Threat Intelligence** category to display a list of available [threat intelligence integrations](https://docs.elastic.co/en/integrations/threat-intelligence-intro). - - :::: - -3. Select an {{agent}} integration, then complete the installation steps. -4. Return to the Threat Intelligence view on the Overview dashboard. If indicator data isn’t displaying, refresh the page or refer to these [troubleshooting steps](../../../troubleshoot/security/indicators-of-compromise.md#troubleshoot-indicators-page). - - -## Add a {{filebeat}} Threat Intel module integration [ti-mod-integration] - -1. Set up the [{{filebeat}} agent](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html) and enable the Threat Intel module. - - ::::{note} - For more information about enabling available threat intelligence filesets, refer to [Threat Intel module](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-threatintel.html). - - :::: - -2. Update the `securitySolution:defaultThreatIndex` [advanced setting](../../../solutions/security/get-started/configure-advanced-settings.md#update-threat-intel-indices) by adding the appropriate index pattern name after the default {{fleet}} threat intelligence index pattern (`logs-ti*`): - - 1. If you’re *only* using {{filebeat}} version 8.x, add the appropriate {{filebeat}} threat intelligence index pattern. For example, `logs-ti*`, `filebeat-8*`. - 2. If you’re using a previous version of Filebeat *and* a current one, differentiate between the threat intelligence indices by using unique index pattern names. For example, if you’re using {{filebeat}} version 7.0.0 and 8.0.0, update the setting to `logs-ti*`,`filebeat-7*`,`filebeat-8*`. - -3. Return to the Threat Intelligence view on the Overview dashboard. Refresh the page if indicator data isn’t displaying. - - -## Add a custom integration [custom-ti-integration] - -1. Set up a way to [ingest data](../../../solutions/security/get-started/ingest-data-to-elastic-security.md) into your system. -2. Update the `securitySolution:defaultThreatIndex` [advanced setting](../../../solutions/security/get-started/configure-advanced-settings.md#update-threat-intel-indices) by adding the appropriate index pattern name after the default {{fleet}} threat intelligence index pattern (`logs-ti*`), for example, `logs-ti*`,`custom-ti-index*`. - - ::::{note} - Threat intelligence indices aren’t required to be ECS compatible. However, we strongly recommend compatibility if you’d like your alerts to be enriched with relevant threat indicator information. You can find a list of ECS-compliant threat intelligence fields at [Threat Fields](https://www.elastic.co/guide/en/ecs/{{ecs_version}}/ecs-threat.html). - - :::: - -3. Return to the Threat Intelligence view on the Overview dashboard (**Dashboards** → **Overview**). Refresh the page if indicator data isn’t displaying. - - ::::{note} - The Threat Intelligence view searches for a `threat.feed.name` field value to define the source name in the **Name** column. If a custom source doesn’t have the `threat.feed.name` field or hasn’t defined a `threat.feed.name` field value, it’s considered unnamed and labeled as **Other**. Dashboards aren’t created for unnamed sources unless the `threat.feed.dashboard_id` field is defined. - - :::: diff --git a/raw-migrated-files/security-docs/security/es-threat-intel-integrations.md b/raw-migrated-files/security-docs/security/es-threat-intel-integrations.md deleted file mode 100644 index f5e2bc69e..000000000 --- a/raw-migrated-files/security-docs/security/es-threat-intel-integrations.md +++ /dev/null @@ -1,68 +0,0 @@ -# Enable threat intelligence integrations [es-threat-intel-integrations] - -The Threat Intelligence view provides a streamlined way to collect threat intelligence data that you can use for threat detection and matching. Threat intelligence data consists of [threat indicators](../../../troubleshoot/security/indicators-of-compromise.md#ti-indicators) ingested from third-party threat intelligence sources. - -Threat indicators describe potential threats, unusual behavior, or malicious activity on a network or in an environment. They are commonly used in indicator match rules to detect and match known threats. When an indicator match rule generates an alert, it includes information about the matched threat indicator. - -::::{note} -To learn more about alerts with threat intelligence, visit [View alert details](../../../solutions/security/detect-and-alert/view-detection-alert-details.md). -:::: - - -You can connect to threat intelligence sources using an [{{agent}} integration](../../../solutions/security/get-started/enable-threat-intelligence-integrations.md#agent-ti-integration), the [Threat Intel module](../../../solutions/security/get-started/enable-threat-intelligence-integrations.md#ti-mod-integration), or a [custom integration](../../../solutions/security/get-started/enable-threat-intelligence-integrations.md#custom-ti-integration). - -:::{image} ../../../images/security-threat-intelligence-view.png -:alt: threat intelligence view -:class: screenshot -::: - -There are a few scenarios when data won’t display in the Threat Intelligence view: - -* If you’ve chosen a time range that doesn’t contain threat indicator event data, you are prompted to choose a different range. Use the date and time picker in the {{security-app}} or Kibana to select a new range to analyze. -* If the {{agent}} or {{filebeat}} agent hasn’t ingested Threat Intel module data yet, the threat indicator event counts won’t load. You can wait for data to be ingested or reach out to your administrator for help resolving this. - - -## Add an {{agent}} integration [agent-ti-integration] - -1. Install a [{{fleet}}-managed {{agent}}](https://www.elastic.co/guide/en/fleet/current/install-fleet-managed-elastic-agent.html) on the hosts you want to monitor. -2. In the Threat Intelligence view, click **Enable sources** to view the Integrations page. Scroll down and select **Elastic Agent only** to filter by {{agent}} integrations. - - ::::{tip} - If you know the name of {{agent}} integration you want to install, you can search for it directly. Alternatively, choose the **Threat Intelligence** category to display a list of available [threat intelligence {{integrations}}](https://docs.elastic.co/en/integrations/threat-intelligence-intro). - - :::: - -3. Select an {{agent}} integration, then complete the installation steps. -4. Return to the Threat Intelligence view on the Overview dashboard. If indicator data isn’t displaying, refresh the page or refer to these [troubleshooting steps](../../../troubleshoot/security/indicators-of-compromise.md#troubleshoot-indicators-page). - - -## Add a {{filebeat}} Threat Intel module integration [ti-mod-integration] - -1. Set up the [{{filebeat}} agent](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html) and enable the Threat Intel module. - - ::::{note} - For more information about enabling available threat intelligence filesets, refer to [Threat Intel module](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-threatintel.html). - :::: - -2. Update the `securitySolution:defaultThreatIndex` [advanced setting](../../../solutions/security/get-started/configure-advanced-settings.md#update-threat-intel-indices) by adding the appropriate index pattern name after the default {{fleet}} threat intelligence index pattern (`logs-ti*`): - - 1. If you’re *only* using {{filebeat}} version 8.x, add the appropriate {{filebeat}} threat intelligence index pattern. For example, `logs-ti*`, `filebeat-8*`. - 2. If you’re using a previous version of Filebeat *and* a current one, differentiate between the threat intelligence indices by using unique index pattern names. For example, if you’re using {{filebeat}} version 7.0.0 and 8.0.0, update the setting to `logs-ti*`,`filebeat-7*`,`filebeat-8*`. - -3. Return to the Threat Intelligence view on the Overview dashboard. Refresh the page if indicator data isn’t displaying. - - -## Add a custom integration [custom-ti-integration] - -1. Set up a way to [ingest data](../../../solutions/security/get-started/ingest-data-to-elastic-security.md) into your system. -2. Update the `securitySolution:defaultThreatIndex` [advanced setting](../../../solutions/security/get-started/configure-advanced-settings.md#update-threat-intel-indices) by adding the appropriate index pattern name after the default {{fleet}} threat intelligence index pattern (`logs-ti*`), for example, `logs-ti*`,`custom-ti-index*`. - - ::::{note} - Threat intelligence indices aren’t required to be ECS compatible. However, we strongly recommend compatibility if you’d like your alerts to be enriched with relevant threat indicator information. You can find a list of ECS-compliant threat intelligence fields at [Threat Fields](https://www.elastic.co/guide/en/ecs/{{ecs_version}}/ecs-threat.html). - :::: - -3. Return to the Threat Intelligence view on the Overview dashboard (**Dashboards → Overview**). Refresh the page if indicator data isn’t displaying. - - ::::{note} - The Threat Intelligence view searches for a `threat.feed.name` field value to define the source name in the **Name** column. If a custom source doesn’t have the `threat.feed.name` field or hasn’t defined a `threat.feed.name` field value, it’s considered unnamed and labeled as **Other**. Dashboards aren’t created for unnamed sources unless the `threat.feed.dashboard_id` field is defined. - :::: diff --git a/raw-migrated-files/toc.yml b/raw-migrated-files/toc.yml index 827073fd5..869b24002 100644 --- a/raw-migrated-files/toc.yml +++ b/raw-migrated-files/toc.yml @@ -508,7 +508,6 @@ toc: - file: docs-content/serverless/security-shared-exception-lists.md - file: docs-content/serverless/security-signals-to-cases.md - file: docs-content/serverless/security-third-party-actions.md - - file: docs-content/serverless/security-threat-intelligence.md - file: docs-content/serverless/security-timeline-templates-ui.md - file: docs-content/serverless/security-timelines-ui.md - file: docs-content/serverless/security-triage-alerts-with-elastic-ai-assistant.md @@ -846,7 +845,6 @@ toc: - file: security-docs/security/environment-variable-capture.md - file: security-docs/security/ers-requirements.md - file: security-docs/security/es-overview.md - - file: security-docs/security/es-threat-intel-integrations.md - file: security-docs/security/es-ui-overview.md - file: security-docs/security/esql-queries-assistant.md - file: security-docs/security/event-filters.md diff --git a/solutions/security/get-started/enable-threat-intelligence-integrations.md b/solutions/security/get-started/enable-threat-intelligence-integrations.md index 6a21391c6..ae07fd31a 100644 --- a/solutions/security/get-started/enable-threat-intelligence-integrations.md +++ b/solutions/security/get-started/enable-threat-intelligence-integrations.md @@ -4,19 +4,77 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/security-threat-intelligence.html --- -# Enable threat intelligence integrations +# Enable threat intelligence integrations [security-enable-threat-intelligence-integrations] -% What needs to be done: Lift-and-shift +The Threat Intelligence view provides a streamlined way to collect threat intelligence data that you can use for threat detection and matching. Threat intelligence data consists of [threat indicators](../../../troubleshoot/security/indicators-of-compromise.md#ti-indicators) ingested from third-party threat intelligence sources. -% Use migrated content from existing pages that map to this page: +Threat indicators describe potential threats, unusual behavior, or malicious activity on a network or in an environment. They are commonly used in indicator match rules to detect and match known threats. When an indicator match rule generates an alert, it includes information about the matched threat indicator. -% - [ ] ./raw-migrated-files/security-docs/security/es-threat-intel-integrations.md -% - [ ] ./raw-migrated-files/docs-content/serverless/security-threat-intelligence.md +::::{note} +To learn more about alerts with threat intelligence, visit [View alert details](../detect-and-alert/view-detection-alert-details.md). -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +:::: -$$$agent-ti-integration$$$ -$$$custom-ti-integration$$$ +You can connect to threat intelligence sources using an [{{agent}} integration](#agent-ti-integration), the [Threat Intel module](#ti-mod-integration), or a [custom integration](#custom-ti-integration). -$$$ti-mod-integration$$$ \ No newline at end of file +:::{image} ../../../images/getting-started-threat-intelligence-view.png +:alt: The Threat Intelligence view on the Overview dashboard +:class: screenshot +::: + +There are a few scenarios when data won’t display in the Threat Intelligence view: + +* If you’ve chosen a time range that doesn’t contain threat indicator event data, you are prompted to choose a different range. Use the date and time picker in the {{security-app}} to select a new range to analyze. +* If the {{agent}} or {{filebeat}} agent hasn’t ingested Threat Intel module data yet, the threat indicator event counts won’t load. You can wait for data to be ingested or reach out to your administrator for help resolving this. + + +## Add an {{agent}} integration [agent-ti-integration] + +1. Install a [{{fleet}}-managed {{agent}}](https://www.elastic.co/guide/en/fleet/current/install-fleet-managed-elastic-agent.html) on the hosts you want to monitor. +2. In the Threat Intelligence view, click **Enable sources** to view the Integrations page. Scroll down and select **Elastic Agent only** to filter by {{agent}} integrations. + + ::::{tip} + If you know the name of {{agent}} integration you want to install, you can search for it directly. Alternatively, choose the **Threat Intelligence** category to display a list of available [threat intelligence integrations](https://docs.elastic.co/en/integrations/threat-intelligence-intro). + + :::: + +3. Select an {{agent}} integration, then complete the installation steps. +4. Return to the Threat Intelligence view on the Overview dashboard. If indicator data isn’t displaying, refresh the page or refer to these [troubleshooting steps](../../../troubleshoot/security/indicators-of-compromise.md#troubleshoot-indicators-page). + + +## Add a {{filebeat}} Threat Intel module integration [ti-mod-integration] + +% Substeps in step 2 will require inline versioning. Remember to update them when we have more guidance on handling line-level differences. + +1. Set up the [{{filebeat}} agent](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html) and enable the Threat Intel module. + + ::::{note} + For more information about enabling available threat intelligence filesets, refer to [Threat Intel module](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-threatintel.html). + + :::: + +2. Update the `securitySolution:defaultThreatIndex` [advanced setting](configure-advanced-settings.md#update-threat-intel-indices) by adding the appropriate index pattern name after the default {{fleet}} threat intelligence index pattern (`logs-ti*`): + + * If you’re *only* using {{filebeat}} version 8.x, add the appropriate {{filebeat}} threat intelligence index pattern. For example, `logs-ti*`, `filebeat-8*`. + * If you’re using a previous version of Filebeat *and* a current one, differentiate between the threat intelligence indices by using unique index pattern names. For example, if you’re using {{filebeat}} version 7.0.0 and 8.0.0, update the setting to `logs-ti*`,`filebeat-7*`,`filebeat-8*`. + +3. Return to the Threat Intelligence view on the Overview dashboard. Refresh the page if indicator data isn’t displaying. + + +## Add a custom integration [custom-ti-integration] + +1. Set up a way to [ingest data](ingest-data-to-elastic-security.md) into your system. +2. Update the `securitySolution:defaultThreatIndex` [advanced setting](configure-advanced-settings.md#update-threat-intel-indices) by adding the appropriate index pattern name after the default {{fleet}} threat intelligence index pattern (`logs-ti*`), for example, `logs-ti*`,`custom-ti-index*`. + + ::::{note} + Threat intelligence indices aren’t required to be ECS compatible. However, we strongly recommend compatibility if you’d like your alerts to be enriched with relevant threat indicator information. You can find a list of ECS-compliant threat intelligence fields at [Threat Fields](https://www.elastic.co/guide/en/ecs/{{ecs_version}}/ecs-threat.html). + + :::: + +3. Return to the Threat Intelligence view on the Overview dashboard (**Dashboards** → **Overview**). Refresh the page if indicator data isn’t displaying. + + ::::{note} + The Threat Intelligence view searches for a `threat.feed.name` field value to define the source name in the **Name** column. If a custom source doesn’t have the `threat.feed.name` field or hasn’t defined a `threat.feed.name` field value, it’s considered unnamed and labeled as **Other**. Dashboards aren’t created for unnamed sources unless the `threat.feed.dashboard_id` field is defined. + + :::: From c31c196138e04f94f1d47e83bfb9e80aa94f25a0 Mon Sep 17 00:00:00 2001 From: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com> Date: Thu, 13 Feb 2025 10:54:17 -0500 Subject: [PATCH 2/3] Edits get started section (#412) Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com> --- get-started/contribute.md | 5 --- get-started/installing-elastic-stack.md | 16 +++---- get-started/introduction.md | 32 +++++++------- get-started/stack-components.md | 10 ++--- get-started/the-stack.md | 12 +---- get-started/toc.yml | 4 +- get-started/use-cases.md | 58 ------------------------- 7 files changed, 31 insertions(+), 106 deletions(-) delete mode 100644 get-started/contribute.md delete mode 100644 get-started/use-cases.md diff --git a/get-started/contribute.md b/get-started/contribute.md deleted file mode 100644 index e8487ef13..000000000 --- a/get-started/contribute.md +++ /dev/null @@ -1,5 +0,0 @@ -# Contribute - -% What needs to be done: Write from scratch - -% Scope notes: stub about building on (e.g. search) and extending/contributing to (e.g. plugins) elastic. link down to the search solution + contribute docs Contributing prior art: - https://www.elastic.co/guide/en/kibana/current/contributing.html - https://www.elastic.co/guide/en/logstash/current/contributing-to-logstash.html - https://www.elastic.co/guide/en/beats/filebeat/current/contributing-to-beats.html \ No newline at end of file diff --git a/get-started/installing-elastic-stack.md b/get-started/installing-elastic-stack.md index 2b9d27226..32971c0d5 100644 --- a/get-started/installing-elastic-stack.md +++ b/get-started/installing-elastic-stack.md @@ -30,23 +30,21 @@ For more information on supported network configurations, refer to [{{es}} Inges Install the Elastic Stack products you want to use in the following order: -1. Elasticsearch ({{ref}}/install-elasticsearch.html[install instructions]) -2. Kibana ({{kibana-ref}}/install.html[install]) -3. Logstash ({{logstash-ref}}/installing-logstash.html[install]) -4. Elastic Agent ({{fleet-guide}}/elastic-agent-installation.html[install instructions]) or Beats ({{beats-ref}}/getting-started.html[install instructions]) -5. APM ({{apm-guide-ref}}/apm-quick-start.html[install instructions]) -6. Elasticsearch Hadoop ({{hadoop-ref}}/install.html[install instructions]) +1. [Elasticsearch]({{ref}}/install-elasticsearch.html) +2. [Kibana]({{kibana-ref}}/install.html) +3. [Logstash]({{logstash-ref}}/installing-logstash.html) +4. [Elastic Agent]({{fleet-guide}}/elastic-agent-installation.html) or [Beats]({{beats-ref}}/getting-started.html) +5. [APM]({{apm-guide-ref}}/apm-quick-start.html) +6. [Elasticsearch Hadoop]({{hadoop-ref}}/install.html) Installing in this order ensures that the components each product depends on are in place. ## Installing on {{ecloud}} [install-elastic-stack-for-elastic-cloud] -{ess-leadin-short} - Installing on {{ecloud}} is easy: a single click creates an {{es}} cluster configured to the size you want, with or without high availability. The subscription features are always installed, so you automatically have the ability to secure and monitor your cluster. {{kib}} is enabled automatically, and a number of popular plugins are readily available. -Some {{ecloud}} features can be used only with a specific subscription. For more information, see [https://www.elastic.co/pricing/](https://www.elastic.co/pricing/). +Some {{ecloud}} features can be used only with a specific subscription. For more information, refer to [https://www.elastic.co/pricing/](https://www.elastic.co/pricing/). diff --git a/get-started/introduction.md b/get-started/introduction.md index 0693fb7e4..fbdf23ba4 100644 --- a/get-started/introduction.md +++ b/get-started/introduction.md @@ -26,9 +26,9 @@ navigation_title: "What is Kibana?" Being able to search, observe, and protect your data is a requirement for any analyst. {{kib}} provides solutions for each of these use cases. -* [**Enterprise Search**](https://www.elastic.co/guide/en/enterprise-search/current/index.html) enables you to create a search experience for your app, workplace, and website. -* [**Elastic Observability**](../solutions/observability/get-started/what-is-elastic-observability.md) enables you to monitor and apply analytics in real time to events happening across all your environments. You can analyze log events, monitor the performance metrics for the host or container that it ran in, trace the transaction, and check the overall service availability. -* Designed for security analysts, [**Elastic Security**](../solutions/security.md) provides an overview of the events and alerts from your environment. Elastic Security helps you defend your organization from threats before damage and loss occur. +* [**Enterprise Search**](https://www.elastic.co/guide/en/enterprise-search/current/index.html) enables you to implement premier search experiences for your apps, organizations, and websites. +* [**Elastic Observability**](../solutions/observability/get-started/what-is-elastic-observability.md) enables you to monitor and apply analytics in real time to events happening across all your environments. You can analyze log events, monitor performance metrics for hosts and containers, trace transactions, and maintain service availability. +* Designed for security analysts, [**Elastic Security**](../solutions/security.md) includes a multitude of prebuilt detection rules and data integrations to help you collect important information in one place, get an overview of events and alerts, and secure your environments from endpoints to the cloud. ## Analyze [visualize-and-analyze] @@ -44,11 +44,11 @@ $$$get-data-into-kibana$$$ | | | | --- | --- | -| **1** | **Add data.** The best way to add data to the Elastic Stack is to use one of our many [integrations](../manage-data/ingest.md).On the **Integrations** page, you can also find options to add sample data sets or to upload a file. | -| **2** | **Explore.** With [**Discover**](../explore-analyze/discover.md), you can search your data for hiddeninsights and relationships. Ask your questions, and then filter the results to just the data you want.You can limit your results to the most recent documents added to {{es}}. | -| **3** | **Visualize.** {{kib}} provides many options to create visualizations of your data, fromaggregation-based data to time series data to geo data.[**Dashboard**](../explore-analyze/dashboards.md) is your starting point to create visualizations,and then pulling them together to show your data from multiple perspectives.Use [**Canvas**](../explore-analyze/visualize/canvas.md),to give your datathe “wow” factor for display on a big screen. Use **Graph** to explore patterns and relationships. | -| **4** | **Model data behavior.**Use [**{{ml-cap}}**](../explore-analyze/machine-learning/machine-learning-in-kibana.md) to model the behavior of your data—forecast unusual behavior andperform outlier detection, regression, and classification analysis. | -| **5** | **Share.** Ready to [share](../explore-analyze/report-and-share.md) your findings with a larger audience? {{kib}} offers many options—embeda dashboard, share a link, export to PDF, and more. | +| **1** | **Add data.** The best way to add data to the Elastic Stack is to use one of our many [integrations](../manage-data/ingest.md). On the **Integrations** page, you can also find options to add sample data sets or to upload a file. | +| **2** | **Explore.** With [**Discover**](../explore-analyze/discover.md), you can search your data for hidden insights and relationships. Ask your questions, and then filter the results using a wide range of criteria. | +| **3** | **Visualize.** {{kib}} provides many options to create visualizations of your data, from aggregation-based data to time series data to geospatial data. Use [**Dashboards**](../explore-analyze/dashboards.md) to visualize your data from multiple perspectives tailored for your use case. Use [**Canvas**](../explore-analyze/visualize/canvas.md) to give your data the “wow” factor for display on a big screen. Use **Graph** to identify patterns and relationships. | +| **4** | **Model data behavior.** Use [**{{ml-cap}}**](../explore-analyze/machine-learning/machine-learning-in-kibana.md) to gain a deeper understanding of your data. You can use a wide range of models for supervised and unsupervised learning on tasks including classification, regression, trend forecasting, and anomaly detection. | +| **5** | **Share.** Ready to [share](../explore-analyze/report-and-share.md) your findings with a larger audience? {{kib}} offers many options—embed a dashboard, share a link, export to PDF, and more. | ## Manage your data [_manage_your_data] @@ -73,7 +73,7 @@ For a full list of data management UIs, refer to [**Stack Management**](../deplo Detecting and acting on significant shifts and signals in your data is a need that exists in almost every use case. Alerting allows you to detect conditions in different {{kib}} apps and trigger actions when those conditions are met. For example, you might trigger an alert when a shift occurs in your business critical KPIs or when memory, CPU, or disk space take a dip. When the alert triggers, you can send a notification to a system that is part of your daily workflow: email, Slack, PagerDuty, ServiceNow, and other third party integrations. -A dedicated view for creating, searching, and editing rules is in [**{{rules-ui}}**](../explore-analyze/alerts-cases/alerts/create-manage-rules.md). +The [**{{rules-ui}}**](../explore-analyze/alerts-cases/alerts/create-manage-rules.md) page provides a dedicated view for creating, searching, and editing rules. ## Organize content [organize-and-secure] @@ -83,7 +83,7 @@ You might be managing tens, hundreds, or even thousands of dashboards, visualiza ### Collect related items in a space [organize-in-spaces] -{{kib}} provides [spaces](../deploy-manage/manage-spaces.md) for organizing your visualizations, dashboards, {{data-sources}}, and more. Think of a space as its own mini {{kib}} installation—it’s isolated from all other spaces, so you can tailor it to your specific needs without impacting others. +{{kib}} [spaces](../deploy-manage/manage-spaces.md) help you organize your visualizations, dashboards, {{data-sources}}, and more. Think of a space as its own mini {{kib}} instance—it’s isolated from all other spaces, so you can tailor it to your specific needs without impacting others. :::{image} ../images/kibana-select-your-space.png :alt: Space selector view @@ -93,9 +93,9 @@ You might be managing tens, hundreds, or even thousands of dashboards, visualiza ### Organize your content with tags [_organize_your_content_with_tags] -Tags are keywords or labels that you assign to saved objects, such as dashboards and visualizations, so you can classify them in a way that is meaningful to you. For example, if you tag objects with “design”, you can search and filter on the tag to see all related objects. Tags are also good for grouping content into categories within a space. +Tags are keywords or labels that you assign to saved objects, such as dashboards and visualizations, so you can classify them in a way that is meaningful to you. For example, if you tag objects with “design”, you can search and filter on the tag to display all related objects. Tags are also good for grouping content into categories within a space. -Don’t worry if you have hundreds of dashboards that need to be tagged. Use [**Tags**](../explore-analyze/find-and-organize/tags.md) in **Stack Management** to create your tags, then assign and delete them in bulk operations. +Don’t worry if you have hundreds of dashboards that need to be tagged. Use [**Tags**](../explore-analyze/find-and-organize/tags.md) in **Stack Management** to create your tags, then assign and delete them using bulk operations. ## Secure {{kib}} [intro-kibana-Security] @@ -105,7 +105,7 @@ Don’t worry if you have hundreds of dashboards that need to be tagged. Use [** ### Log in [_log_in] -{{kib}} supports several [authentication providers](../deploy-manage/users-roles/cluster-or-deployment-auth/user-authentication.md), allowing you to login using {{es}}’s built-in realms, or with your own single sign-on provider. +{{kib}} supports several [authentication providers](../deploy-manage/users-roles/cluster-or-deployment-auth/user-authentication.md), allowing you to login using {{es}} or your own single sign-on provider. :::{image} ../images/kibana-kibana-login.png :alt: Login page @@ -129,7 +129,7 @@ The {{kib}} role management interface allows you to describe these various acces ### Audit access [_audit_access] -Once you have your users and roles configured, you might want to maintain a record of who did what, when. The {{kib}} audit log will record this information for you, which can then be correlated with {{es}} audit logs to gain more insights into your users’ behavior. For more information, see [{{kib}} audit logging](../deploy-manage/monitor/logging-configuration/enabling-kibana-audit-logs.md). +Once you have your users and roles configured, you might want to maintain a record of who did what, when. The {{kib}} audit log will record this information for you, which can then be correlated with {{es}} audit logs to gain more insights into your users’ behavior. For more information, refer to [{{kib}} audit logging](../deploy-manage/monitor/logging-configuration/enabling-kibana-audit-logs.md). ## Find apps and objects [kibana-navigation-search] @@ -143,7 +143,7 @@ To quickly find apps and the objects you create, use the search field in the glo You can search for objects by type, name, and tag. To get the most from the search feature, follow these tips: -* Use the keyboard shortcut—Ctrl+/ on Windows and Linux, Command+/ on MacOS—to focus on the input at any time. +* Use the keyboard shortcut—Ctrl+/ on Windows and Linux, Command+/ on MacOS—to focus on the search bar at any time. * Use the provided syntax keywords. | | | @@ -154,7 +154,7 @@ You can search for objects by type, name, and tag. To get the most from the sear | Advanced searches | `tag:(tagname1 or tagname2) my_dashboard_title`
`type:lens tag:(tagname1 or tagname2)`
`type:(dashboard or canvas-workpad) logs`
| -This example searches for visualizations with the tag `design` . +This example searches for visualizations with the tag `design`. :::{image} ../images/kibana-tags-search.png :alt: Example of searching for tags diff --git a/get-started/stack-components.md b/get-started/stack-components.md index e21591e22..b035c0fb3 100644 --- a/get-started/stack-components.md +++ b/get-started/stack-components.md @@ -16,7 +16,7 @@ Trying to decide which ingest component to use? Refer to [Adding data to {{es}}] $$$stack-components-agent$$$ -{{fleet}} and {agent} +{{fleet}} and {{agent}} : {{agent}} is a single, unified way to add monitoring for logs, metrics, and other types of data to a host. It can also protect hosts from security threats, query data from operating systems, forward data from remote services or hardware, and more. Each agent has a single policy to which you can add integrations for new data sources, security protections, and more. {{fleet}} enables you to centrally manage {{agents}} and their policies. Use {{fleet}} to monitor the state of all your {{agents}}, manage agent policies, and upgrade {{agent}} binaries or integrations. @@ -31,7 +31,7 @@ APM $$$stack-components-beats$$$ -{beats} +{{beats}} : {{beats}} are data shippers that you install as agents on your servers to send operational data to {{es}}. {{beats}} are available for many standard observability data scenarios, including audit data, log files and journals, cloud data, availability, metrics, network traffic, and Windows event logs. [Learn more about {{beats}}](https://www.elastic.co/guide/en/beats/libbeat/current/beats-reference.html). $$$stack-components-ingest-pipelines$$$ @@ -41,7 +41,7 @@ $$$stack-components-ingest-pipelines$$$ $$$stack-components-logstash$$$ -{ls} +{{ls}} : {{ls}} is a data collection engine with real-time pipelining capabilities. It can dynamically unify data from disparate sources and normalize the data into destinations of your choice. {{ls}} supports a broad array of input, filter, and output plugins, with many native codecs further simplifying the ingestion process. [Learn more about {{ls}}](https://www.elastic.co/guide/en/logstash/current/introduction.html). @@ -49,7 +49,7 @@ $$$stack-components-logstash$$$ $$$stack-components-elasticsearch$$$ -{es} +{{es}} : {{es}} is the distributed search and analytics engine at the heart of the {{stack}}. It provides near real-time search and analytics for all types of data. Whether you have structured or unstructured text, numerical data, or geospatial data, {{es}} can efficiently store and index it in a way that supports fast searches. {{es}} provides a REST API that enables you to store data in {{es}} and retrieve it. The REST API also provides access to {{es}}'s search and analytics capabilities. [Learn more about {{es}}](https://www.elastic.co/guide/en/elasticsearch/reference/current/elasticsearch-intro.html). @@ -59,7 +59,7 @@ Use {{kib}} to query and visualize the data that’s stored in {{es}}. Or, use t $$$stack-components-kibana$$$ -{kib} +{{kib}} : {{kib}} is the tool to harness your {{es}} data and to manage the {{stack}}. Use it to analyze and visualize the data that’s stored in {{es}}. {{kib}} is also the home for the Elastic Enterprise Search, Elastic Observability and Elastic Security solutions. [Learn more about {{kib}}](https://www.elastic.co/guide/en/kibana/current/introduction.html). $$$stack-components-elasticsearch-clients$$$ diff --git a/get-started/the-stack.md b/get-started/the-stack.md index 28efaeb94..bc63796a0 100644 --- a/get-started/the-stack.md +++ b/get-started/the-stack.md @@ -7,15 +7,7 @@ mapped_urls: # The stack -% What needs to be done: Lift-and-shift +This section provides an overview of the {{stack}} and its components — including {{kib}}, the primary user interface for all of Elastic's solutions: Enterprise Search, Elastic Observability, and Elastic Security. It then describes how to install the {{stack}}. -% Use migrated content from existing pages that map to this page: +$$$kibana-navigation-search$$$ -% - [ ] ./raw-migrated-files/tech-content/starting-with-the-elasticsearch-platform-and-its-solutions/stack-components.md -% Notes: Put this first -% - [ ] ./raw-migrated-files/kibana/kibana/introduction.md -% Notes: Put towards the top of the section -% - [ ] ./raw-migrated-files/stack-docs/elastic-stack/installing-elastic-stack.md -% Notes: hints about install order - -$$$kibana-navigation-search$$$ \ No newline at end of file diff --git a/get-started/toc.yml b/get-started/toc.yml index 7c8793312..e27385a1a 100644 --- a/get-started/toc.yml +++ b/get-started/toc.yml @@ -6,6 +6,4 @@ toc: - file: stack-components.md - file: introduction.md - file: installing-elastic-stack.md - - file: deployment-options.md - - file: use-cases.md - - file: contribute.md \ No newline at end of file + - file: deployment-options.md \ No newline at end of file diff --git a/get-started/use-cases.md b/get-started/use-cases.md deleted file mode 100644 index e7d5c102a..000000000 --- a/get-started/use-cases.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -mapped_pages: - - https://www.elastic.co/guide/en/elasticsearch/reference/current/elasticsearch-intro-what-is-es.html ---- - -# Use cases [elasticsearch-intro-what-is-es] - -[{{es}}](https://github.com/elastic/elasticsearch/) is a distributed search and analytics engine, scalable data store, and vector database built on Apache Lucene. It’s optimized for speed and relevance on production-scale workloads. Use {{es}} to search, index, store, and analyze data of all shapes and sizes in near real time. - -{{es}} is the heart of the [Elastic Stack](the-stack.md). Combined with [{{kib}}](https://www.elastic.co/kibana), it powers the following Elastic solutions: - -* [Observability](https://www.elastic.co/observability) -* [Search](https://www.elastic.co/enterprise-search) -* [Security](https://www.elastic.co/security) - -::::{tip} -{{es}} has a lot of features. Explore the full list on the [product webpage](https://www.elastic.co/elasticsearch/features). - -:::: - - -::::{admonition} What is the Elastic Stack? -:name: elasticsearch-intro-elastic-stack - -{{es}} is the core component of the Elastic Stack, a suite of products for collecting, storing, searching, and visualizing data. [Learn more about the Elastic Stack](the-stack.md). - -:::: - - - -## Use cases [elasticsearch-intro-use-cases] - -{{es}} is used for a wide and growing range of use cases. Here are a few examples: - -**Observability** - -* **Logs, metrics, and traces**: Collect, store, and analyze logs, metrics, and traces from applications, systems, and services. -* **Application performance monitoring (APM)**: Monitor and analyze the performance of business-critical software applications. -* **Real user monitoring (RUM)**: Monitor, quantify, and analyze user interactions with web applications. -* **OpenTelemetry**: Reuse your existing instrumentation to send telemetry data to the Elastic Stack using the OpenTelemetry standard. - -**Search** - -* **Full-text search**: Build a fast, relevant full-text search solution using inverted indexes, tokenization, and text analysis. -* **Vector database**: Store and search vectorized data, and create vector embeddings with built-in and third-party natural language processing (NLP) models. -* **Semantic search**: Understand the intent and contextual meaning behind search queries using tools like synonyms, dense vector embeddings, and learned sparse query-document expansion. -* **Hybrid search**: Combine full-text search with vector search using state-of-the-art ranking algorithms. -* **Build search experiences**: Add hybrid search capabilities to apps or websites, or build enterprise search engines over your organization’s internal data sources. -* **Retrieval augmented generation (RAG)**: Use {{es}} as a retrieval engine to supplement generative AI models with more relevant, up-to-date, or proprietary data for a range of use cases. -* **Geospatial search**: Search for locations and calculate spatial relationships using geospatial queries. - -**Security** - -* **Security information and event management (SIEM)**: Collect, store, and analyze security data from applications, systems, and services. -* **Endpoint security**: Monitor and analyze endpoint security data. -* **Threat hunting**: Search and analyze data to detect and respond to security threats. - -This is just a sample of search, observability, and security use cases enabled by {{es}}. Refer to Elastic [customer success stories](https://www.elastic.co/customers/success-stories) for concrete examples across a range of industries. From 9fe72b67cc46b787e75d68c5deccbbf6b24ebfa0 Mon Sep 17 00:00:00 2001 From: Colleen McGinnis Date: Thu, 13 Feb 2025 10:22:34 -0600 Subject: [PATCH 3/3] [solutions] Replace AsciiDoc API links (#433) * replace api links * fix links Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com> --------- Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com> --- .../apps/agent-configuration-api.md | 2 +- .../observability/apps/annotation-api.md | 2 +- solutions/observability/apps/api-keys.md | 4 +- .../observability/apps/apm-agent-key-api.md | 2 +- .../observability/apps/apm-server-binary.md | 2 +- solutions/observability/apps/apm-ui-api.md | 2 +- .../apps/control-access-to-apm-data.md | 6 +- ...ssign-feature-roles-to-apm-server-users.md | 2 +- .../observability/apps/custom-filters.md | 8 +-- .../apps/delete-sensitive-data.md | 10 ++-- .../apps/fleet-managed-apm-server.md | 2 +- .../apps/grant-access-using-api-keys.md | 10 ++-- .../apps/index-lifecycle-management.md | 2 +- .../observability/apps/rum-source-map-api.md | 2 +- .../apps/tutorial-monitor-java-application.md | 2 +- .../apps/view-elasticsearch-index-template.md | 2 +- ...l-profiling-index-life-cycle-management.md | 2 +- .../observability-ai-assistant.md | 2 +- .../logs-index-template-reference.md | 4 +- solutions/search/async-search-api.md | 2 +- solutions/search/cross-cluster-search.md | 42 +++++++------- .../search/elasticsearch-basics-quickstart.md | 8 +-- .../search-relevance/consistent-scoring.md | 4 +- .../search/full-text/search-with-synonyms.md | 10 ++-- solutions/search/hybrid-semantic-text.md | 2 +- ...bacloud-ai-search-inference-integration.md | 2 +- .../amazon-bedrock-inference-integration.md | 4 +- .../anthropic-inference-integration.md | 2 +- .../azure-ai-studio-inference-integration.md | 4 +- .../azure-openai-inference-integration.md | 4 +- .../cohere-inference-integration.md | 4 +- .../elasticsearch-inference-integration.md | 6 +- .../elser-inference-integration.md | 4 +- .../google-ai-studio-inference-integration.md | 2 +- .../google-vertex-ai-inference-integration.md | 2 +- .../huggingface-inference-integration.md | 4 +- .../mistral-inference-integration.md | 4 +- .../openai-inference-integration.md | 4 +- .../watsonx-inference-integration.md | 2 +- .../querydsl-full-text-filter-tutorial.md | 4 +- solutions/search/rag/playground-query.md | 2 +- solutions/search/rag/playground.md | 2 +- solutions/search/ranking.md | 2 +- .../learning-to-rank-model-training.md | 4 +- .../search/ranking/semantic-reranking.md | 10 ++-- solutions/search/retrievers-overview.md | 21 ++++--- solutions/search/search-applications.md | 6 +- .../search-application-api.md | 56 +++++++++---------- .../search-application-client.md | 48 ++++++++-------- .../search-application-security.md | 8 +-- .../near-real-time-search.md | 2 +- solutions/search/search-pipelines.md | 2 +- solutions/search/search-templates.md | 20 +++---- solutions/search/semantic-search.md | 2 +- solutions/search/semantic-search/cohere-es.md | 2 +- .../semantic-search-inference.md | 32 +++++------ .../semantic-search-semantic-text.md | 6 +- .../site-or-app/behavioral-analytics-api.md | 2 +- .../site-or-app/behavioral-analytics-start.md | 2 +- .../site-or-app/behavioral-analytics.md | 2 +- solutions/search/the-search-api.md | 6 +- ...er-endpoint-before-cross-cluster-search.md | 4 +- solutions/search/vector/bring-own-vectors.md | 4 +- .../dense-versus-sparse-ingest-pipelines.md | 4 +- solutions/search/vector/knn.md | 18 +++--- 65 files changed, 227 insertions(+), 228 deletions(-) diff --git a/solutions/observability/apps/agent-configuration-api.md b/solutions/observability/apps/agent-configuration-api.md index 893e823fe..b15d0a7b3 100644 --- a/solutions/observability/apps/agent-configuration-api.md +++ b/solutions/observability/apps/agent-configuration-api.md @@ -23,7 +23,7 @@ Interact with APM APIs using cURL or another API tool. All APM APIs are Kibana A For all APM APIs, you must use a request header. Supported headers are `Authorization`, `kbn-xsrf`, and `Content-Type`. `Authorization: ApiKey {{credentials}}` -: Kibana supports token-based authentication with the Elasticsearch API key service. The API key returned by the [Elasticsearch create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) can be used by sending a request with an `Authorization` header that has a value of `ApiKey` followed by the `{{credentials}}`, where `{{credentials}}` is the base64 encoding of `id` and `api_key` joined by a colon. +: Kibana supports token-based authentication with the Elasticsearch API key service. The API key returned by the [Elasticsearch create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) can be used by sending a request with an `Authorization` header that has a value of `ApiKey` followed by the `{{credentials}}`, where `{{credentials}}` is the base64 encoding of `id` and `api_key` joined by a colon. Alternatively, you can create a user and use their username and password to authenticate API access: `-u $USER:$PASSWORD`. diff --git a/solutions/observability/apps/annotation-api.md b/solutions/observability/apps/annotation-api.md index dca4275db..4ab269f99 100644 --- a/solutions/observability/apps/annotation-api.md +++ b/solutions/observability/apps/annotation-api.md @@ -22,7 +22,7 @@ Interact with APM APIs using cURL or another API tool. All APM APIs are Kibana A For all APM APIs, you must use a request header. Supported headers are `Authorization`, `kbn-xsrf`, and `Content-Type`. `Authorization: ApiKey {{credentials}}` -: Kibana supports token-based authentication with the Elasticsearch API key service. The API key returned by the [Elasticsearch create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) can be used by sending a request with an `Authorization` header that has a value of `ApiKey` followed by the `{{credentials}}`, where `{{credentials}}` is the base64 encoding of `id` and `api_key` joined by a colon. +: Kibana supports token-based authentication with the Elasticsearch API key service. The API key returned by the [Elasticsearch create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) can be used by sending a request with an `Authorization` header that has a value of `ApiKey` followed by the `{{credentials}}`, where `{{credentials}}` is the base64 encoding of `id` and `api_key` joined by a colon. Alternatively, you can create a user and use their username and password to authenticate API access: `-u $USER:$PASSWORD`. diff --git a/solutions/observability/apps/api-keys.md b/solutions/observability/apps/api-keys.md index 4f96d6247..af06ae1cc 100644 --- a/solutions/observability/apps/api-keys.md +++ b/solutions/observability/apps/api-keys.md @@ -241,7 +241,7 @@ A full list of `apikey` subcommands and flags is available in the [API key comma ### {{es}} API key workflow [apm-create-api-key-workflow-es] -It is also possible to create API keys using the {{es}} [create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html). +It is also possible to create API keys using the {{es}} [create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key). This example creates an API key named `java-002`: @@ -286,7 +286,7 @@ The `credential` string, which is what agents use to communicate with APM Server echo -n GnrUT3QB7yZbSNxKET6d:RhHKisTmQ1aPCHC_TPwOvw | base64 ``` -You can verify your API key has been base64-encoded correctly with the [Authenticate API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-authenticate.html): +You can verify your API key has been base64-encoded correctly with the [Authenticate API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-authenticate): ```sh curl -H "Authorization: ApiKey R0gzRWIzUUI3eVpiU054S3pYSy06bXQyQWl4TlZUeEcyUjd4cUZDS0NlUQ==" localhost:9200/_security/_authenticate diff --git a/solutions/observability/apps/apm-agent-key-api.md b/solutions/observability/apps/apm-agent-key-api.md index 62c40d684..88e363939 100644 --- a/solutions/observability/apps/apm-agent-key-api.md +++ b/solutions/observability/apps/apm-agent-key-api.md @@ -20,7 +20,7 @@ Interact with APM APIs using cURL or another API tool. All APM APIs are Kibana A For all APM APIs, you must use a request header. Supported headers are `Authorization`, `kbn-xsrf`, and `Content-Type`. `Authorization: ApiKey {{credentials}}` -: Kibana supports token-based authentication with the Elasticsearch API key service. The API key returned by the [Elasticsearch create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) can be used by sending a request with an `Authorization` header that has a value of `ApiKey` followed by the `{{credentials}}`, where `{{credentials}}` is the base64 encoding of `id` and `api_key` joined by a colon. +: Kibana supports token-based authentication with the Elasticsearch API key service. The API key returned by the [Elasticsearch create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) can be used by sending a request with an `Authorization` header that has a value of `ApiKey` followed by the `{{credentials}}`, where `{{credentials}}` is the base64 encoding of `id` and `api_key` joined by a colon. Alternatively, you can create a user and use their username and password to authenticate API access: `-u $USER:$PASSWORD`. diff --git a/solutions/observability/apps/apm-server-binary.md b/solutions/observability/apps/apm-server-binary.md index 5929ebcb9..701eebe53 100644 --- a/solutions/observability/apps/apm-server-binary.md +++ b/solutions/observability/apps/apm-server-binary.md @@ -168,7 +168,7 @@ elasticApm { 1. You can find the latest version in the [Gradle plugin portal](https://plugins.gradle.org/plugin/co.elastic.apm.android). 2. Defaults to your `android.defaultConfig.applicationId` value. 3. Defaults to your `android.defaultConfig.versionName` value. -4. Defaults to null. More info on API Keys [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html). +4. Defaults to null. More info on API Keys [here](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key). 5. Defaults to null. diff --git a/solutions/observability/apps/apm-ui-api.md b/solutions/observability/apps/apm-ui-api.md index 62f221868..1e6c103ca 100644 --- a/solutions/observability/apps/apm-ui-api.md +++ b/solutions/observability/apps/apm-ui-api.md @@ -20,7 +20,7 @@ Interact with APM APIs using cURL or another API tool. All APM APIs are Kibana A For all APM APIs, you must use a request header. Supported headers are `Authorization`, `kbn-xsrf`, and `Content-Type`. `Authorization: ApiKey {{credentials}}` -: Kibana supports token-based authentication with the Elasticsearch API key service. The API key returned by the [Elasticsearch create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) can be used by sending a request with an `Authorization` header that has a value of `ApiKey` followed by the `{{credentials}}`, where `{{credentials}}` is the base64 encoding of `id` and `api_key` joined by a colon. +: Kibana supports token-based authentication with the Elasticsearch API key service. The API key returned by the [Elasticsearch create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) can be used by sending a request with an `Authorization` header that has a value of `ApiKey` followed by the `{{credentials}}`, where `{{credentials}}` is the base64 encoding of `id` and `api_key` joined by a colon. Alternatively, you can create a user and use their username and password to authenticate API access: `-u $USER:$PASSWORD`. diff --git a/solutions/observability/apps/control-access-to-apm-data.md b/solutions/observability/apps/control-access-to-apm-data.md index 2673ee43f..9ec5368b3 100644 --- a/solutions/observability/apps/control-access-to-apm-data.md +++ b/solutions/observability/apps/control-access-to-apm-data.md @@ -49,7 +49,7 @@ To separate `staging` and `production` APM data, we’d need to create six filte The `production--apm` aliases will contain a filter that only provides access to documents where the `service.environment` is `production`. Similarly, the `staging--apm` aliases will contain a filter that only provides access to documents where the `service.environment` is `staging`. -To create these six filtered aliases, use the {{es}} [Aliases API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html). In {{kib}}, open **Dev Tools** and run the following POST requests. +To create these six filtered aliases, use the {{es}} [Aliases API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-update-aliases). In {{kib}}, open **Dev Tools** and run the following POST requests. ::::{dropdown} `traces-apm*` production alias example ```console @@ -271,7 +271,7 @@ Using the table below, assign each role the following privileges: :class: screenshot ::: -Alternatively, you can use the {{es}} [Create or update roles API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-put-role.html): +Alternatively, you can use the {{es}} [Create or update roles API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-put-role): ::::{dropdown} Create a `production_apm_viewer` role This request creates a `production_apm_viewer` role: @@ -350,7 +350,7 @@ The last thing to do is assign users to the newly created roles above. Users wil For information on how to create users and assign them roles with the {{kib}} UI, see [Securing access to Kibana](../../../deploy-manage/users-roles/cluster-or-deployment-auth/quickstart.md). -Alternatively, you can use the {{es}} [Create or update users API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-put-user.html). +Alternatively, you can use the {{es}} [Create or update users API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-put-user). This example creates a new user and assigns them the `production_apm_viewer` role created in the previous step. This user will only have access to the production space and data with a `service.environment` of `production`. Remember to change the `password`, `full_name`, and `email` fields. diff --git a/solutions/observability/apps/create-assign-feature-roles-to-apm-server-users.md b/solutions/observability/apps/create-assign-feature-roles-to-apm-server-users.md index f75166f7b..49cd9af89 100644 --- a/solutions/observability/apps/create-assign-feature-roles-to-apm-server-users.md +++ b/solutions/observability/apps/create-assign-feature-roles-to-apm-server-users.md @@ -95,7 +95,7 @@ The privileges included in this role are **required** for all users when [centra $$$apm-privileges-agent-central-config-server$$$ APM Server acts as a proxy between your APM agents and the Applications UI. The Applications UI communicates any changed settings to APM Server so that your agents only need to poll the Server to determine which central configuration settings have changed. -To create a role with the required privileges for managing central configuration in {{es}} without {{kib}}, you must use the [Roles API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-put-role.html) (the required privileges can’t be assigned to a role in Kibana): +To create a role with the required privileges for managing central configuration in {{es}} without {{kib}}, you must use the [Roles API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-put-role) (the required privileges can’t be assigned to a role in Kibana): ```console POST /_security/role/apm_agentcfg diff --git a/solutions/observability/apps/custom-filters.md b/solutions/observability/apps/custom-filters.md index 71a507913..cd62e12df 100644 --- a/solutions/observability/apps/custom-filters.md +++ b/solutions/observability/apps/custom-filters.md @@ -57,7 +57,7 @@ To obfuscate the passwords stored in the request body, you can use a series of [ ### Create a pipeline [_create_a_pipeline] ::::{tip} -This tutorial uses the [Ingest APIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest-apis.html), but it’s also possible to create a pipeline using the UI. In Kibana, go to **Stack Management** → **Ingest Pipelines** → **Create pipeline** → **New pipeline** or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). +This tutorial uses the [Ingest APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-ingest), but it’s also possible to create a pipeline using the UI. In Kibana, go to **Stack Management** → **Ingest Pipelines** → **Create pipeline** → **New pipeline** or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). :::: @@ -140,7 +140,7 @@ Finally, use the [remove processor](https://www.elastic.co/guide/en/elasticsearc #### Register the pipeline [_register_the_pipeline] -Then put it all together, and use the [create or update pipeline API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-pipeline-api.html) to register the new pipeline in {{es}}. Name the pipeline `apm_redacted_body_password`: +Then put it all together, and use the [create or update pipeline API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ingest-put-pipeline) to register the new pipeline in {{es}}. Name the pipeline `apm_redacted_body_password`: ```console PUT _ingest/pipeline/apm_redacted_body_password @@ -184,7 +184,7 @@ PUT _ingest/pipeline/apm_redacted_body_password ### Test the pipeline [_test_the_pipeline] -Prior to enabling this new pipeline, you can test it with the [simulate pipeline API](https://www.elastic.co/guide/en/elasticsearch/reference/current/simulate-pipeline-api.html). This API allows you to run multiple documents through a pipeline to ensure it is working correctly. +Prior to enabling this new pipeline, you can test it with the [simulate pipeline API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ingest-simulate). This API allows you to run multiple documents through a pipeline to ensure it is working correctly. The request below simulates running three different documents through the pipeline: @@ -295,7 +295,7 @@ The final step in this process is to call the newly created `apm_redacted_body_p To match a custom ingest pipeline with a data stream, follow the `-@custom` template, or replace `-namespace` with `@custom` in the table above. For example, to target application traces, you’d create a pipeline named `traces-apm@custom`. -Use the [create or update pipeline API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-pipeline-api.html) to register the new pipeline in {{es}}. Name the pipeline `traces-apm@custom`: +Use the [create or update pipeline API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ingest-put-pipeline) to register the new pipeline in {{es}}. Name the pipeline `traces-apm@custom`: ```console PUT _ingest/pipeline/traces-apm@custom diff --git a/solutions/observability/apps/delete-sensitive-data.md b/solutions/observability/apps/delete-sensitive-data.md index 2fe452e38..03cfcca13 100644 --- a/solutions/observability/apps/delete-sensitive-data.md +++ b/solutions/observability/apps/delete-sensitive-data.md @@ -17,7 +17,7 @@ If you accidentally ingest sensitive data, follow these steps to remove or redac ## Redact specific fields [apm-redact-field-data] -To redact sensitive data in a specific field, use the [update by query API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html). +To redact sensitive data in a specific field, use the [update by query API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-update-by-query). For example, the following query removes the `client.ip` address from APM documents in the `logs-apm.error-default` data stream: @@ -59,17 +59,17 @@ POST /logs-apm.error-default/_update_by_query } ``` -See [update by query API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html) for more information and examples. +See [update by query API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-update-by-query) for more information and examples. ## Delete {{es}} documents [apm-delete-doc-data] ::::{warning} -This will permanently delete your data. You should test your queries with the [search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html) prior to deleting data. +This will permanently delete your data. You should test your queries with the [search API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search) prior to deleting data. :::: -To delete an {{es}} document, you can use the [delete by query API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html). +To delete an {{es}} document, you can use the [delete by query API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-delete-by-query). For example, to delete all documents in the `apm-traces-*` data stream with a `user.email` value, run the following query: @@ -84,5 +84,5 @@ POST /apm-traces-*/_delete_by_query } ``` -See [delete by query API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html) for more information and examples. +See [delete by query API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-delete-by-query) for more information and examples. diff --git a/solutions/observability/apps/fleet-managed-apm-server.md b/solutions/observability/apps/fleet-managed-apm-server.md index 5fdadbe80..fc1b78766 100644 --- a/solutions/observability/apps/fleet-managed-apm-server.md +++ b/solutions/observability/apps/fleet-managed-apm-server.md @@ -237,7 +237,7 @@ elasticApm { 1. You can find the latest version in the [Gradle plugin portal](https://plugins.gradle.org/plugin/co.elastic.apm.android). 2. Defaults to your `android.defaultConfig.applicationId` value. 3. Defaults to your `android.defaultConfig.versionName` value. -4. Defaults to null. More info on API Keys [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html). +4. Defaults to null. More info on API Keys [here](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key). 5. Defaults to null. diff --git a/solutions/observability/apps/grant-access-using-api-keys.md b/solutions/observability/apps/grant-access-using-api-keys.md index f66c74c0c..b4999f063 100644 --- a/solutions/observability/apps/grant-access-using-api-keys.md +++ b/solutions/observability/apps/grant-access-using-api-keys.md @@ -130,7 +130,7 @@ monitoring.elasticsearch: ## Create an API key with {{es}} APIs [apm-beats-api-key-es] -You can also use {{es}}'s [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to create a new API key. For example: +You can also use {{es}}'s [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) to create a new API key. For example: ```console POST /_security/api_key @@ -171,13 +171,13 @@ POST /_security/api_key 2. Granted privileges, see [Use feature roles](create-assign-feature-roles-to-apm-server-users.md) -See the [Create API key](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) reference for more information. +See the [Create API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) reference for more information. ## Learn more about API keys [apm-learn-more-api-keys] See the {{es}} API key documentation for more information: -* [Create API key](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) -* [Get API key information](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-get-api-key.html) -* [Invalidate API key](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-invalidate-api-key.html) +* [Create API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) +* [Get API key information](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-get-api-key) +* [Invalidate API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-invalidate-api-key) diff --git a/solutions/observability/apps/index-lifecycle-management.md b/solutions/observability/apps/index-lifecycle-management.md index 56cdfe740..e719cb1e0 100644 --- a/solutions/observability/apps/index-lifecycle-management.md +++ b/solutions/observability/apps/index-lifecycle-management.md @@ -138,7 +138,7 @@ The result should include the following: 2. The name of the {{ilm-init}} policy applied to the new component template in step two -New {{ilm-init}} policies only take effect when new indices are created, so you either must wait for a rollover to occur (usually after 30 days or when the index size reaches 50 GB), or force a rollover using the [{{es}} rollover API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-rollover-index.html): +New {{ilm-init}} policies only take effect when new indices are created, so you either must wait for a rollover to occur (usually after 30 days or when the index size reaches 50 GB), or force a rollover using the [{{es}} rollover API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover): ```bash POST /traces-apm-default/_rollover/ diff --git a/solutions/observability/apps/rum-source-map-api.md b/solutions/observability/apps/rum-source-map-api.md index f3404daff..a59f69832 100644 --- a/solutions/observability/apps/rum-source-map-api.md +++ b/solutions/observability/apps/rum-source-map-api.md @@ -36,7 +36,7 @@ Interact with APM APIs using cURL or another API tool. All APM APIs are Kibana A For all APM APIs, you must use a request header. Supported headers are `Authorization`, `kbn-xsrf`, and `Content-Type`. `Authorization: ApiKey {{credentials}}` -: Kibana supports token-based authentication with the Elasticsearch API key service. The API key returned by the [Elasticsearch create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) can be used by sending a request with an `Authorization` header that has a value of `ApiKey` followed by the `{{credentials}}`, where `{{credentials}}` is the base64 encoding of `id` and `api_key` joined by a colon. +: Kibana supports token-based authentication with the Elasticsearch API key service. The API key returned by the [Elasticsearch create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) can be used by sending a request with an `Authorization` header that has a value of `ApiKey` followed by the `{{credentials}}`, where `{{credentials}}` is the base64 encoding of `id` and `api_key` joined by a colon. Alternatively, you can create a user and use their username and password to authenticate API access: `-u $USER:$PASSWORD`. diff --git a/solutions/observability/apps/tutorial-monitor-java-application.md b/solutions/observability/apps/tutorial-monitor-java-application.md index ba0773e87..3276faf26 100644 --- a/solutions/observability/apps/tutorial-monitor-java-application.md +++ b/solutions/observability/apps/tutorial-monitor-java-application.md @@ -846,7 +846,7 @@ If a new log file is created, older log files are gzipped as well to take less s The built-in modules are almost entirely using the [Ingest node](../../../manage-data/ingest/transform-enrich/ingest-pipelines.md) feature of {{es}} instead of the {{beats}} processors. -One of the most helpful parts of the ingest pipeline is the ability to debug by using the [Simulate Pipeline API](https://www.elastic.co/guide/en/elasticsearch/reference/current/simulate-pipeline-api.html). +One of the most helpful parts of the ingest pipeline is the ability to debug by using the [Simulate Pipeline API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ingest-simulate). 1. Let’s write a pipeline that is similar to our {{filebeat}} processors using the {{dev-tools-app}} panel in {{kib}}, run the following: diff --git a/solutions/observability/apps/view-elasticsearch-index-template.md b/solutions/observability/apps/view-elasticsearch-index-template.md index 3636ba80b..b00ccbcce 100644 --- a/solutions/observability/apps/view-elasticsearch-index-template.md +++ b/solutions/observability/apps/view-elasticsearch-index-template.md @@ -73,7 +73,7 @@ In the **Index settings** step, you can specify custom [index settings](https:// ## Roll over the data stream [apm-custom-index-template-rollover] -Changes to component templates are not applied retroactively to existing indices. For changes to take effect, you must create a new write index for the data stream. This can be done with the {{es}} [Rollover API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-rollover-index.html). For example, to roll over the `traces-apm-default` data stream, run: +Changes to component templates are not applied retroactively to existing indices. For changes to take effect, you must create a new write index for the data stream. This can be done with the {{es}} [Rollover API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover). For example, to roll over the `traces-apm-default` data stream, run: ```console POST /traces-apm-default/_rollover/ diff --git a/solutions/observability/infra-and-hosts/universal-profiling-index-life-cycle-management.md b/solutions/observability/infra-and-hosts/universal-profiling-index-life-cycle-management.md index a95bf686a..0fdc968f4 100644 --- a/solutions/observability/infra-and-hosts/universal-profiling-index-life-cycle-management.md +++ b/solutions/observability/infra-and-hosts/universal-profiling-index-life-cycle-management.md @@ -129,7 +129,7 @@ If the custom policy is already applied, the result should include the following } ``` -If the result is empty, the custom {{ilm-init}} policy is not yet in use. New {{ilm-init}} policies only take effect when new indices are created, so either wait for a rollover to occur (usually after 30 days or when the index size reaches 50 GB), or force a rollover using the [{{es}} rollover API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-rollover-index.html): +If the result is empty, the custom {{ilm-init}} policy is not yet in use. New {{ilm-init}} policies only take effect when new indices are created, so either wait for a rollover to occur (usually after 30 days or when the index size reaches 50 GB), or force a rollover using the [{{es}} rollover API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover): ```bash POST /profiling-events-5pow01/_rollover/ diff --git a/solutions/observability/observability-ai-assistant.md b/solutions/observability/observability-ai-assistant.md index ffab08011..23478acc9 100644 --- a/solutions/observability/observability-ai-assistant.md +++ b/solutions/observability/observability-ai-assistant.md @@ -154,7 +154,7 @@ The [search connectors](https://www.elastic.co/guide/en/elasticsearch/reference/ [Connectors](https://www.elastic.co/guide/en/elasticsearch/reference/current/es-connectors.html) allow you to index content from external sources thereby making it available for the AI Assistant. This can greatly improve the relevance of the AI Assistant’s responses. Data can be integrated from sources such as GitHub, Confluence, Google Drive, Jira, AWS S3, Microsoft Teams, Slack, and more. -UI affordances for creating and managing search connectors are available in the Search Solution in {{kib}}. You can also use the {{es}} [Connector APIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/connector-apis.html) to create and manage search connectors. +UI affordances for creating and managing search connectors are available in the Search Solution in {{kib}}. You can also use the {{es}} [Connector APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-connector) to create and manage search connectors. The infrastructure for deploying connectors can be managed by Elastic or self-managed. Managed connectors require an [Enterprise Search](https://www.elastic.co/guide/en/enterprise-search/current/server.html) server connected to the Elastic Stack. Self-managed connectors are run on your own infrastructure and don’t require the Enterprise Search service. diff --git a/solutions/observability/unknown-bucket/logs-index-template-reference.md b/solutions/observability/unknown-bucket/logs-index-template-reference.md index c4bb01db2..c61bd555f 100644 --- a/solutions/observability/unknown-bucket/logs-index-template-reference.md +++ b/solutions/observability/unknown-bucket/logs-index-template-reference.md @@ -5,7 +5,7 @@ mapped_pages: # Logs index template reference [logs-index-template] -Index templates are used to configure the backing indices of data streams as they’re created. These index templates are composed of multiple [component templates](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-component-template.html)—reusable building blocks that configure index mappings, settings, and aliases. +Index templates are used to configure the backing indices of data streams as they’re created. These index templates are composed of multiple [component templates](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-component-template)—reusable building blocks that configure index mappings, settings, and aliases. You can view the default `logs` index template in {{kib}}. To open **Index Management**, find **Stack Management** in the main menu or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). Select **Index Templates** and search for `logs`. Select the `logs` index templates to view relevant component templates. @@ -27,7 +27,7 @@ You can use the `logs@custom` component template to customize your {{es}} indice 4. Name the component template logs@custom. 5. Add any custom metadata, index settings, or mappings. -Changes to component templates are not applied retroactively to existing indices. For changes to take effect, create a new write index for impacted data streams by triggering a rollover. Do this using the {{es}} [Rollover API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-rollover-index.html). For example, to roll over the `logs-generic-default` data stream, run: +Changes to component templates are not applied retroactively to existing indices. For changes to take effect, create a new write index for impacted data streams by triggering a rollover. Do this using the {{es}} [Rollover API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover). For example, to roll over the `logs-generic-default` data stream, run: ```console POST /logs-generic-default/_rollover/ diff --git a/solutions/search/async-search-api.md b/solutions/search/async-search-api.md index a9d172414..face7c25b 100644 --- a/solutions/search/async-search-api.md +++ b/solutions/search/async-search-api.md @@ -11,5 +11,5 @@ applies: {{es}} generally allows you to quickly search across big amounts of data. There are situations where a search executes on many shards, possibly against large data sets or multiple [remote clusters](../../deploy-manage/remote-clusters.md), for which results are not expected to be returned in milliseconds. When you need to execute long-running searches, synchronously waiting for its results to be returned is not ideal. Instead, Async search lets you submit a search request that gets executed *asynchronously*, monitor the progress of the request, and retrieve results at a later stage. You can also retrieve partial results as they become available but before the search has completed. -You can submit an async search request using the [submit async search](https://www.elastic.co/guide/en/elasticsearch/reference/current/async-search.html#submit-async-search) API. The [get async search](https://www.elastic.co/guide/en/elasticsearch/reference/current/async-search.html#get-async-search) API allows you to monitor the progress of an async search request and retrieve its results. An ongoing async search can be deleted through the [delete async search](https://www.elastic.co/guide/en/elasticsearch/reference/current/async-search.html#delete-async-search) API. +You can submit an async search request using the [submit async search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-async-search-submit) API. The [get async search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-async-search-submit) API allows you to monitor the progress of an async search request and retrieve its results. An ongoing async search can be deleted through the [delete async search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-async-search-submit) API. diff --git a/solutions/search/cross-cluster-search.md b/solutions/search/cross-cluster-search.md index afd00ac91..4fe8c686a 100644 --- a/solutions/search/cross-cluster-search.md +++ b/solutions/search/cross-cluster-search.md @@ -15,17 +15,17 @@ applies: The following APIs support {{ccs}}: -* [Search](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html) -* [Async search](https://www.elastic.co/guide/en/elasticsearch/reference/current/async-search.html) -* [Multi search](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-multi-search.html) +* [Search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search) +* [Async search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-async-search-submit) +* [Multi search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-msearch) * [Search template](search-templates.md) -* [Multi search template](https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-search-template.html) -* [Field capabilities](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-field-caps.html) +* [Multi search template](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-msearch-template) +* [Field capabilities](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-field-caps) * [Painless execute API](https://www.elastic.co/guide/en/elasticsearch/painless/current/painless-execute-api.html) -* [Resolve Index API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-resolve-index-api.html) -* [preview] [EQL search](https://www.elastic.co/guide/en/elasticsearch/reference/current/eql-search-api.html) -* [preview] [SQL search](https://www.elastic.co/guide/en/elasticsearch/reference/current/sql-search-api.html) -* [preview] [Vector tile search](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-vector-tile-api.html) +* [Resolve Index API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-resolve-index) +* [preview] [EQL search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-eql-search) +* [preview] [SQL search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-sql-query) +* [preview] [Vector tile search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-mvt) * [preview] [ES|QL](../../explore-analyze/query-filter/languages/esql-cross-clusters.md) @@ -52,7 +52,7 @@ The following APIs support {{ccs}}: ### Remote cluster setup [ccs-remote-cluster-setup] -The following [cluster update settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html) API request adds three remote clusters: `cluster_one`, `cluster_two`, and `cluster_three`. +The following [cluster update settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings) API request adds three remote clusters: `cluster_one`, `cluster_two`, and `cluster_three`. ```console PUT _cluster/settings @@ -91,7 +91,7 @@ PUT _cluster/settings In the search request, you specify data streams and indices on a remote cluster as `:`. -The following [search](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html) API request searches the `my-index-000001` index on a single remote cluster, `cluster_one`. +The following [search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search) API request searches the `my-index-000001` index on a single remote cluster, `cluster_one`. ```console GET /cluster_one:my-index-000001/_search @@ -180,7 +180,7 @@ The API returns the following response. Note that when you search one or more re ### Search multiple remote clusters [ccs-search-multi-remote-cluster] -The following [search](https://www.elastic.co/guide/en/elasticsearch/reference/current/search.html) API request searches the `my-index-000001` index on three clusters: +The following [search](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-search) API request searches the `my-index-000001` index on three clusters: * The local ("querying") cluster, with 10 shards * Two remote clusters, `cluster_one`, with 12 shards and `cluster_two` with 6 shards. @@ -328,7 +328,7 @@ The API returns the following response: ## Using async search for {{ccs}} with ccs_minimize_roundtrips=true [ccs-async-search-minimize-roundtrips-true] -Remote clusters can be queried asynchronously using the [async search](https://www.elastic.co/guide/en/elasticsearch/reference/current/async-search.html) API. A {{ccs}} accepts a [`ccs_minimize_roundtrips`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#ccs-minimize-roundtrips) parameter. For asynchronous searches it defaults to `false`. (Note: for synchronous searches it defaults to `true`.) See [Considerations for choosing whether to minimize roundtrips in a {{ccs}}](#ccs-min-roundtrips) to learn more about this option. +Remote clusters can be queried asynchronously using the [async search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-async-search-submit) API. A {{ccs}} accepts a [`ccs_minimize_roundtrips`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-ccs_minimize_roundtrips) parameter. For asynchronous searches it defaults to `false`. (Note: for synchronous searches it defaults to `true`.) See [Considerations for choosing whether to minimize roundtrips in a {{ccs}}](#ccs-min-roundtrips) to learn more about this option. The following request does an asynchronous search of the `my-index-000001` index using `ccs_minimize_roundtrips=true` against three clusters (same ones as the previous example). @@ -405,7 +405,7 @@ The API returns the following response: 3. The `_clusters` section indicates that 3 clusters are in scope for the search and all are currently in the "running" state. -If you query the [get async search](https://www.elastic.co/guide/en/elasticsearch/reference/current/async-search.html#get-async-search) endpoint while the query is still running, you will see an update in the `_clusters` and `_shards` section of the response as each cluster finishes its search. +If you query the [get async search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-async-search-submit) endpoint while the query is still running, you will see an update in the `_clusters` and `_shards` section of the response as each cluster finishes its search. If you set `ccs_minimize_roundtrips=false`, then you will also see partial aggregation results from shards (from any cluster) that have finished, but no results are shown in "hits" section until the search has completed. @@ -490,7 +490,7 @@ Response: 3. Number of hits from the completed searches so far. Final hits are not shown until searches on all clusters have been completed and merged. Thus, the "hits" section can change as you call this endpoint until the search is completely done. -After searches on all the clusters have completed, querying the [get async search](https://www.elastic.co/guide/en/elasticsearch/reference/current/async-search.html#get-async-search) endpoint will show the final status of the `_clusters` and `_shards` section as well as the hits and any aggregation results. +After searches on all the clusters have completed, querying the [get async search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-async-search-submit) endpoint will show the final status of the `_clusters` and `_shards` section as well as the hits and any aggregation results. ```console GET /_async_search/FklQYndoTDJ2VEFlMEVBTzFJMGhJVFEaLVlKYndBWWZSMUdicUc4WVlEaFl4ZzoxNTU= @@ -888,7 +888,7 @@ POST /my-index-000001,cluster_one:my-index-000001,cluster_two:my-index-000001/_a } ``` -The API returns the following response if the query takes longer than the `wait_for_completion_timeout` duration (see [Async search](https://www.elastic.co/guide/en/elasticsearch/reference/current/async-search.html)). +The API returns the following response if the query takes longer than the `wait_for_completion_timeout` duration (see [Async search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-async-search-submit)). ```console-result { @@ -980,7 +980,7 @@ If `skip_unavailable` is `true`, a {{ccs}}: * Skips the remote cluster if its nodes are unavailable during the search. The response’s `_clusters.skipped` value contains a count of any skipped clusters and the `_clusters.details` section of the response will show a `skipped` status. * Ignores errors returned by the remote cluster, such as errors related to unavailable shards or indices. This can include errors related to search parameters such as [`allow_no_indices`](https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#api-multi-index) and [`ignore_unavailable`](https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#api-multi-index). -* Ignores the [`allow_partial_search_results`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-partial-responses) parameter and the related `search.default_allow_partial_results` cluster setting when searching the remote cluster. This means searches on the remote cluster may return partial results. +* Ignores the [`allow_partial_search_results`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-allow_partial_search_results) parameter and the related `search.default_allow_partial_results` cluster setting when searching the remote cluster. This means searches on the remote cluster may return partial results. You can modify the `skip_unavailable` setting by editing the `cluster.remote.` settings in the elasticsearch.yml config file. For example: @@ -995,7 +995,7 @@ cluster: skip_unavailable: true ``` -Or you can set the cluster.remote settings via the [cluster update settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html) API as shown [here](#ccs-remote-cluster-setup). +Or you can set the cluster.remote settings via the [cluster update settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings) API as shown [here](#ccs-remote-cluster-setup). When a remote cluster configured with `skip_unavailable: true` (such as `cluster_two` above) is disconnected or unavailable during a {{ccs}}, {{es}} won’t include matching documents from that cluster in the final results and the search will be considered successful (HTTP status 200 OK). @@ -1013,13 +1013,13 @@ Because {{ccs}} involves sending requests to remote clusters, any network delays [Don’t minimize network roundtrips](#ccs-unmin-roundtrips) -: For search requests that include a scroll or inner hits, {{es}} sends multiple outgoing and ingoing requests to each remote cluster. You can also choose this option by setting the [`ccs_minimize_roundtrips`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#ccs-minimize-roundtrips) parameter to `false`. While typically slower, this approach may work well for networks with low latency. +: For search requests that include a scroll or inner hits, {{es}} sends multiple outgoing and ingoing requests to each remote cluster. You can also choose this option by setting the [`ccs_minimize_roundtrips`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search) parameter to `false`. While typically slower, this approach may work well for networks with low latency. See [Don’t minimize network roundtrips](#ccs-unmin-roundtrips) to learn how this option works. ::::{note} -The [vector tile search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-vector-tile-api.html) always minimizes network roundtrips and doesn’t include the `ccs_minimize_roundtrips` parameter. +The [vector tile search API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-mvt) always minimizes network roundtrips and doesn’t include the `ccs_minimize_roundtrips` parameter. :::: @@ -1138,7 +1138,7 @@ $$$ccs-version-compatibility$$$ | 8.17 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ::::{important} -For the [EQL search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/eql-search-api.html), the local and remote clusters must use the same {{es}} version if they have versions prior to 7.17.7 (included) or prior to 8.5.1 (included). +For the [EQL search API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-eql-search), the local and remote clusters must use the same {{es}} version if they have versions prior to 7.17.7 (included) or prior to 8.5.1 (included). :::: diff --git a/solutions/search/elasticsearch-basics-quickstart.md b/solutions/search/elasticsearch-basics-quickstart.md index 0057b4c78..ab1cff037 100644 --- a/solutions/search/elasticsearch-basics-quickstart.md +++ b/solutions/search/elasticsearch-basics-quickstart.md @@ -115,7 +115,7 @@ The response includes metadata that {{es}} generates for the document, including ### Add multiple documents [getting-started-add-multiple-documents] -Use the [`_bulk` endpoint](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html) to add multiple documents in one request. Bulk data must be formatted as newline-delimited JSON (NDJSON). +Use the [`_bulk` endpoint](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) to add multiple documents in one request. Bulk data must be formatted as newline-delimited JSON (NDJSON). ```console POST /_bulk @@ -252,7 +252,7 @@ POST /books/_doc 1. The new field. -View the mapping for the `books` index with the [Get mapping API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html). The new field `language` has been added to the mapping with a `text` data type. +View the mapping for the `books` index with the [Get mapping API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-mapping). The new field `language` has been added to the mapping with a `text` data type. ```console GET /books/_mapping @@ -345,7 +345,7 @@ PUT /my-explicit-mappings-books ### Combine dynamic and explicit mappings [getting-started-combined-mapping] -Explicit mappings are defined at index creation, and documents must conform to these mappings. You can also use the [Update mapping API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html). When an index has the `dynamic` flag set to `true`, you can add new fields to documents without updating the mapping. +Explicit mappings are defined at index creation, and documents must conform to these mappings. You can also use the [Update mapping API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-mapping). When an index has the `dynamic` flag set to `true`, you can add new fields to documents without updating the mapping. This allows you to combine explicit and dynamic mappings. Learn more about [managing and updating mappings](../../manage-data/data-store/mapping.md#mapping-manage-update). @@ -474,7 +474,7 @@ GET books/_search ## Step 5: Delete your indices (optional) [getting-started-delete-indices] -When following along with examples, you might want to delete an index to start from scratch. You can delete indices using the [Delete index API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-delete-index.html). +When following along with examples, you might want to delete an index to start from scratch. You can delete indices using the [Delete index API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-delete). For example, run the following command to delete the indices created in this tutorial: diff --git a/solutions/search/full-text/search-relevance/consistent-scoring.md b/solutions/search/full-text/search-relevance/consistent-scoring.md index 90c50f41a..72704be69 100644 --- a/solutions/search/full-text/search-relevance/consistent-scoring.md +++ b/solutions/search/full-text/search-relevance/consistent-scoring.md @@ -17,7 +17,7 @@ Say the same user runs the same request twice in a row and documents do not come Now why is it a problem? Index statistics are an important part of the score. And these index statistics may be different across copies of the same shard due to deleted documents. As you may know when documents are deleted or updated, the old document is not immediately removed from the index, it is just marked as deleted and it will only be removed from disk on the next time that the segment this old document belongs to is merged. However for practical reasons, those deleted documents are taken into account for index statistics. So imagine that the primary shard just finished a large merge that removed lots of deleted documents, then it might have index statistics that are sufficiently different from the replica (which still have plenty of deleted documents) so that scores are different too. -The recommended way to work around this issue is to use a string that identifies the user that is logged in (a user id or session id for instance) as a [preference](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-preference). This ensures that all queries of a given user are always going to hit the same shards, so scores remain more consistent across queries. +The recommended way to work around this issue is to use a string that identifies the user that is logged in (a user id or session id for instance) as a [preference](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search). This ensures that all queries of a given user are always going to hit the same shards, so scores remain more consistent across queries. This work around has another benefit: when two documents have the same score, they will be sorted by their internal Lucene doc id (which is unrelated to the `_id`) by default. However these doc ids could be different across copies of the same shard. So by always hitting the same shard, we would get more consistent ordering of documents that have the same scores. @@ -34,7 +34,7 @@ then there are good chances that all shards that are involved in the search requ If you have a small dataset, the easiest way to work around this issue is to index everything into an index that has a single shard (`index.number_of_shards: 1`), which is the default. Then index statistics will be the same for all documents and scores will be consistent. -Otherwise the recommended way to work around this issue is to use the [`dfs_query_then_fetch`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#dfs-query-then-fetch) search type. This will make Elasticsearch perform an initial round trip to all involved shards, asking them for their index statistics relatively to the query, then the coordinating node will merge those statistics and send the merged statistics alongside the request when asking shards to perform the `query` phase, so that shards can use these global statistics rather than their own statistics in order to do the scoring. +Otherwise the recommended way to work around this issue is to use the [`dfs_query_then_fetch`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-search_type) search type. This will make Elasticsearch perform an initial round trip to all involved shards, asking them for their index statistics relatively to the query, then the coordinating node will merge those statistics and send the merged statistics alongside the request when asking shards to perform the `query` phase, so that shards can use these global statistics rather than their own statistics in order to do the scoring. In most cases, this additional round trip should be very cheap. However in the event that your query contains a very large number of fields/terms or fuzzy queries, beware that gathering statistics alone might not be cheap since all terms have to be looked up in the terms dictionaries in order to look up statistics. diff --git a/solutions/search/full-text/search-with-synonyms.md b/solutions/search/full-text/search-with-synonyms.md index 7df306630..f31337d10 100644 --- a/solutions/search/full-text/search-with-synonyms.md +++ b/solutions/search/full-text/search-with-synonyms.md @@ -50,7 +50,7 @@ Your synonyms sets need to be stored in {{es}} so your analyzers can refer to th ### Synonyms API [synonyms-store-synonyms-api] -You can use the [synonyms APIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/synonyms-apis.html) to manage synonyms sets. This is the most flexible approach, as it allows to dynamically define and modify synonyms sets. +You can use the [synonyms APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-synonyms) to manage synonyms sets. This is the most flexible approach, as it allows to dynamically define and modify synonyms sets. Changes in your synonyms sets will automatically reload the associated analyzers. @@ -99,7 +99,7 @@ foo => foo bar, baz To update an existing synonyms set, upload new files to your cluster. Synonyms set files must be kept in sync on every cluster node. -When a synonyms set is updated, search analyzers that use it need to be refreshed using the [reload search analyzers API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-reload-analyzers.html) +When a synonyms set is updated, search analyzers that use it need to be refreshed using the [reload search analyzers API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-reload-search-analyzers) This manual syncing and reloading makes this approach less flexible than using the [synonyms API](../../../solutions/search/full-text/search-with-synonyms.md#synonyms-store-synonyms-api). @@ -147,7 +147,7 @@ Check each synonym token filter documentation for configuration details and inst ### Test your analyzer [synonyms-test-analyzer] -You can test an analyzer configuration without modifying your index settings. Use the [analyze API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html) to test your analyzer chain: +You can test an analyzer configuration without modifying your index settings. Use the [analyze API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-analyze) to test your analyzer chain: ```console GET /_analyze @@ -171,8 +171,8 @@ Analyzers can be applied at [index time or search time](../../../manage-data/dat You need to decide when to apply your synonyms: -* Index time: Synonyms are applied when the documents are indexed into {{es}}. This is a less flexible alternative, as changes to your synonyms require [reindexing](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html). -* Search time: Synonyms are applied when a search is executed. This is a more flexible approach, which doesn’t require reindexing. If token filters are configured with `"updateable": true`, search analyzers can be [reloaded](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-reload-analyzers.html) when you make changes to your synonyms. +* Index time: Synonyms are applied when the documents are indexed into {{es}}. This is a less flexible alternative, as changes to your synonyms require [reindexing](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-reindex). +* Search time: Synonyms are applied when a search is executed. This is a more flexible approach, which doesn’t require reindexing. If token filters are configured with `"updateable": true`, search analyzers can be [reloaded](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-reload-search-analyzers) when you make changes to your synonyms. Synonyms sets created using the [synonyms API](../../../solutions/search/full-text/search-with-synonyms.md#synonyms-store-synonyms-api) can only be used at search time. diff --git a/solutions/search/hybrid-semantic-text.md b/solutions/search/hybrid-semantic-text.md index 4b7f71cb6..236c91fbf 100644 --- a/solutions/search/hybrid-semantic-text.md +++ b/solutions/search/hybrid-semantic-text.md @@ -44,7 +44,7 @@ PUT semantic-embeddings ::::{note} -If you want to run a search on indices that were populated by web crawlers or connectors, you have to [update the index mappings](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html) for these indices to include the `semantic_text` field. Once the mapping is updated, you’ll need to run a full web crawl or a full connector sync. This ensures that all existing documents are reprocessed and updated with the new semantic embeddings, enabling hybrid search on the updated data. +If you want to run a search on indices that were populated by web crawlers or connectors, you have to [update the index mappings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-mapping) for these indices to include the `semantic_text` field. Once the mapping is updated, you’ll need to run a full web crawl or a full connector sync. This ensures that all existing documents are reprocessed and updated with the new semantic embeddings, enabling hybrid search on the updated data. :::: diff --git a/solutions/search/inference-api/alibabacloud-ai-search-inference-integration.md b/solutions/search/inference-api/alibabacloud-ai-search-inference-integration.md index 65fb2c310..cc73e63fe 100644 --- a/solutions/search/inference-api/alibabacloud-ai-search-inference-integration.md +++ b/solutions/search/inference-api/alibabacloud-ai-search-inference-integration.md @@ -43,7 +43,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `alibabaclou ## {{api-request-body-title}} [infer-service-alibabacloud-ai-search-api-request-body] `chunking_settings` -: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#infer-chunking-config) to learn more about chunking. +: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to learn more about chunking. `max_chunk_size` : (Optional, integer) Specifies the maximum size of a chunk in words. Defaults to `250`. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). diff --git a/solutions/search/inference-api/amazon-bedrock-inference-integration.md b/solutions/search/inference-api/amazon-bedrock-inference-integration.md index bdd4d5bd4..3de3f4444 100644 --- a/solutions/search/inference-api/amazon-bedrock-inference-integration.md +++ b/solutions/search/inference-api/amazon-bedrock-inference-integration.md @@ -41,7 +41,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `amazonbedro ## {{api-request-body-title}} [infer-service-amazon-bedrock-api-request-body] `chunking_settings` -: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#infer-chunking-config) to learn more about chunking. +: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to learn more about chunking. `max_chunk_size` : (Optional, integer) Specifies the maximum size of a chunk in words. Defaults to `250`. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). @@ -72,7 +72,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `amazonbedro ::::{important} -You need to provide the access and secret keys only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-inference-api.html) does not retrieve your access or secret keys. After creating the {{infer}} model, you cannot change the associated key pairs. If you want to use a different access and secret key pair, delete the {{infer}} model and recreate it with the same name and the updated keys. +You need to provide the access and secret keys only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-get) does not retrieve your access or secret keys. After creating the {{infer}} model, you cannot change the associated key pairs. If you want to use a different access and secret key pair, delete the {{infer}} model and recreate it with the same name and the updated keys. :::: diff --git a/solutions/search/inference-api/anthropic-inference-integration.md b/solutions/search/inference-api/anthropic-inference-integration.md index 4068b92e1..13dbe2604 100644 --- a/solutions/search/inference-api/anthropic-inference-integration.md +++ b/solutions/search/inference-api/anthropic-inference-integration.md @@ -40,7 +40,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `anthropic` ## {{api-request-body-title}} [infer-service-anthropic-api-request-body] `chunking_settings` -: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#infer-chunking-config) to learn more about chunking. +: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to learn more about chunking. `max_chunk_size` : (Optional, integer) Specifies the maximum size of a chunk in words. Defaults to `250`. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). diff --git a/solutions/search/inference-api/azure-ai-studio-inference-integration.md b/solutions/search/inference-api/azure-ai-studio-inference-integration.md index 971aff4f2..1f23b73d0 100644 --- a/solutions/search/inference-api/azure-ai-studio-inference-integration.md +++ b/solutions/search/inference-api/azure-ai-studio-inference-integration.md @@ -41,7 +41,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `azureaistud ## {{api-request-body-title}} [infer-service-azure-ai-studio-api-request-body] `chunking_settings` -: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#infer-chunking-config) to learn more about chunking. +: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to learn more about chunking. `max_chunk_size` : (Optional, integer) Specifies the maximum size of a chunk in words. Defaults to `250`. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). @@ -68,7 +68,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `azureaistud : (Required, string) A valid API key of your Azure AI Studio model deployment. This key can be found on the overview page for your deployment in the management section of your [Azure AI Studio](https://ai.azure.com/) account. ::::{important} - You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-inference-api.html) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. + You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-get) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. :::: diff --git a/solutions/search/inference-api/azure-openai-inference-integration.md b/solutions/search/inference-api/azure-openai-inference-integration.md index aae300b43..4e75ab8ad 100644 --- a/solutions/search/inference-api/azure-openai-inference-integration.md +++ b/solutions/search/inference-api/azure-openai-inference-integration.md @@ -41,7 +41,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `azureopenai ## {{api-request-body-title}} [infer-service-azure-openai-api-request-body] `chunking_settings` -: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#infer-chunking-config) to learn more about chunking. +: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to learn more about chunking. `max_chunk_size` : (Optional, integer) Specifies the maximum size of a chunk in words. Defaults to `250`. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). @@ -68,7 +68,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `azureopenai : (Required, string) You must provide *either* an API key or an Entra ID. If you do not provide either, or provide both, you will receive an error when trying to create your model. See the [Azure OpenAI Authentication documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#authentication) for more details on these authentication types. ::::{important} - You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-inference-api.html) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. + You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-get) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. :::: diff --git a/solutions/search/inference-api/cohere-inference-integration.md b/solutions/search/inference-api/cohere-inference-integration.md index 803401219..2f8b32456 100644 --- a/solutions/search/inference-api/cohere-inference-integration.md +++ b/solutions/search/inference-api/cohere-inference-integration.md @@ -42,7 +42,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `cohere` ser ## {{api-request-body-title}} [infer-service-cohere-api-request-body] `chunking_settings` -: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#infer-chunking-config) to learn more about chunking. +: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to learn more about chunking. `max_chunk_size` : (Optional, integer) Specifies the maximum size of a chunk in words. Defaults to `250`. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). @@ -69,7 +69,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `cohere` ser : (Required, string) A valid API key of your Cohere account. You can find your Cohere API keys or you can create a new one [on the API keys settings page](https://dashboard.cohere.com/api-keys). ::::{important} - You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-inference-api.html) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. + You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-get) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. :::: diff --git a/solutions/search/inference-api/elasticsearch-inference-integration.md b/solutions/search/inference-api/elasticsearch-inference-integration.md index d8610ec38..967713295 100644 --- a/solutions/search/inference-api/elasticsearch-inference-integration.md +++ b/solutions/search/inference-api/elasticsearch-inference-integration.md @@ -18,7 +18,7 @@ For the most up-to-date API details, refer to [{{infer-cap}} APIs](https://www.e Creates an {{infer}} endpoint to perform an {{infer}} task with the `elasticsearch` service. ::::{note} -* Your {{es}} deployment contains [preconfigured ELSER and E5 {{infer}} endpoints](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#default-enpoints), you only need to create the enpoints using the API if you want to customize the settings. +* Your {{es}} deployment contains [preconfigured ELSER and E5 {{infer}} endpoints](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference), you only need to create the enpoints using the API if you want to customize the settings. * If you use the ELSER or the E5 model through the `elasticsearch` service, the API request will automatically download and deploy the model if it isn’t downloaded yet. :::: @@ -49,7 +49,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `elasticsear ## {{api-request-body-title}} [infer-service-elasticsearch-api-request-body] `chunking_settings` -: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#infer-chunking-config) to learn more about chunking. +: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to learn more about chunking. `max_chunk_size` : (Optional, integer) Specifies the maximum size of a chunk in words. Defaults to `250`. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). @@ -146,7 +146,7 @@ Refer to this [Python notebook](https://github.com/elastic/elasticsearch-labs/bl :::: -The API request below will automatically download the Elastic Rerank model if it isn’t already downloaded and then deploy the model. Once deployed, the model can be used for semantic re-ranking with a [`text_similarity_reranker` retriever](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#text-similarity-reranker-retriever-example-elastic-rerank). +The API request below will automatically download the Elastic Rerank model if it isn’t already downloaded and then deploy the model. Once deployed, the model can be used for semantic re-ranking with a [`text_similarity_reranker` retriever](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever). ```console PUT _inference/rerank/my-elastic-rerank diff --git a/solutions/search/inference-api/elser-inference-integration.md b/solutions/search/inference-api/elser-inference-integration.md index c9b8f8ca4..22a740575 100644 --- a/solutions/search/inference-api/elser-inference-integration.md +++ b/solutions/search/inference-api/elser-inference-integration.md @@ -18,7 +18,7 @@ For the most up-to-date API details, refer to [{{infer-cap}} APIs](https://www.e Creates an {{infer}} endpoint to perform an {{infer}} task with the `elser` service. You can also deploy ELSER by using the [Elasticsearch {{infer}} integration](elasticsearch-inference-integration.md). ::::{note} -* Your {{es}} deployment contains [a preconfigured ELSER {{infer}} endpoint](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#default-enpoints), you only need to create the enpoint using the API if you want to customize the settings. +* Your {{es}} deployment contains [a preconfigured ELSER {{infer}} endpoint](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference), you only need to create the enpoint using the API if you want to customize the settings. * The API request will automatically download and deploy the ELSER model if it isn’t already downloaded. :::: @@ -55,7 +55,7 @@ The `elser` service is deprecated and will be removed in a future release. Use t ## {{api-request-body-title}} [infer-service-elser-api-request-body] `chunking_settings` -: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#infer-chunking-config) to learn more about chunking. +: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to learn more about chunking. `max_chunk_size` : (Optional, integer) Specifies the maximum size of a chunk in words. Defaults to `250`. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). diff --git a/solutions/search/inference-api/google-ai-studio-inference-integration.md b/solutions/search/inference-api/google-ai-studio-inference-integration.md index 28878fb89..819103807 100644 --- a/solutions/search/inference-api/google-ai-studio-inference-integration.md +++ b/solutions/search/inference-api/google-ai-studio-inference-integration.md @@ -41,7 +41,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `googleaistu ## {{api-request-body-title}} [infer-service-google-ai-studio-api-request-body] `chunking_settings` -: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#infer-chunking-config) to learn more about chunking. +: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to learn more about chunking. `max_chunk_size` : (Optional, integer) Specifies the maximum size of a chunk in words. Defaults to `250`. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). diff --git a/solutions/search/inference-api/google-vertex-ai-inference-integration.md b/solutions/search/inference-api/google-vertex-ai-inference-integration.md index 59e250b3a..aeff1bd62 100644 --- a/solutions/search/inference-api/google-vertex-ai-inference-integration.md +++ b/solutions/search/inference-api/google-vertex-ai-inference-integration.md @@ -41,7 +41,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `googleverte ## {{api-request-body-title}} [infer-service-google-vertex-ai-api-request-body] `chunking_settings` -: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#infer-chunking-config) to learn more about chunking. +: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to learn more about chunking. `max_chunk_size` : (Optional, integer) Specifies the maximum size of a chunk in words. Defaults to `250`. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). diff --git a/solutions/search/inference-api/huggingface-inference-integration.md b/solutions/search/inference-api/huggingface-inference-integration.md index 42031a092..c77e6e351 100644 --- a/solutions/search/inference-api/huggingface-inference-integration.md +++ b/solutions/search/inference-api/huggingface-inference-integration.md @@ -40,7 +40,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `hugging_fac ## {{api-request-body-title}} [infer-service-hugging-face-api-request-body] `chunking_settings` -: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#infer-chunking-config) to learn more about chunking. +: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to learn more about chunking. `max_chunk_size` : (Optional, integer) Specifies the maximum size of a chunk in words. Defaults to `250`. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). @@ -67,7 +67,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `hugging_fac : (Required, string) A valid access token of your Hugging Face account. You can find your Hugging Face access tokens or you can create a new one [on the settings page](https://huggingface.co/settings/tokens). ::::{important} - You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-inference-api.html) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. + You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-get) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. :::: diff --git a/solutions/search/inference-api/mistral-inference-integration.md b/solutions/search/inference-api/mistral-inference-integration.md index e6de3cbeb..47d240c12 100644 --- a/solutions/search/inference-api/mistral-inference-integration.md +++ b/solutions/search/inference-api/mistral-inference-integration.md @@ -40,7 +40,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `mistral` se ## {{api-request-body-title}} [infer-service-mistral-api-request-body] `chunking_settings` -: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#infer-chunking-config) to learn more about chunking. +: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to learn more about chunking. `max_chunk_size` : (Optional, integer) Specifies the maximum size of a chunk in words. Defaults to `250`. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). @@ -67,7 +67,7 @@ Creates an {{infer}} endpoint to perform an {{infer}} task with the `mistral` se : (Required, string) A valid API key for your Mistral account. You can find your Mistral API keys or you can create a new one [on the API Keys page](https://console.mistral.ai/api-keys/). ::::{important} - You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-inference-api.html) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. + You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-get) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. :::: diff --git a/solutions/search/inference-api/openai-inference-integration.md b/solutions/search/inference-api/openai-inference-integration.md index 90400131b..f6fdb0ac9 100644 --- a/solutions/search/inference-api/openai-inference-integration.md +++ b/solutions/search/inference-api/openai-inference-integration.md @@ -50,7 +50,7 @@ For more information on how to use the `chat_completion` task type, please refer ## {{api-request-body-title}} [infer-service-openai-api-request-body] `chunking_settings` -: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html#infer-chunking-config) to learn more about chunking. +: (Optional, object) Chunking configuration object. Refer to [Configuring chunking](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to learn more about chunking. `max_chunk_size` : (Optional, integer) Specifies the maximum size of a chunk in words. Defaults to `250`. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). @@ -77,7 +77,7 @@ For more information on how to use the `chat_completion` task type, please refer : (Required, string) A valid API key of your OpenAI account. You can find your OpenAI API keys in your OpenAI account under the [API keys section](https://platform.openai.com/api-keys). ::::{important} - You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-inference-api.html) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. + You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-get) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. :::: diff --git a/solutions/search/inference-api/watsonx-inference-integration.md b/solutions/search/inference-api/watsonx-inference-integration.md index eb823f581..cc34f7634 100644 --- a/solutions/search/inference-api/watsonx-inference-integration.md +++ b/solutions/search/inference-api/watsonx-inference-integration.md @@ -53,7 +53,7 @@ You need an [IBM Cloud® Databases for Elasticsearch deployment](https://cloud.i : (Required, string) A valid API key of your Watsonx account. You can find your Watsonx API keys or you can create a new one [on the API keys page](https://cloud.ibm.com/iam/apikeys). ::::{important} - You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-inference-api.html) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. + You need to provide the API key only once, during the {{infer}} model creation. The [Get {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-get) does not retrieve your API key. After creating the {{infer}} model, you cannot change the associated API key. If you want to use a different API key, delete the {{infer}} model and recreate it with the same name and the updated API key. :::: diff --git a/solutions/search/querydsl-full-text-filter-tutorial.md b/solutions/search/querydsl-full-text-filter-tutorial.md index ee96aed78..6cd04b127 100644 --- a/solutions/search/querydsl-full-text-filter-tutorial.md +++ b/solutions/search/querydsl-full-text-filter-tutorial.md @@ -10,7 +10,7 @@ applies: # Tutorial: Full-text search and filtering in {{es}} [full-text-filter-tutorial] -This is a hands-on introduction to the basics of [full-text search](full-text.md) with {{es}}, also known as *lexical search*, using the [`_search` API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html) and [Query DSL](../../explore-analyze/query-filter/languages/querydsl.md). You’ll also learn how to filter data, to narrow down search results based on exact criteria. +This is a hands-on introduction to the basics of [full-text search](full-text.md) with {{es}}, also known as *lexical search*, using the [`_search` API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search) and [Query DSL](../../explore-analyze/query-filter/languages/querydsl.md). You’ll also learn how to filter data, to narrow down search results based on exact criteria. In this scenario, we’re implementing a search function for a cooking blog. The blog contains recipes with various attributes including textual content, categorical data, and numerical ratings. @@ -114,7 +114,7 @@ Full-text search is powered by [text analysis](full-text/text-analysis-during-se ## Step 2: Add sample blog posts to your index [full-text-filter-tutorial-index-data] -Now you’ll need to index some example blog posts using the [Bulk API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html#bulk). Note that `text` fields are analyzed and multi-fields are generated at index time. +Now you’ll need to index some example blog posts using the [Bulk API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-settings). Note that `text` fields are analyzed and multi-fields are generated at index time. ```console POST /cooking_blog/_bulk?refresh=wait_for diff --git a/solutions/search/rag/playground-query.md b/solutions/search/rag/playground-query.md index 1d71bb803..31dfc2c4e 100644 --- a/solutions/search/rag/playground-query.md +++ b/solutions/search/rag/playground-query.md @@ -43,7 +43,7 @@ Remember that the next step in the workflow is to send the retrieved documents t [Troubleshooting](playground-troubleshooting.md) provides tips on how to diagnose and fix relevance issues. ::::{note} -Playground uses the [`retriever`](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html) syntax for {{es}} queries. Retrievers make it easier to compose and test different retrieval strategies in your search pipelines. Refer to [documentation](../querying-for-search.md) for a high level overview of retrievers. +Playground uses the [`retriever`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever) syntax for {{es}} queries. Retrievers make it easier to compose and test different retrieval strategies in your search pipelines. Refer to [documentation](../querying-for-search.md) for a high level overview of retrievers. :::: diff --git a/solutions/search/rag/playground.md b/solutions/search/rag/playground.md index 1c8355c65..a04d191be 100644 --- a/solutions/search/rag/playground.md +++ b/solutions/search/rag/playground.md @@ -147,7 +147,7 @@ There are many options for ingesting data into {{es}}, including: * The [Elastic crawler](https://www.elastic.co/guide/en/enterprise-search/current/crawler.html) for web content (**NOTE**: Not yet available in *Serverless*) * [Elastic connectors](https://www.elastic.co/guide/en/elasticsearch/reference/current/es-connectors.html) for data synced from third-party sources -* The {{es}} [Bulk API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html) for JSON documents +* The {{es}} [Bulk API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) for JSON documents ::::{dropdown} **Expand** for example To add a few documents to an index called `books` run the following in Dev Tools Console: diff --git a/solutions/search/ranking.md b/solutions/search/ranking.md index 43ee10117..5d14097be 100644 --- a/solutions/search/ranking.md +++ b/solutions/search/ranking.md @@ -45,7 +45,7 @@ When using the following advanced re-ranking pipelines, first-stage retrieval me #### Semantic re-ranking [re-ranking-overview-semantic] -[*Semantic re-ranking*](ranking/semantic-reranking.md) uses machine learning models to reorder search results based on their semantic similarity to a query. Models can be hosted directly in your {{es}} cluster, or you can use [inference endpoints](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html) to call models provided by third-party services. Semantic re-ranking enables out-of-the-box semantic search capabilities on existing full-text search indices. +[*Semantic re-ranking*](ranking/semantic-reranking.md) uses machine learning models to reorder search results based on their semantic similarity to a query. Models can be hosted directly in your {{es}} cluster, or you can use [inference endpoints](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) to call models provided by third-party services. Semantic re-ranking enables out-of-the-box semantic search capabilities on existing full-text search indices. #### Learning to Rank (LTR) [re-ranking-overview-ltr] diff --git a/solutions/search/ranking/learning-to-rank-model-training.md b/solutions/search/ranking/learning-to-rank-model-training.md index 5a8a4ea79..c14b72379 100644 --- a/solutions/search/ranking/learning-to-rank-model-training.md +++ b/solutions/search/ranking/learning-to-rank-model-training.md @@ -142,7 +142,7 @@ MLModel.import_ltr_model( ) ``` -This method will serialize the trained model and the Learning To Rank configuration (including feature extraction) in a format that {{es}} can understand. The model is then deployed to {{es}} using the [Create Trained Models API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-trained-models.html). +This method will serialize the trained model and the Learning To Rank configuration (including feature extraction) in a format that {{es}} can understand. The model is then deployed to {{es}} using the [Create Trained Models API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-put-trained-model). The following types of models are currently supported for LTR with {{es}}: @@ -157,5 +157,5 @@ More model types will be supported in the future. ## Learning To Rank model management [learning-to-rank-model-management] -Once your model is deployed in {{es}} you can manage it using the [trained model APIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-df-trained-models-apis.html). You’re now ready to work with your LTR model as a rescorer at [search time](learning-to-rank-search-usage.md). +Once your model is deployed in {{es}} you can manage it using the [trained model APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-ml-trained-model). You’re now ready to work with your LTR model as a rescorer at [search time](learning-to-rank-search-usage.md). diff --git a/solutions/search/ranking/semantic-reranking.md b/solutions/search/ranking/semantic-reranking.md index 8a37b0071..dcc92e2db 100644 --- a/solutions/search/ranking/semantic-reranking.md +++ b/solutions/search/ranking/semantic-reranking.md @@ -14,7 +14,7 @@ This functionality is in technical preview and may be changed or removed in a fu ::::{tip} -This overview focuses more on the high-level concepts and use cases for semantic re-ranking. For full implementation details on how to set up and use semantic re-ranking in {{es}}, see the [reference documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#text-similarity-reranker-retriever) in the Search API docs. +This overview focuses more on the high-level concepts and use cases for semantic re-ranking. For full implementation details on how to set up and use semantic re-ranking in {{es}}, see the [reference documentation](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever) in the Search API docs. :::: @@ -86,7 +86,7 @@ The following is a non-exhaustive list of considerations when choosing between c ## Semantic re-ranking in {{es}} [semantic-reranking-in-es] -In {{es}}, semantic re-rankers are implemented using the {{es}} [Inference API](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html) and a [retriever](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html). +In {{es}}, semantic re-rankers are implemented using the {{es}} [Inference API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) and a [retriever](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever). To use semantic re-ranking in {{es}}, you need to: @@ -99,7 +99,7 @@ To use semantic re-ranking in {{es}}, you need to: Refer to [the Elastic NLP model reference](../../../explore-analyze/machine-learning/nlp/ml-nlp-model-ref.md#ml-nlp-model-ref-text-similarity) for a list of third party text similarity models supported by {{es}} for semantic re-ranking. -2. **Create a `rerank` endpoint using the [{{es}} Inference API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)**. The Inference API creates an inference endpoint and configures your chosen machine learning model to perform the re-ranking task. +2. **Create a `rerank` endpoint using the [{{es}} Inference API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put)**. The Inference API creates an inference endpoint and configures your chosen machine learning model to perform the re-ranking task. 3. **Define a `text_similarity_reranker` retriever in your search request**. The retriever syntax makes it simple to configure both the retrieval and re-ranking of search results in a single API call. ::::{dropdown} Example search request with semantic reranker @@ -135,8 +135,8 @@ POST _search ## Learn more [semantic-reranking-learn-more] -* Read the [retriever reference documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html) for syntax and implementation details +* Read the [retriever reference documentation](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever) for syntax and implementation details * Learn more about the [retrievers](../querying-for-search.md) abstraction -* Learn more about the Elastic [Inference APIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html) +* Learn more about the Elastic [Inference APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) * Check out our [Python notebook](https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/integrations/cohere/cohere-elasticsearch.ipynb) for using Cohere with {es} diff --git a/solutions/search/retrievers-overview.md b/solutions/search/retrievers-overview.md index e3c779b6c..1bd29d80c 100644 --- a/solutions/search/retrievers-overview.md +++ b/solutions/search/retrievers-overview.md @@ -1,11 +1,10 @@ --- +mapped_urls: + - https://www.elastic.co/guide/en/elasticsearch/reference/current/retrievers-overview.html applies: stack: serverless: --- ---- -mapped_urls: - - https://www.elastic.co/guide/en/elasticsearch/reference/current/retrievers-overview.html # Retrievers [retrievers-overview] @@ -14,7 +13,7 @@ A retriever is an abstraction that was added to the `_search` API in **8.14.0** This syntax enables the configuration of multi-stage retrieval pipelines within a single `_search` call. This simplifies your search application logic, because you no longer need to configure complex searches via multiple {{es}} calls or implement additional client-side logic to combine results from different queries. ::::{note} -This document provides a general overview of the retriever abstraction. For a full syntax reference and implementation overview, check out the [reference documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html) in the `_search` API docs. +This document provides a general overview of the retriever abstraction. For a full syntax reference and implementation overview, check out the [reference documentation](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever) in the `_search` API docs. :::: ::::{tip} @@ -25,12 +24,12 @@ Prefer to start with some sample code? Check out [](retrievers-examples.md) for Retrievers come in various types, each tailored for different search operations. The following retrievers are currently available: -* [**Standard Retriever**](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#standard-retriever). Returns top documents from a traditional [query](https://www.elastic.co/guide/en/elasticsearch/reference/master/query-dsl.html). Mimics a traditional query but in the context of a retriever framework. This ensures backward compatibility as existing `_search` requests remain supported. That way you can transition to the new abstraction at your own pace without mixing syntaxes. -* [**kNN Retriever**](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#knn-retriever). Returns top documents from a [knn search](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-api-knn), in the context of a retriever framework. -* [**Linear Retriever**](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#linear-retriever). Combines the top results from multiple sub-retrievers using a weighted sum of their scores. Allows to specify different weights for each retriever, as well as independently normalize the scores from each result set. -* [**RRF Retriever**](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#rrf-retriever). Combines and ranks multiple first-stage retrievers using the reciprocal rank fusion (RRF) algorithm. Allows you to combine multiple result sets with different relevance indicators into a single result set. An RRF retriever is a **compound retriever**, where its `filter` element is propagated to its sub retrievers. -* [**Rule Retriever**](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#rule-retriever). Applies [query rules](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-using-query-rules.html#query-rules) to the query before returning results. -* [**Text Similarity Re-ranker Retriever**](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#text-similarity-reranker-retriever). Used for [semantic reranking](ranking/semantic-reranking.md). Requires first creating a `rerank` task using the [{{es}} Inference API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html). +* [**Standard Retriever**](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever). Returns top documents from a traditional [query](https://www.elastic.co/guide/en/elasticsearch/reference/master/query-dsl.html). Mimics a traditional query but in the context of a retriever framework. This ensures backward compatibility as existing `_search` requests remain supported. That way you can transition to the new abstraction at your own pace without mixing syntaxes. +* [**kNN Retriever**](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever). Returns top documents from a [knn search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-knn), in the context of a retriever framework. +* [**Linear Retriever**](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever). Combines the top results from multiple sub-retrievers using a weighted sum of their scores. Allows to specify different weights for each retriever, as well as independently normalize the scores from each result set. +* [**RRF Retriever**](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever). Combines and ranks multiple first-stage retrievers using the reciprocal rank fusion (RRF) algorithm. Allows you to combine multiple result sets with different relevance indicators into a single result set. An RRF retriever is a **compound retriever**, where its `filter` element is propagated to its sub retrievers. +* [**Rule Retriever**](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever). Applies [query rules](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-using-query-rules.html#query-rules) to the query before returning results. +* [**Text Similarity Re-ranker Retriever**](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever). Used for [semantic reranking](ranking/semantic-reranking.md). Requires first creating a `rerank` task using the [{{es}} Inference API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put). ## What makes retrievers useful? [retrievers-overview-why-are-they-useful] @@ -145,6 +144,6 @@ Refer to the [Playground documentation](rag/playground.md) for more information. ## API reference [retrievers-overview-api-reference] -For implementation details, including notable restrictions, check out the [reference documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html) in the Search API docs. +For implementation details, including notable restrictions, check out the [reference documentation](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever) in the Search API docs. diff --git a/solutions/search/search-applications.md b/solutions/search/search-applications.md index 016dbb908..02ab866c9 100644 --- a/solutions/search/search-applications.md +++ b/solutions/search/search-applications.md @@ -14,13 +14,13 @@ applies: *Search Applications* enable users to build search-powered applications that leverage the full power of {{es}} and its Query DSL, with a simplified user experience. Create search applications based on your {{es}} indices, build queries using search templates, and easily preview your results directly in the {{kib}} Search UI. -You can also interact with your search applications using the [Search Application APIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-application-apis.html). Search Applications are designed to simplify building unified search experiences across a range of enterprise search use cases, using the Elastic platform. +You can also interact with your search applications using the [Search Application APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-search_application). Search Applications are designed to simplify building unified search experiences across a range of enterprise search use cases, using the Elastic platform. ::::{admonition} Search Applications documentation Documentation for the Search Applications feature lives in two places: * The documentation in this section covers the basics of Search Applications, information about working with Search Applications in the {{kib}} UI, and use case examples. -* The [{{es}} API documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-application-apis.html) contains the API references for working with Search Applications programmatically. Jump there if you’re only interested in the APIs. +* The [{{es}} API documentation](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-search_application) contains the API references for working with Search Applications programmatically. Jump there if you’re only interested in the APIs. :::: @@ -74,7 +74,7 @@ Once created, you can explore the documents in your search application under **S ### Option 2: Get started with the API [search-application-overview-get-started-api] -Use the {{es}} [Put Search Application API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-search-application.html) to create a search application. +Use the {{es}} [Put Search Application API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-application-put) to create a search application. The following example creates a search application named `my_search_application` that searches over the `my_search_index1` and `my_search_index2` indices, along with defining a simple search template (Refer to [Default template example](search-applications/search-application-api.md#search-application-api-default-template)). diff --git a/solutions/search/search-applications/search-application-api.md b/solutions/search/search-applications/search-application-api.md index 3c9871d04..479aa13aa 100644 --- a/solutions/search/search-applications/search-application-api.md +++ b/solutions/search/search-applications/search-application-api.md @@ -12,7 +12,7 @@ applies: # Search API and templates [search-application-api] -Your [search applications](../search-applications.md) use [search templates](../search-templates.md) to perform searches. Templates help reduce complexity by exposing only template parameters, while using the full power of {{es}}'s query DSL to formulate queries. Templates may be set when creating or updating a search application, and can be customized. This template can be edited or updated at any time using the [Put Search Application API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-search-application.html) API call. +Your [search applications](../search-applications.md) use [search templates](../search-templates.md) to perform searches. Templates help reduce complexity by exposing only template parameters, while using the full power of {{es}}'s query DSL to formulate queries. Templates may be set when creating or updating a search application, and can be customized. This template can be edited or updated at any time using the [Put Search Application API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-application-put) API call. In a nutshell, you create search templates with parameters instead of specific hardcoded search values. At search time, you pass in the actual values for these parameters, enabling customized searches without rewriting the entire query structure. Search Application templates: @@ -22,7 +22,7 @@ In a nutshell, you create search templates with parameters instead of specific h This document provides information and sample templates to get you started using [search applications](../search-applications.md) for additional use cases. These templates are designed to be easily modified to meet your needs. Once you’ve created a search application with a template, you can search your search application using this template. -::::{tip} +::::{tip} Search templates use the [Mustache](https://mustache.github.io/) templating language. Mustache variables are typically enclosed in double curly brackets like this: `{{my-var}}`. Learn more by reading about [search templates](../search-templates.md). @@ -31,11 +31,11 @@ Learn more by reading about [search templates](../search-templates.md). -## Default template example [search-application-api-default-template] +## Default template example [search-application-api-default-template] If no template is stored with a search application, a minimal [default search template](#search-application-api-default-template) will be applied at search time. The default template implements a simple search use case. -To create a search application with the default template, issue a [create or update Search Application](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-search-application.html) request without specifying a template: +To create a search application with the default template, issue a [create or update Search Application](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-application-put) request without specifying a template: ```console PUT _application/search_application/my_search_application @@ -44,7 +44,7 @@ PUT _application/search_application/my_search_application } ``` -You can then use the [get search application](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-search-application.html) API call to view your newly created search application, which will also include the default template that was created for you: +You can then use the [get search application](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-application-get) API call to view your newly created search application, which will also include the default template that was created for you: ```console GET _application/search_application/my_search_application @@ -106,14 +106,14 @@ The default template is very minimal: This may be useful for initial exploration of search templates, but you’ll likely want to update this. -::::{note} +::::{note} This template does not support additional parameters, including `from`, `size` or `boost`. If you need to use these, you can customize the template associated with your search application accordingly to include them as parameters. :::: -You can see the parameters and their default values by viewing the template, but it also may be valuable to view the query that will be generated if you [search your search application](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-application-search.html) with various parameters. +You can see the parameters and their default values by viewing the template, but it also may be valuable to view the query that will be generated if you [search your search application](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-application-search) with various parameters. -You can use the [render search application query](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-application-render-query.html) to view the query this template would generate, including with default parameters. For example, searching the search application with no parameters: +You can use the [render search application query](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-application-render-query) to view the query this template would generate, including with default parameters. For example, searching the search application with no parameters: ```console POST _application/search_application/my_search_application/_render_query @@ -168,7 +168,7 @@ POST _application/search_application/my_search_application/_search Searching with the `query_string` and/or `default_field` parameters will perform a [`query_string`](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html) query. -::::{warning} +::::{warning} The default template is subject to change in future versions of the Search Applications feature. :::: @@ -177,14 +177,14 @@ The default template is subject to change in future versions of the Search Appli Try some of the other examples in this document to experiment with specific use cases, or try creating your own! -## Searching a search application [search-application-api-searching] +## Searching a search application [search-application-api-searching] -### Template search [search-application-api-searching-templates] +### Template search [search-application-api-searching-templates] The simplest way to interact with a search application is to use the search template that’s created and stored with it. Each search application has a single template associated with it, which defines search criteria, parameters and defaults. -You send search requests to a search application using the [Search Application Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-application-search.html). +You send search requests to a search application using the [Search Application Search API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-application-search). With the default template, a search looks like this: @@ -200,30 +200,30 @@ POST _application/search_application/my_search_application/_search In this example, we’ve overridden the `query_string` parameter’s default value of `*`. Since we didn’t specify `default_field` the value of this parameter will still be `*`. -### Alias search [search-application-api-searching-alias] +### Alias search [search-application-api-searching-alias] If you don’t want to set up a search template for your search application, an alias will be created with the same name as your search application. This may be helpful when experimenting with specific search queries that you want to use when building your search application’s search template. -If your search application’s name is `my_search_application`, your alias will be `my_search_application`. You can search this using the [search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html). +If your search application’s name is `my_search_application`, your alias will be `my_search_application`. You can search this using the [search API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search). -### Cross cluster search [search-application-cross-cluster-search] +### Cross cluster search [search-application-cross-cluster-search] Search applications do not currently support {{ccs}} because it is not possible to add a remote cluster’s index or index pattern to an index alias. -::::{note} -You should use the Search Applications management APIs to update your application and *not* directly use {{es}} APIs such as the alias API. For example, use [PUT Search Application](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-search-application.html) with the `indices` parameter. This will automatically keep the associated alias up to date and ensure that indices are added to the search application correctly. +::::{note} +You should use the Search Applications management APIs to update your application and *not* directly use {{es}} APIs such as the alias API. For example, use [PUT Search Application](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-application-put) with the `indices` parameter. This will automatically keep the associated alias up to date and ensure that indices are added to the search application correctly. :::: -## Search template examples [search-application-api-examples] +## Search template examples [search-application-api-examples] We have created a number of examples to explore specific use cases. Use these as a starting point for creating your own search templates. -### Text search example [search-application-api-bm25-template] +### Text search example [search-application-api-bm25-template] The following template supports a `multi_match` search over specified fields and boosts: @@ -282,7 +282,7 @@ POST _application/search_application/my_search_application/_search The `text_fields` parameters can be overridden with new/different fields and boosts to experiment with the best configuration for your use case. This template also supports pagination and `explain` via parameters. -### Text search + ELSER with RRF [search-application-api-rrf-template] +### Text search + ELSER with RRF [search-application-api-rrf-template] This example supports the [reciprocal rank fusion (RRF)]](https://www.elastic.co/guide/en/elasticsearch/reference/current/rrf.html) method for combining BM25 and [ELSER](../../../explore-analyze/machine-learning/nlp/ml-nlp-elser.md) searches. Reciprocal Rank Fusion consistently improves the combined results of different search algorithms. It outperforms all other ranking algorithms, and often surpasses the best individual results, without calibration. @@ -345,7 +345,7 @@ PUT _application/search_application/my-search-app } ``` -::::{note} +::::{note} Replace `` with the model ID of your ELSER deployment. :::: @@ -368,7 +368,7 @@ POST _application/search_application/my-search-app/_search ``` -### Text search + ELSER [search-application-api-catchall-template] +### Text search + ELSER [search-application-api-catchall-template] The Elastic Learned Sparse EncodeR ([ELSER](../../../explore-analyze/machine-learning/nlp/ml-nlp-elser.md)) improves search relevance through text-expansion, which enables semantic search. This experimental template requires ELSER to be enabled for one or more fields. Refer to [Semantic search with ELSER](/solutions/search/semantic-search/semantic-search-elser-ingest-pipelines.md) for more information on how to use ELSER. In this case, ELSER is enabled on the `title` and `description` fields. @@ -512,14 +512,14 @@ POST _application/search_application/my_search_application/_search } ``` -::::{tip} +::::{tip} Text search results and ELSER search results are expected to have significantly different scores in some cases, which makes ranking challenging. To find the best search result mix for your dataset, we suggest experimenting with the boost values provided in the example template: * `text_query_boost` to boost the BM25 query as a whole * [`boost`](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#_boosting) fields to boost individual text search fields -* [`min_score`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-api-min-score) parameter to omit significantly low confidence results +* [`min_score`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-min_score) parameter to omit significantly low confidence results -The above boosts should be sufficient for many use cases, but there are cases when adding a [rescore](https://www.elastic.co/guide/en/elasticsearch/reference/current/filter-search-results.html#rescore) query or [index boost](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-multiple-indices.html#index-boost) to your template may be beneficial. Remember to update your search application to use the new template using the [put search application command](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-search-application.html). +The above boosts should be sufficient for many use cases, but there are cases when adding a [rescore](https://www.elastic.co/guide/en/elasticsearch/reference/current/filter-search-results.html#rescore) query or [index boost](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-multiple-indices.html#index-boost) to your template may be beneficial. Remember to update your search application to use the new template using the [put search application command](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-application-put). :::: @@ -531,7 +531,7 @@ POST _application/search_application/my_search_application/_search ``` -### ELSER search [search-application-api-elser-template] +### ELSER search [search-application-api-elser-template] This example supports a streamlined version of ELSER search. @@ -582,7 +582,7 @@ PUT _application/search_application/my_search_application } ``` -::::{note} +::::{note} Replace `` with the model ID of your ELSER deployment. :::: @@ -599,7 +599,7 @@ POST _application/search_application/my_search_application/_search ``` -### kNN search [search-applications-knn-template] +### kNN search [search-applications-knn-template] This example supports [k-nearest neighbor (kNN) search](../vector/knn.md). diff --git a/solutions/search/search-applications/search-application-client.md b/solutions/search/search-applications/search-application-client.md index ed284de60..9fad36a94 100644 --- a/solutions/search/search-applications/search-application-client.md +++ b/solutions/search/search-applications/search-application-client.md @@ -14,7 +14,7 @@ applies: This document is a how-to guide to building a search experience with a [search application](../search-applications.md), using the [Search Application client](https://github.com/elastic/search-application-client). The client is a JavaScript library designed to be used in the browser. You’ll integrate this library into your web app to simplify querying your search application. -::::{tip} +::::{tip} A [sandbox environment](https://github.com/elastic/search-application-client/blob/main/examples/sandbox/README.md) is available for testing and experimenting with the `search-application-client` library. Jump there if you’d like to try out the client without setting up your own web app. Clone the [repository](https://github.com/elastic/search-application-client) and follow the instructions in the README to get started. @@ -23,7 +23,7 @@ Clone the [repository](https://github.com/elastic/search-application-client) and -## Goal [search-application-client-client-goal] +## Goal [search-application-client-client-goal] This guide assumes you want to build a web app with the following search features: @@ -34,7 +34,7 @@ This guide assumes you want to build a web app with the following search feature You can think of the search application as the "server side" that persists changes to {{es}}. Your web app acts as the "client side" that queries the search application. You’ll be making edits to both your search application and your web app to complete the implementation. -## Prerequisites [search-application-client-client-prerequisites] +## Prerequisites [search-application-client-client-prerequisites] To follow this guide, you’ll need: @@ -44,15 +44,15 @@ To follow this guide, you’ll need: * A **search application**. - * Create and manage search applications in the [{{kib}} UI](../search-applications.md#search-application-overview-get-started-ui) or using the [API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-search-application.html). + * Create and manage search applications in the [{{kib}} UI](../search-applications.md#search-application-overview-get-started-ui) or using the [API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-application-put). * A **web app** to query your search application, using [Search Application client](https://github.com/elastic/search-application-client#installation). -## Install and configure the client [search-application-client-client-configuration] +## Install and configure the client [search-application-client-client-configuration] -### Install the client [search-application-client-client-configuration-install] +### Install the client [search-application-client-client-configuration-install] [Install](https://github.com/elastic/search-application-client/blob/main/README.md#installation) the client using npm, yarn, or a CDN. @@ -79,7 +79,7 @@ Alternatively, you can install the client using a CDN. Add the following ` { ``` -### Pagination [search-application-client-client-features-pagination] +### Pagination [search-application-client-client-features-pagination] To use pagination, set the page number and the page size. By default, the page size is 10. The `size` and `from` parameters allow you to control the page and number of hits returned in the response. @@ -491,7 +491,7 @@ const results = await request() ``` -## Sorting [search-application-client-client-features-sorting] +## Sorting [search-application-client-client-features-sorting] To use sorting, specify the field name and the sort order or `pass _score` to sort by relevance. Requires the `_es_sort_fields_fields` param in the search template. Refer to our [example template](#search-application-client-client-template-example) to see where this is used. @@ -504,7 +504,7 @@ const results = await request() ``` -### Filtering [search-application-client-client-features-filter] +### Filtering [search-application-client-client-features-filter] The Search application client also supports filters and facets. To use these, you need to add two parameters: @@ -514,7 +514,7 @@ The Search application client also supports filters and facets. To use these, yo Refer to our [example template](#search-application-client-client-template-example) to see where these are used. -#### Base Filtering [search-application-client-client-features-filter-base] +#### Base Filtering [search-application-client-client-features-filter-base] With a template that’s configured to use filters, use the `setFilter` method to add filters to your query. @@ -535,7 +535,7 @@ const results = await request() ``` -### Facets [search-application-client-client-features-facets] +### Facets [search-application-client-client-features-facets] The client supports the ability to configure facets with your results. Specify facets in the client initialization call. For example, say we want to add facets for actors, directors and IMDB rating. @@ -566,7 +566,7 @@ const request = Client( ) ``` -::::{note} +::::{note} In {{es}}, the `keyword` type is used for fields that need to be searchable in their exact, unmodified form. This means these queries are case-sensitive. We use this type for facets because facets require aggregating and filtering data based on exact values or terms. :::: diff --git a/solutions/search/search-applications/search-application-security.md b/solutions/search/search-applications/search-application-security.md index b49038ee1..ffa5f556e 100644 --- a/solutions/search/search-applications/search-application-security.md +++ b/solutions/search/search-applications/search-application-security.md @@ -17,7 +17,7 @@ When building a frontend application for search use cases, there are two main ap 1. The client (user’s browser) makes API requests to the application backend, which in turn makes a request to {{es}}. The {{es}} cluster is not exposed to the end user. 2. **The client (user’s browser) makes API requests directly to the search service - in this case the {{es}} cluster is reachable to the client.** -This guide describes best practices when taking the second approach. Specifically, we will explain how to use search applications with frontend apps that make direct requests to the [Search Application Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-application-search.html). +This guide describes best practices when taking the second approach. Specifically, we will explain how to use search applications with frontend apps that make direct requests to the [Search Application Search API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-application-search). This approach has a few advantages: @@ -97,7 +97,7 @@ curl -XPOST "http://localhost:9200/_application/search_application/website-produ ``` ::::{tip} -If `expiration` is not present, by default {{es}} API keys never expire. The API key can be invalidated using the [invalidate API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-invalidate-api-key.html). +If `expiration` is not present, by default {{es}} API keys never expire. The API key can be invalidated using the [invalidate API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-invalidate-api-key). :::: @@ -218,7 +218,7 @@ If you are unable to enable CORS on {{es}}, you can proxy the request through a * [Field level security](../../../deploy-manage/users-roles/cluster-or-deployment-auth/controlling-access-at-document-field-level.md) * [APIs](search-application-api.md) - * [PUT Search Application API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-search-application.html) - * [Search Application Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-application-search.html) + * [PUT Search Application API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-application-put) + * [Search Application Search API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-application-search) diff --git a/solutions/search/search-approaches/near-real-time-search.md b/solutions/search/search-approaches/near-real-time-search.md index e4a15e686..bf6ffe22a 100644 --- a/solutions/search/search-approaches/near-real-time-search.md +++ b/solutions/search/search-approaches/near-real-time-search.md @@ -32,7 +32,7 @@ In {{es}}, this process of writing and opening a new segment is called a *refres * Waiting for the refresh interval * Setting the [?refresh](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-refresh.html) option -* Using the [Refresh API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-refresh.html) to explicitly complete a refresh (`POST _refresh`) +* Using the [Refresh API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-refresh) to explicitly complete a refresh (`POST _refresh`) By default, {{es}} periodically refreshes indices every second, but only on indices that have received one search request or more in the last 30 seconds. This is why we say that {{es}} has *near* real-time search: document changes are not visible to search immediately, but will become visible within this timeframe. diff --git a/solutions/search/search-pipelines.md b/solutions/search/search-pipelines.md index c164f2dfa..1d9aa8ed0 100644 --- a/solutions/search/search-pipelines.md +++ b/solutions/search/search-pipelines.md @@ -100,7 +100,7 @@ If the pipeline is not specified, the underscore-prefixed fields will actually b ### `search-default-ingestion` Reference [ingest-pipeline-search-details-generic-reference] -You can access this pipeline with the [Elasticsearch Ingest Pipelines API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-pipeline-api.html) or via Kibana’s [Stack Management > Ingest Pipelines](/manage-data/ingest/transform-enrich/ingest-pipelines.md#create-manage-ingest-pipelines) UI. +You can access this pipeline with the [Elasticsearch Ingest Pipelines API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ingest-get-pipeline) or via Kibana’s [Stack Management > Ingest Pipelines](/manage-data/ingest/transform-enrich/ingest-pipelines.md#create-manage-ingest-pipelines) UI. ::::{warning} This pipeline is a "managed" pipeline. That means that it is not intended to be edited. Editing/updating this pipeline manually could result in unintended behaviors, or difficulty in upgrading in the future. If you want to make customizations, we recommend you utilize index-specific pipelines (see below), specifically [the `@custom` pipeline](#ingest-pipeline-search-details-specific-custom-reference). diff --git a/solutions/search/search-templates.md b/solutions/search/search-templates.md index 4e2dbda15..f5ee85fa5 100644 --- a/solutions/search/search-templates.md +++ b/solutions/search/search-templates.md @@ -17,9 +17,9 @@ If you use {{es}} for a custom application, search templates let you change your ### Create a search template [create-search-template] -To create or update a search template, use the [create stored script API](https://www.elastic.co/guide/en/elasticsearch/reference/current/create-stored-script-api.html). +To create or update a search template, use the [create stored script API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-put-script). -The request’s `source` supports the same parameters as the [search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-search-api-request-body)'s request body.`source` also accepts [Mustache](https://mustache.github.io/) variables, from an open source project [mustache.java](https://github.com/spullara/mustache.java). +The request’s `source` supports the same parameters as the [search API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search)'s request body.`source` also accepts [Mustache](https://mustache.github.io/) variables, from an open source project [mustache.java](https://github.com/spullara/mustache.java). Typically [Mustache](https://mustache.github.io/) variables are enclosed in double curly brackets: `{{my-var}}`. When you run a templated search, {{es}} replaces these variables with values from `params`. To learn more about mustache syntax - see [Mustache.js manual](http://mustache.github.io/mustache.5.md) Search templates must use a `lang` of `mustache`. @@ -49,7 +49,7 @@ PUT _scripts/my-search-template ### Validate a search template [validate-search-template] $$$_validating_templates$$$ -To test a template with different `params`, use the [render search template API](https://www.elastic.co/guide/en/elasticsearch/reference/current/render-search-template-api.html). +To test a template with different `params`, use the [render search template API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-render-search-template). ```console POST _render/template @@ -63,7 +63,7 @@ POST _render/template } ``` -When rendered, the template outputs a [search request body](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-search-api-request-body). +When rendered, the template outputs a [search request body](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search). ```console-result { @@ -104,7 +104,7 @@ POST _render/template ### Run a templated search [run-templated-search] -To run a search with a search template, use the [search template API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-template-api.html). You can specify different `params` with each request. +To run a search with a search template, use the [search template API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search-template). You can specify different `params` with each request. ```console GET my-index/_search/template @@ -118,7 +118,7 @@ GET my-index/_search/template } ``` -The response uses the same properties as the [search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html)'s response. +The response uses the same properties as the [search API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search)'s response. ```console-result { @@ -153,7 +153,7 @@ The response uses the same properties as the [search API](https://www.elastic.co ### Run multiple templated searches [run-multiple-templated-searches] -To run multiple templated searches with a single request, use the [multi search template API](https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-search-template.html). These requests often have less overhead and faster speeds than multiple individual searches. +To run multiple templated searches with a single request, use the [multi search template API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-msearch-template). These requests often have less overhead and faster speeds than multiple individual searches. ```console GET my-index/_msearch/template @@ -166,13 +166,13 @@ GET my-index/_msearch/template ### Get search templates [get-search-templates] -To retrieve a search template, use the [get stored script API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-stored-script-api.html). +To retrieve a search template, use the [get stored script API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-get-script). ```console GET _scripts/my-search-template ``` -To get a list of all search templates and other stored scripts, use the [cluster state API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-state.html). +To get a list of all search templates and other stored scripts, use the [cluster state API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-state). ```console GET _cluster/state/metadata?pretty&filter_path=metadata.stored_scripts @@ -181,7 +181,7 @@ GET _cluster/state/metadata?pretty&filter_path=metadata.stored_scripts ### Delete a search template [delete-search-template] -To delete a search template, use the [delete stored script API](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-stored-script-api.html). +To delete a search template, use the [delete stored script API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-delete-script). ```console DELETE _scripts/my-search-template diff --git a/solutions/search/semantic-search.md b/solutions/search/semantic-search.md index 1199cd699..05acd3dd0 100644 --- a/solutions/search/semantic-search.md +++ b/solutions/search/semantic-search.md @@ -35,7 +35,7 @@ This diagram summarizes the relative complexity of each workflow: ### Option 1: `semantic_text` [_semantic_text_workflow] -The simplest way to use NLP models in the {{stack}} is through the [`semantic_text` workflow](semantic-search/semantic-search-semantic-text.md). We recommend using this approach because it abstracts away a lot of manual work. All you need to do is create an {{infer}} endpoint and an index mapping to start ingesting, embedding, and querying data. There is no need to define model-related settings and parameters, or to create {{infer}} ingest pipelines. Refer to the [Create an {{infer}} endpoint API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html) documentation for a list of supported services. +The simplest way to use NLP models in the {{stack}} is through the [`semantic_text` workflow](semantic-search/semantic-search-semantic-text.md). We recommend using this approach because it abstracts away a lot of manual work. All you need to do is create an {{infer}} endpoint and an index mapping to start ingesting, embedding, and querying data. There is no need to define model-related settings and parameters, or to create {{infer}} ingest pipelines. Refer to the [Create an {{infer}} endpoint API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put) documentation for a list of supported services. For an end-to-end tutorial, refer to [Semantic search with `semantic_text`](semantic-search/semantic-search-semantic-text.md). diff --git a/solutions/search/semantic-search/cohere-es.md b/solutions/search/semantic-search/cohere-es.md index 8e5eabb7b..51c849f16 100644 --- a/solutions/search/semantic-search/cohere-es.md +++ b/solutions/search/semantic-search/cohere-es.md @@ -79,7 +79,7 @@ print(client.info()) ## Create the {{infer}} endpoint [cohere-es-infer-endpoint] -[Create the {{infer}} endpoint](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html) first. In this example, the {{infer}} endpoint uses Cohere’s `embed-english-v3.0` model and the `embedding_type` is set to `byte`. +[Create the {{infer}} endpoint](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put) first. In this example, the {{infer}} endpoint uses Cohere’s `embed-english-v3.0` model and the `embedding_type` is set to `byte`. ```py COHERE_API_KEY = "cohere_api_key" diff --git a/solutions/search/semantic-search/semantic-search-inference.md b/solutions/search/semantic-search/semantic-search-inference.md index 894ad04d3..fe0335eb7 100644 --- a/solutions/search/semantic-search/semantic-search-inference.md +++ b/solutions/search/semantic-search/semantic-search-inference.md @@ -92,7 +92,7 @@ An [OpenAI account](https://openai.com/) is required to use the {{infer}} API wi ## Create an inference endpoint [infer-text-embedding-task] -Create an {{infer}} endpoint by using the [Create {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html): +Create an {{infer}} endpoint by using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put): :::::::{tab-set} @@ -110,7 +110,7 @@ PUT _inference/text_embedding/cohere_embeddings <1> ``` 1. The task type is `text_embedding` in the path and the `inference_id` which is the unique identifier of the {{infer}} endpoint is `cohere_embeddings`. -2. The API key of your Cohere account. You can find your API keys in your Cohere dashboard under the [API keys section](https://dashboard.cohere.com/api-keys). You need to provide your API key only once. The [Get {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-inference-api.html) does not return your API key. +2. The API key of your Cohere account. You can find your API keys in your Cohere dashboard under the [API keys section](https://dashboard.cohere.com/api-keys). You need to provide your API key only once. The [Get {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-get) does not return your API key. 3. The name of the embedding model to use. You can find the list of Cohere embedding models [here](https://docs.cohere.com/reference/embed). @@ -174,7 +174,7 @@ PUT _inference/text_embedding/openai_embeddings <1> ``` 1. The task type is `text_embedding` in the path and the `inference_id` which is the unique identifier of the {{infer}} endpoint is `openai_embeddings`. -2. The API key of your OpenAI account. You can find your OpenAI API keys in your OpenAI account under the [API keys section](https://platform.openai.com/api-keys). You need to provide your API key only once. The [Get {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-inference-api.html) does not return your API key. +2. The API key of your OpenAI account. You can find your OpenAI API keys in your OpenAI account under the [API keys section](https://platform.openai.com/api-keys). You need to provide your API key only once. The [Get {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-get) does not return your API key. 3. The name of the embedding model to use. You can find the list of OpenAI embedding models [here](https://platform.openai.com/docs/guides/embeddings/embedding-models). @@ -198,7 +198,7 @@ PUT _inference/text_embedding/azure_openai_embeddings <1> ``` 1. The task type is `text_embedding` in the path and the `inference_id` which is the unique identifier of the {{infer}} endpoint is `azure_openai_embeddings`. -2. The API key for accessing your Azure OpenAI services. Alternately, you can provide an `entra_id` instead of an `api_key` here. The [Get {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-inference-api.html) does not return this information. +2. The API key for accessing your Azure OpenAI services. Alternately, you can provide an `entra_id` instead of an `api_key` here. The [Get {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-get) does not return this information. 3. The name our your Azure resource. 4. The id of your deployed model. @@ -310,7 +310,7 @@ PUT _inference/text_embedding/alibabacloud_ai_search_embeddings <1> ``` 1. The task type is `text_embedding` in the path and the `inference_id` which is the unique identifier of the {{infer}} endpoint is `alibabacloud_ai_search_embeddings`. -2. The API key for accessing the AlibabaCloud AI Search API. You can find your API keys in your AlibabaCloud account under the [API keys section](https://opensearch.console.aliyun.com/cn-shanghai/rag/api-key). You need to provide your API key only once. The [Get {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-inference-api.html) does not return your API key. +2. The API key for accessing the AlibabaCloud AI Search API. You can find your API keys in your AlibabaCloud account under the [API keys section](https://opensearch.console.aliyun.com/cn-shanghai/rag/api-key). You need to provide your API key only once. The [Get {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-get) does not return your API key. 3. The AlibabaCloud AI Search embeddings model name, for example `ops-text-embedding-zh-001`. 4. The name our your AlibabaCloud AI Search host address. 5. The name our your AlibabaCloud AI Search workspace. @@ -619,7 +619,7 @@ PUT _ingest/pipeline/cohere_embeddings_pipeline } ``` -1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html), it’s referred to as `inference_id` in that step. +1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put), it’s referred to as `inference_id` in that step. 2. Configuration object that defines the `input_field` for the {{infer}} process and the `output_field` that will contain the {{infer}} results. :::::: @@ -641,7 +641,7 @@ PUT _ingest/pipeline/elser_embeddings_pipeline } ``` -1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html), it’s referred to as `inference_id` in that step. +1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put), it’s referred to as `inference_id` in that step. 2. Configuration object that defines the `input_field` for the {{infer}} process and the `output_field` that will contain the {{infer}} results. :::::: @@ -663,7 +663,7 @@ PUT _ingest/pipeline/hugging_face_embeddings_pipeline } ``` -1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html), it’s referred to as `inference_id` in that step. +1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put), it’s referred to as `inference_id` in that step. 2. Configuration object that defines the `input_field` for the {{infer}} process and the `output_field` that will contain the {{infer}} results. :::::: @@ -685,7 +685,7 @@ PUT _ingest/pipeline/openai_embeddings_pipeline } ``` -1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html), it’s referred to as `inference_id` in that step. +1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put), it’s referred to as `inference_id` in that step. 2. Configuration object that defines the `input_field` for the {{infer}} process and the `output_field` that will contain the {{infer}} results. :::::: @@ -707,7 +707,7 @@ PUT _ingest/pipeline/azure_openai_embeddings_pipeline } ``` -1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html), it’s referred to as `inference_id` in that step. +1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put), it’s referred to as `inference_id` in that step. 2. Configuration object that defines the `input_field` for the {{infer}} process and the `output_field` that will contain the {{infer}} results. :::::: @@ -729,7 +729,7 @@ PUT _ingest/pipeline/azure_ai_studio_embeddings_pipeline } ``` -1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html), it’s referred to as `inference_id` in that step. +1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put), it’s referred to as `inference_id` in that step. 2. Configuration object that defines the `input_field` for the {{infer}} process and the `output_field` that will contain the {{infer}} results. :::::: @@ -751,7 +751,7 @@ PUT _ingest/pipeline/google_vertex_ai_embeddings_pipeline } ``` -1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html), it’s referred to as `inference_id` in that step. +1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put), it’s referred to as `inference_id` in that step. 2. Configuration object that defines the `input_field` for the {{infer}} process and the `output_field` that will contain the {{infer}} results. :::::: @@ -773,7 +773,7 @@ PUT _ingest/pipeline/mistral_embeddings_pipeline } ``` -1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html), it’s referred to as `inference_id` in that step. +1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put), it’s referred to as `inference_id` in that step. 2. Configuration object that defines the `input_field` for the {{infer}} process and the `output_field` that will contain the {{infer}} results. :::::: @@ -795,7 +795,7 @@ PUT _ingest/pipeline/amazon_bedrock_embeddings_pipeline } ``` -1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html), it’s referred to as `inference_id` in that step. +1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put), it’s referred to as `inference_id` in that step. 2. Configuration object that defines the `input_field` for the {{infer}} process and the `output_field` that will contain the {{infer}} results. :::::: @@ -817,7 +817,7 @@ PUT _ingest/pipeline/alibabacloud_ai_search_embeddings_pipeline } ``` -1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html), it’s referred to as `inference_id` in that step. +1. The name of the inference endpoint you created by using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put), it’s referred to as `inference_id` in that step. 2. Configuration object that defines the `input_field` for the {{infer}} process and the `output_field` that will contain the {{infer}} results. :::::: @@ -834,7 +834,7 @@ Download the file and upload it to your cluster using the [Data Visualizer](../. ## Ingest the data through the {{infer}} ingest pipeline [reindexing-data-infer] -Create embeddings from the text by reindexing the data through the {{infer}} pipeline that uses your chosen model. This step uses the [reindex API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html) to simulate data ingestion through a pipeline. +Create embeddings from the text by reindexing the data through the {{infer}} pipeline that uses your chosen model. This step uses the [reindex API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-reindex) to simulate data ingestion through a pipeline. :::::::{tab-set} diff --git a/solutions/search/semantic-search/semantic-search-semantic-text.md b/solutions/search/semantic-search/semantic-search-semantic-text.md index 3dc8a8d79..896d77500 100644 --- a/solutions/search/semantic-search/semantic-search-semantic-text.md +++ b/solutions/search/semantic-search/semantic-search-semantic-text.md @@ -26,7 +26,7 @@ This tutorial uses the [`elasticsearch` service](../inference-api/elasticsearch- ## Requirements [semantic-text-requirements] -This tutorial uses the [`elasticsearch` service](../inference-api/elasticsearch-inference-integration.md) for demonstration, which is created automatically as needed. To use the `semantic_text` field type with an {{infer}} service other than `elasticsearch` service, you must create an inference endpoint using the [Create {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html). +This tutorial uses the [`elasticsearch` service](../inference-api/elasticsearch-inference-integration.md) for demonstration, which is created automatically as needed. To use the `semantic_text` field type with an {{infer}} service other than `elasticsearch` service, you must create an inference endpoint using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put). ## Create the index mapping [semantic-text-index-mapping] @@ -47,11 +47,11 @@ PUT semantic-embeddings ``` 1. The name of the field to contain the generated embeddings. -2. The field to contain the embeddings is a `semantic_text` field. Since no `inference_id` is provided, the default endpoint `.elser-2-elasticsearch` for the [`elasticsearch` service](../inference-api/elasticsearch-inference-integration.md) is used. To use a different {{infer}} service, you must create an {{infer}} endpoint first using the [Create {{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html) and then specify it in the `semantic_text` field mapping using the `inference_id` parameter. +2. The field to contain the embeddings is a `semantic_text` field. Since no `inference_id` is provided, the default endpoint `.elser-2-elasticsearch` for the [`elasticsearch` service](../inference-api/elasticsearch-inference-integration.md) is used. To use a different {{infer}} service, you must create an {{infer}} endpoint first using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put) and then specify it in the `semantic_text` field mapping using the `inference_id` parameter. ::::{note} -If you’re using web crawlers or connectors to generate indices, you have to [update the index mappings](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html) for these indices to include the `semantic_text` field. Once the mapping is updated, you’ll need to run a full web crawl or a full connector sync. This ensures that all existing documents are reprocessed and updated with the new semantic embeddings, enabling semantic search on the updated data. +If you’re using web crawlers or connectors to generate indices, you have to [update the index mappings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-mapping) for these indices to include the `semantic_text` field. Once the mapping is updated, you’ll need to run a full web crawl or a full connector sync. This ensures that all existing documents are reprocessed and updated with the new semantic embeddings, enabling semantic search on the updated data. :::: diff --git a/solutions/search/site-or-app/behavioral-analytics-api.md b/solutions/search/site-or-app/behavioral-analytics-api.md index ef6526320..30f95f73e 100644 --- a/solutions/search/site-or-app/behavioral-analytics-api.md +++ b/solutions/search/site-or-app/behavioral-analytics-api.md @@ -16,7 +16,7 @@ This page outlines all the APIs available for behavioral analytics and links to ## Behavioral Analytics REST APIs [behavioral-analytics-api-es-rest] -Behavioral Analytics relies on a number of {{es}} APIs to manage analytics collections. Refer to the [API documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/behavioral-analytics-apis.html) for the details. +Behavioral Analytics relies on a number of {{es}} APIs to manage analytics collections. Refer to the [API documentation](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-analytics) for the details. ## Behavioral Analytics Clients [behavioral-analytics-api-clients] diff --git a/solutions/search/site-or-app/behavioral-analytics-start.md b/solutions/search/site-or-app/behavioral-analytics-start.md index cceebd11f..e4fbf401b 100644 --- a/solutions/search/site-or-app/behavioral-analytics-start.md +++ b/solutions/search/site-or-app/behavioral-analytics-start.md @@ -23,7 +23,7 @@ Using behavioral analytics is a three-step process: ## Collections [behavioral-analytics-start-collections] ::::{note} -This guide focuses on using the Behavioral Analytics UI in {{kib}} to create and manage collections. You can also use the [Behavioral Analytics APIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/behavioral-analytics-apis.html) to create, list, and delete collections, as well as post events to a collection. +This guide focuses on using the Behavioral Analytics UI in {{kib}} to create and manage collections. You can also use the [Behavioral Analytics APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-analytics) to create, list, and delete collections, as well as post events to a collection. :::: diff --git a/solutions/search/site-or-app/behavioral-analytics.md b/solutions/search/site-or-app/behavioral-analytics.md index 4909b66df..c0c89bdcf 100644 --- a/solutions/search/site-or-app/behavioral-analytics.md +++ b/solutions/search/site-or-app/behavioral-analytics.md @@ -57,7 +57,7 @@ Additional documentation is available in the following places: * Read the [Browser tracker README](https://github.com/elastic/behavioral-analytics-tracker/blob/main/packages/browser-tracker/README.md). * The Search UI documentation contains information about the [Search UI Analytics Plugin](https://docs.elastic.co/search-ui/api/core/plugins/analytics-plugin). -* Behavioral Analytics uses a number of [APIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/behavioral-analytics-apis.html) to manage analytics collections. +* Behavioral Analytics uses a number of [APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-analytics) to manage analytics collections. ::::{admonition} GDPR compliance :class: note diff --git a/solutions/search/the-search-api.md b/solutions/search/the-search-api.md index 4c48ea248..0fe72b96a 100644 --- a/solutions/search/the-search-api.md +++ b/solutions/search/the-search-api.md @@ -13,7 +13,7 @@ A *search* consists of one or more queries that are combined and sent to {{es}}. A search may also contain additional information used to better process its queries. For example, a search may be limited to a specific index or only return a specific number of results. -You can use the [search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html) to search and [aggregate](../../explore-analyze/query-filter/aggregations.md) data stored in {{es}} data streams or indices. The API’s `query` request body parameter accepts queries written in [Query DSL](../../explore-analyze/query-filter/languages/querydsl.md). +You can use the [search API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search) to search and [aggregate](../../explore-analyze/query-filter/aggregations.md) data stored in {{es}} data streams or indices. The API’s `query` request body parameter accepts queries written in [Query DSL](../../explore-analyze/query-filter/languages/querydsl.md). ## Run a search [run-an-es-search] @@ -184,12 +184,12 @@ GET /my-index-000001/_search } ``` -To set a cluster-wide default timeout for all search requests, configure `search.default_search_timeout` using the [cluster settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html). This global timeout duration is used if no `timeout` argument is passed in the request. If the global search timeout expires before the search request finishes, the request is cancelled using [task cancellation](https://www.elastic.co/guide/en/elasticsearch/reference/current/tasks.html#task-cancellation). The `search.default_search_timeout` setting defaults to `-1` (no timeout). +To set a cluster-wide default timeout for all search requests, configure `search.default_search_timeout` using the [cluster settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings). This global timeout duration is used if no `timeout` argument is passed in the request. If the global search timeout expires before the search request finishes, the request is cancelled using [task cancellation](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-tasks). The `search.default_search_timeout` setting defaults to `-1` (no timeout). ## Search cancellation [global-search-cancellation] -You can cancel a search request using the [task management API](https://www.elastic.co/guide/en/elasticsearch/reference/current/tasks.html#task-cancellation). {{es}} also automatically cancels a search request when your client’s HTTP connection closes. We recommend you set up your client to close HTTP connections when a search request is aborted or times out. +You can cancel a search request using the [task management API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-tasks). {{es}} also automatically cancels a search request when your client’s HTTP connection closes. We recommend you set up your client to close HTTP connections when a search request is aborted or times out. ## Track total hits [track-total-hits] diff --git a/solutions/search/using-resolve-cluster-endpoint-before-cross-cluster-search.md b/solutions/search/using-resolve-cluster-endpoint-before-cross-cluster-search.md index b461dac7f..06a4c6811 100644 --- a/solutions/search/using-resolve-cluster-endpoint-before-cross-cluster-search.md +++ b/solutions/search/using-resolve-cluster-endpoint-before-cross-cluster-search.md @@ -113,10 +113,10 @@ GET /_resolve/cluster/not-present,clust*:my-index*,oldcluster:*?ignore_unavailab ``` 1. The local cluster has no index called `not_present`. Searching against it using the specified `ignore_unavailable=false` param will return a "no such index" error. Other types of errors can show up here as well, such as security permission errors when the user does not have authorization to search the specified index. -2. The `cluster_one` remote cluster has no indices that match the pattern `my-index*`. There may be no indices that match the pattern or the index could be closed. (You can check this by using the [resolve index](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-resolve-index-api.html) API.) +2. The `cluster_one` remote cluster has no indices that match the pattern `my-index*`. There may be no indices that match the pattern or the index could be closed. (You can check this by using the [resolve index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-resolve-index) API.) 3. The `cluster_two` remote cluster is not connected (the attempt to connect failed). Since this cluster is marked as `skip_unavailable=false`, you should probably exclude this cluster from the search by adding `-cluster_two:*` to the search index expression. 4. For `cluster_three`, the error message indicates that this remote cluster did not respond within the 5-second timeout window specified, so it is also marked as not connected. -5. The `oldcluster` remote cluster shows that it has matching indices, but no version information is included. This indicates that the cluster version predates the introduction of the `_resolve/cluster` API in 8.13.0., so you may want to exclude it from your {{ccs}}. (Note: the endpoint was able to tell there were matching indices because it fell back to using the [resolve index](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-resolve-index-api.html) API.) +5. The `oldcluster` remote cluster shows that it has matching indices, but no version information is included. This indicates that the cluster version predates the introduction of the `_resolve/cluster` API in 8.13.0., so you may want to exclude it from your {{ccs}}. (Note: the endpoint was able to tell there were matching indices because it fell back to using the [resolve index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-resolve-index) API.) diff --git a/solutions/search/vector/bring-own-vectors.md b/solutions/search/vector/bring-own-vectors.md index 2e782da33..08048c24e 100644 --- a/solutions/search/vector/bring-own-vectors.md +++ b/solutions/search/vector/bring-own-vectors.md @@ -82,7 +82,7 @@ PUT /amazon-reviews/_doc/1 ### Bulk index multiple documents [_bulk_index_multiple_documents] -In a production scenario, you’ll want to index many documents at once using the [`_bulk` endpoint](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html). +In a production scenario, you’ll want to index many documents at once using the [`_bulk` endpoint](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk). Here’s an example of indexing multiple documents in a single `_bulk` request. @@ -101,7 +101,7 @@ POST /_bulk ## Step 3: Search documents with embeddings [bring-your-own-vectors-search-documents] -Now you can query these document vectors using a [`knn` retriever](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#knn-retriever). `knn` is a type of vector search, which finds the `k` most similar documents to a query vector. Here we’re simply using a raw vector for the query text, for demonstration purposes. +Now you can query these document vectors using a [`knn` retriever](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever). `knn` is a type of vector search, which finds the `k` most similar documents to a query vector. Here we’re simply using a raw vector for the query text, for demonstration purposes. ```console POST /amazon-reviews/_search diff --git a/solutions/search/vector/dense-versus-sparse-ingest-pipelines.md b/solutions/search/vector/dense-versus-sparse-ingest-pipelines.md index 707db3865..0e3d9ead0 100644 --- a/solutions/search/vector/dense-versus-sparse-ingest-pipelines.md +++ b/solutions/search/vector/dense-versus-sparse-ingest-pipelines.md @@ -12,7 +12,7 @@ applies: ::::{important} * For the easiest way to perform semantic search in the {{stack}}, refer to the [`semantic_text`](../semantic-search/semantic-search-semantic-text.md) end-to-end tutorial. -* This tutorial was written before the [{{infer}} endpoint](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html) and [`semantic_text` field type](https://www.elastic.co/guide/en/elasticsearch/reference/current/semantic-text.html) was introduced. Today we have simpler options for performing semantic search. +* This tutorial was written before the [{{infer}} endpoint](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference) and [`semantic_text` field type](https://www.elastic.co/guide/en/elasticsearch/reference/current/semantic-text.html) was introduced. Today we have simpler options for performing semantic search. :::: @@ -233,7 +233,7 @@ Combining semantic and lexical search into one hybrid search request using [reci :::::::{tab-set} ::::::{tab-item} ELSER -Hybrid search between a semantic and lexical query can be achieved by using an [`rrf` retriever](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#rrf-retriever) as part of your search request. Provide a `sparse_vector` query and a full-text query as [`standard` retrievers](https://www.elastic.co/guide/en/elasticsearch/reference/current/retriever.html#standard-retriever) for the `rrf` retriever. The `rrf` retriever uses [reciprocal rank fusion](https://www.elastic.co/guide/en/elasticsearch/reference/current/rrf.html) to rank the top documents. +Hybrid search between a semantic and lexical query can be achieved by using an [`rrf` retriever](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever) as part of your search request. Provide a `sparse_vector` query and a full-text query as [`standard` retrievers](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-retriever) for the `rrf` retriever. The `rrf` retriever uses [reciprocal rank fusion](https://www.elastic.co/guide/en/elasticsearch/reference/current/rrf.html) to rank the top documents. ```console GET my-index/_search diff --git a/solutions/search/vector/knn.md b/solutions/search/vector/knn.md index 654eb0cfb..c4c2b2b45 100644 --- a/solutions/search/vector/knn.md +++ b/solutions/search/vector/knn.md @@ -58,7 +58,7 @@ Exact, brute-force kNN guarantees accurate results but doesn’t scale well with Compared to other types of search, approximate kNN search has specific resource requirements. In particular, all vector data must fit in the node’s page cache for it to be efficient. Please consult the [approximate kNN search tuning guide](/deploy-manage/production-guidance/optimize-performance/approximate-knn-search.md) for important notes on configuration and sizing. :::: -To run an approximate kNN search, use the [`knn` option](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-api-knn) to search one or more `dense_vector` fields with indexing enabled. +To run an approximate kNN search, use the [`knn` option](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-knn) to search one or more `dense_vector` fields with indexing enabled. 1. Explicitly map one or more `dense_vector` fields. Approximate kNN search requires the following mapping options: @@ -103,7 +103,7 @@ To run an approximate kNN search, use the [`knn` option](https://www.elastic.co/ ... ``` -3. Run the search using the [`knn` option](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-api-knn) or the [`knn` query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-knn-query.html) (expert case). +3. Run the search using the [`knn` option](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-knn) or the [`knn` query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-knn-query.html) (expert case). ```console POST image-index/_search @@ -119,7 +119,7 @@ To run an approximate kNN search, use the [`knn` option](https://www.elastic.co/ ``` -The [document `_score`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-api-response-body-score) is determined by the similarity between the query and document vector. See [`similarity`](https://www.elastic.co/guide/en/elasticsearch/reference/current/dense-vector.html#dense-vector-similarity) for more information on how kNN search scores are computed. +The document `_score` is a positive 32-bit floating point number used to score the relevance of the returned document, determined by the similarity between the query and document vector. See [`similarity`](https://www.elastic.co/guide/en/elasticsearch/reference/current/dense-vector.html#dense-vector-similarity) for more information on how kNN search scores are computed. ::::{note} Support for approximate kNN search was added in version 8.0. Before this, `dense_vector` fields did not support enabling `index` in the mapping. If you created an index prior to 8.0 containing `dense_vector` fields, then to support approximate kNN search the data must be reindexed using a new field mapping that sets `index: true` which is the default option. @@ -162,7 +162,7 @@ Similarly, you can decrease `num_candidates` for faster searches with potentiall ### Approximate kNN using byte vectors [approximate-knn-using-byte-vectors] -The approximate kNN search API supports `byte` value vectors in addition to `float` value vectors. Use the [`knn` option](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-api-knn) to search a `dense_vector` field with [`element_type`](https://www.elastic.co/guide/en/elasticsearch/reference/current/dense-vector.html#dense-vector-params) set to `byte` and indexing enabled. +The approximate kNN search API supports `byte` value vectors in addition to `float` value vectors. Use the [`knn` option](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-knn) to search a `dense_vector` field with [`element_type`](https://www.elastic.co/guide/en/elasticsearch/reference/current/dense-vector.html#dense-vector-params) set to `byte` and indexing enabled. 1. Explicitly map one or more `dense_vector` fields with [`element_type`](https://www.elastic.co/guide/en/elasticsearch/reference/current/dense-vector.html#dense-vector-params) set to `byte` and indexing enabled. @@ -196,7 +196,7 @@ The approximate kNN search API supports `byte` value vectors in addition to `flo { "byte-image-vector": [11, 23], "title": "full moon" } ``` -3. Run the search using the [`knn` option](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-api-knn) ensuring the `query_vector` values are integers within the range [-128, 127]. +3. Run the search using the [`knn` option](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-knn) ensuring the `query_vector` values are integers within the range [-128, 127]. ```console POST byte-image-index/_search @@ -273,7 +273,7 @@ PUT quantized-image-index { "image-vector": [1.2, 0.1], "title": "full moon" } ``` -2. Run the search using the [`knn` option](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-api-knn). When searching, the `float` vector is automatically quantized to a `byte` vector. +2. Run the search using the [`knn` option](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-knn). When searching, the `float` vector is automatically quantized to a `byte` vector. ```console POST quantized-image-index/_search @@ -366,7 +366,7 @@ To avoid significant performance drawbacks, Lucene implements the following stra ### Combine approximate kNN with other features [_combine_approximate_knn_with_other_features] -You can perform *hybrid retrieval* by providing both the [`knn` option](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-api-knn) and a [`query`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#request-body-search-query): +You can perform *hybrid retrieval* by providing both the [`knn` option](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-body-application-json-knn) and a [`query`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#operation-search-query): ```console POST image-index/_search @@ -891,7 +891,7 @@ Now the result will contain the nearest found paragraph when searching. * {{es}} uses the [HNSW algorithm](https://arxiv.org/abs/1603.09320) to support efficient kNN search. Like most kNN algorithms, HNSW is an approximate method that sacrifices result accuracy for improved search speed. This means the results returned are not always the true *k* closest neighbors. ::::{note} -Approximate kNN search always uses the [`dfs_query_then_fetch`](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#dfs-query-then-fetch) search type in order to gather the global top `k` matches across shards. You cannot set the `search_type` explicitly when running kNN search. +Approximate kNN search always uses the [`dfs_query_then_fetch`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search) search type in order to gather the global top `k` matches across shards. You cannot set the `search_type` explicitly when running kNN search. :::: @@ -1075,7 +1075,7 @@ To run an exact kNN search, use a `script_score` query with a vector function. ... ``` -3. Use the [search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html) to run a `script_score` query containing a [vector function](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-script-score-query.html#vector-functions). +3. Use the [search API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search) to run a `script_score` query containing a [vector function](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-script-score-query.html#vector-functions). ::::{tip} To limit the number of matched documents passed to the vector function, we recommend you specify a filter query in the `script_score.query` parameter. If needed, you can use a [`match_all` query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-all-query.html) in this parameter to match all documents. However, matching all documents can significantly increase search latency.