forked from opensearch-project/documentation-website
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge remote-tracking branch 'upstream/main'
- Loading branch information
Showing
118 changed files
with
3,491 additions
and
308 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -34,8 +34,7 @@ This project has adopted an [Open Source Code of Conduct](https://opensearch.org | |
|
|
||
| ## Security | ||
|
|
||
| If you discover a potential security issue in this project, we ask that you notify AWS/Amazon Security using our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Do **not** create a public GitHub issue. | ||
|
|
||
| If you discover a potential security issue in this project, notify OpenSearch Security directly by emailing [email protected]. To prevent any additional risk caused by the potential issue, do **not** create a public GitHub issue. | ||
|
|
||
| ## License | ||
|
|
||
|
|
||
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,158 @@ | ||
| --- | ||
| layout: default | ||
| title: Median absolute deviation | ||
| parent: Metric aggregations | ||
| grand_parent: Aggregations | ||
| nav_order: 65 | ||
| redirect_from: | ||
| - /query-dsl/aggregations/metric/median-absolute-deviation/ | ||
| --- | ||
|
|
||
| # Median absolute deviation aggregations | ||
|
|
||
| The `median_absolute_deviation` metric is a single-value metric aggregation that returns a median absolute deviation field. Median absolute deviation is a statistical measure of data variability. Because the median absolute deviation measures dispersion from the median, it provides a more robust measure of variability that is less affected by outliers in a dataset. | ||
|
|
||
| Median absolute deviation is calculated as follows:<br> | ||
| median_absolute_deviation = median(|X<sub>i</sub> - Median(X<sub>i</sub>)|) | ||
|
|
||
| The following example calculates the median absolute deviation of the `DistanceMiles` field in the sample dataset `opensearch_dashboards_sample_data_flights`: | ||
|
|
||
|
|
||
| ```json | ||
| GET opensearch_dashboards_sample_data_flights/_search | ||
| { | ||
| "size": 0, | ||
| "aggs": { | ||
| "median_absolute_deviation_DistanceMiles": { | ||
| "median_absolute_deviation": { | ||
| "field": "DistanceMiles" | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
| {% include copy-curl.html %} | ||
|
|
||
| #### Example response | ||
|
|
||
| ```json | ||
| { | ||
| "took": 35, | ||
| "timed_out": false, | ||
| "_shards": { | ||
| "total": 1, | ||
| "successful": 1, | ||
| "skipped": 0, | ||
| "failed": 0 | ||
| }, | ||
| "hits": { | ||
| "total": { | ||
| "value": 10000, | ||
| "relation": "gte" | ||
| }, | ||
| "max_score": null, | ||
| "hits": [] | ||
| }, | ||
| "aggregations": { | ||
| "median_absolute_deviation_distanceMiles": { | ||
| "value": 1829.8993624441966 | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Missing | ||
|
|
||
| By default, if a field is missing or has a null value in a document, it is ignored during computation. However, you can specify a value to be used for those missing or null fields by using the `missing` parameter, as shown in the following request: | ||
|
|
||
| ```json | ||
| GET opensearch_dashboards_sample_data_flights/_search | ||
| { | ||
| "size": 0, | ||
| "aggs": { | ||
| "median_absolute_deviation_distanceMiles": { | ||
| "median_absolute_deviation": { | ||
| "field": "DistanceMiles", | ||
| "missing": 1000 | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
| {% include copy-curl.html %} | ||
|
|
||
| #### Example response | ||
|
|
||
| ```json | ||
| { | ||
| "took": 7, | ||
| "timed_out": false, | ||
| "_shards": { | ||
| "total": 1, | ||
| "successful": 1, | ||
| "skipped": 0, | ||
| "failed": 0 | ||
| }, | ||
| "hits": { | ||
| "total": { | ||
| "value": 10000, | ||
| "relation": "gte" | ||
| }, | ||
| "max_score": null, | ||
| "hits": [] | ||
| }, | ||
| "aggregations": { | ||
| "median_absolute_deviation_distanceMiles": { | ||
| "value": 1829.6443646143355 | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Compression | ||
|
|
||
| The median absolute deviation is calculated using the [t-digest](https://github.com/tdunning/t-digest/tree/main) data structure, which balances between performance and estimation accuracy through the `compression` parameter (default value: `1000`). Adjusting the `compression` value affects the trade-off between computational efficiency and precision. Lower `compression` values improve performance but may reduce estimation accuracy, while higher values enhance accuracy at the cost of increased computational overhead, as shown in the following request: | ||
|
|
||
| ```json | ||
| GET opensearch_dashboards_sample_data_flights/_search | ||
| { | ||
| "size": 0, | ||
| "aggs": { | ||
| "median_absolute_deviation_DistanceMiles": { | ||
| "median_absolute_deviation": { | ||
| "field": "DistanceMiles", | ||
| "compression": 10 | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
| {% include copy-curl.html %} | ||
|
|
||
| #### Example response | ||
|
|
||
| ```json | ||
| { | ||
| "took": 1, | ||
| "timed_out": false, | ||
| "_shards": { | ||
| "total": 1, | ||
| "successful": 1, | ||
| "skipped": 0, | ||
| "failed": 0 | ||
| }, | ||
| "hits": { | ||
| "total": { | ||
| "value": 10000, | ||
| "relation": "gte" | ||
| }, | ||
| "max_score": null, | ||
| "hits": [] | ||
| }, | ||
| "aggregations": { | ||
| "median_absolute_deviation_DistanceMiles": { | ||
| "value": 1836.265614211182 | ||
| } | ||
| } | ||
| } | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| --- | ||
| layout: default | ||
| title: Refresh index | ||
| parent: Index APIs | ||
| nav_order: 61 | ||
| --- | ||
|
|
||
| # Refresh index | ||
| Introduced 1.0 | ||
| {: .label .label-purple } | ||
|
|
||
| The Refresh Index API refreshes one or more indexes in an OpenSearch cluster. In the case of data streams, the Refresh Index API refreshes a stream's backing indexes. | ||
|
|
||
| OpenSearch's refresh behavior depends on whether or not `index.refresh_interval` is set: | ||
|
|
||
| - When set, indexes are refreshed based on the `index.refresh_interval` setting (in seconds). For more information about `index.refresh_interval` settings, see [Dynamic index-level index settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index-settings/#dynamic-index-level-index-settings). | ||
| - When not set, refreshes occur every second until the shard receives no search requests for at least the amount of time specified by the `index.search.idle.after` setting (in seconds). Default is `30s`. | ||
|
|
||
| After a shard becomes idle, the indexes will not refresh until either the next search request or a Refresh Index API request is sent. The first search request on an idle shard will wait for the refresh operation to complete. | ||
|
|
||
| To use the Refresh Index API, you must have write access to the indexes you want to refresh. | ||
|
|
||
| ## Path and HTTP methods | ||
|
|
||
| ```json | ||
| POST /_refresh | ||
| GET /_refresh | ||
| POST /<index>/_refresh | ||
| GET /<index>/_refresh | ||
| ``` | ||
|
|
||
| ## Path parameters | ||
|
|
||
| The following table lists the available path parameters. All path parameters are optional. | ||
|
|
||
| | Parameter | Data type | Description | | ||
| | :--- | :--- | :--- | | ||
| | `index` | String | A comma-separated list of index names to be refreshed. Wildcards are accepted.| | ||
|
|
||
| ## Query parameters | ||
|
|
||
| The following table lists the available query parameters. All query parameters are optional. | ||
|
|
||
| | Parameter | Data type | Description | | ||
| | :--- | :--- | :--- | | ||
| | `ignore_unavailable` | Boolean | When `false`, the request returns an error when it targets a missing or closed index. Default is `false`. | ||
| | `allow_no_indices` | Boolean | When `false`, the Refresh Index API returns an error when a wildcard expression, index alias, or `_all` targets only closed or missing indexes, even when the request is made against open indexes. Default is `true`. | | ||
| | `expand_wildcard` | String | The type of index that the wildcard patterns can match. If the request targets data streams, this argument determines whether the wildcard expressions match any hidden data streams. Supports comma-separated values, such as `open,hidden`. Valid values are `all`, `open`, `closed`, `hidden`, and `none`. | ||
|
|
||
|
|
||
|
|
||
| #### Example: Refresh several data streams or indexes | ||
|
|
||
| The following example request refreshes two indexes named `my-index-A` and `my-index-B`: | ||
|
|
||
|
|
||
| ``` | ||
| POST /my-index-A,my-index-B/_refresh | ||
| ``` | ||
| {% include copy-curl.html %} | ||
|
|
||
| #### Example: Refresh all data streams and indexes in a cluster | ||
|
|
||
| The following request refreshes all data streams and indexes in a cluster: | ||
|
|
||
| ``` | ||
| POST /_refresh | ||
| ``` | ||
|
|
Oops, something went wrong.