Skip to content

Commit

Permalink
Merge branch 'main' into eck_links_asciidocalypse
Browse files Browse the repository at this point in the history
  • Loading branch information
eedugon authored Feb 20, 2025
2 parents d415db1 + fdb706b commit bf14439
Show file tree
Hide file tree
Showing 1,262 changed files with 8,564 additions and 12,189 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -70,5 +70,5 @@ To create a bearer token:
{ "token": "eyJ0eXa...<very long string>...MgBmsw4s" }
```
2. Specify the bearer token in the Authentication header of your API requests. To learn more, check [accessing the API from the command line](https://www.elastic.co/guide/en/cloud-enterprise/current/ece-api-command-line.html).
2. Specify the bearer token in the Authentication header of your API requests. To learn more, check [accessing the API from the command line](asciidocalypse://docs/cloud/docs/reference/cloud/cloud-enterprise/ece-api-command-line.md).
4 changes: 2 additions & 2 deletions deploy-manage/autoscaling/deployments-autoscaling-on-eck.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Elasticsearch autoscaling requires a valid Enterprise license or Enterprise tria
::::


ECK can leverage the [autoscaling API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-autoscaling) introduced in Elasticsearch 7.11 to adjust automatically the number of Pods and the allocated resources in a tier. Currently, autoscaling is supported for Elasticsearch [data tiers](https://www.elastic.co/guide/en/elasticsearch/reference/current/data-tiers.html) and machine learning nodes.
ECK can leverage the [autoscaling API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-autoscaling) introduced in Elasticsearch 7.11 to adjust automatically the number of Pods and the allocated resources in a tier. Currently, autoscaling is supported for Elasticsearch [data tiers](/manage-data/lifecycle/data-tiers.md) and machine learning nodes.


## Enable autoscaling [k8s-enable]
Expand Down Expand Up @@ -159,7 +159,7 @@ spec:
max: 512Gi
```

You can find [a complete example in the ECK GitHub repository](https://github.com/elastic/cloud-on-k8s/blob/2.16/config/recipes/autoscaling/elasticsearch.yaml) which will also show you how to fine-tune the [autoscaling deciders](https://www.elastic.co/guide/en/elasticsearch/reference/current/autoscaling-deciders.html).
You can find [a complete example in the ECK GitHub repository](https://github.com/elastic/cloud-on-k8s/blob/2.16/config/recipes/autoscaling/elasticsearch.yaml) which will also show you how to fine-tune the [autoscaling deciders](/deploy-manage/autoscaling/autoscaling-deciders.md).


### Change the polling interval [k8s-autoscaling-polling-interval]
Expand Down
2 changes: 1 addition & 1 deletion deploy-manage/autoscaling/ec-autoscaling-api-example.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ mapped_pages:

This example demonstrates how to use the Elasticsearch Service RESTful API to create a deployment with autoscaling enabled.

The example deployment has a hot data and content tier, warm data tier, cold data tier, and a machine learning node, all of which will scale within the defined parameters. To learn about the autoscaling settings, check [Deployment autoscaling](../autoscaling.md) and [Autoscaling example](ec-autoscaling-example.md). For more information about using the Elasticsearch Service API in general, check [RESTful API](https://www.elastic.co/guide/en/cloud/current/ec-restful-api.html).
The example deployment has a hot data and content tier, warm data tier, cold data tier, and a machine learning node, all of which will scale within the defined parameters. To learn about the autoscaling settings, check [Deployment autoscaling](../autoscaling.md) and [Autoscaling example](ec-autoscaling-example.md). For more information about using the Elasticsearch Service API in general, check [RESTful API](asciidocalypse://docs/cloud/docs/reference/cloud/cloud-hosted/ec-api-restful.md).


## Requirements [ec_requirements]
Expand Down
2 changes: 1 addition & 1 deletion deploy-manage/autoscaling/ec-autoscaling.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ When past behavior on a hot tier indicates that the influx of data can increase

* Through ILM policies. For example, if a deployment has only hot nodes and autoscaling is enabled, it automatically creates warm or cold nodes, if an ILM policy is trying to move data from hot to warm or cold nodes.

On machine learning nodes, scaling is determined by an estimate of the memory and CPU requirements for the currently configured jobs and trained models. When a new machine learning job tries to start, it looks for a node with adequate native memory and CPU capacity. If one cannot be found, it stays in an `opening` state. If this waiting job exceeds the queueing limit set in the machine learning decider, a scale up is requested. Conversely, as machine learning jobs run, their memory and CPU usage might decrease or other running jobs might finish or close. In this case, if the duration of decreased resource usage exceeds the set value for `down_scale_delay`, a scale down is requested. Check [Machine learning decider](autoscaling-deciders.md) for more detail. To learn more about machine learning jobs in general, check [Create anomaly detection jobs](https://www.elastic.co/guide/en/machine-learning/current/ml-ad-run-jobs.html).
On machine learning nodes, scaling is determined by an estimate of the memory and CPU requirements for the currently configured jobs and trained models. When a new machine learning job tries to start, it looks for a node with adequate native memory and CPU capacity. If one cannot be found, it stays in an `opening` state. If this waiting job exceeds the queueing limit set in the machine learning decider, a scale up is requested. Conversely, as machine learning jobs run, their memory and CPU usage might decrease or other running jobs might finish or close. In this case, if the duration of decreased resource usage exceeds the set value for `down_scale_delay`, a scale down is requested. Check [Machine learning decider](autoscaling-deciders.md) for more detail. To learn more about machine learning jobs in general, check [Create anomaly detection jobs](/explore-analyze/machine-learning/anomaly-detection/ml-ad-run-jobs.md).

On a highly available deployment, autoscaling events are always applied to instances in each availability zone simultaneously, to ensure consistency.

Expand Down
2 changes: 1 addition & 1 deletion deploy-manage/autoscaling/ece-autoscaling-api-example.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ mapped_pages:

This example demonstrates how to use the Elastic Cloud Enterprise RESTful API to create a deployment with autoscaling enabled.

The example deployment has a hot data and content tier, warm data tier, cold data tier, and a machine learning node, all of which will scale within the defined parameters. To learn about the autoscaling settings, check [Deployment autoscaling](../autoscaling.md) and [Autoscaling example](ece-autoscaling-example.md). For more information about using the Elastic Cloud Enterprise API in general, check [RESTful API](https://www.elastic.co/guide/en/cloud-enterprise/current/ece-restful-api.html).
The example deployment has a hot data and content tier, warm data tier, cold data tier, and a machine learning node, all of which will scale within the defined parameters. To learn about the autoscaling settings, check [Deployment autoscaling](../autoscaling.md) and [Autoscaling example](ece-autoscaling-example.md). For more information about using the Elastic Cloud Enterprise API in general, check [RESTful API](asciidocalypse://docs/cloud/docs/reference/cloud/cloud-enterprise/restful-api.md).


## Requirements [ece_requirements_3]
Expand Down
2 changes: 1 addition & 1 deletion deploy-manage/autoscaling/ece-autoscaling.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ When past behavior on a hot tier indicates that the influx of data can increase

* Through ILM policies. For example, if a deployment has only hot nodes and autoscaling is enabled, it automatically creates warm or cold nodes, if an ILM policy is trying to move data from hot to warm or cold nodes.

On machine learning nodes, scaling is determined by an estimate of the memory and CPU requirements for the currently configured jobs and trained models. When a new machine learning job tries to start, it looks for a node with adequate native memory and CPU capacity. If one cannot be found, it stays in an `opening` state. If this waiting job exceeds the queueing limit set in the machine learning decider, a scale up is requested. Conversely, as machine learning jobs run, their memory and CPU usage might decrease or other running jobs might finish or close. In this case, if the duration of decreased resource usage exceeds the set value for `down_scale_delay`, a scale down is requested. Check [Machine learning decider](autoscaling-deciders.md) for more detail. To learn more about machine learning jobs in general, check [Create anomaly detection jobs](https://www.elastic.co/guide/en/machine-learning/current/ml-ad-run-jobs.html).
On machine learning nodes, scaling is determined by an estimate of the memory and CPU requirements for the currently configured jobs and trained models. When a new machine learning job tries to start, it looks for a node with adequate native memory and CPU capacity. If one cannot be found, it stays in an `opening` state. If this waiting job exceeds the queueing limit set in the machine learning decider, a scale up is requested. Conversely, as machine learning jobs run, their memory and CPU usage might decrease or other running jobs might finish or close. In this case, if the duration of decreased resource usage exceeds the set value for `down_scale_delay`, a scale down is requested. Check [Machine learning decider](autoscaling-deciders.md) for more detail. To learn more about machine learning jobs in general, check [Create anomaly detection jobs](/explore-analyze/machine-learning/anomaly-detection/ml-ad-run-jobs.md).

On a highly available deployment, autoscaling events are always applied to instances in each availability zone simultaneously, to ensure consistency.

Expand Down
2 changes: 1 addition & 1 deletion deploy-manage/autoscaling/ech-autoscaling.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ When past behavior on a hot tier indicates that the influx of data can increase

* Through ILM policies. For example, if a deployment has only hot nodes and autoscaling is enabled, it automatically creates warm or cold nodes, if an ILM policy is trying to move data from hot to warm or cold nodes.

On machine learning nodes, scaling is determined by an estimate of the memory and CPU requirements for the currently configured jobs and trained models. When a new machine learning job tries to start, it looks for a node with adequate native memory and CPU capacity. If one cannot be found, it stays in an `opening` state. If this waiting job exceeds the queueing limit set in the machine learning decider, a scale up is requested. Conversely, as machine learning jobs run, their memory and CPU usage might decrease or other running jobs might finish or close. In this case, if the duration of decreased resource usage exceeds the set value for `down_scale_delay`, a scale down is requested. Check [Machine learning decider](autoscaling-deciders.md) for more detail. To learn more about machine learning jobs in general, check [Create anomaly detection jobs](https://www.elastic.co/guide/en/machine-learning/current/ml-ad-run-jobs.html).
On machine learning nodes, scaling is determined by an estimate of the memory and CPU requirements for the currently configured jobs and trained models. When a new machine learning job tries to start, it looks for a node with adequate native memory and CPU capacity. If one cannot be found, it stays in an `opening` state. If this waiting job exceeds the queueing limit set in the machine learning decider, a scale up is requested. Conversely, as machine learning jobs run, their memory and CPU usage might decrease or other running jobs might finish or close. In this case, if the duration of decreased resource usage exceeds the set value for `down_scale_delay`, a scale down is requested. Check [Machine learning decider](autoscaling-deciders.md) for more detail. To learn more about machine learning jobs in general, check [Create anomaly detection jobs](/explore-analyze/machine-learning/anomaly-detection/ml-ad-run-jobs.md).

On a highly available deployment, autoscaling events are always applied to instances in each availability zone simultaneously, to ensure consistency.

Expand Down
2 changes: 1 addition & 1 deletion deploy-manage/cloud-organization/billing/billing-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ $$$faq-dts$$$What are the data transfer and storage charges and how can I contro
$$$faq-taxes$$$What taxes will be applied on my invoice?
: Customers within the United States, and US territories, will be billed from Elasticsearch Inc., based out of the United States. The US Sales Tax rate will be based on the SaaS tax rates in the local jurisdiction (state/county/city) of the billing address of your subscription.

Customers outside the United States, will be billed from Elasticsearch BV, based out of the Netherlands. Customers with a billing address in countries with applicable EU VAT will have VAT applied based on their country and status as a business or private customer. Elastic collects VAT Numbers associated with EU VAT to determine your status as a business (B2B) or private / non-business customer (B2C), as this is a key factor to determine Elastic’s liability to charge VAT on your subscription. To update your VAT Number follow the instructions provided in [Add your billing details](https://www.elastic.co/guide/en/cloud/current/ec-billing-details.html). Customers located in countries without EU VAT will not be applied VAT on their invoices.
Customers outside the United States, will be billed from Elasticsearch BV, based out of the Netherlands. Customers with a billing address in countries with applicable EU VAT will have VAT applied based on their country and status as a business or private customer. Elastic collects VAT Numbers associated with EU VAT to determine your status as a business (B2B) or private / non-business customer (B2C), as this is a key factor to determine Elastic’s liability to charge VAT on your subscription. To update your VAT Number follow the instructions provided in [Add your billing details](/deploy-manage/cloud-organization/billing/add-billing-details.md). Customers located in countries without EU VAT will not be applied VAT on their invoices.



Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Data inter-node charges are currently waived for Azure deployments.

Data transfer out of deployments and between nodes of the cluster is hard to control, as it is a function of the use case employed for the cluster and cannot always be tuned. Use cases such as batch queries executed at a frequent interval may be revisited to help lower transfer costs, if applicable. Watcher email alerts also count towards data transfer out of the deployment, so you may want to reduce their frequency and size.

The largest contributor to inter-node data transfer is usually shard movement between nodes in a cluster. The only way to prevent shard movement is by having a single node in a single availability zone. This solution is only possible for clusters up to 64GB RAM and is not recommended as it creates a risk of data loss. [Oversharding](https://www.elastic.co/guide/en/elasticsearch/reference/current/size-your-shards.html) can cause excessive shard movement. Avoiding oversharding can also help control costs and improve performance. Note that creating snapshots generates inter-node data transfer. The *storage* cost of snapshots is detailed later in this document.
The largest contributor to inter-node data transfer is usually shard movement between nodes in a cluster. The only way to prevent shard movement is by having a single node in a single availability zone. This solution is only possible for clusters up to 64GB RAM and is not recommended as it creates a risk of data loss. [Oversharding](/deploy-manage/production-guidance/optimize-performance/size-shards.md) can cause excessive shard movement. Avoiding oversharding can also help control costs and improve performance. Note that creating snapshots generates inter-node data transfer. The *storage* cost of snapshots is detailed later in this document.

The exact root cause of unusual data transfer is not always something we can identify as it can have many causes, some of which are out of our control and not associated with Cloud configuration changes. It may help to [enable monitoring](../../monitor/stack-monitoring/elastic-cloud-stack-monitoring.md) and examine index and shard activity on your cluster.

Expand Down
8 changes: 4 additions & 4 deletions deploy-manage/cloud-organization/tools-and-apis.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,15 @@ Most Elastic resources can be accessed and managed through RESTful APIs. While t
Elasticsearch Service API
: You can use the Elasticsearch Service API to manage your deployments and all of the resources associated with them. This includes performing deployment CRUD operations, scaling or autoscaling resources, and managing traffic filters, deployment extensions, remote clusters, and Elastic Stack versions. You can also access cost data by deployment and by organization.

To learn more about the Elasticsearch Service API, read through the [API overview](https://www.elastic.co/guide/en/cloud/current/ec-restful-api.html), try out some [getting started examples](https://www.elastic.co/guide/en/cloud/current/ec-api-examples.html), and check our [API reference documentation](https://www.elastic.co/docs/api/doc/cloud).
To learn more about the Elasticsearch Service API, read through the [API overview](asciidocalypse://docs/cloud/docs/reference/cloud/cloud-hosted/ec-api-restful.md), try out some [getting started examples](asciidocalypse://docs/cloud/docs/reference/cloud/cloud-hosted/api-examples.md), and check our [API reference documentation](https://www.elastic.co/docs/api/doc/cloud).

Calls to the Elasticsearch Service API are subject to [Rate limiting](https://www.elastic.co/guide/en/cloud/current/ec-api-rate-limiting.html).
Calls to the Elasticsearch Service API are subject to [Rate limiting](asciidocalypse://docs/cloud/docs/reference/cloud/cloud-hosted/ec-api-rate-limiting.md).


Elasticsearch APIs
: This set of APIs allows you to interact directly with the Elasticsearch nodes in your deployment. You can ingest data, run search queries, check the health of your clusters, manage snapshots, and more.

To use these APIs in Elasticsearch Service read our topic [Access the Elasticsearch API console](https://www.elastic.co/guide/en/cloud/current/ec-api-console.html), and to learn about all of the available endpoints check the [Elasticsearch API reference documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/rest-apis.html).
To use these APIs in Elasticsearch Service read our topic [Access the Elasticsearch API console](asciidocalypse://docs/cloud/docs/reference/cloud/cloud-hosted/ec-api-console.md), and to learn about all of the available endpoints check the [Elasticsearch API reference documentation](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/rest-apis/index.md).

Some [restrictions](../deploy/elastic-cloud/restrictions-known-problems.md#ec-restrictions-apis-elasticsearch) apply when using the Elasticsearch APIs in Elasticsearch Service.

Expand All @@ -37,7 +37,7 @@ Other Products
* [APM event intake API Reference](/solutions/observability/apps/elastic-apm-events-intake-api.md)
* [App Search API Reference](https://www.elastic.co/guide/en/app-search/current/api-reference.html)
* [Elastic Security APIs](https://www.elastic.co/guide/en/security/current/security-apis.html)
* [Fleet APIs](https://www.elastic.co/guide/en/fleet/current/fleet-api-docs.html)
* [Fleet APIs](asciidocalypse://docs/docs-content/docs/reference/ingestion-tools/fleet/fleet-api-docs.md)
* [Logstash APIs](https://www.elastic.co/guide/en/logstash/current/monitoring-logstash.html)
* [Workplace Search API Reference](https://www.elastic.co/guide/en/workplace-search/current/workplace-search-api-overview.html)

Expand Down
1 change: 1 addition & 0 deletions deploy-manage/deploy/cloud-enterprise.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
mapped_urls:
- https://www.elastic.co/guide/en/cloud-enterprise/current/index.html
- https://www.elastic.co/guide/en/cloud-enterprise/current/Elastic-Cloud-Enterprise-overview.html
- https://www.elastic.co/guide/en/cloud-enterprise/current/ece-administering-ece.html
---
Expand Down
Loading

0 comments on commit bf14439

Please sign in to comment.