diff --git a/deploy-manage/api-keys.md b/deploy-manage/api-keys.md index 9f3a35c05..d72c08bf3 100644 --- a/deploy-manage/api-keys.md +++ b/deploy-manage/api-keys.md @@ -4,4 +4,6 @@ % GitHub issue: https://github.com/elastic/docs-projects/issues/349 -% Scope notes: Elasticsearch & Kibana authentication API Keys \ No newline at end of file +% Scope notes: Elasticsearch & Kibana authentication API Keys + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/autoscaling/autoscaling-deciders.md b/deploy-manage/autoscaling/autoscaling-deciders.md index d2646d233..1289aa165 100644 --- a/deploy-manage/autoscaling/autoscaling-deciders.md +++ b/deploy-manage/autoscaling/autoscaling-deciders.md @@ -14,7 +14,7 @@ mapped_urls: % What needs to be done: Refine -% GitHub issue: https://github.com/elastic/docs-projects/issues/344 +% GitHub issue: https://github.com/elastic/docs-projects/issues/344 % Scope notes: Collapse to a single page, explain what deciders are @@ -27,4 +27,17 @@ mapped_urls: % - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-frozen-storage-decider.md % - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-frozen-existence-decider.md % - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-machine-learning-decider.md -% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-fixed-decider.md \ No newline at end of file +% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-fixed-decider.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-deciders.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-deciders.md) +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-reactive-storage-decider.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-reactive-storage-decider.md) +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-proactive-storage-decider.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-proactive-storage-decider.md) +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-frozen-shards-decider.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-frozen-shards-decider.md) +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-frozen-storage-decider.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-frozen-storage-decider.md) +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-frozen-existence-decider.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-frozen-existence-decider.md) +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-machine-learning-decider.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-machine-learning-decider.md) +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-fixed-decider.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/autoscaling-fixed-decider.md) \ No newline at end of file diff --git a/deploy-manage/deploy.md b/deploy-manage/deploy.md index c600a815b..1e64c731b 100644 --- a/deploy-manage/deploy.md +++ b/deploy-manage/deploy.md @@ -15,4 +15,11 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/docs-content/serverless/intro.md -% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/elasticsearch-intro-deploy.md \ No newline at end of file +% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/elasticsearch-intro-deploy.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/docs-content/serverless/intro.md](/raw-migrated-files/docs-content/serverless/intro.md) +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/elasticsearch-intro-deploy.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/elasticsearch-intro-deploy.md) \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-enterprise.md b/deploy-manage/deploy/cloud-enterprise.md index 18419a885..e8bc2710d 100644 --- a/deploy-manage/deploy/cloud-enterprise.md +++ b/deploy-manage/deploy/cloud-enterprise.md @@ -17,4 +17,11 @@ mapped_urls: % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/Elastic-Cloud-Enterprise-overview.md % Notes: 2 child docs % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-administering-ece.md -% Notes: redirect only \ No newline at end of file +% Notes: redirect only + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-enterprise/Elastic-Cloud-Enterprise-overview.md](/raw-migrated-files/cloud/cloud-enterprise/Elastic-Cloud-Enterprise-overview.md) +* [/raw-migrated-files/cloud/cloud-enterprise/ece-administering-ece.md](/raw-migrated-files/cloud/cloud-enterprise/ece-administering-ece.md) \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-enterprise/air-gapped-install.md b/deploy-manage/deploy/cloud-enterprise/air-gapped-install.md index 3c6544dc0..06fc1afe3 100644 --- a/deploy-manage/deploy/cloud-enterprise/air-gapped-install.md +++ b/deploy-manage/deploy/cloud-enterprise/air-gapped-install.md @@ -16,4 +16,11 @@ mapped_urls: % - [ ] ./raw-migrated-files/stack-docs/elastic-stack/air-gapped-install.md % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-install-offline.md -% Notes: 3 child docs \ No newline at end of file +% Notes: 3 child docs + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/stack-docs/elastic-stack/air-gapped-install.md](/raw-migrated-files/stack-docs/elastic-stack/air-gapped-install.md) +* [/raw-migrated-files/cloud/cloud-enterprise/ece-install-offline.md](/raw-migrated-files/cloud/cloud-enterprise/ece-install-offline.md) \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-enterprise/create-deployment.md b/deploy-manage/deploy/cloud-enterprise/create-deployment.md index 912923c97..5c2b4ee19 100644 --- a/deploy-manage/deploy/cloud-enterprise/create-deployment.md +++ b/deploy-manage/deploy/cloud-enterprise/create-deployment.md @@ -17,4 +17,12 @@ mapped_urls: % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-create-deployment.md % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-access-kibana.md -% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-manage-kibana.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-manage-kibana.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-enterprise/ece-create-deployment.md](/raw-migrated-files/cloud/cloud-enterprise/ece-create-deployment.md) +* [/raw-migrated-files/cloud/cloud-enterprise/ece-access-kibana.md](/raw-migrated-files/cloud/cloud-enterprise/ece-access-kibana.md) +* [/raw-migrated-files/cloud/cloud-enterprise/ece-manage-kibana.md](/raw-migrated-files/cloud/cloud-enterprise/ece-manage-kibana.md) \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-enterprise/deploy-an-orchestrator.md b/deploy-manage/deploy/cloud-enterprise/deploy-an-orchestrator.md index 3fc28c128..4ba2431a8 100644 --- a/deploy-manage/deploy/cloud-enterprise/deploy-an-orchestrator.md +++ b/deploy-manage/deploy/cloud-enterprise/deploy-an-orchestrator.md @@ -4,4 +4,6 @@ % GitHub issue: https://github.com/elastic/docs-projects/issues/339 -% Scope notes: Introduction about the content of this big section (which covers install and configuration possibilities of the orchestrator) \ No newline at end of file +% Scope notes: Introduction about the content of this big section (which covers install and configuration possibilities of the orchestrator) + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-enterprise/generate-roles-tokens.md b/deploy-manage/deploy/cloud-enterprise/generate-roles-tokens.md index 8dbffee49..045dcaa01 100644 --- a/deploy-manage/deploy/cloud-enterprise/generate-roles-tokens.md +++ b/deploy-manage/deploy/cloud-enterprise/generate-roles-tokens.md @@ -15,4 +15,11 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-generate-roles-token.md -% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-revoke-roles-token.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-revoke-roles-token.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-enterprise/ece-generate-roles-token.md](/raw-migrated-files/cloud/cloud-enterprise/ece-generate-roles-token.md) +* [/raw-migrated-files/cloud/cloud-enterprise/ece-revoke-roles-token.md](/raw-migrated-files/cloud/cloud-enterprise/ece-revoke-roles-token.md) \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-enterprise/manage-integrations-server.md b/deploy-manage/deploy/cloud-enterprise/manage-integrations-server.md index eb03e7430..d6ba014c3 100644 --- a/deploy-manage/deploy/cloud-enterprise/manage-integrations-server.md +++ b/deploy-manage/deploy/cloud-enterprise/manage-integrations-server.md @@ -11,4 +11,11 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-manage-integrations-server.md -% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-integrations-server-api-example.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-integrations-server-api-example.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-enterprise/ece-manage-integrations-server.md](/raw-migrated-files/cloud/cloud-enterprise/ece-manage-integrations-server.md) +* [/raw-migrated-files/cloud/cloud-enterprise/ece-integrations-server-api-example.md](/raw-migrated-files/cloud/cloud-enterprise/ece-integrations-server-api-example.md) \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-enterprise/tools-apis.md b/deploy-manage/deploy/cloud-enterprise/tools-apis.md index 6502ce5d4..dc9b34e50 100644 --- a/deploy-manage/deploy/cloud-enterprise/tools-apis.md +++ b/deploy-manage/deploy/cloud-enterprise/tools-apis.md @@ -2,4 +2,6 @@ % What needs to be done: Write from scratch -% GitHub issue: https://github.com/elastic/docs-projects/issues/310 \ No newline at end of file +% GitHub issue: https://github.com/elastic/docs-projects/issues/310 + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-enterprise/working-with-deployments.md b/deploy-manage/deploy/cloud-enterprise/working-with-deployments.md index 19f8a8738..f4d171762 100644 --- a/deploy-manage/deploy/cloud-enterprise/working-with-deployments.md +++ b/deploy-manage/deploy/cloud-enterprise/working-with-deployments.md @@ -21,4 +21,13 @@ mapped_urls: % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-change-deployment.md % Notes: another redirect % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-monitoring-deployments.md -% Notes: mostly redirect \ No newline at end of file +% Notes: mostly redirect + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-enterprise/ece-stack-getting-started.md](/raw-migrated-files/cloud/cloud-enterprise/ece-stack-getting-started.md) +* [/raw-migrated-files/cloud/cloud-enterprise/ece-administering-deployments.md](/raw-migrated-files/cloud/cloud-enterprise/ece-administering-deployments.md) +* [/raw-migrated-files/cloud/cloud-enterprise/ece-change-deployment.md](/raw-migrated-files/cloud/cloud-enterprise/ece-change-deployment.md) +* [/raw-migrated-files/cloud/cloud-enterprise/ece-monitoring-deployments.md](/raw-migrated-files/cloud/cloud-enterprise/ece-monitoring-deployments.md) \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-on-k8s.md b/deploy-manage/deploy/cloud-on-k8s.md index dff4d8ffc..46972c321 100644 --- a/deploy-manage/deploy/cloud-on-k8s.md +++ b/deploy-manage/deploy/cloud-on-k8s.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_urls: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-overview.html - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-advanced-topics.html @@ -6,18 +8,86 @@ mapped_urls: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s_learn_more_about_eck.html --- -# Elastic Cloud on Kubernetes +# Elastic Cloud on Kubernetes [k8s-overview] -% What needs to be done: Refine +Built on the Kubernetes Operator pattern, {{eck}} (ECK) extends the basic Kubernetes orchestration capabilities to support the setup and management of Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash on Kubernetes. -% GitHub issue: https://github.com/elastic/docs-projects/issues/357 +## ECK overview -% Scope notes: Maybe we can even leave it as it is. +With Elastic Cloud on Kubernetes, you can streamline critical operations, such as: -% Use migrated content from existing pages that map to this page: +1. Managing and monitoring multiple clusters +2. Scaling cluster capacity and storage +3. Performing safe configuration changes through rolling upgrades +4. Securing clusters with TLS certificates +5. Setting up hot-warm-cold architectures with availability zone awareness -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-overview.md -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-advanced-topics.md -% Notes: redirect only -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-supported.md -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_learn_more_about_eck.md \ No newline at end of file +This section provides everything you need to install, configure, and manage Elastic Stack applications with ECK, including: + +- [](./cloud-on-k8s/deploy-an-orchestrator.md): ECK installation methods and configuration options. Deploy ECK on managed Kubernetes platforms like GKE, AKS, and EKS, on self-managed Kubernetes clusters, on OpenShift, and even in air-gapped environments. +- [](./cloud-on-k8s/manage-deployments.md): Handle {{es}} clusters and {{kib}} instances through ECK. +- [](./cloud-on-k8s/orchestrate-other-elastic-applications.md): Run APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash on Kubernetes. +- [](./cloud-on-k8s/tools-apis.md): A collection of tools and APIs available in ECK based environments. + +Other sections of the documentation include the following important topics around ECK: + +- [Logging and Monitoring](../monitor.md): Configure stack monitoring and logs forwarding with the help of ECK. +- [Remote Clusters](../remote-clusters.md): Configure remote clusters on ECK. +- [](../tools.md): Add snapshot repositories to your {{es}} clusters for automatic snapshots. +- [Security](../security.md): Secure communications, manage HTTP certificates, or add secure settings to your applications. +- [Users and Roles](../users-roles.md): Configure authentication and authorization mechanisms, built-in users, external providers, and more. +- [Autoscaling](../autoscaling.md): Learn how to use {{es}} autoscaling on ECK, or use Horizontal Pod Autoscaler functionality for stateless workloads. +- [Licensing](../license/manage-your-license-in-eck.md): Manage licenses on ECK. + +::::{important} +ECK is an Elastic self-managed product offered in two licensing tiers: Basic and Enterprise. For more details refer to [Elastic subscriptions](https://www.elastic.co/subscriptions) and [](/deploy-manage/license/manage-your-license-in-eck.md) documentation. +:::: + +## Quickstart [eck-quickstart] + +If you want to get started quickly, follow these guides to deploy ECK and set up an {{es}} cluster: + +* [Install ECK using YAML manifests](./cloud-on-k8s/install-using-yaml-manifest-quickstart.md) +* [Deploy an {{es}} cluster](./cloud-on-k8s/elasticsearch-deployment-quickstart.md) +* [Deploy a {{kib}} instance](./cloud-on-k8s/kibana-instance-quickstart.md) + +Afterwards, you can: + +* Learn how to [update your deployment](./cloud-on-k8s/update-deployments.md) +* Check out [our recipes](./cloud-on-k8s/recipes.md) for multiple use cases +* Find further sample resources [in the project repository](https://github.com/elastic/cloud-on-k8s/tree/2.16/config/samples) + +## Supported versions [k8s-supported] + +This section outlines the supported Kubernetes and Elastic Stack versions for ECK. Check the full [Elastic support matrix](https://www.elastic.co/support/matrix#matrix_kubernetes) for more information. + +### Kubernetes compatibility + +ECK is compatible with the following Kubernetes distributions and related technologies: + +* Kubernetes 1.28-1.32 +* OpenShift 4.12-4.17 +* Google Kubernetes Engine (GKE), Azure Kubernetes Service (AKS), and Amazon Elastic Kubernetes Service (EKS) +* Helm: 3.2.0+ + +ECK should work with all conformant **installers** listed in these [FAQs](https://github.com/cncf/k8s-conformance/blob/master/faq.md#what-is-a-distribution-hosted-platform-and-an-installer). Distributions include source patches and so may not work as-is with ECK. + +### Elastic Stack compatibility + +ECK is compatible with the following Elastic Stack applications: + +* Elasticsearch, Kibana, APM Server: 6.8+, 7.1+, 8+ +* Enterprise Search: 7.7+, 8+ +* Beats: 7.0+, 8+ +* Elastic Agent: 7.10+ (standalone), 7.14+ (Fleet), 8+ +* Elastic Maps Server: 7.11+, 8+ +* Logstash: 8.7+ + +Elastic Stack application images for the OpenShift-certified Elasticsearch (ECK) Operator are only available from version 7.10 and later. + +## Learn more about ECK [k8s_learn_more_about_eck] + +* [Orchestrate Elasticsearch on Kubernetes](https://www.elastic.co/elasticsearch-kubernetes) +* [ECK post on the Elastic Blog](https://www.elastic.co/blog/introducing-elastic-cloud-on-kubernetes-the-elasticsearch-operator-and-beyond?elektra=products&storm=sub1) +* [Getting Started With Elastic Cloud on Kubernetes (ECK)](https://www.youtube.com/watch?v=PIJmlYBIFXM) +* [Running the Elastic Stack on Kubernetes with ECK](https://www.youtube.com/watch?v=Wf6E3vkvEFM) diff --git a/deploy-manage/deploy/cloud-on-k8s/accessing-services.md b/deploy-manage/deploy/cloud-on-k8s/accessing-services.md index 7b6d059ee..deb0a8086 100644 --- a/deploy-manage/deploy/cloud-on-k8s/accessing-services.md +++ b/deploy-manage/deploy/cloud-on-k8s/accessing-services.md @@ -1,28 +1,133 @@ --- +applies: + eck: all mapped_urls: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-accessing-elastic-services.html - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-request-elasticsearch-endpoint.html - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-services.html - - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-security.html --- -# Accessing services +# Accessing services [k8s-accessing-elastic-services] -% What needs to be done: Refine +To provide access to {{es}}, {{kib}}, and other {{stack}} applications when applicable, ECK relies on [Kubernetes services](https://kubernetes.io/docs/concepts/services-networking/service/). -% GitHub issue: https://github.com/elastic/docs-projects/issues/357 +All Elastic Stack resources deployed by the ECK operator are secured by default. The operator sets up basic authentication and TLS to encrypt network traffic to, from, and within your Elasticsearch cluster. -% Scope notes: Merge the selected docs into one: - First describe how to access Elasticsearch. - Describe the services that ECK creates for ES. - Provide the example and instructions +This section explains how to access and customize the Kubernetes services and secrets created by ECK, covering topics such as: -% Use migrated content from existing pages that map to this page: +* [Retrieving the `elastic` user password for basic authentication](#k8s-authentication) +* [Managing Kubernetes services](#k8s-kubernetes-service) +* [Obtaining the CA certificate and accessing the endpoint](#k8s-request-elasticsearch-endpoint) -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-accessing-elastic-services.md -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-request-elasticsearch-endpoint.md -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-services.md -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-security.md +For advanced use cases related to exposing and accessing orchestrated applications, see: + +* [](/deploy-manage/security/secure-http-communications.md): Configuration options for the HTTP SSL certificates, including integration with certificate management systems such as [cert-manager](https://cert-manager.io/). +* [](./service-meshes.md): Connect ECK and your managed deployments to service mesh implementations such as [Istio](https://istio.io) and [Linkerd](https://linkerd.io). +* [](./requests-routing-to-elasticsearch-nodes.md): Create custom services to expose different node types. +* [Use Ingress to expose {{es}} or {{kib}}](./managing-deployments-using-helm-chart.md#k8s-eck-stack-ingress): Helm based installation also facilitates the creation of Ingress resources. + +## Retrieve the `elastic` user password [k8s-authentication] + +To access Elastic resources, the operator manages a default user named `elastic` with the `superuser` role. Its password is stored in a `Secret` named `-elastic-user`. + +Run the following command to obtain the password of the `elastic` user: + +```sh +> kubectl get secret hulk-es-elastic-user -o go-template='{{.data.elastic | base64decode }}' +42xyz42citsale42xyz42 +``` + +::::{note} +Beware of copying this Secret as-is into a different namespace. Check [Common Problems: Owner References](../../../troubleshoot/deployments/cloud-on-k8s/common-problems.md#k8s-common-problems-owner-refs) for more information. +:::: + +For more information about handling built-in users on ECK deployments, refer to [](/deploy-manage/users-roles/cluster-or-deployment-auth/built-in-users.md). + +## Managing Kubernetes services [k8s-kubernetes-service] + +You can access Elastic resources by using native Kubernetes services that are not reachable from the public Internet by default. + +For each resource, the operator manages a Kubernetes service named `-[es|kb|apm|ent|agent]-http`, which is of type `ClusterIP` by default. `ClusterIP` exposes the service on a cluster-internal IP and makes the service only reachable within the cluster. + +```sh +> kubectl get svc + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +hulk-apm-http ClusterIP 10.19.212.105 8200/TCP 1m +hulk-es-http ClusterIP 10.19.252.160 9200/TCP 1m +hulk-kb-http ClusterIP 10.19.247.151 5601/TCP 1m +``` + +### Allow public access [k8s-allow-public-access] + +You can expose services in [different ways](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types) by specifying an `http.service.spec.type` in the `spec` of the resource manifest. On cloud providers which support external load balancers, you can set the `type` field to `LoadBalancer` to provision a load balancer for the `Service`, and populate the column `EXTERNAL-IP` after a short delay. Depending on the cloud provider, it may incur costs. + +By default, the Elasticsearch service created by ECK is configured to route traffic to all Elasticsearch nodes in the cluster. Depending on your cluster configuration, you may want more control over the set of nodes that handle different types of traffic (query, ingest, and so on). Refer to [](./requests-routing-to-elasticsearch-nodes.md) for more information. + +::::{warning} +When you change the `clusterIP` setting of the service, ECK will delete and re-create the service as `clusterIP` is an immutable field. Depending on your client implementation, this might result in a short disruption until the service DNS entries refresh to point to the new endpoints. +:::: + +```yaml +apiVersion: .k8s.elastic.co/v1 +kind: +metadata: + name: hulk +spec: + version: 8.16.1 + http: + service: + spec: + type: LoadBalancer +``` + +```sh +> kubectl get svc + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +hulk-apm-http LoadBalancer 10.19.212.105 35.176.227.106 8200:31000/TCP 1m +hulk-es-http LoadBalancer 10.19.252.160 35.198.131.115 9200:31320/TCP 1m +hulk-kb-http LoadBalancer 10.19.247.151 35.242.197.228 5601:31380/TCP 1m +``` + +## Access the endpoint [k8s-request-elasticsearch-endpoint] + +You can access the Elasticsearch endpoint within or outside the Kubernetes cluster. + +**Within the Kubernetes cluster** + +1. Retrieve the CA certificate. +2. Retrieve the password of the `elastic` user. +3. Use the service name to access the endpoint. + +```sh +NAME=hulk + +kubectl get secret "$NAME-es-http-certs-public" -o go-template='{{index .data "tls.crt" | base64decode }}' > tls.crt +PW=$(kubectl get secret "$NAME-es-elastic-user" -o go-template='{{.data.elastic | base64decode }}') + +curl --cacert tls.crt -u elastic:$PW https://$NAME-es-http:9200/ +``` + +::::{tip} +You can also use the examples in this section to access {{kib}} instead of {{es}} by adapting the secret and service names. +:::: + +**Outside the Kubernetes cluster** + +1. Retrieve the CA certificate. +2. Retrieve the password of the `elastic` user. +3. Retrieve the IP of the `LoadBalancer` service. + +```sh +NAME=hulk + +kubectl get secret "$NAME-es-http-certs-public" -o go-template='{{index .data "tls.crt" | base64decode }}' > tls.crt +IP=$(kubectl get svc "$NAME-es-http" -o jsonpath='{.status.loadBalancer.ingress[].ip}') +PW=$(kubectl get secret "$NAME-es-elastic-user" -o go-template='{{.data.elastic | base64decode }}') + +curl --cacert tls.crt -u elastic:$PW https://$IP:9200/ +``` -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): -$$$k8s-allow-public-access$$$ -$$$k8s-setting-up-your-own-certificate$$$ \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-logstash.md b/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-logstash.md index 6972733ee..2d0183b1c 100644 --- a/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-logstash.md +++ b/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-logstash.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-advanced-configuration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-maps-server.md b/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-maps-server.md index 9478d3d32..f789eaee8 100644 --- a/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-maps-server.md +++ b/deploy-manage/deploy/cloud-on-k8s/advanced-configuration-maps-server.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-maps-advanced-configuration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/advanced-configuration.md b/deploy-manage/deploy/cloud-on-k8s/advanced-configuration.md index e69e1d0fa..3390f9e23 100644 --- a/deploy-manage/deploy/cloud-on-k8s/advanced-configuration.md +++ b/deploy-manage/deploy/cloud-on-k8s/advanced-configuration.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-apm-advanced-configuration.html --- @@ -166,8 +168,6 @@ Now that you know how to use the APM keystore and customize the server configura By default the operator manages a private CA and generates a self-signed certificate used to secure the communication between APM agents and the server. -This behavior and the relevant configuration is identical to what is done for Elasticsearch and Kibana. Check [Setting up your own certificate](accessing-services.md#k8s-setting-up-your-own-certificate) for more information on how to use your own certificate to configure the TLS endpoint of the APM Server. +This behavior and the relevant configuration is identical to what is done for Elasticsearch and Kibana. Check [Setting up your own certificate](/deploy-manage/security/secure-http-communications.md) for more information on how to use your own certificate to configure the TLS endpoint of the APM Server. For more details on how to configure the APM agents to work with custom certificates, check the [APM agents documentation](https://www.elastic.co/guide/en/apm/agent/index.html). - - diff --git a/deploy-manage/deploy/cloud-on-k8s/advanced-elasticsearch-node-scheduling.md b/deploy-manage/deploy/cloud-on-k8s/advanced-elasticsearch-node-scheduling.md index 02dfffb5e..f7993f358 100644 --- a/deploy-manage/deploy/cloud-on-k8s/advanced-elasticsearch-node-scheduling.md +++ b/deploy-manage/deploy/cloud-on-k8s/advanced-elasticsearch-node-scheduling.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-advanced-node-scheduling.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/air-gapped-install.md b/deploy-manage/deploy/cloud-on-k8s/air-gapped-install.md index 0fae72fba..8d1e878aa 100644 --- a/deploy-manage/deploy/cloud-on-k8s/air-gapped-install.md +++ b/deploy-manage/deploy/cloud-on-k8s/air-gapped-install.md @@ -1,4 +1,7 @@ --- +navigation_title: Air gapped environments +applies: + eck: all mapped_urls: - https://www.elastic.co/guide/en/elastic-stack/current/air-gapped-install.html - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-air-gapped.html @@ -15,6 +18,8 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/stack-docs/elastic-stack/air-gapped-install.md + +% already removed % - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-air-gapped.md % Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): @@ -24,3 +29,132 @@ $$$air-gapped-install$$$ $$$k8s-container-registry-override$$$ $$$k8s-eck-diag-air-gapped$$$ + +% There are two concepts and areas to explore here: +% ECK installation on air-gapped. This has no complexity as it's all a matter of docker registry and docker images. +% Managing deployments on an ECK running on air-gapped is something not really covered in the official ECK book and partly covered in stack-docs + +% In this doc we will focus on ECK operator installation in air gapped environments, and we will link to Manage Deployments -> Air gapped (doesn't exist yet) for the content and examples about the rest. + +% from fleet air-gapped +% Kibana is able to reach the Elastic Package Registry to download package metadata and content. +% Elastic Agents are able to download binaries during upgrades from the Elastic Artifact Registry. + +% what about Elasticsearch requirements for example for GeoIP database, etc? + +Pending to determine what to do with this: +* Syncing container images for ECK and all other {{stack}} components over to a locally-accessible container repository. +* Modifying the ECK helm chart configuration so that ECK is aware that it is supposed to use your offline container repository instead of the public Elastic repository. +* Optionally, disabling ECK telemetry collection in the ECK helm chart. This configuration propagates to all other Elastic components, such as {{kib}}. +* Building your custom deployment container image for the {{artifact-registry}}. +* Building your custom deployment container image for the Elastic Endpoint Artifact Repository. + +# Running in air-gapped environments [k8s-air-gapped] + +The ECK operator can be run in an air-gapped environment without access to the open internet when it is configured not to pull container images from `docker.elastic.co`. + +By default ECK does not require you to specify the container image for each Elastic Stack application you deploy. + +```yaml +apiVersion: elasticsearch.k8s.elastic.co/v1 +kind: Elasticsearch +metadata: + name: quickstart +spec: + version: 8.16.1 + # image: docker.elastic.co/elasticsearch/elasticsearch:8.16.1 <1> + nodeSets: + - name: default + count: 1 + # podTemplate: + # spec: + # imagePullSecrets: <2> + # - name: private-registry-credentials-secret +``` + +1. The ECK operator will set this value by default. You can explicitly set it to your mirrored container image when running in an air-gapped environment +2. You can provide credentials to your private container registry by setting the `imagePullSecrets` field through the `spec.podTemplate` section of your Elastic resource specification, check [how to customize the Elastic resources Pods](../../../deploy-manage/deploy/cloud-on-k8s/customize-pods.md) and [how to setup a Secret containing your registry credentials](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). + + +ECK will automatically set the correct container image for each application. When running in an air-gapped or offline environment you will have to mirror the official Elastic container images in a private container image registry. To make use of your mirrored images you can either set the image for each application explicitly as shown in the preceding example or more conveniently override the default container registry as explained in the next section. + + +## Use a mirrored image of the ECK operator [k8s-use-mirrored-operator-image] + +To deploy the ECK operator in an air-gapped environment, you first have to mirror the operator image itself from `docker.elastic.co` to a private container registry, for example `my.registry`. + +Once the ECK operator image is copied internally, replace the original image name `docker.elastic.co/eck/eck-operator:2.16.1` with the private name of the image, for example `my.registry/eck/eck-operator:2.16.1`, in the [operator manifests](../../../deploy-manage/deploy/cloud-on-k8s/install-using-yaml-manifest-quickstart.md). When using [Helm charts](../../../deploy-manage/deploy/cloud-on-k8s/install-using-helm-chart.md), replace the `image.repository` Helm value with, for example, `my.registry/eck/eck-operator`. + + +## Override the default container registry [k8s-container-registry-override] + +When creating custom resources (Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash), the operator defaults to using container images pulled from the `docker.elastic.co` registry. If you are in an environment where external network access is restricted, you can configure the operator to use a different default container registry by starting the operator with the `--container-registry` command-line flag. Check [*Configure ECK*](../../../deploy-manage/deploy/cloud-on-k8s/configure-eck.md) for more information on how to configure the operator using command-line flags and environment variables. + +The operator expects container images to be located at specific repositories in the default container registry. Make sure that your container images are stored in the right repositories and are tagged correctly with the Stack version number. For example, if your private registry is `my.registry` and you wish to deploy components from Stack version 8.16.1, the following image names should exist: + +* `my.registry/elasticsearch/elasticsearch:8.16.1` +* `my.registry/kibana/kibana:8.16.1` +* `my.registry/apm/apm-server:8.16.1` + + +## Use a global container repository [k8s-container-repository-override] + +If you cannot follow the default Elastic image repositories naming scheme, you can configure the operator to use a different container repository by starting the operator with the `--container-repository` command-line flag. Check [*Configure ECK*](../../../deploy-manage/deploy/cloud-on-k8s/configure-eck.md) for more information on how to configure the operator using command-line flags and environment variables. + +For example, if your private registry is `my.registry` and all Elastic images are located under the `elastic` repository, the following image names should exist: + +* `my.registry/elastic/elasticsearch:8.16.1` +* `my.registry/elastic/kibana:8.16.1` +* `my.registry/elastic/apm-server:8.16.1` + + +## ECK Diagnostics in air-gapped environments [k8s-eck-diag-air-gapped] + +The [eck-diagnostics tool](../../../troubleshoot/deployments/cloud-on-k8s/run-eck-diagnostics.md) optionally runs diagnostics for Elastic Stack applications in a separate container that is deployed into the Kubernetes cluster. + +In air-gapped environments with no access to the `docker.elastic.co` registry, you should copy the latest support-diagnostics container image to your internal image registry and then run the tool with the additional flag `--diagnostic-image `. To find out which support diagnostics container image matches your version of eck-diagnostics run the tool once without arguments and it will print the default image in use. + + +% FROM THE OTHER CONTENT (ELASTIC-STACK): + +### 2. Kubernetes & OpenShift Install [air-gapped-kubernetes-and-openshift] + +Setting up air-gapped Kubernetes or OpenShift installs of the {{stack}} has some unique concerns, but the general dependencies are the same as in the self-managed install case on a regular Linux machine. + + +#### 2.1. Elastic Kubernetes Operator (ECK) [air-gapped-k8s-os-elastic-kubernetes-operator] + +The Elastic Kubernetes operator is an additional component in the Kubernetes OpenShift install that, essentially, does a lot of the work in installing, configuring, and updating deployments of the {{stack}}. For details, refer to the [{{eck}} install instructions](../../../deploy-manage/deploy/cloud-on-k8s/air-gapped-install.md). + +The main requirements are: + +* Syncing container images for ECK and all other {{stack}} components over to a locally-accessible container repository. +* Modifying the ECK helm chart configuration so that ECK is aware that it is supposed to use your offline container repository instead of the public Elastic repository. +* Optionally, disabling ECK telemetry collection in the ECK helm chart. This configuration propagates to all other Elastic components, such as {{kib}}. +* Building your custom deployment container image for the {{artifact-registry}}. +* Building your custom deployment container image for the Elastic Endpoint Artifact Repository. + + +#### 2.2. Elastic Package Registry [air-gapped-k8s-os-elastic-package-registry] + +The container image can be downloaded from the official Elastic Docker repository, as described in the {{fleet}} and {{elastic-agent}} [air-gapped environments](https://www.elastic.co/guide/en/fleet/current/air-gapped.html) documentation. + +This container would, ideally, run as a Kubernetes deployment. Refer to [Appendix C - EPR Kubernetes Deployment](../../../deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-epr-kubernetes-example) for examples. + + +#### 2.3. {{artifact-registry}} [air-gapped-k8s-os-elastic-artifact-registry] + +A custom container would need to be created following similar instructions to setting up a web server in the [self-managed install case](../../../deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-artifact-registry). For example, a container file using an NGINX base image could be used to run a build similar to the example described in [Appendix B - {{artifact-registry}}](../../../deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-artifact-registry-example). + + +#### 2.4. Elastic Endpoint Artifact Repository [air-gapped-k8s-os-elastic-endpoint-artifact-repository] + +Just like the {{artifact-registry}}. A custom container needs to be created following similar instructions to setting up a web server for the [self-managed install case](../../../deploy-manage/deploy/self-managed/air-gapped-install.md#air-gapped-elastic-artifact-registry). + + +#### 2.5. Ironbank Secure Images for Elastic [air-gapped-k8s-os-ironbank-secure-images] + +Besides the public [Elastic container repository](https://www.docker.elastic.co), most {{stack}} container images are also available in Platform One’s [Iron Bank](https://ironbank.dso.mil/repomap?vendorFilters=Elastic&page=1&sort=1). + + + diff --git a/deploy-manage/deploy/cloud-on-k8s/apm-server.md b/deploy-manage/deploy/cloud-on-k8s/apm-server.md index 8a9f9144e..288d73905 100644 --- a/deploy-manage/deploy/cloud-on-k8s/apm-server.md +++ b/deploy-manage/deploy/cloud-on-k8s/apm-server.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-apm-server.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/beats.md b/deploy-manage/deploy/cloud-on-k8s/beats.md index 25f3de6c5..88685012e 100644 --- a/deploy-manage/deploy/cloud-on-k8s/beats.md +++ b/deploy-manage/deploy/cloud-on-k8s/beats.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-beat.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-beats.md b/deploy-manage/deploy/cloud-on-k8s/configuration-beats.md index ffb89ba39..216150127 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-beats.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-beats.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-beat-configuration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-beats.md b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-beats.md index ae4c75789..eaa702b99 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-beats.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-beats.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-beat-configuration-examples.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-fleet.md b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-fleet.md index 9cea9876e..ef004f294 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-fleet.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-fleet.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-fleet-configuration-examples.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-logstash.md b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-logstash.md index 1d7ca54ae..9640071f3 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-logstash.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-logstash.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-configuration-examples.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-standalone.md b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-standalone.md index 73f6d7f12..24a031202 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-examples-standalone.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-examples-standalone.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-configuration-examples.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-fleet.md b/deploy-manage/deploy/cloud-on-k8s/configuration-fleet.md index c953f78d2..b7760578b 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-fleet.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-fleet.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-fleet-configuration.html --- @@ -221,7 +223,7 @@ To deploy {{agent}} in clusters with the Pod Security Policy admission controlle ## Customize {{fleet-server}} Service [k8s-elastic-agent-fleet-configuration-customize-fleet-server-service] -By default, ECK creates a Service for {{fleet-server}} that {{agents}} can connect through. You can customize it using the `http` configuration element. Check more information on how to [make changes](accessing-services.md) to the Service and [customize](tls-certificates.md) the TLS configuration. +By default, ECK creates a Service for {{fleet-server}} that {{agents}} can connect through. You can customize it using the `http` configuration element. Check more information on how to [make changes](accessing-services.md) to the Service and [customize](/deploy-manage/security/secure-http-communications.md) the TLS configuration. ## Control {{fleet}} policy selection [k8s-elastic-agent-control-fleet-policy-selection] diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-logstash.md b/deploy-manage/deploy/cloud-on-k8s/configuration-logstash.md index 87d1c9f18..eaa6135de 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-logstash.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-logstash.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-configuration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configuration-standalone.md b/deploy-manage/deploy/cloud-on-k8s/configuration-standalone.md index 0c2516d2c..f05f3b8bf 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configuration-standalone.md +++ b/deploy-manage/deploy/cloud-on-k8s/configuration-standalone.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-configuration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/configure-deployments.md b/deploy-manage/deploy/cloud-on-k8s/configure-deployments.md index 857a783f2..ec4294b64 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configure-deployments.md +++ b/deploy-manage/deploy/cloud-on-k8s/configure-deployments.md @@ -1,26 +1,35 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-orchestrating-elastic-stack-applications.html + - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-update-deployment.html --- # Configure deployments [k8s-orchestrating-elastic-stack-applications] -* [*Run Elasticsearch on ECK*](elasticsearch-configuration.md) -* [*Run {{kib}} on ECK*](kibana-configuration.md) -* [*Run APM Server on ECK*](apm-server.md) -* [*Run standalone Elastic Agent on ECK*](standalone-elastic-agent.md) -* [*Run {{fleet}}-managed {{agent}} on ECK*](fleet-managed-elastic-agent.md) -* [*Run Elastic Maps Server on ECK*](elastic-maps-server.md) -* [*Run Beats on ECK*](beats.md) -* [*Run {{ls}} on ECK*](logstash.md) -* [*Elastic Stack Helm Chart*](managing-deployments-using-helm-chart.md) -* [*Recipes*](recipes.md) -* [*Secure the Elastic Stack*](../../security.md) -* [*Access Elastic Stack services*](accessing-services.md) -* [*Customize Pods*](customize-pods.md) -* [*Manage compute resources*](manage-compute-resources.md) -* [*Autoscaling stateless applications*](../../autoscaling/autoscaling-stateless-applications-on-eck.md) -* [*Elastic Stack configuration policies*](elastic-stack-configuration-policies.md) -* [*Upgrade the Elastic Stack version*](../../upgrade/deployment-or-cluster.md) -* [*Connect to external Elastic resources*](connect-to-external-elastic-resources.md) +This section provides details around {{kib}} and {{es}} configuration when running on ECK. For general information about how ECK applies configuration changes and the syntax to use in the YAML manifests, refer to [](./update-deployments.md). +* [**{{es}} configuration**](elasticsearch-configuration.md): Review configuration possibilities to tune your {{es}} cluster running on ECK, learn how [nodes orchestration](./nodes-orchestration.md) work, [storage recommendations](./storage-recommendations.md), and more. + +* [**{{kib}} configuration**](kibana-configuration.md): Learn how to connect {{kib}} to an {{es}} cluster, apply advanced configuration settings, and tune the HTTP configuration. + +Additionally, the following topics apply to both {{es}} and {{kib}}, and in some cases, to other applications supported by ECK: + +* [**Access services**](accessing-services.md): Learn how to access to the orchestrated clusters and how to adapt the Kubernetes services to your needs. + +* [**Customize Pods**](customize-pods.md): Learn how to adapt the `podTemplate` field to your needs. + +* [**Manage compute resources**](manage-compute-resources.md): Important considerations around CPU and memory `requests` and `limits` when running production workloads. + +* [**Recipes**](recipes.md): Advanced use cases examples available in our GitHub repository. + +* [**Connect to external Elastic resources**](connect-to-external-elastic-resources.md): Use custom `secrets` for the `elasticsearchRef` and `kibanaRef` parameters. + +ECK also facilitates configuration and operation activities with advanced features, such as: + +* [**Elastic Stack configuration policies**](elastic-stack-configuration-policies.md): Organize your {{es}} and {{kib}} configuration settings through `StackConfigPolicy` resources that can be referenced within your deployments. This helps to keep your manifests simplified. + +::::{important} +Explore the [Security](/deploy-manage/security.md) and [Users and roles](/deploy-manage/users-roles.md) sections to to learn more about how to secure and control access your deployments. +:::: diff --git a/deploy-manage/deploy/cloud-on-k8s/configure-eck.md b/deploy-manage/deploy/cloud-on-k8s/configure-eck.md index 949749690..c2c6cc018 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configure-eck.md +++ b/deploy-manage/deploy/cloud-on-k8s/configure-eck.md @@ -1,93 +1,96 @@ --- +navigation_title: Apply configuration settings +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-operator-config.html --- -# Configure ECK [k8s-operator-config] - -ECK can be configured using either command line flags or environment variables. - -| Flag | Default | Description | -| --- | --- | --- | -| `ca-cert-rotate-before` | `24h` | Duration representing how long before expiration CA certificates should be re-issued. | -| `ca-cert-validity` | `8760h` | Duration representing the validity period of a generated CA certificate. | -| `ca-dir` | `""` | Path to a directory containing a CA certificate (tls.crt) and its associated private key (tls.key) to be used for all managed resources. Effectively disables the CA rotation and validity options. | -| `cert-rotate-before` | `24h` | Duration representing how long before expiration TLS certificates should be re-issued. | -| `cert-validity` | `8760h` | Duration representing the validity period of a generated TLS certificate. | -| `config` | `""` | Path to a file containing the operator configuration. | -| `container-registry` | `docker.elastic.co` | Container registry to use for pulling Elastic Stack container images. | -| `container-repository` | `""` | Container repository to use for pulling Elastic Stack container images. | -| `container-suffix` | `""` | Suffix to be appended to container images by default. Cannot be combined with `--ubi-only` flag. | -| `disable-config-watch` | `false` | Watch the configuration file for changes and restart to apply them. Only effective when the `--config` flag is used to set the configuration file. | -| `disable-telemetry` | `false` | Disable periodically updating ECK telemetry data for Kibana to consume. | -| `elasticsearch-client-timeout` | `180s` | Default timeout for requests made by the Elasticsearch client. | -| `enable-leader-election` | `true` | Enable leader election. Must be set to true if using multiple replicas of the operator | -| `enable-tracing` | `false` | Enable APM tracing in the operator process. Use environment variables to configure APM server URL, credentials, and so on. Check [Apm Go Agent reference](https://www.elastic.co/guide/en/apm/agent/go/current/configuration.html) for details. | -| `enable-webhook` | `false` | Enables a validating webhook server in the operator process. | -| `enforce-rbac-on-refs` | `false` | Enables restrictions on cross-namespace resource association through RBAC. | -| `exposed-node-labels` | `""` | List of Kubernetes node labels which are allowed to be copied as annotations on the Elasticsearch Pods. Check [Topology spread constraints and availability zone awareness](advanced-elasticsearch-node-scheduling.md#k8s-availability-zone-awareness) for more details. | -| `ip-family` | `""` | Set the IP family to use. Possible values: IPv4, IPv6, "" (= auto-detect) | -| `kube-client-qps` | `0` | Set the maximum number of queries per second to the Kubernetes API. Default value is inherited from the [Go client](https://github.com/kubernetes/client-go/blob/e6538dd42b4fe55b6c754e41c66b43133ba41a59/rest/config.go#L44). | -| `kube-client-timeout` | `60s` | Set the request timeout for Kubernetes API calls made by the operator. | -| `log-verbosity` | `0` | Verbosity level of logs. `-2`=Error, `-1`=Warn, `0`=Info, `0` and above=Debug. | -| `manage-webhook-certs` | `true` | Enables automatic webhook certificate management. | -| `max-concurrent-reconciles` | `3` | Maximum number of concurrent reconciles per controller (Elasticsearch, Kibana, APM Server). Affects the ability of the operator to process changes concurrently. | -| `metrics-cert-dir` | `"{{TempDir}}/k8s-metrics-server/serving-certs"` | Location of TLS certs for the metrics server. Directory needs to contain tls.key and tls.crt. If empty self-signed certificates are used. Only effective when combined with metrics-port and metrics-secure. | -| `metrics-host` | `0.0.0.0` | The host to which the operator should bind to serve metrics in the Prometheus format. Will be combined with metrics-port. | -| `metrics-port` | `0` | Prometheus metrics port. Set to 0 to disable the metrics endpoint. | -| `metrics-secure` | `false` | Enables TLS for the metrics server. Only effective combined with metrics-port. | -| `namespaces` | `""` | Namespaces in which this operator should manage resources. Accepts multiple comma-separated values. Defaults to all namespaces if empty or unspecified. | -| `operator-namespace` | `""` | Namespace the operator runs in. Required. | -| `password-hash-cache-size` | `5 x max-concurrent-reconciles` | Sets the size of the password hash cache. Caching is disabled if explicitly set to 0 or any negative value. | -| `set-default-security-context` | `auto-detect` | Enables adding a default Pod Security Context to Elasticsearch Pods in Elasticsearch `8.0.0` and later. `fsGroup` is set to `1000` by default to match Elasticsearch container default UID. This behavior might not be appropriate for OpenShift and PSP-secured Kubernetes clusters, so it can be disabled. | -| `ubi-only` | `false` | Use only UBI container images to deploy Elastic Stack applications. UBI images are only available from 7.10.0 onward. Cannot be combined with `--container-suffix` flag. | -| `validate-storage-class` | `true` | Specifies whether the operator should retrieve storage classes to verify volume expansion support. Can be disabled if cluster-wide storage class RBAC access is not available. | -| `webhook-cert-dir` | `"{{TempDir}}/k8s-webhook-server/serving-certs"` | Path to the directory that contains the webhook server key and certificate. | -| `webhook-name` | `"elastic-webhook.k8s.elastic.co"` | Name of the Kubernetes ValidatingWebhookConfiguration resource. Only used when `enable-webhook` is true. | -| `webhook-secret` | `""` | K8s secret mounted into the path designated by webhook-cert-dir to be used for webhook certificates. | -| `webhook-port` | `9443` | Port to listen for incoming validation requests. | - -Unless noted otherwise, environment variables can be used instead of flags to configure the operator as well. Simply convert the flag name to upper case and replace any dashes (`-`) with underscores (`_`). For example, the `log-verbosity` flag can be set by an environment variable named `LOG_VERBOSITY`. - -Duration values should be specified as numeric values suffixed by the time unit. For example, a duration of 10 hours should be specified as `10h`. Acceptable time unit suffixes are: - -| Suffix | Unit | -| --- | --- | -| `ms` | Milliseconds | -| `s` | Seconds | -| `m` | Minutes | -| `h` | Hours | +# Apply ECK configuration settings [k8s-operator-config] -If you have a large number of configuration options to specify, use the `--config` flag to point to a file containing those options. For example, assume you have a file named `eck-config.yaml` with the following content: +This page explains the various methods for configuring and applying ECK settings. + +::::{tip} +For a detailed list and description of all available settings in ECK, refer to asciidocalypse://reference/cloud/cloud-on-k8s/eck-configuration-flags.md. +:::: + +By default, the ECK installation includes a [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) with an `eck.yaml` key where you can add, remove, or update configuration settings. This ConfigMap is mounted into the operator’s container as a file, and provided to the application through the `--config` flag. + +::::{note} +If you use [Operator Lifecycle Manager](https://github.com/operator-framework/operator-lifecycle-manager), refer to [Configure ECK under Operator Lifecycle Manager](#k8s-operator-config-olm) +:::: + +To configure ECK settings, follow the instructions in the next sections depending on whether you installed ECK through the Helm chart or the YAML manifests. + +## Using the operator Helm chart + +If you installed ECK through the Helm chart commands listed in [](./install-using-helm-chart.md), add your configuration parameters under the `config` key in your values file, or set them inline using the equivalent `--set config.=` flags when updating or installing the release. + +For example, to add the `ca-cert-validity` setting with a value of `43800h`, you can use any of the following methods: + +### Option 1: Use a values file and reference it in the helm upgrade command: + +Create a values file with the following content: ```yaml -log-verbosity: 2 -metrics-port: 6060 -namespaces: [ns1, ns2, ns3] +config: + ca-cert-validity: 43800h ``` -The operator can be started using any of the following methods to achieve the same end result: +Then, update the installed release pointing to the values file: ```sh -./elastic-operator manager --config=eck-config.yaml +helm upgrade elastic-operator elastic/eck-operator -f my-values-file.yaml -n elastic-system ``` -```sh -./elastic-operator manager --log-verbosity=2 --metrics-port=6060 --namespaces=ns1,ns2,ns3 -``` +### Option 2: Use `--set` in the helm upgrade command ```sh -LOG_VERBOSITY=2 METRICS_PORT=6060 NAMESPACES="ns1,ns2,ns3" ./elastic-operator manager +helm upgrade elastic-operator elastic/eck-operator --set config.ca-cert-validity=43800h -n elastic-system ``` -If you use a combination of all or some of the these methods, the descending order of precedence in case of a conflict is as follows: +## Using the operator YAML manifests -* Flag -* Environment variable -* File +If you installed ECK using the manifests and the commands listed in [Deploy ECK](./install-using-yaml-manifest-quickstart.md), you can configure it by editing the `eck.yaml` key of the `elastic-operator` ConfigMap. Add, remove or update any configuration setting there and the operator will restart automatically to apply the new changes unless the `--disable-config-watch` flag is set. + +You can update the ConfigMap directly using the command `kubectl edit configmap elastic-operator -n elastic-operator` or modify the installation manifests and reapply them with `kubectl apply -f `. -You can edit the `elastic-operator` ConfigMap to change the operator configuration. Unless the `--disable-config-watch` flag is set, the operator should restart automatically to apply the new changes. Alternatively, you can edit the `elastic-operator` StatefulSet and add flags to the `args` section — which will trigger an automatic restart of the operator pod by the StatefulSet controller. +The following shows the default `elastic-operator` ConfigMap, for reference purposes. Refer to asciidocalypse://reference/cloud/cloud-on-k8s/eck-configuration-flags.md for a complete list of available settings. +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: elastic-operator + namespace: elastic-system +data: + eck.yaml: |- + log-verbosity: 0 + metrics-port: 0 + container-registry: docker.elastic.co + max-concurrent-reconciles: 3 + ca-cert-validity: 8760h + ca-cert-rotate-before: 24h + cert-validity: 8760h + cert-rotate-before: 24h + disable-config-watch: false + exposed-node-labels: [topology.kubernetes.io/.*,failure-domain.beta.kubernetes.io/.*] + set-default-security-context: auto-detect + kube-client-timeout: 60s + elasticsearch-client-timeout: 180s + disable-telemetry: false + distribution-channel: all-in-one + validate-storage-class: true + enable-webhook: true + webhook-name: elastic-webhook.k8s.elastic.co + webhook-port: 9443 + operator-namespace: elastic-system + enable-leader-election: true + elasticsearch-observation-interval: 10s + ubi-only: false +``` + +Alternatively, you can edit the `elastic-operator` StatefulSet and add flags to the `args` section of the operator container — which will trigger an automatic restart of the operator pod by the StatefulSet controller. ## Configure ECK under Operator Lifecycle Manager [k8s-operator-config-olm] @@ -127,7 +130,7 @@ If you use [Operator Lifecycle Manager (OLM)](https://github.com/operator-framew name: elastic-cloud-eck source: elastic-operators sourceNamespace: openshift-marketplace - startingCSV: elastic-cloud-eck.v2.16.1 + startingCSV: elastic-cloud-eck.v{{eck_version}} config: volumes: - name: config @@ -139,4 +142,42 @@ If you use [Operator Lifecycle Manager (OLM)](https://github.com/operator-framew readOnly: true ``` +## Advanced configuration methods + +ECK can be configured using either command-line flags, environment variables or a file containing the operator configuration, pointed by `--config` flag. + +::::{important} +For most use cases, Elastic recommends configuring ECK through the `elastic-operator` ConfigMap, which is included by default in all installation methods. +This section provides a low-level overview of alternative configuration methods, primarily intended for developers or advanced users who might need to start the operator binary manually or adjust its configuration without modifying the ConfigMap. The implementation of these methods through Kubernetes manifests is out of the scope of this document. +:::: + +To pass configuration options as environment variables, convert the flag name to upper case and replace any dashes (`-`) with underscores (`_`). For example, the `log-verbosity` flag can be set by an environment variable named `LOG_VERBOSITY`. + +If you use a combination of all or some of the these methods, the descending order of precedence in case of a conflict is as follows: + +* Flag +* Environment variable +* File + +If you have a large number of configuration options to specify, use the `--config` flag to point to a file containing those options. For example, assume you have a file named `eck-config.yaml` with the following content: + +```yaml +log-verbosity: 2 +metrics-port: 6060 +namespaces: [ns1, ns2, ns3] +``` + +The operator can be started using any of the following methods to achieve the same end result: + +```sh +./elastic-operator manager --config=eck-config.yaml +``` + +```sh +./elastic-operator manager --log-verbosity=2 --metrics-port=6060 --namespaces=ns1,ns2,ns3 +``` + +```sh +LOG_VERBOSITY=2 METRICS_PORT=6060 NAMESPACES="ns1,ns2,ns3" ./elastic-operator manager +``` diff --git a/deploy-manage/deploy/cloud-on-k8s/configure-validating-webhook.md b/deploy-manage/deploy/cloud-on-k8s/configure-validating-webhook.md index 25f5523f5..fb35b29fe 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configure-validating-webhook.md +++ b/deploy-manage/deploy/cloud-on-k8s/configure-validating-webhook.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-webhook.html --- @@ -24,7 +26,7 @@ Validating webhooks are defined using a `ValidatingWebhookConfiguration` object When using the default `operator.yaml` manifest, ECK is installed with a `ValidatingWebhookConfiguration` configured as follows: -* Validate all known Elastic custom resources (Elasticsearch, Kibana, APM Server, Enterprise Search, Beats, Elastic Agent, Elastic Maps Server, and Logstash) on create and update. +* Validate all known Elastic custom resources (Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash) on create and update. * The operator itself is the webhook server — which is exposed through a service named `elastic-webhook-server` in the `elastic-system` namespace. * The operator generates a certificate for the webhook and stores it in a secret named `elastic-webhook-server-cert` in the `elastic-system` namespace. This certificate is automatically rotated by the operator when it is due to expire. diff --git a/deploy-manage/deploy/cloud-on-k8s/configure.md b/deploy-manage/deploy/cloud-on-k8s/configure.md index dd31d9655..d7c8f7f22 100644 --- a/deploy-manage/deploy/cloud-on-k8s/configure.md +++ b/deploy-manage/deploy/cloud-on-k8s/configure.md @@ -1,18 +1,43 @@ --- +navigation_title: Configure +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-operating-eck.html --- -# Configure [k8s-operating-eck] +# Configure ECK [k8s-operating-eck] -* [*Configure ECK*](configure-eck.md) -* [*Required RBAC permissions*](required-rbac-permissions.md) -* [*Configure the validating webhook*](configure-validating-webhook.md) -* [*Configure the metrics endpoint*](../../monitor/orchestrators/eck-metrics-configuration.md) -* [*Restrict cross-namespace resource associations*](restrict-cross-namespace-resource-associations.md) -* [*Manage licenses in ECK*](../../license/manage-your-license-in-eck.md) -* [*Install ECK*](install.md) -* [*Upgrade ECK*](../../upgrade/orchestrator/upgrade-cloud-on-k8s.md) -* [*Uninstall ECK*](../../uninstall/uninstall-elastic-cloud-on-kubernetes.md) -* [*Running in air-gapped environments*](air-gapped-install.md) +This section covers ECK configuration mechanisms and use cases, starting with the basic setup of the operator using the provided `ConfigMap` and extending to more advanced configuration scenarios that require detailed procedures. +::::{tip} +This content focuses on ECK operator configuration. For details on available features and how to configure your {{es}} and {{kib}} deployments, refer to [](./configure-deployments.md). +:::: + +The following guides cover common ECK configuration tasks: + +* [](./configure-eck.md): Apply configuration changes, such the CA certificate validity period, the namespaces where the operator is allowed to work, or the log verbosity level for ECK. + +* [Configure the validating webhook](configure-validating-webhook.md): Enable or disable the webhook, and configure multiple SSL certificate generation options. + +* [Restrict cross-namespace resource associations](restrict-cross-namespace-resource-associations.md): Configure access control rules for cross-namespace associations. This functionality is disabled by default. + +* [Create custom images](./create-custom-images.md): Use your own images with {{es}} plugins already installed rather than installing them through init containers. + +* [Service meshes](./service-meshes.md): Connect ECK and managed Elastic Stack applications to some of the most popular [service mesh](https://www.cncf.io/blog/2017/04/26/service-mesh-critical-component-cloud-native-stack/) implementations in the Kubernetes ecosystem. + +* [Network policies](./network-policies.md): Use [Kubernetes network policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) to isolate pods by restricting incoming and outgoing network connections to a trusted set of sources and destinations. + +* [](./webhook-namespace-selectors.md): Restrict the namespaces that the validation webhook applies to, allowing multiple operators to coexist efficiently in the same cluster. + +Other sections of the Elastic documentation cover additional topics related to ECK configuration: + +* **Monitoring** + * [Configure the metrics endpoint](/deploy-manage/monitor/orchestrators/eck-metrics-configuration.md) (monitor an orchestrator) + +* **Licensing** + * [Manage licenses in ECK](../../license/manage-your-license-in-eck.md) + +* **Maintenance** + * [Upgrade ECK](../../upgrade/orchestrator/upgrade-cloud-on-k8s.md) + * [Uninstall ECK](../../uninstall/uninstall-elastic-cloud-on-kubernetes.md) \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-on-k8s/connect-to-apm-server.md b/deploy-manage/deploy/cloud-on-k8s/connect-to-apm-server.md index 1eb744a0b..83bb20453 100644 --- a/deploy-manage/deploy/cloud-on-k8s/connect-to-apm-server.md +++ b/deploy-manage/deploy/cloud-on-k8s/connect-to-apm-server.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-apm-connecting.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/connect-to-external-elastic-resources.md b/deploy-manage/deploy/cloud-on-k8s/connect-to-external-elastic-resources.md index 4353ef8e6..9ec084e1f 100644 --- a/deploy-manage/deploy/cloud-on-k8s/connect-to-external-elastic-resources.md +++ b/deploy-manage/deploy/cloud-on-k8s/connect-to-external-elastic-resources.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-connect-to-unmanaged-resources.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/create-custom-images.md b/deploy-manage/deploy/cloud-on-k8s/create-custom-images.md index d789ef3ef..8f20ed39e 100644 --- a/deploy-manage/deploy/cloud-on-k8s/create-custom-images.md +++ b/deploy-manage/deploy/cloud-on-k8s/create-custom-images.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-custom-images.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/custom-configuration-files-plugins.md b/deploy-manage/deploy/cloud-on-k8s/custom-configuration-files-plugins.md index 20d08630c..1b966ab92 100644 --- a/deploy-manage/deploy/cloud-on-k8s/custom-configuration-files-plugins.md +++ b/deploy-manage/deploy/cloud-on-k8s/custom-configuration-files-plugins.md @@ -1,11 +1,13 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-bundles-plugins.html --- # Custom configuration files and plugins [k8s-bundles-plugins] -To run Elasticsearch with specific plugins or configuration files installed on ECK, you have two options. Each option has its own pros and cons. +To run Elasticsearch with specific plugins or configuration files installed on ECK, you have multiple options. Each option has its own pros and cons. 1. Create a custom container image with the required plugins and configuration files. @@ -33,9 +35,26 @@ To run Elasticsearch with specific plugins or configuration files installed on E * Each Elasticsearch node needs to repeat the download, wasting bandwidth and slowing startup. * Deployment manifests are more complicated. +3. Use ConfigMaps or Secrets together with volumes and volume mounts for configuration files. + + * **Pros** + + * Best choice for injecting configuration files into your {{es}} nodes. + * Follows standard Kubernetes methodology to mount files into Pods. + + * **Cons** + + * Not valid for plugins installation. + * Requires to maintain the ConfigMaps or Secrets with the content of the files. + +The following sections provide examples for each of the mentioned options. + +## Create a custom image Refer to [Creating custom images](create-custom-images.md) for instructions on how to build custom Docker images based on the official Elastic images. +## Use init containers for plugins installation + The following example describes option 2, using a repository plugin. To install the plugin before the Elasticsearch nodes start, use an init container to run the [plugin installation tool](https://www.elastic.co/guide/en/elasticsearch/plugins/current/installation.html). ```yaml @@ -55,9 +74,9 @@ spec: bin/elasticsearch-plugin install --batch repository-azure ``` -To install custom configuration files you can use volumes and volume mounts. +### Note when using Istio [istio-note] -The next example shows how to add a synonyms file for the [synonym token filter](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-synonym-tokenfilter.html) in Elasticsearch. But you can use the same approach for any kind of file you want to mount into the configuration directory of Elasticsearch. +When using Istio, init containers do **not** have network access, as the Envoy sidecar that provides network connectivity is not started yet. In this scenario, custom containers are the best option. If custom containers are simply not a viable option, then it is possible to adjust the startup command for the {{es}} container itself to run the plugin installation before starting {{es}}, as the following example describes. Note that this approach will require updating the startup command if it changes in the {{es}} image, which could potentially cause failures during upgrades. ```yaml spec: @@ -67,24 +86,45 @@ spec: podTemplate: spec: containers: - - name: elasticsearch <1> - volumeMounts: - - name: synonyms - mountPath: /usr/share/elasticsearch/config/dictionaries - volumes: - - name: synonyms - configMap: - name: synonyms <2> + - name: elasticsearch + command: + - /usr/bin/env + - bash + - -c + - | + #!/usr/bin/env bash + set -e + bin/elasticsearch-plugin remove --purge repository-s3 || true + bin/elasticsearch-plugin install --batch repository-s3 + /bin/tini -- /usr/local/bin/docker-entrypoint.sh ``` -1. Elasticsearch runs by convention in a container called *elasticsearch*. -2. Assuming you have created a config map in the same namespace as Elasticsearch with the name *synonyms* containing the synonyms file(s). +## Use a volume and volume mount together with a ConfigMap or Secret + +To install custom configuration files you can: +1. Add the configuration data into a ConfigMap or Secret. +2. Use volumes and volume mounts in your manifest to mount the contents of the ConfigMap or Secret as files in your {{es}} nodes. -$$$istio-note$$$ -**Note when using Istio** +The next example shows how to add a synonyms file for the [synonym token filter](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-synonym-tokenfilter.html) in Elasticsearch. But you can **use the same approach for any kind of file you want to mount into the configuration directory of Elasticsearch**, like adding CA certificates of external systems. -When using Istio, init containers do **not** have network access, as the Envoy sidecar that provides network connectivity is not started yet. In this scenario, custom containers are the best option. If custom containers are simply not a viable option, then it is possible to adjust the startup command for the elasticsearch container itself to run the plugin installation before starting Elasticsearch, as the following example describes. Note that this approach will require updating the startup command if it changes in the Elasticsearch image, which could potentially cause failures during upgrades. +1. Create the ConfigMap or Secret with the data: + +There are multiple ways to create and mount [ConfigMaps](https://kubernetes.io/docs/concepts/configuration/configmap/) and [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) on Kubernetes. Refer to the official documentation for more details. + +This example shows how to create a ConfigMap named `synonyms` with the content of a local file named `my-synonyms.txt` added into the `synonyms-elasticsearch.txt` key of the ConfigMap. + +```sh +kubectl create configmap synonyms -n --from-file=my-synonyms.txt=synonyms-elasticsearch.txt +``` + +::::{tip} +Create the ConfigMap or Secret in the same namespace where your {{es}} cluster runs. +:::: + +2. Declare the ConfigMap as a volume and mount it in the Elasticsearch containers. + +In this example, modify your {{es}} manifest to mount the contents of the `synonyms` ConfigMap into `/usr/share/elasticsearch/config/dictionaries` on the {{es}} nodes. ```yaml spec: @@ -94,15 +134,19 @@ spec: podTemplate: spec: containers: - - name: elasticsearch - command: - - /usr/bin/env - - bash - - -c - - | - #!/usr/bin/env bash - set -e - bin/elasticsearch-plugin remove --purge repository-s3 || true - bin/elasticsearch-plugin install --batch repository-s3 - /bin/tini -- /usr/local/bin/docker-entrypoint.sh + - name: elasticsearch <1> + volumeMounts: + - name: synonyms + mountPath: /usr/share/elasticsearch/config/dictionaries <2> + volumes: + - name: synonyms + configMap: <3> + name: synonyms <4> ``` + +1. Elasticsearch runs by convention in a container called `elasticsearch`. Do not change that value. +2. Use always a path under `/usr/share/elasticsearch/config`. +3. Use `secret` instead of `configMap` if you used a secret to store the data. +4. The ConfigMap name must be the same as the ConfigMap created in the previous step. + +After the changes are applied, {{es}} nodes should be able to access `dictionaries/synonyms-elasticsearch.txt` and use it in any [configuration setting](./node-configuration.md). diff --git a/deploy-manage/deploy/cloud-on-k8s/customize-pods.md b/deploy-manage/deploy/cloud-on-k8s/customize-pods.md index c48aac2c3..3bee22931 100644 --- a/deploy-manage/deploy/cloud-on-k8s/customize-pods.md +++ b/deploy-manage/deploy/cloud-on-k8s/customize-pods.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-customize-pods.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/deploy-an-orchestrator.md b/deploy-manage/deploy/cloud-on-k8s/deploy-an-orchestrator.md index d942cbfa4..b64f75198 100644 --- a/deploy-manage/deploy/cloud-on-k8s/deploy-an-orchestrator.md +++ b/deploy-manage/deploy/cloud-on-k8s/deploy-an-orchestrator.md @@ -1,18 +1,22 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-quickstart.html --- # Deploy an orchestrator [k8s-quickstart] -With Elastic Cloud on Kubernetes (ECK) you can extend the basic Kubernetes orchestration capabilities to easily deploy, secure, upgrade your {{es}} cluster, and much more. +With Elastic Cloud on Kubernetes (ECK), you can extend the basic Kubernetes orchestration capabilities to easily deploy, secure, upgrade your {{es}} cluster, along with other Elastic applications. -Eager to get started? This quickstart guide shows you how to: +In this section, you'll learn how to do the following: -* [Deploy ECK in your Kubernetes cluster](install-using-yaml-manifest-quickstart.md) -* [Deploy an {{es}} cluster](elasticsearch-deployment-quickstart.md) -* [Deploy a {{kib}} instance](kibana-instance-quickstart.md) -* [Update your deployment](update-deployments.md) +- [**Installing the ECK Operator**](./install.md): Learn different installation methods, including Helm and YAML manifests. +- [**Deploying in air-gapped environments**](./air-gapped-install.md): Follow best practices for installing and operating ECK in restricted networks. +- [**Configuring ECK**](./configure.md): Understand the available configuration options to optimize your ECK deployment. -Afterwards, you can find further sample resources [in the project repository](https://github.com/elastic/cloud-on-k8s/tree/2.16/config/samples) or by checking out [our recipes](recipes.md). +To learn how to deploy {{es}}, {{kib}}, or other Elastic applications using ECK, refer to [](./manage-deployments.md). +::::{tip} +For a quickstart experience covering installation of ECK and deployment of an {{es}} cluster with a {{kib}} instance, refer to [](../cloud-on-k8s.md#eck-quickstart) +:::: \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md b/deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md index b3f6bb82f..077fd7069 100644 --- a/deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md +++ b/deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md @@ -1,6 +1,12 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-autopilot.html + - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-autopilot-setting-virtual-memory.html + - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-autopilot-deploy-the-operator.html + - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-autopilot-deploy-elasticsearch.html + - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-autopilot-deploy-agent-beats.html --- # Deploy ECK on GKE Autopilot [k8s-autopilot] @@ -8,17 +14,54 @@ mapped_pages: This page shows how to run ECK on GKE Autopilot. 1. It is recommended that each Kubernetes host’s virtual memory kernel settings be modified. Refer to [Virtual memory](virtual-memory.md). -2. It is recommended that Elasticsearch Pods have an `initContainer` that waits for virtual memory settings to be in place. Refer to [Deploy an Elasticsearch instance](k8s-autopilot-deploy-elasticsearch.md). -3. For Elastic Agent/Beats there are storage limitations to be considered. Refer to [Deploy a standalone Elastic Agent and/or Beats](k8s-autopilot-deploy-agent-beats.md) -4. Ensure you are using a node class that is applicable for your workload by adding a `cloud.google.com/compute-class` label in a `nodeSelector`. Refer to [GKE Autopilot documentation.](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-compute-classes) +2. It is recommended that Elasticsearch Pods have an `initContainer` that waits for virtual memory settings to be in place. +3. For Elastic Agent/Beats there are storage limitations to be considered. +4. Ensure you are using a node class that is applicable for your workload by adding a `cloud.google.com/compute-class` label in a `nodeSelector`. Refer to [GKE Autopilot documentation.](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-compute-classes). - * [Ensuring virtual memory kernel settings](k8s-autopilot-setting-virtual-memory.md) - * [Installing the ECK Operator](k8s-autopilot-deploy-operator.md) - * [Deploy an Elasticsearch instance](k8s-autopilot-deploy-elasticsearch.md) - * [Deploy a standalone Elastic Agent and/or Beats](k8s-autopilot-deploy-agent-beats.md) +## Ensuring virtual memory kernel settings [k8s-autopilot-setting-virtual-memory] +If you are intending to run production workloads on GKE Autopilot then `vm.max_map_count` should be set. The recommended way to set this kernel setting on the Autopilot hosts is with a `Daemonset` as described in the [Virtual memory](virtual-memory.md) section. You must be running at least version 1.25 when on the `regular` channel or using the `rapid` channel, which currently runs version 1.27. +::::{warning} +Only use the provided `Daemonset` exactly as specified or it could be rejected by the Autopilot control plane. +:::: +## Install the ECK Operator [k8s-autopilot-deploy-the-operator] +Refer to [*Install ECK*](install.md) for more information on installation options. +## Deploy an Elasticsearch cluster [k8s-autopilot-deploy-elasticsearch] + +Create an Elasticsearch cluster. If you are using the `Daemonset` described in the [Virtual memory](virtual-memory.md) section to set `max_map_count` you can add the `initContainer` below is also used to ensure the setting is set prior to starting Elasticsearch. + +```shell +cat < | -| Helm Charts | `app.kubernetes.io/name: elastic-operator`
| - -::::{note} -The examples in this section assume that the ECK operator has been installed using the Helm chart. -:::: - - - -## Kubernetes API server IP [k8s_kubernetes_api_server_ip] - -Run `kubectl get endpoints kubernetes -n default` to obtain the API server IP address for your cluster. - -::::{note} -The following examples assume that the Kubernetes API server IP address is `10.0.0.1`. -:::: - - - -## Isolating the operator [k8s-network-policies-operator-isolation] - -The minimal set of permissions required are as follows: - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 443 of the Kubernetes API server.
* UDP port 53 for DNS lookup.
* TCP port 9200 of {{es}} nodes on managed namespace.
| -| Ingress (incoming) | * TCP port 9443 for webhook requests from the Kubernetes API server.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: elastic-operator - namespace: elastic-system -spec: - egress: - - ports: - - port: 53 - protocol: UDP - - ports: - - port: 443 - protocol: TCP - to: - - ipBlock: - cidr: 10.0.0.1/32 - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchExpressions: - - key: eck.k8s.elastic.co/tenant - operator: In - values: - - team-a - - team-b - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - ingress: - - from: - - ipBlock: - cidr: 10.0.0.1/32 - ports: - - port: 9443 - protocol: TCP - podSelector: - matchLabels: - app.kubernetes.io/name: elastic-operator -``` - - -## Isolating {{es}} [k8s-network-policies-elasticsearch-isolation] - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9300 to other {{es}} nodes in the namespace (transport port).
* UDP port 53 for DNS lookup.
| -| Ingress (incoming) | * TCP port 9200 from the operator and other pods in the namespace.
* TCP port 9300 from other {{es}} nodes in the namespace (transport port).
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-elasticsearch - namespace: team-a -spec: - egress: - - ports: - - port: 9300 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - - ports: - - port: 53 - protocol: UDP - ingress: - - from: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/operator-name: elastic-operator - podSelector: - matchLabels: - app.kubernetes.io/name: elastic-operator - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - # [Optional] Allow ingress controller pods from the ingress-nginx namespace. - #- namespaceSelector: - # matchLabels: - # name: ingress-nginx - ports: - - port: 9200 - protocol: TCP - - from: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - ports: - - port: 9300 - protocol: TCP - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch -``` - - -## Isolating {{kib}} [k8s-network-policies-kibana-isolation] - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* UDP port 53 for DNS lookup.
| -| Ingress (incoming) | * TCP port 5601 from other pods in the namespace.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-kibana - namespace: team-a -spec: - egress: - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - # [Optional] Restrict to a single {{es}} cluster named hulk. - # elasticsearch.k8s.elastic.co/cluster-name=hulk - - ports: - - port: 53 - protocol: UDP - # [Optional] If Agent is deployed, this is to allow Kibana to access the Elastic Package Registry (https://epr.elastic.co). - # - port: 443 - # protocol: TCP - ingress: - - from: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - # [Optional] Allow ingress controller pods from the ingress-nginx namespace. - #- namespaceSelector: - # matchLabels: - # name: ingress-nginx - ports: - - port: 5601 - protocol: TCP - podSelector: - matchLabels: - common.k8s.elastic.co/type: kibana -``` - - -## Isolating APM Server [k8s-network-policies-apm-server-isolation] - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* TCP port 5601 to {{kib}} instances in the namespace.
* UDP port 53 for DNS lookup.
| -| Ingress (incoming) | * TCP port 8200 from other pods in the namespace.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-apm-server - namespace: team-a -spec: - egress: - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - - ports: - - port: 5601 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: kibana - - ports: - - port: 53 - protocol: UDP - ingress: - - from: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - # [Optional] Allow ingress controller pods from the ingress-nginx namespace. - #- namespaceSelector: - # matchLabels: - # name: ingress-nginx - ports: - - port: 8200 - protocol: TCP - podSelector: - matchLabels: - common.k8s.elastic.co/type: apm-server -``` - - -## Isolating Enterprise Search [k8s-network-policies-enterprise-search-isolation] - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* UDP port 53 for DNS lookup.
| -| Ingress (incoming) | * TCP port 3002 from other pods in the namespace.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-enterprise-search - namespace: team-a -spec: - egress: - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - - ports: - - port: 53 - protocol: UDP - ingress: - - from: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - # [Optional] Allow ingress controller pods from the ingress-nginx namespace. - #- namespaceSelector: - # matchLabels: - # name: ingress-nginx - ports: - - port: 3002 - protocol: TCP - podSelector: - matchLabels: - common.k8s.elastic.co/type: enterprise-search -``` - - -## Isolating {{beats}} [k8s-network-policies-beats-isolation] - -::::{note} -Some {{beats}} may require additional access rules than what is listed here. For example, {{heartbeat}} will require a rule to allow access to the endpoint it is monitoring. -:::: - - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* TCP port 5601 to {{kib}} instances in the namespace.
* UDP port 53 for DNS lookup.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-beats - namespace: team-a -spec: - egress: - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - - ports: - - port: 5601 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: kibana - - ports: - - port: 53 - protocol: UDP - podSelector: - matchLabels: - common.k8s.elastic.co/type: beat -``` - - -## Isolating {{agent}} and {{fleet}} [k8s-network-policies-agent-isolation] - -::::{note} -Some {{agent}} policies may require additional access rules other than those listed here. -:::: - - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* TCP port 5601 to {{kib}} instances in the namespace.
* TCP port 8220 to {{fleet}} instances in the namespace.
* UDP port 53 for DNS lookup.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-agent - namespace: team-a -spec: - egress: - - ports: - - port: 8220 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: agent - - ports: - - port: 5601 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: kibana - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - - ports: - - port: 53 - protocol: UDP - - ports: - - port: 443 - protocol: TCP - to: - - ipBlock: - cidr: 10.0.0.1/32 - ingress: - - from: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - ports: - - port: 8220 - protocol: TCP - podSelector: - matchLabels: - common.k8s.elastic.co/type: agent -``` - - -## Isolating {{ls}} [k8s-network-policies-logstash-isolation] - -::::{note} -{{ls}} may require additional access rules than those listed here, depending on plugin usage. -:::: - - -| | | -| --- | --- | -| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* UDP port 53 for DNS lookup.
| - -```yaml -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: eck-logstash - namespace: team-a -spec: - egress: - - ports: - - port: 9200 - protocol: TCP - to: - - namespaceSelector: - matchLabels: - eck.k8s.elastic.co/tenant: team-a - podSelector: - matchLabels: - common.k8s.elastic.co/type: elasticsearch - - ports: - - port: 53 - protocol: UDP - podSelector: - matchLabels: - common.k8s.elastic.co/type: logstash -``` - - diff --git a/deploy-manage/deploy/cloud-on-k8s/kibana-configuration.md b/deploy-manage/deploy/cloud-on-k8s/kibana-configuration.md index af59b323a..086b015f6 100644 --- a/deploy-manage/deploy/cloud-on-k8s/kibana-configuration.md +++ b/deploy-manage/deploy/cloud-on-k8s/kibana-configuration.md @@ -1,11 +1,13 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-kibana.html --- # Kibana configuration [k8s-kibana] -The [quickstart](kibana-instance-quickstart.md) is a good starting point to quickly setup a {{kib}} instance with ECK. The following sections describe how to customize a {{kib}} deployment to suit your requirements. +The following sections describe how to customize a {{kib}} deployment to suit your requirements. * [Connect to an {{es}} cluster](k8s-kibana-es.md) @@ -26,9 +28,6 @@ The [quickstart](kibana-instance-quickstart.md) is a good starting point to quic * [Disable TLS](k8s-kibana-http-configuration.md#k8s-kibana-http-disable-tls) * [Install {{kib}} plugins](k8s-kibana-plugins.md) - - - - +* [Autoscaling stateless applications](../../autoscaling/autoscaling-stateless-applications-on-eck.md): Use [Horizontal Pod Autoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) for {{kib}} or other stateless applications. diff --git a/deploy-manage/deploy/cloud-on-k8s/kibana-instance-quickstart.md b/deploy-manage/deploy/cloud-on-k8s/kibana-instance-quickstart.md index 680343c7a..57a8238fb 100644 --- a/deploy-manage/deploy/cloud-on-k8s/kibana-instance-quickstart.md +++ b/deploy-manage/deploy/cloud-on-k8s/kibana-instance-quickstart.md @@ -1,9 +1,12 @@ --- +navigation_title: Deploy a Kibana instance +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-deploy-kibana.html --- -# Kibana instance quickstart [k8s-deploy-kibana] +# Deploy a Kibana instance [k8s-deploy-kibana] To deploy a simple [{{kib}}](https://www.elastic.co/guide/en/kibana/current/introduction.html#introduction) specification, with one {{kib}} instance: @@ -53,7 +56,7 @@ To deploy a simple [{{kib}}](https://www.elastic.co/guide/en/kibana/current/intr kubectl port-forward service/quickstart-kb-http 5601 ``` - Open `https://localhost:5601` in your browser. Your browser will show a warning because the self-signed certificate configured by default is not verified by a known certificate authority and not trusted by your browser. You can temporarily acknowledge the warning for the purposes of this quick start but it is highly recommended that you [configure valid certificates](tls-certificates.md#k8s-setting-up-your-own-certificate) for any production deployments. + Open `https://localhost:5601` in your browser. Your browser will show a warning because the self-signed certificate configured by default is not verified by a known certificate authority and not trusted by your browser. You can temporarily acknowledge the warning for the purposes of this quick start but it is highly recommended that you [configure valid certificates](/deploy-manage/security/secure-http-communications.md#k8s-setting-up-your-own-certificate) for any production deployments. Login as the `elastic` user. The password can be obtained with the following command: @@ -68,5 +71,12 @@ For a full description of each `CustomResourceDefinition` (CRD), refer to the [* kubectl describe crd kibana ``` -This completes the quickstart of deploying an {{kib}} instance on top of [the ECK operator](install-using-yaml-manifest-quickstart.md) and [deployed {{es}} cluster](elasticsearch-deployment-quickstart.md). We recommend continuing to [updating your deployment](update-deployments.md). For more {{kib}} configuration options, refer to [Running {{kib}} on ECK](kibana-configuration.md). +## Next steps + +This completes the quickstart of deploying an {{kib}} instance on top of [the ECK operator](install-using-yaml-manifest-quickstart.md) and [deployed {{es}} cluster](elasticsearch-deployment-quickstart.md). + +We recommend continuing to: +* [Updating your deployment](update-deployments.md). +* For more {{kib}} configuration options, refer to [{{kib}} configuration on ECK](kibana-configuration.md) and [](./configure-deployments.md). + diff --git a/deploy-manage/deploy/cloud-on-k8s/known-limitations.md b/deploy-manage/deploy/cloud-on-k8s/known-limitations.md index 0185402de..f4cc067a1 100644 --- a/deploy-manage/deploy/cloud-on-k8s/known-limitations.md +++ b/deploy-manage/deploy/cloud-on-k8s/known-limitations.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-fleet-known-limitations.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/logstash-plugins.md b/deploy-manage/deploy/cloud-on-k8s/logstash-plugins.md index a3736cb6d..33b0b2005 100644 --- a/deploy-manage/deploy/cloud-on-k8s/logstash-plugins.md +++ b/deploy-manage/deploy/cloud-on-k8s/logstash-plugins.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-plugins.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/logstash.md b/deploy-manage/deploy/cloud-on-k8s/logstash.md index 3a614827b..74b078da4 100644 --- a/deploy-manage/deploy/cloud-on-k8s/logstash.md +++ b/deploy-manage/deploy/cloud-on-k8s/logstash.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/manage-compute-resources.md b/deploy-manage/deploy/cloud-on-k8s/manage-compute-resources.md index b818afa95..d0bdbe21e 100644 --- a/deploy-manage/deploy/cloud-on-k8s/manage-compute-resources.md +++ b/deploy-manage/deploy/cloud-on-k8s/manage-compute-resources.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-managing-compute-resources.html --- @@ -102,7 +104,7 @@ A [known Kubernetes issue](https://github.com/kubernetes/kubernetes/issues/51135 -### Set compute resources for Kibana, Enterprise Search, Elastic Maps Server, APM Server and Logstash [k8s-compute-resources-kibana-and-apm] +### Set compute resources for Kibana, Elastic Maps Server, APM Server and Logstash [k8s-compute-resources-kibana-and-apm] ```yaml apiVersion: kibana.k8s.elastic.co/v1 @@ -285,7 +287,6 @@ If `resources` is not defined in the specification of an object, then the operat | Beat | `300Mi` | `300Mi` | | Elastic Agent | `400Mi` | `400Mi` | | Elastic Maps Server | `200Mi` | `200Mi` | -| Enterprise Search | `4Gi` | `4Gi` | | Logstash | `2Gi` | `2Gi` | If the Kubernetes cluster is configured with [LimitRanges](https://kubernetes.io/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/) that enforce a minimum memory constraint, they could interfere with the operator defaults and cause object creation to fail. diff --git a/deploy-manage/deploy/cloud-on-k8s/manage-deployments.md b/deploy-manage/deploy/cloud-on-k8s/manage-deployments.md index 19c32a12f..6e3f51aae 100644 --- a/deploy-manage/deploy/cloud-on-k8s/manage-deployments.md +++ b/deploy-manage/deploy/cloud-on-k8s/manage-deployments.md @@ -1,7 +1,31 @@ +--- +applies: + eck: all +--- # Manage deployments -% What needs to be done: Write from scratch +This section provides detailed guidance on deploying, configuring, and managing Elasticsearch and Kibana within ECK. A **deployment** refers to an {{es}} cluster, optionally with one or more {{kib}} instances connected to it. -% GitHub issue: https://github.com/elastic/docs-projects/issues/357 +::::{tip} +This content focuses on Elasticsearch and Kibana deployments. To orchestrate other Elastic Stack applications such as APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash, refer to the [Orchestrating other Elastic Stack applications](./orchestrate-other-elastic-applications.md). +:::: -% Scope notes: To be decided... \ No newline at end of file +## What You'll Learn + +In this section, you'll learn how to perform the following tasks in ECK: + +- [**Deploy an Elasticsearch cluster**](./elasticsearch-deployment-quickstart.md): Orchestrate an {{es}} cluster in Kubernetes. +- [**Deploy Kibana instances**](./kibana-instance-quickstart.md): Set up and connect Kibana to an existing Elasticsearch cluster. +- [**Manage deployments using Elastic Stack Helm chart**](./managing-deployments-using-helm-chart.md): Use Helm to deploy clusters and other stack applications. +- [**Apply updates to your deployments**](./update-deployments.md): Modify existing deployments, scale clusters, and update configurations, while ensuring minimal disruption. +- [**Configure access to your deployments**](./accessing-services.md): Use and adapt Kubernetes services to your needs. +- [**Advanced configuration**](./configure-deployments.md): Explore available settings for Elasticsearch and Kibana, including storage, networking, security, and scaling options. + +For a complete reference on configuration possibilities for {{es}} and {{kib}}, see: + +- [](./elasticsearch-configuration.md) +- [](./kibana-configuration.md) + +Other references for managing deployments: + +* [**Upgrade the Elastic Stack version**](../../upgrade/deployment-or-cluster.md): Upgrade orchestrated applications on ECK. diff --git a/deploy-manage/deploy/cloud-on-k8s/managing-deployments-using-helm-chart.md b/deploy-manage/deploy/cloud-on-k8s/managing-deployments-using-helm-chart.md index 2780ad8ea..883c9d780 100644 --- a/deploy-manage/deploy/cloud-on-k8s/managing-deployments-using-helm-chart.md +++ b/deploy-manage/deploy/cloud-on-k8s/managing-deployments-using-helm-chart.md @@ -1,9 +1,12 @@ --- +navigation_title: Elastic Stack Helm chart +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-stack-helm-chart.html --- -# Managing deployments using a Helm chart [k8s-stack-helm-chart] +# Elastic Stack Helm chart [k8s-stack-helm-chart] Starting from ECK 2.4.0, a Helm chart is available for managing Elastic Stack resources using the ECK Operator. It is available from the Elastic Helm repository and can be added to your Helm repository list by running the following command: @@ -16,21 +19,26 @@ helm repo update The minimum supported version of Helm is 3.2.0. :::: +The Elastic Stack (`eck-stack`) Helm chart is built on top of individual charts such as `eck-elasticsearch` and `eck-kibana`. For more details on its structure and dependencies, refer to the [chart repository](https://github.com/elastic/cloud-on-k8s/tree/main/deploy/eck-stack/). +The chart enables you to deploy the core components ({{es}} and {{kib}}) together, along with other {{stack}} applications if needed, under the same chart release. The following sections guide you through the installation process for multiple use cases. Choose the command that best fits your setup. -## Installing Elasticsearch and Kibana using the eck-stack Helm Chart [k8s-install-elasticsearch-kibana-helm] +::::{tip} +All the provided examples deploy the applications in a namespace named `elastic-stack`. Consider adapting the commands to your use case. +:::: + +## Elasticsearch and Kibana [k8s-install-elasticsearch-kibana-helm] -Similar to the [quickstart](elasticsearch-deployment-quickstart.md), the following section describes how to setup an Elasticsearch cluster with a simple Kibana instance managed by ECK, and how to customize a deployment using the eck-stack Helm chart’s values. +Similar to the quickstart examples for {{es}} and {{kib}}, this section describes how to setup an {{es}} cluster with a simple {{kib}} instance managed by ECK, and how to customize a deployment using the eck-stack Helm chart’s values. ```sh # Install an eck-managed Elasticsearch and Kibana using the default values, which deploys the quickstart examples. helm install es-kb-quickstart elastic/eck-stack -n elastic-stack --create-namespace ``` +### Customize Elasticsearch and Kibana installation with example values [k8s-eck-stack-helm-customize] -### Customizing Kibana and Elasticsearch using the eck-stack Helm Chart’s example values [k8s-eck-stack-helm-customize] - -There are example Helm values files for installing and managing a more advanced Elasticsearch and/or Kibana [in the project repository](https://github.com/elastic/cloud-on-k8s/tree/2.16/deploy/eck-stack/examples). +You can find example Helm values files for deploying and managing more advanced Elasticsearch and Kibana setups [in the project repository](https://github.com/elastic/cloud-on-k8s/tree/2.16/deploy/eck-stack/examples). To use one or more of these example configurations, use the `--values` Helm option, as seen in the following section. @@ -41,8 +49,7 @@ helm install es-quickstart elastic/eck-stack -n elastic-stack --create-namespace --values https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/deploy/eck-stack/examples/kibana/http-configuration.yaml ``` - -## Installing Fleet Server with Elastic Agents along with Elasticsearch and Kibana using the eck-stack Helm Chart [k8s-install-fleet-agent-elasticsearch-kibana-helm] +## Fleet Server with Elastic Agents along with Elasticsearch and Kibana [k8s-install-fleet-agent-elasticsearch-kibana-helm] The following section builds upon the previous section, and allows installing Fleet Server, and Fleet-managed Elastic Agents along with Elasticsearch and Kibana. @@ -52,8 +59,7 @@ helm install eck-stack-with-fleet elastic/eck-stack \ --values https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/deploy/eck-stack/examples/agent/fleet-agents.yaml -n elastic-stack ``` - -## Installing Logstash along with Elasticsearch, Kibana and Beats using the eck-stack Helm Chart [k8s-install-logstash-elasticsearch-kibana-helm] +## Logstash along with Elasticsearch, Kibana and Beats [k8s-install-logstash-elasticsearch-kibana-helm] The following section builds upon the previous sections, and allows installing Logstash along with Elasticsearch, Kibana and Beats. @@ -63,8 +69,7 @@ helm install eck-stack-with-logstash elastic/eck-stack \ --values https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/deploy/eck-stack/examples/logstash/basic-eck.yaml -n elastic-stack ``` - -## Installing a standalone Elastic APM Server along with Elasticsearch and Kibana using the eck-stack Helm Chart [k8s-install-apm-server-elasticsearch-kibana-helm] +## Standalone Elastic APM Server along with Elasticsearch and Kibana [k8s-install-apm-server-elasticsearch-kibana-helm] The following section builds upon the previous sections, and allows installing a standalone Elastic APM Server along with Elasticsearch and Kibana. @@ -74,12 +79,12 @@ helm install eck-stack-with-apm-server elastic/eck-stack \ --values https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/deploy/eck-stack/examples/apm-server/basic.yaml -n elastic-stack ``` -### Installing individual components of the Elastic Stack using the Helm Charts [k8s-eck-stack-individual-components] +## Install individual components of the Elastic Stack [k8s-eck-stack-individual-components] You can install individual components in one of two ways using the provided Helm Charts. 1. Using Helm values -2. Using the individual Helm Charts directly +2. Using the individual Helm Charts directly (not the `eck-stack` helm chart) **Using Helm values to install only Elasticsearch** @@ -93,12 +98,90 @@ helm install es-quickstart elastic/eck-stack -n elastic-stack --create-namespace helm install es-quickstart elastic/eck-elasticsearch -n elastic-stack --create-namespace ``` +## Adding Ingress to the Elastic stack [k8s-eck-stack-ingress] + +Both {{es}} and {{kib}} support [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/), which can be enabled using the following options: -### Adding Ingress to the Elastic stack using the Helm Charts [k8s-eck-stack-ingress] +**If an individual chart is used (not eck-stack)** -Both Elasticsearch and Kibana support Ingress, which can be enabled using the following options: +The following command installs an {{es}} cluster using the `eck-elasticsearch` chart and configures an ingress resource: ```sh -helm install es-quickstart elastic/eck-elasticsearch -n elastic-stack --create-namespace --set=ingress.enabled=true --set=ingress.hosts[0].host=elasticsearch.example.com --set=ingress.hosts[0].path="/" +helm install es-quickstart elastic/eck-elasticsearch -n elastic-stack --create-namespace \ + --set=ingress.enabled=true --set=ingress.hosts[0].host=elasticsearch.example.com --set=ingress.hosts[0].path="/" ``` +**If eck-stack chart is used** + +The following command deploys the basic {{es}} and {{kib}} example with ingress resources for both components: + +```sh +helm install es-kb-quickstart elastic/eck-stack -n elastic-stack --create-namespace \ + --set=eck-elasticsearch.ingress.enabled=true --set=eck-elasticsearch.ingress.hosts[0].host=elasticsearch.example.com --set=eck-elasticsearch.ingress.hosts[0].path="/" \ + --set=eck-kibana.ingress.enabled=true --set=eck-kibana.ingress.hosts[0].host=kibana.example.com --set=eck-kibana.ingress.hosts[0].path="/" +``` + +For illustration purposes, the ingress objects created by the previous command will look similar to the following: + +```yaml +# Source: eck-stack/charts/eck-elasticsearch/templates/ingress.yaml +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: elasticsearch + labels: + helm.sh/chart: eck-elasticsearch-0.14.1 + app.kubernetes.io/name: eck-elasticsearch + app.kubernetes.io/instance: es-kb-quickstart + app.kubernetes.io/managed-by: Helm +spec: + rules: + - host: "elasticsearch.example.com" + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: elasticsearch-es-http + port: + number: 9200 +--- +# Source: eck-stack/charts/eck-kibana/templates/ingress.yaml +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: es-kb-quickstart-eck-kibana + labels: + helm.sh/chart: eck-kibana-0.14.1 + app.kubernetes.io/name: eck-kibana + app.kubernetes.io/instance: es-kb-quickstart + app.kubernetes.io/managed-by: Helm +spec: + rules: + - host: "kibana.example.com" + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: es-kb-quickstart-eck-kibana-kb-http + port: + number: 5601 +``` + +## View available configuration options [k8s-install-helm-show-values-stack] + +You can view all configurable values of the Elastic Stack helm chart of the individual charts by running the following: + +```sh +helm show values elastic/eck-stack +helm show values elastic/eck-elasticsearch +helm show values elastic/eck-kibana +helm show values elastic/eck-agent +helm show values elastic/eck-beats +helm show values elastic/eck-apm-server +helm show values elastic/eck-fleet-server +helm show values elastic/eck-logstash +``` diff --git a/deploy-manage/deploy/cloud-on-k8s/map-data.md b/deploy-manage/deploy/cloud-on-k8s/map-data.md index cbff49066..7e22059b3 100644 --- a/deploy-manage/deploy/cloud-on-k8s/map-data.md +++ b/deploy-manage/deploy/cloud-on-k8s/map-data.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-maps-data.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/network-policies.md b/deploy-manage/deploy/cloud-on-k8s/network-policies.md index 0f0bbfdb8..2c26e0633 100644 --- a/deploy-manage/deploy/cloud-on-k8s/network-policies.md +++ b/deploy-manage/deploy/cloud-on-k8s/network-policies.md @@ -1,17 +1,25 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-network-policies.html + - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s_prerequisites.html --- +% this section could be moved entirely to security. to be considered. + # Network policies [k8s-network-policies] -[Network policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) allow you to isolate pods by restricting incoming and outgoing network connections to a trusted set of sources and destinations. This section describes how to use network policies to isolate the ECK operator and the {{stack}} applications to a set of namespaces to implement a form of soft multi-tenancy. Soft multi-tenancy is a term used to describe a scenario where a group of trusted users (different teams within an organization, for example) share a single resource such as a Kubernetes cluster. Note that network policies alone are not sufficient for security. You should complement them with strict RBAC policies, resource quotas, node taints, and other available security mechanisms to ensure that tenants cannot access, modify, or disrupt resources belonging to each other. +Kubernetes [network policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) allow you to isolate pods by restricting incoming and outgoing network connections to a trusted set of sources and destinations. + +This section describes how to use network policies to isolate the ECK operator and the {{stack}} applications to a set of namespaces to implement a form of soft multi-tenancy. Soft multi-tenancy is a term used to describe a scenario where a group of trusted users (different teams within an organization, for example) share a single resource such as a Kubernetes cluster. + +Note that network policies alone are not sufficient for security. You should complement them with strict RBAC policies, resource quotas, node taints, and other available security mechanisms to ensure that tenants cannot access, modify, or disrupt resources belonging to each other. ::::{note} There are several efforts to support multi-tenancy on Kubernetes, including the [official working group for multi-tenancy](https://github.com/kubernetes-sigs/multi-tenancy) and community extensions such as [loft](https://loft.sh) and [kiosk](https://github.com/kiosk-sh/kiosk), that can make configuration and management easier. You might need to employ network policies such the ones described in this section to have fine-grained control over {{stack}} applications deployed by your tenants. :::: - The following sections assume that the operator is installed in the `elastic-system` namespace with the [`namespaces` configuration](configure-eck.md) set to `team-a,team-b`. Each namespace is expected to be labelled as follows: ```sh @@ -20,4 +28,405 @@ kubectl label namespace team-a eck.k8s.elastic.co/tenant=team-a kubectl label namespace team-b eck.k8s.elastic.co/tenant=team-b ``` +## Prerequisites [k8s_prerequisites] + +To set up the network policies correctly you must know the operator Pod selector and the Kubernetes API server IP. They may vary depending on your environment and how the operator has been installed. + +### Operator Pod selector [k8s_operator_pod_selector] + +The operator Pod label depends on how the operator has been installed. Check the following table to know which label name is used in the network policies. + +| Installation method | Pod selector | +| --- | --- | +| YAML manifests | `control-plane: elastic-operator`
| +| Helm Charts | `app.kubernetes.io/name: elastic-operator`
| + +::::{note} +The examples in this section assume that the ECK operator has been installed using the Helm chart. +:::: + +### Kubernetes API server IP [k8s_kubernetes_api_server_ip] + +Run `kubectl get endpoints kubernetes -n default` to obtain the API server IP address for your cluster. + +::::{note} +The following examples assume that the Kubernetes API server IP address is `10.0.0.1`. +:::: + +## Isolating the operator [k8s-network-policies-operator-isolation] + +The minimal set of permissions required are as follows: + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 443 of the Kubernetes API server.
* UDP port 53 for DNS lookup.
* TCP port 9200 of {{es}} nodes on managed namespace.
| +| Ingress (incoming) | * TCP port 9443 for webhook requests from the Kubernetes API server.
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: elastic-operator + namespace: elastic-system +spec: + egress: + - ports: + - port: 53 + protocol: UDP + - ports: + - port: 443 + protocol: TCP + to: + - ipBlock: + cidr: 10.0.0.1/32 + - ports: + - port: 9200 + protocol: TCP + to: + - namespaceSelector: + matchExpressions: + - key: eck.k8s.elastic.co/tenant + operator: In + values: + - team-a + - team-b + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + ingress: + - from: + - ipBlock: + cidr: 10.0.0.1/32 + ports: + - port: 9443 + protocol: TCP + podSelector: + matchLabels: + app.kubernetes.io/name: elastic-operator +``` + + +## Isolating Elasticsearch [k8s-network-policies-elasticsearch-isolation] + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 9300 to other {{es}} nodes in the namespace (transport port).
* UDP port 53 for DNS lookup.
| +| Ingress (incoming) | * TCP port 9200 from the operator and other pods in the namespace.
* TCP port 9300 from other {{es}} nodes in the namespace (transport port).
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: eck-elasticsearch + namespace: team-a +spec: + egress: + - ports: + - port: 9300 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + - ports: + - port: 53 + protocol: UDP + ingress: + - from: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/operator-name: elastic-operator + podSelector: + matchLabels: + app.kubernetes.io/name: elastic-operator + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + # [Optional] Allow ingress controller pods from the ingress-nginx namespace. + #- namespaceSelector: + # matchLabels: + # name: ingress-nginx + ports: + - port: 9200 + protocol: TCP + - from: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + ports: + - port: 9300 + protocol: TCP + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch +``` + + +## Isolating Kibana [k8s-network-policies-kibana-isolation] + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* UDP port 53 for DNS lookup.
| +| Ingress (incoming) | * TCP port 5601 from other pods in the namespace.
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: eck-kibana + namespace: team-a +spec: + egress: + - ports: + - port: 9200 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + # [Optional] Restrict to a single {es} cluster named hulk. + # elasticsearch.k8s.elastic.co/cluster-name=hulk + - ports: + - port: 53 + protocol: UDP + # [Optional] If Agent is deployed, this is to allow Kibana to access the Elastic Package Registry (https://epr.elastic.co). + # - port: 443 + # protocol: TCP + ingress: + - from: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + # [Optional] Allow ingress controller pods from the ingress-nginx namespace. + #- namespaceSelector: + # matchLabels: + # name: ingress-nginx + ports: + - port: 5601 + protocol: TCP + podSelector: + matchLabels: + common.k8s.elastic.co/type: kibana +``` + + +## Isolating APM Server [k8s-network-policies-apm-server-isolation] + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* TCP port 5601 to {{kib}} instances in the namespace.
* UDP port 53 for DNS lookup.
| +| Ingress (incoming) | * TCP port 8200 from other pods in the namespace.
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: eck-apm-server + namespace: team-a +spec: + egress: + - ports: + - port: 9200 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + - ports: + - port: 5601 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: kibana + - ports: + - port: 53 + protocol: UDP + ingress: + - from: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + # [Optional] Allow ingress controller pods from the ingress-nginx namespace. + #- namespaceSelector: + # matchLabels: + # name: ingress-nginx + ports: + - port: 8200 + protocol: TCP + podSelector: + matchLabels: + common.k8s.elastic.co/type: apm-server +``` + +## Isolating Beats [k8s-network-policies-beats-isolation] + +::::{note} +Some {{beats}} may require additional access rules than what is listed here. For example, {{heartbeat}} will require a rule to allow access to the endpoint it is monitoring. +:::: + + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* TCP port 5601 to {{kib}} instances in the namespace.
* UDP port 53 for DNS lookup.
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: eck-beats + namespace: team-a +spec: + egress: + - ports: + - port: 9200 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + - ports: + - port: 5601 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: kibana + - ports: + - port: 53 + protocol: UDP + podSelector: + matchLabels: + common.k8s.elastic.co/type: beat +``` + + +## Isolating Elastic Agent and Fleet [k8s-network-policies-agent-isolation] + +::::{note} +Some {{agent}} policies may require additional access rules other than those listed here. +:::: + + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* TCP port 5601 to {{kib}} instances in the namespace.
* TCP port 8220 to {{fleet}} instances in the namespace.
* UDP port 53 for DNS lookup.
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: eck-agent + namespace: team-a +spec: + egress: + - ports: + - port: 8220 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: agent + - ports: + - port: 5601 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: kibana + - ports: + - port: 9200 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + - ports: + - port: 53 + protocol: UDP + - ports: + - port: 443 + protocol: TCP + to: + - ipBlock: + cidr: 10.0.0.1/32 + ingress: + - from: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + ports: + - port: 8220 + protocol: TCP + podSelector: + matchLabels: + common.k8s.elastic.co/type: agent +``` + +## Isolating Logstash [k8s-network-policies-logstash-isolation] + +::::{note} +{{ls}} may require additional access rules than those listed here, depending on plugin usage. +:::: + + +| | | +| --- | --- | +| Egress (outgoing) | * TCP port 9200 to {{es}} nodes in the namespace.
* UDP port 53 for DNS lookup.
| + +```yaml +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: eck-logstash + namespace: team-a +spec: + egress: + - ports: + - port: 9200 + protocol: TCP + to: + - namespaceSelector: + matchLabels: + eck.k8s.elastic.co/tenant: team-a + podSelector: + matchLabels: + common.k8s.elastic.co/type: elasticsearch + - ports: + - port: 53 + protocol: UDP + podSelector: + matchLabels: + common.k8s.elastic.co/type: logstash +``` diff --git a/deploy-manage/deploy/cloud-on-k8s/node-configuration.md b/deploy-manage/deploy/cloud-on-k8s/node-configuration.md index ece4a7918..68d572bed 100644 --- a/deploy-manage/deploy/cloud-on-k8s/node-configuration.md +++ b/deploy-manage/deploy/cloud-on-k8s/node-configuration.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-node-configuration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/nodes-orchestration.md b/deploy-manage/deploy/cloud-on-k8s/nodes-orchestration.md index 1c59c707c..def6e10a8 100644 --- a/deploy-manage/deploy/cloud-on-k8s/nodes-orchestration.md +++ b/deploy-manage/deploy/cloud-on-k8s/nodes-orchestration.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-orchestration.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/orchestrate-other-elastic-applications.md b/deploy-manage/deploy/cloud-on-k8s/orchestrate-other-elastic-applications.md index 0ab9e93bd..2376f6ff0 100644 --- a/deploy-manage/deploy/cloud-on-k8s/orchestrate-other-elastic-applications.md +++ b/deploy-manage/deploy/cloud-on-k8s/orchestrate-other-elastic-applications.md @@ -1,3 +1,28 @@ # Orchestrate other Elastic applications -% What needs to be done: Write from scratch \ No newline at end of file +This section explains how to deploy and configure various Elastic Stack applications within Elastic Cloud on Kubernetes (ECK). + +::::{tip} +This content applies to APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash applications. To orchestrate an {{es}} cluster or {{kib}}, refer to [](./manage-deployments.md). +:::: + +The following guides provide specific instructions for deploying and configuring each application on ECK: +* [APM Server](apm-server.md) +* [Standalone Elastic Agent](standalone-elastic-agent.md) +* [{{fleet}}-managed {{agent}}](fleet-managed-elastic-agent.md) +* [Elastic Maps Server](elastic-maps-server.md) +* [Beats](beats.md) +* [{{ls}}](logstash.md) + +When orchestrating any of these applications, also consider the following topics: + +* [Elastic Stack Helm Chart](managing-deployments-using-helm-chart.md) +* [Recipes](recipes.md) +* [Secure the Elastic Stack](../../security.md) +* [Access Elastic Stack services](accessing-services.md) +* [Customize Pods](customize-pods.md) +* [Manage compute resources](manage-compute-resources.md) +* [Autoscaling stateless applications](../../autoscaling/autoscaling-stateless-applications-on-eck.md) +* [Elastic Stack configuration policies](elastic-stack-configuration-policies.md) +* [Upgrade the Elastic Stack version](../../upgrade/deployment-or-cluster.md) +* [Connect to external Elastic resources](connect-to-external-elastic-resources.md) \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-on-k8s/pod-disruption-budget.md b/deploy-manage/deploy/cloud-on-k8s/pod-disruption-budget.md index 661b5f91f..f19af44c6 100644 --- a/deploy-manage/deploy/cloud-on-k8s/pod-disruption-budget.md +++ b/deploy-manage/deploy/cloud-on-k8s/pod-disruption-budget.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-pod-disruption-budget.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/pod-prestop-hook.md b/deploy-manage/deploy/cloud-on-k8s/pod-prestop-hook.md index 2609be706..627bc5734 100644 --- a/deploy-manage/deploy/cloud-on-k8s/pod-prestop-hook.md +++ b/deploy-manage/deploy/cloud-on-k8s/pod-prestop-hook.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-prestop.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/quickstart-beats.md b/deploy-manage/deploy/cloud-on-k8s/quickstart-beats.md index bf98e7c60..e55d5e6e5 100644 --- a/deploy-manage/deploy/cloud-on-k8s/quickstart-beats.md +++ b/deploy-manage/deploy/cloud-on-k8s/quickstart-beats.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-beat-quickstart.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/quickstart-fleet.md b/deploy-manage/deploy/cloud-on-k8s/quickstart-fleet.md index 809b99ac1..26b4423ee 100644 --- a/deploy-manage/deploy/cloud-on-k8s/quickstart-fleet.md +++ b/deploy-manage/deploy/cloud-on-k8s/quickstart-fleet.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-fleet-quickstart.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/quickstart-logstash.md b/deploy-manage/deploy/cloud-on-k8s/quickstart-logstash.md index 396875dde..e382b877f 100644 --- a/deploy-manage/deploy/cloud-on-k8s/quickstart-logstash.md +++ b/deploy-manage/deploy/cloud-on-k8s/quickstart-logstash.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-quickstart.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/quickstart-standalone.md b/deploy-manage/deploy/cloud-on-k8s/quickstart-standalone.md index e2144da96..8ec7c87b7 100644 --- a/deploy-manage/deploy/cloud-on-k8s/quickstart-standalone.md +++ b/deploy-manage/deploy/cloud-on-k8s/quickstart-standalone.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-quickstart.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/readiness-probe.md b/deploy-manage/deploy/cloud-on-k8s/readiness-probe.md index 93454872c..ec7ef13f5 100644 --- a/deploy-manage/deploy/cloud-on-k8s/readiness-probe.md +++ b/deploy-manage/deploy/cloud-on-k8s/readiness-probe.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-readiness.html --- @@ -39,9 +41,10 @@ spec: Note that this requires restarting the Pods. - ## Elasticsearch versions 8.2.0 and later [k8s_elasticsearch_versions_8_2_0_and_later] +% this feature might have disappeared, we will need to investigate this a bit more, as the link below doesn't work anymore but it does for 8.15 for example. + We do not recommend overriding the default readiness probe on Elasticsearch 8.2.0 and later. ECK configures a socket based readiness probe using the Elasticsearch [readiness port feature](https://www.elastic.co/guide/en/elasticsearch/reference/current/advanced-configuration.html#readiness-tcp-port) which is not influenced by the load on the Elasticsearch cluster. diff --git a/deploy-manage/deploy/cloud-on-k8s/recipes.md b/deploy-manage/deploy/cloud-on-k8s/recipes.md index 3fd402848..6d546cab1 100644 --- a/deploy-manage/deploy/cloud-on-k8s/recipes.md +++ b/deploy-manage/deploy/cloud-on-k8s/recipes.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-recipes.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md b/deploy-manage/deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md index 1b5ac483b..79cc20359 100644 --- a/deploy-manage/deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md +++ b/deploy-manage/deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-traffic-splitting.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/required-rbac-permissions.md b/deploy-manage/deploy/cloud-on-k8s/required-rbac-permissions.md index 27bedd92d..6ff4dceb6 100644 --- a/deploy-manage/deploy/cloud-on-k8s/required-rbac-permissions.md +++ b/deploy-manage/deploy/cloud-on-k8s/required-rbac-permissions.md @@ -1,11 +1,13 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-eck-permissions.html --- # Required RBAC permissions [k8s-eck-permissions] -Installing and running ECK, as well as using ECK-managed resources requires the following Kubernetes [permissions](https://kubernetes.io/docs/reference/access-authn-authz/rbac): +Installing and running ECK, as well as using ECK-managed resources, requires the following Kubernetes [permissions](https://kubernetes.io/docs/reference/access-authn-authz/rbac): * [Installing CRDs](#k8s-eck-permissions-installing-crds) * [Installing the ECK operator](#k8s-eck-permissions-installing-operator) diff --git a/deploy-manage/deploy/cloud-on-k8s/restrict-cross-namespace-resource-associations.md b/deploy-manage/deploy/cloud-on-k8s/restrict-cross-namespace-resource-associations.md index ddbf13377..60b96bbda 100644 --- a/deploy-manage/deploy/cloud-on-k8s/restrict-cross-namespace-resource-associations.md +++ b/deploy-manage/deploy/cloud-on-k8s/restrict-cross-namespace-resource-associations.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-restrict-cross-namespace-associations.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/securing-logstash-api.md b/deploy-manage/deploy/cloud-on-k8s/securing-logstash-api.md index 421346f41..2de0724b6 100644 --- a/deploy-manage/deploy/cloud-on-k8s/securing-logstash-api.md +++ b/deploy-manage/deploy/cloud-on-k8s/securing-logstash-api.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-securing-api.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/security-context.md b/deploy-manage/deploy/cloud-on-k8s/security-context.md index cc8460d7c..3abb59085 100644 --- a/deploy-manage/deploy/cloud-on-k8s/security-context.md +++ b/deploy-manage/deploy/cloud-on-k8s/security-context.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-security-context.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/service-meshes.md b/deploy-manage/deploy/cloud-on-k8s/service-meshes.md index 1c943134a..4530b8dd3 100644 --- a/deploy-manage/deploy/cloud-on-k8s/service-meshes.md +++ b/deploy-manage/deploy/cloud-on-k8s/service-meshes.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-service-meshes.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/settings-managed-by-eck.md b/deploy-manage/deploy/cloud-on-k8s/settings-managed-by-eck.md index 58b6e22bb..617935e8b 100644 --- a/deploy-manage/deploy/cloud-on-k8s/settings-managed-by-eck.md +++ b/deploy-manage/deploy/cloud-on-k8s/settings-managed-by-eck.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-reserved-settings.html --- @@ -10,8 +12,8 @@ The following Elasticsearch settings are managed by ECK: * `cluster.name` * `discovery.seed_hosts` * `discovery.seed_providers` -* `discovery.zen.minimum_master_nodes` [7.0] -* `cluster.initial_master_nodes` [7.0] +* `discovery.zen.minimum_master_nodes` +* `cluster.initial_master_nodes` * `network.host` * `network.publish_host` * `path.data` diff --git a/deploy-manage/deploy/cloud-on-k8s/standalone-elastic-agent.md b/deploy-manage/deploy/cloud-on-k8s/standalone-elastic-agent.md index 7f597dec2..dfccd861e 100644 --- a/deploy-manage/deploy/cloud-on-k8s/standalone-elastic-agent.md +++ b/deploy-manage/deploy/cloud-on-k8s/standalone-elastic-agent.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/storage-recommendations.md b/deploy-manage/deploy/cloud-on-k8s/storage-recommendations.md index 0505593e0..2a3f64f97 100644 --- a/deploy-manage/deploy/cloud-on-k8s/storage-recommendations.md +++ b/deploy-manage/deploy/cloud-on-k8s/storage-recommendations.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-storage-recommendations.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/tls-certificates.md b/deploy-manage/deploy/cloud-on-k8s/tls-certificates.md deleted file mode 100644 index 324713c7b..000000000 --- a/deploy-manage/deploy/cloud-on-k8s/tls-certificates.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -mapped_pages: - - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-tls-certificates.html ---- - -# TLS Certificates [k8s-tls-certificates] - -This section only covers TLS certificates for the HTTP layer. TLS certificates for the transport layer that are used for internal communications between Elasticsearch nodes are managed by ECK and cannot be changed. You can however set your own certificate authority for the [transport layer](transport-settings.md#k8s-transport-ca). - -## Default self-signed certificate [k8s-default-self-signed-certificate] - -By default, the operator manages a self-signed certificate with a custom CA for each resource. The CA, the certificate and the private key are each stored in a separate `Secret`. - -```sh -> kubectl get secret | grep es-http -hulk-es-http-ca-internal Opaque 2 28m -hulk-es-http-certs-internal Opaque 2 28m -hulk-es-http-certs-public Opaque 1 28m -``` - -The public certificate is stored in a secret named `-[es|kb|apm|ent|agent]-http-certs-public`. - -```sh -> kubectl get secret hulk-es-http-certs-public -o go-template='{{index .data "tls.crt" | base64decode }}' ------BEGIN CERTIFICATE----- -MIIDQDCCAiigAwIBAgIQHC4O/RWX15a3/P3upsm3djANBgkqhkiG9w0BAQsFADA6 -... -QLYL4zLEby3vRxq65+xofVBJAaM= ------END CERTIFICATE----- -``` - -### Reserve static IP and custom domain [k8s-static-ip-custom-domain] - -To use a custom domain name with the self-signed certificate, you can reserve a static IP and/or use an Ingress instead of a `LoadBalancer` `Service`. Whatever you use, your DNS must be added to the certificate SAN in the `spec.http.tls.selfSignedCertificate.subjectAltNames` section of your Elastic resource manifest. - -```yaml -spec: - http: - service: - spec: - type: LoadBalancer - tls: - selfSignedCertificate: - subjectAltNames: - - ip: 160.46.176.15 - - dns: hulk.example.com -``` - - - -## Setup your own certificate [k8s-setting-up-your-own-certificate] - -You can bring your own certificate to configure TLS to ensure that communication between HTTP clients and the Elastic Stack application is encrypted. - -Create a Kubernetes secret with: - -* `ca.crt`: CA certificate (optional if `tls.crt` was issued by a well-known CA). -* `tls.crt`: The certificate. -* `tls.key`: The private key to the first certificate in the certificate chain. - -::::{warning} -If your `tls.crt` is signed by an intermediate CA you may need both the Root CA and the intermediate CA combined within the `ca.crt` file depending on whether the Root CA is globally trusted. -:::: - - -```sh -kubectl create secret generic my-cert --from-file=ca.crt --from-file=tls.crt --from-file=tls.key -``` - -Alternatively you can also bring your own CA certificate including a private key and let ECK issue certificates with it. Any certificate SANs you have configured as decribed in [Reserve static IP and custom domain](#k8s-static-ip-custom-domain) will also be respected when issuing certificates with this CA certificate. - -Create a Kubernetes secret with: - -* `ca.crt`: CA certificate. -* `ca.key`: The private key to the CA certificate. - -```sh -kubectl create secret generic my-cert --from-file=ca.crt --from-file=ca.key -``` - -In both cases, you have to reference the secret name in the `http.tls.certificate` section of the resource manifest. - -```yaml -spec: - http: - tls: - certificate: - secretName: my-cert -``` - - -## Disable TLS [k8s-disable-tls] - -You can explicitly disable TLS for Kibana, APM Server, and the HTTP layer of Elasticsearch. - -```yaml -spec: - http: - tls: - selfSignedCertificate: - disabled: true -``` - - diff --git a/deploy-manage/deploy/cloud-on-k8s/tools-apis.md b/deploy-manage/deploy/cloud-on-k8s/tools-apis.md index 6502ce5d4..dc9b34e50 100644 --- a/deploy-manage/deploy/cloud-on-k8s/tools-apis.md +++ b/deploy-manage/deploy/cloud-on-k8s/tools-apis.md @@ -2,4 +2,6 @@ % What needs to be done: Write from scratch -% GitHub issue: https://github.com/elastic/docs-projects/issues/310 \ No newline at end of file +% GitHub issue: https://github.com/elastic/docs-projects/issues/310 + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-on-k8s/transport-settings.md b/deploy-manage/deploy/cloud-on-k8s/transport-settings.md index 32a5dea1b..37ff1f3ba 100644 --- a/deploy-manage/deploy/cloud-on-k8s/transport-settings.md +++ b/deploy-manage/deploy/cloud-on-k8s/transport-settings.md @@ -1,11 +1,13 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-transport-settings.html --- # Transport settings [k8s-transport-settings] -The transport module in Elasticsearch is used for internal communication between nodes within the cluster as well as communication between remote clusters. Check the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html) for details. For customization options of the HTTP layer, check [Services](accessing-services.md) and [TLS certificates](tls-certificates.md). +The transport module in Elasticsearch is used for internal communication between nodes within the cluster as well as communication between remote clusters. Check the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html) for details. For customization options of the HTTP layer, check [Services](accessing-services.md) and [TLS certificates](/deploy-manage/security/secure-http-communications.md). ## Customize the Transport Service [k8s_customize_the_transport_service] @@ -29,7 +31,6 @@ When you change the `clusterIP` setting of the service, ECK deletes and re-creat :::: - ## Configure a custom Certificate Authority [k8s-transport-ca] Elasticsearch uses X.509 certificates to establish encrypted and authenticated connections across nodes in the cluster. By default, ECK creates a self-signed CA certificate to issue a certificate [for each node in the cluster](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-basic-setup.html#encrypt-internode-communication). @@ -120,8 +121,6 @@ spec: 3. If the remote cluster server is enabled, then the DNS names must also include both:* The DNS name for the related Kubernetes `Service`: `-es-remote-cluster.${POD_NAMESPACE}.svc` * The Pod DNS name: `${POD_NAME}.-es-.${POD_NAMESPACE}.svc` - - The following manifest is only provided to illustrate how these certificates can be configured in principle, using the trust-manager Bundle resource and cert-manager provisioned certificates: ```yaml diff --git a/deploy-manage/deploy/cloud-on-k8s/troubleshooting-beats.md b/deploy-manage/deploy/cloud-on-k8s/troubleshooting-beats.md index df5a2f81a..b395f520e 100644 --- a/deploy-manage/deploy/cloud-on-k8s/troubleshooting-beats.md +++ b/deploy-manage/deploy/cloud-on-k8s/troubleshooting-beats.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-beat-troubleshooting.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/update-deployments.md b/deploy-manage/deploy/cloud-on-k8s/update-deployments.md index a0f05f43e..8ce6fabaa 100644 --- a/deploy-manage/deploy/cloud-on-k8s/update-deployments.md +++ b/deploy-manage/deploy/cloud-on-k8s/update-deployments.md @@ -1,4 +1,7 @@ --- +navigation_title: Applying updates +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-update-deployment.html --- @@ -25,7 +28,4 @@ spec: EOF ``` -ECK will automatically schedule the requested update. Changes can be monitored with the [ECK operator logs](install-using-yaml-manifest-quickstart.md), [`events`](https://kubernetes.io/docs/reference/kubernetes-api/cluster-resources/event-v1/), and applicable product’s [pod `logs`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_logs/). These will either report successful application of changes or provide context for further troubleshooting. Kindly note, Kubernetes restricts some changes, for example refer to [Updating Volume Claims](volume-claim-templates.md#k8s-volume-claim-templates-update). - -This completes our quickstart guide for deploying an {{es}} cluster and {{kib}} instance with our ECK operator. We recommend continuing to [Orchestrating Elastic Stack applications](configure-deployments.md) for more configuration options - +ECK will automatically schedule the requested update. Changes can be monitored with the [ECK operator logs](install-using-yaml-manifest-quickstart.md), [`events`](https://kubernetes.io/docs/reference/kubernetes-api/cluster-resources/event-v1/), and applicable product’s [pod `logs`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_logs/). These will either report successful application of changes or provide context for further troubleshooting. Kindly note, Kubernetes restricts some changes, for example refer to [Updating Volume Claims](volume-claim-templates.md#k8s-volume-claim-templates-update). \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-on-k8s/update-strategy-logstash.md b/deploy-manage/deploy/cloud-on-k8s/update-strategy-logstash.md index 574908c0d..6a93577fe 100644 --- a/deploy-manage/deploy/cloud-on-k8s/update-strategy-logstash.md +++ b/deploy-manage/deploy/cloud-on-k8s/update-strategy-logstash.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-logstash-update-strategy.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/update-strategy.md b/deploy-manage/deploy/cloud-on-k8s/update-strategy.md index decd1f262..09d1c393f 100644 --- a/deploy-manage/deploy/cloud-on-k8s/update-strategy.md +++ b/deploy-manage/deploy/cloud-on-k8s/update-strategy.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-update-strategy.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/use-an-elasticsearch-cluster-managed-by-eck.md b/deploy-manage/deploy/cloud-on-k8s/use-an-elasticsearch-cluster-managed-by-eck.md index a4a93e0aa..8810ca43e 100644 --- a/deploy-manage/deploy/cloud-on-k8s/use-an-elasticsearch-cluster-managed-by-eck.md +++ b/deploy-manage/deploy/cloud-on-k8s/use-an-elasticsearch-cluster-managed-by-eck.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-apm-eck-managed-es.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/virtual-memory.md b/deploy-manage/deploy/cloud-on-k8s/virtual-memory.md index 52ebf84db..b8b11dffb 100644 --- a/deploy-manage/deploy/cloud-on-k8s/virtual-memory.md +++ b/deploy-manage/deploy/cloud-on-k8s/virtual-memory.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-virtual-memory.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md b/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md index 0b713c23d..fa1819228 100644 --- a/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md +++ b/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-volume-claim-templates.html --- diff --git a/deploy-manage/deploy/cloud-on-k8s/webhook-namespace-selectors.md b/deploy-manage/deploy/cloud-on-k8s/webhook-namespace-selectors.md index 41e052ff2..661a32e7e 100644 --- a/deploy-manage/deploy/cloud-on-k8s/webhook-namespace-selectors.md +++ b/deploy-manage/deploy/cloud-on-k8s/webhook-namespace-selectors.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-webhook-namespace-selectors.html --- @@ -15,6 +17,5 @@ Webhook resources are cluster-scoped, therefore `createClusterScopedResources` m It is not recommended to deploy webhook resources in environments where operators are run by untrusted users and need to be locked down tightly. :::: - For more information, check [Configure the validating webhook](configure-validating-webhook.md) and [Dynamic Admission Control](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/). diff --git a/deploy-manage/deploy/elastic-cloud/add-plugins-provided-with-elastic-cloud-hosted.md b/deploy-manage/deploy/elastic-cloud/add-plugins-provided-with-elastic-cloud-hosted.md index 62b781fac..f2545619e 100644 --- a/deploy-manage/deploy/elastic-cloud/add-plugins-provided-with-elastic-cloud-hosted.md +++ b/deploy-manage/deploy/elastic-cloud/add-plugins-provided-with-elastic-cloud-hosted.md @@ -11,4 +11,11 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-adding-plugins.md -% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-adding-elastic-plugins.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-adding-elastic-plugins.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-heroku/ech-adding-plugins.md](/raw-migrated-files/cloud/cloud-heroku/ech-adding-plugins.md) +* [/raw-migrated-files/cloud/cloud-heroku/ech-adding-elastic-plugins.md](/raw-migrated-files/cloud/cloud-heroku/ech-adding-elastic-plugins.md) \ No newline at end of file diff --git a/deploy-manage/deploy/elastic-cloud/configure.md b/deploy-manage/deploy/elastic-cloud/configure.md index 1cb52996a..de5a2762d 100644 --- a/deploy-manage/deploy/elastic-cloud/configure.md +++ b/deploy-manage/deploy/elastic-cloud/configure.md @@ -11,4 +11,11 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/cloud/cloud/ec-customize-deployment.md -% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-configure-settings.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-configure-settings.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud/ec-customize-deployment.md](/raw-migrated-files/cloud/cloud/ec-customize-deployment.md) +* [/raw-migrated-files/cloud/cloud-heroku/ech-configure-settings.md](/raw-migrated-files/cloud/cloud-heroku/ech-configure-settings.md) \ No newline at end of file diff --git a/deploy-manage/deploy/elastic-cloud/custom-endpoint-aliases.md b/deploy-manage/deploy/elastic-cloud/custom-endpoint-aliases.md index d64decd3f..e463ea00d 100644 --- a/deploy-manage/deploy/elastic-cloud/custom-endpoint-aliases.md +++ b/deploy-manage/deploy/elastic-cloud/custom-endpoint-aliases.md @@ -11,4 +11,11 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/cloud/cloud/ec-regional-deployment-aliases.md -% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-regional-deployment-aliases.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-regional-deployment-aliases.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud/ec-regional-deployment-aliases.md](/raw-migrated-files/cloud/cloud/ec-regional-deployment-aliases.md) +* [/raw-migrated-files/cloud/cloud-heroku/ech-regional-deployment-aliases.md](/raw-migrated-files/cloud/cloud-heroku/ech-regional-deployment-aliases.md) \ No newline at end of file diff --git a/deploy-manage/deploy/elastic-cloud/heroku.md b/deploy-manage/deploy/elastic-cloud/heroku.md index e5b337984..c5bf8bbe7 100644 --- a/deploy-manage/deploy/elastic-cloud/heroku.md +++ b/deploy-manage/deploy/elastic-cloud/heroku.md @@ -15,4 +15,11 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-getting-started.md -% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-about.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-about.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-heroku/ech-getting-started.md](/raw-migrated-files/cloud/cloud-heroku/ech-getting-started.md) +* [/raw-migrated-files/cloud/cloud-heroku/ech-about.md](/raw-migrated-files/cloud/cloud-heroku/ech-about.md) \ No newline at end of file diff --git a/deploy-manage/deploy/elastic-cloud/keep-track-of-deployment-activity.md b/deploy-manage/deploy/elastic-cloud/keep-track-of-deployment-activity.md index f16dc754e..dae450e0d 100644 --- a/deploy-manage/deploy/elastic-cloud/keep-track-of-deployment-activity.md +++ b/deploy-manage/deploy/elastic-cloud/keep-track-of-deployment-activity.md @@ -13,4 +13,11 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/cloud/cloud/ec-activity-page.md -% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-activity-page.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-activity-page.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud/ec-activity-page.md](/raw-migrated-files/cloud/cloud/ec-activity-page.md) +* [/raw-migrated-files/cloud/cloud-heroku/ech-activity-page.md](/raw-migrated-files/cloud/cloud-heroku/ech-activity-page.md) \ No newline at end of file diff --git a/deploy-manage/deploy/elastic-cloud/serverless.md b/deploy-manage/deploy/elastic-cloud/serverless.md index cf5538c38..8a888a9ce 100644 --- a/deploy-manage/deploy/elastic-cloud/serverless.md +++ b/deploy-manage/deploy/elastic-cloud/serverless.md @@ -14,4 +14,11 @@ mapped_urls: % - [ ] ./raw-migrated-files/docs-content/serverless/intro.md % - [ ] ./raw-migrated-files/docs-content/serverless/general-serverless-status.md -% Notes: also in troubleshooting \ No newline at end of file +% Notes: also in troubleshooting + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/docs-content/serverless/intro.md](/raw-migrated-files/docs-content/serverless/intro.md) +* [/raw-migrated-files/docs-content/serverless/general-serverless-status.md](/raw-migrated-files/docs-content/serverless/general-serverless-status.md) \ No newline at end of file diff --git a/deploy-manage/deploy/elastic-cloud/tools-apis.md b/deploy-manage/deploy/elastic-cloud/tools-apis.md index 2210a9f78..1d32278f7 100644 --- a/deploy-manage/deploy/elastic-cloud/tools-apis.md +++ b/deploy-manage/deploy/elastic-cloud/tools-apis.md @@ -16,4 +16,10 @@ mapped_urls: % - [ ] ./raw-migrated-files/docs-content/serverless/elasticsearch-http-apis.md % - [ ] https://www.elastic.co/guide/en/tpec/current/index.html -% Notes: reference only, this page wasn't migrated, but you can pull from the live URL if needed. \ No newline at end of file +% Notes: reference only, this page wasn't migrated, but you can pull from the live URL if needed. + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/docs-content/serverless/elasticsearch-http-apis.md](/raw-migrated-files/docs-content/serverless/elasticsearch-http-apis.md) \ No newline at end of file diff --git a/deploy-manage/deploy/kibana-reporting-configuration.md b/deploy-manage/deploy/kibana-reporting-configuration.md index 806b6eb90..b1d3045a0 100644 --- a/deploy-manage/deploy/kibana-reporting-configuration.md +++ b/deploy-manage/deploy/kibana-reporting-configuration.md @@ -15,4 +15,11 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/kibana/kibana/secure-reporting.md -% - [ ] ./raw-migrated-files/kibana/kibana/reporting-production-considerations.md \ No newline at end of file +% - [ ] ./raw-migrated-files/kibana/kibana/reporting-production-considerations.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/kibana/kibana/secure-reporting.md](/raw-migrated-files/kibana/kibana/secure-reporting.md) +* [/raw-migrated-files/kibana/kibana/reporting-production-considerations.md](/raw-migrated-files/kibana/kibana/reporting-production-considerations.md) \ No newline at end of file diff --git a/deploy-manage/deploy/self-managed/install-kibana.md b/deploy-manage/deploy/self-managed/install-kibana.md index feeb8d10e..a6bc5ebd5 100644 --- a/deploy-manage/deploy/self-managed/install-kibana.md +++ b/deploy-manage/deploy/self-managed/install-kibana.md @@ -16,4 +16,11 @@ mapped_urls: % - [ ] ./raw-migrated-files/kibana/kibana/setup.md % Notes: 5 child docs, all needed -% - [ ] ./raw-migrated-files/kibana/kibana/install.md \ No newline at end of file +% - [ ] ./raw-migrated-files/kibana/kibana/install.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/kibana/kibana/setup.md](/raw-migrated-files/kibana/kibana/setup.md) +* [/raw-migrated-files/kibana/kibana/install.md](/raw-migrated-files/kibana/kibana/install.md) \ No newline at end of file diff --git a/deploy-manage/deploy/self-managed/other-configuration-settings.md b/deploy-manage/deploy/self-managed/other-configuration-settings.md index 0e5554f28..d03724905 100644 --- a/deploy-manage/deploy/self-managed/other-configuration-settings.md +++ b/deploy-manage/deploy/self-managed/other-configuration-settings.md @@ -4,4 +4,6 @@ % GitHub issue: https://github.com/elastic/docs-projects/issues/340 -% Scope notes: Link to reference documentation? Where are we going to allocate the rest of the config settings? Reference? \ No newline at end of file +% Scope notes: Link to reference documentation? Where are we going to allocate the rest of the config settings? Reference? + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/deploy/self-managed/tools-apis.md b/deploy-manage/deploy/self-managed/tools-apis.md index 6502ce5d4..dc9b34e50 100644 --- a/deploy-manage/deploy/self-managed/tools-apis.md +++ b/deploy-manage/deploy/self-managed/tools-apis.md @@ -2,4 +2,6 @@ % What needs to be done: Write from scratch -% GitHub issue: https://github.com/elastic/docs-projects/issues/310 \ No newline at end of file +% GitHub issue: https://github.com/elastic/docs-projects/issues/310 + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/distributed-architecture/shard-allocation-relocation-recovery/index-level-shard-allocation.md b/deploy-manage/distributed-architecture/shard-allocation-relocation-recovery/index-level-shard-allocation.md index bc0ccdfc2..cb03cae8a 100644 --- a/deploy-manage/distributed-architecture/shard-allocation-relocation-recovery/index-level-shard-allocation.md +++ b/deploy-manage/distributed-architecture/shard-allocation-relocation-recovery/index-level-shard-allocation.md @@ -18,4 +18,12 @@ mapped_urls: % - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/shard-allocation-filtering.md % Notes: conceptual content % - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/recovery-prioritization.md -% Notes: conceptual content \ No newline at end of file +% Notes: conceptual content + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/index-modules-allocation.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/index-modules-allocation.md) +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/shard-allocation-filtering.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/shard-allocation-filtering.md) +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/recovery-prioritization.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/recovery-prioritization.md) \ No newline at end of file diff --git a/deploy-manage/license/manage-your-license-in-eck.md b/deploy-manage/license/manage-your-license-in-eck.md index 3cd98799a..a1fd8687e 100644 --- a/deploy-manage/license/manage-your-license-in-eck.md +++ b/deploy-manage/license/manage-your-license-in-eck.md @@ -1,4 +1,6 @@ --- +applies: + eck: all mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-licensing.html --- diff --git a/deploy-manage/maintenance/ece.md b/deploy-manage/maintenance/ece.md index 536f8426b..34c4b70c5 100644 --- a/deploy-manage/maintenance/ece.md +++ b/deploy-manage/maintenance/ece.md @@ -4,4 +4,6 @@ % GitHub issue: https://github.com/elastic/docs-projects/issues/353 -% Scope notes: Introduction about ECE maintenance and activities / actions. Explain the difference between deployments maintenance and ECE hosts infrastructure maintenance. \ No newline at end of file +% Scope notes: Introduction about ECE maintenance and activities / actions. Explain the difference between deployments maintenance and ECE hosts infrastructure maintenance. + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/maintenance/ece/maintenance-activities.md b/deploy-manage/maintenance/ece/maintenance-activities.md index 43bd56c89..7ac140925 100644 --- a/deploy-manage/maintenance/ece/maintenance-activities.md +++ b/deploy-manage/maintenance/ece/maintenance-activities.md @@ -4,4 +4,6 @@ % GitHub issue: https://github.com/elastic/docs-projects/issues/353 -% Scope notes: summarize the list of activites \ No newline at end of file +% Scope notes: summarize the list of activites + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/maintenance/ece/start-stop-routing-requests.md b/deploy-manage/maintenance/ece/start-stop-routing-requests.md index 064f97ce7..f0e42a2a1 100644 --- a/deploy-manage/maintenance/ece/start-stop-routing-requests.md +++ b/deploy-manage/maintenance/ece/start-stop-routing-requests.md @@ -13,4 +13,11 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-maintenance-mode-routing.md -% - [ ] ./raw-migrated-files/cloud/cloud/ec-maintenance-mode-routing.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud/ec-maintenance-mode-routing.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-enterprise/ece-maintenance-mode-routing.md](/raw-migrated-files/cloud/cloud-enterprise/ece-maintenance-mode-routing.md) +* [/raw-migrated-files/cloud/cloud/ec-maintenance-mode-routing.md](/raw-migrated-files/cloud/cloud/ec-maintenance-mode-routing.md) \ No newline at end of file diff --git a/deploy-manage/maintenance/start-stop-services.md b/deploy-manage/maintenance/start-stop-services.md index 59059deeb..758a44ecc 100644 --- a/deploy-manage/maintenance/start-stop-services.md +++ b/deploy-manage/maintenance/start-stop-services.md @@ -4,4 +4,6 @@ % GitHub issue: https://github.com/elastic/docs-projects/issues/353 -% Scope notes: Super brief summary \ No newline at end of file +% Scope notes: Super brief summary + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/maintenance/start-stop-services/restart-cloud-hosted-deployment.md b/deploy-manage/maintenance/start-stop-services/restart-cloud-hosted-deployment.md index 549659e41..f6dba06f8 100644 --- a/deploy-manage/maintenance/start-stop-services/restart-cloud-hosted-deployment.md +++ b/deploy-manage/maintenance/start-stop-services/restart-cloud-hosted-deployment.md @@ -14,4 +14,11 @@ mapped_urls: % - [ ] ./raw-migrated-files/cloud/cloud/ec-restart-deployment.md % - [ ] ./raw-migrated-files/cloud/cloud/ec-api-deployment-other.md -% Notes: api example \ No newline at end of file +% Notes: api example + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud/ec-restart-deployment.md](/raw-migrated-files/cloud/cloud/ec-restart-deployment.md) +* [/raw-migrated-files/cloud/cloud/ec-api-deployment-other.md](/raw-migrated-files/cloud/cloud/ec-api-deployment-other.md) \ No newline at end of file diff --git a/deploy-manage/monitor.md b/deploy-manage/monitor.md index a2d231bcc..93fee1b46 100644 --- a/deploy-manage/monitor.md +++ b/deploy-manage/monitor.md @@ -22,4 +22,11 @@ applies: % - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/monitor-elasticsearch-cluster.md % Notes: Existing articles about monitoring: Elasticsearch, Cloud, Cloud-enterprise, Cloud on Kubernetes, Kibana books Might need a new landing page -% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/secure-monitoring.md \ No newline at end of file +% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/secure-monitoring.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/monitor-elasticsearch-cluster.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/monitor-elasticsearch-cluster.md) +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/secure-monitoring.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/secure-monitoring.md) \ No newline at end of file diff --git a/deploy-manage/monitor/logging-configuration.md b/deploy-manage/monitor/logging-configuration.md index 80a04a2b2..9f2e5c81b 100644 --- a/deploy-manage/monitor/logging-configuration.md +++ b/deploy-manage/monitor/logging-configuration.md @@ -9,4 +9,6 @@ applies: % What needs to be done: Write from scratch -% GitHub issue: https://github.com/elastic/docs-projects/issues/350 \ No newline at end of file +% GitHub issue: https://github.com/elastic/docs-projects/issues/350 + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs-in-orchestrated-deployments.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs-in-orchestrated-deployments.md index 06e2ffd2d..f2eac00a2 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs-in-orchestrated-deployments.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs-in-orchestrated-deployments.md @@ -21,4 +21,12 @@ applies: % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-enable-auditing.md % - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md -% - [ ] ./raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-enterprise/ece-enable-auditing.md](/raw-migrated-files/cloud/cloud-enterprise/ece-enable-auditing.md) +* [/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md](/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md) +* [/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md](/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md) \ No newline at end of file diff --git a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md index 7d12c21f7..71f489ef9 100644 --- a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md +++ b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md @@ -11,4 +11,6 @@ applies: % GitHub issue: https://github.com/elastic/docs-projects/issues/350 -% Scope notes: Landing page about audit logs in Kibana and Elasticsearch, explaining how they can be enabled and configured, and also linking to the page about correlating information. We can create a doc to explain how to enable audit logging in both Elasticsearch and Kibana, and considering also ECE and orchestrated deployments. Kibana audit events list should be moved to reference content. \ No newline at end of file +% Scope notes: Landing page about audit logs in Kibana and Elasticsearch, explaining how they can be enabled and configured, and also linking to the page about correlating information. We can create a doc to explain how to enable audit logging in both Elasticsearch and Kibana, and considering also ECE and orchestrated deployments. Kibana audit events list should be moved to reference content. + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/monitor/monitoring-data.md b/deploy-manage/monitor/monitoring-data.md index 240512af5..26e53e9ef 100644 --- a/deploy-manage/monitor/monitoring-data.md +++ b/deploy-manage/monitor/monitoring-data.md @@ -14,3 +14,6 @@ applies: % GitHub issue: https://github.com/elastic/docs-projects/issues/350 % Scope notes: we can review the name of this section... + + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/monitor/monitoring-data/access-performance-metrics-on-elastic-cloud.md b/deploy-manage/monitor/monitoring-data/access-performance-metrics-on-elastic-cloud.md index 7b56e1240..a95c7e22b 100644 --- a/deploy-manage/monitor/monitoring-data/access-performance-metrics-on-elastic-cloud.md +++ b/deploy-manage/monitor/monitoring-data/access-performance-metrics-on-elastic-cloud.md @@ -15,4 +15,11 @@ applies: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/cloud/cloud/ec-saas-metrics-accessing.md -% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-saas-metrics-accessing.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-saas-metrics-accessing.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud/ec-saas-metrics-accessing.md](/raw-migrated-files/cloud/cloud/ec-saas-metrics-accessing.md) +* [/raw-migrated-files/cloud/cloud-heroku/ech-saas-metrics-accessing.md](/raw-migrated-files/cloud/cloud-heroku/ech-saas-metrics-accessing.md) \ No newline at end of file diff --git a/deploy-manage/monitor/orchestrators.md b/deploy-manage/monitor/orchestrators.md index 2d671dbd2..b0fdaf278 100644 --- a/deploy-manage/monitor/orchestrators.md +++ b/deploy-manage/monitor/orchestrators.md @@ -10,4 +10,6 @@ applies: % GitHub issue: https://github.com/elastic/docs-projects/issues/350 -% Scope notes: Landing page to monitoring orchestrators (not deployments) \ No newline at end of file +% Scope notes: Landing page to monitoring orchestrators (not deployments) + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/monitor/stack-monitoring/elasticsearch-monitoring-self-managed.md b/deploy-manage/monitor/stack-monitoring/elasticsearch-monitoring-self-managed.md index 5d7c38ad5..37788d76b 100644 --- a/deploy-manage/monitor/stack-monitoring/elasticsearch-monitoring-self-managed.md +++ b/deploy-manage/monitor/stack-monitoring/elasticsearch-monitoring-self-managed.md @@ -16,4 +16,11 @@ applies: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/monitoring-production.md -% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/secure-monitoring.md \ No newline at end of file +% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/secure-monitoring.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/monitoring-production.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/monitoring-production.md) +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/secure-monitoring.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/secure-monitoring.md) \ No newline at end of file diff --git a/deploy-manage/security/encrypt-deployment.md b/deploy-manage/security/encrypt-deployment.md index 37095fb51..e0fe9ba44 100644 --- a/deploy-manage/security/encrypt-deployment.md +++ b/deploy-manage/security/encrypt-deployment.md @@ -2,4 +2,6 @@ % What needs to be done: Write from scratch -% GitHub issue: https://github.com/elastic/docs-projects/issues/346 \ No newline at end of file +% GitHub issue: https://github.com/elastic/docs-projects/issues/346 + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/security/private-link-traffic-filters.md b/deploy-manage/security/private-link-traffic-filters.md index f7d0556d1..68db09211 100644 --- a/deploy-manage/security/private-link-traffic-filters.md +++ b/deploy-manage/security/private-link-traffic-filters.md @@ -2,4 +2,6 @@ % What needs to be done: Write from scratch -% GitHub issue: https://github.com/elastic/docs-projects/issues/346 \ No newline at end of file +% GitHub issue: https://github.com/elastic/docs-projects/issues/346 + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/security/secure-http-communications.md b/deploy-manage/security/secure-http-communications.md index 933f4a529..43675d22b 100644 --- a/deploy-manage/security/secure-http-communications.md +++ b/deploy-manage/security/secure-http-communications.md @@ -20,6 +20,9 @@ mapped_urls: % - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-custom-http-certificate.md % - [ ] ./raw-migrated-files/kibana/kibana/Security-production-considerations.md +% EEDUGON NOTE: security section might miss a section to secure the transport layer (not the HTTP). +% There we should integrate the content of https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-transport-settings.html which is currently in ECK (/deploy-manage) doc. + % Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): $$$encrypt-kibana-browser$$$ @@ -46,4 +49,6 @@ $$$csp-strict-mode$$$ $$$k8s-setting-up-your-own-certificate$$$ -$$$k8s-static-ip-custom-domain$$$ \ No newline at end of file +$$$k8s-static-ip-custom-domain$$$ + +$$$k8s-disable-tls$$$ \ No newline at end of file diff --git a/deploy-manage/security/traffic-filtering.md b/deploy-manage/security/traffic-filtering.md index 63a9e9723..58c1312a1 100644 --- a/deploy-manage/security/traffic-filtering.md +++ b/deploy-manage/security/traffic-filtering.md @@ -15,4 +15,12 @@ mapped_urls: % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-traffic-filtering-deployment-configuration.md % - [ ] ./raw-migrated-files/cloud/cloud/ec-traffic-filtering-deployment-configuration.md -% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-traffic-filtering-deployment-configuration.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-traffic-filtering-deployment-configuration.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-enterprise/ece-traffic-filtering-deployment-configuration.md](/raw-migrated-files/cloud/cloud-enterprise/ece-traffic-filtering-deployment-configuration.md) +* [/raw-migrated-files/cloud/cloud/ec-traffic-filtering-deployment-configuration.md](/raw-migrated-files/cloud/cloud/ec-traffic-filtering-deployment-configuration.md) +* [/raw-migrated-files/cloud/cloud-heroku/ech-traffic-filtering-deployment-configuration.md](/raw-migrated-files/cloud/cloud-heroku/ech-traffic-filtering-deployment-configuration.md) \ No newline at end of file diff --git a/deploy-manage/toc.yml b/deploy-manage/toc.yml index 3224f254f..15ef9b9e0 100644 --- a/deploy-manage/toc.yml +++ b/deploy-manage/toc.yml @@ -232,6 +232,7 @@ toc: children: - file: deploy/cloud-on-k8s/install-using-yaml-manifest-quickstart.md - file: deploy/cloud-on-k8s/install-using-helm-chart.md + - file: deploy/cloud-on-k8s/required-rbac-permissions.md - file: deploy/cloud-on-k8s/deploy-eck-on-openshift.md children: - file: deploy/cloud-on-k8s/k8s-openshift-deploy-operator.md @@ -241,27 +242,18 @@ toc: - file: deploy/cloud-on-k8s/k8s-openshift-beats.md - file: deploy/cloud-on-k8s/k8s-openshift-agent.md - file: deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md - children: - - file: deploy/cloud-on-k8s/k8s-autopilot-setting-virtual-memory.md - - file: deploy/cloud-on-k8s/k8s-autopilot-deploy-operator.md - - file: deploy/cloud-on-k8s/k8s-autopilot-deploy-elasticsearch.md - - file: deploy/cloud-on-k8s/k8s-autopilot-deploy-agent-beats.md - file: deploy/cloud-on-k8s/deploy-fips-compatible-version-of-eck.md - file: deploy/cloud-on-k8s/air-gapped-install.md - file: deploy/cloud-on-k8s/configure.md children: - file: deploy/cloud-on-k8s/configure-eck.md - - file: deploy/cloud-on-k8s/required-rbac-permissions.md - file: deploy/cloud-on-k8s/configure-validating-webhook.md - file: deploy/cloud-on-k8s/restrict-cross-namespace-resource-associations.md - - file: deploy/cloud-on-k8s/create-custom-images.md - file: deploy/cloud-on-k8s/service-meshes.md children: - file: deploy/cloud-on-k8s/k8s-service-mesh-istio.md - file: deploy/cloud-on-k8s/k8s-service-mesh-linkerd.md - file: deploy/cloud-on-k8s/network-policies.md - children: - - file: deploy/cloud-on-k8s/k8s_prerequisites.md - file: deploy/cloud-on-k8s/webhook-namespace-selectors.md - file: deploy/cloud-on-k8s/manage-deployments.md children: @@ -274,21 +266,22 @@ toc: children: - file: deploy/cloud-on-k8s/elasticsearch-configuration.md children: + - file: deploy/cloud-on-k8s/nodes-orchestration.md + - file: deploy/cloud-on-k8s/storage-recommendations.md - file: deploy/cloud-on-k8s/node-configuration.md - file: deploy/cloud-on-k8s/volume-claim-templates.md - - file: deploy/cloud-on-k8s/storage-recommendations.md - - file: deploy/cloud-on-k8s/transport-settings.md - file: deploy/cloud-on-k8s/virtual-memory.md - file: deploy/cloud-on-k8s/settings-managed-by-eck.md - file: deploy/cloud-on-k8s/custom-configuration-files-plugins.md - file: deploy/cloud-on-k8s/init-containers-for-plugin-downloads.md + - file: deploy/cloud-on-k8s/transport-settings.md - file: deploy/cloud-on-k8s/update-strategy.md - file: deploy/cloud-on-k8s/pod-disruption-budget.md - - file: deploy/cloud-on-k8s/nodes-orchestration.md - file: deploy/cloud-on-k8s/advanced-elasticsearch-node-scheduling.md - file: deploy/cloud-on-k8s/readiness-probe.md - file: deploy/cloud-on-k8s/pod-prestop-hook.md - file: deploy/cloud-on-k8s/security-context.md + - file: deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md - file: deploy/cloud-on-k8s/kibana-configuration.md children: - file: deploy/cloud-on-k8s/k8s-kibana-es.md @@ -296,13 +289,11 @@ toc: - file: deploy/cloud-on-k8s/k8s-kibana-secure-settings.md - file: deploy/cloud-on-k8s/k8s-kibana-http-configuration.md - file: deploy/cloud-on-k8s/k8s-kibana-plugins.md - - file: deploy/cloud-on-k8s/tls-certificates.md - - file: deploy/cloud-on-k8s/recipes.md - - file: deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md - file: deploy/cloud-on-k8s/customize-pods.md - file: deploy/cloud-on-k8s/manage-compute-resources.md - - file: deploy/cloud-on-k8s/elastic-stack-configuration-policies.md + - file: deploy/cloud-on-k8s/recipes.md - file: deploy/cloud-on-k8s/connect-to-external-elastic-resources.md + - file: deploy/cloud-on-k8s/elastic-stack-configuration-policies.md - file: deploy/cloud-on-k8s/orchestrate-other-elastic-applications.md children: - file: deploy/cloud-on-k8s/apm-server.md @@ -342,6 +333,7 @@ toc: - file: deploy/cloud-on-k8s/configuration-examples-logstash.md - file: deploy/cloud-on-k8s/update-strategy-logstash.md - file: deploy/cloud-on-k8s/advanced-configuration-logstash.md + - file: deploy/cloud-on-k8s/create-custom-images.md - file: deploy/cloud-on-k8s/tools-apis.md - file: deploy/self-managed.md children: @@ -855,4 +847,4 @@ toc: - file: uninstall/uninstall-elastic-cloud-enterprise.md - file: uninstall/uninstall-elastic-cloud-on-kubernetes.md - file: uninstall/uninstall-a-self-managed-cluster.md - - file: uninstall/delete-a-cloud-deployment.md \ No newline at end of file + - file: uninstall/delete-a-cloud-deployment.md diff --git a/deploy-manage/tools/snapshot-and-restore/elastic-cloud-hosted.md b/deploy-manage/tools/snapshot-and-restore/elastic-cloud-hosted.md index eef2e7d74..6aed00f7f 100644 --- a/deploy-manage/tools/snapshot-and-restore/elastic-cloud-hosted.md +++ b/deploy-manage/tools/snapshot-and-restore/elastic-cloud-hosted.md @@ -13,4 +13,11 @@ mapped_urls: % - [ ] ./raw-migrated-files/cloud/cloud/ec-custom-repository.md % Notes: 3 children % - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-custom-repository.md -% Notes: redirects only \ No newline at end of file +% Notes: redirects only + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud/ec-custom-repository.md](/raw-migrated-files/cloud/cloud/ec-custom-repository.md) +* [/raw-migrated-files/cloud/cloud-heroku/ech-custom-repository.md](/raw-migrated-files/cloud/cloud-heroku/ech-custom-repository.md) \ No newline at end of file diff --git a/deploy-manage/uninstall.md b/deploy-manage/uninstall.md index df04a3d75..99847c256 100644 --- a/deploy-manage/uninstall.md +++ b/deploy-manage/uninstall.md @@ -2,4 +2,6 @@ % What needs to be done: Write from scratch -% GitHub issue: https://github.com/elastic/docs-projects/issues/362 \ No newline at end of file +% GitHub issue: https://github.com/elastic/docs-projects/issues/362 + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/uninstall/delete-a-cloud-deployment.md b/deploy-manage/uninstall/delete-a-cloud-deployment.md index 8e1b9c677..cc0073629 100644 --- a/deploy-manage/uninstall/delete-a-cloud-deployment.md +++ b/deploy-manage/uninstall/delete-a-cloud-deployment.md @@ -22,4 +22,14 @@ mapped_urls: % - [ ] ./raw-migrated-files/cloud/cloud/ec-delete-deployment.md % - [ ] ./raw-migrated-files/docs-content/serverless/general-billing-stop-project.html % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-restore-deployment.md -% Notes: if you need to restore a deleted deployment ... \ No newline at end of file +% Notes: if you need to restore a deleted deployment ... + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-heroku/ech-delete-deployment.md](/raw-migrated-files/cloud/cloud-heroku/ech-delete-deployment.md) +* [/raw-migrated-files/cloud/cloud-enterprise/ece-delete-deployment.md](/raw-migrated-files/cloud/cloud-enterprise/ece-delete-deployment.md) +* [/raw-migrated-files/cloud/cloud-enterprise/ece-terminate-deployment.md](/raw-migrated-files/cloud/cloud-enterprise/ece-terminate-deployment.md) +* [/raw-migrated-files/cloud/cloud/ec-delete-deployment.md](/raw-migrated-files/cloud/cloud/ec-delete-deployment.md) +* [/raw-migrated-files/cloud/cloud-enterprise/ece-restore-deployment.md](/raw-migrated-files/cloud/cloud-enterprise/ece-restore-deployment.md) \ No newline at end of file diff --git a/deploy-manage/uninstall/uninstall-a-self-managed-cluster.md b/deploy-manage/uninstall/uninstall-a-self-managed-cluster.md index 67922a5f0..114f023b5 100644 --- a/deploy-manage/uninstall/uninstall-a-self-managed-cluster.md +++ b/deploy-manage/uninstall/uninstall-a-self-managed-cluster.md @@ -2,4 +2,6 @@ % What needs to be done: Write from scratch -% GitHub issue: https://github.com/elastic/docs-projects/issues/362 \ No newline at end of file +% GitHub issue: https://github.com/elastic/docs-projects/issues/362 + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/upgrade.md b/deploy-manage/upgrade.md index 9da1ca6e8..bd9fe06e9 100644 --- a/deploy-manage/upgrade.md +++ b/deploy-manage/upgrade.md @@ -4,4 +4,6 @@ % GitHub issue: https://github.com/elastic/docs-projects/issues/355 -% Scope notes: Can have version-specific child pages Connection between upgrading your orchestrator and your cluster \ No newline at end of file +% Scope notes: Can have version-specific child pages Connection between upgrading your orchestrator and your cluster + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/upgrade/internal-upgrade-processes.md b/deploy-manage/upgrade/internal-upgrade-processes.md index fabe5e160..a8ce64547 100644 --- a/deploy-manage/upgrade/internal-upgrade-processes.md +++ b/deploy-manage/upgrade/internal-upgrade-processes.md @@ -1,3 +1,5 @@ # Internal upgrade processes -% What needs to be done: Write from scratch \ No newline at end of file +% What needs to be done: Write from scratch + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/upgrade/orchestrator.md b/deploy-manage/upgrade/orchestrator.md index e3658756f..15ba07a03 100644 --- a/deploy-manage/upgrade/orchestrator.md +++ b/deploy-manage/upgrade/orchestrator.md @@ -2,4 +2,6 @@ % What needs to be done: Write from scratch -% GitHub issue: https://github.com/elastic/docs-projects/issues/355 \ No newline at end of file +% GitHub issue: https://github.com/elastic/docs-projects/issues/355 + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/upgrade/orchestrator/upgrade-cloud-enterprise.md b/deploy-manage/upgrade/orchestrator/upgrade-cloud-enterprise.md index 34181933b..093990370 100644 --- a/deploy-manage/upgrade/orchestrator/upgrade-cloud-enterprise.md +++ b/deploy-manage/upgrade/orchestrator/upgrade-cloud-enterprise.md @@ -13,4 +13,11 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-upgrade.md -% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece_re_running_the_ece_upgrade.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece_re_running_the_ece_upgrade.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-enterprise/ece-upgrade.md](/raw-migrated-files/cloud/cloud-enterprise/ece-upgrade.md) +* [/raw-migrated-files/cloud/cloud-enterprise/ece_re_running_the_ece_upgrade.md](/raw-migrated-files/cloud/cloud-enterprise/ece_re_running_the_ece_upgrade.md) \ No newline at end of file diff --git a/deploy-manage/upgrade/prepare-to-upgrade.md b/deploy-manage/upgrade/prepare-to-upgrade.md index 0ea12c3f8..75c76c291 100644 --- a/deploy-manage/upgrade/prepare-to-upgrade.md +++ b/deploy-manage/upgrade/prepare-to-upgrade.md @@ -2,4 +2,6 @@ % What needs to be done: Write from scratch -% Scope notes: Prerequisites and requirements \ No newline at end of file +% Scope notes: Prerequisites and requirements + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/users-roles/cloud-enterprise-orchestrator.md b/deploy-manage/users-roles/cloud-enterprise-orchestrator.md index 4488b5e2a..5fbaaf03d 100644 --- a/deploy-manage/users-roles/cloud-enterprise-orchestrator.md +++ b/deploy-manage/users-roles/cloud-enterprise-orchestrator.md @@ -2,4 +2,6 @@ % What needs to be done: Write from scratch -% GitHub issue: https://github.com/elastic/docs-projects/issues/347 \ No newline at end of file +% GitHub issue: https://github.com/elastic/docs-projects/issues/347 + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/users-roles/cluster-or-deployment-auth.md b/deploy-manage/users-roles/cluster-or-deployment-auth.md index 7d1f2403e..70d45367c 100644 --- a/deploy-manage/users-roles/cluster-or-deployment-auth.md +++ b/deploy-manage/users-roles/cluster-or-deployment-auth.md @@ -13,4 +13,11 @@ mapped_urls: % Use migrated content from existing pages that map to this page: % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-securing-clusters.md -% - [ ] ./raw-migrated-files/cloud/cloud/ec-security.md \ No newline at end of file +% - [ ] ./raw-migrated-files/cloud/cloud/ec-security.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-enterprise/ece-securing-clusters.md](/raw-migrated-files/cloud/cloud-enterprise/ece-securing-clusters.md) +* [/raw-migrated-files/cloud/cloud/ec-security.md](/raw-migrated-files/cloud/cloud/ec-security.md) \ No newline at end of file diff --git a/deploy-manage/users-roles/cluster-or-deployment-auth/external-authentication.md b/deploy-manage/users-roles/cluster-or-deployment-auth/external-authentication.md index a2bbca6e0..9202a19c4 100644 --- a/deploy-manage/users-roles/cluster-or-deployment-auth/external-authentication.md +++ b/deploy-manage/users-roles/cluster-or-deployment-auth/external-authentication.md @@ -1,3 +1,5 @@ # External authentication -% What needs to be done: Write from scratch \ No newline at end of file +% What needs to be done: Write from scratch + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/users-roles/cluster-or-deployment-auth/internal-authentication.md b/deploy-manage/users-roles/cluster-or-deployment-auth/internal-authentication.md index 835d5e397..5e442a92f 100644 --- a/deploy-manage/users-roles/cluster-or-deployment-auth/internal-authentication.md +++ b/deploy-manage/users-roles/cluster-or-deployment-auth/internal-authentication.md @@ -1,3 +1,5 @@ # Internal authentication -% What needs to be done: Write from scratch \ No newline at end of file +% What needs to be done: Write from scratch + +⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file diff --git a/deploy-manage/users-roles/cluster-or-deployment-auth/kerberos.md b/deploy-manage/users-roles/cluster-or-deployment-auth/kerberos.md index 8d71b1972..8b211fcf0 100644 --- a/deploy-manage/users-roles/cluster-or-deployment-auth/kerberos.md +++ b/deploy-manage/users-roles/cluster-or-deployment-auth/kerberos.md @@ -17,4 +17,13 @@ mapped_urls: % - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-secure-clusters-kerberos.md % - [ ] ./raw-migrated-files/cloud/cloud/ec-secure-clusters-kerberos.md % - [ ] ./raw-migrated-files/cloud/cloud-heroku/ech-secure-clusters-kerberos.md -% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/kerberos-realm.md \ No newline at end of file +% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/kerberos-realm.md + +⚠️ **This page is a work in progress.** ⚠️ + +The documentation team is working to combine content pulled from the following pages: + +* [/raw-migrated-files/cloud/cloud-enterprise/ece-secure-clusters-kerberos.md](/raw-migrated-files/cloud/cloud-enterprise/ece-secure-clusters-kerberos.md) +* [/raw-migrated-files/cloud/cloud/ec-secure-clusters-kerberos.md](/raw-migrated-files/cloud/cloud/ec-secure-clusters-kerberos.md) +* [/raw-migrated-files/cloud/cloud-heroku/ech-secure-clusters-kerberos.md](/raw-migrated-files/cloud/cloud-heroku/ech-secure-clusters-kerberos.md) +* [/raw-migrated-files/elasticsearch/elasticsearch-reference/kerberos-realm.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/kerberos-realm.md) \ No newline at end of file diff --git a/manage-data/lifecycle/index-lifecycle-management.md b/manage-data/lifecycle/index-lifecycle-management.md index 5fb70203c..9840bad15 100644 --- a/manage-data/lifecycle/index-lifecycle-management.md +++ b/manage-data/lifecycle/index-lifecycle-management.md @@ -2,43 +2,83 @@ mapped_urls: - https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html - https://www.elastic.co/guide/en/elasticsearch/reference/current/overview-index-lifecycle-management.html - - https://www.elastic.co/guide/en/serverless/current/elasticsearch-differences.html#elasticsearch-differences-serverless-features-replaced - - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-migrate-index-management.html - - https://www.elastic.co/guide/en/cloud/current/ec-configure-index-management.html - - https://www.elastic.co/guide/en/cloud/current/ec-migrate-index-management.html + - https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-concepts.html --- # Index lifecycle management -% What needs to be done: Refine +{{ilm-cap}} ({{ilm-init}}) provides an integrated and streamlined way to manage time-based data such as logs and metrics, making it easier to follow best practices for managing your indices. -% GitHub issue: docs-projects#376 +You can configure {{ilm-init}} policies to automatically manage indices according to your performance, resiliency, and retention requirements. For example, you could use {{ilm-init}} to: -% Scope notes: Combine the linked resources. The "migrate to ILM" one is probably outdated now. +* Spin up a new index when an index reaches a certain size or number of documents +* Create a new index each day, week, or month and archive previous ones +* Delete stale indices to enforce data retention standards -% Use migrated content from existing pages that map to this page: +::::{tip} +{{ilm-init}} is not available on {{es-serverless}}. -% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/index-lifecycle-management.md -% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/overview-index-lifecycle-management.md -% - [ ] ./raw-migrated-files/docs-content/serverless/elasticsearch-differences.md -% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-migrate-index-management.md -% - [ ] ./raw-migrated-files/cloud/cloud/ec-configure-index-management.md -% - [ ] ./raw-migrated-files/cloud/cloud/ec-migrate-index-management.md +:::{dropdown} Why? +In an {{ecloud}} or self-managed environment, ILM lets you automatically transition indices through data tiers according to your performance needs and retention requirements. This allows you to balance hardware costs with performance. {{es-serverless}} eliminates this complexity by optimizing your cluster performance for you. -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +Data stream lifecycle is an optimized lifecycle tool that lets you focus on the most common lifecycle management needs, without unnecessary hardware-centric concepts like data tiers. +::: +:::: -$$$ilm-gs-alias-bootstrap$$$ +::::{important} +To use {{ilm-init}}, all nodes in a cluster must run the same version. Although it might be possible to create and apply policies in a mixed-version cluster, there is no guarantee they will work as intended. Attempting to use a policy that contains actions that aren’t supported on all nodes in a cluster will cause errors. +:::: -$$$manage-time-series-data-without-data-streams$$$ +## Actions -$$$ilm-gs-create-policy$$$ +{{ilm-init}} policies can trigger actions like: -$$$ilm-gs-apply-policy$$$ +* **Rollover**: Creates a new write index when the current one reaches a certain size, number of docs, or age. +* **Shrink**: Reduces the number of primary shards in an index. +* **Force merge**: Triggers a [force merge](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-forcemerge.html) to reduce the number of segments in an index’s shards. +* **Delete**: Permanently remove an index, including all of its data and metadata. +* [And more](https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-actions.html) -$$$ilm-gs-create-the-data-stream$$$ +Each action has options you can use to specify index behavior and characteristics like: -$$$ilm-gs-check-progress$$$ +* The maximum shard size, number of documents, or age at which you want to roll over to a new index. +* The point at which the index is no longer being updated and the number of primary shards can be reduced. +* When to force a merge to permanently remove documents marked for deletion. +* The point at which the index can be moved to less performant hardware. +* The point at which the availability is not as critical and the number of replicas can be reduced. +* When the index can be safely deleted. -$$$ilm-gs-alias-apply-policy$$$ +For example, if you are indexing metrics data from a fleet of ATMs into Elasticsearch, you might define a policy that says: -$$$ilm-gs-alias-check-progress$$$ +1. When the total size of the index’s primary shards reaches 50GB, roll over to a new index. +2. Move the old index into the warm phase, mark it read only, and shrink it down to a single shard. +3. After 7 days, move the index into the cold phase and move it to less expensive hardware. +4. Delete the index once the required 30 day retention period is reached. + +**Learn about all available actions in [Index lifecycle actions](https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-actions.html).** + +## Create and manage {{ilm-init}} policies + +You can create and manage index lifecycle policies through [{{kib}} Management](/manage-data/lifecycle/index-lifecycle-management/index-management-in-kibana.md) or the [{{ilm-init}} APIs](https://www.elastic.co/docs/api/doc/elasticsearch/v8/group/endpoint-ilm). For more details on creating and managing index lifecycle policies refer to: + +* [Configure a lifecycle policy](/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md) +* [Update a lifecycle policy](/manage-data/lifecycle/index-lifecycle-management/policy-updates.md) +* [Start and stop index lifecycle management](/manage-data/lifecycle/index-lifecycle-management/start-stop-index-lifecycle-management.md) +* [Restore a managed data stream or index](/manage-data/lifecycle/index-lifecycle-management/restore-managed-data-stream-index.md) +* [Customize built-in policies](/manage-data/lifecycle/index-lifecycle-management/tutorial-customize-built-in-policies.md) + +Default {{ilm}} policies are created automatically when you use {{agent}}, {{beats}}, or the {{ls}} {{es}} output plugin to send data to the {{stack}}. + +![index lifecycle policies](../../images/elasticsearch-reference-index-lifecycle-policies.png) + +::::{tip} +To automatically back up your indices and manage snapshots, use [snapshot lifecycle policies](/deploy-manage/tools/snapshot-and-restore/create-snapshots.md#automate-snapshots-slm). +:::: + +## Migrate to {{ilm-init}} + +For existing hot-warm deployments that are currently using index curation, migrating to ILM gives you more fine-grained control over the lifecycle of each index. Read more in: + +* [Manage existing indices](/manage-data/lifecycle/index-lifecycle-management/manage-existing-indices.md) +* [Migrate to index lifecycle management](/manage-data/lifecycle/index-lifecycle-management/migrate-index-management.md) +* [Migrate index allocation filters to node roles](/manage-data/lifecycle/index-lifecycle-management/migrate-index-allocation-filters-to-node-roles.md) diff --git a/manage-data/lifecycle/index-lifecycle-management/concepts.md b/manage-data/lifecycle/index-lifecycle-management/concepts.md deleted file mode 100644 index ec1637001..000000000 --- a/manage-data/lifecycle/index-lifecycle-management/concepts.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -navigation_title: "Concepts" -mapped_pages: - - https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-concepts.html ---- - - - -# Concepts [ilm-concepts] - - -* [Index lifecycle](index-lifecycle.md) -* [Rollover](rollover.md) -* [Policy updates](policy-updates.md) - - - - diff --git a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md index a6ddfec93..773867a7a 100644 --- a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md +++ b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md @@ -1,4 +1,5 @@ --- +navigation_title: Configure a policy mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/set-up-lifecycle-policy.html --- @@ -18,7 +19,6 @@ When you enable {{ilm}} for {{beats}} or the {{ls}} {{es}} output plugin, the ne :::: - ## Create lifecycle policy [ilm-create-policy] To create a lifecycle policy from {{kib}}, open the menu and go to **Stack Management > Index Lifecycle Policies**. Click **Create policy**. diff --git a/raw-migrated-files/cloud/cloud/ec-migrate-index-management.md b/manage-data/lifecycle/index-lifecycle-management/migrate-index-management.md similarity index 70% rename from raw-migrated-files/cloud/cloud/ec-migrate-index-management.md rename to manage-data/lifecycle/index-lifecycle-management/migrate-index-management.md index ea7d82e60..40dca8e06 100644 --- a/raw-migrated-files/cloud/cloud/ec-migrate-index-management.md +++ b/manage-data/lifecycle/index-lifecycle-management/migrate-index-management.md @@ -1,10 +1,16 @@ -# Migrate to index lifecycle management [ec-migrate-index-management] +--- +mapped_pages: + - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-migrate-index-management.html + - https://www.elastic.co/guide/en/cloud/current/ec-configure-index-management.html + - https://www.elastic.co/guide/en/cloud/current/ec-migrate-index-management.html +--- + +# Migrate to index lifecycle management ::::{important} Index curation is deprecated. Any deployments using index curation are prompted to migrate to ILM. :::: - The index lifecycle management (ILM) feature of the {{stack}} provides an integrated and streamlined way to manage time-based data, making it easier to follow best practices for managing your indices. Compared to index curation, migrating to ILM gives you more fine-grained control over the lifecycle of each index. For existing hot-warm deployments that are currently using index curation, there are a couple of options for migrating to index lifecycle management (ILM). You can: @@ -12,13 +18,11 @@ For existing hot-warm deployments that are currently using index curation, there * Use the migration process in the console to change an existing deployment to ILM. * Take a snapshot and restore your data to a new Elastic Stack deployment that has ILM enabled. -To learn more about configuring index lifecycle management for Elasticsearch Service or about all of the features that are available with ILM, see: - -* [Create your index lifecyle policy](https://www.elastic.co/guide/en/elasticsearch/reference/current/set-up-lifecycle-policy.html) -* [Managing the index lifecycle](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html) - To configure ILM Migration in the console: +::::{tab-set} + +:::{tab-item} {{ech}} 1. Log in to the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body). 2. From the **Deployments** page, select your deployment. @@ -27,12 +31,25 @@ To configure ILM Migration in the console: 3. Near the top of the deployment overview, you should get a message to migrate from index curation to index lifecycle management (ILM) along with a **Start migration** button. 4. Select which index curation pattern you wish to migrate. 5. Set the ILM policy name for each data view (formerly *index pattern*). +::: + +:::{tab-item} Elastic Cloud Enterprise +1. [Log into the Cloud UI](../../../deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). +2. From the **Deployments** page, select your deployment. + + On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. + +3. Near the top of the deployment overview, you should get a message to migrate from index curation to index lifecycle management (ILM) along with a **Start migration** button. +4. Select which index curation pattern you wish to migrate. +5. Set the ILM policy name for each data view (formerly *index pattern*). +::: + +:::: ::::{note} Beginning with Elastic Stack version 8.0, Kibana *index patterns* have been renamed to *data views*. To learn more, check the Kibana [What’s new in 8.0](https://www.elastic.co/guide/en/kibana/8.0/whats-new.html#index-pattern-rename) page. :::: - 1. Set the shard allocation attribute for the data view. * You can set different node attributes per data view to allow for more in-depth configuration in Kibana, or diff --git a/manage-data/lifecycle/index-lifecycle-management/policy-updates.md b/manage-data/lifecycle/index-lifecycle-management/policy-updates.md index 8ffb1906b..e19df1846 100644 --- a/manage-data/lifecycle/index-lifecycle-management/policy-updates.md +++ b/manage-data/lifecycle/index-lifecycle-management/policy-updates.md @@ -1,12 +1,10 @@ --- -navigation_title: "Policy updates" +navigation_title: "Update a policy" mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/update-lifecycle-policy.html --- - - -# Policy updates [update-lifecycle-policy] +# Update a lifecycle policy You can change how the lifecycle of an index or collection of rolling indices is managed by modifying the current policy or switching to a different policy. @@ -16,7 +14,7 @@ To ensure that policy updates don’t put an index into a state where it can’t When the index advances to the next phase, it uses the phase definition from the updated policy. -## How changes are applied [ilm-apply-changes] +## How changes are applied [ilm-apply-changes] When a policy is initially applied to an index, the index gets the latest version of the policy. If you update the policy, the policy version is bumped and {{ilm-init}} can detect that the index is using an earlier version that needs to be updated. @@ -25,7 +23,7 @@ Changes to `min_age` are not propagated to the cached definition. Changing a pha For example, if you create a policy that has a hot phase that does not specify a `min_age`, indices immediately enter the hot phase when the policy is applied. If you then update the policy to specify a `min_age` of 1 day for the hot phase, that has no effect on indices that are already in the hot phase. Indices created *after* the policy update won’t enter the hot phase until they are a day old. -## How new policies are applied [ilm-apply-new-policy] +## How new policies are applied [ilm-apply-new-policy] When you apply a different policy to a managed index, the index completes the current phase using the cached definition from the previous policy. The index starts using the new policy when it moves to the next phase. diff --git a/manage-data/lifecycle/index-lifecycle-management/start-stop-index-lifecycle-management.md b/manage-data/lifecycle/index-lifecycle-management/start-stop-index-lifecycle-management.md index 4be9e5815..c8c287706 100644 --- a/manage-data/lifecycle/index-lifecycle-management/start-stop-index-lifecycle-management.md +++ b/manage-data/lifecycle/index-lifecycle-management/start-stop-index-lifecycle-management.md @@ -1,4 +1,5 @@ --- +navigation_title: Start and stop {{ilm-init}} mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/start-stop-ilm.html --- @@ -9,13 +10,13 @@ By default, the {{ilm-init}} service is in the `RUNNING` state and manages all i You can stop {{ilm}} to suspend management operations for all indices. For example, you might stop {{ilm}} when performing scheduled maintenance or making changes to the cluster that could impact the execution of {{ilm-init}} actions. -::::{important} +::::{important} When you stop {{ilm-init}}, [{{slm-init}}](../../../deploy-manage/tools/snapshot-and-restore/create-snapshots.md#automate-snapshots-slm) operations are also suspended. No snapshots will be taken as scheduled until you restart {{ilm-init}}. In-progress snapshots are not affected. :::: -## Get {{ilm-init}} status [get-ilm-status] +## Get {{ilm-init}} status [get-ilm-status] To see the current status of the {{ilm-init}} service, use the [Get Status API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-get-status): @@ -32,7 +33,7 @@ Under normal operation, the response shows {{ilm-init}} is `RUNNING`: ``` -## Stop {{ilm-init}} [stop-ilm] +## Stop {{ilm-init}} [stop-ilm] To stop the {{ilm-init}} service and pause execution of all lifecycle policies, use the [Stop API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-stop): @@ -57,7 +58,7 @@ Once all policies are at a safe stopping point, {{ilm-init}} moves into the `STO ``` -## Start {{ilm-init}} [_start_ilm_init] +## Start {{ilm-init}} [_start_ilm_init] To restart {{ilm-init}} and resume executing policies, use the [Start API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-start). This puts the {{ilm-init}} service in the `RUNNING` state and {{ilm-init}} begins executing policies from where it left off. diff --git a/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md b/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md index 43eef61ab..e5c62b7ed 100644 --- a/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md +++ b/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md @@ -1,29 +1,23 @@ --- -navigation_title: "Tutorial" mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle-management.html --- - - # Tutorial: Automate rollover [getting-started-index-lifecycle-management] - When you continuously index timestamped documents into {{es}}, you typically use a [data stream](../../data-store/index-types/data-streams.md) so you can periodically [roll over](rollover.md) to a new index. This enables you to implement a [hot-warm-cold architecture](../data-tiers.md) to meet your performance requirements for your newest data, control costs over time, enforce retention policies, and still get the most out of your data. ::::{tip} -[Data streams](../../data-store/index-types/data-streams.md) are best suited for [append-only](../../data-store/index-types/data-streams.md#data-streams-append-only) use cases. If you need to update or delete existing time series data, you can perform update or delete operations directly on the data stream backing index. If you frequently send multiple documents using the same `_id` expecting last-write-wins, you may want to use an index alias with a write index instead. You can still use [ILM](../index-lifecycle-management.md) to manage and [roll over](rollover.md) the alias’s indices. Skip to [Manage time series data without data streams](../index-lifecycle-management.md#manage-time-series-data-without-data-streams). +[Data streams](../../data-store/index-types/data-streams.md) are best suited for [append-only](../../data-store/index-types/data-streams.md#data-streams-append-only) use cases. If you need to update or delete existing time series data, you can perform update or delete operations directly on the data stream backing index. If you frequently send multiple documents using the same `_id` expecting last-write-wins, you may want to use an index alias with a write index instead. You can still use [ILM](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md) to manage and [roll over](rollover.md) the alias’s indices. Skip to [Manage time series data without data streams](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-without-data-streams). :::: - - ## Manage time series data with data streams [manage-time-series-data-with-data-streams] To automate rollover and management of a data stream with {{ilm-init}}, you: -1. [Create a lifecycle policy](../index-lifecycle-management.md#ilm-gs-create-policy) that defines the appropriate [phases](index-lifecycle.md) and [actions](https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-actions.html). -2. [Create an index template](../index-lifecycle-management.md#ilm-gs-apply-policy) to [create the data stream](../index-lifecycle-management.md#ilm-gs-create-the-data-stream) and apply the ILM policy and the indices settings and mappings configurations for the backing indices. -3. [Verify indices are moving through the lifecycle phases](../index-lifecycle-management.md#ilm-gs-check-progress) as expected. +1. [Create a lifecycle policy](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#ilm-gs-create-policy) that defines the appropriate [phases](index-lifecycle.md) and [actions](https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-actions.html). +2. [Create an index template](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#ilm-gs-apply-policy) to [create the data stream](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#ilm-gs-create-the-data-stream) and apply the ILM policy and the indices settings and mappings configurations for the backing indices. +3. [Verify indices are moving through the lifecycle phases](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#ilm-gs-check-progress) as expected. ::::{important} When you enable {{ilm}} for {{beats}} or the {{ls}} {{es}} output plugin, lifecycle policies are set up automatically. You do not need to take any other actions. You can modify the default policies through [{{kib}} Management](tutorial-customize-built-in-policies.md) or the {{ilm-init}} APIs. @@ -220,10 +214,10 @@ If you frequently send multiple documents using the same `_id` expecting last-wr To automate rollover and management of time series indices with {{ilm-init}} using an index alias, you: -1. Create a lifecycle policy that defines the appropriate phases and actions. See [Create a lifecycle policy](../index-lifecycle-management.md#ilm-gs-create-policy) above. -2. [Create an index template](../index-lifecycle-management.md#ilm-gs-alias-apply-policy) to apply the policy to each new index. -3. [Bootstrap an index](../index-lifecycle-management.md#ilm-gs-alias-bootstrap) as the initial write index. -4. [Verify indices are moving through the lifecycle phases](../index-lifecycle-management.md#ilm-gs-alias-check-progress) as expected. +1. Create a lifecycle policy that defines the appropriate phases and actions. See [Create a lifecycle policy](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#ilm-gs-create-policy) above. +2. [Create an index template](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#ilm-gs-alias-apply-policy) to apply the policy to each new index. +3. [Bootstrap an index](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#ilm-gs-alias-bootstrap) as the initial write index. +4. [Verify indices are moving through the lifecycle phases](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#ilm-gs-alias-check-progress) as expected. ### Create an index template to apply the lifecycle policy [ilm-gs-alias-apply-policy] @@ -291,7 +285,7 @@ This process repeats each time rollover conditions are met. You can search acros ### Check lifecycle progress [ilm-gs-alias-check-progress] -Retrieving the status information for managed indices is very similar to the data stream case. See the data stream [check progress section](../index-lifecycle-management.md#ilm-gs-check-progress) for more information. The only difference is the indices namespace, so retrieving the progress will entail the following api call: +Retrieving the status information for managed indices is very similar to the data stream case. See the data stream [check progress section](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#ilm-gs-check-progress) for more information. The only difference is the indices namespace, so retrieving the progress will entail the following api call: ```console GET timeseries-*/_ilm/explain diff --git a/manage-data/lifecycle/index-lifecycle-management/tutorial-customize-built-in-policies.md b/manage-data/lifecycle/index-lifecycle-management/tutorial-customize-built-in-policies.md index d17936eff..28aae4233 100644 --- a/manage-data/lifecycle/index-lifecycle-management/tutorial-customize-built-in-policies.md +++ b/manage-data/lifecycle/index-lifecycle-management/tutorial-customize-built-in-policies.md @@ -1,13 +1,9 @@ --- -navigation_title: "Tutorial" mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/example-using-index-lifecycle-policy.html --- - - -# Tutorial: Customize built-in policies [example-using-index-lifecycle-policy] - +# Tutorial: Customize built-in policies {{es}} includes the following built-in {{ilm-init}} policies: diff --git a/manage-data/toc.yml b/manage-data/toc.yml index 2fd971e5d..6d686b598 100644 --- a/manage-data/toc.yml +++ b/manage-data/toc.yml @@ -118,20 +118,20 @@ toc: - file: lifecycle/data-tiers.md - file: lifecycle/index-lifecycle-management.md children: - - file: lifecycle/index-lifecycle-management/tutorial-customize-built-in-policies.md - - file: lifecycle/index-lifecycle-management/tutorial-automate-rollover.md - - file: lifecycle/index-lifecycle-management/index-management-in-kibana.md - - file: lifecycle/index-lifecycle-management/concepts.md + - file: lifecycle/index-lifecycle-management/index-lifecycle.md + - file: lifecycle/index-lifecycle-management/rollover.md children: - - file: lifecycle/index-lifecycle-management/index-lifecycle.md - - file: lifecycle/index-lifecycle-management/rollover.md - - file: lifecycle/index-lifecycle-management/policy-updates.md + - file: lifecycle/index-lifecycle-management/skip-rollover.md + - file: lifecycle/index-lifecycle-management/tutorial-automate-rollover.md - file: lifecycle/index-lifecycle-management/configure-lifecycle-policy.md - - file: lifecycle/index-lifecycle-management/migrate-index-allocation-filters-to-node-roles.md + - file: lifecycle/index-lifecycle-management/policy-updates.md - file: lifecycle/index-lifecycle-management/start-stop-index-lifecycle-management.md - - file: lifecycle/index-lifecycle-management/manage-existing-indices.md - - file: lifecycle/index-lifecycle-management/skip-rollover.md - file: lifecycle/index-lifecycle-management/restore-managed-data-stream-index.md + - file: lifecycle/index-lifecycle-management/tutorial-customize-built-in-policies.md + - file: lifecycle/index-lifecycle-management/index-management-in-kibana.md + - file: lifecycle/index-lifecycle-management/manage-existing-indices.md + - file: lifecycle/index-lifecycle-management/migrate-index-management.md + - file: lifecycle/index-lifecycle-management/migrate-index-allocation-filters-to-node-roles.md - file: lifecycle/data-stream.md children: - file: lifecycle/data-stream/tutorial-create-data-stream-with-lifecycle.md diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-accessing-elastic-services.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-accessing-elastic-services.md deleted file mode 100644 index 280cfd2ae..000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-accessing-elastic-services.md +++ /dev/null @@ -1,19 +0,0 @@ -# Access Elastic Stack services [k8s-accessing-elastic-services] - -To access the Elastic Stack services, you need to: - -* Retrieve the `elastic` user password for basic authentication. -* Specify the IP of the service, if you want to access the service from outside the Kubernetes cluster. -* Decide if you want to use the self-signed certificate generated by ECK, or configure your own certificate. - -The following sections will guide you through this process: - -* [Security](../../../deploy-manage/security/secure-cluster-communications.md) -* [Services](../../../deploy-manage/deploy/cloud-on-k8s/accessing-services.md) -* [TLS certificates](../../../deploy-manage/security/secure-http-communications.md) -* [Access the Elasticsearch endpoint](../../../deploy-manage/deploy/cloud-on-k8s/accessing-services.md) - - - - - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-advanced-topics.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-advanced-topics.md deleted file mode 100644 index 227a31684..000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-advanced-topics.md +++ /dev/null @@ -1,12 +0,0 @@ -# Advanced topics [k8s-advanced-topics] - -* [*Deploy ECK on OpenShift*](../../../deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-openshift.md) -* [*Deploy ECK on GKE Autopilot*](../../../deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md) -* [*Create custom images*](../../../deploy-manage/deploy/cloud-on-k8s/create-custom-images.md) -* [*Service meshes*](../../../deploy-manage/deploy/cloud-on-k8s/service-meshes.md) -* [*Traffic Splitting*](../../../deploy-manage/deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md) -* [*Network policies*](../../../deploy-manage/deploy/cloud-on-k8s/network-policies.md) -* [*Webhook namespace selectors*](../../../deploy-manage/deploy/cloud-on-k8s/webhook-namespace-selectors.md) -* [*Stack Monitoring*](../../../deploy-manage/monitor/stack-monitoring/eck-stack-monitoring.md) -* [*Deploy a FIPS compatible version of ECK*](../../../deploy-manage/deploy/cloud-on-k8s/deploy-fips-compatible-version-of-eck.md) - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-air-gapped.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-air-gapped.md deleted file mode 100644 index da626e0ff..000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-air-gapped.md +++ /dev/null @@ -1,65 +0,0 @@ -# Running in air-gapped environments [k8s-air-gapped] - -The ECK operator can be run in an air-gapped environment without access to the open internet when it is configured not to pull container images from `docker.elastic.co`. - -By default ECK does not require you to specify the container image for each Elastic Stack application you deploy. - -```yaml -apiVersion: elasticsearch.k8s.elastic.co/v1 -kind: Elasticsearch -metadata: - name: quickstart -spec: - version: 8.16.1 - # image: docker.elastic.co/elasticsearch/elasticsearch:8.16.1 <1> - nodeSets: - - name: default - count: 1 - # podTemplate: - # spec: - # imagePullSecrets: <2> - # - name: private-registry-credentials-secret -``` - -1. The ECK operator will set this value by default. You can explicitly set it to your mirrored container image when running in an air-gapped environment -2. You can provide credentials to your private container registry by setting the `imagePullSecrets` field through the `spec.podTemplate` section of your Elastic resource specification, check [how to customize the Elastic resources Pods](../../../deploy-manage/deploy/cloud-on-k8s/customize-pods.md) and [how to setup a Secret containing your registry credentials](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). - - -ECK will automatically set the correct container image for each application. When running in an air-gapped or offline environment you will have to mirror the official Elastic container images in a private container image registry. To make use of your mirrored images you can either set the image for each application explicitly as shown in the preceding example or more conveniently override the default container registry as explained in the next section. - - -## Use a mirrored image of the ECK operator [k8s-use-mirrored-operator-image] - -To deploy the ECK operator in an air-gapped environment, you first have to mirror the operator image itself from `docker.elastic.co` to a private container registry, for example `my.registry`. - -Once the ECK operator image is copied internally, replace the original image name `docker.elastic.co/eck/eck-operator:2.16.1` with the private name of the image, for example `my.registry/eck/eck-operator:2.16.1`, in the [operator manifests](../../../deploy-manage/deploy/cloud-on-k8s/install-using-yaml-manifest-quickstart.md). When using [Helm charts](../../../deploy-manage/deploy/cloud-on-k8s/install-using-helm-chart.md), replace the `image.repository` Helm value with, for example, `my.registry/eck/eck-operator`. - - -## Override the default container registry [k8s-container-registry-override] - -When creating custom resources (Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash), the operator defaults to using container images pulled from the `docker.elastic.co` registry. If you are in an environment where external network access is restricted, you can configure the operator to use a different default container registry by starting the operator with the `--container-registry` command-line flag. Check [*Configure ECK*](../../../deploy-manage/deploy/cloud-on-k8s/configure-eck.md) for more information on how to configure the operator using command-line flags and environment variables. - -The operator expects container images to be located at specific repositories in the default container registry. Make sure that your container images are stored in the right repositories and are tagged correctly with the Stack version number. For example, if your private registry is `my.registry` and you wish to deploy components from Stack version 8.16.1, the following image names should exist: - -* `my.registry/elasticsearch/elasticsearch:8.16.1` -* `my.registry/kibana/kibana:8.16.1` -* `my.registry/apm/apm-server:8.16.1` - - -## Use a global container repository [k8s-container-repository-override] - -If you cannot follow the default Elastic image repositories naming scheme, you can configure the operator to use a different container repository by starting the operator with the `--container-repository` command-line flag. Check [*Configure ECK*](../../../deploy-manage/deploy/cloud-on-k8s/configure-eck.md) for more information on how to configure the operator using command-line flags and environment variables. - -For example, if your private registry is `my.registry` and all Elastic images are located under the `elastic` repository, the following image names should exist: - -* `my.registry/elastic/elasticsearch:8.16.1` -* `my.registry/elastic/kibana:8.16.1` -* `my.registry/elastic/apm-server:8.16.1` - - -## ECK Diagnostics in air-gapped environments [k8s-eck-diag-air-gapped] - -The [eck-diagnostics tool](../../../troubleshoot/deployments/cloud-on-k8s/run-eck-diagnostics.md) optionally runs diagnostics for Elastic Stack applications in a separate container that is deployed into the Kubernetes cluster. - -In air-gapped environments with no access to the `docker.elastic.co` registry, you should copy the latest support-diagnostics container image to your internal image registry and then run the tool with the additional flag `--diagnostic-image `. To find out which support diagnostics container image matches your version of eck-diagnostics run the tool once without arguments and it will print the default image in use. - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-deploy-eck.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-deploy-eck.md deleted file mode 100644 index fa2966fd2..000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-deploy-eck.md +++ /dev/null @@ -1,57 +0,0 @@ -# Deploy ECK in your Kubernetes cluster [k8s-deploy-eck] - -Things to consider before you start: - -* For this quickstart guide, your Kubernetes cluster is assumed to be already up and running. Before you proceed with the ECK installation, make sure you check the [supported versions](../../../deploy-manage/deploy/cloud-on-k8s.md). -* If you are using GKE, make sure your user has `cluster-admin` permissions. For more information, check [Prerequisites for using Kubernetes RBAC on GKE](https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control#iam-rolebinding-bootstrap). -* If you are using Amazon EKS, make sure the Kubernetes control plane is allowed to communicate with the Kubernetes nodes on port 443. This is required for communication with the Validating Webhook. For more information, check [Recommended inbound traffic](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.md). -* Refer to [*Install ECK*](../../../deploy-manage/deploy/cloud-on-k8s/install.md) for more information on installation options. -* Check the [upgrade notes](../../../deploy-manage/upgrade/orchestrator/upgrade-cloud-on-k8s.md) if you are attempting to upgrade an existing ECK deployment. - -To deploy the ECK operator: - -1. Install [custom resource definitions](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) with [`create`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_create/): - - ```sh - kubectl create -f https://download.elastic.co/downloads/eck/2.16.1/crds.yaml - ``` - - This will output similar to the following upon Elastic resources' creation: - - ```sh - customresourcedefinition.apiextensions.k8s.io/agents.agent.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/apmservers.apm.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/beats.beat.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/elasticmapsservers.maps.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/elasticsearches.elasticsearch.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/enterprisesearches.enterprisesearch.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/kibanas.kibana.k8s.elastic.co created - customresourcedefinition.apiextensions.k8s.io/logstashes.logstash.k8s.elastic.co created - ``` - -2. Install the operator with its RBAC rules with [`apply`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_apply/): - - ```sh - kubectl apply -f https://download.elastic.co/downloads/eck/2.16.1/operator.yaml - ``` - - ::::{note} - The ECK operator runs by default in the `elastic-system` namespace. It is recommended that you choose a dedicated namespace for your workloads, rather than using the `elastic-system` or the `default` namespace. - :::: - -3. Monitor the operator’s setup from its logs through [`logs`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_logs/): - - ```sh - kubectl -n elastic-system logs -f statefulset.apps/elastic-operator - ``` - -4. Once ready, the operator will report as `Running` as shown with [`get`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_get/), replacing default `elastic-system` with applicable installation namespace as needed: * - -``` -$ kubectl get -n elastic-system pods -NAME READY STATUS RESTARTS AGE -elastic-operator-0 1/1 Running 0 1m -``` - -This completes the quickstart of the ECK operator. We recommend continuing to [Deploying an {{es}} cluster](../../../deploy-manage/deploy/cloud-on-k8s/elasticsearch-deployment-quickstart.md); but for more configuration options as needed, navigate to [Operating ECK](../../../deploy-manage/deploy/cloud-on-k8s/configure.md). - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-fips.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-fips.md deleted file mode 100644 index c861994ce..000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-fips.md +++ /dev/null @@ -1,10 +0,0 @@ -# Deploy a FIPS compatible version of ECK [k8s-fips] - -The Federal Information Processing Standard (FIPS) Publication 140-2, (FIPS PUB 140-2), titled "Security Requirements for Cryptographic Modules" is a U.S. government computer security standard used to approve cryptographic modules. Since version 2.6 ECK offers a FIPS-enabled image that is a drop-in replacement for the standard image. - -For the ECK operator, adherence to FIPS 140-2 is ensured by: - -* Using FIPS approved / NIST recommended cryptographic algorithms. -* Compiling the operator using the [BoringCrypto](https://github.com/golang/go/blob/dev.boringcrypto/README.boringcrypto.md) library for various cryptographic primitives. - - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-install-yaml-manifests.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-install-yaml-manifests.md deleted file mode 100644 index 89840c378..000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-install-yaml-manifests.md +++ /dev/null @@ -1,10 +0,0 @@ -# Install ECK using the YAML manifests [k8s-install-yaml-manifests] - -This method is the quickest way to get started with ECK if you have full administrative access to the Kubernetes cluster. The [Quickstart](../../../deploy-manage/deploy/cloud-on-k8s/deploy-an-orchestrator.md) document describes how to proceed with this method. When you run the `kubectl` command listed in [*Deploy ECK in your Kubernetes cluster*](../../../deploy-manage/deploy/cloud-on-k8s/install-using-yaml-manifest-quickstart.md), the following components are installed or updated: - -* `CustomResourceDefinition` objects for all supported resource types (Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash). -* `Namespace` named `elastic-system` to hold all operator resources. -* `ServiceAccount`, `ClusterRole` and `ClusterRoleBinding` to allow the operator to manage resources throughout the cluster. -* `ValidatingWebhookConfiguration` to validate Elastic custom resources on admission. -* `StatefulSet`, `ConfigMap`, `Secret` and `Service` in `elastic-system` namespace to run the operator application. - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-overview.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-overview.md deleted file mode 100644 index 69d1ad6b4..000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-overview.md +++ /dev/null @@ -1,12 +0,0 @@ -# Overview [k8s-overview] - -Built on the Kubernetes Operator pattern, Elastic Cloud on Kubernetes (ECK) extends the basic Kubernetes orchestration capabilities to support the setup and management of Elasticsearch, Kibana, APM Server, Beats, Elastic Agent, Elastic Maps Server, and Logstash on Kubernetes. - -With Elastic Cloud on Kubernetes you can streamline critical operations, such as: - -1. Managing and monitoring multiple clusters -2. Scaling cluster capacity and storage -3. Performing safe configuration changes through rolling upgrades -4. Securing clusters with TLS certificates -5. Setting up hot-warm-cold architectures with availability zone awareness - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-request-elasticsearch-endpoint.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-request-elasticsearch-endpoint.md deleted file mode 100644 index c73311711..000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-request-elasticsearch-endpoint.md +++ /dev/null @@ -1,34 +0,0 @@ -# Access the Elasticsearch endpoint [k8s-request-elasticsearch-endpoint] - -You can access the Elasticsearch endpoint within or outside the Kubernetes cluster. - -**Within the Kubernetes cluster** - -1. Retrieve the CA certificate. -2. Retrieve the password of the `elastic` user. - -```sh -NAME=hulk - -kubectl get secret "$NAME-es-http-certs-public" -o go-template='{{index .data "tls.crt" | base64decode }}' > tls.crt -PW=$(kubectl get secret "$NAME-es-elastic-user" -o go-template='{{.data.elastic | base64decode }}') - -curl --cacert tls.crt -u elastic:$PW https://$NAME-es-http:9200/ -``` - -**Outside the Kubernetes cluster** - -1. Retrieve the CA certificate. -2. Retrieve the password of the `elastic` user. -3. Retrieve the IP of the `LoadBalancer` `Service`. - -```sh -NAME=hulk - -kubectl get secret "$NAME-es-http-certs-public" -o go-template='{{index .data "tls.crt" | base64decode }}' > tls.crt -IP=$(kubectl get svc "$NAME-es-http" -o jsonpath='{.status.loadBalancer.ingress[].ip}') -PW=$(kubectl get secret "$NAME-es-elastic-user" -o go-template='{{.data.elastic | base64decode }}') - -curl --cacert tls.crt -u elastic:$PW https://$IP:9200/ -``` - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-security.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-security.md deleted file mode 100644 index 068c54755..000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-security.md +++ /dev/null @@ -1,19 +0,0 @@ -# Security [k8s-security] - -All Elastic Stack resources deployed by the ECK operator are secured by default. The operator sets up basic authentication and TLS to encrypt network traffic to, from, and within your Elasticsearch cluster. - -## Authentication [k8s-authentication] - -To access Elastic resources, the operator manages a default user named `elastic` with the `superuser` role. Its password is stored in a `Secret` named `-elastic-user`. - -```sh -> kubectl get secret hulk-es-elastic-user -o go-template='{{.data.elastic | base64decode }}' -42xyz42citsale42xyz42 -``` - -::::{note} -Beware of copying this Secret as-is into a different namespace. Check [Common Problems: Owner References](../../../troubleshoot/deployments/cloud-on-k8s/common-problems.md#k8s-common-problems-owner-refs) for more information. -:::: - - - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-services.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-services.md deleted file mode 100644 index f4a1791f2..000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-services.md +++ /dev/null @@ -1,50 +0,0 @@ -# Services [k8s-services] - -You can access Elastic resources by using native Kubernetes services that are not reachable from the public Internet by default. - -## Manage Kubernetes services [k8s-kubernetes-service] - -For each resource, the operator manages a Kubernetes service named `-[es|kb|apm|ent|agent]-http`, which is of type `ClusterIP` by default. `ClusterIP` exposes the service on a cluster-internal IP and makes the service only reachable from the cluster. - -```sh -> kubectl get svc - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -hulk-apm-http ClusterIP 10.19.212.105 8200/TCP 1m -hulk-es-http ClusterIP 10.19.252.160 9200/TCP 1m -hulk-kb-http ClusterIP 10.19.247.151 5601/TCP 1m -``` - - -## Allow public access [k8s-allow-public-access] - -You can expose services in [different ways](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types) by specifying an `http.service.spec.type` in the `spec` of the resource manifest. On cloud providers which support external load balancers, you can set the `type` field to `LoadBalancer` to provision a load balancer for the `Service`, and populate the column `EXTERNAL-IP` after a short delay. Depending on the cloud provider, it may incur costs. - -By default, the Elasticsearch service created by ECK is configured to route traffic to all Elasticsearch nodes in the cluster. Depending on your cluster configuration, you may want more control over the set of nodes that handle different types of traffic (query, ingest, and so on). Check [*Traffic Splitting*](../../../deploy-manage/deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes.md) for more information. - -::::{warning} -When you change the `clusterIP` setting of the service, ECK will delete and re-create the service as `clusterIP` is an immutable field. Depending on your client implementation, this might result in a short disruption until the service DNS entries refresh to point to the new endpoints. -:::: - - -```yaml -apiVersion: .k8s.elastic.co/v1 -kind: -metadata: - name: hulk -spec: - version: 8.16.1 - http: - service: - spec: - type: LoadBalancer -``` - -```sh -> kubectl get svc - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -hulk-apm-http LoadBalancer 10.19.212.105 35.176.227.106 8200:31000/TCP 1m -hulk-es-http LoadBalancer 10.19.252.160 35.198.131.115 9200:31320/TCP 1m -hulk-kb-http LoadBalancer 10.19.247.151 35.242.197.228 5601:31380/TCP 1m -``` diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-supported.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-supported.md deleted file mode 100644 index 58850c32e..000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s-supported.md +++ /dev/null @@ -1,24 +0,0 @@ -# Supported versions [k8s-supported] - -ECK is compatible with: - -* Kubernetes 1.28-1.32 -* OpenShift 4.12-4.17 -* Google Kubernetes Engine (GKE), Azure Kubernetes Service (AKS), and Amazon Elastic Kubernetes Service (EKS) -* Helm: 3.2.0+ -* Elasticsearch, Kibana, APM Server: 6.8+, 7.1+, 8+ -* Enterprise Search: 7.7+, 8.x (Enterprise Search is not available in {{stack}} 9.0+) - -* Beats: 7.0+, 8+ -* Elastic Agent: 7.10+ (standalone), 7.14+ (Fleet), 8+ -* Elastic Maps Server: 7.11+, 8+ -* Logstash: 8.7+ - -ECK should work with all conformant installers as listed in these [FAQs](https://github.com/cncf/k8s-conformance/blob/master/faq.md#what-is-a-distribution-hosted-platform-and-an-installer). Distributions include source patches and so may not work as-is with ECK. - -Alpha, beta, and stable API versions follow the same [conventions used by Kubernetes](https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-versioning). - -Elastic Stack application images for the OpenShift-certified Elasticsearch (ECK) Operator are only available from version 7.10 and later. - -Check the full [Elastic support matrix](https://www.elastic.co/support/matrix#matrix_kubernetes) for more information. - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_installation.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_installation.md deleted file mode 100644 index 36ed69985..000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_installation.md +++ /dev/null @@ -1,33 +0,0 @@ -# Installation [k8s_installation] - -## FIPS compliant installation using Helm [k8s_fips_compliant_installation_using_helm] - -Set `image.fips=true` to install a FIPS-enabled version of the ECK Operator. Refer to [Install ECK using the Helm chart](../../../deploy-manage/deploy/cloud-on-k8s/install-using-helm-chart.md) for full Helm installation instructions. - -```sh -helm install elastic-operator elastic/eck-operator \ - -n elastic-system --create-namespace \ - --set=image.fips=true -``` - - -## FIPS compliant installation using manifests [k8s_fips_compliant_installation_using_manifests] - -The `StatefulSet` definition within the yaml installation manifest will need to be patched prior to installation to append `-fips` to the `spec.template.spec.containers[*].image` to install a FIPS-enabled version of the ECK Operator. Refer to [Install ECK using the YAML manifests](../../../deploy-manage/deploy/cloud-on-k8s/install-using-yaml-manifest-quickstart.md) for full manifest installation instructions. - -::::{note} -`${ECK_VERSION}` in the following command needs to be replaced with the version of the Operator that is to be installed. -:::: - - -```sh -curl -s https://download.elastic.co/downloads/eck/${ECK_VERSION}/operator.yaml | sed -r 's#(image:.*eck-operator)(:.*)#\1-fips\2#' | kubectl apply -f - -``` - -If the Operator has already been installed using the manifests, the installation can be patched instead: - -```sh -kubectl patch sts elastic-operator -n elastic-system -p '{"spec":{"template":{"spec":{"containers":[{"name":"manager", "image":"docker.elastic.co/eck/eck-operator-fips:${ECK_VERSION}"}]}}}}' -``` - - diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_learn_more_about_eck.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_learn_more_about_eck.md deleted file mode 100644 index 0501980ca..000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_learn_more_about_eck.md +++ /dev/null @@ -1,17 +0,0 @@ -# Learn more about ECK [k8s_learn_more_about_eck] - -* [Orchestrate Elasticsearch on Kubernetes](https://www.elastic.co/elasticsearch-kubernetes) -* [ECK post on the Elastic Blog](https://www.elastic.co/blog/introducing-elastic-cloud-on-kubernetes-the-elasticsearch-operator-and-beyond?elektra=products&storm=sub1) -* [Getting Started With Elastic Cloud on Kubernetes (ECK)](https://www.youtube.com/watch?v=PIJmlYBIFXM) -* [Running the Elastic Stack on Kubernetes with ECK](https://www.youtube.com/watch?v=Wf6E3vkvEFM) - - -## Ask for help [k8s-ask-for-help] - -If you are an existing Elastic customer with an active support contract, you can create a case in the [Elastic Support Portal](https://support.elastic.co/). Kindly attach an [ECK diagnostic](../../../troubleshoot/deployments/cloud-on-k8s/run-eck-diagnostics.md) when opening your case. - -Alternatively, or if you do not have a support contract, and if you are unable to find a solution to your problem with the information provided in these documents, ask for help: - -* [ECK Discuss forums](https://discuss.elastic.co/c/eck) to ask any question -* [Github issues](https://github.com/elastic/cloud-on-k8s/issues) for bugs and feature requests - diff --git a/raw-migrated-files/cloud/cloud-enterprise/ece-migrate-index-management.md b/raw-migrated-files/cloud/cloud-enterprise/ece-migrate-index-management.md deleted file mode 100644 index 78b67d0ce..000000000 --- a/raw-migrated-files/cloud/cloud-enterprise/ece-migrate-index-management.md +++ /dev/null @@ -1,46 +0,0 @@ -# Migrate to index lifecycle management [ece-migrate-index-management] - -::::{important} -Index curation is deprecated. Any deployments using index curation are prompted to migrate to ILM. -:::: - - -The index lifecycle management (ILM) feature of the {{stack}} provides an integrated and streamlined way to manage time-based data, making it easier to follow best practices for managing your indices. Compared to index curation, migrating to ILM gives you more fine-grained control over the lifecycle of each index. - -For existing hot-warm deployments that are currently using index curation, there are a couple of options for migrating to index lifecycle management (ILM). You can: - -* Use the migration process in the console to change an existing deployment to ILM. -* Take a snapshot and restore your data to a new Elastic Stack deployment that has ILM enabled. - -To learn more about configuring index lifecycle management for Elastic Cloud Enterprise or about all of the features that are available with ILM, see: - -* [Create your index lifecyle policy](https://www.elastic.co/guide/en/elasticsearch/reference/current/set-up-lifecycle-policy.html) -* [Managing the index lifecycle](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html) - -To configure ILM Migration in the console: - -1. [Log into the Cloud UI](../../../deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). -2. From the **Deployments** page, select your deployment. - - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. - -3. Near the top of the deployment overview, you should get a message to migrate from index curation to index lifecycle management (ILM) along with a **Start migration** button. -4. Select which index curation pattern you wish to migrate. -5. Set the ILM policy name for each data view (formerly *index pattern*). - -::::{note} -Beginning with Elastic Stack version 8.0, Kibana *index patterns* have been renamed to *data views*. To learn more, check the Kibana [What’s new in 8.0](https://www.elastic.co/guide/en/kibana/8.0/whats-new.html#index-pattern-rename) page. -:::: - - -1. Set the shard allocation attribute for the data view. - - * You can set different node attributes per data view to allow for more in-depth configuration in Kibana, or - * You may choose to add one node attribute that applies to all data views. - * If you do not wish to migrate a certain data view to ILM, you can deselect the checkbox in the associated row. - * You may also wish to migrate to ILM without carrying over any of your current data views by deselecting all patterns. This means that those data views will no longer be curated, and you will have the option to set up new ILM policies in Kibana. - -2. Select **Migrate**. - -After you get the notification that confirms that migration was completed successfully, you can view your ILM policies in Kibana. - diff --git a/raw-migrated-files/cloud/cloud/ec-configure-index-management.md b/raw-migrated-files/cloud/cloud/ec-configure-index-management.md deleted file mode 100644 index e0eb7d4aa..000000000 --- a/raw-migrated-files/cloud/cloud/ec-configure-index-management.md +++ /dev/null @@ -1,17 +0,0 @@ -# Configure index management [ec-configure-index-management] - -::::{important} -Index curation is deprecated. Any deployments using index curation will be prompted to migrate to ILM. -:::: - - -The index lifecycle management (ILM) feature of the Elastic Stack provides an integrated and streamlined way to manage time-based data, making it easier to follow best practices for managing your indices. Compared to index curation, migrating to ILM gives you more fine-grained control over the lifecycle of each index. - -For existing hot-warm deployments that are currently using index curation, there are a couple of options for migrating to index lifecycle management (ILM). You can: - -* Use the migration process in the console to change an existing deployment to ILM. -* Take a snapshot and restore your data to a new Elastic Stack 6.7+ deployment that has ILM enabled. -* [Create your index lifecyle policy](https://www.elastic.co/guide/en/elasticsearch/reference/current/set-up-lifecycle-policy.html) -* [Managing the index lifecycle](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html) - - diff --git a/raw-migrated-files/elasticsearch/elasticsearch-reference/data-streams.md b/raw-migrated-files/elasticsearch/elasticsearch-reference/data-streams.md index befa980ff..f648c339f 100644 --- a/raw-migrated-files/elasticsearch/elasticsearch-reference/data-streams.md +++ b/raw-migrated-files/elasticsearch/elasticsearch-reference/data-streams.md @@ -18,7 +18,7 @@ To determine whether you should use a data stream for your data, you should cons * You mostly perform indexing requests, with occasional updates and deletes. * You index documents without an `_id`, or when indexing documents with an explicit `_id` you expect first-write-wins behavior. -For most time series data use-cases, a data stream will be a good fit. However, if you find that your data doesn’t fit into these categories (for example, if you frequently send multiple documents using the same `_id` expecting last-write-wins), you may want to use an index alias with a write index instead. See documentation for [managing time series data without a data stream](../../../manage-data/lifecycle/index-lifecycle-management.md#manage-time-series-data-without-data-streams) for more information. +For most time series data use-cases, a data stream will be a good fit. However, if you find that your data doesn’t fit into these categories (for example, if you frequently send multiple documents using the same `_id` expecting last-write-wins), you may want to use an index alias with a write index instead. See documentation for [managing time series data without a data stream](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-without-data-streams) for more information. Keep in mind that some features such as [Time Series Data Streams (TSDS)](../../../manage-data/data-store/index-types/tsdb.md) and [data stream lifecycles](../../../manage-data/lifecycle/data-stream.md) require a data stream. @@ -98,7 +98,7 @@ Data streams are designed for use cases where existing data is rarely updated. Y If you need to update a larger number of documents in a data stream, you can use the [update by query](../../../manage-data/data-store/index-types/use-data-stream.md#update-docs-in-a-data-stream-by-query) and [delete by query](../../../manage-data/data-store/index-types/use-data-stream.md#delete-docs-in-a-data-stream-by-query) APIs. ::::{tip} -If you frequently send multiple documents using the same `_id` expecting last-write-wins, you may want to use an index alias with a write index instead. See [Manage time series data without data streams](../../../manage-data/lifecycle/index-lifecycle-management.md#manage-time-series-data-without-data-streams). +If you frequently send multiple documents using the same `_id` expecting last-write-wins, you may want to use an index alias with a write index instead. See [Manage time series data without data streams](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-without-data-streams). :::: diff --git a/raw-migrated-files/elasticsearch/elasticsearch-reference/index-lifecycle-management.md b/raw-migrated-files/elasticsearch/elasticsearch-reference/index-lifecycle-management.md deleted file mode 100644 index 066e1c305..000000000 --- a/raw-migrated-files/elasticsearch/elasticsearch-reference/index-lifecycle-management.md +++ /dev/null @@ -1,31 +0,0 @@ -# {{ilm-init}}: Manage the index lifecycle [index-lifecycle-management] - -You can configure {{ilm}} ({{ilm-init}}) policies to automatically manage indices according to your performance, resiliency, and retention requirements. For example, you could use {{ilm-init}} to: - -* Spin up a new index when an index reaches a certain size or number of documents -* Create a new index each day, week, or month and archive previous ones -* Delete stale indices to enforce data retention standards - -You can create and manage index lifecycle policies through {{kib}} Management or the {{ilm-init}} APIs. Default {{ilm}} policies are created automatically when you use {{agent}}, {{beats}}, or the {{ls}} {{es}} output plugin to send data to the {{stack}}. - -![index lifecycle policies](../../../images/elasticsearch-reference-index-lifecycle-policies.png "") - -::::{tip} -To automatically back up your indices and manage snapshots, use [snapshot lifecycle policies](../../../deploy-manage/tools/snapshot-and-restore/create-snapshots.md#automate-snapshots-slm). -:::: - - -* [Tutorial: Customize built-in policies](../../../manage-data/lifecycle/index-lifecycle-management/tutorial-customize-built-in-policies.md) -* [Tutorial: Automate rollover](../../../manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md) -* [Overview](../../../manage-data/lifecycle/index-lifecycle-management.md) -* [Concepts](../../../manage-data/lifecycle/index-lifecycle-management/concepts.md) -* [*Configure a lifecycle policy*](../../../manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md) -* [*Migrate index allocation filters to node roles*](../../../manage-data/lifecycle/index-lifecycle-management/migrate-index-allocation-filters-to-node-roles.md) -* [*Troubleshooting {{ilm}} errors*](../../../troubleshoot/elasticsearch/elasticsearch-reference/index-lifecycle-management-errors.md) -* [*Start and stop {{ilm}}*](../../../manage-data/lifecycle/index-lifecycle-management/start-stop-index-lifecycle-management.md) -* [*Manage existing indices*](../../../manage-data/lifecycle/index-lifecycle-management/manage-existing-indices.md) -* [*Skip rollover*](../../../manage-data/lifecycle/index-lifecycle-management/skip-rollover.md) -* [*Restore a managed data stream or index*](../../../manage-data/lifecycle/index-lifecycle-management/restore-managed-data-stream-index.md) -* [*{{ilm-cap}} APIs*](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-ilm) -* [*Index lifecycle actions*](https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-actions.html) - diff --git a/raw-migrated-files/elasticsearch/elasticsearch-reference/overview-index-lifecycle-management.md b/raw-migrated-files/elasticsearch/elasticsearch-reference/overview-index-lifecycle-management.md deleted file mode 100644 index 8ca368ac6..000000000 --- a/raw-migrated-files/elasticsearch/elasticsearch-reference/overview-index-lifecycle-management.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -navigation_title: "Overview" ---- - -# {{ilm-init}} overview [overview-index-lifecycle-management] - - -You can create and apply {{ilm-cap}} ({{ilm-init}}) policies to automatically manage your indices according to your performance, resiliency, and retention requirements. - -Index lifecycle policies can trigger actions such as: - -* **Rollover**: Creates a new write index when the current one reaches a certain size, number of docs, or age. -* **Shrink**: Reduces the number of primary shards in an index. -* **Force merge**: Triggers a [force merge](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-forcemerge) to reduce the number of segments in an index’s shards. -* **Delete**: Permanently remove an index, including all of its data and metadata. - -{{ilm-init}} makes it easier to manage indices in hot-warm-cold architectures, which are common when you’re working with time series data such as logs and metrics. - -You can specify: - -* The maximum shard size, number of documents, or age at which you want to roll over to a new index. -* The point at which the index is no longer being updated and the number of primary shards can be reduced. -* When to force a merge to permanently remove documents marked for deletion. -* The point at which the index can be moved to less performant hardware. -* The point at which the availability is not as critical and the number of replicas can be reduced. -* When the index can be safely deleted. - -For example, if you are indexing metrics data from a fleet of ATMs into Elasticsearch, you might define a policy that says: - -1. When the total size of the index’s primary shards reaches 50GB, roll over to a new index. -2. Move the old index into the warm phase, mark it read only, and shrink it down to a single shard. -3. After 7 days, move the index into the cold phase and move it to less expensive hardware. -4. Delete the index once the required 30 day retention period is reached. - -::::{important} -To use {{ilm-init}}, all nodes in a cluster must run the same version. Although it might be possible to create and apply policies in a mixed-version cluster, there is no guarantee they will work as intended. Attempting to use a policy that contains actions that aren’t supported on all nodes in a cluster will cause errors. - -:::: diff --git a/raw-migrated-files/toc.yml b/raw-migrated-files/toc.yml index 1685bfbe5..7004b43b8 100644 --- a/raw-migrated-files/toc.yml +++ b/raw-migrated-files/toc.yml @@ -14,26 +14,13 @@ toc: - file: cloud-on-k8s/cloud-on-k8s/index.md children: - file: cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md - - file: cloud-on-k8s/cloud-on-k8s/k8s_installation.md - - file: cloud-on-k8s/cloud-on-k8s/k8s_learn_more_about_eck.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-accessing-elastic-services.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-advanced-topics.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-air-gapped.md - file: cloud-on-k8s/cloud-on-k8s/k8s-custom-http-certificate.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-deploy-eck.md - file: cloud-on-k8s/cloud-on-k8s/k8s-es-secure-settings.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-fips.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-install-yaml-manifests.md - file: cloud-on-k8s/cloud-on-k8s/k8s-installing-eck.md - file: cloud-on-k8s/cloud-on-k8s/k8s-orchestration.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-overview.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-request-elasticsearch-endpoint.md - file: cloud-on-k8s/cloud-on-k8s/k8s-rotate-credentials.md - file: cloud-on-k8s/cloud-on-k8s/k8s-saml-authentication.md - file: cloud-on-k8s/cloud-on-k8s/k8s-securing-stack.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-security.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-services.md - - file: cloud-on-k8s/cloud-on-k8s/k8s-supported.md - file: cloud-on-k8s/cloud-on-k8s/k8s-tls-certificates.md - file: cloud-on-k8s/cloud-on-k8s/k8s-upgrading-stack.md - file: cloud-on-k8s/cloud-on-k8s/k8s-users-and-roles.md @@ -71,7 +58,6 @@ toc: - file: cloud/cloud-enterprise/ece-manage-integrations-server.md - file: cloud/cloud-enterprise/ece-manage-kibana-settings.md - file: cloud/cloud-enterprise/ece-manage-kibana.md - - file: cloud/cloud-enterprise/ece-migrate-index-management.md - file: cloud/cloud-enterprise/ece-monitoring-deployments.md - file: cloud/cloud-enterprise/ece-password-reset-elastic.md - file: cloud/cloud-enterprise/ece-restful-api-examples-configuring-keystore.md @@ -154,7 +140,6 @@ toc: - file: cloud/cloud/ec-autoscaling.md - file: cloud/cloud/ec-billing-stop.md - file: cloud/cloud/ec-cloud-ingest-data.md - - file: cloud/cloud/ec-configure-index-management.md - file: cloud/cloud/ec-configuring-keystore.md - file: cloud/cloud/ec-custom-bundles.md - file: cloud/cloud/ec-custom-repository.md @@ -182,7 +167,6 @@ toc: - file: cloud/cloud/ec-manage-enterprise-search-settings.md - file: cloud/cloud/ec-manage-kibana-settings.md - file: cloud/cloud/ec-metrics-memory-pressure.md - - file: cloud/cloud/ec-migrate-index-management.md - file: cloud/cloud/ec-monitoring-setup.md - file: cloud/cloud/ec-monitoring.md - file: cloud/cloud/ec-password-reset.md @@ -505,7 +489,6 @@ toc: - file: elasticsearch/elasticsearch-reference/fips-140-compliance.md - file: elasticsearch/elasticsearch-reference/how-monitoring-works.md - file: elasticsearch/elasticsearch-reference/ignore_missing_component_templates.md - - file: elasticsearch/elasticsearch-reference/index-lifecycle-management.md - file: elasticsearch/elasticsearch-reference/index-mgmt.md - file: elasticsearch/elasticsearch-reference/index-modules-allocation.md - file: elasticsearch/elasticsearch-reference/index-modules-mapper.md @@ -522,7 +505,6 @@ toc: - file: elasticsearch/elasticsearch-reference/native-realm.md - file: elasticsearch/elasticsearch-reference/oidc-guide.md - file: elasticsearch/elasticsearch-reference/oidc-realm.md - - file: elasticsearch/elasticsearch-reference/overview-index-lifecycle-management.md - file: elasticsearch/elasticsearch-reference/recovery-prioritization.md - file: elasticsearch/elasticsearch-reference/role-mapping-resources.md - file: elasticsearch/elasticsearch-reference/saml-guide-stack.md diff --git a/troubleshoot/elasticsearch/elasticsearch-reference/index-lifecycle-management-errors.md b/troubleshoot/elasticsearch/elasticsearch-reference/index-lifecycle-management-errors.md index 6f5aa8a43..116e91727 100644 --- a/troubleshoot/elasticsearch/elasticsearch-reference/index-lifecycle-management-errors.md +++ b/troubleshoot/elasticsearch/elasticsearch-reference/index-lifecycle-management-errors.md @@ -161,7 +161,7 @@ Problems with rollover aliases are a common cause of errors. Consider using [dat ### Rollover alias [x] can point to multiple indices, found duplicated alias [x] in index template [z] [_rollover_alias_x_can_point_to_multiple_indices_found_duplicated_alias_x_in_index_template_z] -The target rollover alias is specified in an index template’s `index.lifecycle.rollover_alias` setting. You need to explicitly configure this alias *one time* when you [bootstrap the initial index](../../../manage-data/lifecycle/index-lifecycle-management.md#ilm-gs-alias-bootstrap). The rollover action then manages setting and updating the alias to [roll over](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover#rollover-index-api-desc) to each subsequent index. +The target rollover alias is specified in an index template’s `index.lifecycle.rollover_alias` setting. You need to explicitly configure this alias *one time* when you [bootstrap the initial index](/manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md#ilm-gs-alias-bootstrap). The rollover action then manages setting and updating the alias to [roll over](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover#rollover-index-api-desc) to each subsequent index. Do not explicitly configure this same alias in the aliases section of an index template.