From da96a4fb0bf21d6eb43734b4fe8b6eecb9e39309 Mon Sep 17 00:00:00 2001 From: Miles Garnsey <11435896+Miles-Garnsey@users.noreply.github.com> Date: Tue, 28 May 2024 17:23:36 +1000 Subject: [PATCH] Docs for data plane registration (#1332) --- README.md | 2 +- docs/content/en/install/local/_index.md | 6 +-- .../local/multi-cluster-helm/_index.md | 48 ++----------------- .../local/multi-cluster-kustomize/_index.md | 19 ++------ .../multi-cluster/_index.md | 38 ++------------- .../tasks/data-plane-registration/_index.md | 27 +++++++++++ 6 files changed, 42 insertions(+), 98 deletions(-) rename docs/content/en/{tasks => reference}/multi-cluster/_index.md (80%) create mode 100644 docs/content/en/tasks/data-plane-registration/_index.md diff --git a/README.md b/README.md index af7ff9a2f..257f3b406 100644 --- a/README.md +++ b/README.md @@ -93,7 +93,7 @@ This section needs to be moved elsewhere, probably a dedicated page of its own. ## Connecting to remote clusters The control plane needs to establish client connections to remote cluster where the data plane runs. Credentials are provided via a [kubeconfig](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/) file that is stored in a Secret. That secret is then referenced via a `ClientConfig` custom resource. -A kubeconfig entry for a cluster hosted by a cloud provider with include an auth token for authenticated with the cloud provider. That token expires. If you use one of these kubeconfigs be aware that the operator will not be able to access the remote cluster once that token expires. For this reason it is recommended that you use the [create-clientconfig.sh](https://github.com/k8ssandra/k8ssandra-operator/blob/main/scripts/create-clientconfig.sh) script for configuring a connection to the remote cluster. This script is discussed in more detail in a later section. +A kubeconfig entry for a cluster hosted by a cloud provider with include an auth token for authenticated with the cloud provider. That token expires. If you use one of these kubeconfigs be aware that the operator will not be able to access the remote cluster once that token expires. For this reason it is recommended that you use the [k8ssandra-client](https://github.com/k8ssandra/k8ssandra-client) for configuring a connection to the remote cluster. --> ## Installing the operator diff --git a/docs/content/en/install/local/_index.md b/docs/content/en/install/local/_index.md index 2f7dc002f..dfa97ee73 100644 --- a/docs/content/en/install/local/_index.md +++ b/docs/content/en/install/local/_index.md @@ -30,7 +30,7 @@ Make sure you have the following installed before going through the related inst * [kubectl](https://kubernetes.io/docs/tasks/tools/) and [helm v3+](https://helm.sh/docs/intro/install/) on your preferred OS. -Also, note that the [setup-kind-multicluster.sh](#setup-kind-multicluster) and [create-clientconfig.sh](#create-clientconfig) scripts mentioned below are available in the K8ssandra Operator [GitHub repo](https://github.com/k8ssandra/k8ssandra-operator/tree/main/scripts). +Also, note that the [setup-kind-multicluster.sh](#setup-kind-multicluster) script mentioned below is available in the K8ssandra Operator [GitHub repo](https://github.com/k8ssandra/k8ssandra-operator/tree/main/scripts). The [k8ssandra-client](https://github.com/k8ssandra/k8ssandra-client) tool is also helpful. #### kind @@ -64,9 +64,9 @@ The `make NUM_CLUSTERS= create-kind-multicluster` command, which is show **Tip:** kind generates a `kubeconfig` with the IP address of the API server set to `localhost` because the cluster is intended for local development. We need a `kubeconfig` with the IP address set to the internal address of the API server. The `setup-kind-mulitcluster.sh` script takes care of this requirement for you. -#### create-clientconfig +#### k8ssandra-client -[create-clientconfig.sh](https://github.com/k8ssandra/k8ssandra-operator/blob/main/scripts/create-clientconfig.sh) is in the k8ssandra-operator repo. This script is used to configure access to remote clusters, as described in subsequent topics. +The k8ssandra-client can be used to register data planes to control planes for multi-cluster use cases. See [here]({{< relref "tasks/data-plane-registration" >}}) for more details. ## Next steps diff --git a/docs/content/en/install/local/multi-cluster-helm/_index.md b/docs/content/en/install/local/multi-cluster-helm/_index.md index eeb299ac5..cf13152a7 100644 --- a/docs/content/en/install/local/multi-cluster-helm/_index.md +++ b/docs/content/en/install/local/multi-cluster-helm/_index.md @@ -263,9 +263,9 @@ kubectl -n k8ssandra-operator get deployment k8ssandra-operator \ false ``` -### Generate and install ClientConfigs +### Register the data plane to the control plane -[create-clientconfig.sh](https://github.com/k8ssandra/k8ssandra-operator/blob/main/scripts/create-clientconfig.sh) lives in the k8ssandra-operator repo. It is used to configure access to remote clusters. +[k8ssandra-client](https://github.com/k8ssandra/k8ssandra-client) is used to register data planes to control planes. It does this by creating a ClientConfig custom resource on the control plane and a secret with a kubeconfig entry providing access to the dataplane. **Note:** K8ssandra Operator restarts automatically whenever there is a change to a `ClientConfig` (a create, update, or delete operation). This restart is done in order to update connections to remote clusters. @@ -275,49 +275,7 @@ First, set the context to `kind-k8ssandra-0`, the control plane cluster. kubectl config use-context kind-k8ssandra-0 ``` -Run the create-clientconfig.sh script, once per data plane cluster. - -```bash -scripts/create-clientconfig.sh --namespace k8ssandra-operator \ - --src-kubeconfig ./build/kind-kubeconfig \ - --dest-kubeconfig ./build/kind-kubeconfig \ - --src-context kind-k8ssandra-1 \ - --dest-context kind-k8ssandra-0 \ - --output-dir clientconfig -``` - -**Output:** - -```bash -Creating clientconfig/kubeconfig -Creating secret kind-k8ssandra-1-config -Error from server (NotFound): secrets "kind-k8ssandra-1-config" not found -secret/kind-k8ssandra-1-config created -Creating ClientConfig clientconfig/kind-k8ssandra-1.yaml -clientconfig.config.k8ssandra.io/kind-k8ssandra-1 created -``` - -Then enter: - -```bash -scripts/create-clientconfig.sh --namespace k8ssandra-operator \ - --src-kubeconfig ./build/kind-kubeconfig \ - --dest-kubeconfig ./build/kind-kubeconfig \ - --src-context kind-k8ssandra-2 \ - --dest-context kind-k8ssandra-0 \ - --output-dir clientconfig -``` - -**Output:** - -```bash -Creating clientconfig/kubeconfig -Creating secret kind-k8ssandra-2-config -Error from server (NotFound): secrets "kind-k8ssandra-2-config" not found -secret/kind-k8ssandra-2-config created -Creating ClientConfig clientconfig/kind-k8ssandra-2.yaml -clientconfig.config.k8ssandra.io/kind-k8ssandra-2 created -``` +Run k8ssandra-client as described [here]({{< relref "/tasks/data-plane-registration" >}}). ### Deploy the K8ssandraCluster diff --git a/docs/content/en/install/local/multi-cluster-kustomize/_index.md b/docs/content/en/install/local/multi-cluster-kustomize/_index.md index 4887b4317..990a91a58 100644 --- a/docs/content/en/install/local/multi-cluster-kustomize/_index.md +++ b/docs/content/en/install/local/multi-cluster-kustomize/_index.md @@ -175,12 +175,11 @@ Verify that the `K8SSANDRA_CONTROL_PLANE` environment variable is set to `false` kubectl -n k8ssandra-operator get deployment k8ssandra-operator -o jsonpath='{.spec.template.spec.containers[0].env[?(@.name=="K8SSANDRA_CONTROL_PLANE")].value}' ``` -### Create a ClientConfig -Now we need to create a `ClientConfig` for the `kind-k8ssandra-1` cluster. We will use the -`create-clientconfig.sh` script which can be found -[here](https://github.com/k8ssandra/k8ssandra-operator/blob/main/scripts/create-clientconfig.sh). +### Register the data plane -Here is a summary of what the script does: +[k8ssandra-client](https://github.com/k8ssandra/k8ssandra-client) is used to register data planes to control planes. It does this by installing a ClientConfig on the control plane. K8ssandra-client data plane registration is described [here]({{< relref "/tasks/data-plane-registration" >}}). + +Here is a summary of what the k8ssandra-client's `register` command does': * Get the k8ssandra-operator service account from the data plane cluster * Extract the service account token @@ -192,16 +191,6 @@ Here is a summary of what the script does: Create a `ClientConfig` in the `kind-k8ssandra-0` cluster using the service account token and CA cert from `kind-k8ssandra-1`: -```sh -scripts/create-clientconfig.sh --namespace k8ssandra-operator \ - --src-kubeconfig ./build/kind-kubeconfig \ - --dest-kubeconfig ./build/kind-kubeconfig \ - --src-context kind-k8ssandra-1 \ - --dest-context kind-k8ssandra-0 \ - --output-dir clientconfig -``` -The script stores all of the artifacts that it generates in a directory which is specified with the `--output-dir` option. If not specified, a temp directory is created. - ### Restart the control plane There is a controller in the operator that watches for ClientConfig changes. When it detects a change (create/update/delete), it automatically restarts the operator. diff --git a/docs/content/en/tasks/multi-cluster/_index.md b/docs/content/en/reference/multi-cluster/_index.md similarity index 80% rename from docs/content/en/tasks/multi-cluster/_index.md rename to docs/content/en/reference/multi-cluster/_index.md index 2b09bcf41..7a7ea118d 100644 --- a/docs/content/en/tasks/multi-cluster/_index.md +++ b/docs/content/en/reference/multi-cluster/_index.md @@ -1,9 +1,10 @@ --- -title: "Multi Cluster Connection Management" +title: "Multi Cluster Connectivity" linkTitle: "Multi Cluster" -description: "Managing connections between multiple Kubernetes clusters in a multi cluster environment." +description: "Multi-cluster connectivity architecture." --- + ## Background K8ssandra Operator is capable of creating and managing a multi-datacenter Cassandra cluster that spans multiple Kubernetes clusters. The operator uses Kubernetes APIs to manage objects in remote clusters; therefore, it needs to create client connections with appropriate permissions to remote clusters. @@ -157,9 +158,6 @@ As stated earlier, the operator only processes ClientConfigs at startup. If you Proceed with caution before deleting a ClientConfig. If there are any K8ssandraClusters that use the kube config provided by the ClientConfig, then the operator won't be able to properly manage them. -#### Creating a ClientConfig -Creating a ClientConfig involves creating the kubeconfig file and secret. This can be error prone if done by hand. Instead use the `create-clientconfig.sh` script which can be found [here](https://github.com/k8ssandra/k8ssandra-operator/blob/main/scripts/create-clientconfig.sh). - The operator should already be installed in the remote cluster for which you want to create a ClientConfig. The operator service account is created when the operator is installed. Creating the ClientConfig requires access to that service account. Here is a brief summary of what the script does: @@ -171,32 +169,4 @@ Here is a brief summary of what the script does: * Create a secret for the kubeconfig in the control plane custer * Create a ClientConfig in the control plane cluster that references the secret. -Suppose we have two clusters with context names `kind-k8ssandra-0` and `kind-k8ssandra-1` and the control plane is running -in `kind-k8ssandra-0`. We want to create a ClientConfig for `kind-k8ssandra-1`. This can be accomplished by running the -following: - -```sh -./create-clientconfig.sh \ - --namespace k8ssandra-operator \ - --src-kubeconfig ./kubeconfigs/k8ssandra-1 \ - --dest-kubeconfig ./kubeconfigs/k8ssandra-0 -``` - -The above assumes that each cluster has its configuration stored in distinct files. If you have one single kubeconfig -for both clusters, then you can create the ClientConfig as follows: - -```sh -./create-clientconfig.sh \ - --namespace k8ssandra-operator \ - --src-kubeconfig ./kubeconfig \ - --dest-kubeconfig ./kubeconfig \ - --src-context kind-k8ssandra-1 \ - --dest-context kind-k8ssandra-0 -``` - -Here `./kubeconfig` refers to the kubeconfig file containing configurations to access both contexts `kind-k8ssandra-0` -and `kind-k8ssandra-1`. - -You can specify the namespace where the secret and ClientConfig are created with the `--namespace` option. - -**Note:** Remember to restart the operator in the control plane cluster after creating the ClientConfig. \ No newline at end of file +See [here]({{< relref "/tasks/data-plane-registration" >}}) for more information on the dataplane registration procedure. \ No newline at end of file diff --git a/docs/content/en/tasks/data-plane-registration/_index.md b/docs/content/en/tasks/data-plane-registration/_index.md new file mode 100644 index 000000000..415d09fcf --- /dev/null +++ b/docs/content/en/tasks/data-plane-registration/_index.md @@ -0,0 +1,27 @@ +--- +title: "Data plane registration" +linkTitle: "Data Plane Registration" +description: "Managing connections between multiple Kubernetes clusters in a multi cluster environment." +--- + +## Background + +Details of the multi-cluster connectivity architecture can be found [here]({{< relref "reference/multi-cluster" >}}). Details regarding ClientConfigs, ServiceAccounts, and Kubeconfig secrets are dealt with there. + +Proceed with caution before deleting a ClientConfig or the secrets backing it. If there are any K8ssandraClusters that use the kube config provided by the ClientConfig, then the operator won't be able to properly manage them. + +#### Creating a ClientConfig and Kubeconfig secret +Registering a data plane to a control plane can be error prone if done by hand. Instead use the `k8ssandra-client` CLI tool which can be found [here](https://github.com/k8ssandra/k8ssandra-client). It can be installed as a kubectl plugin by placing it on your `$PATH` (invoked with `kubectl k8ssandra ...`) or run completely separately (invoked with `./kubectl-k8ssandra ...`). + +An example command to register a data plane to a control plane would be `kubectl k8ssandra register --source-context gke_k8ssandra_us-central1-c_registration-2 --dest-context gke_k8ssandra_us-central1-c_registration-1` where the source context (`gke_k8ssandra_us-central1-c_registration-2`) is the data plane (because that's the source of the credentials) and the destination context (`gke_k8ssandra_us-central1-c_registration-1`) is the control plane. + +If the operator was installed in a different namespace than `k8ssandra-operator` in either the source or the destination cluster, use the following flags to indicate the actual namespaces: + +- `--dest-namespace ` +- `--source-namespace ` + +A full list of options for this command can be found by using the `--help` flag. + +#### Referencing a remote data plane within a K8ssandraCluster + +Within the K8ssandraCluster DC list, the k8sContext can be used to reference a remote data plane. When creating ClientConfigs using k8ssandra-client, you will find that you can simply refer to the data plane using the name of the ClientConfig resource. However, if the ClientConfig has been created by hand, the ClientConfig `spec.contextName` and `meta.name` may be different, in which case the `spec.contextName` should be used. We recommend using k8ssandra-client to register data planes due to complexities like these. \ No newline at end of file