|
| 1 | +--- |
| 2 | +title: "Azure Cloud Controller Manager and CSI driver for storage" |
| 3 | +description: "Guide on how to install the Azure Cloud Controller Manager and Container Storage Interface driver in Kubernetes" |
| 4 | +aliases: |
| 5 | + - ../../guides/azure-ccm-csi |
| 6 | +--- |
| 7 | + |
| 8 | +This is a guide for installing the Azure Cloud Provider and Azure CSI. |
| 9 | + |
| 10 | +The `cloud-provider-azure` module is used for interacting with Azure cloud resources through Kubernetes and this guide will also walk through setting up the CSI storage component to set up a StorageClass for workloads to use on the cluster. |
| 11 | + |
| 12 | +The steps in this guide could be used for any Kubernetes cluster with the addition of the patch applied to a Talos cluster. |
| 13 | + |
| 14 | +## Pre -requisites |
| 15 | + |
| 16 | +This guide assumes a Talos cluster is already available and the user has an Azure account set up. |
| 17 | + |
| 18 | +- Instructions for installing Talos can be found in [Talos Docs (Installation)](https://www.talos.dev/v1.4/talos-guides/install/). |
| 19 | +- Instructions for installing **talosctl** and **kubectl** can be found in [Talos Docs (Quickstart)](https://www.talos.dev/v1.4/introduction/quickstart/#talosctl). |
| 20 | + |
| 21 | +The applications in this guide will be installed using Helm. |
| 22 | + |
| 23 | +- Instructions for install **helm** can be found in the [Helm Documentation](https://helm.sh/docs/intro/install/). |
| 24 | + |
| 25 | +## Apply patch to Talos |
| 26 | + |
| 27 | +There is an option in the Talos machine config to tell the control-plane to use an external controller manager. |
| 28 | + |
| 29 | +This will apply an uninitialized label to a node when it registers to make it impossible to schedule workloads until the CCM has discovered that there is a new node in the cluster. |
| 30 | + |
| 31 | +This configuration is referenced in [Talos Docs (Machine Controller Manager)](https://www.talos.dev/v1.4/reference/configuration/#machinecontrollermanagerconfig). |
| 32 | + |
| 33 | +To apply this to the Talos cluster, create a patch file or edit the machineconfig. |
| 34 | + |
| 35 | +To create a patch file: |
| 36 | + |
| 37 | +```bash |
| 38 | +vim patch.yaml |
| 39 | +``` |
| 40 | + |
| 41 | +Add the following to the **patch.yaml** file: |
| 42 | + |
| 43 | +```yaml |
| 44 | +cluster: |
| 45 | + controllerManager: |
| 46 | + extraArgs: |
| 47 | + cloud-provider: external |
| 48 | +``` |
| 49 | +
|
| 50 | +Then, apply the patch with: |
| 51 | +
|
| 52 | +```bash |
| 53 | +talosctl machineconfig patch patch.yaml |
| 54 | +``` |
| 55 | + |
| 56 | +More information on applying machinconfig patches can be found at [Talos Docs (Machine Config Patch)](https://www.talos.dev/v1.4/reference/cli/#talosctl-machineconfig-patch). |
| 57 | + |
| 58 | +## Azure Configuration File |
| 59 | + |
| 60 | +The Azure Cloud Controller Manager requires a configuration file to gain permissions on the cluster which will require gathering a few values from the Azure Portal and creating an app registration to give the CCM the permissions it needs. |
| 61 | + |
| 62 | +This file is usually placed on the filesystem, but this guide will cover creating a secret to store this configuration instead. |
| 63 | + |
| 64 | +### App Registration |
| 65 | + |
| 66 | +The App Registration is what we will use to authenticate to Azure for uploading blobs and creating resources. |
| 67 | + |
| 68 | +For more information not in this guide or to see changes made to the app registration process, Azure's documentation can be found here: |
| 69 | + |
| 70 | +- [Azure Documentation (App Registration)](https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app) |
| 71 | + |
| 72 | +To create an App Registration in Azure: |
| 73 | + |
| 74 | +- Search for and Select **Azure Active Directory**. |
| 75 | +- Select **App registrations**, then select **New registration**. |
| 76 | +- Name the application, for example "example-app". |
| 77 | +- Select a supported account type, which determines who can use the application. |
| 78 | +- Under **Redirect URI**, select **Web** for the type of application and enter the URI where the access token is sent to. |
| 79 | +- Select **Register**. |
| 80 | + |
| 81 | +Collect the following values from Azure, as they will be needed for the Azure CCM configuration file. |
| 82 | + |
| 83 | +- **Tenant ID** |
| 84 | +- **Subscription ID** |
| 85 | +- **Client ID** |
| 86 | +- **Client Secret** |
| 87 | + |
| 88 | +#### Add permissions for App Registration |
| 89 | + |
| 90 | +The App registration only needs permissions to the Compute Gallery and the Storage Account. |
| 91 | + |
| 92 | +- Select the **Resource Group** the Talos cluster is deployed in |
| 93 | +- Select **Access control (IAM)** |
| 94 | +- Select **Add** role assignment |
| 95 | +- Select the role needed for the account. |
| 96 | + |
| 97 | +> **NOTE:** This will vary depending on what the CCM is being used for, but **Virtual Machine Contributor** is enough for the purposes if this installation guide. |
| 98 | +
|
| 99 | +### Collect additional information |
| 100 | + |
| 101 | +In the Azure Portal, collected the following values to be used in the configuration file, **specific to the cluster the CCM is being installed on**: |
| 102 | + |
| 103 | +- **Resource Group** |
| 104 | +- **Location** |
| 105 | +- **Virtual Network name** |
| 106 | +- **Route Table name** |
| 107 | + |
| 108 | +### Create the configuration file |
| 109 | + |
| 110 | +Create a configuration file named **azure.cfg** |
| 111 | + |
| 112 | +```shell |
| 113 | +vim cloud.conf |
| 114 | +``` |
| 115 | + |
| 116 | +Add the following to the **azure.cfg** file, but **replace the values with the values gathered at the beginning of this guide**. |
| 117 | + |
| 118 | +```shell |
| 119 | +{ |
| 120 | + "cloud":"AzurePublicCloud", |
| 121 | + "tenantId": "${TENANT_ID}$", |
| 122 | + "subscriptionId": "${SUBSCRIPTION_ID}$", |
| 123 | + "aadClientId": "${CLIENT_ID}$", |
| 124 | + "aadClientSecret": "${CLIENT_SECRET}$", |
| 125 | + "resourceGroup": "${RESOURCE_GROUP}$", |
| 126 | + "location": "${LOCATION}", |
| 127 | + "loadBalancerSku": "standard", |
| 128 | + "securityGroupName": "${SECURITY_GROUP_NAME}", |
| 129 | + "vnetName": "${VIRTUAL_NETWORK_NAME}", |
| 130 | + "routeTableName": "${ROUTE_TABLE_NAME}" |
| 131 | +} |
| 132 | + |
| 133 | +``` |
| 134 | + |
| 135 | +Additional configurations can be found in the CCM docs here: [Cloud Provider Azure configs](https://github.com/kubernetes-sigs/cloud-provider-azure/blob/documentation/content/en/install/configs.md). |
| 136 | + |
| 137 | +A secret can be created in Kubernetes using the following command: |
| 138 | + |
| 139 | +> **NOTE**: This secret is created in the **kube-system** namespace because that is where the CCM and CSI components will be installed. |
| 140 | +
|
| 141 | +```bash |
| 142 | +kubectl create secret generic azure-cloud-provider --from-file=cloud-config=./cloud.conf -n kube-system |
| 143 | +``` |
| 144 | + |
| 145 | +## Install the Azure Cloud Controller Manager |
| 146 | + |
| 147 | +Find the version compatible with the Kubernetes version installed with the Talos cluster https://github.com/kubernetes-sigs/cloud-provider-azure/blob/master/README.md |
| 148 | + |
| 149 | +To use the latest release add the following helm repo: |
| 150 | + |
| 151 | +> **NOTE**: To use a release specific to the Kubernetes version other than the latest version, replace **master** with the branch name specified in the version matrix above. |
| 152 | +
|
| 153 | +```bash |
| 154 | +helm repo add cloud-provider-azure https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/helm/repo |
| 155 | +``` |
| 156 | + |
| 157 | +Update helm repositories: |
| 158 | + |
| 159 | +```bash |
| 160 | +helm repo update |
| 161 | +``` |
| 162 | + |
| 163 | +Install the helm chart for `cloud-provider-azure`: |
| 164 | + |
| 165 | +```bash |
| 166 | +helm install azure-ccm cloud-provider-azure/cloud-provider-azure \ |
| 167 | +--set cloud-provider-azure.infra.clusterName="christian-tf" \ |
| 168 | +--set cloud-provider-azure.cloudControllerManager.cloudConfig='' \ |
| 169 | +--set cloud-provider-azure.cloudControllerManager.cloudConfigSecretName="azure-cloud-provider" \ |
| 170 | +--set cloud-provider-azure.cloudControllerManager.enableDynamicReloading="true" \ |
| 171 | +--set cloud-provider-azure.cloudControllerManager.configureCloudRoutes="true" \ |
| 172 | +--set cloud-provider-azure.cloudControllerManager.allocateNodeCidrs="true" \ |
| 173 | +--set cloud-provider-azure.cloudControllerManager.imageRepository="mcr.microsoft.com/oss/kubernetes" |
| 174 | +``` |
| 175 | + |
| 176 | +## Install the Azure CSI Driver |
| 177 | + |
| 178 | +dependencies: |
| 179 | + |
| 180 | +- name: azuredisk-csi-driver |
| 181 | + repository: https://raw.githubusercontent.com/kubernetes-sigs/azuredisk-csi-driver/master/charts |
| 182 | + version: v1.27.1 |
| 183 | + |
| 184 | +Add the Azure CSI helm repo: |
| 185 | + |
| 186 | +```bash |
| 187 | +helm repo add azuredisk-csi-driver https://raw.githubusercontent.com/kubernetes-sigs/azuredisk-csi-driver/master/charts |
| 188 | +``` |
| 189 | + |
| 190 | +Update helm repositories |
| 191 | + |
| 192 | +```bash |
| 193 | +helm repo update |
| 194 | +``` |
| 195 | + |
| 196 | +```bash |
| 197 | +helm install azure-csi azuredisk-csi-driver/azuredisk-csi-driver -n kube-system |
| 198 | +``` |
| 199 | + |
| 200 | +Lastly, create a file for a StorageClass to use the CSI: |
| 201 | + |
| 202 | +```bash |
| 203 | +vim azure-ssd-lrs.yaml |
| 204 | +``` |
| 205 | + |
| 206 | +Add the following contents to the file: |
| 207 | + |
| 208 | +```yaml |
| 209 | +apiVersion: storage.k8s.io/v1 |
| 210 | +kind: StorageClass |
| 211 | +metadata: |
| 212 | + name: azuredisk-standard-ssd-lrs |
| 213 | +provisioner: disk.csi.azure.com |
| 214 | +parameters: |
| 215 | + skuName: StandardSSD_LRS |
| 216 | +reclaimPolicy: Delete |
| 217 | +volumeBindingMode: Immediate |
| 218 | +allowVolumeExpansion: true |
| 219 | +``` |
| 220 | +
|
| 221 | +Create the storageclass: |
| 222 | +
|
| 223 | +```bash |
| 224 | +kubectl apply -f azure-ssd-lrs.yaml |
| 225 | +``` |
| 226 | + |
| 227 | +Persistent Volume Claims can now be created for workloads in the cluster using the StorageClass created. |
0 commit comments