Skip to content

Commit

Permalink
Add instructions to install Antrea in encap mode in AKS
Browse files Browse the repository at this point in the history 
AKS has supported a "Bring Your Own CNI" mode for a while now:
https://learn.microsoft.com/en-us/azure/aks/use-byo-cni?tabs=azure-cli

With this support, Antrea can easily be installed in encap mode in AKS
clusters, with no limitation (unlike for EKS where there is no more
connectivity from the control-plane to the Nodes). The only requirement
is to enable Antrea NodeIPAM.

This support relies on the
[Konnectivity Service](https://kubernetes.io/docs/tasks/extend-kubernetes/setup-konnectivity/),
which is deployed by default in "recent" AKS versions, and guarantees
connectivity from the control-plane to the Nodes, regardless of which
CNI is used.

Fixes #5817

Signed-off-by: Antonin Bas <[email protected]>
  • Loading branch information
@antoninbas
antoninbas committed Jan 22, 2024
1 parent 2ecbe79 commit 2de3dd4
Showing 1 changed file with 119 additions and 6 deletions.
125 changes: 119 additions & 6 deletions docs/aks-installation.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,32 @@ Engine cluster.

## Deploy Antrea to an AKS cluster

Antrea can be deployed to an AKS cluster in `networkPolicyOnly` mode, in which
Antrea enforces NetworkPolicies and implements other services for the AKS
cluster, while Azure CNI takes care of Pod IPAM and traffic routing across Nodes.
For more information about `networkPolicyOnly` mode, refer to [this design document](design/policy-only.md).
Antrea can be deployed to an AKS cluster either in `networkPolicyOnly` mode or
in `encap` mode.

In `networkPolicyOnly` mode, Antrea enforces NetworkPolicies and implements
other services for the AKS cluster, while the Azure CNI takes care of Pod IPAM
and traffic routing across Nodes. For more information about `networkPolicyOnly`
mode, refer to [this design document](design/policy-only.md).

In `encap` mode, Antrea is in charge of Pod IPAM and of all the networking
functions on the Nodes. Using `encap` mode provides access to additional Antrea
features, such as Multicast, as inter-Node Pod traffic is encapsulated, and is
not handled directly by the Azure Virtual Network. Note that the [caveats](eks-installation.md#deploying-antrea-in-encap-mode)
which apply when deploying Antrea in `encap` mode on EKS do *not* apply for AKS.

We recommend `encap` mode, as it will give you access to the most Antrea
features.

### AKS Prerequisites

Install the Azure Cloud CLI. Refer to [Azure CLI installation guide](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest)

### Creating the cluster
We recommend using the latest version available (use at least version 2.39.0).

### Deploying Antrea in `networkPolicyOnly` mode

#### Creating the cluster

You can use any method to create an AKS cluster. The example given here is using the Azure Cloud CLI.

Expand Down Expand Up @@ -50,7 +66,7 @@ You can use any method to create an AKS cluster. The example given here is using
aks-nodepool1-84330359-vmss000001 Ready agent 6m25s v1.16.10
```

### Deploying Antrea
#### Deploying Antrea

1. Prepare the Cluster Nodes

Expand Down Expand Up @@ -110,6 +126,103 @@ you should be able to see these Pods running in your cluster:
pod "tunnelfront-649ff5fb55-5lxg7" deleted
```

### Deploying Antrea in `encap` mode

AKS now officially supports [Bring your own Container Network Interface (BYOCNI)](https://learn.microsoft.com/en-us/azure/aks/use-byo-cni).
Thanks to this, you can deploy Antrea on EKS in `encap` mode, and you will not
lose access to any functionality. Check the AKS BYOCNI documentation for
prerequisites, in particular for AKS version requirements.

#### Creating the cluster

You can use any method to create an AKS cluster. The example given here is using the Azure Cloud CLI.

1. Create an AKS Cluster

```bash
export RESOURCE_GROUP_NAME=aks-antrea-cluster
export CLUSTER_NAME=aks-antrea-cluster
export LOCATION=westus
az group create --name $RESOURCE_GROUP_NAME --location $LOCATION
az aks create \
--resource-group $RESOURCE_GROUP_NAME \
--name $CLUSTER_NAME \
--node-count 2 \
--network-plugin none
```

Notice `--network-plugin none`, which tells AKS not to install any CNI plugin.

2. Get AKS Cluster Credentials

```bash
az aks get-credentials --name $CLUSTER_NAME --resource-group $RESOURCE_GROUP_NAME
```

3. Access your Cluster

```bash
kubectl get nodes
NAME STATUS ROLES AGE VERSION
aks-nodepool1-40948307-vmss000000 NotReady agent 18m v1.27.7
aks-nodepool1-40948307-vmss000001 NotReady agent 17m v1.27.7
```

The Nodes are supposed to report a `NotReady` Status, since no CNI plugin is
installed yet.

#### Deploying Antrea

You can use Helm to easily install Antrea (or any other supported installation
method). Just make sure that you configure Antrea NodeIPAM:

```bash
# you may not need this:
helm repo add antrea https://charts.antrea.io
helm repo update
cat <<EOF >> values-aks.yml
nodeIPAM:
enable: true
clusterCIDRs: ["10.10.0.0/16"]
EOF
helm install -n kube-system -f values-aks.yml antrea antrea/antrea
```

For more information about how to configure Antrea Node IPAM, please refer to
[Antrea Node IPAM guide](antrea-ipam.md#running-nodeipam-within-antrea-controller).

After a while, make sure that all your Nodes report a `Ready` Status and that
all your Pods are running correctly. Some Pods, and in particular the
`metrics-server` Pods, may restart once after installing Antrea; this is not an
issue.

After a successful installation, Pods should look like this:

```bash
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system antrea-agent-bpskv 2/2 Running 0 7m34s
kube-system antrea-agent-pfqrn 2/2 Running 0 7m34s
kube-system antrea-controller-555b8c799d-wk8zz 1/1 Running 0 7m34s
kube-system cloud-node-manager-2nszz 1/1 Running 0 31m
kube-system cloud-node-manager-wj68q 1/1 Running 0 31m
kube-system coredns-789789675-2nwd7 1/1 Running 0 6m48s
kube-system coredns-789789675-lbkfn 1/1 Running 0 31m
kube-system coredns-autoscaler-649b947bbd-j5wqc 1/1 Running 0 31m
kube-system csi-azuredisk-node-4bnnl 3/3 Running 0 31m
kube-system csi-azuredisk-node-52nwd 3/3 Running 0 31m
kube-system csi-azurefile-node-2h66l 3/3 Running 0 31m
kube-system csi-azurefile-node-dhrf2 3/3 Running 0 31m
kube-system konnectivity-agent-5fc7989878-6nhwl 1/1 Running 0 31m
kube-system konnectivity-agent-5fc7989878-t2n6h 1/1 Running 0 30m
kube-system kube-proxy-96c9p 1/1 Running 0 31m
kube-system kube-proxy-x8g8s 1/1 Running 0 31m
kube-system metrics-server-5955767688-2hjvn 2/2 Running 0 3m45s
kube-system metrics-server-5955767688-vmcq7 2/2 Running 0 3m45s
```

## Deploy Antrea to an AKS Engine cluster

Antrea is an integrated CNI of AKS Engine, and can be installed in
Expand Down

0 comments on commit 2de3dd4

Please sign in to comment.