Skip to content

Commit

Permalink
Add support for NodeNetworkPolicy data plane
Browse files Browse the repository at this point in the history
Signed-off-by: Hongliang Liu <[email protected]>
  • Loading branch information
hongliangl committed Jan 8, 2024
1 parent fed260a commit 912aa68
Show file tree
Hide file tree
Showing 38 changed files with 4,643 additions and 377 deletions.
3 changes: 3 additions & 0 deletions build/charts/antrea/conf/antrea-agent.conf
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,9 @@ featureGates:
# Allow users to allocate Egress IPs from a different subnet from the default Node subnet.
{{- include "featureGate" (dict "featureGates" .Values.featureGates "name" "EgressSeparateSubnet" "default" false) }}

# Allow users to apply ClusterNetworkPolicy to Kubernetes Nodes.
{{- include "featureGate" (dict "featureGates" .Values.featureGates "name" "NodeNetworkPolicy" "default" false) }}

# Name of the OpenVSwitch bridge antrea-agent will create and use.
# Make sure it doesn't conflict with your existing OpenVSwitch bridges.
ovsBridge: {{ .Values.ovs.bridgeName | quote }}
Expand Down
7 changes: 5 additions & 2 deletions build/yamls/antrea-aks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5625,6 +5625,9 @@ data:
# Allow users to allocate Egress IPs from a different subnet from the default Node subnet.
# EgressSeparateSubnet: false
# Allow users to apply ClusterNetworkPolicy to Kubernetes Nodes.
# NodeNetworkPolicy: false
# Name of the OpenVSwitch bridge antrea-agent will create and use.
# Make sure it doesn't conflict with your existing OpenVSwitch bridges.
ovsBridge: "br-int"
Expand Down Expand Up @@ -6925,7 +6928,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: fe9081d7718e258905728726bb8f3a8d42a332d7f4e0d8faaa9731b3c4b51aa4
checksum/config: f4ad8910666191c02982d1b7b202e3c4bd20fb4a8179dcb5696119f3b1490a72
labels:
app: antrea
component: antrea-agent
Expand Down Expand Up @@ -7163,7 +7166,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: fe9081d7718e258905728726bb8f3a8d42a332d7f4e0d8faaa9731b3c4b51aa4
checksum/config: f4ad8910666191c02982d1b7b202e3c4bd20fb4a8179dcb5696119f3b1490a72
labels:
app: antrea
component: antrea-controller
Expand Down
7 changes: 5 additions & 2 deletions build/yamls/antrea-eks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5625,6 +5625,9 @@ data:
# Allow users to allocate Egress IPs from a different subnet from the default Node subnet.
# EgressSeparateSubnet: false
# Allow users to apply ClusterNetworkPolicy to Kubernetes Nodes.
# NodeNetworkPolicy: false
# Name of the OpenVSwitch bridge antrea-agent will create and use.
# Make sure it doesn't conflict with your existing OpenVSwitch bridges.
ovsBridge: "br-int"
Expand Down Expand Up @@ -6925,7 +6928,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: fe9081d7718e258905728726bb8f3a8d42a332d7f4e0d8faaa9731b3c4b51aa4
checksum/config: f4ad8910666191c02982d1b7b202e3c4bd20fb4a8179dcb5696119f3b1490a72
labels:
app: antrea
component: antrea-agent
Expand Down Expand Up @@ -7164,7 +7167,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: fe9081d7718e258905728726bb8f3a8d42a332d7f4e0d8faaa9731b3c4b51aa4
checksum/config: f4ad8910666191c02982d1b7b202e3c4bd20fb4a8179dcb5696119f3b1490a72
labels:
app: antrea
component: antrea-controller
Expand Down
7 changes: 5 additions & 2 deletions build/yamls/antrea-gke.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5625,6 +5625,9 @@ data:
# Allow users to allocate Egress IPs from a different subnet from the default Node subnet.
# EgressSeparateSubnet: false
# Allow users to apply ClusterNetworkPolicy to Kubernetes Nodes.
# NodeNetworkPolicy: false
# Name of the OpenVSwitch bridge antrea-agent will create and use.
# Make sure it doesn't conflict with your existing OpenVSwitch bridges.
ovsBridge: "br-int"
Expand Down Expand Up @@ -6925,7 +6928,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 997259cac105a193d671880b165e203a9954f33009766df5eceed753509c46b9
checksum/config: a54768c79d693083be554386f268c93bbbd0fdf5b334edd9aff31c13151c4e29
labels:
app: antrea
component: antrea-agent
Expand Down Expand Up @@ -7161,7 +7164,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 997259cac105a193d671880b165e203a9954f33009766df5eceed753509c46b9
checksum/config: a54768c79d693083be554386f268c93bbbd0fdf5b334edd9aff31c13151c4e29
labels:
app: antrea
component: antrea-controller
Expand Down
7 changes: 5 additions & 2 deletions build/yamls/antrea-ipsec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5638,6 +5638,9 @@ data:
# Allow users to allocate Egress IPs from a different subnet from the default Node subnet.
# EgressSeparateSubnet: false
# Allow users to apply ClusterNetworkPolicy to Kubernetes Nodes.
# NodeNetworkPolicy: false
# Name of the OpenVSwitch bridge antrea-agent will create and use.
# Make sure it doesn't conflict with your existing OpenVSwitch bridges.
ovsBridge: "br-int"
Expand Down Expand Up @@ -6938,7 +6941,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 4364ee1520a24d9a465a405536736498119269c0fc81d4dc01e83d7fdd462913
checksum/config: 7ce7d85bc08079d1cef3b1d44f31e2139961f9ae49f71d79ff3b28e7e9ad6325
checksum/ipsec-secret: d0eb9c52d0cd4311b6d252a951126bf9bea27ec05590bed8a394f0f792dcb2a4
labels:
app: antrea
Expand Down Expand Up @@ -7220,7 +7223,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 4364ee1520a24d9a465a405536736498119269c0fc81d4dc01e83d7fdd462913
checksum/config: 7ce7d85bc08079d1cef3b1d44f31e2139961f9ae49f71d79ff3b28e7e9ad6325
labels:
app: antrea
component: antrea-controller
Expand Down
7 changes: 5 additions & 2 deletions build/yamls/antrea.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5625,6 +5625,9 @@ data:
# Allow users to allocate Egress IPs from a different subnet from the default Node subnet.
# EgressSeparateSubnet: false
# Allow users to apply ClusterNetworkPolicy to Kubernetes Nodes.
# NodeNetworkPolicy: false
# Name of the OpenVSwitch bridge antrea-agent will create and use.
# Make sure it doesn't conflict with your existing OpenVSwitch bridges.
ovsBridge: "br-int"
Expand Down Expand Up @@ -6925,7 +6928,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: d8e5dd6cc4bd55eeba224229fd8207045291ab3dfafe5f3c3e100c31003d2887
checksum/config: 290f0c748863a7dad1e9d53d62c74f8108a44c5cc803306d351c108062cc1378
labels:
app: antrea
component: antrea-agent
Expand Down Expand Up @@ -7161,7 +7164,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: d8e5dd6cc4bd55eeba224229fd8207045291ab3dfafe5f3c3e100c31003d2887
checksum/config: 290f0c748863a7dad1e9d53d62c74f8108a44c5cc803306d351c108062cc1378
labels:
app: antrea
component: antrea-controller
Expand Down
11 changes: 10 additions & 1 deletion cmd/antrea-agent/agent.go
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ func run(o *Options) error {
enableAntreaIPAM := features.DefaultFeatureGate.Enabled(features.AntreaIPAM)
enableBridgingMode := enableAntreaIPAM && o.config.EnableBridgingMode
l7NetworkPolicyEnabled := features.DefaultFeatureGate.Enabled(features.L7NetworkPolicy)
nodeNetworkPolicyEnabled := features.DefaultFeatureGate.Enabled(features.NodeNetworkPolicy)
enableMulticlusterGW := features.DefaultFeatureGate.Enabled(features.Multicluster) && o.config.Multicluster.EnableGateway
enableMulticlusterNP := features.DefaultFeatureGate.Enabled(features.Multicluster) && o.config.Multicluster.EnableStretchedNetworkPolicy
enableFlowExporter := features.DefaultFeatureGate.Enabled(features.FlowExporter) && o.config.FlowExporter.Enable
Expand Down Expand Up @@ -219,7 +220,13 @@ func run(o *Options) error {
egressConfig := &config.EgressConfig{
ExceptCIDRs: exceptCIDRs,
}
routeClient, err := route.NewClient(networkConfig, o.config.NoSNAT, o.config.AntreaProxy.ProxyAll, connectUplinkToBridge, multicastEnabled, serviceCIDRProvider)
routeClient, err := route.NewClient(networkConfig,
o.config.NoSNAT,
o.config.AntreaProxy.ProxyAll,
connectUplinkToBridge,
nodeNetworkPolicyEnabled,
multicastEnabled,
serviceCIDRProvider)
if err != nil {
return fmt.Errorf("error creating route client: %v", err)
}
Expand Down Expand Up @@ -462,6 +469,7 @@ func run(o *Options) error {
networkPolicyController, err := networkpolicy.NewNetworkPolicyController(
antreaClientProvider,
ofClient,
routeClient,
ifaceStore,
afero.NewOsFs(),
nodeKey,
Expand All @@ -471,6 +479,7 @@ func run(o *Options) error {
groupIDUpdates,
antreaPolicyEnabled,
l7NetworkPolicyEnabled,
nodeNetworkPolicyEnabled,
o.enableAntreaProxy,
statusManagerEnabled,
multicastEnabled,
Expand Down
51 changes: 51 additions & 0 deletions docs/antrea-network-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
- [ACNP for IGMP traffic](#acnp-for-igmp-traffic)
- [ACNP for multicast egress traffic](#acnp-for-multicast-egress-traffic)
- [ACNP for HTTP traffic](#acnp-for-http-traffic)
- [ACNP for Kubernetes Node traffic](#acnp-for-kubernetes-node-traffic)
- [ACNP with log settings](#acnp-with-log-settings)
- [Behavior of <em>to</em> and <em>from</em> selectors](#behavior-of-to-and-from-selectors)
- [Key differences from K8s NetworkPolicy](#key-differences-from-k8s-networkpolicy)
Expand Down Expand Up @@ -524,6 +525,56 @@ spec:

Please refer to [Antrea Layer 7 NetworkPolicy](antrea-l7-network-policy.md) for extra information.

#### ACNP for Kubernetes Node traffic

```yaml
apiVersion: crd.antrea.io/v1beta1
kind: ClusterNetworkPolicy
metadata:
name: acnp-node-egress-traffic-drop
spec:
priority: 5
tier: securityops
appliedTo:
- nodeSelector:
matchLabels:
kubernetes.io/os: linux
egress:
- action: Drop
to:
- ipBlock:
cidr: 192.168.1.0/24
ports:
- protocol: TCP
port: 80
name: dropHTTPTrafficToCIDR
```

```yaml
apiVersion: crd.antrea.io/v1beta1
kind: ClusterNetworkPolicy
metadata:
name: acnp-node-ingress-traffic-drop
spec:
priority: 5
tier: securityops
appliedTo:
- nodeSelector:
matchLabels:
kubernetes.io/os: linux
ingress:
- action: Drop
from:
- ipBlock:
cidr: 192.168.1.0/24
ports:
- protocol: TCP
port: 22
name: dropSSHTrafficFromCIDR
```

Please refer to [Antrea Node NetworkPolicy](antrea-node-network-policy.md) for more information.

#### ACNP with log settings

```yaml
Expand Down
116 changes: 116 additions & 0 deletions docs/antrea-node-network-policy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Antrea Node NetworkPolicy

## Table of Contents

<!-- toc -->
- [Introduction](#introduction)
- [Prerequisites](#prerequisites)
- [Usage](#usage)
- [Limitations](#limitations)
<!-- /toc -->

## Introduction

Node NetworkPolicy is designed to secure the Kubernetes Nodes traffic. It is supported by Antrea starting with Antrea
v1.15. This guide demonstrates how to configure Node NetworkPolicies.

## Prerequisites

Node NetworkPolicy was introduced in v1.15 as an alpha feature and is disabled by default. A feature gate,
`NodeNetworkPolicy`, must be enabled in antrea-agent.conf in the `antrea-config` ConfigMap.

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: antrea-config
namespace: kube-system
data:
antrea-agent.conf: |
featureGates:
NodeNetworkPolicy: true
```
Alternatively, you can use the following helm installation command to enable the feature gate:
```bash
helm install antrea antrea/antrea --namespace kube-system --set featureGates.NodeNetworkPolicy=true
```

## Usage

Node NetworkPolicy is an extension of Antrea ClusterNetworkPolicy (ACNP). By specifying a `nodeSelector` in the
policy-level `appliedTo`, an ACNP is applied to the selected Kubernetes Nodes.

An example Node NetworkPolicy that blocks ingress traffic from Pods with label `app=client` to Nodes with label
`kubernetes.io/hostname: k8s-node-control-plane`:

```yaml
apiVersion: crd.antrea.io/v1beta1
kind: ClusterNetworkPolicy
metadata:
name: ingress-drop-pod-to-node
spec:
priority: 5
tier: application
appliedTo:
- nodeSelector:
matchLabels:
kubernetes.io/hostname: k8s-node-control-plane
ingress:
- name: drop-80
action: Drop
from:
- podSelector:
matchLabels:
app: client
ports:
- protocol: TCP
port: 80
```
An example Node NetworkPolicy that blocks egress traffic from Nodes with the label
`kubernetes.io/hostname: k8s-node-control-plane` to Nodes with the label `kubernetes.io/hostname: k8s-node-worker-1`
and some IP blocks:

```yaml
apiVersion: crd.antrea.io/v1beta1
kind: ClusterNetworkPolicy
metadata:
name: egress-drop-node-to-node
spec:
priority: 5
tier: application
appliedTo:
- nodeSelector:
matchLabels:
kubernetes.io/hostname: k8s-node-control-plane
egress:
- name: drop-22
action: Drop
to:
- nodeSelector:
matchLabels:
kubernetes.io/hostname: k8s-node-worker-1
- ipBlock:
cidr: 192.168.77.0/24
- ipBlock:
cidr: 10.10.0.0/24
ports:
- protocol: TCP
port: 22
```

## Limitations

- This feature is currently only supported for Linux Nodes.
- Be cautious when you configure policies to Nodes, in particular, when configuring a default-deny policy applied to
Nodes. You should ensure Kubernetes and Antrea control-plane communication is exempt from the deny rules, otherwise
the cluster may go out-of-service and you may lose connectivity to the Nodes.
- Only ACNPs can be applied to Nodes. ANPs cannot be applied to Nodes.
- `nodeSelector` can only be specified in the policy-level `appliedTo` field, not in the rule-level `appliedTo`, and not
in a `Group` or `ClusterGroup`.
- ACNPs applied to Nodes cannot be applied to Pods at the same time.
- FQDN is not supported for ACNPs applied to Nodes.
- Layer 7 NetworkPolicy is not supported yet.
- For UDP or SCTP, when the `Reject` action is specified in an egress rule, it behaves identical to the `Drop` action.
9 changes: 9 additions & 0 deletions docs/feature-gates.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ edit the Agent configuration in the
| `AdminNetworkPolicy` | Controller | `false` | Alpha | v1.13 | N/A | N/A | Yes | |
| `EgressTrafficShaping` | Agent | `false` | Alpha | v1.14 | N/A | N/A | Yes | OVS meters should be supported |
| `EgressSeparateSubnet` | Agent | `false` | Alpha | v1.15 | N/A | N/A | No | |
| `NodeNetworkPolicy` | Agent | `false` | Alpha | v1.15 | N/A | N/A | Yes | |

## Description and Requirements of Features

Expand Down Expand Up @@ -405,6 +406,14 @@ this [document](antrea-l7-network-policy.md#prerequisites) for more information
The `AdminNetworkPolicy` API (which currently includes the AdminNetworkPolicy and BaselineAdminNetworkPolicy objects)
complements the Antrea-native policies and help cluster administrators to set security postures in a portable manner.

### NodeNetworkPolicy

`NodeNetworkPolicy` allows users to protect their Kubernetes Nodes.

#### Requirements for this Feature

This feature is only supported for Linux Nodes at the moment.

### EgressTrafficShaping

The `EgressTrafficShaping` feature gate of Antrea Agent enables traffic shaping of Egress, which could limit the
Expand Down
Loading

0 comments on commit 912aa68

Please sign in to comment.