Skip to content

Commit

Permalink
Enable disabling TX checksum offload for Antrea host gateway
Browse files Browse the repository at this point in the history 
This commit introduces the ability to disable TX checksum offload
for the host gateway interface (default: `antrea-gw0`) by setting the
`disableTXChecksumOffload` option to `true`.

Note: If this option is later set to false, Antrea will not restore
the original TX checksum state, as it does not retain the original
configuration. Users are responsible for manually reconfiguring the
setting if needed.

Signed-off-by: Hongliang Liu <[email protected]>
  • Loading branch information
@hongliangl
hongliangl committed Dec 4, 2024
1 parent b646bbd commit e92e5ab
Show file tree
Hide file tree
Showing 11 changed files with 127 additions and 77 deletions.
10 changes: 7 additions & 3 deletions build/charts/antrea/conf/antrea-agent.conf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -179,9 +179,13 @@ trafficEncryptionMode: {{ .Values.trafficEncryptionMode | quote }}
# `trafficEncapMode` is `noEncap`, and `noSNAT` is true.
enableBridgingMode: {{ .Values.enableBridgingMode }}

# Disable TX checksum offloading for container network interfaces. It's supposed to be set to true when the
# datapath doesn't support TX checksum offloading, which causes packets to be dropped due to bad checksum.
# It affects Pods running on Linux Nodes only.
# Disable TX checksum offloading for container network interfaces and the host gateway interface (default:
# antrea-gw0). It's supposed to be set to true when the datapath doesn't support TX checksum offloading,
# which causes packets to be dropped due to bad checksum.
# If this option is later set to false, for the host gateway interface, Antrea will not restore its
# original TX checksum state, as Antrea does not retain the original configuration. Users are responsible
# for manually reconfiguring the setting if needed.
# This option affects Linux Nodes only.
disableTXChecksumOffload: {{ .Values.disableTXChecksumOffload }}

# Default MTU to use for the host gateway interface and the network interface of each Pod.
Expand Down
14 changes: 9 additions & 5 deletions build/yamls/antrea-aks.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4122,9 +4122,13 @@ data:
# `trafficEncapMode` is `noEncap`, and `noSNAT` is true.
enableBridgingMode: false
# Disable TX checksum offloading for container network interfaces. It's supposed to be set to true when the
# datapath doesn't support TX checksum offloading, which causes packets to be dropped due to bad checksum.
# It affects Pods running on Linux Nodes only.
# Disable TX checksum offloading for container network interfaces and the host gateway interface (default:
# antrea-gw0). It's supposed to be set to true when the datapath doesn't support TX checksum offloading,
# which causes packets to be dropped due to bad checksum.
# If this option is later set to false, for the host gateway interface, Antrea will not restore its
# original TX checksum state, as Antrea does not retain the original configuration. Users are responsible
# for manually reconfiguring the setting if needed.
# This option affects Linux Nodes only.
disableTXChecksumOffload: false
# Default MTU to use for the host gateway interface and the network interface of each Pod.
Expand Down Expand Up @@ -5394,7 +5398,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 5dd823245aab41ce7ca74d05693aa96e1537615f6966b6b78879cde5d3a0b215
checksum/config: c33eb2561c88bdaa139731764e64f7e2e82a352e565277e91a3e8d6b7cc5ad8e
labels:
app: antrea
component: antrea-agent
Expand Down Expand Up @@ -5632,7 +5636,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 5dd823245aab41ce7ca74d05693aa96e1537615f6966b6b78879cde5d3a0b215
checksum/config: c33eb2561c88bdaa139731764e64f7e2e82a352e565277e91a3e8d6b7cc5ad8e
labels:
app: antrea
component: antrea-controller
Expand Down
14 changes: 9 additions & 5 deletions build/yamls/antrea-eks.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4122,9 +4122,13 @@ data:
# `trafficEncapMode` is `noEncap`, and `noSNAT` is true.
enableBridgingMode: false
# Disable TX checksum offloading for container network interfaces. It's supposed to be set to true when the
# datapath doesn't support TX checksum offloading, which causes packets to be dropped due to bad checksum.
# It affects Pods running on Linux Nodes only.
# Disable TX checksum offloading for container network interfaces and the host gateway interface (default:
# antrea-gw0). It's supposed to be set to true when the datapath doesn't support TX checksum offloading,
# which causes packets to be dropped due to bad checksum.
# If this option is later set to false, for the host gateway interface, Antrea will not restore its
# original TX checksum state, as Antrea does not retain the original configuration. Users are responsible
# for manually reconfiguring the setting if needed.
# This option affects Linux Nodes only.
disableTXChecksumOffload: false
# Default MTU to use for the host gateway interface and the network interface of each Pod.
Expand Down Expand Up @@ -5394,7 +5398,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 5dd823245aab41ce7ca74d05693aa96e1537615f6966b6b78879cde5d3a0b215
checksum/config: c33eb2561c88bdaa139731764e64f7e2e82a352e565277e91a3e8d6b7cc5ad8e
labels:
app: antrea
component: antrea-agent
Expand Down Expand Up @@ -5633,7 +5637,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 5dd823245aab41ce7ca74d05693aa96e1537615f6966b6b78879cde5d3a0b215
checksum/config: c33eb2561c88bdaa139731764e64f7e2e82a352e565277e91a3e8d6b7cc5ad8e
labels:
app: antrea
component: antrea-controller
Expand Down
14 changes: 9 additions & 5 deletions build/yamls/antrea-gke.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4122,9 +4122,13 @@ data:
# `trafficEncapMode` is `noEncap`, and `noSNAT` is true.
enableBridgingMode: false
# Disable TX checksum offloading for container network interfaces. It's supposed to be set to true when the
# datapath doesn't support TX checksum offloading, which causes packets to be dropped due to bad checksum.
# It affects Pods running on Linux Nodes only.
# Disable TX checksum offloading for container network interfaces and the host gateway interface (default:
# antrea-gw0). It's supposed to be set to true when the datapath doesn't support TX checksum offloading,
# which causes packets to be dropped due to bad checksum.
# If this option is later set to false, for the host gateway interface, Antrea will not restore its
# original TX checksum state, as Antrea does not retain the original configuration. Users are responsible
# for manually reconfiguring the setting if needed.
# This option affects Linux Nodes only.
disableTXChecksumOffload: false
# Default MTU to use for the host gateway interface and the network interface of each Pod.
Expand Down Expand Up @@ -5394,7 +5398,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: e9ea48fa57cb11513f69a1fb2b44dd3c6cb96aa739598c1db5091ea91f097f4b
checksum/config: d1bc82a4105dd68d41a246fdeaa94f523a312d25a015a4af642db3dd921d9ca2
labels:
app: antrea
component: antrea-agent
Expand Down Expand Up @@ -5630,7 +5634,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: e9ea48fa57cb11513f69a1fb2b44dd3c6cb96aa739598c1db5091ea91f097f4b
checksum/config: d1bc82a4105dd68d41a246fdeaa94f523a312d25a015a4af642db3dd921d9ca2
labels:
app: antrea
component: antrea-controller
Expand Down
14 changes: 9 additions & 5 deletions build/yamls/antrea-ipsec.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4135,9 +4135,13 @@ data:
# `trafficEncapMode` is `noEncap`, and `noSNAT` is true.
enableBridgingMode: false
# Disable TX checksum offloading for container network interfaces. It's supposed to be set to true when the
# datapath doesn't support TX checksum offloading, which causes packets to be dropped due to bad checksum.
# It affects Pods running on Linux Nodes only.
# Disable TX checksum offloading for container network interfaces and the host gateway interface (default:
# antrea-gw0). It's supposed to be set to true when the datapath doesn't support TX checksum offloading,
# which causes packets to be dropped due to bad checksum.
# If this option is later set to false, for the host gateway interface, Antrea will not restore its
# original TX checksum state, as Antrea does not retain the original configuration. Users are responsible
# for manually reconfiguring the setting if needed.
# This option affects Linux Nodes only.
disableTXChecksumOffload: false
# Default MTU to use for the host gateway interface and the network interface of each Pod.
Expand Down Expand Up @@ -5407,7 +5411,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 38e19ea8db3838e3f5cff4aaa2684db1586fb457d095ac3ea49e8bf405a04e41
checksum/config: 27b65f250725362ac4a550513f781b82c6b6c6145c0ce660cc3beaebe221bb68
checksum/ipsec-secret: d0eb9c52d0cd4311b6d252a951126bf9bea27ec05590bed8a394f0f792dcb2a4
labels:
app: antrea
Expand Down Expand Up @@ -5689,7 +5693,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 38e19ea8db3838e3f5cff4aaa2684db1586fb457d095ac3ea49e8bf405a04e41
checksum/config: 27b65f250725362ac4a550513f781b82c6b6c6145c0ce660cc3beaebe221bb68
labels:
app: antrea
component: antrea-controller
Expand Down
14 changes: 9 additions & 5 deletions build/yamls/antrea.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4122,9 +4122,13 @@ data:
# `trafficEncapMode` is `noEncap`, and `noSNAT` is true.
enableBridgingMode: false
# Disable TX checksum offloading for container network interfaces. It's supposed to be set to true when the
# datapath doesn't support TX checksum offloading, which causes packets to be dropped due to bad checksum.
# It affects Pods running on Linux Nodes only.
# Disable TX checksum offloading for container network interfaces and the host gateway interface (default:
# antrea-gw0). It's supposed to be set to true when the datapath doesn't support TX checksum offloading,
# which causes packets to be dropped due to bad checksum.
# If this option is later set to false, for the host gateway interface, Antrea will not restore its
# original TX checksum state, as Antrea does not retain the original configuration. Users are responsible
# for manually reconfiguring the setting if needed.
# This option affects Linux Nodes only.
disableTXChecksumOffload: false
# Default MTU to use for the host gateway interface and the network interface of each Pod.
Expand Down Expand Up @@ -5394,7 +5398,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 59fb1ea496577015058d4e99e4f64136aa68d5340db13c00ced565da750a22fc
checksum/config: f61c0982305ae124c954dc3980b6bacabc23663e7eab46da17dd9d65fbf8a997
labels:
app: antrea
component: antrea-agent
Expand Down Expand Up @@ -5630,7 +5634,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 59fb1ea496577015058d4e99e4f64136aa68d5340db13c00ced565da750a22fc
checksum/config: f61c0982305ae124c954dc3980b6bacabc23663e7eab46da17dd9d65fbf8a997
labels:
app: antrea
component: antrea-controller
Expand Down
3 changes: 2 additions & 1 deletion cmd/antrea-agent/agent.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,8 @@ func run(o *Options) error {
connectUplinkToBridge,
o.enableAntreaProxy,
l7NetworkPolicyEnabled,
l7FlowExporterEnabled)
l7FlowExporterEnabled,
o.config.DisableTXChecksumOffload)
err = agentInitializer.Initialize()
if err != nil {
return fmt.Errorf("error initializing agent: %v", err)
Expand Down
96 changes: 51 additions & 45 deletions pkg/agent/agent.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -109,27 +109,28 @@ var (

// Initializer knows how to setup host networking, OpenVSwitch, and Openflow.
type Initializer struct {
client clientset.Interface
crdClient versioned.Interface
ovsBridgeClient ovsconfig.OVSBridgeClient
ovsCtlClient ovsctl.OVSCtlClient
ofClient openflow.Client
routeClient route.Interface
wireGuardClient wireguard.Interface
ifaceStore interfacestore.InterfaceStore
ovsBridge string
hostGateway string // name of gateway port on the OVS bridge
mtu int
networkConfig *config.NetworkConfig
nodeConfig *config.NodeConfig
wireGuardConfig *config.WireGuardConfig
egressConfig *config.EgressConfig
serviceConfig *config.ServiceConfig
l7NetworkPolicyConfig *config.L7NetworkPolicyConfig
enableL7NetworkPolicy bool
enableL7FlowExporter bool
connectUplinkToBridge bool
enableAntreaProxy bool
client clientset.Interface
crdClient versioned.Interface
ovsBridgeClient ovsconfig.OVSBridgeClient
ovsCtlClient ovsctl.OVSCtlClient
ofClient openflow.Client
routeClient route.Interface
wireGuardClient wireguard.Interface
ifaceStore interfacestore.InterfaceStore
ovsBridge string
hostGateway string // name of gateway port on the OVS bridge
mtu int
networkConfig *config.NetworkConfig
nodeConfig *config.NodeConfig
wireGuardConfig *config.WireGuardConfig
egressConfig *config.EgressConfig
serviceConfig *config.ServiceConfig
l7NetworkPolicyConfig *config.L7NetworkPolicyConfig
enableL7NetworkPolicy bool
enableL7FlowExporter bool
connectUplinkToBridge bool
enableAntreaProxy bool
disableTXChecksumOffload bool
// podNetworkWait should be decremented once the Node's network is ready.
// The CNI server will wait for it before handling any CNI Add requests.
podNetworkWait *utilwait.Group
Expand Down Expand Up @@ -166,32 +167,34 @@ func NewInitializer(
enableAntreaProxy bool,
enableL7NetworkPolicy bool,
enableL7FlowExporter bool,
disableTXChecksumOffload bool,
) *Initializer {
return &Initializer{
ovsBridgeClient: ovsBridgeClient,
ovsCtlClient: ovsCtlClient,
client: k8sClient,
crdClient: crdClient,
ifaceStore: ifaceStore,
ofClient: ofClient,
routeClient: routeClient,
ovsBridge: ovsBridge,
hostGateway: hostGateway,
mtu: mtu,
networkConfig: networkConfig,
wireGuardConfig: wireGuardConfig,
egressConfig: egressConfig,
serviceConfig: serviceConfig,
l7NetworkPolicyConfig: &config.L7NetworkPolicyConfig{},
podNetworkWait: podNetworkWait,
flowRestoreCompleteWait: flowRestoreCompleteWait,
stopCh: stopCh,
nodeType: nodeType,
externalNodeNamespace: externalNodeNamespace,
connectUplinkToBridge: connectUplinkToBridge,
enableAntreaProxy: enableAntreaProxy,
enableL7NetworkPolicy: enableL7NetworkPolicy,
enableL7FlowExporter: enableL7FlowExporter,
ovsBridgeClient: ovsBridgeClient,
ovsCtlClient: ovsCtlClient,
client: k8sClient,
crdClient: crdClient,
ifaceStore: ifaceStore,
ofClient: ofClient,
routeClient: routeClient,
ovsBridge: ovsBridge,
hostGateway: hostGateway,
mtu: mtu,
networkConfig: networkConfig,
wireGuardConfig: wireGuardConfig,
egressConfig: egressConfig,
serviceConfig: serviceConfig,
l7NetworkPolicyConfig: &config.L7NetworkPolicyConfig{},
podNetworkWait: podNetworkWait,
flowRestoreCompleteWait: flowRestoreCompleteWait,
stopCh: stopCh,
nodeType: nodeType,
externalNodeNamespace: externalNodeNamespace,
connectUplinkToBridge: connectUplinkToBridge,
enableAntreaProxy: enableAntreaProxy,
enableL7NetworkPolicy: enableL7NetworkPolicy,
enableL7FlowExporter: enableL7FlowExporter,
disableTXChecksumOffload: disableTXChecksumOffload,
}
}

Expand Down Expand Up @@ -706,6 +709,9 @@ func (i *Initializer) setupGatewayInterface() error {
return err
}
}
if err := i.setTXChecksumOffload(); err != nil {
return err
}

return nil
}
Expand Down
11 changes: 11 additions & 0 deletions pkg/agent/agent_linux.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
package agent

import (
"antrea.io/antrea/pkg/agent/util/ethtool"
"context"
"fmt"
"net"
Expand Down Expand Up @@ -262,3 +263,13 @@ func (i *Initializer) prepareL7EngineInterfaces() error {
}
return nil
}

func (i *Initializer) setTXChecksumOffload() error {
if i.disableTXChecksumOffload {
if err := ethtool.EthtoolTXHWCsumOff(i.hostGateway); err != nil {
return fmt.Errorf("error when disabling TX checksum offload on %s: %v", i.hostGateway, err)
}
klog.Warningf("Disabled TX checksum offload on %s", i.hostGateway)
}
return nil
}
4 changes: 4 additions & 0 deletions pkg/agent/agent_windows.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -512,3 +512,7 @@ func (i *Initializer) installVMInitialFlows() error {
func (i *Initializer) prepareL7EngineInterfaces() error {
return nil
}

func (i *Initializer) setTXChecksumOffload() error {
return nil
}
10 changes: 7 additions & 3 deletions pkg/config/agent/config.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -120,9 +120,13 @@ type AgentConfig struct {
// IPv4 and Linux Nodes, and can be enabled only when `ovsDatapathType` is `system`,
// `trafficEncapMode` is `noEncap`, and `noSNAT` is true.
EnableBridgingMode bool `yaml:"enableBridgingMode,omitempty"`
// Disable TX checksum offloading for container network interfaces. It's supposed to be set to true when the
// datapath doesn't support TX checksum offloading, which causes packets to be dropped due to bad checksum.
// It affects Pods running on Linux Nodes only.
// Disable TX checksum offloading for container network interfaces and the host gateway interface (default:
// antrea-gw0). It's supposed to be set to true when the datapath doesn't support TX checksum offloading,
// which causes packets to be dropped due to bad checksum.
// If this option is later set to false, for the host gateway interface, Antrea will not restore its
// original TX checksum state, as Antrea does not retain the original configuration. Users are responsible
// for manually reconfiguring the setting if needed.
// This option affects Linux Nodes only.
DisableTXChecksumOffload bool `yaml:"disableTXChecksumOffload,omitempty"`
// APIPort is the port for the antrea-agent APIServer to serve on.
// Defaults to 10350.
Expand Down

0 comments on commit e92e5ab

Please sign in to comment.