Skip to content

Commit

Permalink
Only bind to IPv6 addresses when disableIPv6=false (#12938)
Browse files Browse the repository at this point in the history
## Problem

When the IPv6 stack in Linux is disabled, the proxy will crash at startup.

## Repro

In a Linux machine, disable IPv6 networking through the `net.ipv6.conf.*` sysctl kernel tunables, and restart the system:

- In /etc/sysctl.conf add:
```
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
```

- In /etc/default/grub set:
```
GRUB_CMDLINE_LINUX="ipv6.disable=1"
```

- Don't forget to update grub before rebooting:
```
sudo update-grub
```

In a default k3d cluster, install Linkerd. You should see the following error in any proxy log:

```
thread 'main' panicked at /__w/linkerd2-proxy/linkerd2-proxy/linkerd/app/src/lib.rs:245:14:
Failed to bind inbound listener: Os { code: 97, kind: Uncategorized, message: "Address family not supported by protocol" }
```

## Cause

Even if a k8s cluster didn't support IPv6, we were counting on the nodes having an IPv6 stack, which allowed us to bind to the inbound proxy to [::] (although not to [::1] for the outbound proxy, as seen in GKE). This was the case in the major cloud providers we tested, but it turns out there are folks running nodes with IPv6 disabled and so we have to cater that case as well.

## Fix

The current change undoes some of the changes from 7cbe2f5 (for the proxy config), 7cbe2f5 (for the policy controller) and 6603409 (for linkerd-cni), binding back again to 0.0.0.0 unless `disableIPv6` is false.
  • Loading branch information
alpeb committed Aug 5, 2024
1 parent 58a4e67 commit 332c4ef
Show file tree
Hide file tree
Showing 60 changed files with 383 additions and 376 deletions.
2 changes: 1 addition & 1 deletion charts/linkerd-control-plane/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ Kubernetes: `>=1.22.0-0`
| linkerdVersion | string | `"linkerdVersionValue"` | control plane version. See Proxy section for proxy version |
| networkValidator.connectAddr | string | `""` | Address to which the network-validator will attempt to connect. This should be an IP that the cluster is expected to be able to reach but a port it should not, e.g., a public IP for public clusters and a private IP for air-gapped clusters with a port like 20001. If empty, defaults to 1.1.1.1:20001 and [fd00::1]:20001 for IPv4 and IPv6 respectively. |
| networkValidator.enableSecurityContext | bool | `true` | Include a securityContext in the network-validator pod spec |
| networkValidator.listenAddr | string | `"[::]:4140"` | Address to which network-validator listens to requests from itself |
| networkValidator.listenAddr | string | `""` | Address to which network-validator listens to requests from itself. If empty, defaults to 0.0.0.0:4140 and [::]:4140 for IPv4 and IPv6 respectively. |
| networkValidator.logFormat | string | plain | Log format (`plain` or `json`) for network-validator |
| networkValidator.logLevel | string | debug | Log level for the network-validator |
| networkValidator.timeout | string | `"10s"` | Timeout before network-validator fails to validate the pod's network connectivity |
Expand Down
6 changes: 3 additions & 3 deletions charts/linkerd-control-plane/templates/destination.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -315,10 +315,10 @@ spec:
name: sp-tls
readOnly: true
- args:
- --admin-addr=[::]:9990
- --admin-addr={{ if .Values.disableIPv6 }}0.0.0.0{{ else }}[::]{{ end }}:9990
- --control-plane-namespace={{.Release.Namespace}}
- --grpc-addr=[::]:8090
- --server-addr=[::]:9443
- --grpc-addr={{ if .Values.disableIPv6 }}0.0.0.0{{ else }}[::]{{ end }}:8090
- --server-addr={{ if .Values.disableIPv6 }}0.0.0.0{{ else }}[::]{{ end }}:9443
- --server-tls-key=/var/run/linkerd/tls/tls.key
- --server-tls-certs=/var/run/linkerd/tls/tls.crt
- --cluster-networks={{.Values.clusterNetworks}}
Expand Down
5 changes: 3 additions & 2 deletions charts/linkerd-control-plane/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -328,8 +328,9 @@ networkValidator:
# for public clusters and a private IP for air-gapped clusters with a port like 20001.
# If empty, defaults to 1.1.1.1:20001 and [fd00::1]:20001 for IPv4 and IPv6 respectively.
connectAddr: ""
# -- Address to which network-validator listens to requests from itself
listenAddr: "[::]:4140"
# -- Address to which network-validator listens to requests from itself.
# If empty, defaults to 0.0.0.0:4140 and [::]:4140 for IPv4 and IPv6 respectively.
listenAddr: ""
# -- Timeout before network-validator fails to validate the pod's network connectivity
timeout: "10s"
# -- Include a securityContext in the network-validator pod spec
Expand Down
2 changes: 1 addition & 1 deletion charts/linkerd2-cni/templates/cni-plugin.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -313,7 +313,7 @@ spec:
- /usr/lib/linkerd/linkerd-cni-repair-controller
args:
- --admin-addr
- "[::]:9990"
- "{{ if .Values.disableIPv6 }}0.0.0.0{{ else }}[::]{{ end }}:9990"
- --log-format
- {{ .Values.repairController.logFormat }}
- --log-level
Expand Down
6 changes: 6 additions & 0 deletions charts/partials/templates/_network-validator.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,13 @@ args:
- "[fd00::1]:20001"
{{- end }}
- --listen-addr
{{- if .Values.networkValidator.listenAddr }}
- {{ .Values.networkValidator.listenAddr | quote }}
{{- else if .Values.disableIPv6}}
- "0.0.0.0:4140"
{{- else }}
- "[::]:4140"
{{- end }}
- --timeout
- {{ .Values.networkValidator.timeout }}

Expand Down
6 changes: 3 additions & 3 deletions charts/partials/templates/_proxy.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -81,16 +81,16 @@ env:
value: "365d"
{{ end -}}
- name: LINKERD2_PROXY_CONTROL_LISTEN_ADDR
value: "[::]:{{.Values.proxy.ports.control}}"
value: "{{ if .Values.disableIPv6 }}0.0.0.0{{ else }}[::]{{ end }}:{{.Values.proxy.ports.control}}"
- name: LINKERD2_PROXY_ADMIN_LISTEN_ADDR
value: "[::]:{{.Values.proxy.ports.admin}}"
value: "{{ if .Values.disableIPv6 }}0.0.0.0{{ else }}[::]{{ end }}:{{.Values.proxy.ports.admin}}"
{{- /* Deprecated, superseded by LINKERD2_PROXY_OUTBOUND_LISTEN_ADDRS since proxy's v2.228.0 (deployed since edge-24.4.5) */}}
- name: LINKERD2_PROXY_OUTBOUND_LISTEN_ADDR
value: "127.0.0.1:{{.Values.proxy.ports.outbound}}"
- name: LINKERD2_PROXY_OUTBOUND_LISTEN_ADDRS
value: "127.0.0.1:{{.Values.proxy.ports.outbound}}{{ if not .Values.disableIPv6}},[::1]:{{.Values.proxy.ports.outbound}}{{ end }}"
- name: LINKERD2_PROXY_INBOUND_LISTEN_ADDR
value: "[::]:{{.Values.proxy.ports.inbound}}"
value: "{{ if .Values.disableIPv6 }}0.0.0.0{{ else }}[::]{{ end }}:{{.Values.proxy.ports.inbound}}"
- name: LINKERD2_PROXY_INBOUND_IPS
valueFrom:
fieldRef:
Expand Down
6 changes: 3 additions & 3 deletions cli/cmd/testdata/inject-filepath/expected/injected_nginx.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -66,15 +66,15 @@ spec:
- name: LINKERD2_PROXY_INBOUND_DISCOVERY_IDLE_TIMEOUT
value: 90s
- name: LINKERD2_PROXY_CONTROL_LISTEN_ADDR
value: '[::]:4190'
value: 0.0.0.0:4190
- name: LINKERD2_PROXY_ADMIN_LISTEN_ADDR
value: '[::]:4191'
value: 0.0.0.0:4191
- name: LINKERD2_PROXY_OUTBOUND_LISTEN_ADDR
value: 127.0.0.1:4140
- name: LINKERD2_PROXY_OUTBOUND_LISTEN_ADDRS
value: 127.0.0.1:4140
- name: LINKERD2_PROXY_INBOUND_LISTEN_ADDR
value: '[::]:4143'
value: 0.0.0.0:4143
- name: LINKERD2_PROXY_INBOUND_IPS
valueFrom:
fieldRef:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,15 +66,15 @@ spec:
- name: LINKERD2_PROXY_INBOUND_DISCOVERY_IDLE_TIMEOUT
value: 90s
- name: LINKERD2_PROXY_CONTROL_LISTEN_ADDR
value: '[::]:4190'
value: 0.0.0.0:4190
- name: LINKERD2_PROXY_ADMIN_LISTEN_ADDR
value: '[::]:4191'
value: 0.0.0.0:4191
- name: LINKERD2_PROXY_OUTBOUND_LISTEN_ADDR
value: 127.0.0.1:4140
- name: LINKERD2_PROXY_OUTBOUND_LISTEN_ADDRS
value: 127.0.0.1:4140
- name: LINKERD2_PROXY_INBOUND_LISTEN_ADDR
value: '[::]:4143'
value: 0.0.0.0:4143
- name: LINKERD2_PROXY_INBOUND_IPS
valueFrom:
fieldRef:
Expand Down Expand Up @@ -296,15 +296,15 @@ spec:
- name: LINKERD2_PROXY_INBOUND_DISCOVERY_IDLE_TIMEOUT
value: 90s
- name: LINKERD2_PROXY_CONTROL_LISTEN_ADDR
value: '[::]:4190'
value: 0.0.0.0:4190
- name: LINKERD2_PROXY_ADMIN_LISTEN_ADDR
value: '[::]:4191'
value: 0.0.0.0:4191
- name: LINKERD2_PROXY_OUTBOUND_LISTEN_ADDR
value: 127.0.0.1:4140
- name: LINKERD2_PROXY_OUTBOUND_LISTEN_ADDRS
value: 127.0.0.1:4140
- name: LINKERD2_PROXY_INBOUND_LISTEN_ADDR
value: '[::]:4143'
value: 0.0.0.0:4143
- name: LINKERD2_PROXY_INBOUND_IPS
valueFrom:
fieldRef:
Expand Down
6 changes: 3 additions & 3 deletions cli/cmd/testdata/inject-filepath/expected/injected_redis.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -66,15 +66,15 @@ spec:
- name: LINKERD2_PROXY_INBOUND_DISCOVERY_IDLE_TIMEOUT
value: 90s
- name: LINKERD2_PROXY_CONTROL_LISTEN_ADDR
value: '[::]:4190'
value: 0.0.0.0:4190
- name: LINKERD2_PROXY_ADMIN_LISTEN_ADDR
value: '[::]:4191'
value: 0.0.0.0:4191
- name: LINKERD2_PROXY_OUTBOUND_LISTEN_ADDR
value: 127.0.0.1:4140
- name: LINKERD2_PROXY_OUTBOUND_LISTEN_ADDRS
value: 127.0.0.1:4140
- name: LINKERD2_PROXY_INBOUND_LISTEN_ADDR
value: '[::]:4143'
value: 0.0.0.0:4143
- name: LINKERD2_PROXY_INBOUND_IPS
valueFrom:
fieldRef:
Expand Down
6 changes: 3 additions & 3 deletions cli/cmd/testdata/inject_contour.golden.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 12 additions & 12 deletions cli/cmd/testdata/inject_emojivoto_already_injected.golden.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions cli/cmd/testdata/inject_emojivoto_deployment.golden.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 332c4ef

Please sign in to comment.