Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🐛 capd: fix ignition to also set the kube-proxy configuration to skip setting sysctls #9894

Merged
merged 1 commit into from
Dec 18, 2023
Merged

🐛 capd: fix ignition to also set the kube-proxy configuration to skip setting sysctls #9894

merged 1 commit into from
Dec 18, 2023

Conversation

chrischdi
Copy link
Member

What this PR does / why we need it:

While developing for k8s v1.29 support (#9872 and #9890) ignition tests started to fail because kube-proxy was not able to start due to the following error:

❯ k logs -n kube-system kube-proxy-brfk2
I1218 12:12:16.663929       1 server_others.go:72] "Using iptables proxy"
I1218 12:12:16.670800       1 server.go:1050] "Successfully retrieved node IP(s)" IPs=["172.18.0.5"]
I1218 12:12:16.671585       1 conntrack.go:118] "Set sysctl" entry="net/netfilter/nf_conntrack_max" value=196608
E1218 12:12:16.671607       1 server.go:556] "Error running ProxyServer" err="open /proc/sys/net/netfilter/nf_conntrack_max: permission denied"
E1218 12:12:16.671633       1 run.go:74] "command failed" err="open /proc/sys/net/netfilter/nf_conntrack_max: permission denied"

It turns out that we are adding custom kube-proxy configuration when using cloud-init in capd already for a long time. This did not happen for ignition.
With k8s v1.29, kube-proxy seems to have changed its behaviour so it wanted to set the sysctls when maxPerCore was set to nil.

This fixes the issue by adding the configuration similar to what we do for cloud-init in CAPD.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

/area provider/infrastructure-docker

@k8s-ci-robot k8s-ci-robot added the area/provider/infrastructure-docker Issues or PRs related to the docker infrastructure provider label Dec 18, 2023
@chrischdi
Copy link
Member Author

/test help

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Dec 18, 2023
@k8s-ci-robot
Copy link
Contributor

@chrischdi: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test pull-cluster-api-build-main
  • /test pull-cluster-api-e2e-full-dualstack-and-ipv6-main
  • /test pull-cluster-api-e2e-full-main
  • /test pull-cluster-api-e2e-main
  • /test pull-cluster-api-e2e-mink8s-main
  • /test pull-cluster-api-e2e-workload-upgrade-1-28-latest-main
  • /test pull-cluster-api-test-main
  • /test pull-cluster-api-test-mink8s-main
  • /test pull-cluster-api-verify-main

The following commands are available to trigger optional jobs:

  • /test pull-cluster-api-apidiff-main
  • /test pull-cluster-api-e2e-scale-main-experimental

Use /test all to run the following jobs that were automatically triggered:

  • pull-cluster-api-apidiff-main
  • pull-cluster-api-build-main
  • pull-cluster-api-e2e-main
  • pull-cluster-api-test-main
  • pull-cluster-api-verify-main

In response to this:

/test help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Dec 18, 2023
@chrischdi
Copy link
Member Author

/cherry-pick release-1.6

@k8s-infra-cherrypick-robot
Copy link

@chrischdi: once the present PR merges, I will cherry-pick it on top of release-1.6 in a new PR and assign it to you.

In response to this:

/cherry-pick release-1.6

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@chrischdi
Copy link
Member Author

/test help

@k8s-ci-robot
Copy link
Contributor

@chrischdi: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test pull-cluster-api-build-main
  • /test pull-cluster-api-e2e-full-dualstack-and-ipv6-main
  • /test pull-cluster-api-e2e-full-main
  • /test pull-cluster-api-e2e-main
  • /test pull-cluster-api-e2e-mink8s-main
  • /test pull-cluster-api-e2e-workload-upgrade-1-28-latest-main
  • /test pull-cluster-api-test-main
  • /test pull-cluster-api-test-mink8s-main
  • /test pull-cluster-api-verify-main

The following commands are available to trigger optional jobs:

  • /test pull-cluster-api-apidiff-main
  • /test pull-cluster-api-e2e-scale-main-experimental

Use /test all to run the following jobs that were automatically triggered:

  • pull-cluster-api-apidiff-main
  • pull-cluster-api-build-main
  • pull-cluster-api-e2e-main
  • pull-cluster-api-test-main
  • pull-cluster-api-verify-main

In response to this:

/test help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@chrischdi chrischdi changed the title capd: fix ignition to also set the kube-proxy configuration to skip setting sysctls 🐛 capd: fix ignition to also set the kube-proxy configuration to skip setting sysctls Dec 18, 2023
@chrischdi
Copy link
Member Author

/test pull-cluster-api-e2e-full-main

@chrischdi
Copy link
Member Author

/assign sbueringer

@chrischdi chrischdi force-pushed the pr-ignition-fix-kube-proxy branch from 7b99223 to 8ded744 Compare December 18, 2023 14:17
@chrischdi
Copy link
Member Author

/test pull-cluster-api-e2e-full-main

@sbueringer
Copy link
Member

/lgtm
/approve

Thank you very much!! Very nice troubleshooting!

/hold
feel free to merge

@k8s-ci-robot k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Dec 18, 2023
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: f1e7708ef8e2b06327ae8e7e8b179aee02c61488

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbueringer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 18, 2023
@chrischdi
Copy link
Member Author

Tested locally, looks good to merge after successful e2e.

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 18, 2023
@k8s-ci-robot k8s-ci-robot merged commit ec45bdb into kubernetes-sigs:main Dec 18, 2023
22 checks passed
@k8s-ci-robot k8s-ci-robot added this to the v1.7 milestone Dec 18, 2023
@k8s-infra-cherrypick-robot
Copy link

@chrischdi: new pull request created: #9895

In response to this:

/cherry-pick release-1.6

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-infra-cherrypick-robot k8s-infra-cherrypick-robot mentioned this pull request Dec 18, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sbueringer sbueringer sbueringer left review comments

@elmiko elmiko Awaiting requested review from elmiko

@JoelSpeed JoelSpeed Awaiting requested review from JoelSpeed

Assignees

@sbueringer sbueringer

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/provider/infrastructure-docker Issues or PRs related to the docker infrastructure provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Milestone
v1.7
Development

Successfully merging this pull request may close these issues.
4 participants
@chrischdi @k8s-ci-robot @k8s-infra-cherrypick-robot @sbueringer