Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: make kind-with-registry.sh work with podman #3731

Closed
wants to merge 1 commit into from
Closed

fix: make kind-with-registry.sh work with podman #3731

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion site/static/examples/kind-with-registry.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ EOF
# We want a consistent name that works from both ends, so we tell containerd to
# alias localhost:${reg_port} to the registry container when pulling images
REGISTRY_DIR="/etc/containerd/certs.d/localhost:${reg_port}"
for node in $(kind get nodes); do
for node in $(kubectl get nodes -o jsonpath='{.items[*].metadata.name}'); do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't the right fix to make kind get nodes work with podman?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

podman works with kind get nodes, it logs the experimental disclaimer ... podman is not stable enough and still lacks some important capabilities #1778

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's some small discussion on the linked issue that highlights why the current kind get nodes output is a challenge with podman.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wbrefvem I didn't realize, but the script does not rely on kubectl commands, since you can not guarantee you are targeting the right cluster

Since all kind nodes have a -control-plane or -worker suffix we may use that inside the loop to skip those, WDYT @BenTheElder

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, my bad again, it does rely on kubectl at the end of the script ... nevermind, let me reassing this to @BenTheElder , he knows better this script

/assign @BenTheElder

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be stderr, which should not be captured:

kind/pkg/internal/runtime/runtime.go

Line 16 in 0483056

logger.Warn("using podman due to KIND_EXPERIMENTAL_PROVIDER")

kind/pkg/cmd/logger.go

Lines 31 to 36 in 0483056

func NewLogger() log.Logger {
var writer io.Writer = os.Stderr
if env.IsSmartTerminal(writer) {
writer = cli.NewSpinner(writer)
}
return cli.NewLogger(writer, 0)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kind/pkg/cluster/internal/providers/podman/provider.go

Line 45 in 0483056

logger.Warn("enabling experimental podman provider")

Copy link
Member

@BenTheElder BenTheElder Sep 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

$ (out="$(KIND_EXPERIMENTAL_PROVIDER=podman kind get nodes || true)"; echo "out: $out")
using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
ERROR: failed to list nodes: command "podman ps -a --filter label=io.x-k8s.kind.cluster=kind --format '{{.Names}}'" failed with error: exec: "podman": executable file not found in $PATH
Command Output: 
out:

Are we sure the script is broken?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure the script is broken?

For podman running on macOS, yes, but I may have misunderstood the reason. Here's what I'm seeing when the script starts to docker exec the nodes:

using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
Error: can only create exec sessions on running containers: container state improper

@BenTheElder you're right that the two lines about podman are not going to stdout. It must be a race condition, and since my solution involves an API call, it's slow enough to lose the race.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, we really should be safe to exec once kind create cluster succeeds ... uhhh

Did the container crash or something?

Let's follow up more on the issue.

docker exec "${node}" mkdir -p "${REGISTRY_DIR}"
cat <<EOF | docker exec -i "${node}" cp /dev/stdin "${REGISTRY_DIR}/hosts.toml"
[host."http://${reg_name}:5000"]
Expand Down