The Best OS For Kubernetes · Mircea Anton

[giscus[bot]]

The Best OS For Kubernetes · Mircea Anton

Typically, when we deploy a Kubernetes cluster, we pick a generic OS, like RHEL or Ubuntu as our "base image", and then we start installing Kubernetes using tools like kubeadm, k3s or <insert-your-favorite-tool here>. While this is not inherently wrong, I'd say it's a relic of the past and not the best way to go moving forward. In this blog post I aim to explain why and to present a better alternative.

https://mirceanton.com/posts/2023-11-28-the-best-os-for-kubernetes/

[torches2004]

Thank you for this great document.
I believe there is a typo, the file created as patches/interface-names-and-dhcp.yaml should be vip.yaml as refrenced in the talosconfig command.

Where does the VIP IP reside ? is that a fourth host?

[mircea-pavel-anton]

Good catch! That's indeed a typo. I've fixed it now - the changes should be up in a bit.

As for the VIP, it's not residing on any host. Virtual IP addresses are IP addresses that are not tethered to particular machines. Thus, they can rotate among nodes. Basically, when the first node comes up, it takes ownership of that VIP, so if you look in the Talos dashboard for example, you will see that it reports having them both.
If something were to happen to that node and it becomes unresponsive, one of the other controlplane nodes will then take ownership of the VIP, thus ensuring HA for the cluster.

[torches2004]

why does the VIP and the other two nodes answer to the nginx URL running on one node? even a worker node I added answers to it. Is this normal behavior?

[mircea-pavel-anton]

So there are a couple individual questions to address here.

Why do the other two nodes answer to the nginx URL running on one node? Even a worker node I added answers to it

So apparently you are right. According to the official documentation:

If you set the type field to NodePort, the Kubernetes control plane allocates a port from a range specified by --service-node-port-range flag. Each node proxies that port (the same port number on every Node) into your Service.

I was under the impression that a NodePort service would only listen to the given port on the node on which the application was running, and I did not check the official docs. I also learned something new today! Good catch!

Why does the VIP answer to the nginx URL?

The VIP itself loadbalances all requests to the controlplane nodes. In this particular edge case, the controlplane nodes are also the workers, so the VIP acts as a loadbalancer for your workloads as well (by coincidence though). If you were to separate your nodes into dedicated controlplanes and dedicated worker nodes, this would no longer work as the VIP would not loadbalance your workers.