[CPDEV-106078] - Control plane kubelet local mode kuber 1.31

[Imadzuma]

Description

Kubernetes version 1.31 provides kubelet local mode feature, that is needed for kubelet to do requests to local api-server ip instead of vrrp. ip. It helps to resolve the floating issue with kubernetes upgrade to 1.30 version. It's enabled in kubeadm config using featureGates.ControlPlaneKubeletLocalMode=true in ClusterConfiguration.

Fixes # (issue)

Solution

• Add featureGates.ControlPlaneKubeletLocalMode=true for install procedure with kubernetes version v1.31+. This feature can be turned off by user in cluster.yaml configuration;
• Set featureGates.ControlPlaneKubeletLocalMode=true for upgrade procedure to kubernetes version v1.31+ before the upgrade.
• Include featureGates section to claster.yaml lson schema;
• Add unittests for featureGates enrichment;
• Add doc mention about featureGates.ControlPlaneKubeletLocalMode;

Test Cases

TestCase 1

Test Configuration:

• Hardware:
• OS:
• Inventory: 1.31.0 kubernetes version

Steps:

1. Run kubemarine install procedure;
2. Check kubeadm-config in cluster;
3. Check kubelet configuration in /etc/kubernetes/kubelet.conf;

Results:

Before	After
No featureGates field in kubeadm-config	featureGates.ControlPlaneKubeletLocalMode=true in kubeadm-config
cluster name is specisied as server in kubelet config	local kubeapi-server endpoint is used in kubelet config

TestCase 2

Test Configuration:

• Hardware:
• OS:
• Inventory: 2 control plane nodes + 1 balancer node and 1.30.3 kubernetes version

Steps:

1. Run kubemarine install procedure;
2. Remove first control plane node IP from control plane haproxy configuration on balancer node (all kubeapi requests should go to second control-plane node) and restart haproxy;
3. Run kubemarine upgrade` procedure to 1.31 version;
4. Check kubeadm-config in cluster;
5. Check kubelet configuration in /etc/kubernetes/kubelet.conf;

Results:

Before	After
Upgrade procedure fails on first control-plane, because some pods have CreateContainerConfigError status	Successful upgrade to 1.31 version
No featureGates field in kubeadm-config	featureGates.ControlPlaneKubeletLocalMode=true in kubeadm-config
cluster name is specisied as server in kubelet config	local kubeapi-server endpoint is used in kubelet config

Checklist

• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• Integration CI passed
• Unit tests. If Yes list of new/changed tests with brief description
• There is no merge conflicts

Unit tests

Indicate new or changed unit tests and what they do, if any.