Goal: k8s maintenance

[consideRatio]

I would greatly value catching up with k8s operation maintenance. This would involve upgrading a lot of clusters and establishing documentation on the technical steps, as well as documentation on the community interactions leading up to an upgrade. This can involve figuring out planned maintenance windows or similar.

Initial catch up upgrades

helm and kubectl tooling:

• helm and kubectl version pinning policies, cd: upgrade helm 3.8 -> 3.11 and kubectl 1.23 -> 1.24 #2314

k8s clusters:

• AWS EKS, Document procedure and migrate existing AWS EKS based hubs from k8s 1.21+ to 1.24+ #2057
• GCP GKE, Migrate existing GCP GKE based hubs from k8s 1.22+ to 1.24+ #2157
• Azure AKS, Document procedure and migrate existing Azure AKS based hubs from k8s 1.22+ to 1.24+ #2158

Documentation of technical steps during upgrades

• AWS EKS, docs: add an aws k8s cluster upgrade guide #2142
• GCP GKE, Document a k8s upgrade procedure for GKE #3250
• Azure AKS, Document a k8s upgrade procedure for AKS #3251

Documentation of related community interactions

During ugprades of k8s clusters, we will cause a disruption that take a while. We should have some routines for how to communicate and such when planning a larger maintenance operation. This can involve having a clear guidance on how to get in contact with the community representatives among other things.

Notes

• After agreement with a community represenative on when I could perform disruptive maintenance, I added an announcement: leap: maintenance notice (basehub: fix override of template_vars) #2318. The community representative also communicated that: a) running servers would be forcefully stopped if running, b) service shouldn't be expected to be available during this time.

[consideRatio]

Closing as the umbrella issue is no longer relevant, but retaining #3250 and #3251 open