kairos upgrade via kubernetes

[cristian98149]

Hi all,
I see that performing kairos upgrade via kubernetes consists in one/more POD/s that:

1. Downloads target image (that must be public)
2. Executes /usr/sbin/suc-upgrade script that checks diff between /etc/os-release and /host/ect/os-release

My question is: If I have installed a custom image with v1.0 and I need to upgrade to my custom image with v2.0 that doesn't change OS but just some customization (apps, configs, etc), should upgrade work correctly?

I've tried it in these ways:
TEST 1

• Start image: Custom image (v1.0) based on debian v2.0.0-rc5-k3sv1.26.3+k3s1
• Target image: Official Debian image from quay.io (latest tag)
• RESULT: POD for upgrade created, target image downloaded, nothing happens. The system doesn't reboot

TEST 2

• Start image: Custom image (v1.0) based on debian v2.0.0-rc5-k3sv1.26.3+k3s1
• Target image: Official Ubuntu image from quay.io (latest tag)
• RESULT: POD for upgrade created, target image downloaded, after a while the system reboots and apply new OS.

Maybe, as /etc/os-release file doesn't contain information about custom image (trivially image tag), in case of upgrade to another custom image with the same OS-release, the upgrade doesn't happen. Could be this reason valid?

/etc/os-release file

kairos@node:~$` cat /etc/os-release
NAME="kairos-debian"
VERSION="v2.0.0-rc5-k3sv1.26.3+k3s1"
ID="kairos"
ID_LIKE="kairos-debian"
VERSION_ID="v2.0.0-rc5-k3sv1.26.3+k3s1"
PRETTY_NAME="kairos-debian v2.0.0-rc5-k3sv1.26.3+k3s1"
ANSI_COLOR="0;32"
BUG_REPORT_URL="https://github.com/kairos-io/kairos/issues/new/choose"
HOME_URL="https://github.com/kairos-io/provider-kairos"
IMAGE_REPO="quay.io/kairos/kairos-debian"
IMAGE_LABEL="latest"
GITHUB_REPO="kairos-io/provider-kairos"
VARIANT="core"
FLAVOR="opensuse-leap"kairos

[jimmykarily]

@cristian98149 sorry for the late reply. Did you ever find a fix? Maybe there are logs in upgrade Pod that would help understand why it doesn't work?

[PaSharks9]

I'm facing the same problem with the same use case.

I'm trying to upgrade my custom image based on quay.io/kairos/kairos-ubuntu-20-lts:v2.2.2-k3sv1.25.10-k3s1 to another version of my image where only the customization changes and actually happens like you describe in TEST 1 .
Another problem in my case is that if i try to apply again the Plan config file to my k3s single-node cluster, nothing happens, no more pods are created. This let me think that maybe the "upgrade process" is somehow stuck somewhere.

Edit 1:
In my use case, my images are inside a private registry.
To pull my images from a private registry i had to add to the system-upgrade-controller Service Account the field imagePullSecrets in order to let the Plan pod to use the Secret that contains my auth credentials (https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#add-imagepullsecrets-to-a-service-account).

Edit 2:
I also watched the logs of the Plan Pod and they say that the new image is pulled succesfully, after that start the upgrade container succesfully and nothing more, after a while the Plan pod and the relative Job.Batch pod disappear

[cristian98149]

Hi all, I've recently tested again this type of upgrade (same base image) and it worked.

@PaSharks9 can I sugget you to add a custom field into /etc/os-release which its value changes between custom images and retry the upgrade?

Thank you.

[PaSharks9]

@cristian98149 It worked! Thank you very much!

[jimmykarily]

Any key makes this work? So there is a check somewhere that os-release file should not be unchanged between upgrades? 🤔

[PaSharks9]

@jimmykarily from what I understand, in Plan's Pod are compared the /etc/os-release files of the currently installed OS and the version to which you want to upgrade. If there are no differences then the upgrade is not performed, if there are differences then the upgrade is performed.

Since in my case the upgrade involves modifying only some of my customizations (and therefore not the source OS, in my case kairos-ubuntu-20-lts:v2.2.2-k3sv1.25.10-k3s1), following @cristian98149 's advice I added a custom field inside /etc/os-release so that it would identify the versions of the Dockerimages I would like to use.

What I'm getting at is that the current mechanism is implemented to check if the update only and exclusively affects the OS

[jimmykarily]

You are right: https://github.com/kairos-io/packages/blob/821de2dded0c2f590b539261002c5d257fb8ea07/packages/system/suc-upgrade/suc-upgrade.sh#L6