Arch Linux test runner

[klausenbusk]

Hi

It has been brought to my attention that you would like some test runners for Arch Linux and Debian.

We (Arch Linux) might be able to help and as part of Arch Linux's DevOps team, I would like some more details on what exactly you need.

How powerful must the runner be? Would a Hetzner CX11 VM be sufficient and what kind of software must the VM run? Zuul?

[debarshiray]

First of all, let me thank you deeply for responding to the call for help! It's much appreciated.

I suspect that we don't need a particularly powerful runner.

We are currently using an instance of Zuul CI called Software Factory for Fedora and CentOS Stream 9 runners, and GitHub Actions workflow for a Ubuntu 22.04 runner. Software Factory has CI runners (or nodes) with different hardware capabilities. We are using the ones called cloud-fedora-rawhide, cloud-fedora-39, etc., but I don't know their exact configuration off hand.

I suppose we have two ways to add Arch Linux to the Toolbx CI. We can either try to add Arch Linux images to Software Factory, which, I think, will then run on Software Factory's CI runners (or nodes), or we can try to add a runner for GitHub Actions workflows. Here are the existing image definitions known to Software Factory.

I am not an expert in either, so I am happy to leave it to your tastes and preferences. @danpawlik and @TristanCacqueray are my contacts for questions about Zuul CI and Software Factory. I hope they will correct me if I said anything wrong above.

[TristanCacqueray]

The Zuul CI system relies on cloud providers (like OpenStack/IBMCloud/AWS) to create ephemeral instances per build, so a dedicated VM would not be sufficient. We could either add a new cloud provider, or perhaps we have enough capacity to handle the new jobs. In any-case, we either need an existing image-id/ami, or a recipe to build the image that will be uploaded to the cloud prior to running job. As you can see in the link shared by @debarshiray , the image definition uses an existing qcow to perform minor modification for CI purpose.

Note that Zuul also support cloud native providers (like Kubernetes), but that's not practical to run toolbox test (which would need nested containerization).

[debarshiray]

/cc @Foxboron

[debarshiray]

Ping @klausenbusk

[klausenbusk]

Sorry for not getting back to you @debarshiray. Providing access to our cloud provider (Hetzner), so you can create ephemeral instances, is not a option due to cost, labor and security concerns. It is vastly different from providing a few static runners :)

So the only realistic option is adding a Arch Linux image to existing setup. We are already building "cloud images" and they can be downloaded from the mirrors (e.g. https://geo.mirror.pkgbuild.com/images/latest/).

Do you have the manpower to add the image to Software Factory? It is not exactly the same task as providing a server or two and I'm not sure how much work it entails.

[debarshiray]

Sorry for not getting back to you @debarshiray. Providing access to our cloud provider (Hetzner), so you can create ephemeral instances, is not a option due to cost, labor and security concerns. It is vastly different from providing a few static runners :)

Ok, understood. :)

So the only realistic option is adding a Arch Linux image to existing setup. We are already building "cloud images" and they can be downloaded from the mirrors (e.g. https://geo.mirror.pkgbuild.com/images/latest/).

These look useful to me. @danpawlik @TristanCacqueray what do you think?

Do you have the manpower to add the image to Software Factory? It is not exactly the same task as providing a server or two and I'm not sure how much work it entails.

I am not an expert in Zuul, but I can definitely make some time for this since I am the one who needs Arch Linux hosts for testing. I will probably end up asking a lot of questions as I find my way forward.

[debarshiray]

Ping @danpawlik @TristanCacqueray

[danpawlik]

@klausenbusk @debarshiray hi, I will talk with my team. So far, I don't see any issue to add Arch qcow2 image to our CI.

[danpawlik]

Added an image + label: https://softwarefactory-project.io/r/c/config/+/32385 . It needs to be merged to be available later.
Let's see what other will say for that idea. 🤞

[danpawlik]

So the change has been merged. One more is required, but you can test the arch label now.