supported platform lists

[kroeckx]

I see that on a pull request we cross build for various platforms. I'm wondering if any of those qualify for our primary or secondary platform list. That all seem to be using Ubuntu.

I have access to all official ports of Debian and am willing to support them. That's currently: amd64, arm64, armel, armhf, i386, mips64el, mipsel, ppc64el and s390x. I most likely have access or can get access to all unofficial ports too.

[paulidale]

We cross build for these targets on PRs and execute via QEMU on push to the main repo.
The choice of Ubuntu was just one of expediency on my part.

Generally, IMO, the more platforms we test against the better.

[levitte]

Not sure what you heard there, @paulidale... me, I heard an offer to help build and test on some sort of regular basis.

[levitte]

That is to say, one doesn't preclude the other.

[paulidale]

I was simply clarifying what was currently done. Nothing more.

I do have one concern, out of these three:

• Build
• Test
• Fix

the commitment was only to the first two 😀

[kroeckx]

It's not very clear to me what we think is needed. Is just compiling in CI to be enough for secondary? Or does it also need to run the test suite? One of my questions was if they qualify for the lists, which comes to: - Do we just compile or also run the test suite for those that our in our current CI? - Our they running on our infrastructure?

[paulidale]

I think we do need to run the test suite in addition to just compiling to qualify something as a secondary platform. I am willing to leave the timing of the test suite runs somewhat flexible.

For the cross compiled platforms, we don't currently run the suite on every PR (it takes way way too long with qemu) but we do run them when merging. This means we've some coverage but I still wouldn't consider them secondary platforms. For that, we'd want real hardware rather than an emulation and more thought put into the process.

[levitte]

For the cross compiled platforms, we don't currently run the suite on every PR (it takes way way too long with qemu) but we do run them when merging. This means we've some coverage but I still wouldn't consider them secondary platforms. For that, we'd want real hardware rather than an emulation and more thought put into the process.

How do you plan on turning that into more than a recommendation (which should be self evident to most anyway)?

I rather think that we will have to accept that some secondary platforms will only build and test on merges, or even just once a day (because they're slow in today's measures, even non-emulated).

[mattcaswell]

Is just compiling in CI to be enough for secondary? Or does it also need to run the test suite?

From the definition of secondary:

A platform that is regularly tested through project CI on a system that is not owned or managed by the project. At least one project committer must have access to the system and be able and willing to support it.

So regular testing is the requirement. How regular is not specified (so it doesn't necessarily have to be on every push).

• Our they running on our infrastructure?

To qualify for secondary it needs to be integrated into the project CI (which will mean buildbot once its setup - but this is still being worked on). It does not need to be on project owned hardware.

[paulidale]

@levitte, I was deliberately flexible in the wording in the part you didn't quote. However, I believe that being a secondary platform means that it has to run on real hardware, not an emulation. That was the critical intent.

Yes, there is a grey area with VMs. I'm happy to accept that an XXX VM running on an XXX cpu is effectively equivalent to running on the XXX cpu directly. I'm not so happy about an XXX VM running on a YYY cpu being sufficient to claim XXX is supported.

[levitte]

I'm not so happy about an XXX VM running on a YYY cpu being sufficient to claim XXX is supported.

Er... why not? It's not like I want to do that, but as a matter of principal, I'm curious why not. How strict do you want to make this proposal (if that's what it is)? And if pretty strict, what do you want to do in case the choice is between XXX VM on YYY CPU or no support at all? Shall we have a procedure?

(re XXX VM on XXX cpu, I do not see a problem... at least with QEMU, that's usually not a CPU emulation, at least as far as I understand. Quite beside the point, in other words)

[paulidale]

(re XXX VM on XXX cpu, I do not see a problem... at least with QEMU, that's usually not a CPU emulation, at least as far as I understand. Quite beside the point, in other words)

I believe that this makes my point for me.

I'll accept emulation at a pinch but would much prefer not to.

[kroeckx]

It's my understanding that all those platforms that are tested on our infrastructure could be looked at as primary. But I really think not all of them should be primary, since we don't have actual access to real hardware. But maybe, if a committer wants to adopt it, it can be on the secondary list. This assumes the committer has access, is willing to support it, but doesn't actually run the CI.