Use no-content manifests to decide if rebuilds are needed

[achilleas-k]

EXPERIMENTAL PR FOR CI SHENANIGANS

The idea behind this is that we can generate manifests without resolving content and be reasonably certain¹ that any changes in these manifests reflect changes in the real ones.

¹ This is based on the assumption that the content doesn't change if the address is the same. When gen-manifests is asked to not resolve content, --packages=false, --commits=false, and/or --containers=false, instead of depsolving, fetching the commit ID, or resolving the container ID, it generates a checksum of each item based on the provided info (name, url, etc). This is a safe alternative under certain conditions:

• rpms: In our tests we use rpm repo snapshots. That means that for a given timestamped URL, the content will never change. This isn't true in the general case. If we ever change a test repo to point to CDNs for example, or anything that's not a pinned snapshot, then this assumption falls apart.
• commits: For our tests, any build that requires an ostree commit uses an ostree-container built by the manifest at the same commit. It's not really important what these images point to for this check: if their manifest changes, but the corresponding ostree-container doesn't, it means that something changed in the definition of the dependent manifest (e.g., iot-qcow2 or iot-installer), so a rebuild will be needed. If a dependency changes (e.g., iot-container), we would start a rebuild of that, and then automatically also rebuild any dependants, since the content hash in their manifests will change in the real case.
• containers: This is a bit stranger for cases where we embed containers. Right now we use a couple of containers that we keep pinned in the GitLab registry. If we change one of those, this new method will never detect the change because it doesn't try to resolve them. We could just enable container resolves (it probably wont be that slow), but also we almost never change those and when we do, we can just push them with a new tag or name and update the test configs.

Note here that in the current example there's no CI cache to compare against. Instead we just generate manifests on both the tip of the branch (HEAD of the PR) and the merge-base with the base ref (i.e. git merge-base HEAD main). Right now, with the CI cache, we have a file lifetime that sort of ensures that images will be rebuilt even if nothing changes after a certain time (10 days currently). Realistically, changes always happen much more frequently, but if manifests don't change for a while for whatever reason, they wont be rebuilt and the cache will expire. Whether this is a problem or not depends on whether we intend to reuse the cache for other things (like tests in osbuild-composer).

---

Generate manifests without content to decide if we should trigger builds
in GitLab CI.

• Generate manifests without content in /manifests/HEAD
• Switch to the merge-base with the base branch
• Generate manifests without content in /manifests/merge-base
• Diff the directories

If there are no differences exit with 0, otherwise print a message.

The real version of this job will write a file or set a variable to make
the gitlab trigger skip the gitlab CI if no builds are needed.

Alternatively, it can create the top-level gitlab CI file so that we
don't need to have two nested dynamic pipelines in gitlab. It can
probably even generate the whole low level one so we don't need to have
dynamic pipelines there and it would make thing a tiny bit saner.

It should also take into account the other things that affect builds:

• osbuild version in Schutzfile
• Seed arg (load it into env when generating manifests)
• bib container ID

[github-actions]

This PR is stale because it has been open 30 days with no activity. Remove "Stale" label or comment or this will be closed in 7 days.

[github-actions]

This PR is stale because it has been open 30 days with no activity. Remove "Stale" label or comment or this will be closed in 7 days.

[github-actions]

This PR was closed because it has been stalled for 30+7 days with no activity.