Enable testing merge queues @ GitHub Actions CI/CD

[webknjaz]

This allows org-hosted projects to start enabling merge queues in the repository settings. With that, GitHub would trigger a separate event against a merge commit derived from merging several pull requests with the target branch.

Summary

$sbj. This is preparatory work for the possibility to enable the merge queues feature, but enabling it can be done through the branch protection settings. I'm starting to roll this out across my repos and I like it so far. If you decide to go ahead and enable it, merge this PR first. Also, uncheck the box saying “don't allow queuing failed PRs” ­— it's useful for cases, when several PRs fix CI failures if only merged together.

Pull Request Check List

• Added tests for changed code.
  Our CI fails if coverage is not 100%.
• New features have been added to our Hypothesis testing strategy.
• Changes or additions to public APIs are reflected in our type stubs (files ending in “.pyi`“.
  • ...and used in the stub test file tests/typing_example.py.
  • If they've been added to attr/__init__.pyi, they've also been re-imported in attrs/__init__.pyi.
• Updated documentation for changed code.
  • New functions/classes have to be added to docs/api.rst by hand.
  • Changes to the signature of @attr.s() have to be added by hand too.
  • Changed/added classes/methods/functions have appropriate versionadded, versionchanged, or deprecated directives.
    The next version is the second number in the current release + 1.
    The first number represents the current year.
    So if the current version on PyPI is 22.2.0, the next version is gonna be 22.3.0.
    If the next version is the first in the new year, it'll be 23.1.0.
• Documentation in .rst files is written using semantic newlines.
• Changes (and possible deprecations) have news fragments in changelog.d.
• Consider granting push permissions to the PR branch, so maintainers can fix minor issues themselves without pestering you.

[hynek]

why not

[hynek]

Also, uncheck the box saying “don't allow queuing failed PRs” ­— it's useful for cases, when several PRs fix CI failures if only merged together.

there is no such checkbox

[webknjaz]

@hynek I didn't remember the exact words when I was writing that, but it's this one:

[webknjaz]

@hynek I also noticed the RTD integration being required. It does not yet support merge queues. Document your interest here: readthedocs/readthedocs.org#10021.

For now, though, I think it's okay to remove RTD from required checks given that the strict docs build is happening in GHA too.

Let's try merging something soon to test if there's any other blockers. I detected a few in aiohttp, for example: https://github.com/orgs/aio-libs/discussions/27#discussioncomment-7396866. If there's a deal-breaker, you'll need to temporarily disable the feature.

[hynek]

I have removed RTD but why would I allow merges of failing PRs?

[webknjaz]

Sometimes, there's situations when the CI is broken in several places. Fixing each is a separate PR, but since part of the fixes are in different PRs, each of individual fixes wouldn't be mergeable. Though, when combined, it would result in a green CI. That checkbox stands in the way of achieving this, though.

The ideas behind this are coming from the concept of gating and are essential to having an evergreen main branch: https://gating.dev.

Adding a broken PR to the queue doesn't mean, it'll inevitably end up on the main branch. If the checks for the combined commit fail, GH will drop the PR from the queue.

[hynek]

I'm sorry but I don't get it (as in: lack imagination for the mechanics in practice). Can you point me to an example of that in practice? That gating.dev page is not very useful unless my ad blocker is hiding something – I knew that computers are good at running tests before. ;)

[webknjaz]

I'll try to come up with a better explanation. But have you seen this illustration
https://bors.tech/essay/2017/02/02/pitch/? Any maybe heard of https://zuul-ci.org? The latter implements this on the huge scale, coming from the OpenStack ecosystem.

[webknjaz]

The gating-based CIs (which the Merge Queues implement) make speculative merges of a bunch of changes into a temporary branch, test it, and if that ends up being green, they fast-forward the main branch to the same commit. This way, main gets exactly what was tested.
But individual PRs run CI at the time of creation/updates so their green or red status indicates the PR state as of time when the main branch might've looked different. With gating, you're guaranteed not to rely on the outdated check reports.

[hynek]

But have you seen this illustration
https://bors.tech/essay/2017/02/02/pitch/?

That sounds like stacked diffs? 🤔

[webknjaz]

Might be. But the main idea is that this is the workflow to prevent cases of untested code combos from being merged, causing unexpected CI statuses.

[hynek]

This is a concern I’ve never understood. I’ve set all my projects to that branches have to be up to date before merged. So this is basically a practical simplification of that?

[webknjaz]

Yep. You don't have to force rebasing/merging-in each PR with merge queues. So said setting doesn't make sense anymore, in this case.