Mini-RFC: Tagging GNOME collection for updating purposes

[jtojnar]

Issue description

GNOME makes a release twice a year, which includes the releases of packages outside of the gnome scope (such as glib or gtk4 or more). To run GNOME updates, we currently query packages based on update script:

nixpkgs/maintainers/scripts/update.nix

Line 166 in 45eb64f

% nix-shell maintainers/scripts/update.nix --arg predicate '(path: pkg: pkg.updateScript.name or null == "gnome-update-script")'

But this this does not find packages that do not use update script so it is a lot of manual work. Also, for point releases, we want to separate updates targetting staging, again requiring manual filtering.

Proposal

I propose we add passthru.updateCollections attribute, which will contain a list of strings. For our use case, we would use the values "gnome" and "staging".

Additionally, we would add support for this to maintainers/scripts/update.nix.

Alternatives

• Using RFC 146: meta.categories
  • That is more for user-facing categories meta.tags was not approved.
  • Since GNOME 47 is approaching quickly, I would go with an interim passthru.updateCollections for now and migrate to a global solution later, if one emerges.
• Having a attribute list somewhere in Nixpkgs
  • This would be faster to evaluate but that’s probably not an issue for updating.
  • It would be easy to forget to add stuff.
• Modifying maintainer teams infrastructure so that update.nix can detect lib.teams.gnome
  • Would not handle staging distinction.
  • Not clear how to do that in a backwards compatible way.

Not going through the RFC process since this would only affect GNOME packages and I feel like these kind of local changes are better to evolve organically.

cc @bobby285271 @getchoo @doronbehar

[doronbehar]

GNOME makes a release twice a year, which includes the releases of packages outside of the gnome scope (such as glib or gtk4 or more). To run GNOME updates, we currently query packages based on update script:

nixpkgs/maintainers/scripts/update.nix

Line 166 in 45eb64f

% nix-shell maintainers/scripts/update.nix --arg predicate '(path: pkg: pkg.updateScript.name or null == "gnome-update-script")'

So the goal is to be able to run nix-shell maintainers/scripts/update.nix (with a certain argument) upon the gnome packages that need to go through the staging -> staging-next -> master cycle, and then once that finishes to run nix-shell maintainers/scripts/update.nix again for the rest of the gnome packages?

[doronbehar]

I just encountered a similar problem when thinking of writing a certain automatic update related script - where I want to be able to know in advance whether an update should go through the staging branch, without running an ofborg evaluation locally (which is too resource demanding). The approaches I was thinking of are:

• Count how many direct reverse dependencies exist for the package.
• Lookup for previous PRs of the package and see if they targeted staging.
• Parse the commit log from the last update commit up to the current HEAD, and see if there was a staging-next related merge.

Related effort:

https://github.com/NixOS/GSoC/blob/main/ideas/2024.md#outpath-cache-for-faster-automation

[jtojnar]

So the goal is to be able to run nix-shell maintainers/scripts/update.nix (with a certain argument) upon the gnome packages that need to go through the staging -> staging-next -> master cycle, and then once that finishes to run nix-shell maintainers/scripts/update.nix again for the rest of the gnome packages?

Currently, for minor releases, I typically do the following:

1. Run update scripts for all GNOME packages.
2. Create two branches gnome and gnome-staging
3. For each of them, drop commits unsuitable for their target branch using git rebase --interactive. (Hoping I remember the where each package goes correctly.)
4. Open both PRs simultaneously. (Since for minor releases, lockstep update are not usually necessary except for maybe stuff like gnome-shell × mutter.)

[doronbehar]

I propose we add passthru.updateCollections attribute, which will contain a list of strings. For our use case, we would use the values "gnome" and "staging".

Maybe the belonging to the "gnome" collection is trivial to figure out, since the predicate you quoted already checks whether updateScript.name == "gnome-update-script". I think therefor that the only real information that would be so nice to have encoded in the expressions is whether a derivation change should go through staging or not. Hence I suggest a much simpler interface:

passthru = {
  updateScript = gnome.updateScript {
    packageName = "...";
    attrPath = "...";
  };
  updateTargetBranch = "staging";
};

It'd also be very useful for users that run their own update scripts.

[jtojnar]

The issue is that not all packages we consider part of gnome collection use the update script for one reason or another. This was the primary thing I wanted to address since filtering commits during rebase is not that hard, compared to gathering all the packages for update in the first place.

It might be similarly useful for e.g. Budgie, which uses nix-update-script.

[doronbehar]

The issue is that not all packages we consider part of gnome collection use the update script for one reason or another.

And you still want to be able to update them in the same nix-shell maintainers/scripts/update.nix command when a Gnome version is released? Because they would belong to the "gnome" update collection? You could sort of "add them to this 'collection'" by giving them an updateScript attribute with a .name == "gnome-update-script".

Another option would be indeed to add an additional attribute to the updateScript (perhaps name it collection (not plural)), and use that attribute for the update.nix predicate.

In either case I'd separate the staging/master target branch topic, from the "collection(s)" topic, since the former can be easily grasped by beginner Nixpkgs contributors without delving into the complexity of the Nixpkgs packages collections such as Gnome and Budgie, that is related to the later topic.