An organization wide release pulse every X month?

[consideRatio]

@yuvipanda suggested that we could perhaps adopt a regular release cycle for Z2JH, so that we would cut a release every 2-3rd month for example. @yuvipanda quoted documentation from pip

Our release months are January, April, July, October. The release date within that month will be up to the release manager for that release. If there are no changes, then that release month is skipped and the next release will be 3 months later.

I think this sounds good, but want to consider adopting something similar across our organization's repos and have a "pulse" of releases that starts with leaf projects that doesn't depend on other projects and finishes with the distributions that depend on a lot of projects.

A release pulse

• A release pulse is a week long event.
• A pre-pulse week can optionally have taken place to nudge PRs to be merged before the release pulse, but the release pulse should not be blocked by PRs.
• A release pulse starts in our org's repos that doesn't depend on others, and finishes with repos that only depends on others, and we use a nice dependency graph for this!
• The release pulse finish with our distribution projects like tljh/z2jh at the end of the week.

A progressive adoption

• All projects must not be part of the release pulse, but I'd like to have easy managed z2jh dependencies like oauthenticator/kubespawner/configurable-http-proxy part of a pilot at least.
• We can plan release pulses every 2-3 months, but adjust the interval and phase initially

[choldgraf]

I'm a big fan of anything that:

1. Reduces the amount of human judgment time needed to make decisions
2. As long as it doesn't harm the stability or reputation of the tools we release (e.g. by releasing half-baked tools)

I'm not too worried about 2 here because it's quite common to release frequently in this world, so I'm +1 on this.

Maybe something to discuss at team meeting?

[consideRatio]

@choldgraf I've added an agenda item for this in the upcoming team meeting!

[yuvipanda]

My summary from the meeting is:

1. Let's try this out just for z2jh, just shipping what's in master in set times. We don't have extra maintainer time to try and get things 'to a particular state' before releases, so just do them point-in-time.
2. Don't explicitly sync other projects releases to this 'distribution release', but try to make releases more often.

@choldgraf suggested that we try and automate (1) as much as possible.

It this an accurate summary, @manics @minrk @sgibson91?

[minrk]

I believe so - I think there are some less defined tasks for point 2, toward keeping releases flowing on other repos.

Things discussed:

• automate releases as much as possible
  • most repos already publish to PyPI from CI via GitHub Actions, which means pushing a tag is all it takes. Any repos that don't do this yet can and should by adding this github action and a PYPI_TOKEN to github secrets.
  • @manics brought up automating this further with versioneer or setuptools scm so that we don't need to make and push a version-bump commit. For instance, repo2docker, which uses versioneer, can be released just by clicking a button on github to make a tag, with no local clone necessary.
  • @blink1073 mentioned tbump which is a bit like bump2version which doesn't eliminate the explicit bump & push step, but helps automate it.
  • (not mentioned, but bringing it up) using github-activity to generate changelogs helps summarize what changed - I'm still a fan of human-curated changelogs, but generating the full list of changes is a good baseline, and consolidates the inputs to any curated highlights to put in our own words, saving lots of work!
• surfacing information about what repos have unreleased work waiting to go. Long ago (I think on our old rackspace multipurpose server), I had this release-page app running to give a quick overview of repos with unreleased work. We could revive this or merely take inspiration. I suspect the github API can be used without maintaining clones like that one does, and this could perhaps even be a purely client-side app like all my pulls.

This is all related to our ongoing discussion of stale pull request reviews and issue triage, and a general challenge of staying on top of lots of repos. Making release is 'just another task' to stay on top of, with its own inputs and challenges.

[blink1073]

💯 to github_activity: jupyter/notebook#6014

[choldgraf]

I think the simplest thing to start with is make a GitHub action in the Team Compass that opens an "It's time to create a release 🎉" issue. This could either simply be a checklist of instructions to release Z2JH, or it could be something a bit more full of information, like lists of pull requests that need attention etc.

I just wrote something similar to this for the 2i2c weekly team syncs, do people think something like that could be useful for our purposes as well? (say, every 3 months or something, and focused around releases and such?) e.g. here is one update issue and it is generated by this gh action which runs this python file to grab lists of issues

One challenge with the "stale pull requests" situation is that you don't want those lists to become overwhelming. E.g., I think BinderHub has like 34 open PRs, so knowing where to start is a challenge.

[yuvipanda]

his could either simply be a checklist of instructions to release Z2JH

I like this, especially if it's as simple as possible. Maybe it'll add output from github activities, and make a full release when merged?

Releases take work separate from regular maintenance (merging PRs, etc), so I want us to decouple these as much as possible

[betatim]

Instead of some fancy GitHub Action to remind us would a recurring calendar appointment do?

I think the solution to "too many open PRs" is to pick one, anyone, by any method you like and work on that PR. I don't think displaying, categorising, automating and slicing&dicing them in a new way will help with moving open PRs forward.

[yuvipanda]

Agree re: PR work, but I want to separate that from automating releases. I think GitHub action opening a PR that you can hit 'merge' on to make a release will be very helpful in making releases. Making a release should be mostly independent of any currnetly open PRs - otherwise it becomes difficult to stick to timeframes for releases given limited maintainer time.

[manics]

Maybe we could do something with `workflow_dispatch, with the tag passed in as a parameter?
https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/

[consideRatio]

Towards action

Trying to drive this onward I'm delineating the following action points as first steps to:

• facilitate release work,
• at more regular time intervals,
• and help dependency projects to align releases in relation to distribution releases.

First steps

• To deploy or make release-page app logic results accessible in any way.

  surfacing information about what repos have unreleased work waiting to go. Long ago (I think on our old rackspace multipurpose server), I had this release-page app running to give a quick overview of repos with unreleased work. We could revive this or merely take inspiration. I suspect the github API can be used without maintaining clones like that one does, and this could perhaps even be a purely client-side app like all my pulls.

• Create a release pulse issue template focused on z2jh and dependencies initially in jupyterhub/team-compass
  • To be used to initializing a release pulse
  • Should summarize what releases needs to be cut and suggest a release day for each project
  • Should be an issue that is closed when the release pulse is complete

Relevant but not the focus of this issue

• Ensure RELEASE.md in projects are updated
• Ensure GitHub actions automate publishing of a tagged commit
• Setup release assistance tooling such as tbump that can be incorperated in the RELEASE.md instructions
• To make repos catch up on PR reviews

Future steps

• To perhaps align tljh releases with z2jh releases as the shared dependencies will release make sense to release in a similar time
• Start finding a release pulse rhythm, every 2 or every 3 months
• Add some suitable automation, such as giving us a reminder in a calendar or github issue to intitiate a release pulse
• To declare this process to provide transparency to users depending on z2jh/tljh
  • Ensure to make it clear this shouldn't exclude cutting additional releases in between

[sgibson91]

On a weakly-related note, I like the concept of "do-nothing scripting" as a prompt for humans to go through the correct steps of a manual process, e.g. making a release, as it lowers the activation energy for more people to take on the responsibility of carrying out that procedure. It also works as a skeleton to begin automating processes around, as you fill in the different sections to actually do the task instead of "do nothing". https://blog.danslimmon.com/2019/07/15/do-nothing-scripting-the-key-to-gradual-automation/

[minrk]

I love the do-nothing scripting idea!

[blink1073]

Jupyter Releaser has now been tested against jupyter_server, we'll make a discourse post and PYPI release next week.

[consideRatio]

We have not adopted a "release pulse" or similar, and I don't think we will want to coordinate this much around releases. But, we have also got a lot of projects into a state where making releases is easier by having automation and RELEASE.md notes where the notes often end up saying "write a changelog using github_activity and wait for approval/merge (see notes) and use tbump like this".

I suggest we go for a close here

[GeorgianaElena]

There's #394 that addresses this issue. Looking over it now, it looks it's more of a release reminder with some helpful links? Similar to these release planner tracking issues jupyterhub/zero-to-jupyterhub-k8s#3091

@consideRatio, do you there's value into updating #394 to have it as a release planner tracking issue that we could either trigger ourselves or have it as a cron job (or both), instead of opening them from scratch?

[consideRatio]

@consideRatio, do you there's value into updating #394 to have it as a release planner tracking issue that we could either trigger ourselves or have it as a cron job (or both), instead of opening them from scratch?

@GeorgianaElena I don't think so currently, but maybe pivoted to something similar. I'll mention the pivot idea later but I suggest #394 is closed to be re-opened if pivoting.

I currently (but not before) believe that a top-to-bottom approach to drive releases isn't going to work well. I believe the biggest challenge with a top-to-bottom / org-to-project approach is that we end up with a too large task.

I think we would benefit from pivoting towards helping individual projects get releases out more regularly by providing org-wide standards and other things that can be done org-wide to assist individual projects on the project level. I think the creation of jupyterhub/jupyterhub-python-repo-template was such work and has helped a lot already.

The repo template allowed us to define a RELEASE.md and a .github/workflows/release.yaml file that we could be adopted across projects. For some project it meant getting releases out via gitops for the first time, for others it meant that we got the procedure a bit streamlined via tbump. Overall the jupyterhub-python-repo-template helps us standardize release practices among other things, which helps individual projects get releases out easier.

Pivot idea

I see #394 as attempting to assist z2jh get a release out by providing overview. What if we try to do something similar for all projects, not making it z2jh specific?

I'm thinking that org-wide automation could perhaps help individual projects get an issue regularly updated with the state since last release. I'm thinking such issue would be similar to what #394 aimed to generate, for example latest release and date and opened and merged PRs since latest release. For comparison, here is a screenshot from #394: