Skip to content

Release process

Jump to bottom
Alyssa Dai edited this page Jan 12, 2024 · 13 revisions

Neurobagel Release process

For all tools, our goal is to have the main branch releasable at all times. That means:

  • all linters and static tools are passing
  • all tests are passing
  • all new features since last release have appropriate documentation (as part of PR)
  • the build is successful

Note: For features that take a lot of work to implement, having main always be releasable requires an extra step. For example a feature to "make this experiment reproducible" might take several PRs to implement. To still have main be in a releasable state, we will have to hide the in-progress feature behind a feature flag.

The goal is that we could release always, but we choose to release only when it makes sense. Related: the work to do an actual release should be minimal, and only involve the actual release process:

  • changelog editing
  • version increment picking
  • celebrating 🎉

It should not include any fixes, changes, or the writing of new documentation.

When to release

We will decide when to make a release as a team. The goal for releases is:

  • Release often - A release should not be a major thing that we only do every couple of months.
  • Release regularly - As much as we can, the releases should be at regular intervals. This may mean that different tools have different release intervals.
  • Release when it makes sense - The idea is that we only push released versions to our deployments. So not every merged PR will get deployed. But we will want to deploy and thus release whenever we fix an important bug or a new feature.

Auto-releasing using intuit/auto

Context

  • This is done using a workflow called release.yaml and the .autorc configuration file in each repo
    • Note that the workflow uses PATs (not GITHUB_TOKEN, which doesn't have sufficient permissions) that have, among other permissions, R+W permissions to both issues and pull requests

About PR labels

  • auto uses custom labels applied to PRs in a repo to determine when a release is made, which PRs are included in the release (and under what CHANGELOG heading), and to calculate the version bump for the release (major/minor/patch)
    • Each of these PR labels has one associated releaseType (defined in .autorc) that basically describes the version bump that should happen if a release were to be made containing only a PR with that label applied, and also the version bump if the label were paired with others (of potentially different releaseTypes) on the same PR
      • Possible label releaseTypes: "major", "minor", "patch", "skip", "none", "release" (note that these do not refer to the labels themselves, which can have different names)
      • Multiple labels can have the same releaseType
      • Labels with a releaseType of "major"/"minor"/"patch" are called SEMVER labels - these are used to calculate the final version bump for a release

Automatically creating the PR labels in a repo

Note: as long as the label properties don't change, this step only needs to be done once in the planning repo, since the labels will then be propagated to all other repos via our label syncing workflow.

  1. Install auto using binary on local machine
# replace the auto version if needed
curl -LJO https://github.com/intuit/auto/releases/download/v11.0.4/auto-linux.gz

gunzip auto-linux.gz ~/auto
chmod +x ~/auto

# check that it works
~/auto --help
  1. cd into the repo in which you want to create the labels
cd neurobagel/planning

  1. Ensure the .autorc file is available at the root of the repo

  2. Create a .env file in the repo root with the variable GH_TOKEN containing the value of a PAT with repo R+W access to at least issues and code (contents)

GH_TOKEN=value_of_your_pat_here

  1. Run auto create-labels from the repo root

  2. Check that the list of created labels corresponds to the ones in your .autorc

Our configuration

You can find our PR label configuration here: [ADD LINK WHEN READY]
The default PR label, if no labels are applied, is pr-patch (which also has "releaseType": "patch").

Our current auto configuration only releases when a PR with the release label applied has been merged.

When this happens, all PRs that have been merged since the last release are collected and auto looks at the PR label(s) on each to figure out:

  • if the PR should be included in the changelog
    • if a PR has a label of skip-release (has "releaseType": "skip") this always takes precedence over other SEMVER labels on the same PR
  • if so, what changelog heading it should go under (based "changelogTitle" label attribute in .autorc)

Finally, auto looks for the highest-priority releaseType among the included PRs based on their labels, and increments either the major, minor, or patch version accordingly.

Create a release

  1. To ensure that PRs since the last release end up under the appropriate heading in the changelog, when opening a PR, the author should always manually apply at least one PR label from our .autorc config (note: the pr-patch label is used by default behind the hood if no labels are applied, but we should aim to intentionally apply these to avoid having to modify changelog sections after a release)
    • For most changes, the author should choose the most relevant* label starting with the pr- prefix
    • If a PR should be excluded from the changelog (e.g., some small automation PR that is not user-relevant), the 'skip-release' label should be applied
  2. To auto-release after the PR you are working on is merged, apply the release label to the PR along with a relevant pr- prefixed label (note to double check: if no pr- labels are applied, the default one used should be pr-patch)
  3. To include extra release notes in the changelog, you can do so by modifying the PR description following these instructions: https://intuit.github.io/auto/docs/generated/changelog#additional-release-notes
  4. Once your release PR is merged, check that the release was created successfully with the expected tag + SEMVER (you can also double check that the release.yaml workflow in the repo succeeded), and double check the changelog

*Each PR included in the release will be assigned to a single changelog section based upon the applied label with the highest releaseType that has a changelogTitle. So, be careful when applying multiple pr- labels to the same PR - make sure that the label with the highest priority releaseType actually has the changelog section you want the PR to be listed under. See also https://intuit.github.io/auto/docs/configuration/autorc#changelog-titles)

Docs: https://intuit.github.io/auto/docs

Auto-pushing to DockerHub [WIP]

Once you have made a new release, a new GH workflow will start to:

  1. Build a new Docker image from the tagged commit
  2. Ensure that the build and test succeed (if not -> 🛑 -> 😱 -> 🛠️)
  3. Tag the new Docker image with two tags:
    • latest to show that it is the latest released version
    • <version> to show that it is the specified version
  4. Push the Docker image to docker hub
  5. (where applicable) -> push the new version to production
    • this may be a manual process

If we encounter a bug or notice things failing somewhere during this process, fixing this bug becomes the top priority for everyone. The fix for the bug becomes the next release and then gets deployed. While the fix is being worked on we will deploy :oldstable.

How to release manually (OLD)

At any given moment, the main branch is in a releasable state (see above). To actually make a release, we use the normal GitHub release workflow. Specifically:

  1. Click the "new Release" button
  2. Add a new "tag"
    • the tag has to follow semantic versioning: https://semver.org/ (see below)
    • check out the changelog (see below) to decide whether to make a patch, minor, or major version increase
  3. Add a title. The title has to match the "tag"
    • i.e. for a new release called "v0.2.4" the Release title would also be "v0.2.4"
  4. Use the "generate release notes" button to add the git log since the last release
    • This also adds a section about new contributors!
  5. Edit the changelog (see also the Example release notes template)
    • for minor and major versions (optional for patches) add a short description of the release on top of the changelog
    • e.g. "This release introduces new functionality for EEG data" or "This release introduces a breaking change in the XYZ workflow ..."
    • sort the changelog by type of change (i.e. a section with all of the [FIX] changes, one for all the [FEAT], and so on)
    • remove irrelevant changes (e.g. dependabot version bumps?)
  6. Store the release as a draft
  7. Discuss the ready-to-go release during standup
    • gives folks a chance to take a last look
    • possibility for feedback on the release notes
  8. Release if no objections 🎉

Example release notes template (OLD)

Summary

This release introduces X. This release includes a breaking change to Y.

What's Changed

(Aim to group changes of the same type/prefix)

New or improved features ✨

Changes to the data model for inputs/outputs ⚙️

Documentation updates 📜

Bug fixes 🛠️

Other changes 🧹

Clone this wiki locally