diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 469f28c50..ac25022b1 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -141,36 +141,7 @@ We integrate new features in the `master` branch. If you are an external contributor, you generally only care about that. However, if you are part of the OONI team, you also need to care about releasing. -In terms of branching, the release process is roughly the following: - -1. we use the [routine sprint releases template]( -https://github.com/ooni/probe/blob/master/.github/ISSUE_TEMPLATE/routine-sprint-releases.md) -to create an issue describing the activities bound to an -upcoming OONI Probe release; - -2. the first part of the procedure happens inside the `master` branch -until we reach a point where we tag an `alpha` release (e.g., `v3.21.0-alpha`); - -3. once we have tagged an `alpha` release, we create and push a branch -named `release/X.Y` (e.g., `release/3.21`); - -4. we commit to the `master` branch and bump the `internal/version/version.go` -version number to be the next `alpha` release, such that we can distinguish -measurements from the `master` branch taken after tagging the `alpha`; - -5. we finish preparing the release and eventually tag a stable release -(e.g., `v3.21.0`) inside the `release/X.Y` branch; - -6. we keep the `release/X.Y` around forever and we keep it as the -branching point from which to create patch releases (e.g., `v.3.21.1`). - -The `release/X.Y` branches run many more CI checks than the `master` branch -and this allows us to ensure that everything is in order for releasing. We run -fewer checks in the `master` branch to make the development process leaner. - -We prefer backporting from `master` to `release/X.Y` to forward porting from -a `release/X.Y` to `master`. When backporting, the commit name should start -with `[backport]` to identify it as a backporting commit. +See [docs/releasing.md](docs/releasing.md) for additional information. ## Releases diff --git a/docs/releasing.md b/docs/releasing.md new file mode 100644 index 000000000..6dd80160a --- /dev/null +++ b/docs/releasing.md @@ -0,0 +1,197 @@ +# Releasing OONI Probe CLI + +In terms of branching, the release process is roughly the following: + +1. we use the [routine sprint releases template]( +https://github.com/ooni/probe/blob/master/.github/ISSUE_TEMPLATE/routine-sprint-releases.md) +to create an issue describing the activities bound to an +upcoming OONI Probe release; + +2. the first part of the procedure happens inside the `master` branch +until we reach a point where we tag an `alpha` release (e.g., `v3.21.0-alpha`); + +3. once we have tagged an `alpha` release, we create and push a branch +named `release/X.Y` (e.g., `release/3.21`); + +4. we commit to the `master` branch and bump the `internal/version/version.go` +version number to be the next `alpha` release, such that we can distinguish +measurements from the `master` branch taken after tagging the `alpha`; + +5. we finish preparing the release and eventually tag a stable release +(e.g., `v3.21.0`) inside the `release/X.Y` branch; + +6. we keep the `release/X.Y` around forever and we keep it as the +branching point from which to create patch releases (e.g., `v.3.21.1`). + +The `release/X.Y` branches run many more CI checks than the `master` branch +and this allows us to ensure that everything is in order for releasing. We run +fewer checks in the `master` branch to make the development process leaner. + +We prefer backporting from `master` to `release/X.Y` to forward porting from +a `release/X.Y` to `master`. When backporting, the commit name should start +with `[backport]` to identify it as a backporting commit. + +The rest of this document discusses what you should do for each high-level +group of the [routine sprint releases template]( +https://github.com/ooni/probe/blob/master/.github/ISSUE_TEMPLATE/routine-sprint-releases.md). + +## Psiphon + +We need to precisely pin to the Psiphon dependency by using the latest +commit in the `staging-client` branch with this command: + +```bash +./script/go.bash get -u -v github.com/Psiphon-Labs/psiphon-tunnel-core@COMMIT +./script/go.bash mod tidy +./script/go.bash build -v ./... +./script/go.bash test ./... +``` + +Psiphon developers sometime use `replace` in their `go.mod` file. Because `replace` +only applies to the main module, we should probably ask to Psiphon developers what is +the impact of new `replace` directives on our integration. + +## Go version + +We MAY need to update `.github/workflows/gobash.yml` if a new minor version of Go is +available, to make sure the `./script/go.bash` tool is still able to build if run using +such a new minor version of Go. For example, I added `go1.22` after it was released. + +We also typically want to update the Go version we use. The general rule is that OONI +Probe SHOULD build for any version greater than this version. We indicate the expected +Go version inside the `GOVERSION` file. We try to use a stable, still supported Go +version that is compatible with our dependencies. + +After updating `GOVERSION`, we need to update our stdlib forks: + +- https://github.com/ooni/oocrypto +- https://github.com/ooni/oohttp + +When tagging `oocrypto` and `oohttp` you should bump the patch version number if you +are _updating_ from go1.X.Y to go1.X.Y+M and bump the minor version number if you are +_upgrading_ from go1.X.Z to go1.X+N.Y. For example, for `oohttp`, we bumped from +`v0.6.8` to `v0.7.0` when we upgraded from go1.20 to go1.21; we bumped from `v0.6.7` +to `v0.6.8` when we updated from go1.20.12 to go1.20.14. + +Once these two packages have been updated, use: + +```bash +./script/go.bash get -u -v github.com/ooni/oocrypto@TAG +./script/go.bash get -u -v github.com/ooni/oohttp@TAG +./script/go.bash mod tidy +./script/go.bash build -v ./... +./script/go.bash test ./... +``` + +to update `go.mod` and `go.sum`. + +Remember to also update mentions to the correct Go version in `Readme.md`. + +## Android + +We use `NDKVERSION` to track the expected version of the NDK to use. We should also +update this file to point to the latest stable version when releasing. + +We should also update `./MOBILE/android/ensure` to update the versions of the +`build-tools`, and `platforms` that we're using for building locally. + +## Dependencies other than Psiphon + +Ideally, one should be able to update all dependencies using: + +```bash +./script/go.bash get -u -v ./... +``` + +However, this MAY break the tree. For example, while writing this document, running +such a command would update to a version of uTLS that breaks Psiphon. + +A less aggressive approach is that of using https://github.com/icholy/gomajor to +list the dependencies needing updating and updating each of them manually by editing +`go.mod` and then running: + +```bash +./script/go.bash mod tidy +``` + +It's also important to update the C dependencies required to build `tor` for mobile. To this +end, just open the following files in `internal/cmd/buildtool/` and use the links to homebrew +to update the version number and the `SHA256SUM`: + +- `cdepslibevent.go` +- `cdepsopenssl.go` +- `cdepstor.go` +- `cdepszlib.go` + +Make sure you also update the corresponding tests in `internal/cmd/buildtool`. + +At the end, before committing and pusing, one should check that it's all good using: + +```bash +./script/go.bash build -v ./... +./script/go.bash test ./... +``` + +## Updating assets and definitions + +We save some measurements results locally to test the `./internal/minipipeline` package. We +regenerate these files before releasing using this command: + +```bash +./script/updateminipipeline.bash +``` + +We also run: + +```bash +./script/go.bash generate ./... +``` + +to update the bundled certs. + +Then, we follow the instructions in `./internal/model/http.go` to update the +`User-Agent` header that we're using when measuring. + +Finally, we need to bless a new https://github.com/ooni/probe-assets release +and integrate it as a dependency. To this end, one needs to follow this procedure: + +1. make sure https://github.com/ooni/historical-geoip has run and otherwise, if +the GitHub Action was paused, restart it to obtain a new database published using the +Internet Archive API on the Internet Archive cloud; + +2. obtain the link to the latest database build and its SHA256; + +3. follow https://github.com/ooni/probe-assets instructions in its README.md. + +As usual, before committing and pushing: + + +```bash +./script/go.bash build -v ./... +./script/go.bash test ./... +``` + +## Maintenance + +The check-list should be self explanatory. + +## QA and alpha releasing + +This stage is where we tag an `alpha` release on `master` and then create +the `release/X.Y` branch, as explained above. + +## Releasing proper + +This check-list stage should also be self explanatory. + +## Publishing stable packages + +We use `./script/autoexport.bash` in https://github.com/ooni/probe-engine to +export the latest engine to community members. Then we commit, push on master +and tag a new 0.Y.0 release. It's fine to use as target for the autoexport +the `alpha` tag or the final release tag. Alphas are pretty close to releases anyway. + +For publishing for Android and Debian, head out to the +https://github.com/ooni/probe-releases private repository +and follow the `README.md` instructions. +