Skip to content

Commit 7900b4d

Browse files
alambalamb
authored
Update object_store release documentation (#6565)
* Update object_store release documentation * Update object store * prettier * tweak * Update object_store/dev/release/README.md * update instructions * fix * Update object_store/dev/release/README.md
1 parent 280ac21 commit 7900b4d

File tree

3 files changed

+219
-85
lines changed

3 files changed

+219
-85
lines changed

‎dev/release/README.md

Lines changed: 13 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -21,29 +21,24 @@
2121

2222
## Overview
2323

24-
This file documents the release process for:
25-
26-
1. The "Rust Arrow Crates": `arrow`, `arrow-flight`, `parquet`, and `parquet-derive`.
27-
2. The `object_store` crate.
24+
This file documents the release process for the "Rust Arrow Crates": `arrow`, `arrow-flight`, `parquet`, and `parquet-derive`.
2825

2926
### The Rust Arrow Crates
3027

31-
The Rust Arrow Crates are interconnected (e.g. `parquet` has an optional dependency on `arrow`) so we increment and release all of them together. We try to release a new version of "Rust Arrow Crates" every two weeks. This cadence balances getting new features into the community without overwhelming downstream projects with too frequent changes or overly burdening maintainers.
32-
33-
If any code has been merged to master that has a breaking API change, as defined in [Rust RFC 1105](https://github.com/rust-lang/rfcs/blob/master/text/1105-api-evolution.md), the major version number incremented changed (e.g. `9.0.2` to `9.0.2`). Otherwise the new minor version incremented (e.g. `9.0.2` to `7.1.0`).
34-
35-
### `object_store` crate
28+
The Rust Arrow Crates are interconnected (e.g. `parquet` has an optional dependency on `arrow`) so we increment and release all of them together.
3629

37-
At the time of writing, we release a new version of `object_store` on demand rather than on a regular schedule.
30+
If any code has been merged to master that has a breaking API change, as defined
31+
in [Rust RFC 1105] he major version number is incremented (e.g. `9.0.2` to `10.0.2`).
32+
Otherwise the new minor version incremented (e.g. `9.0.2` to `9.1.0`).
3833

39-
As we are still in an early phase, we use the 0.x version scheme. If any code has been merged to master that has a breaking API change, as defined in [Rust RFC 1105](https://github.com/rust-lang/rfcs/blob/master/text/1105-api-evolution.md), the minor version number incremented changed (e.g. `0.3.0` to `0.4.0`). Otherwise the patch version is incremented (e.g. `0.3.0` to `0.3.1`).
34+
[rust rfc 1105]: https://github.com/rust-lang/rfcs/blob/master/text/1105-api-evolution.md
4035

4136
# Release Mechanics
4237

4338
## Process Overview
4439

4540
As part of the Apache governance model, official releases consist of
46-
signed source tarballs approved by the PMC.
41+
signed source tarballs approved by the Arrow PMC.
4742

4843
We then use the code in the approved source tarball to release to
4944
crates.io, the Rust ecosystem's package manager.
@@ -60,9 +55,7 @@ labels associated with them.
6055

6156
Now prepare a PR to update `CHANGELOG.md` and versions on `master` to reflect the planned release.
6257

63-
For the Rust Arrow crates, do this in the root of this repository. For example [#2323](https://github.com/apache/arrow-rs/pull/2323)
64-
65-
For `object_store` the same process is done in the `object_store` directory. Examples TBD
58+
Do this in the root of this repository. For example [#2323](https://github.com/apache/arrow-rs/pull/2323)
6659

6760
```bash
6861
git checkout master
@@ -90,17 +83,13 @@ python dev/release/label_issues.py
9083
# review change log / edit issues and labels if needed, rerun
9184
git commit -a -m 'Create changelog'
9285

93-
9486
# Manually edit ./dev/release/update_change_log.sh to reflect the release version
9587
# Create the changelog
9688
CHANGELOG_GITHUB_TOKEN=<TOKEN> ./dev/release/update_change_log.sh
9789
# Review change log / edit issues and labels if needed, rerun
9890
git commit -a -m 'Create changelog'
9991

10092
git push
101-
102-
# File the release PR
103-
export BRANCH=<RELEASE_BRANCH> && export GITHUB_USERNAME=<USERNAME> && export GITHUB_TOKEN=<TOKEN> && ./file_release_pr.sh
10493
```
10594

10695
Note that when reviewing the change log, rather than editing the
@@ -120,9 +109,7 @@ distribution servers.
120109

121110
While the official release artifact is a signed tarball, we also tag the commit it was created for convenience and code archaeology.
122111

123-
For a Rust Arrow Crates release, use a string such as `4.0.1` as the `<version>`.
124-
125-
For `object_store` releases, use a string such as `object_store_0.4.0` as the `<version>`.
112+
Use a string such as `43.0.0` as the `<version>`.
126113

127114
Create and push the tag thusly:
128115

@@ -147,12 +134,6 @@ Rust Arrow Crates:
147134
./dev/release/create-tarball.sh 4.1.0 2
148135
```
149136

150-
`object_store`:
151-
152-
```shell
153-
./object_store/dev/release/create-tarball.sh 4.1.0 2
154-
```
155-
156137
The `create-tarball.sh` script
157138

158139
1. creates and uploads a release candidate tarball to the [arrow
@@ -198,7 +179,7 @@ For the release to become "official" it needs at least three Apache Arrow PMC me
198179

199180
## Verifying release candidates
200181

201-
The `dev/release/verify-release-candidate.sh` or `object_store/dev/release/verify-release-candidate.sh` are scripts in this repository that can assist in the verification process. Run it like:
182+
The `dev/release/verify-release-candidate.sh` script in this repository can assist in the verification process. Run it like:
202183

203184
```
204185
./dev/release/verify-release-candidate.sh 4.1.0 2
@@ -218,19 +199,13 @@ Rust Arrow Crates:
218199
./dev/release/release-tarball.sh 4.1.0 2
219200
```
220201

221-
`object_store`
222-
223-
```shell
224-
./object_store/dev/release/release-tarball.sh 4.1.0 2
225-
```
226-
227202
Congratulations! The release is now official!
228203

229204
### Publish on Crates.io
230205

231-
Only approved releases of the tarball should be published to
232-
crates.io, in order to conform to Apache Software Foundation
233-
governance standards.
206+
It is important that only approved releases of the tarball should be published
207+
to crates.io, in order to conform to Apache Software Foundation governance
208+
standards.
234209

235210
An Arrow committer can publish this crate after an official project release has
236211
been made to crates.io using the following instructions.
@@ -269,9 +244,3 @@ Rust Arrow Crates:
269244
(cd parquet_derive && cargo publish)
270245
(cd arrow-integration-test && cargo publish)
271246
```
272-
273-
`object_store`
274-
275-
```shell
276-
cargo publish
277-
```

‎dev/release/file_release_pr.sh

Lines changed: 0 additions & 40 deletions
This file was deleted.

‎object_store/dev/release/README.md

Lines changed: 206 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,4 +17,209 @@
1717
under the License.
1818
-->
1919

20-
See instructions in [`/dev/release/README.md`](../../../dev/release/README.md)
20+
21+
# Release Process
22+
23+
## Overview
24+
25+
This file documents the release process for the `object_store` crate.
26+
27+
At the time of writing, we release a new version of `object_store` on demand rather than on a regular schedule.
28+
29+
As we are still in an early phase, we use the 0.x version scheme. If any code has
30+
been merged to master that has a breaking API change, as defined in [Rust RFC 1105]
31+
the minor version number is incremented changed (e.g. `0.3.0` to `0.4.0`).
32+
Otherwise the patch version is incremented (e.g. `0.3.0` to `0.3.1`).
33+
34+
[Rust RFC 1105]: https://github.com/rust-lang/rfcs/blob/master/text/1105-api-evolution.md
35+
# Release Mechanics
36+
37+
## Process Overview
38+
39+
As part of the Apache governance model, official releases consist of
40+
signed source tarballs approved by the PMC.
41+
42+
We then use the code in the approved source tarball to release to
43+
crates.io, the Rust ecosystem's package manager.
44+
45+
We create a `CHANGELOG.md` so our users know what has been changed between releases.
46+
47+
The CHANGELOG is created automatically using
48+
[update_change_log.sh](https://github.com/apache/arrow-rs/blob/master/object_store/dev/release/update_change_log.sh)
49+
50+
This script creates a changelog using github issues and the
51+
labels associated with them.
52+
53+
## Prepare CHANGELOG and version:
54+
55+
Now prepare a PR to update `CHANGELOG.md` and versions on `master` to reflect the planned release.
56+
57+
Note this process is done in the `object_store` directory. See [#6227] for an example
58+
59+
[#6227]: https://github.com/apache/arrow-rs/pull/6227
60+
61+
```bash
62+
# NOTE: Run commands in object_store sub directory (not main repo checkout)
63+
# cd object_store
64+
65+
git checkout master
66+
git pull
67+
git checkout -b <RELEASE_BRANCH>
68+
69+
# Update versions. Make sure to run it before the next step since we do not want CHANGELOG-old.md affected.
70+
sed -i '' -e 's/0.11.0/0.11.1/g' `find . -name 'Cargo.toml' -or -name '*.md' | grep -v CHANGELOG`
71+
git commit -a -m 'Update version'
72+
73+
# ensure your github token is available
74+
export CHANGELOG_GITHUB_TOKEN=<TOKEN>
75+
76+
# manually edit ./dev/release/update_change_log.sh to reflect the release version
77+
# create the changelog
78+
./dev/release/update_change_log.sh
79+
80+
# review change log / and edit associated issues and labels if needed, rerun update_change_log.sh
81+
82+
# Commit changes
83+
git commit -a -m 'Create changelog'
84+
85+
# push changes to fork and create a PR to master
86+
git push
87+
```
88+
89+
Note that when reviewing the change log, rather than editing the
90+
`CHANGELOG.md`, it is preferred to update the issues and their labels
91+
(e.g. add `invalid` label to exclude them from release notes)
92+
93+
Merge this PR to `master` prior to the next step.
94+
95+
## Prepare release candidate tarball
96+
97+
After you have merged the updates to the `CHANGELOG` and version,
98+
create a release candidate using the following steps. Note you need to
99+
be a committer to run these scripts as they upload to the apache `svn`
100+
distribution servers.
101+
102+
### Create git tag for the release:
103+
104+
While the official release artifact is a signed tarball, we also tag the commit it was created for convenience and code archaeology.
105+
106+
For `object_store` releases, use a string such as `object_store_0.4.0` as the `<version>`.
107+
108+
Create and push the tag thusly:
109+
110+
```shell
111+
git fetch apache
112+
git tag <version> apache/master
113+
# push tag to apache
114+
git push apache <version>
115+
```
116+
117+
### Pick an Release Candidate (RC) number
118+
119+
Pick numbers in sequential order, with `1` for `rc1`, `2` for `rc2`, etc.
120+
121+
### Create, sign, and upload tarball
122+
123+
Run `create-tarball.sh` with the `<version>` tag and `<rc>` and you found in previous steps.
124+
125+
```shell
126+
./object_store/dev/release/create-tarball.sh 0.11.1 1
127+
```
128+
129+
The `create-tarball.sh` script
130+
131+
1. creates and uploads a release candidate tarball to the [arrow
132+
dev](https://dist.apache.org/repos/dist/dev/arrow) location on the
133+
apache distribution svn server
134+
135+
2. provide you an email template to
136+
send to [email protected] for release voting.
137+
138+
### Vote on Release Candidate tarball
139+
140+
Send an email, based on the output from the script to [email protected]. The email should look like
141+
142+
```
143+
Draft email for [email protected] mailing list
144+
145+
---------------------------------------------------------
146+
To: [email protected]
147+
Subject: [VOTE][RUST] Release Apache Arrow Rust Object Store 0.11.1 RC1
148+
149+
Hi,
150+
151+
I would like to propose a release of Apache Arrow Rust Object
152+
Store Implementation, version 0.11.1.
153+
154+
This release candidate is based on commit: b945b15de9085f5961a478d4f35b0c5c3427e248 [1]
155+
156+
The proposed release tarball and signatures are hosted at [2].
157+
158+
The changelog is located at [3].
159+
160+
Please download, verify checksums and signatures, run the unit tests,
161+
and vote on the release. There is a script [4] that automates some of
162+
the verification.
163+
164+
The vote will be open for at least 72 hours.
165+
166+
[ ] +1 Release this as Apache Arrow Rust Object Store
167+
[ ] +0
168+
[ ] -1 Do not release this as Apache Arrow Rust Object Store because...
169+
170+
[1]: https://github.com/apache/arrow-rs/tree/b945b15de9085f5961a478d4f35b0c5c3427e248
171+
[2]: https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-object-store-rs-0.11.1-rc1
172+
[3]: https://github.com/apache/arrow-rs/blob/b945b15de9085f5961a478d4f35b0c5c3427e248/object_store/CHANGELOG.md
173+
[4]: https://github.com/apache/arrow-rs/blob/master/object_store/dev/release/verify-release-candidate.sh
174+
```
175+
176+
For the release to become "official" it needs at least three Apache Arrow PMC members to vote +1 on it.
177+
178+
## Verifying release candidates
179+
180+
The `object_store/dev/release/verify-release-candidate.sh` script can assist in the verification process. Run it like:
181+
182+
```
183+
./object_store/dev/release/verify-release-candidate.sh 0.11.0 1
184+
```
185+
186+
#### If the release is not approved
187+
188+
If the release is not approved, fix whatever the problem is and try again with the next RC number
189+
190+
### If the release is approved,
191+
192+
Move tarball to the release location in SVN, e.g. https://dist.apache.org/repos/dist/release/arrow/arrow-4.1.0/, using the `release-tarball.sh` script:
193+
194+
195+
```shell
196+
./object_store/dev/release/release-tarball.sh 4.1.0 2
197+
```
198+
199+
Congratulations! The release is now official!
200+
201+
### Publish on Crates.io
202+
203+
Only approved releases of the tarball should be published to
204+
crates.io, in order to conform to Apache Software Foundation
205+
governance standards.
206+
207+
An Arrow committer can publish this crate after an official project release has
208+
been made to crates.io using the following instructions.
209+
210+
Follow [these
211+
instructions](https://doc.rust-lang.org/cargo/reference/publishing.html) to
212+
create an account and login to crates.io before asking to be added as an owner
213+
of the [arrow crate](https://crates.io/crates/arrow).
214+
215+
Download and unpack the official release tarball
216+
217+
Verify that the Cargo.toml in the tarball contains the correct version
218+
(e.g. `version = "0.11.0"`) and then publish the crate with the
219+
following commands
220+
221+
222+
```shell
223+
cargo publish
224+
```
225+

0 commit comments

Comments
 (0)