ci: improve release workflow

[DaniBodor]

An automated workflow can be triggered from the Actions tab (once this is merged to main) that creates a draft GitHub release. The draft release (notes) can be edited if needed, and then manually converted to an actual Release here, which will trigger the PyPi release (note that a draft release does not trigger PyPi release).

The workflow has the following logic:

• Take inputs:
  • version update level ("patch", "minor", or "major")
  • release branch
    • Unfortunately, the default branch cannot be anything else than main, which is almost never the branch we want to release from. Therefore, if main is selected, the workflow fails, because it was likely by accident forgotten to change the default. Releasing from main can still be done manually if needed.
• Merge release-branch into main
  • A conflict will terminate the process before release
• Bump the version
  • install bump-my-version, bump the version on main, push to remote.
  • NOTE: bump2version is no longer maintained, so its maintained fork bump-my-version is used instead.
• Create draft release
  • the draft can then manually be converted into a published release.
• If above is successful, a separate job is started to clean the workspace:
  • In this way, the "release" will not be listed as failed if below doesn't work (usually due to conflict with dev branch)
  • changes are merged into dev
  • the associated PR is closed
  • if released from a different branch than dev, the hotfix branch is deleted.

This workflow was adapted from here.

fix #170

[gcroci2]

Very nice improvements to the workflow :D

I added some comments, and requested changes since I'd like to be aware of them (the flow is a bit more complex now, I am like a new developer testing it). Also I wonder, has this workflow being tested? Maybe we can try to do a draft release (that then we cancel) before merging it into dev.

[DaniBodor]

Very nice improvements to the workflow :D

I added some comments, and requested changes since I'd like to be aware of them (the flow is a bit more complex now, I am like a new developer testing it). Also I wonder, has this workflow being tested? Maybe we can try to do a draft release (that then we cancel) before merging it into dev.

I've made some improvements, mainly to the documentation.

I've tested this in the eitprocessing repo, but not yet here.
There are a few difficulties in testing this:

• Each time the workflow is triggered, changes are merged into main (even if the release is just a draft). To avoid this, I had created test branches in the other repo and pretended those were the main to keep the true main clean.
• Until the workflow is not on main, it cannot be triggered from the actions tab. This can be somewhat circumvented by using GitHub CLI with this command gh workflow run "Draft GitHub Release" --ref 170_release_workflow_dbodor -f version_level="patch", but it will always read main as the release branch and fail.
• both issues above mean temporarily changing the workflow while testing, which is time consuming.

EDIT: newest tests on eitprocessing aren't working after all 💩 . I will figure it out there and then make changes here. Until then, this PR can stay open.

[gcroci2]

EDIT: newest tests on eitprocessing aren't working after all 💩 . I will figure it out there and then make changes here. Until then, this PR can stay open.

Okay, I've added a couple of comments still to be addressed. Just ask for my review again when you'll discover what is the fix needed :)

[DaniBodor]

EDIT: newest tests on eitprocessing aren't working after all 💩 . I will figure it out there and then make changes here. Until then, this PR can stay open.

Okay, I've added a couple of comments still to be addressed. Just ask for my review again when you'll discover what is the fix needed :)

Will do.
I think the best way to test this is to create a fork of the repo, test everything there, and then if it works implement it here as well.

[gcroci2]

Is this ready to be reviewed again? @DaniBodor If not, maybe either remove me from the reviewers or make this PR a draft (otherwise I keep receiving notifications ;))

[github-actions]

This PR is stale because it has been open for 14 days with no activity.

[DaniBodor]

This action can now be tested on this test fork: https://github.com/EIT-ALIVE/deeprank2_test_release/. I tested it already and it worked. See if you can follow from the dev readme how to do it and if it's explained clearly enough.

Note that in the forked repo, I bugged the release-on-PyPi action, so that we can still check that it gets triggered, but then fails (I don't think it would have been allowed to release a different repo, but didnt want to find out that somehow it does).

Also, I believe all the issues you @gcroci2 flagged have been addressed, or did I miss anything?

[github-actions]

This PR is stale because it has been open for 14 days with no activity.

[gcroci2]

Let's have a chat about the remaining comments, and let's test this on the fork together

[gcroci2]

I added the last suggestions as discussed.

I approved before the PR, but I just noticed some issues. I run the release workflow on the fork and I got this:

The issue is due to the toml file, where the current version of the package seen from bumpversion does not get updated, and thus the error.

Other remaining things before merging:

• Non-expiring token
• Suggestions

[DaniBodor]

I request changes to remember that we need to look at the security thing again. I added a couple of minor suggestions for the instructions. Also let me know if I have to test everything on the fork again (if you say is all good I'll avoid that)

OK, let me know what you decide.
If you want to test again (only minor changes occurred to workflow itself), the please use this new fork instead of the old one. It was quicker to create a new fork rather than update the other one.

• Once we are done testing, we should delete both test forks.

[gcroci2]

I request changes to remember that we need to look at the security thing again. I added a couple of minor suggestions for the instructions. Also let me know if I have to test everything on the fork again (if you say is all good I'll avoid that)

OK, let me know what you decide. If you want to test again (only minor changes occurred to workflow itself), the please use this new fork instead of the old one. It was quicker to create a new fork rather than update the other one.

• Once we are done testing, we should delete both test forks.

I tried doing the test, but it gives me: "Releasing from main or dev branch is not permitted, please select a valid release branch." even if I am using a branch called gcroci2-release-test. See here the PR and here the failed action. @DaniBodor

[DaniBodor]

I request changes to remember that we need to look at the security thing again. I added a couple of minor suggestions for the instructions. Also let me know if I have to test everything on the fork again (if you say is all good I'll avoid that)

OK, let me know what you decide. If you want to test again (only minor changes occurred to workflow itself), the please use this new fork instead of the old one. It was quicker to create a new fork rather than update the other one.

• Once we are done testing, we should delete both test forks.

I tried doing the test, but it gives me: "Releasing from main or dev branch is not permitted, please select a valid release branch." even if I am using a branch called gcroci2-release-test. See here the PR and here the failed action. @DaniBodor

it's fixed

[gcroci2]

Now the automated release process works great :D the only minor thing left can be to improve the error's triggering if the user selects the main branch from the actions (as I did), because it would fail in this case but it is not clear why (unless you know precisely how the workflow works) @DaniBodor

[DaniBodor]

Now the automated release process works great :D the only minor thing left can be to improve the error's triggering if the user selects the main branch from the actions (as I did), because it would fail in this case but it is not clear why (unless you know precisely how the workflow works) @DaniBodor

@gcroci2

It now displays a message on one of the first steps stating: "Releasing from main or dev branch is not permitted, please select a different release branch."

The workflow in the test repo is now the same as the one here in case you want to test it again.
Apologies for having to go back and forth on this so often.

On the test repo, I've now tested trying to release from main, dev, or random other branch name, and it gives expected result each time.

[gcroci2]

Now the automated release process works great :D the only minor thing left can be to improve the error's triggering if the user selects the main branch from the actions (as I did), because it would fail in this case but it is not clear why (unless you know precisely how the workflow works) @DaniBodor

@gcroci2

It now displays a message on one of the first steps stating: "Releasing from main or dev branch is not permitted, please select a different release branch."

The workflow in the test repo is now the same as the one here in case you want to test it again. Apologies for having to go back and forth on this so often.

On the test repo, I've now tested trying to release from main, dev, or random other branch name, and it gives expected result each time.

Great, merge this :D

[DaniBodor]

Now the automated release process works great :D the only minor thing left can be to improve the error's triggering if the user selects the main branch from the actions (as I did), because it would fail in this case but it is not clear why (unless you know precisely how the workflow works) @DaniBodor

@gcroci2
It now displays a message on one of the first steps stating: "Releasing from main or dev branch is not permitted, please select a different release branch."
The workflow in the test repo is now the same as the one here in case you want to test it again. Apologies for having to go back and forth on this so often.
On the test repo, I've now tested trying to release from main, dev, or random other branch name, and it gives expected result each time.

Great, merge this :D

Finally :D