Using Nox inside our GitHub CI actions #40
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jherland
changed the title
jherland
force-pushed
the
jherland/nox-for-github-ci
branch
4 times, most recently
from
486d485
to
1bca1bd
Compare
|
At this point in time, the CI actions all complete successfully, and they are not obviously slower than the previous CI actions (comparing CI actions runtimes between this PR and #23), so I think this is ready to go out of draft mode. The remaining question posed by this PR is really: do we prefer duplicating the commands run in CI actions between Nox and GitHub Actions, or do we want a slightly more complicated GitHub Actions that reuses the Nox actions? |
jherland
force-pushed
the
jherland/nox-for-local-ci
branch
from
406205a
to
6c2f30b
Compare
|
🎉 All dependencies have been resolved ! |
jherland
force-pushed
the
jherland/nox-for-local-ci
branch
from
6c2f30b
to
3c77318
Compare
This reduces duplication of GitHub vs local/Nox CI actions by changing the GitHub actions to call out to Nox. That said, this is not a full-fledged embrace of the Nox way: Nox will usually install dependencies in its own virtualenvs. However, these would come in addition to Poetry's virtualenv, and we have not currently set up caching for anything but the Poetry virtualenvs. Work around this by setting up the appropriate Python version and installing all the necessary dependencies in the Poetry virtualenv (i.e. outside of Nox). We can then run Nox with --no-venv --no-install to explicitly disable the Nox virtualenvs (which also disables the Python version parametrization inside Nox). For this to work, we also need to adjust our install_groups() helper in noxfile.py to detect when it's running without Nox virtualenvs and skip installation in this case. In summary, this adds Nox into the Github Actions mix, but avoids adding an extra layer of virtualenvs and also keeps the same useful caching of the Poetry virtualenvs. The downside is that we run the Nox sessions in a slightly different way compared to running Nox in our local development environment: We do Python version parametrization and dependency installation the Github Actions + Poetry way, and then use Nox merely to abstract away the actual commands run in each CI action. Time will tell if this setup is stable enough to run without issues. We can always revert to duplicating Nox actions inside GitHub Actions later. We can even consider generating .github/workflows/ci.yaml semi- automatically from the sessions defined in noxfile.py.
jherland
force-pushed
the
jherland/nox-for-github-ci
branch
from
1bca1bd
to
97541a4
Compare
mknorps
approved these changes
Jan 10, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(This PR depends on #23 and adds the following commit on top)
Rephrase GitHub Actions in terms of Nox actions
This reduces duplication of GitHub vs local/Nox CI actions by changing
the GitHub actions to call out to Nox.
That said, this is not a full-fledged embrace of the Nox way: Nox will
usually install dependencies in its own virtualenvs. However, these
would come in addition to Poetry's virtualenv, and we have not currently
set up caching for anything but the Poetry virtualenvs.
Work around this by setting up the appropriate Python version and
installing all the necessary dependencies in the Poetry virtualenv
(i.e. outside of Nox). We can then run Nox with
--no-venv --no-installto explicitly disable the Nox virtualenvs (which also disables the
Python version parametrization inside Nox).
For this to work, we also need to adjust our
install_groups()helper innoxfile.pyto detect when it's running without Nox virtualenvs and skipinstallation in this case.
In summary, this avoids an extra layer of virtualenvs and should also
retain the same useful caching of the Poetry virtualenvs, but at the
cost of running the Nox sessions in a slightly different way compared
to running Nox in your local development environment: We do Python
version parametrization and dependency installation the Github Actions +
Poetry way, and then only use Nox to abstract away the actual commands
run inside each CI action.
Time will tell if this creates too many issues to be worthwhile.