Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate template setup process via CI for GitHub users #143

Merged
merged 42 commits into from
May 3, 2024
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
25f6f85
Allow to bypass steps on CI with `CI` env variable
Ndpnt Apr 24, 2024
c29c903
Avoid modifying main workflow in bootstrap script
Ndpnt Apr 24, 2024
5f54fae
Perform bootstrapping when used as GitHub template
Ndpnt Apr 24, 2024
176574b
Use a less technical naming for better accessibility
Ndpnt Apr 24, 2024
668beab
Update documentation
Ndpnt Apr 24, 2024
00260ce
Unify commit message with first-time-setup script
Ndpnt Apr 26, 2024
4361f2a
Decompose monolithic workflow into specialized workflows
Ndpnt Apr 26, 2024
a6d1175
Remove obsolete dependency
Ndpnt Apr 26, 2024
24d6c03
Update job condition
Ndpnt Apr 26, 2024
721d53f
Add changelog entry
Ndpnt Apr 26, 2024
ee2baed
Update version
Ndpnt Apr 26, 2024
90a7d70
Improve documentation
Ndpnt Apr 26, 2024
6397718
Title case the jurisdiction name
Ndpnt Apr 26, 2024
7b3e90f
Allow to manually trigger first time setup workflow
Ndpnt Apr 26, 2024
de9206f
Improve documentation
Ndpnt Apr 26, 2024
e1924a8
Remove spaces
Ndpnt Apr 26, 2024
fd141ad
Fix typos and improve some phrasings
MattiSG Apr 29, 2024
c906b86
Improve wording
Ndpnt May 3, 2024
5c65f27
Fix condition
Ndpnt May 3, 2024
dcec073
Unify syntax
Ndpnt May 3, 2024
fdeb428
Add comment
Ndpnt May 3, 2024
6b5f74d
Delete file via Git to ensure the removal will be committed
Ndpnt May 3, 2024
718255e
Merge remote-tracking branch 'openfisca/main'
Ndpnt May 3, 2024
93bb53e
Update stefanzweifel/git-auto-commit-action to v5
Ndpnt May 3, 2024
1238ffb
Remove obsolete IDs
Ndpnt May 3, 2024
faea0bc
Improve naming
Ndpnt May 3, 2024
01be175
Remove obsolete checkout option
Ndpnt May 3, 2024
0dcdbcf
Remove manual trigger on `build` and `validate` workflows
Ndpnt May 3, 2024
4130dfe
Update Ubuntu to `v22.04` on CI
Ndpnt May 3, 2024
7d13315
Update CI dependency `actions/setup-python` to `v5`
Ndpnt May 3, 2024
b3e4ebb
Update Python to `v3.9.12` on CI
Ndpnt May 3, 2024
71b05d3
Update CI dependency `actions/checkout` to `v4`
Ndpnt May 3, 2024
830be46
Update CI dependency `actions/cache` to `v4`
Ndpnt May 3, 2024
ce8f18d
Merge pull request #11 from Ndpnt/update-actions-dependencies
Ndpnt May 3, 2024
6df2cb8
Update changelog
Ndpnt May 3, 2024
3e01209
Improve copywriting
MattiSG May 3, 2024
5b43f1a
Fix step name
Ndpnt May 3, 2024
50b8408
Trigger deployment on any push to `main`
Ndpnt May 3, 2024
dd4f13b
Simplify code
Ndpnt May 3, 2024
3f35133
Use consistent line breaks across workflows
Ndpnt May 3, 2024
73ba03b
Reduce logs
Ndpnt May 3, 2024
e70eaf8
Improve job name
Ndpnt May 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Build

on:
workflow_call:
workflow_dispatch:
MattiSG marked this conversation as resolved.
Show resolved Hide resolved

jobs:
build:
runs-on: ubuntu-20.04
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
steps:
- name: Checkout
uses: actions/checkout@v2
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
- name: Set up Python
uses: actions/setup-python@v2
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
with:
python-version: 3.9.9 # Patch version must be specified to avoid any cache confusion, since the cache key depends on the full Python version. Any potentiel difference in patches between jobs will lead to a cache not found error.
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
- name: Cache build
id: restore-build
uses: actions/cache@v2
with:
path: ${{ env.pythonLocation }}
key: build-${{ env.pythonLocation }}-${{ hashFiles('pyproject.toml') }}-${{ github.sha }} # Cache the entire build Python environment
restore-keys: |
build-${{ env.pythonLocation }}-${{ hashFiles('pyproject.toml') }}
build-${{ env.pythonLocation }}-
- name: Build package
run: make build
- name: Cache release
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that there could be a performance optimisation by specifying the release cache before the build step, in case the build would optimise and spare itself from generating artefacts that would already be present in the dist folder.

id: restore-release
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
uses: actions/cache@v2
with:
path: dist
key: release-${{ env.pythonLocation }}-${{ hashFiles('pyproject.toml') }}-${{ github.sha }}
62 changes: 62 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
name: Deploy

on:
pull_request_target:
branches:
- main
types: [ closed ]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unify syntax.

Suggested change
branches:
- main
types: [ closed ]
branches: [ main ]
types: [ closed ]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we deploy on any push to main, even if they did not originate from a merged pull request? 🤔
This would ensure a consistent mapping between commits on main and deployments, and simplify the code by removing the need to check if: github.event.pull_request.merged == true.

If changed, this should be reflected in the CHANGELOG.


jobs:
validate:
uses: "./.github/workflows/validate.yml"

# GitHub Actions does not have a halt job option, to stop from deploying if no functional changes were found.
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
# We build a separate job to substitute the halt option.
# The `deploy` job is dependent on the output of the `check-for-functional-changes`job.
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
check-for-functional-changes:
runs-on: ubuntu-20.04
if: github.event.pull_request.merged == true
outputs:
status: ${{ steps.stop-early.outputs.status }}
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0 # Fetch all the tags
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.9.9
- id: stop-early
run: if "${GITHUB_WORKSPACE}/.github/has-functional-changes.sh" ; then echo "::set-output name=status::success" ; fi

deploy:
runs-on: ubuntu-20.04
needs: [ validate, check-for-functional-changes ]
if: github.event.pull_request.merged == true && needs.check-for-functional-changes.outputs.status == 'success'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This condition is redundant since check-for-functional-changes already checks that github.event.pull_request.merged == true.

env:
PYPI_USERNAME: openfisca-bot
PYPI_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0 # Fetch all the tags
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.9.9
- name: Cache build
id: restore-build
uses: actions/cache@v2
with:
path: ${{ env.pythonLocation }}
key: build-${{ env.pythonLocation }}-${{ hashFiles('pyproject.toml') }}-${{ github.sha }}
- name: Cache release
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
id: restore-release
uses: actions/cache@v2
with:
path: dist
key: release-${{ env.pythonLocation }}-${{ hashFiles('pyproject.toml') }}-${{ github.sha }}
- name: Upload a Python package to PyPi
run: twine upload dist/* --username $PYPI_USERNAME --password $PYPI_PASSWORD
- name: Publish a git tag
run: "${GITHUB_WORKSPACE}/.github/publish-git-tag.sh"
36 changes: 36 additions & 0 deletions .github/workflows/first-time-setup.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: First time setup

on:
create:
workflow_dispatch:

permissions:
actions: write
checks: write
contents: write

jobs:
first-time-setup:
# Ensure this job does not run on the template repository or when the repository is forked
if: ${{ !github.event.repository.is_template && !github.event.repository.fork }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}

Ndpnt marked this conversation as resolved.
Show resolved Hide resolved
- name: Set environment variables
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
run: |
echo "TITLE_CASE_JURISDICTION_NAME=$(echo ${{ github.event.repository.name }} | sed 's/openfisca-//' | sed 's/[-|_]/ /g' | awk '{for (i=1; i<=NF; i++) $i=toupper(substr($i,1,1)) substr($i,2)} 1')" >> $GITHUB_ENV

- name: Execute the first-time-setup script
run: CI=true JURISDICTION_NAME="$TITLE_CASE_JURISDICTION_NAME" REPOSITORY_URL="${{ github.repositoryUrl }}" ./first-time-setup.sh

- name: Commit changes
uses: stefanzweifel/git-auto-commit-action@v4
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
with:
commit_message: "Customise country-template through CI"
tagging_message: "0.0.1"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
83 changes: 83 additions & 0 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
name: Validate

on:
pull_request:
types: [ assigned, opened, reopened, synchronize, ready_for_review ]
workflow_dispatch:
workflow_call:

jobs:
build:
uses: "./.github/workflows/build.yml"

lint-files:
runs-on: ubuntu-20.04
needs: [ build ]
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0 # Fetch all the tags
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.9.9
- name: Cache build
id: restore-build
uses: actions/cache@v2
with:
path: ${{ env.pythonLocation }}
key: build-${{ env.pythonLocation }}-${{ hashFiles('pyproject.toml') }}-${{ github.sha }}
- run: make check-syntax-errors
- run: make check-style
- name: Lint Python files
run: "${GITHUB_WORKSPACE}/.github/lint-changed-python-files.sh"
- name: Lint YAML tests
run: "${GITHUB_WORKSPACE}/.github/lint-changed-yaml-tests.sh"
MattiSG marked this conversation as resolved.
Show resolved Hide resolved

test-yaml:
runs-on: ubuntu-20.04
needs: [ build ]
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.9.9
- name: Cache build
id: restore-build
uses: actions/cache@v2
with:
path: ${{ env.pythonLocation }}
key: build-${{ env.pythonLocation }}-${{ hashFiles('pyproject.toml') }}-${{ github.sha }}
- run: make test

test-api:
runs-on: ubuntu-20.04
needs: [ build ]
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.9.9
- name: Cache build
id: restore-build
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
uses: actions/cache@v2
with:
path: ${{ env.pythonLocation }}
key: build-${{ env.pythonLocation }}-${{ hashFiles('pyproject.toml') }}-${{ github.sha }}
- name: Test the Web API
run: "${GITHUB_WORKSPACE}/.github/test-api.sh"

check-version-and-changelog:
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0 # Fetch all the tags
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.9.9
- name: Check version number has been properly updated
run: "${GITHUB_WORKSPACE}/.github/is-version-number-acceptable.sh"
161 changes: 0 additions & 161 deletions .github/workflows/workflow.yml

This file was deleted.

Loading
Loading