Automate template setup process via CI for GitHub users #143

Ndpnt · 2024-04-26T13:35:46Z

Technical improvement
Minor change
Impacted areas: Setup script and CI workflows
Details:
- Automate template setup process via CI for GitHub users:
  - When creating a new repository by using this template on GitHub, the setup script is automatically executed by the CI on the resulting generated repository.
- Improve accessibility by adopting less technical terminology:
  - Replaced "bootstrap" with "first-time setup," clarifying its purpose as a one-time project initialization, particularly useful for users unfamiliar with the technical vocabulary.
- Decompose GitHub Actions monolithic workflow into specialized workflows:
  - Split the Country-Template workflow into three distinct workflows: build, validate and deploy enhancing clarity and organization.
- Enhance accuracy of workflow triggers:
  - Trigger deployment exclusively when a PR is merged on the main branch
  - Trigger validation on PR events or as a dependency of the deployment workflow

These changes:

Change non-functional parts of this repository: CI, setup script and documentation

The automation of template setup process via CI can be tested on my fork here

MattiSG

Thank you for this work! It's going to significantly improve the first time setup experience 😃

Many of the GitHub Actions referenced are outdated, and I made line-by-line remarks.
I believe it is worth including their updates in this PR, as it is usually quite painless to do, but I am totally fine if you prefer to separate the version upgrades in a separate PR. It should still be done as part of the work we do now on CI maintenance and improvements 🙂

.github/workflows/build.yml

.github/workflows/deploy.yml

.github/workflows/validate.yml

Co-authored-by: Matti Schneider <[email protected]>

* openfisca/main: Fix changelog hierarchy Use PyPi token for deployment

Python `v3.9.9` is not available for Ubuntu `v22.04`

Update actions dependencies

Ndpnt · 2024-05-03T13:44:50Z

This PR will imply to update the expected validations in the branch protection settings.

MattiSG

Beautiful! 🤩
Thank you for this cloud first time setup, and for the overall upgrades to the CI pipeline! I could have stayed more focused on the main goal of this changeset in my comments, apologies for that.
It is the same with the bunch of comments I make here now. I will apply some of the changes in wording myself, especially when they touch upon pre-existing content.

One question though: I see that the current workflow layout has a nested build job, and it is the only nested one. Do you understand where this is coming from? Is there any way we can make it appear at the same level as other entries in the build?

.github/workflows/build.yml

MattiSG · 2024-05-03T14:04:33Z

.github/workflows/deploy.yml

+    branches: 
+      - main
+    types: [ closed ]


Unify syntax.

Suggested change

branches:

- main

types: [ closed ]

branches: [ main ]

types: [ closed ]

.github/workflows/deploy.yml

MattiSG · 2024-05-03T14:13:19Z

.github/workflows/deploy.yml

+  pull_request_target:
+    branches: 
+      - main
+    types: [ closed ]


Shouldn't we deploy on any push to main, even if they did not originate from a merged pull request? 🤔
This would ensure a consistent mapping between commits on main and deployments, and simplify the code by removing the need to check if: github.event.pull_request.merged == true.

If changed, this should be reflected in the CHANGELOG.

MattiSG · 2024-05-03T14:14:55Z

.github/workflows/deploy.yml

+  deploy:
+    runs-on: ubuntu-22.04
+    needs: [ validate, check-for-functional-changes ]
+    if: github.event.pull_request.merged == true && needs.check-for-functional-changes.outputs.status == 'success'


This condition is redundant since check-for-functional-changes already checks that github.event.pull_request.merged == true.

.github/workflows/deploy.yml

.github/workflows/first-time-setup.yml

README.md

MattiSG · 2024-05-03T14:28:43Z

first-time-setup.sh

+echo -e "${PURPLE}*  ${PURPLE}Remove single use \033[0m${BLUE}first-time-setup.yml\033[0m${PURPLE} GitHub Action\033[0m"
+git rm .github/workflows/first-time-setup.yml > /dev/null 2>&1
+
+echo -e "${PURPLE}*  ${PURPLE}Remove single use \033[0m${BLUE}first-time-setup.sh\033[0m${PURPLE} script\033[0m"


Let's group them and reduce the amount of logs we expose our users to 🙂

Suggested change

echo -e "${PURPLE}* ${PURPLE}Remove single use \033[0m${BLUE}first-time-setup.yml\033[0m${PURPLE} GitHub Action\033[0m"

git rm .github/workflows/first-time-setup.yml > /dev/null 2>&1

echo -e "${PURPLE}* ${PURPLE}Remove single use \033[0m${BLUE}first-time-setup.sh\033[0m${PURPLE} script\033[0m"

echo -e "${PURPLE}* ${PURPLE}Remove single use first time setup files\033[0m"

git rm .github/workflows/first-time-setup.yml > /dev/null 2>&1

MattiSG · 2024-05-03T14:35:25Z

I have switched off required status for all status checks except check-version-and-changelog 👍

Ndpnt · 2024-05-03T14:43:17Z

One question though: I see that the current workflow layout has a nested build job, and it is the only nested one. Do you understand where this is coming from? Is there any way we can make it appear at the same level as other entries in the build?

I'm unsure why, and I don't know how to change it. My guess is that this arises because the build job is defined within the build workflow file, and this job is triggered only as a dependency of other workflows, not directly.

Co-authored-by: Matti Schneider <[email protected]>

MattiSG · 2024-05-03T14:45:13Z

My guess is that this arises because the build job is defined within the build workflow file, and this job is triggered only as a dependency of other workflows, not directly.

That makes sense… let's try to find a workaround this UI limitation and rename one or the other to make it look clearer 🙂

Ndpnt · 2024-05-03T14:52:13Z

That makes sense… let's try to find a workaround this UI limitation and rename one or the other to make it look clearer 🙂

What do you think if we rename the job build into build-and-cache?

MattiSG · 2024-05-03T15:01:19Z

rename the job build into build-and-cache?

Very good idea, as indeed the goal is to cache and fill in the dependency for all other steps!
Let's make sure that this is the name of the top-level entry though, so I assume this should be the name of the workflow and not job (but I'm not sure).

Ndpnt · 2024-05-03T15:04:09Z

Very good idea, as indeed the goal is to cache and fill in the dependency for all other steps! Let's make sure that this is the name of the top-level entry though, so I assume this should be the name of the workflow and not job (but I'm not sure).

I'm not sure to understand, what do you prefer build / build-and-cache or build-and-cache / build?

MattiSG · 2024-05-03T15:06:12Z

Thanks for pointing that out… I prefer build / build-and-cache even though I initially suggested the opposite 😅

Ndpnt · 2024-05-03T15:07:45Z

Thanks for pointing that out… I prefer build / build-and-cache even though I initially suggested the opposite 😅

Ok so I properly understood 😄. I also prefer build / build-and-cache

MattiSG

The code LGTM, but I spot something odd in the workflow view when applying this changeset, where the dependencies seem to be wrong.

Before

After

MattiSG · 2024-05-03T15:39:27Z

As discussed synchronously:

The parallelisation between check-changelog and build is voluntary and I had already suggested it, in order to shorten the feedback loop for users.
The deploy tasks are now only triggered on main and thus voluntarily do not appear on other branches. The dependency layout on a deployment branch can be seen on the source repository and below.

MattiSG

🚀

Ndpnt added 16 commits April 26, 2024 10:09

Allow to bypass steps on CI with CI env variable

25f6f85

Avoid modifying main workflow in bootstrap script

c29c903

Perform bootstrapping when used as GitHub template

5f54fae

Use a less technical naming for better accessibility

176574b

Update documentation

668beab

Unify commit message with first-time-setup script

00260ce

Decompose monolithic workflow into specialized workflows

4361f2a

Remove obsolete dependency

a6d1175

Update job condition

24d6c03

Add changelog entry

721d53f

Update version

ee2baed

Improve documentation

90a7d70

Title case the jurisdiction name

6397718

Allow to manually trigger first time setup workflow

7b3e90f

Improve documentation

de9206f

Remove spaces

e1924a8

Ndpnt requested a review from MattiSG April 26, 2024 13:35

MattiSG requested changes Apr 29, 2024

View reviewed changes

MattiSG and others added 12 commits April 29, 2024 14:57

Fix typos and improve some phrasings

fd141ad

Improve wording

c906b86

Co-authored-by: Matti Schneider <[email protected]>

Fix condition

5c65f27

Co-authored-by: Matti Schneider <[email protected]>

Unify syntax

dcec073

Co-authored-by: Matti Schneider <[email protected]>

Add comment

fdeb428

Co-authored-by: Matti Schneider <[email protected]>

Delete file via Git to ensure the removal will be committed

6b5f74d

Merge remote-tracking branch 'openfisca/main'

718255e

* openfisca/main: Fix changelog hierarchy Use PyPi token for deployment

Update stefanzweifel/git-auto-commit-action to v5

93bb53e

Remove obsolete IDs

1238ffb

Improve naming

faea0bc

Remove obsolete checkout option

01be175

Remove manual trigger on build and validate workflows

0dcdbcf

Ndpnt added 5 commits May 3, 2024 15:13

Update Python to v3.9.12 on CI

b3e4ebb

Python `v3.9.9` is not available for Ubuntu `v22.04`

Update CI dependency actions/checkout to v4

71b05d3

Update CI dependency actions/cache to v4

830be46

Merge pull request #11 from Ndpnt/update-actions-dependencies

ce8f18d

Update actions dependencies

Update changelog

6df2cb8

Ndpnt requested a review from MattiSG May 3, 2024 13:45

Ndpnt force-pushed the main branch from c1224b4 to 6df2cb8 Compare May 3, 2024 13:54

MattiSG reviewed May 3, 2024

View reviewed changes

Improve copywriting

3e01209

Fix step name

5b43f1a

Co-authored-by: Matti Schneider <[email protected]>

Ndpnt added 3 commits May 3, 2024 16:57

Trigger deployment on any push to main

50b8408

Simplify code

dd4f13b

Use consistent line breaks across workflows

3f35133

Reduce logs

73ba03b

Improve job name

e70eaf8

Ndpnt requested a review from MattiSG May 3, 2024 15:12

Ndpnt enabled auto-merge May 3, 2024 15:13

MattiSG reviewed May 3, 2024

View reviewed changes

MattiSG approved these changes May 3, 2024

View reviewed changes

Ndpnt merged commit 717800a into openfisca:main May 3, 2024
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Automate template setup process via CI for GitHub users #143

Automate template setup process via CI for GitHub users #143

Ndpnt commented Apr 26, 2024

MattiSG left a comment

Ndpnt commented May 3, 2024

MattiSG left a comment

MattiSG May 3, 2024

MattiSG May 3, 2024

MattiSG May 3, 2024

MattiSG May 3, 2024

MattiSG commented May 3, 2024

Ndpnt commented May 3, 2024

MattiSG commented May 3, 2024

Ndpnt commented May 3, 2024

MattiSG commented May 3, 2024

Ndpnt commented May 3, 2024

MattiSG commented May 3, 2024

Ndpnt commented May 3, 2024

MattiSG left a comment

MattiSG commented May 3, 2024

MattiSG left a comment

Automate template setup process via CI for GitHub users #143

Automate template setup process via CI for GitHub users #143

Conversation

Ndpnt commented Apr 26, 2024

MattiSG left a comment

Choose a reason for hiding this comment

Ndpnt commented May 3, 2024

MattiSG left a comment

Choose a reason for hiding this comment

MattiSG May 3, 2024

Choose a reason for hiding this comment

MattiSG May 3, 2024

Choose a reason for hiding this comment

MattiSG May 3, 2024

Choose a reason for hiding this comment

MattiSG May 3, 2024

Choose a reason for hiding this comment

MattiSG commented May 3, 2024

Ndpnt commented May 3, 2024

MattiSG commented May 3, 2024

Ndpnt commented May 3, 2024

MattiSG commented May 3, 2024

Ndpnt commented May 3, 2024

MattiSG commented May 3, 2024

Ndpnt commented May 3, 2024

MattiSG left a comment

Choose a reason for hiding this comment

Before

After

MattiSG commented May 3, 2024

MattiSG left a comment

Choose a reason for hiding this comment