Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Github Actions CI Changes #187

Draft
wants to merge 19 commits into
base: master
Choose a base branch
from
Draft

Conversation

shihab-dls
Copy link

CI workflow jobs now spread out into reusable workflows. Some repetitive steps (i.e., checkouts, rclone installations, S3 mounting, and registry login) moved to a composite actions. S3 mounting and registry login removed from actions, however, due to secrets becoming visible as inputs.

PR opened as draft until IRIS runners are available for CI jobs.

@Araneidae
Copy link
Contributor

Please squash these commits into a sensible commit history! I suspect a lot of the churn in this will be due to debugging history, which doesn't need to go into the final PR.

@shihab-dls shihab-dls force-pushed the actions_workflow_changes branch 2 times, most recently from 076e6dd to 15738c8 Compare March 28, 2024 13:45
@shihab-dls
Copy link
Author

All jobs are now on IRIS runners, but make_boot and test jobs require more attention:

  • make_boot currently hangs between successful build completion and artefact upload (for 4hrs). A 60min timeout and zip file check is set to appropriately end the job.
  • Running tests on IRIS take ~3.5x longer than on GH runners, with each test (of 161) sometimes requiring a couple of minutes.

The /scripts directory has been deleted, and an NFS mount at /opt/arc-panda on cs05r-sc-cloud-30 (rw for DCS group) is used instead of the S3 bucket. An adjusted dev-container is used as the runner's base image, a PR of which will be opened on the rootfs repository.

@tomtrafford tomtrafford self-requested a review May 9, 2024 14:17
@shihab-dls
Copy link
Author

A work-around for the 4hr test job is now in place

  • The /scripts directory was reintroduced
  • Two scripts are included: Find_Tests.sh and Group_Tests.py
    --Find_Tests.sh produced an array of module names and their respective test counts (i.e., number of tests)
    --Group_Tests.py allocates modules to IRIS runners, targeting runners with least load first, resulting in a (almost) even spread of tests.

Overall duration of the matrix of test jobs is < 1hr; thus, no longer the limiting job. Overall duration of complete workflow is ~1hr.

Shihab Suliman and others added 3 commits May 17, 2024 13:35
Current CI jobs run similar blocks of code to checkout repos, install rclone and go, and mount the S3 bucket. This composite action YAML file will be called for instead.
External call to code.yml make_boot job
shihab-dls and others added 15 commits May 17, 2024 13:46
Make Boot and Test jobs require more attention. Make Boot currently hangs on completion, requiring a timeout and zip file check to successfully end job. Running tests on IRIS takes 3.5X as long.

Running all jobs on IRIS with NFS mount, and removing jobs/scripts related to S3 or rclone

Make Boot and Test jobs require more attention. Make Boot currently hangs on completion, requiring a timeout and zip file check to successfully end job. Running tests on IRIS seem to take 3.5X as long.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants