Skip to content

Commit

Permalink
#4: improve documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
cwschilly committed Jan 7, 2025
1 parent 3d318a4 commit 99fdc2b
Show file tree
Hide file tree
Showing 3 changed files with 39 additions and 9 deletions.
44 changes: 37 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# DARMA Workflows

This repository unifies all CI/CD workflows and containers across the DARMA-tasking organization.
This repository unifies all CI/CD workflows and containers across the [DARMA-tasking](https://github.com/DARMA-tasking) organization.

Jump to:
- [Standard Workflows](standard-workflows)
Expand All @@ -16,16 +16,24 @@ All DARMA-tasking repositories should include the following workflows:
* [*check-commit-format*](https://github.com/DARMA-tasking/check-commit-format) - checking if commit message is properly formatted - either starts with "*Merge ...*" or fullfils template: "*#issue_number: short commit description*"
* [*action-git-diff-check*](https://github.com/joel-coffman/action-git-diff-check) - checking if changes introduce conflict markers or whitespace errors

These workflows are included by default when a new repository is created based on [DARMA-tasking/template-repository](https://github.com/DARMA-tasking/template-repository).
> [!TIP]
> These workflows are included by default when a new repository is created based on [DARMA-tasking/template-repository](https://github.com/DARMA-tasking/template-repository).
This repository periodically runs the `check_repository` action (`check-repositories.yml`), which locates all of the aforementioned workflows within every DARMA-tasking repository.
This repository periodically runs a check ([`DARMA repositories check`](https://github.com/DARMA-tasking/workflows/actions/workflows/check-repositories.yml)) to locate all of the above workflows within every DARMA-tasking repository.
Any repositories that fail this check should be corrected as soon as possible to meet the requirements for DARMA-tasking.

> [!NOTE]
> To ignore a repository during this check, add it to the `EXCLUDE` list in `ci/list_repositories.sh`.
## Standard Containers

This repository builds and pushes Docker containers that should be used throughout all DARMA-tasking repositories for CI.

### General Overview
This section explains both 1) how to create a container within this repository and 2) how to use any created containers throughout the DARMA-tasking organization.

### 1. Creating New Containers

#### Overview

All test environments are defined in `ci/config.yaml`, which also includes commented documentation.

Expand All @@ -36,15 +44,37 @@ Some tools are provided as Python scripts:
- `ci/build-docker-image.py`: Build a Docker image from the list of available images described in the configuration file
- This script includes interactive support for local builds as well as a non-interactive mode for CI

### How To Use
#### Step By Step

To create a new container, you only need to edit the `ci/config.yaml` file and run the `build` helper scripts above.

1. **Setup**: Add a new configuration to the `setup` section of `ci/config.yaml`.

2. **Image**: Add a new Docker image to the `images` section of `ci/config.yaml`, referencing the setup that you just created.

3. **Runner**: Add a new runner to the `runners` section of `ci/config.yaml`, referencing the docker image tag you just created.

4. **Generate**: Build the setup and matrix files with these commands:

```sh
python ci/build-setup.py
python ci/build-matrix.py
```

> [!NOTE]
> If you wish to build the new image locally, run `python ci/build-docker-image.py` and select the new image when prompted. Otherwise, the next step will build and push the image in CI.
5. **Merge**: The `build-docker-image.yml` workflow is triggered by a merge to `master`. This will 1) build all Docker containers defined by the matrices in the `shared/matrix` directory and 2) push to Dockerhub.

### 2. Using Containers

The following steps explain how to use the standardized Docker containers in the CI of other DARMA-tasking repositories.

1. Copy the YAML workflow/pipeline

- **GitHub**: Copy/paste the contents of `gh_build_and_test.yml` workflow from the `docs` directory into the `.github/workflows` directory of the desired repository.
- **GitHub**: Copy/paste the contents of `docs/gh_build_and_test.yml` workflow into the `.github/workflows` directory of the desired repository.

- **Azure**: Copy/paste the contents of `azure_build_and-test.yml` workflow from the `docs` directory into the `.azure/pipelines` directory of the desired repository.
- **Azure**: Copy/paste the contents of `docs/azure_build_and-test.yml` workflow into the `.azure/pipelines` directory of the desired repository.

2. Update the definition of `CMD` in the "PR tests" step to reflect the correct build and test commands for the repository.
- Search for "PR tests" in the workflow or pipeline file to find the `CMD` definition
Expand Down
2 changes: 1 addition & 1 deletion ci/build-docker-image.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ class DockerBuilder:
"""Dockerfile generator class"""

def build(self, args: list):
"""Build an image using a given docker configuration fro the config file"""
"""Build an image using a given docker configuration from the config file"""

raw_config: dict = {}
with open(os.path.dirname(__file__) + "/config.yaml", 'r', encoding="utf-8") as file:
Expand Down
2 changes: 1 addition & 1 deletion ci/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

# Dependencies:
# The dependencies are command or scripts to run during a setup process
# - dependencies scripts: located under the `deps` directory. Use script same (no prefix) as key
# - dependencies scripts: located under the `deps` directory. Use script name (no prefix) as key
# - simple commands: use `cmd` as key. It will be added as is in the setup script.

# List of defaults and/or re-usable configuration nodes
Expand Down

0 comments on commit 99fdc2b

Please sign in to comment.