Workflows need re-engineering

[ktyle]

We will need to cleanup/rewrite existing GH actions in our various cookbooks/foundations/etc .github/workflows folders. Currently when "MySTifying" a cookbook, we create a new deploy.yml file. We will need to examine the other .yml files.

[erogluorhan]

From @mgrover1:

These workflows have been mystified - feel free to use

https://github.com/openradar/erad2024/tree/main/.github/workflows

https://github.com/openradar/erad2024/blob/main/.github/workflows/build-book.yaml

https://github.com/openradar/erad2024/blob/main/.github/workflows/build-book-pullrequest.yaml

[jukent]

When we make the new workflow repository, I'd like us to name it something like pythia-actions instead of cookbook-actions. I've been confused a few times before remembering to check that the portal and foundation actions (that I was chasing down) lived in the cookbooks titled repo.

[brian-rose]

My $0.02: the naming should reflect what the workflows do / what they are for.

We have developed reusable workflows with Pythia-specific goals in mind, but there is actually nothing specific to Pythia content in how the workflows operate, and people are using them outside of Pythia (e.g. ProjectPythia/cookbook-actions#120).

This is good thing. Our software is filling a more general need for building and publishing documentation. This is an important output from our project. It would be ideal to track usage somehow for reporting back to funders, but I'm not sure how this should be done.

Regardless of the reporting, I encourage everyone to see our continued investment in maintaining these workflows as a core piece of our infrastructure work that contributes directly to a project deliverable. Along with the shift to MyST-ified workflows, we should be striving to keep things well documented.

Question for all: Should we continue to house these workflows in a separate repository?

It was originally my choice to do it this way, and my intent was simply to get code out of the Cookbook Template repo so future maintenance would be easier. But the workflows could really live anywhere aside from the Template, e.g.

• in the Foundations repo
• in the Portal repo
• in the new pythia-config repo that will house reusable config stuff for all our cookbooks and other sites.

[jukent]

Great points @brian-rose . I feel like actions used for more than one of our repositories should live in a 3rd space (maybe the config. repo?). Otherwise, the portal seems like a more natural home than foundations.

[r-ford]

I'm currently trying to modify the deploy workflow in the projectpythia-mystmd/eofs-cookbook repo to execute the notebooks using the Pythia BinderHub. Based on the documentation for executing during build and connecting to a Binder, I added the --execute flag in the deploy workflow and a link to the Pythia Binder in myst.yml. I'm not sure if these are the correct changes to make (or if this functionality is even possible at the moment), but the GitHub Action is showing that the "Jupyter server did not start".

It seems like ERAD2024 workflows linked above are working, so should we just copy those over and modify them?

[brian-rose]

I made some backwards-compatible changes to https://github.com/ProjectPythia/cookbook-actions so that they now support building with MyST. This is currently demoed in https://github.com/projectpythia-mystmd/cookbook-template.

I haven't started to look at binderbot options.

[brian-rose]

I would support re-homing the reusable workflows into the new config repo, although not until we finish migrating https://github.com/projectpythia-mystmd/pythia-config into the main Pythia org.