Add independent ingest and phylogenetic GH Action workflows #55
Conversation
Create a reusable GH Action workflows for ingest and phylogenetic that are called within the ingest-to-phylogenetic workflow. Future commits will update the independent GH Action workflows to allow manual workflow dispatches and trial runs.
This will allow manual triggers of the independent ingest and phylogenetic workflows. Currently just includes the existing `image` input but subsequent commits will allow trial runs of each workflow.
Allow trial builds of the ingest and phylogenetic workflows to be uploaded to trial S3 URLs. The `trial_name` inputs are _not_ added to the `workflow_call` trigger since the `workflow_call` trigger is only being used by our automated ingest-to-phylogenetic workflow. This may change in the future, but leaving it out for now. For phylogenetic workflow to run with trial outputs from the ingest workflow, the phylogenetic workflow needs to accept inputs from the config. I'll make these changes separately.
Allows us to easily update the input URLs both from the config YAML and `--config` args. This is mostly motivated by the desire to run phylogenetic builds from trial ingest outputs.
Allows customization of input sequences and metadata URLs. This will allow us to test trial outputs from ingest workflows.
|
The trial ingest outputs are available at The trial phylogenetic bulid that used the trial ingest output was deployed to staging, available at https://nextstrain.org/staging/zika/trials/reusable-workflows/zika |
Make it easier to set up GH Action workflows for pathogen repos, but still allow customizations within them that are not possible with reusable workflows. Based on the GH Action workflows added in nextstrain/zika#55.
Co-authored-by: Victor Lin <[email protected]>
|
Merging to allow @j23414 and @kimandrews to see the GitHub Actions UI for the 3 different workflows. Post-merge feedback is always welcome! |
|
Worked as expected during my tests (ingest, phylogenetic). Thanks! |
|
Oh hmm, just found a side effect of this PR. The Pathogen status page no longer includes the automated workflow:
|
|
@joverlee521 The criteria used for that status page, "workflows which use |
For sure! Just need to figure out what the appropriate proxy is for the workflows of this PR. I'll write up something in the status repo. |
|
I completed trial runs and it went fine for me too: ingest, phylogenetic |
Create a reusable GH Action workflows for ingest that can be manually called to test branches, and can be called within the ingest-to-phylogenetic workflow. Copied from the zika PR and modified for dengue. nextstrain/zika#55
Create a reusable GH Action workflows for ingest that can be manually called to test branches, and can be called within the ingest-to-phylogenetic workflow. Copied from the zika PR and modified for dengue. nextstrain/zika#55
Create a reusable GH Action workflows for phylogenetic that can be manually called to test branches, and can be called within the ingest-to-phylogenetic workflow. Copied from the zika PR and modified for dengue. nextstrain/zika#55
Create a reusable GH Action workflows for ingest that can be manually called to test branches, and can be called within the ingest-to-phylogenetic workflow. Copied from the zika PR and modified for dengue. nextstrain/zika#55
Create a reusable GH Action workflows for phylogenetic that can be manually called to test branches, and can be called within the ingest-to-phylogenetic workflow. Copied from the zika PR and modified for dengue. nextstrain/zika#55
Create a reusable GH Action workflows for ingest that can be manually called to test branches, and can be called within the ingest-to-phylogenetic workflow. Copied from the zika PR and modified for dengue. nextstrain/zika#55
Create a reusable GH Action workflows for phylogenetic that can be manually called to test branches, and can be called within the ingest-to-phylogenetic workflow. Copied from the zika PR and modified for dengue. nextstrain/zika#55
Description of proposed changes
Splits the
ingest-to-phylogeneticGH Action workflow added in #52 into independent reusable workflows.The reusable GH Action workflows
ingestandphylogeneticare then called from withiningest-to-phylogenetic, but they can also be run independently and with trial names.This splits the use of the workflows as
ingest-to-phylogenetic.yamlfor scheduled, automated runsingest.yamlfor manual trial runs of the ingest workflowphylogenetic.yamlfor manual trial runs of the phylogenetic workflow OR forced runs that don't check the cacheChecklist
ingest-to-phylogenetic(with cache)ingest-to-phylogenetic(after manually deleting cache)ingest(with trial name)phylogenetic(with trial name and custom input URLs that point to output files of the trial ingest run)