From 7d99965e72897fcc4aa908ae1789da3cf299dc79 Mon Sep 17 00:00:00 2001 From: Kim Andrews <17375001+kimandrews@users.noreply.github.com> Date: Mon, 1 Apr 2024 16:53:24 -0700 Subject: [PATCH 1/4] phylo: Update config section of README to reflect reality --- phylogenetic/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/phylogenetic/README.md b/phylogenetic/README.md index 8ead834..dd23d64 100644 --- a/phylogenetic/README.md +++ b/phylogenetic/README.md @@ -27,10 +27,10 @@ Once you've run the build, you can view the results with: ## Configuration -Configuration takes place entirely with the `Snakefile`. This can be read -top-to-bottom, each rule specifies its file inputs and output and also its -parameters. There is little redirection and each rule should be able to be -reasoned with on its own. +Configuration for the workflow takes place entirely within the [defaults/config.yaml](defaults/config.yaml). +The analysis pipeline is contained in [Snakefile](Snakefile) with included [rules](rules). +Each rule specifies its file inputs and output and pulls its parameters from the config. +There is little redirection and each rule should be able to be reasoned with on its own. ### Using GenBank data From b3eb84ee54d3a51a16fc16d2d6cf8b2214fa916d Mon Sep 17 00:00:00 2001 From: Kim Andrews <17375001+kimandrews@users.noreply.github.com> Date: Mon, 1 Apr 2024 17:01:45 -0700 Subject: [PATCH 2/4] phylo: Add build-config/nextstrain-automation Add config and rules for Nextstrain automation to deploy the measles dataset Followed commit from zika: nextstrain/zika@d7838ad As duplicating nextstrain/zika#51 --- phylogenetic/README.md | 14 ++++++++++++++ .../nextstrain-automation/config.yaml | 4 ++++ .../nextstrain-automation/deploy.smk | 15 +++++++++++++++ 3 files changed, 33 insertions(+) create mode 100644 phylogenetic/build-configs/nextstrain-automation/config.yaml create mode 100644 phylogenetic/build-configs/nextstrain-automation/deploy.smk diff --git a/phylogenetic/README.md b/phylogenetic/README.md index dd23d64..5b8abd8 100644 --- a/phylogenetic/README.md +++ b/phylogenetic/README.md @@ -48,3 +48,17 @@ example data provided in this repository. To run the build by copying the example sequences into the `data/` directory, use the following: nextstrain build . --configfile profiles/ci/profiles_config.yaml + +### Deploying build + +To run the workflow and automatically deploy the build to nextstrain.org, +you will need to have AWS credentials to run the following: + +``` +nextstrain build \ + --env AWS_ACCESS_KEY_ID \ + --env AWS_SECRET_ACCESS_KEY \ + . \ + deploy_all \ + --configfile build-configs/nextstrain-automation/config.yaml +``` diff --git a/phylogenetic/build-configs/nextstrain-automation/config.yaml b/phylogenetic/build-configs/nextstrain-automation/config.yaml new file mode 100644 index 0000000..e549405 --- /dev/null +++ b/phylogenetic/build-configs/nextstrain-automation/config.yaml @@ -0,0 +1,4 @@ +custom_rules: + - build-configs/nextstrain-automation/deploy.smk + +deploy_url: "s3://nextstrain-data" diff --git a/phylogenetic/build-configs/nextstrain-automation/deploy.smk b/phylogenetic/build-configs/nextstrain-automation/deploy.smk new file mode 100644 index 0000000..6869346 --- /dev/null +++ b/phylogenetic/build-configs/nextstrain-automation/deploy.smk @@ -0,0 +1,15 @@ +""" +This part of the workflow handles automatic deployments of the measles build. +Uploads the build defined as the default output of the workflow through +the `all` rule from Snakefille +""" + +rule deploy_all: + input: *rules.all.input + output: touch("results/deploy_all.done") + params: + deploy_url = config["deploy_url"] + shell: + """ + nextstrain remote upload {params.deploy_url} {input} + """ From a3f5d32194348c07a56465f6d550d656b123e144 Mon Sep 17 00:00:00 2001 From: Kim Andrews <17375001+kimandrews@users.noreply.github.com> Date: Tue, 2 Apr 2024 11:48:30 -0700 Subject: [PATCH 3/4] Fix `nextstrain build` command in README.md --- phylogenetic/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/phylogenetic/README.md b/phylogenetic/README.md index 5b8abd8..01fa18e 100644 --- a/phylogenetic/README.md +++ b/phylogenetic/README.md @@ -47,7 +47,7 @@ Alternatively, you can run the build using the example data provided in this repository. To run the build by copying the example sequences into the `data/` directory, use the following: - nextstrain build . --configfile profiles/ci/profiles_config.yaml + nextstrain build . --configfile build-configs/ci/config.yaml ### Deploying build From 22136e8b8597af474a15b6edb405b0ae8bd17e6f Mon Sep 17 00:00:00 2001 From: Kim Andrews <17375001+kimandrews@users.noreply.github.com> Date: Tue, 2 Apr 2024 13:25:38 -0700 Subject: [PATCH 4/4] Update Changelog --- CHANGES.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGES.md b/CHANGES.md index f563361..f65d6d8 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,4 +1,5 @@ # CHANGELOG +* 2 April 2024: Add nextstrain-automation build-configs for deploying the final Auspice dataset of the phylogenetic workflow * 1 April 2024: Create a tree using the 450 nucleotides encoding the carboxyl-terminal 150 amino acids of the nucleoprotein (N450), which is highly represented on NCBI for measles. [PR #20](https://github.com/nextstrain/measles/pull/20) * 15 March 2024: Connect ingest and phylogenetic workflows to follow the pathogen-repo-guide by uploading ingest output to S3, downloading ingest output from S3 to phylogenetic directory, using "accession" column as the ID column, and using a color scheme that matches the new region name format. [PR #19](https://github.com/nextstrain/measles/pull/19) * 1 March 2024: Add phylogenetic directory to follow the pathogen-repo-guide, and update the CI workflow to match the new file structure. [PR #18](https://github.com/nextstrain/measles/pull/18)