Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create separate staging and production profiles #821

Open
jashapiro opened this issue Feb 11, 2025 · 0 comments
Open

Create separate staging and production profiles #821

jashapiro opened this issue Feb 11, 2025 · 0 comments

Comments

@jashapiro
Copy link
Member

jashapiro commented Feb 11, 2025

To better ensure that the results from scpca-nf are stable and allow easier comparisons between the production results and any changes in progress, we should create separate staging and production profiles, with separate output locations for checkpoints and results.

The current ccdl profile uses an output location that is effectively a testing ground; there are no particular controls on the data that are found there. When we are performing a production run, we currently set the output location to the scpca-prod directory manually with the --outdir parameter, but this is a bit brittle and, well, manual.

Instead, we should define ccdl_staging and ccdl_prod profiles that set appropriate output paths.

For extra protection, we may want to see if we can restrict the production profile so that will only run with a versioned tag (so we do not accidentally overwrite results from a development version of the workflow).

This issue should also include:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant