Snakemake upgrade #471
Open
Snakemake upgrade #471
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Alputer
added a commit
to Alputer/reana-commons
that referenced
this pull request
Oct 15, 2024
Alputer
force-pushed
the
snakemake-upgrade
branch
from
a4ab1b0
to
20a01a4
Compare
Alputer
added a commit
to Alputer/reana-commons
that referenced
this pull request
Oct 15, 2024
Alputer
force-pushed
the
snakemake-upgrade
branch
from
20a01a4
to
46bd3b6
Compare
Alputer
added a commit
to Alputer/reana-commons
that referenced
this pull request
Oct 15, 2024
Alputer
force-pushed
the
snakemake-upgrade
branch
from
46bd3b6
to
c243ae1
Compare
Alputer
added a commit
to Alputer/reana-commons
that referenced
this pull request
Oct 23, 2024
Alputer
force-pushed
the
snakemake-upgrade
branch
from
c243ae1
to
88cc0dd
Compare
tiborsimko
reviewed
Oct 23, 2024
reana_commons/snakemake.py
Outdated
| ) as snakemake_api: | ||
| try: | ||
| workflow_api = snakemake_api.workflow( | ||
| resource_settings=ResourceSettings(nodes=300), |
There was a problem hiding this comment.
For the nodes settings, we already have SNAKEMAKE_MAX_PARALLEL_JOBS environmental variable that could be used instead of the hard-coded value of 300, even though this might "fully' work only when the validation would be moved to the server-side.
Still, you could preventively introduce in reana_commons/config.py something like:
SNAKEMAKE_MAX_PARALLEL_JOBS = int(os.getenv("SNAKEMAKE_MAX_PARALLEL_JOBS", "300"))
"""Snakemake maximum number of jobs that can run in parallel."""which might be better that hard-coding the value of 300.
(Similarly as we do in reana_workflow_engine_snakemake/config.py.)
setup.py
Outdated
| # see https://github.com/snakemake/snakemake/issues/2657 | ||
| "snakemake @ git+https://github.com/mdonadoni/snakemake.git@cea31624976989ad0645eb2e1751260d32259506", # branch `7.32.4-python3.12` | ||
| "pulp>=2.7.0,<2.8.0", | ||
| "snakemake==8.23.0", |
There was a problem hiding this comment.
There is already Snakemake 8.24.0 out.
reana_commons/snakemake.py
Outdated
| workdir=workdir, | ||
| ) | ||
| # Try to create dag and it throws an error in case of a failure. | ||
| # Seems to be enough but might not be an extensive validation. |
There was a problem hiding this comment.
Perhaps amend the docstring to:
Seems to be enough for the first validation. We may move to snakemake --dry-run when the validation process will be fully moved to the server-side.
Alputer
added a commit
to Alputer/reana-commons
that referenced
this pull request
Oct 23, 2024
Alputer
force-pushed
the
snakemake-upgrade
branch
from
88cc0dd
to
dfa4283
Compare
Alputer
added a commit
to Alputer/reana-commons
that referenced
this pull request
Oct 23, 2024
Alputer
force-pushed
the
snakemake-upgrade
branch
from
dfa4283
to
804e084
Compare
Alputer
added a commit
to Alputer/reana-commons
that referenced
this pull request
Oct 23, 2024
Alputer
force-pushed
the
snakemake-upgrade
branch
from
804e084
to
e23bba8
Compare
Alputer
added a commit
to Alputer/reana-commons
that referenced
this pull request
Oct 23, 2024
Alputer
added a commit
to Alputer/reana-commons
that referenced
this pull request
Oct 23, 2024
Alputer
added a commit
to Alputer/reana-commons
that referenced
this pull request
Oct 23, 2024
Alputer
force-pushed
the
snakemake-upgrade
branch
from
6ab9917
to
5d54f47
Compare
Alputer
added a commit
to Alputer/reana-commons
that referenced
this pull request
Oct 23, 2024
Alputer
added a commit
to Alputer/reana-commons
that referenced
this pull request
Oct 23, 2024
Alputer
force-pushed
the
snakemake-upgrade
branch
from
5d54f47
to
939f501
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #471 +/- ##
==========================================
- Coverage 44.23% 41.79% -2.45%
==========================================
Files 28 28
Lines 1978 2010 +32
==========================================
- Hits 875 840 -35
- Misses 1103 1170 +67
|
Alputer
force-pushed
the
snakemake-upgrade
branch
from
939f501
to
150afd2
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.