Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update resource allocations #289

Merged
merged 9 commits into from
May 15, 2024
Merged

update resource allocations #289

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
68792e7
update resource allocations
sorelfitzgibbon May 13, 2024
fea2590
update changelog
sorelfitzgibbon May 13, 2024
81d7dfb
Merge branch 'main' of github.com:uclahs-cds/pipeline-call-sSNV into …
sorelfitzgibbon May 13, 2024
1272a77
merge submodule update
sorelfitzgibbon May 13, 2024
6d842f8
Merge remote-tracking branch 'origin' into sfitz-update-resources
sorelfitzgibbon May 13, 2024
80f5403
update config submodule
sorelfitzgibbon May 13, 2024
d5bc47b
update pipeline description
sorelfitzgibbon May 13, 2024
6e2cbeb
Autofix Nextflow configuration regression tests
sorelfitzgibbon May 13, 2024
1c02893
Update CHANGELOG.md
sorelfitzgibbon May 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Grant explicit permissions for Nextflow configuration test workflow
- Update Nextflow configuration test workflows

### Changed
- Update resource allocations

## [8.0.0] - 2024-01-29

### Changed
Expand Down
16 changes: 8 additions & 8 deletions config/F16.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ process {
retry_strategy {
memory {
strategy = 'add'
operand = 3.GB
operand = 5.GB
}
}
}
Expand All @@ -32,7 +32,7 @@ process {
retry_strategy {
memory {
strategy = 'add'
operand = 3.GB
operand = 5.GB
}
}
}
Expand All @@ -42,7 +42,7 @@ process {
retry_strategy {
memory {
strategy = 'add'
operand = 3.GB
operand = 5.GB
}
}
}
Expand All @@ -58,7 +58,7 @@ process {
}
withName: call_sSNV_Mutect2 {
cpus = 1
memory = 3.GB
memory = 2.GB
retry_strategy {
memory {
strategy = 'add'
Expand All @@ -71,8 +71,8 @@ process {
memory = 8.GB
retry_strategy {
memory {
strategy = 'exponential'
operand = 2
strategy = 'add'
operand = 20.GB
}
}
}
Expand All @@ -88,11 +88,11 @@ process {
}
withName: run_sump_MuSE {
cpus = 8
memory = 24.GB
memory = 16.GB
retry_strategy {
memory {
strategy = 'add'
operand = 8.GB
operand = 16.GB
}
}
}
Expand Down
12 changes: 6 additions & 6 deletions config/F32.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,21 +58,21 @@ process {
}
withName: call_sSNV_Mutect2 {
cpus = 1
memory = 3.GB
memory = 2.GB
retry_strategy {
memory {
strategy = 'add'
operand = 3.GB
operand = 2.GB
yashpatel6 marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
withName: run_LearnReadOrientationModel_GATK {
cpus = 1
memory = 16.GB
memory = 12.GB
retry_strategy {
memory {
strategy = 'exponential'
operand = 2
strategy = 'add'
operand = 20.GB
}
}
}
Expand All @@ -88,7 +88,7 @@ process {
}
withName: run_sump_MuSE {
cpus = 12
memory = 48.GB
memory = 18.GB
yashpatel6 marked this conversation as resolved.
Show resolved Hide resolved
retry_strategy {
memory {
strategy = 'add'
Expand Down
18 changes: 9 additions & 9 deletions config/F72.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ process {
}
withName: call_sSNV_SomaticSniper {
cpus = 1
memory = 1.GB
memory = 2.GB
retry_strategy {
memory {
strategy = 'add'
Expand All @@ -18,11 +18,11 @@ process {
}
withName: convert_BAM2Pileup_SAMtools {
cpus = 1
memory = 1.GB
memory = 2.GB
retry_strategy {
memory {
strategy = 'add'
operand = 5.GB
operand = 16.GB
}
}
}
Expand Down Expand Up @@ -68,11 +68,11 @@ process {
}
withName: run_LearnReadOrientationModel_GATK {
cpus = 1
memory = 16.GB
memory = 12.GB
retry_strategy {
memory {
strategy = 'exponential'
operand = 2
strategy = 'add'
operand = 20.GB
}
}
}
Expand All @@ -87,12 +87,12 @@ process {
}
}
withName: run_sump_MuSE {
cpus = 24
memory = 80.GB
cpus = 12
memory = 18.GB
retry_strategy {
memory {
strategy = 'add'
operand = 20.GB
operand = 24.GB
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion external/pipeline-Nextflow-config
Open in desktop
Submodule pipeline-Nextflow-config updated 2 files
+0 −24 .github/workflows/cicd-base.yaml
+17 −0 .github/workflows/static-analysis.yml
2 changes: 1 addition & 1 deletion metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
Category: 'pipeline'
Description: 'A Nextflow pipeline to identify the somatic single nucleotide variants (sSNVs) by comparing a pair of tumor/normal samples.'
Description: 'Nextflow pipeline to identify the somatic single nucleotide variants (sSNVs) by comparing a pair of tumor/normal samples.'
Maintainers: [[email protected]]
Contributors: ['Mao Tian', 'Bugh Caden', 'Helena Winata', 'Yash Patel', 'Sorel Fitz-Gibbon']
Languages: ['Docker', 'Nextflow']
Expand Down
50 changes: 25 additions & 25 deletions test/configtest-F16.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@
"memory": "6 GB",
"retry_strategy": {
"memory": {
"operand": "3 GB",
"operand": "5 GB",
"strategy": "add"
}
}
Expand All @@ -130,7 +130,7 @@
},
"call_sSNV_Mutect2": {
"cpus": "1",
"memory": "3 GB",
"memory": "2 GB",
"retry_strategy": {
"memory": {
"operand": "2 GB",
Expand Down Expand Up @@ -176,7 +176,7 @@
"memory": "1 GB",
"retry_strategy": {
"memory": {
"operand": "3 GB",
"operand": "5 GB",
"strategy": "add"
}
}
Expand All @@ -186,7 +186,7 @@
"memory": "1 GB",
"retry_strategy": {
"memory": {
"operand": "3 GB",
"operand": "5 GB",
"strategy": "add"
}
}
Expand All @@ -206,17 +206,17 @@
"memory": "8 GB",
"retry_strategy": {
"memory": {
"operand": "2",
"strategy": "exponential"
"operand": "20 GB",
"strategy": "add"
}
}
},
"run_sump_MuSE": {
"cpus": "8",
"memory": "24 GB",
"memory": "16 GB",
"retry_strategy": {
"memory": {
"operand": "8 GB",
"operand": "16 GB",
"strategy": "add"
}
}
Expand Down Expand Up @@ -329,9 +329,9 @@
"cpus": "6",
"memory": {
"1": "6 GB",
"2": "9 GB",
"3": "12 GB",
"closure": "retry_updater(6 GB, add, 3 GB, $task.attempt, memory)"
"2": "11 GB",
"3": "16 GB",
"closure": "retry_updater(6 GB, add, 5 GB, $task.attempt, memory)"
}
},
"withName:call_sSNV_MuSE": {
Expand All @@ -346,10 +346,10 @@
"withName:call_sSNV_Mutect2": {
"cpus": "1",
"memory": {
"1": "3 GB",
"2": "5 GB",
"3": "7 GB",
"closure": "retry_updater(3 GB, add, 2 GB, $task.attempt, memory)"
"1": "2 GB",
"2": "4 GB",
"3": "6 GB",
"closure": "retry_updater(2 GB, add, 2 GB, $task.attempt, memory)"
}
},
"withName:call_sSNV_SomaticSniper": {
Expand Down Expand Up @@ -386,18 +386,18 @@
"cpus": "1",
"memory": {
"1": "1 GB",
"2": "4 GB",
"3": "7 GB",
"closure": "retry_updater(1 GB, add, 3 GB, $task.attempt, memory)"
"2": "6 GB",
"3": "11 GB",
"closure": "retry_updater(1 GB, add, 5 GB, $task.attempt, memory)"
}
},
"withName:create_IndelCandidate_SAMtools": {
"cpus": "1",
"memory": {
"1": "1 GB",
"2": "4 GB",
"3": "7 GB",
"closure": "retry_updater(1 GB, add, 3 GB, $task.attempt, memory)"
"2": "6 GB",
"3": "11 GB",
"closure": "retry_updater(1 GB, add, 5 GB, $task.attempt, memory)"
}
},
"withName:plot_VennDiagram_R": {
Expand All @@ -413,18 +413,18 @@
"cpus": "1",
"memory": {
"1": "8 GB",
"2": "16 GB",
"2": "28 GB",
"3": "31 GB",
"closure": "retry_updater(8 GB, exponential, 2, $task.attempt, memory)"
"closure": "retry_updater(8 GB, add, 20 GB, $task.attempt, memory)"
}
},
"withName:run_sump_MuSE": {
"cpus": "8",
"memory": {
"1": "24 GB",
"1": "16 GB",
"2": "31 GB",
"3": "31 GB",
"closure": "retry_updater(24 GB, add, 8 GB, $task.attempt, memory)"
"closure": "retry_updater(16 GB, add, 16 GB, $task.attempt, memory)"
}
},
"withName:run_validate_PipeVal": {
Expand Down
34 changes: 17 additions & 17 deletions test/configtest-F32.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,10 +130,10 @@
},
"call_sSNV_Mutect2": {
"cpus": "1",
"memory": "3 GB",
"memory": "2 GB",
"retry_strategy": {
"memory": {
"operand": "3 GB",
"operand": "2 GB",
"strategy": "add"
}
}
Expand Down Expand Up @@ -203,17 +203,17 @@
},
"run_LearnReadOrientationModel_GATK": {
"cpus": "1",
"memory": "16 GB",
"memory": "12 GB",
"retry_strategy": {
"memory": {
"operand": "2",
"strategy": "exponential"
"operand": "20 GB",
"strategy": "add"
}
}
},
"run_sump_MuSE": {
"cpus": "12",
"memory": "48 GB",
"memory": "18 GB",
"retry_strategy": {
"memory": {
"operand": "16 GB",
Expand Down Expand Up @@ -346,10 +346,10 @@
"withName:call_sSNV_Mutect2": {
"cpus": "1",
"memory": {
"1": "3 GB",
"2": "6 GB",
"3": "9 GB",
"closure": "retry_updater(3 GB, add, 3 GB, $task.attempt, memory)"
"1": "2 GB",
"2": "4 GB",
"3": "6 GB",
"closure": "retry_updater(2 GB, add, 2 GB, $task.attempt, memory)"
}
},
"withName:call_sSNV_SomaticSniper": {
Expand Down Expand Up @@ -412,19 +412,19 @@
"withName:run_LearnReadOrientationModel_GATK": {
"cpus": "1",
"memory": {
"1": "16 GB",
"1": "12 GB",
"2": "32 GB",
"3": "64 GB",
"closure": "retry_updater(16 GB, exponential, 2, $task.attempt, memory)"
"3": "52 GB",
"closure": "retry_updater(12 GB, add, 20 GB, $task.attempt, memory)"
}
},
"withName:run_sump_MuSE": {
"cpus": "12",
"memory": {
"1": "48 GB",
"2": "64 GB",
"3": "64 GB",
"closure": "retry_updater(48 GB, add, 16 GB, $task.attempt, memory)"
"1": "18 GB",
"2": "34 GB",
"3": "50 GB",
"closure": "retry_updater(18 GB, add, 16 GB, $task.attempt, memory)"
}
},
"withName:run_validate_PipeVal": {
Expand Down