This PR closes #598

🗑️ This dev branch should be deleted after merging to main.

🧠 Summary

The Ivar Consensus workflow was facing a silent error due to a pipefail on input files that were very large.

The changes in this PR do the following:

1. Add pipefail parameters set -euo pipefail to both the ivar consensus task and the ivar variant calling task.
2. Expose cpu, memory, disk_size, and docker parameters for the following tasks in the ivar consensus wf:

• bwa alignment
• primer trimming
• stats and coverage for primer trimming
• variant calling
• consensus
• stats and coverage for the consensus wf

3.Improve error handling so that tasks will now fail correctly instead of producing silent errors.

⚡ Impacted Workflows/Tasks

This PR may lead to different results in pre-existing outputs: No

This PR uses an element that could cause duplicate runs to have different results: No

🛠️ Changes

⚙️ Algorithm

set -euo pipefail has been added to the task_ivar_consensus.wdl and to task_ivar_variant_call.wdl

➡️ Inputs

The following optional inputs have been added to the ivar consensus workflow:

Int?        ivar_bwa_cpu
Int?        ivar_bwa_memory
Int?        ivar_bwa_disk_size
String?     ivar_bwa_docker
Int?        ivar_trim_primers_cpu
Int?        ivar_trim_primers_memory
Int?        ivar_trim_primers_disk_size
String?     ivar_trim_primers_docker
Int?        ivar_variant_cpu
Int?        ivar_variant_memory
Int?        ivar_variant_disk_size
String?     ivar_variant_docker
Int?        ivar_consensus_cpu
Int?        ivar_consensus_memory
Int?        ivar_consensus_disk_size
String?     ivar_consensus_docker
Int?        stats_n_coverage_cpu
Int?        stats_n_coverage_memory
Int?        stats_n_coverage_disk_size
String?     stats_n_coverage_docker

⬅️ Outputs

🧪 Testing

This PR was tested on the input data that originally brought up this bug by Marc Perry from UCSC. The following set of rsv fastq inputs: RSV00100_A & RSV00122_A, these inputs can be exported from the ucsc-rsv data table found here.

This data was first tested with the workflow defaults, which lead to the correct failures being flagged on terra.

Then the data was tested with the following parameters adjusted for the ivar consensus task on the TheiaCov Illumina PE wf:
ivar_consensus_memory: 16
ivar_variant_memory: 16

To further confirm the updates did not cause any breaking changes, the workflow was tested with defaults against WNV and MPXV validation data from the theiagen validation illumina pe data set.

Suggested Scenarios for Reviewer to Test

Please test with simple inputs with defualt and one with large files >=1.5GB per fastq with the following adjustments:
ivar_consensus_memory: 16
ivar_variant_memory: 16

I can provide the source to the large datasets.

🔬 Final Developer Checklist

• The workflow/task has been tested and results, including file contents, are as anticipated
• The CI/CD has been adjusted and tests are passing (Theiagen developers)
• Code changes follow the style guide
• Documentation and/or workflow diagrams have been updated if applicable (Theiagen developers only)

🎯 Reviewer Checklist

• All changed results have been confirmed
• You have tested the PR appropriately (see the testing guide for more information)
• All code adheres to the style guide
• MD5 sums have been updated
• The PR author has addressed all comments
• The documentation has been updated