Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parabricks/applybqsr version updated and tests migrated to nf-test #6881

Open
wants to merge 62 commits into
base: master
Choose a base branch
from

Conversation

gitanoqevaporelmundoentero
Copy link
Contributor

@gitanoqevaporelmundoentero gitanoqevaporelmundoentero commented Oct 29, 2024

Tools applybqsr is updated to latest version (4.3.2-1) and tests were migrated to new convention (from pytest to nf-test).

PR checklist

Closes #6861

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the module conventions in the contribution docs
  • If necessary, include test data in your PR.
  • Remove all TODO statements.
  • Emit the versions.yml file.
  • Follow the naming conventions.
  • Follow the parameters requirements.
  • Follow the input/output options guidelines.
  • Add a resource label
  • Use BioConda and BioContainers if possible to fulfil software requirements.
  • Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
    • For modules:
      • nf-core modules test <MODULE> --profile docker
      • nf-core modules test <MODULE> --profile singularity
      • nf-core modules test <MODULE> --profile conda
    • For subworkflows:
      • nf-core subworkflows test <SUBWORKFLOW> --profile docker
      • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
      • nf-core subworkflows test <SUBWORKFLOW> --profile conda

Copy link
Contributor

@famosab famosab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good so far.

.github/workflows/test.yml Show resolved Hide resolved
@famosab
Copy link
Contributor

famosab commented Oct 29, 2024

We need to change the way the GPUs are accessed in the main.nf file. Maybe we can do that here aswell.

so the line
--num-gpus $task.accelerator.request \\

needs to be
$num_gpus \\

and add this above the script block
def num_gpus = task.accelerator ? "--num-gpus $task.accelerator.request" : ''


And we need to add below the label expression on the top of the file
label 'process_gpu'

@gitanoqevaporelmundoentero gitanoqevaporelmundoentero changed the title parabricks/applybqsr version updated and tests migrated to pytest parabricks/applybqsr version updated and tests migrated to nf-test Oct 29, 2024
@famosab famosab self-assigned this Oct 30, 2024
@mashehu
Copy link
Contributor

mashehu commented Nov 7, 2024

I think I finally got everything running, and I blame outdated snapshots for the failing tests. Am I wrong there, @gitanoqevaporelmundoentero, @famosab

@famosab
Copy link
Contributor

famosab commented Nov 8, 2024

@sateeshperi @sateeshperi (not sure which one is the correct handle)
For fq2bammeth there is also no distinction between docker & singularity. Do you know why?

@mashehu @gitanoqevaporelmundoentero I will try to update the snaps on our cluster and push them.

@sateeshperi
Copy link
Contributor

honestly no idea since I didn't test using singularity. I just borrowed that config block from nf-core/scdownstream

@mashehu mashehu mentioned this pull request Nov 8, 2024
@famosab
Copy link
Contributor

famosab commented Nov 11, 2024

Ok small update from me: Our cluster does not support docker, only singularity and for some reason just adding the --nv flag to the singularity options did not work for me.

@sateeshperi
Copy link
Contributor

could you try what we had previously ?

@famosab
Copy link
Contributor

famosab commented Nov 12, 2024

could you try what we had previously ?

No, I think the AWS instance was closed. I am trying to run it on other stuff we have locally but each one of those has some limitations.

@mashehu
Copy link
Contributor

mashehu commented Nov 12, 2024

restarted the test instance

@famosab
Copy link
Contributor

famosab commented Nov 12, 2024

Snaps are updated. I could not solve the weird fq2bammeth behaviour.
Seems like the test for applybqsr somehow does not pick up and the singularity one for fq2bam fails?

@mashehu
Copy link
Contributor

mashehu commented Nov 12, 2024

singularity hiccup is fixed. the other tests shouldn't be picked up...

@famosab
Copy link
Contributor

famosab commented Nov 12, 2024

@sateeshperi Now the fq2bammeth test fails with
.command.sh: line 2: bwameth.py: command not found

any ideas?

@sateeshperi
Copy link
Contributor

that means the container profile is not invoked properly....hmmmm

@famosab
Copy link
Contributor

famosab commented Nov 12, 2024

Maybe we can remove it from this PR and tackle it separately. I would like to merge this to have the GPU profile in the CI ;)

@famosab
Copy link
Contributor

famosab commented Nov 12, 2024

@sateeshperi if you are fine with that then give us the approval :D

@famosab
Copy link
Contributor

famosab commented Nov 12, 2024

singularity hiccup is fixed. the other tests shouldn't be picked up...

I think the issue is back, back again @mashehu

@mashehu
Copy link
Contributor

mashehu commented Nov 12, 2024

There is a weird ci run, where the profile is empty. No idea how that can be. Will investigate tomorrow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

update module: parabricks/applybqsr
4 participants