Workflow issue with DSL2

[hkaspersen]

Hello everyone,

I have a pipeline that I wrote with DSL2, and I am having an issue with defining workflows. The issue started when I wanted to convert my pipeline to using singularity images rather than conda environments. I updated my nextflow.config file with lines like this:

process {
        withName: PARSNP {
                container = "https://depot.galaxyproject.org/singularity/parsnp:1.5.6--h9a82719_1"
        }
        withName: SNIPPY {
                container = "https://depot.galaxyproject.org/singularity/snippy:4.6.0--hdfd78af_1"
        }

singularity {
        singularity.enabled     = true
}

However, when I added this, I got the warnings:

WARN: There's no process matching config selector: PARSNP
WARN: There's no process matching config selector: SNIPPY

I read on some github issues that this is fixed in a newer version of nextflow (as mentioned here), so I updated to the newest version (21.10.6). However, now I got another warning that I cannot seem to get around:
WARN: No entry workflow specified

My workflow looks like this (a condensed version at least):

if (params.type == "mapping") {
        include { SNIPPY } from "${params.module_dir}/SNIPPY.nf"
  
        workflow MAPPING {
                reads_ch=channel.fromPath(params.reads, checkIfExists: true)
                                .collect()

                SNIPPY(reads_ch)
        }
}
if (params.type == "core_genome") {
      include { PARSNP } from "${params.module_dir}/PARSNP.nf"

      workflow CORE_GENOME {
             assemblies_ch=channel.fromPath(params.assemblies, checkIfExists: true)
                                      .collect()

            PARSNP(assemblies_ch)
       }
}

I tried to look in the update log for any changes in the syntax here, but I cannot seem to find anything. Has there been an update changing the way we define/use workflows?

Any help is greatly appreciated!

[manuelesimi]

Hi @hkaspersen,

With DSL2 you must define an entry point of execution, the entry workflow. More or less it's like the main static method in a Java app. There are two ways to define it:

1. a workflow with no name (you could wrap the code above into a workflow {} block).
2. specify the entry workflow with -entry.

See here for details.

[hkaspersen]

Thank you for a quick reply!
I tried this, however now I get the error message:
Include statement is not allowed within a workflow definition

If I have an if statement that selects which workflow to run based on a param, where should I put the include parameters?

[manuelesimi]

include must be outside any workflow block :)

[manuelesimi]

include { SNIPPY } from "${params.module_dir}/SNIPPY.nf"
include { PARSNP } from "${params.module_dir}/PARSNP.nf"

workflow {
 if (params.type == "mapping") {  
        workflow MAPPING {
                reads_ch=channel.fromPath(params.reads, checkIfExists: true)
                                .collect()

                SNIPPY(reads_ch)
        }
 }
 if (params.type == "core_genome") {

      workflow CORE_GENOME {
             assemblies_ch=channel.fromPath(params.assemblies, checkIfExists: true)
                                      .collect()

            PARSNP(assemblies_ch)
       }
  }
}

You might want to do not include the process you're not using. For that, I'd define the 2 sub-workflows in 2 external modules and import each process in its respective module.

[hkaspersen]

I see. I have now moved the workflows to separate .nf files, and put them in a directoy called workflows. In the main.nf file i then have the following:

// Modules
include { SNIPPY } from "${params.module_dir}/SNIPPY.nf"
include { PARSNP } from "${params.module_dir}/PARSNP.nf"

// Workflows
include { MAPPING } from "${params.workflow_dir}/MAPPING.nf"
include { CORE_GENOME } from "${params.workflow_dir}/CORE_GENOME.nf"

workflow {
    if (params.type == "mapping") {  
            MAPPING()
    }
    if (params.type == "core_genome") {
            CORE_GENOME()
    }
}

However, with this, it cannot seem to find the modules connected to the workflows. For example, I get the error:
Missing process or function with name 'PARSNP'
Note that no include statements are in the workflow definitions in the workflow directory, they are only defined in main.nf.
What am I missing here?

[hkaspersen]

Nevermind, I moved the incude statements inside the workflow definitions instead, it was a referencing issue to the module directory from the workflow directory. Is there any smart way to refer to the main run directory (i.e. the directory where the nextflow.config is)?

[manuelesimi]

Probably workflow.projectDir or workflow.launchDir (depending on your needs) would do it.
See https://www.nextflow.io/docs/latest/metadata.html#runtime-metadata