Skip to content

Commit

Permalink
Add pipeline parameters section
Browse files Browse the repository at this point in the history
  • Loading branch information
fasterius committed Mar 18, 2022
1 parent bf6e5a1 commit f8b755d
Showing 1 changed file with 87 additions and 1 deletion.
88 changes: 87 additions & 1 deletion _episodes/01-getting-started-with-nextflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ To run a Nextflow script use the command `nextflow run <script_name>`.
> > You should see output similar to the text shown below:
> >
> > ~~~
> > N E X T F L O W ~ version 20.10.0
> > N E X T F L O W ~ version 21.04.3
> > Launching `word_count.nf` [fervent_babbage] - revision: c54a707593
> > executor > local (1)
> > [21/b259be] process > NUM_LINES (1) [100%] 1 of 1 ✔
Expand All @@ -281,4 +281,90 @@ To run a Nextflow script use the command `nextflow run <script_name>`.
> {: .solution}
{: .challenge}
## Pipeline parameters
The Nextflow `word_count.nf` script defines a pipeline parameter
`params.input`. Pipeline parameters enable you to change the input to the
workflow at runtime, via the command line or a configuration file, so they are
not hard-coded into the script.
Pipeline parameters are declared in the workflow by prepending the prefix
`params`, separated by dot character, to a variable name e.g., `params.input`.
Their value can be specified on the command line by prefixing the parameter
name with a double dash character, e.g., `--input`.
We can also use wild cards to specify multiple input files. In the example
below we use the `*` to match any sequence of characters with `.fastq.gz`. Note
that if you use wild card characters on the command line you must enclose the
value in quotes.
Re-run the Nextflow script by entering the following command in your terminal:
~~~
nextflow run word_count.nf --input 'data/untrimmed_fastq/*.fastq.gz'
~~~~
{: .language-bash}
~~~
N E X T F L O W ~ version 21.04.3
Launching `word_count.nf` [desperate_agnesi] - revision: bf77afb9d7
executor > local (6)
[60/584db8] process > NUM_LINES (5) [100%] 6 of 6 ✔
SRR2589044_1.fastq.gz 4428360
SRR2589044_2.fastq.gz 4428360
SRR2584863_1.fastq.gz 6213036
SRR2584863_2.fastq.gz 6213036
SRR2584866_2.fastq.gz 11073592
SRR2584866_1.fastq.gz 11073592
~~~~
{: .output}

The pipeline executes the process 6 times; one process for each file matching
the string. Since each process is executed in parallel, there is no guarantee
of which output is reported first. When you run this script, you may see the
process output in a different order.

Nextflow stores intermediate files in a `work` sub-directory in your current
directory, which might look like this:

~~~
work/
├── 13
│ └── 46936e3927b74ea6e5555ce7b7b56a
│ └── SRR2589044_2.fastq.gz -> nextflow_tutorial/data/untrimmed_fastq/SRR2589044_2.fastq.gz
├── 43
│ └── 3819a4bc046dd7dc528de8eae1c6b8
│ └── SRR2584863_1.fastq.gz -> nextflow_tutorial/data/untrimmed_fastq/SRR2584863_1.fastq.gz
├── 5a
│ └── d446a792db2781ccd0c7aaafdff329
│ └── SRR2584866_2.fastq.gz -> nextflow_tutorial/data/untrimmed_fastq/SRR2584866_2.fastq.gz
├── 76
│ └── b1e11f2e706b901e0c57768a2af59f
│ └── SRR2589044_1.fastq.gz -> nextflow_tutorial/data/untrimmed_fastq/SRR2589044_1.fastq.gz
├── 9c
│ └── 1b1ebc2ea11a395e4e9dcf805b2c7d
│ └── SRR2584863_2.fastq.gz -> nextflow_tutorial/data/untrimmed_fastq/SRR2584863_2.fastq.gz
└── ce
└── b94ef609ee54f4b7ea79dc23eb32bb
└── SRR2584866_1.fastq.gz -> nextflow_tutorial/data/untrimmed_fastq/SRR2584866_1.fastq.gz
12 directories, 6 files
~~~~
{: .output}
Nextflow has a log command to display the logs of all previous executions. Type
the following on the command line to display an output similar as below:
~~~
nextflow log
~~~~
{: .language-bash}
~~~
TIMESTAMP DURATION RUN NAME STATUS REVISION ID SESSION ID COMMAND
2021-11-16 07:17:23 5.9s irreverent_leakey OK bf77afb9d7 17d06cd0-3bb9-4d32-9d75-48bfdf5401a9 nextflow run word_count.nf
2021-11-16 07:23:00 11.1s desperate_agnesi OK bf77afb9d7 41f78242-27d7-462a-a88d-80b6ec9dc5db nextflow run word_count.nf --input 'data/untrimmed_fastq/*.fastq.gz'
~~~~
{: .output}

{% include links.md %}

0 comments on commit f8b755d

Please sign in to comment.