The `work` directory

When running otb a sub-directory of the main directory called work will be created as a sub-directory of otb. This is directory is where nextflow computes each step of it processes. otb might add /busco and /singularity directories if configured to do so. He're is what a full /work directory might look like:

work/
├── 01
│   └── 5fd9920526ce4cd891b6181ddb1a70
│       ├── any2fasta_stats.flag.txt
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       ├── .exitcode
│       ├── right.fastq.gz -> /90daydata/project/ag100pest/software/OTB_test/0_Xylecopa_micans/RawHiC/JNBK_HiC_NA_NA_ACAGTG_Xylecopa_micans-Xylecopa_micans_HiC_I1143_L4_R2.fastq.gz
│       └── right.fastq.gz.stats
├── 1b
│   └── 1d5bf40157d16bca1b352914141131
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       ├── .exitcode
│       ├── jellyfish_version.flag.txt
│       └── version.txt -> /90daydata/project/ag100pest/software/OTB_test/0_Xylecopa_micans/work/42/0283004d9768bf75bf2ff72fef1031/version.txt
├── 22
│   └── 5d18f4291708f512b7a1be9148dd2a
│       ├── any2fasta_version.flag.txt
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       └── .exitcode
├── 39
│   └── e8f238662358a6b400104fd9dd0f0a
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       ├── .exitcode
│       └── hifiadapterfilt_version.flag.txt
├── 3c
│   └── 9475be059b3bd92f42172edda5d853
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       ├── .exitcode
│       └── shhquis_version.flag.txt
├── 42
│   └── 0283004d9768bf75bf2ff72fef1031
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       ├── .exitcode
│       ├── jellyfish.flag.txt
│       ├── left.fastq.gz -> /90daydata/project/ag100pest/software/OTB_test/0_Xylecopa_micans/RawHiC/JNBK_HiC_NA_NA_ACAGTG_Xylecopa_micans-Xylecopa_micans_HiC_I1143_L4_R1.fastq.gz
│       ├── reads.jf
│       ├── right.fastq.gz -> /90daydata/project/ag100pest/software/OTB_test/0_Xylecopa_micans/RawHiC/JNBK_HiC_NA_NA_ACAGTG_Xylecopa_micans-Xylecopa_micans_HiC_I1143_L4_R2.fastq.gz
│       ├── version.txt
│       └── Xylecopa_micans.histo
├── 43
│   └── 611cc48f608f83c5ddda5d8f8090ac
│       ├── bbtools_version.flag.txt
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       └── .exitcode
├── 51
│   └── 318c0111ff007513bca8b4b5df1416
│       ├── bcftools_version.flag.txt
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       └── .exitcode
├── 56
│   └── 7e48f7c96d4cefcdec1eda5bf4ac6b
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       ├── .exitcode
│       ├── genomescope_version.flag.txt
│       └── version.txt -> /90daydata/project/ag100pest/software/OTB_test/0_Xylecopa_micans/work/e8/22e87b3c09c5cd59a28b99cf21e31a/version.txt
├── 5c
│   └── cbff3d6fd1b76022d7f1ec52a00934
│       ├── busco_version.flag.txt
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       └── .exitcode
├── 67
│   └── 766c5a6cc10fc153b8f5b37c40a826
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       ├── .exitcode
│       └── hicstuff_version.flag.txt
├── 81
│   └── 2b38330f70f55413734e6a140cb3dc
│       ├── any2fasta_stats.flag.txt
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       ├── .exitcode
│       ├── left.fastq.gz -> /90daydata/project/ag100pest/software/OTB_test/0_Xylecopa_micans/RawHiC/JNBK_HiC_NA_NA_ACAGTG_Xylecopa_micans-Xylecopa_micans_HiC_I1143_L4_R1.fastq.gz
│       └── left.fastq.gz.stats
├── a3
│   └── 1a6a8bf4fb272772353cd89c763cb1
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       ├── .exitcode
│       └── ragtag_version.flag.txt
├── b3
│   └── a1899efe376be7777d1f6fe8d64baf
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       ├── .exitcode
│       └── samtools_version.flag.txt
├── ba
│   └── fe4de41ba6a5e575d6079beef371f5
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       ├── .exitcode
│       └── HiFiASM_version.flag.txt
├── e8
│   └── 22e87b3c09c5cd59a28b99cf21e31a
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       ├── .exitcode
│       ├── genomescope.flag.txt
│       ├── kcov.txt
│       ├── version.txt
│       ├── Xylecopa_micans
│       │   ├── fitted_hist.png
│       │   ├── linear_plot.png
│       │   ├── log_plot.png
│       │   ├── lookup_table.txt
│       │   ├── model.txt
│       │   ├── progress.txt
│       │   ├── summary.txt
│       │   ├── transformed_linear_plot.png
│       │   └── transformed_log_plot.png
│       └── Xylecopa_micans.histo -> /90daydata/project/ag100pest/software/OTB_test/0_Xylecopa_micans/work/42/0283004d9768bf75bf2ff72fef1031/Xylecopa_micans.histo
├── ec
│   └── efe6d84cbf60d95363237250212e5e
│       ├── check_fastq.flag.txt
│       ├── .command.begin
│       ├── .command.err
│       ├── .command.log
│       ├── .command.out
│       ├── .command.run
│       ├── .command.sh
│       ├── .exitcode
│       ├── JNBK_HiC_NA_NA_ACAGTG_Xylecopa_micans-Xylecopa_micans_HiC_I1143_L4_R1.fastq.gz -> /90daydata/project/ag100pest/software/OTB_test/0_Xylecopa_micans/RawHiC/JNBK_HiC_NA_NA_ACAGTG_Xylecopa_micans-Xylecopa_micans_HiC_I1143_L4_R1.fastq.gz
│       ├── JNBK_HiC_NA_NA_ACAGTG_Xylecopa_micans-Xylecopa_micans_HiC_I1143_L4_R2.fastq.gz -> /90daydata/project/ag100pest/software/OTB_test/0_Xylecopa_micans/RawHiC/JNBK_HiC_NA_NA_ACAGTG_Xylecopa_micans-Xylecopa_micans_HiC_I1143_L4_R2.fastq.gz
        └── out
            ├── left.fastq.gz -> ../JNBK_HiC_NA_NA_ACAGTG_Xylecopa_micans-Xylecopa_micans_HiC_I1143_L4_R1.fastq.gz
            └── right.fastq.gz -> ../JNBK_HiC_NA_NA_ACAGTG_Xylecopa_micans-Xylecopa_micans_HiC_I1143_L4_R2.fastq.gz

We can see that a hex code (eg. 1b, see above) is used, followed by a more unique identifier (eg. 1d5bf40157d16bca1b352914141131, see above) for each subprocess that nextflow runs. Since each of these directories is a process, we can just use one as an example, lets take a look at 1b:

1b/
└── 1d5bf40157d16bca1b352914141131
    ├── .command.begin
    ├── .command.err
    ├── .command.log
    ├── .command.out
    ├── .command.run
    ├── .command.sh
    ├── .exitcode
    ├── jellyfish_version.flag.txt
    └── version.txt -> /90daydata/project/ag100pest/software/OTB_test/0_Xylecopa_micans/work/42/0283004d9768bf75bf2ff72fef1031/version.txt

This process outputs the version of jellyfish. the .command.xxx files and exitcode files are created by nextflow. .command.log is the log from running that particular command. .command.out is the commands standard output and .command.err is that commands standard error. .command.run is the command that nextflow is running, it's a bash script that has some extras, where as .command.sh is the actual command to be run.

In this example we have jellyfish_version.flag.txt which is a flag that otb adds to tell which subprocess was run in which directory, and version.txt nextflow links files from one process to the next, instead of copying files, and this file came from the output of another process.