For Slurm cluster, how do I stage files local to the launcher node?

[siddharthab]

My pipeline inputs are not on an NFS, but on the launcher node. This is so that I can use watchPath more easily with them. When using the Slurm executor, because the paths are absolute posix paths, the staging process simply creates a dangling symlink rather than copying the file from my launcher node to the work directory.

Is there a way to specify that the file should be copied to the work directory on the launcher node first?

[siddharthab]

A workaround is to have a dummy process with executor directive set to local and stageInMode set to copy. Then if we declare the output of this process to be the same as the input, the resulting out channel will be all properly staged files.

[bentsherman]

I think input files basically need to be accessible to both the launcher node and compute nodes for Nextflow's staging logic to work.

For remote inputs like HTTP or S3, Nextflow will download the file into the work directory. But for local files, Nextflow assumes that the compute nodes can already access it (there is no way for it to know otherwise), so it just makes a symlink