FileExistsError in medaka smolecule upon execution of the pipeline with the example data

[camcl]

Hi,

I have been interested in using your pipeline with ONT long-read data with UMIs that are generated by colleagues in my lab.
To begin with, I tried to execute the pipeline with the example data that are provided in the repository. I followed the instructions provided in the README, that is:

1. Clone the repository:
   git clone [email protected]:camcl/pipeline-umi-amplicon.git

2. Navigate to the cloned repository and finish the configuration and installation. I used the latest miniconda3:

cd pipeline-umi-amplicon
conda env create -f environment.yml
conda activate pipeline-umi-amplicon
cd lib && pip install . && cd ..

This ran without error and I have the following components in the conda environment:

(pipeline-umi-amplicon) camcl:~/pipeline-umi-amplicon$ conda list
# packages in environment at ~/miniconda3/envs/pipeline-umi-amplicon:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge                                                    
_openmp_mutex             4.5                       2_gnu    conda-forge                                                    
_tflow_select             2.3.0                       mkl                                                                   
absl-py                   2.1.0              pyhd8ed1ab_0    conda-forge                                                    
aioeasywebdav             2.4.0           py38h578d9bd_1001    conda-forge                                                  
aiohappyeyeballs          2.4.0              pyhd8ed1ab_0    conda-forge                                                    
aiohttp                   3.10.5           py38h5eee18b_0                                                                   
aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge                                                    
alsa-lib                  1.2.6.1              h7f98852_0    conda-forge                                                    
amply                     0.1.6              pyhd8ed1ab_0    conda-forge                                                    
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge                                                    
astunparse                1.6.3              pyhd8ed1ab_0    conda-forge                                                    
async-timeout             4.0.3              pyhd8ed1ab_0    conda-forge                                                    
attmap                    0.13.2             pyhd8ed1ab_0    conda-forge                                                    
attrs                     24.2.0             pyh71513ae_0    conda-forge                                                    
bcftools                  1.17                 h3cc50cf_1    bioconda                                                       
bcrypt                    3.2.0            py38h5eee18b_1                                                                   
bedtools                  2.31.1               hf5e1c6e_2    bioconda                                                       
blas                      1.0                    openblas                                                                   
blinker                   1.8.2              pyhd8ed1ab_0    conda-forge                                                    
boto3                     1.35.19            pyhd8ed1ab_0    conda-forge                                                    
botocore                  1.35.19         pyge38_1234567_0    conda-forge                                                   
bottleneck                1.3.7            py38ha9d4c09_0                                                                   
brotli-python             1.0.9            py38h6a678d5_8                                                                   
bzip2                     1.0.8                h4bc722e_7    conda-forge                                                    
c-ares                    1.33.1               heb4867d_0    conda-forge                                                    
ca-certificates           2024.8.30            hbcca054_0    conda-forge                                                    
cached-property           1.5.2                hd8ed1ab_1    conda-forge                                                    
cached_property           1.5.2              pyha770c72_1    conda-forge                                                    
cachetools                5.5.0              pyhd8ed1ab_0    conda-forge                                                    
cairo                     1.16.0            h18b612c_1001    conda-forge                                                    
certifi                   2024.8.30        py38h06a4308_0                                                                   
cffi                      1.16.0           py38h5eee18b_1                                                                   
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge                                                    
click                     8.1.7           unix_pyh707e725_0    conda-forge                                                  
cni                       1.0.1                ha975731_1    conda-forge                                                    
cni-plugins               1.3.0                ha8f183a_0    conda-forge                                                    
coin-or-cbc               2.10.12              h8b142ea_1    conda-forge                                                    
coin-or-cgl               0.60.7               h516709c_0    conda-forge                                                    
coin-or-clp               1.17.8               h1ee7a9c_0    conda-forge                                                    
coin-or-osi               0.108.10             haf5fa05_0    conda-forge                                                    
coin-or-utils             2.11.11              hee58242_0    conda-forge                                                    
coincbc                   2.10.12           1_metapackage    conda-forge                                                    
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge                                                    
configargparse            1.7                pyhd8ed1ab_0    conda-forge
connection_pool           0.0.3              pyhd3deb0d_0    conda-forge
cryptography              41.0.3           py38h130f0dd_0  
datrie                    0.8.2            py38h27cfd23_0  
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
docutils                  0.18.1           py38h06a4308_3  
dpath                     2.2.0              pyha770c72_0    conda-forge
dropbox                   12.0.2             pyhd8ed1ab_0    conda-forge
eido                      0.2.2              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.2.2              pyhd8ed1ab_0    conda-forge
expat                     2.6.3                h5888daf_0    conda-forge
filechunkio               1.8                        py_2    conda-forge
flatbuffers               24.3.25              h59595ed_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_2    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
frozenlist                1.4.0            py38h5eee18b_0  
ftputil                   5.1.0              pyhd8ed1ab_0    conda-forge
gast                      0.4.0              pyh9f0ad1d_0    conda-forge
giflib                    5.2.2                hd590300_0    conda-forge
gitdb                     4.0.11             pyhd8ed1ab_0    conda-forge
gitpython                 3.1.43             pyhd8ed1ab_0    conda-forge
glib                      2.78.4               h6a678d5_0  
glib-tools                2.78.4               h6a678d5_0  
google-api-core           2.19.2             pyhd8ed1ab_0    conda-forge
google-api-python-client  2.145.0            pyhd8ed1ab_0    conda-forge
google-auth               2.34.0             pyhff2d567_0    conda-forge
google-auth-httplib2      0.2.0              pyhd8ed1ab_0    conda-forge
google-auth-oauthlib      0.4.6              pyhd8ed1ab_0    conda-forge
google-cloud-core         2.4.1              pyhd8ed1ab_0    conda-forge
google-cloud-storage      2.18.2             pyhff2d567_0    conda-forge
google-crc32c             1.5.0            py38h5eee18b_0  
google-pasta              0.2.0              pyhd8ed1ab_1    conda-forge
google-resumable-media    2.7.2              pyhd8ed1ab_1    conda-forge
googleapis-common-protos  1.65.0             pyhd8ed1ab_0    conda-forge
graphite2                 1.3.14               h295c915_1  
grpc-cpp                  1.46.4               hbad87ad_7    conda-forge
grpcio                    1.42.0           py38hce63b2e_0  
gsl                       2.7                  he838d99_0    conda-forge
h5py                      3.11.0           py38hbe37b52_0  
harfbuzz                  4.3.0                hf52aaf7_1  
hdf5                      1.12.1               h70be1eb_2  
htslib                    1.17                 h6bc39ce_1    bioconda
httplib2                  0.22.0             pyhd8ed1ab_0    conda-forge
humanfriendly             10.0               pyhd8ed1ab_6    conda-forge
icu                       58.2              hf484d3e_1000    conda-forge
idna                      3.10               pyhd8ed1ab_0    conda-forge
importlib-metadata        8.5.0              pyha770c72_0    conda-forge
importlib_resources       6.4.5              pyhd8ed1ab_0    conda-forge
iniconfig                 2.0.0              pyhd8ed1ab_0    conda-forge
intervaltree              3.1.0              pyhd8ed1ab_1    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
jmespath                  1.0.1              pyhd8ed1ab_0    conda-forge
jpeg                      9e                   h0b41bf4_3    conda-forge
jq                        1.7.1                hd590300_0    conda-forge
jsonschema                4.23.0             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.12.1          pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2            py38h06a4308_0  
k8                        0.2.5                hdcf5f25_4    bioconda
keras                     2.10.0             pyhd8ed1ab_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.20.1               hf9c8cef_0    conda-forge
lcms2                     2.15                 hfd0df8a_0    conda-forge
ld_impl_linux-64          2.40                 hf3520f5_7    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libabseil                 20220623.0      cxx17_h05df665_6    conda-forge
libaec                    1.1.3                h59595ed_0    conda-forge
libarchive                3.5.2                hb890918_3    conda-forge
libblas                   3.9.0           23_linux64_openblas    conda-forge
libcblas                  3.9.0           23_linux64_openblas    conda-forge
libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
libcups                   2.3.3                h36d4200_3    conda-forge
libcurl                   7.87.0               h6312ad2_0    conda-forge
libdeflate                1.17                 h0b41bf4_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libexpat                  2.6.3                h5888daf_0    conda-forge
libffi                    3.4.4                h6a678d5_1  
libgcc                    14.1.0               h77fa898_1    conda-forge
libgcc-ng                 14.1.0               h69a702a_1    conda-forge
libgfortran               14.1.0               h69a702a_1    conda-forge
libgfortran-ng            14.1.0               h69a702a_1    conda-forge
libgfortran5              14.1.0               hc5f4f2c_1    conda-forge
libglib                   2.78.4               hdc74915_0  
libgomp                   14.1.0               h77fa898_1    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
liblapack                 3.9.0           23_linux64_openblas    conda-forge
liblapacke                3.9.0           23_linux64_openblas    conda-forge
libnghttp2                1.51.0               hdcd2b5c_0    conda-forge
libopenblas               0.3.27          pthreads_hac2b453_1    conda-forge
libpng                    1.6.43               h2797004_0    conda-forge
libprotobuf               3.20.3               h3eb15da_0    conda-forge
libseccomp                2.4.4                hd590300_4    conda-forge
libsodium                 1.0.20               h4ab18f5_0    conda-forge
libsqlite                 3.46.0               hde9e2c9_0    conda-forge
libssh2                   1.10.0               haa6b8db_3    conda-forge
libstdcxx                 14.1.0               hc0a3c3a_1    conda-forge
libstdcxx-ng              14.1.0               h4852527_1    conda-forge
libtiff                   4.5.0                h6adf6a1_2    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp-base              1.4.0                hd590300_0    conda-forge
libxcb                    1.16                 hb9d3cd8_1    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxml2                   2.9.14               h74e7548_0  
libzlib                   1.2.13               h4ab18f5_6    conda-forge
logmuse                   0.2.6              pyh8c360ce_0    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
lzo                       2.10              hd590300_1001    conda-forge
mappy                     2.28             py38h6bfa29d_2    bioconda
markdown                  3.6                pyhd8ed1ab_0    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.3            py38h5eee18b_0  
mdurl                     0.1.2              pyhd8ed1ab_0    conda-forge
medaka                    1.11.3           py38h2e44183_0    bioconda
minimap2                  2.28                 he4a0461_3    bioconda
multidict                 6.0.4            py38h5eee18b_0  
nbformat                  5.10.4             pyhd8ed1ab_0    conda-forge
ncurses                   6.5                  he02047a_1    conda-forge
numexpr                   2.8.4            py38hd2a5715_1  
numpy                     1.24.3           py38hf838250_0  
numpy-base                1.24.3           py38h1e6e340_0  
oauth2client              4.1.3                      py_0    conda-forge
oauthlib                  3.2.2              pyhd8ed1ab_0    conda-forge
oniguruma                 6.9.9                hd590300_0    conda-forge
ont-fast5-api             4.1.3              pyhdfd78af_0    bioconda
openjdk                   11.0.15              hc6918da_0    conda-forge
openssl                   1.1.1w               h7f8727e_0  
opt_einsum                3.3.0              pyhc1e730c_2    conda-forge
packaging                 24.1               pyhd8ed1ab_0    conda-forge
pandas                    2.0.3            py38h1128e8f_0  
paramiko                  2.8.1              pyhd3eb1b0_0  
parasail-python           1.3.4            py38h40d3509_2    bioconda
pcre                      8.45                 h9c3ff4c_0    conda-forge
pcre2                     10.42                hebb0a14_1  
peppy                     0.40.6             pyhd8ed1ab_0    conda-forge
perl                      5.32.1          7_hd590300_perl5    conda-forge
pip                       24.2               pyh8b19718_1    conda-forge
pixman                    0.38.0            h516909a_1003    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
plac                      1.4.3              pyhd8ed1ab_0    conda-forge
platformdirs              4.3.3              pyhd8ed1ab_0    conda-forge
pluggy                    1.5.0              pyhd8ed1ab_0    conda-forge
ply                       3.11               pyhd8ed1ab_2    conda-forge
pooch                     1.7.0            py38h06a4308_0  
prettytable               3.11.0             pyhd8ed1ab_0    conda-forge
progressbar33             2.4                        py_0    conda-forge
proto-plus                1.23.0             pyhd8ed1ab_0    conda-forge
protobuf                  3.20.3           py38h6a678d5_0  
psutil                    5.9.0            py38h5eee18b_0  
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pulp                      2.7.0            py38h578d9bd_1    conda-forge
pyabpoa                   1.5.2            py38h6bfa29d_1    bioconda
pyasn1                    0.6.1              pyhd8ed1ab_0    conda-forge
pyasn1-modules            0.4.1              pyhd8ed1ab_0    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pyjwt                     2.9.0              pyhd8ed1ab_1    conda-forge
pynacl                    1.3.0           py38h516909a_1001    conda-forge
pyopenssl                 23.2.0             pyhd8ed1ab_1    conda-forge
pyparsing                 3.1.4              pyhd8ed1ab_0    conda-forge
pysam                     0.21.0           py38h1c8baaf_0    bioconda
pysftp                    0.2.9                      py_1    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
pyspoa                    0.2.1            py38hd638cd3_3    bioconda
pytest                    8.3.3              pyhd8ed1ab_0    conda-forge
python                    3.8.16               h7a1cb2a_3  
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-edlib              1.3.9.post1      py38h2123bcc_0    bioconda
python-fastjsonschema     2.20.0             pyhd8ed1ab_0    conda-forge
python-flatbuffers        24.3.25            pyh59ac667_0    conda-forge
python-irodsclient        2.1.0              pyhd8ed1ab_0    conda-forge
python-tzdata             2024.1             pyhd8ed1ab_0    conda-forge
python_abi                3.8                      2_cp38    conda-forge
pytz                      2024.2             pyhd8ed1ab_0    conda-forge
pyu2f                     0.1.5              pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.1            py38h5eee18b_0  
re2                       2022.06.01           h27087fc_1    conda-forge
readline                  8.2                  h8228510_1    conda-forge
referencing               0.35.1             pyhd8ed1ab_0    conda-forge
requests                  2.32.3             pyhd8ed1ab_0    conda-forge
requests-oauthlib         2.0.0              pyhd8ed1ab_0    conda-forge
reretry                   0.11.8             pyhd8ed1ab_0    conda-forge
rich                      13.8.1             pyhd8ed1ab_0    conda-forge
rpds-py                   0.10.6           py38hb02cf49_0  
rsa                       4.9                pyhd8ed1ab_0    conda-forge
s3transfer                0.10.2             pyhd8ed1ab_0    conda-forge
samtools                  1.18                 hd87286a_0    bioconda
scipy                     1.10.1           py38h32ae08f_1  
seqkit                    2.8.2                h9ee0642_1    bioconda
setuptools                73.0.1             pyhd8ed1ab_0    conda-forge
singularity               3.8.6                h9c2343c_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
slacker                   0.14.0                     py_0    conda-forge
smart_open                7.0.4              pyhd8ed1ab_0    conda-forge
smmap                     5.0.0              pyhd8ed1ab_0    conda-forge
snakemake                 7.32.4               hdfd78af_1    bioconda
snakemake-minimal         7.32.4             pyhdfd78af_1    bioconda
snappy                    1.2.1                ha2e4443_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
sqlite                    3.46.0               h6d4b2fc_0    conda-forge
squashfs-tools            4.6.1                hdfefc0d_0    conda-forge
stone                     3.3.2              pyhd8ed1ab_0    conda-forge
stopit                    1.1.2                      py_0    conda-forge
tabix                     1.11                 hdfd78af_0    bioconda
tabulate                  0.9.0              pyhd8ed1ab_1    conda-forge
tar                       1.34                 hb2e2bae_1    conda-forge
tensorboard               2.10.1             pyhd8ed1ab_0    conda-forge
tensorboard-data-server   0.6.1            py38h52d8a92_0  
tensorboard-plugin-wit    1.8.1              pyhd8ed1ab_0    conda-forge
tensorflow                2.10.0          mkl_py38hd2379f1_0  
tensorflow-base           2.10.0          mkl_py38hb9daa73_0  
tensorflow-estimator      2.10.0           py38h06a4308_0  
termcolor                 2.4.0              pyhd8ed1ab_0    conda-forge
throttler                 1.2.2              pyhd8ed1ab_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
toposort                  1.10               pyhd8ed1ab_0    conda-forge
traitlets                 5.14.3             pyhd8ed1ab_0    conda-forge
typing-extensions         4.12.2               hd8ed1ab_0    conda-forge
typing_extensions         4.12.2             pyha770c72_0    conda-forge
tzdata                    2024a                h8827d51_1    conda-forge
ubiquerg                  0.8.0              pyhd8ed1ab_0    conda-forge
uritemplate               4.1.1              pyhd8ed1ab_0    conda-forge
urllib3                   1.26.19            pyhd8ed1ab_0    conda-forge
varscan                   2.4.6                hdfd78af_0    bioconda
veracitools               0.1.3                      py_0    conda-forge
vsearch                   2.28.1               h6a68c12_1    bioconda
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
werkzeug                  3.0.4              pyhd8ed1ab_0    conda-forge
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
wrapt                     1.14.1           py38h5eee18b_0  
wurlitzer                 3.1.1              pyhd8ed1ab_0    conda-forge
xorg-fixesproto           5.0               h7f98852_1002    conda-forge
xorg-inputproto           2.3.2             h7f98852_1002    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.9                hb711507_1    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxfixes            5.0.3             h7f98852_1004    conda-forge
xorg-libxi                1.7.10               h4bc722e_1    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-libxtst              1.2.5                h4bc722e_0    conda-forge
xorg-recordproto          1.14.2            h7f98852_1002    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.10               h5eee18b_1  
yaml                      0.2.5                h7f98852_2    conda-forge
yarl                      1.11.0           py38h5eee18b_0  
yte                       1.5.4              pyha770c72_0    conda-forge
zipp                      3.20.2             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               h4ab18f5_6    conda-forge
zstd                      1.5.6                ha6fb4c9_0    conda-forge

3. Testing the installation with snakemake -j 1 -pr --configfile config.yml does not produce any error:

Targets: EGFR_917
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job                   count
------------------  -------
copy_bed                  1
reads                     1
seqkit_bam_acc_tsv        1
total                     3

Select jobs to execute...

[Tue Sep 17 16:20:28 2024]
rule copy_bed:
    input: data/example_egfr_amplicon.bed
    output: example_egfr_single_read_run/targets.bed
    jobid: 1
    reason: Missing output files: example_egfr_single_read_run/targets.bed
    wildcards: name=example_egfr_single_read_run
    resources: tmpdir=/tmp

cp data/example_egfr_amplicon.bed example_egfr_single_read_run/targets.bed
[Tue Sep 17 16:20:28 2024]
Finished job 1.
1 of 3 steps (33%) done
Select jobs to execute...

[Tue Sep 17 16:20:28 2024]
rule seqkit_bam_acc_tsv:
    input: example_egfr_single_read_run/align/EGFR_917_consensus.bam
    output: example_egfr_single_read_run/stats/EGFR_917_consensus_size_vs_acc.tsv
    jobid: 13
    reason: Missing output files: example_egfr_single_read_run/stats/EGFR_917_consensus_size_vs_acc.tsv
    wildcards: name=example_egfr_single_read_run, target=EGFR_917, stage=consensus
    resources: tmpdir=/tmp


        echo -e "Read	Cluster_size	Ref	MapQual	Acc	ReadLen	RefLen	RefAln	RefCov	ReadAln	ReadCov	Strand	MeanQual	LeftClip	RightClip	Flags	IsSec	IsSup" > example_egfr_single_read_run/stats/EGFR_917_consensus_size_vs_acc.tsv && seqkit bam example_egfr_single_read_run/align/EGFR_917_consensus.bam 2>&1 | sed 's/_/	/' | tail -n +2 >> example_egfr_single_read_run/stats/EGFR_917_consensus_size_vs_acc.tsv
        
[Tue Sep 17 16:20:29 2024]
Finished job 13.
2 of 3 steps (67%) done
Select jobs to execute...

[Tue Sep 17 16:20:29 2024]
localrule reads:
    input: example_egfr_single_read_run/targets.bed, example_egfr_single_read_run/align/EGFR_917_final.bam.bai, example_egfr_single_read_run/stats/EGFR_917_vsearch_cluster_stats.tsv, example_egfr_single_read_run/stats/EGFR_917_consensus_size_vs_acc.tsv
    jobid: 0
    reason: Input files updated by another job: example_egfr_single_read_run/stats/EGFR_917_consensus_size_vs_acc.tsv, example_egfr_single_read_run/targets.bed
    resources: tmpdir=/tmp

[Tue Sep 17 16:20:29 2024]
Finished job 0.
3 of 3 steps (100%) done
Complete log: .snakemake/log/2024-09-17T162028.348925.snakemake.log

4. Without editing anything in config.yml, I ran the command snakemake -j 30 reads --configfile config.yml. All steps until the rule polish clusters complete but the execution terminates upon polishing with the following output:

[Tue Sep 17 16:42:44 2024]
Error in rule polish_clusters:
    jobid: 6
    input: example_egfr_single_read_run/clustering/EGFR_917/clusters_fa, example_egfr_single_read_run/clustering/EGFR_917/smolecule_clusters.fa
    output: example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp, example_egfr_single_read_run/fasta/EGFR_917_consensus.bam, example_egfr_single_read_run/fasta/EGFR_917_consensus.fasta
    shell:
        
	rm -rf example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp
	medaka smolecule --threads 30 --length 50 --depth 2 --model r941_min_high_g360 --method spoa example_egfr_single_read_run/clustering/EGFR_917/smolecule_clusters.fa example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp 2> example_egfr_single_read_run/fasta/EGFR_917_consensus.bam_smolecule.log
	cp example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp/consensus.fasta example_egfr_single_read_run/fasta/EGFR_917_consensus.fasta
	cp example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp/subreads_to_spoa.bam example_egfr_single_read_run/fasta/EGFR_917_consensus.bam && cp example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp/subreads_to_spoa.bam.bai example_egfr_single_read_run/fasta/EGFR_917_consensus.bam.bai
	
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-09-17T164240.824646.snakemake.log

The contents of the file example_egfr_single_read_run/fasta/EGFR_917_consensus.bam_smolecule.log provide more information about the error:

Traceback (most recent call last):
  File "~/miniconda3/envs/pipeline-umi-amplicon/bin/medaka", line 11, in <module>
    sys.exit(main())
  File "~/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/medaka.py", line 814, in main
    args.func(args)
  File "~/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/smolecule.py", line 429, in main
    medaka.common.mkdir_p(args.output, info='Results will be overwritten.')
  File "~/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/common.py", line 763, in mkdir_p
    os.makedirs(path)
  File "~/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: 'example_egfr_single_read_run/clustering/EGFR_917/smolecule_clusters.fa'

What have I done wrong?

Regards,

Camille C.

[camcl]

Fixes here: #35