Increase in runtime

[ramirobarrantes]

Hello, first for all, thank you so much for providing this great resource. We have been noticing that for the last couple of months or so the runs have been taking considerably longer, it used to be 6-7 hours, now it's more than 30. Is there a reason for that? Thank you again very much.

[erinyoung]

Below is a graph of the average time a process takes. This one is from mid-September. The process that takes the longest to run per sample is Freyja.

There was an attempt to alleviate this problem in version 3.820230926 by requiring a minimum depth. This hasn't really made an impact, however.

[erinyoung]

I have thought about dropping Freyja boot.

[ramirobarrantes]

Hi Erin, Thank you. So I think that the issues started with version 3.8.20230915<https://github.com/UPHL-BioNGS/Cecret/releases/tag/3.8.20230915> . Before that things were taking 6-7 hours and in this one they started to take more than 24 hours. It seems that it is the Freyja upgrade and there is not much we can do at the moment correct? Thanks, Ramiro From: Young ***@***.***> Reply-To: UPHL-BioNGS/Cecret ***@***.***> Date: Wednesday, November 1, 2023 at 11:40 AM To: UPHL-BioNGS/Cecret ***@***.***> Cc: Ramiro ***@***.***>, Author ***@***.***> Subject: Re: [UPHL-BioNGS/Cecret] Increase in runtime (Issue #249) Resent-From: ***@***.***> Resent-Date: Wednesday, November 1, 2023 at 11:40 AM [This message was sent from an address outside the Larner College of Medicine. Please exercise caution when clicking links or opening attachments from this source. ] Below is a graph of the average time a process takes. This one is from mid-September. The process that takes the longest to run per sample is Freyja. There was an attempt to alleviate this problem in version 3.820230926<https://github.com/UPHL-BioNGS/Cecret/releases/tag/3.8.20230926'> by requiring a minimum depth. This hasn't really made an impact, however. [newplot]<https://user-images.githubusercontent.com/28793649/279720688-df648e36-b8a7-47ca-882f-7e133ed2711a.png> — Reply to this email directly, view it on GitHub<#249 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADS2KRYTBPOH3PCHFKQTLGDYCJUNRAVCNFSM6AAAAAA6ZMWKOCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBZGE4DCOJQGQ>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[erinyoung]

If you don't need the results from Freyja, you can change the freyja parameter to false.

In a config file add a params.freyja = false line, and on the command line it's something like

nextflow run UPHL-BioNGS/Cecret <everything else you normally run> --freyja false

This should skip Freyja altogether?

[ramirobarrantes]

Thank you very much! This is very clear and it’s not a problem, we just need to make sure that we are not doing anything wrong and having the option to not do this is helpful. Just curious, do you know the reason why Freyja started to take so much longer after the update? From: Young ***@***.***> Reply-To: UPHL-BioNGS/Cecret ***@***.***> Date: Wednesday, November 1, 2023 at 12:28 PM To: UPHL-BioNGS/Cecret ***@***.***> Cc: Ramiro ***@***.***>, Author ***@***.***> Subject: Re: [UPHL-BioNGS/Cecret] Increase in runtime (Issue #249) Resent-From: ***@***.***> Resent-Date: Wednesday, November 1, 2023 at 12:28 PM [This message was sent from an address outside the Larner College of Medicine. Please exercise caution when clicking links or opening attachments from this source. ] If you don't need the results from Freyja, you can change the freyja parameter to false. In a config file add a params.freyja = false line, and on the command line it's something like nextflow run UPHL-BioNGS/Cecret <everything else you normally run> --freyja false This should skip Freyja altogether? — Reply to this email directly, view it on GitHub<#249 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADS2KR4ZAXYGF6IWPYHBBDTYCJ2ATAVCNFSM6AAAAAA6ZMWKOCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBZGI3DKMJYGY>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[erinyoung]

I'm not sure. I didn't even notice until this summer, so I'm thinking it's something with version 1.4.5+.

[erinyoung]

As an update to this issue, I have created a PR where freyja boot has been removed : #250

This makes the workflow much faster.

As soon as this PR passes testing, we'll merge it and it will be the default workflow.