Code

host_filter.wdl modernization#70

Code

Merged

rzlim08 merged 83 commits intomainchanzuckerberg/czid-workflows:mainfrom mlin/modernize-host-filterchanzuckerberg/czid-workflows:mlin/modernize-host-filterCopy head branch name to clipboardApr 25, 2023

+1,894-733

Commits

Commits on Jan 27, 2023

• fastp

  

  mlin

  authored and

  valenzuelaomar

  committed
  e92a5e0

  Copy full SHA for e92a5e0

• fastp single

  

  mlin

  authored and

  valenzuelaomar

  committed
  24f6841

  Copy full SHA for 24f6841

• bowtie2 run

  

  mlin

  authored and

  valenzuelaomar

  committed
  06517ea

  Copy full SHA for 06517ea

• hisat2 run

  

  mlin

  authored and

  valenzuelaomar

  committed
  6c485e4

  Copy full SHA for 6c485e4

• dedup run

  

  mlin

  authored and

  valenzuelaomar

  committed
  623a906

  Copy full SHA for 623a906

• run subsample

  

  mlin

  authored and

  valenzuelaomar

  committed
  0c75c04

  Copy full SHA for 0c75c04

• run kallisto

  

  mlin

  authored and

  valenzuelaomar

  committed
  c45b5c3

  Copy full SHA for c45b5c3

• adjust index tar filenames

  

  mlin

  authored and

  valenzuelaomar

  committed
  18917cc

  Copy full SHA for 18917cc

• polishing

  

  mlin

  authored and

  valenzuelaomar

  committed
  029ab72

  Copy full SHA for 029ab72

• polishing

  

  mlin

  authored and

  valenzuelaomar

  committed
  15d65fe

  Copy full SHA for 15d65fe

• count reads in each step

  

  mlin

  authored and

  valenzuelaomar

  committed
  bc8aebe

  Copy full SHA for bc8aebe

• Create host_filter_indexing.wdl

  

  mlin

  authored and

  valenzuelaomar

  committed
  db223c8

  Copy full SHA for db223c8

• boost fastp complexity threshold

  

  mlin

  authored and

  valenzuelaomar

  committed
  d0a5ebe

  Copy full SHA for d0a5ebe

• output fastp report

  

  mlin

  authored and

  valenzuelaomar

  committed
  b60afa1

  Copy full SHA for b60afa1

• build fastp from our fork with SDUST complexity filtering

  

  mlin

  authored and

  valenzuelaomar

  committed
  8e308bd

  Copy full SHA for 8e308bd

• use fastp --sdust_complexity_filter

  

  mlin

  authored and

  valenzuelaomar

  committed
  17e31c5

  Copy full SHA for 17e31c5

• bump

  

  mlin

  authored and

  valenzuelaomar

  committed
  7bcb00f

  Copy full SHA for 7bcb00f

• bump

  

  mlin

  authored and

  valenzuelaomar

  committed
  b70ce8f

  Copy full SHA for b70ce8f

• tune

  

  mlin

  authored and

  valenzuelaomar

  committed
  052120e

  Copy full SHA for 052120e

• stub the remaining step descriptions

  

  mlin

  authored and

  valenzuelaomar

  committed
  169336f

  Copy full SHA for 169336f

• wire to tests

  

  mlin

  authored and

  valenzuelaomar

  committed
  f2e27ca

  Copy full SHA for f2e27ca

• and auto_benchmark

  

  mlin

  authored and

  valenzuelaomar

  committed
  d7b3959

  Copy full SHA for d7b3959

• fixup tests

  

  mlin

  authored and

  valenzuelaomar

  committed
  22cc6c2

  Copy full SHA for 22cc6c2

• fixup tests

  

  mlin

  authored and

  valenzuelaomar

  committed
  0089204

  Copy full SHA for 0089204

• fixup tests

  

  mlin

  authored and

  valenzuelaomar

  committed
  a17a508

  Copy full SHA for a17a508

• fixup tests

  

  mlin

  authored and

  valenzuelaomar

  committed
  f96344e

  Copy full SHA for f96344e

• fixup tests

  

  mlin

  authored and

  valenzuelaomar

  committed
  97fbb83

  Copy full SHA for 97fbb83

• fixup tests

  

  mlin

  authored and

  valenzuelaomar

  committed
  101c8dc

  Copy full SHA for 101c8dc

• add back in picard CollectInsertSizeMetrics

  

  mlin

  authored and

  valenzuelaomar

  committed
  4b2fbea

  Copy full SHA for 4b2fbea

• picard step description

  

  mlin

  authored and

  valenzuelaomar

  committed
  5317bb8

  Copy full SHA for 5317bb8

• host_filter_2022.wdl => host_filter.wdl

  

  mlin

  authored and

  valenzuelaomar

  committed
  6ef7035

  Copy full SHA for 6ef7035

• polish

  

  mlin

  authored and

  valenzuelaomar

  committed
  5062f19

  Copy full SHA for 5062f19

• restore fastqs_0 and fastqs_1 to minimize collateral changes

  

  mlin

  authored and

  valenzuelaomar

  committed
  713d414

  Copy full SHA for 713d414

• add minimap2 index build

  

  mlin

  authored and

  valenzuelaomar

  committed
  9e2612e

  Copy full SHA for 9e2612e

• picard_insert_metrics.txt

  

  mlin

  authored and

  valenzuelaomar

  committed
  ca47e36

  Copy full SHA for ca47e36

• amr/run.wdl workaround

  

  mlin

  authored and

  valenzuelaomar

  committed
  378573c

  Copy full SHA for 378573c

• index multiple transcripts_fasta_gz

  

  mlin

  authored and

  valenzuelaomar

  committed
  aeeb7bc

  Copy full SHA for aeeb7bc

• make gtf optional

  

  mlin

  authored and

  valenzuelaomar

  committed
  a897fe6

  Copy full SHA for a897fe6

• allow uncompressed genome fasta

  

  mlin

  authored and

  valenzuelaomar

  committed
  0d66e58

  Copy full SHA for 0d66e58

• allow uncompressed genome fasta

  

  mlin

  authored and

  valenzuelaomar

  committed
  1606cc9

  Copy full SHA for 1606cc9

• allow uncompressed genome fasta

  

  mlin

  authored and

  valenzuelaomar

  committed
  0d0cd69

  Copy full SHA for 0d0cd69

• bump minimap2 memory

  

  mlin

  authored and

  valenzuelaomar

  committed
  de5b5c1

  Copy full SHA for de5b5c1

• bump minimap2 memory

  

  mlin

  authored and

  valenzuelaomar

  committed
  9307f80

  Copy full SHA for 9307f80

• step descriptions -- first draft

  

  mlin

  authored and

  valenzuelaomar

  committed
  03d367e

  Copy full SHA for 03d367e

• add indexing driver & draft readme

  

  Mike Lin

  authored and

  valenzuelaomar

  committed
  10d3219

  Copy full SHA for 10d3219

• include invocations in step descriptions

  

  mlin

  authored and

  valenzuelaomar

  committed
  7126058

  Copy full SHA for 7126058

• rebase amr fix

  

  mlin

  authored and

  valenzuelaomar

  committed
  afcb3fb

  Copy full SHA for afcb3fb

• load card_json

  valenzuelaomar

  committed
  9fb0824

  Copy full SHA for 9fb0824

• run kallisto every time

  valenzuelaomar

  committed
  0071184

  Copy full SHA for 0071184

• fix amr wdl

  valenzuelaomar

  committed
  06db702

  Copy full SHA for 06db702

• fix short-read-mngs rebase weirdness

  valenzuelaomar

  committed
  14ba43b

  Copy full SHA for 14ba43b

• add final things

  valenzuelaomar

  committed
  884b8a2

  Copy full SHA for 884b8a2

Commits on Jan 30, 2023

• [modernized host filter] add ERCC and gene-level outputs to kallisto (#175)

  Show description for ff8b815
  mlin

  authored
  ff8b815

  Copy full SHA for ff8b815

• load card_json explicitly

  valenzuelaomar

  committed
  c29ed76

  Copy full SHA for c29ed76

• add ~

  valenzuelaomar

  committed
  448f997

  Copy full SHA for 448f997

Commits on Feb 1, 2023

• fix host_filter unit tests

  mlin

  committed
  61d8767

  Copy full SHA for 61d8767

• fix host_filter unit tests

  mlin

  committed
  9e1ab88

  Copy full SHA for 9e1ab88

• bowtie2: sort by read name for better reproducibility

  mlin

  committed
  f9c49ac

  Copy full SHA for f9c49ac

Commits on Feb 2, 2023

• update minimap2 indexing invocation

  mlin

  committed
  33421ed

  Copy full SHA for 33421ed

Commits on Feb 3, 2023

• add chelonia_mydas, drosophila_melanogaster, gray_whale, pea-aphid

  mlin

  committed
  7cc1bbe

  Copy full SHA for 7cc1bbe

Commits on Feb 4, 2023

• copy-paste {bowtie2,hisat2}_human_filter to support pipeline viz

  mlin

  committed
  fa870a6

  Copy full SHA for fa870a6

Commits on Feb 5, 2023

• allow kallisto nonzero exit

  mlin

  committed
  110574c

  Copy full SHA for 110574c

Commits on Feb 9, 2023

• rename modern host filtering inputs/outputs and create a 1-1 mapping between inputs/outputs

  valenzuelaomar

  committed
  cea0ba7

  Copy full SHA for cea0ba7

• fix lint issue

  valenzuelaomar

  committed
  20fd60d

  Copy full SHA for 20fd60d

• Merge remote-tracking branch 'origin/main' into mlin/modernize-host-filter

  mlin

  committed
  a1ae1c2

  Copy full SHA for a1ae1c2

• rename reads_in_count to input_read_count

  mlin

  committed
  2a41734

  Copy full SHA for 2a41734

• auto_benchmark updates

  mlin

  committed
  96f77b7

  Copy full SHA for 96f77b7

• fix test_RunCZIDDedup_safe_csv

  mlin

  committed
  bf3a134

  Copy full SHA for bf3a134

• rename kallisto output files

  mlin

  committed
  1aba733

  Copy full SHA for 1aba733

Commits on Feb 12, 2023

• update mosquitos with several Culicidae

  mlin

  committed
  35636e4

  Copy full SHA for 35636e4

Commits on Feb 16, 2023

• add files to wdl output for pipeline viz compatibility

  valenzuelaomar

  committed
  90cea20

  Copy full SHA for 90cea20

• convert headers in descriptions to bolded text

  valenzuelaomar

  committed
  81ea0ad

  Copy full SHA for 81ea0ad

Commits on Mar 31, 2023

• Merge branch 'main' into mlin/modernize-host-filter

  mlin

  authored
  40d2052

  Copy full SHA for 40d2052

• delete host_filter_indexing since it's subsumed in #182

  mlin

  committed
  e35cd04

  Copy full SHA for e35cd04

Commits on Apr 14, 2023

• Merge branch 'main' into mlin/modernize-host-filter

  valenzuelaomar

  authored
  8e981c4

  Copy full SHA for 8e981c4

Commits on Apr 18, 2023

• fix glob patterns in read counting

  mlin

  committed
  aeb234f

  Copy full SHA for aeb234f

• Revert "fix glob patterns in read counting"

  Show description for 74304e0
  mlin

  committed
  74304e0

  Copy full SHA for 74304e0

• [Bug] fix count expansion for single file short-read-mngs (#216)

  Show description for ec91bf3
  

  rzlim08

  and

  valenzuelaomar

  authored
  ec91bf3

  Copy full SHA for ec91bf3

Commits on Apr 19, 2023

• Merge branch 'main' into mlin/modernize-host-filter

  valenzuelaomar

  authored
  021d183

  Copy full SHA for 021d183

• Remove AMR changes that are a WIP from modern host filtering branch (#219)

  Show description for 227a489
  valenzuelaomar

  authored
  227a489

  Copy full SHA for 227a489

Commits on Apr 21, 2023

• tune hisat2 memory usage (#223)

  rzlim08

  authored
  3ad5ad0

  Copy full SHA for 3ad5ad0

Commits on Apr 25, 2023

• Legacy Host Filter initial commit (#224)

  Show description for 3976349
  rzlim08

  authored
  3976349

  Copy full SHA for 3976349

• Revert "Remove AMR changes that are a WIP from modern host filtering branch (#219)" (#226)

  Show description for ba46f53
  rzlim08

  authored
  ba46f53

  Copy full SHA for ba46f53