[FEATURE EPIC] Call SNPs, Indels & SVs with iGenVar

[Irallia]

This is an overview over all epics.

As a geneticist for rare diseases, I would like to analyze the deletions in a patient's genome so that the events characterized by the disease can be detected with the help of databases. This helps to narrow down the diagnosis and to initiate tailored therapies corresponding to the genon type.

This includes the following aspects:

• Input: We want to allow short and long reads and deal with them differently.
  • Create a Structure for BAM Indexing cxx20: return-type-requirement is not a type-constraint  product_backlog#88
• Algorithms: We want to use various algorithms for the various inputs and outputs.
  • Call SNPs & Indels: Since GATK is currently the best standard, we want to use these methods. The tool HaplotypeCaller, written here in Java, should be translated into C ++ for this purpose.
    => As a python binding writer I want dynamic search configurations so I can handle the different configurations with runtime dispatching. product_backlog#31
  • Call SVs: For larger structural variations, we want to combine the various known methods to call deletions.
    => Balanced Binning Directory product_backlog#32
  • Add all Methods of Vaquita
    => Minimiser View works with reverse k-mers product_backlog#84
• Output: We want to output the deletions in VCF format using a VCF parser from SeqAn3 (needs to be implemented).
  => [Search] Use a buffer to cache the intermediate results between search incovations such that memory allocations are reduced on multiple invocations of search. product_backlog#29 ✅
• We want to modularise the different parts of IGenVar so that the user can decide which methods to use and so that we can compare different combinations of methods more easily.
  => As application developer I want built application binaries put into a bin directory so I can easily find the binaries after building. product_backlog#44 ✅
• Testing: We want to test all functionalities and also prove this with code coverage.
  => Improve the documentation for the search module product_backlog#30 ✅
• Refinements, bugs, and requests
  => [Search] Add support for generalised CSAs in the sdsl, such that we can index text collections without artificially adding a sentinel character to our indices. product_backlog#24

---

Input

Differentiating between the inputs will be processed in the course of Issue seqan/product_backlog#17. ✅
Create a Structure for BAM Indexing seqan/product_backlog#88

Algorithms

Call SNPs & Indels:

-> generating candidate haplotypes
-> local realignment using the pair HMM Model against the candidate haplotypes -> matrix of likelihoods for each read
-> local assembly: assemble these window aligned reads into an assembly graph of local variation
-> infer variants from assembled haplotypes: "Despite its name, HaplotypeCaller does not actually call haplotypes. Rather, it generates haplotypes as an intermediate step to discover variants at individual loci. Here we describe how the GATK engine determines which alt alleles exist in locally assembled haplotypes." (-> variant qualtiy score model)

Call SVs:

• Distinguish between long and short reads in the input As application developer I want automatic fetching of test data so I can run my tests without worrying about where the test data is downloaded. product_backlog#43 ✅

Call Deletions from long reads seqan/product_backlog#32 ✅

• Call Deletions via the CIGAR string ✅
• Call Deletion using split alignment EPR Dictionaries product_backlog#18 ✅
• Cluster Deletions by hierarchical clustering [Alignment] Add basic vectorisation support for the dna global affine alignment product_backlog#54 ✅

Call Insertions from long reads seqan/product_backlog#93 ✅

Add all Methods of Vaquita seqan/product_backlog#84

Call SVs in short reads seqan/product_backlog#17

Cluster SVs: seqan/product_backlog#26

• hirachical clustering [Alignment] Add basic vectorisation support for the dna global affine alignment product_backlog#54 Ambiguous overload in debug stream operator for tuple and aligned sequence product_backlog#125 ✅
• ...

Refinement

TODO... (sViper, ...)

Output

We need to decouple the output from the functionality #6 so that we can write it to an output file #8 with an output option seqan/product_backlog#21.
Then a VCF parser has to be developed in SeqAn3 #9 #10, which we want to use for iGenVar #11. ✅

Testing ✅

We want to check the code with CLI seqan/product_backlog#4 and API tests seqan/product_backlog#12 seqan/product_backlog#13 and cover it completely.
-> We now have a codecoverage of > 85%! seqan/product_backlog#116 ✅
In order to implement the CodeCoverage, we are waiting for an update in the app template: seqan/app-template#30.
Update: CLI tests are implemented. 🎉

Refinements, bugs, and requests

• We want fully documented code. As an application developer I want to use CLI tests to test the interface of my program such that incompatible options or invalid inputs can be tested automatically. product_backlog#3
• A verbose option would be nice. -> [FEATURE] Add verbose option to the argument_parser sharg-parser#78 -> Port Lambda's search engine to use EPR dictionaries within SeqAn3/SDSL product_backlog#20, [Search] Add a sentinel-free implementation of the CSA to the SDSL such that we don't have to transform the text by increasing the ranks by one and increase the performance. product_backlog#23