Add XY filtration workflow

[Faizal-Eeman]

Description

ADD XY filter workflow

Closes #190

Testing Results

• N-T paired WGS (sample_sex = XY)

  • samples: NA24149, NA24143
  • input YAML: /hot/software/pipeline/pipeline-call-gSNP/Nextflow/development/pre-10.1.0-rc.1/mmootor-add-xy-filtration/pipeline_test/NA24143.yaml
  • config: /hot/software/pipeline/pipeline-call-gSNP/Nextflow/development/pre-10.1.0-rc.1/mmootor-add-xy-filtration/pipeline_test/filter-xy-gSNP.config
  • output: /hot/software/pipeline/pipeline-call-gSNP/Nextflow/development/pre-10.1.0-rc.1/mmootor-add-xy-filtration/pipeline_test/call-gSNP-10.1.0-rc.1/NA24143/GATK-4.5.0.0/output/

• N-T paired WGS (sample_sex = XX)

  • samples: NA24149, NA24143
  • input YAML: /hot/software/pipeline/pipeline-call-gSNP/Nextflow/development/pre-10.1.0-rc.1/mmootor-add-xy-filtration/pipeline_test/NA24143.yaml
  • config: /hot/software/pipeline/pipeline-call-gSNP/Nextflow/development/pre-10.1.0-rc.1/mmootor-add-xy-filtration/pipeline_test/test-XX/filter-xx-gSNP.config
  • output: /hot/software/pipeline/pipeline-call-gSNP/Nextflow/development/pre-10.1.0-rc.1/mmootor-add-xy-filtration/pipeline_test/test-XX/call-gSNP-10.1.0-rc.1/NA24143/GATK-4.5.0.0/output/

Checklist

• I have read the code review guidelines and the code review best practice on GitHub check-list.

• I have reviewed the Nextflow pipeline standards.

• The name of the branch is meaningful and well formatted following the standards, using [AD_username (or 5 letters of AD if AD is too long)]-[brief_description_of_branch].

• I have set up or verified the branch protection rule following the github standards before opening this pull request.

• I have added my name to the contributors listings in the manifest block in the nextflow.config as part of this pull request, am listed
  already, or do not wish to be listed. (This acknowledgement is optional.)

• I have added the changes included in this pull request to the CHANGELOG.md under the next release version or unreleased, and updated the date.

• I have updated the version number in the metadata.yaml and manifest block of the nextflow.config file following semver, or the version number has already been updated. (Leave it unchecked if you are unsure about new version number and discuss it with the infrastructure team in this PR.)

• I have tested the pipeline on at least one A-mini sample.

[alkaZeltser]

Will this be a standardized reference in /hot/resource/ ?

[Faizal-Eeman]

I'll defer this to @yashpatel6 as I do not have permission to create a dir in /hot/resource/

Here's the GRCh38 version of PAR BED. You can remove the commented lines from this file when you make a copy in /hot/resource/ - /hot/project/method/AlgorithmEvaluation/BNCH-000122-GIABSexChrGermlineFilter/GIAB/AshkenazimTrio/germline-small-variant/filter_XY/pseudoautosomal_regions_hg38.bed

[alkaZeltser]

I don't think we would ever encounter this in DNA-metapipeline, but just FYI I have seen X/Y encoded as chr23 and chr24 in some genetic data.

[Faizal-Eeman]

Yeah I'm aware. However to keep consistent with variant calls in the DNA-metapipeline, chrX and chrY notation should be fine in pipeline outputs.

[alkaZeltser]

@Faizal-Eeman Would it be possible to add a line to the outputed VCF header documenting the XY filtration, similar to how bcftools appends every operation to the header? It would be a good way to maintain a record of what has been done to the file.

[alkaZeltser]

Test run complete.

XY filtered call-gSNP output - /hot/software/pipeline/pipeline-call-gSNP/Nextflow/development/pre-10.1.0-rc.1/mmootor-add-xy-filtration/pipeline_test/call-gSNP-10.1.0-rc.1/NA24143/GATK-4.5.0.0/output/Hail-branch-mmootor-fix-spark-permission_GATK-4.5.0.0_TEST_NA24143_Hail-branch-mmootor-fix-spark-permission-GATK-4.5.0.0-TEST-XY-filtered.vcf.bgz

pipeline log - /hot/software/pipeline/pipeline-call-gSNP/Nextflow/development/pre-10.1.0-rc.1/mmootor-add-xy-filtration/pipeline_test/xy-filter.log

I skimmed through the output file and everything looks as expected: PARs remain diploid in both X and Y, non-PARs are haploid in X and Y. Missing genotypes are always in diploid notation: ./. but I don't think that's an issue. @Faizal-Eeman for completeness' sake, would be good to test a female sample? Unless one of the two is already female?

[Faizal-Eeman]

@Faizal-Eeman Would it be possible to add a line to the outputed VCF header documenting the XY filtration, similar to how bcftools appends every operation to the header? It would be a good way to maintain a record of what has been done to the file.

@alkaZeltser for now I'm appending the script command like GATK does to VCF header

##source=ApplyVQSR
##source=CombineGVCFs
##XYFiltration=<CommandLine=script/filter_xy_call.py --sample_name Hail-0.2.133_GATK-4.5.0.0_TEST_NA24143 --input_vcf GATK-4.5.0.0_TEST_NA24143_VQSR-SNP-AND-INDEL.vcf.gz --vcf_source_file ./vcf_source.txt --sample_sex XY --par_bed pseudoautosomal_regions_hg38.bed --genome_build GRCh38 --output_dir .>

As to the steps in the workflow, I've added a document to ./docs in the repo and referenced it in call-gSNP README.

[Faizal-Eeman]

for completeness' sake, would be good to test a female sample? Unless one of the two is already female?

The same sample was treated as an XX case,

• output VCF - /hot/software/pipeline/pipeline-call-gSNP/Nextflow/development/pre-10.1.0-rc.1/mmootor-add-xy-filtration/pipeline_test/test-XX/call-gSNP-10.1.0-rc.1/NA24143/GATK-4.5.0.0/output/Hail-0.2.133_GATK-4.5.0.0_TEST_NA24143_XY_filtered.vcf.bgz

By the time this test finished, I updated the python script to name the output file based on the sample_sex. Latest python script test output is below,
/hot/software/pipeline/pipeline-call-gSNP/Nextflow/development/pre-10.1.0-rc.1/mmootor-add-xy-filtration/script_test/multisample_XX_filtered.vcf.bgz