Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

include MiXCR toolkit in nf-core/airrflow #346

Open
wants to merge 36 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
262ff1a
black
mapo9 Dec 7, 2023
0c1942b
linting
mapo9 Dec 7, 2023
d679d50
prettier
mapo9 Dec 7, 2023
b84b997
first mixcr implementations
mapo9 Jan 30, 2024
31a26ac
mixcr sc upstream
mapo9 Feb 8, 2024
0c0bcde
Merge pull request #337 from nf-core/dev
ggabernet Jun 1, 2024
b91a4cf
merge dev into mixcr branch
mapo9 Jun 7, 2024
55e45b1
add library param
mapo9 Jun 7, 2024
681ab2c
initial mixcr pr
mapo9 Jun 18, 2024
04999f2
Merge branch 'nf-core:master' into mixcr
mapo9 Jun 25, 2024
ace38fc
imgt for postanalysis
mapo9 Jun 25, 2024
8a48982
Merge branch 'mixcr' of https://github.com/mapo9/airrflow into mixcr
mapo9 Jun 25, 2024
495a374
mixcr docs
mapo9 Sep 23, 2024
b9c1d23
mixcr tests
mapo9 Sep 23, 2024
ee55055
better description
mapo9 Sep 23, 2024
7eb5ccc
some error catches
mapo9 Sep 23, 2024
ab50cfb
Merge branch 'dev' into mixcr
mapo9 Sep 23, 2024
8189450
linting
mapo9 Sep 23, 2024
4073ab5
prettier
mapo9 Sep 23, 2024
73167ca
trailing whitespace
mapo9 Sep 23, 2024
ededf1e
trailing whitespace
mapo9 Sep 23, 2024
a10a621
bumped actions/upload-artifact to v4 to circumvent deprecated version…
mapo9 Sep 23, 2024
2a155de
set module to process_high
mapo9 Oct 1, 2024
15f8074
metromap updated
mapo9 Nov 7, 2024
2c24b05
indentation
mapo9 Nov 7, 2024
172a595
remove extra def
mapo9 Nov 7, 2024
32ef9ae
Merge branch 'dev' into mixcr
mapo9 Nov 7, 2024
1b51912
structure
mapo9 Nov 7, 2024
a5140b7
definition to
mapo9 Nov 7, 2024
cec63d4
schema
mapo9 Nov 7, 2024
e55f7d6
schema
mapo9 Nov 7, 2024
32101f1
schema
mapo9 Nov 7, 2024
8a928f4
schema
mapo9 Nov 7, 2024
366ad16
syntax fixed
mapo9 Nov 8, 2024
3de8d49
secrets profile added
mapo9 Nov 8, 2024
c47ba1c
prettier
mapo9 Nov 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,8 @@ jobs:
"test_assembled_hs",
"test_assembled_mm",
"test_10x_sc",
"test_mixcr_bulk",
"test_mixcr_sc",
"test_clontech_umi",
"test_nebnext_umi",
"test_rnaseq_bulk",
Expand Down Expand Up @@ -135,3 +137,52 @@ jobs:
/home/runner/work
!/home/runner/work/conda
!/home/runner/work/singularity
test_profile_download:
name: nf-test with profiles needing mixcr license
# Only run on PRs from branches of nf-core/airrflow repository or on push if this is the nf-core dev branch (merged PRs)
# (GitHub secrets are not accessible for workflows from forks!)
if: "${{ github.event.pull_request.head.repo.full_name == 'nf-core/airrflow' || (github.event_name == 'push' && github.repository == 'nf-core/airrflow') }}"
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
NXF_VER:
- "23.04.0"
- "latest-everything"
profile: ["test_mixcr_bulk", "test_mixcr_sc"]

steps:
- name: Check out pipeline code
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4

- name: Install Nextflow
uses: nf-core/setup-nextflow@v1
with:
version: "${{ matrix.NXF_VER }}"

- name: Disk space cleanup
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1

- name: Install nf-test
uses: nf-core/setup-nf-test@v1
with:
version: ${{ env.NFT_VER }}

- name: Run nf-test
env:
NCBI_EMAIL: ${{ secrets.MIXCR_LICENSE }}
run: |
nextflow secrets set MIXCR_LICENSE ${{ secrets.MIXCR_LICENSE }}
nf-test test --tag ${{ matrix.profile }} --profile ${{ matrix.profile }},docker --junitxml=test.xml --verbose

- name: Output log on failure
if: failure()
run: |
sudo apt install bat > /dev/null
batcat --decorations=always --color=always ${{ github.workspace }}/.nf-test/tests/*/output/pipeline_info/software_versions.yml

- name: Publish Test Report
uses: mikepenz/action-junit-report@v3
if: always() # always run even if the previous step fails
with:
report_paths: "*.xml"
36 changes: 36 additions & 0 deletions conf/modules.config
Original file line number Diff line number Diff line change
Expand Up @@ -731,4 +731,40 @@ process {
]
}

// -------------------------------
// MiXCR
// -------------------------------

withName: MIXCR_IND_POSTANALYSIS {
publishDir = [
path: { "${params.outdir}/mixcr/individual_postanalysis" },
mode: params.publish_dir_mode,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
]
}

withName: MIXCR_IND_PLOTS {
publishDir = [
path: { "${params.outdir}/mixcr/individual_postanalysis/plots" },
mode: params.publish_dir_mode,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
]
}

withName: MIXCR_OVERLAP_POSTANALYSIS {
publishDir = [
path: { "${params.outdir}/mixcr/overlap_postanalysis" },
mode: params.publish_dir_mode,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
]
}

withName: MIXCR_OVERLAP_PLOTS {
publishDir = [
path: { "${params.outdir}/mixcr/overlap_postanalysis/plots" },
mode: params.publish_dir_mode,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
]
}

}
40 changes: 40 additions & 0 deletions conf/test_mixcr_bulk.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
/*
* -------------------------------------------------
* Nextflow config file for running tests
* -------------------------------------------------
* Defines bundled input files and everything required
* to run a fast and simple test. Use as follows:
* nextflow run nf-core/airrflow -profile test_mixcr_bulk,<docker/singularity>
*/

process {
resourceLimits = [
cpus: 2,
memory: '6.GB',
time: '1.h'
]
}

params {
config_profile_name = 'Test MiXCR toolset on bulk data.'
config_profile_description = 'Minimal test dataset to check pipeline functionality with raw bulk data using the MiXCR toolset'


// params
mode = 'fastq'
library_generation_method = 'mixcr'
kit = "irepertoire-human-rna-xcr-repseq-lr"
clonal_threshold = 0
mixcr_postanalysis = true
mixcr_downsampling = "count-read-auto"
mixcr_weightfunction = "none"
mixcr_productive_only = true
mixcr_drop_outliers = true
mixcr_overlap_criteria = "CDR3|AA|V|J"
mixcr_diversity_plottype = "boxplot"


// Input data
input = pipelines_testdata_base_path + 'testdata-tcr/TCR_metadata_airr.tsv'
imgt_mixcr = "https://github.com/repseqio/library-imgt/releases/download/v8/imgt.202312-3.sv8.json.gz"
}
42 changes: 42 additions & 0 deletions conf/test_mixcr_sc.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
/*
* -------------------------------------------------
* Nextflow config file for running tests
* -------------------------------------------------
* Defines bundled input files and everything required
* to run a fast and simple test. Use as follows:
* nextflow run nf-core/airrflow -profile test_mixcr_sc,<docker/singularity>
*/

includeConfig "https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/testdata-mixcr/10x_sc.conf"

process {
resourceLimits = [
cpus: 2,
memory: '6.GB',
time: '1.h'
]
}

params {
config_profile_name = 'Test MiXCR sc analysis.'
config_profile_description = 'Minimal test dataset to check pipeline functionality with raw single cell data from 10xGenomics using the MiXCR toolset'

// params
mode = 'fastq'
library_generation_method = 'mixcr'
clonal_threshold = 0
kit = '10x-sc-xcr-vdj'
mixcr_postanalysis = true
mixcr_downsampling = "count-read-auto"
mixcr_weightfunction = "none"
mixcr_productive_only = true
mixcr_drop_outliers = true
mixcr_overlap_criteria = "CDR3|AA|V|J"
mixcr_diversity_plottype = "boxplot"


// Input data
input = pipelines_testdata_base_path + 'testdata-sc/10x_sc_raw.tsv'
reference_10x = pipelines_testdata_base_path + 'testdata-sc/refdata-cellranger-vdj-GRCh38-alts-ensembl-5.0.0.tar.gz'
imgt_mixcr = "https://github.com/repseqio/library-imgt/releases/download/v8/imgt.202312-3.sv8.json.gz"
}
Binary file modified docs/images/metro-map-airrflow.pdf
Binary file not shown.
Binary file modified docs/images/metro-map-airrflow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading