Skip to content

Commit 701c015

Browse files
drpatelhdrpatelh
committed
Fix merge conflicts
2 parents d2d38ee + ef3ffb3 commit 701c015

File tree

170 files changed

+20513
-503
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

170 files changed

+20513
-503
lines changed

‎.editorconfig

+3
Original file line numberDiff line numberDiff line change
@@ -22,3 +22,6 @@ indent_size = unset
2222

2323
[/assets/email*]
2424
indent_size = unset
25+
26+
[/assets/blacklists/GRCh37-blacklist.bed]
27+
trim_trailing_whitespace = unset

‎.github/PULL_REQUEST_TEMPLATE.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/chip
1717
- [ ] If you've fixed a bug or added code that should be tested, add tests!
1818
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/chipseq/tree/master/.github/CONTRIBUTING.md)- [ ] If necessary, also make a PR on the nf-core/chipseq _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
1919
- [ ] Make sure your code lints (`nf-core lint`).
20-
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
20+
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker` --outdir <OUTDIR>`).
2121
- [ ] Usage Documentation in `docs/usage.md` is updated.
2222
- [ ] Output Documentation in `docs/output.md` is updated.
2323
- [ ] `CHANGELOG.md` is updated.

‎.github/workflows/awsfulltest.yml

+4-3
Original file line numberDiff line numberDiff line change
@@ -12,12 +12,12 @@ jobs:
1212
name: Run AWS full tests
1313
if: github.repository == 'nf-core/chipseq'
1414
runs-on: ubuntu-latest
15+
strategy:
16+
matrix:
17+
aligner: ["bwa", "bowtie2", "chromap", "star"]
1518
steps:
1619
- name: Launch workflow via tower
1720
uses: nf-core/tower-action@v3
18-
# TODO nf-core: You can customise AWS full pipeline tests as required
19-
# Add full size test data (but still relatively small datasets for few samples)
20-
# on the `test_full.config` test runs with only one set of parameters
2121
with:
2222
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
2323
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
@@ -26,5 +26,6 @@ jobs:
2626
parameters: |
2727
{
2828
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/chipseq/results-${{ github.sha }}"
29+
"aligner": "${{ matrix.aligner }}"
2930
}
3031
profiles: test_full,aws_tower

‎.github/workflows/ci.yml

100644100755
+45-3
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,50 @@ jobs:
3232
version: "${{ matrix.NXF_VER }}"
3333

3434
- name: Run pipeline with test data
35-
# TODO nf-core: You can customise CI pipeline run tests as required
36-
# For example: adding multiple test runs with different parameters
37-
# Remember that you can parallelise this by using strategy.matrix
3835
run: |
3936
nextflow run ${GITHUB_WORKSPACE} -profile test,docker --outdir ./results
37+
38+
parameters:
39+
name: Test workflow parameters
40+
if: ${{ github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'nf-core/chipseq') }}
41+
runs-on: ubuntu-latest
42+
strategy:
43+
matrix:
44+
parameters:
45+
- "--skip_trimming"
46+
- "--skip_consensus_peaks"
47+
steps:
48+
- name: Check out pipeline code
49+
uses: actions/checkout@v2
50+
51+
- name: Install Nextflow
52+
run: |
53+
wget -qO- get.nextflow.io | bash
54+
sudo mv nextflow /usr/local/bin/
55+
56+
- name: Run pipeline with various parameters
57+
run: |
58+
nextflow run ${GITHUB_WORKSPACE} -profile test,docker ${{ matrix.parameters }} --outdir ./results
59+
60+
aligners:
61+
name: Test available aligners
62+
if: ${{ github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'nf-core/chipseq') }}
63+
runs-on: ubuntu-latest
64+
strategy:
65+
matrix:
66+
aligner:
67+
- "bowtie2"
68+
- "chromap"
69+
- "star"
70+
steps:
71+
- name: Check out pipeline code
72+
uses: actions/checkout@v2
73+
74+
- name: Install Nextflow
75+
run: |
76+
wget -qO- get.nextflow.io | bash
77+
sudo mv nextflow /usr/local/bin/
78+
79+
- name: Run pipeline with the different aligners available
80+
run: |
81+
nextflow run ${GITHUB_WORKSPACE} -profile test,docker --aligner ${{ matrix.aligner }} --outdir ./results

‎.github/workflows/linting.yml

100644100755
File mode changed.

‎.nf-core.yml

+7
Original file line numberDiff line numberDiff line change
@@ -1 +1,8 @@
11
repository_type: pipeline
2+
lint:
3+
files_unchanged:
4+
- .github/workflows/branch.yml
5+
- .github/workflows/linting_comment.yml
6+
- .github/workflows/linting.yml
7+
- .github/PULL_REQUEST_TEMPLATE.md
8+
- bin/check_samplesheet.py

‎CHANGELOG.md

+233-2
Original file line numberDiff line numberDiff line change
@@ -3,14 +3,245 @@
33
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
44
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
55

6-
## v2.0.0 - [date]
6+
## [[2.0.0](https://github.com/nf-core/chipseq/releases/tag/2.0.0)] - 2022-09-30
77

8-
Initial release of nf-core/chipseq, created with the [nf-core](https://nf-co.re/) template.
8+
### Enhancements & fixes
9+
10+
- Pipeline has been re-implemented in [Nextflow DSL2](https://www.nextflow.io/docs/latest/dsl2.html)
11+
- All software containers are now exclusively obtained from [Biocontainers](https://biocontainers.pro/#/registry)
12+
- Updated pipeline template to [nf-core/tools 2.5.1](https://github.com/nf-core/tools/releases/tag/2.5.1)
13+
- [[#128](https://github.com/nf-core/chipseq/issues/128)] - Filter files with no peaks to avoid errors in downstream processes
14+
- [[#220](https://github.com/nf-core/chipseq/issues/220)] - Fix `phantompeakqualtools` protection stack overflow error
15+
- [[#233](https://github.com/nf-core/chipseq/issues/233)] - Add `chromap` to the available aligners
16+
- Bump minimum Nextflow version from `21.04.0` -> `21.10.3`
17+
- Added `python3` shebang to appropriate scripts in `bin/` directory
18+
- [[#160](https://github.com/nf-core/chipseq/issues/160)] - Add `bowtie2` and `star` as available aligners, via the `--aligner` parameter
19+
- Add `--save_unaligned` parameter (only available for `bowtie2` and `star`)
20+
- Update `igenomes.config` to fetch whole `BWAIndex/version0.6.0/` folder
21+
- [[228](https://github.com/nf-core/chipseq/issues/228)] - Update blacklist bed files.
22+
- [nf-core/tools#1415](https://github.com/nf-core/tools/issues/1415) - Make `--outdir` a mandatory parameter
23+
- [[282](https://github.com/nf-core/chipseq/issues/282)] - Fix `genome.fa` publication for IGV.
24+
- [[280](https://github.com/nf-core/chipseq/issues/280)] - Update `macs_gsize` in `igenomes.config`, create a new `--read_length` parameter and implement the logic to calculate `--macs_gsize` when the parameter is missing
25+
- Eliminate `if` conditions from `deseq2_qc` and `macs2_consensus` (local module and use `ext.when` instead)
26+
- Remove `deseq2` differential binding analysis of consensus peaks.
27+
- [[280](https://github.com/nf-core/chipseq/issues/291) - Filter paired-end files produced by `chromap` since the resulting `BAM` files can not be processed downstream.
28+
- Remove <ANTIBODY> from the macs2 consensus publish directory since it can not be referred as input from the IGV process (meta.id not resolved at execution time)
29+
- Add bytesize link to readme.
30+
31+
### Parameters
32+
33+
| Old parameter | New parameter |
34+
| ---------------------- | ----------------------- |
35+
| `--conda` | `--enable_conda` |
36+
| `--skip_diff_analysis` | `--skip_deseq2_qc` |
37+
| | `--skip_qc` |
38+
| | `--aligner` |
39+
| | `--save_unaligned` |
40+
| | `--read_length` |
41+
| | `--multiqc_title` |
42+
| | `--gff` |
43+
| | `--bowtie2_index` |
44+
| | `--chromap_index` |
45+
| | `--star_index` |
46+
| | `--validate_params` |
47+
| | `--show_hidden_params` |
48+
| | `--config_profile_name` |
49+
| `--clusterOptions` | |
50+
| `--single_end` | |
51+
| `--name` | |
52+
| `--hostnames` | |
53+
54+
> **NB:** Parameter has been **updated** if both old and new parameter information is present.
55+
> **NB:** Parameter has been **added** if just the new parameter information is present.
56+
> **NB:** Parameter has been **removed** if parameter information isn't present.
57+
58+
### Software dependencies
59+
60+
Note, since the pipeline is now using Nextflow DSL2, each process will be run with its own [Biocontainer](https://biocontainers.pro/#/registry). This means that on occasion it is entirely possible for the pipeline to be using different versions of the same tool. However, the overall software dependency changes compared to the last release have been listed below for reference.
61+
62+
| Dependency | Old version | New version |
63+
| ----------------------- | ----------- | ----------- |
64+
| `samtools` | 1.10 | 1.15.1 |
65+
| `picard` | 2.23.1 | 2.27.4 |
66+
| `bamtools` | 2.5.1 | 2.5.2 |
67+
| `pysam` | 0.15.3 | 0.19.0 |
68+
| `bedtools` | 2.29.2 | 2.30.0 |
69+
| `ucsc-bedgraphtobigwig` | 357 | 377 |
70+
| `deeptools` | 3.4.3 | 3.5.1 |
71+
| `pigz` | 2.3.4 | 2.6 |
72+
| `preseq` | 2.0.3 | 3.1.2 |
73+
| `multiqc` | 1.9 | 1.13a |
74+
| `r-base` | 3.6.1 | 4.0.3 |
75+
| `r-ggplot2` | 3.3.2 | 3.3.3 |
76+
| `bioconductor-deseq2` | 1.26.0 | 1.28.0 |
77+
| `trim-galore` | 0.6.5 | 0.6.7 |
78+
| `r-optparse` | - | 1.7.1 |
79+
| `chromap` | - | 0.2.1 |
80+
| `bowtie2` | - | 2.4.4 |
81+
| `star` | - | 2.6.1d |
82+
| `r-tidyr` | - | - |
83+
| `r-lattice` | - | - |
84+
| `r-xfun` | - | - |
85+
| `bioconductor-vsn` | - | - |
86+
87+
> **NB:** Dependency has been **updated** if both old and new version information is present.
88+
> **NB:** Dependency has been **added** if just the new version information is present.
89+
> **NB:** Dependency has been **removed** if version information isn't present.
90+
91+
## [[1.2.2](https://github.com/nf-core/chipseq/releases/tag/1.2.2)] - 2021-04-22
92+
93+
- [#206](https://github.com/nf-core/chipseq/issues/206) - Minor patch release to fix Conda environment
94+
95+
### `Dependencies`
96+
97+
- Update r-base `3.6.2` -> `3.6.3`
98+
- Update r-xfun `0.15` -> `0.20`
99+
100+
## [[1.2.1](https://github.com/nf-core/chipseq/releases/tag/1.2.1)] - 2020-07-29
101+
102+
- [#171](https://github.com/nf-core/chipseq/issues/171) - Minor patch release to update pipeline schema
103+
104+
## [[1.2.0](https://github.com/nf-core/chipseq/releases/tag/1.2.0)] - 2020-07-02
9105

10106
### `Added`
11107

108+
- [#138](https://github.com/nf-core/chipseq/issues/138) - Add social preview image
109+
- [#153](https://github.com/nf-core/chipseq/issues/153) - Add plotHeatmap
110+
- [#159](https://github.com/nf-core/chipseq/issues/159) - expose bwa mem -T parameter
111+
- [nf-core/atacseq#63](https://github.com/nf-core/atacseq/issues/63) - Added multicore support for Trim Galore!
112+
- [nf-core/atacseq#75](https://github.com/nf-core/atacseq/issues/75) - Include gene annotation versions in multiqc report
113+
- [nf-core/atacseq#76](https://github.com/nf-core/atacseq/issues/76) - featureCounts coupled to DESeq2
114+
- [nf-core/atacseq#79](https://github.com/nf-core/atacseq/issues/79) - Parallelize DESeq2
115+
- [nf-core/atacseq#97](https://github.com/nf-core/atacseq/issues/97) - PBC1, PBC2 from pipeline?
116+
- [nf-core/atacseq#107](https://github.com/nf-core/atacseq/issues/107) - Add options to change MACS2 parameters
117+
- Regenerated screenshots and added collapsible sections for output files in `docs/output.md`
118+
- Update template to tools `1.9`
119+
- Replace `set` with `tuple` and `file()` with `path()` in all processes
120+
- Capitalise process names
121+
- Parameters:
122+
- `--bwa_min_score` to set minimum alignment score for BWA MEM
123+
- `--macs_fdr` to provide FDR threshold for MACS2 peak calling
124+
- `--macs_pvalue` to provide p-value threshold for MACS2 peak calling
125+
- `--skip_peak_qc` to skip MACS2 peak QC plot generation
126+
- `--skip_peak_annotation` to skip annotation of MACS2 and consensus peaks with HOMER
127+
- `--skip_consensus_peaks` to skip consensus peak generation
128+
- `--deseq2_vst` to use variance stabilizing transformation (VST) instead of regularized log transformation (rlog) with DESeq2
129+
- `--publish_dir_mode` to customise method of publishing results to output directory [nf-core/tools#585](https://github.com/nf-core/tools/issues/585)
130+
131+
### `Removed`
132+
133+
- `--tss_bed` parameter
134+
12135
### `Fixed`
13136

137+
- [#118](https://github.com/nf-core/chipseq/issues/118) - Running on with SGE
138+
- [#132](https://github.com/nf-core/chipseq/issues/132) - BigWig Error: sort: cannot create temporary file in '': Read-only file system
139+
- [#154](https://github.com/nf-core/chipseq/issues/154) - computeMatrix.val.mat.gz files not zipped
140+
- [nf-core/atacseq#71](https://github.com/nf-core/atacseq/issues/71) - consensus_peaks.mLb.clN.boolean.intersect.plot.pdf not generated
141+
- [nf-core/atacseq#73](https://github.com/nf-core/atacseq/issues/73) - macs_annotatePeaks.mLb.clN.summary.txt file is not created
142+
- [nf-core/atacseq#86](https://github.com/nf-core/atacseq/issues/86) - bug in the plot_homer_annotatepeaks.r script
143+
- [nf-core/atacseq#102](https://github.com/nf-core/atacseq/issues/102) - Incorrect Group ID assigned by featurecounts_deseq2.r
144+
- [nf-core/atacseq#109](https://github.com/nf-core/atacseq/issues/109) - Specify custom gtf but gene bed is not generated from that gtf?
145+
- Make executables in `bin/` compatible with Python 3
146+
147+
### `Dependencies`
148+
149+
- Add bioconductor-biocparallel `1.20.0`
150+
- Add markdown `3.2.2`
151+
- Add pigz `2.3.4`
152+
- Add pygments `2.6.1`
153+
- Add pymdown-extensions `7.1`
154+
- Add python `3.7.6`
155+
- Add r-reshape2 `1.4.4`
156+
- Add r-tidyr `1.1.0`
157+
- Update bedtools `2.27.1` -> `2.29.2`
158+
- Update bioconductor-deseq2 `1.20.0` -> `1.26.0`
159+
- Update bioconductor-vsn `3.46.0` -> `3.54.0`
160+
- Update deeptools `3.2.1` -> `3.4.3`
161+
- Update fastqc `0.11.8` -> `0.11.9`
162+
- Update gawk `4.2.1` -> `5.1.0`
163+
- Update homer `4.9.1` -> `4.11`
164+
- Update macs2 `2.1.2` -> `2.2.7.1`
165+
- Update multiqc `1.7` -> `1.8`
166+
- Update phantompeakqualtools `1.2` -> `1.2.2`
167+
- Update picard `2.19.0` -> `2.23.1`
168+
- Update pysam `0.15.2` -> `0.15.3`
169+
- Update r-base `3.4.1` -> `3.6.2`
170+
- Update r-ggplot2 `3.1.0` -> `3.3.2`
171+
- Update r-lattice `0.20_35` -> `0.20_41`
172+
- Update r-optparse `1.6.0` -> `1.6.6`
173+
- Update r-pheatmap `1.0.10` -> `1.0.12`
174+
- Update r-scales `1.0.0` -> `1.1.1`
175+
- Update r-upsetr `1.3.3` -> `1.4.0`
176+
- Update r-xfun `0.3` -> `0.15`
177+
- Update samtools `1.9` -> `1.10`
178+
- Update subread `1.6.4` -> `2.0.1`
179+
- Update trim-galore `0.5.0` -> `0.6.5`
180+
- Update ucsc-bedgraphtobigwig `377` -> `357`
181+
182+
## [[1.1.0](https://github.com/nf-core/chipseq/releases/tag/1.1.0)] - 2019-11-05
183+
184+
### `Added`
185+
186+
- [nf-core/atacseq#46](https://github.com/nf-core/atacseq/issues/46) - Missing gene_bed path in igenomes config
187+
- Update template to tools `1.7`
188+
- Add `--trim_nextseq` parameter
189+
- Add `CITATIONS.md` file
190+
- Capitalised process names
191+
192+
### `Fixed`
193+
194+
- Change all parameters from `camelCase` to `snake_case` (see [Deprecated](#Deprecated))
195+
- [nf-core/atacseq#44](https://github.com/nf-core/atacseq/issues/44) - Output directory missing: macs2/consensus/deseq2
196+
- [nf-core/atacseq#45](https://github.com/nf-core/atacseq/issues/45) - Wrong x-axis scale for the HOMER: Peak annotation Counts tab plot?
197+
- [nf-core/atacseq#46](https://github.com/nf-core/atacseq/issues/46) - Stage blacklist file in channel properly
198+
- [nf-core/atacseq#50](https://github.com/nf-core/atacseq/issues/50) - HOMER number of peaks does not correspond to found MACS2 peaks
199+
- Fixed bug in UpSetR peak intersection plot
200+
- Increase default resource requirements in `base.config`
201+
- Increase process-specific requirements based on user-reported failures
202+
14203
### `Dependencies`
15204

205+
- Update Nextflow `0.32.0` -> `19.10.0`
206+
16207
### `Deprecated`
208+
209+
| Deprecated | Replacement |
210+
| ---------------------------- | ------------------------- |
211+
| `--design` | `--input` |
212+
| `--singleEnd` | `--single_end` |
213+
| `--saveGenomeIndex` | `--save_reference` |
214+
| `--skipTrimming` | `--skip_trimming` |
215+
| `--saveTrimmed` | `--save_trimmed` |
216+
| `--keepDups` | `--keep_dups` |
217+
| `--keepMultiMap` | `--keep_multi_map` |
218+
| `--saveAlignedIntermediates` | `--save_align_intermeds` |
219+
| `--narrowPeak` | `--narrow_peak` |
220+
| `--saveMACSPileup` | `--save_macs_pileup` |
221+
| `--skipDiffAnalysis` | `--skip_diff_analysis` |
222+
| `--skipFastQC` | `--skip_fastqc` |
223+
| `--skipPicardMetrics` | `--skip_picard_metrics` |
224+
| `--skipPreseq` | `--skip_preseq` |
225+
| `--skipPlotProfile` | `--skip_plot_profile` |
226+
| `--skipPlotFingerprint` | `--skip_plot_fingerprint` |
227+
| `--skipSpp` | `--skip_spp` |
228+
| `--skipIGV` | `--skip_igv` |
229+
| `--skipMultiQC` | `--skip_multiqc` |
230+
231+
## [[1.0.0](https://github.com/nf-core/chipseq/releases/tag/1.0.0)] - 2019-06-06
232+
233+
Initial release of nf-core/chipseq pipeline.
234+
235+
### `Added`
236+
237+
- Raw read QC (FastQC)
238+
- Adapter trimming (Trim Galore!)
239+
- Map and filter reads (BWA, picard, SAMtools, BEDTools, BAMTools, Pysam)
240+
- Create library-size normalised bigWig tracks (BEDTools, bedGraphToBigWig)
241+
- Alignment QC metrics (Preseq, picard)
242+
- ChIP-seq QC metrics (deepTools, phantompeakqualtools)
243+
- Call and annotate broad/narrow peaks (MACS2, HOMER)
244+
- Create consensus set of peaks per antibody (BEDTools)
245+
- Quantification and differential binding analysis (featureCounts, DESeq2)
246+
- Collate appropriate files for genome browser visualisation (IGV)
247+
- Collate and present various QC metrics (MultiQC, R)

0 commit comments

Comments
 (0)