Skip to content

Commit

Permalink
Update of README.
Browse files Browse the repository at this point in the history
  • Loading branch information
ny-shao committed Sep 5, 2014
1 parent 65a7b75 commit 2e1a6ed
Showing 1 changed file with 30 additions and 35 deletions.
65 changes: 30 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
Pipeline for ChIP-seq preprocessing
===================================
# Pipeline for ChIP-seq preprocessing

### Overview

Here is the pipeline I used for ChIP-seq preprocessing, including:

- align the fastq data to reference genome by bowtie or bowtie2.
- run FastQC to check the sequencing quality.
- remove all reads duplications of the aligned data.
- generate TDF files for browsing in IGV.
- run PhantomPeak to check the quality of ChIP.
- run ngs.plot to investigate the enrichment of ChIP-seq data at TSS, TES, and genebody.
* align the fastq data to reference genome by bowtie or bowtie2.
* run FastQC to check the sequencing quality.
* remove all reads duplications of the aligned data.
* generate TDF files for browsing in IGV.
* run PhantomPeak to check the quality of ChIP.
* run ngs.plot to investigate the enrichment of ChIP-seq data at TSS, TES, and genebody.

The pipeline work flow is:

Expand All @@ -20,14 +19,14 @@ The pipeline work flow is:

The softwares used in this pipeline are:

- [ruffus](https://code.google.com/p/ruffus/)
- [Bowtie](http://bowtie-bio.sourceforge.net/index.shtml)
- [FastQC](http://www.bioinformatics.babraham.ac.uk/projects/fastqc/)
- [samtools](http://samtools.sourceforge.net/)
- [IGVTools](http://www.broadinstitute.org/igv/igvtools)
- [PhantomPeak](http://code.google.com/p/phantompeakqualtools/) **In fact, the script **run_spp_nodups.R** is from PhantomPeak, but PhantomPeak still need to be installed in R.**
- [ngs.plot](https://code.google.com/p/ngsplot/)
- If cluster supporting needed, [drmaa_for_python](https://pypi.python.org/pypi/drmaa) is needed. Now LSF and SGE are supported, but it is easy to modify it to fit your demands.
* [ruffus](https://code.google.com/p/ruffus/)
* [Bowtie](http://bowtie-bio.sourceforge.net/index.shtml)
* [FastQC](http://www.bioinformatics.babraham.ac.uk/projects/fastqc/)
* [samtools](http://samtools.sourceforge.net/)
* [IGVTools](http://www.broadinstitute.org/igv/igvtools)
* [PhantomPeak](http://code.google.com/p/phantompeakqualtools/) __In fact, the script **run_spp_nodups.R** is from PhantomPeak, but PhantomPeak still need to be installed in R.__
* [ngs.plot](https://code.google.com/p/ngsplot/)
* If cluster supporting needed, [drmaa_for_python](https://pypi.python.org/pypi/drmaa) is needed. Now LSF and SGE are supported, but it is easy to modify it to fit your demands.

Install above softwares and make sure they are in $PATH.

Expand Down Expand Up @@ -57,42 +56,38 @@ python results_parser.py config.yaml

For the organization of projects, I generally follow this paper: [A Quick Guide to Organizing Computational Biology Projects](http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1000424). Here because it is preprocessing, and real analysis will be peak calling, chromatin segmentation, and differential enrichment detection, so I just put the results of the preprocess in the data folder.

For the configuration yaml file, **project_dir: `~/projects/test_ChIP-seq`** and **data_dir: "data"** mean the data folder is `~/projects/test_ChIP-seq/data`, and the results will be put in the same folder. Fastq files should be under `~/projects/test_ChIP-seq/data/fastq` folder. Now *.fastq, *.fq, *.gz (compressed fastq) files are acceptable. `aligner` now could be `bowtie` or `bowtie2`, if not assigned, then default aligner is `bowtie`. For `bowtie2`, the system variable `$BOWTIE2_INDEXES` should be set before running.
For the configuration yaml file, __project_dir: `~/projects/test_ChIP-seq`__ and __data_dir: "data"__ mean the data folder is `~/projects/test_ChIP-seq/data`, and the results will be put in the same folder. Fastq files should be under `~/projects/test_ChIP-seq/data/fastq` folder. Now *.fastq, *.fq, *.gz (compressed fastq) files are acceptable. `aligner` now could be `bowtie` or `bowtie2`, if not assigned, then default aligner is `bowtie`. For `bowtie2`, the system variable `$BOWTIE2_INDEXES` should be set before running.

The position of pipeline.py, results_parser.py, and config.yaml doesn't matter at all. But I prefer to put them under project/script/preprocess folder.

**Important:**

- To make ngs.plot part work, please name the fastq files in this way:

+ To make ngs.plot part work, please name the fastq files in this way:
```
Say condition A, B, each with 2 replicates, and one DNA input per condition.
Name the files as A_rep1.fastq, A_rep2.fastq, A_input.fastq, B_rep1.fastq,
Say condition A, B, each with 2 replicates, and one DNA input per condition.
Name the files as A_rep1.fastq, A_rep2.fastq, A_input.fastq, B_rep1.fastq,
B_rep2.fastq, and B_input.fastq.The key point is to make the same condition
samples with common letters and input samples contain "input" or "Input"
strings.
```

- If use want to only run to some specific step, just modify the function name in `pipeline_run` in pipeline.py.
- If the data are pair-end, follow this step:
- Modify the `config.yaml` file, change "pair_end" to "yes".
- Modify the `config.yaml` file, change "input_files" to "*R1*.fastq.gz" or "*R1*.fastq".
- Make sure the fastq files named as "*R1*" and "*R2*" pattern.
- if you want to use cluster:
- Edit '~/.bash_profile' to make sure all paths in $PATH.
- Modify `config.yaml` to fit your demands.
- `multithread` in `pipeline.py` determines the number of concurrent jobs to be submitted to cluster nodes by ruffus. A default value of 10 is used.
+ If use want to only run to some specific step, just modify the function name in `pipeline_run` in pipeline.py.
+ If the data are pair-end, follow this step:
+ Modify the `config.yaml` file, change "pair_end" to "yes".
+ Modify the `config.yaml` file, change "input_files" to "\*R1\*.fastq.gz" or "\*R1\*.fastq".
+ Make sure the fastq files named as "\*R1\*" and "\*R2\*" pattern.
+ if you want to use cluster:
+ Edit '~/.bash_profile' to make sure all paths in $PATH.
+ Modify `config.yaml` to fit your demands.
+ `multithread` in `pipeline.py` determines the number of concurrent jobs to be submitted to cluster nodes by ruffus. A default value of 10 is used.

**Warning:**

`Bowtie2` allows multiple hits reads, and breaks the assumption of `phantomPeak`:

`Bowtie2` allows multiple hits reads, and breaks the assumption of `phantomPeak`:
```
It is EXTREMELY important to filter out multi-mapping reads from the BAM/tagAlign
files. Large number of multimapping reads can severly affect the phantom peak
coefficient and peak calling results.
```

So be careful to interpret `NSC` and `RSC` in `Bowtie2` alignment results.

### Notes
Expand All @@ -103,4 +98,4 @@ In Bowtie2, default parameters are used.

### ToDos

- Method to skip some steps if the user doesn't run.
+ Method to skip some steps if the user doesn't run.

0 comments on commit 2e1a6ed

Please sign in to comment.