modelbased_summarization.Rmd

---
title: "**Summarization** strategy comparison for **Model-based** analysis of isobarically labeled proteomic data."
author: "Piotr Prostko, Joris Van Houtven"
date: '`r format(Sys.time(), "%B %d, %Y,%H:%M")`'
output: 
  html_document:
    toc: true
    toc_depth: 2
    toc_float: true
    number_sections: true
    theme: flatly
    code_folding: "hide"
editor_options: 
  chunk_output_type: console
params:
  input_data_p: 'data/input_data.rds'
  suffix_p: ''
  load_outputdata_p: FALSE
  save_outputdata_p: FALSE
  subsample_p: 0
---

```{r, setup, include=FALSE}
# default knitr options for all chunks
knitr::opts_chunk$set(
  message=FALSE,
  warning=FALSE,
  fig.width=12,
  fig.height=7
)
```

<span style="color: grey;">
_This notebook is one in a series of many, where we explore how different data analysis strategies affect the outcome of a proteomics experiment based on isobaric labeling and mass spectrometry. Each analysis strategy or 'workflow' can be divided up into different components; it is recommend you read more about that in the [introduction notebook](intro.html)._
</span>

In this notebook specifically, we investigate the effect of varying the **Summarization** component on the outcome of the differential expression results. The three component variants are: **no summarization**, **Median summarization (PSM to peptide)**, **Sum summarization (PSM to peptide)**.

<span style="color: grey;">
_The R packages and helper scripts necessary to run this notebook are listed in the next code chunk: click the 'Code' button. Each code section can be expanded in a similar fashion. You can also download the [entire notebook source code](modelbased_unit.Rmd)._
</span>

```{r}
library(caret)
library(lme4)
library(lmerTest)
library(ggplot2)
library(stringi)
library(gridExtra)
library(ggfortify)
library(dendextend)
library(psych)
library(kableExtra)
library(tidyverse)
library(dtplyr)
source('util/other_functions.R')
source('util/plotting_functions.R')
```

Let's load our PSM-level data set:

```{r}
data.list <- readRDS(params$input_data_p)
dat.l <- data.list$dat.l # data in long format
dat.w <- data.list$dat.w # data in wide format
display_dataframe_head(dat.l)
```

After the filtering done in `data_prep.R`, there are 19 UPS1 proteins remaining, even though 48 were originally spiked in.

```{r}
# which proteins were spiked in?
spiked.proteins <- dat.l %>% distinct(Protein) %>% filter(stri_detect(Protein, fixed='ups')) %>% pull %>% as.character
tmp=dat.l %>% distinct(Protein) %>% pull %>% as.character
# protein subsampling
if (params$subsample_p>0 & params$subsample_p==floor(params$subsample_p) & params$subsample_p<=length(tmp)){
  sub.prot <- tmp[sample(1:length(tmp), size=params$subsample_p)]
  if (length(spiked.proteins)>0) sub.prot <- c(sub.prot,spiked.proteins)
  dat.l <- dat.l %>% filter(Protein %in% sub.prot)
}
```

We store the metadata in `sample.info` and show some entries below. We also pick technical replicates with a dilution factor of 0.5 as the reference condition of interest. Each condition is represented by two of eight reporter Channels in each Run. 

```{r}
# specify # of varying component variants and their names
variant.names <- c('no_summ', 'median', 'sum')
n.comp.variants <- length(variant.names)

# get some data parameters created in the data_prep script
referenceCondition <- data.list$data.params$referenceCondition
condition.color <- data.list$data.params$condition.color
ma.onesample.num <- data.list$data.params$ma.onesample.num
ma.onesample.denom <- data.list$data.params$ma.onesample.denom
ma.allsamples.num <- data.list$data.params$ma.allsamples.num
ma.allsamples.denom <- data.list$data.params$ma.allsamples.denom
# create data frame with sample information
sample.info <- get_sample_info(dat.l, condition.color)
# get channel names
channelNames <- remove_factors(unique(sample.info$Channel))
```

```{r}
display_dataframe_head(sample.info)
referenceCondition
channelNames
```

# Unit scale component: log2 transformation of reporter ion intensities

We use the default unit scale: the log2-transformed reporter ion intensities.

```{r}
dat.unit.l <- dat.l %>% mutate(response=log2(intensity)) %>% select(-intensity)
```

# Summarization component

In the next three subsections, let's closely look at three selected ways of data summarization of isobarically labelled proteomics data.

```{r}
dat.summ.l <- emptyList(variant.names)
```

## No summarization

Even though it is widely adopted approach to have only one quantification value per peptide within a sample (this could be a summary value such as mean or median, or the one with the highest value of some score provided by a database search engine), in this variant we are going to analyse all available PSM-level data.

```{r}
dat.summ.l$no_summ <- dat.unit.l
```

## Median summarization (PSM to peptide)

Median summarization is simple: within each Run and within each Channel, we replace multiple related observations with their median such that we end up with a single value per peptide.

```{r, eval=!params$load_outputdata_p}
dat.summ.l$median <- aggFunc(dat.unit.l, 'response', group.vars=c('Mixture', 'TechRepMixture', 'Run', 'Channel', 'Condition', 'BioReplicate', 'Protein',  'Peptide'), 'median')
```

## Sum summarization (PSM to peptide)

Sum summarization is completely analogous to the Median summarization, except that we sum values instead of taking the median. Note that sum normalization is _not_ equivalent to mean normalization: yes, rows containing NA values are removed, but there may be multiple PSMs per peptide. Since we know that there is a strong peptide-run interaction, summing PSM values per peptide may result in bias by run.

```{r, eval=!params$load_outputdata_p}
dat.summ.l$sum <- aggFunc(dat.unit.l, 'response', group.vars=c('Mixture', 'TechRepMixture', 'Run', 'Channel', 'Condition', 'BioReplicate', 'Protein',  'Peptide'), 'sum')
```

# Normalization component: linear mixed-effects model

The manuscripts of [Hill et al.](https://doi.org/10.1021/pr070520u) and [Oberg et al.](https://doi.org/10.1021/pr700734f) illustrated the application of linear models for removing various biases potentially present in isobarically labelled data. Inspired by this approach, we fit the following linear mixed-effect model, which corrects the observed reporter ion intensities $y_{i, j(i), q, l, s}$ for imbalance stemming from run $b_q$ and run-channel $v_{l(q)}$ fixed effects, as well as protein $p_i$ and run-protein $b_q \times f_{j(i)}$ random effects:

$$ \log_2y_{i, j(i), q, l, s} = u + b_q + v_{l(q)} + p_i + (b_q \times f_{j(i)}) + \varepsilon_{i, j(i), q, l, s} $$
where $p_i \sim N(0, \sigma_p^2),\, (b_q \times f_{j(i)}) \sim N(0, \sigma_f^2),\, \varepsilon_{i, j(i), q, l, s} \sim N(0, \sigma^2)$

The model is fitted using the `lmer()` function based on the REML criterion. Afterwards, the "subject-specific" residuals (which involve subtraction of the empirical bayes estimates of the random effects) of the model are treated as normalized values and used in further analyses.

```{r, eval=!params$load_outputdata_p}
dat.norm.l <- lapply(dat.summ.l, function(x) {
  mod <- lmer(response ~ Run + Run:Channel + (1|Protein)  + (1|Run:Peptide), data=x)
  x$response <- residuals(mod)
  return(x)})
```

# QC plots

Before getting to the DEA section, let's do some basic quality control and take a sneak peek at the differences between the component variants we've chosen. First, however, we should make the data completely wide, so that each sample gets it's own unique column.

```{r, eval=!params$load_outputdata_p}
dat.nonnorm.summ.l <- lapply(dat.summ.l, function(x) aggFunc(x, 'response', group.vars=c('Mixture', 'TechRepMixture', 'Run', 'Channel', 'Condition', 'BioReplicate', 'Protein', 'Peptide'), 'median')) # before normalization (you still need to aggregate the data because of 'dat.summ.l <- dat.unit.l' operation)
dat.nonnorm.summ.l <- lapply(dat.nonnorm.summ.l, function(x) aggFunc(x, 'response', group.vars=c('Mixture', 'TechRepMixture', 'Run', 'Channel', 'Condition', 'BioReplicate', 'Protein'), 'median'))
dat.norm.summ.l <- lapply(dat.norm.l, function(x) aggFunc(x, 'response', group.vars=c('Mixture', 'TechRepMixture', 'Run', 'Channel', 'Condition', 'BioReplicate', 'Protein', 'Peptide'), 'median')) # after normalization
dat.norm.summ.l <- lapply(dat.norm.summ.l, function(x) aggFunc(x, 'response', group.vars=c('Mixture', 'TechRepMixture', 'Run', 'Channel', 'Condition', 'BioReplicate', 'Protein'), 'median')) # after normalization

# make data completely wide (also across runs)
## normalized data
dat.norm.summ.w2 <- lapply(dat.norm.summ.l, function(x) {
  dat.tmp <- pivot_wider(data=x, id_cols=Protein, names_from=Run:Channel, values_from=response, names_sep=':')
  return(dat.tmp) } )

## non-normalized data
dat.nonnorm.summ.w2 <- lapply(dat.nonnorm.summ.l, function(x) {
  dat.tmp <- pivot_wider(data=x, id_cols=Protein, names_from=Run:Channel, values_from=response, names_sep=':')
  return(dat.tmp) } )
```

```{r, echo=FALSE, eval=params$load_outputdata_p}
load(paste0('modelbased_summarization_outdata', params$suffix_p, '.rda'))
```

## Boxplots

The boxplots of all three summarization variants are very alike.

```{r}
par(mfrow=c(2,2))
for (i in 1:n.comp.variants){
  boxplot_ils(dat.nonnorm.summ.l[[variant.names[i]]], paste('raw', variant.names[i], sep='_'))
  boxplot_ils(dat.norm.summ.l[[variant.names[i]]], paste('normalized', variant.names[i], sep='_'))}
```

## MA plots

We then make MA plots of two single samples taken from condition `r ma.allsamples.num` and condition `r ma.allsamples.denom`, measured in different MS runs (samples *`r ma.onesample.num`* and *`r ma.onesample.denom`*, respectively). Clearly, the normalization had a strong variance-reducing effect on the fold changes. One more time, the plots of normalized values are almost identical. 

```{r}
for (i in 1:n.comp.variants){
  p1 <- maplot_ils(dat.nonnorm.summ.w2[[variant.names[i]]], ma.onesample.num, ma.onesample.denom, scale='log', paste('raw', variant.names[i], sep='_'), spiked.proteins)
  p2 <- maplot_ils(dat.norm.summ.w2[[variant.names[i]]], ma.onesample.num, ma.onesample.denom, scale='log', paste('normalized', variant.names[i], sep='_'), spiked.proteins)
  grid.arrange(p1, p2, ncol=2)}
```

To increase the robustness of these results, let's make some more MA plots, but now for all samples from condition `r ma.allsamples.num` and condition `r ma.allsamples.denom` (quantification values averaged within condition).
Both the unnormalized and normalized data now show less variability as using more samples (now 8 in both the enumerator and denominator instead of just one) in the fold change calculation makes the rolling average more robust. A similar conclusion still holds as in the case of single-sample MA plots, with one exception that now it seems the spike-in proteins induce a small positive bias (blue curve is rolling average) for low abundance proteins.

```{r}
channels.num <- sample.info %>% filter(Condition==ma.allsamples.num) %>% distinct(Sample) %>% pull
channels.denom <- sample.info %>% filter(Condition==ma.allsamples.denom) %>% distinct(Sample) %>% pull
for (i in 1:n.comp.variants){
  p1 <- maplot_ils(dat.nonnorm.summ.w2[[variant.names[i]]], channels.num, channels.denom, scale='log', paste('raw', variant.names[i], sep='_'), spiked.proteins)
  p2 <- maplot_ils(dat.norm.summ.w2[[variant.names[i]]], channels.num, channels.denom, scale='log', paste('normalized', variant.names[i], sep='_'), spiked.proteins)
  grid.arrange(p1, p2, ncol=2)}
```

## PCA plots

Now, let's check if these multi-dimensional data contains some kind of grouping; It's time to make PCA plots.

### Using all proteins

Even though PC1 does seem to capture the conditions, providing a gradient for the dilution number, only the 0.125 condition is completely separable in the normalized data. Moreover, all variants lead to comparable PCA plots of the normalized data, with perhaps slightly worse separation in case of Sum summarization. Also, as we previously warned, using sum summarization in the presence of a peptide-run interaction effect gives rise to strong sample clustering by run (see the raw_sum PCA plot). Therefore, a normalization method that is able to correct for the peptide-run interaction becomes essential.

```{r}
par(mfrow=c(1, 2))
for (i in 1:n.comp.variants){
  pcaplot_ils(dat.nonnorm.summ.w2[[variant.names[i]]] %>% select(-'Protein'), info=sample.info, paste('raw', variant.names[i], sep='_'))
  pcaplot_ils(dat.norm.summ.w2[[variant.names[i]]] %>% select(-'Protein'), info=sample.info, paste('normalized', variant.names[i], sep='_'))}
par(mfrow=c(1, 1))
```

There are only 19 proteins supposed to be differentially expressed in this data set, which is only a very small amount in both relative (to the 4083 proteins total) and absolute (for a biological sample) terms.

### Using spiked proteins only

Therefore, let's see what the PCA plots look like if we were to only use the spiked proteins in the PCA. 
Now, the separation quality is very comparable between the three variants.

```{r, eval=length(spiked.proteins)>0}
par(mfrow=c(1, 2))
for (i in 1:n.comp.variants){
  pcaplot_ils(dat.nonnorm.summ.w2[[variant.names[i]]] %>% filter(Protein %in% spiked.proteins) %>% select(-'Protein'), info=sample.info, paste('raw', variant.names[i], sep='_'))
  pcaplot_ils(dat.norm.summ.w2[[variant.names[i]]] %>% filter(Protein %in% spiked.proteins) %>% select(-'Protein'), info=sample.info, paste('normalized', variant.names[i], sep='_'))}
par(mfrow=c(1, 1))
```

Notice how for all PCA plots, the percentage of variance explained by PC1 is now much greater than when using data from all proteins.
In a real situation without spiked proteins, you might plot data corresponding to the top X most differential proteins instead.

## HC (hierarchical clustering) plots

The PCA plots of all proteins has a rather lower fraction of variance explained by PC1. We can confirm this using the hierarchical clustering dendrograms below: when considering the entire multidimensional space, the different conditions are not very separable at all. This is not surprising as there is little biological variation between the conditions: there are only 19 truly differential proteins, and they all (ought to) covary in exactly the same manner (i.e., their variation can be captured in one dimension).

### Using all proteins

```{r}
par(mfrow=c(1, 2))
for (i in 1:n.comp.variants){
  dendrogram_ils(dat.nonnorm.summ.w2[[variant.names[i]]] %>% select(-'Protein'), info=sample.info, paste('raw', variant.names[i], sep='_'))
  dendrogram_ils(dat.norm.summ.w2[[variant.names[i]]] %>% select(-'Protein'), info=sample.info, paste('normalized', variant.names[i], sep='_'))}
par(mfrow=c(1, 1))
```

## Run effect p-value plot

Our last quality check involves a measure of how well each variant was able to assist in removing the run effect. 
Below are the distributions of p-values from a linear model for the `response` variable with `Run` as a covariate.
If the run effect was removed successfully, these p-values ought to be large. Clearly, the raw data contains a run effect, which all variants are able to partially remove. 

```{r}
plots <- vector('list', n.comp.variants)
for (i in 1:n.comp.variants){
dat <- list(dat.nonnorm.summ.l[[variant.names[i]]], dat.norm.summ.l[[variant.names[i]]])
names(dat) <- c(paste('raw', variant.names[i], sep='_'), paste('normalized', variant.names[i], sep='_'))
plots[[i]] <- run_effect_plot(dat)}
grid.arrange(grobs = plots, nrow=n.comp.variants) 
```

# DEA component: linear (mixed-effects) model

A typical approach to Differential Expression Analysis, which we also employ here, assumes testing only one protein at a time. Therefore, for each slice of the normalized data corresponding to a certain protein $i$, we fit another linear mixed-effect model given by:

$$ w_{j(i), q, l, s} = m + r_c + z_{l(q)} + \eta_{j(i), c, q, l, s} $$

with $w$ as the normalized values (the subject-specific residuals of the normalization model), $m$ as the model intercept, $r_c$ as the difference in expression levels between the biological conditions, $z_{l(q)}$ as the random effect accounting for the potential correlation within each sample induced by the protein repeated measurements, and $\eta_{j(i),c,q,l,s}$ as the random error. Note the index $s$ which implies the PSM-level data (i.e. not aggregated data). 

**Technical comment 1**: obtaining log fold changes corresponding to the contrasts of interest when working with log intensities and 'treatment' model parametrization (i.e., model intercept represents the reference condition) is immediately straightforward: these are coefficients corresponding to the $r_c$ effect.

**Technical comment 2**: while introducing the $z_{l(q)}$ random effect into the DEA model is justified, not every protein will have enough repeated measurements (i.e., multiple peptides and/or PSMs corresponding to different peptide modifications, charge states and retention times) for the random effect being estimable. However, in such cases the fixed effect is estimable and its inference remain valid. 

**Technical comment 3**: after testing, we make a correction for multiple testing using the Benjamini-Hochberg method in order to keep the FDR under control.

```{r, eval=!params$load_outputdata_p}
dat.dea <- lapply(dat.norm.l, function(x){
  return(lmm_dea(dat=x, mod.formula='response ~ Condition + (1|Run:Channel)', referenceCondition, scale='log'))})
```

```{r}
# character vectors containing logFC and p-values columns
dea.cols <- colnames(dat.dea[[1]])
logFC.cols <- dea.cols[stri_detect_fixed(dea.cols, 'logFC')]
significance.cols <- dea.cols[stri_detect_fixed(dea.cols, 'q.mod')]
n.contrasts <- length(logFC.cols)
```

For each condition, we now get the fold changes, p-values, q-values (BH-adjusted p-values), and some other details (head of dataframe below).

```{r}
display_dataframe_head(dat.dea[[1]])
```

```{r, echo=FALSE, eval=params$save_outputdata_p}
# save output data
save(dat.nonnorm.summ.l
     ,dat.norm.summ.l
     ,dat.nonnorm.summ.w2
     ,dat.norm.summ.w2
     ,dat.norm.l
     ,dat.summ.l
     ,dat.dea, file=paste0('modelbased_summarization_outdata', params$suffix_p, '.rda'))
```

# Results comparison

Now, the most important part: let's find out how our component variants have affected the outcome of the DEA.

## Confusion matrix

A confusion matrix shows how many true and false positives/negatives each variant has given rise to. Spiked proteins that are DE are true positives, background proteins that are not DE are true negatives. We calculate this matrix for all conditions and then calculate some other informative metrics based on the confusion matrices: accuracy, sensitivity, specificity, positive predictive value and negative predictive value. 

In case of the `0.125 vs 0.5` and `1 vs 0.5` contrasts, all summarization methods return analagous results. The biological difference in the `0.667 vs 0.5` contrast, however, seems to be too small to be picked by the proposed modelling approach, regardless of the summarization method.

```{r, results='asis'}
cm <- conf_mat(dat.dea, 'q.mod', 0.05, spiked.proteins)
print_conf_mat(cm, referenceCondition)
```

## Scatter plots

To see whether the three summarization approaches produce similar results on the detailed level of individual proteins, we make scatter plots and check the correlation between their fold changes and between their significance estimates (q-values, in our case). 

In general, not summarizing quantification values, summarization with median or sum lead to well correlated q-values and log fold changes.

```{r}
scatterplot_ils(dat.dea, significance.cols, 'q-values', spiked.proteins, referenceCondition)
scatterplot_ils(dat.dea, logFC.cols, 'log2FC', spiked.proteins, referenceCondition)
```

## Volcano plots

The volcano plot combines information on fold changes and statistical significance. The spike-in proteins are colored blue; the magenta, dashed line indicates the theoretical fold change of the spike-ins. 

These plots are also nearly undistinguishable between the three variants.

```{r}
for (i in 1:n.contrasts){
  volcanoplot_ils(dat.dea, i, spiked.proteins, referenceCondition)}
```

## Violin plots

A good way to assess the general trend of the fold change estimates on a more 'macroscopic' scale is to make a violin plot. Ideally, there will be some spike-in proteins that attain the expected fold change  (red dashed line) that corresponds to their condition, while most (background) protein log2 fold changes are situated around zero.

Clearly, the empirical results _tend towards_ the theoretical truth, but not a single observation attained the fold change it should have attained. There is clearly a strong bias towards zero fold change, which may partly be explained by the ratio compression phenomenon in mass spectrometry, although the effect seems quite extreme here.

Although, shape of spike-in protein log fold changes somewhat differ between the three variants, the underlying message is simple - there are no substantial differences, at least in this particular dataset.

```{r}
# plot theoretical value (horizontal lines) and violin per variant
if (length(spiked.proteins)>0) violinplot_ils(lapply(dat.dea, function(x) x[spiked.proteins, logFC.cols]), referenceCondition) else violinplot_ils(lapply(dat.dea, function(x) x[,logFC.cols]), referenceCondition,  show_truth = FALSE)
```

# Conclusions

In this investigation of three selected summarization variants, we could not show any definite advantage of using all available information contained in PSM-level data (no summarization) as compared with popular approaches that exploit median and sum aggregation to the peptide level. Perhaps other datasets should be studied to answer this question. Our last recommendation pertains choosing an appropriate/effective normalization method while opting for using sum summarization.

# Session information
```{r}
sessionInfo()
```