modelbased_unit.Rmd

---
title: "**Unit scaling** strategy comparison for **Model-based** analysis of isobarically labeled proteomic data."
author: "Piotr Prostko, Joris Van Houtven"
date: '`r format(Sys.time(), "%B %d, %Y,%H:%M")`'
output: 
  html_document:
    toc: true
    toc_depth: 2
    toc_float: true
    number_sections: true
    theme: flatly
    code_folding: "hide"
editor_options: 
  chunk_output_type: console
params:
  input_data_p: 'data/input_data.rds'
  suffix_p: ''
  load_outputdata_p: FALSE
  save_outputdata_p: FALSE
  subsample_p: 0
---

```{r, setup, include=FALSE}
# default knitr options for all chunks
knitr::opts_chunk$set(
  message=FALSE,
  warning=FALSE,
  fig.width=12,
  fig.height=7
)
```

<span style="color: grey;">
_This notebook is one in a series of many, where we explore how different data analysis strategies affect the outcome of a proteomics experiment based on isobaric labeling and mass spectrometry. Each analysis strategy or 'workflow' can be divided up into different components; it is recommend you read more about that in the [introduction notebook](intro.html)._
</span>

In this notebook specifically, we investigate the effect of varying the **Unit scale** component on the outcome of the differential expression results. The three component variants are: **log2 intensity**, **(untransformed) intensity**, **ratio**.

<span style="color: grey;">
_The R packages and helper scripts necessary to run this notebook are listed in the next code chunk: click the 'Code' button. Each code section can be expanded in a similar fashion. You can also download the [entire notebook source code](modelbased_unit.Rmd)._
</span>

```{r}
library(caret)
library(lme4)
library(lmerTest)
library(ggplot2)
library(stringi)
library(gridExtra)
library(ggfortify)
library(dendextend)
library(psych)
library(kableExtra)
library(tidyverse)
library(dtplyr)
source('util/other_functions.R')
source('util/plotting_functions.R')
```

Let's load our PSM-level data set:

```{r}
data.list <- readRDS(params$input_data_p)
dat.l <- data.list$dat.l # data in long format
dat.w <- data.list$dat.w # data in wide format
display_dataframe_head(dat.l)
```

After the filtering done in `data_prep.R`, there are 19 UPS1 proteins remaining, even though 48 were originally spiked in.

```{r}
# which proteins were spiked in?
spiked.proteins <- dat.l %>% distinct(Protein) %>% filter(stri_detect(Protein, fixed='ups')) %>% pull %>% as.character
tmp=dat.l %>% distinct(Protein) %>% pull %>% as.character
# protein subsampling
if (params$subsample_p>0 & params$subsample_p==floor(params$subsample_p) & params$subsample_p<=length(tmp)){
  sub.prot <- tmp[sample(1:length(tmp), size=params$subsample_p)]
  if (length(spiked.proteins)>0) sub.prot <- c(sub.prot,spiked.proteins)
  dat.l <- dat.l %>% filter(Protein %in% sub.prot)
  dat.w <- dat.w %>% filter(Protein %in% sub.prot)
}
```

We store the metadata in `sample.info` and show some entries below. We also pick technical replicates with a dilution factor of 0.5 as the reference condition of interest. Each condition is represented by two of eight reporter Channels in each Run. 

```{r}
# specify # of varying component variants and their names
variant.names <- c('log2_intensity', 'intensity', 'ratio')
n.comp.variants <- length(variant.names)
scale.vec <- c('log', 'raw','raw')  # ratios are considered raw, because they are basically mean-normalized intensities

# get some data parameters created in the data_prep script
referenceCondition <- data.list$data.params$referenceCondition
channelsOrdered <- data.list$data.params$channelsOrdered
condition.color <- data.list$data.params$condition.color
ma.onesample.num <- data.list$data.params$ma.onesample.num
ma.onesample.denom <- data.list$data.params$ma.onesample.denom
ma.allsamples.num <- data.list$data.params$ma.allsamples.num
ma.allsamples.denom <- data.list$data.params$ma.allsamples.denom
# create data frame with sample information
sample.info <- get_sample_info(dat.l, condition.color)
# get channel names
channelNames <- remove_factors(unique(sample.info$Channel))
```

```{r}
display_dataframe_head(sample.info)
referenceCondition
channelNames
```

# Unit scale component

In the next three subsections, let's look at the three different unit scales of reporter ion intensities.

```{r}
dat.unit.l <- emptyList(variant.names)
```

## log2 transformation of reporter ion intensities

Taking the log2 transform is a widely used approach, especially if downstream analysis involves linear regression. It renders the distribution of values more symmetrical, which is often presumed to be log-normal.

```{r}
dat.unit.l$log2_intensity <- dat.l %>% mutate(response=log2(intensity)) %>% select(-intensity)
```

Alternatively, one can use untransformed intensities.

## intensities on the original scale
```{r}
dat.unit.l$intensity <- dat.l %>% rename(response=intensity)
```

## intensity ratios

Using ratios is an approach that has been around for a long time, as it inherently mitigates some of the unwanted variance. There are many ways to define ratios, some of which are wiser than others. We take one of the best approaches that we assume may be commonly used by researchers, which is to divide each sample's quantification values by the average of the corresponding feature values of the reference condition samples in that same run.

```{r}
# use half-wide data to compute within-run average of PSM channels corresponding to the reference Condition
refCols <- sample.info %>% filter(Condition==referenceCondition) %>% distinct(Channel) %>% pull
denom.df=dat.l %>% filter(Condition==referenceCondition) %>% pivot_wider(id_cols=-one_of('Condition', 'BioReplicate'),names_from='Channel', values_from='intensity')
denom.df$denom=apply(denom.df[,refCols], 1, function(x) mean(x, na.rm=T))
denom.df=denom.df[,c('Run', 'Protein', 'Peptide', 'RT', 'Charge', 'PTM', 'denom')]
dat.unit.l$ratio <- dat.l %>% left_join(denom.df, by=c('Run', 'Protein', 'Peptide', 'RT', 'Charge', 'PTM')) %>% mutate(response=intensity/denom) %>% select(-c(intensity, denom)) 
```

# Summarization component: no summarization

As a default approach (consult the manuscript or the [introduction notebook](intro.html)) we opt for no summarization, meaning that all PSM-level data is going to be exploited in further analyses. This also means that multiple  variants of the same peptide within a sample carrying different charge, modifications or detected at different retention times are kept as is.

```{r}
# no summarization 
dat.summ.l <- dat.unit.l
```

# Normalization component: linear mixed-effects model

The manuscripts of [Hill et al.](https://doi.org/10.1021/pr070520u) and [Oberg et al.](https://doi.org/10.1021/pr700734f) illustrated the application of linear models for removing various biases potentially present in isobarically labelled data. Inspired by this approach, we fit the following linear mixed-effect model, which corrects the observed reporter ion intensities $y_{i, j(i), q, l, s}$ for imbalance stemming from run $b_q$ and run-channel $v_{l(q)}$ fixed effects, as well as protein $p_i$ and run-protein $b_q \times f_{j(i)}$ random effects:

$$ \log_2y_{i, j(i), q, l, s} = u + b_q + v_{l(q)} + p_i + (b_q \times f_{j(i)}) + \varepsilon_{i, j(i), q, l, s} $$
where $p_i \sim N(0, \sigma_p^2),\, (b_q \times f_{j(i)}) \sim N(0, \sigma_f^2),\, \varepsilon_{i, j(i), q, l, s} \sim N(0, \sigma^2)$

The model is fitted using the `lmer()` function based on the REML criterion. Afterwards, the "subject-specific" residuals (which involve subtraction of the empirical bayes estimates of the random effects) of the model are treated as normalized values and used in further analyses.

```{r, eval=!params$load_outputdata_p}
dat.norm.l <- dat.summ.l
# fit normalization model
norm.models <- lapply(dat.summ.l, function(x) return(lmer(response ~ Run + Run:Channel + (1|Protein) + (1|Run:Peptide), data=x)))
# assign normalized values
for (i in 1:n.comp.variants){ 
  dat.norm.l[[variant.names[i]]]$response <- residuals(norm.models[[variant.names[i]]])}
```

# QC plots

Before getting to the DEA section, let's do some basic quality control and take a sneak peek at the differences between the component variants we've chosen. First, however, we should make the data completely wide, so that each sample gets it's own unique column.

```{r, eval=!params$load_outputdata_p}
dat.nonnorm.summ.l <- lapply(dat.summ.l, function(x) aggFunc(x, 'response', group.vars=c('Mixture', 'TechRepMixture', 'Run', 'Channel', 'Condition', 'BioReplicate', 'Protein', 'Peptide'), 'median')) 
dat.nonnorm.summ.l <- lapply(dat.nonnorm.summ.l, function(x) aggFunc(x, 'response', group.vars=c('Mixture', 'TechRepMixture', 'Run', 'Channel', 'Condition', 'BioReplicate', 'Protein'), 'median'))

dat.norm.summ.l <- lapply(dat.norm.l, function(x) aggFunc(x, 'response', group.vars=c('Mixture', 'TechRepMixture', 'Run', 'Channel', 'Condition', 'BioReplicate', 'Protein', 'Peptide'), 'median')) 
dat.norm.summ.l <- lapply(dat.norm.summ.l, function(x) aggFunc(x, 'response', group.vars=c('Mixture', 'TechRepMixture', 'Run', 'Channel', 'Condition', 'BioReplicate', 'Protein'), 'median')) 

# make data completely wide (also across runs)
## normalized data
dat.norm.summ.w2 <- lapply(dat.norm.summ.l, function(x){
  dat.tmp <- pivot_wider(data=x, id_cols=Protein, names_from=Run:Channel, values_from=response, names_sep=':')
  return(dat.tmp)})
colnames(dat.norm.summ.w2[[1]])

## non-normalized data
dat.nonnorm.summ.w2 <- lapply(dat.nonnorm.summ.l, function(x){
  dat.tmp <- pivot_wider(data=x, id_cols=Protein, names_from=Run:Channel, values_from=response, names_sep=':')
  return(dat.tmp)})
```

```{r, echo=FALSE, eval=params$load_outputdata_p}
load(paste0('modelbased_unit_outdata', params$suffix_p, '.rda'))
```

## Boxplot

The median of normalized intensity values are set to zero for all of the three unit variants, as indicated by the red line. However, some large outlying values enters the picture in case of untransformed intensities. 

```{r}
par(mfrow=c(1,2))
for (i in 1:n.comp.variants){
  boxplot_ils(dat.nonnorm.summ.l[[variant.names[i]]], paste('raw', variant.names[i], sep='_'))
  boxplot_ils(dat.norm.summ.l[[variant.names[i]]], paste('normalized', variant.names[i], sep='_'))}
```

## MA plots

We then make MA plots of two single samples taken from condition `r ma.allsamples.num` and condition `r ma.allsamples.denom`, measured in different MS runs (samples *`r ma.onesample.num`* and *`r ma.onesample.denom`*, respectively). Clearly, the normalization had a strong variance-reducing effect on the fold changes. However, the visualisations of untransformed intensities and ratios are not reliable as some proteins were ommited. This is due to that the $X$ and $Y$ normalized values in the $M=\log_2(X/Y)$ and $A=0.5(\log_2X+\log_2Y)$ calculations can have opposite signs and then $\log_2$ would be undefined. Hence only the $\log_2$ unit component resulted in a correctly centered around zero MA plot. 

```{r}
for (i in 1:n.comp.variants){
  p1 <- maplot_ils(dat.nonnorm.summ.w2[[variant.names[i]]], ma.onesample.num, ma.onesample.denom, scale.vec[i], paste('raw', variant.names[i], sep='_'), spiked.proteins)
  p2 <- maplot_ils(dat.norm.summ.w2[[variant.names[i]]], ma.onesample.num, ma.onesample.denom, scale.vec[i], paste('normalized', variant.names[i], sep='_'), spiked.proteins)
  grid.arrange(p1, p2, ncol=2)}  
```

To increase the robustness of these results (but the opposite signs problem cannot be remedied), let's make some more MA plots, but now for all samples from condition `r ma.allsamples.num` and condition `r ma.allsamples.denom` (quantification values averaged within condition).
Both the unnormalized and normalized data now show less variability, and extremely so in the case of unnormalized ratios. Using more samples (now 8 in both the enumerator and denominator instead of just one) in the fold change calculation makes the rolling average more robust. Here again, only the $\log_2$ unit plot is "trustworthy", and in that plot, it seems the spike-in proteins induce a small positive bias (blue curve is rolling average) for low abundance proteins.

```{r}
channels.num <- sample.info %>% filter(Condition==ma.allsamples.num) %>% distinct(Run:Channel) %>% pull
channels.denom <- sample.info %>% filter(Condition==ma.allsamples.denom) %>% distinct(Run:Channel) %>% pull
for (i in 1:n.comp.variants){
  p1 <- maplot_ils(dat.nonnorm.summ.w2[[variant.names[i]]], channels.num, channels.denom, scale=scale.vec[i], paste('raw', variant.names[i], sep='_'), spiked.proteins)
  p2 <- maplot_ils(dat.norm.summ.w2[[variant.names[i]]], channels.num, channels.denom, scale=scale.vec[i], paste('normalized', variant.names[i], sep='_'), spiked.proteins)
grid.arrange(p1, p2, ncol=2)}
```

## PCA plots

Now, let's check if these multi-dimensional data contains some kind of grouping; It's time to make PCA plots.

### Using all proteins

In the PCA plot of the log transformed intensities variant, PC1 seems to capture the conditions, providing a gradient for the dilution number. Clearly, this is not the case for the other two variants. 

```{r}
par(mfrow=c(1, 2))
for (i in 1:n.comp.variants){
  pca.scale=FALSE
  if (variant.names[i] %in% c('intensity', 'intensity_fix')) pca.scale=TRUE
  pcaplot_ils(dat.nonnorm.summ.w2[[variant.names[i]]] %>% select(-'Protein'), info=sample.info, paste('raw', variant.names[i], sep='_'), scale=pca.scale)
  pcaplot_ils(dat.norm.summ.w2[[variant.names[i]]] %>% select(-'Protein'), info=sample.info, paste('normalized', variant.names[i], sep='_'), scale=pca.scale)}
```

There are only 19 proteins supposed to be differentially expressed in this data set, which is only a very small amount in both relative (to the 4083 proteins total) and absolute (for a biological sample) terms. 

### Using spiked proteins only

Therefore, let's see what the PCA plots look like if we were to only use the spiked proteins in the PCA. 
Now, in the PCA plots of the two variants that previously failed miserably to group samples according to the dilution factor, the sample separation has improved.

```{r, eval=length(spiked.proteins)>0}
par(mfrow=c(1, 2))
for (i in 1:n.comp.variants){
  pca.scale=FALSE
  if (variant.names[i] %in% c('intensity', 'intensity_fix')) pca.scale=TRUE
  pcaplot_ils(dat.nonnorm.summ.w2[[variant.names[i]]] %>% filter(Protein %in% spiked.proteins) %>% select(-'Protein'), info=sample.info, paste('raw', variant.names[i], sep='_'), scale=pca.scale)
  pcaplot_ils(dat.norm.summ.w2[[variant.names[i]]] %>% filter(Protein %in% spiked.proteins) %>% select(-'Protein'), info=sample.info, paste('normalized', variant.names[i], sep='_'), scale=pca.scale)}
```

Notice how for all PCA plots, the percentage of variance explained by PC1 is now much greater than when using data from all proteins.
In a real situation without spiked proteins, you might plot data corresponding to the top X most differential proteins instead.

## HC (hierarchical clustering) plots

The PCA plots of all proteins has a rather lower fraction of variance explained by PC1. We can confirm this using the hierarchical clustering dendrograms below: when considering the entire multidimensional space, the different conditions are not very separable at all. This is not surprising as there is little biological variation between the conditions: there are only 19 truly differential proteins, and they all (ought to) covary in exactly the same manner (i.e., their variation can be captured in one dimension).

### Using all proteins

```{r}
par(mfrow=c(1,2))
for (i in 1:n.comp.variants){
  dendrogram_ils(dat.nonnorm.summ.w2[[variant.names[i]]] %>% select(-'Protein'), info=sample.info, paste('raw', variant.names[i], sep='_'))
  dendrogram_ils(dat.norm.summ.w2[[variant.names[i]]] %>% select(-'Protein'), info=sample.info, paste('normalized', variant.names[i], sep='_'))}
```

## Run effect p-value plot

Our last quality check involves a measure of how well each variant was able to assist in removing the run effect. 
Below are the distributions of p-values from a linear model for the `response` variable with `Run` as a covariate.
If the run effect was removed successfully, these p-values ought to be large. Clearly, the raw data contains a run effect, which all variants are able to partially remove. 

```{r, fig.width=12, fig.height=11}
plots <- vector('list', n.comp.variants)
for (i in 1:n.comp.variants){
dat <- list(dat.nonnorm.summ.l[[variant.names[i]]], dat.norm.summ.l[[variant.names[i]]])
names(dat) <- c(paste('raw', variant.names[i], sep='_'), paste('normalized', variant.names[i], sep='_'))
plots[[i]] <- run_effect_plot(dat)}
grid.arrange(grobs = plots, nrow=n.comp.variants)
```

# DEA component: linear mixed-effects model

A typical approach to Differential Expression Analysis, which we also employ here, assumes testing only one protein at a time. Therefore, to each slice of the normalized data corresponding to a certain protein $i$, we fit another linear mixed-effect model given by:

$$ w_{j(i), q, l, s} = m + r_c + z_{l(q)} + \eta_{j(i), c, q, l, s} $$
with $w$ as the normalized values (the subject-specific residuals of the normalization model), $m$ as the model intercept, $r_c$ as the difference in expression levels between the biological conditions, $z_{l(q)}$ as the random effect accounting for the potential correlation within each sample induced by the protein repeated measurements, and $\eta_{j(i),c,q,l,s}$ as the random error. Note the index $s$ which implies the PSM-level data (i.e. not aggregated data). 

**Technical comment 1**: performing DEA analysis on different unit scales is like asking for problems. This is because the mean difference of log2 intensity values is not the same as the log2 of the ratio of mean raw intensities, nor the mean ratio of raw intensities, symbolically shown below:

$$mean(\log_2 B) - mean(\log_2 A) \neq log_2 (\frac{mean(B)}{mean(A)}) \neq \log_2 (mean(\frac{B}{A}))$$

**Technical comment 2**: obtaining log fold changes corresponding to the contrasts of interest when working with log intensities and 'treatment' model parametrization (i.e., model intercept represents the reference condition) is immediately straightforward: these are coefficients corresponding to the $r_c$ effect. Things get more complicated if one opts for untransformed intensities or ratios; then one has to apply this simple manipulation: $\log FC = \log \frac {m + r_c}{m}$. 

**Technical comment 3**: while introducing the $z_{l(q)}$ random effect into the DEA model is justified, not every protein will have enough repeated measurements (i.e., PSMs corresponding to different peptide modifications, charge states and retention times) for the random effect being estimable. However, in such cases the fixed effect is estimable and its inference remain valid. 

**Technical comment 4**: after testing, we make a correction for multiple testing using the Benjamini-Hochberg method in order to keep the FDR under control.

```{r, eval=!params$load_outputdata_p}
dat.dea <- emptyList(variant.names)
for(i in seq_along(dat.dea)){
  dat.dea[[variant.names[i]]] <- lmm_dea(dat=dat.norm.l[[variant.names[i]]], mod.formula='response ~ Condition + (1|Run:Channel)', scale=scale.vec[i], referenceCondition)}
```

```{r}
# character vectors containing logFC and p-values columns
dea.cols <- colnames(dat.dea[[1]])
logFC.cols <- dea.cols[stri_detect_fixed(dea.cols, 'logFC')]
significance.cols <- dea.cols[stri_detect_fixed(dea.cols, 'q.mod')]
n.contrasts <- length(logFC.cols)
```

For each condition, we now get the fold changes, p-values, q-values (BH-adjusted p-values), and some other details (head of dataframe below).

```{r}
display_dataframe_head(dat.dea[[1]])
```

```{r, echo=FALSE, eval=params$save_outputdata_p}
# save output data
save(dat.nonnorm.summ.l
     ,dat.norm.summ.l
     ,dat.nonnorm.summ.w2
     ,dat.norm.summ.w2
     ,dat.norm.l
     ,dat.summ.l
     ,dat.dea, file=paste0('modelbased_unit_outdata', params$suffix_p, '.rda'))
```

# Results comparison

Now, the most important part: let's find out how our component variants have affected the outcome of the DEA.

## Confusion matrix

A confusion matrix shows how many true and false positives/negatives each variant has given rise to. Spiked proteins that are DE are true positives, background proteins that are not DE are true negatives. We calculate this matrix for all conditions and then calculate some other informative metrics based on the confusion matrices: accuracy, sensitivity, specificity, positive predictive value and negative predictive value. 

In case of the `0.125 vs 0.5` and `1 vs 0.5` contrasts, only the log of intensities provides good outcomes. The biological difference in the `0.667 vs 0.5` contrast, however, seems to be too small to be picked by the proposed modelling approach, regardless of the unit scale. Moreover, the reason for the subpar performance of untransformed intensities and ratios that we observe here can lie in the flawed log fold change estimation and/or erroneous variance structure in the data. Hopefully, this conundrum can be explained after inspecting the forthcoming visualisations.

```{r, results='asis'}
cm <- conf_mat(dat.dea, 'q.mod', 0.05, spiked.proteins)
print_conf_mat(cm, referenceCondition)
```

## Scatter plots

To see whether the three Unit scales produce similar results on the detailed level of individual proteins, we make scatter plots and check the correlation between their fold changes and between their significance estimates (q-values, in our case). 

First, the q-values of spike-in proteins associated with intensities on the original scale and ratios are generally larger. 

```{r}
scatterplot_ils(dat.dea, significance.cols, 'q-values', spiked.proteins, referenceCondition)
```

Second, the distribution of fold changes stemming from the analysis of untransformed intensities and ratios are unnaturally wide, both for the spike-in and background proteins. If we recall that the boxplots of normalized intensity and ratio values were centered around zero and the technical comment #2, we can conjecture that the terms $m$ and $m + r_c$ can be also close to zero, and therefore the entire $\log FC = \log \frac {m + r_c}{m}$ can be unstable or even undefined.

```{r}
scatterplot_ils(dat.dea, logFC.cols, 'log2FC', spiked.proteins, referenceCondition)
```

## Volcano plots

The volcano plot combines information on fold changes and statistical significance. The spike-in proteins are colored blue; the magenta, dashed line indicates the theoretical fold change of the spike-ins. 

It again becomes evident that only the log transformed intensities yield sensible log fold changes as well as manage to detect the largest number of the spike-in proteins (though not in the case of the `0.667 vs 0.5` contrast, as discussed earlier). 

```{r, fig.width=12, fig.height=10}
for (i in 1:n.contrasts){
  volcanoplot_ils(dat.dea, i, spiked.proteins, referenceCondition)}
```

## Violin plots

A good way to assess the general trend of the fold change estimates on a more 'macroscopic' scale is to make a violin plot. Ideally, there will be some spike-in proteins that attain the expected fold change  (red dashed line) that corresponds to their condition, while most (background) protein log2 fold changes are situated around zero.

Clearly, the empirical results _tend towards_ the theoretical truth, but not a single observation attained the fold change it should have attained. There is clearly a strong bias towards zero fold change, which may partly be explained by the ratio compression phenomenon in mass spectrometry, although the effect seems quite extreme here.

Due to the unstability of fold changes of untransformed intensities and ratios, which is an artifact dragging from applying a not well suited normalization technique, the log fold changes of the spike proteins of the `0.667 vs 0.5` and `1 vs 0.5` contrasts could virtually not be computed.

```{r}
# plot theoretical value (horizontal lines) and violin per variant
if (length(spiked.proteins)>0) violinplot_ils(lapply(dat.dea, function(x) x[spiked.proteins, logFC.cols]), referenceCondition) else violinplot_ils(lapply(dat.dea, function(x) x[,logFC.cols]), referenceCondition,  show_truth = FALSE)
```

# Conclusions

In this notebook we demonstrated that one should not use (untransformed) intensities in a model-based normalization as the former putatively exhibit multiplicative biases, but the latter can only apply additive corrections. Mixing additive and multiplicative scales tends to give distorted results, including the usage of (intensity-based) ratios. 

# Session information

```{r}
sessionInfo()
```