bonferroni gRNA integration explanation / significant non-targeting pairs

[sweavs111]

Hello, big fan of sceptre so far. What is practically different from using the gRNA integration strategy "union" vs "bonferroni". I couldn't find the answer in the e-book, I apologize if it is in there and I missed it.

I have this question because when I run the same sceptre analysis with all three gRNA integration strategies (data frame with 1699 discovery pairs), I get very different numbers of negative control pairs that turn up as significant.

union: 45/978 negative control pairs as significant
singleton: 26/3938 negative control pairs as significant
bonferroni: 2/871 negative control pairs as significant

I was hoping to learn what is happening in the bonferroni integration strategy. Additionally, is bonferroni clearly the gRNA integration strategy that I should use because it produces the least amount of false positives?

[ekatsevi]

For more information on gRNA integration strategies, please read Section 2.4 of the e-book. In short, the union strategy treats cells with any gRNA targeting a given element as all the same and carries out one test comparing cells with any gRNA targeting a given element against a set of control cells, whereas the Bonferroni strategy tests each gRNA targeting a given element separately, and then combines those p-values via the Bonferroni global test. We expect the union strategy to have better power if gRNAs targeting the same element have roughly similar effects, whereas we expect the Bonferroni strategy to have better power if gRNAs have very different effects (e.g. due to some of your gRNAs not working). In our experience, we have found the union strategy to have better power more often than the Bonferroni strategy, which is why it is our default. Having said that, it appears you have some non-negligible miscalibration present in your union analysis. If you want to get a reliable answer as quickly as possible, you could choose the Bonferroni strategy since it appears to have much better calibration. Otherwise, I would recommend investing some effort in improving the calibration of the union strategy by following the suggestions in Section 5.4 of the e-book. If you were successful, you could then compare the power of the union and Bonferroni strategies if you have any positive control pairs in your data, choosing the one with the better power. (Note that comparing power in the presence of miscalibration can be misleading, because one method may have more true positives simply by virtue of having more false positives.) Otherwise, if you have no positive control pairs, I would recommend using the union strategy if you can improve its calibration. Finally, if you are unable to improve the calibration of the union strategy, then you can go ahead and use the Bonferroni strategy.

[timothy-barry]

Thanks for the kind words. I just wanted to add that section 10.4 of the book also contains information about gRNA integration strategies that may be relevant.

[sweavs111]

Thank you both for the responses. I will look into fixing some of the mis-calibration issues and re-run with the union parameter

[timothy-barry]

Hey Seth,

In addition to the strategies outlined in the book for improving calibration, you may want to try including the top PCs of the gene expression matrix as covariates in the model. We have found that doing this can improve calibration on some datasets. We likely will add a module for this purpose to the package in the nearish future. Below is some example code that does this. I used Seurat to compute the top PCs. Perhaps you could give this a shot as well and let us know how it goes?

Tim

library(Seurat)
library(sceptre)

# load the data
response_matrix <- readRDS("THP_i_threewell_ge-sceptre-matrix.rds")
grna_matrix <- readRDS("THP_i_threewell_grna-sceptre-matrix.rds")
grna_target_data_frame <- readRDS("THP_i_threewell_gRNAi-target.rds")
batch_df <- readRDS("THP_i_threewell_extra-covariate-df.rds")
response_names <- readRDS("THP_i_threewell_gene-names.rds")
discovery_pairs <- readRDS("THP_i_threewell_gRNAi-discovery-pairs.rds")

# 0. run an initial Seurat analysis to obtain the top PCs
pbmc <- CreateSeuratObject(counts = response_matrix, project = "pbmc3k", min.cells = 3, min.features = 200)
pbmc <- NormalizeData(pbmc)
pbmc <- FindVariableFeatures(pbmc, selection.method = "vst", nfeatures = 2000)
pbmc <- ScaleData(pbmc)
pbmc <- RunPCA(pbmc)
DimPlot(pbmc, reduction = "pca") + NoLegend() # clearly some low-dimensional structure in the data (cell cycle or type)?
top_pcs <- pbmc[["pca"]]@cell.embeddings[, 1:5]

# add the top pcs to the extra covariats
extra_covariates <- cbind(batch_df, top_pcs)

# 1. Create sceptre object
sceptre_object <- import_data(
  response_matrix = response_matrix,
  grna_matrix = grna_matrix,
  grna_target_data_frame = grna_target_data_frame,
  moi = "high",
  extra_covariates = extra_covariates,
  response_names = response_names
)

# 2. Set the analysis params
my_formula <- formula(~log(response_n_nonzero) + log(response_n_umis) + log(grna_n_nonzero + 1) + log(grna_n_umis + 1) + batch + PC_1 + PC_2 + PC_3 + PC_4 + PC_5)
sceptre_object <- set_analysis_parameters(sceptre_object, discovery_pairs,
                                          side = "left",
                                          formula_object = my_formula)

# Continue with rest of analysis

[sweavs111]

Thanks for the suggestion and the code! I added the top 5 PCs as covariates and it cut the number of false positives in half.