workshop1.obsolete

---
title: "Workshop Mutation PSPG 245B"
author: "Lee and Martell"
date: "`r format(Sys.time(), '%d %B, %Y')`"
output:
  html_document:
    df_print: paged
editor_options:
  chunk_output_type: console
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE, warning=FALSE )
```

# PSPG 245B
In this workshop you will learn about the fundamentals of how to download and utilize cancer genomic data. By the end of the course you should be able to generate your own full R markdown for your assigned TCGA ID, including plots and identification of actionable mutations and/or gene.

### this work shop will be based on a study for Invasive Breast Cancer  https://pubmed.ncbi.nlm.nih.gov/26451490/ 
Ciriello et al, 2015. 

  * Breast cancer is an ideal dataset for study, as there is a sufficient amount of knowledge available to validate our results, yet there remains ample opportunity for interesting and valuable discoveries.
  * This workshop will be entirely focused on the analysis of public datasets. The data will be collected and analyzed in the context of current knowledge.
  * The primary GOAL of this workshop is to provide you with the foundation to pursue further research projects. While there are two main goals, we hope that you will be able to use and expand upon what you learn here. In this 2-3 hour works shop we will attempt to: 
    + "discover" a general molecular profile for the TCGA Breast Cancer dataset.  
    + https://pubmed.ncbi.nlm.nih.gov/26451490/
    + Each participant has been assigned a TCGA patient identifier for this dataset, and your task is to use the tools provided in this workshop to gather as much information as possible. Atthe end of the workshop you should have a complete report for your assigned sample. 


### Wile the assignment is an important aspect of this workshop, the primary objective is to spark your enthusiasm for bioinformatics and illustrate the depth of information that can be obtained even with a cursory examination of data. The potential is limitless, so feel free to ask questions and embrace the learning experience. We encourage you to fully participate and take advantage of this. Contact information below. 
  + Alex: alex.lee2@ucsf.edu
  + Henry: henry.martell@ucsf.edu

### It is assumed that you have a basic understanding of R, but if not, that's okay. We will provide guidance and support as needed throughout the workshop.

## The following are required for this workshop
<div style="background-color:#b6d2db">

Software requirements  

* R >3.5 and have the below packages installed. 
* use BiocManager::install() when possible. 
    + cgdsr
    + ggplot2
    + knitr
    + kableExtra
    + dplyr
    + VennDiagram
    + reshape2
    + gridExtra
    + ggrepel
    + DT
    + ggpubr
    
Please make sure to either clone the github repo or download the entire dropbox folder. Unzip this to a folder.  


* https://github.com/ahdee/workshop.PSPG-245B.2.git
* https://github.com/ahdee/workshop.PSPG-245B.2/archive/refs/heads/master.zip

In this workshop, we will primarily utilize the cBioPortal API to obtain the necessary data. It is therefore important that you have an internet connection.

We will be using the TCGA study on Breast Invasive Carcinoma (TCGA, Cell 2015), https://pubmed.ncbi.nlm.nih.gov/26451490/

* Each student should had been assigned a TCGA ID prior to this.
    + TCGA-C8-A3M7-01
    + TCGA-GM-A2D9-01
    + TCGA-BH-A1FL-01
    + TCGA-E2-A14Z-01
	+ TCGA-BH-A1FC-01
	+ TCGA-AC-A2QJ-01
	+ TCGA-EW-A1P8-01
	+ TCGA-E2-A1LE-01
	+ TCGA-E2-A1LK-01
	+ TCGA-AC-A2FE-01


	
</div>


<hr>

<font color="red"> IMPORTANT: please remember to set the working directory to your source by clicking Sesssion-> Set working directory -> to source file location</font>

```{r, include=T, echo=T, message=FALSE, warning=FALSE, fig=TRUE}

# make sure to set to working directory 
# if you don't do this your script will fail to run. 
# Session->set working directory -> to source file location 

# load require package and "source" an auxiliary R script design for this workshop.  
 

# library ( cgdsr), cgdsr now depecrated -- good lesson here 
library(cBioPortalData)
library ( ggplot2)
library(knitr)
library(kableExtra)
library ( dplyr)
library ( reshape2)
library(VennDiagram)
library(gridExtra)
library (ggrepel)
library ( DT )
library ( ggpubr )

# the auxiliary script will help manage some of the more complex/redundant tasks
source("auxi.R")

# initiate cbioportal API

cbiop <- cBioPortal(
  hostname = "www.cbioportal.org",
  protocol = "https",
  api. = "/api/v2/api-docs",
  token = character()
)

```

# Learn how to communicate with cbioportal and study what is available. 
* This portal is hierarchically structured. This is the workflow. 
    + List of all the cancer studies available on this portal. 
    + Identify the study you want
    + List all the "cases" for that study, a case is a description of both data available for the study and/or subsets of certain phenotypes
	+ With each case you can get the list of all the associated TCGA identify
	+ which you can then use to get all the clinical data associated with the TCGA samples. 
	+ moreover you use the study to get specific genetic profile ( datasets available for downloading ). 
	+ From here you can then download the actual data. 

# comment key 
  + QQQ = rhetorical, however answer it if you could.  
  + HINT 
  + X: bonus exercises  
  + note: extra info you may or may not need

```{r }

# Start by getting a list of cancer studies available at cbio
studies <- getStudies(cbiop, buildReport = TRUE)

# This will return a table with a list of available cancer studies to draw from 
k2( head ( studies, 10) , "example list of data available in cbioportal")

# note: k2 is one of those cheat codes to make tables print out nicely 
# Want to know more about a function? use "?" and the function name 

? t.test
? getStudies

# lets find out how many studies are available to the public
dim ( studies ) # dim is a function to tabulate a dataframe
nrow ( studies ) # nrow tells you how many rows 

# QQQ how do you think we can use this to look for the dataset we need among > 300 studies?
## we can use the grepl function which uses regular expression to search within objects and returns a 
## a vector of T and F 

grepl("Breast", studies$name, ignore.case = T )
## an index is a position in a vector, eg apple, pear, peach
which ( grepl("Breast", studies$name, ignore.case = T ) == T)


studies$name [grepl("Breast", studies$name, ignore.case = T ) ]

# QQQ, what is the T for? what if you use lower case instead of upper case
## you can subset a dataframe by telling it which row you want, either with a positional ( index ) or logical vector 

# According to the syllabus we are looking for TCGA study on Breast Invasive Carcinoma (TCGA, Cell 2015) 
# However lets first search for all the breast cancer studies available so that we can look to see what is available. 
# X, find how many studies are their with the Breast Invasive Carcinoma in the name column 



breast = studies[ grepl("Breast", studies$name, ignore.case = T ), ]
DT::datatable(breast )

# QQQ can you tell which is the study id you need  -
# we can be more specific here
## Breast Invasive Carcinoma (TCGA, Cell 2015)
DT::datatable ( studies[ grepl("Breast Invasive Carcinoma", studies$name, ignore.case = T )  , ] )



## study id is:  brca_tcga_pub2015 


brca.study = "brca_tcga_pub2015"

# note: each institution has different ways of organizing their data
# from the study we can identify all the available data by using a case list API 


mycaselist = getCaseLists(cbiop,brca.study)




# QQQ: Which case do you think is most relevant for this workshop? 
# hint: look through the case_list_name
# hint: the last column which was not displayed are all the TCGA ids associated with the each of the case 
# note: the descriptions contains not only type of samples but sometimes it includes what type of data they are.  
k2 (mycaselist[ , 1:3] )
# View ( mycaselist)


# X how would you get only samples labeled as 	Her2-positive breast tumors

 



```

## For **reproducibility** we need to consider either the "freeze" case or store a local copy.  This is an important point that is often overlook!
* Also be aware that cbio is only 1 of many different portals available to the public. It is not necessarily the best.  

```{r}

# here we see that brca_tcga_pub2015_all is probably the best to use because it includes ALL Complete Tumors 
case.list.id = "brca_tcga_pub2015_all"



# now using All Complete Tumors lets see if your sample is present. 

# first lets get all the TCGA ID associated with this case 
mysample = mycaselist[ mycaselist$case_list_id == case.list.id, ]$case_ids
# lets spit it up into a nice vector by delimiting space
mysample = unlist ( strsplit(mysample, " ") )

#k2 ( mysample )

# here is the list of samples that were assigned. 
## important. Do you see your sample? 
cat ( samples, sep="\n")
```

## Here I chose a random sample to test 
* Important here: you need to replace this with your own assigned TCGA ID. Let us know immediately if your assigned TCGA is not in the list.  

```{r}

# QQQ mysample variable now contains all the TCGA samples in this study, can you check if your sample is exists? 

myid = "TCGA-C8-A131-01" # put your assigned id here 

mysample[mysample== myid]

# here is another way
intersect ( mysample, myid)
# yet another 
mysample[grepl(myid, mysample)]



### Now lets check if the total samples matches with what is expected: 818 
length ( mysample)
```

## With this list of TCGA ids we can also get clinical data for each of the samples. 

```{r}
# lets get clinical some data
myclinicaldata = getClinicalData(cbiop,getCaseLists(cbiop,brca.study)[1,1])
# clean up the names a bit 
myclinicaldata$pid = gsub ( "\\.", "-", row.names(myclinicaldata))

# lets get the clinical information for your specific TCGA id.
# QQQ what does your sample say. 
# lets look through the clinical data and see what info do we have. 
# QQQ can you think of any usage for these data? 

DT::datatable ( t ( myclinicaldata[myclinicaldata$pid %in% myid, ] ))

# important fields to consider. 
# AGE, AJCC_PATHOLOGIC_TUMOR_STAGE, Ethnicity
# DFS ( disease free), OS ( overall survival)
# this is quite extraordinary how much data is in here. 

# QQQ: think about your experimental designs and what you can do with this! 


### X Lets have some fun here, even with just a table of clinical data we can do some investigation. 

# For example how would you figure out if one of your sample ( the assigned TCGA) is a Triple-negative or HER2+. 
# hint" look at the caselist and work backwards using the intersect function

k2 ( head ( mycaselist[ ,c("case_list_id",	 "case_list_name",	 "case_list_description",	 "cancer_study_id"	)], 50), "list of cases for the BRCA CELL 2015 study" ) 

t3.study = "brca_tcga_pub2015_trineg"
t3 = mycaselist[ mycaselist$case_list_id == t3.study, ]$case_ids
t3 = unlist ( strsplit(t3, " ") )


her2p.study = "brca_tcga_pub2015_her2pos"
her2p = mycaselist[ mycaselist$case_list_id == her2p.study, ]$case_ids
her2p = unlist ( strsplit(her2p, " ") )

er_neg.study = "brca_tcga_pub2015_erneg"
er_neg = mycaselist[ mycaselist$case_list_id == er_neg.study, ]$case_ids
er_neg = unlist ( strsplit(er_neg, " ") )


er_POS.study = "brca_tcga_pub2015_erpos"
er_POS = mycaselist[ mycaselist$case_list_id == er_POS.study, ]$case_ids
er_POS = unlist ( strsplit(er_POS, " ") )


# answer. 

t3[t3==myid]
her2p[her2p==myid]
er_POS[er_POS==myid]


# X: suppose you ask how many are HER2 positive but also ER +? 

venn1 = data.frame ( her2p= length ( her2p)
                    , er_neg= length ( er_POS )
                    , int= length ( intersect(er_POS,her2p))  
                        )
names ( venn1 )[1:2] = c("Her2+","ER+") 

venn.this ( 
    venn1, 
    type = 2, cp= c("#3C8A9B" , "#9B445D"), 
    dgg=180
) 



# Now out of curiosity we want to know if there is a difference in age based on these attributes. 

age.both = intersect (er_POS,her2p )
age.both=myclinicaldata[ myclinicaldata$pid %in% age.both, ]$AGE
age.Her2only.noER = setdiff( her2p, er_POS)
age.Her2only.noER = myclinicaldata[ myclinicaldata$pid %in% age.Her2only.noER, ]$AGE

temp = rbind ( 
data.frame ( age = age.both, type= rep("both", length(age.both)) )    
,data.frame ( age = age.Her2only.noER, type= rep("Heronly", length(age.Her2only.noER)))   
)

median ( age.Her2only.noER )
median ( age.both )
t.test( age.Her2only.noER , age.both)

ggplot( temp, aes(x=age, fill=type)) +
    geom_histogram( color="#e9ecef", alpha=0.6, position = 'identity') +
    scale_fill_manual(values=c("#69b3a2", "#404080")) 

# X can you think of where there might be difference in age? 
# hint: There is evidence of a correlation between age and mutation burden in certain cancer types

```

BONUS question to take home. As a pharmacologist how can you use DFS and OS? For example 
the histogram above only shows age of onset.  However, question what if you hypothesize
that having HER2 positive is much worse of survival then having both HER2 and ER+?


```{r}
 

# Now lets see what information is available for your study 
mygeneticprofile = getGeneticProfiles(cbiop,brca.study)
k2 ( mygeneticprofile, "types of data available for chosen case" )


# so looking at that we now know it contains several interesting modalities.  
# lets pick up 3 types of analysis that may be of interests 

mutation = mygeneticprofile[grepl( "Mutation", mygeneticprofile$genetic_profile_name), ]$genetic_profile_id 

# QQQ what other modalities do you think may be of interest

cna = mygeneticprofile[grepl( "Putative copy", mygeneticprofile$genetic_profile_name),1 ]

exp = mygeneticprofile[grepl( "V2 RSEM", mygeneticprofile$genetic_profile_name),1 ]
exp = exp[ grepl ( "mrna$", exp)]

```



# Now that we have a basic understanding of what is available lets go and download the actual data. 
**Important** Due to the nature of the API and having so many people use it at once the connection might fail. If that is the case retry or if all else fails use the frozen version which has already been previously downloaded. 

```{r}

# lets collect this through a loop so not to overwhelm the system
# its important to be mindful of resources when donwloading from public data.  We need to play nice and consider the server load. 
# QQQ what do you think we can do to miminize server load? 

# For this class we are going to just download genes that are relevant to cancer. 
# genes that had already been implicated in cancer.
# We do this by looking through different organization and cross reference a list of genes. 

k2 ( cancer.list[17:25, ], "cancer list")

length ( cancer.gene)


# lets check this list to make sure genes that are relevant to breast cancer exists 

imp = as.character ( cancer.list[grepl("^BRCA|^ATM$|^BARD1$|^CDH1$|^CHEK2$|^NBN$|^NF1$|^PALB2$|^PTEN$|^TP53$", cancer.list$gene), ]$gene )

# ? why did we add ^ in front and $ for only some genes and not others?

k2 ( cancer.list[cancer.list$gene %in% imp, ], "breast cancer genes")



total = ceiling ( length( cancer.gene)/100  )  *100
mutations = data.frame (   stringsAsFactors = F )
expression = data.frame (   stringsAsFactors = F )
e = 1
for(i in seq(from=150, to=total, by=150) ){
   
    if ( i > length(cancer.gene)){
        i = length(cancer.gene)
    }
    
    print ( paste ( e, i ))
    temp = getMutationData(cbiop , brca.study, mutation, cancer.gene[e:i])

    
    
    mutations = rbind ( mutations, temp)
    Sys.sleep (2) # lets give the system a break
        e2 = 1
        exp.temp2 = 1
        for(i2 in seq(from=150, to=length ( myclinicaldata$pid), by=150)){
        exp.temp = getProfileData(cbiop , cancer.gene[e:i]
                              , exp
                              , getCaseLists(cbiop,brca.study)[1,1] , myclinicaldata$pid[e2:i2])
        if ( exp.temp2 == 1){
            exp.temp2 =   t ( exp.temp)
        }else {
            exp.temp2 =  cbind ( exp.temp2,  t ( exp.temp)) 
        }
        e2 = i2+ 1 
        Sys.sleep (1) # lets give the system a break
        }
        
    e = i + 1    
    expression = rbind ( expression, exp.temp2)    
    
    
    
}



# create coordinates
mutations$igv = paste ( mutations$chr, mutations$start_position, mutations$end_position, mutations$reference_allele, mutations$variant_allele)

# there is an error in the mutation table.  The links are obsolete.  Here we fix it. 
# by replacing the older url with this: http://mutationassessor.org/r3

mutations$xvar_link = gsub('getma.org','http://mutationassessor.org/r3', mutations$xvar_link)
mutations$xvar_link_msa = gsub('getma.org','http://mutationassessor.org/r3', mutations$xvar_link_msa )
mutations$xvar_link_pdb = gsub('getma.org','http://mutationassessor.org/r3', mutations$xvar_link_pdb  )


# for backup and moreover we want to save this to ensure 
# reproducibility

cache = 0 
if ( cache == 1){
  saveRDS( list (
    mutatios = mutations, 
    expression = expression 
    
  ), "freeze.rds")
}

```


# lets study the mutation table

```{r}

dim ( mutations )

# lets take a few minutes here to go over the different fields. 
names ( mutations )


unique( mutations$mutation_type)

#################### BACK to lecture 

# QQQ based on the lecture previously can identify the different mutation_type
# QQQ can you figure out what each of these mean?  
# for example which of these are nonsynonymous substitution?
# QQQ for druggability which one of these do you think its more likely to be useful

# QQQ can you guess what is functional_impact_score
data.frame ( table ( mutations$functional_impact_score)  )

# make sure that your sample has mutations. 
# www if you don't see any or obvious mutation where else can look at for the driver signal? 
mt = nrow ( mutations [ mutations$case_id ==  myid , ] )
mt = mutations [ mutations$case_id ==  myid , ] 
mt = mt %>% replace(is.na(.), '')  %>% data.frame()

# get an idea of functional score distrubution for your sample. 
data.frame ( table (  mt$functional_impact_score) )


# lets clear up junk 
gc()
                          

```


# Now lets talk a bit about how to identify pathogenic mutations. 


```{r}

# QQQ: What are some of things that we can look look for? 

# For example we can detect important mutations is to see if other resources are available. 
# Here we use cosmic a manually curated list of recurrent mutations in cancer and look for keyword breast 

## WARNING THIS IS NOT A COMPREHENSIVE LIST. AND IT IS OUTDATED. 
## only for demo 

k2 ( head ( cosmic.70[ grepl("breast",  cosmic.70$description ), ] ), "breast cancer in cosmic")
# note there are are several other databases such as clinvar that we can look for 

# lets integrate cosmic into our mutation table
# all coordnates are are based on hg19 

# note: how we are merging by coordinates.  This is the most accurate way to do this. 
# ? why do you think all.x = T and all.y=F 

mutations = merge ( mutations, cosmic.70, by="igv", all.x=T, all.y=F )



# let us find out how many mutations we have that is also found in cosmic 
# the description field should not be empty if there was a cosmic entry

nrow ( mutations[ ! is.na ( mutations$description ),  ] ) / nrow (mutations)


# X check if cosmic contain BRCA mutations.  This is a good sanity check. 

k2 ( head ( mutations[ grepl("^BRCA", mutations$gene_symbol), c("amino_acid_change","chr.x","start_position","end_position","reference_allele", "variant_allele",
"gene_symbol","case_id","mutation_status","mutation_type")] ) )

# X go to https://cancer.sanger.ac.uk/cosmic to get latest dataset since this is very outdated.  



# X can you find all BRCA mutation that is Missense_Mutation AND found is cosmic in the mutation db?
# hint, mutations$mutation_type == "Missense_Mutation"

```

# Now that the data has been succesfully donwloaded and annotated lets study it a bit more. 

```{r}



# lets tabulate the types of mutations  

mutation.type = data.frame ( table ( mutations$mutation_type))

mutation.type = mutation.type[ order ( mutation.type$Freq), ]

mutation.type$Var1 = factor ( mutation.type$Var1, levels = mutation.type$Var1)

ggplot(mutation.type, aes(Var1,Freq), label=Freq ) +
    geom_bar(aes(fill = Freq), stat="identity", position = "dodge") +
    coord_flip() +
    scale_fill_distiller(palette = "RdBu") + xlab("") + ylab("") +
    theme(strip.text.y = element_text(angle = 0), legend.position="none") +
    geom_text(aes(label=Freq), position=position_dodge(width=0.9), vjust=.4, hjust = .5, size=5) +
    theme(panel.grid.major = element_blank(), 
          panel.grid.minor = element_blank(),
          panel.background = element_blank(), 
          axis.line = element_line(colour = "black"), 
          text = element_text(size=16),  # size of label
          axis.text.x = element_text(angle=0, hjust=1) ) + ggtitle ( "All mutation types")


# As mentioned previously one reliable way to look for relevant mutations is to match it against what has been previously observed and/or study. Here we can try to use Cosmic however other organization may be relevant as well including clinvar

cosmic.mutation = mutations [ !is.na(mutations$description), ]

mutation.type = data.frame ( table ( cosmic.mutation$mutation_type))

mutation.type = mutation.type[ order ( mutation.type$Freq), ]
mutation.type$Var1 = factor ( mutation.type$Var1, levels = mutation.type$Var1)

ggplot(mutation.type, aes(Var1,Freq), label=Freq ) +
    geom_bar(aes(fill = Freq), stat="identity", position = "dodge") +
    coord_flip() +
    scale_fill_distiller(palette = "RdBu") + xlab("") + ylab("") +
    theme(strip.text.y = element_text(angle = 0), legend.position="none") +
    geom_text(aes(label=Freq), position=position_dodge(width=0.9), vjust=.4, hjust = .5, size=5) +
    theme(panel.grid.major = element_blank(), 
          panel.grid.minor = element_blank(),
          panel.background = element_blank(), 
          axis.line = element_line(colour = "black"), 
          text = element_text(size=16),  # size of label
          axis.text.x = element_text(angle=0, hjust=1) ) + ggtitle ( "Mutation types found in cosmic")


# QQQ seems like missense mutation is the most recurrent.  Can you think of any way to filter the data further 

# in the lecture we talked about depth.  So lets calculate this. 

mutations$dp = mutations$variant_read_count_tumor + mutations$reference_read_count_tumor 
quantile ( mutations$dp)

 ggplot(mutations, aes(x=dp)) + 
  geom_density( color="darkblue", fill="steelblue" ) + geom_vline(xintercept = 21, linetype="dotted", 
                color = "grey", size=1.5) + ggtitle ( " density plot, total depth")

# QQQ by looking at this plot and quartiles we can see that there is a good amount of mutations that are under 21 
# QQQ how would you remove this?

# before  
dim ( mutations )
# after 
mutations = mutations[ mutations$dp > 21, ]
dim (mutations )


# another thing we can look at is Allele frequency
# QQQ: do you remember what af is? 
# lets calculate allele freqeuncy here as well 

mutations$af = mutations$variant_read_count_tumor / mutations$dp

 ggplot(mutations, aes(x=af)) + 
  geom_density( color="darkblue", fill="#e8975a" ) + geom_vline(xintercept = .1, linetype="dotted", 
                color = "grey", size=1.5) + ggtitle ( " density plot, allele freqeuncy")

# QQQ what do you think it means when af is higher for a particular mutation 
# X pick the top 10 mutations ranked by af 

mutations = mutations[ order ( - mutations$af ) , ]

# clean mut head
cleanmut = c ("gene_symbol", "mutation_type", "amino_acid_change", "reference_allele", "variant_allele" , "reference_read_count_tumor", "variant_read_count_tumor" , "dp", "af")  

DT::datatable ( head ( mutations [ , c( cleanmut )]) )



# lets further filter the mutations with any af > .1
mutations = mutations[ mutations$af > .1, ]
dim (mutations)


# plot 20 highest recurrent gene mutations

mutation.gene = data.frame ( table ( mutations$gene_symbol))

mutation.gene = mutation.gene[ order ( - mutation.gene$Freq), ]
mutation.gene$Var1 = factor ( mutation.gene$Var1, levels = rev ( mutation.gene$Var1) )

ggplot(mutation.gene [ 1:20, ] , aes(Var1,Freq), label=Freq ) +
    geom_bar(aes(fill = Freq), stat="identity", position = "dodge") +
    coord_flip() +
    scale_fill_distiller(palette = "RdBu") + xlab("") + ylab("") +
    theme(strip.text.y = element_text(angle = 0), legend.position="none") +
    geom_text(aes(label=Freq), position=position_dodge(width=0.9), vjust=.4, hjust = .5, size=5) +
    theme(panel.grid.major = element_blank(), 
          panel.grid.minor = element_blank(),
          panel.background = element_blank(), 
          axis.line = element_line(colour = "black"), 
          text = element_text(size=16),  # size of label
          axis.text.x = element_text(angle=0, hjust=1) ) + ggtitle ( "Most recurrent genes")



# based on literature these are some of the genes involved in breast cancer. 
imp 

# lets study them and see what types of mutations corresponds most with these genes. 

mutation.brca = data.frame ( table ( mutations[ mutations$gene_symbol %in% imp, ] $mutation_type))

mutation.brca = mutation.brca[ order ( mutation.brca$Freq), ]
mutation.brca$Var1 = factor ( mutation.brca$Var1, levels = mutation.brca$Var1)

ggplot(mutation.brca, aes(Var1,Freq), label=Freq ) +
    geom_bar(aes(fill = Freq), stat="identity", position = "dodge") +
    coord_flip() +
    scale_fill_distiller(palette = "RdBu") + xlab("") + ylab("") +
    theme(strip.text.y = element_text(angle = 0), legend.position="none") +
    geom_text(aes(label=Freq), position=position_dodge(width=0.9), vjust=.4, hjust = .5, size=5) +
    theme(panel.grid.major = element_blank(), 
          panel.grid.minor = element_blank(),
          panel.background = element_blank(), 
          axis.line = element_line(colour = "black"), 
          text = element_text(size=16),  # size of label
          axis.text.x = element_text(angle=0, hjust=1) ) + ggtitle ( "Mutation-types in genes associated with breast cancer")

# lets assume that we don't know these genes 
# X can you think of another way to rediscover these genes and/or find novel ones?

# X HERE lets have some fun The discovery part.  And there are many many ways to do this.  
# so be creative! 

# here is a 2 sec way to do this. 
# simply look in cosmic to see how often these genes appears in breast cancer. 
imp.brac = data.frame ( table ( mutations[ grepl( "breast", mutations$description), ]$gene_symbol ) )
imp.brac$fraction = imp.brac$Freq / nrow ( mutations)
imp.brac = imp.brac [ order ( -imp.brac$fraction) , ]

k2 ( head ( imp.brac, 20), "top cosmic genes found in breast cancer")

# let look at the mutation breakdown 
top5 = as.character ( imp.brac$Var1[1:5] )
top5 = mutations[ mutations$gene_symbol %in% top5, c ( "amino_acid_change", "gene_symbol" )]
top5 = data.frame ( table ( top5 ) )
top5 = top5 [ order ( - top5$Freq), ]

# quick hack to add url, will not work with *

k2 ( head ( top5, 20), "top 20 recurrent mutations")

# X take the first mutation and google it what does it say? 
# take the second, what does google say? 

# part of studying mutations is look for clinical significance.  One way to do this is to match your mutations to actionable 
# targets


mutations$drug = paste ( mutations$gene_symbol, mutations$amino_acid_change)

action = drug[ drug$match %in% unique ( mutations$drug  ), ]
action.f = c("gene" ,"variant" , "disease", "drugs", "evidence_type", 
             "clinical_significance", "evidence_statement", "evidence_civic_url", "gene_civic_url" , "match"
)
DT::datatable ( head ( action[ ,action.f], 10)  )

# X can you figure out which samples have druggable targets?  
# hint look at evidence type. Can yo figure out which of these are diagnostic vs targetable? 

DT::datatable ( head ( action[action$clinical_significance == "Sensitivity/Response" ,action.f], 10) )



```


# Besides Mutation data we also downloaded expression RNAseq data in the form of TPM. 
* Here we review two types of very important pathway analysis.
  + Gene set Enrichment Analysis ( GSEA )
  + Over-Representation Analysis 
## IMPORTANT 
  * for demon only the group comparison we are doing here should never be attempted in a real experiment 
  * TPM values we donwload are only for certain analysis like cluster, correlation etc... and attempts to compare groups are at best, an estimation 
  * moreover the comparisons we are making should be completed with all genes, however here we are only using ~1.7 K cosmic genes
* Here is an example of comparing samples that test positive for HER2 vs. those that don't
* _NOTE_ ERBB2 is the HER2 gene
* EGFR is the HER1 gene 

```{r fig=TRUE,fig.width=12, fig.height=12}

# clean up 
expressionc <-expression  %>% 
    select(where(~!all(is.na(.)))) %>% data.frame()
# log2 transform 
expressionc= data.frame ( log2 ( expressionc + 1 ))

expressionc = na.omit(expressionc)

her2p2 = gsub ( "-", ".", her2p)

last_j = which ( colnames (expressionc ) %in% her2p2 )
last_b = which ( ! colnames (expressionc ) %in% her2p2 )

results = apply (expressionc, 1, function(x) wtest(x))
results = data.frame ( t ( results ))
colnames(results) = c( "pv", "logfc", "her2_ave","base_ave", "all_ave" )
results$fdr = p.adjust(results$pv,  method = "fdr",  n=nrow ( results))
results = results[ order ( results$fdr), ]

results = na.omit(results)

results$class  = ifelse(  results$fdr > 0.05  , "no-change",
                         ifelse( 
                            results$logfc >= .5, "up", 
                            
                            ifelse( 
                            results$logfc <= -.5, "down",
                            "no-change"
                           
                           )
                         
                         ) )

# lets check the distribution 
table ( results$class)
results[results$class == "up", ]
label_this <- head ( results[results$class == "up", ] , 10)
label_this$label = row.names ( label_this)

results = data.frame ( results)

ggplot(results , aes(logfc, -log10(fdr))) +
    geom_point(aes(col=class), size=3, alpha=.6) +
    scale_color_manual(values=c("up"="#5EAE64", "no-change"="grey", "down"= "#D55A3D"), breaks=c("up", "down", "no-change"), labels=c(paste0("up in ",'her2',"(",nrow(results [results $class=='up',]),")"), paste0("down in ", 'her2' ,"(",nrow(results[results $class=='down',]),")"), "no-change")) + 
    ggtitle(paste0( " Volcano plot ")) +theme_bw(base_size = 30)  +  theme(legend.text=element_text(size=15), legend.position = "bottom") + guides(color = guide_legend(override.aes = list(size=10)))

results$gene = row.names ( results )

#results = merge ( results, ens[!duplicated ( ens$external_gene_name), c("external_gene_name",  "entrezgene_id")] , by.x = "gene", by.y="external_gene_name", all.x=T, all.y=F )

results = results[ order ( -results$logfc), ]

glist = setNames( results$logfc, results$gene)
glist = sort(glist, decreasing = TRUE)
```

```{r, echo=FALSE, include=FALSE}

library(clusterProfiler)
library (DOSE)
library( 'org.Hs.eg.db', character.only = TRUE)

## WARNING DEMO ONLY  - for this to be done correctl you need to entire list 

gse <- gseGO(geneList=glist, 
             ont ="ALL", 
             keyType = "SYMBOL", 
             nPerm = 10000, 
             minGSSize = 3, 
             maxGSSize = 800, 
             pvalueCutoff = 0.05, 
             verbose = TRUE, 
             OrgDb = org.Hs.eg.db, 
             pAdjustMethod = "none")


gsea_dot = dotplot(gse, showCategory=10, split=".sign") + facet_grid(.~.sign)
gsea_ridge = ridgeplot(gse) + labs(x = "enrichment distribution")


# over representation 
# demo only

go_enrich_bp <- enrichGO(gene = as.character ( results[ results$class == "up", ]$gene ),
                      universe = as.character ( results$gene),
                      OrgDb = org.Hs.eg.db, 
                      keyType = 'SYMBOL',
                      readable = F,
                      ont = "BP",
                      pvalueCutoff = 0.05, 
                      qvalueCutoff = 0.10
                      )

go_1 = barplot(go_enrich_bp, 
        drop = TRUE, 
        showCategory = 10, 
        title = "GO Biological Pathways",
        font.size = 8)

go_2 =dotplot(go_enrich_bp)

```


## demo only 
* please refer to lecture 
* this is only to show show case 2 basica pathway analysis and the analysis is not complete or be taken seriously since we only used a subset of genes and DEG was not completed with the proper dataset. 

### GSEA summary1 

```{r fig=TRUE,fig.width=15, fig.height=8}
gsea_dot
```

### GSEA summary2
```{r fig=TRUE,fig.width=15, fig.height=8}
gsea_ridge
```

### GO 1  
```{r fig=TRUE,fig.width=5, fig.height=5}
go_1
```

### GO 2
```{r fig=TRUE,fig.width=12, fig.height=5}
go_2
```

  
```{r, fig.width=14, fig.height=17 }  
samplesnice = gsub ( "-",".", samples) # common issue when converting from dataframe to data matrix. 

expressionc[ row.names (expressionc ) %in% c(  "TP53" , "KRAS"), samplesnice ]

do.these = head ( names ( glist ) )
plot.out = list()

for ( gene in do.these  ){

g1 = melt (  expression[gene,  ]  )
colnames ( g1 )= c("pid","value")
g1$pid = gsub ( "\\.","-", g1$pid)

g1$HER2p = ifelse ( g1$pid %in% her2p, "HER2P", "NO.HER2")
g1$value = log2 ( g1$value + 1 )

mann_whit = wilcox.test(g1[g1$HER2p == "NO.HER2", ]$value,
                        g1[g1$HER2p != "NO.HER2", ]$value
                        ,paired=FALSE) 

p = round ( mann_whit$p.value, 2 )  
bon = p * length ( do.these )

gthis = ggplot(g1, aes(y=value, x=HER2p, fill=HER2p)) +
        geom_violin()+ 
        geom_jitter(shape=19, position=position_jitter(0.07), size=3 ) +
        theme_bw() +
        ylab("log2 (tpm + 1 ) ") +
        xlab("") +
        theme(legend.position="none", legend.title=element_blank(), legend.key = element_blank(),
              
              axis.text.y = element_text(size=12),
              axis.text.x = element_text(angle = 90, size=11.5),
              axis.title.x = element_text(size=22),
              
              axis.title.y     = element_text(size=22), 
              legend.text      =element_text(size=12)
        ) + stat_summary(fun.y = mean, fun.ymin = mean, fun.ymax = mean,
                         geom = "crossbar", width = .5)  + ggtitle ( gene ) +
    scale_fill_manual(values = c("#a6cee3","#b2df8a", "#fdbf6f") ) +
    ggtitle ( paste ( gene, "p.value", p, "corr", bon ))
plot.out[[gene]] = gthis
#plot ( gthis )
}

do.call("grid.arrange", c(plot.out, ncol=2))



```

# Now lets see if there are genes that correlate well with the HER2 gene 

```{r fig=TRUE,fig.width=14, fig.height=17}

exp = data.frame ( t ( expressionc ))
exp =exp [ , unique ( c( "ERBB2", names ( exp)))]
temp = cor(exp, exp$ERBB2, method="spearman"  ) 

her2.c = as.numeric(temp )
names ( her2.c) = row.names ( temp )
her2.c = sort ( abs ( her2.c ), decreasing = T )
her2.c = round ( her2.c, 3)


do.these = head ( names ( her2.c ), 7)
do.these = do.these[-1]
plot.out = list()

for ( gene in do.these  ){



gthis = ggplot(exp, aes_string (x="ERBB2", y=gene)) +
  geom_point(position = position_jitter(width = 0.5, height = 0.5)) + 
  geom_smooth(method=lm
              , se=FALSE
              , fullrange=TRUE
              )+
  theme_classic() +  stat_cor(method = "spearman") + ggtitle(gene)

plot.out[[gene]] = gthis
#plot ( gthis )
}

do.call("grid.arrange", c(plot.out, ncol=2))




```


# study sample id and see if there are any alteration in expression

```{r}

scaled.dat <- data.frame ( t ( scale(exp) )) 
myid2 = gsub ( "-", ".", myid )
my.dat = scaled.dat[, myid2, drop=T] 


names ( my.dat)= row.names (scaled.dat )
my.dat = sort ( abs ( my.dat ), decreasing = T )
my.dat = round ( my.dat, 3)

gene = names (  my.dat[1] )
g1 = melt (  as.matrix ( expressionc[ gene, ]  ) )
colnames ( g1 )= c("gene","pid","value")

g1 = g1[ order ( g1$value), ]
g1$pid = factor ( g1$pid, levels=g1$pid)

ggplot(g1 , aes(x=pid, y=value, color=  value  )) +
  geom_bar(stat="identity") +
 scale_fill_brewer(palette = "Set1")    +
  xlab("") + ylab("") +
  theme(axis.title.x=element_blank(),
        axis.text.x=element_blank(),
        axis.ticks.x=element_blank()

  ) + theme_void()  + 
  geom_text_repel(
        data = g1[g1$pid == myid2,  ], 
        aes(
          label = pid
        )
        , 
        fontface="bold", 
        color="black",
        size = 5, 
        nudge_x = 20.15,
    box.padding = 1.5,
    nudge_y = -.5,
  
      ) + ggtitle ( paste ( "Expression of ", gene, "relative to all BRCA samples" )  )




```

# top expressing genes for your TCGA ID

```{r,  fig.width= 7 , fig.height=10  }
# list of "overexpressed" genes for id 
my.dat[my.dat> 1.96 ]

gene = names (  my.dat[my.dat> 1.96 ] )[1:12]
g1 = melt (  as.matrix ( expressionc[ gene, ]  ) )
colnames ( g1 )= c("gene","pid","value")
g1$pid = as.character ( g1$pid)
g1$group = ifelse ( g1$pid == myid2, myid2, "base")

ccc = setNames( c("red", "grey"), c(myid2, "base"))
ggplot(g1, aes(y=value, x=gene , colour=group )) +
        geom_violin()+ 
        geom_jitter(shape=19, position=position_jitter(0.07), size=3 ) +
        theme_bw() +
        ylab("log2 (tpm + 1 ) ") +
        xlab("") +
        theme(legend.position="bottom", legend.title=element_blank(), legend.key = element_blank(),
              
              axis.text.y = element_text(size=12),
              axis.text.x = element_text(angle = 90, size=11.5),
              axis.title.x = element_text(size=22),
              
              axis.title.y     = element_text(size=22), 
              legend.text      =element_text(size=12)
        )  + ggtitle ( paste ( "Top expression gene", myid2 ) ) + scale_colour_manual(values=ccc) + 
  facet_wrap(gene ~., scales = "free" ) 





```





# HW at this point please use your own TCGA ID, study it using what you learn above.  Assess your sample, study what genes are most common and which if any are targetable. Study what mutations it contains and what type. 
## Bonus, do a survival analysis looking for certain mutations or expression specific to you sample 


```{r}
# here is example where I chose, myid = "TCGA-C8-A131-01"

 
actionid = drug[ drug$match %in% unique ( mutations[mutations$case_id == myid, ]  $drug  ), ]

actionid = actionid %>% dplyr::group_by(variant) %>%
        dplyr::summarise(
            gene = paste(unique ( gene ), collapse = "," ) ,
            disease = paste(unique ( disease ), collapse = "," ),
            clinical_significance =  paste(unique ( clinical_significance ), collapse = "," ),
            drugs = paste(unique ( drugs ), collapse = "," ) 
            
        ) %>%
        data.frame()

k2( actionid, myid)
 
```