Setup REAL TIME analysis

Installation (only for OSX)

Clone the popcycle git repository into your computer. This will create a popcycle-master folder in the root of the computer. Open the terminal and type

$ cd ~
$ git clone https://github.com/uwescience/popcycle.git popcycle-master

Install popcycle package and its dependencies, such as RSQLite and splancs packages if they are not already installed

$ cd ~/popcyle-master/
$ Rscript setup.R

Then copy the computer folder in popcycle-master to the root to install the files requested by the different cron jobs .

$ cp ~/popcyle-master/computer/* ~

On the SeaFlow computer (not Mac mini), share that SeaFlow data folder.
- Open the Command Prompt window, click Start, point to All Programs, click Accessories, right-click Command Prompt, and then click Run as administrator.
- If the User Account Control dialog box appears, confirm that the action it displays is what you want, and then click Yes.
- Type:
```
$ net share evt=C:Users\seaflow\Desktop\seaflow
```
Then back to Mac computer, mount a disk to have access to the SeaFlow data folder.

$ mkdir /Volumes/SeaFlow
$ mount -t smbfs //admin@SEAFLOW/SeaFlow /Volumes/SeaFlow

Make a folder for a local copy of Seaflow data

$ mkdir -p ~/SeaFlow/datafiles/evt

Initialization

The first step is to start R and load the popcycle library.
```
library(popcycle)
```

Since we're using the default configuration for realtime analysis there's no need to set the project location, cruise name, evt location, etc.

The second step is to set the parameters for the filtration method, i.e., the width(to adjust the alignment of the instrument) and the notch (to adjust the focus of the instrument) . The notch represents the the ratio D/fsc_small and depends on how the PMTs of D1/D2 and fsc_small were set up, the width represents the acceptable difference between D1 and D2 for a particle to be considered 'aligned', it is usually set between 0.1 and 0.5. For this example, we are going to choose the notch using the latest evt file collected by the instrument (but you choose any evt file that you want, of course). The width is set to 0.2. Open an R session and type:
```
# SELECT AN EVT FILE

evt.name <- get.latest.evt.with.day() # to get the last evt file of the list
# OR
# evt.list <- get.evt.list() # to get the entire list of evt files
# evt.name <- evt.list[10] # then select the evt file (e.g., the 10th evt file in the list)

# LOAD THE EVT FILE
evt <- readSeaflow(evt.name)

# SET the WIDTH and NOTCH parameter
width <- 0.2 # usually between 0.1 and 0.5
notch <- 1 # usually between 0.5 and 1
plot.filter.cytogram(evt, notch=notch, width=width) # to plot the filtration steps
```
NOTE that if you have trouble finding the optimal NOTCH, you can use the function find.filter.notch()
```
width <- 0.2
notch <- find.filter.notch(evt, notch=seq(0.5, 1.5, by=0.1),width=width, do.plot=TRUE)
plot.filter.cytogram(evt, notch=notch, width=width)
```
Once you are satisfied with the filter parameters, you can filter evt to get opp
```
setFilterParams(width, notch) #To save the filter parameters so filter parameters will be apply to all new evt files
opp <- filter.notch(evt, notch=notch, width=width)
upload.opp(opp.to.db.opp(opp, cruise.id, basename(evt.name)))  # to save to DB
```
The setFilterParams(width, notch) function saves the parameters in ~/popcycle/params/filter/filter.csv. Note that every changes in the filter parameters are automatically saved in the logs (~popcycle/logs/filter/filter.csv).
Third step is to set the gating for the different populations. WARNINGS: The order in which you gate the different populations is very important, choose it wisely. The gating has to be performed over optimally positioned particles only, not over an evt file. In this example, you are going to first gate the beads (this is always the first population to be gated.). Then we will gate Synechococcus population (this population needs to be gated before you gate Prochlorococcus or picoeukaryote), and finally Prochlorococcus and picoeukaryote population. After drawing your gate on the plot, right-click to finalize. In the R session, type:
```
# Using the opp data frame from previous the filtering step
setGateParams(opp, popname='beads', para.x='fsc_small', para.y='pe')
setGateParams(opp, popname='synecho', para.x='fsc_small', para.y='pe')
setGateParams(opp, popname='prochloro', para.x='fsc_small', para.y='chl_small')
setGateParams(opp, popname='picoeuk', para.x='fsc_small', para.y='chl_small')
```
Similar to the setFilterParams function, setGateParams saves the gating parameters and order in which the gating was performed in ~/popcycle/params/params.RData, parameters for each population are also separately saved as a .csv file. Note that every changes in the gating parameters are automatically saved in the logs (~popcycle/logs/params/).

Note: If you want to change the order of the gating, delete a population, or simply restart over, use the function resetGateParams()
To cluster the different population according to your manual gating, type:
```
vct <- classify.opp(opp, ManualGating)
```

To plot the cytogram with clustered populations, use the following function:

opp$pop <- vct
par(mfrow=c(1,2))
plot.vct.cytogram(opp, para.x='fsc_small', para.y='chl_small')
plot.vct.cytogram(opp, para.x='fsc_small', para.y='pe')

Play

Now that the filter and gating parameters are set, you can tell the computer to apply these parameters to every new files collected by the instrument and back up the data to the external hard disk named seaflow_backup. To do that, you will need to set a 'cron' job. Open the terminal and type

$ crontab -e

This will open the vi text editor. Type i then copy and paste this line:

*/3 * * * * ~/RT_analysis.sh >~/RT_analysis.log 2>&1

Press Esc, type :wq, then press return to save and quit the editor.

Visualization

To display the plots, type in the terminal:

$ open ~/*html

Reset the database for a new cruise

Before you start, make sure that there is no evt files left from previous cruises in the SeaFlow folder located 1) on the local machine (~/SeaFlow/datafiles/evt) AND 2) on the SeaFlow computer (//admin@SEAFLOW/SeaFlow).
Start an R session and type

library(popcycle)
reset.db()

Open the terminal and type

$ crontab -e

This will open the vi text editor. Type i then copy and paste this line:

*/3 * * * * ~/RT_analysis.sh >~/RT_analysis.log 2>&1

Press Esc, type :wq, then press return to save and quit the editor.

That's it. The script will automatically look for new data on the SeaFlow computer, copy data form SeaFlow computer to local machine, analyze new files and update plots.

Provide feedback

Saved searches