Pulling all changes from private repo up to commit 2939b75

docs/final_report/final_report_refs.bib

@@ -88,6 +88,16 @@ @article{doi:10.1098/rsta.2011.0550
 }
 
 
+@book{rasmussen2006gaussian,
+  title = {Gaussian Processes for Machine Learning},
+  author = {Rasmussen, Carl Edward and Williams, Christopher K. I.},
+  year = {2006},
+  publisher = {The MIT Press},
+  address = {Cambridge, MA},
+  isbn = {026218253X}
+}
+
+
 
 @phdthesis{kernels,
 author = {Duvenaud, David},
@@ -160,23 +170,24 @@ @article{Hoyer-2017
  year = {2017}
 }
 
+@article{hourdin2020lmdz6a,
+  title={LMDZ6A: The atmospheric component of the IPSL climate model with improved and better tuned physics},
+  author={Hourdin, Fr{\'e}d{\'e}ric and Rio, Catherine and Grandpeix, Jean-Yves and Madeleine, Jean-Baptiste and Cheruy, Fr{\'e}d{\'e}rique and Rochetin, Nicolas and Jam, Arnaud and Musat, Ionela and Idelkadi, Abderrahmane and Fairhead, Laurent and others},
+  journal={Journal of Advances in Modeling Earth Systems},
+  volume={12},
+  number={7},
+  pages={e2019MS001892},
+  year={2020},
+  publisher={Wiley Online Library}
+}
+
 @article{mulvaney2004past,
   title={How are past temperatures determined from an ice core?},
   author={Mulvaney, Robert},
   journal={Scientific American},
   year={2004}
 }
 
-@incollection{NEURIPS2019_9015,
-title = {PyTorch: An Imperative Style, High-Performance Deep Learning Library},
-author = {Paszke, Adam and Gross, Sam and Massa, Francisco and Lerer, Adam and Bradbury, James and Chanan, Gregory and Killeen, Trevor and Lin, Zeming and Gimelshein, Natalia and Antiga, Luca and Desmaison, Alban and Kopf, Andreas and Yang, Edward and DeVito, Zachary and Raison, Martin and Tejani, Alykhan and Chilamkurthy, Sasank and Steiner, Benoit and Fang, Lu and Bai, Junjie and Chintala, Soumith},
-booktitle = {Advances in Neural Information Processing Systems 32},
-pages = {8024--8035},
-year = {2019},
-publisher = {Curran Associates, Inc.},
-url = {http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf}
-}
-
 @article{oshea-2015,
   author       = {Keiron O'Shea and Ryan Nash},
   title        = {An Introduction to Convolutional Neural Networks},
@@ -267,4 +278,29 @@ @article{yoshimura2008historical
   number={D19},
   year={2008},
   publisher={Wiley Online Library}
+}
+
+@incollection{NEURIPS2019_9015,
+title = {PyTorch: An Imperative Style, High-Performance Deep Learning Library},
+author = {Paszke, Adam and Gross, Sam and Massa, Francisco and Lerer, Adam and Bradbury, James and Chanan, Gregory and Killeen, Trevor and Lin, Zeming and Gimelshein, Natalia and Antiga, Luca and Desmaison, Alban and Kopf, Andreas and Yang, Edward and DeVito, Zachary and Raison, Martin and Tejani, Alykhan and Chilamkurthy, Sasank and Steiner, Benoit and Fang, Lu and Bai, Junjie and Chintala, Soumith},
+booktitle = {Advances in Neural Information Processing Systems 32},
+pages = {8024--8035},
+year = {2019},
+publisher = {Curran Associates, Inc.},
+url = {http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf}
+}
+
+@incollection{HOLTON2013453,
+title = {Chapter 13 - Numerical Modeling and Prediction},
+editor = {James R. Holton and Gregory J. Hakim},
+booktitle = {An Introduction to Dynamic Meteorology (Fifth Edition)},
+publisher = {Academic Press},
+edition = {Fifth Edition},
+address = {Boston},
+pages = {453-490},
+year = {2013},
+isbn = {978-0-12-384866-6},
+doi = {https://doi.org/10.1016/B978-0-12-384866-6.00013-1},
+url = {https://www.sciencedirect.com/science/article/pii/B9780123848666000131},
+author = {James R. Holton and Gregory J. Hakim}
 }

docs/final_report/title.sty

@@ -7,10 +7,10 @@
 \vspace*{5\baselineskip}
 \LARGE{\textbf{Data Science for Polar Ice Core Climate Reconstructions}}\\
 \vspace*{0\baselineskip}
-\Large{Capstone Project Final Report Draft}\\
+\Large{Capstone Project Final Report}\\
 
 \vspace*{2\baselineskip}
-\normalsize{{June 21, 2023}}\\
+\normalsize{{June 28, 2023}}\\
 
 \vspace*{2\baselineskip}
 \normalsize{\textbf{Authors}}\\

docs/guides/sockeye.md

@@ -9,10 +9,7 @@ Date: June 2023
 > 1. Introduction
 > 2. Logging in
 > 3. Navigating and setting up the directories
-> 4. Creating environments 
-> 5. Writing the job script
-> 6. Submitting and monitoring and tracking 
-> 7. Moving outputs to project space, and local space
+
 
 ## 1. Introduction
 
@@ -21,31 +18,13 @@ This document serves as a comprehensive guide to using UBC ARC's Sockeye high co
 For a more simplified version of using Sockeye to run Gaussian Process models, see the following notebook: []. The Sockeye tutorial in this notebook assumes that you are part of the `st-aorsi-1` allocation and have access to the group's directories. 
 
 **Overview of Workflow:**
-- Apply to obtain resources from UBC ARC's Sockeye cloud computing platform. Professor Anais Orsi did this for us. Then, obtain permission to connect to Sockeye with your CWL. Professor Anais Orsi will have to contact Sockeye and give them your cwl ... DOUBLE CHECK THIS ...
+- Apply to obtain resources from UBC ARC's Sockeye cloud computing platform. Professor Anais Orsi did this for us. Then, obtain permission to connect to Sockeye with your CWL. (Professor Anais Orsi will have to contact Sockeye and give them your cwl). 
 - Then, do the first-time set-up stuff where you install Cisco AnyConnect and enhanced CWL (two-factor authentication).
 - Next, try ssh-ing into Sockeye.
 - Make a plan of the GP jobs you would like to submit. It would be helpful to create a spreadsheet.
 - Next, make sure all the necessary scripts and data are set up in the directories. You can use the already set-up st-aorsi-1 group or set-up your own if you have a new allocation.
 - Create an environment, or use the one already set-up for st-aorsi-1. 
 
-**Terms and Definitions:**
-
-- job
-- allocation code
-- ssh
-- home 
-- scratch
-- project 
-- vi
-- cwl
-- vpn
-- enhanced cwl
-- terminal
-- purging scratch space
-
-**More resources on further information:** 
-
-- ... 
 
 > SLIPP's allocation code: `st-aorsi-1`
 
@@ -70,7 +49,7 @@ $ ssh [email protected]
 
 ---
 
-## 3. Navigating and setting up the directories 
+## 3. Navigating the directories 
 
 #### **General overview** 
 
@@ -94,71 +73,4 @@ There are three main directories:
 > **Access:** Shared with everyone in your allocation/team (readable/writable)
 > **Storage:** 5 TB + limit of 1,000,000 files
 
-#### **Directory set-up \[new allocation\]**
-
-Follow the steps below to set up the allocation space with the files required to run jobs for this project: 
-
-... 
-
-Each team member who plans on submitting jobs on Sockeye should 
-
-
-#### **`st-aorsi-1` directories \[st-aorsi-1 allocation\]**
-
-The `st-aorsi-1` Sockeye space is already set-up, below shows the breakdown of the directories: 
-
-**Project:** `/arc/project/st-aorsi-1/`
-
-- **`/arc/project/st-aorsi-1/data/`**
-    - Contains the full IsoGSM raw dataset
-        - `IsoGSM/Total.IsoGSM.ERA5.monmean.nc`
-    - Contains data required for preprocessing (land-sea mask, and orography across Antarctica)
-        - `IsoGSM_land_sea_mask.nc`
-        - `IsoGSM_orogrd.nc`
-    - Contains the full three preprocessed data files
-        - `preprocessed/preprocessed_test_ds.nc`
-        - `preprocessed/preprocessed_train_ds.nc`
-        - `preprocessed/preprocessed_valid_ds.nc`
-- **`/arc/project/st-aorsi-1/outputs/`**
-    - Used to store finalized outputs you want to keep
-    - ...
-- `/arc/project/st-aorsi-1/shared/`
-    - Stores environment files able to be used by everyone in the allocation group
-    - ... 
-
-**Scratch:** `/scratch/st-aorsi-1/` 
-
-From May - June 2023, Shirley Zhang (cwl: `wenlansz`) submitted jobs through her directory in the scratch space. As of June 2023, the directory is set up as follows (may be different if Sockeye purges the scratch space): 
-
-- **`/scratch/st-aorsi-1/wenlansz/job-scripts/`**
-    - ... 
-
-- **`/scratch/st-aorsi-1/wenlansz/outputs/`**
-    - ... 
-
-- **`/scratch/st-aorsi-1/wenlansz/e-files/`**
-    - ... 
-
-- **`/scratch/st-aorsi-1/wenlansz/o-files/`**
-    - ...
 
-
-
-## 4. Creating environments
-
-
-... 
-
-
-## 5. Writing the job script
-
-...
-
-
-## 6. Submitting and monitoring and tracking 
-
-...
-
-## 7. Moving outputs to project space, and local space
-
-...