Skip to content

Commit 2aae923

Browse files
authored
Add CMIP6 simpler example (#24)
1 parent 0cab2a9 commit 2aae923

File tree

8 files changed

+2450
-5
lines changed

8 files changed

+2450
-5
lines changed

examples/cmip6/README.md

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
Coupled Model Intercomparison Project Phase 6 (CMIP6) Examples
2+
==============================================
3+
4+
5+
These example use AWS CloudFormation to create an Amazon SageMaker Jupyter Notebook instance with the appropriate Python libraries installed (zarr, xarray, dask, intake) for working with the CMIP6 datasets.
6+
7+
[![cloudformation-launch-stack](cloudformation/cloudformation-launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/new?stackName=CMIP6&templateURL=https://s3.amazonaws.com/docs.opendata.aws/cloudformation/cmip6.yaml)
8+
9+
The Jupyter notebook `cmip6_airtemperature.ipynb` shows an example of how to query the intake-esm catalog for CMIP6 and select the appropriate ensemble members for a specific institution. Then it plots the mean difference in temperature at the end of one of the scenarios from the beginning showing how the temperature is changing over the 100 year period. The notebook also shows how to extract the time series of temperature at a location for all of the scenarios and plot that as well.
Loading
+89
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
AWSTemplateFormatVersion: '2010-09-09'
2+
Description: This runs Sagemaker for a Jupyter notebook to work with the NOAA ISD open dataset.
3+
Resources:
4+
5+
ExecutionRole:
6+
Type: "AWS::IAM::Role"
7+
Properties:
8+
AssumeRolePolicyDocument:
9+
Version: "2012-10-17"
10+
Statement:
11+
-
12+
Effect: "Allow"
13+
Principal:
14+
Service:
15+
- "sagemaker.amazonaws.com"
16+
Action:
17+
- "sts:AssumeRole"
18+
Path: "/"
19+
ManagedPolicyArns:
20+
- "arn:aws:iam::aws:policy/AmazonS3FullAccess"
21+
22+
NotebookLifecycleConfig:
23+
Type: AWS::SageMaker::NotebookInstanceLifecycleConfig
24+
Properties:
25+
OnCreate:
26+
- Content:
27+
Fn::Base64: |
28+
#!/bin/sh
29+
30+
set -e
31+
cat > /home/ec2-user/setup.sh << EOF
32+
#!/bin/bash
33+
sleep 10
34+
echo "Creating daskpy3 conda environment"
35+
conda create --name daskpy3 python="3.7.9" -y
36+
source activate daskpy3
37+
conda install -c conda basemap proj4 nodejs ipywidgets -y
38+
pip install zarr==2.7.1 rechunker==0.3.2 ipykernel boto3 dask==2021.1.1 distributed==2021.1.1 tornado==6.1 cloudpickle==1.6.0 msgpack==1.0.2 blosc==1.10.2 numpy==1.19.5 pandas==1.2.1 lz4==3.1.3 netcdf4==1.5.5.1 xarray==0.17.0 bokeh==2.2.3 s3fs==0.4.2 fsspec==2021.4.0 h5netcdf==0.8.1 h5py==3.1.0 intake==0.6.2 intake-esm==2021.1.15
39+
python -m ipykernel install --user --name daskpy3 --display-name "conda_daskpy3"
40+
echo "Updating Jupyter config options"
41+
sed -i.bak 's/^#c.NotebookApp.iopub_data_rate_limit.*$/c.NotebookApp.iopub_data_rate_limit = 1.0e10/' ~/.jupyter/jupyter_notebook_config.py
42+
sed -i.bak 's/^#c.NotebookApp.iopub_msg_rate_limit.*$/c.NotebookApp.iopub_msg_rate_limit = 1.0e10/' ~/.jupyter/jupyter_notebook_config.py
43+
source /home/ec2-user/anaconda3/bin/deactivate
44+
source activate JupyterSystemEnv
45+
jupyter labextension install @jupyter-widgets/[email protected]
46+
source /home/ec2-user/anaconda3/bin/deactivate
47+
echo "Finished!"
48+
EOF
49+
50+
chown ec2-user:ec2-user /home/ec2-user/setup.sh
51+
chmod +x /home/ec2-user/setup.sh
52+
53+
sudo -u ec2-user -i nohup /home/ec2-user/setup.sh >/home/ec2-user/output.log 2>&1 &
54+
55+
56+
NotebookRepository:
57+
Type: AWS::SageMaker::CodeRepository
58+
Properties:
59+
GitConfig:
60+
RepositoryUrl: "https://github.com/awslabs/amazon-asdi"
61+
Branch: "main"
62+
63+
NotebookInstance:
64+
Type: "AWS::SageMaker::NotebookInstance"
65+
Properties:
66+
NotebookInstanceName: !Join
67+
- ''
68+
- - CMIP6-
69+
- !Select
70+
- 6
71+
- !Split
72+
- '-'
73+
- !Ref 'AWS::StackId'
74+
InstanceType: "ml.c5.xlarge"
75+
DefaultCodeRepository: !GetAtt NotebookRepository.CodeRepositoryName
76+
RoleArn: !GetAtt ExecutionRole.Arn
77+
LifecycleConfigName: !GetAtt NotebookLifecycleConfig.NotebookInstanceLifecycleConfigName
78+
79+
Outputs:
80+
JupyterNotebook:
81+
Value: !Join
82+
- ''
83+
- - https://
84+
- !Ref 'AWS::Region'
85+
- .console.aws.amazon.com/sagemaker/home?region=
86+
- !Ref 'AWS::Region'
87+
- '#/notebook-instances/openNotebook/'
88+
- !GetAtt 'NotebookInstance.NotebookInstanceName'
89+
- '?view=lab'

0 commit comments

Comments
 (0)