@@ -89,12 +89,12 @@ If you are new to MED and are wondering [*"What is Model Evaluation and Diagnost
### Tools in development
-We are currently setting up a range of tools that will help you to better evaluate and diagnose climate models:
+For the evaluation and diagnosis of ACCESS climate models, the following tools are currently being setup:
* Data format processing tools like APP4
* An Evaluation Recipe Gallery with searching functionality
-While we are working on these, we have collected a number of links to existing tools in our [community tab](../community_resources/index.md) (note that we are not currating them).
+While this work is in progress, you can refer to a collection of links to existing tools (not curated by ACCESS-NRI) in the [community tab](../community_resources/index.md).
diff --git a/docs/model_evaluation/model_evaluation_getting_started/index.md b/docs/model_evaluation/model_evaluation_getting_started/index.md
index 20ff3e60b..9be25276e 100644
--- a/docs/model_evaluation/model_evaluation_getting_started/index.md
+++ b/docs/model_evaluation/model_evaluation_getting_started/index.md
@@ -1,16 +1,25 @@
# Getting Started with Model Evaluation at NCI
-Welcome to Model Evaluation and Diagnostics!
+
-## What is MED about?
+## What is MED?
-**Evaluation** involves scrutinizing the model through Model/Observation confrontations, checking its performance against real-world observations. It also includes experiment comparisons, testing the model under different scenarios, and inter-model comparisons like the Coupled Model Intercomparison Project (CMIP), assessing how the ACCESS-NRI model fares when compared with other climate models.
+Model Evaluation and Diagnostics (MED) of ACCESS models includes:
-**Diagnostics** involves constant monitoring of model runs to detect any anomalies or inconsistencies and a thorough analysis of outputs to verify the model's accuracy over time.
+
-## How can you get started with MED?
+## How to get started with MED?
-Here, we provide you the important information to give you access to the large data that we curate at NCI's storage and show you how you can use it to figure out how fit for purpose specific models are, in particular when you compare them to osbervational data:
+Here you can find information on how to access curated data stored at NCI and how to use it to evaluate specific models, such as comparing model output with observational data.
diff --git a/docs/model_evaluation/model_evaluation_getting_started/model_evaluation_getting_started.md b/docs/model_evaluation/model_evaluation_getting_started/model_evaluation_getting_started.md
index b6c48b535..a2db037cd 100644
--- a/docs/model_evaluation/model_evaluation_getting_started/model_evaluation_getting_started.md
+++ b/docs/model_evaluation/model_evaluation_getting_started/model_evaluation_getting_started.md
@@ -1,30 +1,37 @@
-# `conda` Environment for Model Evaluation on NIC's Gadi
+# `conda` Environment for Model Evaluation on Gadi
-At this stage of *Getting Started*, we assume that you already have access to NCI's Gadi via `ssh`. If this is not the case, please go to our instructions on [how to get access to NCI's Gadi](../../getting_started/index.md).
+If you do not yet have `ssh` access to Gadi, refer to instructions on how to login to Gadi.
-The instructions below explain how to load our curated `python` environment, with packages and scripts which are supported by ACCESS-NRI. Once these instructions have been followed you will be able to use all pacakges and scripts when running directly on Gadi via `ssh`, in `PBS` scripts, or in JupyterLab.
+The following instructions explain how to load the curated `python` environment on NCI, which includes packages and scripts supported by ACCESS-NRI. Once loaded, these can be run directly on
Gadi via `ssh`, `PBS` scripts, or in `JupyterLab`.
-???+ warning "ACCESS-NRI provides code and support, but not computing resources"
- As mentioned in the [Getting Started pages](../../../getting_started), you do not automatically have access to all of Gadi's storage at `/g/data/`, but need to be part of a `$PROJECT` to see files at `/g/data/$PROJECT`. For model evaluation and diagnostics, you need to be part of projects `xp65` and `hh5` for code access and a project with compute resources.
+???+ warning "ACCESS-NRI can provide code and support, but not computing resources"
+ You do not automatically have access to all `/g/data/` storage on
Gadi. You need to
join an NCI project to view files on `/g/data/$PROJECT`.
+
+ For model evaluation and diagnostics, you need to join projects `xp65` and `hh5` for code access and a `$PROJECT` with sufficient compute resources.
-## What is part of the `access-med` enrivonment?
+## What is the `access-med` environment?
-The complete list of dependencies can be found in
this `environment.yml` file of our
GitHub repository and includes `intake`, `esmvaltool`, and `ilamb`:
+The complete list of dependencies for the `access-med` environment can be found in the
environment.yml file of the
ACCESS-NRI MED GitHub repository. These include `intake`, `esmvaltool` and `ilamb`:
-## Running our `access-med` environment on Gadi
+## Running the `access-med` environment on Gadi
-To avoid running code on Gadi with incompatible packages we provide you with a conda environment called access-med.
-In order to change to this curated environment, please run the following commands everytime after you log into Gadi (and as part of a
PBS scripts):
+To avoid running code on
Gadi with incompatible packages, a conda environment called `access-med` is provided.
+
+To change to this curated environment, run the following commands after logging into
Gadi and
edit your `PBS` script accordingly:
```
module use /g/data/xp65/public/modules
module load conda/access-med
```
-This will load the latest version of `access-med` (in this tutorial `0.3`). To check which conda version you are using, you can run `which python`:
-
+This will load the latest version of `access-med`, e.g. version `access-med-0.3`.
+
+To check which `conda` version you are using, run the following command:
+```
+which python
+```
module use /g/data/xp65/public/modules
@@ -36,7 +43,7 @@ This will load the latest version of `access-med` (in this tutorial `0.3`). To c
/g/data/xp65/public/apps/med_conda_scripts/access-med-0.3.d/bin/python
-To test everything is working correctly, import the packages in `python3`:
+To test everything is working correctly, import the packages in `python3` as follows:
```python
import numpy as np
@@ -49,13 +56,7 @@ print(intake.__version__)
print(esmvaltool.__version__)
```
-If you are planning to run your code on Gadi with a Portable Batch System (`PBS`) job, you will need to add in the `module use` and `module load` commands to your PBS script as well.
-
-
-If you are not familiar with PBS jobs on NCI, you can find NCI's guide
here.
-
-
-You could for example create an `example_pbs.sh` file with the content:
+If you want to run your code on
Gadi using a Portable Batch System (`PBS`) job, add the `module use` and `module load` commands to your `PBS` script as shown in the `example_pbs.sh` `PBS` script below:
```
#!/bin/bash
@@ -74,44 +75,61 @@ module load conda/access-med
python3 your_code.py
```
-The content of `your_code.py` could be as simple as the import and version print from our example above. submit this job, you then only need to execute
+The content of `your_code.py` could simply comprise the `import` and `which version` lines from our above example.
+
+To submit this `PBS` job, execute the following command:
```
qsub example_pbs.sh
```
In brief: this PBS script will submit a job to Gadi with the job name (`#PBS -N`) *example_pbs* under compute project (`#PBS -P`) `iq82` with a
normal queue (`#PBS -q normalbw`), for 1 CPU (`#PBS -l ncpus=1`) with 2 GB RAM (`#PBS -l mem=2GB`), a walltime of 10 minutes (`#PBS -l walltime=00:10:00`) and data storage access to projects `xp65`. Note that for this example to work, you have to be
member of the NCI project `xp65` and `iq82`. Adjust the `#PBS -P` option to match your compute project. Upon starting the job, it will change into to the working directory that you submitted the job from (`#PBS -l wd`) and load the access-med conda environment.
-
-## Running our `access-med` environment on NCI's Interactive ARE (JupyterLab)
-
-NCI also supports an interactive coding environment called the Australian Research Environment (ARE). It's use is quite similar to submitting a computing job via `qsub -I`, but it comes with dedicated graphical user interfaces for jupyter notebooks. To use it, you need an NCI account and be part of a project that gives you computing resources (see our [getting started](../../getting_started/index.md)).
-
-You can then go to
https://are.nci.org.au to log in. In the "Featured Apps" section, click on "JupyterLab" and to a JupyterLab instance.
-
-
-If you are not familiar with ARE at NCI, you can find NCI's guide
here.
-
-
-Below we have provided example values, however you must change these values to match your project and use case:
+This will submit a job to
Gadi with the job name (`#PBS -N`) *example_pbs* under compute project (`#PBS -P`) *iq82* with a normalbw
normal queue (`#PBS -q`). The number of CPUs requested is 1 CPU (`#PBS -l ncpus=1`) with 2 GB RAM (`#PBS -l mem=2GB`) and a
walltime of 10 minutes (`#PBS -l walltime=00:10:00`). The
data storage (`#PBS -l storage=gdata/xp65`) is data storage access to project `xp65`.
+
+
+
Note: to run this example, you need to be a
member of an NCI project, in this case `xp65` and `iq82` projects.
+
+Adjust the `#PBS -P` option to match your compute project.
+
+When the job starts, it will change to the working directory from where you submitted the job (`#PBS -l wd`) and load the access-med `conda` environment.
+
+
+For more information on running `PBS` jobs on NCI, refer to
PBS Jobs.
+
+## Running the `access-med` environment on ARE
+
+NCI also supports an interactive coding environment called the Australian Research Environment (
ARE). Its use is similar to that for submitting a `PBS` job via `qsub -I`, but with an added bonus of a dedicated graphical user interface for `Jupyter` notebooks.
+
+
+To use
ARE, you must have an NCI account and be a member of a project with computing resources (see section on [getting started](../../getting_started/index.md)).
+
+Once you
login to ARE, click on
JupyterLab in the
Featured Apps section to launch a `JupyterLab` instance.
+
+Below are some example values that you should change to match your `$PROJECT` and use case:
- **Walltime (hours)** `1`
- **Queue** `normalbw`
- **Compute Size** `tiny`
-- **Project** `iq82` (This should match your project with computing resources)
-- **Storage** `gdata/xp65+gdata/hh5` (Select all which match your project's gdata storage)
+- **Project** `iq82` (This should match your `$PROJECT` with compute resources)
+- **Storage** `gdata/xp65+gdata/hh5` (Select all that match your project's `/g/data` storage)
- *Advanced Options ...* (click button to expand)
- **Module directories** `/g/data/xp65/public/modules`
- **Modules** `conda/are`
- *Launch* (click to submit)
-This will launch a JupyterLab session with a Session ID and display it in the list of interactive sessions (you can also find it under **My Interactive Sessions** in the top left of the ARE website).
-The session appears blue while it is loading, yellow or red in case of warnings or errors, and green when it is successfully running, as in the following example:
+This will launch a `JupyterLab` session with a
Session ID, which will appear in the list of interactive sessions. (You can also find it under
My Interactive Sessions at the top-left of the ARE window).
+
+The session appears blue while it is loading, yellow or red in case of warnings or errors, and green when it is successfully running:
-You can then **Open JupyterLab** via a button at the bottom of the session. This will bring you to a window with a directory structure to the left and a jupyter notebook to the right (see the example below). If you have loaded the modules from `hh5` or `xp65`, you should then be able to import python packes like `numpy`, `xarray` or `intake`, as shown in the screenshot below.
+You can then
Open JupyterLab by clicking on the button at the bottom of the session.
+
+This will open a window which contains a directory structure on the left and a `Jupyter` notebook on the right, as shown below.
+
+If you loaded the modules from `hh5` or `xp65`, you should be able to import python packages such as `numpy`, `xarray` or `intake`, as shown below:
diff --git a/docs/model_evaluation/model_evaluation_getting_started/model_variables/index.md b/docs/model_evaluation/model_evaluation_getting_started/model_variables/index.md
index 7aee013db..5fab894af 100644
--- a/docs/model_evaluation/model_evaluation_getting_started/model_variables/index.md
+++ b/docs/model_evaluation/model_evaluation_getting_started/model_variables/index.md
@@ -2,63 +2,77 @@
-For climate modelling, we need to store multidimensional scientific data (variables) such as temperature, humidity, pressure, wind speed, and direction.
+For climate modelling, we need to store multidimensional scientific data (variables) such as temperature, humidity, pressure, wind speed and direction.
-
+
-ACCESS-NRI strives for comparable model outputs for easily model evaluation.
+As comparable model outputs simplify Model evaluation, ACCESS-NRI supports Coupled Model Intercomparison Projects (CMIP) and the use of common data formats and variables.
-We therefore support projects like the Coupled Model Intercomparison Projects (CMIP) and the use of common data formats and variables.
+## Network Common Data Format (NetCDF)
-## 1 Network Common Data Format (NetCDF)
+Numerous organisations and scientific groups worldwide have adopted a file format called
NetCDF as a standard way to store multidimensional scientific data.
-Many organizations and scientific groups in different countries have adopted a data format called
NetCDF by the Unidata Community as a standard way to represent some forms of scientific data.
+
NetCDF, which has the file extension `*.nc`, is a self-describing, machine-independent data format of array-oriented scientific data.
-NetCDF (with file extensions `*.nc`) is a self-describing, machine-independent data format of array-oriented scientific data.
+
+- Self-describing
+
+ *.nc
files include not only the data itself, but also a header with metadata that describes the data layout.
-- **Self-describing** means that the `*.nc` files include not only the data itself, but also a *header* with `metadata` that describes the layout of the data.
-- **Machine-independent** means that the `*.nc` files can be accessed by computers with different ways of storing integers, characters, and floating-point numbers.
-- **Array-oriented** means that the `*.nc` data is typically spanning multiple dimensions with the same lengths (like latitude and longitude or time) and variables (like temperature or humidity) which are stored in arrays.
+ - Machine-independent
+
+ *.nc
files can be accessed by computers with different ways of storing integers, characters and floating-point numbers.
-
-
-
+ - Array-oriented
+
+ *.nc
data typically spans multiple dimensions with the same lengths (e.g., latitude, longitude and time) and variables, such as temperature or humidity, which are stored in arrays.
+
+
+
+
+
+
-### 1.1 NetCDF metadata
+### NetCDF metadata
-Metadata is typically described as *information about the data* and enables users of data from different sources to decide which quantities are comparable. This facilitates building applications with powerful extraction, regridding, and display capabilities.
+
Metadata, which is typically described as
information about the data, enables users of data from different sources to decide which quantities are comparable. This facilitates building applications with powerful extraction, regridding and display capabilities.
-To facilitate this process, there are conventions for CF (Climate and Forecast) metadata. These are designed to promote the processing and sharing of files created with NetCDF. The conventions define metadata that provide a definitive description of what the data in each variable represents, and the spatial and temporal properties of the data. Learn more about the CF metadata conventions
here.
+To facilitate this process, there are
conventions for Climate and Forecast metadata. These are designed to promote the processing and sharing of
NetCDF files. The conventions define metadata that provide a definitive description of what the data in each variable represents, and the spatial and temporal properties of the data.
-### 1.2 NetCDF data and variables
+### NetCDF data and variables
-The data in a netCDF file is stored in the form of arrays. For example, temperature varying over time at a location is stored as a one-dimensional array. Temperature over an area for a given time is stored as a two-dimensional array. Three-dimensional (3D) data, like temperature over an area varying with time, or four-dimensional (4D) data, like temperature over an area varying with time and altitude, is stored as a series of two-dimensional arrays.
+Data in a
netCDF file is stored in the form of arrays, where each
netCDF dimension has a name and a length.
+
+For example, temperature variation over time at a fixed location is stored as a one-dimensional array, whereas temperature over a region (i.e. varying location) at a fixed time is stored as a two-dimensional array. Thus, three-dimensional (3D) data would be temperature varying with time over a region, and four-dimensional (4D) data would be temperature varying with time over a region with varying altitude.
-## 2 Common Variables
+## Common variables
-Using common variables is key for a united climate modelling community. While we are trying to bring together modellers from different communities, we understand that the variables used for climate modelling may differ for historical reasons. Here, we are collating lists of different widely used variable formats.
+Variables used in climate modelling can differ in terms of naming conventions, units, etc. While this may be for historical reasons, the use of common variables is key not only for ease and compatibilty when working with the data, but also to unite the climate modelling community. Hence, there are collated lists of different widely used variable formats, such as:
-### 2.1 Variables of CMIP6 (Coupled Model Intercomparison Project Version 6)
-
-You can search the extensive list of Coupled Model Intercomparison Project version 6 on
this website.
-
-### 2.2 Variables of the ERA5 atmospheric reanalysis
-
-ERA5 is the fifth generation ECMWF atmospheric reanalysis of the global climate covering the period from January 1940 to present. ERA5 is produced by the Copernicus Climate Change Service (C3S) at ECMWF. ERA5 provides hourly estimates of a large number of atmospheric, land and oceanic climate variables.
+### CMIP6 variables
+You can search the extensive list of Coupled Model Intercomparison Project phase 6
(CMIP6) variables by either the MIP variable name or associated CF standard name.
+### ERA5 variables
+ERA5 is the fifth generation European Centre for Medium-Range Weather Forecasts (ECMWF) atmospheric reanalysis of the global climate, which spans a period from January 1940 to present. ERA5 provides hourly estimates of a significant number of atmospheric, land and oceanic climate variables.
+
+
A full list of ERA5 parameters is available on the
ECMWF database. It covers both the
ERA5 parameter listings as well as the
ERA5-LAND parameter listings.
-## 3 Loading NetCDF files
+## Loading NetCDF files
-Our Model Evaluation and Diagnostics tools are based around the reading and storing of files via the Python package `xarray`. You can find a quick overview
here and tutorials
here.
+Our Model Evaluation and Diagnostics tools are based on the reading and storing of files via the Python package `xarray`.
+
+For more information, refer to a
quick overview of xarray and
xarray tutorials.
-We provide `xarray` through our `conda` environment, so you can use it either directly as shown below, or through the dataset capabilities of our catalog [ACCESS-NRI Model Intake Catalog Tool](../../model_evaluation_model_catalogs/index.md.
+`xarray` is a python package avaliable through the `conda` environment on NCI.
+
+Hence, you can either use it directly (as shown below) or through the dataset capabilities of the [ACCESS-NRI Model Intake Catalog Tool](../../model_evaluation_model_catalogs/index.md).
```
import xarray as xr
@@ -67,5 +81,5 @@ dataset
```
-
+
diff --git a/docs/model_evaluation/model_evaluation_model_catalogs/index.md b/docs/model_evaluation/model_evaluation_model_catalogs/index.md
index 68226ba25..47fadd42d 100644
--- a/docs/model_evaluation/model_evaluation_model_catalogs/index.md
+++ b/docs/model_evaluation/model_evaluation_model_catalogs/index.md
@@ -1,6 +1,8 @@
-# ACCESS-NRI intake Model Catalog
+# ACCESS-NRI Intake catalog
-The ACCESS-NRI intake catalog aims to provide a way for Python users to discover and load data across a broad range of climate data products available on the Australian NCI supercomputer Gadi. For detailed information, tutorials and more, please go to the
+The ACCESS-NRI Intake catalog aims to provide a way for Python users to discover and load data across a broad range of climate data products available on
Gadi.
+
+For detailed information, tutorials and more, please go to:
@@ -10,13 +12,19 @@ The ACCESS-NRI intake catalog aims to provide a way for Python users to discover
-## What is the ACCESS-NRI intake Model Catalog?
+## What is the ACCESS-NRI Intake catalog?
+
+The ACCESS-NRI Intake catalog is essentially a table of climate data products available on Gadi.
+
+Each entry in the table corresponds to a different product, where the columns contain attributes associated with that product (e.g., available models, frequencies and data variables). Users can search on the attributes to find the products that might be useful to them. For example, a user may want to know which data products contain variables X, Y and Z at monthly frequency.
-The ACCESS-NRI catalog is essentially a table of climate data products that exist on Gadi. Each entry in the table corresponds to a different product, and the columns contain attributes associated with each product–things like the models, frequencies and variables available. Users can search on the attributes to find the products that might be useful to them. For example, a user might want to know which data products contain variables X, Y and Z at monthly frequency. The ACCESS-NRI catalog enables users to find products that satisfy their query and to subsequently load their data without having to know the location and structure of the underlying files.
+The ACCESS-NRI Intake catalog enables users to find products that satisfy their query and to subsequently load their data without having to know the location and structure of the underlying files.
-## Showcase: use intake to easily find, load and plot data
+## Showcase: use ACCESS-NRI Intake to find, load and plot data
-In this showcase, we'll demonstrate one of the simplest use-cases of the ACCESS-NRI intake catalog: a user wants to plot a timeseries of a variable from a specific data product. Here, the variable is a scalar ocean variable called "temp_global_ave" and the product is an [ACCESS-ESM1-5](../../models/configurations/access-esm) run called "HI_CN_05", which is an historical run using same configuration as CMIP6 ACCESS-ESM1-5 historical r1i1p1f1, but with phosphorus limitation disabled within CASA-CNP.
+A simple use case of the ACCESS-NRI Intake catalog is a user wants to plot a timeseries of a variable from a specific data product.
+
+For example, the user is interested in plotting a scalar ocean variable called temp_global_ave for an [ACCESS-ESM1-5(/models/configurations/access-esm)] run called `HI_CN_05` (data product), which is an historical run using the same configuration as CMIP6 ACCESS-ESM1.5 historical `r1i1p1f1`, but with phosphorus limitation disabled within CASA-CNP.
First we load the catalog using
diff --git a/docs/model_evaluation/model_evaluation_observational_catalogs.md b/docs/model_evaluation/model_evaluation_observational_catalogs.md
index b8d88823c..4a7660eea 100644
--- a/docs/model_evaluation/model_evaluation_observational_catalogs.md
+++ b/docs/model_evaluation/model_evaluation_observational_catalogs.md
@@ -1,8 +1,8 @@
# Observational Data Catalogue
-The National Computational Infrastructure (NCI) is hosting numerous datasets for climate research and furthermore manages and optimises currated data collections.
+NCI not only hosts numerous datasets for climate research, it also manages and optimises curated data collections.
-You can browse and search the available data collections on the NCI Data Collections website:
+Here, you can browse and search the available NCI data collections:
-Examples of the NCI Data Collections include:
+Some examples of NCI data collections include:
+
- Data of the Earth Systems Grid Federation hosted at the
NCI ESGF Node
- Data of the
fifth generation of ECMWF atmospheric reanalyses (ERA5) with more information on the
NCI ERA5 Community Page
- Data of the European Space Agency’s multi-petabyte Sentinel satellite data via the
Sentinel Australasia Regional Access (SARA)
-NCI is also providing a user guide for finding, accessing, and citing data
here.
+NCI also has a
user guide for finding, accessing and citing data.
-Below you can see a screenshot of an example catalogue (NCI project `ct11`) of ACCESS-NRI Replicated Datasets for Climate Model Evaluation (you can find this particular entry by searching `ACCESS-NRI`):
+For example, the catalogue of ACCESS-NRI Replicated Datasets for Climate Model Evaluation (NCI project `ct11`) shown below can be found by entering `ACCESS-NRI` in the
NCI Data Catalogue Search field: