diff --git a/_sources/dataset_access/CONUS404_ACCESS.md b/_sources/dataset_access/CONUS404_ACCESS.md index 4d551c25..3ad383cd 100644 --- a/_sources/dataset_access/CONUS404_ACCESS.md +++ b/_sources/dataset_access/CONUS404_ACCESS.md @@ -1,11 +1,20 @@ -# CONUS404 Access +# CONUS404 and CONUS404 Bias-Adjusted Data Access -This section contains notebooks that demonstrate how to access and perform basic data manipulation for the [CONUS404 dataset](https://doi.org/10.5066/P9PHPK4F). +This section contains notebooks that demonstrate how to access and perform basic data manipulation for the [CONUS404 dataset](https://doi.org/10.5066/P9PHPK4F). The examples can also be applied to the [CONUS404 bias-adjusted dataset](https://doi.org/10.5066/P9JE61P7). -In the CONUS404 intake sub-catalog (see [here](../dataset_catalog/README.md) for an explainer of our intake data catalog), you will see entries for four CONUS404 datasets: conus404-hourly, conus404-daily, conus404-monthly, and conus404-daily-diagnostic data. Each of these datasets is duplicated in three different storage locations (as the [intake catalog section](../dataset_catalog/README.md) also describes). The conus404-hourly data is a subset of the wrfout model output and conus404-daily-diagnostic is a subset from the wrfxtrm model output, both of which are described in the official [CONUS404 data release](https://doi.org/10.5066/P9PHPK4F). We also have conus404-daily and conus404-monthly files, which are just resampled from the conus404-hourly data. +In the CONUS404 intake sub-catalog (see [here](../dataset_catalog/README.md) for an explainer of our intake data catalog), you will see entries for four CONUS404 datasets: `conus404-hourly`, `conus404-daily`, `conus404-monthly`, and `conus404-daily-diagnostic` data, as well as two CONUS404 bias-adjusted datasets: `conus404-hourly-ba`, `conus404-daily-ba`. Each of these datasets is duplicated in up to three different storage locations (as the [intake catalog section](../dataset_catalog/README.md) also describes). + +## CONUS404 Data +The `conus404-hourly` data is a subset of the wrfout model output and `conus404-daily-diagnostic` is a subset from the wrfxtrm model output, both of which are described in the official [CONUS404 data release](https://doi.org/10.5066/P9PHPK4F). We also have `conus404-daily` and `conus404-monthly` files, which are just resampled from the `conus404-hourly` data. **Please note that the values in the ACLWDNB, ACLWUPB, ACSWDNB, ACSWDNT, and ACSWUPB variables available in the zarr store differ from the original model output.** These variables have been re-calculated to reflect the accumulated value since the model start, as directed in the WRF manual. An attribute has been added to each of these variables in the zarr store to denote the accumulation period for the variable. +**We recommend that you regularly check our [CONUS404 changelog](./CONUS404_CHANGELOG) to see any updates that have been made to the zarr stores.** We do not anticipate regular changes to the dataset, but we may need to fix an occasional bug or update the dataset with additional years of data. + +## CONUS404 Bias-Adjusted Data +The `conus404-hourly-ba` data contains bias-adjusted temperature and precipiation data from the CONUS404 dataset, which is described in the official [CONUS404 bias adjusted data release](https://doi.org/10.5066/P9JE61P7). The `conus404-daily-ba` files are resampled from the `conus404-hourly-ba` data. + +## Example Notebooks We currently have five notebooks to help demonstrate how to work with these datasets in a python workflow: - [Explore CONUS404 Dataset](./conus404_explore.ipynb): opens the CONUS404 dataset, loads and plots the entire spatial domain of a specified variable at a specfic time step, and loads and plots a time series of a variable at a specified coordinate pair. @@ -14,8 +23,6 @@ We currently have five notebooks to help demonstrate how to work with these data - [CONUS404 Point Selection](./conus404_point_selection.ipynb): samples the CONUS404 data at a selection of gage locations using their lat/lon point coordinates. - [CONUS404 Regridding (Curvilinear => Rectilinear)](./conus404_regrid.ipynb): regrids a subset of the CONUS404 dataset from a curvilinear grid to a rectilinear grid and saves the output to a netcdf file. The package used in this demo is not compatible with Windows. We hope to improve upon this methodology, and will likely update the package/technique used in the future. -**We recommend that you regularly check our [CONUS404 changelog](./CONUS404_CHANGELOG) to see any updates that have been made to the zarr stores.** We do not anticipate regular changes to the dataset, but we may need to fix an occasional bug or update the dataset with additional years of data. - These methods are likely applicable to many of the other key HyTEST datasets that can be opened with xarray. *Note: If you need help setting up a computing environment where you can run these notebooks, you should review the [Computing Environments](../environment_set_up/README.md) section of the documentation.* \ No newline at end of file diff --git a/_sources/dataset_catalog/README.md b/_sources/dataset_catalog/README.md index 573631bd..74dd9e48 100644 --- a/_sources/dataset_catalog/README.md +++ b/_sources/dataset_catalog/README.md @@ -36,19 +36,33 @@ list(cat) ``` produces a list of datasets, for example: ``` -['conus404-drb-eval-tutorial-catalog', +['conus404-catalog', + 'conus404-drb-eval-tutorial-catalog', 'nhm-v1.0-daymet-catalog', 'nhm-v1.1-c404-bc-catalog', 'nhm-v1.1-gridmet-catalog', - 'conus404-catalog', + 'trends-and-drivers-catalog', + 'nhm-prms-v1.1-gridmet-format-testing-catalog', 'nwis-streamflow-usgs-gages-onprem', - 'nwis-streamflow-usgs-gages-cloud', + 'nwis-streamflow-usgs-gages-osn', 'nwm21-streamflow-usgs-gages-onprem', - 'nwm21-streamflow-usgs-gages-cloud', + 'nwm21-streamflow-usgs-gages-osn', 'nwm21-streamflow-cloud', + 'geofabric_v1_1-zip-osn', + 'geofabric_v1_1_POIs_v1_1-osn', + 'geofabric_v1_1_TBtoGFv1_POIs-osn', + 'geofabric_v1_1_nhru_v1_1-osn', + 'geofabric_v1_1_nhru_v1_1_simp-osn', + 'geofabric_v1_1_nsegment_v1_1-osn', + 'gages2_nndar-osn', + 'wbd-zip-osn', + 'huc12-geoparquet-osn', + 'huc12-gpkg-osn', 'nwm21-scores', 'lcmap-cloud', - 'rechunking-tutorial-cloud'] + 'rechunking-tutorial-osn', + 'pointsample-tutorial-sites-osn', + 'pointsample-tutorial-output-osn'] ``` The characteristics of indivdual datasets can be explored: ```python diff --git a/_sources/dataset_catalog/subcatalogs/README.md b/_sources/dataset_catalog/subcatalogs/README.md index a4960b4c..d5b0a280 100644 --- a/_sources/dataset_catalog/subcatalogs/README.md +++ b/_sources/dataset_catalog/subcatalogs/README.md @@ -14,14 +14,28 @@ produces a list of datasets and sub-catalogs in the main HyTEST data catalog, fo 'nhm-v1.0-daymet-catalog', 'nhm-v1.1-c404-bc-catalog', 'nhm-v1.1-gridmet-catalog', + 'trends-and-drivers-catalog', + 'nhm-prms-v1.1-gridmet-format-testing-catalog', 'nwis-streamflow-usgs-gages-onprem', - 'nwis-streamflow-usgs-gages-cloud', + 'nwis-streamflow-usgs-gages-osn', 'nwm21-streamflow-usgs-gages-onprem', - 'nwm21-streamflow-usgs-gages-cloud', + 'nwm21-streamflow-usgs-gages-osn', 'nwm21-streamflow-cloud', + 'geofabric_v1_1-zip-osn', + 'geofabric_v1_1_POIs_v1_1-osn', + 'geofabric_v1_1_TBtoGFv1_POIs-osn', + 'geofabric_v1_1_nhru_v1_1-osn', + 'geofabric_v1_1_nhru_v1_1_simp-osn', + 'geofabric_v1_1_nsegment_v1_1-osn', + 'gages2_nndar-osn', + 'wbd-zip-osn', + 'huc12-geoparquet-osn', + 'huc12-gpkg-osn', 'nwm21-scores', 'lcmap-cloud', - 'rechunking-tutorial-cloud'] + 'rechunking-tutorial-osn', + 'pointsample-tutorial-sites-osn', + 'pointsample-tutorial-output-osn'] ``` We can then open the CONUS404 sub-catalog with: @@ -42,7 +56,11 @@ producing a list of all the CONUS404 dataset versions: 'conus404-daily-osn', 'conus404-monthly-onprem', 'conus404-monthly-cloud', - 'conus404-monthly-osn'] + 'conus404-monthly-osn', + 'conus404-hourly-ba-onprem', + 'conus404-hourly-ba-osn', + 'conus404-daily-ba-onprem', + 'conus404-daily-ba-osn'] ``` The characteristics of indivdual datasets can be explored: diff --git a/dataset_access/CONUS404_ACCESS.html b/dataset_access/CONUS404_ACCESS.html index ed9a4a54..919d61c7 100644 --- a/dataset_access/CONUS404_ACCESS.html +++ b/dataset_access/CONUS404_ACCESS.html @@ -8,7 +8,7 @@ - CONUS404 Access — HyTEST Project + CONUS404 and CONUS404 Bias-Adjusted Data Access — HyTEST Project @@ -32,7 +32,7 @@ - + @@ -216,7 +216,7 @@
  • HyTEST Intake Sub-Catalogs
  • -
  • CONUS404 Access