diff --git a/.gitignore b/.gitignore index c9d3f6e12..8f7abb420 100644 --- a/.gitignore +++ b/.gitignore @@ -9,7 +9,7 @@ # USE CAUTION WHEN ADDING WILDCARDS, as some builds use different filename # # conventions than others # ############################################################################## -build*/ +build/*/ install*/ *.[aox] diff --git a/.readthedocs.yml b/.readthedocs.yml index 32872d19e..817d1e9ef 100644 --- a/.readthedocs.yml +++ b/.readthedocs.yml @@ -11,7 +11,7 @@ build: # Build documentation in the docs/ directory with Sphinx sphinx: - configuration: docs/UserGuide/conf.py + configuration: doc/UserGuide/conf.py # Build documentation with MkDocs #mkdocs: @@ -22,6 +22,5 @@ formats: all # Optionally set the version of Python and requirements required to build your docs python: -# version: 3.7 install: - - requirements: docs/UserGuide/requirements.txt + - requirements: doc/UserGuide/requirements.txt diff --git a/docs/2D-decomp.md b/doc/2D-decomp.md similarity index 100% rename from docs/2D-decomp.md rename to doc/2D-decomp.md diff --git a/docs/CMakeLists.txt b/doc/CMakeLists.txt similarity index 100% rename from docs/CMakeLists.txt rename to doc/CMakeLists.txt diff --git a/docs/Doxyfile.in b/doc/Doxyfile.in similarity index 100% rename from docs/Doxyfile.in rename to doc/Doxyfile.in diff --git a/docs/Release_Notes.ncep_post.v7.0.0 b/doc/Release_Notes.ncep_post.v7.0.0 similarity index 100% rename from docs/Release_Notes.ncep_post.v7.0.0 rename to doc/Release_Notes.ncep_post.v7.0.0 diff --git a/docs/UserGuide/BackgroundInfo/Acknowledgments.rst b/doc/UserGuide/BackgroundInfo/Acknowledgments.rst similarity index 100% rename from docs/UserGuide/BackgroundInfo/Acknowledgments.rst rename to doc/UserGuide/BackgroundInfo/Acknowledgments.rst diff --git a/docs/UserGuide/BackgroundInfo/CodeOverview.rst b/doc/UserGuide/BackgroundInfo/CodeOverview.rst similarity index 100% rename from docs/UserGuide/BackgroundInfo/CodeOverview.rst rename to doc/UserGuide/BackgroundInfo/CodeOverview.rst diff --git a/docs/UserGuide/BackgroundInfo/Introduction.rst b/doc/UserGuide/BackgroundInfo/Introduction.rst similarity index 100% rename from docs/UserGuide/BackgroundInfo/Introduction.rst rename to doc/UserGuide/BackgroundInfo/Introduction.rst diff --git a/docs/UserGuide/BackgroundInfo/index.rst b/doc/UserGuide/BackgroundInfo/index.rst similarity index 100% rename from docs/UserGuide/BackgroundInfo/index.rst rename to doc/UserGuide/BackgroundInfo/index.rst diff --git a/docs/UserGuide/BuildingRunningTesting/BuildingUPP.rst b/doc/UserGuide/BuildingRunningTesting/BuildingUPP.rst similarity index 99% rename from docs/UserGuide/BuildingRunningTesting/BuildingUPP.rst rename to doc/UserGuide/BuildingRunningTesting/BuildingUPP.rst index 0916e8ff2..ca11fa689 100644 --- a/docs/UserGuide/BuildingRunningTesting/BuildingUPP.rst +++ b/doc/UserGuide/BuildingRunningTesting/BuildingUPP.rst @@ -3,6 +3,8 @@ .. role:: bolditalic :class: bolditalic +.. _building-upp: + ************************* Building UPP Stand-Alone ************************* diff --git a/docs/UserGuide/BuildingRunningTesting/InputsOutputs.rst b/doc/UserGuide/BuildingRunningTesting/InputsOutputs.rst similarity index 100% rename from docs/UserGuide/BuildingRunningTesting/InputsOutputs.rst rename to doc/UserGuide/BuildingRunningTesting/InputsOutputs.rst diff --git a/docs/UserGuide/BuildingRunningTesting/RunningUPP.rst b/doc/UserGuide/BuildingRunningTesting/RunningUPP.rst similarity index 99% rename from docs/UserGuide/BuildingRunningTesting/RunningUPP.rst rename to doc/UserGuide/BuildingRunningTesting/RunningUPP.rst index 064bba5aa..f72df10be 100644 --- a/docs/UserGuide/BuildingRunningTesting/RunningUPP.rst +++ b/doc/UserGuide/BuildingRunningTesting/RunningUPP.rst @@ -3,6 +3,8 @@ .. role:: bolditalic :class: bolditalic +.. _running-upp: + *********************** Running UPP Stand-Alone *********************** diff --git a/doc/UserGuide/BuildingRunningTesting/TestingUPP.rst b/doc/UserGuide/BuildingRunningTesting/TestingUPP.rst new file mode 100644 index 000000000..1014f26fc --- /dev/null +++ b/doc/UserGuide/BuildingRunningTesting/TestingUPP.rst @@ -0,0 +1,53 @@ +.. role:: underline + :class: underline +.. role:: bolditalic + :class: bolditalic + +.. _testing-upp: + +***************** +Testing the UPP +***************** + +Running UPP Regression Tests +============================= + +To run the full regression test (RT) suite in preparation for opening a pull request (PR): + + #. Navigate to the local clone of your UPP fork containing the changes you would like to introduce, and run the included RT script within ``/ci`` + + .. code-block:: console + + cd /path/to/UPP/ci + nohup ./rt.sh -a -r $PWD/rundir -t $PWD/../ & + + where ``my_account`` is the name of an account where you have permissions to run jobs. The terminal will print a message like: + + .. code-block:: console + + nohup: ignoring input and appending output to ‘nohup.out’ + + The user can continue to issue commands in the Terminal while the RTs run in the background. + + .. note:: + + The time it takes for tests to run is queue-dependent. RTs can take as little as half an hour to run, but on machines with long queue times, it can take several hours to complete the full set of tests. + + #. Check ``rt.log./nohup.out`` for a short summary of any changes in results. The tests are finished when there are 16 timestamps and a final results summary (e.g., "No changes in test results detected."). + + * The ``/work`` directory generated in ``UPP/ci`` contains ``out.post.`` files, which list output from each test, including any unexpected errors during runtime. + * The ``/rundir`` directory generated within ``UPP/ci`` will include test case results, and ``.diff`` files located within each test's directory will outline changes in fields with the current baselines. + * Confirm expected changes within the run directory ``.diff`` files if any are present. + + * Changes in the ``rap_pe_test`` case only consisting of field 708 Convective Cloud Layer may be ignored; this is a known bug and will always be present within the ``WRFPRS.diff`` file. + +Additional Configuration +========================= +For repeated regression test runs, users can edit the ``rt.sh`` file and disable the specified test cases by changing their respective values to “no.” Users can disable the build step as well with the same value for the build variable above the tests. Please be sure to enable all test cases and build settings and conduct a full RT run in preparation for a pull request so that code managers (CMs) can confirm all changes in results are expected and consistent with the developer's results. + +``rt.sh`` will allow for changing the configuration of the regression tests if users desire to do so with the following available options: + +* ``w`` -- specify the work directory for test case job output +* ``r`` -- specify the run directory containing baselines and ``.diff`` files for comparison of changes in results + +The following are legacy options for when ``rt.sh`` was not included within the UPP repository and may be ignored by developers: ``-b``, ``-u``, ``-c``, ``-t``. \ No newline at end of file diff --git a/docs/UserGuide/BuildingRunningTesting/index.rst b/doc/UserGuide/BuildingRunningTesting/index.rst similarity index 92% rename from docs/UserGuide/BuildingRunningTesting/index.rst rename to doc/UserGuide/BuildingRunningTesting/index.rst index 2d6e1fc4d..02599d803 100644 --- a/docs/UserGuide/BuildingRunningTesting/index.rst +++ b/doc/UserGuide/BuildingRunningTesting/index.rst @@ -7,3 +7,4 @@ Building, Running, and Testing the UPP InputsOutputs BuildingUPP RunningUPP + TestingUPP diff --git a/docs/UserGuide/CustomizingTheUPP/AddNewVariable.rst b/doc/UserGuide/CustomizingTheUPP/AddNewVariable.rst similarity index 100% rename from docs/UserGuide/CustomizingTheUPP/AddNewVariable.rst rename to doc/UserGuide/CustomizingTheUPP/AddNewVariable.rst diff --git a/docs/UserGuide/CustomizingTheUPP/Regridding.rst b/doc/UserGuide/CustomizingTheUPP/Regridding.rst similarity index 97% rename from docs/UserGuide/CustomizingTheUPP/Regridding.rst rename to doc/UserGuide/CustomizingTheUPP/Regridding.rst index b6a540be7..088ed7b38 100644 --- a/docs/UserGuide/CustomizingTheUPP/Regridding.rst +++ b/doc/UserGuide/CustomizingTheUPP/Regridding.rst @@ -8,7 +8,7 @@ Users who wish to interpolate their UPP output to a different grid may do so wit *Wgrib2* is a versatile program that has the ability to convert grib2 files from one grid to another for various user-defined grids as well as predefined :term:`NCEP` grids. Complete documentation with examples -of regridding for all available grid definitions can be found at: http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/new_grid.html +of regridding for all available grid definitions can be found at: https://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/new_grid.html .. _Examples-of-wgrib2: @@ -125,4 +125,4 @@ the variable. .. note:: *wgrib2* is not distributed as part of the :term:`UFS`, but it can be installed via :term:`spack-stack` or :term:`HPC-Stack` along with other UFS prerequisite software. - Users may also download and install it directly from http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/. + Users may also download and install it directly from https://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/. diff --git a/docs/UserGuide/CustomizingTheUPP/index.rst b/doc/UserGuide/CustomizingTheUPP/index.rst similarity index 100% rename from docs/UserGuide/CustomizingTheUPP/index.rst rename to doc/UserGuide/CustomizingTheUPP/index.rst diff --git a/doc/UserGuide/Makefile b/doc/UserGuide/Makefile new file mode 100644 index 000000000..d3987c6e8 --- /dev/null +++ b/doc/UserGuide/Makefile @@ -0,0 +1,28 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXOPTS = -a -n #-W +SPHINXBUILD ?= sphinx-build +SOURCEDIR = . +BUILDDIR = build +LINKCHECKDIR = $(BUILDDIR)/linkcheck + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile linkcheck + +doc: + make clean + $(MAKE) linkcheck + $(MAKE) html + +linkcheck: + $(SPHINXBUILD) -b linkcheck $(SPHINXOPTS) $(SOURCEDIR) $(LINKCHECKDIR) + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) -w $(BUILDDIR)/warnings.log \ No newline at end of file diff --git a/docs/UserGuide/Reference/FAQ.rst b/doc/UserGuide/Reference/FAQ.rst similarity index 100% rename from docs/UserGuide/Reference/FAQ.rst rename to doc/UserGuide/Reference/FAQ.rst diff --git a/docs/UserGuide/Reference/Glossary.rst b/doc/UserGuide/Reference/Glossary.rst similarity index 88% rename from docs/UserGuide/Reference/Glossary.rst rename to doc/UserGuide/Reference/Glossary.rst index 011f0adf6..1c86e92df 100644 --- a/docs/UserGuide/Reference/Glossary.rst +++ b/doc/UserGuide/Reference/Glossary.rst @@ -67,7 +67,7 @@ Glossary `Gridpoint Statistical Interpolation `__ (GSI) is a variational data assimilation system, designed to be flexible, state-of-art, and run efficiently on various parallel computing platforms. It supports :term:`RRFS` features. GSI code is publicly available `on GitHub `__, and fix file data is publicly available `here `__. HPC-Stack - `HPC-Stack `__ is a repository that provides a unified, shell script-based build system for building the software stack required for numerical weather prediction (NWP) tools such as the `Unified Forecast System (UFS) `__ and the `Joint Effort for Data assimilation Integration (JEDI) `__ framework. It is being phased out in favor of :term:`spack-stack`. `HPC-Stack documentation `__ is available, but the repository and documentation is rarely updated since it is being deprecated. + `HPC-Stack `__ is a repository that provides a unified, shell script-based build system for building the software stack required for numerical weather prediction (NWP) tools such as the `Unified Forecast System (UFS) `__ and the `Joint Effort for Data assimilation Integration (JEDI) `__ framework. It is being phased out in favor of :term:`spack-stack`. `HPC-Stack documentation `__ is available, but the repository and documentation is rarely updated since it is being deprecated. HRRR `High Resolution Rapid Refresh `__. The HRRR is a NOAA real-time 3-km resolution, hourly updated, cloud-resolving, convection-allowing atmospheric model initialized by 3-km grids with 3-km radar assimilation. Radar data is assimilated in the HRRR every 15 min over a 1-hour period adding further detail to that provided by the hourly data assimilation from the 13-km radar-enhanced Rapid Refresh. @@ -137,7 +137,7 @@ Glossary `Spack `__ is a package management tool designed to support multiple versions and configurations of software on a wide variety of platforms and environments. It was designed for large supercomputing centers, where many users and application teams share common installations of software on clusters with exotic architectures. spack-stack - The `spack-stack `__ is a collaborative effort between the NOAA Environmental Modeling Center (:term:`EMC`), the UCAR Joint Center for Satellite Data Assimilation (:term:`JCSDA`), and the Earth Prediction Innovation Center (:term:`EPIC`). *spack-stack* is a repository that provides a :term:`Spack`-based method for building the software stack required for numerical weather prediction (NWP) tools such as the `Unified Forecast System (UFS) `__ and the `Joint Effort for Data assimilation Integration (JEDI) `__ framework. *spack-stack* uses the Spack package manager along with custom Spack configuration files and Python scripts to simplify installation of the libraries required to run various applications. The *spack-stack* can be installed on a range of platforms and comes pre-configured for many systems. Users can install the necessary packages for a particular application and later add the missing packages for another application without having to rebuild the entire stack. + The `spack-stack `__ is a collaborative effort between the NOAA Environmental Modeling Center (:term:`EMC`), the UCAR Joint Center for Satellite Data Assimilation (:term:`JCSDA`), and the Earth Prediction Innovation Center (:term:`EPIC`). *spack-stack* is a repository that provides a :term:`Spack`-based method for building the software stack required for numerical weather prediction (NWP) tools such as the `Unified Forecast System (UFS) `__ and the `Joint Effort for Data assimilation Integration (JEDI) `__ framework. *spack-stack* uses the Spack package manager along with custom Spack configuration files and Python scripts to simplify installation of the libraries required to run various applications. The *spack-stack* can be installed on a range of platforms and comes pre-configured for many systems. Users can install the necessary packages for a particular application and later add the missing packages for another application without having to rebuild the entire stack. UFS The Unified Forecast System is a community-based, coupled, comprehensive Earth modeling diff --git a/docs/UserGuide/Reference/index.rst b/doc/UserGuide/Reference/index.rst similarity index 100% rename from docs/UserGuide/Reference/index.rst rename to doc/UserGuide/Reference/index.rst diff --git a/docs/UserGuide/_static/custom.css b/doc/UserGuide/_static/custom.css similarity index 100% rename from docs/UserGuide/_static/custom.css rename to doc/UserGuide/_static/custom.css diff --git a/docs/UserGuide/_static/theme_overrides.css b/doc/UserGuide/_static/theme_overrides.css similarity index 100% rename from docs/UserGuide/_static/theme_overrides.css rename to doc/UserGuide/_static/theme_overrides.css diff --git a/docs/UserGuide/conf.py b/doc/UserGuide/conf.py similarity index 78% rename from docs/UserGuide/conf.py rename to doc/UserGuide/conf.py index b18cef9d7..e0524f2a0 100644 --- a/docs/UserGuide/conf.py +++ b/doc/UserGuide/conf.py @@ -10,9 +10,9 @@ # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # -# import os -# import sys -# sys.path.insert(0, os.path.abspath('.')) +import os +import sys +sys.path.insert(0, os.path.abspath('.')) # -- Project information ----------------------------------------------------- @@ -53,6 +53,19 @@ # The master toctree document. master_doc = 'index' +# -- Linkcheck options ------------------------------------------------- + +# Avoid a 403 Forbidden error when accessing certain links (e.g., noaa.gov) +# Can be found using navigator.userAgent inside a browser console. +user_agent = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" + +# Ignore working links that cause a linkcheck 403 error. +linkcheck_ignore = [] + +linkcheck_allowed_redirects = {r"https://github.com/JCSDA/crtm/wiki/.*/.*": + r"https://raw.githubusercontent.com/wiki/JCSDA/crtm/.*/.*", + } + # -- Options for HTML output ------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for @@ -60,9 +73,13 @@ # html_theme = 'sphinx_rtd_theme' html_theme_path = ["_themes", ] +html_logo = 'https://github.com/ufs-community/ufs/wiki/images/ufs.png' # html_theme_options = {} -html_theme_options = {"body_max_width": "none"} +html_theme_options = { + "body_max_width": "none", + "navigation_depth": 6, + } # html_sidebar_options = {} html_sidebars = { '**': ['globaltoc.html', 'relations.html', 'sourcelink.html', 'searchbox.html'] } diff --git a/docs/UserGuide/index.rst b/doc/UserGuide/index.rst similarity index 100% rename from docs/UserGuide/index.rst rename to doc/UserGuide/index.rst diff --git a/docs/UserGuide/requirements.txt b/doc/UserGuide/requirements.in similarity index 73% rename from docs/UserGuide/requirements.txt rename to doc/UserGuide/requirements.in index 7be32f526..26c778f4a 100644 --- a/docs/UserGuide/requirements.txt +++ b/doc/UserGuide/requirements.in @@ -1,3 +1,3 @@ -sphinxcontrib-bibtex +sphinx>=6.0.0 sphinx_rtd_theme -docutils==0.16 \ No newline at end of file +sphinxcontrib-bibtex diff --git a/doc/UserGuide/requirements.txt b/doc/UserGuide/requirements.txt new file mode 100644 index 000000000..b4642f587 --- /dev/null +++ b/doc/UserGuide/requirements.txt @@ -0,0 +1,82 @@ +# +# This file is autogenerated by pip-compile with Python 3.9 +# by the following command: +# +# pip-compile requirements.in +# +alabaster==0.7.16 + # via sphinx +babel==2.14.0 + # via sphinx +certifi==2024.2.2 + # via requests +charset-normalizer==3.3.2 + # via requests +docutils==0.20.1 + # via + # pybtex-docutils + # sphinx + # sphinx-rtd-theme + # sphinxcontrib-bibtex +idna==3.6 + # via requests +imagesize==1.4.1 + # via sphinx +importlib-metadata==7.0.1 + # via + # sphinx + # sphinxcontrib-bibtex +jinja2==3.1.3 + # via sphinx +latexcodec==2.0.1 + # via pybtex +markupsafe==2.1.5 + # via jinja2 +packaging==23.2 + # via sphinx +pybtex==0.24.0 + # via + # pybtex-docutils + # sphinxcontrib-bibtex +pybtex-docutils==1.0.3 + # via sphinxcontrib-bibtex +pygments==2.17.2 + # via sphinx +pyyaml==6.0.1 + # via pybtex +requests==2.31.0 + # via sphinx +six==1.16.0 + # via + # latexcodec + # pybtex +snowballstemmer==2.2.0 + # via sphinx +sphinx==7.2.6 + # via + # -r requirements.in + # sphinx-rtd-theme + # sphinxcontrib-bibtex + # sphinxcontrib-jquery +sphinx-rtd-theme==2.0.0 + # via -r requirements.in +sphinxcontrib-applehelp==1.0.8 + # via sphinx +sphinxcontrib-bibtex==2.6.2 + # via -r requirements.in +sphinxcontrib-devhelp==1.0.6 + # via sphinx +sphinxcontrib-htmlhelp==2.0.5 + # via sphinx +sphinxcontrib-jquery==4.1 + # via sphinx-rtd-theme +sphinxcontrib-jsmath==1.0.1 + # via sphinx +sphinxcontrib-qthelp==1.0.7 + # via sphinx +sphinxcontrib-serializinghtml==1.1.10 + # via sphinx +urllib3==2.2.1 + # via requests +zipp==3.17.0 + # via importlib-metadata diff --git a/docs/UserGuide/tables/UFS_unified_variables_table.csv b/doc/UserGuide/tables/UFS_unified_variables_table.csv similarity index 100% rename from docs/UserGuide/tables/UFS_unified_variables_table.csv rename to doc/UserGuide/tables/UFS_unified_variables_table.csv diff --git a/docs/UserGuide/tables/UFS_unified_variables_table.rst b/doc/UserGuide/tables/UFS_unified_variables_table.rst similarity index 100% rename from docs/UserGuide/tables/UFS_unified_variables_table.rst rename to doc/UserGuide/tables/UFS_unified_variables_table.rst diff --git a/docs/UserGuide/tables/UPP_GRIB2_Table_byID.csv b/doc/UserGuide/tables/UPP_GRIB2_Table_byID.csv similarity index 100% rename from docs/UserGuide/tables/UPP_GRIB2_Table_byID.csv rename to doc/UserGuide/tables/UPP_GRIB2_Table_byID.csv diff --git a/docs/UserGuide/tables/UPP_GRIB2_Table_byID.rst b/doc/UserGuide/tables/UPP_GRIB2_Table_byID.rst similarity index 100% rename from docs/UserGuide/tables/UPP_GRIB2_Table_byID.rst rename to doc/UserGuide/tables/UPP_GRIB2_Table_byID.rst diff --git a/docs/module-load-make-howto.txt b/doc/module-load-make-howto.txt similarity index 100% rename from docs/module-load-make-howto.txt rename to doc/module-load-make-howto.txt diff --git a/docs/user_guide.md b/doc/user_guide.md similarity index 100% rename from docs/user_guide.md rename to doc/user_guide.md