Create gw_setup script and extend to other machines (#1706)

* Create gw_setup script and extend to other machines
Add a new script to handle setting up the lmod environment needed to run all the setup for
checkout and experiment generation. C
`checkout.sh` now uses this script as more recent versions of `git` are needed
on some machines. 
Documentation has been updated to recommend this method on all machines rather
than the previous instructions.
Updates documentation to add the checkout with UFSDA option.
Closes #1677

docs/source/clone.rst

@@ -31,6 +31,19 @@ For cycled (w/ data assimilation):
    ./build_all.sh
    ./link_workflow.sh
 
+For coupled cycling (include new UFSDA):
+
+[Currently only available on Hera and Orion]
+
+::
+
+   git clone https://github.com/NOAA-EMC/global-workflow.git
+   cd global-workflow/sorc
+   ./checkout.sh -gu
+   ./build_all.sh
+   ./link_workflow.sh
+
+
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Clone workflow and component repositories
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -104,6 +117,14 @@ Or with the ``-g`` switch to include data assimilation (GSI) for cycling:
    cd sorc
    ./checkout.sh -g
 
+Or also with the ``-u`` swtich to include coupled DA (via UFSDA):
+[Currently only available on Hera and Orion]
+
+::
+
+   cd sorc
+   ./checkout.sh -gu
+
 Each component cloned via checkout.sh will have a log (``/sorc/logs/checkout-COMPONENT.log``). Check the screen output and logs for clone errors.
 
 ^^^^^^^^^^^^^^^^

docs/source/setup.rst

@@ -2,69 +2,13 @@
 Experiment Setup
 ================
 
- Global workflow uses a set of scripts to help configure and set up the drivers (also referred to as Workflow Manager) that run the end-to-end system. While currently we use a `ROCOTO <https://github.com/christopherwharrop/rocoto/wiki/documentation>`__ based system and that is documented here, an `ecFlow <https://www.ecmwf.int/en/learning/training/introduction-ecmwf-job-scheduler-ecflow>`__ based systm is also under development and will be introduced to the Global Workflow when it is mature. To run the setup scripts, you need to make sure to have a copy of ``python3`` with ``numpy`` available. The easiest way to guarantee this is to load python from the `official hpc-stack installation <https://github.com/NOAA-EMC/hpc-stack/wiki/Official-Installations>`_ for the machine you are on:
+ Global workflow uses a set of scripts to help configure and set up the drivers (also referred to as Workflow Manager) that run the end-to-end system. While currently we use a `ROCOTO <https://github.com/christopherwharrop/rocoto/wiki/documentation>`__ based system and that is documented here, an `ecFlow <https://www.ecmwf.int/en/learning/training/introduction-ecmwf-job-scheduler-ecflow>`__ based systm is also under development and will be introduced to the Global Workflow when it is mature. To run the setup scripts, you need to have rocoto and a python3 environment with several specific libraries. The easiest way to guarantee this is to source the following script, which will load the necessary modules for your machine:
 
-.. list-table:: Python Module Load Commands
-   :widths: 25 120
-   :header-rows: 1
+ ::
 
-   * - **MACHINE**
-     - **COMMAND(S)**
-   * - Hera
-     - ::
+   # Note: this will wipe your existing lmod environment
+   source workflow/gw_setup.sh
 
-           module use -a /contrib/anaconda/modulefiles
-           module load anaconda/anaconda3-5.3.1
-   * - Orion
-     - ::
-
-           module load python/3.7.5
-   * - WCOSS2
-     - ::
-
-           module load python/3.8.6
-   * - S4
-     - ::
-
-           module load miniconda/3.8-s4
-
-   * - Jet
-     - ::
-
-           module use /mnt/lfs4/HFIP/hfv3gfs/role.epic/miniconda3/modulefiles
-           module load miniconda3/4.12.0
-           conda activate ufswm
-
-If running with Rocoto make sure to have a Rocoto module loaded before running setup scripts:
-
-.. list-table:: ROCOTO Module Load Commands
-   :widths: 25 120
-   :header-rows: 1
-
-   * - **MACHINE**
-     - **COMMAND(S)**
-   * - Hera
-     - ::
-
-           module load rocoto/1.3.3
-   * - Orion
-     - ::
-
-           module load contrib
-           module load rocoto/1.3.3
-   * - WCOSS2
-     - ::
-
-           module use /apps/ops/test/nco/modulefiles/
-           module load core/rocoto/1.3.5
-   * - S4
-     - ::
-
-           module load rocoto/1.3.4
-   * - Jet
-     - ::
-
-           module load rocoto/1.3.3
 
 ^^^^^^^^^^^^^^^^^^^^^^^^
 Forecast-only experiment

modulefiles/module_gwsetup.jet.lua

@@ -0,0 +1,22 @@
+help([[
+Load environment to run GFS workflow setup scripts on Jet
+]])
+
+load(pathJoin("rocoto", "1.3.3"))
+
+if (mode() == "unload") then
+    -- `execute` delays commands until last, but we need conda deactivated
+    --   before unloading miniconda. `print` (bizarrely) still executes the
+    --   command, but does it immediately. The semicolon is necessary 
+    --   because otherwise other commands get tacked onto the same line.
+    print("conda deactivate;")
+end
+
+-- Temporary until official hpc-stack is updated
+prepend_path("MODULEPATH", "/mnt/lfs4/HFIP/hfv3gfs/role.epic/miniconda3/modulefiles")
+load(pathJoin("miniconda3", "4.12.0"))
+if (mode() == "load") then
+    execute{cmd="conda activate ufswm", modeA={"load"}}
+end
+
+whatis("Description: GFS run setup environment")

modulefiles/module_gwsetup.orion.lua

@@ -2,13 +2,12 @@ help([[
 Load environment to run GFS workflow ci scripts on Orion
 ]])
 
--- Temporary until official hpc-stack is updated
-
 prepend_path("MODULEPATH", "/apps/modulefiles/core")
 load(pathJoin("contrib","0.1"))
 load(pathJoin("rocoto","1.3.3"))
 load(pathJoin("git","2.28.0"))
 
+-- Temporary until official hpc-stack is updated
 prepend_path("MODULEPATH", "/work2/noaa/global/wkolczyn/save/hpc-stack/modulefiles/stack")
 load(pathJoin("hpc", "1.2.0"))
 load(pathJoin("hpc-miniconda3", "4.6.14"))

modulefiles/module_gwsetup.s4.lua

@@ -0,0 +1,9 @@
+help([[
+Load environment to run GFS workflow setup scripts on S4
+]])
+
+load(pathJoin("miniconda", "3.8-s4"))
+load(pathJoin("rocoto","1.3.5"))
+load(pathJoin("git","2.30.0"))
+
+whatis("Description: GFS run setup environment")

modulefiles/module_gwsetup.wcoss2.lua

@@ -0,0 +1,10 @@
+help([[
+Load environment to run GFS workflow ci scripts on WCOSS2
+]])
+
+load(pathJoin("git","2.29.0"))
+
+prepend_path("MODULEPATH", "/apps/ops/test/nco/modulefiles/core")
+load(pathJoin("rocoto","1.3.5"))
+
+whatis("Description: GFS run ci top-level sripts environment")

sorc/checkout.sh

@@ -144,6 +144,9 @@ export topdir
 export logdir="${topdir}/logs"
 mkdir -p "${logdir}"
 
+# Setup lmod environment
+source "${topdir}/../workflow/gw_setup.sh"
+
 # The checkout version should always be a speciifc commit (hash or tag), not a branch
 errs=0
 checkout "gfs_utils.fd"    "https://github.com/NOAA-EMC/gfs-utils"              "8965258"                    ; errs=$((errs + $?))

workflow/gw_setup.sh

@@ -0,0 +1,15 @@
+#! /bin/bash
+
+#
+# Resets the lmod environment and loads the modules necessary to run all the
+#   scripts necessary to prepare the workflow for use (checkout, experiment 
+#   setup, etc.).
+#
+# This script should be SOURCED to properly setup the environment.
+#
+
+HOMEgfs="$(cd "$(dirname  "${BASH_SOURCE[0]}")/.." >/dev/null 2>&1 && pwd )"
+source "${HOMEgfs}/ush/detect_machine.sh"
+source "${HOMEgfs}/ush/module-setup.sh"
+module use "${HOMEgfs}/modulefiles"
+module load "module_gwsetup.${MACHINE_ID}"