Use spack-stack-1.6.0 environment, in accordance with the current UFS-WM stack

[natalie-perlin]

DESCRIPTION OF CHANGES:

Modulefiles are updated to use spack-stack-1.6.0 environment
as currently used by UFS-WM and UFS-SRW App.

The main difference of the software package versions as related to the UFS_UTILS is an updated version of g2/3.5.1. It would further enable implement changes to the UFS_UTILS that require this higher version of g2, such as reading large grib2 files.

Files adapted for platforms and compilers, except for Stampede (no longer supported or used by UFS as Tier 1 platforms) and WCOSS2.

TESTS CONDUCTED:

• Compile branch on all Tier 1 machines using Intel and lntelllvm (Orion, Jet, Hera, Hercules) - except for WCOSS2
• Compile branch on WCOSS2. Done using 2d42c42.
• Compile branch on Hera using GNU. Done using 2d42c42.
• Compile branch in 'Debug' mode on WCOSS2. Done using 2d42c42.
• Compile with Doxygen on any machine with no errors.
• Run unit tests locally on any Tier 1 machine. Done on Hera using Gnu and 2d42c42.
• Run relevant consistency tests locally on all Tier 1 machines. Done on Hera using 2d42c42. All tests passed.

Describe any additional tests performed.

DEPENDENCIES:

DOCUMENTATION:

• Doxygen is updated.

ISSUE:

This PR is the first step to contribute in resolving the issue ufs-community/ufs-srweather-app#1191 that requires reading large grib2 files.

[GeorgeGayno-NOAA]

@DavidHuber-NOAA - let us know if this is a problem for the global workflow.

[GeorgeGayno-NOAA]

@natalie-perlin - I only have access to the Tier 1 machines (Hera, Jet, Orion, Hercules and WCOSS2). You will need to test on the other machines.

[GeorgeGayno-NOAA]

On Hera, when I compile and run the chgres_cube consistency tests using Gnu, I get this slurm error:

The application appears to have been direct launched using "srun",
but OMPI was not built with SLURM's PMI support and therefore cannot
execute. There are several options for building PMI support under
SLURM, depending upon the SLURM version you are using:

  version 16.05 or later: you can use SLURM's PMIx support. This
  requires that you configure and build SLURM --with-pmix.

  Versions earlier than 16.05: you must use either SLURM's PMI-1 or
  PMI-2 support. SLURM builds PMI-1 by default, or you can manually
  install PMI-2. You must then build Open MPI using --with-pmi pointing
  to the SLURM PMI library location.

Please configure as appropriate and try again.

'develop' uses Gnu v9.2. This branch uses v13.3.0.

[natalie-perlin]

On Hera, when I compile and run the chgres_cube consistency tests using Gnu, I get this slurm error:

The application appears to have been direct launched using "srun",
but OMPI was not built with SLURM's PMI support and therefore cannot
execute. There are several options for building PMI support under
SLURM, depending upon the SLURM version you are using:

  version 16.05 or later: you can use SLURM's PMIx support. This
  requires that you configure and build SLURM --with-pmix.

  Versions earlier than 16.05: you must use either SLURM's PMI-1 or
  PMI-2 support. SLURM builds PMI-1 by default, or you can manually
  install PMI-2. You must then build Open MPI using --with-pmi pointing
  to the SLURM PMI library location.

Please configure as appropriate and try again.

'develop' uses Gnu v9.2. This branch uses v13.3.0.

Will take a look. Some additional env. variables and or/linking flags could be required (as for other repos), looking into this.

[natalie-perlin]

@GeorgeGayno-NOAA - updated the build.hera.gnu.lua modulefile, adding cpp flags, library paths and preloaded library to address the runtime issues.
When building the executables, it is important to open a fresh terminal to avoid gnu module conflicts with any system default gnu modules.

The following has been added to the modulefile:

prepend_path("CPPFLAGS", " -I/apps/slurm_hera/23.11.3/include/slurm"," ")
prepend_path("LD_LIBRARY_PATH", "/apps/slurm_hera/23.11.3/lib")
setenv("LD_PRELOAD", "/scratch2/NCEPDEV/stmp1/role.epic/installs/gnu/13.3.0/lib64/libstdc++.so.6")

Building the executable was done as following:

export target=hera
export compiler=gnu
./build_all.sh >& log.build.hera.gnu13.3. &

The log file is attached: llog.build.hera.gnu13.3.txt

[GeorgeGayno-NOAA]

@GeorgeGayno-NOAA - updated the build.hera.gnu.lua modulefile, adding cpp flags, library paths and preloaded library to address the runtime issues. When building the executables, it is important to open a fresh terminal to avoid gnu module conflicts with any system default gnu modules.

The following has been added to the modulefile:

prepend_path("CPPFLAGS", " -I/apps/slurm_hera/23.11.3/include/slurm"," ")
prepend_path("LD_LIBRARY_PATH", "/apps/slurm_hera/23.11.3/lib")
setenv("LD_PRELOAD", "/scratch2/NCEPDEV/stmp1/role.epic/installs/gnu/13.3.0/lib64/libstdc++.so.6")

Building the executable was done as following:

export target=hera
export compiler=gnu
./build_all.sh >& log.build.hera.gnu13.3. &

The log file is attached: llog.build.hera.gnu13.3.txt

That fixed it. I was able to run the unit tests and chgres_cube consistency tests using Gnu.

[GeorgeGayno-NOAA]

@natalie-perlin - be sure to merge in the latest updates from 'develop'.

[GeorgeGayno-NOAA]

@DavidHuber-NOAA - are these changes OK with the global workflow?

[DavidHuber-NOAA]

@GeorgeGayno-NOAA Yes, these are fine for the global workflow.

Thanks @natalie-perlin for the updates! Approve.

[natalie-perlin]

@DavidHuber-NOAA @GeorgeGayno-NOAA -
brought the recent commits in develop into the feature/ss-1.6.0-fms branch, and it is now up-to-date.