Intel install with eckit set to gcc does not create a module for eckit

[stiggy87]

Describe the bug

Creating an spack-stack image on Ubuntu 24.04 with intel classic, I run into the problem where after I setup the meta-modules, the eckit (which is forced to be compiled by gcc) does not get loaded even though the required modules of stack-intel and stack-intel-oneapi-mpi are met.

To Reproduce
Steps to reproduce the behavior:

Follow sites/tier2/aws/ubuntu2404/README.md for Intel setup (before concretize and install) from PR #1393

Add this to spack.yaml in the env:

packages:
...
  eckit:
    require:
    - '%gcc'

Concretize and install

spack module lmod -y refresh
spack stack setup-meta-modules

module purge
module use install/modulefiles/Core
module load stack-intel/2021.10.0
module load stack-intel-oneapi-mpi/2021.10.0
module load base-env
module load jedi-mpas-env # <--- Get error here

Expected behavior
eckit should be loaded properly even if gcc built the module.

System:
What system(s) are you running the code on?

Ubuntu 24.04 (AWS AMI)

Additional context

$ spack module lmod find eckit
eckit/1.27.0

module debug=1 load eckit/1.27.0 result = eckit_module_load.txt

spack spec eckit result = eckit_spec.txt

spider.log

@climbfuji This looks like a weird meta-module issue, but I think there could be something else wrong here.

[climbfuji]

This is not expected to work, in fact not even supposed to work. eckit requires MPI if I am not mistaken. The MPI library is compiled with the principal=preferred compiler. Therefore, everything that depends on the MPI library must be compiled with the preferred compiler. There are notes in the setup-meta-modules extension in the spack-ext/lib directory, but probably not in the documentation (we should add that).

[climbfuji]

Found it: See

spack-stack/spack-ext/lib/jcsda-emc/spack-stack/stack/meta_modules.py

Lines 361 to 367 in 71aa767

	# Determine the preferred compiler and sort the flattened list of compilers
	# such that the preferred compiler comes last. This is so that all other
	# compilers populate the MODULEPATHS_SAVE list before the preferred compiler
	# takes it and adds it to the stack-COMPILER metamodule. Likewise, we need
	# to save the list of compiler substitutions from the preferred compiler
	# so that we have access to it when we build the MPI meta module. Note that
	# by definition, only the preferred compiler can be used for MPI dependencies.

[climbfuji]

The real question is why you would want to compile eckit with GCC when your main compiler is Intel?

[stiggy87]

Thanks for showing me the info in the lib! I do agree that we should add to documentation to it to discourage users from doing it (even though it might be a rarity).

The issue I was running into is that, at least, on Ubuntu 24.04 the intel build of eckit failed to find the standard c++ library, even if the paths are present in the compiiler.yaml. This was fine on Rocky8 that had similar paths added.