From c0a8f409bdd34c81b1bc4ad8a043f2665e02d42d Mon Sep 17 00:00:00 2001 From: torri Date: Tue, 30 Apr 2024 15:18:50 +0200 Subject: [PATCH 01/19] adding documentation on how to build and run on top of EESSI without EB --- docs/using_eessi/building_on_eessi.md | 31 ++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 92d599bb1..983a6fb3c 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -67,7 +67,32 @@ Finally, you should be able to load our newly build module: module load netCDF/4.9.0-gompi-2022a ``` -## Manually building software op top of EESSI -Building software on top of EESSI would require your linker to use the same system-dependencies as the software in EESSI does. In other words: it requires you to link against libraries from the compatibility layer, instead of from your host OS. +## Manually building software op top of EESSI (without EasyBuild) + +Building and running software on top of EESSI without EasyBuild is not as straightforward and requires some considerations to take care of. + +Build time is trivial as long as EESSI toolchains are loaded, so the runtime linker from the compat-layer is used. The safest way to make sure all libraries will point to the required locations is starting an EESSI shell before building. Steps would be: + +* First of all, load the environment by starting an EESSI shell as described [here](https://www.eessi.io/docs/using_eessi/setting_up_environment). +* Load all dependencies you need to build your software. You must use at least a toolchain from EESSI to compile it. +* Compile and make sure the library resolution points to the EESSI stack. For this, `ldd` from compatibility layer and **not** `/usr/bin/ldd` should be used when checking the binary. +* Once done, exit prefix shell or use another terminal to run your software. + +For runtime, as EESSI does not set `LD_LIBRARY_PATH`, the executable will need help finding the proper libraries. The easiest approach is to set this variable manually before running. Thus, assuming a new terminal with no EESSI shell loaded, you will need to: + +* Load dependencies for your software to run +* Set manually `LD_LIBRARY_PATH` to resolve libraries at runtime +```sh +export LD_LIBRARY_PATH=$LIBRARY_PATH:$EBROOTGCCcore/lib64 +``` +* Run! + +Note how this has to be done every time you want to run your software. A long term approach is to set add any library used by your software to the RPATH in your executable. This is also more reliable as your executable won't pick unexpected libraries from LD_LIBRARY_PATH. + +!!! Note No RPATH should point to a compatibility layer directory, only to software layer ones, as all resolving is done via `ld-linux*.so`. + +The biggest downside of this approach is that your executable becomes bound to the architecture you linked your libraries for. I. e, if you add to your executable RPATH a `libhdf5.so`compiled for `intel_avx512`, you will not be able to run that binary in a machine with a different architecture. + +!!! Note This documentation is intended for experts in runtime linker/runtime linker behaviour and most cases are untested. Any feedback on this topic is highly appreciated. + -While we plan to support this in the future, manually building on top of EESSI is currently not supported yet in a trivial way. From 0b5b8d038abf6e474ea2f3f0ee4bace5a6a6dcf6 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Mon, 27 May 2024 14:13:00 +0200 Subject: [PATCH 02/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 983a6fb3c..4b552fa33 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -71,7 +71,7 @@ module load netCDF/4.9.0-gompi-2022a Building and running software on top of EESSI without EasyBuild is not as straightforward and requires some considerations to take care of. -Build time is trivial as long as EESSI toolchains are loaded, so the runtime linker from the compat-layer is used. The safest way to make sure all libraries will point to the required locations is starting an EESSI shell before building. Steps would be: +It is expected that you will have loaded all of your required dependencies as modules from the EESSI environment. Since EESSI sets `LIBRARY_PATH` for all of the modules and the `GCC` compiler is configured to use the compat layer, there should be no additional configuration required to execute a standard build process. The safest way to make sure all libraries will point to the required locations (and do not leak in from the host operating system) is starting an EESSI prefix shell before building. To do this: * First of all, load the environment by starting an EESSI shell as described [here](https://www.eessi.io/docs/using_eessi/setting_up_environment). * Load all dependencies you need to build your software. You must use at least a toolchain from EESSI to compile it. From b4819b8350f2e6d8dd0e4d64976713a611b263f1 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Mon, 27 May 2024 14:13:16 +0200 Subject: [PATCH 03/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 4b552fa33..8b6b4f82e 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -74,7 +74,7 @@ Building and running software on top of EESSI without EasyBuild is not as straig It is expected that you will have loaded all of your required dependencies as modules from the EESSI environment. Since EESSI sets `LIBRARY_PATH` for all of the modules and the `GCC` compiler is configured to use the compat layer, there should be no additional configuration required to execute a standard build process. The safest way to make sure all libraries will point to the required locations (and do not leak in from the host operating system) is starting an EESSI prefix shell before building. To do this: * First of all, load the environment by starting an EESSI shell as described [here](https://www.eessi.io/docs/using_eessi/setting_up_environment). -* Load all dependencies you need to build your software. You must use at least a toolchain from EESSI to compile it. +* Load all dependencies you need to build your software. You must use at least a toolchain from EESSI to compile it (`foss` is a good option as it will also include MPI with OpenMPI and math libraries via FlexiBLAS/FFTW). * Compile and make sure the library resolution points to the EESSI stack. For this, `ldd` from compatibility layer and **not** `/usr/bin/ldd` should be used when checking the binary. * Once done, exit prefix shell or use another terminal to run your software. From c99b7973ebd62a4750d4ea6b82689a2739cd2a88 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Mon, 27 May 2024 14:13:27 +0200 Subject: [PATCH 04/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 8b6b4f82e..c18542587 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -76,7 +76,7 @@ It is expected that you will have loaded all of your required dependencies as mo * First of all, load the environment by starting an EESSI shell as described [here](https://www.eessi.io/docs/using_eessi/setting_up_environment). * Load all dependencies you need to build your software. You must use at least a toolchain from EESSI to compile it (`foss` is a good option as it will also include MPI with OpenMPI and math libraries via FlexiBLAS/FFTW). * Compile and make sure the library resolution points to the EESSI stack. For this, `ldd` from compatibility layer and **not** `/usr/bin/ldd` should be used when checking the binary. -* Once done, exit prefix shell or use another terminal to run your software. +* Once done, exit the EESSI prefix shell (with `exit`) or use another terminal to run your software. For runtime, as EESSI does not set `LD_LIBRARY_PATH`, the executable will need help finding the proper libraries. The easiest approach is to set this variable manually before running. Thus, assuming a new terminal with no EESSI shell loaded, you will need to: From 820c2d462fc5d5f9141ca0a7c2d5742230f802f2 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Mon, 27 May 2024 14:13:40 +0200 Subject: [PATCH 05/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index c18542587..c0cb64aa6 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -78,7 +78,7 @@ It is expected that you will have loaded all of your required dependencies as mo * Compile and make sure the library resolution points to the EESSI stack. For this, `ldd` from compatibility layer and **not** `/usr/bin/ldd` should be used when checking the binary. * Once done, exit the EESSI prefix shell (with `exit`) or use another terminal to run your software. -For runtime, as EESSI does not set `LD_LIBRARY_PATH`, the executable will need help finding the proper libraries. The easiest approach is to set this variable manually before running. Thus, assuming a new terminal with no EESSI shell loaded, you will need to: +EESSI does not set `LD_LIBRARY_PATH`, so at runtime the executable will need help finding the proper libraries that it needs. The easiest approach is to set this variable manually before running your compiled application. Thus, assuming a new terminal with no EESSI shell loaded, you will need to: * Load dependencies for your software to run * Set manually `LD_LIBRARY_PATH` to resolve libraries at runtime From 1a66170fc16948c5a51f1a03ab62fb13f21f5e04 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Mon, 27 May 2024 14:13:51 +0200 Subject: [PATCH 06/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index c0cb64aa6..09d7b0485 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -87,7 +87,7 @@ export LD_LIBRARY_PATH=$LIBRARY_PATH:$EBROOTGCCcore/lib64 ``` * Run! -Note how this has to be done every time you want to run your software. A long term approach is to set add any library used by your software to the RPATH in your executable. This is also more reliable as your executable won't pick unexpected libraries from LD_LIBRARY_PATH. +Note how this has to be done every time you want to run your software. A long term approach is to set add any library used by your software to the RPATH in your executable. This is also more reliable as your executable won't pick unexpected libraries from `LD_LIBRARY_PATH`. !!! Note No RPATH should point to a compatibility layer directory, only to software layer ones, as all resolving is done via `ld-linux*.so`. From d7edeaa29b3d26c2084ee1dac6168244ed95be18 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Mon, 27 May 2024 14:14:04 +0200 Subject: [PATCH 07/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 09d7b0485..45fe9d6ae 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -89,7 +89,7 @@ export LD_LIBRARY_PATH=$LIBRARY_PATH:$EBROOTGCCcore/lib64 Note how this has to be done every time you want to run your software. A long term approach is to set add any library used by your software to the RPATH in your executable. This is also more reliable as your executable won't pick unexpected libraries from `LD_LIBRARY_PATH`. -!!! Note No RPATH should point to a compatibility layer directory, only to software layer ones, as all resolving is done via `ld-linux*.so`. +!!! Note RPATH should never point to a compatibility layer directory, only to software layer ones, as all resolving is done via the runtime linker (`ld-linux*.so`) that is shipped with EESSI, which automatically searches these locations. The biggest downside of this approach is that your executable becomes bound to the architecture you linked your libraries for. I. e, if you add to your executable RPATH a `libhdf5.so`compiled for `intel_avx512`, you will not be able to run that binary in a machine with a different architecture. From 99ba8759cfbc64e3ddbc2da5b480318dcfc4c298 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Mon, 27 May 2024 14:14:32 +0200 Subject: [PATCH 08/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 45fe9d6ae..7a2b40295 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -91,7 +91,7 @@ Note how this has to be done every time you want to run your software. A long te !!! Note RPATH should never point to a compatibility layer directory, only to software layer ones, as all resolving is done via the runtime linker (`ld-linux*.so`) that is shipped with EESSI, which automatically searches these locations. -The biggest downside of this approach is that your executable becomes bound to the architecture you linked your libraries for. I. e, if you add to your executable RPATH a `libhdf5.so`compiled for `intel_avx512`, you will not be able to run that binary in a machine with a different architecture. +The biggest downside of this approach is that your executable becomes bound to the architecture you linked your libraries for, i.e., if you add to your executable RPATH a `libhdf5.so`compiled for `intel_avx512`, you will not be able to run that binary in a machine with a different architecture. If this is an issue for you, you should look into how EESSI itself organises the location of binaries and perhaps leverage the relevant environment variables (e.g., `EESSI_SOFTWARE_SUBDIR`). !!! Note This documentation is intended for experts in runtime linker/runtime linker behaviour and most cases are untested. Any feedback on this topic is highly appreciated. From dcb85085cc6e1e4ffb76e22e1889a5d8935f2e82 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Mon, 27 May 2024 14:16:47 +0200 Subject: [PATCH 09/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 7a2b40295..5dcc3cd9c 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -85,6 +85,7 @@ EESSI does not set `LD_LIBRARY_PATH`, so at runtime the executable will need hel ```sh export LD_LIBRARY_PATH=$LIBRARY_PATH:$EBROOTGCCcore/lib64 ``` + With this approach we leverage the fact that `LIBRARY_PATH` should contain all the paths we need, and we also need to include the path to `libstdc++` from our GCC installation * Run! Note how this has to be done every time you want to run your software. A long term approach is to set add any library used by your software to the RPATH in your executable. This is also more reliable as your executable won't pick unexpected libraries from `LD_LIBRARY_PATH`. From b06e3d5373ed3beab65913ff56ffc28cac74da98 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Mon, 27 May 2024 14:28:53 +0200 Subject: [PATCH 10/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 5dcc3cd9c..498586b71 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -69,6 +69,18 @@ module load netCDF/4.9.0-gompi-2022a ## Manually building software op top of EESSI (without EasyBuild) +!!! warning + + We are working on a module file that should make building on top of EESSI (without using EasyBuild) + more straightforward, particularly when using `Autotools` or `CMake`. Right now, it is a little convoluted + and requires you to have a decent grasp of + * What a runtime linker (`ld-linux*.so`) is and does + * How to influence the behaviour of the runtime linker with `LD_LIBRARY_PATH` + * The difference between `LIBRARY_PATH` and `LD_LIBRARY_PATH` + + As such, this documentation is intended for "experts" in the runtime linker and it's behaviour, + and most cases are untested. Any feedback on this topic is highly appreciated. + Building and running software on top of EESSI without EasyBuild is not as straightforward and requires some considerations to take care of. It is expected that you will have loaded all of your required dependencies as modules from the EESSI environment. Since EESSI sets `LIBRARY_PATH` for all of the modules and the `GCC` compiler is configured to use the compat layer, there should be no additional configuration required to execute a standard build process. The safest way to make sure all libraries will point to the required locations (and do not leak in from the host operating system) is starting an EESSI prefix shell before building. To do this: From bd837944defe82363836562a35fb46db9d9c1489 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Mon, 27 May 2024 14:28:58 +0200 Subject: [PATCH 11/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 498586b71..805b0dad6 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -106,6 +106,3 @@ Note how this has to be done every time you want to run your software. A long te The biggest downside of this approach is that your executable becomes bound to the architecture you linked your libraries for, i.e., if you add to your executable RPATH a `libhdf5.so`compiled for `intel_avx512`, you will not be able to run that binary in a machine with a different architecture. If this is an issue for you, you should look into how EESSI itself organises the location of binaries and perhaps leverage the relevant environment variables (e.g., `EESSI_SOFTWARE_SUBDIR`). -!!! Note This documentation is intended for experts in runtime linker/runtime linker behaviour and most cases are untested. Any feedback on this topic is highly appreciated. - - From ed0c5dd839e380f1ba4188d7ccb6c2ea48f37b0f Mon Sep 17 00:00:00 2001 From: torri Date: Mon, 17 Jun 2024 17:09:56 +0200 Subject: [PATCH 12/19] updating on LD_RUN_PATH --- docs/using_eessi/building_on_eessi.md | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 805b0dad6..5d8377f8c 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -83,24 +83,21 @@ module load netCDF/4.9.0-gompi-2022a Building and running software on top of EESSI without EasyBuild is not as straightforward and requires some considerations to take care of. -It is expected that you will have loaded all of your required dependencies as modules from the EESSI environment. Since EESSI sets `LIBRARY_PATH` for all of the modules and the `GCC` compiler is configured to use the compat layer, there should be no additional configuration required to execute a standard build process. The safest way to make sure all libraries will point to the required locations (and do not leak in from the host operating system) is starting an EESSI prefix shell before building. To do this: +It is expected that you will have loaded all of your required dependencies as modules from the EESSI environment. Since EESSI sets `LIBRARY_PATH` for all of the modules and the `GCC` compiler is configured to use the compat layer, there should be no additional configuration required to execute a standard build process. On the other hand, EESSI does not set `LD_LIBRARY_PATH`, so at runtime, the executable will need help finding the proper libraries that it needs. The easiest way to circumvent this is setting the environment variable `LD_RUN_PATH` during compile time as well. Thus, the dynamic linker will search for those paths when the program is executed. + +The safest way to make sure all libraries will point to the required locations (and do not leak in from the host operating system) is starting an EESSI prefix shell before building. To do this: * First of all, load the environment by starting an EESSI shell as described [here](https://www.eessi.io/docs/using_eessi/setting_up_environment). * Load all dependencies you need to build your software. You must use at least a toolchain from EESSI to compile it (`foss` is a good option as it will also include MPI with OpenMPI and math libraries via FlexiBLAS/FFTW). +* Set manually `LD_RUN_PATH` to resolve libraries at runtime. `LIBRARY_PATH` should contain all the paths we need, and we also need to include the path to `libstdc++` from our GCC installation +```sh +export LD_RUN_PATH=$LIBRARY_PATH:$EBROOTGCCcore/lib64 +``` * Compile and make sure the library resolution points to the EESSI stack. For this, `ldd` from compatibility layer and **not** `/usr/bin/ldd` should be used when checking the binary. * Once done, exit the EESSI prefix shell (with `exit`) or use another terminal to run your software. -EESSI does not set `LD_LIBRARY_PATH`, so at runtime the executable will need help finding the proper libraries that it needs. The easiest approach is to set this variable manually before running your compiled application. Thus, assuming a new terminal with no EESSI shell loaded, you will need to: - -* Load dependencies for your software to run -* Set manually `LD_LIBRARY_PATH` to resolve libraries at runtime -```sh -export LD_LIBRARY_PATH=$LIBRARY_PATH:$EBROOTGCCcore/lib64 -``` - With this approach we leverage the fact that `LIBRARY_PATH` should contain all the paths we need, and we also need to include the path to `libstdc++` from our GCC installation * Run! -Note how this has to be done every time you want to run your software. A long term approach is to set add any library used by your software to the RPATH in your executable. This is also more reliable as your executable won't pick unexpected libraries from `LD_LIBRARY_PATH`. !!! Note RPATH should never point to a compatibility layer directory, only to software layer ones, as all resolving is done via the runtime linker (`ld-linux*.so`) that is shipped with EESSI, which automatically searches these locations. From f1fd3416f1c44834e3b15dee6e6ae6986685eff0 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Mon, 30 Sep 2024 16:24:57 +0200 Subject: [PATCH 13/19] tab indent fix --- docs/using_eessi/building_on_eessi.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 5d8377f8c..9cd903988 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -90,9 +90,9 @@ The safest way to make sure all libraries will point to the required locations ( * First of all, load the environment by starting an EESSI shell as described [here](https://www.eessi.io/docs/using_eessi/setting_up_environment). * Load all dependencies you need to build your software. You must use at least a toolchain from EESSI to compile it (`foss` is a good option as it will also include MPI with OpenMPI and math libraries via FlexiBLAS/FFTW). * Set manually `LD_RUN_PATH` to resolve libraries at runtime. `LIBRARY_PATH` should contain all the paths we need, and we also need to include the path to `libstdc++` from our GCC installation -```sh -export LD_RUN_PATH=$LIBRARY_PATH:$EBROOTGCCcore/lib64 -``` + ```sh + export LD_RUN_PATH=$LIBRARY_PATH:$EBROOTGCCcore/lib64 + ``` * Compile and make sure the library resolution points to the EESSI stack. For this, `ldd` from compatibility layer and **not** `/usr/bin/ldd` should be used when checking the binary. * Once done, exit the EESSI prefix shell (with `exit`) or use another terminal to run your software. From f70fff407ddee7d3a85ce93bafa7fc93a7f42993 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Thu, 10 Oct 2024 08:33:10 +0200 Subject: [PATCH 14/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 9cd903988..4b5319b4c 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -74,7 +74,7 @@ module load netCDF/4.9.0-gompi-2022a We are working on a module file that should make building on top of EESSI (without using EasyBuild) more straightforward, particularly when using `Autotools` or `CMake`. Right now, it is a little convoluted and requires you to have a decent grasp of - * What a runtime linker (`ld-linux*.so`) is and does + * What a runtime dynamic linker (`ld-linux*.so`) is and does * How to influence the behaviour of the runtime linker with `LD_LIBRARY_PATH` * The difference between `LIBRARY_PATH` and `LD_LIBRARY_PATH` From 1b7f1eb0a3a31f5d73618f758e410ebf6889c6d8 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Thu, 10 Oct 2024 08:33:30 +0200 Subject: [PATCH 15/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 4b5319b4c..5aa571ef1 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -81,7 +81,7 @@ module load netCDF/4.9.0-gompi-2022a As such, this documentation is intended for "experts" in the runtime linker and it's behaviour, and most cases are untested. Any feedback on this topic is highly appreciated. -Building and running software on top of EESSI without EasyBuild is not as straightforward and requires some considerations to take care of. +Building and running software on top of EESSI without EasyBuild is not straightforward and requires some considerations to take care of. It is expected that you will have loaded all of your required dependencies as modules from the EESSI environment. Since EESSI sets `LIBRARY_PATH` for all of the modules and the `GCC` compiler is configured to use the compat layer, there should be no additional configuration required to execute a standard build process. On the other hand, EESSI does not set `LD_LIBRARY_PATH`, so at runtime, the executable will need help finding the proper libraries that it needs. The easiest way to circumvent this is setting the environment variable `LD_RUN_PATH` during compile time as well. Thus, the dynamic linker will search for those paths when the program is executed. From 6523accb789f462783506632130ecb06c18ba318 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Thu, 10 Oct 2024 08:45:25 +0200 Subject: [PATCH 16/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 5aa571ef1..753110924 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -83,7 +83,7 @@ module load netCDF/4.9.0-gompi-2022a Building and running software on top of EESSI without EasyBuild is not straightforward and requires some considerations to take care of. -It is expected that you will have loaded all of your required dependencies as modules from the EESSI environment. Since EESSI sets `LIBRARY_PATH` for all of the modules and the `GCC` compiler is configured to use the compat layer, there should be no additional configuration required to execute a standard build process. On the other hand, EESSI does not set `LD_LIBRARY_PATH`, so at runtime, the executable will need help finding the proper libraries that it needs. The easiest way to circumvent this is setting the environment variable `LD_RUN_PATH` during compile time as well. Thus, the dynamic linker will search for those paths when the program is executed. +It is expected that you will have loaded all of your required dependencies as modules from the EESSI environment. Since EESSI sets `LIBRARY_PATH` for all of the modules and the `GCC` compiler is configured to use the compat layer, there should be no additional configuration required to execute a standard build process. On the other hand, EESSI does not set `LD_LIBRARY_PATH` so, _at runtime_, the executable will need help finding the libraries that it needs to actually execute. The easiest way to circumvent this requirement is by setting the environment variable `LD_RUN_PATH` during compile time as well. With `LD_RUN_PATH` set, the program will be able to tell the dynamic linker to search in those paths when the program is being executed. The safest way to make sure all libraries will point to the required locations (and do not leak in from the host operating system) is starting an EESSI prefix shell before building. To do this: From 6a98870b0b41c36a5fc0274e1ec7355e0a1556b6 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Thu, 10 Oct 2024 08:47:16 +0200 Subject: [PATCH 17/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 753110924..daa604e48 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -85,7 +85,7 @@ Building and running software on top of EESSI without EasyBuild is not straightf It is expected that you will have loaded all of your required dependencies as modules from the EESSI environment. Since EESSI sets `LIBRARY_PATH` for all of the modules and the `GCC` compiler is configured to use the compat layer, there should be no additional configuration required to execute a standard build process. On the other hand, EESSI does not set `LD_LIBRARY_PATH` so, _at runtime_, the executable will need help finding the libraries that it needs to actually execute. The easiest way to circumvent this requirement is by setting the environment variable `LD_RUN_PATH` during compile time as well. With `LD_RUN_PATH` set, the program will be able to tell the dynamic linker to search in those paths when the program is being executed. -The safest way to make sure all libraries will point to the required locations (and do not leak in from the host operating system) is starting an EESSI prefix shell before building. To do this: +EESSI uses a [compatibility layer](../compatibility_layer.md) to ensure that it takes as few libraries from the host as possible. The safest way to make sure all libraries will point to the required locations in the compatibility layer (and do not leak in from the host operating system) is starting an EESSI prefix shell before building. To do this: * First of all, load the environment by starting an EESSI shell as described [here](https://www.eessi.io/docs/using_eessi/setting_up_environment). * Load all dependencies you need to build your software. You must use at least a toolchain from EESSI to compile it (`foss` is a good option as it will also include MPI with OpenMPI and math libraries via FlexiBLAS/FFTW). From 08d9aa5122f9ca735eb93d28bad43ceb8b7ddd4f Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Thu, 10 Oct 2024 09:03:01 +0200 Subject: [PATCH 18/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index daa604e48..83fa6d7f0 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -101,5 +101,5 @@ EESSI uses a [compatibility layer](../compatibility_layer.md) to ensure that it !!! Note RPATH should never point to a compatibility layer directory, only to software layer ones, as all resolving is done via the runtime linker (`ld-linux*.so`) that is shipped with EESSI, which automatically searches these locations. -The biggest downside of this approach is that your executable becomes bound to the architecture you linked your libraries for, i.e., if you add to your executable RPATH a `libhdf5.so`compiled for `intel_avx512`, you will not be able to run that binary in a machine with a different architecture. If this is an issue for you, you should look into how EESSI itself organises the location of binaries and perhaps leverage the relevant environment variables (e.g., `EESSI_SOFTWARE_SUBDIR`). +The biggest downside of this approach is that your executable becomes bound to the architecture you linked your libraries for, i.e., if you add to your executable RPATH a `libhdf5.so`compiled for `intel_avx512`, you will not be able to run that binary on a machine with a different architecture. If this is an issue for you, you should look into how EESSI itself organises the location of binaries and perhaps leverage the relevant environment variables (e.g., `EESSI_SOFTWARE_SUBDIR`). From 3db1ac65143c02159ebe8d918cbeb085e549e9c8 Mon Sep 17 00:00:00 2001 From: Helena Vela Beltran <47674829+hvelab@users.noreply.github.com> Date: Thu, 10 Oct 2024 09:03:24 +0200 Subject: [PATCH 19/19] Update docs/using_eessi/building_on_eessi.md Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 83fa6d7f0..957f64d3a 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -89,7 +89,7 @@ EESSI uses a [compatibility layer](../compatibility_layer.md) to ensure that it * First of all, load the environment by starting an EESSI shell as described [here](https://www.eessi.io/docs/using_eessi/setting_up_environment). * Load all dependencies you need to build your software. You must use at least a toolchain from EESSI to compile it (`foss` is a good option as it will also include MPI with OpenMPI and math libraries via FlexiBLAS/FFTW). -* Set manually `LD_RUN_PATH` to resolve libraries at runtime. `LIBRARY_PATH` should contain all the paths we need, and we also need to include the path to `libstdc++` from our GCC installation +* Set manually `LD_RUN_PATH` to resolve libraries at runtime. `LIBRARY_PATH` should contain all the paths we need, and we also need to include the path to `libstdc++` from our GCC installation to avoid picking up the one from the host: ```sh export LD_RUN_PATH=$LIBRARY_PATH:$EBROOTGCCcore/lib64 ```