Adding documentation on how to build and run on top of EESSI without EB

docs/using_eessi/building_on_eessi.md

@@ -67,7 +67,39 @@ Finally, you should be able to load our newly build module:
 module load netCDF/4.9.0-gompi-2022a
 ```
 
-## Manually building software op top of EESSI
-Building software on top of EESSI would require your linker to use the same system-dependencies as the software in EESSI does. In other words: it requires you to link against libraries from the compatibility layer, instead of from your host OS.
+## Manually building software op top of EESSI (without EasyBuild)
+
+!!! warning
+
+    We are working on a module file that should make building on top of EESSI (without using EasyBuild)
+    more straightforward, particularly when using `Autotools` or `CMake`. Right now, it is a little convoluted
+    and requires you to have a decent grasp of
+    * What a runtime dynamic linker (`ld-linux*.so`) is and does
+    * How to influence the behaviour of the runtime linker with `LD_LIBRARY_PATH`
+    * The difference between `LIBRARY_PATH` and `LD_LIBRARY_PATH`
+
+    As such, this documentation is intended for "experts" in the runtime linker and it's behaviour,
+    and most cases are untested. Any feedback on this topic is highly appreciated. 
+
+Building and running software on top of EESSI without EasyBuild is not straightforward and requires some considerations to take care of. 
+
+It is expected that you will have loaded all of your required dependencies as modules from the EESSI environment. Since EESSI sets `LIBRARY_PATH` for all of the modules and the `GCC` compiler is configured to use the compat layer, there should be no additional configuration required to execute a standard build process. On the other hand, EESSI does not set `LD_LIBRARY_PATH` so, _at runtime_, the executable will need help finding the libraries that it needs to actually execute. The easiest way to circumvent this requirement is by setting the environment variable `LD_RUN_PATH` during compile time as well. With `LD_RUN_PATH` set, the program will be able to tell the dynamic linker to search in those paths when the program is being executed. 
+
+EESSI uses a [compatibility layer](../compatibility_layer.md) to ensure that it takes as few libraries from the host as possible. The safest way to make sure all libraries will point to the required locations in the compatibility layer (and do not leak in from the host operating system) is starting an EESSI prefix shell before building. To do this: 
+
+* First of all, load the environment by starting an EESSI shell as described [here](https://www.eessi.io/docs/using_eessi/setting_up_environment). 
+* Load all dependencies you need to build your software. You must use at least a toolchain from EESSI to compile it (`foss` is a good option as it will also include MPI with OpenMPI and math libraries via FlexiBLAS/FFTW). 
+* Set manually `LD_RUN_PATH` to resolve libraries at runtime. `LIBRARY_PATH` should contain all the paths we need, and we also need to include the path to `libstdc++` from our GCC installation to avoid picking up the one from the host:
+    ```sh
+        export LD_RUN_PATH=$LIBRARY_PATH:$EBROOTGCCcore/lib64
+    ```
+* Compile and make sure the library resolution points to the EESSI stack. For this, `ldd` from compatibility layer and **not** `/usr/bin/ldd` should be used when checking the binary.
+* Once done, exit the EESSI prefix shell (with `exit`) or use another terminal to run your software.
+
+* Run! 
+
+
+!!! Note RPATH should never point to a compatibility layer directory, only to software layer ones, as all resolving is done via the runtime linker (`ld-linux*.so`)  that is shipped with EESSI, which automatically searches these locations.
+
+The biggest downside of this approach is that your executable becomes bound to the architecture you linked your libraries for, i.e., if you add to your executable RPATH a `libhdf5.so`compiled for `intel_avx512`, you will not be able to run that binary on a machine with a different architecture. If this is an issue for you, you should look into how EESSI itself organises the location of binaries and perhaps leverage the relevant environment variables (e.g., `EESSI_SOFTWARE_SUBDIR`).
 
-While we plan to support this in the future, manually building on top of EESSI is currently not supported yet in a trivial way.