Merge pull request easybuilders#273 from boegel/main

update docs for EasyBuild v4.9.3 release

docs/common-toolchains.md

@@ -61,9 +61,8 @@ graph LR
 
 Note: following notes apply for the generations listed and those older than it:
 
-- `2022a` - `iimkl` not present yet
-- `2021b` - `gfbf` not present yet
-- `2020b` - `foss` uses OpenBLAS instead of FlexiBLAS, `iccifort` is used instead of `intel-compilers`
+- <= `2021b` - `gfbf` not present yet
+- <= `2020b` - `foss` uses OpenBLAS instead of FlexiBLAS, `iccifort` is used instead of `intel-compilers`
 
 
 Keep in mind that when creating an Easyconfig, you need to look at what toolchain "level" (e.g. `foss` vs `GCC`) your
@@ -180,6 +179,7 @@ most recent revision of the common toolchains at that time.
 | `2022b` | Dec '22  | 2.39       | 12.2.0 | 4.1.4      | 3.2.1       | 0.3.21     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10 |
 | `2023a` | Jun '23  | 2.40       | 12.3.0 | 4.1.5      | 3.3.1       | 0.3.23     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10 |
 | `2023b` | Dec '23  | 2.40       | 13.2.0 | 4.1.6      | 3.3.1       | 0.3.24     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10 |
+| `2024a` | Aug '24  | 2.42       | 13.3.0 | 5.0.3      | 3.4.4       | 0.3.27     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10 |
 
 ### Component versions in `intel` toolchain {: #common_toolchains_overview_intel }
 
@@ -191,6 +191,7 @@ most recent revision of the common toolchains at that time.
 | `2022b` | Dec '22  | 2.39       | 12.2.0 | 2022.2.1          | 2021.7.1    | 2022.2.1    |
 | `2023a` | Jun '23  | 2.40       | 12.3.0 | 2023.1.0          | 2021.9.1    | 2023.1.0    |
 | `2023b` | Dec '23  | 2.40       | 13.2.0 | 2023.2.1          | 2021.10.1   | 2023.2.0    |
+| `2024a` | Aug '24  | 2.42       | 13.3.0 | 2024.2.0          | 2021.13.0   | 2024.2.0    |
 
 ## Overview of common toolchains (deprecated versions) {: #common_toolchains_overview_deprecated }
 

docs/hooks.md

@@ -357,3 +357,16 @@ def post_run_shell_cmd_hook(cmd, work_dir=None, interactive=None, exit_code=None
         cmd_type = 'interactive' if interactive else 'non-interactive'
         fp.write("%s command '%s' in %s exited with %s - output: %s\n" % (cmd_type, cmd, work_dir, exit_code, output))
 ```
+
+### Adding a hook conditional on EasyBuild version
+
+If an unknown hook is used then EasyBuild will error (see [Available hooks](#available-hooks)). The following example only
+adds the `pre_build_and_install_loop_hook` if the EasyBuild version is `>= "4.8.1"`.
+
+```py
+from easybuild.tools.version import VERSION
+
+if VERSION >= "4.8.1":
+    def pre_build_and_install_loop_hook(ecs, *args, **kwargs):
+        pass
+```

docs/installation.md

@@ -369,13 +369,13 @@ Supported module tools:
 
 Additional notes:
 
-* Tcl(/C) environment-modules requires [Tcl](https://www.tcl.tk/) to be
+* Tcl(/C) environment-modules requires [Tcl](https://www.tcl-lang.org/) to be
   installed (with header files and development libraries)
 * Lmod requires [Lua](https://www.lua.org/) and a couple of non-standard Lua libraries
   (`lua-posix`, `lua-filesystem`) to be available
     * Tcl (`tclsh`) must also be available for Lmod to support module files in `Tcl` syntax
 * a guide to installing Tcl/C environment modules without having root
-  permissions is available at [Installing environment modules without root permissions][installing_env_mod_c].
+  permissions is available at [Installing environment modules without root permissions][installing_env_mod].
 * a guide to installing Lmod without having root permissions is available at
   [Installing Lmod without root permissions][installing_lmod].
 

docs/installing-environment-modules-without-root-permissions.md

@@ -1,25 +1,22 @@
-# Installing environment modules without root permissions {: #installing_env_mod_c }
+# Installing Environment Modules without root permissions {: #installing_env_mod }
 
-This short guide will explain how to install the standard environment
-modules Tcl/C software package without root permissions on a Linux or
+This short guide will explain how to install the standard Environment
+Modules software package without root permissions on a Linux or
 Mac OS X system, together with Tcl on which it depends.
 
 ## Tcl
 
-1. Go to <https://www.tcl.tk> and download the latest Tcl sources. At
-    the time of writing, the latest available Tcl version was 8.5.15,
+1. Go to <https://www.tcl-lang.org> and download the latest Tcl sources. At
+    the time of writing, the latest available Tcl version was 8.6.14,
     which can be downloaded from
-    [here](https://prdownloads.sourceforge.net/tcl/tcl8.5.15-src.tar.gz).
-    The remainder of these commands will assume Tcl v8.5.15 is being
-    installed, you may need to adjust them accordingly. **Note**: Stick
-    to Tcl v8.5.x, don't consider using the more recent v8.6.x or
-    higher, since the environment modules package is not compatible with
-    those Tcl versions.
+    [here](https://prdownloads.sourceforge.net/tcl/tcl8.6.14-src.tar.gz).
+    The remainder of these commands will assume Tcl v8.6.14 is being
+    installed, you may need to adjust them accordingly.
 
 1. Unpack the Tcl source tarball:
 
     ``` shell
-    tar xfvz tcl8.5.15-src.tar.gz
+    tar xfvz tcl8.6.14-src.tar.gz
     ```
 
 1. Pick a location where you will install Tcl. It should be a directory
@@ -30,12 +27,12 @@ Mac OS X system, together with Tcl on which it depends.
     the `configure` script using the `--prefix` option:
 
     ``` shell
-    cd tcl8.5.15/unix
+    cd tcl8.6.14/unix
     ./configure --prefix=$HOME/.local/Tcl
     ```
 
-    If you're building Tcl and environment modules on Mac, you should
-    run `configure` in the `tcl8.5.15/macosx` directory instead.
+    If you're building Tcl and Environment Modules on Mac, you should
+    run `configure` in the `tcl8.6.14/macosx` directory instead.
 
 1. Next, build Tcl using the `make` command. If the system you are
     building on has multiple cores, running make in parallel will speed
@@ -54,38 +51,37 @@ Mac OS X system, together with Tcl on which it depends.
     make install
     ```
 
-**All done!** Now you are ready to build the environment modules
+**All done!** Now you are ready to build the Environment Modules
 package, which requires Tcl.
 
 ## Environment Modules
 
-1. Download the latest source tarball for the environment modules tools
+1. Download the latest source tarball for the Environment Modules tools
     from <https://modules.sourceforge.net>. At the time of writing, the
-    latest available version is 3.2.10 which can be downloaded [from
-    here](https://prdownloads.sourceforge.net/modules/modules-3.2.10.tar.gz).
+    latest available version is 5.4.0 which can be downloaded [from
+    here](https://prdownloads.sourceforge.net/modules/modules-5.4.0.tar.gz).
 
 1. Unpack the downloaded source tarball:
 
     ``` shell
-    tar xfvz modules-3.2.10.tar.gz
+    tar xfvz modules-5.4.0.tar.gz
     ```
 
 1. Configure the build, again use `--prefix` to specify where to
-    install the environment modules tool in the end. If you needed to
+    install the Environment Modules tool in the end. If you needed to
     install Tcl by hand as outlined in the previous section, you'll also
     need to specify where it was installed using the `--with-tcl`
     option:
 
     ``` shell
-    cd modules-3.2.10
+    cd modules-5.4.0
     ./configure --prefix=$HOME/.local/environment-modules --with-tcl=$HOME/.local/Tcl/lib
     ```
 
-1. Build with `make`, consider parallel build if your system is recent
-    enough:
+1. Build with `make`:
 
     ``` shell
-    make -j 4
+    make
     ```
 
 1. Install:
@@ -98,17 +94,17 @@ Alright, now just one more thing...
 
 ## Set up your environment
 
-Because you've installed environment modules and Tcl in a non-default
+Because you've installed Environment Modules and Tcl in a non-default
 location, you need to make sure your environment is setup up correctly
 to use them.
 
 To make a long story short, these are the commands you need to execute:
 
 ``` shell
-export PATH=$HOME/.local/environment-modules/Modules/3.2.10/bin:$PATH
+export PATH=$HOME/.local/environment-modules/bin:$PATH
 export LD_LIBRARY_PATH=$HOME/.local/Tcl/lib:$LD_LIBRARY_PATH
 # adjust line below if you're using a shell other than bash, check with 'echo $SHELL'
-source $HOME/.local/environment-modules/Modules/3.2.10/init/bash
+source $HOME/.local/environment-modules/init/bash
 ```
 
 !!! tip

docs/patch-files.md

@@ -0,0 +1,109 @@
+# Patch files
+
+## What are patch files (patches)?
+
+A patch file, or just *patch* for short, is a text file that can be used to modify an existing software sources.
+They are often used to fix bugs, but can also be used to improve the usability or performance.
+
+## Including patches in an easyconfig file
+
+Patches can be provided via the `patches` easyconfig parameter (list). A
+patch can be defined as:
+
+- a `string`,
+- a `tuple` of 2 elements, or
+- a `dict`
+
+The most straight-forward use-case is `string`, which contains the name
+of the patch file (and must have `.patch` extension).
+
+A `tuple` adds the possibility to specify the subdirectory in which the patch should be applied.
+This is mostly needed if a patch file adds new files or it is generally
+not possible to determine the appropriate directory to apply the patch in.
+The first element of this tuple is
+the name of patch file. The second element is either the *patch level* (as an integer value)
+which is used in the patch command (`patch -p<level>`), or a directory relative to the unpacked source directory.
+
+!!! note
+    `tuple` also has a special use case if the patch file has any other extension than `.patch`.
+    In this case, the first tuple element is the name of a file which should be
+    copied to the unpacked source directory, and the second element is
+    the target path, where the files should be copied to (relative to
+    the unpacked source directory). See below for an example of this use case.
+
+A `dict` value can be used pass additional arguments to the `patch` command.
+For example, the `--binary` flag may be needed for patch files
+with CRLF endings. For a `dict` value the `filename` key is required.
+`level` and `opts` are supported (optional) keys.
+
+!!! note
+    Specifying only `filename` in `dict` is the same as using a plain `string` definition.
+    Specifying `filename` and `level` is same as using a `tuple`
+    definition.
+
+Example:
+
+```python
+patches = [
+  # a simple patch file
+  'example.patch',
+
+  # when creating only new files by patch file, you need to specify the level:
+  ('example-add-files.patch', 1),
+
+  # apply the patch in a (sub-)subdirectory inside the source tree
+  ('example-subfolder.patch', 'src/subfolder'),
+
+  # copy file to target_path folder
+  ('Makefile', 'path/to/target/'),
+
+  # specify patch file and optionally level and opts for patch command
+  {'filename': 'example.patch', 'level': 1, 'opts': '--binary'}
+]
+```
+
+## Creating a patch file
+
+To create a patch file to use with EasyBuild, follow the steps below.
+
+### 1) Copy unpacked sources
+
+Start by copying the unpacked sources, before making any changes, so we can determine which changes were made:
+
+```bash
+cp -a unpacked_sources unpacked_sources.orig
+
+```
+
+### 2) Make necessary changes
+
+Make the necessary changes in the `unpacked_sources` directory.
+
+### 3) Create patch file
+
+Create the patch file by comparing the original sources with the modified sources using the `diff` command:
+
+```bash
+diff -ruN unpacked_sources.orig unpacked_sources > example.patch
+```
+
+The `-ruN` options here are important:
+
+- `r` instructs the `diff` command to look for changes between the specified directories *recursively*;
+- `u` indicates that `diff` should produce a *unified* diff as output;
+- `N` indicates that files only found in one directory should be treated as new files;
+
+### 4) Add description, author, reference
+
+Please also add a description on top of the patch file, along with the author and/or reference to the source of the patch.
+
+For example:
+
+```diff
+example of a patch description
+author: John Doe (Earth University)
+diff -ru unpacked_sources.orig/example.c unpacked_sources/example.c
+--- unpacked_sources.orig/example.c	2011-08-05 05:08:07.000000000 +0200
++++ unpacked_sources/example.c	2024-03-14 12:31:16.218067000 +0100
+...
+```