Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Monitor dependency versions #80

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
148 changes: 94 additions & 54 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,76 +7,114 @@

# Chaste Dependency Modules

Utility scripts for installing Chaste dependencies as Environment Modules.
This repository contains utility scripts for building and installing Chaste's software dependencies as environment modules. It also contains Dockerfiles for building GitHub runner Docker images with specified dependency versions.

## Usage

### 1. Install Environment Modules

[Environment Modules](https://modules.readthedocs.io/) enables switching between software versions by reconfiguring the shell environment.
The [Environment Modules](https://modules.readthedocs.io/) system allows users to switch between different software versions installed on the same system by reconfiguring the shell environment.

Installation on Ubuntu:
See the Environment Modules [documentation](https://modules.readthedocs.io/en/latest/INSTALL.html) for installation instructions on different systems. On Ubuntu, `environment-modules` can be installed from the `apt` repository:

``` bash
apt-get install environment-modules
```

To activate environment modules, close and open a new bash shell, or alternatively run:
```bash
source /etc/profile.d/modules.sh
```
> [!IMPORTANT]
> To activate the Environment Modules system after installation, close the current shell and start a new session. Alternatively, load the activation script directly into the current shell:
> ```bash
> source /etc/profile.d/modules.sh
> ```

See [Installing Modules on Unix](https://modules.readthedocs.io/en/latest/INSTALL.html) for more details.
### 2. Create a modules directory

### 2. Prepare modulefiles location
[Modulefiles](https://modules.readthedocs.io/en/latest/modulefile.html) are recipes used to reconfigure the shell environment for alternative software versions. The `MODULEPATH` environment variable is a list of locations where modulefiles are stored on the system.

[Modulefiles](https://modules.readthedocs.io/en/latest/modulefile.html) are recipes for configuring the shell environment to access specific software versions. Environment Modules uses modulefiles from locations on `MODULEPATH`.
> [!TIP]
> Directories containing modulefiles can be added to `MODULEPATH` using the command `module use <path/to/modulefiles>`.

``` bash
The commands below create a directory for modules and adds it to the `MODULEPATH`.

```sh
# Create a directory for storing modulefiles
MODULES_DIR=${HOME}/modules
mkdir -p ${MODULES_DIR}/modulefiles

# Add the directory to MODULEPATH
module use ${MODULES_DIR}/modulefiles

# Add the directory to MODULEPATH automatically in future bash sessions
echo "module use ${MODULES_DIR}/modulefiles" >> ${HOME}/.bashrc
```

The command `module use directory` prepends `directory` to `MODULEPATH`.

### 3. Install Chaste dependencies

Clone the repository and navigate to the custom build scripts
``` bash
Clone this repository and navigate to the build scripts

```sh
git clone https://github.com/Chaste/dependency-modules.git
cd dependency-modules/scripts/custom
```

Install the dependencies
``` bash
> [!NOTE]
> Running the build scripts will build and install software in this directory structure:
>```
><MODULES_DIR>
>|-- modulefiles/
>|-- opt/
>`-- src/
>```
> Software will be downloaded and built in `src/` and installed in `opt/`.
> A modulefile for each software built will be created in `modulefiles/`.

Install XSD
```sh
./install_xsd.sh --version=4.0.0 --modules-dir=${MODULES_DIR}
```

Install Xerces-C
```sh
./install_xercesc.sh --version=3.2.4 --modules-dir=${MODULES_DIR}
```

Install SUNDIALS
```sh
./install_sundials.sh --version=6.4.0 --modules-dir=${MODULES_DIR}
./install_boost.sh --version=1.83.0 --modules-dir=${MODULES_DIR}
./install_vtk.sh --version=9.3.1 --modules-dir=${MODULES_DIR}
./install_petsc_hdf5.sh --petsc-version=3.19.6 --hdf5-version=1.10.10 \
--petsc-arch=linux-gnu-opt --modules-dir=${MODULES_DIR}
```

The scripts will build and install dependencies following this directory structure:
Install Boost
```sh
./install_boost.sh --version=1.83.0 --modules-dir=${MODULES_DIR}
```

``` bash
<modules-dir>
|-- modulefiles
|-- opt
`-- src
Install VTK
```sh
./install_vtk.sh --version=9.3.1 --modules-dir=${MODULES_DIR}
```

Builds are done from `src`.
Install PETSc + HDF5
```sh
./install_petsc_hdf5.sh --petsc-version=3.19.6 --hdf5-version=1.10.10 --petsc-arch=linux-gnu-opt --modules-dir=${MODULES_DIR}
```

Software versions are installed to `opt`.
> [!TIP]
> After installation empty the `src/` directory as the build files are no longer needed.
> ```sh
> cd ${MODULES_DIR} && rm -rI src/*
> ```

Modulefiles are placed under `modulefiles`.
### 4. Load installed software modules

### 4. Load installed dependencies
Use `module avail` to show available software modules
```
---------------- /home/<user>/modules/modulefiles ----------------
boost/1.83.0 vtk/9.3.1
petsc_hdf5/3.19.6_1.10.810/linux-gnu-opt xercesc/3.2.4
sundials/6.4.0 xsd/4.0.0
```

Use `module load` to activate software modules
``` bash
module load xsd/4.0.0
module load xercesc/3.2.4
Expand All @@ -88,26 +126,28 @@ module load petsc_hdf5/3.19.6_1.10.10/linux-gnu-opt

### 5. Build Chaste

See the [Chaste Guides](https://chaste.github.io/docs/installguides/ubuntu-package/) for detailed instructions on building Chaste.

## Useful commands

`module use directory` enables using modulefiles located in `directory`.

`module load modulefile` loads modulefile into the environment.

`module unload modulefile` unloads modulefile from the environment.

`module switch [modulefile1] modulefile2` switches version to modulefile2.

`module list` lists all currently loaded modulefiles.

`module purge` unloads all currently loaded modulefiles.

`module avail` lists all installed modulefiles.

`module avail string` searches for modulefiles that contain `string`.

`module show modulefile` prints the environment changes prescribed by modulefile.

See the [module command help](https://modules.readthedocs.io/en/latest/module.html) for more details.
Configure and build Chaste as normal following the instructions in the [documentation](https://chaste.github.io/docs/installguides/).

## Module commands

Below is a subset of commonly used `module` commands. See the environment modules [documentation](https://modules.readthedocs.io/en/latest/module.html) for a more comprehensive manual.

| Command |  Description |
| ------------------------------------------- | ----------------------------------------------------------------- |
| `module use <path/to/modulefiles>` | Make software modules located on the specified path available. |
| | |
| `module avail` |  List all available software modules. |
| | |
| `module avail <search_string>` |  Search for software modules that match the search string. |
| | |
| `module load <module>` |  Load a software module into the environment. |
| | |
| `module list` |  List all currently loaded software modules. |
| | |
| `module unload <module>` |  Unload a software module from the environment. |
| | |
| `module purge` |  Unload all currently loaded software modules. |
| | |
| `module switch <module/ver0> <module/ver1>` |  Unload `module/ver0` and load `module/ver1`. |
| | |
| `module show <module>` |  Show the environment settings for a module. |
143 changes: 143 additions & 0 deletions scripts/common.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
#!/bin/bash -eu

# Split version string, setting minor and patch to 0 if missing.
#
# Usage: split_version <version>
#
# Returns: version major minor patch[ rc]
#
# Examples:
# `split_version 1` -> 1.0.0 1 0 0
# `split_version 1.2` -> 1.2.0 1 2 0
# `split_version 1.2.3` -> 1.2.3 1 2 3
# `split_version 1.2.3-rc1` -> 1.2.3-rc1 1 2 3 rc1
split_version()
{
local varr parr major minor=0 patch=0 rc
varr=(${1//\./ }) # split version string on '.'
major=${varr[0]}
if [ ${#varr[@]} -ge 2 ]; then
minor=${varr[1]}
if [ ${#varr[@]} -ge 3 ]; then
parr=(${varr[2]//-/ }) # split patch substring on '-'
patch=${parr[0]}
if [ ${#parr[@]} -ge 2 ]; then
rc=${parr[1]}
fi
fi
fi
if [ -z "${rc}" ]; then
echo "${major}.${minor}.${patch}" "${major}" "${minor}" "${patch}"
else
echo "${major}.${minor}.${patch}-${rc}" "${major}" "${minor}" "${patch}" "${rc}"
fi
}

# Compare two version strings, ignoring release candidate.
#
# Usage: compare_version <version_x> <version_y>
#
# Returns:
# -1 if version_x < version_y
# 0 if version_x == version_y
# 1 if version_x > version_y
#
# Examples:
# `compare_version 1.2.3 1.2.3` -> 0
# `compare_version 1.2.3 1.2.4` -> -1
# `compare_version 1.2.4 1.2.3` -> 1
compare_version()
{
local arr_x maj_x min_x patch_x
local arr_y maj_y min_y patch_y

read -r _ maj_x min_x patch_x _ < <(split_version $1)
read -r _ maj_y min_y patch_y _ < <(split_version $2)

arr_x=("${maj_x}" "${min_x}" "${patch_x}")
arr_y=("${maj_y}" "${min_y}" "${patch_y}")

for i in $(seq 0 2); do
if ((arr_x[i] > arr_y[i])); then
echo 1
return
elif ((arr_x[i] < arr_y[i])); then
echo -1
return
fi
done

echo 0
}

# Check if version_x is equal to version_y.
#
# Usage: version_eq <version_x> <version_y>
#
# Returns: true if version_x == version_y, false otherwise
#
# Examples:
# `version_eq 1.2.3 1.2.3` -> true
# `version_eq 1.2.3 1.2.4` -> false
version_eq()
{
test "$(compare_version $1 $2)" -eq 0
}

# Check if version_x is less than version_y.
#
# Usage: version_lt <version_x> <version_y>
#
# Returns: true if version_x < version_y, false otherwise
#
# Examples:
# `version_lt 1.2.3 1.2.3` -> false
# `version_lt 1.2.3 1.2.4` -> true
version_lt()
{
test "$(compare_version $1 $2)" -eq -1
}

# Check if version_x is greater than version_y.
#
# Usage: version_gt <version_x> <version_y>
#
# Returns: true if version_x > version_y, false otherwise
#
# Examples:
# `version_gt 1.2.3 1.2.3` -> false
# `version_gt 1.2.4 1.2.3` -> true
version_gt()
{
test "$(compare_version $1 $2)" -eq 1
}

# Check if version_x is less than or equal to version_y.
#
# Usage: version_le <version_x> <version_y>
#
# Returns: true if version_x <= version_y, false otherwise
#
# Examples:
# `version_le 1.2.3 1.2.3` -> true
# `version_le 1.2.3 1.2.4` -> true
# `version_le 1.2.4 1.2.3` -> false
version_le()
{
! version_gt $1 $2
}

# Check if version_x is greater than or equal to version_y.
#
# Usage: version_ge <version_x> <version_y>
#
# Returns: true if version_x >= version_y, false otherwise
#
# Examples:
# `version_ge 1.2.3 1.2.3` -> true
# `version_ge 1.2.4 1.2.3` -> true
# `version_ge 1.2.3 1.2.4` -> false
version_ge()
{
! version_lt $1 $2
}
25 changes: 0 additions & 25 deletions scripts/custom/common.sh

This file was deleted.

Loading