Skip to content

Latest commit

 

History

History
executable file
·
167 lines (114 loc) · 6.97 KB

README.md

File metadata and controls

executable file
·
167 lines (114 loc) · 6.97 KB
Raw

JHPCE module config

This repository contains the Lmod module configuration files (.lua) for modules contributed to JHPCE mostly by members of the R/Bioconductor-powered Team Data Science at the Lieber Institute for Brain Development. Though other people from LIBD can contribute to these modules and request to be added to the lieber_modules user group.

The source files are hosted in the jhpce_module_source GitHub repository.

Questions

If you have any questions, please send an email to the JHPCE bithelp mailing list.

Usage

To use these modules, you have to tell Lmod module where this files are listed with the module use command. Then you can run the module avail command to see all the list of modules.

module use /jhpce/shared/libd/core
module avail

Example

As an example, if we run module avail first we don't see the LIBD modules listed.

$ module avail
--------------------------------- /jhpce/shared/jhpce/modulefiles --------------
(output deleted)

  Where:
   L:  Module is loaded

If the avail list is too long consider trying:

"module --default avail" or "ml -d av" to just list the default modules.
"module overview" or "ml ov" to display the number of modules for each name.

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

Then we run the commands listed above and we notice that we have a module for HISAT2 version 2.0.4 listed.

$ module use /jhpce/shared/libd/modulefiles
$ module avail

--------------------- /jhpce/shared/libd/modulefiles ---------------------------
   hisat2/2.2.1

Without loading the module, we don't see HISAT2 on the $PATH environment variable.

$ which hisat2
/usr/bin/which: no hisat2 in (deleted output)

If we load the module, then we find HISAT2 in the $PATH and we can start using it.

$ module use /jhpce/shared/libd/modulefiles
$ module load hisat2/2.2.1
$ which hisat2
/jhpce/shared/libd/core/hisat2/2.2.1/hisat2-2.2.1/hisat2
$ hisat2 --version
/jhpce/shared/libd/core/hisat2/2.2.1/hisat2-2.2.1/hisat2-align-s version 2.2.1
64-bit
Built on compute-094.cm.cluster
Thu Jun 29 02:28:02 PM EDT 2023
Compiler: gcc version 11.3.1 20221121 (Red Hat 11.3.1-4) (GCC) 
Options: -O3 -m64 -msse2 -funroll-loops -g3 -DPOPCNT_CAPABILITY -std=c++11
Sizeof {int, long, long long, void*, size_t, off_t}: {4, 8, 8, 8, 8, 8}

Recurrent usage

If you want to always include our modules, you can edit your .bashrc file and add the following lines:

## List the LIBD modules by default when inside srun/sbatch
if [[ $HOSTNAME == compute-* ]] || [[ $HOSTNAME == transfer-* ]]; then
    echo "Adding LIBD modules"
    module use /jhpce/shared/libd/modulefiles
fi

Be cautious about export statements in your .bashrc! In particular, setting variables like PATH, R_HOME, LD_LIBRARY_PATH, and other path-related environment variables can interfere with the functioning of modules or cause other issues that other users won't be able to replicate!

Create a new module

Step 1. Create a directory in lowercase with the name of the source you are installing. In this example, we are creating a module for HISAT2 version 2.2.1 that was installed at jhpce_module_source/tree/SLURM/hisat2/2.2.1.

mkdir hisat2

Step 2. Create a .lua configuration file with the version number of the software you are installing. In this example, that would be 2.2.1.lua.

touch hisat2/2.2.1.lua

To jump start this process, you can simply copy one of the existing .lua files. For example, you could copy the .lua file for the hisat2/2.2.1 module.

## You could use the HISAT2 v2.0.4 module file for future use
cp /jhpce/shared/libd/modulefiles/hisat2/2.2.1.lua your_software/your_version.lua

Step 3. Edit the .lua file you just created and test it. As an example, check the hisat2/2.2.1.lua file.

Step 4. Make sure that the permissions are set correctly at the end.

## Use 7 for group: so Nick and Leo can edit the files
chmod 775 -R hisat2

lieber_modules membership rules

To be added to lieber_modules JHPCE user group you have to agree to:

  1. Be subscribed to the bithelp mailing list.
  2. Respond to questions from users who have questions about modules you contributed.
  3. Be willing to create modules for new versions of the software or help advice new maintainers if you are passing down the torch to someone new.

If you do not agree to these rules, worry not! We will still gladly train and guide you to make your own private modules with files hosted elsewhere. Please sign up for a Data Science guidance session with Nick Eagles after having watched the relevant module-related videos from the LIBD rstats club - JHPCE playlist.

As noted by Mark Miller from JHPCE BitSupport, using shared modules has some advantages, such as:

  • It prevents duplication of effort, where 2 people on 2 different teams need the same software and build the same duplicate module.
  • It allows for the effort put into building the module to be used and appreciated by others.
  • Having a module used by more people will help to encourage the maintainer to keep the module up to date.
  • We do have the annual JHPCE High Achiever award that has been awarded to those that have contributed modules. 😊

To be added to lieber_modules, please send an email to BitHelp mentioning that you agree to the rules stated above and cc Leonardo Collado Torres.

Welcome to the club! We are glad to have you! 🙌🏽

PS This whole process is inspired by the rules you have to agree to when submitting an R package to Bioconductor.

Contributors

Current

Former

JHPCE location

/jhpce/shared/libd/modulefiles

Useful aliases for your ~/.bashrc file:

## Creating modules
# https://lmod.readthedocs.io/en/latest/050_lua_modulefiles.html
alias modsrc="cd /jhpce/shared/libd/core"
alias modlua="cd /jhpce/shared/libd/modulefiles"