User Tools

Site Tools


cluster-lbt:environment-modules

Environment-Modules

In general, when you are using UNIX/GNU-Linux in a production environment, it's not unusually that you need other releases of software products or libraries than pre-installed ones. One method, after having compiled it (and installed wherever you want), is to surcharge your own environment re-SETing you environment variables directly in your own .bash_profile file; but what happens when you have compiled several releases of these and wish to switch between? The good method is, I guess, to use Environment-Modules letting you load and unload what you need, avoiding any conflict, etc..

Modules list display

$ module avail [<module-name>]

$ module avail

--------------------------------------------------------------------------- /shared/modulefiles/compilers ---------------------------------------------------------------------------
intel/12.0.5               mvapich2/gnu/1.9           mvapich2/gnu/2.0b_32b      openmpi/gnu/1.6.5          openmpi/intel/1.6.5        openmpi/intel_12.0.5/1.6.5
intel/14.0                 mvapich2/gnu/1.9_gpu       openmpi/gnu/1.4.4          openmpi/gnu/1.8.4          openmpi/intel/1.8.4        python/gnu/2.7.5
mpiexec/gnu/0.84           mvapich2/gnu/2.0b          openmpi/gnu/1.5.4_centos   openmpi/gnu/1.8.4_gpu      openmpi/intel/1.8.4_gpu

--------------------------------------------------------------------------- /shared/modulefiles/libraries ---------------------------------------------------------------------------
atlas/gnu/3.10.1  blas/gnu          boost/gnu/1.57.0  cblas/gnu         fftw/gnu/3.3.3    fftw/gnu/3.3.4    fftw/intel/3.3.4  lapack/gnu/3.4.2  libxml2/gnu/2.7.8 openblas/gnu

--------------------------------------------------------------------------- /shared/modulefiles/softwares ---------------------------------------------------------------------------
amber/gnu/12                   gromacs/gnu/4.6.3              gromacs/intel/5.0.4_intelnodes namd2/intel/2.10_mc_gpu        sassena/gnu/1.4.1
amber/gnu/14                   gromacs/gnu/4.6.3_gpu          namd2/gnu/2.8                  namd2/intel/2.9                vmd/gnu/1.9.2
amber/gnu/14_gpu               gromacs/gnu/4.6.3_intelnodes   namd2/gnu/2.9                  namd2/intel/2.9_gpu
amber/intel/14_gpu             gromacs/gnu/5.0.4              namd2/gnu/2.9_gpu              namd2/multicore
amber/intel/14_mc_gpu          gromacs/gnu/5.0.4_gpu          namd2/intel/2.10_gpu           R/gnu/2.14.2
gromacs/gnu/4.6.1              gromacs/intel/5.0.4_gpu        namd2/intel/2.10_mc            R/gnu/3.0.1

--------------------------------------------------------------------------- /shared/modulefiles/utilities ---------------------------------------------------------------------------
bonnie++/gnu/1.03e    ddt/gnu/8             emacs/gnu/24.3        hdf5/gnu/1.8.14       hwloc/gnu/1.7.1       iperf/gnu/3.0.3       tar/gnu/1.28
cmake/gnu/2.8.11.2    ddt/gnu/9             fpart/gnu/0.9         HPL-Linpack/gnu/2.1   iozone/gnu/3          parallel/gnu/20140822 torque/gnu/2.5.12
$ module avail gromacs

--------------------------------------------------------------------------- /shared/modulefiles/softwares ---------------------------------------------------------------------------
gromacs/gnu/4.6.1              gromacs/gnu/4.6.3_gpu          gromacs/gnu/5.0.4              gromacs/intel/5.0.4_gpu
gromacs/gnu/4.6.3              gromacs/gnu/4.6.3_intelnodes   gromacs/gnu/5.0.4_gpu          gromacs/intel/5.0.4_intelnodes

Module loading

$ module load <module-name>

$ module load gromacs/intel/5.0.4_gpu 
Loads the gromacs-5.0.4 compiled using intel-14.0.
You should also run these following shell commands:
	module load  intel/14.0
	module load  openmpi/intel/1.8.4

Note: As you can note in returns, you may need to load some additional modules by yourself to satisfy dependencies.

Module unloading

$ module unload <module-name>

$ module unload gromacs/intel/5.0.4_gpu 

Loaded modules display

$ module list

$ module list
Currently Loaded Modulefiles:
  1) torque/gnu/2.5.12         2) openmpi/gnu/1.6.5

Module information display

$ module show [<module-name>]

$ module show openmpi/gnu/1.6.5
-------------------------------------------------------------------
/shared/modulefiles/compilers/openmpi/gnu/1.6.5:

conflict	 current/intel current/openmpi/intel/1.4.3 
module-whatis	 Loads the openmpi-1.6.5 compiled using gnu-4.4.6 
conflict	 openmpi 
prepend-path	 MANPATH /shared/compilers/openmpi/1.6.5/gnu/share/man 
prepend-path	 PATH /shared/compilers/openmpi/1.6.5/gnu/bin 
prepend-path	 LD_LIBRARY_PATH /shared/compilers/openmpi/1.6.5/gnu/lib 
-------------------------------------------------------------------

Module purging

$ module purge

$ module list
Currently Loaded Modulefiles:
  1) torque/gnu/2.5.12   2) openmpi/gnu/1.6.5
$ module purge
$ module list
No Modulefiles Currently Loaded.

Module replacement

$ module switch <new module-name>

$ module list
Currently Loaded Modulefiles:
  1) torque/gnu/2.5.12   2) openmpi/gnu/1.6.5
$ module switch openmpi/gnu/1.8.4
$ module list
Currently Loaded Modulefiles:
  1) torque/gnu/2.5.12   2) openmpi/gnu/1.8.4

Due to our modules organisation, you can only switch modules between versions, not between compilers.

How to create your own modules

In order to create your own modulefiles, you need to create a new directory tree available from everywhere (example: /workdir/<you own directory>/.modules) and add this line into your .bash_profile file:

$ echo "export MODULEPATH=<your module directory>:${MODULEPATH}" >> <your own workdir>/.bash_profile

Note: here we use a .bash_profile file located in your own workdir since, because /home is not mounted on computing nodes, your homedir because your workdir for computations.

Now you can create in your own module files using this following template:

module.template
proc ModulesHelp { } {
        global mod_desc
	puts stderr "$mod_desc"
}
 
# The following variables will be used to compose
# paths to basic directory and binaries for different
# versions of the software you are installing
 
# app name
set     app_name     <application name>
# app version
set     app_version  <application version>
# compiler used to build the software
set     compiler     <compiler: (gnu|intel)>
# app root
set     app_root     <application path>
 
#####################################################################
# basic help line that will be shown on "module what-is" command
set     mod_desc "\tLoads the $app_name-$app_version"
 
#####################################################################
 
module-whatis  $mod_desc
 
#####################################################################
 
# this will write a description of the module
# on "module load" command
if { [ module-info mode load ] } {
	puts stderr "$mod_desc"
}
 
# here follows a list of pre-requisites module
# you need to load in order to set the proper environment
# notice that you can use variable to easily compose 
# module names and paths (so that the same "template" for
# current software can be used for other flavour or versions
 
# APPS
prepend-path    MANPATH         $app_root/share/man
prepend-path    PATH            $app_root/bin
prepend-path    LD_LIBRARY_PATH $app_root/lib
 
# If this module can be required to compile something else, these following lines can help you, by uncommenting them:
# prepend-path      --delim " " CPPFLAGS  -I$app_root/include
# prepend-path      --delim " " CFLAGS    -I$app_root/include
# prepend-path      --delim " " LDFLAGS   -L$app_root/lib

Because /home is not mounted on computing nodes, if you want to use your own modules for computation, I advise you to copy your ~/.bash_profile on your personal workdir (/workdir/<groupname>/<username>/). Indeed, your real homedir will be binded into this.

cluster-lbt/environment-modules.txt ยท Last modified: 2017/07/20 10:31 by admin