Modules and applications

< Storage and data transfer

Home

Job management with LSF >

Several software tools, each with several versions, are offered on the cluster. How to find applications you need and set up the computing environment specific for you? Use Modules A Modules package is a tool to let you to easily configure your computing environment. We employ two types of Modules packages on the clusters: LMOD Modules implemented on Euler and Leonhard Open Environment Modules implemented on Euler

Modules commands

Here are the most frequently used module commands.

List all modules that match the given module name, e.g., list all available Python

$ module avail python

Load modules, e.g., load GCC 6.3.0 and Python 3.8.5

$ module load gcc/6.3.0 python/3.8.5

List all currently loaded modules

$ module list

Currently Loaded Modules:
 1) StdEnv   2) gcc/6.3.0   3) openblas/0.2.20   4) python/3.8.5

Other commonly used module commands

module                    # get info about module sub-commands
module avail              # List all modules available on the cluster
module key keyword        # list all modules whose description contains keyword
module help name          # get information about module name
module show name          # show what module name does (without loading it)
module unload name        # unload module name
module purge              # unload all modules at once

Where are applications installed?

Centrally installed Central applications are installed in /cluster/apps Applications that are needed by many users should be installed centrally, like compilers and libraries Visible and accessible to all users via modules Installed and maintained by cluster support Commercial licenses provided by the IT shop of ETH or by research groups

In $HOME Users can install additional applications in their home directory, but only if the quotas (space: 16GB, files/directories: 100'000) are not exceeded Avoid anaconda installations as they often conflict with the files/directories quota For Python and R, packages can easily be installed locally

Two software stacks on Euler

Old software stack with Env Modules Upon your login on Euler, the old software stack is set by default. Switch from the old to the new software stack with the command:		New software stack with LMOD Modules Switch from the new to old software stack:
$ env2lmod		$ lmod2env

You can set the default software stack to the new one with the command:

$ set_software_stack.sh new

All new software is installed exclusively in the new software stack, mostly done with SPACK package manager.

You can find instructions how to load modules and examples on application pages, for instance, R, Python, Matlab, etc.

The structure of LMOD Modules

LMOD Modules comprise of three layers The core layer contains software which are independent of compilers and MPI libraries, e.g., commercials software which come with their own runtime libraries $ module load comsol/5.6 The compiler layer contains software which are dependent of compilers. $ module load gcc/6.3.0 hdf5/1.10.1 The MPI layer contains software which are dependent of compilers and MPI libraries $ module load gcc/6.3.0 openmpi/3.0.1 openblas

There are four main toolchains GCC 4.8.5 (supports C++11 standard) GCC 6.3.0 (supports C++14 standard) GCC 8.2.0 (supports C++17 standard) Intel 18.0.1 Those compilers can be combined with OpenMPI 3.0.1 or 4.0.2 and OpenBLAS

Use a Python GPU module

In the new software stack, we have installed a Python module which is linked with CUDA, cuDNN and NCCL libraries and contains machine learning / deep learning packages such as scikit-Learn, TensorFlow and Pytorch. This module can be loaded with the following commands:

$ env2lmod
$ module load gcc/6.3.0 python_gpu/3.8.5

Application lists

• Euler - New software stack
• Euler - Old software stack
• Leonhard Open
• Most recently installed software

Table: Examples of centrally-installed commercial and open source applications

Bioinformatics & life science	Finite element methods	Machine learning	Multi-physics phenomena	Quantum chemistry & molecular dynamics	Symbolic, numerical and statistical mathematics	Visualization
Bamtools BLAST Bowtie CLC Genomics Server RAxML Relion TopHat	Ansys Abaqus FEniCS	PyTorch Scikit-Learn TensorFlow Theano	AnsysEDT COMSOL Multiphysics STAR-CCM+	ADF Ambertools Gaussian Molcas Octopus Orca Qbox Turbomole	Gurobi Mathematica MATLAB R Stata	Ffmpeg ParaView VisIT VTK

Table: Examples of centrally-installed software development tools

Compiler	Programming languages	Scientific libraries	Solvers	MPI libraries	Build systems	Version control
GCC Intel	C, C++ Fortran, Go Java, Julia Perl, Python Ruby, Scal	Boost, Eigen FFTW, GMP GSL, HDF5 MKL, NetCDF NumPy, OpenBLAS SciPy	PETSc Gurobi Hypre Trilino	Open MPI Intel MPI MPICH	GNU Autotools Cmake qmake make	CVS Git Mercurial SVN

Example

• Python virtual environment

Further reading

• User guide: Setting up your environment
• Setting up a software stack for a research group
• Creating a local module directory
• Unpacking RPM packages in users space

< Storage and data transfer

Home

Job management with LSF >