Modules and applications

__NOTOC__

On the cluster, we provide many centrally installed software and for some software even multiple versions. To configure the environment for a particular software version, we use modules. Modules configure your current computing environment (PATH, LD_LIBRARY_PATH, MANPATH, etc.) to make sure all required binaries and libraries are found. We employ the following Modules packages on the cluster: - LMOD Modules

Modules commands¶

Here are the most frequently used module commands.

List all modules that match the given module name, e.g., list all available Python

$ module spider python

You can also use avail to see the one available with the current module loaded:

$ module avail python

List all currently loaded modules

$ module list

Currently Loaded Modules:
1) gcc/12.2.0 2) stack/2024-06

Other commonly used module commands

module                    # get info about module sub-commands
module avail              # List all modules available on the cluster
module keykeyword# list all modules whose description containskeyword
module helpname# get information about modulename
module showname# show what modulenamedoes (without loading it)
module unloadname# unload modulename
module purge              # unload all modules at once

You can also search through the full stack for a package with

module spider # report all modules
module spidername# report all versions of the modulename
module --show_hidden spidername# Show also the hidden modules

Please note that you first need to load a stack. The stack determines the GCC version of the packages that can be loaded.

stack/2024-04 -> GCC 8.5.0
stack/2024-06 -> GCC 12.2.0

The 2024-06 also contains packages for Intel OneAPI 2023.2.0

module load stack/2024-06 intel-oneapi-compilers/2023.2.0

All available modules are listed on our wiki.

Where are applications installed?¶

### Centrally installed {#centrally_installed} Central applications are installed in /cluster/software Applications that are needed by many users should be installed centrally, like compilers and libraries - Visible and accessible to all users via modules - Installed and maintained by cluster support - Commercial licenses provided by the IT shop of ETH or by research groups ### In $HOME {#in_home} Users can install additional applications in their home directory, but only if the quotas (space: 50GB, files/directories: 500'000) are not exceeded - Avoid anaconda installations as they often conflict with the files/directories quota. Alternatively, you can create a Python virtual environment. - For Python and R, packages can easily be installed locally, e.g., for Python $ pip3 install --user packagename

The structure of LMOD Modules¶

LMOD Modules use a hierarchy of modules with three layers to avoid conflicts when multiple modules are loaded at the same time. - The core layer contains software which are independent of compilers and MPI libraries, e.g., commercials software which come with their own runtime libraries $ module load comsol/5.6 - The stack layer contains software which are dependent of compilers. $ module load stack/2024-06 hdf5/1.14.3 - The MPI layer contains software which are dependent of compilers and MPI libraries $ module load stack/2024-06 openmpi/4.1.6 openblas/0.3.24

Modules for GPUs¶

We have installed a Python module which is linked with CUDA and NCCL libraries and contains machine learning / deep learning packages such as scikit-Learn, TensorFlow and Pytorch. This module can be loaded with the following commands:

$ module load stack/2024-06 python_cuda/3.11.6

You can also find and load CUDA, cuDNN and NCCL libraries available on the cluster matching your needs.

$ module load stack/2024-06
$ module avail cuda
-------- /cluster/software/stacks/2024-06/spack/share/spack/lmod/linux-ubuntu22.04-x86_64/gcc/12.2.0 ------
cuda/11.8.0 cuda/12.1.1 (D)

Application lists¶

Ubuntu:

Euler - Ubuntu software stacks

CentOS (deprecated):

Euler - New software stack

Example¶

Python virtual environment