Setting up your environment

From ScientificComputing
Revision as of 11:33, 17 October 2022 by Sfux (talk | contribs)

Jump to: navigation, search


Most applications, compilers and libraries rely on environment variables to function properly. These variables are usually set by the operating system, the administrator, or by the user. Typical examples include:

  • PATH — location of system commands and user programs
  • LD_LIBRARY_PATH — location of the dynamic (=shared) libraries needed by these commands and programs
  • MANPATH — location of man (=manual) pages for these commands
  • Program specific environment variables

The majority of problems encountered by users are caused by incorrect or missing environment variables. People often copy initialization scripts — .profile, .bashrc, .cshrc — from one machine to the next, without verifying that the variables defined in these scripts are correct (or even meaningful!) on the target system.

If setting environment variables is difficult, modifying them at run-time is even more complex and error-prone. Changing the contents of PATH to use a different compiler than the one set by default, for example, is not for the casual user. The situation can quickly become a nightmare when one has to deal with multiple compilers and libraries (e.g. MPI) at the same time.

Environment modules — modules in short — offer an elegant and user-friendly solution to all these problems. Modules allow a user to load all the settings needed by a particular application on demand, and to unload them when they are no longer needed. Switching from one compiler to the other; or between different releases of the same application; or from one MPI library to another can be done in a snap, using just one command — module.

Software stacks

On our clusters we provide multiple software stacks.

  • The old software stack on Euler uses environment modules
  • The new software stack on Euler and the software stack on Leonahrd Open uses LMOD modules

When you login to Euler, then the old software stack is set as default. You can change to the new software stack using the command


For information regarding the new software stack, please have a look at our wiki page regarding SPACK and LMOD

Module commands

Module avail

The module avail command lists all available modules of the supported module category. If you load the new or the legacy module, it will also list all modules of these categories. It can be used to get a quick overview of all centrally installed software. If you are interested in a particular software and would like to know which versions are available, then you can specify the name of the software as a parameter for the module avail command

[sfux@eu-login-34 ~]$ module avail gcc

--------------------------------- /cluster/apps/lmodules/Compiler/gcc/4.8.5 ----------------------------------
   gcc/5.4.0    gcc/6.3.0    gcc/7.3.0    gcc/8.2.0    gcc/9.3.0 (D)

---------------------------------------- /cluster/apps/lmodules/Core -----------------------------------------
   gcc/4.8.2    gcc/4.8.5 (L)    gcc/5.4.0    gcc/6.3.0    gcc/7.3.0    gcc/8.2.0    gcc/9.3.0

   L:  Module is loaded
   D:  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

[sfux@eu-login-34 ~]$

Module show

The module show command provides you some information on what environment variables are changed and set by the module file. Further more it also contains a short string with information about the name and the version of the application or library.

[sfux@eu-login-34 ~]$ module show python/3.7.4
whatis("Name : Python")
whatis("Version : 3.7.4")
help(Python version 3.7.4)

[sfux@eu-login-34 ~]$

Module load

The module load command load the corresponding and prepares the environment for using this application or library, by applying the instructions, which can be shown by running the module show command.

[sfux@eu-login-45 ~]$ module load gcc/8.2.0 python/3.10.4

The following have been reloaded with a version change:
  1) gcc/4.8.5 => gcc/8.2.0

[sfux@eu-login-45 ~]$ which python
[sfux@eu-login-45 ~]$

Module list

The module list command displays the currently loaded modules files.

[sfux@eu-login-44 ~]$ module list

Currently Loaded Modules:
  1) StdEnv   2) gcc/4.8.5

[sfux@eu-login-44 ~]$ module load gcc/8.2.0 python/3.9.9

The following have been reloaded with a version change:
  1) gcc/4.8.5 => gcc/8.2.0

[sfux@eu-login-44 ~]$ module list

Currently Loaded Modules:
  1) StdEnv   2) gcc/8.2.0   3) openblas/0.2.20   4) python/3.9.9

[sfux@eu-login-44 ~]$

Module purge

The module purge command unload all currently loaded modules and cleans up the environment of your shell. In some cases, it might be better to log out and log in again, in order to get a really clean shell.

[sfux@eu-login-44 ~]$ module list

Currently Loaded Modules:
  1) StdEnv   2) gcc/8.2.0   3) openblas/0.2.20   4) python/3.9.9

[sfux@eu-login-44 ~]$ module purge
[sfux@eu-login-44 ~]$ module list
No modules loaded
[sfux@eu-login-44 ~]$ module load StdEnv
[sfux@eu-login-44 ~]$ module list

Currently Loaded Modules:
  1) gcc/4.8.5   2) StdEnv

[sfux@eu-login-44 ~]$

Please make sure that after purging all modules you again load the StdEnv module, which contains some important paths for your environment.

Naming scheme

Please find the general naming scheme of module files below.


Instead of specifying a version directly, it is also possible to use aliases.

program_name/alias == program_name/version

The special alias default indicates which version is taken by default (if neither version nor alias is specified)

program_name/default == program_name

If no default is specified for a particular software, then the most recent version (i.e. that with the largest number) is taken by default.


For the Leonhard cluster and the new software stack on Euler, we decided to switch from the environment modules that are used for the old software stack on Euler to Lmod modules, which provide some additional features. You should barely notice the transition from environment modules to Lmod modules as the commands are mostly the same. Therefore please refer to the Setting up your environment tutorial for a general documentation about the module commands.

[sfux@eu-login-02 ~]$ module avail boost

----------------------------------------- /cluster/apps/lmodules/Compiler/gcc/4.8.5 ------------------------------------------

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

[sfux@eu-login-02 ~]$ module load boost/1.63.0
[sfux@eu-login-02 ~]$ module list

Currently Loaded Modules:
  1) gcc/4.8.5   2) StdEnv   3) boost/1.63.0

[sfux@eu-login-02 ~]$ 

Please note that this is work in progress and the module names might change. Currently, the number of software packages provided on Leonhard is not comparable to the software we provide on the Euler cluster, but it will grow over time.

Hierarchical modules

LMOD allows to define a hierarchy of modules containing 3 layers (Core, Compiler, MPI). The core layer contains all module files which are not depending on any compiler/MPI. The compiler layer contains all modules which are depending on a particular compilers, but not on any MPI library. The MPI layer contains modules that are depending on a particular compiler/MPI combination.

When you login to the Leonhard cluster, the standard compiler gcc/4.8.5 is automatically loaded. Running the module avail command displays all modules that are available for gcc/4.8.5. If you would like to see the modules available for a different compiler, for instance gcc/6.3.0, then you would need to load the compiler module and run module avail again. For checking out the available modules for gcc/4.8.5 openmpi/2.1.0, you would load the corresponding compiler and MPI module and run again module avail'.

As a consequence of the module hierarchy, you can never have two different versions of the same module loaded at the same time. This helps to avoid problems arising due to misconfiguration of the environment.