Self Managed Software Stack

Introduction

We use the Spack package manager to deploy and manage the software stack and the module system (LMOD) on Euler. You have the possibility to extend the centrally provided software stack, by installing Spack locally in their home directory or in a project storage share. This way users can install software with Spack, reusing all the centrally installed packages as dependencies. We provide a few scripts to facilitate creating your own local software stack.

Creating a local software stack

By running the command

software-stack-create-local

and following its instructions, you can create a software stack in any directory you have write access. You can get into your stack environment with the script

__STACK_DIRECTORY__/init.sh.

For help regarding the usage of Spack, please refer to the documentation. You can also find a overview of all packages that can be built with Spack.

Installing software

Feel free to use Spack in anyway you wish to use it. We just provide some scripts to help you manage the stack in a similar way than us.

When you are in your stack environment, you can add a software to the install list with the command

spack add SOFTWARE_NAME

where SOFTWARE_NAME is the name of the corresponding Spack package. Once all the required software have been added, you can concretize the dependencies with

software-stack-concretize

and install them with

export SLURM_TIME=24:00:00
software-stack-install

Setting the SLURM_TIME variable is only required until the long queues in the Ubuntu partition are available.

If you wish to install anything without the provided scripts, then don't install on the login nodes as it impacts other users and the CPU architecture will not be the same than the central stack (which will prevent spack from reusing it).

Synchronizing Modules

When we add a library in the central stack, the module will be generated only within our stack. If you wish to synchronize the modules, then you need to run spack module lmod refresh.

Warnings

• Do not run heavy spack commands on the login nodes (e.g. install, concretize, spec). It will not only impact other users but you will also have CPU incompatibilities as the CPU are not the same in the compute nodes and the login nodes.
• Currently, module dependency loading is disabled (if module a depends on module b, module load a does not load module b). To enable this, you have to edit __STACK_DIRECTORY__/home/.spack/modules.yaml and comment out autoload: "none".

By default, when you want to use your custom stack by

module use __STACK_DIRECTORY__/spack/share/spack/lmod/linux-*

modules will be loadable using their containing directory as a prefix. Example: A typical lmod folder looks like:

lmod
└── linux-ubuntu22.04-x86_64
├── Core
│   ├── mod-a
│   │   └── a.a.a.lua
│   ...
├── gcc
│   └── x.x.x
│       ├── mod-b
│       │   └── b.b.b.lua
│       ...
└── openmpi
└── y.y.y
└── gcc
└── x.x.x
├── mod-c
│   └── c.c.c.lua
...

Here you would have to load mod-a, mod-b, mod-c like

module load Core/mod-a gcc/x.x.x/mod-b openmpi/y.y.y/gcc/x.x.x/mod-c

If you would like to make mod-a, mod-b, mod-c available without their prefix, you have to add the specific directories to the MODULEPATH environment variable. A convenient way how this can be done automatically: Let overlays in your custom stack of the modules gcc/x.x.x, openmpi/y.y.y, ... in the official stack extend the MODULEPATH.

In order to do so, add the following overlays:

lmod
└── linux-ubuntu22.04-x86_64
├── Core
│   └── gcc
│       └── x.x.x.lua
└── gcc
└── x.x.x
└── openmpi
└── y.y.y.lua

Core/gcc/x.x.x.lua

-- -*- lua -*-
-- Alias to main stack

whatis(``Name : gcc``)
whatis(``Version : x.x.x``)
whatis(``Target : x86_64_v3``)
whatis(``Short description : The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages.``)

help(``Name : gcc``)
help(``Version: x.x.x``)
help(``Target : x86_64_v3``)
help()
help(``The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages.``)

inherit()

-- Loading this module unlocks the path below unconditionally
prepend_path("MODULEPATH", "__STACK_DIRECTORY__/spack/share/spack/lmod/linux-ubuntu22.04-x86_64/gcc/x.x.x")

gcc/x.x.x/openmpi/y.y.y.lua

-- -*- lua -*-
-- Alias to main stack

whatis(``Name : openmpi``)
whatis(``Version : y.y.y``)
whatis(``Target : x86_64_v3``)
whatis(``Short description : An open source Message Passing Interface implementation.``)

help(``Name : openmpi``)
help(``Version: y.y.y``)
help(``Target : x86_64_v3``)
help()
help(``An open source Message Passing Interface implementation. The Open MPI Project is an open source Message Passing Interface implementation that is developed and maintained by a consortium of academic, research, and industry partners. Open MPI is therefore able to combine the expertise, technologies, and resources from all across the High Performance Computing community in order to build the best MPI library available. Open MPI offers advantages for system and software vendors, application developers and computer science researchers.``)

inherit()

-- Loading this module unlocks the path below unconditionally
prepend_path("MODULEPATH", "__STACK_DIRECTORY__/spack/share/spack/lmod/linux-ubuntu22.04-x86_64/openmpi/y.y.y/gcc/x.x.x")