Setting up a software stack for research group
Research groups that own a storage share in any of the HPC cluster of ETH have the possibility to setup their own software stack, which can then be shared within the group. They can also setup their own modules to manage such a software stack.
We recommend that you create the subdirectories apps and modules in your storage share. In the apps directories, you would then manage the software stack, whereas the modules subdirecotry would be used the manage your own modules.
On the Euler cluster, we recommend to use storage shares in /cluster/project as the NetApp file system can better handle small files than Lustre (/cluster/work).
[sfux@eu-login-11-ng ~]$ cd /cluster/project/__YOUR_SHARE__ [sfux@eu-login-11-ng __YOUR_SHARE__]$ mkdir apps modules [sfux@eu-login-11-ng __YOUR_SHARE__]$ chown sfux:sfux-group /cluster/project/__YOUR_SHARE__/apps /cluster/project/__YOUR_SHARE__/modules
Here you would need to replace __YOUR_SHARE__ by the name of your project share and sfux and sfux-group with your account name and personal group.
In this example we use a home directory as top-level directory for the software stack
[sfux@lo-login-01 ~]$ cd $HOME [sfux@lo-login-01 ~]$ mkdir apps modules [sfux@lo-login-01 ~]$ chown sfux:sfux-group $HOME/apps $HOME/modules
Here you would need to replace sfux and sfux-group with your account name and personal group (or any other IAM group that is suitable for this purpose).
Structure inside the apps directory
We recommend that you use the following naming scheme to manage the applications in your software stack:
If you plan to install your own version of Python, such a structure could look like:
This is also the naming scheme that we use in /cluster/apps