Difference between revisions of "Creating local module directory"

From ScientificComputing
Jump to: navigation, search
(→‎Creating a custom module directory)
Line 1: Line 1:
 
== Introduction ==
 
== Introduction ==
 +
You can use your own custom modules to setup your environment to run programs that you have installed yourself. This tutorial will guide you through creating a custom module directory, adding versioned modules for programs, and automatically having these modules available to you upon login. The tutorial will assume you will be managing a set of modules for a whole group, though it is equally applicable to managing a personal set of modules.
  
−
You can use your own custom modules to setup your environment to run programs that you have installed yourself.
 
  
−
This tutorial will guide you through creating a custom module directory, adding versioned modules for programs, and automatically having these modules available to you upon login. The tutorial will assume you will be managing a set of modules for a whole group, though it is equally applicable to managing a personal set of modules.
+
Please note that on Euler, we are using environment modules, whereas on Leonhard we are using LMOD modules. When extending the centrally provided modules, please make sure that you use the correct module type for your own modules.
  
 
== Creating a custom module directory ==
 
== Creating a custom module directory ==
 +
You need to decide on the top-level directory (we will use the placeholder __TLD__ for the examples) in which to store the module files. You need to create the directory and set the user and group ownership:
  
−
You need to decide on the top-level directory in which to store the module files.
+
===Euler===
  
−
===Euler===
+
[sfux@eu-login-01-ng ~]$ '''cd __TLD__'''
−
Let us assume it is <tt>/cluster/project/__YOUR_SHARE__/modules</tt>, where <tt>__YOUR_SHARE__</tt> should be replaced with your group's storage location. This can also be an arbitrary directory to which you have access, e.g., <tt>$HOME/modules</tt>. First create the directory and set proper permissions:
+
[sfux@eu-login-01-ng __TLD__]$ '''chown sfux:sfux-group modules'''
  
−
[leonhard@euler00 ~]$ '''mkdir /cluster/project/leonhard/modules'''
+
Here you would need to replace <tt>sfux</tt> and <tt>sfux-group</tt> with your account name and personal group (or any other IAM group that is suitable for this purpose).
−
[leonhard@euler00 ~]$ '''chown leonhard:leonhard-group /cluster/project/leonhard/modules'''
 
  
 
===Leonhard===
 
===Leonhard===
 +
 +
[sfux@lo-login-01 ~]$ '''cd __TLD__'''
 +
[sfux@lo-login-01-ng __TLD__]$ '''chown sfux:sfux-group modules'''
 +
 +
Here you would need to replace <tt>sfux</tt> and <tt>sfux-group</tt> with your account name and personal group (or any other IAM group that is suitable for this purpose).
  
 
== Using the custom module directory ==
 
== Using the custom module directory ==
  
−
Now you need to let the <tt>module</tt> command know about your custom directory. Add the following line to your <tt>$HOME/.bash_profile</tt> file.
+
Now you need to let the <tt>module</tt> command know about your custom directory. Add the following line to your <tt>$HOME/.bash_profile</tt> file in case you are the only person using the additional modules. Or you can create a script that can be sourced in case multiple people use the additional modules.
  
 
For the Euler cluster:
 
For the Euler cluster:
  
−
  module use /cluster/project/leonhard/modules
+
  module use __TLD__/leonhard/modules
  
 
For the Leonhard cluster:
 
For the Leonhard cluster:
  
−
  export MODULEPATH=$HOME/modules:$MODULEPATH
+
  export MODULEPATH=__TLD__/modules:$MODULEPATH
  
 
The <tt>module</tt> command will now see your custom modules the next time you log in. You can also type this command into your current shell to have immediate access.
 
The <tt>module</tt> command will now see your custom modules the next time you log in. You can also type this command into your current shell to have immediate access.
Line 33: Line 38:
 
Every group members will need to do this setup step to see the modules.
 
Every group members will need to do this setup step to see the modules.
  
−
== Creating a versioned module file ==
+
==Example module files==
 +
 
 +
===Environment modules===
  
−
Next, you need to create a module file for a program. Assume you have version 1.0 of the <tt>basel</tt> program located in <tt>/cluster/project/leonhard/apps/basel_problem/1.0/bin</tt> and it depends on the <tt>gmp</tt> module. Create the program's module directory:
 
−
[leonhard@euler00 ~]$ '''mkdir /cluster/project/leonhard/modules/basel'''
 
−
[leonhard@euler00 ~]$ '''chown leonhard:leonhard-group /cluster/project/leonhard/modules/basel'''
 
−
In your favorite editor, edit the <tt>/cluster/project/leonhard/modules/basel/1.0</tt> file to contain the following contents:
 
 
  #%Module1.0
 
  #%Module1.0
−
  module-whatis "Solve Basel problem"
+
  module-whatis "This is my software PROGRAM version 1.0"
−
  set helpmsg "Solves the Basel problem."
+
  set helpmsg "PROGRAM is used used to achieve the following task ..."
−
module load gmp
+
  set topdir "__TLD__/apps/PROGRAM/1.0"
−
  set topdir "/cluster/project/leonhard/apps/basel_problem/1.0"
+
setenv IMPORTANT_VARIABLE "Some string"
 
  prepend-path PATH $topdir/bin
 
  prepend-path PATH $topdir/bin
 +
prepend-path LD_LIBRARY_PATH $topdir/lib64
 +
prepend-path C_INCLUDE_PATH $topdir/include
 +
 +
===LMOD modules===
  
−
Test that it works:
+
-- -*- lua -*-
−
  [leonhard@euler00 ~]$ '''module avail basel/1.0'''
+
--
−
  [leonhard@euler00 ~]$ '''module load basel/1.0'''
+
-- Any sort of comment can be put here
−
  [leonhard@euler00 ~]$ '''module list'''
+
--
−
  Currently Loaded Modulefiles:
+
  --
−
  1) modules                  2) gcc/4.8.2(default:4.8)  3) gmp/5.1.3(default:5)    4) basel/1.0
+
  whatis([[Name : This is my software PROGRAM version 1.0]])
 +
  help([[PROGRAM is used to achieve the following task ...]])
 +
  setenv("IMPORTANT_VARIABLE", "Some string")
 +
prepend_path("PATH", "__TLD__/apps/PROGRAM/1.0/bin")
 +
prepend_path("LD_LIBRARY_PATH", "__TLD__/apps/PROGRAM/1.0/lib64")
 +
prepend_path("C_INCLUDE_PATH", "__TLD__/apps/PROGRAM/1.0/include")

Revision as of 13:27, 28 June 2019

Introduction

You can use your own custom modules to setup your environment to run programs that you have installed yourself. This tutorial will guide you through creating a custom module directory, adding versioned modules for programs, and automatically having these modules available to you upon login. The tutorial will assume you will be managing a set of modules for a whole group, though it is equally applicable to managing a personal set of modules.


Please note that on Euler, we are using environment modules, whereas on Leonhard we are using LMOD modules. When extending the centrally provided modules, please make sure that you use the correct module type for your own modules.

Creating a custom module directory

You need to decide on the top-level directory (we will use the placeholder __TLD__ for the examples) in which to store the module files. You need to create the directory and set the user and group ownership:

Euler

[sfux@eu-login-01-ng ~]$ cd __TLD__
[sfux@eu-login-01-ng __TLD__]$ chown sfux:sfux-group modules

Here you would need to replace sfux and sfux-group with your account name and personal group (or any other IAM group that is suitable for this purpose).

Leonhard

[sfux@lo-login-01 ~]$ cd __TLD__
[sfux@lo-login-01-ng __TLD__]$ chown sfux:sfux-group modules

Here you would need to replace sfux and sfux-group with your account name and personal group (or any other IAM group that is suitable for this purpose).

Using the custom module directory

Now you need to let the module command know about your custom directory. Add the following line to your $HOME/.bash_profile file in case you are the only person using the additional modules. Or you can create a script that can be sourced in case multiple people use the additional modules.

For the Euler cluster: 

module use __TLD__/leonhard/modules

For the Leonhard cluster: 

export MODULEPATH=__TLD__/modules:$MODULEPATH

The module command will now see your custom modules the next time you log in. You can also type this command into your current shell to have immediate access.

Every group members will need to do this setup step to see the modules.

Example module files

Environment modules

#%Module1.0
module-whatis "This is my software PROGRAM version 1.0"
set helpmsg "PROGRAM is used used to achieve the following task ..."
set topdir "__TLD__/apps/PROGRAM/1.0"
setenv IMPORTANT_VARIABLE "Some string"
prepend-path PATH $topdir/bin
prepend-path LD_LIBRARY_PATH $topdir/lib64
prepend-path C_INCLUDE_PATH $topdir/include

LMOD modules

-- -*- lua -*-
-- 
-- Any sort of comment can be put here
--
--
whatis(Name : This is my software PROGRAM version 1.0)
help(PROGRAM is used to achieve the following task ...)
setenv("IMPORTANT_VARIABLE", "Some string")
prepend_path("PATH", "__TLD__/apps/PROGRAM/1.0/bin")
prepend_path("LD_LIBRARY_PATH", "__TLD__/apps/PROGRAM/1.0/lib64")
prepend_path("C_INCLUDE_PATH", "__TLD__/apps/PROGRAM/1.0/include")
Retrieved from "https://scicomp.ethz.ch/w/index.php?title=Creating_local_module_directory&oldid=3921"