Python/Extension
The Python programming language can make use of packages, which extend the functionality of the core language. We provide a central Python package repository, with a range of installed packages that are available to all cluster users. If a package is not installed centrally, or if you need a newer version than the centrally installed one, then you can also install a Python package locally in your home directory, that can then be used together with the centrally installed Python interpreter.
Checking installed packages with pip
Before installing a new package locally with pip, you might want to check first if the package is already available. For Python 3.4.3 and newer versions, all centrally installed packages were installed with pip. To check the available packages for a particular Python version, you would need to first load the corresponding Python module and then run the command
pip3 list
The output will display a list of all installed Python packages.
Installing a Python package, using PIP
Additionally, you can also install a package locally using pip
pip3 install --user package
The packages will be installed in a local repository at $HOME/.local. Since this is the default path for user-installed python packages, there is no need to adjust PYTHONPATH.
In order to install a particular version of a package, you can use the command
pip3 install --user package==version
Where "version" denotes the version number that you would like to install.
But be aware that using pip to install a newer version of a package that has originally been installed with easy_install causes problems in the loading order, such that always the version that was installed with easy_install will be loaded unless you change the sys.path in the current Python session.
Installing a Python package locally, using distutils
Please don't install packages with distutils any more. Since Python 3.4 only pip should be used to install packages
You can create your personal Python package repository. As an example, we create this directory in $HOME:
mkdir $HOME/python
You are free to choose any other location for your package repository, but it has to contain a given sub-directory structure (which is shown here for python/2.7.6. In case of python/3.3.3, exchange the version number in the directory names from 2.7 to 3.3):
cd $HOME/python mkdir -p lib64/python2.7/site-packages
In order to make python aware of the local package repository, you should add a line to the $HOME/.bash_profile file:
export PYTHONPATH=$HOME/python/lib64/python2.7/site-packages:$PYTHONPATH
and run this same command on the command line to make your current interactive shell aware of the additional python path. All of the the above steps should be done only once: before you install your first package.
After this preparation, one needs to load the Python module:
module load python/2.7.6
As a next step, you can download and extract the tarball with the source code of the additional package. Go into the directory containing the setup.py file (the directory is usually named package-version). Finally, you can run the command to install the package:
python setup.py install --prefix=$HOME/python
Here the prefix option assures that the package is installed in your local package repository.
With this setup, it is possible to have two versions of a package side-by-side. Python will always search in the locations given in PYTHONPATH first, and if a package is not found there, it will fall back to the central repository.
