Euler workshop 10.10.2016

Introduction

The ID SIS HPC group organizes a Getting started with Euler workshop for the group of Prof. Dr. Karsten Borgwardt.

Location

"Manser" seminar room, building 1078/P.03

Time

13:30 - 16:30

Agenda

The workshop takes place 13:30-16:30 and will cover the following topics:

• introduction to Euler (history, specification, shareholders)
• accessing the cluster (policies, login, security)
• data management (storage systems, file transfer)
• environment modules (purpose, usage, categories)
• batch system (job submission, resources, monitoring)
• applications (development tools, centrally installed apps, compiling an application)
• user support (documentation, help desk)

Examples

blitz++

Installation:

export BASEDIR=/cluster/project/borgwardt/workshop/apps/blitz++/0.10
mkdir -p $BASEDIR/source $BASEDIR/x86_64
cd $BASEDIR/source
wget https://sourceforge.net/projects/blitz/files/blitz/Blitz++%200.10/blitz-0.10.tar.gz
tar -xzvf blitz-0.10.tar.gz
cd blitz-0.10
module load gcc/4.8.2 openblas/0.2.13_seq boost/1.59.0
sed -i 's|-lblas|-lopenblas|g;s|BOOSTLIBDIR=.*|BOOSTLIBDIR=$BOOST_LIBRARYDIR|g' configure
./configure --prefix=$BASEDIR/x86_64 --enable-fortran --enable-64bit \
--enable-shared --enable-optimize --enable-serialization \
--with-blas=$GOTOBLAS2/lib --with-boost=$BOOST_ROOT/lib64
make all
make install

Creating the module:

mkdir -p /cluster/project/borgwardt/workshop/modules/blitz

cat > /cluster/project/borgwardt/workshop/modules/blitz/0.10 << 'EOF'
#%Module1.0
set version 0.10
module-whatis "blitz++ version $version"
set helpmsg "blitz++ version $version"
set topdir "/cluster/project/borgwardt/workshop/apps/blitz++/$version/x86_64"
prepend-path LD_LIBRARY_PATH $topdir/lib
prepend-path INCLUDE $topdir/include
prepend-path C_INCLUDE_PATH $topdir/include
prepend-path CPLUS_INCLUDE_PATH $topdir/include
EOF

module use /cluster/project/borgwardt/workshop/modules
module load gcc/4.8.2 openblas/0.2.13_seq boost/1.59.0 blitz/0.10

tensorflow

#this must be executed in a batch interactive job
bsub -Is -n 24 -W 4:00 bash

cd $TMPDIR
module load new gcc/4.8.2 java/1.8.0_91 bazel/0.3.0 swig/3.0.5 python/3.4.3
module load eth_proxy
git clone -b r0.10 https://github.com/tensorflow/tensorflow
cd tensorflow
sed -i '/def .*linko/,+1 s|\[\]|["-lrt"]|' tensorflow/tensorflow.bzl
./configure
bazel build --verbose_failures -c opt //tensorflow/tools/pip_package:build_pip_package
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
pip install /tmp/tensorflow_pkg/tensorflow-0.10.0-py3-none-any.whl --user

Slides

Getting started with Euler (October 2016).pdf

Helper tool for LSF batch system

LSF submission line advisor