Difference between revisions of "Jupyter on Euler and Leonhard Open"

From ScientificComputing
Jump to: navigation, search
(16 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
==Introduction==
 
==Introduction==
−
Since Jupyter notebooks are becoming more widely used among the scientific community, the HPC group developed a script that you can run on your local computer. This shell script then starts a Jupyter notebook in a batch job on Euler/Leonhard Open (depending on which cluster you choose) and connects your local browser with it.
+
Jupyter notebooks and Jupyter Lab are becoming more widely used among the scientific community at ETH. We therefore developed a script that you can run on your local computer to start Jupyter notebooks or Jupyter lab on Euler. The shell script starts a Jupyter in a batch job on Euler and connects your local browser with it.
  
−
At the moment, '''the script can be used with Linux and Mac computers'''. '''There is no support for Windows computers'''. Maybe Windows user can try to run the script using Windows subsystem for Linux (WSL), but this has not been tested yet.
+
The script runs on Linux, macOS and Windows (using WSL/WSL2 or git bash).
  
 
Please note, that with this script we are addressing beginners that start to use Jupyter notebooks on the cluster. It is not addressing advanced users that need a wide range of additional features going beyond simple Jupyter notebooks. Advanced users can take the script and adapt it, such that it can be used with other Python versions (centrally installed, or local installations) and add support for GPU, adding new kernels etc.
 
Please note, that with this script we are addressing beginners that start to use Jupyter notebooks on the cluster. It is not addressing advanced users that need a wide range of additional features going beyond simple Jupyter notebooks. Advanced users can take the script and adapt it, such that it can be used with other Python versions (centrally installed, or local installations) and add support for GPU, adding new kernels etc.
−
 
−
Please note, that the script uses the old software stack on Euler and can in its current state not be used with tools from the new software stack.
 
  
 
==Installation==
 
==Installation==
 +
Download the repository with the commnad
 +
git clone https://gitlab.ethz.ch/sfux/Jupyter-on-Euler-or-Leonhard-Open
 +
Mac OS X:
 +
git clone https://gitlab.ethz.ch/sfux/Jupyter-on-Euler-or-Leonhard-Open.git
 +
After downloading the script from gitlab.ethz.ch, you need to change its permissions to make it executable
 +
cd Jupyter-on-Euler-or-Leonhard-Open/
 +
chmod 755 start_jupyter_nb.sh
  
 
===Prerequisites===
 
===Prerequisites===
Line 15: Line 20:
 
  https://scicomp.ethz.ch/wiki/Accessing_the_clusters#SSH_keys
 
  https://scicomp.ethz.ch/wiki/Accessing_the_clusters#SSH_keys
  
−
Please note that the example on the wiki refers to the Euler cluster and for Leonhard Open, then hostname needs to be changed from
+
Please make sure that xdg-open is installed. This package is used to automatically start your default browser. You can install it with the following command:
−
 
 
−
euler.ethz.ch
 
−
to
 
−
 
 
−
login.leonhard.ethz.ch
 
−
 
 
−
please make sure that xdg-open is installed. This package is used to automatically start your default browser. You can install it with the following command:
 
  
 
CentOS:
 
CentOS:
Line 33: Line 31:
  
 
Further more, the script requires that there is a Python installation available, which is usually included in the Linux distribution or Mac OS.
 
Further more, the script requires that there is a Python installation available, which is usually included in the Linux distribution or Mac OS.
 +
 +
===Security token vs. password setup===
 +
Please note that a part of the script (parsing of the ports) requires that you use jupyter notebooks with the security tokens. If you configure a password instead, such that you can use the jupyter notebook without the security token, then the script will not work anymore (it cannot parse the port on the remote compute node) without adapting it.
  
 
===Download and setup===
 
===Download and setup===
Line 51: Line 52:
 
  chmod 755 start_jupyter_nb.sh
 
  chmod 755 start_jupyter_nb.sh
  
−
===Updating the script===
+
===Software stack===
−
* 01 Oct 2019 &mdash; Today the script has been updated, such that the jupyter notebooks have next to the Python 3.6 kernel also a bash and an R kernel (3.6.0 on Euler, 3.5.1 on Leonhard Open) available. If you use an older version of the script and you would like to use the newly added kernels, then you need to update your script from the gitlab repository with the command <tt>git pull origin master</tt>
+
Please note that currently the old software stack is still set a default (this will change). The script is using the new software stack (unless you explicitly request the old software stack with the option -s old (or --softwarestack old). Therefore please make sure that you set the new software stack as permanent default by using the command
  
−
  samfux@bullvalene:~/Jupyter-on-Euler-or-Leonhard-Open$ git pull origin master
+
  set_software_stack.sh new
−
warning: redirecting to https://gitlab.ethz.ch/sfux/Jupyter-on-Euler-or-Leonhard-Open.git/
 
−
From https://gitlab.ethz.ch/sfux/Jupyter-on-Euler-or-Leonhard-Open
 
−
  * branch            master    -> FETCH_HEAD
 
−
Already up to date.
 
−
samfux@bullvalene:~/Jupyter-on-Euler-or-Leonhard-Open$
 
  
−
===Using SSH keys with non-default names===
+
You can find more information about this script on our wiki:
−
Since the reopening of Euler and Leonhard Open after the cyber attack in May 2020, we recommend to the cluster users to use SSH keys. We recommend to use different keys for Euler and Leonhard Open, with according names
 
  
−
  $HOME/.ssh/id_ed25519_euler
+
  https://scicomp.ethz.ch/wiki/Setting_permanent_default_for_software_stack_upon_login
−
$HOME/.ssh/id_ed25519_leonhard
 
  
−
In order to use those keys with the jupyter script, you would need to edit the following section at the beginning of the script and add the path to your SSH keys. In the example below we show how this would look like for Euler:
+
===Creating SSH keys for the Jupyter script with a Windows computer (using git-bash)===
 +
With git-bash, you can use the same command as for Linux/Mac to create your SSH keys:
  
−
#########################
+
* Start git-bash
−
# Configuration options #
+
* Run the command ssh-keygen to create the SSH key pair
−
  #########################
+
  ssh-keygen -t ed25519 -f $HOME/.ssh/id_ed25519_euler
−
+
* Enter a passphrase to protect your SSH keys
−
# SSH key location is the path to your SSH key. Please specify the path if you are using a non-standard name for your SSH key
+
* Use the ssh-copy-id command to copy your '''public''' (never your private) key to Euler
−
  SSH_KEY_LOCATION="'''$HOME/.ssh/id_ed255519_euler'''"
+
  ssh-copy-id -i ~/.ssh/id_ed25519_euler.pub '''username'''@euler.ethz.ch
−
+
* (here you need to replace '''username''' with your ETH username)
−
# Waiting time interval after starting the jupyter notebook. Check every $WAITING_TIME_INTERVAL seconds if the job already started
+
* Enter your password
−
WAITING_TIME_INTERVAL=60
 
−
 
−
#############################
 
−
# End configuration options #
 
−
#############################
 
  
−
This is required to use SSH keys with non-default names.
+
After creating the keys, we strongly recommend to create an SSH config file in $HOME/.ssh
  
−
===Installing additional Python and R packages locally===
+
* Change to your .ssh directory
−
When starting a Jupyter notebook with this script, then it will use a central Python and R installation:
+
cd $HOME/.ssh
 +
* Create an SSH config file to specify the path to your SSH key
 +
echo -e "Host euler\nHostName euler.ethz.ch\nUser '''username'''\nIdentityFile ~/.ssh/id_ed25519_euler" > config
 +
* (here you need to replace '''username''' with your ETH username)
  
−
* Euler: python/3.6.1, r/3.6.0
+
At this stage, passwordless login to Euler should work. You can test this with the command
−
* Leonhard Open: python_cpu/3.6.4, r/3.5.1
 
  
−
Therefore you can only use packages that are centrally installed out-of-the-box. But you have the option to install additional packages locally in your home directory, which can afterwards be used.
+
ssh username@euler.ethz.ch
  
−
For installing a Python package from inside a Jupyter notebook, you would need to run the following command:
+
* (here you need to replace '''username''' with your ETH username)
  
−
!pip install --user package_name
+
But now you still have to enter your passphrase. In order to not have to type your passphrase all the time, you can use an SSH agent, which is already included in git-bash
  
−
This will install <tt>package_name</tt> into <tt>$HOME/.local</tt>, as described on our wiki page about Python:
+
* Start git-bash
 +
* Start the SSH agent for the current terminal
 +
ssh-agent /bin/bash
 +
* Add your SSH key to the SSH agent
 +
ssh-add ~/.ssh/id_ed25519_euler
 +
* Enter your passphrase
  
−
  https://scicomp.ethz.ch/wiki/Python#Installing_a_Python_package.2C_using_PIP
+
As long as this terminal is open, you can access Euler with
 +
  ssh username@euler.ethz.ch
 +
(here you need to replace '''username''' with your ETH username)
 +
without having to type a password or passphrase. As soon as you exit the terminal, then the SSH agent will be stopped.
  
−
The command to locally install an R package:
+
Now you are ready to clone the Jupyter script and to execute it in the terminal with the SSH agent. Please note that you only need to create the SSH once, but you need to setup the SSH agent every time you would like to run the Jupyter script.
  
−
install.packages("package_name")
+
===Using SSH keys with non-default names===
 +
Since the reopening of Euler and Leonhard Open after the cyber attack in May 2020, we recommend to the cluster users to use SSH keys. We recommend to use different keys for Euler and Leonhard Open, with according names
  
−
Then follow the instructions provided on our wiki:
+
$HOME/.ssh/id_ed25519_euler
  
−
https://scicomp.ethz.ch/wiki/R#Extensions
+
In order to use this keys with the jupyter script, then there are two options. You can configure your ssh client to use this option automatically by adding the following lines in the <tt>$HOME/.ssh/config</tt> file on your local computer:
 +
 +
Host euler.ethz.ch
 +
IdentityFile ~/.ssh/id_ed25519_euler
  
 +
Both options work and are equivalent.
  
−
==Running the script==
+
===Installing additional Python packages locally===
−
The start_jupyer_nb.sh script needs to be executed on your local computer:
+
When starting a Jupyter notebook with this script, then it will use a central Python installation:
  
−
./start_jupyter_nb.sh CLUSTER ETH_USERNAME NUM_CORES RUN_TIME MEM_PER_CORE
+
* New software stack: module load gcc/6.3.0 python/3.8.5
 +
* Old software stack: module load new gcc/4.8.2 python/3.6.1
  
−
{| class="wikitable
+
Therefore you can only use packages that are centrally installed out-of-the-box. But you have the option to install additional packages locally in your home directory, which can afterwards be used.
−
!Parameter
 
−
!Description
 
−
|-
 
−
|CLUSTER
 
−
|Name of the cluster (Euler or LeoOpen)
 
−
|-
 
−
|ETH_USERNAME
 
−
|ETH username for which the notebook should be started
 
−
|-
 
−
|NUM_CORES
 
−
|Number of cores to be used on the cluster (maximum: 36)
 
−
|-
 
−
|RUN_TIME
 
−
|Run time limit for the jupyter notebook on the cluster (HH:MM)
 
−
|-
 
−
|MEM_PER_CORE
 
−
|Memory limit in MB per core
 
−
|}
 
  
−
Example:
+
For installing a Python package from inside a Jupyter notebook, you would need to run the following command:
  
−
  ./start_jupyter_nb.sh Euler sfux 4 01:20 2048
+
  !pip install --user package_name
  
−
[[File:Jupyter_on_Euler_and_Leonhard_Open.png|thumb|none|800px|Example for running a Jupyter notebook on the Euler cluster]]
+
This will install <tt>package_name</tt> into <tt>$HOME/.local</tt>, as described on our wiki page about Python:
  
−
===Starting in a different location than your home directory===
+
https://scicomp.ethz.ch/wiki/Python#Installing_a_Python_package.2C_using_PIP
−
By default, the Jupyter notebook will start in your home directory. It is also possible to start in a different location. For this you would need to change line 122 in the script from
 
  
−
jupyter notebook --no-browser --ip "\$IP_REMOTE" &> /cluster/home/$USERNAME/jnbinfo
+
==Running the script==
  
−
to
+
The start_jupyer_nb.sh script needs to be executed on your local computer. Please find below the list of options that can be used with the script:
  
−
  jupyter notebook --no-browser --ip "\$IP_REMOTE" --notebook-dir PATH &> /cluster/home/$USERNAME/jnbinfo
+
  $ ./start_jupyter_nb.sh -h
 +
./start_jupyter_nb.sh: Script to start jupyter notebook/lab on Euler from a local computer
 +
Usage: start_jupyter_nb.sh [options]
 +
 +
Options:
 +
 +
        -u | --username      USERNAME        ETH username for SSH connection to Euler
 +
        -n | --numcores      NUM_CPU          Number of CPU cores to be used on the cluster
 +
        -W | --runtime        RUN_TIME        Run time limit for jupyter notebook/lab in hours and minutes HH:MM
 +
        -m | --memory        MEM_PER_CORE    Memory limit in MB per core
 +
 +
Optional arguments:
 +
 +
        -c | --config        CONFIG_FILE      Configuration file for specifying options
 +
        -e | --environment    ENV              Python virtual environment
 +
        -g | --numgpu        NUM_GPU          Number of GPUs to be used on the cluster
 +
        -h | --help                            Display help for this script and quit
 +
        -i | --interval      INTERVAL        Time interval for checking if the job on the cluster already started
 +
        -k | --key            SSH_KEY_PATH    Path to SSH key with non-standard name
 +
        -l | --lab                            Start jupyter lab instead of a jupyter notebook
 +
        -s | --softwarestack  SOFTWARE_STACK  Software stack to be used (old, new)
 +
        -v | --version                        Display version of the script and exit
 +
        -w | --workdir        WORKING_DIR      Working directory for the jupyter notebook/lab
 +
 +
Examlples:
 +
 +
        ./start_jupyter_nb.sh -u sfux -n 4 -W 04:00 -m 2048 -w /cluster/scratch/sfux
 +
 +
        ./start_jupyter_nb.sh --username sfux --numcores 2 --runtime 01:30 --memory 2048 --softwarestack new
 +
 +
        ./start_jupyter_nb.sh -c $HOME/.jnb_config
 +
 +
Format of configuration file:
 +
 +
JNB_USERNAME=""            # ETH username for SSH connection to Euler
 +
JNB_NUM_CPU=1              # Number of CPU cores to be used on the cluster
 +
JNB_NUM_GPU=0              # Number of GPUs to be used on the cluster
 +
JNB_RUN_TIME="01:00"        # Run time limit for jupyter notebook/lab in hours and minutes HH:MM
 +
JNB_MEM_PER_CPU_CORE=1024  # Memory limit in MB per core
 +
JNB_WAITING_INTERVAL=60    # Time interval to check if the job on the cluster already started
 +
JNB_SSH_KEY_PATH=""        # Path to SSH key with non-standard name
 +
JNB_SOFTWARE_STACK="new"    # Software stack to be used (old, new)
 +
JNB_WORKING_DIR=""          # Working directory for jupyter notebook/lab
 +
JNB_ENV=""                  # Path to virtual environment
 +
JNB_JLAB=""                # "lab" -> start jupyter lab; "" -> start jupyter notebook
  
−
where PATH needs to be replaced with the path in which the Jupyter notebook should start.
+
[[File:jupyter_on_Euler.png|thumb|none|800px|Example for running a Jupyter notebook on the Euler cluster]]
  
 
===Reconnect to a Jupyter notebook===
 
===Reconnect to a Jupyter notebook===
−
When running the script, it creates a local file called reconnect_info in the installation directory, which contains all information regarding the used ports, the remote ip address, the command for the SSH tunnel and the URL for the browser. This information should be sufficient to reconnect to a Jupyter notebook if connection was lost.
+
When running the script, it creates a local file called reconnect_info in the installation directory, which contains all information regarding the used ports, the remote IP address, the command for the SSH tunnel and the URL for the browser. This information should be sufficient to reconnect to a Jupyter notebook if connection was lost.
−
 
 
−
===Running multiple notebooks in a single Jupyter instance===
 
−
If you run Jupyter on the Leonhard cluster, using GPUs (the default version of the uses a python_cpu module, which does not support GPU usage. You would need to change the Python version in the script to enable GPU usage), then you need to make sure a notebook is correctly terminated before you can start another one.
 
−
 
 
−
If you don't properly close the first notebook and run a second one, then the previous notebook will still occupy some GPU memory and have processes running, which will throw some errors, when executing the second notebook.
 
−
 
 
−
Therefore please make sure that you stop running kernels in the "running" tab in the browser, before starting a new notebook.
 
  
 
===Terminate the Jupyter session===
 
===Terminate the Jupyter session===
Line 170: Line 194:
 
  samfux    8729  0.0  0.0  59404  6636 pts/5    S    13:46  0:00 ssh sfux@euler.ethz.ch -L 51339:10.205.4.122:8888 -N
 
  samfux    8729  0.0  0.0  59404  6636 pts/5    S    13:46  0:00 ssh sfux@euler.ethz.ch -L 51339:10.205.4.122:8888 -N
 
  samfux@bullvalene:~/jupyter-on-Euler-or-Leonhard-Open$ kill 8729
 
  samfux@bullvalene:~/jupyter-on-Euler-or-Leonhard-Open$ kill 8729
 +
 +
==Troubleshooting==
 +
===Logs===
 +
If the script does not work, then we recommend to check the logs. There is on one hand the file <tt>$HOME/jnbinfo</tt> which contains logs, and on the other hand there is the lsf.o* logfile of the job that was running the jupyter notebook.
 +
 +
===Mismatch of software stacks===
 +
In many cases it turns out that there are issues when the software stack used in the script does not match the default software stack that you have set. You can check which software stack is set as default by running the command
 +
 +
set_software_stack.sh -i
 +
 +
The script has an option (-s/--softwarestack) for which you can specify either "old" or "new". Please either adapt the software stack that you are requesting for the script or change the default setting for the software stack that you use on the cluster.
 +
 +
===Conflict with locally installed Python packages===
 +
When the jnbinfo contains a Python error with a backtrace, then it could be that some of the Python packages that you installed locally in <tt>$HOME/.local</tt> are conflicting with the centrally provided Jupyter installation.
 +
 +
==Jupyter in VSCode==
 +
Another possibility to work with jupyter notebooks on Euler is to run them in VSCode (https://code.visualstudio.com/docs/datascience/jupyter-notebooks). You can run VSCode (actually code-server) on Euler using the script:
 +
 +
https://scicomp.ethz.ch/wiki/VSCode

Revision as of 07:34, 11 August 2022

Introduction

Jupyter notebooks and Jupyter Lab are becoming more widely used among the scientific community at ETH. We therefore developed a script that you can run on your local computer to start Jupyter notebooks or Jupyter lab on Euler. The shell script starts a Jupyter in a batch job on Euler and connects your local browser with it.

The script runs on Linux, macOS and Windows (using WSL/WSL2 or git bash).

Please note, that with this script we are addressing beginners that start to use Jupyter notebooks on the cluster. It is not addressing advanced users that need a wide range of additional features going beyond simple Jupyter notebooks. Advanced users can take the script and adapt it, such that it can be used with other Python versions (centrally installed, or local installations) and add support for GPU, adding new kernels etc.

Installation

Download the repository with the commnad 

git clone https://gitlab.ethz.ch/sfux/Jupyter-on-Euler-or-Leonhard-Open

Mac OS X: 

git clone https://gitlab.ethz.ch/sfux/Jupyter-on-Euler-or-Leonhard-Open.git

After downloading the script from gitlab.ethz.ch, you need to change its permissions to make it executable cd Jupyter-on-Euler-or-Leonhard-Open/ chmod 755 start_jupyter_nb.sh

Prerequisites

In order to use this script, users need to make sure, that they have set up SSH keys for passwordless access to the cluster: 

https://scicomp.ethz.ch/wiki/Accessing_the_clusters#SSH_keys

Please make sure that xdg-open is installed. This package is used to automatically start your default browser. You can install it with the following command:

CentOS: 

yum install xdg-utils

Ubuntu: 

apt-get install xdg-utils

Further more, the script requires that there is a Python installation available, which is usually included in the Linux distribution or Mac OS.

Security token vs. password setup

Please note that a part of the script (parsing of the ports) requires that you use jupyter notebooks with the security tokens. If you configure a password instead, such that you can use the jupyter notebook without the security token, then the script will not work anymore (it cannot parse the port on the remote compute node) without adapting it.

Download and setup

The script is available on the Gitlab instance of ETH Zurich: 

https://gitlab.ethz.ch/sfux/Jupyter-on-Euler-or-Leonhard-Open

Download the repository with the command 

git clone https://gitlab.ethz.ch/sfux/Jupyter-on-Euler-or-Leonhard-Open

Mac OS X: 

git clone https://gitlab.ethz.ch/sfux/Jupyter-on-Euler-or-Leonhard-Open.git

After downloading the script from gitlab.ethz.ch, you need to change its permissions to make it executable 

chmod 755 start_jupyter_nb.sh

Software stack

Please note that currently the old software stack is still set a default (this will change). The script is using the new software stack (unless you explicitly request the old software stack with the option -s old (or --softwarestack old). Therefore please make sure that you set the new software stack as permanent default by using the command 

set_software_stack.sh new

You can find more information about this script on our wiki: 

https://scicomp.ethz.ch/wiki/Setting_permanent_default_for_software_stack_upon_login

Creating SSH keys for the Jupyter script with a Windows computer (using git-bash)

With git-bash, you can use the same command as for Linux/Mac to create your SSH keys:

  • Start git-bash
  • Run the command ssh-keygen to create the SSH key pair
ssh-keygen -t ed25519 -f $HOME/.ssh/id_ed25519_euler
  • Enter a passphrase to protect your SSH keys
  • Use the ssh-copy-id command to copy your public (never your private) key to Euler
ssh-copy-id -i ~/.ssh/id_ed25519_euler.pub username@euler.ethz.ch
  • (here you need to replace username with your ETH username)
  • Enter your password

After creating the keys, we strongly recommend to create an SSH config file in $HOME/.ssh

  • Change to your .ssh directory
cd $HOME/.ssh
  • Create an SSH config file to specify the path to your SSH key
echo -e "Host euler\nHostName euler.ethz.ch\nUser username\nIdentityFile ~/.ssh/id_ed25519_euler" > config
  • (here you need to replace username with your ETH username)

At this stage, passwordless login to Euler should work. You can test this with the command 

ssh username@euler.ethz.ch
  • (here you need to replace username with your ETH username)

But now you still have to enter your passphrase. In order to not have to type your passphrase all the time, you can use an SSH agent, which is already included in git-bash

  • Start git-bash
  • Start the SSH agent for the current terminal
ssh-agent /bin/bash
  • Add your SSH key to the SSH agent
ssh-add ~/.ssh/id_ed25519_euler
  • Enter your passphrase

As long as this terminal is open, you can access Euler with 

ssh username@euler.ethz.ch

(here you need to replace username with your ETH username) without having to type a password or passphrase. As soon as you exit the terminal, then the SSH agent will be stopped.

Now you are ready to clone the Jupyter script and to execute it in the terminal with the SSH agent. Please note that you only need to create the SSH once, but you need to setup the SSH agent every time you would like to run the Jupyter script.

Using SSH keys with non-default names

Since the reopening of Euler and Leonhard Open after the cyber attack in May 2020, we recommend to the cluster users to use SSH keys. We recommend to use different keys for Euler and Leonhard Open, with according names 

$HOME/.ssh/id_ed25519_euler

In order to use this keys with the jupyter script, then there are two options. You can configure your ssh client to use this option automatically by adding the following lines in the $HOME/.ssh/config file on your local computer: 

Host euler.ethz.ch
IdentityFile ~/.ssh/id_ed25519_euler

Both options work and are equivalent.

Installing additional Python packages locally

When starting a Jupyter notebook with this script, then it will use a central Python installation:

  • New software stack: module load gcc/6.3.0 python/3.8.5
  • Old software stack: module load new gcc/4.8.2 python/3.6.1

Therefore you can only use packages that are centrally installed out-of-the-box. But you have the option to install additional packages locally in your home directory, which can afterwards be used.

For installing a Python package from inside a Jupyter notebook, you would need to run the following command: 

!pip install --user package_name

This will install package_name into $HOME/.local, as described on our wiki page about Python: 

https://scicomp.ethz.ch/wiki/Python#Installing_a_Python_package.2C_using_PIP

Running the script

The start_jupyer_nb.sh script needs to be executed on your local computer. Please find below the list of options that can be used with the script: 

$ ./start_jupyter_nb.sh -h
./start_jupyter_nb.sh: Script to start jupyter notebook/lab on Euler from a local computer
Usage: start_jupyter_nb.sh [options]

Options:

        -u | --username       USERNAME         ETH username for SSH connection to Euler
        -n | --numcores       NUM_CPU          Number of CPU cores to be used on the cluster
        -W | --runtime        RUN_TIME         Run time limit for jupyter notebook/lab in hours and minutes HH:MM
        -m | --memory         MEM_PER_CORE     Memory limit in MB per core

Optional arguments:

        -c | --config         CONFIG_FILE      Configuration file for specifying options
        -e | --environment    ENV              Python virtual environment
        -g | --numgpu         NUM_GPU          Number of GPUs to be used on the cluster
        -h | --help                            Display help for this script and quit
        -i | --interval       INTERVAL         Time interval for checking if the job on the cluster already started
        -k | --key            SSH_KEY_PATH     Path to SSH key with non-standard name
        -l | --lab                             Start jupyter lab instead of a jupyter notebook
        -s | --softwarestack  SOFTWARE_STACK   Software stack to be used (old, new)
        -v | --version                         Display version of the script and exit
        -w | --workdir        WORKING_DIR      Working directory for the jupyter notebook/lab

Examlples:

        ./start_jupyter_nb.sh -u sfux -n 4 -W 04:00 -m 2048 -w /cluster/scratch/sfux

        ./start_jupyter_nb.sh --username sfux --numcores 2 --runtime 01:30 --memory 2048 --softwarestack new

        ./start_jupyter_nb.sh -c $HOME/.jnb_config

Format of configuration file:

JNB_USERNAME=""             # ETH username for SSH connection to Euler
JNB_NUM_CPU=1               # Number of CPU cores to be used on the cluster
JNB_NUM_GPU=0               # Number of GPUs to be used on the cluster
JNB_RUN_TIME="01:00"        # Run time limit for jupyter notebook/lab in hours and minutes HH:MM
JNB_MEM_PER_CPU_CORE=1024   # Memory limit in MB per core
JNB_WAITING_INTERVAL=60     # Time interval to check if the job on the cluster already started
JNB_SSH_KEY_PATH=""         # Path to SSH key with non-standard name
JNB_SOFTWARE_STACK="new"    # Software stack to be used (old, new)
JNB_WORKING_DIR=""          # Working directory for jupyter notebook/lab
JNB_ENV=""                  # Path to virtual environment
JNB_JLAB=""                 # "lab" -> start jupyter lab; "" -> start jupyter notebook
Example for running a Jupyter notebook on the Euler cluster

Reconnect to a Jupyter notebook

When running the script, it creates a local file called reconnect_info in the installation directory, which contains all information regarding the used ports, the remote IP address, the command for the SSH tunnel and the URL for the browser. This information should be sufficient to reconnect to a Jupyter notebook if connection was lost.

Terminate the Jupyter session

Please note that when you finish working with the jupyter notebook, you need to click on the "Quit" or "Logout" button in your Browser. "Quit" will stop the batch job running on Euler, "Logout" will just log you out from the session but not stop the batch job (in this case you need to login to the cluster, identify the job with bjobs and then kill it with the bkill command, using the jobid as parameter). Afterwards you also need to clean up the SSH tunnel that is running in the background.

Example: 

samfux@bullvalene:~/Jupyter-on-Euler-or-Leonhard-Open$ ps -u | grep -m1 -- "-L" | grep -- "-N"
samfux    8729  0.0  0.0  59404  6636 pts/5    S    13:46   0:00 ssh sfux@euler.ethz.ch -L 51339:10.205.4.122:8888 -N
samfux@bullvalene:~/jupyter-on-Euler-or-Leonhard-Open$ kill 8729

Troubleshooting

Logs

If the script does not work, then we recommend to check the logs. There is on one hand the file $HOME/jnbinfo which contains logs, and on the other hand there is the lsf.o* logfile of the job that was running the jupyter notebook.

Mismatch of software stacks

In many cases it turns out that there are issues when the software stack used in the script does not match the default software stack that you have set. You can check which software stack is set as default by running the command 

set_software_stack.sh -i

The script has an option (-s/--softwarestack) for which you can specify either "old" or "new". Please either adapt the software stack that you are requesting for the script or change the default setting for the software stack that you use on the cluster.

Conflict with locally installed Python packages

When the jnbinfo contains a Python error with a backtrace, then it could be that some of the Python packages that you installed locally in $HOME/.local are conflicting with the centrally provided Jupyter installation.

Jupyter in VSCode

Another possibility to work with jupyter notebooks on Euler is to run them in VSCode (https://code.visualstudio.com/docs/datascience/jupyter-notebooks). You can run VSCode (actually code-server) on Euler using the script: 

https://scicomp.ethz.ch/wiki/VSCode
Retrieved from "https://scicomp.ethz.ch/w/index.php?title=Jupyter_on_Euler_and_Leonhard_Open&oldid=8146"