Storage systems

From ScientificComputing
Revision as of 06:03, 31 August 2016 by Sfux (talk | contribs)

Jump to: navigation, search

Introduction

On our cluster, we provide multiple storage systems, which are optimized for different purposes. Since the available storage space on our clusters is limited and shared between all users, we set quotas in order to prevent single users from filling up an entire storage system with their data.

A summary of general questions about file systems, storage and file transfer can be found in our FAQ. If you have questions or encounter problems with the storage systems provided on our clusters or file transfer, then please contact cluster support.

Personal storage (everyone)

Home

On our clusters, we provide a home directory (folder) for every user that can be used for safe long term storage of important and critical data (program source, script, input file, etc.). It is created on your first login to the cluster and accessible through the path

/cluster/home/username

The path is also saved in the variable $HOME. The permissions are set that only you can access the data in your home directory and no other user. Your home directory is limited to 16 GB and a maximum of 100'000 files and directories (inodes). The content of your home is saved every hour and there is also a nightly backup (tape).

Scratch

We also provide a personal scratch directory (folder) for every user, that can be used for short-term storage of larger amounts of data. It is created, when you access it the first time through the path

/cluster/scratch/username

The path is also saved in the variable $SCRATCH. It is visible (mounted), only when you access it. If you try to access it with a graphical tool, then you need to specify the full path as it is might not visible in the /cluster/scratch top-level directory. Before you use your personal scratch directory, please carefully read the usage rules to avoid misunderstandings. The usage rules can also be displayed directly on the cluster with the following command.

cat $SCRATCH/__USAGE_RULES__

Your personal scratch directory has a disk quota of 2.5 TB and a maximum of 1'000'000 files and directories (inodes). There is no backup for the personal scratch directories and they are purged on a regular basis (see usage rules).

Group storage (sharholders only)

Project

Work

Local scratch (on each compute node)

/scratch
$TMPDIR

External storage

Central NAS

  • Mountable via NFS
  • Export to Euler subnet
  • Contact Systemdienste

Local NAS

  • NFSv3
  • NETHZ user names and groups
  • Export to Euler subnet (put example for export file), check if load balancer is part of the Euler subnet.

Comparison

Put table from presentation here

Choosing the optimal storage system

File transfer

Command line tools

Graphical tools

Quotas