Difference between revisions of "Storage systems"

From ScientificComputing
Jump to: navigation, search
Line 8: Line 8:
 
{| class="wikitable sortable" border=1
 
{| class="wikitable sortable" border=1
 
|-
 
|-
! Brutus ! Mountpoint !! Filesystem type !! Capacity !! Backup !! Purged !! Should be used for
+
! Brutus !! Mountpoint !! Filesystem type !! Capacity !! Backup !! Purged !! Should be used for
 
|-
 
|-
 
|   | /cluster/home || NFS (ZFS) || 32TB || Yes || Never || Important files, medium and large files
 
|   | /cluster/home || NFS (ZFS) || 32TB || Yes || Never || Important files, medium and large files

Revision as of 08:50, 30 August 2016

Introduction

On our cluster, we provide multiple storage systems, which are optimized for different purposes. On this page, we try to give you an overview on the different storage systems and provide you all the information that is needed to choose the most optimal file system for your purpose. Some of the storage systems are optimized for large files and high performance, whereas others can also deal with small files. Some are for permanent storage of data (homes) whereas the scratch file systems are only for temporary storage of data. Since the available storage space on our clusters is limited and shared between all users, we set quotas in order to prevent single users from filling up an entire storage system with their data. A summary of general questions about file systems and storage can be found in our FAQ. If you have questions or encounter problems with the storage systems provided on our clusters, then please contact cluster support.

Available storage systems

First of all, we would like to start with an overview on the available storage systems in Brutus. The description of the storage systems contains link that redirect you to more detailed wiki pages on these systems.

Brutus:

Brutus Mountpoint Filesystem type Capacity Backup Purged Should be used for
/cluster/home NFS (ZFS) 32TB Yes Never Important files, medium and large files
/cluster/work Panasas 76TB No scr[1-6] : files older than 15 days Parallel I/O, medium and large files (> 10MB)
/cluster/scratch_xp Lustre 407TB No public : files older than 15 days Parallel I/O, large files (> 50MB)
/cluster/scratch_xl Lustre 407TB No No Parallel I/O, large files (> 50MB)
/scratch ext3 (LOCAL scratch) 10GB - 1TB No After your job finishes Single node I/O during a job, small files OK

Central storage from ID storage

You can buy central NAS shares from the ID storage group. These shares are also accessible on our clusters. When you access your central NAS share on one of our cluster, then it is mounted automatically. Please make sure that the permissions on the NAS share and the NFS export are done properly.

User storage systems

If you have your own NAS storage, managed by yourself, then you can also mount it on our clusters, but we do not provide any support for it. We only support NAS shares that are managed by the ID storage group.

Choosing the optimal storage system

File transfer

Quotas