Storage and data transfer
$ cd $HOME $ pwd /cluster/home/username
$HOME is a safe, long-term storage for critical data (program source, scripts, etc.) and is accessible only by the user (owner). This means other people cannot read its contents.
There is a disk quota of 16/20 GB and a maximum of 80’000/100’000 files (soft/hard quota). You can check the quota with the command
Its content is saved every hour/day/week using snapshot, which is stored in the hidden .snapshot directory.
$ cd $SCRATCH $ pwd /cluster/scratch/username
$SCRATCH is a fast, short-term storage for computations running on the cluster. It is created automatically upon first access (cd $SCRATCH) and visible (mounted) only when accessed.
It has strict usage rules (see $SCRATCH/__USAGE_RULES__ for details) and has no backup.
/scratch on each compute node ($TMPDIR)
The local scratch is intended for serial, I/O-intensive applications. Therefore, it has a very short life span. Data are deleted automatically when the job ends.
Scratch space must be requested by the job and has no backup.
$ cd /cluster/project/groupname
Similar to $HOME, but for groups, it is a safe, long-term storage for critical data.
$ cd /cluster/work/groupname
Similar to global scratch, but without purge, it is a fast, short-or medium-term storage for large computations.
The folder is visible only when accessed.
Groups who have purchased storage on the central NAS of ETH can access it on our clusters.
The NAS share needs to be mountable via NFS (shares that only support CIFS cannot be mounted on the HPC clusters), and exported to the subnet of our HPC clusters (please contact ID Systemdienste and ask them for an NFS export of your NAS share).
The NAS share is mounted automatically on our clusters under
Groups who are operating their own NAS can export a shared file system via NFS to Euler. The user and group ID's on the NAS needs to be consistent with ETH user names and groups.
NAS needs to support NFSv3 (this is currently the only NFS version that is supported from our side) and needs to be exported to the subnet of our HPC clusters.
The NAS is then mounted automatically on our clusters under