Group for files and directories in work and project shares

Introduction

Shareholder groups have the possibility to purchase project (/cluster/project) and work (/cluster/work) storage on the Euler cluster. The group provides an ETH group that contains all users that should have access to this storage location.

The group ownership of top-level directory is set to this group and we also set the set group id bit (https://de.wikipedia.org/wiki/Setgid). This way files and directories created under this top-level directory inherit the group from the parent directory:

[sfux@eu-login-29 ~]$ ls -ltrd /cluster/project/sis/hpc
drwxrwsr-t 10 byrdeo 06087 4096 Jun  2  2022 /cluster/project/sis/hpc
[sfux@eu-login-29 ~]$ ls -ltrd /cluster/project/sis/hpc/sfux
drwxr-sr-x 3 sfux 06087 4096 Jan 28  2022 /cluster/project/sis/hpc/sfux
[sfux@eu-login-29 ~]$

This setup facilitates sharing data within a research group as you only have to change the group permissions to make your files accessible to your colleagues.

Inheriting the group from the parent directory (due to setgid) in Linux

This feature of the setgid bit depends on what command is used to bring in the data to the storage share.

• Directories created with mkdir inherit the group
• Files copied with cp inherit the group
• Files transfered with mv do not inherit the group
• For files transferred with rsync, it depends on the command line options if the group is inherited or not. Using the option -p or any option that applies -p (like -a) should preserve the setgid bit and the group in the destination

When you for instance use mv to transfer files, then the files will keep the original group instead of inheriting the group from the parent directory.

Example:

[sfux@eu-login-20 ~]$ cd /cluster/project/sis/hpc/sfux
[sfux@eu-login-20 sfux]$ ls -ltrd .
drwxr-sr-x 2 sfux 06087 4096 Mar  6  2020 .
[sfux@eu-login-20 sfux]$ mkdir test1
[sfux@eu-login-20 sfux]$ cp -r $HOME/test2 .
[sfux@eu-login-20 sfux]$ mv $HOME/test3 .
[sfux@eu-login-20 sfux]$ ls -ltrd *
drwxr-sr-x 2 sfux 06087      4096 Sep 16 11:38 test1
drwxr-sr-x 2 sfux 06087      4096 Sep 16 11:38 test2
drwxr-xr-x 2 sfux sfux-group 4096 Sep 16 11:38 test3
[sfux@eu-login-20 sfux]$

Therefore, please carefully check how to transfer data to your work or project storage share in case you would like to share those with your colleagues.

How to change the group back

If you notice that some files in your project/work share have a group that is different from the group of the top-level directory, then you can change this back using the command chgrp or chown

[sfux@eu-login-29 sfux]$ pwd
/cluster/project/sis/hpc/sfux
[sfux@eu-login-29 sfux]$ ls -ltrd .
drwxr-sr-x 2 sfux 06087 4096 Feb 23 14:47 .
[sfux@eu-login-29 sfux]$ ls -l
total 0
-rw-r--r-- 1 sfux sfux-group 9 Feb 23 14:47 test1
-rw-r--r-- 1 sfux sfux-group 9 Feb 23 14:47 test2
[sfux@eu-login-29 sfux]$ chgrp 06087 test1
[sfux@eu-login-29 sfux]$ chown sfux:06087 test2
[sfux@eu-login-29 sfux]$ ls -l
total 0
-rw-r--r-- 1 sfux 06087 9 Feb 23 14:47 test1
-rw-r--r-- 1 sfux 06087 9 Feb 23 14:47 test2
[sfux@eu-login-29 sfux]$

Please note that for changing the group with chgrp it is sufficient to specify the group, whereas for chown you would need to specify the username and the group.