Unified quota wrapper

From ScientificComputing
Jump to: navigation, search

Introduction

We have implemented a unified quota wrapper lquota that can be used to query the space and file quota on all file systems on our clusters:

  • /cluster/home/$USER ($HOME, home directory)
  • /cluster/scratch/$USER ($SCRATCH, personal scratch directory)
  • /cluster/shadow (buffering stdout of running batch jobs)
  • /cluster/work (shareholder storage)
  • /cluster/project (shareholder storage)

Units

The quota wrapper can display the space quotas either in bytes, in base 10 units or base 2 units.

Base 10

Name Prefix Size
Kilobyte kB 10^3 bytes
Megabyte MB 10^6 bytes
Gigabyte GB 10^9 bytes
Terabyte TB 10^{12} bytes
Petabyte PB 10^{15} bytes

Base 2

Name Prefix Size
Kibibyte KiB 2^{10} bytes
Mebibyte MiB 2^{20} bytes
Gibibyte GiB 2^{30} bytes
Tebibyte TiB 2^{40} bytes
Pebibyte PiB 2^{50} bytes

Documentation

You can display the documentation of the command by running it with the parameter -h:

[sfux@eu-login-11-ng ~]$ lquota -h
/cluster/apps/local/lquota: Command to check quotas on distruted filesystems
Default: values are diplayed in decimal units

Usage: lquota [-h] [-2] [-b] [-p] [directory]

-h    display this help message and exit
-2    display space in binary units
-b    display space in bytes
-p    display the output in parsable format

If "directory" is not specified user's home and scratch directory are used as argument
[sfux@eu-login-11-ng ~]$ 

There are space quotas and inode quotas. An inode is a data structure to organize data in a file system. Each file or directory has one inode. Therefore a quota for inodes actually means that you can only have a limited number of files and directories in a storage share. For the home directories, we use a quota of 100'000 inodes, which implies that you cannot have more than 100'000 files and directories in your home directory.

Examples

If the command is run without specifying a particular path, then it displays the quota for the home directory, the personal scratch directory and the shadow file system in base 10 units:

[sfux@eu-login-11-ng ~]$ lquota
+-----------------------------+-------------+------------------+------------------+------------------+
| Storage location:           | Quota type: | Used:            | Soft quota:      | Hard quota:      |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/home/sfux          | space       |          7.66 GB |         17.18 GB |         21.47 GB |
| /cluster/home/sfux          | files       |            20724 |            80000 |           100000 |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/shadow             | space       |          4.10 kB |          2.15 GB |          2.15 GB |
| /cluster/shadow             | files       |                2 |            50000 |            50000 |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/scratch/sfux       | space       |        237.57 kB |          2.50 TB |          2.70 TB |
| /cluster/scratch/sfux       | files       |               29 |          1000000 |          1500000 |
+-----------------------------+-------------+------------------+------------------+------------------+
[sfux@eu-login-11-ng ~]$ 

For output in base 2 units, please use the option -2:

[sfux@eu-login-11-ng ~]$ lquota -2
+-----------------------------+-------------+------------------+------------------+------------------+
| Storage location:           | Quota type: | Used:            | Soft quota:      | Hard quota:      |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/home/sfux          | space       |         7.14 GiB |        16.00 GiB |        20.00 GiB |
| /cluster/home/sfux          | files       |            20724 |            80000 |           100000 |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/shadow             | space       |         4.00 KiB |         2.00 GiB |         2.00 GiB |
| /cluster/shadow             | files       |                2 |            50000 |            50000 |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/scratch/sfux       | space       |       232.00 KiB |         2.27 TiB |         2.46 TiB |
| /cluster/scratch/sfux       | files       |               29 |          1000000 |          1500000 |
+-----------------------------+-------------+------------------+------------------+------------------+
[sfux@eu-login-11-ng ~]$

For output in bytes, please use the option -b

[sfux@eu-login-11-ng ~]$ lquota -b
+-----------------------------+-------------+------------------+------------------+------------------+
| Storage location:           | Quota type: | Used:            | Soft quota:      | Hard quota:      |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/home/sfux          | space       |       7662727168 |      17179869184 |      21474836480 |
| /cluster/home/sfux          | files       |            20723 |            80000 |           100000 |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/shadow             | space       |             4096 |       2147483648 |       2147483648 |
| /cluster/shadow             | files       |                2 |            50000 |            50000 |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/scratch/sfux       | space       |           237568 |    2499670966272 |    2699386945536 |
| /cluster/scratch/sfux       | files       |               29 |          1000000 |          1500000 |
+-----------------------------+-------------+------------------+------------------+------------------+
[sfux@eu-login-11-ng ~]$

For parsable output, please use the option -p

[sfux@eu-login-11-ng ~]$ lquota -p
  Storage location:             Quota type:   Used:              Soft quota:        Hard quota:       
  /cluster/home/sfux            space               7662727168        17179869184        21474836480  
  /cluster/home/sfux            files                    20723              80000             100000  
  /cluster/shadow               space                     4096         2147483648         2147483648  
  /cluster/shadow               files                        2              50000              50000  
  /cluster/scratch/sfux         space                   237568      2499670966272      2699386945536  
  /cluster/scratch/sfux         files                       29            1000000            1500000  
[sfux@eu-login-11-ng ~]$ 

If a path is specified as argument to the script, then it will display the inode and space quota for this particular storage share:

[sfux@eu-login-11-ng ~]$ lquota /cluster/project/sis
+-----------------------------+-------------+------------------+------------------+------------------+
| Storage location:           | Quota type: | Used:            | Soft quota:      | Hard quota:      |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/project/sis        | space[B]    |          6.17 TB |                - |         10.41 TB |
| /cluster/project/sis        | files       |          1155583 |                - |         30721113 |
+-----------------------------+-------------+------------------+------------------+------------------+
[sfux@eu-login-11-ng ~]$
[sfux@eu-login-11-ng ~]$ lquota /cluster/work/sis
+-----------------------------+-------------+------------------+------------------+------------------+
| Storage location:           | Quota type: | Used:            | Soft quota:      | Hard quota:      |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/work/sis           | space       |          8.36 TB |         10.00 TB |         11.00 TB |
| /cluster/work/sis           | files       |          1142478 |         10000000 |         11000000 |
+-----------------------------+-------------+------------------+------------------+------------------+
[sfux@eu-login-11-ng ~]$

For the moment, you might need to use the full path /cluster/apps/local/lquota to run the script. We will add it to the users path soon, such that you will not have to use the full path.