Difference between revisions of "Unified quota wrapper"

Revision as of 16:20, 6 March 2019

Introduction

We have implemented a unified quota wrapper lquota that can be used to query the space and file quota on all file systems on our clusters:

• /cluster/home/$USER ($HOME, home directory)
• /cluster/scratch/$USER ($SCRATCH, personal scratch directory)
• /cluster/shadow (buffering stdout of running batch jobs)
• /cluster/work (shareholder storage)
• /cluster/project (shareholder storage)

Units

The quota wrapper can display the values either in Bytes, human readable format (base 10) or human readable format (base 2)

Base 10

Name	Prefix	Size
Kilobyte	kB	bytes
Megabyte	MB	bytes
Gigabyte	GB	bytes
Terabyte	TB	bytes
Petabyte	PB	bytes

Base 2

Name	Prefix	Size
Kibibyte	KiB	bytes
Mebibyte	MiB	bytes
Gibibyte	GiB	bytes
Tebibyte	TiB	bytes
Pebibyte	PiB	bytes

Documentation

You can display the documentation of the command by running it with the parameter -h:

[sfux@eu-login-11-ng ~]$ lquota -h
/cluster/apps/local/lquota: Command to check quotas on distruted filesystems
Default: values are diplayed in decimal units

Usage: lquota [-h] [-2] [-b] [-p] [directory]

-h    display this help message and exit
-2    display space in binary units
-b    display space in bytes
-p    display the output in parsable format

If "directory" is not specified user's home and scratch directory are used as argument
[sfux@eu-login-11-ng ~]$ 

There are space quotas and inode quotas. An inode is a data structure to organize data in a file system. Each file or directory has one inode. Therefore a quota for inodes actually means that you can only have a limited number of files and directories in a storage share. For the home directories, we use a quota of 100'000 inodes, which implies that you cannot have more than 100'000 files and directories in your home directory.

Examples

If the command is run without specifying a particular path, then it displays the quota for the home directory, the personal scratch directory and the shadow file system in base 10 units:

[sfux@eu-login-11-ng ~]$ lquota
+-----------------------------+-------------+------------------+------------------+------------------+
| Storage location:           | Quota type: | Used:            | Soft quota:      | Hard quota:      |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/home/sfux          | space       |          7.66 GB |         17.18 GB |         21.47 GB |
| /cluster/home/sfux          | files       |            20724 |            80000 |           100000 |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/shadow             | space       |          4.10 kB |          2.15 GB |          2.15 GB |
| /cluster/shadow             | files       |                2 |            50000 |            50000 |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/scratch/sfux       | space       |        237.57 kB |          2.50 TB |          2.70 TB |
| /cluster/scratch/sfux       | files       |               29 |          1000000 |          1500000 |
+-----------------------------+-------------+------------------+------------------+------------------+
[sfux@eu-login-11-ng ~]$ 

For output in base 2 units, please use the option -2:

[sfux@eu-login-11-ng ~]$ lquota -2
+-----------------------------+-------------+------------------+------------------+------------------+
| Storage location:           | Quota type: | Used:            | Soft quota:      | Hard quota:      |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/home/sfux          | space       |         7.14 GiB |        16.00 GiB |        20.00 GiB |
| /cluster/home/sfux          | files       |            20724 |            80000 |           100000 |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/shadow             | space       |         4.00 KiB |         2.00 GiB |         2.00 GiB |
| /cluster/shadow             | files       |                2 |            50000 |            50000 |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/scratch/sfux       | space       |       232.00 KiB |         2.27 TiB |         2.46 TiB |
| /cluster/scratch/sfux       | files       |               29 |          1000000 |          1500000 |
+-----------------------------+-------------+------------------+------------------+------------------+
[sfux@eu-login-11-ng ~]$

For output in bytes, please use the option -b

[sfux@eu-login-11-ng ~]$ lquota -b
+-----------------------------+-------------+------------------+------------------+------------------+
| Storage location:           | Quota type: | Used:            | Soft quota:      | Hard quota:      |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/home/sfux          | space       |       7662727168 |      17179869184 |      21474836480 |
| /cluster/home/sfux          | files       |            20723 |            80000 |           100000 |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/shadow             | space       |             4096 |       2147483648 |       2147483648 |
| /cluster/shadow             | files       |                2 |            50000 |            50000 |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/scratch/sfux       | space       |           237568 |    2499670966272 |    2699386945536 |
| /cluster/scratch/sfux       | files       |               29 |          1000000 |          1500000 |
+-----------------------------+-------------+------------------+------------------+------------------+
[sfux@eu-login-11-ng ~]$

For parsable output, please use the option -p

[sfux@eu-login-11-ng ~]$ lquota -p
  Storage location:             Quota type:   Used:              Soft quota:        Hard quota:       
  /cluster/home/sfux            space               7662727168        17179869184        21474836480  
  /cluster/home/sfux            files                    20723              80000             100000  
  /cluster/shadow               space                     4096         2147483648         2147483648  
  /cluster/shadow               files                        2              50000              50000  
  /cluster/scratch/sfux         space                   237568      2499670966272      2699386945536  
  /cluster/scratch/sfux         files                       29            1000000            1500000  
[sfux@eu-login-11-ng ~]$ 

If a path is specified as argument to the script, then it will display the inode and space quota for this particular storage share:

[sfux@eu-login-11-ng ~]$ lquota /cluster/project/sis
+-----------------------------+-------------+------------------+------------------+------------------+
| Storage location:           | Quota type: | Used:            | Soft quota:      | Hard quota:      |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/project/sis        | space[B]    |          6.17 TB |                - |         10.41 TB |
| /cluster/project/sis        | files       |          1155583 |                - |         30721113 |
+-----------------------------+-------------+------------------+------------------+------------------+
[sfux@eu-login-11-ng ~]$

[sfux@eu-login-11-ng ~]$ lquota /cluster/work/sis
+-----------------------------+-------------+------------------+------------------+------------------+
| Storage location:           | Quota type: | Used:            | Soft quota:      | Hard quota:      |
+-----------------------------+-------------+------------------+------------------+------------------+
| /cluster/work/sis           | space       |          8.36 TB |         10.00 TB |         11.00 TB |
| /cluster/work/sis           | files       |          1142478 |         10000000 |         11000000 |
+-----------------------------+-------------+------------------+------------------+------------------+
[sfux@eu-login-11-ng ~]$

For the moment, you might need to use the full path /cluster/apps/local/lquota to run the script. We will add it to the users path soon, such that you will not have to use the full path.