Difference between revisions of "GPU job submission"
From ScientificComputing
| Line 13: | Line 13: | ||
</tr> | </tr> | ||
</table> | </table> | ||
| + | |||
| Line 25: | Line 26: | ||
While your jobs will see all GPUs, LSF will set the [https://devblogs.nvidia.com/parallelforall/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/ CUDA_VISIBLE_DEVICES] environment variable, which is honored by CUDA programs. | While your jobs will see all GPUs, LSF will set the [https://devblogs.nvidia.com/parallelforall/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/ CUDA_VISIBLE_DEVICES] environment variable, which is honored by CUDA programs. | ||
| + | |||
| + | ==Available GPU node types== | ||
| + | {| class="wikitable" | ||
| + | |- | ||
| + | ! GPU Model !! Specifier !! GPU memory per GPU !! CPU cores per node !! CPU memory per node | ||
| + | |- | ||
| + | | NVIDIA GeForce GTX 1080 || <tt>GeForceGTX1080</tt> || 8 GiB || 20 || 256 GiB | ||
| + | |- | ||
| + | | NVIDIA GeForce GTX 1080 Ti || <tt>GeForceGTX1080Ti</tt> || 11 GiB || 20 || 256 GiB | ||
| + | |- | ||
| + | | NVIDIA GeForce RTX 2080 Ti || <tt>GeForceRTX2080Ti</tt> || 11 GiB || 36 || 384 GiB | ||
| + | |- | ||
| + | | NVIDIA GeForce RTX 2080 Ti || <tt>GeForceRTX2080Ti</tt> || 11 GiB || 128 || 512 GiB | ||
| + | |- | ||
| + | | NVIDIA TITAN RTX || <tt>TITANRTX</tt> || 24 GiB || 128 || 512 GiB | ||
| + | |- | ||
| + | | [[Nvidia_DGX-1_with_Tensor_Cores| NVIDIA Tesla V100-SXM2 32 GB]] || <tt>TeslaV100_SXM2_32GB</tt> || 32 GiB || 48 || 768 GiB | ||
| + | |- | ||
| + | | NVIDIA Tesla A100 || <tt> A100_PCIE_40GB </tt> || 40 GiB || 48 || 768 GiB | ||
| + | |} | ||
| + | |||
== Example == | == Example == | ||
Revision as of 12:33, 17 August 2021
To use the GPUs for a job node you need to request the ngpus_excl_p resource. It refers to the number of GPUs per node. This is unlike other resources, which are requested per core.
For example, to run a serial job with one GPU,
$ bsub -R "rusage[ngpus_excl_p=1]" ./my_cuda_program
or on a full node with all 8 GeForce GTX 1080 Ti GPUs and up to 90 GB of RAM,
$ bsub -n 20 -R "rusage[mem=4500,ngpus_excl_p=8]" -R "select[gpu_model0==GeForceGTX1080Ti]" ./my_cuda_program
or on two full nodes:
$ bsub -n 40 -R "rusage[mem=4500,ngpus_excl_p=8]" -R "select[gpu_model0==GeForceGTX1080Ti]" -R "span[ptile=20]" ./my_cuda_program
While your jobs will see all GPUs, LSF will set the CUDA_VISIBLE_DEVICES environment variable, which is honored by CUDA programs.
Available GPU node types
| GPU Model | Specifier | GPU memory per GPU | CPU cores per node | CPU memory per node |
|---|---|---|---|---|
| NVIDIA GeForce GTX 1080 | GeForceGTX1080 | 8 GiB | 20 | 256 GiB |
| NVIDIA GeForce GTX 1080 Ti | GeForceGTX1080Ti | 11 GiB | 20 | 256 GiB |
| NVIDIA GeForce RTX 2080 Ti | GeForceRTX2080Ti | 11 GiB | 36 | 384 GiB |
| NVIDIA GeForce RTX 2080 Ti | GeForceRTX2080Ti | 11 GiB | 128 | 512 GiB |
| NVIDIA TITAN RTX | TITANRTX | 24 GiB | 128 | 512 GiB |
| NVIDIA Tesla V100-SXM2 32 GB | TeslaV100_SXM2_32GB | 32 GiB | 48 | 768 GiB |
| NVIDIA Tesla A100 | A100_PCIE_40GB | 40 GiB | 48 | 768 GiB |
Example
Further reading
