Difference between revisions of "GPU job submission"

From ScientificComputing
Jump to: navigation, search
Line 13: Line 13:
 
</tr>
 
</tr>
 
</table>
 
</table>
 +
  
  
Line 25: Line 26:
  
 
While your jobs will see all GPUs, LSF will set the [https://devblogs.nvidia.com/parallelforall/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/ CUDA_VISIBLE_DEVICES] environment variable, which is honored by CUDA programs.
 
While your jobs will see all GPUs, LSF will set the [https://devblogs.nvidia.com/parallelforall/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/ CUDA_VISIBLE_DEVICES] environment variable, which is honored by CUDA programs.
 +
 +
==Available GPU node types==
 +
{| class="wikitable"
 +
|-
 +
! GPU Model !! Specifier !! GPU memory per GPU !! CPU cores per node !! CPU memory per node
 +
|-
 +
| NVIDIA GeForce GTX 1080 || <tt>GeForceGTX1080</tt> || 8&nbsp;GiB || 20 || 256&nbsp;GiB
 +
|-
 +
| NVIDIA GeForce GTX 1080 Ti || <tt>GeForceGTX1080Ti</tt> || 11&nbsp;GiB || 20 || 256&nbsp;GiB
 +
|-
 +
| NVIDIA GeForce RTX 2080 Ti || <tt>GeForceRTX2080Ti</tt> || 11&nbsp;GiB || 36 || 384&nbsp;GiB
 +
|-
 +
| NVIDIA GeForce RTX 2080 Ti || <tt>GeForceRTX2080Ti</tt> || 11&nbsp;GiB || 128 || 512&nbsp;GiB
 +
|-
 +
| NVIDIA TITAN RTX || <tt>TITANRTX</tt> || 24&nbsp; GiB || 128 || 512&nbsp;GiB
 +
|-
 +
| [[Nvidia_DGX-1_with_Tensor_Cores| NVIDIA Tesla V100-SXM2 32 GB]] || <tt>TeslaV100_SXM2_32GB</tt> || 32&nbsp;GiB || 48 || 768&nbsp;GiB
 +
|-
 +
| NVIDIA Tesla A100 || <tt> A100_PCIE_40GB </tt> || 40&nbsp;GiB || 48 || 768&nbsp;GiB
 +
|}
 +
  
 
== Example ==
 
== Example ==

Revision as of 12:33, 17 August 2021

< Submit a parallel job

Home

Monitor a job >


To use the GPUs for a job node you need to request the ngpus_excl_p resource. It refers to the number of GPUs per node. This is unlike other resources, which are requested per core.

For example, to run a serial job with one GPU,

$ bsub -R "rusage[ngpus_excl_p=1]" ./my_cuda_program

or on a full node with all 8 GeForce GTX 1080 Ti GPUs and up to 90 GB of RAM,

$ bsub -n 20 -R "rusage[mem=4500,ngpus_excl_p=8]" -R "select[gpu_model0==GeForceGTX1080Ti]" ./my_cuda_program

or on two full nodes:

$ bsub -n 40 -R "rusage[mem=4500,ngpus_excl_p=8]" -R "select[gpu_model0==GeForceGTX1080Ti]" -R "span[ptile=20]" ./my_cuda_program

While your jobs will see all GPUs, LSF will set the CUDA_VISIBLE_DEVICES environment variable, which is honored by CUDA programs.

Available GPU node types

GPU Model Specifier GPU memory per GPU CPU cores per node CPU memory per node
NVIDIA GeForce GTX 1080 GeForceGTX1080 8 GiB 20 256 GiB
NVIDIA GeForce GTX 1080 Ti GeForceGTX1080Ti 11 GiB 20 256 GiB
NVIDIA GeForce RTX 2080 Ti GeForceRTX2080Ti 11 GiB 36 384 GiB
NVIDIA GeForce RTX 2080 Ti GeForceRTX2080Ti 11 GiB 128 512 GiB
NVIDIA TITAN RTX TITANRTX 24  GiB 128 512 GiB
NVIDIA Tesla V100-SXM2 32 GB TeslaV100_SXM2_32GB 32 GiB 48 768 GiB
NVIDIA Tesla A100 A100_PCIE_40GB 40 GiB 48 768 GiB


Example

Further reading


< Submit a parallel job

Home

Monitor a job >