Difference between revisions of "Job management with LSF"

From ScientificComputing
Jump to: navigation, search
 
(19 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
__NOTOC__
 
__NOTOC__
 +
<table style="width: 100%;">
 +
<tr valign=top>
 +
<td style="width: 30%; text-align:left">
 +
< [[Modules and applications]]
 +
</td>
 +
<td style="width: 35%; text-align:center">
 +
[[Main_Page | Home]]
 +
</td>
 +
<td style="width: 35%; text-align:right">
 +
[[Job submission | Submit a job]] >
 +
</td>
 +
</tr>
 +
</table>
 +
 +
 
<table style="width: 100%;">
 
<table style="width: 100%;">
 
<tr valign=top>
 
<tr valign=top>
Line 5: Line 20:
 
<td style="width: 30%; background: white; text-align: center;">
 
<td style="width: 30%; background: white; text-align: center;">
  
[[File:batch_system_icon.png|250px]]
+
[[File:batch_system.png|500px]]
 +
</td>
 +
<td style="width: 5%;>
 
</td>
 
</td>
<td style="width: 40%; background: white; text-align: left;>
+
<td style="width: 35%; background: white; text-align: left;>
 +
 
 +
<div style="background: #B2D9EA; text-align: left" class="button">
 +
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[[Job submission|'''1. Submit a job''']]
 +
</div>
  
<div style="width: 60%; background: #BFE5D9; height: 35px; border-radius: 10px; padding: 5px; margin:5px">
+
<div style="background: #B2D9EA; text-align: left" class="button">
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[[Job submission|1. Submit a job]]
+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[[Parallel job submission|'''1.1 Submit a parallel job''']]
 
</div>
 
</div>
  
<div style="width: 60%; background: #BFE5D9; height: 35px; border-radius: 10px; padding: 5px; margin:5px">
+
<div style="background: #B2D9EA; text-align: left" class="button">
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[[Parallel job submission|2. Submit a parallel job]]
+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[[GPU job submission|'''1.2 Submit a GPU job''']]
 
</div>
 
</div>
  
<div style="width: 60%; background: #BFE5D9; height: 35px; border-radius: 10px; padding: 5px; margin:5px">
+
<div style="background: #B2D9EA; text-align: left" class="button">
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[[Job monitoring|3. Monitor a job]]
+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[[Job monitoring|'''2. Monitor a job''']]
 
</div>
 
</div>
  
<div style="width: 60%; background: #BFE5D9; height: 35px; border-radius: 10px; padding: 5px; margin:5px">
+
<div style="background: #B2D9EA; text-align: left" class="button">
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[[Job output | 4. Job output]]
+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[[Job output |'''3. Job output''']]
 
</div>
 
</div>
 +
  
 
</td>
 
</td>
  
<td style="width: 20%; background: white; text-align: center;">
+
<td style="width: 30%; background: white; text-align: center;">
  
 
</td>
 
</td>
Line 35: Line 57:
 
</table>
 
</table>
  
== Quick examples ==
+
== Why use the batch system? ==
=== Submit a job with a command line ===
+
Users can access the computing resources on the cluster solely through the batch system. On the ETH HPC clusters, we use the LSF batch system to manage computing jobs.
$ env2lmod
 
$ module load gcc/6.3.0 openmpi/4.0.2 python/3.8.5
 
$ bsub -n 4 -W 4:00 -R "rusage[mem=2048]" "python myscript.py"
 
  
=== Submit with a job script ===
 
Create a job script called job_script.bsub
 
#!/bin/bash
 
#BSUB -n 4                    # 4 cores
 
#BSUB -W 4:00                  # 8-hour run-time
 
#BSUB -R "rusage[mem=2048]"    # 2048 MB per core
 
 
source /cluster/apps/local/env2lmod.sh
 
module load gcc/6.3.0 openmpi/4.0.2 python/3.8.5
 
python myscript.py
 
  
Submit the script
+
<table style="width: 100%;">
$ bsub < job_script.bsub
+
<tr valign=top>
 
+
<td style="width: 30%; text-align:left">
=== Monitor submitted job ===
+
< [[Modules and applications]]
Check the status of your submitted job
+
</td>
$ bjobs
+
<td style="width: 35%; text-align:center">
 +
[[Main_Page | Home]]
 +
</td>
 +
<td style="width: 35%; text-align:right">
 +
[[Job submission | Submit a job]] >
 +
</td>
 +
</tr>
 +
</table>

Latest revision as of 09:24, 1 October 2021

< Modules and applications

Home

Submit a job >


Batch system.png

     1. Submit a job

       1.2 Submit a GPU job

     2. Monitor a job

     3. Job output


Why use the batch system?

Users can access the computing resources on the cluster solely through the batch system. On the ETH HPC clusters, we use the LSF batch system to manage computing jobs.


< Modules and applications

Home

Submit a job >