Difference between revisions of "Job submission"
From ScientificComputing
| Line 78: | Line 78: | ||
|} | |} | ||
| − | == Job script == | + | == Job script and #BSUB pragmas == |
| + | Create a job script called job_script.bsub | ||
| + | |||
#!/bin/bash | #!/bin/bash | ||
#BSUB -n 24 # 24 cores | #BSUB -n 24 # 24 cores | ||
#BSUB -W 8:00 # 8-hour run-time | #BSUB -W 8:00 # 8-hour run-time | ||
#BSUB -R "rusage[mem=4000]" # 4000 MB per core | #BSUB -R "rusage[mem=4000]" # 4000 MB per core | ||
| + | #BSUB -J analysis1 | ||
| + | #BSUB -o analysis1.out | ||
| + | #BSUB -e analysis1.err | ||
| + | #BSUB -N | ||
| + | |||
| + | module load gcc/6.3.0 openmpi/3.0.2 | ||
cd /path/to/execution/folder | cd /path/to/execution/folder | ||
| − | + | mpirun myprogram arg1 | |
| + | Submit a job | ||
bsub < job_script.bsub | bsub < job_script.bsub | ||
Revision as of 23:03, 20 January 2021
Basic job submission
A basic BSUB job submission command consists of three parts:
| bsub | LSF options | job |
- The BSUB executable command
- LSF options requesting resources and defining job-related options
- A job to be submitted
Here is an example:
| bsub | -n 4 -W 4:00 -R "rusage[mem=4096]" | "python myscript.py" |
Job
A job can be one of the following:
| Job | Command | Examples of job submission command |
|---|---|---|
| a single Linux command | cmd | |
| a program with its path | /path/to/myprogram | bsub ./bin/hello |
| a command or program with its arguments | cmd arg1 arg2 | bsub echo hello |
| multiple commands | "cmd1 ; cmd2" | bsub "date; pwd; ls -l" |
| piped command | "cmd1 | cmd2" | |
| a command with I/O redirection, quote | "cmd<in >out" | bsub "du -sk/scratch > du.out" |
| a here document, passed via "<<" | << EOF ... EOF | |
| a shell script, passed via "<" | < script | bsub < hello.sh |
Command line advisor
LSF options
Requesting resources
| Resources | Format | Default values |
|---|---|---|
| Maximum run time | -W HH:MM | 04:00 (4 hours) |
| Number of processors | -n nprocs | 1 processor |
| Memory | -R "rusage[mem=2048]" | 1024 MB per core |
| Scratch space | -R "rusage[scratch=10000]" |
Other LSF options
| -o outfile | append job’s standard output to outfile |
| -e errfile | append job’s error messages to errfile |
| -R "rusage[...]" | advanced resource requirement (memory,...) |
| -J jobname | assign a jobname to the job |
| -w "depcond" | wait until dependency condition is satisfied |
| -Is | submit an interactive job with pseudo-terminal |
| -B /-N | send an email when the job begins/ends |
| -u user@domain | use this address instead of username@ethz.ch |
Job script and #BSUB pragmas
Create a job script called job_script.bsub
#!/bin/bash #BSUB -n 24 # 24 cores #BSUB -W 8:00 # 8-hour run-time #BSUB -R "rusage[mem=4000]" # 4000 MB per core #BSUB -J analysis1 #BSUB -o analysis1.out #BSUB -e analysis1.err #BSUB -N module load gcc/6.3.0 openmpi/3.0.2 cd /path/to/execution/folder mpirun myprogram arg1
Submit a job
bsub < job_script.bsub
Interactive session
bsub -n 4 -W 4:00 -R "rusage[mem=4096]" -Is bash
