Difference between revisions of "MATLAB/Parallel"

From ScientificComputing
Jump to: navigation, search
(Local pool size has increased in 9.1.)
(6 intermediate revisions by 2 users not shown)
Line 1: Line 1:
MATLAB's [http://www.mathworks.com/products/parallel-computing/ Parallel Computing Toolbox] can be used to run suitably-written programs in parallel—on multiple cores at once.  For this the newest version of MATLAB that is available is recommended.
+
MATLAB's [http://www.mathworks.com/products/parallel-computing/ Parallel Computing Toolbox] lets you run suitably-written programs in parallel. Several cores calculate different parts of a problem at the same time to reduce the time-to-solution.
  
For suitable MATLAB programs (such as those containing <tt>parfor</tt> loops), using the Parallel Computing Toolbox requires two steps
+
For running MATLAB in parallel, you can either use a local parpool (up to 24 cores), an LSF parpool (24+ cores) or use the MATLAB distributed computing server (MDCS), which allows to offload jobs from a local MATLAB instance running on your computer to the Euler cluster. Please find below the documentation about using a local parpool. Instructions for using an [[MATLAB_PCT| LSF parpool]] as well as for using the [[Using_the_MATLAB_service|MDCS]] are provided on separate wiki pages.
# opening a ''local'' matlabpool and
 
# telling Euler's bsub scheduler to request multiple cores.
 
  
=== Use the ''local'' matlabpool ===
+
=== Use a parpool ===
Use the ''local'' matlabpool in your MATLAB program and specify the number of cores (e.g., 4):
 
matlabpool('local', 4);
 
when done, close the pool:
 
matlabpool close;
 
Note that the ''local'' matlabpool is limited to 12 cores in releases up to R2016a (8.7/9.0). From release R2016b (9.1) on, you can use all the cores of Euler nodes.
 
  
=== Submit a Parallel Job ===
+
For suitable MATLAB programs (such as those containing <tt>parfor</tt> loops), using the Parallel Computing Toolbox requires two steps:
Pass the number of cores (e.g., 4) to bsub's <tt>-n</tt> argument:
+
# use a parpool in your MATLAB program and
  bsub -n 4 -W "1:00" matlab -nodisplay -singleCompThread -r simulation
+
# request multiple cores from Euler's LSF batch scheduler.
Note that you must ''not'' use the <tt>-nojvm</tt> argument but you ''should'' include the <tt>-singleCompThread</tt> argument.
+
 
 +
A trivial parallel program (<tt>simulation.m</tt>) is shown below:
 +
euler = parcluster('local');
 +
squares = zeros(10,1);
 +
pool = parpool(euler,4);
 +
parfor i = 1:10
 +
    squares(i) = i^2;
 +
end
 +
disp(squares)
 +
pool.delete()
 +
Note that the ''local'' parpool is limited to 12 cores in releases up to R2016a (8.7/9.0). From release R2016b (9.1) on, you can use all the cores of Euler nodes (effectively&nbsp;24).
 +
 
 +
Older versions of MATLAB used a matlabpool instead of a parpool. For using a local pool there is no need to load a cluster profile.
 +
 
 +
=== Submit a parallel job ===
 +
 
 +
Pass the number of cores (e.g., 4) to bsub's <tt>-n</tt> argument. This should match the size of the pool requested in your MATLAB script.
 +
  bsub -n 4 -W "1:00" -R "rusage[mem=2048]" matlab -nodisplay -singleCompThread -r simulation
 +
Note that you must ''not'' use the <tt>-nojvm</tt> argument but you ''should'' include the <tt>-singleCompThread</tt> argument. MATLAB is quite memory-hungry, so request at least 2&nbsp;GB of memory.
 +
 
 +
=== Troubleshoot parallel jobs ===
 +
 
 +
Using parallel pools often results in hard-to-diagnose errors. Many of these errors are related to running several pools at the same time, which is not what MATLAB expects.
 +
If you encounter persistent problems starting pools, try to perform one of these commands. Before running them, make sure that you do not have a MATLAB processes running.
 +
# Remove the <tt>matlab_metadat.mat</tt> file in your current working directory.
 +
# Remove the <tt>$HOME/.matlab/local_cluster_jobs</tt> directory.
 +
# Remove the entire <tt>$HOME/.matlab</tt> directory. '''Warning''': Your MATLAB settings on Euler will be lost.
 +
 
 +
In case this does not solve the problem, then please contact {{Cluster_support}}.

Revision as of 06:26, 19 June 2017

MATLAB's Parallel Computing Toolbox lets you run suitably-written programs in parallel. Several cores calculate different parts of a problem at the same time to reduce the time-to-solution.

For running MATLAB in parallel, you can either use a local parpool (up to 24 cores), an LSF parpool (24+ cores) or use the MATLAB distributed computing server (MDCS), which allows to offload jobs from a local MATLAB instance running on your computer to the Euler cluster. Please find below the documentation about using a local parpool. Instructions for using an LSF parpool as well as for using the MDCS are provided on separate wiki pages.

Use a parpool

For suitable MATLAB programs (such as those containing parfor loops), using the Parallel Computing Toolbox requires two steps:

  1. use a parpool in your MATLAB program and
  2. request multiple cores from Euler's LSF batch scheduler.

A trivial parallel program (simulation.m) is shown below:

euler = parcluster('local');
squares = zeros(10,1);
pool = parpool(euler,4);
parfor i = 1:10
    squares(i) = i^2;
end
disp(squares)
pool.delete()

Note that the local parpool is limited to 12 cores in releases up to R2016a (8.7/9.0). From release R2016b (9.1) on, you can use all the cores of Euler nodes (effectively 24).

Older versions of MATLAB used a matlabpool instead of a parpool. For using a local pool there is no need to load a cluster profile.

Submit a parallel job

Pass the number of cores (e.g., 4) to bsub's -n argument. This should match the size of the pool requested in your MATLAB script.

bsub -n 4 -W "1:00" -R "rusage[mem=2048]" matlab -nodisplay -singleCompThread -r simulation

Note that you must not use the -nojvm argument but you should include the -singleCompThread argument. MATLAB is quite memory-hungry, so request at least 2 GB of memory.

Troubleshoot parallel jobs

Using parallel pools often results in hard-to-diagnose errors. Many of these errors are related to running several pools at the same time, which is not what MATLAB expects. If you encounter persistent problems starting pools, try to perform one of these commands. Before running them, make sure that you do not have a MATLAB processes running.

  1. Remove the matlab_metadat.mat file in your current working directory.
  2. Remove the $HOME/.matlab/local_cluster_jobs directory.
  3. Remove the entire $HOME/.matlab directory. Warning: Your MATLAB settings on Euler will be lost.

In case this does not solve the problem, then please contact cluster support.