Using the MATLAB service

Introduction

The MATLAB Distributed Computing Server (MDCS) is a service for offloading computationally-intensive calculations from your workstation to the Euler computer cluster as transparently as possible.

The most common use case is offloading a computationally-intensive parfor loop, which takes more than several hours to run on a normal workstation.

Setup

Quick setup:

1. Install MATLAB version 9.3 (R2017b) on your workstation.
2. (optional but recommended) Open your firewall from 10.205.0.0/19 and 10.205.96.0/19 to ports 27370–27470 on your workstation.
3. Unpack the MATLAB interface files for Euler into Documents\MATLAB (Windows) or ~/Documents/MATLAB (Linux, Mac).
4. Import the Euler_R2017b_9.3.settings cluster profile into MATLAB.
5. Log in to Euler to accept the usage agreement (if you are not already an Euler user)
6. Validate the Euler cluster profile.

Did you encounter problems? The full setup instructions and Troubleshooting are on a separate page.

Usage

Refer to Mathwork's Parallel Computing Toolbox (PCT) documentation on how to make use of Euler in your code.

The batch() function

The batch() function runs a script or function on Calculus. Its use is pretty straightforward:

cluster = parcluster('Euler');
job = batch(cluster, 'my_script');
wait(job);
diary(job);
delete(job);

For a simple function, such as sin, returning a variable and with a single argument:

job = batch(cluster,@sin,1,{0.5});
job.wait();
results=job.fetchOutputs();
ans=results{1};
delete(job);

The submit() function

The submit() function submits a job or several jobs to Calculus. You have to prepare tasks, add them to a job, then submit the job. A useful example is if you have a function with an argument that needs to be evaluated for many different values:

cluster = parcluster('Euler');
job = createJob(cluster);
for i = 1:10
    createTask(job,@(x)x^2,1,{i});
end
submit(job);
wait(job);
squares = job.fetchOutputs();
delete(job);
squares

The parfor statement and parpool

Code that uses parfor or other PCT constructs can use the Euler MDCS by referring to the Euler cluster profile. For example,

cluster = parcluster('Euler');
squares = zeros(10,1);
parpool(cluster,4);
parfor i = 1:10
    squares(i) = i^2;
end
disp(squares)

If you set Euler to be the default cluster profile, then you need no changes to your code: all of the PCT constructs will use Euler by default.

Setting job time limits

The default settings of how long a job can run (24 hours) or how much memory it needs (2500 MB/core) should suffice for most cases. You can specify other values for how long the jobs can run or how much memory it needs. Set the global calculusTimeLimit variable to the maximum allowable time (in seconds) a job can run:

global calculusTimeLimit; calculusTimeLimit=60

Set the global calculusMemory variable to the maximum memory (RAM) a worker will use (in MB/core):

global calculusMemory; calculusMemory=2000