Using the MATLAB service

Introduction

The MATLAB Distributed Computing Server (MDCS) is a service for offloading computationally-intensive calculations from your workstation to the Euler computer cluster as transparently as possible.

The most common use case is offloading a computationally-intensive parfor loop, which takes more than several hours to run on a normal workstation.

Setup

Quick setup:

1. Install MATLAB version 8.5 (R2015a) on your workstation.
2. (optional but recommended) Open your firewall from 10.205.0.0/19 and 10.205.96.0/19 to ports 27370–27470 on your workstation.
3. Unpack the MATLAB interface files for Euler into Documents\MATLAB (Windows) or ~/Documents/MATLAB (Linux, Mac).
4. Import the Euler_8.5.settings cluster profile into MATLAB.
5. Log in to Euler to accept the usage agreement (if you are not already an Euler user)
6. Validate the Euler cluster profile.

Still having problems? The full setup instructions are on a separate page.

Usage

Refer to Mathwork's Parallel Computing Toolbox (PCT) documentation on how to make use of Euler in your code.

The batch() function

The batch() function runs a script or function on Calculus. Its use is pretty straightforward:

cluster = parcluster('Euler');
job = batch(cluster, 'my_script');
wait(job);
diary(job);
delete(job);

For a simple function, such as sin, returning a variable and with a single argument:

job = batch(cluster,@sin,1,{0.5});
job.wait();
results=job.fetchOutputs();
ans=results{1};
delete(job);

The submit() function

The submit() function submits a job or several jobs to Calculus. You have to prepare tasks, add them to a job, then submit the job. A useful example is if you have a function with an argument that needs to be evaluated for many different values:

cluster = parcluster('Euler');
job = createJob(cluster);
for i = 1:10
    createTask(job,@(x)x^2,1,{i});
end
submit(job);
wait(job);
squares = job.fetchOutputs();
delete(job);
squares

The parfor statement and parpool

Code that uses parfor or other PCT constructs can use the Euler MDCS by referring to the Euler cluster profile. For example,

cluster = parcluster('Euler');
squares = zeros(10,1);
parpool(cluster,4);
parfor i = 1:10
    squares(i) = i^2;
end
disp(squares)

If you set Euler to be the default cluster profile, then you need no changes to your code: all of the PCT constructs will use Euler by default.

Setting job time limits

The default settings of how long a job can run (24 hours) or how much memory it needs (2500 MB/core) should suffice for most cases. You can specify other values for how long the jobs can run or how much memory it needs. Set the global calculusTimeLimit variable to the maximum allowable time (in minutes) a job can run:

global calculusTimeLimit; calculusTimeLimit=60

Set the global calculusMemory variable to the maximum memory (RAM) a worker will use (in MB/core):

global calculusMemory; calculusMemory=2000

Troubleshooting

Using parallel pools often results in hard-to-diagnose errors. Many of these errors are related to running several pools at the same time, which is not what MATLAB expects. If you encounter persistent problems starting pools, try to perform one of these commands. Before running them, make sure that you do not have a MATLAB processes running.

1. Remove the matlab_metadat.mat file in your current working directory on your workstation.
2. Remove the $HOME/.matlab/local_cluster_jobs directory on your workstation. The actual location may depend on your operating system or installation options.
3. Remove the entire $HOME/.matlab directory on your workstation. Warning: Your MATLAB settings will be lost.

Resetting or Forgetting the Username

Your username is saved as a MATLAB preference in the ETHCalculus preference group. If you have mistyped it or want to delete the saved username, then issue the

rmpref('ETHCalculus')

MATLAB command to clear all the Calculus preferences. Contrary to the username, your password is not saved as a preference. It remains valid for the entire MATLAB session but you will need to retype it every time your restart MATLAB.