MATLAB Parallel Server

MATLAB Parallel Server (MPS) allows you to create and use parallel pools that can scale to many CPUs more than in a single node. Currently MPS jobs must be submitted from a GUI or command line MATLAB session running on one of the ARC Graphical (NX) nodes or a session on an interactive node. You must not run MPS from a cluster login node.

Starting MATLAB from an NX node

If you are using the ARC Graphical (NX) nodes to run MATLAB. You need to ensure you select which cluster you wish to use, HTC or ARC.

Open the Konsole and load both the MATLAB and the cluster definition module:

module load MATLAB/R2022b cluster/arc

or

module load MATLAB/R2022b cluster/htc

You can then type matlab at the command line to start MATLAB.

Importing cluster profiles

We have generated some basic cluster profile definition files for each cluster ARC and HTC, these act as a starting point for your own profiles which can support custom resource requirements.

You need to import the ARC or HTC cluster profile into your MATLAB environment and set it as the default before you can submit MPS jobs.

This only needs doing once. The imported profile will be saved in your MATLAB settings directory.

Importing the profile can be done either by calling MATLAB functions or via the graphical interface. The profiles are stored here (for R2022b):

/apps/common/commercial/MATLAB/mps_profiles/R2022b

The profiles are configured for specific partitions or resources, as follows:

Partition/Resource

ARC

HTC

devel

arc_devel.mlsettings

htc_devel.mlsettings

short

arc_short.mlsettings

htc_short.mlsettings

medium

arc_medium.mlsettings

htc_medium.mlsettings

Importing cluster profiles directly from the MATLAB GUI

For these examples we will use the arc_devel profile.

To import using the graphical interface:

From the MATLAB Home tab select the Parallel menu and click Create and Manage Clusters... The Cluster Profile Manager window will open:

Cluster Window

Select Import and from within the Import Profiles from file window navigate to the arc_devel.mlsettings file described above and select Open

Select the resulting arc_devel profile and click Set as Default.

The Cluster Profile Manager window should now look like this:

Cluster Imported

Note

After you exit MATLAB, your default cluster profile is saved for future use.

To test the loaded profile, click the Validate button and the validation process will begin. Once the process completes the screen should look as follows:

Cluster Validation

Import using MATLAB functions

Instead of selecting and activating the cluster profile from the GUI you can run these functions from a command line MATLAB session:

arc_profile = parallel.importProfile ('/apps/common/commercial/MATLAB/mps_profiles/R2022b/arc_devel.mlsettings');
parallel.defaultClusterProfile ('arc_devel');

The above could be used as part of a MATLAB script. See next example…

Full MPS Example

In this example we are going to use the arc_short partition. So, follow the instructions above to load the file named arc_short.mlsettings

Once this is loaded. From the MATLAB GUI click New Script and paste the following script into the editor:

%
% ARC Parallel MATLAB example
%
% Either have arc-short cluster profile set as default in GUI or uncomment the
% following two lines:
%
%parprof = parallel.importProfile("/apps/common/commercial/MATLAB/mps_profiles/R2022b/arc_short.mlsettings")
%parallel.defaultProfile(parprof)


primeNumbers = primes(uint64(2^20));
compositeNumbers = primeNumbers.*primeNumbers(randperm(numel(primeNumbers)));
factors = zeros(numel(primeNumbers),2);

% Create parallel pool
%
poolObj=parpool('arc_short',96);

numWorkers = [1 24 48 72 96];
tCluster = zeros(size(numWorkers));

for w = 1:numel(numWorkers)
    tic;
    parfor (idx = 1:numel(compositeNumbers), numWorkers(w))
        factors(idx,:) = factor(compositeNumbers(idx));
    end
    tCluster(w) = toc;
end

% Shutdown parallel pool.
delete(poolObj);

f = figure;
figure(f);
hold on
speedup = tCluster(1)./tCluster;
plot(numWorkers, speedup);
title('Speedup with the number of workers');
xlabel('Number of workers');
xticks(numWorkers(2:end));
ylabel('Speedup');

Once you have done this, click Run from the Editor. The script will start a job in the ARC short queue, and use this as a MATLAB worker pool.

If the job is able to run immediately the run time is around 5 minutes and you should be presented with the following plot when it completes:

Cluster Speedup

Note

When you create the parallel pool with parpool() a batch job is submitted to SLURM. Your MATLAB script will only continue once this job is scheduled by SLURM and is in the RUNNING state. This is why we recommend running MATLAB from the ARC Graphical (NX) nodes where you can disconnect your session without terminating your client MATLAB session.