MATLAB Parallel Server
MATLAB Parallel Server (MPS) allows you to create and use parallel pools that can scale to many CPUs more than in a single node. Currently MPS jobs must be submitted from a GUI or command line MATLAB session running on one of the ARC Graphical (NX) nodes or a session on an interactive node. You must not run MPS from a cluster login node.
Starting MATLAB from an NX node
If you are using the ARC Graphical (NX) nodes to run MATLAB. You need to ensure you select which cluster you wish to use, HTC or ARC.
Open the Konsole
and load both the MATLAB and the cluster definition module:
module load MATLAB/R2022b cluster/arc
or
module load MATLAB/R2022b cluster/htc
You can then type matlab
at the command line to start MATLAB.
Importing cluster profiles
We have generated some basic cluster profile definition files for each cluster ARC and HTC, these act as a starting point for your own profiles which can support custom resource requirements.
You need to import the ARC or HTC cluster profile into your MATLAB environment and set it as the default before you can submit MPS jobs.
This only needs doing once. The imported profile will be saved in your MATLAB settings directory.
Importing the profile can be done either by calling MATLAB functions or via the graphical interface. The profiles are stored here (for R2022b):
/apps/common/commercial/MATLAB/mps_profiles/R2022b
The profiles are configured for specific partitions or resources, as follows:
Partition/Resource |
ARC |
HTC |
devel |
arc_devel.mlsettings |
htc_devel.mlsettings |
short |
arc_short.mlsettings |
htc_short.mlsettings |
medium |
arc_medium.mlsettings |
htc_medium.mlsettings |
Importing cluster profiles directly from the MATLAB GUI
For these examples we will use the arc_devel
profile.
To import using the graphical interface:
From the MATLAB Home tab select the Parallel
menu and click Create and Manage Clusters...
The Cluster Profile Manager window will open:
Select Import
and from within the Import Profiles from file
window navigate to the arc_devel.mlsettings
file described
above and select Open
Select the resulting arc_devel
profile and click Set as Default
.
The Cluster Profile Manager window should now look like this:
Note
After you exit MATLAB, your default cluster profile is saved for future use.
To test the loaded profile, click the Validate
button and the validation process will begin. Once the process completes the screen should look as follows:
Import using MATLAB functions
Instead of selecting and activating the cluster profile from the GUI you can run these functions from a command line MATLAB session:
arc_profile = parallel.importProfile ('/apps/common/commercial/MATLAB/mps_profiles/R2022b/arc_devel.mlsettings');
parallel.defaultClusterProfile ('arc_devel');
The above could be used as part of a MATLAB script. See next example…
Full MPS Example
In this example we are going to use the arc_short
partition. So, follow the instructions above to load the file named arc_short.mlsettings
Once this is loaded. From the MATLAB GUI click New Script
and paste the following script into the editor:
%
% ARC Parallel MATLAB example
%
% Either have arc-short cluster profile set as default in GUI or uncomment the
% following two lines:
%
%parprof = parallel.importProfile("/apps/common/commercial/MATLAB/mps_profiles/R2022b/arc_short.mlsettings")
%parallel.defaultProfile(parprof)
primeNumbers = primes(uint64(2^20));
compositeNumbers = primeNumbers.*primeNumbers(randperm(numel(primeNumbers)));
factors = zeros(numel(primeNumbers),2);
% Create parallel pool
%
poolObj=parpool('arc_short',96);
numWorkers = [1 24 48 72 96];
tCluster = zeros(size(numWorkers));
for w = 1:numel(numWorkers)
tic;
parfor (idx = 1:numel(compositeNumbers), numWorkers(w))
factors(idx,:) = factor(compositeNumbers(idx));
end
tCluster(w) = toc;
end
% Shutdown parallel pool.
delete(poolObj);
f = figure;
figure(f);
hold on
speedup = tCluster(1)./tCluster;
plot(numWorkers, speedup);
title('Speedup with the number of workers');
xlabel('Number of workers');
xticks(numWorkers(2:end));
ylabel('Speedup');
Once you have done this, click Run
from the Editor. The script will start a job in the ARC short queue, and use this as a MATLAB worker pool.
If the job is able to run immediately the run time is around 5 minutes and you should be presented with the following plot when it completes:
Note
When you create the parallel pool with parpool()
a batch job is submitted to SLURM. Your MATLAB script will only continue once this job is scheduled by SLURM
and is in the RUNNING state. This is why we recommend running MATLAB from the ARC Graphical (NX) nodes where you can disconnect your session without terminating your
client MATLAB session.