Distributed Matlab
Usage with PBS / Interactive Usage
Overview
The Distributed Computing Toolbox and the MATLAB Distributed Computing Engine enable you to coordinate and execute independent MATLAB operations simultaneously on a cluster of computers, speeding up execution of large MATLAB jobs.
The MATLAB Distributed Computing Engine (DCE) provides the backend (behind-the-scenes) support for the Distributed Computing Toolbox. The DCE environment is transparent to users of our clusters, as the environment is set up with the 'dmatlab' wrapper explained below.
The Distributed Computing Toolbox provides the user functionality to perform parallel computations using MATLAB. There are three steps to using this functionality:
- Parallelize your calculation/algorithm. This is beyond the scope of this article - Please visit Mathworks' documentation for further information on this topic.
- Utilize the environment variables set by 'dmatlab' (explained in "Setup").
- Submit your job using the 'dmatlab' command (explained in "Setup").
Setup for Use with PBS
A wrapper script has been written to facilitate efficient usage of Matlab's distributed computing tools within the PBS environment on our clusters. The script handles setting up the environment as well as starting the appropriate services and workers, so you can concentrate on your research and getting results.
To utilize MATLAB's event handler, a fake vnc environment must be created. For security reasons, a password must be set for this environment prior to running dmatlab. This is done by running the following command and following the prompts:
**Setting a vnc password must only be done once. If you do not set your vnc password, you will receive an error when attempting to run dmatlab.**
To run matlab in a distributed environment, you can run this command within your PBS script:
dmatlab [-32] [-v] [-matlab_args "<matlab arguments>"] [<-function function_name>|<input.m>]
|
Where 'input.m' is the name of your input (.m) file. On 64-bit systems, the '32' flag causes dmatlab to run in 32-bit mode (this is only useful if you need to utilize the Symbolic Math Toolkit). '-v' turns on verbose output, and the '-matlab_args' flag will pass the double-quoted arguments following it to MATLAB. To find out what switches and arguments are available on the cluster you wish to use, simply run the following command on the head-node:
The dmatlab wrapper sets three environment variables that can be used to automatically allocate the correct number of processes in your PBS environment and utilize the jobmanager associated with your session. These variables are named DMATLAB_NPROCS, JOBMANAGER and JOBMANAGERHOST.
To access the DMATLAB_NPROCS value, you can use the following lines in your .m file:
nprocs = [ getenv('DMATLAB_NPROCS') ]
np = sscanf( nprocs, '%d' )
|
To access the JOBMANAGER and JOBMANAGERHOST variables, the following lines should be present in your .m file:
mgr_name = [ getenv('JOBMANAGER') ]
mgr_host = [ getenv('JOBMANAGERHOST') ]
jm = findResource('jobmanager','Name',mgr_name,'LookupURL',mgr_host)
|
Where 'jm' is the variable used when referencing the jobmanager object within Matlab.
Please reference the Examples section for an example of a simple Matlab input that sets these variables and does a very elementary calculation.
Interactive Development Usage
In order to facilitate efficient development of distributed MATLAB programs, we have made available an interactive version of dmatlab, called 'idmatlab' which is available on our hammer cluster. The development environment is not intended for calculations or long runtimes, and is limited to either 2 or 4 processes, the only command-line argument accepted by idmatlab. The environment simulates the dmatlab environment by setting the same environment variables explained above, and starting and stopping the Distributed MATLAB Engine components. When logged in to hammer, the environment can be invoked as follows:
idmatlab [-np <n processes (1-4)>] [-nogui] [-32] [-matlab_args "<matlab arguments>"]
|
For instance, to run interactive dmatlab with 3 processes, and the matlab nosplash and nodesktop arguments, you would issue the command as below:
idmatlab -np 3 -matlab_args "-nosplash -nodesktop"
|
The environment will provide an instance of the MATLAB GUI (or command line if -nogui is specified) that you can use to run small development tests in real time.
Examples
The following is a simple distributed calculation that uses the above-mentioned wrapper features and prints a result at each worker, saved as file "dist2.m":
nprocs = [ getenv('DMATLAB_NPROCS') ]
np = sscanf( nprocs, '%d' )
mgr_name = [ getenv('JOBMANAGER') ]
mgr_host = [ getenv('JOBMANAGERHOST') ]
jm = findResource('jobmanager','Name',mgr_name,'LookupURL',mgr_host);
j = createJob(jm);
for p = 1 : np
createTask(j, @sum, 1, {[22*p p]});
end
submit(j);
get(jm);
waitForState(j);
results = getAllOutputArguments(j)
|
The script used to submit this job is as follows:
#PBS -l nodes=2:ppn=1,walltime=00:30:00
#PBS -joe
#PBS -q lionxo
cd $PBS_O_WORKDIR
dmatlab dist2.m
|
The result of running this within a PBS environment using 2 processors will output the following (neglecting the utility output):
< M A T L A B >
Copyright 1984-2006 The MathWorks, Inc.
Version 7.3.0.298 (R2006b)
August 03, 2006
To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.
pbsjobid =
9999
>>
nprocs =
2
>>
np =
2
>> >> >> >>
mgr_name =
lionxo_9999_jobmanager
mgr_host =
lionxo121:47350
>> >> >> >> >> >> >> >> >> >> >> >>
Name: 'lionxo_9999_jobmanager'
Hostname: 'lionxo121.hpc.aset.psu.edu'
HostAddress: {2x1 cell}
Jobs: [0x1 double]
State: 'running'
Configuration: ''
NumberOfBusyWorkers: 0
BusyWorkers: [0x1 double]
NumberOfIdleWorkers: 2
IdleWorkers: [2x1 distcomp.worker]
>> >> >> >> >> >> >> >>
results =
[ 23]
[ 46]
>>
|
Documentation
For further information on using the Distributed Computing Toolkit, please visit Mathworks' documentation.
Please send questions or suggestions about this web page to beatnic@aset.psu.edu
ASET | ITS | Penn State
|
