Execution, Remote Execution through Command

crashclappergapSoftware and s/w Development

Dec 13, 2013 (3 years and 5 days ago)

102 views

A Comparative Analysis of Localized Command Line
Execution, Remote Execution through Command
Line, and Torque Submissions of MATLAB® Scripts for
the Charting of
CReSIS

Flight Path Data

Team Members

JerNettie

Burney

Robyn Evans

Mentor

Je’aime

Powell

Nature and Background of the Study

Abstract


The Polar Grid team was tasked with providing the Center for the Remote Sensing of Ice
Sheets (CReSIS) with data that would allow signal processing through the CReSIS Synthetic
Aperture RADAR Processor (CSARP) to utilize clustered computing resources without the
need of
MATLAB’s
® proprietary Distributed Computing Environment. This research
centered on the use of MATLAB® through command line, and scripted distribution
through TORQUE high performance computing scheduling.



The team used flight path information from the Greenland 2007 field deployment. This
data was imported into MATLAB® so that they could be converted from text files into
actual MATLAB® script files. With these MEX files, the team was able to create a script
within MATLAB® that could plot the flight path data into a graph with the axes of the
graph being labeled latitude for the x
-
axis and longitude for the y
-
axis.



The team took the master script for the creation of the chart and ran jobs through the
command line of MATLAB® to
Madogo

[Elizabeth City State University’s Cluster] and
Quarry [Indiana University’s Cluster]. The team was then able to compare execution times
from the jobs of
Madogo

versus Quarry. A second comparison was then tested with
TORQUE job submission versus MATLAB® submission to see which performed with
greater efficiency. Lastly the average execution times of all three data sets were statistically
compared with a 5% significance level to determine if there was a statistically significant
difference between the use of command line jobs verses TORQUE submissions.

Background of the Problem


Versatility


Affordability


Better Efficiency



Dr. Prasad
Gogineni


Director of
CReSIS

Twin Otter Carrying
SAR RADAR

Vocabulary


ANOVA


Binary


Cluster


Jobs


Linux


MATLAB


Node


Perl Script


Ssh

key



TORQUE

Hypothesis


Multiple hypotheses:


Submission through TORQUE would be quicker


Submission of the scripts through the Quarry’s
command line would be the slowest to submit


Methodology

Definition of the Population


Types of Data


CReSIS

Datasets


Run times


Madogo

Quarry

Definition of Population

Research Design


Defined the problem


Analyzed a solution


Performed experiment


Analyzed results



Statistical Methods & Tests used


ANOVA in Excel


Null hypothesis

H
0
= μ
1

2


3

or

H
1
≠ μ
1
≠μ
2

≠μ
3



Analysis of Data

Visual Representation Data

0
5
10
15
20
25
30
35
40
45
1
3
5
7
9
11
13
15
17
19
TORQUE
IU MATLAB ®
ECSU MATLAB ®
Job Submission Trial

Time in Seconds

Summary, Conclusions, and Recommendations

Conclusions based off Statistical
Analysis of the Data










Anova
:

Single Factor

SUMMARY

Groups

Count

Sum

Average

Variance

ECSU

MATLAB ®

20

84.75

4.2375

0.044536

IU

MATLAB ®

20

392.6

19.63

0.216116

TORQUE

20

223

11.15

36.13421

ANOVA

Source of Variation

SS

df

MS

F

P
-
value

F
crit

Between Groups

2377.481

2

1188.74

97.98694

3.587E
-
19

3.158843

Within Groups

691.5024

57

12.121362

Total

3068.98
3

59

Visual Representation Data

0
5
10
15
20
25
30
35
40
45
1
3
5
7
9
11
13
15
17
19
TORQUE
IU MATLAB ®
ECSU MATLAB ®
Challenges


MATLAB ® complier


Libraries


TORQUE on
Madogo


No user shared home directory


No shared
ssh

keys


Future works



Create standalone applications using MATLAB®


Research why TORQUE submissions were
slower


Direct comparisons of the clusters


MATLAB
®
Distributed Computing Toolkit vs.
using TORQUE to access MATLAB
®


Test using CSARP
-
Lite



Acknowledgements


William Blake of the University of Kansas


MathWorks

Support Team


Je’aime Powell


Jefferson Davis of Indiana University