D C U

boundlessbazaarΔιακομιστές

9 Δεκ 2013 (πριν από 4 χρόνια και 23 μέρες)

165 εμφανίσεις

1
/19


D
EPLOYING

C
ONDOR

U
SING

V
IRTUAL
B
OX
®
ON

W
INDOWS
® H
OSTS

David Herzfeld
1
, Craig A. Struble, Ph.D
2
, and Lars E. Olson, Ph.D
1

Marquette University

1
Department of Biomedical Engineering

2
Department of Mathematics, Statistics, and Computer Science

Southeast Wisconsin High Performance (SeWHiP) Computing
Consortium (www.sewhip.org)


Tuesday, Apr. 21, 2009


2
/19


I
NTRODUCTION


Most systems on Marquette’s campus run
Windows XP® or Windows Vista®


Most researchers at Marquette require Linux
based environments


Most sys
-
admins have primarily Windows
®

based experience

2

Tuesday, Apr. 21, 2009


3
/19


P
ROJECT

G
OALS


Allow researchers to use underutilized
Marquette systems


Requires Linux environment


Allow researchers full use of the Condor
system


check pointing, migration, etc.


Do not change Windows® users’ experience


Easy installation and maintenance for
Windows® sys
-
admins


3

Tuesday, Apr. 21, 2009


4
/19


P
OSSIBLE

S
OLUTIONS


Use Condor for Windows®


Very few researchers using Windows® based
applications


Most require a GUI


Windows®


Linux dual boot systems


Management is difficult


Windows® users will be affected


Condor Virtual Universe


VMware® = $


Xen

does not support Windows® hosts


Configuration Overhead


Virtualization




4

Tuesday, Apr. 21, 2009


5
/19


I
NITIAL

S
OLUTION
:
C
O
L
INUX


Based on previous work by Neeman, Serverini (U. of
Oklahoma), and Sumanth (Nebraska
-
Lincoln), 2006


Runs natively in Windows®


Near native performance


Small hardware dependencies


Open source

×
Relatively unsupported


small community

×
Only supports 32 bit environments

×
No current SMP support

×
Few recent developments

5

Tuesday, Apr. 21, 2009


6
/19


C
URRENT

S
OLUTION
: S
UN

V
IRTUAL
B
OX
®


Free & “Open Source”


Capable or running under 32 &
64 bit

Windows®


Guest Additions allow for near native speeds


Some disk intensive operations may see a speed up


Supported & continually developed


Large

community support

×
No current SMP support


planned for future release


6

Tuesday, Apr. 21, 2009


7
/19


7

V
IRTUAL
B
OX
® & W
INDOWS
®

Tuesday, Apr. 21, 2009


8
/19


C
ONDOR
MAC

&
C
O
M
ON


CondorMAC:


Allows for distribution via images


Replaces the VirtualBox® MAC address with a modification
of the host MAC address


CoMon:


Determine sec. since last key stroke


Determine total processor utilization


Determine VirtualBox® processor utilization


Write data to file in host
-
guest shared folder


Visual Basic monitor service

8

Tuesday, Apr. 21, 2009


9
/19


HCLUST
_
SERVICE


Fork headless VirtualBox® machine at
Windows® startup.


Remove any VirtualBox® “remnant” machines


“Nicely” shut down guest system when
requested (via ACPI)

9

Tuesday, Apr. 21, 2009


10
/19


10

C
ONDOR

V
IRTUAL

M
ACHINE

Tuesday, Apr. 21, 2009


11
/19


V
IRTUAL

M
ACHINE

B
ASICS


Guest hostname is based on assigned IP address


Central Manager name & IP are determined


Personalized text file in shared folder


Variables replaced in condor_config.local


For remote administration


ssh key in shared folder


Condor launches after guest startup


init.d script


11

Tuesday, Apr. 21, 2009


12
/19


B
ENEFITS

OF

I
MMUTABLE

H
ARD

D
RIVES


Defines an upper limit on the amount of space Condor
users can utilize (100 GB).


Removal of the snapshots


by hclust_service


returns
the VM to an original state.


Returns unused hard drive space back to the
Windows® host.


This space would not be returned if it were simply a
dynamically expanding .vdi image

12

Tuesday, Apr. 21, 2009


13
/19


13

C
ONDOR

V
IRTUAL

M
ACHINE

Tuesday, Apr. 21, 2009


14
/19


VM A
DDITIONS


Additions included in immutable hard drive


Software packages utilized by researchers


Geant, OpenFOAM, AutoDock, etc.


Sets up execution path for use in Condor jobs


Package names and versions added to Condor
ClassAds


14

Tuesday, Apr. 21, 2009


15
/19


I
NSTALLATION


Distributed as a zip file, batch installer


Installs VirtualBox®


Creates a new virtual machine


MAC Address set by CondorMAC


Memory computed as fraction of total system memory


Bridged networking set up


Adds CentOS image to virtual machine


Sets services to run at system startup


Starts the new VM


Installation time is minutes

15

Tuesday, Apr. 21, 2009


16
/19


V
IRTUALIZATION

I
SSUES


Networking Problems


Current bridged connection requires an additional
IP address for each worker


Doubles

the on campus IP addresses!



16

Tuesday, Apr. 21, 2009


17
/19


F
UTURE

W
ORK


SMP Support


Support for different networking modes


using NAT, OpenVPN, etc.


Specific VM updates via repositories


Tailored distributions


Bioinformatics


Fluid Dynamics


Neural Simulations


17

Tuesday, Apr. 21, 2009


18
/19


A
CKNOWLELDGMENTS


Information Technology Services; Marquette
University


Kathy Lang; CIO, Clifford Brown, Patrick Blume,
Aaron Gember, Chad Gorectke


Lingtao Zeng; Marquette MSCS System
Administrator


Brad Bonczkiewicz; Marquette Engineering System
Administrator


Biomedical Engineering Department


Mathematics, Statistics and Computer Science
Department


18

19
/19


Q
UESTIONS
/C
OMMENTS
?

Thank you!