Possibilities and Practice

wakefulchardΛογισμικό & κατασκευή λογ/κού

17 Φεβ 2014 (πριν από 3 χρόνια και 3 μήνες)

65 εμφανίσεις

Cloud Computing:

Possibilities and Practice

A presentation of cloud computing techniques
and applications


Presented by technology professionals from
Northeastern University, MIT, and Wheelock
College.


Copyright Greg Banwarth, Rick MicKool, and Justin Riley, 2009. This
work is the intellectual property of the authors. Permission is
granted for this material to be shared for non
-
commercial,
educational purposes, provided that this copyright statement appears
on the reproduced materials and notice is given that the copying is
by permission of the author. To disseminate otherwise or to
republish requires written permission from the author. Some images
in this Powerpoint where taken from public web sites.

Speakers


Rick Mickool, Executive Director & CTO,
Information Services, Northeastern
University


Greg Banwarth, Director of Technology /
CIO, Wheelock College


Justin Riley, Programmer Analyst III, MIT



Northeastern University


in the Cloud



Why?




How?

Northeastern University


Private university founded 1898 in Boston, MA


National interdisciplinary research university that is
student
-
centered, practice
-
oriented and urban


25,000 applications nationally for 2,800 freshman seats


Major research initiatives: Technology development, economic
growth, social progress, and health care.


18,000 FT and 5,500 PT students across 7 colleges and Law
School


1,000 faculty, 3,000 support staff. Over 160,000 registered
alumni


Top co
-
op school per US News & World Report


Focused on experiential learning

including co
-
op, student
research, service learning, and global learning


Site of the first World Series in 1903 (Pittsburgh Pirates vs.
Boston Red Sox
*
)



The Vision


How do we deliver the same, personalized ‘campus
experience’ to our customers wherever they are,
whenever they want it, on whatever device is
available?


Residence Hall

Classroom

Home

Staff

Alumni

Parents

Mobile

Device

Where

-

branch
locations,
telecommuters,
disconnected
users, foreign
countries

What

Faculty

How

Office

Curricular

Co
-
Curricular

Extra
-
Curricular

Students

Browser

Desktop

Set top

Device

Who

Labs

When

12:00

24:00

The Challenge


Data Centers

The Challenge


Variety of
Devices

The Challenge


The Need to
Integrate

How
-

Evolution


Application Virtualization


Softricity/Microsoft
Softgrid


Reduced application conflicts


Faster introduction and upgrade of applications


Server Virtualization


VMware / AIX LPAR


Consolidated data center


Faster and higher availability of servers



Terminal Services / Remote Desktop


Microsoft


NU On
-
Demand


Software as a Service

SaaS Experience



Implemented Salesforce.com (SFDC) in 2006


Used for alumni development and contributor
relations


Previous experiences with ASP, managed and
hosted solutions


SFDC consultant
-

RAD development


LDAP authentication / SFDC profiles


Integration in 30 days and live in 90 days


Used CastIron integration “appliance”


No local server and storage infrastructure


Next


Platform as a Service
(PaaS)?

* reference: http://blogs.zdnet.com/Hinchcliffe/?p=166&tag=btxcsim

Next


Desktop as a Service?


Desktop as a Service


MokaFive


Desktone


G.HO.ST

Considerations
-

Good or Bad?


Rapid deployment
-

little infrastructure


Network connectivity


Authentication


“Open” platform for development and integration


Web services / APIs


Lock
-
in


Your data
-

their location & retention schedule?


Security & Liability


Licensing & Intellectual Property


OpEx versus CapEx

My Final Thoughts



May 2003, Nicholas Carr published “IT Doesn’t Matter”
and highlighted IT as a utility


Cloud computing services are evolving based on new
and maturing grid and virtualization technologies


Some cloud services can bring value today


It is not ‘if” but “when” to use cloud services


SOA & BPO focus


The cloud will be an increasingly important contributor to
delivering agile on
-
demand environments in the future


start your plan to plug
-
in now

Disaster Recovery On the
Cloud

Greg Banwarth

Director of Technology / CIO

About Wheelock




Approximately 900 undergraduate, 300 graduate
students


Consolidated campus in downtown Boston, graduate
campus a ¼ mile away, off
-
campus programs in South
Carolina and Singapore


Technology department handles all technology
services with a full time employee count of 5


We are driven to new or innovative technologies to overcome
resource issues while still providing full service to our
faculty/staff/students.



Infrastructure



40 active servers (primarily Windows based)


ERP (Jenzabar), Mail (Exchange 2003), File
Server, DNS, Web, etc.


Main services (excluding ERP) are on a
VMWare environment


Backup options were non
-
enterprise


Backed up essential data to Iron Mountain “Live
Vault”



Funding and Project Goals



One time capital expense budget to develop an
on and off
-
site comprehensive backup set


Funding came with requirement to save data out of
state


Identified Amazon’s S3 service as the most
cost efficient (Approximately 75% cheaper than
traditional co
-
location means)


How to get our data there?




Disaster Recovery Foundation
On The Cloud



Rolling out Zmanda as an Enterprise wide
backup solution


Allows local disk and


Amazon S3 storage


Full backup sets flow to

on
-
site backup, ‘essential’

data continues onto the

S3 cloud


Fully supported system


(akin to Symantec) that


takes programming off


of department staff


Standard backup sets: saves as ZIP, TAR, etc.




Disaster Recovery: The Next
Steps



Essential data is on S3


can we connect it to Amazon’s EC2
(Elastic Cloud Computing) service? (yes!)







Setting up a suite of servers (Windows and Linux based) that are
in use only when testing or during a disaster recovery scenario


For EC2 ‘CPU Instances’ that power the services, we pay 10 to 12.5 cents an
hour for testing for each instance, can scale processes in a disaster


Grab S3 backup data for a recent data set in a disaster scenario


Easy to manipulate/extract given standard ZIP and TAR formats




Obstacles and Potential Growth


Obstacles

(see consideration slide in NorthEastern section)


We are currently focusing on: Licensing issues, IP
addressing, ‘rooted services’ (Active Directory)


Collaborative resources for effective Testing

Potential Growth


Cloud Bursting (via enomalism?)


Full business crucial DR is possible... For less money
than we were paying for existing (Live Vault) remote
back
-
up storage



S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

Cloud Computing in the Classroom

Justin Riley (
jtriley@mit.edu
)


Software Tools for Academics and Researchers

Office of Educational Innovation and Technology

Massachusetts Institute of Technology

77 Massachusetts Ave.

Cambridge, MA 02139

S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

Outline


The STAR Group


HPC Use
-
Cases in the Classroom


Faculty Concerns


Cloud Computing as a Solution


Our Cloud Computing Experiences at MIT


Converting Cloud Computing
Configurations into Products (StarCluster)

S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

The STAR Group

What's your biggest problem bringing your research into the classroom?

S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

HPC use
-
cases in the classroom

Constantinos Evangelinos, PhD

Researcher in Earth,
Atmospheric, and Planetary
Sciences. (MIT)



Parallel Programming

Course
:



Parallel

Programming

for

Multi
-
core

Machines

Using

OpenMP

and

MPI

S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

HPC use
-
cases in the classroom

Markus Buehler, PhD

Assistant Professor in Civil
and Environmental
Engineering (MIT)



Nicola Marzari, PhD

Associate Professor in
Computational Materials
Science (MIT)

Materials Modeling

Courses
:



Introduction to Modeling and Simulation (undergraduate)




Atomistic Computer Modeling of Materials (graduate)

S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

Faculty Concerns

Administrative

overhead



Teaching Assistants acting as volatile system admins



What happens when TA's/Graduate Students graduate?



Reliability of resource configurations



Has the cluster changed since we've last used it?



Does the software still work?


Ability to scale with the number of students



I need to accommodate X students but only have N resources



I just need a few resources for a small class/demonstration.


Resource Accessibility



How do I give every student access to these resources?



Do we make accounts for students on someone's cluster?



How do these accounts change from year to year?



Who manages student accounts?

S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

Cloud Computing as a Solution



Amazon Elastic Compute Cloud (
EC2
)



Amazon Simple Storage Service (
S3
)



Amazon Elastic Block Storage (
EBS
)

EC2 Features
:



Dynamically allocate/terminate Linux “virtual” machines




Pay only for what you use (i.e. machine hours and data transfer)




Capture software configurations into Amazon Machine Images (AMI) for later use.



EBS Features
:



Highly available/reliable volumes that can be attached to a running EC2 machine (ie like
plugging in a USB key to the machine).




Allows you to create point
-
in
-
time snapshots of your data.



S3 Features
:



Amazon S3 provides a simple web services interface that can be used to store and retrieve
any amount of data, at any time, from anywhere on the web.

:

S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

Our Experiences with Cloud
Computing at MIT

StarHPC
-

an on demand compute cluster for parallel programming with
both OpenMP and OpenMPI technologies. It provides a virtual desktop
environment, hosted on EC2, configured with all the necessary tools for
programming in OpenMP/OpenMPI.


http://web.mit.edu/star/hpc



StarMolsim

-

a web application used to run materials modeling research
software. It enables the user to run various simulations on a distributed
compute cluster and retrieve the results, all from a web browser.


http://web.mit.edu/star/molsim

S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

StarHPC

Use case
: students have direct access to the cluster to actively develop
parallel programs using the Message Passing Interface (MPI)

StarHPC was used for 2 weeks in an Independent Activities Period (IAP)
course for parallel programming using OpenMP and OpenMPI.

Result
: Creating a 4
-
node cluster for two weeks came out to about $25 per
student using Amazon EC2.

SSH/VNC

Virtual Desktop Environment

EC2 OpenMPI
Cluster

User

S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

StarHPC

Using EC2 allowed us to fully automate:




Launching new nodes (or instances)




Generating user accounts/passwords




Configuring passwordless ssh access between the nodes for
each user (a strict requirement for OpenMPI)




Generating example OpenMP/OpenMPI projects in Eclipse
and Netbeans that are immediately available to the user and
will run on the spot.




Customizing desktop configurations with easy links/menus to
the parallel programming development environments

All at the press of a button!

S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

StarMolsim

Use case
: students log in to a web application as a proxy to the computing
resources. The web application handles communicating with the cluster to
submit jobs, retrieve the results, etc.

Result
: EC2 is being used to replace a traditional 9
-
node HPC cluster for ~3
months in the Intro to Modeling and Simulation course at MIT. Depending on
the instance type and data transfer, the cost for using the 9 node EC2 cluster
for the semester is around $2,000
-
4,000.

User

EC2 OpenMPI
Cluster

Web Server hosting GenePattern from
the Broad Institute of MIT and Harvard

S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

StarMolsim

Using EC2/EBS allowed us to fully automate:




Launching new nodes (or instances)




Configuring passwordless ssh between nodes




Setting up network file system for shared folders (NFS)




Configuring Sun Grid Engine




Attaching the software needed for the course using EBS




Creating backups via EBS point
-
in
-
time snapshotting




Providing cluster usage statistics to the user via a web
interface


All at the press of a button!

S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

StarCluster

http://web.mit.edu/stardev/cluster

StarCluster is a utility for creating and managing general purpose compute
clusters hosted on Amazon's Elastic Compute Cloud (EC2).

StarCluster makes it easy for a user to create their own compute cluster on
EC2 and pay only for what they use.

Software included in the virtual machine
:



OpenMPI



NFS'd /home directory



Sun Grid Engine



Compilers for installing your own custom software



Linux OS with package manager for installing additional OS software

StarCluster

Dependencies
:



Registered and fully configured EC2 account.



Python 2.4+



Paramiko library for Python

S
oftware

T
ools

for

A
cademics

and

R
esearchers


http://web.mit.edu/star

Acknowledgements



Professor Buehler (MIT)




Professor Marzari (MIT)




Constantinos Evangelinos (MIT)




Broad Institute (MIT and Harvard)

Questions?

Thank you


Greg Banwarth,
gbanwarth@wheelock.edu

Rick Mickool,
r.mickool@neu.edu

Justin Riley,
jtriley@mit.edu