Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

meatcologneInternet και Εφαρμογές Web

3 Νοε 2013 (πριν από 3 χρόνια και 7 μήνες)

84 εμφανίσεις

1
/17

Distributed Systems Architecture Research Group

Universidad Complutense de Madrid

Execution of SGE Clusters on top of

Hybrid Clouds using OpenNebula

Constantino Vázquez Blanco

Sun HPC Software Workshop

Regensburg, Germany

7
-
10 September 2009

2
/17

Objectives


Benefits

of the latest OpenNebula version


OpenNebula
features
for the different cloud
types


Compare OpenNebula
with other similar
technologies


Ecosystem

of the technology


SGE Clusters

on top of
Hybrid Clouds
using
OpenNebula


Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

3
/17

What is OpenNebula?


Orchestrates

storage,

network

and

virtualization

technologies

to

enable

the

dynamic

placement

of

multi
-
tier

services

on

distributed

infrastructures,

combining

both

data

center

resources

and

remote

cloud

resources
,

according

to

allocation

policies


Provides

internal

and

Cloud

administration

and

user

interfaces

for

the

full

management

of

the

IaaS

Cloud

platform

Open
-
Source Toolkit for Building Cloud Infrastructures


Private Cloud
: Management of
virtual infrastructure in the data
-
center or cluster



Hybrid Cloud
: Combination of
private with Cloud resources



Public Cloud
: Cloud interfaces for
the full management of services


Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

4
/17

Benefits


Faster

response

to infrastructure needs for services


Centralized management
of the virtual and physical distributed infrastructure


Higher utilization
of existing resources


Operational saving
with server consolidation


Lower infrastructure expenses

Infrastructure Manager

Infrastructure User


Faster

delivery and scalability
of services to meet dynamic demands


Support for heterogeneous
execution environments


Full control
of the lifecycle of virtualized services management

System Integrators


Fits into any existing data center


Builds

any type of Cloud deployment


Open source software
, Apache license


Seamless integration
with any product and service in the virtualization/cloud
ecosystem and management tool in the data center

Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

5
/17

Private Clouds

Features


Private Clouds


The infrastructure is owned and used by a single organization


Private clouds enable
a flexible and agile management of local infrastructure


Not a new model
, datacenter management has been around for a while


Internal interfaces expose additional functionality
for managing virtualized
resources and controlling data center operation, not exposed by cloud interfaces


Cloud interfaces
may be also provided for users requiring higher abstraction

Internal Interface


Centralized management


VM placement optimization


Dynamic resizing and partitioning
of the infrastructure


Support for heterogeneous
workloads

Administration Domain

Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

6
/17

Features


Private Clouds

Feature

Function

Internal Interface


Unix
-
like CLI for fully management of VM life
-
cycle and physical boxes


XML
-
RPC API and libvirt virtualization API

Scheduler


Requirement/rank matchmaker allowing the definition of workload and
resource
-
aware allocation policies


Support for advance reservation of capacity through Haizea

Virtualization
Management


Xen, KVM, and VMware


Generic libvirt connector (VirtualBox planned for 1.4.2)

Image Management


General mechanisms to transfer and clone VM images

Network Management


Definition of isolated virtual networks to interconnect VMs

Service Management
and Contextualization


Support for multi
-
tier services consisting of groups of inter
-
connected
VMs, and their auto
-
configuration at boot time

Security


Management of users by the infrastructure administrator

Fault

Tolerance


Persistent database backend to store host and VM information

Scalability


Tested in the management of medium scale infrastructures with hundreds
of servers and
VMs

(no scalability issues has been reported)

Installation


Installation on a UNIX cluster front
-
end without requiring new services


Distributed in Ubuntu 9.04 (Jaunty Jackalope)

Flexibility and
Extensibility


Open, flexible and extensible architecture, interfaces and components,
allowing its integration with any product or tool

Private Clouds

Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

7
/17

Hybrid Clouds

Features


Hybrid Clouds


Extension of Private Clouds to
combine private with public Cloud
-
based
infrastructure to enable highly scalable hosting environments


Access to remote cloud is
fully transparent
to private cloud users


Hybrid Clouds enable the
dynamic scaling of capacity to meet peak or
fluctuating demands


Remote Cloud

Administration Domain

Internal Interface

Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

8
/17

Features


Hybrid Clouds

Feature

Function

Cloud

Plugins


Amazon EC2 and ElasticHosts connectors

Federation


Support for simultaneous access to several remote clouds


Flexibility


Modular approach to develop new connectors


Hybrid Clouds

Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

9
/17

Public Clouds

Features


Public Clouds


The infrastructure is
owned by a single commercial organization and used
by customers


Public clouds enable the deployment of an entire IT infrastructure
without the
associated capital costs, paying only for the used capacity


Cloud interface:
Simple remote management of virtualized server instances

Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula


Remote Cloud

Administration Domain

Internal Interface

Cloud Interface

10
/17

Features


Public Clouds

Feature

Function

Cloud

Interfaces

for

Users


Implementation

of a
subset

of the EC2
Query

API

Flexibility


The Cloud Service allows the implementation of new Cloud interfaces
(OCCI planned for 1.4.2)


Public Clouds

Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

11
/17

Comparison with Similar Technologies

Platform ISF

VMware Vsphere

Eucalyptus

Nimbus

OpenNebula

Virtualization
Management

VMware, Xen

VMware

Xen, KVM

Xen

Xen, KVM,
VMware

Virtual Network
Management

Yes

Yes

No

Yes

Yes

Image
Management

Yes

Yes

Yes

Yes

Yes

Service
Contextualizatio
n

No

No

No

Yes

Yes

Scheduling

Yes

Yes

No

No

Yes

Administration
Interface

Yes

Yes

No

No

Yes

Hybrid Cloud
Computing

No

No

No

No

Yes

Cloud Interfaces

No

vCloud

EC2

WSRF, EC2

EC2 Query

Flexibility and
Extensibility

Yes

No

Yes

Yes

Yes

Open Source

No

No

GPL

Apache

Apache

Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

12
/17

RESERVOIR

Flagship of Cloud Computing Research in FP7


Open source technology
to enable deployment and management of
complex
IT services
across

different administrative domains


Functionality for Service Management


Definition


Lifecycle


Billing/accounting


Elasticity/SLAs


Resources and Services Virtualization without Barriers

VM

VM

VM

VM

VM


Focus on technologies that enable to build a
federation of cooperating
computing clouds


A project driven by business use cases
: SAP business application, Telco
application, utility computing and eGov application


17
-
million and 3
-
year project partially
funded by the European Commission


Partners: IBM, Telefónica, UCL, Umea, SAP, Thales, SUN, Elsag Datamat,
UCM, CETIC, University of Lugano, University of Messina, OGF.eeig.

Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

13
/17

RESERVOIR

RESERVOIR Architecture

Service Provider

Service Manager

VEE Manager

(OpenNebula)

VEE Host

Service Provider

Service Provider

VEE Host

VEE Host

VEE Manager

VEE Host

VEE Host

VEE Host

VHI

VMI

VMI


Infrastructure Providers

Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

14
/17

Ecosystem

Related Technologies


Haizea
(Uchicago)
:
Open
-
source VM
-
based lease management architecture


Several tools
for service elasticity management, VM scheduling…
being developed
around OpeNebula in RESERVOIR (IBM, Telefonica I+D,…)

Infrastructure Technology


KVM
Management Tools


Xen

Community Project


Cloud Services


Technology Partner of
ElasticHosts


Project in the
Amazon EC2 Solutions Catalog


Cloud Solutions and Tools


Libvirt
CLI and Desktop Applications

Open
-
source Distributions


Ubuntu

9.04 (Jaunty Jackalope)


Chapter on Cloud Technologies in the
Morfeo

open
-
source community

Standardization Bodies


OGF Open Cloud Computing Interface

Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

15
/17

Scaling SGE cluster in Hybrid Clouds

Infrastructure Perspective

Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

16
/17

Scaling SGE cluster in Hybrid Clouds

Service Perspective

Execution of SGE Clusters on top of Hybrid Clouds using OpenNebula

17
/17

THANK YOU FOR YOUR ATTENTION!!!

More info, downloads, mailing lists at

www.OpenNebula.org

Acknowledgements



OpenNebula is partially funded by the

“RESERVOIR


Resources and
Services Virtualization without Barriers” project

EU grant agreement 215605

www.reservoir
-
fp7.eu/


Ignacio M. Llorente


Rubén S. Montero




Javier Fontán


Rafael Moreno


Scaling SGE cluster in Hybrid Clouds