Cloud Federationsx - contrail-project

italiansaucyΛογισμικό & κατασκευή λογ/κού

13 Δεκ 2013 (πριν από 3 χρόνια και 10 μήνες)

90 εμφανίσεις

Cloud Federations

Patrizio Dazzi

(ISTI
-
CNR
) [Overall Presentation]

p
atrizio.dazzi@isti.cnr.it


Gaetano
Anastasi

(ISTI
-
CNR) [Hands
-
On]

g
aetano.anastasi@isti.cnr.it


contrail

is co
-
funded by the EC 7th
Framework
Programme

under
Grant Agreement nr. 257438

http://contrail
-
project.eu

Presentation
Outline


Cloud

Federations



Contrail

Federations



Contrail

Federation

Architecture



Federations

and
Resources



A
few

details

about

the
current

status of
Contrail

Cloud

Federations



Introduction

to the
hands
-
on

session

Cloud

Federations

What

is

Cloud

Computing ?



Computing
as

a public utility


Clouds

are the
computing


power

plants




Do
not

manage

resources
, just use
them



Choose

the
interaction

model:
SaaS
,
PaaS
,
IaaS



4

Principal

Advantages

of
Cloud

Comuting



Advantages

f
or
Tenants
:


Pay

just
what

you

get


Pay

only

when

you

get


Reduce
costs

for
maintenance



Advantages

f
or Providers


maximizing

the
effectiveness

of the
shared

resources
.


dynamically

re
-
allocation

depending

on
actual

usage

5

Anyway



Providers
have
:


A
limited

amount

of
resources


A
limited

range

of
resource

types


Resources

placed

only

in
same

countries



Providers
want
:


to
avoid

that

their

customers

leave

them


To
make

(more)
money



Users

want


To
have

all

the
resources

they

need

(
possibly

more)


To
pay

less
….



How can
we

manage

this

???

6

Cloud

Federations


A
Federation

of
Clouds

is

(
maybe
) the
answer


By
federating
, providers can
offer

more
resources


More
resources

means



T
o be
able

to
run

bigger

applications


To be more
elastic


To
provide

(
potentially

more)
different

resources


A
federation

(
may
)
allow

users

to


Choose

among

a
wider

range

of providers


Seamlessly

use more providers for the
same

application


Migrate from a
provide

to
another

7

Federation

of
IaaS

providers


Infrastructure as a Service


provide computation, storage and network



IaaS

Providers


distinct Clouds may have different interfaces,
access rules, capabilities, prices, …



Special Providers


Peculiar computational supports


Specific Storage capabilities


High performance networking

8

How
Federation

of
IaaS

impacts

on
PaaS

and
SaaS


The background:


Platform and Software as a service offer an even
more complex landscape


PaaS

and
SaaS

provider may be distinct from
IaaS

ones



IaaS

Federation can


Allow properly instrumented
PaaS

and
SaaS

to
run on top of a huge amount of resources


Need to translate
PaaS

and
SaaS

requirements


To address heterogeneity of providers


Enforce
QoS

guarantees

9

Unified

Identity and Billing management



Federations should be almost invisible from
the point of view of users but:


User’s identity should be automatically managed


In a cross
-
cloud fashion


Creating and managing proper identities in each cloud
(if needed)


Map by keeping proper capabilities and permissions



Users have to be properly billed


Unified monitoring


Unified accounting


10

Security
Mechanisms


Authentication


Allow the access of users to Clouds


Exploit existing identity infrastructures


E.g.
OpenID


Enact an identity federation



Authorization


Access Control


Usage Control


Particularly useful for long lasting applications

11

Cross
-
Cloud

management of Service Level
Agreements


Cloud Applications have
QoS

requirements


Performance, reliability, security


Typically expressed as SLAs



SLA management is a complex activity also in a
single Cloud


Violations may occur and need to be managed



Even more complex in a Federation of Clouds


Not all providers provide the same degree of


guarantees


strictness


penalties


Compensation


An overall coordination activity shall be performed


12

Last
but

not

Least
:
Brokering


Cloud Federations also behave as brokers


To find for each user’s application the best
Cloud(s) depending on


Application structure and requirements


Cloud reputation


Cost


Splitting the user’s application in order to


Choose the best Cloud for each part of the application


Allow a better exploitation of Specialized Clouds

13

Contrail Federations

14

Contrail
Iaas

Federation


A Contrail Federation integrates in a common platform
multiple Clouds, both public and private ones.

To perform this task it provides:



A common support for authentication, authorization and
billing


M
echanisms for policy definition,
monitoring
, and
enforcing

for all the
QoS
-
related aspects


An automated selection of providers depending on the
user applications


A support for resource provisioning to applications able
to deal with heterogeneous sets of resources

Contrail development methodology



Do not rebuild and reinvent


Contrail exploits Standards


OVF


OCCI


OpenID



Exploit existing platforms and functionalities


Open Nebula


SLA@SOI


Etc.

16

Develop

a Federation support that integrates and
actively coordinates SLA management provided by
single Cloud providers



Do not disrupt provider’s business model


Allow exploiting a Federation seamlessly


Federation

Support must
be

scalable

Exploit Providers’ SLA
support

Main Contrail Federation Innovations (1)

Federation = more than a simple broker, or
portal, or decentralized cloud
-
bursting



Interoperability


Heterogeneous providers


Dynamically
choosing best providers


At deploy time and at
runtime


Allow to
combine
providers


Migration, elasticity


Security and privacy framework

Main Contrail Federation Innovations
(2)

19



Sophisticated SLA/
QoS



QoS

via SLA


Via provider selection
and
integration


Enforcement mechanisms


Federation as a
mediator
and a 3
rd

party


Federation also acts as a
coordinator

Contrail

Federation

Architecture

20

Simplified

Blocks

Architecture

Final

Users

ConPaaS

Cloud

Federation

Cloud

Provider
1

Cloud

Provider
2



Cloud

Provider
N

21

Distributed Federation Access Points


Several Federation access points (FAPs)


FAP act as brokers, but share a common
view


Security, status of resources, users and providers
reputation

F

F

F

F


Hierarchical structure


Common Policies


Detailed resource
allocation is on providers


AP
may be

hosted by
providers

Federation Interfaces

SLA

Auth

AuthZ

Federation Core

Coarse
-
grain view of Contrail Federation
Architecture


Functionalities which extend horizontally in the platform

Provisioning Manager

Concrete
Architecture

In the next slide



Module View of a single Access point



Interactions, some modules not shown



Interfaces sit on top of this core



Auth
/
AutZ

mechanisms included

24

Contrail

federation

25

Contrail Provider
Interface layer
HTTP
REST
Federation support
User Identity
GAFS driver
VIN driver
CLI
External Cloud
adapters
Core layer
Adapters layer
VEP
driver
State
SLA
Coordination
SLA
Negotiation
SLA
Organizer
Provider W
atcher
SLA

Management
SLA
T
emplate
Repository
Federation Runtime
Manager
Mapping
Attribute
Authority
Policy
Administration Point
Policy Decision Point
Authentication
Security
SLA

Management
External Provider
Image Registry
Image Manager
Provisioning

Manager

Provisioning

Manager

Federations

and
Resources

26

Basic concepts


What’s an application for
IaaS
?


A set of software entities which need to be
deployed on a suitable set of resources for
execution


The parties involved


User


Who submit applications


Provider


Any entity managing physical resources which may be
used to run applications


Federation


The union of many providers under a common set of
APIs and functionalities, which can be exploited as a
single Cloud

Application Description


The requirements so far can be expressed as
a Task Interaction Graph


An undirected graph G(N,E) can model a
distributed application


Each Node in N is a task (needs a resource)


Edges imply relations between tasks (need links)


Heavily labeled graph


Nodes state all resource

constraints
and
SLA measures


Edges labeled with

communication needs


Cloud applications can get

hard to describe


Types of resources


Computation resources


Available VMs slots on top of physical hardware


Storage resources


Shared
filesystems


Shared Databases


Networking resources


Virtual networks connect machine within an
application


Behaviour

of the joining points between the
internet and the federation is important

How resources are measured ?


Static specification by analogy with the
physical counterparts


Memory size, CPU type and speed (peak)


Size, FS of storage


Nominal bandwidth and latency of networks


Dynamic specification


Available computing power, memory


Actual (average, peak, used) storage speed


Observed in
-
Cloud bandwidth and latency


Observed
bw
/lat to the outside


Application
-
specific metrics



Provider’s View on resources


Providers know exactly the layout and state
of all physical (virtual) resources


Dedicated link bandwidths, physical memory…



They can greatly optimize resource
management


Turn off unused resources, exploit cheapest


They have their own goal (revenues)



QoS

properties of resources


Extend the set of characteristics to be
measured on the platform


Protection


Type of security mechanisms which are in place


Auth. Protocols, Encryption mechanisms, Isolation


Privacy


Guarantees offered by storage holder, network
infrastructure


Geo
-
localization


Can have deep legal implications


Power consumption


Overall power, power efficiency

QoS

expressed via SLAs


As the SLA is signed, the user should be able
to trust resources from the platform


But not all Providers may offer the same reliability



How reliability can be measured ?


failures, SLA violations with different providers


lengthy task with poor reward for single users


Provider Reputation


Management information


Available resources per kind


Features granted


Amount of users/apps ongoing


State of
SLAs

controlled by the federation


Static level of “trust” given from federation to the
provider


Past History


History of SLA violation per user / type of app


Average level of satisfaction of SLA



Contrail approach to SLA


Reuse SLA@SOI framework as a starting point


Integration with Contrail internal interfaces and
components


Integration with domain
-
specific reasoning/monitoring
plugins


Extend SLA@SOI with:


Federation

support


Integration
with

external

providers (
and

their

SLA
management systems)


Reputation model for providers


Cost
-
based
QoS

enforcement


Federation
-
level SLA Enforcing


Federation will act as a SLA coordinator over
providers


As much as possible the single providers is in charge of the
local SLA


Reduce reaction delay



Federation evaluates the provider and the app



Extend the SLA@SOI monitoring infrastructure to the
federation


Keep track of main application parameters



Receive SLA violations from providers



Reallocate some resources on a provider basis


May require a new negotiation between federation and provider

What if… a SLA is violated in a single
provider scenario ?

The application is deployed on a single
provider, which may still violate SLA




Actions:


The provider resize the set of allocated resources


If the application is no longer violating SLAs


The application will be up and running and that’s it


Otherwise


A renegotiation phase is conducted and if will not be
successfully

t
he application will be terminated

…and
what

if

the
violation

occurs

in a multi
-
provider scenario ?

The
application

has

been

sent

by the
federation

to a provider for the
execution

and
a SLA
violation

occurs



Actions
:


Previous

slide shows
what

happens

when

the
provider
is

able

to
manage

the
violation

by
itself


If

it

is

not

able
, the
federation

can
still

migrate
(part of) the
application

to a
different

provider

38

Contrail S. School 2012


P. Dazzi
-

Cloud Federations

Applications Running in Multiple Providers

Some
a
pplications could be also decomposed in
parts and each deployed in a different provider


Actions


resize
part(s
)


managed by providers


migrate
part(s
)


may need to stop the application


violate constraints


By leveraging the whole federation we push away the limit
where a violation is unavoidable


rebalance constraints


By renegotiating with more providers, overall SLO may be
achieved


Increase resource availability where they are
cheaper/more available at the moment

Anyway SLA splitting is still an open issue


Necessary to leverage SLA management at
single providers


How to derive a combination of SLA for
separate parts of the application which
allow to manage the application overall


Not yet addressed in literature


Not hard if providers are specialized


Compute, network, storage = SLA aspects


As the user expresses SLA terms on parts of
the application (task groups) this will
become easier

Contrail Federation, the 3
rd

Player


A Contrail Federation sits in the middle


Aims


Serving the users


Exploit efficiently the providers


If economic gain is sought, it comes form
intermediation


Can efficiently gather provider information


Gathers from all users and all providers


Can make better informed choices than users


Can afford to launch directed tests in doubt


Can trigger corrective actions impossible to
single providers

A
few

details

about

the
c
urrent

status of
Contrail

Cloud

Federations

42

How

it

is

used


Federation Interfaces use REST


The Web portal and command line tools access
the REST layer


Different roles


federation coordinator


cloud administrator


end user


Main steps


account creation


SLA formation


Application upload (
VMs

and descriptor)


SLA negotiation and agreement


Deploy

43

What

is

implemented

today


User management


Registration, mapping are working


some integration activities are still ongoing


SLA Management


Negotiation with providers counterpart is working


A complete integration is expected in the next
release


Application management


Applications can be submitted and executed


So far only one provider at a time can be used
for a single application

44

Aligning with standard formats


OVF (Open Virtualization Format)


Open format for describing application


Allows to describe structured applications


Directly expresses only HW constraint and
deployment information


Partially overlaps with full SLA specification


OVF is extendable


As of v1.1.0 it does not target Application
Management


SLA@SOI provides a general formalization


Includes monitoring hierarchy and negotiation


Exploit a combination of OVF and SLA@SOI
mechanisms


Securing
” the
implementation


The
current

implementation

of
federation

authentication


Support

for
OpenID


Additional

details

in
Jens’s

keynote





The
current

implementation

of
federation

authorization


Advanced
support

to
access

control


Ongoing

work to
finalize

the
support

to
usage

control

46

Ongoing

work
about

advanced

features


Multi
-
criteria

mapping

algorithms


Genetic

algorithms

for
application

mapping


Evolving

mapping

plans

to
achieve

better

allocations



Application
splitting


Interplay

with SLA
splitting



Monitoring

data
aggregation

and
filtering

47

Introduction

to the
hands
-
on

session

Contrail S. School 2012


P. Dazzi
-

Cloud Federations

48

Sample Application


OVF Representation


2 Appliances

49

Contrail S. School 2012
-

M. Coppola
-

Cloud Federations

<
VirtualSystem

ovf:id
="VirtualSystem1">


<
VirtualHardwareSection
>


<
Item>



<
rasd:Description
>
Number

of virtual
CPUs
</
rasd:Description
>



<
rasd:ElementName
>1 virtual
CPU</
rasd:ElementName
>


<
rasd:InstanceID
>1</
rasd:InstanceID
>


<
rasd:ResourceType
>3</
rasd:ResourceType
>


<
rasd:VirtualQuantity
>1</
rasd:VirtualQuantity
>


</
Item>


<
Item>



<
rasd:AllocationUnits
>byte *
2^20</
rasd:AllocationUnits
>


<
rasd:Description
>Memory
Size
</
rasd:Description
>


<
rasd:ElementName
>512 MB of
memory</
rasd:ElementName
>


<
rasd:InstanceID
>2</
rasd:InstanceID
>


<
rasd:ResourceType
>4</
rasd:ResourceType
>


<
rasd:VirtualQuantity
>512</
rasd:VirtualQuantity
>


</
Item>





Simplified Deployment Chain


Users submit an OVF


Federation mapping phase:


Which provider(s) for the application?


Federation Application Submission


Provider Deployment


Provisioning Manager
: receives requests coming
from the federation


VEP: manages provider resources for
deployment


For simplicity do not consider SLAs

50

Contrail S. School 2012
-

M. Coppola
-

Cloud Federations

Thanks for you attention!

51

Questions?

Funded under: FP7 (Seventh Framework
Programme
)

Area: Internet of Services, Software & Virtualization (ICT
-
2009.1.2)

Project reference: FP7
-
IST
-
257438

Total cost: 11,29 million euro

EU contribution: 8,3 million euro

Execution: From 2010
-
10
-
01 till 2013
-
09
-
30

Duration: 36 months

Contract type: Collaborative project (generic)

contrail

is co
-
funded by the

EC 7th Framework
Programme