The CloudStack development story and future vision

joeneetscompetitiveSecurity

Nov 3, 2013 (3 years and 9 months ago)

105 views

The CloudStack development
story and future vision

Sheng Liang, CTO Cloud Platforms, Citrix Systems

August 29, 2012

AWS
is setting the standard…

as measured by capacity…

Every day
through 2011,
AWS added the same
amount of server
processing capacity, on
average, that it took to run
the Amazon online retailing
operation in 2000, when it
was a $2.76bn company.


Prickett
-
Morgan. “AWS Cloud Double Fluffs in 2011.”
The
Register
, 6 Jan 2012.

2.9B

14B

40B

102B

262B

762B

Q4 2006
Q4 2007
Q4 2008
Q4 2009
Q4 2010
Q4 2011
Total Number of Objects

Stored in Amazon S3

Peak Requests:
500,000+

per second

Source: UBS

…data center footprint and geographic
distribution…


the company said that
with the opening of its AWS
data center in São Paulo,
Brazil in mid
-
December, the
company has
doubled

its
AWS data
-
center footprint
.


Prickett
-
Morgan. “AWS Cloud Double Fluffs in 2011.” The
Register, 6 Jan 2012.

AWS Regions

Amazon Edge Locations (
CloudFront

& Route 53)

$0
$200
$400
$600
$800
$1,000
$1,200
$1,400
2006e
2007e
2008e
2009e
2010e
2011e
All Other
…and, most importantly, revenue…

It
has been estimated
that AWS could be a
$1
billion business

for the
online retailer come
next year
…could
hit
$2.5B in
2014
.



Hickey, Andrew. “Amazon Q3 Cloud Revenue
Skyrockets” CRN. 26 Oct 2011.

Amazon Web Services Revenue Model

$M

Source: UBS

H
ow did Amazon build its Cloud?

Servers

Storage

Networking

Open Source
Xen

Hypervisor

Amazon Proprietary Orchestration Software

AWS API (EC2, S3, …)

Amazon
eCommerce

Platform

How can we build a cloud using
CloudStack
?

“Cloud OS or
Data Center
OS”

Servers

Storage

Networking

Open Source
Xen

Hypervisor

Amazon Proprietary Orchestration Software

AWS API (EC2, S3, …)

Amazon
eCommerce

Platform

XenServer

Apache
CloudStack

User Portal

Cloud API

VMware KVM OVM Hyper
-
V Bare
-
Metal

The Virtual Datacenter OS allows businesses to efficiently
pool all types of hardware resources
-

servers, storage and
network


into an aggregated on
-
premise
cloud

VMware press release Sept 15, 2008

Eucalyptus is the only cloud architecture to support the
same application programming interfaces (APIs) as public
clouds, and today Eucalyptus is fully compatible with the
Amazon AWS public cloud infrastructure.



Eucalyptus Systems Press Release April 2009

OpenNebula

Zimory

Cassatt

3Tera

Enomaly

Joyent

Nimbula

Abiquo

Yunteq

OnApp

Eucalyptus

VMware

Cloud.com June 2009

Prototype

1.0 GA

2.0
Refactor

AWS
Compatibility

2.2
Refactor

3.0 Quality
Improvements

2008

Sept 2008:
VMOps

Founded

2009

Nov 2009:
CloudStack

1.0 GA

2010

May 2010:
Cloud.com

Launch &

CloudStack

2.0 GA

2011

July 2011:
Citrix
Acquires
Cloud.com

2012

April 2012:
Apache
CloudStack

4.X

Refactor


Initial target: hosting companies like Rackspace and
Savvis


3 engineers built a fully functional prototype in 5 months


Use the demo to sell to early customers (
ReliaCloud
,
CloudCentral
,
1800hosting.com, Go Daddy, etc.)

Prototype

1.0 GA

2.0
Refactor

AWS
Compatibility

2.2
Refactor

3.0 Quality
Improvements

4.X

Refactor

2008

2009

2010

2011

2012


Took 6 more months to make 1.0 software production ready


Deployed on 5 production customers

2.0
Refactor

AWS
Compatibility

2.2
Refactor

3.0 Quality
Improvements

4.X

Refactor

2008

2009

2010

2011

2012

Prototype

1.0 GA


Product first, architecture second


From web hosting to enterprise workload


Multi
-
hypervisor, SAN, and VLAN support


Learn needs of enterprise workload from: Tata Communications, Korea
Telecom, Macquarie Teleco
m


Competition:
vCloud

Express

2.0
Refactor

AWS
Compatibility

2.2
Refactor

3.0 Quality
Improvements

4.X

Refactor

2008

2009

2010

2011

2012

Prototype

1.0 GA


Private cloud demand picked up


Zynga

wanted private cloud


Support Amazon
-
style flat networking and security groups


Competition: Eucalyptus

2.0
Refactor

AWS
Compatibility

2.2
Refactor

3.0 Quality
Improvements

4.X

Refactor

2008

2009

2010

2011

2012

Prototype

1.0 GA


Second major refactoring of
CloudStack

code


Network
-
as
-
a
-
service combing both Amazon and traditional style networking


More flexible
orchestration engine

2.0
Refactor

2.2
Refactor

3.0 Quality
Improvements

4.X

Refactor

2008

2009

2010

2011

2012

Prototype

1.0 GA

AWS
Compatibility


Citrix acquisition


Rapid growth of
CloudStack

user base


Quality is more important than new features

2.0
Refactor

3.0 Quality
Improvements

4.X

Refactor

2008

2009

2010

2011

2012

Prototype

1.0 GA

AWS
Compatibility

2.2
Refactor


Third major refactoring of
CloudStack

code


Apache contribution drive rapid growth of
CloudStack

developer base


Apache license compliance


Services framework


Hadoop

integration

2.0
Refactor

4.X

Refactor

2008

2009

2010

2011

2012

Prototype

1.0 GA

AWS
Compatibility

2.2
Refactor

3.0 Quality
Improvements


Optimize
Hadoop

on cloud infrastructure


Use HDFS as object store


+

How is cloud different from legacy
infrastructure?

How to handle failures

#
CitrixSynergy

23

8
%

Kashi

Venkatesh

Vishwanath

and
Nachiappan

Nagappan
,
Characterizing
Cloud Computing Hardware
Reliability,
SoCC’10

Annual Failure Rate of servers


Server failure comes from:


70%
-

hard disk


6%
-

RAID controller


5%
-

memory


18%
-

other factors


Application can still fail for
other reasons:


Network failure


Software bugs


Human admin error

Internet

Core Routers

Access Routers

Aggregation Switches

Load Balancers

Top of Rack Switches





Servers

#
CitrixSynergy

25

40
%

Phillipa

Gill,
Navendu

Jain &
Nachiappan

Nagappan
,
Understanding Network Failures
in Data
Centers: Measurement
, Analysis
and
Implications
, SIGCOMM
2011

Effectiveness of network
redundancy in reducing failures


Bugs in failover
mechanism


Incorrect configuration


Protocol issues such
as TCP back
-
off,
timeouts, and spanning
tree
reconfiguration


Cloud workloads

Traditional
-
Style

Reliable hardware, backup entire
cloud, and restore for users when
failure happens

Amazon
-
Style

Tell users to expect failure.
Users to build apps that can
withstand infrastructure
failure

Link Aggregation

Storage Multi
-
pathing

VM HA, Fault Tolerance

VM Live Migration

VM Backup/Snapshots

Multi
-
site Redundancy

Chaos Monkey

Ephemeral Resources

Designing a zone for a traditional workload

vCenter
/
XenCenter

Hypervisor
Cluster

Hypervisor
Cluster

Hypervisor
Cluster

Enterprise Networking (e.g., VLAN)

Enterprise Storage (e.g., SAN)


Hypervisor



Storage


SAN


Networking

L2 VLANs


Network Services


Load Balancing

VPN


Multi
-
tier Apps


Multi
-
tier VLANs

OVF

vSphere

or
XenServer

Enterprise

Traditional
-
Style Availability Zone

Designing a zone for an Amazon
-
style workload


Hypervisor



Storage


Local

EBS

Networking


L3

SDN based L2

Elastic IP


Network Services


Security Groups

ELB


Multi
-
tier Apps


3
rd

Party Tools (e.g.,
RightScale,
enStratus
)

XenServer

Advanced

Object store

GSLB

CloudFormation

Software Defined Networks

(e.g., Security Groups,
EIP, ELB,...)

Amazon
-
Style Availability Zone

Server
Racks

Server
Racks

Server
Racks

Server
Racks

Server
Racks

Server
Racks

Server
Racks

Server
Racks

Server
Racks

Server
Racks

Server
Racks

Server
Racks

Elastic Block Storage

AWS
-
style
Availability
Zone

AWS
-
style
Availability
Zone

AWS
-
style
Availability
Zone

CloudStack

can Support Both Styles

Traditional
Style
Availability
Zone

Apache
CloudStack

Traditional
Style
Availability
Zone

CloudStack

Future

146 Companies

238 Developers

100’s of Production Clouds

32,000 Community Members

Global User Groups

Service Providers

Enterprises

Universities

Apache
CloudStack

community
p
rojects


SDN


Nicira


Midokura


Big Switch Networks


Stratosphere


Backup/DR


Sungard


Networking


Cisco (VXLAN, Nexus)


Brocade (ADX)


Smart Storage


Hadoop

+ S3 API for object store


NetApp

(
FlexPod
,
o
bject
s
tore)


Basho RIAK CS


Caringo object store


Cloudian

S3


PaaS


CloudFoundry

implementation through
IronFoundry

and
Stackato

teams


Engine Yard


Cumulogic


GigaSpaces


“The Apache Way”


C
ollaborative
software development


Commercial
-
friendly
standard license


Consistently
high quality software


R
espectful
, honest, technical
-
based interaction


Faithful
implementation of standards


Security
as a mandatory feature




Innovative Cloud Applications and Services

Servers

Storage

Networking

Innovative Cloud Infrastructure

More information:
http://cloudstack.org

http://cloudstack.jp