Dynamic Provisioning for Multi-tier Internet Applications

groundcombInternet και Εφαρμογές Web

31 Οκτ 2013 (πριν από 3 χρόνια και 11 μήνες)

75 εμφανίσεις

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

Dynamic Provisioning for Multi
-
tier
Internet Applications

Bhuvan Urgaonkar,
Prashant Shenoy,
Abhishek Chandra, Pawan Goyal


University of Massachusetts

University of Minnesota

Veritas Software India Pvt. Ltd.

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

2

Internet Applications


Proliferation of Internet applications

auction site

online game

online store


Growing significance in personal, business affairs


Focus: Internet server applications

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

3

Multi
-
tiered Internet Applications



Internet applications: multiple tiers


Example: 3 tiers: HTTP, J2EE app server, database


Replicable components


Individual tiers: partially or fully replicable


Example: clustered HTTP, J2EE server, shared
-
nothing db


Each tier uses a dispatcher: load balancing


requests

http

J2EE

database


Load

balancer

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

4

Internet Workloads Are Dynamic



Multi
-
time
-
scale variations


Time
-
of
-
day, hour
-
of
-
day




Flash crowds









Key issue
: How to provide


desired response time


under varying workloads?

0

20000

40000

60000

80000

100000

120000

140000

0

5

10

15

20

Time (hrs)

Request Rate (req/min)


0 12 24

Time (hours)

Time (days)

0

1

2

3

4

5

Arrivals per min

0

0

140K

1200

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

5

Internet Data Center


Internet applications run on


data centers


Server farms

o
Provide computational and


storage resources



Applications share data


center resources


Problem:

How should the platform allocate
resources to absorb workload variations?

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

6

Our Provisioning Approach



Flexible queuing theoretic model


Captures all tiers in the application


Predictive provisioning


Long
-
term workload variations


Reactive provisioning


Short
-
term variations, flash crowds

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

7

Talk Outline


Introduction

Internet data center model


Existing provisioning approaches


Dynamic capacity provisioning


Implementation and evaluation


Summary

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

8

Data Center Model


Dedicated hosting:

each application runs on a subset of
servers in the data center


Subsets are mutually exclusive: no server sharing


Data center hosts multiple applications



Free server pool: unused servers


Retail

Web site

streaming

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

9

Single
-
tier Provisioning


Single tier provisioning well studied [Muse]


Non
-
trivial to extend to multiple
-
tiers



Strawman #1: use single
-
tier provisioning
independently at each tier


Problem:

independent tier provisioning may not
increase goodput

C=15

C=10

C=10.1

14 req/s

14

10

10

dropped


4 req/s

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

10

Single
-
tier Provisioning


Single tier provisioning well studied [Muse]


Non
-
trivial to extend to multiple
-
tiers



Strawman #1: use single
-
tier provisioning
independently at each tier


Problem:

independent tier provisioning may not
increase goodput

C=15

C=10.1

14 req/s

14

C=20

14

dropped


3.9 req/s

10.1

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

11

Model
-
based Provisioning


Black box approach


Treat application as a black box


Measure response time from outside


Increase allocation if response time > SLA

o
Use a model to determine how much to allocate


Strawman #2: use black box for multi
-
tier apps


Problems:


Unclear which tier needs more capacity


May not increase goodput if bottleneck tier is not replicable

14 req/s

C=15

C=10.1

14

C=20

14

10.1

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

12

Provisioning Multi
-
tier Apps


Approach:

holistic view of multi
-
tier application


Determine tier
-
specific capacity independently


Allocate capacity by looking at all tiers (and other apps)



Predictive provisioning


Long
-
term provisioning: time scale of hours


Maintain long
-
term workload statistics


Predict and provision for the next few hours



Reactive provisioning


Short term provisioning: time scale of several minutes


React to “current” workload trends


Correct errors of long
-
term provisioning


Handle flash crowds (inherently unpredictable)

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

13

Predictive Provisioning


Workload predictor


Predicts workload based on past observations



Application model


Infers capacity needed to handle given workload

Predictor

Model

past

workload

predicted

workload

required

capacity

response time target

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

14

Workload Prediction


Long term workload monitoring
and prediction


Monitor workload for multiple
days


Maintain a histogram for each
hour of the day

o
Capture time of day effects


Forecast based on

o
Observed workload for that
hour in the past

o
Observed workload for the
past few hours of the current
day


Predict a high percentile of
expected workload


Mon

Tue

Wed

Today

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

15

Model
-
based Capacity Inference


Queuing theoretic application model


Each individual server is a G/G/1 queue






Derive per
-
tier E(r) from end
-
to
-
end SLA


Monitor other parameters and determine
l (
per
-
server capacity)


Use predicted workload
l
pred

to determine # servers per tier

o
Assumes
perfect load balancing

in each tier

G/G/1

G/G/1

G/G/1

l
pred

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

16

Reactive Provisioning


Idea:

react to current conditions


Useful for capturing
significant

short
-
term fluctuations


Can correct errors in predictions


Track error between long
-
term predictions and actual


Allocate additional servers if error exceeds a threshold


Account for prediction errors


Can be invoked if request drop rate exceeds a threshold


Handles sudden flash crowds


Operates over time scale of a few minutes


Pure reactive provisioning: lags workload


Reactive + predictive more effective!

Prediction

error

l
pred

l
actual

l
error


>
t

Invoke

reactor

time series

allocate


servers

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

17

Talk Outline


Introduction


Internet data center model


Existing provisioning approaches


Dynamic capacity provisioning

Implementation and evaluation


Summary

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

18

Prototype Data Center


40+ Linux servers


Gigabit switches


Multi
-
tier applications


Auction (RUBiS)


Bulletin
-
board (RUBBoS)


Apache, Tomcat (replicable)


Mysql database


Control Plane

Dynamic provisioning

Nucleus

Apps


OS

Server Node

Applications

Resource monitoring

Parameter estimation

Nucleus

Apps


OS

Nucleus

Apps


OS

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

19

Only Predictive Provisioning

Workload

Response time


Auction application RUBiS


Factor of 4 increase in 30 min


Predictor fails during [15, 30] resulting in under
-
provisioning


Response time violations occur

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

20

Only Reactive Provisioning

Workload

Response time


Auction application RUBiS


Factor of 4 increase in 30 min


Response time shows oscillatory behavior


Several response time violations occur

Time (min)

Resp time (msec)

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

21

Predictive + Reactive Provisioning

Workload

Response time

Server allocations


Auction application RUBiS


Factor of 4 increase in 30 min


Server allocations increased to match increased workload


Response time kept below 2 seconds

20

40

60

80

100

120

140

160

0

10

20

30

40

50

60

Arrivals per min

Time (min)

0

1000

2000

3000

4000

5000

6000

7000

0

10

20

30

40

50

60

Resp time (msec)

Time (min)

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

22

Summary


Dynamic provisioning for multi
-
tier applications


Flexible queuing theoretic model

o
Captures all tiers in the application


Predictive provisioning


Reactive provisioning



Implementation and evaluation on a Linux
cluster

U
NIVERSITY

OF

M
ASSACHUSETTS,
A
MHERST


Department of Computer Science

23

Thank you!

More information at:


http://www.cs.umass.edu/~bhuvan