NETS 212: Scalable and Cloud Computing

compliantprotectiveΛογισμικό & κατασκευή λογ/κού

1 Δεκ 2013 (πριν από 3 χρόνια και 8 μήνες)

194 εμφανίσεις

© 2013 A. Haeberlen, Z. Ives

NETS 212: Scalable and Cloud Computing

1

University of Pennsylvania

Programming at scale; Concurrency and consistency


September 5, 2013

© 2013 A. Haeberlen, Z. Ives

Announcements


HW0 will be due next Tuesday (10:00pm)


Any problems with the VM image?


If you still haven't received your svn password, please come
see me after class!



HW1 should be available next Tuesday

2

University of Pennsylvania

© 2013 A. Haeberlen, Z. Ives

Where are we?


Basics: Scalability, concurrency, consistency...


Cloud basics: EC2, EBS, S3, SimpleDB, ...


Cloud programming: MapReduce, Hadoop, ...


Algorithms: PageRank, adsorption, ...


Web programming, servlets, XML, Ajax...


Beyond MapReduce: Dryad, Hive, PigLatin, ...

3

University of Pennsylvania

© 2013 A. Haeberlen, Z. Ives

Scale increases complexity

4

University of Pennsylvania

Single
-
core

machine

Cluster

Multicore

server

Large
-
scale distributed
system

Wide
-
area

network

More challenges

Known from

CIS120

True

concurrency

Network

Message passing

More failure modes

(faulty nodes, ...)

Wide
-
area network

Even more failure

modes

Incentives, laws, ...

© 2013 A. Haeberlen, Z. Ives

Fear not!


You will hear about many tricky challenges



Packet loss, faulty machine, network partition, inconsistency,
variable memory latencies, deadlocks...



But there are frameworks that will help you


This course is
NOT

about low
-
level programming



Nevertheless, it is important to know about
these challenges


Why?

5

University of Pennsylvania

© 2013 A. Haeberlen, Z. Ives

Symmetric Multiprocessing (SMP)


For now, assume we have multiple cores that
can access the same shared memory


Any core can access any byte; speed is uniform (no byte
takes longer to read or write than any other)


Not all machines are like that
--

other models discussed later


6

University of Pennsylvania

Memory bus

Cache

Cache

Cache

Cache

© 2013 A. Haeberlen, Z. Ives

Plan for the next two lectures


Parallel programming and its challenges


Parallelization and scalability, Amdahl's law


Synchronization, consistency


Mutual exclusion, locking, issues related to locking


Architectures: SMP, NUMA, Shared
-
nothing


All about the Internet in 30 minutes


Structure; packet switching; some important protocols


Latency, packet loss, bottlenecks, and why they matter


Distributed programming and its challenges


Network partitions and the CAP theorem


Faults, failures, and what we can do about them

7

University of Pennsylvania

NEXT

© 2013 A. Haeberlen, Z. Ives

What is scalability?


A system is
scalable

if it can easily adapt to
increased (or reduced) demand


Example: A storage system might start with a capacity of
just 10TB but can grow to many PB by adding more nodes


Scalability is usually limited by some sort of
bottleneck



Often, scalability also means...


the ability to operate at a very large scale


the ability to grow efficiently


Example:
4x as many nodes


~4x capacity (not just 2x!)

8

University of Pennsylvania

© 2013 A. Haeberlen, Z. Ives

Parallelization


The left algorithm works fine on one core


Can we make it faster on multiple cores?


Difficult
-

need to find something for the other cores to do


There are other sorting algorithms where this is much easier


Not all algorithms are equally parallelizable


Can you have scalability without parallelism?

9

University of Pennsylvania

void bubblesort(int nums[]) {


boolean done = false;


while (!done) {


done = true;


for (int i=1; i<nums.length; i++) {


if (nums[i
-
1] > nums[i]) {


swap(nums[i
-
1], nums[i]);


done = false;


}


}


}

}

int[] mergesort(int nums[]) {


int numPieces = 10;


int pieces[][] = split(nums, numPieces);


for (int i=0; i<numPieces; i++)


sort(pieces[i]);


return merge(pieces);

}

Can be done in parallel!

© 2013 A. Haeberlen, Z. Ives

Scalability


If we increase the number of processors, will
the speed also increase?


Yes, but (in almost all cases) only up to a point


Why?


10

University of Pennsylvania

Numbers

sorted per

second

Cores used

Ideal

Expected

Speedup:

n
N
T
T
S
1

Completion time

with one core

Completion time

with n cores

© 2013 A. Haeberlen, Z. Ives

part
overall
S
f
f
S



)
1
(
1
Amdahl's law


Usually, not all parts of the algorithm can be parallelized


Let f be the fraction of the algorithm that can be
parallelized, and let S
part

be the corresponding speedup


Then

11

University of Pennsylvania

Time

Time

Time

....

Parallel

part

Sequential

parts

Core #1

Core #2

Core #3

Core #1

Core #2

Core #3

Core #4

Core #5

Core #6

© 2013 A. Haeberlen, Z. Ives

Is more parallelism always better?


Increasing parallelism beyond a certain point
can cause performance to decrease! Why?


Time for serial parts can depend on #cores


Example: Need to send a message to each core to tell it what
to do

12

University of Pennsylvania

Numbers

sorted per

second

Cores

Ideal

Expected

Reality (often)

Sweet

spot

© 2013 A. Haeberlen, Z. Ives

Granularity


How big a task should we assign to each core?


Coarse
-
grain vs. fine
-
grain parallelism



Frequent coordination creates overhead


Need to send messages back and forth, wait for other cores...


Result: Cores spend most of their time communicating



Coarse
-
grain parallelism is usually more
efficient


Bad:
Ask each core to sort three numbers


Good:
Ask each core to sort a million numbers


13

University of Pennsylvania

© 2013 A. Haeberlen, Z. Ives

Dependencies


What if tasks depend on other tasks?


Example: Need to sort lists
before

merging them


Limits the degree of parallelism


Minimum completion time (and thus maximum speedup) is
determined by the longest path from start to finish


Assumes resources are plentiful; actual speedup may be lower

14

University of Pennsylvania

START

DONE

START

DONE

...

"Embarrassingly parallel"

Individual

tasks

With dependencies

Dependencies

© 2013 A. Haeberlen, Z. Ives

Heterogeneity


What if...


some tasks are larger than others?


some tasks are harder than others?


some tasks are more urgent than others?


not all cores are equally fast, or have different resources?



Result: Scheduling problem


Can be very difficult

15

University of Pennsylvania

© 2013 A. Haeberlen, Z. Ives

Recap: Parallelization


Parallelization is hard


Not all algorithms are equally parallelizable
--

need to pick
very carefully



Scalability is limited by many things


Amdahl's law


Dependencies between tasks


Communication overhead


Heterogeneity


...







16

University of Pennsylvania

© 2013 A. Haeberlen, Z. Ives

Plan for the next two lectures


Parallel programming and its challenges


Parallelization and scalability, Amdahl's law


Synchronization, consistency


Mutual exclusion, locking, issues related to locking


Architectures: SMP, NUMA, Shared
-
nothing


All about the Internet in 30 minutes


Structure; packet switching; some important protocols


Latency, packet loss, bottlenecks, and why they matter


Distributed programming and its challenges


Network partitions and the CAP theorem


Faults, failures, and what we can do about them

17

University of Pennsylvania

NEXT

© 2013 A. Haeberlen, Z. Ives

Why do we need synchronization?


Simple example: Accounting system in a bank


Maintains the current balance of each customer's account


Customers can transfer money to other customers


18

University of Pennsylvania

void transferMoney(customer A, customer B, int amount)

{


showMessage("Transferring "+amount+" to "+B);


int balanceA = getBalance(A);


int balanceB = getBalance(B);


setBalance(B, balanceB + amount);


setBalance(A, balanceA
-

amount);


showMessage("Your new balance: "+(balanceA
-
amount));

}

© 2013 A. Haeberlen, Z. Ives

Why do we need synchronization?


What can happen if this code runs concurrently?

19

University of Pennsylvania

1) B=Balance(Bob)

2) A=Balance(Alice)

3) SetBalance(Bob,B+100)

4) SetBalance(Alice,A
-
100)

1) A=Balance(Alice)

2) B=Balance(Bob)

3) SetBalance(Alice,A+500)

4) SetBalance(Bob,B
-
500)

Alice

Bob

$100

$500

Time

Alice's balance:

Bob's balance:

1

2

$200

$800

1

2

3

4

4

3

$200

$900

$700

$900

$700

$300

$100

$300

© 2013 A. Haeberlen, Z. Ives

Problem


What happened?


Race condition:
Result of the computation depends on the
exact timing of the two threads of execution, i.e., the order
in which the instructions are executed


Reason: Concurrent
updates

to the same state


Can you get a race condition when all the threads are reading the data,
and none of them are updating it?

20

University of Pennsylvania

void transferMoney(customer A, customer B, int amount)

{


showMessage("Transferring "+amount+" to "+B);


int balanceA = getBalance(A);


int balanceB = getBalance(B);


setBalance(B, balanceB + amount);


setBalance(A, balanceA
-

amount);


showMessage("Your new balance: "+(balanceA
-
amount));

}

Alice's and Bob's

threads of execution

: Race condition

© 2013 A. Haeberlen, Z. Ives

Goal


What
should

have happened?


Intuition: It shouldn't make a difference whether the
requests are executed concurrently or not



How can we formalize this?


Need a
consistency model
that specifies how the system
should behave in the presence of concurrency

21

University of Pennsylvania

: Consistency

© 2013 A. Haeberlen, Z. Ives

Sequential consistency


Sequential consistency:


The result of any execution is the same as if the operations
of all the cores had been executed in some sequential order,
and the operations of each individual processor appear in
this sequence in the order specified by the program

22

University of Pennsylvania

T1

T3

T6

T2

T4

T5

Core #1:

Core #2:

Time

Time

Single core:

T1

T2

T3

T4

T5

T6

Same start

state

Same

result

Actual

execution

Hypothetical

execution

© 2013 A. Haeberlen, Z. Ives

Other consistency models


Strong

consistency


After update completes, all subsequent accesses will return
the updated value


Weak

consistency


After update completes, accesses do not necessarily return
the updated value; some condition must be safisfied first


Example: Update needs to reach all the replicas of the object


Eventual

consistency


Specific form of weak consistency: If no more updates are
made to an object, then eventually all reads will return the
latest value


Variants: Causal consistency, read
-
your
-
writes, monotonic writes, ...


More of this later in the course!


How do we build systems that achieve this?

23

University of Pennsylvania

© 2013 A. Haeberlen, Z. Ives

Plan for the next two lectures


Parallel programming and its challenges


Parallelization and scalability, Amdahl's law


Synchronization, consistency


Mutual exclusion, locking, issues related to locking


Architectures: SMP, NUMA, Shared
-
nothing


All about the Internet in 30 minutes


Structure; packet switching; some important protocols


Latency, packet loss, bottlenecks, and why they matter


Distributed programming and its challenges


Network partitions and the CAP theorem


Faults, failures, and what we can do about them

24

University of Pennsylvania

NEXT

© 2013 A. Haeberlen, Z. Ives

Mutual exclusion


How can we achieve better consistency?


Key insight: Code has a
critical section
where accesses from
other cores to the same resources will cause problems


Approach: Mutual exclusion


Enforce restriction that only one core (or machine) can
execute the critical section at any given time


What does this mean for scalability?

25

University of Pennsylvania

void transferMoney(customer A, customer B, int amount)

{


showMessage("Transferring "+amount+" to "+B);


int balanceA = getBalance(A);


int balanceB = getBalance(B);


setBalance(B, balanceB + amount);


setBalance(A, balanceA
-

amount);


showMessage("Your new balance: "+(balanceA
-
amount));

}

Critical section

© 2013 A. Haeberlen, Z. Ives

Locking


Idea: Implement locks


If LOCK(X) is called and X is not locked, lock X and continue


If LOCK(X) is called and X is locked,
wait

until X is unlocked


If UNLOCK(X) is called and X is locked, unlock X


How many locks, and where do we put them?


Option #1: One lock around the critical section


Option #2: One lock per variable (A's and B's balance)


Pros and cons? Other options?

26

University of Pennsylvania

void transferMoney(customer A, customer B, int amount)

{


showMessage("Transferring "+amount+" to "+B);


int balanceA = getBalance(A);


int balanceB = getBalance(B);


setBalance(B, balanceB + amount);


setBalance(A, balanceA
-

amount);


showMessage("Your new balance: "+(balanceA
-
amount));

}

Critical section

© 2013 A. Haeberlen, Z. Ives

Locking helps!

27

University of Pennsylvania

1)
LOCK(Bob)

2)
LOCK(Alice)

3) B=Balance(Bob)

4) A=Balance(Alice)

5) SetBalance(Bob,B+100)

6) SetBalance(Alice,A
-
100)

7)
UNLOCK(Alice)

8)
UNLOCK(Bob)

1)
LOCK(Alice)

2)
LOCK(Bob)

3) A=Balance(Alice)

4) B=Balance(Bob)

5) SetBalance(Alice,A+500)

6) SetBalance(Bob,B
-
500)

7)
UNLOCK(Bob)

8)
UNLOCK(Alice)

Alice

Bob

$100

$500

Time

Alice's balance:

Bob's balance:

1

2

$200

$800

5

2

3

4

4

3

$200

$900

$100

$900

$600

$400

$600

$900

5

6

7

1

1

2

8

6

7

8

blocked

© 2013 A. Haeberlen, Z. Ives

Problem

28

University of Pennsylvania

1) LOCK(Bob)

2) LOCK(Alice)

3) B=Balance(Bob)

4) A=Balance(Alice)

5) SetBalance(Bob,B+100)

6) SetBalance(Alice,A
-
100)

7) UNLOCK(Alice)

8) UNLOCK(Bob)

1) LOCK(Alice)

2) LOCK(Bob)

3) A=Balance(Alice)

4) B=Balance(Bob)

5) SetBalance(Alice,A+500)

6) SetBalance(Bob,B
-
500)

7) UNLOCK(Bob)

8) UNLOCK(Alice)

Alice

Bob

$100

$500

Time

1

2

1

2

blocked (waiting for lock on Bob)

blocked (waiting for lock on Alice)


Neither processor can make progress!

: Deadlock

© 2013 A. Haeberlen, Z. Ives

The dining philosophers problem

29

University of Pennsylvania

Philosopher:


repeat


think


pick up left fork


pick up right fork


eat


put down forks

forever

Philosophers

© 2013 A. Haeberlen, Z. Ives

What to do about deadlocks


Many possible solutions, including:


Lock manager:
Hire a waiter and require that philosophers
must ask the waiter before picking up any forks


Consequences for scalability?



Resource hierarchy:
Number forks 1
-
5 and require that each
philosopher pick up the fork with the lower number first


Problem?



Chandy/Misra solution:


Forks can either be dirty or clean; initially all forks are dirty


After the philosopher has eaten, all his forks are dirty


When a philosopher needs a fork he can't get, he asks his neighbor


If a philosopher is asked for a dirty fork, he cleans it and gives it up


If a philosopher is asked for a clean fork, he keeps it

30

University of Pennsylvania

© 2013 A. Haeberlen, Z. Ives

Plan for the next two lectures


Parallel programming and its challenges


Parallelization and scalability, Amdahl's law


Synchronization, consistency


Mutual exclusion, locking, issues related to locking


Architectures: SMP, NUMA, Shared
-
nothing


All about the Internet in 30 minutes


Structure; packet switching; some important protocols


Latency, packet loss, bottlenecks, and why they matter


Distributed programming and its challenges


Network partitions and the CAP theorem


Faults, failures, and what we can do about them

31

University of Pennsylvania

NEXT

© 2013 A. Haeberlen, Z. Ives

Other architectures


Earlier assumptions:


All cores can access the same memory


Access latencies are uniform



Why is this problematic?


Processor speeds in GHz, speed of light is 299 792 458 m/s



Processor can do 1 addition while signal travels 30cm



Putting memory very far away is not a good idea



Let's talk about other ways to organize cores
and memory!


SMP, NUMA, Shared
-
nothing

32

University of Pennsylvania

?

© 2013 A. Haeberlen, Z. Ives

Symmetric Multiprocessing (SMP)


All processors share the same memory


Any CPU can access any byte; latency is always the same


Pros: Simplicity, easy load balancing


Cons: Limited scalability (~10 processors), expensive


33

University of Pennsylvania

Memory bus

Cache

Cache

Cache

Cache


Pros:


Cons:

© 2013 A. Haeberlen, Z. Ives


Memory is local to a specific processor


Each CPU can still access any byte, but accesses to 'local'
memory are considerably faster (2
-
3x)


Pros: Better scalability


Cons: Complicates programming a bit, scalability still limited

Non
-
Uniform Memory Architecture (NUMA)

34

University of Pennsylvania

Consistent cache interconnect

Cache

Cache

Cache

Cache


Pros:


Cons:

© 2013 A. Haeberlen, Z. Ives

Example: Intel Nehalem


Access to remote memory is slower


In this case, 105ns vs 65ns

35

University of Pennsylvania

Source: The Architecture of the Nehalem Microprocessor and

Nehalem
-
EP SMP Platforms, M. Thomadakis, Texas A&M

© 2013 A. Haeberlen, Z. Ives

Shared
-
Nothing


Independent machines connected by network


Each CPU can only access its local memory; if it needs data
from a remote machine, it must send a message there


Pros: Much better scalability


Cons: Nontrivial programming model

36

University of Pennsylvania

Cache

Cache

Cache

Cache

Network


Pros:


Cons:

© 2013 A. Haeberlen, Z. Ives

Plan for the next two lectures


Parallel programming and its challenges


Parallelization and scalability, Amdahl's law


Synchronization, consistency


Mutual exclusion, locking, issues related to locking


Architectures: SMP, NUMA, Shared
-
nothing


All about the Internet in 30 minutes


Structure; packet switching; some important protocols


Latency, packet loss, bottlenecks, and why they matter


Distributed programming and its challenges


Network partitions and the CAP theorem


Faults, failures, and what we can do about them

37

University of Pennsylvania

NEXT

© 2013 A. Haeberlen, Z. Ives

Stay tuned

Next time you will learn about:

Internet basics; faults and failures


Assigned reading: "The Antifragile Organization" by Ariel Tseitlin


38

University of Pennsylvania

http://www.flickr.com/photos/brandoncripps/623631374/sizes/l/in/photostream/