pptx - Center for Parallel computing at Utah

prettybadelyngeSoftware and s/w Development

Nov 18, 2013 (3 years and 10 months ago)

47 views

Madan Musuvathi


Joint work with

Sebastian Burckhardt, Chris
Dern
, Roy Tan


LineUp
:

Automatic Thread Safety Checking

Motivation


Concurrent applications are common place



Correct synchronization is tricky



Modular programming with “thread
-
safe”
components


Hide synchronization details within each component


Expose a simpler sequential interface


Can be called concurrently without additional
synchronization


Example of a Thread
-
Safe Component


Component = state + set of operations



Clients can concurrently call these operations without
additional synchronization



Concurrent Queue still behaves like a “queue”


Concurrent Queue

State = List of
elements

push(…)

pop(…)



In this talk



A precise formalization of thread safety


Deterministic
Linearizability



An automatic method for checking thread safety



Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t =
q.pop
();

Assert( ?
)

Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t =
q.pop
();

Assert:

q.size
() is 0 or 1

Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t =
q.pop
();

Assert:

q.size
() is 0 or 1

a
nd t is 10 or <fail>

Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t =
q.pop
();

Assert:


t = fail &&
q.size
() = 1 &&
q.peek
() == 10

||

t = 10 &&
q.size
() = 0

Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t =
q.pop
();

q.push
(20);

u

=
q.pop
();

Assert ( ? )

Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t =
q.pop
();

q.push
(20);

u

=
q.pop
();

Assert:

q.size
() == 0 &&

t = 10 || t = 20 &&

u = 10 || t = 20 &&

u != t

Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t1 =
q.pop
();

t
2 =
q.peek
();

q.push
(20);

Assert ( ? )

q.push
(30);

u
1 =
q.peek
();

q.push
(40
);

u2 =
q.pop
();

v1 =
q.pop
();

q.push
(50);

v
2 =
q.peek
();

q.push
(60);

We want to simply say…


q = new
ConcurrentQueue
();

q.push
(10);

t1 =
q.pop
();

t
2 =
q.peek
();

q.push
(20);

Assert:

ConcurrentQueue

behaves

l
ike a queue

q.push
(30);

u
1 =
q.peek
();

q.push
(40
);

u2 =
q.pop
();

v1 =
q.pop
();

q.push
(50);

v
2 =
q.peek
();

q.push
(60);

Linearizability

[
Herlihy

& Wing ‘90]







ConcurrentQueue

behaves like a queue

Concurrent
behaviors of
ConcurrentQueue

are

consistent

with

a sequential
specification
of a queue

Sequential Specification of a Queue


A concise way to specify the behavior when
operations are performed one at a time








And so on …

[]

[10]

push(10)

pop()
-
> <fail>

[9]

push(9)

pop()
-
> 10

pop()
-
> 9

[10, 9]

push(9)

[9, 10]

push(10)

Linearizability

[
Herlihy

& Wing ‘90]







ConcurrentQueue

behaves like a queue

Concurrent
behaviors of
ConcurrentQueue

are

consistent

with

a sequential
specification
of a queue

Every operation appears to occur
atomically at some point between the
call and return

Linearizability


Component is
linearizable

if every operation
appears to occur atomically at some point between
the call and return

Thread 1

Thread 2

Thread 3

push 10

return

push 20

return

pop

return10

pop

return “empty”

A
Linearizability

Violation


History shown below is not
linearizable


Thread 1

push 20

return

pop

r
eturn 20

Thread 2

p
ush 10

return

pop

r
eturn empty

Solution using
Linearizability


q = new
ConcurrentQueue
();

q.push
(10);

t1 =
q.pop
();

t
2 =
q.peek
();

q.push
(20);

Assert:

Every concurrent behavior
(for this test)

is
linearizable

q.push
(30);

u
1 =
q.peek
();

q.push
(40
);

u2 =
q.pop
();

v1 =
q.pop
();

q.push
(50);

v
2 =
q.peek
();

q.push
(60);

Providing Sequential Specification is Hard


Complexity:


API for
java.util.concurrent.BlockingQueue
<T> contains
28 functions



Logic:


Even simple specifications require sophisticated logics


Need to make the accessible to programmers



Insight


We have the code for the component


L
earn the specification automatically

Deterministic
Linearizability







ConcurrentQueue

behaves like a queue

Concurrent
behaviors of
ConcurrentQueue

are

consistent

with

a sequential
specification
of a queue

Some
determinstic

sequential
specification of
a queue

Learn this by observing the code
under sequential runs

The
LineUp

Proposition


Be complete


all reported violations are conclusive refutations



Be automatic


User specifies
component interface (gives list of operation
calls), tool
does the
rest



Be reasonably sound


quantify unsoundness sources (missed bugs)


Demonstrate empirically that we find enough real bugs

Implementation
(part 1: generate Unit Tests)

Random

Testcase

Generator

Collection of

Tests

Thread 1

Thread

2

Thread 3

q.Add
(10)

q.Add
(20)

q.TryTake
()


q.TryTake
()

q.Add
(10),

q.Add
(20),

q.TryTake
(),

q.Clear
()

specify

operations,

# tests,

size of tests

Example of a concurrent unit test:

Example: queue

operations

complete
successfully

Implementation
(part 2: check Each
TEst
)

Thread 1

Thread

2

Thread 3

q.Add
(10)

q.Remove
()

q.Clear
()

q.Remove
()

q.Clear
()

q.Add
(10)

q.Remove
()

q.Remove
()

q.Add
(20)

u
nit test

report

violating

execution

component

under test

(
b
ytecode
)

2
-
Phase
Check

THE Two
-
Phase check (Phase 1)


Enumerate Serial Histories


record all observations (incl.
return values) in file


we are synthesizing a
sequential specification!

Unit Test

CHESS

stateless
model

checker

Thread 1

Thread

2

Add(200)

TryTake
()

Add(400)

TryTake
()

Queue Implementation

(
Bytecode
)

200

4
00

200

4
00

4
00

200

4
00

200

200

4
00

200

4
00

PHASE 1

THE Two
-
Phase check (Phase 2)


Enumerate Concurrent
Histories


check against specification


report violations

Unit Test

CHESS

stateless
model

checker

# of fair executions is finite!
[PLDI 07]

Thread 1

Thread

2

Add(200)

TryTake
()

Add(400)

TryTake
()

200

200

200

4
00

200

4
00

4
00

200

4
00

200

200

4
00

200

4
00

PHASE 2

check

Queue Implementation

(
Bytecode
)

The Two
-
Phase Check: theorems


Completeness


A counterexample refutes deterministic
linearizability

(i.e.
proves that no deterministic sequential specification exists).


Restricted Soundness


If the component is not deterministically
linearizable
, there
exists a unit test that fails.



How sound in practice?

E.g. how good are 100 random 3x3 tests?

Results: Phase 1 / Phase 2

Results

Example: incorrect CAS


volatile
int

state
;



...


int

localstate

= state;


int

newstate

=
f(state
); // compute new value


compare_and_swap
(&state,
localstate
,
newstate
);


...

}


7 Classical

Concurrency

Bugs

Results


C
ancel is not
linearizable

(may delay
past return)


Barrier is not
linearizable

(rendezvous
not
equivalent to
any
interleaved
commit)

Results


nondet
.:

bag may
return any
element


nondet
.:

(weak spec)
Count may
return 0 and
TryTake

may
return false
even if not
empty

R
elated Work / Other Approaches


Two
-
Phase Check


CheckFence

[PLDI 07], less automatic, only SC


Race Detection


The bugs we found do not manifest as data races


Atomicity (Conflict
-
Serializability
) Checking


M
any false alarms (programmers are creative)


Traditional
Linearizability

Verification


less automatic (proofs, specs, commit points)


does not detect incorrect
blocking

Extending Classic
Linearizability

We check deadlocks more tightly
.



[
Herlihy
, Moss]:
D
eadlocked histories are
considered
linearizable

even if operations do not
block in sequential specification.



[Line
-
Up]:

Deadlocked histories qualify as
linearizable

only
if sequential specification allows
all pending operations to block.

Conclusion


Success:


Found tricky concurrency bugs in public production code


Bugs were fixed


Deterministic
linearizability

is a

useful thread
-
safety criterion


Can be checked automatically


In our case, better than race detection or atomicity checking


High
-
value
addition to CHESS


Dramatically automates the process


CHESS source release now available!



Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t =
q.pop
();

Assert( ?
)

Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t =
q.pop
();

Assert:

q.size
() is 0 or 1

Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t =
q.pop
();

Assert:

q.size
() is 0 or 1

a
nd t is 10 or <fail>

Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t =
q.pop
();

Assert:


t = fail &&
q.size
() = 1 &&
q.peek
() == 10

||

t = 10 &&
q.size
() = 0

Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t =
q.pop
();

q.push
(20);

u

=
q.pop
();

Assert ( ? )

Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t =
q.pop
();

q.push
(20);

u

=
q.pop
();

Assert:

q.size
() == 0 &&

t = 10 || t = 20 &&

u = 10 || t = 20 &&

u != t

Let’s Write a Test


q = new
ConcurrentQueue
();

q.push
(10);

t1 =
q.pop
();

t
2 =
q.peek
();

q.push
(20);

Assert ( ? )

q.push
(30);

u
1 =
q.peek
();

q.push
(40
);

u2 =
q.pop
();

v1 =
q.pop
();

q.push
(50);

v
2 =
q.peek
();

q.push
(60);

Wouldn’t it be nice if we could just say…


q = new
ConcurrentQueue
();

q.push
(10);

t1 =
q.pop
();

t
2 =
q.peek
();

q.push
(20);

Assert:

ConcurrentQueue

behaves

l
ike a queue

q.push
(30);

u
1 =
q.peek
();

q.push
(40
);

u2 =
q.pop
();

v1 =
q.pop
();

q.push
(50);

v
2 =
q.peek
();

q.push
(60);

Informally, this is “thread safety”


ConcurrentQueue

behaves like a
queue

A piece of code is thread
-
safe if it
functions correctly during simultaneous
execution by multiple threads.
Formally, this is “
Linearizability
” [
Herlihy

& Wing ‘90]







ConcurrentQueue

behaves like a queue

Concurrent
behaviors of
ConcurrentQueue

are

consistent

with

a sequential
specification
of a queue

Every operation appears to occur
atomically at some point between the
call and return

So, simply check
linearizability


q = new
ConcurrentQueue
();

q.push
(10);

t1 =
q.pop
();

t
2 =
q.peek
();

q.push
(20);

Assert:

Linearizability
wrt


a

given sequential
specification

q.push
(30);

u
1 =
q.peek
();

q.push
(40
);

u2 =
q.pop
();

v1 =
q.pop
();

q.push
(50);

v
2 =
q.peek
();

q.push
(60);

Automatic Thread Safety Checking


q = new
ConcurrentQueue
();

q.push
(10);

t1 =
q.pop
();

t
2 =
q.peek
();

q.push
(20);

Assert:

ConcurrentQueue

is
Thread Safe

q.push
(30);

u
1 =
q.peek
();

q.push
(40
);

u2 =
q.pop
();

v1 =
q.pop
();

q.push
(50);

v
2 =
q.peek
();

q.push
(60);

Automatically learn how “a
queue” behaves

Conclusions


Beware of race conditions when you are designing
your programs


Think of all source of
nondeterminism


Reason about the space of program behaviors



Use tools to explore the space


Cuzz
: Randomized algorithm for large programs


CHESS: systematic algorithm for unit testing


Thread
-
safety as a correctness criterion