Searching for Errors in

rangaleclickSoftware and s/w Development

Nov 4, 2013 (3 years and 9 months ago)

119 views

Searching for Errors in
Actor
-
based Programs


Steven Lauterburg

University of Illinois at Urbana
-
Champaign



23 March 2011

University of Illinois at Urbana
-
Champaign

2

Concurrency


Growing use of multi
-
core systems and
increased emphasis on parallelism and
concurrency in application development.


Increased focus on concurrency based on
shared memory models.


Notoriously difficult to get right


Common problems include data races,
atomicity violations, deadlocks

23 March 2011

University of Illinois at Urbana
-
Champaign

3

Data Race Example

X = 0

a = X

a = a + 1

X = a

a = X

a = a + 1

X = a

X = ???

Thread 2

Thread 1

23 March 2011

University of Illinois at Urbana
-
Champaign

4

Actor Model


Actor Model offers an alternative based on
message passing


Not just
multicore

systems…


distributed systems


“super” computers


cloud computing


sensor networks


etc.



23 March 2011

University of Illinois at Urbana
-
Champaign

5

Actor Model


Increasing number of actor languages and
libraries:
ActorFoundry,
AmbientTalk
, Axum,
Charm++, E,
Erlang
,
Jetlang
,
Kilim
, Newspeak,
Ptolemy II,
Revactor
,
ThAL
, SALSA,
Scala
,
Singularity, Asynchronous Agents Framework
(Microsoft Visual Studio 2010)…


Real
-
world applications:
Twitter’s message
queuing system, Lift Web Framework,
Facebook’s

chat system, Vendetta’s game
engine…



23 March 2011

University of Illinois at Urbana
-
Champaign

6

What are Actors?

Actor Components…


Behavior


State


Independent thread of
control


Mail queue


Unique name allows
mobility and location
transparency!

23 March 2011

University of Illinois at Urbana
-
Champaign

7

Actor Message Processing

Processing messages…


Send messages


Create new actors


Update local state


Change data


Change behavior

23 March 2011

University of Illinois at Urbana
-
Champaign

8

Actor Model Semantics


Actors do not share state! Actors
communicate
with
each
other
only

using
(asynchronous, non
-
blocking) messages
.


The actor model does
not

guarantee in
-
order delivery of
messages… but does assume that message will
eventually

be delivered


Actors
process only one message at a time… access is
serialized
.


The execution of actor code from receipt of a message until
immediately before the receipt of the next message can be
viewed as
atomic
. This atomic execution of code is called
a “
macro
-
step



Non
-
determinism
is
a result of
message delivery order

not

the ordering of shared memory accesses.

23 March 2011

University of Illinois at Urbana
-
Champaign

9

Actor Model Synchronization

23 March 2011

University of Illinois at Urbana
-
Champaign

10

Actor Model Synchronization


RPC
-
like Messaging



The sender of a message waits for a reply before
processing other messages”


23 March 2011

University of Illinois at Urbana
-
Champaign

11

Actor Model Synchronization


RPC
-
like Messaging



The sender of a message waits for a reply before
processing other messages”


1.
Client sends a request

2.
Client checks incoming messages

3.
If the message is the expected reply, the message is
processed

4.
If the message is not the expected reply, it must be
buffered for future processing, and the client continues to
check incoming messages

23 March 2011

University of Illinois at Urbana
-
Champaign

12

Actor Model Synchronization

Local Synchronization Constraints:



Actors can determine which messages they will process
using by considering the state of the actor and the type of a
message delivered to its mailbox.


For Example:


ActorFoundry uses logical formulas (encoded as Java
methods) to determine if a message should be processed


Scala

and
Erlang

use pattern matching to do the same



23 March 2011

University of Illinois at Urbana
-
Champaign

13

Example


Simple Client and Server

Class Server extends Actor {


int value = 0;


@message void set(int v) {value = v}


@message int get() {return value;}


@message void kill() {destroy("server is done");}

}


Class Client extends Actor {


@message void start() {


ActorName server = createActor(Server.class);


send(server, "set", 1);


int v1 = call(server, "get");


int v2 = call(server, "get");


assert(v1 == v2);


send(server, "kill");


}

}

Based on an actor application
written in Scala from the
ScalaWiki

website.

http:

//scala.sygneca.com/


23 March 2011

University of Illinois at Urbana
-
Champaign

14

Example


Simple Client and Server

Class Server extends Actor {


int value = 0;


@message void set(int v) {value = v}


@message int get() {return value;}


@message void kill() {destroy("server is done");}

}


Class Client extends Actor {


@message void start() {


ActorName server = createActor(Server.class);


send(server, "set", 1);


int v1 = call(server, "get");


int v2 = call(server, "get");


assert(v1 == v2);


send(server, "kill");


}

}

23 March 2011

University of Illinois at Urbana
-
Champaign

15

Expected Message Schedule

Client

Server

set(1)

get

ret(0)

get

ret(1)

kill

v1

=

1

v2

=

1

value

=

1

Expected

Behavior!

------

v1 = v2

Creation and Delivery of


Messages over Time

23 March 2011

University of Illinois at Urbana
-
Champaign

16

Expected Message Schedule

Client

Server

set(1)

get

ret(0)

get

ret(1)

kill

v1

=

1

v2

=

1

value

=

1

Expected

Behavior!

------

v1 = v2

This example also has several other
message schedules
, some of which lead

to
incorrect

behaviors
!

23 March 2011

University of Illinois at Urbana
-
Champaign

17

Example


Simple Client & Server

Class Server extends Actor {


int value = 0;


@message void set(int v) {value = v}


@message int get() {return value;}


@message void kill() {destroy("server is done");}

}


Class Client extends Actor {


@message void start() {


ActorName server = createActor(Server.class);


send(server, "set", 1);


int v1 = call(server, "get");


int v2 = call(server, "get");


assert(v1 == v2);


send(server, "kill");


}

}

23 March 2011

University of Illinois at Urbana
-
Champaign

18

Example


State Space

6 message schedules

26 states

20
different

states

23 March 2011

University of Illinois at Urbana
-
Champaign

19

Example


State Space

6 message schedules

26 states

20
different

states

23 March 2011

University of Illinois at Urbana
-
Champaign

20

Example


State Space

6 message schedules

26 states

20
different

states

An assertion
violation


23 March 2011

University of Illinois at Urbana
-
Champaign

21

Example


Message Schedule

Client

Server

set(1)

get

ret(0)

get

ret(1)

kill

v1

=

0

v2

=

1

value

=

1

Assertion

violation

------

v1 ≠ v2

23 March 2011

University of Illinois at Urbana
-
Champaign

22

Example


Simple Client & Server

Class Server extends Actor {


int

value = 0;


@message void set(
int

v) {value = v}


@message
int

get() {return value;}


@message void kill() {destroy("server is done");}

}


Class Client extends Actor {


@message void start() {


ActorName

server =
createActor
(
Server.class
);


send(server, "set", 1);


int

v1 = call(server, "get");


int

v2 = call(server, "get");


assert(v1 == v2);


send(server, "kill");


}

}

23 March 2011

University of Illinois at Urbana
-
Champaign

23

Example


Simple Client & Server

Class Server extends Actor {


int value = 0;


@message void set(int v) {value = v}


@message int get() {return value;}


@message void kill() {destroy("server is done");}

}


Class Client extends Actor {


@message void start() {


ActorName server = createActor(Server.class);


send(server, "set", 1);


int v1 = call(server, "get");


int v2 = call(server, "get");


assert(v1 == v2);


send(server, "kill");


}

}

23 March 2011

University of Illinois at Urbana
-
Champaign

24

Example


Simple Client & Server

Class Server extends Actor {


int

value = 0;


@message void set(
int

v) {value = v}


@message
int

get() {return value;}


@message void kill() {destroy("server is done");}

}


Class Client extends Actor {


@message void start() {


ActorName

server =
createActor
(
Server.class
);


send(server, "set", 1);


int

v1 = call(server, "get");


int

v2 = call(server, "get");


assert(v1 == v2);


send(server, "kill");


}

}

23 March 2011

University of Illinois at Urbana
-
Champaign

25

Example


State Space

6 message schedules

26 states

20
different

states

An undeliverable
message


23 March 2011

University of Illinois at Urbana
-
Champaign

26

Example


Message Schedule

Client

Server

set(1)

get

ret(0)

get

ret(0)

kill

v1

=

0

v2

=

0

Undeliverable
message

23 March 2011

University of Illinois at Urbana
-
Champaign

27

Example


Simple Client & Server

Class Server extends Actor {


int

value = 0;


@message void set(
int

v) {value = v}


@message
int

get() {return value;}


@message void kill() {destroy("server is done");}

}


Class Client extends Actor {


@message void start() {


ActorName

server =
createActor
(
Server.class
);


send(server, "set", 1);


int

v1 = call(server, "get");


int

v2 = call(server, "get");


assert(v1 == v2);


send(server, "kill");


}

}

23 March 2011

University of Illinois at Urbana
-
Champaign

28

Example 2


Divide & Conquer

Master

(
Map
)

Worker

#1

Worker

#2

Worker

#N

Master

(
Reduce
)



23 March 2011

University of Illinois at Urbana
-
Champaign

29

Example 3


Deadlock

A

B

call(B, "get");

23 March 2011

University of Illinois at Urbana
-
Champaign

30

Example 3


Deadlock

A

B

call(B, "get");

call(A, “query");

23 March 2011

University of Illinois at Urbana
-
Champaign

31

So How Do We Test Actors?


Testing the possible behaviors of an actor
program requires us to
systematically
explore
the different
message delivery
schedules
.



23 March 2011

University of Illinois at Urbana
-
Champaign

32

State
-
Space Exploration


State
-
space exploration (SSE) is the basis for
systematic testing (program model checking).


Identification of errors.


Verification of desired properties.


Systematic exploration of a program’s state space.


Research and tools focused on exploring the
behavior of shared
-
memory programs.


e.g.,
Chess

[
Musuvathi

&
Qadeer
, 2007
]
,
Java
PathFinder

[
Visser

et al.,

2000]

23 March 2011

University of Illinois at Urbana
-
Champaign

33

Systematic Testing for Actors

Challenges:


Existing approaches/tools are focused on
exploration of multi
-
threaded programs using
shared
-
memory.

23 March 2011

University of Illinois at Urbana
-
Champaign

34

Systematic Testing for Actors

Challenges:


Existing approaches/tools are focused on
exploration of multi
-
threaded programs using
shared
-
memory.


Complex multi
-
threaded runtime architectures
underlie user actor code.

23 March 2011

University of Illinois at Urbana
-
Champaign

35

ActorFoundry Architecture

TCP, UDP

Broker, Shell

23 March 2011

University of Illinois at Urbana
-
Champaign

36

Exploring Scala Actor Code


Exploring a simple actor program written in Scala
using Java PathFinder:


Simple “HelloWorld” program

23 March 2011

University of Illinois at Urbana
-
Champaign

37

Exploring Scala Actor Code


Exploring a simple actor program written in Scala
using Java PathFinder:


Simple “HelloWorld” program


Without modification

to the Scala library or
architecture exploration
did not finish after
running for over an hour
.


With some modification

(e.g., reduced thread
pool size, etc.) exploration still took
over 7
minutes
.

23 March 2011

University of Illinois at Urbana
-
Champaign

38

Systematic Testing for Actors

Challenges:


Existing approaches/tools are focused on
exploration of multi
-
threaded programs using
shared
-
memory.


Complex multi
-
threaded runtime architectures
underlie user actor code.


Development of separate tools for different (yet
similar) languages/libraries is costly, requiring
complex algorithms to be reimplemented for each
actor system.

23 March 2011

University of Illinois at Urbana
-
Champaign

39

Basset

Provides an efficient systematic testing

framework for Java
-
based actor programs


Supports exploration of
actor application code

itself,
not

the
actor libraries and runtime architectures.


Allows dynamic, direct exploration of
unmodified

application code.


Leverages actor semantics to allow
efficient

exploration.


Facilitates reuse of capabilities across multiple languages
and libraries, specifically those that target Java
bytecode
,
though not necessarily the Java language.


Currently we support
ActorFoundry

and
Scala

23 March 2011

University of Illinois at Urbana
-
Champaign

40

The Basset Framework

Actor Programs

Adapter

Basset Core

Java PathFinder

23 March 2011

University of Illinois at Urbana
-
Champaign

41

Architecture


Actor Programs

Actor Programs

Adapter

Basset Core

Java PathFinder


Developed normally in the
supported language (e.g.,
Scala, ActorFoundry).


No source code changes

are necessary for Basset to
explore the programs.


Tester needs to code a
simple test driver

23 March 2011

University of Illinois at Urbana
-
Champaign

42

Example Driver

class Driver extends Actor {


public static void main(String[]
args
) {


explore();


}



public static void explore() {



// create
rootset

actors


ActorName

server =


FrameUtil.createActor
(
Server.class
);


ActorName

client =


FrameUtil.createActor
(
Client.class
, server);




// initialize actors


FrameUtil.send
(client, “start”);


}

}

23 March 2011

University of Illinois at Urbana
-
Champaign

43

Architecture


Adapter

Actor
Programs

Adapter

Basset Core

Java PathFinder


Created for each language
supported.


Preserves the library API for
the application code.


A significantly slimmed
down version of the library
that redirects operations to
Basset.

ActorFoundry Adapter

To create the ActorFoundry adapter:



6 existing classes were modified (out of 100+):
Actor,
ActorImpl
,
ActorMsgRequest
,
ActorName
,
BasicActorImpl
, and
DeepCopy



2 new classes were created:
ActorMessage

and
FoundryItemsFactory

23 March 2011

University of Illinois at Urbana
-
Champaign

44

23 March 2011

University of Illinois at Urbana
-
Champaign

45

Architecture


The Basset Core

Actor
Programs

Adapter

Basset Core

Java PathFinder


Manages overall exploration
of possible message
delivery schedules.


Supports state assertions,
undeliverable message and
deadlock detection, etc.


Facilitates the reuse of
capabilities across multiple
languages.


23 March 2011

University of Illinois at Urbana
-
Champaign

46

Architecture


Java PathFinder

Actor
Programs

Adapter

Basset Core

Java PathFinder


Performs actual bytecode
execution.


Facilitates nondeterministic
choices and thread
management


Provides state saving,
restoring and backtracking.

23 March 2011

University of Illinois at Urbana
-
Champaign

47

Results: Framework Efficiency


Exploring actor program using Java PathFinder


Simple “
HelloWorld
” program


Without modification

to
Scala

library or architecture
exploration
did
not

finish after running for over an hour
.


With some modification

(e.g., reduced thread pool size,
etc.) exploration took
over 7 minutes
.

23 March 2011

University of Illinois at Urbana
-
Champaign

48

Results: Framework Efficiency


Exploring actor program using Java PathFinder


Simple “
HelloWorld
” program


Without modification

to
Scala

library or architecture
exploration
did
not

finish after running for over an hour
.


With some modification

(e.g., reduced thread pool size,
etc.) exploration took
over 7 minutes
.


Exploring the same actor program using the
Scala

instantiation of Basset


Exploration took
less than one second
.

23 March 2011

University of Illinois at Urbana
-
Champaign

49

Experiments: State
-
Space Exploration

State Space

Reduction Method

# of

States

# of Msgs

Delivered

# of Paths

Explored

Time

(sec)

MergeSort

None

113066

113065

33264

2316

ShortestPath

None

10000

9999

3614

245

Note: Times are for
Scala

programs using the Basset instantiation for
Scala
.

Can we do better?

Basset Efficiency

23 March 2011

University of Illinois at Urbana
-
Champaign

50

Can we do better?



Systematic testing of an actor program
involves exploring the program’s different
message schedules.


Challenge
:

There are too many schedules!


Observation
:

Many states are the same!



Basset Efficiency

23 March 2011

University of Illinois at Urbana
-
Champaign

51

23 March 2011

University of Illinois at Urbana
-
Champaign

52


Stateful
” Search for Actors

When are two system states equivalent?


z = 1

x = 4

s
0

s
1

s
2

z = 1

a = S
2

n = S
1

m = S
2

z = 1

x = 4

s
0

s
1

s
5

z = 1

a = S
5

n = S
1

m = S
5

(S
0
, S
1
,
m
a
)

(S
0
, S
2
,
m
b
)

(S
0
, S
5
,
m
b
)

(S
0
, S
1
,
m
a
)

Undelivered messages:

Undelivered messages:

23 March 2011

University of Illinois at Urbana
-
Champaign

53


Stateful
” Search for Actors

When are two system states equivalent?


z = 1

x = 4

s
0

s
1

s
2

z = 1

a = S
2

n = S
1

m = S
2

z = 1

x = 4

s
0

s
1

s
5

z = 1

a =
S
5

n = S
1

m =
S
5

(S
0
, S
1
,
m
a
)

(S
0
, S
2
,
m
b
)

(S
0
, S
5
,
m
b
)

(S
0
, S
1
,
m
a
)

Undelivered messages:

Undelivered messages:

23 March 2011

University of Illinois at Urbana
-
Champaign

54

Experiments: State
-
Space Reduction

State Space

Reduction Method

# of

States

# of Msgs

Delivered

# of Paths

Explored

Time

(sec)

None

113066

113065

33264

2316

MergeSort

JPF Comparison

1054

2729

20

73

Basset Comparison

270

719

4

25

None

10000

9999

3614

245

ShortestPath

JPF Comparison

534

1160

69

126

Basset Comparison

230

556

22

115

Note: Times are for
Scala

programs using the Basset instantiation for
Scala
.

Can we do better?



Systematic testing of an actor program
involves exploring the program’s different
message schedules.


Challenge
:

There are too many schedules!


Observation
:

Many schedules are redundant!

Basset Efficiency

23 March 2011

University of Illinois at Urbana
-
Champaign

55

Bank Example

d = “deposit 1000”

f = “finishedDeposit”

w = “withdraw 1000”

23 March 2011

University of Illinois at Urbana
-
Champaign

56

State Space Exploration

E={d,f}

E={f}

E={w}

E={d,w}

E={d}

d

f

w

f

d

w

d

w

d = “deposit 1000”

f = “finishedDeposit”

w = “withdraw 1000”

E: Set of Enabled Messages

23 March 2011

University of Illinois at Urbana
-
Champaign

57

Partial Order Reduction


State space could be exponentially large


But many schedules are redundant


POR is a
state space reduction

technique


Exploits
independence

between concurrent
transitions to prune the search space


Explore a subset of enabled transitions


POR (static) and DPOR (dynamic) have been widely
applied to testing concurrent software


VeriSoft, SPIN, Java PathFinder, dCUTE

23 March 2011

University of Illinois at Urbana
-
Champaign

58

23 March 2011

University of Illinois at Urbana
-
Champaign

59

Experiments: State
-
Space Reduction

State Space

Reduction Method

# of

States

# of Msgs

Delivered

# of Paths

Explored

Time

(sec)

None

113066

113065

33264

2316

JPF Comparison

1054

2729

20

73

MergeSort

Basset Comparison

270

719

4

25

DPOR (a)

39

38

8

9

DPOR (b)

211

210

54

17

None

10000

9999

3614

245

JPF Comparison

534

1160

69

126

ShortestPath

Basset Comparison

230

556

22

115

DPOR (a)

287

286

98

114

DPOR (b)

1690

1689

408

212

Note: Times are for Scala programs using the Basset instantiation for Scala.

23 March 2011

University of Illinois at Urbana
-
Champaign

60

More Basset…

Related publications:


A Framework for State
-
Space Exploration of Java
-
based Actor Programs

Steven
L
auterburg,
Mirco

Dotta
,
Darko

Marinov
,
Gul

Agha

ASE 2009, Auckland, New Zealand, November 2009.


Evaluating Ordering Heuristics for Dynamic POR Techniques

Steven Lauterburg, Rajesh K.
Karmani
,
Darko

Marinov
,
Gul

Agha

FASE 2010,
Paphos
, Cyprus, March 2010.


Mutation Operators for Actor Systems

Vilas
Jagannath
,
Milos

Gligoric
, Steven Lauterburg,
Darko

Marinov
,
Gul

Agha

Mutation 2010, Paris, France, April 2010.


Basset: A Tool for Systematic Testing of Actor Programs

Steven Lauterburg, Rajesh K.
Karmani
,
Darko

Marinov
,
Gul

Agha

FSE Demo 2010, Santa Fe, New Mexico, USA, November 2010.


Basset (
jpf
-
actor) is available for download at:


http://mir.cs.illinois.edu/basset/


http://babelfish.arc.nasa.gov/trac/jpf


Searching for Errors in
Actor
-
based Programs


Thank You