Testing Multithreaded Java Applications for Synchronization Problems

lightnewsSoftware and s/w Development

Nov 18, 2013 (3 years and 8 months ago)

71 views


www.istabg.org

www.vmware.com


www.istabg.org

www.vmware.com





Testing Multithreaded Java
Applications for
Synchronization Problems




Vassil Popovski



www.istabg.org

www.vmware.com

Why multithreading?

A modern CPU



core 1

cache

core N

cache

shared memory


www.istabg.org

www.vmware.com

Why multithreading?



Thread 1

Thread N

Java heap

A modern Java

Application

synchronization

thread 1

thread N

Java heap


www.istabg.org

www.vmware.com


Non
-
determinism



New
types of defects exist such as
deadlocks,
livelocks
and race
conditions


New concept
-
> new problems


www.istabg.org

www.vmware.com

Thread 2

Thread 1

Thread interleaving

time

T1: Block A

T1: Block B

T2: Block 1

T2: Block 2


www.istabg.org

www.vmware.com

Thread 2

Thread 1

Thread interleaving

time

T1: Block A

T1: Block B

T2: Block 1

T2: Block 2


www.istabg.org

www.vmware.com

T1: Block A

T1: Block B

T2: Block 1

T2: Block 2

T1: Block A

T1: Block B

T2: Block 1

T2: Block 2

T1: Block A

T1: Block B

T2: Block 1

T2: Block 2

T1: Block A

T1: Block B

T2: Block 1

T2: Block 2

T1: Block A

T1: Block B

T2: Block 1

T2: Block 2

T1: Block A

T1: Block B

T2: Block 1

T2: Block 2


www.istabg.org

www.vmware.com

Number of different thread interleavings


= # of threads



= # of atomic blocks per thread




www.istabg.org

www.vmware.com

Q: What is the number of all interleavings for 3
threads with 2 blocks each?

QUIZ#1


www.istabg.org

www.vmware.com


safety: nothing
bad

happens



liveness: something good
eventually

happens

What to test for?


www.istabg.org

www.vmware.com

Image source:
http://www.doc.ic.ac.uk/~jnm/book/ppt/ch7.ppt


Example 1: Single Lane Bridge


safety:


no car crash


liveness:


every car eventually get an opportunity to cross the bridge


www.istabg.org

www.vmware.com

Example 2: BoundedBuffer

Producer 1

Producer 2

Producer 3

Producer N

Consumer 1

Consumer 2

Consumer M






safety:


If empty, must not allow Get(); If full, must not allow Put()


liveness (
quiz#2
):


Put() on empty and Get() on full will eventually be allowed


www.istabg.org

www.vmware.com

How to test for synchronization issues?

Why


NASA’s Remote Agent



Therac
-
25


www.istabg.org

www.vmware.com

How to test for synchronization issues?



Load/Stress testing (blackbox, whitebox)


Specific interleavings testing (whitebox)


All interleavings testing (whitebox)



Instrumentation (blackbox)



www.istabg.org

www.vmware.com


A lot of threads and operations to exercise
different interleavings


Load/Stress testing


www.istabg.org

www.vmware.com

Demo!


www.istabg.org

www.vmware.com

Deadlock example

Thread 2

Thread 1

time

synchronized ( A ) {

synchronized ( B ) {

synchronized ( B ) {

synchronized ( A ) {

Deadlock !!!


www.istabg.org

www.vmware.com


Tools


ExecutorService
-

http://download.oracle.com/javase/6/docs/api/java/util/concur
rent/ExecutorService.html#invokeAll(java.util.Collection)


TestNG
-

http://testng.org/doc/documentation
-
main.html#annotations


JUnit
-

http://www.junit.org/apidocs/junit/extensions/ActiveTestSuite.
html


JUnitPerf
-

http://testng.org/doc/documentation
-
main.html#annotations



Custom Threading


GroboUtils
-

http://groboutils.sourceforge.net/testing
-
junit/using_mtt.html

Load/Stress testing


www.istabg.org

www.vmware.com


Deterministic and repeatable tests




Specific interleavings testing


www.istabg.org

www.vmware.com


Based on internal
clock



Tick increases when all threads are blocked



waitForTick(tick)


blocks the thread until tick reaches
certain value



assertTick(tick)


asserts the current tick

MultithreadedTC


www.istabg.org

www.vmware.com

MultithreadedTC

Thread 2

Thread 1

time

put 42

g
et 42

put 17

(blocks)

BoundedBuffer with size = 1

g
et 17


www.istabg.org

www.vmware.com

MultithreadedTC

Thread 2

Thread 1

time

put 42

g
et 42

BoundedBuffer with size = 1

g
et 17

Tick 0

put 17

(blocks)

Tick 1

waitForTick(1)

assertForTick(1)


www.istabg.org

www.vmware.com

Demo!


www.istabg.org

www.vmware.com


Events based



IMUnit.fireEvent(“event1”)



@Schedule(“event1
-
> event2”)



@
Schedule
(“[event1]
-
> event2
”)



IMUnit


www.istabg.org

www.vmware.com

Demo!


www.istabg.org

www.vmware.com


Two threads only


Main and Secondary



By default: For each line of Main thread (T1)
-
>
block and execute fully Secondary thread (T2)




ThreadWeaver

T1: Block A

T1: Block B

T2: Block 1

T2: Block 2

T1: Block A

T1: Block B

T2: Block 1

T2: Block 2

T1: Block A

T1: Block B

T2: Block 1

T2: Block 2


www.istabg.org

www.vmware.com


Two threads only


Main and Secondary



By default: For each line of Main thread (T1)
-
>
block and execute fully Secondary thread (T2)



Powerful Breakpoints




ThreadWeaver


www.istabg.org

www.vmware.com

Demo!


www.istabg.org

www.vmware.com


Tools


MultithreadedTC
-

http://code.google.com/p/multithreadedtc/


Enhanced version for Junit 4
-

http://code.google.com/p/multithreadedtc
-
junit4/


IMUnit
-

http://mir.cs.illinois.edu/imunit/


ThreadWeaver
-

http://code.google.com/p/thread
-
weaver/



Awaitility
-

http://code.google.com/p/awaitility/

Specific interleavings testing


www.istabg.org

www.vmware.com


Exercise all possible interleavings



All interleavings testing


www.istabg.org

www.vmware.com

JavaPathFinder

Traditional testing

OK


Code

Testing

error

Image source:
http://javapathfinder.sourceforge.net/events/JPF
-
workshop
-
050108/tutorial.ppt



www.istabg.org

www.vmware.com

JavaPathFinder

Model Checking with JavaPathFinder

Image source:
http://javapathfinder.sourceforge.net/events/JPF
-
workshop
-
050108/tutorial.ppt


OK

Code

properties

Model Checking

error trace

Line 5: …

Line 12: …



Line 41:…

Line 47:…


www.istabg.org

www.vmware.com



Program testing can be used to show the presence
of bugs, but never to show their absence!



--
Edsger Dijkstra




JavaPathFinder


www.istabg.org

www.vmware.com

Demo!


www.istabg.org

www.vmware.com


Tools


JavaPathFinder (JPF)
-

http://babelfish.arc.nasa.gov/trac/jpf



JavaRaceFinder
-

http://www.cise.ufl.edu/research/JavaRacefinder/Java_Rac
eFinder/JRF_Home.html


All interleavings testing


www.istabg.org

www.vmware.com


Instrument the code to catch problems easier


Instrumentation


www.istabg.org

www.vmware.com

AspectJ

Original code

.class or .jar

AspectJ Compiler

Instrumented
.class or .jar

Aspect Definition

.aj


www.istabg.org

www.vmware.com

Demo!


www.istabg.org

www.vmware.com


Tools


AspectJ
-

www.eclipse.org/aspectj/



RacerAJ
-

http://www.bodden.de/tools/raceraj/



CalFuzzer
-

http://srl.cs.berkeley.edu/~ksen/calfuzzer/


(commercial)
Flashlight / Jsure
-

http://www.surelogic.com/concurrency
-
tools.html

Instrumentation


www.istabg.org

www.vmware.com


CHESS (native DLLs & managed executables)
-

http://research.microsoft.com/en
-
us/projects/chess/


MoonWalker (.NET)
-

http://code.google.com/p/moonwalker/

Other tools


www.istabg.org

www.vmware.com

Recommended books


www.istabg.org

www.vmware.com

Thank you!



www.istabg.org

www.vmware.com

Q & A


www.istabg.org

www.vmware.com


org.ista2011.multithreaded.common


Several implementations of
AccountManager
,
Counter,
BounderBuffer

and
MultipleReadersSingleWriter


org.ista2011.multithreaded.executorservice


Load/stress tests using
ExecutorService


org.ista2011.multithreaded.testng


Load/stress tests using
TestNG


org.ista2011.multithreaded.mtc


MultithreadedTC

tests


org.ista2011.multithreaded.imunit


IMUnit

tests


org.ista2011.multithreaded.threadweaver


ThreadWeaver

tests

Source code packages (1)


www.istabg.org

www.vmware.com


org.ista2011.multithreaded.javapathfinder


JavaPathFinder

examples


ca.mcgill.sable.racer



RacerAJ

source code


org.ista2011.multithreaded.aspectj


Custom aspect that increase the chance of hitting multithreading problem
during testing


org.ista2011.multithreaded.blockbox.*


Rest based
AccountManager

client/test and server (in .server package)

Source code packages (2)


www.istabg.org