Unit, Regression, and Behavioral Testing

jetmorebrisketΛογισμικό & κατασκευή λογ/κού

15 Αυγ 2012 (πριν από 5 χρόνια και 3 μήνες)

374 εμφανίσεις

Unit, Regression, and
Behavioral Testing

Based On:

Unit Testing with JUnit and CUnit
by

Beth Kirby

beth@apl.washington.edu

206
-
685
-
0310

Dec 13, 2002


Jules White

jules@dre.vanderbilt.edu


One minute summary

‘Code that isn’t tested doesn’t work’


Why test?

Code that isn’t regression tested suffers
from code rot (breaks eventually)


Why regression test?

A unit testing framework is unit &
regression testing on steroids


What is unit testing?

Testing is often divided into categories such as:


Unit testing

Testing an isolatable ‘unit’ of code, usually a class

e.g. AStack


Integration testing

Testing a module of code (e.g. a package)

Testing AStack + Analyzer, LStack + Analyzer


Application testing

Testing the code as the user would see it (black box)

Testing Delimiters

Writing a unit test
-

example

public void testAnalyzer() {










Analyzer analyzer = new Analyzer (….);


analyzer.process (“{“);


analyzer.process (“}”);



//checks to see if the delimiters matched


assertTrue (analyzer.match ());

}


What is a testing framework?

A test framework provides reusable test
functionality which:


Is easier to use (e.g. don’t have to write the
same code for each class)


Is standardized and reusable


Provides a base for regression tests

Why formalize unit testing?

Unit testing has often been done, but in
an ad hoc manner


E.g. adding a main method to a class,
which runs tests on the class

Axiom:


Code that isn’t tested doesn’t work


‘If code has no automated test case written
for it to prove that it works, it must be
assumed not to work.’ (Hightower and
Lesiecki)


Why use a testing framework?

Each class must be tested when it is
developed

Each class needs a regression test

Regression tests need to have standard
interfaces

Thus, we can build the regression test
when building the class and have a better,
more stable product for less work

Regression testing

New code and changes to old code can
affect the rest of the code base


‘Affect’ sometimes means ‘break’

I need to run tests on the old code, to
verify it works


these are regression
tests

Regression testing is required for a
stable, maintainable code base


Regression testing and
refactoring

‘Refactoring is a technique to restructure
code in a disciplined way.’
(Martin Fowler)

Refactoring is an excellent way to break
code.

Regression testing allows developers to
refactor safely


if the refactored code
passes the test suite, it works

Running automated tests

The real power of regression tests happens
when they are automated


This requires they report pass/fail results in a
standardized way

Can set up jobs to


Clean & check out latest build tree


Run tests


Put results on a web page & send mail (if tests fail)

JUnit & ant have code to do all of this

Some background

eXtreme programming (XP) is a
‘programming methodology that stresses
(among other things) testing and refactoring

The Apache community provides open
-
source software, including the Jakarta project
(server side java tools, e.g. tomcat (servlet
engine))

Ant is a build tool (like make)


Part of the Jakarta project


Is becoming the de facto standard for java projects


Is written in java, uses xml to specify build targets


Stateful Testing

What is Stateful Testing?

Example:


public void testAnalyzer() {


Analyzer analyzer = new Analyzer(…..);


analyzer.process (“{“);


assertEqual (“{”, analyzer.getStack().topOfStack());


analyzer.process (“}”);


assertEqual( 0, analyzer.getStack().size() );

}


What is Wrong with Stateful
Testing?

What happens if Analyzer is refactored to
use a data structure other than a stack?

Example:

public void testAnalyzer() {


Analyzer analyzer = new Analyzer(…..);


analyzer.process (“{“);


assertEqual (“{”, analyzer.getStack().topOfStack());


analyzer.process (“}”);


assertEqual( 0, analyzer.getStack().size() );

}

What is Wrong with Stateful
Testing?

Stateful testing does not allow the interface and
implementation to vary.

Stateful testing tightly couples the testing code
to the implementation so that refactoring breaks
the testing code

Stateful testing also prevents testing code to be
reused as new classes are added that
implement the same interface


e.g. a test targeted specifically for AStack cannot be
used by LStack

}

Behavioral Testing

Tests should check the expected behavior (input
/ output)

Tests should be coded to the specification not
the implementation

Example:


public void testAnalyzer() {


Analyzer analyzer = new Analyzer(……);


analyzer.process (“{“);


analyzer.process (“}”);


Boolean expected = new Boolean(true);


expected.equals( analyzer.match () );

}

Mock Objects

How do you isolate an object for unit
testing when it relies on other objects?

Example:

public void testAnalyzer() {


Analyzer analyzer = new Analyzer(……);


analyzer.process (“{“);


analyzer.process(“}”);


assertEqual( true, analyzer.match ()); // A stack is used here

}

Mock Objects

Mock Objects are used to stand in for the
real objects that the class relies on


e.g Stack is replaced by a MockStack

Mock Objects check to ensure that the
dependent object calls the correct
methods in the correct order

They can guarantee the input and output
to the dependent object and isolate the
source of errors



Mock Objects

public void testAnalyzer() {


Analyzer analyzer = new Analyzer(new
MockObjectFactory());


…..

}


MockObjectFactory {

….

Stack make_stack () { return new MockStackObject ();}

}


Mock Objects

MockStackObject {

….

boolean popCalled = false;

boolean pushCalled = false;

boolean pushCalledFirst = false;


public void push (String str) {


pushCalled = true;


pushCalledFirst = !popCalled;

}

public String pop() {


popCalled = true;


return “{“;

}


Mock Objects

Example:

public void testAnalyzer() {


MockObjectFactory fact = new MockObjectFactory()


Analyzer analyzer = new Analyzer(fact);


analyzer.process (“{“);


analyzer.process (“}”);


assertTrue( fact.mockStack.pushCalledFirst());


assertTrue( fact.mockStack.popCalled ());


assertTrue( true, analyzer.match ()); // A MockStack is used here

}

What is Junit?

JUnit is a regression testing framework written
by Erich Gamma and Kent Beck

It is found at
www.junit.org

It consists of classes that the developer can
extend to write a test


notably
junit.framework.TestCase
-

and related
structure to run and report the tests

Tool integration

Java IDEs



Eclipse


http://www.eclipse.org

Extensive support for JUnit



Netbeans
-

http://www.netbeans.org/index.html


Idea
-

http://www.intellij.com/idea/


Writing a test

Directions can be found in the Junit
cookbook
-

http://junit.sourceforge.net/doc/cookbook/cookbook.ht
m

Soundbite summary

1.
Create an instance of
TestCase
:

2.
Write methods which run your tests, calling them test<Foo>

3.
Call a test runner with your test

4.
When you want to check a value, call an assert method (e.g.
assertTrue()) and pass a condition that is true if the test
succeeds


Writing a test
-

example

public void testAnalyzer() {










Analyzer analyzer = new Analyzer(……);


analyzer.process (“{“);


analyzer.process (“}”);


Boolean expected = new Boolean(true);


expected.equals( analyzer.match () );


}


Specifying tests

JUnit


The framework will look in every class
given to it which implements TestCase


It uses reflection to find all methods that
start with ‘test’ (e.g. testMoneyMoneyBag)


It runs each such method as a test case

Test suites

In practice, you will want to run a group of
related tests (e.g. all the tests for a class)

To do so, group your test methods in a
class which extends TestCase

Override the constructor and (if desired)
the setUp and tearDown method

Test Suite example

Package test.mypackage;

import junit.framework.TestCase;

Import mypackage;


Public class MyClassTest extends TestCase {


public MyClassTest(String name) {


super(name);


public void setUp throws Exception {


// set up for each test


}


public void tearDown throws Exception {


// releases resources


}


public void testMyMethod throws Exception {


// Run tests


}

}



Test Suite methods

setUp


does initialization common to all
tests


It is run each time a test is run (can’t store
state in variables that are set in setUp)

tearDown


releases resources, e.g.
database connections.

Can also have other helper methods, just
don’t name them test<anything>

Running tests

Need to call a runner run method, e.g


junit.textui.TestRunner.run(suite());

Example main class for running tests

// imports

public class ATestSuite {



public static TestSuite suite () {


TestSuite suite = new TestSuite(“MyPackageTests”);


suite.addTestSuite(“MyClassTests.class”);


return suite;


}


public static void main(String args[]) {


junit.textui.TestRunner.run(suite());


}

}



Assert methods

There are a variety of assert methods
available, e.g.


AssertTrue, assertFalse, AssertNull,
assertSame, etc

In general, the syntax is


assert<Foo>(<type> expected, <type>
testVal)


Assert<Foo>(String message, <type>
expected, <type> testVal)


Project structure

One way to organize tests is to put them
in a parallel directory tree


E.g. MyPackage. …. Has a parallel tree of
tests.MyPackage. ….

Test suites can be collected, so


each class has a test class


each directory has test suite with a runner,
which runs the tests in that directory and
the suites in all subdirectories

Junit addons

Junit has a bunch of add on stuff that has been
written for it (your mileage may vary)

Examples


Code generators


make guesses at what tests
are needed


Load testers


Code coverage


Testers for thread safety


‘Helpful classes’ (e.g.
RecursiveTestSuite

is a
TestSuite that will recursively walk through the
directory structure looking for subclasses of
TestCase and will add them.)


Cactus, HttpUnit, Mock Objects

Cactus (from jakarta) is a simple test
framework for unit testing server
-
side java
code (Servlets, EJBs, Tag Libs, Filters, ...).

HttpUnit emulates the relevant portions of
browser behavior to allow automated testing
(on sourceforge)

Mock objects emulate objects from other
parts of the application, but have controlled
behaviour



Resources

Videos


http://video.google.com/videosearch?q=techtalks&pa
ge=1&lv=0&so=1

Web sites


Junit
-

http://www.junit.org


CppUnit
-

http://cppunit.sourceforge.net/


Ant
-

http://jakarta.apache.org/ant/index.html

Book


Java Tools for Extreme Programming: Mastering Open
Source Tools Including Ant, JUnit, and Cactus