Unit, Regression, and Behavioral Testing

jetmorebrisketSoftware and s/w Development

Aug 15, 2012 (5 years and 11 months ago)


Unit, Regression, and
Behavioral Testing

Based On:

Unit Testing with JUnit and CUnit

Beth Kirby



Dec 13, 2002

Jules White


One minute summary

‘Code that isn’t tested doesn’t work’

Why test?

Code that isn’t regression tested suffers
from code rot (breaks eventually)

Why regression test?

A unit testing framework is unit &
regression testing on steroids

What is unit testing?

Testing is often divided into categories such as:

Unit testing

Testing an isolatable ‘unit’ of code, usually a class

e.g. AStack

Integration testing

Testing a module of code (e.g. a package)

Testing AStack + Analyzer, LStack + Analyzer

Application testing

Testing the code as the user would see it (black box)

Testing Delimiters

Writing a unit test


public void testAnalyzer() {

Analyzer analyzer = new Analyzer (….);

analyzer.process (“{“);

analyzer.process (“}”);

//checks to see if the delimiters matched

assertTrue (analyzer.match ());


What is a testing framework?

A test framework provides reusable test
functionality which:

Is easier to use (e.g. don’t have to write the
same code for each class)

Is standardized and reusable

Provides a base for regression tests

Why formalize unit testing?

Unit testing has often been done, but in
an ad hoc manner

E.g. adding a main method to a class,
which runs tests on the class


Code that isn’t tested doesn’t work

‘If code has no automated test case written
for it to prove that it works, it must be
assumed not to work.’ (Hightower and

Why use a testing framework?

Each class must be tested when it is

Each class needs a regression test

Regression tests need to have standard

Thus, we can build the regression test
when building the class and have a better,
more stable product for less work

Regression testing

New code and changes to old code can
affect the rest of the code base

‘Affect’ sometimes means ‘break’

I need to run tests on the old code, to
verify it works

these are regression

Regression testing is required for a
stable, maintainable code base

Regression testing and

‘Refactoring is a technique to restructure
code in a disciplined way.’
(Martin Fowler)

Refactoring is an excellent way to break

Regression testing allows developers to
refactor safely

if the refactored code
passes the test suite, it works

Running automated tests

The real power of regression tests happens
when they are automated

This requires they report pass/fail results in a
standardized way

Can set up jobs to

Clean & check out latest build tree

Run tests

Put results on a web page & send mail (if tests fail)

JUnit & ant have code to do all of this

Some background

eXtreme programming (XP) is a
‘programming methodology that stresses
(among other things) testing and refactoring

The Apache community provides open
source software, including the Jakarta project
(server side java tools, e.g. tomcat (servlet

Ant is a build tool (like make)

Part of the Jakarta project

Is becoming the de facto standard for java projects

Is written in java, uses xml to specify build targets

Stateful Testing

What is Stateful Testing?


public void testAnalyzer() {

Analyzer analyzer = new Analyzer(…..);

analyzer.process (“{“);

assertEqual (“{”, analyzer.getStack().topOfStack());

analyzer.process (“}”);

assertEqual( 0, analyzer.getStack().size() );


What is Wrong with Stateful

What happens if Analyzer is refactored to
use a data structure other than a stack?


public void testAnalyzer() {

Analyzer analyzer = new Analyzer(…..);

analyzer.process (“{“);

assertEqual (“{”, analyzer.getStack().topOfStack());

analyzer.process (“}”);

assertEqual( 0, analyzer.getStack().size() );


What is Wrong with Stateful

Stateful testing does not allow the interface and
implementation to vary.

Stateful testing tightly couples the testing code
to the implementation so that refactoring breaks
the testing code

Stateful testing also prevents testing code to be
reused as new classes are added that
implement the same interface

e.g. a test targeted specifically for AStack cannot be
used by LStack


Behavioral Testing

Tests should check the expected behavior (input
/ output)

Tests should be coded to the specification not
the implementation


public void testAnalyzer() {

Analyzer analyzer = new Analyzer(……);

analyzer.process (“{“);

analyzer.process (“}”);

Boolean expected = new Boolean(true);

expected.equals( analyzer.match () );


Mock Objects

How do you isolate an object for unit
testing when it relies on other objects?


public void testAnalyzer() {

Analyzer analyzer = new Analyzer(……);

analyzer.process (“{“);


assertEqual( true, analyzer.match ()); // A stack is used here


Mock Objects

Mock Objects are used to stand in for the
real objects that the class relies on

e.g Stack is replaced by a MockStack

Mock Objects check to ensure that the
dependent object calls the correct
methods in the correct order

They can guarantee the input and output
to the dependent object and isolate the
source of errors

Mock Objects

public void testAnalyzer() {

Analyzer analyzer = new Analyzer(new



MockObjectFactory {


Stack make_stack () { return new MockStackObject ();}


Mock Objects

MockStackObject {


boolean popCalled = false;

boolean pushCalled = false;

boolean pushCalledFirst = false;

public void push (String str) {

pushCalled = true;

pushCalledFirst = !popCalled;


public String pop() {

popCalled = true;

return “{“;


Mock Objects


public void testAnalyzer() {

MockObjectFactory fact = new MockObjectFactory()

Analyzer analyzer = new Analyzer(fact);

analyzer.process (“{“);

analyzer.process (“}”);

assertTrue( fact.mockStack.pushCalledFirst());

assertTrue( fact.mockStack.popCalled ());

assertTrue( true, analyzer.match ()); // A MockStack is used here


What is Junit?

JUnit is a regression testing framework written
by Erich Gamma and Kent Beck

It is found at

It consists of classes that the developer can
extend to write a test


and related
structure to run and report the tests

Tool integration

Java IDEs



Extensive support for JUnit





Writing a test

Directions can be found in the Junit


Soundbite summary

Create an instance of

Write methods which run your tests, calling them test<Foo>

Call a test runner with your test

When you want to check a value, call an assert method (e.g.
assertTrue()) and pass a condition that is true if the test

Writing a test


public void testAnalyzer() {

Analyzer analyzer = new Analyzer(……);

analyzer.process (“{“);

analyzer.process (“}”);

Boolean expected = new Boolean(true);

expected.equals( analyzer.match () );


Specifying tests


The framework will look in every class
given to it which implements TestCase

It uses reflection to find all methods that
start with ‘test’ (e.g. testMoneyMoneyBag)

It runs each such method as a test case

Test suites

In practice, you will want to run a group of
related tests (e.g. all the tests for a class)

To do so, group your test methods in a
class which extends TestCase

Override the constructor and (if desired)
the setUp and tearDown method

Test Suite example

Package test.mypackage;

import junit.framework.TestCase;

Import mypackage;

Public class MyClassTest extends TestCase {

public MyClassTest(String name) {


public void setUp throws Exception {

// set up for each test


public void tearDown throws Exception {

// releases resources


public void testMyMethod throws Exception {

// Run tests



Test Suite methods


does initialization common to all

It is run each time a test is run (can’t store
state in variables that are set in setUp)


releases resources, e.g.
database connections.

Can also have other helper methods, just
don’t name them test<anything>

Running tests

Need to call a runner run method, e.g


Example main class for running tests

// imports

public class ATestSuite {

public static TestSuite suite () {

TestSuite suite = new TestSuite(“MyPackageTests”);


return suite;


public static void main(String args[]) {




Assert methods

There are a variety of assert methods
available, e.g.

AssertTrue, assertFalse, AssertNull,
assertSame, etc

In general, the syntax is

assert<Foo>(<type> expected, <type>

Assert<Foo>(String message, <type>
expected, <type> testVal)

Project structure

One way to organize tests is to put them
in a parallel directory tree

E.g. MyPackage. …. Has a parallel tree of
tests.MyPackage. ….

Test suites can be collected, so

each class has a test class

each directory has test suite with a runner,
which runs the tests in that directory and
the suites in all subdirectories

Junit addons

Junit has a bunch of add on stuff that has been
written for it (your mileage may vary)


Code generators

make guesses at what tests
are needed

Load testers

Code coverage

Testers for thread safety

‘Helpful classes’ (e.g.

is a
TestSuite that will recursively walk through the
directory structure looking for subclasses of
TestCase and will add them.)

Cactus, HttpUnit, Mock Objects

Cactus (from jakarta) is a simple test
framework for unit testing server
side java
code (Servlets, EJBs, Tag Libs, Filters, ...).

HttpUnit emulates the relevant portions of
browser behavior to allow automated testing
(on sourceforge)

Mock objects emulate objects from other
parts of the application, but have controlled




Web sites








Java Tools for Extreme Programming: Mastering Open
Source Tools Including Ant, JUnit, and Cactus