Testing and Tools for Testing

nutmegactSoftware and s/w Development

Nov 10, 2012 (4 years and 1 month ago)

225 views

Testing

RHS


SOC

2

Testing


So far, we have not really created tests in
a systematic manner


Testing has been more like ”probing”


Making a thorough test is often very labor
-
intensive


many SW companies employ
more testers than developers…


Once it is done, it is a
very

useful tool to
use during development

RHS


SOC

3

Testing


The ideal test:


When the test is passed, the product is
ready for delivery!


Ideal


but (almost) impossible


Number of test cases is often very large


How does one test a GUI?


Is functional correctness all that matters?

RHS


SOC

4

Testing


In general, test goes hand in hand with
requirement specification


If we cannot specifiy what the software
should do, how can we test it…?


”It must be easy to use the software”


”The software should respond quickly”


”The software must never crash”

RHS


SOC

5

Testing


Requirements are speci
-
fied at many leves


Business level


User level


Functional level


So we must also perform
tests at many levels

RHS


SOC

6

Testing

RHS


SOC

7

Unit testing


A
Unit Test

is aimed at testing a well
-
defined code module, in Java usually a
single class


Unit tests are at the
functional

level


Define test cases in terms of input to class
methods (public and private)


Define the expected output for each case


Run the test


Compare expected and actual output

RHS


SOC

8

Unit testing


Unit testing is


more or less


what we
have done so far, but informally


NetBeans can create a more ”formal” unit
test framework for a project


Relies on a Java framework called
Junit

(see
www.junit.org
)

RHS


SOC

9

Unit testing in NetBeans


Consider our ”classic” BankAccount class,
with three methods:


deposit


withdraw


getbalance


Having created the class, we can now
create a unit test for the class

RHS


SOC

10

Unit testing in NetBeans

RHS


SOC

11

Unit testing in NetBeans

RHS


SOC

12

Unit testing in NetBeans

RHS


SOC

13

Unit testing in NetBeans


There are quite a lot of options to choose
from when generating a test class…


For now, just leave them as
-
is


When pressing Finish, a unit test class is
generated for us, called
BankAccount
-
Test
(just choose ”Junit 4.x”)


The test class is placed under Test
Packages

RHS


SOC

14

Unit testing in NetBeans


The generated test class does look a bit
complex (remember we chose all options)


However, we are
free to edit it
!


Remove whatever you do not need


NetBeans can only generat a ”skeleton”
for the test class


we must complete it

RHS


SOC

15

Unit testing in NetBeans

@BeforeClass

public static void

setUpClass()
throws

Exception

{}


@AfterClass

public static void

tearDownClass()
throws

Exception

{}

RHS


SOC

16

Unit testing in NetBeans


The two methods
setUpClass

and
tearDownclass

allows us to include any
actions needed before and after running
all

the test class methods, respectively


Dependencies to other classes


Database connection


Etc.


Is often not used


then delete it!

RHS


SOC

17

Unit testing in NetBeans


@Before

public void

setUp() {}


@After

public void

tearDown() {}

RHS


SOC

18

Unit testing in NetBeans


The two methods
setUp

and
tearDown

allows us to include any actions needed
before and after running
each

of the test
class methods, respectively


Initialising/resetting variable values


Cleaning up data structures


Etc.


Is often not used


then delete it!

RHS


SOC

19

Unit testing in NetBeans

@Test

public void

testGetBalance()

{


System.out.println("getBalance");


BankAccount instance =
new

BankAccount();


int

expResult = 0;


int

result = instance.getBalance();


assertEquals(expResult, result);


// TODO review the generated test code and


// remove the default call to fail.


fail("The test case is a prototype.");

}

RHS


SOC

20

Unit testing in NetBeans


Notice that a test method does
not

return a
value (true/false)


Instead, so
-
called
assertions

are used
during the test


An assertion can
succeed

or
fail


A failed assertion throws an exception,
and the test case is considered failed

RHS


SOC

21

Unit testing in NetBeans


Examples of assertions:


assertEquals(expectedValue, ActualValue)


assertTrue(condition)


assertFalse(condition)


assertNotNull(object)


assertNull(object)


assertSame(object, object)


assertNotSame(object, object)


fail()
// ALWAYS fails

RHS


SOC

22

Unit testing in NetBeans


If you inspect the generated test code, you
will find that it is not very useful


We must


almost always


implement the
body

of the test methods ourselves


We are
free to add more test methods

than those initially generated


the test
framework will run them automatically

RHS


SOC

23

Unit testing in NetBeans


Once the test methods have been defined
properly, we can run the test


Choose
Run | Test Project
, or just press
Alt + F6


Result of test is displayed in the output
window, with indicative colors

RHS


SOC

24

Unit testing in NetBeans

RHS


SOC

25

Exercises


Create a new Java project called
BankAccount
, and implement it
as described in the presentation


Generate a Unit test class for
BankAccount
, as described in the
presentation


Now change the code in the generated test class, in order to make it
a useful test for the
BankAccount

class. Note that this may involve
not only changing the code in the test methods, but perhaps also
adding code to some of the setUp/tearDown methods


Run the test, and keep changing the source code and/or test code
until all tests are passed (all lights are green

)


how confident do
you now feel about the correctness of the
BankAccount

class?


Review the test


did you have to make some assumptions during
implementation of the test, that were perhaps questionable?

RHS


SOC

26

Unit testing details


Using assertions for testing is the most
common way of performing a unit test
(also see
http://junit.org/junit/javadoc/4.5
)


However, there are other aspects of test


Exceptions


Execution time


These can also be tested with Unit test

RHS


SOC

27

Unit testing details


In some cases, the correct response from
a method could be to throw an exception


We cannot ”capture” this behavior with an
assert statement


Instead, a special ”annotation” is used on
the test method


(expected =
name of exception class
)

RHS


SOC

28

Unit testing details


@Test (expected = IllegalArgumentException.class)

// Test fails if the specified exception is NOT

// thrown

public void

testNegativeDeposit()

{


// Note that test as such always succeeds


theAccount.deposit(
-
100);

}

RHS


SOC

29

Unit testing details


Also, a usual test cannot ”capture” if a
method call takes too long (as defined by
us) to complete


This is also handled using a special anno
-
tation to the test method

(timeout = M)


Method must complete within M milli
-
seconds, otherwise it has failed

RHS


SOC

30

Unit testing details


@Test (timeout = 100)

// Test fails if NOT completed within 100ms

public void

testFibonacci()

{


// Calculate the first 10000 Fibonacci numbers


// Note that test as such always succeeds


theFib.calculateFibonacci(10000);

}

RHS


SOC

31

Unit testing considerations


In the ideal scenario, all units tests should
be completely self
-
contained


Testing of a particular class should not
depend on other classes


Testing of a particular method should not
depend on other methods


Isolates cause of failed tests


RHS


SOC

32

Unit testing considerations

@test

public void

testDeposit()

{


int

b = theAcc.getBalance();


theAcc.deposit(500);


int

a = theAcc.getBalance();


int

diff = a


b;


assertEquals(diff, 500);

}

BankAccount


deposit

withdraw

getBalance

RHS


SOC

33

Unit testing considerations


Suppose now that
testDeposit

fails


Which method in
BankAccount

contains an error…?


Is it
deposit
, or
getBalance
...?

You are
wrong!

No,
you

are
wrong!

RHS


SOC

34

Unit testing considerations

@test

public void testCubeVolume()

{


int volume = theCube.getVolume();


int expVolume = theMathLib.calcCube(theCube.getSide());


assertEquals(volume, expVolume);

}

Cube


getSide

getVolume

MathLibrary


calcCube



RHS


SOC

35

Unit testing considerations


Suppose now that
testCubeVolume

fails


Which class contains
an error…?


Is it
Cube

or
MathLibrary
…?

You are
wrong, again!

No,
you

are
wrong again!

RHS


SOC

36

Unit testing considerations


Testing one functionality often assumes
that some other functionality already works
correctly…


This is quite hard to avoid in practice


A rigorous approach is to useso
-
called
test stubs

RHS


SOC

37

Unit testing considerations


A test stub is a ”simulation” of the behavior
of a real class or method


(Martin Fowler):
Test stubs provide
canned answers

to calls made during the
test, usually not responding at all to
anything outside what's programmed in for
the test


RHS


SOC

38

Unit testing considerations


Making a test stub


Write the test, calling any external methods
that are needed


Substitute all calls to external methods with
calls to stub methods (Proxy…?)


Implement the stubs as returning the desired
answer
without any calculation

RHS


SOC

39

Unit testing considerations

@test

public void

testCubeVolume()

{


int

volume = theCube.getVolume();


int

expVolume = theMathLibStub.calcCube(theCube.getSide());


assertEquals(volume, expVolume);

}

...

// Code in MathLibraryStub

// Only called with input = 8 in test

public

int

calcCube(
int

input)

{


return

512;

}

RHS


SOC

40

Unit testing considerations


Creating a test using stubs consequently
can be done


but is quite labor
-
intensive


More pragmatic approach is to use a
bottom
-
up

approach


Test basic methods first (methods that do not
call other methods)


When basic methods work, test methods that
only use basic methods


And so on (dependency tree)

RHS


SOC

41

Testing


final remarks


We can (almost) never expect to create a
completely covering test



Testing is about
building confidence

in
the correctness of the program


Always a
compromise

between level of
confidence and required effort

RHS


SOC

42

Testing


final remarks

Confidence

Effort

Student

assignment

Commer
-
cial word
processor

Space
Shuttle
software

RHS


SOC

43

Testing


final remarks


Further reading:


JUnit test in NetBeans


http://www.netbeans.org/kb/docs/java/junit
-
intro.html


More about Junit in general


www.junit.org


…and the Net contains a lot of material
about test in general!

RHS


SOC

44

Exercises


Download the NetBeans project
IntegerStack

from the website
(under Classes, week 44)


The project contains a single class
IntegerStack
, which has a
simple interface with five methods


Implement a Unit Test for the
IntegerStack

class, which tests all five
methods. Use a bottom
-
up approach, so you test the simplest
methods first (you do
not

need to use stubs)


Use the test to discover any errors in the implementation of the
IntegerStack

class


if you find any, fix them!


Keep fixing errors until all test cases are green (i.e passed)