Testing

quaggahooliganInternet and Web Development

Feb 5, 2013 (4 years and 7 months ago)

127 views

1

CS428/9:

Software Engineering II

Darko Marinov


(slides from Ralph Johnson)

428
-
11

2

Administrative info


HW 1 and Demo 1 grades sent today


HW 2 due next Tuesday, Feb 27


Write several kinds of tests in subgroups of 2
-
3


Will be graded by Thursday, Mar 1


ACP revision 2 due on Thursday, Mar 1


If you failed revision 1, meet with Tracy


Mar 1: Guest lecture on testing video games

428
-
11

3

Testing: Covered


Previously


Classic Testing Mistakes

(by Brian Marick)


Kinds of testing (Hamlet & Maybee, ch. 6 & 19)


Black
-

and white
-
box techniques (H & M, ch. 20 & 21)


JUnit


The Impossibility of Complete Testing

(by Cem Kaner)


428
-
11

4

Testing: To cover


Today


Testing non
-
functional properties, OO testing


Acceptance/system tests


Test doubles


Tuesday, Feb 27


Assertions and debugging


Thursday, Mar 1: Guest lecture


Travis Tholen from Volition/THQ on

Quality Assurance: General Software Testing

428
-
11

5

Some techniques for writing tests


Black
-
box (from specifications)


Equivalence partitioning


Boundary value analysis


White
-
box (from code)


Structural code coverage


Every statement, branch, condition, path…

428
-
11

6

More kinds of testing


Performance testing


Load testing


Configuration testing



OO testing

428
-
11

7

Performance testing


Testing requires specification


Performance tests must measure time of
operations


Test fails if it takes too long


Keep database of past results, and track
changes in performance over time

428
-
11

8

Load testing


Some problems only show up under heavy
load


Includes performance, but more than
performance


Often requires computers to generate load

428
-
11

9

Configuration testing


Some systems have many configurations


Do they all work?


How can you tell?

428
-
11

10

Example: Hardware and OS


Amount of memory: 512MB, 1GB, 2GB


Processors: single
-
core, dual
-
core, quad
-
core


OS: Win XP, Win Vista, Linux, Mac OS X



Each configuration is a triple of some amount
of memory, some processor, and some OS


Widely
-
used approach: cover all pairs

428
-
11

11

Planning testing


Testing must be planned


Make sure important testing happens first


Make sure you have enough people


Make sure it starts soon enough

428
-
11

12

OO testing


Structure of tests should match structure of
system


OO systems have extra structure


Units of OO systems are classes


Subclass of TestCase for each class being
tested


Often one unit is a group of classes

428
-
11

13

Testing and reuse

It might be argued that, as OOA and OOD
mature, greater reuse of design patterns will
mitigate the need for heavy testing of OO
systems. Exactly the opposite is true.

428
-
11

14

Testing and reuse


Reuse reduces testing


Use discovers bugs. Reused components have
less bugs.


Reuse does not reduce testing


Inherited methods can behave differently


Must test inherited methods

public class Foo {


private int x;


public String asString() {



… writeOn(aStream); …}


public abstract writeOn(Stream){}

}

public class Fee extends Foo {


public writeOn(Stream aStream) {…}

}

428
-
11

16

Continuous Improvement


Use testing to learn how to improve
development


Why did this bug happen?


Where else might we have made this
mistake?


What could we have done to prevent it?


Change reviews


Change tests we write

428
-
11

17

Summary of programmer’s tests


Lots of techniques for writing tests


Testing shows the presence of bugs, it never
proves their absence


All testing is a guess


Manage testing


Make it effective


Make it cost
-
effective


Use what you learn to improve your process

428
-
11

18

Customer’s tests


Acceptance tests / System tests



Customer alternatives


Hire programmers to automate tests


Manual testing


Testing tool like SilkTest and WinRunner


FIT: Framework for Integrated Test

428
-
11

19

Customer tests


Most customers are not programmers


Customers don’t care about the structure of
the program


Need to write lots of similar tests and know
which ones work and which ones don’t

428
-
11

20

FIT


Framework for Integrated Test:
http://fit.c2.com


Created by Ward Cunningham


Use HTML to write tests and view their outputs


Tests are in tables


Anything outside of a table is a comment


Output is same tables as input, but showing actual
results and success/failure


Customers write tables and run programs

428
-
11

25

FIT Fixtures


A fixture is a Java class


It reads tests from the table and executes them


Fixture


ColumnFixture


ArithmeticColumnFixture


RowFixture


ActionFixture


PrimitiveFixture

public class ArithmeticFixture extends PrimitiveFixture {


int x=0;


int y=0;


public void doRows(Parse rows) {


super.doRows(rows.more); // skip column heads


}


public void doCell(Parse cell, int column) {


switch (column) {


case 0: x = (int)parseLong(cell); break;


case 1: y = (int)parseLong(cell); break;


case 2: check(cell, x+y); break;


case 3: check(cell, x
-
y); break;


case 4: check(cell, x*y); break;


case 5: check(cell, x/y); break;


default: ignore(cell); break;


}


}

}

public class ArithmeticColumnFixture extends ColumnFixture {


public int x;


public int y;


public int plus () {


return x + y;


}


public int minus() {


return x
-

y;


}


public int times () {


return x * y;


}


public int divide () {


return x / y;


}


public float floating () {


return (float)x / (float)y;


}


public ScientificDouble sin () {


return new ScientificDouble(Math.sin(Math.toRadians(x)));


}


public ScientificDouble cos () {


return new ScientificDouble(Math.cos(Math.toRadians(x)));


}

}

428
-
11

28

Using FIT


HTML can be


In a file


On a web server (especially a Wiki)



Run from regular Java program


FileRunner


WikiRunner


Run from cgi script

428
-
11

29

Using FIT


Variety of languages


Java


.NET


Perl


Python


Lisp


Ruby


C++

428
-
11

30

How to test with FIT


Customer writes tables with Excel to
specify the system


Programmer figures out how to write a
fixture for them


(Negotiation)


Customer exports tables to HTML, and
thereafter maintains them with either Word
or Excel

428
-
11

31

Test double


How can you test one class without having
the whole system working?


Easy if class doesn’t depend on anything


Unimportant if class doesn’t do much except
combine other objects


Use mock object / test double


http://xunitpatterns.com/Using%20Test%20Doubles.html

428
-
11

32

System under test


What is your “unit”?


A method?


A class?


A set of classes?


Write test objects for everything else
needed to make the test run

428
-
11

33

Test double


Test double


An object written by a tester that acts like the
real object


Fake object
-

is simpler / faster


Test stub
-

follows test script


Recording test stub
-

records interaction so that test
can check it


Mock object


knows entire expected behavior


428
-
11

34

Test doubles


Often write test doubles for code that
someone else is writing


Don’t want to wait for it to be done


Don’t want to test their code


Don’t trust them


Test doubles start small and grow


Refactor test doubles

428
-
11

35

Next time


Debugging


Assertions


Oracles