Software Testing

aniseedsplashΛογισμικό & κατασκευή λογ/κού

15 Αυγ 2012 (πριν από 5 χρόνια και 11 μήνες)

322 εμφανίσεις


Software Testing

Darko Marinov

January 22, 2009

Course Overview

Introduction to software testing

Systematic, organized approaches to testing

Based on models and coverage criteria

Testing is not (only) about finding “bugs”

Improve your testing (and development) skills

Teaching staff

Insructor: Darko <>

TA: Vilas <>

Administrative Info


No exams: no final, no midterm

Five problem sets (5*15%) and a project (25%)

Project: proposal, two reports, hopefully bug reports

Project is on testing a piece of refactoring engine

Undergrads must be registered for 3 hours

For more, you can do an independent study

If interested in research on testing, contact me

Juniors: ask for permission if you didn’t


Did you receive emails on the list

Wiki permissions should be set now

Did you try to find the textbook?

Did you try out Eclipse/NetBeans?

Course will use Java

If you’re not familiar, please review some intro


Potential categories

The “buggiest” bug found in the course

Hardest to find, most important, realistic case study

Most bugs found or tests generated

Not the best measures of testing effort

Example bug from Matt

Armadillo Aerospace X Prize lunar lender crash

This Lecture: Introduction (

Why look for bugs?

What are bugs?

Where they come from?

How to detect them?

Topics that will be covered in the course

Related topics that will not be covered

“Bugs” in IEEE 610.12


Incorrect lines of code


Faults cause incorrect (unobserved) state


Errors cause incorrect (observed) behavior

Not used consistently in literature!

Correctness and Quality

Common (partial) properties

Segfaults, uncaught exceptions

Resource leaks

Data races, deadlocks

Statistics based

Specific properties



Traditional Waterfall Model






Unit Testing


System Testing


Regression Testing

We will look at general
techniques, applicable in
several phases of testing

Phases (1)


Specify what the software should do

Analysis: eliminate/reduce ambiguities,
inconsistencies, and incompleteness


Specify how the software should work

Split software into modules, write specifications

Checking: check conformance to requirements,
using for example
conformance testing

Phases (2)


Specify how the modules work

Unit testing
: test each module in isolation


Specify how the modules interact

Integration testing
: test module interactions

System testing
: test the entire system


Evolve software as requirements change

Regression testing
: test changes

Testing Effort

Reported to be >50% of development cost
[e.g., Beizer 1990]

Microsoft: 75% time spent testing

50% testers who spend all time testing

50% developers who spend half time testing

When to Test

The later a bug is found, the higher the cost

Orders of magnitude increase in later phases

Also the smaller chance of a proper fix

Old saying: test often, test early

New methodology: test
driven development

(write tests even before writing code)

Software is Complex




Solves complex problems

Interacts with other software and hardware

Not continuous

Software Still Buggy

Folklore: 1
10 (residual) faults per 1000
nbnc lines of code (after testing)

Consensus: total correctness impossible

to achieve for complex software

driven finding/elimination of faults

Focus on specific correctness properties

Approaches to Detecting Bugs

Software testing

Model checking

(Static) program analysis

Software Testing

Dynamic approach

Run code for some inputs, check outputs

Checks correctness for some executions

Main questions

suite adequacy (coverage criteria)

input generation

Test oracles

Other Testing Questions






Fault Characterization

Testing is not (only) about finding faults!

Current Status

Testing remains the most widely used
approach to finding bugs

Validation: are we building the right system?

Verification: are we building the system right?

Testing is gaining importance with test
development and increased reliability needs

A lot of research on testing (part of mine too)

This course is not about research

“Schools” of Software Testing

Bret Pettichord described four schools

Analytic (a branch of CS/Mathematics)

Factory (a managed process)

Quality (a branch of quality assurance)

Driven (a branch of development)

This course focuses on artifacts, not process

Do you want a guest speaker from industry?

Topics Related to “Finding Bugs”

How to “eliminate bugs” (localize faults)?


How to “prevent bugs”?

Programming language design

Software development processes

How to “show absence of bugs”?

Theorem proving

Model checking, program analysis

Testing Topics to Cover

Test coverage and adequacy criteria

Graph, logic, input domains, syntax

input generation

Test oracles

based testing

Testing software with structural inputs

Test automation

Testing in your domain of interest?

Summary of the Introduction

Eliminate bugs to save lives and money

“Bugs” may mean faults, errors, failures

Several approaches for detection: software
testing, model checking, static analysis…

Software testing is the most widely used
approach for validation and verification

We will cover systematic approaches to testing,
based on coverage criteria for various models

Testing is not (only) about revealing faults

Next Lecture

Tuesday, January 27, 2pm, 1302 SC

Example Interactive Testing Session

It should be fun

We can learn from mistakes as we go

We will learn some testing terminology


Sign up on Wiki

Try out Eclipse and/or NetBeans