Model-Based Testing - Twin Cities Quality Assurance Association ...

fanaticalpumaMechanics

Nov 5, 2013 (4 years and 5 days ago)

75 views

Yaxiong Lin

TestOptimal
, LLC


Model
-
based Testing

Effective Test Case Design

And Test Automation

Twin Cities Quality Assurance
Association (TCQAA)


September 12, 2013

Agenda


What is Model
-
Based Testing (MBT)


MBT in Action (demo)


Benefits of MBT


MBT
Process


MBT Use Cases

Testptimal LLC

Agenda


What is Model
-
Based Testing (MBT)


MBT in Action (demo)


Benefits of MBT


MBT
Process


MBT Use Cases


Testptimal LLC

Business Impacts of Software Defects

Source: Crosstalk,

the Journal of Defense Software Engineering


U.S. alone: $60 billion per year

(NIST 2002)

Testptimal LLC

Where Defects are Found

Testptimal LLC

Source Unknown

Costs of Fixing Software Defects

Testptimal LLC

The Real Cost of Software Defects, LKP Consulting Group

http://www.lkpgroup.com/Cost%20of%20Software%20Defects.pdf

Software Testing Challenges


Requirement/design
defect impact: 64%


Non
-
developer testing catches 69% of
defects


Costs 100x to fix defect in production


Gain higher confidence


better test coverage


requirement traceability



Testptimal LLC

What is Model
-
Based Testing?


Model
-
Based Testing is the Automation
of Test Design of Black
-
box Tests”



Testptimal LLC

Practical Model
-
base Testing”, Mark
Utting

and Bruno
Legeard

What is Model
-
Based Testing?


Model
-
Based Testing is the Automation
of Test Design of Black
-
box Tests”



Testptimal LLC

Practical Model
-
base Testing”, Mark
Utting

and Bruno
Legeard

Model
-
based testing

is application of
Model based
design

for designing and optionally also executing
artifacts to perform
software testing
. Models can be
used to represent the desired behavior of a System
Under Test (SUT), or to represent testing strategies
and a test
environment.

wikipedia

Why Model?


Visual representation of complex systems


Ask What
-
If to understand cause
-
effect


Important role in the technology
advancement and software development


Software testing is a complex task



Testptimal LLC

MBT Modeling Notations


Finite State Machine (FSM)


State Diagram (UML)


Extended Finite State Machine (EFSM)


Message Sequence Chart (MSC)


Sequence Diagram (UML)


Control Flow Graph (CFG)


Activity Diagram (UML)


Event Flow Diagram


UML Testing Profile


UTP




Testptimal LLC

Testing Vending Machine


Drink for $1


Accept Quarter and HalfDollar coins


Cancel any time for refund


Testptimal LLC

Vending Machine Test Cases


Quarter + Quarter + Quarter + Quarter => Vend


Quarter + Quarter + HalfDollar => Vend


Quarter + HalfDollar + Quarter => Vend


Quarter + HalfDollar + HalfDollar => Vend


HalfDollar + Quarter + Quarter => Vend


HalfDollar + HalfDollar => Vend


HalfDollar + Quarter + HalfDollar => Vend


Quarter + Cancel => Refund




Testptimal LLC

Example Model

Testptimal LLC

Generated Test Sequence

Testptimal LLC

Agenda


What is Model
-
Based Testing (MBT)


MBT in Action (demo)


Benefits of MBT


MBT
Process


MBT Use Cases

Testptimal LLC

Types of MBT


Offline Testing



Online Testing


Testptimal LLC

Demo:


Test Case Generation for Manual Testing
or Offline Testing

Testptimal LLC

Demo:


Login
Test Automation


Online Testing

Testptimal LLC

Demo:


WebStore
Test Automation


Online
Testing

Testptimal LLC

Testing Web App

Testptimal LLC

WebStore

MBT Model

© 2012 Testptimal LLC

Generated Test Sequence

Testptimal LLC

Agenda


What is Model
-
Based Testing (MBT)


MBT in Action (demo)


Benefits of MBT


MBT
Process


MBT Use Cases

Testptimal LLC

Recent MBT User Survey

by Robert Binder, System Verification Associates,2012



MBT reduced escaped bugs by 59%.


MBT reduced testing costs by 17%.


MBT reduced testing duration by 25%.


Testptimal LLC

Benefits of MBT


Catch requirement / design defects


Catch defects earlier


Reduce defects in fields


Improve efficiency (Agility)


handling changes


better communication on defects


Gain higher confidence


better test coverage


requirement traceability



Testptimal LLC

Agenda


What is Model
-
Based Testing (MBT)


MBT in Action (demo)


Benefits of MBT


MBT
Process


MBT Use Cases

Testptimal LLC

MBT Iterative Development Process

Testptimal LLC

Modeling

Test
Generation

Automation

Execution

Analysis

MBT
Models

Requirement
s

User Stories

MBT Process Input/Output

Testptimal LLC

Input

Requirements
User Stories

Test Objectives

AUT/Spec

Environment
Configuration

Execution Result

MBT Step

Modeling

Test Generation

Automation

Execution

Analysis

Output

Behavior Models

Test Strategies
Test Coverage

Automation Scripts

Test Coverage
Defects, Perform. Stat

Model Modification

Model
-
Based Testing Process


Testptimal LLC

http://journal.thedacs.com/issue/52/145

Department of Defense (
DoD
) Information Analysis Center (IAC)

Agenda


What is Model
-
Based Testing (MBT)


MBT in Action (demo)


Benefits of MBT


MBT
Process


MBT Use
Cases
(more demo)

Testptimal LLC

Applications of Model
-
Based Testing


Conformance Testing


Functional Testing


System Integration Testing


Security Vulnerability Testing


Testptimal LLC

MBT Use Cases
(demo)


Finding Temporal Defects


Realistic Load Simulation


Performance


Load/Stress Testing


Session /Scenario Based Testing


Risk Based Testing


Quick bug fix verification


Testptimal LLC

Questions?


MBT Papers:
http
://
testoptimal.com/MbtReferences.html


Email: yaxiong.lin@testoptimal.com


Testptimal LLC