Software Testing Life Cycle

lumpysteerSoftware and s/w Development

Dec 2, 2013 (3 years and 6 months ago)

61 views

Software Testing Life Cycle

Designed and Compiled by:

Udayakumar Sree

Senior Test Engineer

STLC
-

Definition

The course of software being tested in a well
-
planned way is known as
Software test life cycle.

2

Contract

Signing


Requirement


Analysis


Test


Planning


Test

Development


Test

Execution

Defect

Reporting

Retest

Defects

Product

Delivery

STLC


Stages Involved


Contract Signing:


Process
-

The project is signed with client for testing the software


Documents involved:


SRS


Test Deliverables


Test Metrics etc.

3

STLC


Stages Involved


Requirement Analysis:


Process: Analyzing software for design and implementation methods and
testable aspects are recorded


Documents involved:


Requirement Specification documents


Functional Specification documents


Design Specification documents (use cases, etc)


Use case Documents


Test Trace
-
ability Matrix for identifying Test Coverage

4

STLC


Stages Involved


Test Planning:



Process: To plan, how the testing process should flow


Test Process Flow


Test Scope, Test Environment


Different Test phase and Test Methodologies


Manual and Automation Testing


Defect Mgmt, Configuration Mgmt, Risk Mgmt. Etc


Evaluation & identification


Test, Defect tracking tools


Documents Involved:


Master Test Plan, Test Scenario, SCM

5

STLC


Stages Involved


Test Development


Process:


Test Traceability Matrix and Test coverage


Test Scenarios Identification & Test Case preparation


Test data and Test scripts preparation


Test case reviews and Approval


Base lining under Configuration Management


Documents Involved:


Test Plan, RTM


Test cases

6

STLC


Stages Involved


Test Execution:


Process:


Executing Test cases


Testing Test Scripts


Capture, review and analyze Test Results


Raising the defects and tracking for its closure


Documents Involved:


Test Cases


Test Execution report


Bug report


Requirement traceability matrix


7

STLC


Stages Involved


Defect Reporting


Process:


Defect logging


Assigning defect and fixing


Retesting


Defect closing


Documents involved:


Test report


Bug Report


8

STLC


Stages Involved


Product Delivery


Process:


After the product had undergone several tests, the acceptance test is
done by the user/client i.e. UAT, wherein the use cases were executed
and the product is accepted to go on live


Test Metrics and process Improvements made


Build release


Receiving acceptance


Documents involved


Test summary reports


UAT Test Plan, UAT Test cases

9

Test Plan


What?


Derived from Test Approach, Requirements, Project
Plan, Functional Spec., and Design Spec


Details out project
-
specific Test Approach


Lists general (high level) Test Case areas


Include testing Risk Assessment


Include preliminary Test Schedule


Lists Resource requirements

11

Test Plan


Why?


Identify Risks and Assumptions up front to reduce
surprises later.


Communicate objectives to all team members.


Foundation for Test Spec, Test Cases, and ultimately
the Bugs we find.


Failing to plan = planning to fail.

12

Test Plan


Definition



The test strategy identifies multiple test levels,
which are going to be performed for the project.
Activities at each level must be planned well in
advance and it has to be formally documented. Based
on the individual plans, the individual test levels are
carried out.

13

Test Plan


Consists of…


Unit Testing Tools


Required tool to test at unit level


Priority of Program units


Module
-
wise priority


Naming convention for test cases


Status reporting mechanism


Regression test approach

14

Test Plan


Consists of…


ETVX Criteria


E
ntry means the entry point to that phase.


For example, for unit testing, the coding must be complete and then only
one can start unit testing


T
ask is the activity that is performed


V
alidation is the way in which the progress and correctness and compliance
are verified for that phase


E
x
it tells the completion criteria of that phase, after the validation is done.


For example, the exit criterion for unit testing is all unit test cases must
pass

15

Risk Analysis


A risk is a potential for loss or damage to an
Organization from materialized threats. Risk Analysis
attempts to identify all the risks and then quantify the
severity of the risks


Risk Identification


Software Risks


Business Risks


Testing Risks


Premature Release Risk


Risk Methods

16

Risk Analysis
continued…


Software Risks


Knowledge of the most common risks
associated with Software development, and
the platform you are working on


Business Risks


Most common risks associated with the
business using the Software


Testing Risks


Knowledge of the most common risks
associated with Software Testing for the
platform you are working on, tools being
used, and test methods being applied

17

Risk Analysis
continued…


Premature Release Risk


Ability to determine the risk associated
with releasing unsatisfactory or
untested Software Products


Risk Methods


Strategies and approaches for
identifying risks or problems associated
with implementing and operating
information technology, products and
process; assessing their likelihood, and
initiating strategies to test those risks

18

Test Execution

Software Testing Fundamentals


Testing objectives include


Testing is a process of executing a program with the intent of finding an
error


A good test case is one that has a high probability of finding an as yet
undiscovered error


A successful test is one that uncovers an as yet undiscovered error

20


Testing should systematically uncover different classes of errors
in a minimum amount of time and with a minimum amount of
effort.


A secondary benefit of testing is that it demonstrates that the
software appears to be working as stated in the specifications


When Testing should start?


Testing early in the life cycle reduces the errors.
Test deliverables are associated with every
phase of development. The goal of Software
Tester is to find bugs, find them as early as
possible, and make them sure they are fixed


The number one cause of Software bugs is the
Specification



The next largest source of bugs is the
Design


21

When to Stop Testing?


Some reasons to stop test are:


Deadlines (release deadlines, testing deadlines.)


Test cases completed with certain percentages
passed


Test budget depleted


Coverage of code/functionality/requirements
reaches a specified point


The rate at which Bugs can be found is too small


Beta or Alpha Testing period ends


The risk in the project is under acceptable limit

22


This can be difficult to determine.


Many modern software applications are so complex, and run
in such as interdependent environment, that complete
testing can never be done.

Test Execution


Testing of an application includes:


Unit Testing


Integration testing


System Testing


Acceptance testing

23

These are the functional testing strategies and few other functional, non
-
functional, performance and other testing methods can also be applied on the
software.

Test Execution


Unit testing

The unit test plan is the overall plan to carry out the unit test activities. The
lead tester prepares it and it will be distributed to the individual testers

24


Basic input/output of the units along with their basic functionality will be
tested


input units will be tested for the format, alignment, accuracy and the totals


The UTP will clearly give the rules of what data types are present in the
system, their format and their boundary conditions


Testing the screens, files, database etc., are to be given in proper sequence

Test Execution


Integration testing


The integration test plan is the overall plan for carrying
out the activities in the integration test level

25


This section clearly specifies the kinds of interfaces fall under the scope of
testing internal, external interfaces, with request and response is to be
explained


Two approaches practiced are Top
-
Down and Bottom
-
Up integrations


Given this correctly, the testing activities will lead to the product, slowly
building the product, unit by unit and then integrating them

Test Execution


System testing


The system test plan is the overall plan carrying out the
system test level activities

26


System testing is based on the requirements


All requirements are to be verified in the scope
of system testing


The requirements can be grouped in terms of
the functionality


Based on this, there may be priorities also
among the functional groups


Apart from this what special testing is
performed are also stated here

Test Execution


Non
-
functional testing

Non
-
functional testing includes:


Installation testing


Installation environment, practical obstacles etc.


Compatibility testing


compatibility with other system software


Configuration testing
-

how well the product works with a broad range of
hardware/peripheral equipment configurations as well as on different
operating systems and software


Security testing

-

secure from mistaken/accidental users, hackers, and other
malevolent attackers


Recovery testing
-

how well a system recovers from crashes, hardware
failures, or other catastrophic problems


Usability testing
-

Testing for 'user
-
friendliness'

27

Test Execution


Performance testing


Performance testing includes:


Load

Testing



Testing

with

the

intent

of

determining

how

well

the

product

handles

competition

for

system

resources
.

The

competition

may

come

in

the

form

of

network

traffic,

CPU

utilization

or

memory

allocation

28

Test Execution


Performance testing


Stress testing

-

Testing with the intent of determining how well a product
performs when a load is placed on the system resources that nears and
then exceeds capacity

29

Bug/Defect Management

BUG LIFE CYCLE
-

What is a bug?


A software bug is an error, flaw, mistake, failure, or fault in a computer
program that prevents it from behaving as intended.



31

BUG LIFE CYCLE

In software testing, the term
life cycle refers to the various
stages that a defect/bug
assumes over its life.


32

What is a Bug Life Cycle?

BUG LIFE CYCLE

The different stages involved in a bug life cycle are as follows:


Finding Bugs


Reporting/ Documentation


Fixing


Retesting


Closing


33

Stages involved in Bug Life Cycle

BUG LIFE CYCLE

34

1.
Finding Bugs:


Software Tester finds bug while testing.



It is then logged and assigned to a programmer to be fixed.

2.
Reporting/ Documentation:


In software,

bugs

need to be tracked and managed

to


Communicate bug for reproducibility, resolution, and regression.


Track bug status (open, resolved, closed).


Ensure bug is not forgotten, lost or ignored


Stages explained

BUG LIFE CYCLE

3.
Fixing:


Once the bug is assigned to the developer, he fixes the bug.


Once the programmer fixes the code , he assigns it back to the
tester and the bugs enters the resolved state.

4.
Retesting:


The tester then performs a regression test to confirm that the bug is
indeed fixed
.

5.
Closing:


If the bug is fixed, then the tester closes the bug.


Here the bug then enters its final state, the closed state.

35

Stages explained
Continued…

Different status of a Bug

36

The different status of a bug during its life cycle can be summarized as follows:

New

Deferred

Open

Reopened

Assign

Duplicate

Test

Rejected

Verified

Closed

Description of Various Status of a bug


New:

When the bug is posted for the first time, its state will be “NEW”. This
means that the bug is not yet approved.


Open
:

After a tester has posted a bug, the lead of the tester approves that the
bug is genuine and he changes the state as “OPEN”.



Assign:

Once the lead changes the state as “OPEN”, he assigns the bug to
corresponding developer or developer team. The state of the bug now is
changed to “ASSIGN”.


Test:

Once the developer fixes the bug, he assigns the bug to the testing team
for retesting. Before he releases the software with bug fixed, he changes the
state of bug to “TEST”. It specifies that the bug has been fixed and is released
to testing team.

37

Description of Various Status of a bug


Deferred:

The bug, changed to deferred state means the bug is expected to
be fixed in next releases.


Rejected:

If the developer feels that the bug is not genuine, he rejects the
bug. Then the state of the bug is changed to “REJECTED”.



Duplicate:

If the bug is repeated twice or the two bugs mention the same
concept of the bug, then one bug status is changed to “DUPLICATE”.

38

Description of Various Status of a bug


Verified:

Once the bug is fixed and the status is changed to “TEST”, the
tester tests the bug. If the bug is not present in the software, he approves that
the bug is fixed and changes the status to “VERIFIED”.


Reopened:

If the bug still exists even after the bug is fixed by the developer,
the tester changes the status to “REOPENED”. The bug traverses the life cycle
once again.



Closed:

Once the bug is fixed, it is tested by the tester. If the tester feels that
the bug no longer exists in the software, he changes the status of the bug to
“CLOSED”. This state means that the bug is fixed, tested and approved.


39

Severity of a Bug



It indicates the impact each defect has on
the testing efforts or users and
administrators of the application under test.


This information is used by developers and
management as the basis for assigning
priority of work on defects.

40

Priority Levels of a Bug


Critical

:


An item that prevents further testing of the product or function under test
can be classified as Critical Bug. No workaround is possible for such bugs.


Examples of this include a missing menu option or security permission required
to access a function under test.



Major / High
:



A defect that does not function as expected/designed or cause other
functionality to fail to meet requirements can be classified as Major Bug.
The workaround can be provided for such bugs.



Examples of this include inaccurate calculations; the wrong field being updated,
etc

41

Priority Levels of a Bug


Average / Medium
:



The defects which do not conform to standards and conventions can be
classified as Medium Bugs. Easy workarounds exists to achieve
functionality objectives.



Examples include matching visual and text links which lead to different end
points.


Minor / Low
:



Cosmetic defects which does not affect the functionality of the system can
be classified as Minor Bugs.


42

Various Bug tracking tools

The various bug tracking tools available are:


Quality Center®


from HP


Bugzilla®
-

from Mozilla


Dev Track®


from TechExcel

43

Product Delivery

Product Delivery
-
Test Deliverables


Test Trace
-
ability Matrix


Test Plan


Testing Strategy


Test Cases (for functional testing)


Test Scenarios (for non
-
functional testing)


Test Scripts


Test Data


Test Results


Test Summary Report


Release Notes


Tested Build

45

Product Delivery
-

Test Metrics


Measuring the correctness of the testing process with
measurable is known to be
test metrics.

46

Product Delivery
-

Test Metrics


There

are

several

test

metrics

identified

as

part

of

the

overall

testing

activity

in

order

to

track

and

measure

the

entire

testing

process
.



These

test

metrics

are

collected

at

each

phase

of

the

testing

life

cycle/SDLC,

analyzed

and

appropriate

process

improvements

are

determined

and

implemented
.

The

metrics

should

be

constantly

collected

and

evaluated

as

a

parallel

activity

together

with

testing,

both

for

manual

and

automated

testing

irrespective

of

the

type

of

application

47

Product Delivery
-

Test Metrics
-

Classification

1.
Project Related Metrics



such as


Test Size,


# of Test Cases tested per day

Automated (NTTA)


# of Test Cases tested per day

Manual (NTTM)


# of Test Cases created per day


Manual (TCED)


Total number of review defects (RD)


Total number of testing defects (TD) etc.

48

Product Delivery


Test Metrics


Classification

2.
Process Related Metrics



such as


Schedule Adherence (SA)


Effort Variance (EV)


Schedule Slippage (SS)


Test Cases and Scripts Rework Effort, etc.

3.
Customer related Metrics



such as


Percentage of defects leaked per release (PDLPR)


Percentage of automation per release (PAPR)


Application Stability Index (ASI) etc.

49

Product Delivery


Acceptance testing


UAT


The
client

at their place performs the acceptance testing. It will be very similar
to the system test performed by the Software Development Unit

50


There is no specific clue on the way they will carry out the testing, since the
client

performs this test


It will not differ much from the system testing


This is just one level of testing done by the client for the overall product and
it includes test cases including the unit and integration test level details

51