TEST TOOL SELECTION CRITERIA

foregoinggowpenΛογισμικό & κατασκευή λογ/κού

4 Νοε 2013 (πριν από 3 χρόνια και 10 μήνες)

68 εμφανίσεις


Copyright Sogeti Nederland B.V.
©
, based in Vianen, the Netherlands.

This work (or any part
thereof) may not be reproduced and/or published (for whatever purpose) in print,
photocopy, microfilm, audio tape, electronically or in any other way whatsoever without prior written
permission from Sogeti Nederland B.V. (Sogeti).

TMap® is a registered tra
demark by

Sogeti Nederland B.V.

TEST TOOL SELECTION
CRITERIA




Test Tool Selectio
n Criteria

Error! Use the Home tab to apply Kop 1 to the text that you want to appear here.


Sogeti Nederland B.V.

cc

1


November 2, 2006

Common criteria


System requirements

The minimum hardware and software requirements to install and use the tool.


Supported environments

Environments, technologies, browsers supported by the tool (no
matter if extra
software (“add
-
in”).


Installation

The way how the test tool will be installed (e.g. straight
-
forward installation
through a wizard).


Ease of use

The degree how intuitive and easy the tool is to use for people completely new
to test automa
tion and for people who have already experience with similar
tools.


Integration with other tools

Integration with other tools like record/playback, test management, defect
management, requirements management, change management and
performance tools


Supp
ort

Kind of support delivered by the tool vendor (e.g. online knowledge base,
online support, online discussion forums)


Manuals

Kind of manuals included in the tool or delivered by the tool vendor (e.g.
installation guide, user guide, technical guide, …)


Tool customization

The degree in which the tool allows customization (e.g. add, delete, hide
buttons/fields, change default folders)


Test Tool Selectio
n Criteria

Error! Use the Home tab to apply Kop 1 to the
text that you want to appear here.


Sogeti Nederland B.V.

cc

2


November 2, 2006

Criteria for tools for planning and controlling the test

Test requirements module

The possibility to manage test requir
ements (create, change, delete test
requirements) in the test management tool.

Relevant items for this module are:



possibility to create, change, delete test requirements



possibility to link test requirements with test plans


Test plan module

The possibili
ty to manage test plans (create, change, delete test plan) in the
test management tool.

Relevant items for this module are:



possibility to create manual/automated test plan



possibility to convert manual to automated test plan



possibility to link test plan
to test requirements


Test execution module

The possibility to manage the execution of test cases (manual/automated) in
the test management tool.

Relevant items for this module are:



possibility to execute test cases (manual/automated) on local/remote
mach
ines



possibility to schedule test execution on time criteria



possibility to schedule test execution on condition criteria


Defect module

The possibility to manage defects in the test management tool.

Relevant items for this module are:



possibility to
create, change, delete defects



possibility to link defects to test plans



possibility to link defects to test requirements



Reporting

The possibility to create reports, graphs for a module (requirement, defect, test
execution).

Relevant items for this
criterion are:



possibility to generate and customize graphs and reports



possibility to create views on results


Web access

The possibility to access a module through a web interface.


Import/export

The possibility to import external files (e.g. xls, txt, h
tml, …) into a module
and/or export the information from a module into an external file.


Test Tool Selectio
n Criteria

Error! Use the Home tab to apply Kop 1 to the text that you want to appear here.


Sogeti Nederland B.V.

cc

3


November 2, 2006

Criteria for automated test execution tool


Record/playback facilities

The different possibilities to record an application (e.g. object, analog recording
(mouse mov
ements).


Scripting language facilities

The language used by the record/playback tool and the possibilities to interact
with this language.

(e.g. expand record script with own statements like procedures, functions,
program and debug facilities, incorporat
ion of object models like Word, Excel,
possibility to call API’s).


Object recognition

The technology/methods used by the record/playback tool to handle the object
recognition.


Object mapping

The methodology/technology used by the record/playback tool to
store the
recorded objects with their hierarchy and the possibilities to work with objects.

E.g. creation of virtual objects, alias mapping of objects, edit/merge object
repositories,


Data driven testing

The possibility to replace hard coded values with
data supplied from an
external source which is normally an Excel file but could be also a text file or
database.


Data base testing

The possibility to perform SQL queries to control, alter and extract information
from any database which supports the ODBC (
Open Database Connectivity)
interface.


Checkpoints

The possibility to check (= capture actual value and compare with expected
value) various objects of the applications.

E.g. check property value of the object, check the value or an area of an
image/bitma
p, check characteristics of a web page, check information in a
table, check text, etc.


Recovery scenario

The possibility to recover a run in case of unexpected events, errors and
application crashes which can disrupt the run session and distort the result
s.


Test results analysis

The possibility to consult, analyse and use the results of a run.

E.g. how are the results represented? Is it possible to drill down to each
individual step? How are failures reported and represented? Is it possible to
export the

results? Is it possible to view the results in a graph?


Test Tool Selectio
n Criteria

Error! Use the Home tab to apply Kop 1 to the text that you want to appear here.


Sogeti Nederland B.V.

cc

4


November 2, 2006

Criteria for performance, load and stress test tool


Protocol support

The possibility to record scripts on different protocols.



Record/playback facilities


The possibilities and features to record and use scripts

(e.g. parameterization of variables, functions, data driven feature, re
-
use of
record/playback scripts, vary the access methods (modem, various line speed),
)



Virtual users


The possibility to ha
ndle virtual users in the performance tool.


Relevant items are:



possibility to scale number of virtual users



possibility to schedule (on time
-
interval) load of virtual users



possibility to use virtual users on process/thread base


Monitoring/measuring

The

possibility to monitor/measure various aspects like:



tiers (web, application, database server)



technology (Linux, NT, Unix, Windows)



network segments



resources



client
-
server response times



performance measurements of data loading


Analyse results

The poss
ibility to analyse the results of performance testing.

Relevant items to check are:



possibility to drill down to identify performance issues



possibility to combine results


Reporting

The possibility to create reports, graphs for a module (requirement,
defect, test
execution).

Relevant items for this criterion are:



possibility to generate and customize graphs and reports



possibility to combine graphs and reports