Composing a Framework to Automate Testing of

hopeacceptableSoftware and s/w Development

Oct 28, 2013 (3 years and 7 months ago)

46 views

Composing a Framework
to Automate Testing of
Operational Web
-
Based
Software

Reham Alhejiali, Chris Cederstrom,
Ranjitha Kashyap

Motivation


Web based application
:

Example: E
-
commerce

Characteristic:

1.Short time to market.

2.Extensively used.

3.High reliability expected.

4.Demand for continuous availability.

5.Large user community.


Goal



-
To address the need for an
automated cost
-

effective testing strategy
for web application.

-

To investigate the
scalability

and
evolving the
tests suites to automatically adapt
as the
application operation profile changes.


Outline

1.General framework for testing web
-
based
application.

2. State of the art.

3.The testing framework .

4. Demonstrates the use of the framework with
case study.

5. conclusion.

``````````````````````````````

Figure 1. General Framework for Testing Web
-
based Applications


State of the Art


User
-
Session
-
based testing


-
Test cases:

-

Created by capturing real user interactions and utilizing the user sessions as
representative test cases of user behavior.

-
Testers could use the collected user sessions
during maintenance to enhance the
original test suite.

-
The user sessions provide the test data that represents usage
not anticipated during
earlier testing stages

and that evolves as operational profiles change.

-
Webking and Rational Robot

Generated test cases may not be adequate.

-
Elbaum et al(2003)

Reduction technique employed is not scalable.




continue:




Functional testing:

-
link, form testers

-
Test the functionality of application.


Program
-
based testing

-
Test cases
: based on the data flow between objects in the model.

Liu et al (2000)

-
WATM

-
Test based on the data flow between objects in the model.

-
ICFG

Di Lucca et al(2002)

-
Developed a web application model and set of tools for the
evaluation and
automation

of testing web application.

-
Test case generation not completely automated.


Testing Framework and Initial
Prototype


Test Case Generation


Using a Web server to log user sessions for test case
generation


Modified the
AccessLog
class of the
Resin web
server

to log user sessions in a specified format:

o
IP address

o
Timestamp

o
Requested URL

o
Cookies

o
Name
-
value of GET/POST requests

o
Referrer URL


Each user session is a collection of user requests in the
form of URL and name
-
value pairs.


Replay Tool & Coverage Analysis


Replaying user sessions are viewed using
wget
for
each entry in the user session log


These requests will emulate user interactions and input


Cookies are used to recognize the user and sessions



Coverage analysis is to evaluate the effectiveness of
the generated test suite


Coverage is calculated by Clover


Clover returns the analysis of statements and methods
used

Coverage Analysis with Clover

Clover: http://www.atlassian.com/software/clover/overview

Oracle and Fault Detection


Oracle generate expected results using a properly
working application


Run the user sessions on the working application and
new version and store results in database


Results of working application is compared to output
from test runs to confirm validity and detect faults





Case Study


A medium sized web application like bookstore was
considered for test


Front end had 11 jsps and classes


LOC:9,748, Methods: 385


Collected 123 User sessions



WEB

Access

Log

User














User Session Log









0.197.37.159 [03/Feb/2004:16:14:05 −0500] GET /apps/bookstore/Default.jsp ] −−cookies=off] −

10.197.37.159 [03/Feb/2004:16:16:27 −0500] GET /apps/bookstore/Registration.jsp ] −−cookies=off −−header
"Cookie:JSESSIONID=a7mpavbwGTf6"] http://dwalin.cis.udel.edu:8080/apps/bookstore/Default.jsp


10.197.37.159 [03/Feb/2004:16:17:22 −0500] GET /apps/bookstore/Registration.jsp




?member_login=bobmason& member_password=14921492&member_password2=14921492&
first_name=bob&last_name=mason& email=bobmason%40udel.edu&address=&phone=& card_type_id=&card_number=&
FormName=Reg&FormAction=insert& member_id=&PK_member_id= ]

−−cookies=off −−header "Cookie:JSESSIONID=a7mpavbwGTf6"] http://dwalin.cis.udel.edu:8080/apps/bookstore/Registration.jsp





















Test case generation and execution


Access

Log

Access Log Parser

10.197.37.159:
GET.apps.bookstore.Default.jsp
GET.apps.bookstore.Registration.jsp
GET.apps.bookstore.Registration.jsp
GET.apps.bookstore.Default.jsp
GET.apps.bookstore.Login.jsp
POST.apps.bookstore.Login.jsp

;






10.82.161.133:
GET.apps.bookstore.Default.jsp
GET.apps.bookstore.images.icon_reg.gif
GET.apps.bookstore.images.icon_home.gif
GET.apps.bookstore.Registration.jsp
GET.apps.bookstore.Registration.jsp
GET.apps.bookstore.Default.jsp

;














Concept
Analysis
Tool

objects[001]:

0.82.161.133

attributes[001]:

GET.apps.bookstore.images.icon_home.gif

GET.apps.bookstore.images.icon_reg.gif









Test Suite
Reducer

Replay Tool

Example Of Using Clover



Coverage Report for Bookstore


Seeded Fault

-

Technique to measure Test Coverage

-

Known bugs are randomly added to the source code

-

During testing, the percentage of known bugs not found indicates the real bugs that remain.

Fault

Description

Example

Logic

error in control or data flow

//String bookDir = bookProp.getProperty(title)

bookDir = null;

Link

error in hyperlink location

String link1 = "registration.jsp"

String link2 = "shoppingCart.jsp"

<a href="<%=link2>">Registration</a>

Datastore

error in code that interacts with
data store

/* Select book_id, name, quantity from store
where book_id=123 */

Select title, name, quantity from store where
book_id=123

Cost and Scalability



















Component


Execution Time


Component Output


Output Space


Test Case Generation


19s


Reduced Test Suite


1MB


Replay Tool
Execution


16m56s


Pages Retrieved


152 MB


Coverage Analysis


1m52s


Coverage Reports/Clover
Database


3.5MB


Test Oracle


14s


Diff Output


0.5M


Conclusion


Discussed the framework for automating the testing
process for web
-
based software


Focuses on usability, costs, and scalability to evolving
the test suite


Used as one of the measures of fault detection for
future evaluation


Code Coverage