Evaluating Web Software Reliability

architectgroundhogInternet and Web Development

Dec 4, 2013 (3 years and 6 months ago)

98 views

Evaluating Web
Software Reliability

By Zumrut Akcam, Kim Gero, Allen Chestoski,

Javian Li & Rohan Warkad

CSI518


Group 1

Agile Development

The Agile Development Process


What is Agile?



Focus on business value



Collaboration within team; meetings



Communication is key



Sprints, prototypes, feedback



Changing requirements

The Agile Development Process


How does Agile work?



How did our class use Agile?



Three Sprints



“Stand up” meetings at beginning of each class



Retrospective at the end of each sprint

Overview

Definition of Reliability

What is reliability for Web applications?



The reliability for Web applications can be defined as
the probability of failure
-
free Web operation
completions.[2]



Failure is “the event of a system deviating from its
specified behavior like obtaining or delivering
information”.[1]


Failure Sources


Failures are caused from the following sources:



Host, network or browser failures: computer systems,
network or software failures, etc.



Source content failures: missing, inaccessible files,
JavaScript errors, etc.



User errors: improper usage, mistyped URLs.
[2]


Project Goal


Attempt to extend previous work on testing the
reliability of websites.



Gain experience doing a research project


Sprint 1

Sprint 1 Goals




Read relevant research papers




Identify factors that may effect reliability analysis




Gather status logs


Factors That May Effect

Reliability Analysis


Web Workload Characteristics


Byte Count



User Count



Session Count



Error Count

System to Analyze Reliability On


Reliability analysis via status logs



Commercial and non
-
commercial



We will try to record the technologies the websites
employ (Apache, DNN, ISS, PHP, ColdFusion, etc..)


Sprint 2

Sprint 2 Goals



Collect log files for calculation




Automate processes to extra data (user,


session, byte, and error counts)




Convert them into excel format




Log Parser


Sprint 2 Progress


DNN Logs (10 Websites)



PHP Logs

What is DotNetNuke (DNN)



.NET version of Drupal



An open source platform for building websites


and web applications based on Microsoft .Net


technology.



Leading open source ASP.NET web
content


management



Has been downloaded over 6 million times



~100 employees



5
th

Version



Founded 2006

Our DNN Logs



Logs from 10 Websites



Window Server (Same Server)



SQL Server 2008



~1000 unique visitors per day



Logs contain


User count


Limited Error count


Major Problem



The DNN Logs do not contain


Session count


Byte count


Alternative


Generate our own DNN logs

Sprint 3

Server Side


Technologies Used



Windows XP Professional



Microsoft Internet Information Services (IIS)



Microsoft SQL Server 2008



DotNetNuke

(DNN)

Generating Logs


Clients


Web
-
Crawlers


DotNetNuke

Client API



Inducing Errors



Logs contain:


Client IP’s


Byte Counts (Uploaded & Downloaded)


Time
-
Taken


Status Code


Description of Success

Status Codes

Status Code

Meaning

Description

200

OK

The request was received,

understood, and
being processed.

206

Partial
Content

Response

to a request for part of a document.
Just the section requested is returned.

302

Found

Requested

resource was moved to a new
location. Response includes new location.

304

Not
Modified

Response to

a request for a document on the
condition it is newer than the version the client
has.

Description of Error Codes

Status Code

Meaning

Description

400

Bad Request

The server did not understand the request
due to bad syntax

401

Unauthorized

Indicates

that before a resource can be
accessed, the client must be authorized by
the server.

403

Forbidden

The

client cannot access the requested
resource. Possible incorrect username &
password, or permissions do not allow the
action.

404

Not Found

The

requested resource was not found at the
supplied URL.

500

Internal
Server

Error

Indicates that the server encountered
something unexpected and was unable to
complete the request.

Results

Workload Measurement Facts


Server log data consisted of 23
consecutive days of data.


Page Not Found (Error 404) is the most
common type of error in our logs, with
46% of total recorded errors.


Accessing forbidden data (Error 403)
follows with 41%.


72 unique IPs, 33 thousand hits total, and
each hit associated with about 5kb.




Status Code
-
Bytes Graphic

-
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,000
200
302
404
403
304
401
206
500
400
KB Transferred

HTTP Status Codes

Bytes Downloaded
Bytes Uploaded
Error and Success Ratios

HTTP

Status
Codes*

Description

200

OK

206

Partial Content

302

Found

304

Not Modified

400

Bad Request

401

Unauthorized

403

Forbidden

404

Not Found

500

Internal Server Error

200

89%

302

7%

404

2%

403

1%

304

1%

401

0%

206

0%

500

0%

400

0%

STATUS
-
BYTES
-
Server To Client

200

18%

302

68%

404

1%

403

1%

304

12%

401

0%

206

0%

500

0%

400

0%

STATUS
-
BYTES
-
Client to Server

S

U

C

C

E

S

S

E

R

R

O

R

*HTTP Status Codes defined

By W3C
-
World Wide Web

Consortium.

500
-
Internal Server Error Profile

0
2000
4000
6000
8000
10000
12000
14000
16000
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# of Bytes Transferred

# of Times "500" status happened

"500"
-
Internal Server
Error Byte Transferred
O
ver
T
ime

sc-bytes
cs-bytes
time-taken
Number of Status Codes

16764

8190

6901

514

467

89

43

1

1

0
2000
4000
6000
8000
10000
12000
14000
16000
18000
302
200
304
404
403
401
500
206
400
# of Status Codes

HTTP Status Codes

Status Code Counts

Average Time Taken By Different
Status Codes

691

1171

10

11

100

4

8

6

252

0
200
400
600
800
1000
1200
1400
200
206
302
304
400
401
403
404
500
Average Time Taken (ms)

HTTP Status Codes

Average Time Taken for Each Status Code

Identify a metric to analyze
reliability


The Nelson Model




R
: Reliability


f
: Total number of failures


n
: Number of workload units


r
: Failure rate


Mean Time Between Failures (MTBF)




MTBF = n/f


Conclusions


By Nelson Model, the site software reliability is
R = 0.966, or that 96.6% of access to website is
successful.


This model also shows that MTBF=29.6 hits or
the site averages one error for every 29.6 hits.


From the number of errors chart, we can see that
Server errors are very few among the other
errors which shows what the reliability of the
DNN server is.



Conclusions

[1]
J.Tian
,
S.Rudraraju
,
Z.Li
, “Evaluating Web
Software Reliability
Based on Workload and
Failure Data Extracted
from Server Logs”,2004.

Our Work

Previous Work
[1]

23 days data

26 days data

96.6% success

96.2% success

29.6 hits/error

26.6 hits/error

148,579 bytes per
error

273,927 bytes per
error

Resources


[1] Jeff
Tian
,
Sunita

Rudraraju
, and Zhao Li. 2004. Evaluating
Web Software Reliability Based on Workload and Failure Data
Extracted from Server Logs. IEEE Trans.
Softw
. Eng. 30, 11
(November 2004), 754
-
769. DOI=10.1109/TSE.2004.87
http://dx.doi.org/10.1109/TSE.2004.87


[2]
Toan

Huynh and James Miller. 2009. Another viewpoint on
"evaluating web software reliability based on workload and failure
data extracted from server logs". Empirical
Softw
.
Engg
. 14, 4
(August 2009), 371
-
396. DOI=10.1007/s10664
-
008
-
9084
-
6
http://dx.doi.org/10.1007/s10664
-
008
-
9084
-
6


[3] G.
Albeanu
, A.
Averian
, I.
Duda
, “Web Software Reliability
Engineering”,2009.