Selenium: A framework for testing web applications

butterbeanspipeΛογισμικό & κατασκευή λογ/κού

14 Ιουλ 2012 (πριν από 4 χρόνια και 11 μήνες)

333 εμφανίσεις

9/12/07
RJUG: Selenium
1
Rochester Java Users Group
Selenium:
A framework for testing web
applications
Kurt A.
Kluever
(
kurt@kloover
.com)
Department of Computer Science
Rochester Institute of Technology
9/12/07
RJUG: Selenium
2
About me

Kurt Alfred
Kluever
(
kurt@kloover
.com)

BS/MS

student @ Rochester Institute of Technology

Fifth year at RIT;

Graduate in May 2008 (hopefully)

Excellus BlueCross BlueShield
(Mar 2006 - Mar 2007)

Worked with Tom
Sausner
and Rob Phipps

Had lunch at Dino BBQ

Google, Inc. (Jun 2007 - Aug 2007)

Worked with Jason

Huggins and Patrick Copeland

Had lunch with Larry

Page

Read more

@
kloover
.com

Interests: Computer Vision, Computer Security
9/12/07
RJUG: Selenium
3
Why test?

Verify correctness

program behaves as designed

Verify completeness

all features are present

Verify

security

Confidentially

The info is known only

to those who should know it

Integrity

The info is untainted (not necessarily accurate)

Availability

The info is accessible when necessary

Also

verify capability, reliability, efficiency, portability,
maintainability, compatibility, and usability (Source: ISO
9126)
9/12/07
RJUG: Selenium
4
Testing

Pyramid

Metaphor conceived by Jason Huggins

Relates software testing to the
now defunct food pyramid
So Selenium = Fats/oils/sweets

Recipe for good software health

Generous servings of unit tests

JUnit
on

all methods

Ample servings of functional/acceptance tests

Black box testing of product

A

dab of GUI tests

Selenium fits in here!
<begin argument about whether GUI tests should be placed into functional tests category>
9/12/07
RJUG: Selenium
5
Testing Pyramid Sketch
9/12/07
RJUG: Selenium
6
Testing Web Apps

For

functional testing of web apps, we have four
options:
1.
Outsource manual testing

Very bad / waste of time + money
2.
Mimic the client

Bad for JS heavy apps (read Web 2.0)
3.
Record/Replay

Better
4.
Control the user interface

Best (maybe a bit of a

biased opinion)
9/12/07
RJUG: Selenium
7
Outsource

Manual Testing

Outsource manual testing

Hire slaves interns to manually walk through a set of test cases

Advantages

Test cases performed by real people using real browsers


Adaptable

(Humans can think, machines can

t)

Disadvantages

Test cases are performed on ONLY one browser

Horribly boring and slow

Inaccurate

Expensive (well, maybe

)

Impossible to setup continuous build/regression tests

Example:
Excellus
BCBS

Write up a set of manual test cases

Distribute the test cases to interns before a big release

The test cases won

t actually get done

Interns will tell you they are done
9/12/07
RJUG: Selenium
8
Mimic the

Client

Mimic the client

Can mimic a browser by reading/responding to HTTP headers

Advantages:

Can be run headless (w/o graphics)

Disadvantages:

Read: no actual browser!

Nearly impossible to interpret JavaScript/DOM

Very difficult

for even a relatively simple web app

Example:

HtmlUnit

Supports HTTP/HTTPS, POST, GET, DOM, JS (basic), AJAX

Not actually a browser; simulation

Used in
Canoo WebTest
(clean

XML/Ant testing) and
JWebUnit
(wrapper for
HtmlUnit
and soon to be more drivers)
9/12/07
RJUG: Selenium
9
Record/Replay

Record and replay

Tool records the users interaction with the web app

To run the test, just play back the test

Expressed in code (Java or some other scripting language)

Advantages:

Test is performed once manually (no programming)

Simple to create tests

Operates in


real

browsers

Disadvantages:

Maintenance

is nonexistent; must start from scratch

*VERY* fragile (must run on the same system)

Example: IBM Rational Functional Tester

Commercial product

Scripts in Java or .NET

Browser is controlled by a
plugin
or ActiveX control
9/12/07
RJUG: Selenium
10
Control the User Interface

Control the user interface

Use JavaScript to drive the tests inside of the actual browser

Advantages:

Uses a

real

browser (IE, FF, Safari, Opera)

Control based on UI elements

Disadvantages:

Slower than mimicking since it can

t run headless

Limitations of JS (popup handling,

downloads, uploads, etc)

Issues with XSS and security

Best solution since it

s not as

fragile as record/replay and it

s
closer to a real environment than

mimicking

Example: Selenium

Open source product

Tests in

many different languages
9/12/07
RJUG: Selenium
11
Origins of Selenium

ThoughtWorks
needed a tool to test an expense tracking
web application

Developers were familiar with the testing tool Mercury
(now owned by HP)

Began development on their own testing tool, called it
Selenium

Selenium is


A chemical element (Se)

Key mineral that protects the body from Mercury toxicity

If someone gets Mercury poisoning, they are treated with
Selenium compounds

Note: I am not a doctor, nor do I play one on

television

so
don

t sue me if you

get Mercury toxicity and try to save yourself
by
chowing
down on a

brick of Selenium
9/12/07
RJUG: Selenium
12
About Selenium

Selenium is



a test tool that allows you to write automated web application
UI tests in any programming language against any HTTP website
using any mainstream JavaScript-enabled browser.



a free and open source multi-browser, multi-platform web
application testing framework that will make your users smile.


Clarifications:

Java, .NET,
Perl
, PHP, Python, Ruby, JavaScript, and HTML
(
Selenese
Tables)

IE 6+,
Firefox
0.8+, Opera 8.5+, Safari 1.3+,
Seamonkey
1.0+,
Camino 1.0+,
Konqueror
3.5+

Windows, Mac OS X, Linux,

etc (OS independent)

Developed and maintained by
OpenQA
(
openqa
.org)

Open sourced under Apache 2.0 license
9/12/07
RJUG: Selenium
13
About
OpenQA

First and only open source organization specifically for
QA tools

Most projects Apache licensed

Who?

Jason Huggins - Author of Selenium Core

Full time @ Google since March 2007

Paul
Hammat
- Author of Selenium RC

Senior Technical Architect @
ThoughtWorks

Patrick
Lightbody

Founder of
OpenQA

Co-creator of Struts 2.0 (
WebWork
)

Co-creator of Selenium Remote Control (RC)
9/12/07
RJUG: Selenium
14
OpenQA
Projects

Selenium Core

Used to test web apps; needs access to application server :(

Selenium Remote Control

Used to test web apps; uses

Core as a HTTP proxy to web app

Selenium IDE

Firefox plugin
for recording a test

case and translating into Java

Selenium on Rails (new!)

Watir

Used to automated browser-based tests using Ruby to drive IE

Floyd

Used to test web apps

with a focus on DDT

Frankenstein

Used to test swing applications

pyWinAuto

Pymeter

WET
9/12/07
RJUG: Selenium
15
Selenium Core

Based on a language called

Selenese

(HTML)

100% JavaScript;
works on most browsers and
OS

s

Must put Selenium Core into your AUT (App Under Test)

Create HTML file with a table

Each row is a command (action or
accessor
)

Access
/selenium/TestRunner
.html inside of AUT and run

Disadvantages:

Not ideal for automation

Requires Selenium installed in the AUT

Does not launch and run the tests

Selenese
is

primitive

No ability to reuse code
9/12/07
RJUG: Selenium
16
Selenium Core
9/12/07
RJUG: Selenium
17
Selenium IDE

Firefox plugin
used to record a test case

Must be recorded in FF, but

can be played back in any browser

Supports all Selenium Core commands

Can export tests in HTML, Java, Ruby, etc
package com.example.tests;
import com.
thoughtworks
.selenium.*;
import java.
util
.
regex
.Pattern;
public class
RJUGTest
extends
SeleneseTestCase
{
public void
testRJUG
() throws Exception {
selenium.open("/");
selenium.type(
"q"
, "RJUG");
selenium.click(
"btnG"
);
selenium.
waitForPageToLoad
("30000");
selenium.click(
"//a/b"
);
selenium.
waitForPageToLoad
("30000");
selenium.
selectFrame
(
"toc"
);
verifyTrue
(selenium.
isTextPresent
("Rochester Java Users Group"));
}
}
9/12/07
RJUG: Selenium
18
Selenium IDE
9/12/07
RJUG: Selenium
19
Selenium Remote Control

Two parts

Selenium Server

Selenium Client Driver

Java, .NET,
Perl
, Python, Ruby

Can reuse code

Does not require Selenium Core in AUT

Solves XSS and other security issues
9/12/07
RJUG: Selenium
20
Selenium Server

Treat it like a daemon process

It kind of is; it

s

a

HTTP proxy

java -jar selenium-server.jar

Options include

interactive

port

timeout

multiWindow
9/12/07
RJUG: Selenium
21
Selenium RC Issues

Can

t handle all modal dialog windows

If you can

expect

a JS popup to occur, you can handle it,
otherwise,

not

Can

t handle file uploads

Can

t handle file downloads

Can

t handle

out of page

issues; these are out of scope
of JS

Eg
:

Firefox
crashes and puts up a dialog

Eg
:
Firefox
prompts you to download a new update

Workarounds for handling these dialogs are ugly (read:
Java Robot class)

XSS

issues are due to browser limitations, not Selenium
9/12/07
RJUG: Selenium
22
Chrome and HTA

Used to get around XSS and HTTPS limitations of
browsers

Chrome

Builds a special
Firefox plugin

Accesses Selenium via chrome:// URL namespace

HTA

Launches Selenium

as an HTA app

HTA is a special mode of IE that has no security restrictions

Allow your tests to


traverse

domains

Without it, browsers wouldn

t allow you to execute JS from
domain A on

domain B

s website
9/12/07
RJUG: Selenium
23
Locators

Element Locators are used to refer to specific elements on a page

Id (@id=

myElement

)

eg
: <div id=

myElement

>

Name (@name=

myElement

)

Eg
: <div name=

myElement

>

Identifier (@identifier=

myElement

)

First tries to match on id,

then tries to match on name

DOM (
dom=javascriptExpression
)

Finds an element using JS traversal of HTML DOM

Eg
:
dom=document
.forms[

myForm

].
myDropdown

Xpath
(
xpath=xpathExpression
)

Locates an element using a given
Xpath
; must start with //

Eg
:
xpath=//html/table
[@id=

table1

]
/tr
[4]/td[2]

Link (
link=textPattern
)

Eg
: link=Click here

CSS (
css=cssSelectorSyntax
)

Eg
:
css=a
[
href=

#id3

]
9/12/07
RJUG: Selenium
24
More on Locators

By default, Selenium assumes

a locator is an identifier

If it starts with //, it assumes
xpath

If it starts with document., it assumes
dom

Best practice:

Use

id

s when available

Use

smart


xpaths
when no id

s are available

Don

t use the full

xpath
unless necessary

Doing so will lead to fragile
xpaths
(and tests)

Xpaths
can get UGLY; use a FF
plugin
to help discover them
9/12/07
RJUG: Selenium
25
Current Trends

Need to reduce the amount of time per run of
testsuites

Not an issue with unit tests due to


smallness

of tests

Reason for long run time?
starting the browser
+ logging into the web application
+ waiting for browser to render pages
+ latency of network
= SLOW TESTS
Tests are running
Testing
9/12/07
RJUG: Selenium
26
How to

Speed up Tests

To reduce the run time of a single test, we can:

Avoid logging into the web application by

starting the browser
with a

special

logged in cookie

Use
Firefox
instead of IE for faster rendering times :)

To reduce the run time of a test suite, we can:

Parallelize tests!

But how?

Multiple sessions on the same machine? OK

Multiple

sessions on multiple machines? Much better

:)

By using a large pool of machines, reduce the overall run time to
the length of your longest test

Gmail
went from taking over 40 minutes to execute test suites to
3.5 minutes (length of longest test)
9/12/07
RJUG: Selenium
27
Selenium Farm

Huge pool of machines, all running different browser /
OS combinations

Write your test once, instruct the farm what browser /
OS combination to execute it on.

OS

s
include

Windows XP, Windows Vista, Linux, OS X

Browsers include FF1.5, FF2.0, IE6, IE7 and Safari
9/12/07
RJUG: Selenium
28
Parallelization Issues

Scalability

Selenium wasn

t

really written with parallelization in mind

Now we need to run hundreds of tests at once

Reliability

Crash

recovery?

Many teams depend on a shared resource

Main Problem: Concurrency

Written prior to java.
util
.concurrent libraries

Deadlocking issues, threading issues, etc

Codebase
is a bit messy

Google is recommitting bug fixes (concurrency related)
back to open source community

http://www.youtube.com/watch?v=qxBatJ1N_Og

(10:00
to 16:30)
9/12/07
RJUG: Selenium
29
Demo

Demo of
Firefox
extension (Selenium IDE)

Demo of Selenium RC

Code samples
9/12/07
RJUG: Selenium
30
Questions?
kurt@kloover
.com