BDD principles (continued) - GoogleCode

quaggahooliganInternet and Web Development

Feb 5, 2013 (4 years and 2 months ago)

114 views


1

O
utline


Introducing BDD


BDD principles


BDD tools


What is Concordion?


Why Concordion?


Concordion in use


Getting along with Maven, Ant and Eclipse


BDD with Concordion


Technique


Common Smells


Resources


2

Introducing BDD


3

What is BDD?

BDD:


Stands for behavior
-
driven development.


Was originally conceived in 2003 by Dan North as a response to TDD.


Is an evolution in the thinking behind Test
-
Driven Development and
Acceptance Test
-
Driven Planning.


Aims to help focus development on the delivery of prioritized, verifiable
business value by providing a common vocabulary (also referred to as a
Ubiquitous Language
) that spans the divide between Business and
Technology.


Relies on the use of a very specific (and small) vocabulary to minimize
miscommunication and to ensure that everyone


the business, developers,
testers, analysts and managers


are not only on the same page but using
the same words.


4

BDD principles


Test method names should be sentences

It can do at least some of your documentation for you, so start to write test

methods that are real sentences. What’s more, when you write the method

name in the language of the business domain, the generated documents make

sense to business users, analysts, and testers.


A simple sentence template keeps test methods focused

Start test method names with the word “should.” This sentence template


The

class should do something


means you can only define a test for the current

class. This keeps you focused. If you find yourself writing a test whose name

doesn’t fit this template, it suggests the behavior may belong elsewhere. If a

class is doing more than one thing, take it as an indication that you should

introduce other classes to do some of the work.

BDD principles
(continued)


An expressive test name is helpful when a test fails

If you’re changing code and cause a test to fail, you can look at the test method name

and identify the intended behavior of the code. Typically one of three things happen:


You had introduced a bug.
Solution: Fix the bug.


The intended behavior is still relevant but have moved elsewhere.
Solution: Move
the test and maybe change it.


The behavior is no longer correct


the premise of the system have changed.
Solution: Delete the test.

The latter is likely to happen on agile projects as your understanding evolves.

Unfortunately, novice TDDers have an innate fear of deleting tests, as though it

somehow reduces the quality of their code.


A more subtle aspect of the word should becomes apparent when compared with the

more formal alternatives of will or shall. Should implicitly allows you to challenge the

premise of the test:
“Should it? Really?”

This makes it easier to decide whether a test

is failing due to a bug you have introduced or simply because your previous assumptions

about the system’s behavior are now incorrect.

BDD principles
(continued)


“Behavior” is a more useful word than “test”

BDD gives answers to some of the most vague TDD questions:


What to call your test is easy


it’s a sentence describing the next
behavior in which you are interested.


How much to test becomes moot


you can only describe so much
behavior in a single sentence.


When a test fails, simply work through the process described above


either you introduced a bug, the behavior moved, or the test is no
longer relevant.

BDD principles
(continued)


Determine the next most important behavior

A really useful way to stay focused is to ask: What’s the next most important

thing the system doesn’t do?


This question requires you to identify the value of the features you haven’t yet

implemented and to prioritize them. It also helps you formulate the behavior

method name: The system doesn’t do X (where X is some meaningful

behavior), and X is important, which means it should do X; so your next

behavior method is simply:


public void shouldDoX() { // ... }


Now we have an answer to another TDD question, namely
where to start
.

BDD principles
(continued)


Requirements are behavior, too


BDD provides a “ubiquitous language” for analysis


BDD Tools


10

BDD Tools
(continued)


ASSpec

-

ActionScript 3


BDoc

-

Extracting documentation from unit tests,
supporting behaviour driven development


BDD in Python is core module
doctest


Bumblebee

-

Extract documentation from JUnit
tests with support for adding text, code
-
snippets,
screenshots and more. Puts focus on the end
-
user.


beanSpec

-

Java


cfSpec

-

ColdFusion


CSpec

-

C


dSpec

-

Delphi


Concordion

-

a Java automated testing tool for BDD
that uses plain English to describe behaviors.


Cucumber

-

Plain text + Ruby. Works against Java,
.NET, Ruby, Flex or any web application via Watir
or Selenium.


easyb

-

Groovy/Java


EasySpec

-

Groovy, usable in Java. Developer also
working on Perception a tool for doing
Context/Specification reporting for many different
tools.


GSpec

-

Groovy



Instinct

-

Java


JavaStubs

-

Java
-

BDD framework supporting
partial
-
mocking/method stubbing


JBehave

-

Java


JDave

-

Java


JFXtras Test

-

JavaFX


JSpec

-

JavaScript


JSSpec

-

JavaScript


NBehave

-

.Net


NSpec

-

.Net


NSpecify

-

.Net


NUnit

-

Another implementation of BDD framework
in .Net with focus on specification testing


PHPSpec

-

PHP


Pyccuracy

-

Behavior
-
driven framework in Python.


Pyhistorian

-

General purpose BDD Story Runner in
Python (internal DSL, not plain
-
text)


RSpec

-

Ruby


ScalaTest

-

Scala


specs

-

Scala


spec
-
cpp

-

C++


Specter

-

Another implementation of BDD
framework in .Net with focus on specification
readability


StoryQ

-

.Net 3.5, can be integrated with NUnit to
provide both specification readability and testing


tspec

-

Groovy (Thai syntax)


12

What is Concordion?

Key Features


Concordion is an open source tool for writing automated acceptance tests in
Java


Powerful, yet simple to use


Concordion integrates directly with JUnit.


Highly readable tests


Concordion acceptance tests are so readable they can double up
as system documentation. And, since the tests are linked to the
system, you know the documentation is always up
-
to
-
date.


Separates tests from implementation


Tests that include a lot of implementation detail lock you into
that implementation. Concordion helps you to document the
logic and behavior of your system in a way that does not lock
you in.

What's special about it?


Plain English specifications


Rather than forcing product owners to specify requirements in a specially
structured language, Concordion lets you write them in plain English using
paragraphs, tables and proper punctuation. This makes the specifications much
more natural to read and write, and helps everyone to understand and agree
about what a feature is supposed to do.


Always bang up
-
to
-
date


Concordion specifications are active. Behind the scenes, they are linked to the
system under test and therefore do not go out
-
of
-
date. If a change is made to
the system's behavior then the tests associated with the relevant specification
will fail and let you know.


Complex behaviors can be decomposed


Each question at the bottom of the specification links to another active
specification and so on until you have either run out of questions or the answer
is "Out of Scope". In this way, a complex behavior can be broken down into
small, focused and easily understood requirements.

Concordion in use


Specifications are written in simple HTML. Developers instrument the
concrete examples in each specification with commands (e.g. "set",
"execute", "assertEquals") that allow the examples to be checked against a
real
-
life system.


The instrumentation is invisible to a browser, but is processed by a Java
fixture class that accompanies the specification and acts as a buffer
between the specification and the system under test. The fixture is also a
JUnit test case, so it's easy to run and integrate into an automated build.
The results of running the examples are exported with green and red
indicating successes and failures.


Some example instrumentation:

<p>


When <span
concordion:set="#firstName"
>Bob</span> logs in a greeting


<span
concordion:assertEquals="greetingFor(#firstName)"
>Hello Bob!</span>


should be displayed.

</p>


Concordion commands


A Concordion active specification consists of two parts:


1.
a well
-
formed XHTML document describing the functionality

2.
fixture code written in Java that finds concrete examples in the
document and uses them to verify the system under test. Both files
must be in the same package.


In order for the magic to happen, the document must first be instrumented
with commands.


Concordion commands are specified as attributes on elements in the
XHTML document. Web browsers ignore attributes that they don't
understand, so these commands are effectively invisible.


The commands use a "concordion" namespace defined at the top of each
document as follows:

<html xmlns:concordion="http://www.concordion.org/2007/concordion">

concordion:assertEquals

<html
xmlns:concordion="http://www.concordion.org/2007/concordion"
>


<body>


<p
concordion:assertEquals="getGreeting()"
>Hello World!</p>


</body>

</html>


package

example;


import

org.concordion.integration.junit4.ConcordionRunner;

import

org.junit.runner.RunWith;


@RunWith
(ConcordionRunner.
class
)

public class

HelloWorldTest

{


public

String getGreeting() {


return

"Hello World!"
;


}

}

By default, Concordion outputs to the directory specified by the system

property
java.io.tmpdir
.


concordion:set

<html
xmlns:concordion="http://www.concordion.org/2007/concordion"
>


<body>


<p>


The greeting for user <span
concordion:set="#firstName"
>Bob</span>


will be:


<span
concordion:assertEquals="greetingFor(#firstName)"
>Hello Bob!</span>


</p>


</body>

</html>


package

example
;


import

org.concordion.integration.junit4.ConcordionRunner;

import

org.junit.runner.RunWith;


@RunWith
(
ConcordionRunner
.
class
)

public class

HelloWorldTest {



public

String greetingFor(String firstName) {


return

"Hello "
+

firstName

+

"!";


}

}


concordion:execute

The execute command has three main uses:

1.
Executing an instruction with a "void" result.

2.
Executing an instruction with an object result (to allow multiple properties
of the object to be checked).

3.
Handling unusual sentence structures.

concordion:execute
. Part 1

Executing an instruction with a void result


It can occasionally be useful to execute an instruction that sets up some system
state. Every time you do this, however, alarm bells should ring in your head and you
should question yourself to make sure that you are not inadvertently writing a script
instead of a specification. E.g. a call to
clearDatabase()

would be a blatant
misuse.


As a rule of thumb, methods with a void result called from an execute should start
with the word set or setUp. E.g.
setUpUser(#username)
.

<html
xmlns:concordion="http://www.concordion.org/2007/concordion"
>


<body>


<p>


If the time is


<span
concordion:execute="setCurrentTime(#TEXT)"
>09:00AM</span>


then the greeting will say:


<span
concordion:assertEquals="getGreeting()"
>Good Morning World!</span>


</p>


</body>

</html>

#TEXT special variable contains the text of the current element.

concordion:execute
. Part 2

Executing an instruction with an object result

Sometimes you need to check more than one result of a behavior. For example, here we want to

check that both the first name and the last name are correctly extracted from the full name:

<html
xmlns:concordion="http://www.concordion.org/2007/concordion"
>


<body>


<div class="example">


<p>


The full name


<span
concordion:execute="#result = split(#TEXT)"
>John Smith</span>


will be broken into first name


<span
concordion:assertEquals="#result.firstName"
>John</span>


and last name


<span
concordion:assertEquals="#result.lastName"
>Smith</span>.


</p>


</div>


</body>

</html>


concordion:execute
. Part 2

(continued)

package

example;


import

org.concordion.integration.junit4.ConcordionRunner;

import

org.junit.runner.RunWith;


@RunWith(ConcordionRunner.
class
)

public class

SplittingNamesTest {


public

Result split(String fullName) {


Result result =
new

Result();


String[] words = fullName.split(" ");


result.firstName = words[0];


result.lastName = words[1];


return

result;


}



class

Result {


public

String firstName;


public

String lastName;


}

}


concordion:execute
. Part 3

Handling unusual sentence structures

One of the great things about Concordion is that when you're writing the specifications you do not

have to worry about how you're going to instrument it. You can just concentrate on making the

document as readable as possible.


<p>


Upon login, the greeting for user <span
concordion:set="#firstName"
>Bob</span>


will be:


<span
concordion:assertEquals="greetingFor(#firstName)"
>Hello Bob!</span>

</p>

vs

<p
concordion:execute="#greeting = greetingFor(#firstName)"
>


The greeting "<span
concordion:assertEquals="#greeting"
>Hello Bob!</span>"


should be given to user <span
concordion:set="#firstName"
>Bob</span>


when he logs in.

</p>


The execute command is designed to process commands on its child elements in a

special order. First of all it processes any child set commands then it runs its own

command, then any child execute commands and finally any child assertEquals

commands.


concordion:execute on a <table>

When you want to show several examples of a behavior, repeating the same sentence structure over

and over again probably isn't going to be very nice to read. It would be better to use a table.


<table
concordion:execute="#result = split(#fullName)"
>


<tr>



<th
concordion:set="#fullName"
>Full Name</th>


<th
concordion:assertEquals="#result.firstName"
>First Name</th>


<th
concordion:assertEquals="#result.lastName"
>Last Name</th>


</tr>


<tr>



<td>John Smith</td>


<td>John</td>


<td>Smith</td>



</tr>


<tr>



<td>David Peterson</td>


<td>David</td>


<td>Peterson</td>


</tr>

</table>


concordion:verifyRows

Sometimes you want to check the contents of a collection of results returned from the system. In the

Fit Framework you might use a RowFixture. In Concordion, you use the verifyRows command.


<table
concordion:execute="setUpUser(#username)"
>


<tr><th
concordion:set="#username"
>Username</th></tr>


<tr><td>john.lennon</td></tr>


<tr><td>ringo.starr</td></tr>


<tr><td>george.harrison</td></tr>


<tr><td>paul.mccartney</td></tr>

</table>


<p>Searching for "<b
concordion:set="#searchString"
>arr</b>" will return:</p>


<table
concordion:verifyRows="#username : getSearchResultsFor(#searchString)"
>


<tr><th
concordion:assertEquals="#username"
>Matching Usernames</th></tr>


<tr><td>george.harrison</td></tr>


<tr><td>ringo.starr</td></tr>

</table>


The syntax for a
verifyRows

command is:
#loopVar : expression

Where
expression

returns an Iterable object with a predictable iteration order, (e.g. a List, LinkedHashSet or a

TreeSet). And
#loopVar

provides access to the current object during iteration and allows the
assertEquals

method

to check its value.

concordion:verifyRows

(continued)

The order of the items in the table being verified must match the iteration order of the items returned by the

expression. You may need to sort the items to ensure they are in a known and consistent order. In our example, we

are using alphabetical order ("george" before "ringo").


@RunWith
(ConcordionRunner.
class
)

public

class PartialMatchesTest {



private

Set<String> usernamesInSystem =
new

HashSet<String>();



public void

setUpUser(String username) {


usernamesInSystem.add(username);


}



public

Iterable<String> getSearchResultsFor(String searchString) {


SortedSet<String> matches =
new

TreeSet<String>();


for

(String username : usernamesInSystem) {


if

(username.contains(searchString)) {


matches.add(username);


}


}


return

matches;


}

}

Getting along with Maven, Ant and Eclipse

The simplest configuration to run Concordion tests with maven.


<build>


<plugins>


<plugin>


<artifactId>maven
-
compiler
-
plugin</artifactId>


<configuration>


<source>1.5</source>


<target>1.5</target>


</configuration>


</plugin>


<plugin>


<artifactId>maven
-
surefire
-
plugin</artifactId>


<configuration>


<systemProperties>


<property>


<name>concordion.output.dir</name>


<value>target/concordion</value>


</property>


</systemProperties>


</configuration>


</plugin>


</plugins>

</build>

Getting along with Maven, Ant and Eclipse
(continued)

The simplest configuration to run Concordion tests with Ant.

Since Concordion is built on top of JUnit it takes no additional effort to run Concordion tests with JUnit. The same is

true for the Eclipse IDE as well as any IDE that supports JUnit.


<junit fork="yes" forkmode="once" printsummary="yes" haltonfailure="yes" showoutput="yes">


<jvmarg value="
-
Dconcordion.output.dir=build/concordion
-
output"/>



<classpath>


<path refid="compile.classpath"/>


<pathelement location="build/classes"/>


</classpath>



<formatter type="plain" />



<batchtest todir="build/test
-
output">


<fileset dir="specs">


<include name="**/*Test.java"/>


<exclude name="**/Abstract*"/>


</fileset>


</batchtest>

</junit>

BDD with Concordion


29

Technique

Write specifications, not scripts


Scripts over
-
specify


Test scripts are a list of instructions to be followed. For example:


Clear database


Load database from "sample
-
data.sql"


Start webserver


Open URL: http://localhost:8080/myapp


Enter username: admin


Enter password: admin1


Click the "Login" button


Click the "User Administration" link


Click the "Create User" button:


Enter name: John Smith


Enter username: john


Enter password: john99


Click the "OK" button


Click the "Logout" link


Enter username: john


Enter password: john99


Click the "Login" button


Check page contains text: Hello John!


Write specifications, not scripts

(continued)


Concealed inside the script is a behavior that the test is trying to
demonstrate. But because the requirement is not explicit, it's hard to know
what it is (e.g. "Can we change this link to a button, or is the link part of
the requirement?").


Scripts also tend to suffer badly from duplication. If we introduce an extra
step into the authentication process then all the scripts that mention
logging
-
in will need modification. This is exactly the kind of duplication that
programming languages are designed to address and why plain English is
not a good language for scripting.

Write specifications, not scripts

(continued)


Specifications give you freedom


Specifications tell you the requirements explicitly. They are written at a
higher level of abstraction to test scripts. For example:


When John logs in, a greeting "Hello John!" is displayed.


This can be implemented in multiple ways. All of the details about how
to test the requirement are hidden inside the fixture code where they
can be refactored as the system evolves.

Specifications should be stable


The specifications themselves should rarely change. Even on agile
projects, though new behaviour is added frequently, existing
behaviour is normally maintained from iteration to iteration.

Evolve a domain
-
specific language

As you write the fixture code, and refactor it to remove duplication,

you'll find you gradually build up a scripting API


a domain
-
specific

language (DSL)


that lets you manipulate the system under test.

Eventually, the fixture code will become very stable too.

Isolate behaviours


Each active specification should have a narrow focus and cover a single
behavior as independently as possible from other behaviors. For example,
one specification might describe how text searches are case
-
insensitive,
another will describe how date searches work, and another will describe
how search results are to be presented (what data is displayed). The idea is
to keep each specification very simple and to avoid overlaps, so that we can
change the specification about the way search results are presented without
having to make changes to the other specifications.


If you want to test a combination of behaviors, write a separate
specification of the behavior for the combination. But always write the
specifications for the individual behaviors first. When you combine
behaviors there is a penalty in terms of complexity


of both the
specification and the fixture code to support it.

Think "Given
-
When
-
Then"


This is an excellent way of structuring the concrete examples in the specifications,
and getting into the mindset of specifying instead of scripting.

Given

(some context)

When

(something happens)

then

(some behavior)

Example:

Given a user called John;

When John logs in,

then a greeting is displayed saying "Hello John!".


Once you've written or thought about your behavior using the Given
-
When
-
Then

template, you can reword the sentence to make it less clunky. For example, in this case

we can deduce that John is a user from the phrase "John logs in", so we can drop the

"Given" part and write something more readable like this:



When John logs in, a greeting "Hello John!" is displayed.

Common Smells

Existing specifications are often changed


You're tied to an implementation


If you find you're having to change the contents of
specifications/instrumentation, on a regular basis, this is a strong
indicator that the specifications are too closely coupled to the
implementation.


So, pretend there are multiple implementations


The solution is to describe the underlying behavior in a more abstract
way. It might help to imagine that there are several different
implementations
-

a web application, a Swing application, and a
command
-
line executable, for example. Only describe behavior that is
required so that you leave as many options open as possible for
alternative implementations.

Lots of "execute" commands


You're writing a script


If you have lots of "execute" commands you are probably writing a
script instead of a specification.


So, hide the scripting in the fixture


Avoid describing the steps of how to test the behavior and simply state
the context (as a sentence), and the behavior you expect to see. Don't
explain how to get into that context, don't explain how to perform the
operation and don't explain how to extract the results. All of these
things are implementation
-
specific and should be hidden in the fixture
code.

Complicated instrumentation


You're testing too much in one go


As a rule of thumb, well
-
written fixtures should have no more than
three public methods and no method should have more than one or two
parameters. Complicated instrumentation is usually a sign of trying to
test too many things at once.


So, decompose the behavior


Focus on one tiny behavior at a time. Break the behavior into smaller
and smaller pieces until you cannot break it down any further. Then
write separate specifications for each piece.

Complicated fixture code


Your fixture code is verbose and hard to follow


All the scripting should be done in the fixture code, but this can make
the code seem complicated.


So, create classes to help with the scripting


Begin to create objects to help with the scripting activity
-

push the
implementation details into them. Eventually you'll create a little domain
specific language (DSL).

Examples all have the same structure


Your examples are too generic


If the examples in your specifications
look very similar (i.e. they have the
same kind of context set
-
up, the same
kind of checks etc.) this is a strong
sign that the examples are too generic.


So, focus the examples more carefully


The examples should demonstrate the
particular behavior you are describing
and should not include any irrelevant
details. Push everything you can into
the fixture code.


The layout above is "one size fits all". It contains

lots of context that is not relevant to the

behavior we want to show.

Focusing the examples improves their clarity an

reduces duplication
.

Resources

Useful links


Dan North's article introducing BDD:
http://dannorth.net/introducing
-
bdd


Introduction to Behavior Driven Development:

http://behavior
-
driven.org/


In pursuit of code quality: Adventures in behavior
-
driven development by Andrew
Glover:

http://www.ibm.com/developerworks/java/library/j
-
cq09187/index.html



The RSpec Book: Behaviour Driven Development with RSpec, Cucumber, and Friends:
http://www.pragprog.com/titles/achbd/the
-
rspec
-
book


Concordion site:

http://www.concordion.org/


Concordion page at google code:

http://code.google.com/p/concordion/w/list