The mixed-mode design of the New Zealand Census of Population and Dwellings:

fullfattruckΚινητά – Ασύρματες Τεχνολογίες

10 Δεκ 2013 (πριν από 3 χρόνια και 8 μήνες)

69 εμφανίσεις

1


The mixed
-
mode design of the New Zealand Census of Population and Dwel
lings:
Then, now and the future

Matthew Flanagan

and Craig Lange, Statistics New Zealand

D
esigning

a
data collection system for more than one
mode

of collection is known as mixed
-
mode design
. It
can
be thought of as a balancing act between maximising the potential of each
mode while also attempting to minimise any effects that may be introduced by the
respondent’s choice of mode. In practice, this h
as often meant
the design is
a compromise
between the opportunities and limitations of the various modes. For example, an online form
design allows for design elements
that are
not possible in a paper form. This leads the mixed
-
mode designer to make
decisions about the extent to which they use those elements in the
online form given they cannot b
e replicated in the paper form.

This paper will look at the design of the

online and paper forms used for the

New Zealand
Census of Population of Dwellings si
nce 2006
, when online response was first offered
.
We
provide

background to the census and some of the rationale for design choices for the online
form
,

and explain how

aspects of the original
form
design

have since been challenged
.
We

discuss

the emerging
opportunities and challenges
of

design
ing

for

data

collection using
mobile communications technology


for example, s
martphones and tablet devices.

As our collections strategy
demands we

make much greater use of online data collection, what
will be our

design
philosophy for

the online mode? And as we think of emerging technologie
s,
even
our understanding of the term


online


is being challenged.

Introduction

S
tatistics New Zealand recently
beg
a
n

a programme of work
,

Stat
istics 2020


Te Kāpehu Whetū,
which
includes

a plan to modernise our

data

collection activities.
It

has

a vision of a world class
statistical collection service with quality, efficiency and sustainability at its core



recognis
ing

that
traditional methods of
collecting

data from our respondents

are
inefficient and
unsustainable
.

I
t is becoming obvious
that a key way

to meet
the
aims
of our collections strategy
is to make greater
use of
online

response options
.
To date, Statistics New Zealand’s experience in offering online
collection of social statistics

is limited to the five
-
yearly census.
Combining our experience of online
census collection
with our vision for the future
,

we can start to form a picture
of

wha
t our
future
online
collection
systems
might look like.



2


Background

to the New Zealand Census of Population and Dwellings

The
New Zealand
Census of Population and Dwellings is usually
1

conducted once ev
ery five years.

It
is the primary source of information on the size, composition, distribution, economic activities and
state
of wellbeing of the population.
By law it must

be completed by everyone in New Zealand on
census night
, both usual residents and visitors to New
Zealand.

The New Zealand Census takes a snapshot of the population on
census

day. Census day is typically
a Tuesday in March, with form delivery and collection taking place in the
two weeks

before and after
census day.
We employ a
temporary workforce of ar
ound 7,000 collectors for the d
elivery and
collection period.

In New Zealand, census information is collected using two forms, an
Individual form

and a
Dwelling
form,

which

are delivered to every occupied
dwelling
.

The
paper
forms are available in
English
-
only
and
bilingual

format
s



the bilingual format presents the Māori language alongside the English.

The
online forms

also allow
users to choose

English or Māori formats


Māori language users
can easily

access

an English translation for each quest
ion.

2006


New Zealand’s
first

online census

The 2006 Census was the first where an
online

response option was offered
. D
evelopment of the
online forms beg
an

in late 2002

(Potaka, 2003)
. During the development
,

a key

objective

was to
minimise
the

effects

of different collection modes,

so that there would be no major differences
between the data collected from
online

resp
ondents and
those using paper forms
.
To help achieve
this, d
esigners
very deliberately
aimed to make the task of filling in forms
online

as similar as possible
to
completing them

on paper
.

Replicating the paper
form
design online not only minimised

the

potential for
differences introduced by choice of collection mode

but
also
made the task of completing
the online form familiar


helping to

meet

a second objective of ensuring the

online

forms were easy
to complete
.
These

objectives
,

and
the fact that
all dwellings would receive paper forms
,

influenced
the design of the online form in the following ways:



c
onsistent use of colour


colours use
d for the

online form
s

were matched to th
ose in
th
e

equivalent

paper forms



c
onsistent use of design element
s



for example
,

question numbers
were
in
white

print
on
solid
-
colour

squares
, just as they
we
re in the paper forms



s
ingle scrolling layout


giving
a

respondent

a similar

a sense of t
he size of the form and their
progress through it, in

both the online and paper form
s




1

The 2011 Census was postponed following the devastating Christchurch earthquake of
22 February
2011


just two weeks before the intended census day. The next
census will be held on Tuesday, 5

March 2013

making
a gap of seven years between the two most rec
ent completed censuses.

3




shading

of non
-
applicable questions


respondent
s

w
ere

routed out of a question
by

displaying
non
-
applicable questions

as
‘gre
yed

out’
.

This
mimick
ed

the ‘go to question x’
approach used in the paper forms

where questions
that did not need to be answered
were still
visible but intended to be ignored



m
inimal use of editing and validation



editing and validation fun
ctionality

was limited
to

the
most important demographic questions and

milestone


questions where accurate response
was
required

to route respondents

correctly
.

The following sections provide some rationale for the decision to aim
for a consistent user experience



so an onlin
e user was

essentially completing the paper form online.

Legislative requirement

for dwellings to receive paper forms

The census is collected under the authority of the Statistics Act 1975, which at the time
required

that

physical paper census ‘schedules’ (forms) be delivered to every dwelling.

Given
that
every dwelling in New Zealand received
paper forms, regardless of the respondents’

final
choice of
response
mode, at least some of the user expectation
of an online respo
ndent
would be framed by the
design of the paper forms.

We

expected that some respondents w
ho
chose to complete their forms online would still refer to the

paper forms

as a guide.

Types of
Internet

connection in New Zealand households

A big factor in the d
ecision to have a single scrolling page design was to do with the ways
New Zealanders connected with the
Internet
. In 2006, of the
65

percent

of New Zealand
househo
lds with home
Internet

connections

(
Statistics New Zealand
, 2007)
,
half

were

analogue

or
‘dia
l
-
up’

connections
.
For

dial
-
up

users,

wh
ose

download speeds are so much
slower,
it w
as
seen as
preferable to have one longer load time

at the start of a form than
many shorter load times
required

for

a question
-
by
-
question, or section
-
by
-
section,
design.

User famil
i
arity
with online forms

In 2006 the New Zealand public were a lot less
Internet
-
savvy than they are now.

Despite

New Zealand households

ha
ving

home
Internet
,

a
t that time

it was
still quite
unusual for
people to interact with government
via online forms.

We felt that

having
online census

form
s

that were

visually similar to the paper forms
would give

less technically able
Internet

user
s

the
confidence to attempt to complete the form online.



4


The end result of this was
the

co
mparable

look

and feel for our paper and online
forms
, as
demonstrated by the images below.

Figure 1:
Example of c
omparative look and feel of paper and online forms, 2006 Census

Paper i
ndividual
f
orm, questions 2

4

Online individual f
orm, questions 2

4



Performance of the

2006 online
census
form
s

Being the first offer of an online response mode, the 2006 forms were subjected to many rounds of
development, usability testing and p
ilot testing
.

Following this extensive develop
ment
and pre
-
testing

process
,
we

deployed

the system

into production confident that it would meet its objectives.

About
7
percent of respondent completed their census forms online,

represent
ing

around 100,000
dwelling responses and 250,000 individuals.

The seemingly low uptake is partly

explained by

the
decision that collectors would not actively promote the online
response mode
.
We made
this decision
because of
uncertainty

around the
online collections
system’s capacity to deal with large numbers of
simultaneous
user

session
s
.

However,
the

low
up
take of the online option did not appear to be as a result of poor user experience.
In

t
he

2006 Census follow
-
up research
(Research New Zealand, 2006)

users

were
asked

about the
ir

experience

with

the online census form
.
Respondents were asked
,

“Overall, how easy was it to fill in
the census forms on the
Internet
?” Seventy
-
seven
percent

of the
141 in the sample
who

had
personally

completed their forms on the
Internet

answered

very easy


with a further 20
percent

stating


easy
’.

From Statistics
New Zealand’s perspective the 2006 online census was
a great
success. The
technology performed as expected
, the
online collection
mode was successfully integrated into
census

collection systems
,
and
those
who completed their forms online

responded positively to the
experience
. The concept was proven and
we would offer
an online response option again in
the
2011

Census
.




5


2011 Census


minimum
change to forms,
key change to

strategy

The 2011 Census operated under a minimum change approach

which meant
big

changes to either
paper or online forms were not in scope.

The one major design change was a reduction in the number
of screens a user needed to navigate when entering or exiting the online forms.
This aimed to make
form completion faster and easier
by taking advantage of growing user confidence
.

Strategically
,

the decision was taken to actively promote the online forms and take an ‘online first’
approach to communications. So where the 2006 Census
could be thought of as

proof of concept


the
2011 Census was going to be the chance to realise the benefits of an online census.

From the
7

percent

achieved in 2006,
we sought a 30 percent

uptake of the online response option.

Our c
onfidence

that

this target could be met, or even exceeded,
was underpinned by our new strategy
to actively promote the online census
. Collectors in 2011 were instructed to

leave
Internet

access
codes
at
every dwelling, to
actively promote the online option
. This was

sup
ported by messaging that
was

online first

and communicated the benefits of completing forms online.

As an example, the
Internet

A
ccess
C
ode

envelope, shown below,

was

delivered to all dwellings and
carried three
messages
designed
to

encourage online part
icipation.

Figure
2
: Images of front
,

and back
,

of
Internet

Access Code
envelope
for the 2011 Census

Front


Back




Another key change

that
supported our
confidence
was

the extent to which New Zealanders had
become more ‘
Internet

connected’ since 2006.
B
y 2009,

home
I
nternet connections were around 75
percent, up from 65 percent in 2006

(
Statistics New Zealand
, 2010
)
. But perhaps more importantly
,

the ways people were connecting with the
Internet

had changed.
By 2009, 6
3
percent

of all
households had broadband
Internet

access
, compared with only 33 percent

in 2006
.

This meant
that
respondents were less likely to encounter the
perceived
barriers to completing an onlin
e form
,

such
as concerns about lo
n
g download times, modest data
caps and
the inco
nvenience
of tying up the
telephone line for the duration of an
Internet

session.


6


Because respondents still had paper forms delivered, much of the original

rationale for
design
choices
for the forms
remained.

Our confidence in the online collection system led to the decision to actively
promote this response option in the 2011 Census
.

This promotion was supported by collector training,
media campaigns and messaging on written communications taking an online fi
rst approach.

Performance of the

2011 online census forms

In the days following the 22

February 2011

Christchurch earthquake, Statistics New Zealand made
the decision to cancel the 2011 Census.

So
,

unlike the 2006 Census
,

there is
was no opportunity to
do
a full
-
scale evaluation of the performance of the online forms.


However,
we do have a little information that suggested the forms were on track to reach the 30
percent

target. By the time the decision to suspend form delivery was made

(and

subsequently
to

cancel the census)
, c
ollectors had

already
started to deliver

forms
.
This meant
those

at

dwellings

where forms
had

already
been
delivered were able to access and complete the
ir

forms
online.
By the
time the online forms were made unavailable on
25
February
, 11 days before census day itself, there

were

already

over

68,000

online forms returned.


2013 Census


minimum

change again but with

important legislative change

The
census intended for
8 March

2011 was rescheduled for
5 March

2013.
Because of
th
is

short
interval
, the 2013 Census will again operate under a strategy of minimum change.

However, one very important change will be possible in the 2013 Census. For the first time, Statistics
New Zealand
plans to

trial a mai
l
-
out
only the information nee
ded to compl
ete an online form
. This
means,

rather than have a collector deliver the forms
,

in one small area of New Zealand

the

first
contact with dwellings will be by

letter
. This letter

will
contain
all the codes and necessary instructions

to
enter the
online census system
.
This essentially changes the approach from

one where the
respondent

opts

in


to the online mode to one where the respondent will need to

opt

out


of the online
mode
by

request
ing

paper forms. If successful, this approach will
increase

the uptake of the online
response mode
, and
has the potential to greatly reduce field costs for
future

census
es
.

The ability to trial this approach
is

due to a change in the legislative requirement to deliver paper
forms to all dwellings.
An amend
ment to the Statistics Act, enacted in July 2010, now

means a
dwelling does not need to receive paper forms,
it

can simply receive the “… means to access an
electronic form”.
This
will

allow us to step away from
online form design

that replicates the paper

form
experience


respondents will not necessarily have a paper form to shape their expectations about
the online form.

The publicly stated target for uptake to the online response option for 2013 is 35
percent

which
,

given
indications from 2011, seems
very achievable
.
W
e

keenly

await the 2013 Census so we can conduct
a full evaluation of the online
collection
mode and th
e strategies that supported it.


7


The future


c
hallenges

raised by minimum change in the ever
-
changing world

The

original design choice
s for the online census have been challenged
over time,

especially as

the
assumptions
we based our design

choices

on

change or disappear.
When we

look at our online forms
in today’s context,
their

simple
design

looks
somewhat old
-
fashioned.

But despite
this possible lack of
sophistication we expect the online form to succeed in meeting its objectives

for expected uptake and
respondent usability
.

However
, Statistics New Zealand
anticipates

refreshing the design of the

online

forms to meet the
expectations

of a modern user.
As

we look to the future
of a predominantly

online
census
, what might

we

do differently? What
things that have

changed


societally, legislatively and technologically



can
we exploit in our redesign work to meet these changes
?

Designing

for handheld mobile devices



n
ew challenges in mixed
-
mode design

An interesting
example of

how the current design is challenged by change
is the increasing use of
m
obile devices, primarily

smartphones.
For the 2013 Census, there
was great interest

from
b
oth our
Minister and
Government Statistician in being able to offer

an online census form compatible with
mobile devices
.
One potential benefit we identified was that an
online census
compatible with mobile
devices
might

positive
ly

influence response

rates

among the young adult population
. This group is

traditi
onally both hard to contact in current

census methods
,

and
are
early and confident adopters of
new technology.

So, given the
two
-
year

window of opportunity
following the cancellation of the 2011 Censu
s,
Statistics
New Zealand began some investigations into the feasibility of a census form

that could be completed
on a smartphone
.

The following sections will discuss the investigations
and
conclusions
we made
about the potential for
smartphone

capable
cen
sus

forms
.

Choice of reference
device
:

W
hat

is

a smartphone?

The first step w
as to decide what would be the ‘
reference device


for testing and possible prototyping
.
Given the
domination of the Apple i
Phone
in the New Zealand market
we chose it

as a
reference
device



specifically

the Apple iPhone3GS
,

which at the time w
as the most up
-
to
-
date version.

We also

decided to include the other

dominant operating system for smartphones


Goo
gle’s Android
operating system. We

acquired a Samsung Galaxy

(the dominant Android device at the time)
as a
second device for testing
.

This

meant our original investigations and
any further
prototyping

we did

would be for the two most commonly use
d devices and operating systems

in 2011.




8


Proof of concept

and test
ing

There was
very little

time

to
adequately
develop and test a new design specific for
use on a
mobile
device
. In other words
, we were unlikely to achieve the
options of either a mobile
-
optimised site (ie
m.census.govt.nz) or a purpose
-
built census app

for mobile devices

due

to time and
resource

constraints
.
So we focussed our

investigations

on
evaluating the

user experience
of completing the
existing
online

form
.

Loading the online form onto the
reference device
s

prove
d

that completing the census
on a

smartphone was possible
. A user was able to successfully
log in to the online forms system, select
forms to complete and complete them. However, the user experience was
compromised by the fact
the forms were n
ot designed for a smartphone’s
screen size.
Fo
r

example,
the default view was ‘f
it to
screen width’ which made text

hard to read and
more
difficult to use touch navigation. Zooming the
view meant
a user

could read the
text
, but now

had to
scroll

horizontal
ly

to

perform

some

tasks, which
impact
ed their

ability to respond effectively.

Figure
3
:
Example
s

of online individual form on Samsung Galaxy in 1) default view, 2) zoomed for readability and, 3)
with onscreen keyboard

(
Note:
not to scale)

1.

Default view


2.
Zoomed for readability


3. With o
nscreen keyboard



Other e
lements of the online design
could not be replicated

on the small screen of a smartphone. A
principle of the design for
the existing
online forms was that users would not need to scroll
vertically
to
see all response options for a particular question.
On the reference devices t
his was not possible for
some

questions, which meant these questions were
prone to

response bias.

As a single scrolling
page design,

individual questions could not be forced
t
o vertically fit to the screen size so there were
no easy solutions to this problem.

9


Another
example of a
usability

issue

for
the existing online form on
a smartphone was when making
responses to questions. In cases

where

a pop
-
up keyboard claim
ed

a large
proportion of the visible
screen size
,

a respondent could easily

los
e

the
focus of
the

question.
The possible effects on data
quality and respondent willingness to continue are hard to quantify but are
certain to be

negative.

In summary, the small amount of testing done
was
enough to learn that
while it was possible to
complete the forms using one of the reference devices
,

the user experience was poor and likely to
introduce error into the response process.

Focus groups an
d re
search into public attitude

To investigate the public attitude to a smartphone census, Statistics New Zealand conducted a series
of focus groups in October 2011. The objective
s of the focus group sessions were to:



l
earn about the range of devices used by o
ur focus group subjects



discover

how mobiles are used for online tasks



e
xplore the design fe
atures that characterise mobile
-
friendly sites



g
a
ug
e interest in completing census forms
on a

s
martphone.

The four focus groups were recruited
from the young adult range


recognising them
as

a

group
considered to be
potential use
rs

of a smartphone census. A full report on the focus group

research
has been

published

(Kaye & Saul, 2012)

but

some of the key findings were:



y
oung mo
bile
Internet

users
tend to use their
smartphones

in short sessions

for specific
tasks



s
ocial networking sites (Facebook, Twitter
etc
) are
some of
the most used sites



s
martphone

is the preferred method

when ‘out and about’

or their home computer
is
otherwise unavailabl
e
, but
users still prefer to do
more complex
tasks on a larger
-
screen
device



c
ost

and
the

wise


use of data allowances, are

key consideration
s

for the young
smartphone

user.

When asked specifically what
they thought about

completing their census forms online focus group
participants

were not greatly enthusiastic
. Other than the reasons expressed above, the key reason
given was that it seemed the sort of task that would be harder to do on a phone


this would be one of
the
tasks they would prefer to do on their lap
top.

Despite our expectation,

the focus groups indicated that among
the young adult market

t
here seemed
to be little interest, meaning the costs of developing the mode were unlikely realise any great benefits

for t
he 2013 Census
.


10


Decisions made for 2013 Census

Given the less
-
than
-
satisfactory user experience for testers when using a smartphone to complete the
census and the apparent lack of interest among a key potential user group,
Statistics New Zealand
has tak
en a neutral position regarding the completion of
2013
census forms on smartphones.
Effectively this means their use is neither encouraged nor discouraged for online form completion.
Formal advice to
respondents

to the

2013

Census

will be that, while completing census forms on a
mobile
device
ma
y
be possible
,

the
forms have not been designed for mobile

devices

and so the user
experience will be compromised

when compared with completing on a traditional computer
.

For

larger
tablet d
evices,
with 8 to 10 inch screen sizes, the online census form user experience has

more
in common with the traditional computer interface than a smartphone
.

We believe

tablet users

should encounter little difficulty in completing their online census forms
.

Future of the online census


where
to from here?

This paper has discussed the reasoning for the original design choices for the online
census forms
and also explained how

over time the basis for some of these decisions has been challenged.

But the
online

mode is here to stay and will
, in time,

become the

dominant collection

mode for
census
.

As we
look forward,

p
erhaps the most important change in environment is the legislative change
that
mean
s

respondents do not have to receive paper forms
. It will be
possible in the future to invite respondents
to go directly online and complete their forms.

Without this constraint we will have more scope to do
things differently with the online collection mode
.

The

next census

(
tentatively
in
2018
)

w
ill

take an

onlin
e firs
t’

approach

to design

b
ut we
are
now
challenged to define just what we mean by online.
The current
online forms



faithful to the paper
form user experience

and designed for a screen resolution of 1024 x 768



does not work well on
small mobile devic
es such as
a smartphone

nor is it optimised for touch
. At the time of the original
form design these
were

not
an issue
, but are now just two of the issues

we
face.

As

we design the online form

for future censuses, and
in time for

other social survey collections,

we
must

find a way to not only design for th
ose

technologies we know about, but also somehow bear in
mind those technologies
soon to be in common use
.
Among
those things

we know about


s
martphones,

tablet
s
, traditional com
puters and

Internet

capable televisions


screen sizes

can

r
ange
from 3.5 inches right through to

50 inches

and above
.

The traditional mouse and keyboard is being
replaced by touch screen, voice activation and motion sensing
, and
browsers and
operating systems
in use today
a
re diverse and constantly being upgraded. These examples demonstrate the
already
significant
design challenges

we face
, without even thinking about the
challenges of

things still to be
invented or those that

change from the
‘unusual’ to the norm
.

11


Despite this

un
certain
ty
,
we are sure

that making best use of
online response

will be
a
key

consideration

as
Statistics New Zealand

transform
s

its

collection activities



for census and our
broader survey programme
.
Our respondents

will

continue to become increasingly hard to contact,
more technological
ly

capable and demanding of solutions that suit them.
Future
collections will

offer
choices of collection mode

to respondents, but it is unlikely that any one mode
could

dictate the
d
esign of the others to th
e
same
extent
as

the original online
census
form
design.



12


References

Kaye L & Saul K (2012).
Investigating the value of providing a census
Internet

option for small mobile
devices.
Statistics New Zealand.

Potaka L (2003).

Comparability and Usability: Key issues in the design of
Internet

forms for New
Zealand’s 2006 Census of Population and Dwellings
.

Statistics New Zealand
.

Research New Zealand (2006).

2006 Census follow
-
up research into public attitudes
. Unpublished.

Statistics New Zealand (2007).
Household Use of Information and Communication Technology
Survey
,
2006.
Available from
www.stats.govt.nz
.

Statistics New Zealand

(2010).
Household Use of Information and Communication
Technology 2009
.
Available from
www.stats.govt.nz
.