Avenues to Learning Braille using Smart Phones

chulavistajuniorMobile - Wireless

Dec 10, 2013 (4 years and 7 months ago)


Avenues to Learning Braille using Smart Phones

1. Approach

Modern smart phones like the iPhone and Google And
roid phones have the ability to

output speech and vibrations

which make them suitable as multi
modal devices for blind
children to learn Braille

and other knowledge that will
be useful throughout life. Th
of this Phase I proposal is to take advantage of modern smart phones to make them into
vehicles for learning for blind children.

Game playing is a huge motivator for children to

tting this together, we will design, implement, and evaluate educational games
on smart phone platforms for children to learn Braille and other lifelong
skills and

In 1999, there were about 1.4 million blind children the world under the age



In the same year

it was reported that 95,000 children in the United States under
the age of 18 ha

severe visual impairments [
]. Although the number of blind
children in the United States is small, their education


depends critically on their
learning of Braille.

. Accessible smart


and other mobile technology

Blind adults and older children are becoming heavy users of smart phones because they
have become more and more accessible.

The iPhone VoiceOver s
ystem is very popular
with blind users and the Google Android platform
provides Talk Back and

the accessible
Free Shell.

There are thousands of applications for these platforms, but very few of
them are accessible.

We have found virtually no game
s on either platform that are

Generally, mobile devices are becoming more used by blind people than ever
before. A formative diary and interview study was conducted on how
people with visual
impairments select, adapt,

and use mobile

in their daily lives [

The study
showed that people with these impairments use a variety of strategies to adapt
inaccessible mobile devices and successfully use them to perform every day tasks and
navigate independently. Mainstream devices were expl
icitly preferred by most of the 18
blind and visually
impaired participants, for their cost and sustainability, even if



. Braille

Braille is a method that is widely used by blind and deaf
blind people to read and write.
was invented by a blind man, Louis Braille, in 1821. Each traditional Braille character

or cell

is made up of six dot positions, arranged in a rectangle containing two columns
of three dots each. (Eight
dot Braille also exists, used to extend the single
cell alphabet
for refreshable Braille displays). While there are many blind people, especially in the
US, who do not know Braille and use audio instead, almost all literate deaf
blind people
are familiar with it [AFB1].

character translation
from print letters to B
raille letters is called Grade

Braille, although numerals and capital letters require an extra mode character. The
standard Braille code, called Grade 2, has contractions which reduce text length by about
30%, enabling faster Brail
le reading. Reading rates for experienced Braille readers

vary, but one study with 44 experienced Braille readers found a median
reading speed of 124 words per minute, equivalent to 7.5 characters per second (Grade 2
is read much faster) [
Legge]. Deaf
blind Braille readers are typically much slower
because most learn Braille later in life

due to losing sight as an adult from Usher's

Paperless Braille displays have been in common use since the early 1980s. They are
usually integr
ated into personal devices and computers with a standard QWERTY or
chording keyboard. A chording keyboard allows the user to enter one Braille cell at a
time by pressing several of six keys simultaneously.

We are not trying to improve upon such specialty
devices, since these devices have been
proven to be very useful. Instead, we are interested in situations where smaller,
mainstream mobile devices would be a useful option or alternative. Mainstream mobile
phones are much cheaper and smaller than special p
urpose paperless Braille devices,
which may make them attractive. Designing for a mainstream
phone encourages
universal design and could be used by low
vision, motor
impaired, and able
people. Aside from working towards specific applications,
it is useful to test new
interaction methods. There has been some research on mobile device usage by blind, low
vision, and motor
impaired users [Kane, Seattle Lighthouse], but none specifically
targeted at deaf
blind users.


Alternative tactile moda

There has been work using touch screens and vibrations for non
visual reading. In 1971,
the Optacon (OPtical to TActile CONverter) was invented, an electromechanical device

camera that, with tiny metal rods vibrated by piezoelectric reeds, ena
bled blind
people to read printed material that has not been transcribed into Braille [Sense].

In the
past, some blind and deaf
blind people have used Morse code for communication, but to
the best of our knowledge it is rarely used today.

The Nokia 770 M
obile Internet tablet has a piezoelectric layer in the touch
screen to
mimic the bumps felt in the 3×2 matrix of dots that make up a Braille character
[Ananthaswamy]. Tsukuba University has developed a system that has two terminals
connected to a phone whi
ch vibrate at a specific rate to create a message. This transmits a
linear chain of the Braille dot information per character

The Nokia Braille Reader prototype, developed in 2009, uses the vibrating touch screen
Nokia phones to convey Braille temporally

the user holds their finger on the screen
and a linear off
on vibration will vibrate (e.g., for 'h' which uses Braille dots 1, 2, and 5, a
pattern of on
off will be vibrated) [NokiaBrailleReader].



Braille was developed by our team for the Google G1 phone under the Android
platform. Android is a complete, open, and free mobile platform, supported by the Open
Handset Alliance, a group of more than 30 technology and mobile co
mpanies. The G1
provides a standard speech recognition component. G1 sensors include an accelerometer,
a digital compass, a GPS receiver, a camera, and a microphone. Android provides a text
speech API, as well as the EyesFreeShell and TalkBalk voice int
erface. The diverse
I/O components and open platform make the G1 an ideal test
bed for our mobile phone
accessibility studies, and the Android Market enables developers to easily publish and
distribute applications to users of Android
compatible phones.

Braille is a very simple mechanism for conveying Braille. The screen is divided into
six parts, representing the six dots in a single Braille cell (Figure VBraille). When the
part of the screen touched (that entire 1/6th of the screen) represents a raised

dot, the
phone vibrates. Touching dot areas 2 and 5 presents stronger vibrations than dots 1, 3, 4,
6. This was done to make it easier for users to differentiate between two vertically
adjacent raised dot areas, not thinking they were one. Grade 1 Braille

is used, since it is
simpler and uses no contractions. Grade 2 Braille could be built into the system easily.
The screen can be tapped in different sections at different times or touched in one or a
few strokes in any direction. Our method does not use mu
touch though it could be
easily adapted to.

To initially test V
Braille we developed a very simple application on which we built two
tasks. In Task 1, random Braille characters were presented by vibrations. A new character
was presented after pressing

the menu button. In Task 2, a short sentence was presented
as a string of characters.

In our
studies with 10 people, we verified that
blind people can read V
Braille successfully if they already know Braille. Three separate
participants indep
endently suggested that V
Braille would be a very useful tool in
learning Braille.

Figure 1. Example of VBraille for the letter “p”. The horizontal lines (dotted or not)
indicate that the smart phone vibrates when that area is touched. Cells 2 and 5 h
ave a
slightly different vibration than cells 1, 3, 4, and 6, to make reading easier.

. Connecting to other devices

One of the advantages and potentials of mainstream smart phones is that they can connect
to some other specialized devices like Braille
displays, if they have
luetooth and an
open API. The BrailleSense is one such device, and we have started work on
communicating between the

devices. This idea is not new

HumanWare's DeafBlind

seamless in
person communica
tion and provides three
types of communication for deaf
blind users: face
face, TTY, and SMS texting [DBC].
It costs $6000
8000. The DBC uses Bluetooth to connect a BrailleNote and a standard
mobile phone, both equipped with proprietary deaf
blind softw
are that can only be
installed on the included phone. Texts can be sent to any cell phone. The difference will
be for developers to be able to connect their own smart phone applications to other
Braille displays or devices. Short messages could be read or
input into the small, portable
cell phone, and sensors on the camera could be leveraged (e.g., the camera, GPS,
accelerometer). If more detail is needed for the user, they could receive this on the easier
to read, more familiar Braille display they are usu
ally carrying with them. If they decide
to not carry the bigger devices for certain errands, the mainstream cell phone's Braille
capability, the modalities of audio and vibration, could come in handy.

. Guidelines for
eaching Braille

There is a lot o
f literature out there on how to efficiently and successfully teach Braille to
children, which can differ in strategy based on whether the child had a knowledge of
language and/or print before becoming blind.

Just as sighted children learn the basis of re
ading and writing from an extremely young
age, so must blind children. They do not just go from nothing to becoming Braille
readers. Concepts and training must first be introduced to the child from an early age, to
gradually make successfully reading and w
riting Braille a reality. Preschool experiences
are extremely important to blind
reading readiness [Mangold]. Games are universally
recommended as a way to have repeated exposure to vocabulary while maintaining a high
motivation to read. It is also vital t
hat blind children have the concept of written and
structured language in their environment even if they cannot see it


Many Braille games exist for children, both for fun and for learning. These include but
are not limited to, Braille jigsaw pu
zzles, playing cards, Bingo, board games, Braille
jeopardy, Tic Tac Toe, scrambled sentence g
ames, flash cards, and Braille
no for
learning numbers. Many of these games are quite costly, or are handmade at home. Most
of these games could easily be program
med ont
o mobile devices using Vbraille

can provide
either single player or collaborative gaming experiences. It is also suggested
that children have the concept of technology at an early age [Wormsly]. Many children in
school start to learn with scre
enreading and magnification technology, but it is vital that
they learn Braille at least concurrently, if not before.

To learn Braille, children must have a sense of how to use their fingers in regards to
strength and dexterity, assessing their sense of sp
ace while reading, and fine
tuning their
tactile perception. However, alongside this they must also learn the Braille letters
themselves, possibly separately from the former learning experience. One common
method is to create enlarged Braille cells, using
muffing tins, glue dots, or special Braille
building blocks [Wormsley]. While the spacing of the dots should be proportional, the
size of the Braille cell will be much larger than a standard Braille cell, and be easier to
learn for most children. This make
s our VBraille games seem promising as they do the
same thing, in mobile form.

Some important reading concepts that come out once children have started to learn some
Braille symbols

include learning contractions, numbers, and phonetics. Also, it is
ant to note that reading and writing Braille, while related, must both be treated
seriously when teaching Braille. Knowing one does not presuppose knowing the
other extremely well. More detailed guidlines for learning to read Braille from t
American Foundation for the Blind include: a) Initial reading and decoding

(ages 6
b) Confirmation and fluency

(ages 7
8), c) Reading for learning the new

(ages 9
14), d)
Multiple viewpoints

(ages 15
17), and e) Construction and reconstruction

(ages 18+)

There is literature on assessing Braille literacy skills which will be a part of our Phase I
and II evaluations [Wormsly].

. Games for blind children

Braille games like those listed in the previous section exist as part of some

game kits, which can cost over two thousand dollars. As mentioned before, many of these
games could be programmed in some form onto mobile devices, and this is a great way to
give as many children as possible access to games for a minimal cost [

There are no cell phone games for blind children that we could find out on the
market. There are thousands of cell phone games for sighted children
which are also educational, including games for science, math, spelling, so
lving puzzles,
geography, drawing, and tracing. Many people have

the iPhone, for example,
as an educational kid's toy. Many of the available games could be made in an accessible
form for blind children, and there is also an opportunity
to create many unique, new
specialized games more catered to learning Braille literacy.

Another advantage of using the cell phone for these games is the opportunity for multiple
modalities of learning

vibration, audio, and collaborative gaming can be che
implemented and used. Multiple modalities have been often lauded as interesting,
important ways to reach a child while learning [TechnologytoImprove], and considered
an important research topic. The integration of mobile learning tools with relevant
ducational curriculum is extremely important, as having static access to books and
periodicals can be helpful, but more guided, interactive, and productive learning can be
achieved most efficiently through mobile devices [Lamb].

2. Justification

In this

section we will give as much evidence as we can that
our approach is needed and
will be effective in teaching blind children Braille and other knowledge. We will argue
for the importance of Braille,
games are effective avenues for learning, and

learning Braille through vibrations will most likely be effective.

. Importance of Braille

There is general agreement that learning Braille is critical in the education of blind
children [AFB].

Without Braille

children have to rely only on speech
and hearing in the
learning process. We do not ask sighted children to learn

speech and hearing
only; we generally require them to learn to read and write. Naturally, there are sighted
children with severe learning disabilities whose ability to
read is very limited and others
who cannot produce coherent prose.

The vast majority of blind children are not learning
disabled and can read and write given the tools and instruction to do so.

For more


150 years, around the world, blind children h
ave learned Braille in their native

as the
se children

grow into adulthood
, they
are much more
likely to go to college and have meaningful careers if they know Braille [AFBBraille].

Over the past 35 years, since th
e passage of Public Law 94
142, now called the
Individuals with Disabilities Education Act (IDEA), more and more blind children are in
public school and living at home with their parents, rather than at state residential schools
for the blind.

The vast ma
jority of blind children are now in mainstream schools. In the

the schools for the blind would have teachers well
versed in Braille and trained to
teach children how to read and write Braille. Currently, the teaching of Braille is often
done by itin
erant teachers who go from school to school, so that many children are
exposed to far less Braille instruction than in the past.

In addition to the reduction in
Braille instruction, technology has made it much easier to have access to reading material

speech synthesis. Optical character recognition (OCR) has permitted reading material
to be available digitally. Speech synthesis technology has advanced to the point that most
digitally available texts can be read out loud in a pleasant voice.

The result

of the
reduction in Braille instruction and readily available speech alternatives is that Braille
literacy in the United States has declined over the past 35 years [BrailleLiteracyCrisis].

A recent article in the New York Times even suggests that Braille

is not needed anymore
because of the advances in technology [Aviv].

The purpose of this project is to help turn
this trend around, to help more children learn Braille.

. Games for

It is well understood that young children need to play gam
es and they learn in the process

It has been observed among most mammals that game playing is part of the
natural development process in their young.

Even if a game is not purposefully
educational, games are powerful teaching instruments. Gam
es can teach social skills

ation and friendly competion
. Children’s games that involve the intellect can
teach basic counting and measurement skills, basic mathematical concepts like addition
and substraction, basic story telling, and man
y other building blocks for future learning.
The digital gaming industry is currently capitalizing on children’s and young adult’s inate
desire to play games to the point where many of them are spending too much time
gaming and not taking on the more serio
us tasks of life.

An analogy might be to imagine
that young lion cubs play chasing each other around, tackling each other, and having fun
doing so.

As the cubs mature into adulthood the chasing and tackling strategies they
learned while playing have to b
e transformed into strategies for chasing and tackling prey
so they can survive.

Adult li
s that don’t make the transfer from game playing to
hunting skills won’t survive. Games with an educational purpose should have the same
goal, namely, they should b
e fun and engaging for children

and the skills and concepts
learned playing eventually help the child survive in a highly competitive world where
both hard and soft skills are needed for successful careers.

Most successful teachers in every generation ha
ve used games to help

Over the past 30 years they have begun to use computer games where
students can playfully interact with the game.

There have been some remarkable
successes in educational games for the desk/laptop i
ncluding Oregon Trail, Math Blaster,
and Number Munchers.

A survey of the literature of games in education

extensive list of valuable skills that can be developed from
whether or not they have an educational purpose [Kir

As computers have
become networked

computer games have expanded beyond single players to multiple

Beyond the lap/desk top there is a huge market for portable hand
held computer games as
evidenced by the sales of Nintendo Game Boys and
its successors. Today, thousands of
different games, some educational, are available for download to the iPhone and other
smart phones.

A pioneer in developing educational games for standard programmable
held devices is Eric Klopfer at MIT.

He has a
rgued that hand
held devices are ideal
for educational games because, like Game Boys, they are with the person at all times,
they are much more integrated into the life of the child than one on lap/desk top where
one would go to a specific location to play

or it is time
consuming to start the game on a
portable laptop [Klopfer].

If an educational game for a hand
held is fun and can
be played over short spurts
of time
then the child will be more likely to play in a spare
moment or while traveling.

ur proposal is to develop accessible educational games for the smart phone platform
and make them available for free or very low cost.

The advantages of using the smart
phone are:


It can be with the child at all times, so the fun educational experience c
an occur
anytime and anywhere.


It can hold a virtually unlimited number of games,

a variety of


It is networked so that multi
player games are possible.



can provide tactile, audio, and speech feedback for interaction

cause we are taking an open
source approach

others can build off our work more
easily to increase the number and variety of educational experiences for blind children.

. Learning Braille


An argument that learning Braille

vibration is possible should be viewed with
some skepticism

after all, it is an untested concept.

This is why we are proposing this
approach in a Phase I Steppingstones proposal. Our goal is to test the concept that we
believe is justified by the wor
done on Braille perception and understanding found in the
cognitive science and neurobiology literature. In particular, it has been shown that the
primary visual cortex of the brain is activated when blind people are reading Braille, the
same part of
the brain that is activated when sighted people read [Sadato].

In other
words, the tactile sense of a blind person reading Braille provides information to the brain
that is processed in a visual
spatial way.

This result provides further evidence that

Braille reading and writing and standard reading and writing for sighted people are
essentially the same task from an intellectual standpoint. We are also not the first group
to link the idea of Braille and vibration, which is very promising, and compan
ies such as
Nokia see this as a viable path to follow as well (see 1c).

VBraille is different from standard Braille in two ways.

First, it is larger, taking up the
screen of a hand
held device. Second, it uses vibration rather than raised dots to indicat
which dots in the 3x2 matrix of dots are “active.”

Ultimately, children who learn
raille must translate that skill to reading standard Braille.

We believe learning the
spatial 3x2 patterns of Braille using VBraille will be imprinted in the brain and

the information to the child when reading standard Braille.
As mentioned earlier, it has
been common practice to teach children Braille using larger, hand
made objects before
moving to standard sized Braille.
By analo
gy, sighted children start to

learn to read large

simple fonts, then translate that learning eventually to much smaller and varied fonts.
The spatial imprinting of the font seems fairly independent of size and exact shape.

VBraille the fact that vibration is tactile may

be enough to translate to tactile pressure
when reading standard Braille.

Regardless, this proposal will investigate whether
learning VBraille does aid in learning to read and write standard Braille.

As mentioned earlier


presented VBraille to 10 bli
nd and deaf
blind adults who
already knew Braille. All but one were able to quickly begin reading VBraille on a smart
phone. This gives evidence that the visual model of Braille translates from standard
Braille to vibrating Braille. We believe the visual
model will also translate in the other

3. Plan

This is a Phase I Steppingstones proposal which is developing and evaluating educational
games for the Google Android platform.

A primary focus of the games is the learning of

. Game
design and implementation

A group of undergraduates at the University of Washington

have already designed and
implemented two games using VBraille for the Google Android platform gear
ed to
learning Grade I Braille
. The first is a si
mple V

on that resembles flash
ards, which we call Brai

In the game, a

VBraille letter is displayed to
the child
is asked to identify i

one of several possible ways.

If the letter is correctly
identified then a new letter is presented.

in a different mode of the game,
random spoken letter is presenting and the child is asked to enter the VBraille character
on the touch screen.

This is done by touching the screen in the areas indicated by Braille
positions 1

If an area i
s touched

corresponding dot
is spoken


and a double
tap on the screen will activate that dot.

A final gesture on the screen inputs the dot
pattern. The child is then notified if the answer is correct.

BrailleFlash is not intrinsically self

Some external motivation would cause a
child to practice Braille using BrailleFlash.

For this reason
the students

designed and
implemented a second game called BraillePet which has a lot of similarity to the famous
Tamagotchi series of electroni
c toys which sold in the tens of millions in the 1990s. In
fact, you can think of BraillePet as an accessible verion of the Tamagotchi.

will indicate from time to time whether it is hungry, sad, or in need of exercise by using
vibrations and so

The child can then feed, cheer

or exercise the BraillePet by
answering various questions that test the child's abilities in Braille.

The smart phone
platform is capable of having several pets on board and different pets can help reinforce
rent learning goals.

One could be more Braille learning focused while another could
be math focused.

After a formative evaluation of BrailleFlash and BraillePet, we will make adjustments to
those games to make them more enjoyable and better for learning.

In addition

we will
design two new games for older children who are more adept at Braille.

These new
games will be cooperative games that can be played with other children both sighted and
blind. One game will be a two
player game and the other will be

a multi
player game,
both with an educational purpose

to be determined as part of the research

An inspiration for cooperative games for smart phones are the collaborative games, like
Palmagotchi games, developed by Eric Kl
opfer at MIT [Klopfer].

The P
games were developed for the Pocket PC platform which has many similarities with the
smart phone platform.

One major advantage of the smart phone platform is that smart
phones are connected to each other via the cell phone data network. This wi
ll allow
players to participate anytime, anywhere, which is a big advantage for hand
held games

Fortunately, there are many examples of two
player and multi
player games that exist for
other platforms that we can potentially adapt.

Beyond designing the n
ew games there a
number of daunting technical challenges in implementing such games.

include designing user interfaces for the games that are usable by sighted and blind
children and determining whether the games are client
server based or peer
peer, or
some combination of the two. Our experience of developing smart phone applications
since 2006 will be invaluable in implementing these games.

. Formative and


Using BrailleFlash and BraillePet we will conduct several
formative evaluations that will
help improve the games and set the stage for
the new games.

The formative
evaluation will consist of a series meetings with blind children who are in the early
phases of learning Braille, and their parents.

meeting will be with one child and
his/her parents.

At the initial meeting

parents will be introduced to the games so that
they are completely aware of what the child will be doing and learning.

We will then
introduce the child to the games. The child a
nd parents will take the smart phones home
so the child can play the games in a more natural setting.

We will also ask the parents to
monitor how much time the child is playing the game at home. In a sequence of visits we
will observe the child playing the

games to see what problems the child is having with the

We will also

both the child and his/her parents about what problems
are coming up and use the meetings to come up with potential alternatives. We will also
test the child at each me
eting on his/her knowledge of Braille to determine the learning

This process will be done for about five children so that we can generalize more
easily about what the problems in the games are

so we can fix them and
help formulate a
more formal


A small summative study of the effectiveness of learning Braille through BrailleFlash and

will be done

The reason for the small size samples is to not exhaust all nearby

The goal of the summative study is not necessarily to

obtain statistically
significant results, but to test the format for future summative studies in a Phase II
Steppingstones proposal.

For this study will use 10 children not used in the formative

and divide them into

two groups of 5
. Individuals in
the first group will be matched
as closely as possible with individuals in the second group in terms of their knowledge of
Braille, the kind of Braille instruction they are receiving, and other developmental factors
such as age and language skills. The fir
st group of 5 will be a control initially, but will
eventually be introduced to the games after a month.

During the first mont

the second
group will have full
time use of the games with some initial training. During the first

the control group
nd the test group will continue

with whatever
instruction they are getting.

The games are intended to be supplementary to whatever
Braille instruction they are already getting, not a substitute. At the end of the month the
control group wi
ll be given the games for a month with the same training.

knowledge we gained from the formative study will inform the exact structure of the
summative study. In the end we have two matched pairs results, group one with group
two and group one with i

, the results will be comparable
. In addition to
testing Braille competency, we will survey the parents and children in ways informed by
the formative evaluation.

It is important to note that the summative study is not a strictly controll
ed laboratory
study and the sample size is very small.

Nonetheless, we hope to find some significant
differences between having the games and not.

More important, a controlled laboratory
study might be inappropriate for evaluating these

mobile games, bec
ause the
y are
intended to be used in many very short sessions over a long period of time.

As mentioned earlier, we will develop two more educational games, a two
person and a
person game
. At the time of this proposal we do not have specifics about
games, but will develop and implement them during the grant period.

procedures for the games will also be developed during the grant period.

. Recruiting subjects

As mentioned

we will recruit children and their parents.

The Superin
tendent of the
Washington State School for the Blind, Dean Stenehjem, has agreed to help us locate
parents of blind children throughout the State of Washington (please see the letter from
Dr. Stenehjem).

We will use IRB process at the University of Washin
gton to create both
the formative and summative evaluations.

In essence we will be recruiting parents and
all tasks given to their children will be approved by the parents.

In addition, the parents
themselves are subjects because we expect them to be inv
olved with their children's play
and they will also surveyed.

. Timeline

Year 1


Formative evaluation of BrailleFlash and BraillePet


Design and implement
ation of

a two
player accessible educational game

Year 2


Summative evaluation of BrailleFlash a
nd BraillePet


Formative evaluation of the two
player accessible educational game


Design and implement
ation of

a multi
player accessible educational game

4. Dissemination and Collaboration

We will disseminate our results at the appropriate conferences a
nd journals in two

The first is the education community for blind children which includes

Council for Exceptional Children,

the Association for Education and Rehabilitation of the
Blind and Visually Impaired,

American Foundation for th
e Blind, and others .


is the human
computer interaction community

which includes the ACM Special
Interest Groups on Accessible Computing and Computer
Human Interaction.

important is getting the games to be used in the community. We w
ill give presentations
and demonstrations at CSUN and other conferences on the consumer side of assistive
technology. In addition, we will attend

consumer events, such as the conventions of the
National Federation of the Blind and the American Council of
the Blind to demonstrate
the games. We will make the completed games available to
public by free download
as part of the Android Market.

Instructions for usin
g the games will be part of the


We will request that people register so that we ca
n notify them of updates and
new games.

Since our accessible games will be open source and based on the Android platform, there
is the natural sustainability of it being part of a large open source effort. As long as
Android is maintained, the source code

of our games will have a home where

can be
improved or used as a basis for more accessible games.

During the project we will gi
support to other developers


few companies

like Ideal Group and

are already in th
e space of accessibility applications for smart phones.

These companies
are welcome to refine the games further to make them even better.

We are happy to
cooperate with these companies, if they make the games available free or at low cost.

The project w
ill develop and maintain an accessible web site where downloads of the
games and instructions in accessible formats will be available.

We will collaborate with the Washington State School for the Blind in finding parents
who are willing to work with us i
n evaluating the games.

Please see the letter from Dean
Stenehjem, Superintendent of the Washington State School for the Blind.

In addition

PI has a close relationship with the Washington State Services for the Blind and the
National Federation of t
he Blind.

He plans to develop relationships with the Council for
Exceptional Children.

5. Research Team

The research team consists of the Principal investigator, Professor Richard Ladner, who
will give overall direction and management of the project,
and three computer science
and engineering graduate students

Chandrika Jaya
nt, Shiri Azenkot, and David Ts

who will carry out the game development
programming, and
evaluations. All three are
accomplished programmers on smart phone platforms.

Ms Jayant and Ms. Azenkot are
accomplished human
computer interaction researchers and are capable of carrying out the
formative and summative evaluations. Two of the three graduate students are legally
blind and one of those two is totally blind.

is funding for one graduate student in
the proposal.

Some of the students have fellowships

other sources of funding so that
the project

will be able to get by

with just funding one of them.

Richard Ladner, Principal Investigator

Dr. Ladner, Boeing Pro
fessor in Computer Science and Engingeering, has been at the
University of Washington for 39 years.

Since about 2004 he has developed an
internationally recognized research program in accessible technology, some of
based on
smart phone technology.


MobileASL project
has designed, implemented and
tested two
way video conferencing suitable for sign language conversations [Cavender].

He has worked extensively on research projects around web accessibility [Brudvik,
WebAnywhere, WebInsight, WebInSi
tu] and in tactile graphics [Jayant]. His most recent
project is his MobileAccessibility project which has the goal of developing accessibility
applications for smart phones, particularly for blind, low
vision and deaf
blind people.
He has worked with youn
g people with disabilities since 1994 in various educational
capacities. From 1994 to 2005 he directed a one week workshop as part of the DO
Summer Program at the University of Washington [DO

In the workshop he engaged
high school students with di
sabilities in programming tasks related to Conway's famous
Game of Life. During the one week
students learned the fundamentals of
programming and developed their o
wn variants of the Game of Life

that performed
various image processing and computer

graphics tasks.

Using the funds provided his
2004 Presidential Award for Excellence in Science, Mathematics and Engineering
Mentoring (PAESMEM) he created the Vertical Mentoring Workshop for the Blind in
Science, Technology, Engineering, and Mathematics
that was held in the summer of

This workshop brought in about 50 high school, college, and graduate students
together with professional
, all blind

where each group mentored the group below it in

In breakout sessions students learned about se
advocacy, math accessibility, tactile
graphics, and other topics.

In 2007 and 2009 Dr. Ladner lead the computer science track
at the National Federation of the Blind (NFB) Youth Slam [NFBYS] where, each year,
about 15 blind students learned about compu
ter programming using the vehicle of
programming instant messaging chatbots

[Bigham chatbots]

In 2008, Dr. Ladner lead a
workshop for blind children, ages 8
12, at the NFB Junior Science Academy.

At this

students learned about computer algorit
hms by performing physical activities
along the lines of the well know Computer Science Unplugged curriculum


Dr. Ladner has worked on research projects with over 15 students with disabilities and
written research papers with 10 of them.

erestingly, Dr. Ladner has never worked
extensively with the age group targeted by this proposal, namely preschool and
elementary school.

This will present some challenges, no doubt.

However, because he
has worked with youth aged 8 and above, particularl
y college students with disabilities,
he has gained a great appreciation for what these students can achieve, which is
comparable to their non
disabled peers.

Chandrika Jayant, Graduate Student

Ms. Jayant has been working with Dr. Ladner
since 2006, f
irst on the Tactile Graphics
Project [Jayant], then on the MobileAccessibility Project.

Ms. Jayant will be completing
her Ph.D. dissertation in June 2011 on interaction techniques that enable blind people to
use cameras.

She is very experienced in smart
phone development

developed an
application to take a picture of a barcode and return the name of the product.

She spent the summer of 2007 at IBM Tokyo Research Lab working with Dr. Asakawa
and her Accessibility Group, worked at the

Vertical Mentoring Workshop for the Blind at
UW in 2006, and co
led a workshop at the NFB's Youth Slam in 2009. Although Ms.
Jayant does not have a disability

has worked closely with blind and deaf
the past four years.

Shiri A
zenkot, Graduate Student

Ms. Azenkot is a first
year graduate student with a
commitment to research

accessibility and human
computer interaction.

In the summer of 2009 she held an AT&T
internship where she developed an accessible iPhone application to
help people with low
vision navigate.

This year she has been concentrating on smart phone applications for the
Android platform for blind, low
vision, and deaf
blind people. Ms. Azenkot has low
vision and is legally blind.

David Tseng, Graduate Student

r. Tseng is currently an Apple employee who has been admitted to the Ph.D. program
in Computer Science and Engineering at the University of Washington.

It is likely he
will defer coming to UW until spring 2011.

Before going to Apple, Mr. Tseng worked in
the accessibility group at Microsoft.

At Apple he was involved in the development of
VoiceOver , which is a screen reader for the iPhone

and in accessibility features for the
iPad. Mr. Tseng is blind.