A portable wireless eye movement-controlled Human-Computer Interface for the Disabled

agerasiaetherealΤεχνίτη Νοημοσύνη και Ρομποτική

24 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

88 εμφανίσεις





A portable wireless eye movement
-
controlled
Human
-
Computer

Interface for the Disabled


ABSTRACT


A portable wireless eye movement
-
controlled Human
-
Computer Interface
which can be used for the disabled who
have motor paralysis and who cannot
speak in multiple applications (such as
communication aid and smart home

applications) is described here. This
Interface consists of four major parts:


(1)

Surface electrodes


(2)

A two
-
channel amplifier



(3)

A laptop (or a micro
-
processor)


(4)

A

ZigBee wireless module.


Horizontal and vertical Electro
-
Oculography (EOG) signals are measured
using five surface electrodes placed on
the head .The vertical electrodes are
placed about 1.0 cm above the right
eyebrow and 2.0 cm below the lower lid
of the

right eye, the horizontal electrodes
are placed 2.0 cm lateral to the each side
of outer canthi and the last electrode is
placed on user’s forehead to serve as a
ground. The two
-
channel amplifier is
comprised of instrumentation amplifiers,
band
-
pass filte
rs and shift circuits. The
EOG signals are sampled at the rate of
250Hz and then sent to a laptop or a
micro
-
processor for signal processing
which is based on the method of
mathematical morphology to recognize
the direction of eye movements and
voluntary e
ye blink. The ZigBee wireless
communication technology, which is
proved to be reliable, low
-
power and
cost
-
efficient, is used in the portable
interface. This interface

provides a
flexible method for the disabled to
improve the life quality.


I. INTRODUCTIO
N


Persons
with

severe diseases, such as
amyotrophic

lateral sclerosis (ALS),
brainstem stroke, brain or spinal

cord
injury, cerebral palsy, muscular
dystrophies, multiple

sclerosis, etc., have
difficulty conveying their intentions and

communicating with o
ther people in daily
life. With the

development of Human
-
Computer Interface (HCI), methods

have
been developed to help these people for

communication. Unlike traditional HCIs (a
keyboard, or a

mouse, etc.), modern HCIs
have played an important role in

the
area
of rehabilitation.


HCIs can be divided into cortical (all
interfaces that

exploit information
collected from the human brain cortical

relays) and non
-
cortical (all interfaces
that do not access the

signals generated
by the human cortex directly)
.
In
the
present study, we describe a novel
portable wireless

eye movement
-
controlled HCI for the disabled. This

interface is a real
-
time communication
control system based

on EOG signals.
There are two main differences between

our system and others mentioned a
bove:
(1)

D
esigning and

implementing a
mathematical morphology method to

preprocess original EOG signals.

(2)


I
ncluding a wireless

module based
on the ZigBee protocol to increase the
scope

of applications (communication aid,
smart home
.

applications,
etc.) of this system.


II
. SYSTEM OVERVIEW


The system we have developed consists of
four major

parts:

1. Five surface electrodes

2. A

two
-
channel amplifier,

3. A
la
ptop (or a micro
-
processor)

4. A
ZigBee

wireless module.




Fig 1:

Overview of the EOG
-
based
wireless Human
-
Computer Interface
.


Fig
1 is the schematic diagram of this

system and the whole system adopts the
star topology which

is also used.
In this
system, horizontal and vertical EOG
signals are

measured by five surface
electrodes placed around eyes.

After a
two
-
channel amplifier, the EOG signals
are sampled

at the rate of 250 Hz and
then sent to a coordinator node

which is
connected with a laptop or a micro
-
processor

through ZigBee wireless
communication technology. The

software
on the laptop or micro
-
processor
recognizes the

direction of eye movement
and voluntary eye blinking.

Programs
(typewriter, patient assistant software,
etc.) in

laptop or remote devices (TV,
lamps, telephone, etc.) can be

controlled
by the recogni
zed results.



III.
ELECTRODES AND THE
PRINCIPLE


The cornea of the eye is electrically
positive relative to the

retina of the eye
and the potential is slowly varying when

eyes move. The standing potential can be
measured by

electrodes placed around
the
eyes. The EOG value varies

from 0.05
-
3.5 mV with a frequency range of about 0
-
100 Hz.

In this paper, there are five
electrodes in all which are

classified as
horizontal, vertical and reference
(ground)

electrodes. As showed in Fig
1,
the vertical electrode
s are

placed about
1.0 cm above the right eyebrow and 2.0
cm

below the lower lid of the right eye,
the horizontal electrodes

are placed 2.0
cm lateral to the each side of outer canthi.
And

the last electrode is placed on user’s
forehead to serve as a

groun
d.


If the eyes move left, horizontal EOG
(HEOG) signal

which is the difference
between signals collected by

electrode
HEOL and HEOR acquires a positive
voltage

value. If the eyes turn right, HEOG
signal changes into a

negative voltage
value. Identically,
if the eyes move from

the central position towards upside,
vertical EOG (VEOG)

signal which is the
difference between signal collected by

electrode VEOU and VEOL acquires a
positive voltage

value. If the eyes move
downside, VEOG signal changes

into a
negat
ive voltage value. An eye blinking
can be

described by EOG signals as a peak
in VEOG but a flat in

HEOG. We can
distinguish the voluntary and involuntary

blinking by the value and duration of the
peak mentioned

above.


IV.
AMPLIFIER


The horizontal and ver
tical eye
movement signals

captured by the
electrodes were then transmitted to a

two
-
channel amplifier whic
h consists of
(1) preamplifiers

(2) band
-
pass filters


(3) shift circuits

(4) right
-
leg driven

circuits

(5) power supply.


The preamplifier is a
micro
-
power
instrumentation

amplifier


for accurate
and low noise differential signal

acquisition. The gain of the preamplifier is
set to be 21 with

a single external
resistor. The band
-
pass filter (0.01
-
41 Hz)

is provided with two Sallen
-
Key filters
(One

second
-
order

high
-
pass filter and
one fourth
-
order low
-
pass filters). The

following circuits are secondary amplifier
with variable gain

and shift circuit to
transform the signal level into the range

Fig
1
:

Overview of the EOG
-
based
wireless Human
-
Computer

Interface.

Fig 2:
EOG signals during eye movement
and blanking. (a) HEOG signals.

(b) VEOG
signals.

of 0 V to 3 V for adapting the
following analog
-
to
-
digital

converter
(ADC). Right
-
leg driven circuit connected
with

the reference electrode is used to
redu
ce the common
-
mode

components in
the signal. Power for the board is supplied
by

one common 6V battery, which is then
transformed into
±

3.3 V with AMS1117
and MAX828 respectively.


V
. WIRELESS MODULE


The Wireless module takes responsibility
for transmitti
ng

two
-
channel EOG signals
from one node attached to the

user’s body
to the coordinator node connected with
the

laptop. Meanwhile, the coordinator
can send messages to

other remote
controllers (TV, lamp, telephone, etc). The

ZigBee wireless communication
t
echnology, which is

proved to be reliable,
low
-
power and cost
-
efficient, is used

in
this system.


The module is established using CC2430
,

which is a true

System
-
on
-
Chip solution
specifically tailored for IEEE

802.15.4 and
ZigBee applications. The CC2430
c
ombines

RF transceiver with an
industry
-
standard enhanced 8051

MCU,
32/64/128 KB flash memory, 8 KB RAM
and many

other powerful features. At the
transmission node, analog

EOG signals
from amplifiers are sampled at the rate of

250Hz and transmitted. At the
reception
node, EOG signals

are transported to
laptop with RS232
-
USB interface for

signal processing. In the prototype
software, the protocol is

based on a
ZigBee stack called MSSTATE_LRWPAN

which implements a ZigBee subset
wireless stack. The

program in C
C2430 is
based on this protocol completely.



VI
. EOG SIGNAL PROCESSING


The

method is based on the mathematical
morphology (MM),

differential and
integral algorithms to recognize the

direction of eye movement and voluntary
blinking. VEOG

signals are used
to detect
up/down movement and voluntary

eye
blinking, while HEOG signals are used to
detect

left/right movement.




Fig 3:

The flowchart of EOG signal
processing.


MM Algorithm
: The method of MM is
widely used in

ECG signal pr
ocessing and
other fields
.
It provides a

good way to
remove drift and magnify feature of the
signal.

The operators of MM are dilation,
erosion, opening and

closing
.

Opening
generally smoothes the

contour of an
object, breaks narrow isthmuses, and
eliminate

thin protrusions. Closing
also
tends to smooth sections of

contours and
which generally fuses narrow breaks and
long

thin gulfs, eliminates small holes, and
fills gaps in the

contour. In our work, we
only used symmetrical sequences

as the
structuring element. The result of VEOG
sig
nals after

MM filter.

Differential Algorithm:
The VEOG
signals, after MM

filter, are feed into the
differential module implemented.


Y(n)



(





)




(



)












---------------
> (1)

Where
x (n)
is the VEOG signals after MM
filter, and
y (n)

is the result after the
differential module.


Integral Algorithm:
The difference of
original VEOG

signals (delay
2N
points,
N
is the length of the structuring

element in
MM algorithm) and signals after MM
filter
can be

used for eye blinking recognition.
Because the peak value of

voluntary
blinking is much larger than involuntary
blinking,

we can distinguish those two
kinds of blinking by the

integral module
using (2) and the threshold.


(

)



(



)








------------
> (2)

Where
x(n)
is the difference of original
VEOG signals

(delay
2N
points) and
signals after MM filter,
y(n)
is the

result
after the integral module.

Decision
Module: In Fig. 4,
S1
,
S2
and
S3
are the
r
esults

by the methods mentioned above.
Threshold1
is the

voluntary eye blinking
threshold,
Threshold2
is the

involuntary
eye blinking threshold,
Threshold3

is the

movements (up/down) threshold, and
Threshold4

is the

movements (left/right)
threshold. We can

distinguish eye

blinking (voluntary and involuntary) and
eight
-
direction

movement through these
thresholds.


Fig 4:
Example of VEOG signal processin
g.


VII.
APPLICATION SOFTWARE
TEST


We have developed two application
programs to test this

system: the
typewriter and the patient assistant
software.

The typewriter user interface is
showed in Fig. 7a. Users

make the cursor
move up, down, left and right to select a

letter in the table. The letters selected (by
voluntary eye

blinking) are showed above
the ta
ble. The patient assistant

software is
showed in Fig. 7b. In this application,
users

move the cursor by eight
-
directional eye movements, and

the size
of icon selected is enhanced. At the same
time, the

LED which indicates the
direction of eye movement is

l
ighted by
the controlling of the remote ZigBee
module.

These two applications indicate
that the portable wireless

eye movement
-
controlled we developed is doable. The

performance (bit rate and latency) of this
system can be

calculated as the following
parag
raphs.



VIII. BIT RATE


The bit rate of this s
ystem is calculated.
Where
B
is the bit rate per selection,
N
is
the possible

choices, and
P
is the
accuracy. Bit rate per minute of the

system can be obtained from
B
multiplied
by the number of

selections in
a minute.


B=
lo



+
lo


p+(1
-
p)log2

(1
-
p)/(n
-
1)


----------
> (3)


Several subjects were asked to test the
patient assistant

software (
N
in this
application is 8). They got accuracies all

above 80% and could make at
least 10
selections per minute.

Therefore, the bit
rate of this system is above 17.17 bits per

minute.



IX. LATENCY


The Latency of this system is related with
the method of

real
-
time signal processing.
The delay time of mathematical

morphology, differential, and integral
algorithms is
T1
,
T2

and
T3
, respectively.
The delay time of each algorithm is

calculated by using (4). Thus,
T1
= 0.4 s,
T2
=
T3
= 0.08 s.


T
=
N
/
Fs

---------
>
(4)


Where
T
is the delay time,
N
is the
number
of sampled

points delayed, and
Fs
are the sampling frequency.



T
total
=T1+max (T2, T3)

----------
> (5)


The sampling frequency is 250 Hz, and
100 sampled

points are delayed during
the method of mathematical

morphology.
20 sampled points are delayed both
in the

processing of differential and integral
algorithms. Thus, the

total time delay
Ttotal
calculated by using (5) is 0.48 s.


X.
FACTORS INFLUENCING
PERFORMANCE


The EOG signals are mostly concentrated
on the low

frequency, especially near the
DC
component where lots of

useful
information locates. Therefore, the cut
-
off
frequency

of the high
-
pass filter should be
set as low as possible (0.01

Hz in this
system) otherwise the eye movement
signals

would decline rapidly rather than
be hold for a long t
ime.

Because of the MM
method, the influences of the drift and

other noise were reduced. However, slow
involuntary

blinking (duration above 0.8
s) would be recognized as eye

movement
(Up), which is a mistake. So when using
this

system, slow involuntary bli
nking
should be performed as

few as possible.


XI
. IMPROVEMENT OF THE
SYSTEM IN FUTURE


Four thresholds which are measured in
advance were

set manually at the

initialization stage of the software.

Clearly, it is time
-
consuming and may
result in unnecessary

errors. Later

evelopment should make the thresholds
set

automatically through a test program
before use. Meanwhile,

the thresholds can
be updated during test period

auto
-
adaptively accordin to the user’s current
state. For

instance, the amplitude of the

EOG signals would change

slowly in the
latter stage because of fatigue.

The
current signal process program is
implemented on a

laptop which also
provides user interface on the screen. If

we do not need user interfaces (e.g.
controlling remote

devices), th
e process
program can be carried out only in a

micro
-
processor which is integrated in
the coordinator node.

Then the processed
results were sent wirelessly to the remote

device which is attached with a ZigBee
reception node.