Indoors Pervasive Computing and Outdoors Mobile Computing for Cognitive Assistance and Telemonitoring

wirelessguideΚινητά – Ασύρματες Τεχνολογίες

24 Νοε 2013 (πριν από 3 χρόνια και 6 μήνες)

67 εμφανίσεις

Indoors Pervasive Computing

and Outdoors Mobile Computing

for Cogniti
ve Assistance

and Telemonitoring

Sylvain Giroux, Hélène Pigot, and André Mayers

Department of ComputerScience,

Université de Sherbrooke

2500 boul. Université, Sherbrooke
,
Canada J1K 2R1

{
Sylvain.
Giroux
,
Helene.Pigot
,

Andre.Mayers
,
}@USherbrooke.ca

http://www.dmi.usherb.ca/~sgiroux/domus/

Abstract.

P
eople suffering from cognitive deficits


Alzheimer disease, schi
z-
o
phrenia, brain injuries


are often obliged to live in medical institutions
.
P
e
r-
v
a
sive computing and
mo
bile applications
are key technology
that may help

them
to
stay at home.
They can be at the root of information systems as cogn
i-
tive a
s
sistance and telemonitoring. Inside a

smart house
, cognitive assistance

could
a
l
leviate

cognitiv
e impairments by
giving

personalized environmental
cues.
B
e
sides

telemonitoring

could inform
relatives and medical
staff

of the
disease ev
o
lution and
could

alert them in case of emergency. But
cognitively
impaired pe
o
ple

must not be confined in their home.

Cognitive a
s
sistance

and
telemonitoring must be available
outdoors too
.

This paper
sketches an infr
a-
structure
and pr
o
totypes
making the most of
indoor pervasive compu
t
ing and
outdoor mobile computing
to achieve

cognitive assistanc
e

and telemonito
r
ing.

1

Int
roduction

People suffering from cognitive impairments

Alzheimer disease, schizophrenia,
brain injuries..
.


often have no choice

other
than living in medical institutions.
S
mart
ho
us
es

[10]

and mobile applications, e.g.
activity compass
[5]
,
can play a cen
tral

role

in keeping them at home
.
Inside a

smart house, cognitive assistance

could
alleviate

co
g
nitive impairments by
giving

personalized environmental cues
to

assist
cognitively
impaired
people
in
their

activities of daily living (ADL)
.
Besides

a

smart h
ouse could
inform
rel
a
tives and medical
staff

of the evolution of the disease and
could

alert them
in case of emergency. But
cognitively impaired people

must not be confined in their
home.
Cognitive assistance

and telemon
i
toring must be available
outdoors
too
.


The required

pervasive and mobile infrastructure
for indoors and outdoors ADLs
calls for

specialized
middleware and frameworks for
managing
distributed systems,
dynamic generation of personalized user interfaces for heterogeneous devices, alg
o-
rithms
based on artificial intelligence techniques, localization and geo
-
referenced
i
n
formation.
Though goals and means are different for indoor and ou
t
door cognitive
assistance, both need to rely on common models and a similar infr
a
structure (Fig. 1).
This paper

first
sketches the distributed
layered
architecture we developed

(§2)
.
A
p-
plication layer is
then
described using t
wo prototypes for indoors

3
)

and

one

prot
o-
type for ou
t
doors


4
)
.

It completes with hardware and middleware layers (§5).


Fig.
1
.

C
ogn
i
tive

a
s
si
s
tance
and telemonito
r
ing share the i
n
fr
a
structure and sim
i
lar
mo
d
els.

2

A Layered Distributed
Architecture

The
distributed
architecture

(Fig. 2)
is logically
organized
in three layers: har
d-
ware, middleware and application
[7, 8]
. The a
pplication layer offers services
like

cogn
i
tive assistance
and

telemonitoring,
as well as supporting services like plan
recognition and

user modeling. The hardware layer
deals with

hardware

and low
-
level
software: se
n
sors, devices, domestic appliances, wir
eless ne
t
wor
ks
... The

middleware
layer
copes with hardware

heterogeneity

through dedicated middleware and
fram
e-
works. Middl
e
ware enables to link the application layer services to the hardware.
Frameworks are a structuring
set of tools and software componen
ts that help to d
e
ve
l-
op mobile and pervasive services.
Since i
ndoors and outdoors

se
r
vices face

similar
issues
,
a common generic part
has been

identified

and extracted into the middleware
and fram
e
works
.

3

Indoors Assisted Cognition

Building and deploying i
nformation systems for assisted cognition and telemonito
r-
ing offer

more
options

inside a house than outside. We
gain

more control over the
devices installed and we can expect to collect
more accurate, redundant

and rich i
n-
formation.
In addition w
e
may have

richer mean
s

of interactions with people.
At the
same time
, heterogeneity and multiplicity of devices and networks raise challenging
issues. Security and fault tolerance calls for highly distributed and redundant systems.
Pervasive computing
[
12
]
seemed

t
he right approach. Thus to explore pervasive co
m-
puting and to put the base of the infrastructure, two prot
o
types have been designed.
The first prototype (§
3.1
) focus on distributed computing, code migration, and spo
n-
taneous networks. It also shows how to c
oordinate and intertwine heterogeneous ne
t-
works and protocols to personalize the environment and to deliver the info
r
mation
wherever the user is and whatever the device is. The second prototype (§
3.2
) conce
n-
trates on structure and deployment of distributed

computations to pr
o
vide for local
decisions and feedbacks whilst global perspectives and decisions are also
occu
r
ring
.


Fig.
2
.

A three layers
architecture
.

In practice, the overall infrastructure is highly distributed
and any device may
host pieces of code belon
g
ing to each layer at the same time.

4

Indoors
Assisted Cognition

Building and deploying information systems for assisted cognition and telemonito
r-
ing offers more
potential

inside a house than outside. We got more control over the
de
vices installed and we can expect to collect more precise
, redundant,

and
complex

i
n
formation. We could also hope
t
o get richer mean
s

of interactions with people. At
the same time, heterogeneity and multiplicity of devices and networks raise challen
g-
ing is
sues. Security and fault tolerance calls for highly distributed and redundant sy
s-
tems. Pervasive computing
[
12
] seemed

the right approach. Thus
two pr
o
totypes have
been designed and implemented
to explore pe
r
vasive computing and to put the base of
the infr
astructur
e
. The first
prototype

(
§
4.1
)
focus on

distributed co
m
puting,
code
migration, and spontaneous networks. It also shows how to

coordinate and intertwine
heterog
e
neous
networks
and protocols
to personal
ize

the
environment
and

to deliver
the info
r
mati
on wherever the user is and whatever the device
she uses
. The second
prototype (§
4.2
)

concentrates on

structure

and
deployment

of

distributed
comput
a-
tion
s
. The aim was

to provide for local decisions and feedbacks

whilst
su
p
porting
global perspec
tives and
g
lobal
decisions
.

4.1

Communication with

the User

in a Pervasive Home

I
n a smart home
,
a

patient can move from room to room. Each room may contain a
wide variety of
fixed

or mobile devices. Some devices may serve to interact with the
patient while other may sol
ely
collect

low
-
level information
, e.g. sensors
.
To

be effe
c-
tive
a cognitive assistant

must know the
p
a
tient
position, at least in which room she
stands, the devices she can use, the
devices
features, the information they provide.
Finally code may
need to

migrate to be deli
v
ered on specific devices

since devices
may join or leave the
room
.
We thus
designed and implemented

a pervasive system

that investigates

these
points
. Th
e system has to keep a list of messages and a
p
poin
t-
ments to send to the patient at
given times. At appropriate time, the
system
has to

localize
in which room
the
patient

is,
identify the
appropriate
device
s
, and
“display”
the

mes
sage

to
the patient
. It could then serve as a basis for a cognitive assistant to

recall

to a patient
an appoin
tment or
a task to perform
.

In the current prototype (Fig. 4), d
evices used are

Ethernet
-
based
PCs,
Wifi and
Bluetooth enabled
PDAs, a
B
luetooth

enabled printer
, a WiFi router, wireless mov
e-
ment detectors, an X10
1

infrared (
IR
)

receptor plugged into the h
ouse electrical sy
s-
tem, an X10 decoder / encoder plugged into a wall
electrical
outlet
.
Th
e

information
system
is divided into five cooperating subsy
s
tems
:



Jini
: Jini
[
1
]

provides http servers to deploy Java classes

and services
, Pho
e
nix for
persistence a
nd robustness, a lease service

for resource management
, and service
discovery. Jini
also makes available
spontaneous networking

f
a
cilities.



the Domus federation
:
t
he Domus federation contains services and Java code sp
e-
cific to the application
s deployed ins
ide the house
. It relies on Jini
.



messaging service
s
:
messaging

service
s allow

to deliver a message to somebody
and to
receive

an acknowledge.
For

telemonitoring,

a nurse

may use it

to et a

co
n-
firm
ation

an action has been done by a patient.
For

assisted co
gnition, an agenda
contai
n
ing monitored tasks

may use it to recall a patient to take her medication.
The
lifec
y
cle of message is summarized as follows



a request
to send a message to a given user
is issued to the mes
sage server
;



the message server search
es

in the Domus federation the client devices
able to

display
the
me
s
sage
, and forwards the message to them;



w
hen an acknowledge is received or
when the

message
becomes

meaningless
(for instance
if
out
dated)
,
the
message

is

withdrawn from the Domus federation
.
In any case, m
essages are
always
garbage col
lected
.



X10 service
: this

service act
s

as bridge
s

between the electrical system
used as a
ne
t
work

and
other

devices.
A part of this service is connected through a serial port
to the X10 interface device
.
The ot
her part makes available X10 signals to
other
Domus
services
. This server enables reading and writing X10
message on the
ele
c-



1


X10 is a communications "language" that allows compatible p
roducts to talk to each other
using the existing electrical wiring in the home

[
2
].

trical

wires.
The decoder linked to the server can write X10 signal to the ele
c
trical
wire
s

to control X10 devices, for instance li
ghts.
Domus
services only need to re
g-
ister and to specify the
X10
codes that are relevant for them.
They will be not
i
fied
whenever
corresponding

X10 messages are
transmitted over

the electrical sy
s
tem.



l
ocalization services
:

Wireless movement d
e
tectors are

used to infer the
room in
which a user is located
. These movement detectors gene
r
ate X10 signals. They
communicate their states via infrared (IR) to an X10 r
e
ceptor plugged into the
house electrical system. This receptor transmits the signals through the
house ele
c-
tr
i
cal system.
States of movement detectors can then be analyzed to determine
wether the patient is entering or exiting a room
.



the agenda service
: this service is a pervasive application that keep sessions alive
from device to device. It acts i
n collaboration with messaging services
. It keeps
t
r
ack of appointment
s

or task
s

to do. For instance it can be used to spe
c
ify the
ADL
s

a patient has to perform


taking pills,
breakfast, diner
, etc. The agenda will
remind them to the patient as the day pr
oceeds.

Order of connection
of services
is not an issue. Services and devices can join and
leave the sy
s
tem in any order.
The system is fully decentralized and used on cheap
commercially avai
l
able devices.
When necessary, services, e.g. t
he agenda system
,

can

migrate from device to d
e
vice preserving the state of the
user session
.


Fig.
3
.

Current
prototype
implement
a-
tion

of the
pervasive
agenda
system
.

4.2

A
D
istributed
I
nfrastructure for
P
lan
R
ecognition

Once the ho
me

pervasive informati
on systems can interact with the user and collect
information from sensors, plan recognition
[
11
]

is

the next issue to tackle. Plan reco
g-
n
i
tion is
compulsory

to
achieve cognitive assistance or telemonitoring. Indeed
the
sy
s
tem

must
assess what the patient
is doing
,

what
she

i
n
tends to

do
, wha
t she has
done, how she did it.
Hence
the

second prototype
focuses on

distributed plan recogn
i-
tion
for

maintain
ing

and updat
ing

a
patient
model
.
D
istributed plan re
c
ognition
is
driven by
predefined scripts. Th
e

prototyp
e is based on Ep
i
talk [
6
].

Plan to recognize are represented as task graphs.
These graphs describe
ep
i-
phyte
2

hierarchical
multi
-
agent sy
s
tems
.
For each node there is a

matching

agent.
A
gents of the plan recognition system are distri
b
uted across the device
s of the house,
their

host.
The agents choose the devices to graft onto according to generic descri
p-
tions of devices owned by the environment model
, e.g. sensor type
.

As an activity
pr
o
ceeds, isolated events are detected by sensors.
E
vent
s

are captured
by
the
low
-
level
agents grafted on sensors. These events feed the

reasoning

pro
c
ess
.
Each node may
have a rule base implemented in JESS
3
.
Each node/agent has a local view on the cu
r-
rent situation and owns part of the user model.
When enough, relevant or synth
etic
information is available,
the agent

sent it

to
its

hierarchical superior

node
.

As info
r-
m
a
tion goes u
p
ward,
higher
-
level agents

get a broader view on what is going on.
As
data
is

percolating up, local dec
i
sions are taken and ADLs scripts are inferred.
When a
handicap situation occurs, personalized cues are generated. In case of risk, the tel
e-
m
o
nitoring system sends messages to the medical staff and the relatives.
So a
t each
node, reasoning can generate pieces of a
d
vice for cognitive assistance or can
in
form
medical staff

for telemonitoring.
Thus if a failure occurs, the system will co
n
tinue to
operate partially.
Information is also kept in a remote data
base a do
c
tor may consult.

Apart localized cues, we use plan recogn
i
tion
for
assessment of illness
evol
ution

based on

the
global deterioration
scale
for assessment of primary degenerative deme
n-
tia.[
9
]
.
Since our
plan recognition
systems
enables to detect loss of me
m
ory,
trouble

in
performing task in the proper order…
it

can infer when there is a level chang
e

in the
deterioration scale
. When this occurs, an email is sent to medical staff.
This pr
o
totype
will be integrated with the pervasive messaging systems to build the sought for co
m-
mon framework for cognitive assistance and telemonitoring.

5

Outdoors

For ob
vious reason
s
, patients must
not
be constrained to stay at home. However
outdoors
offer

a completely different
environment

for cognitive assistance and tel
e-
m
o
nitoring. Much less control is possible over devices available. Much fewer devices
and sources of
information can help than in an indoors controlled setting as
is
patient’s
home. Nonetheless

monitoring tools
are as well
helpful

when the patient is not at
home.
Mobile

computing

can
provide the adequate answers
.
A prototype is
currently

under development

for the monito
r
ing of
people suffering of
schizophrenia

(Fig. 5)
.

For them such tools can make the difference between being obliged to stay in the
hospital or being allowed to go outside.

The
pr
o
totype
system comprises a PDA, a
GPS and a
CDMA
network card
.
It has two purposes:
1)
gather ecological data
,

and
2)
a
n
ticipate and help during crisis.

Schizophrenic people usually see their psychiatrist once a month. Often answer
s

of
the patient are vague o
r not representative of the true situation
. Furthermore m
edic
a-
tion has often harsh side
-
effects, and doses must be fine
-
tuned with care. Our prot
o-
type enables a p
atient
to

note
facts
valuable

for

a better cure, e.g. occurrence of
sym
p-



2

Epiphyte plants, as ivy, are plants that grow on to other plants, called hosts.

3

JESS

is a rule engine and scripting environment for the Java platform.

toms

and their intensity
… So the psychiatrist
will have

real ec
o
logical values
when he
meets

the

patient.
In the prototype,

data are also u
p
loaded to a database

thanks to
wireless networking
,
so
the psychiatrist
can adjust the dose of medica
tion

according to
the observation and the inte
n
sity of side
-
effects
.

When they are in crisis
, patients tend to fo
l
low characteristic paths or to stay in
front of specific locations, for instance churches.
The GPS
co
n
nected to the PDA
enables

to fol
low

the
patient mov
e
ments

and to detect
crisis
. Since we
know at that
moment

the patient location
, w
e somebody
can go there and
help
her
. A me
s
saging
service and task monitoring has also been
implemented

to help staff monitor patients
ADLs
.
D
octors and nurses
are notified
when pr
e
cise
tasks (
medication
,
meals
, etc
.
)
have been or not pe
r
formed. If the ta
sk has not been pe
r
formed, medi
cal staff can send
a me
s
sage.

6

Hardware, Middleware, and Frameworks

Hardware involved in indoors pervasive systems and outdoors mobile systems is
truly different in nature and processing power (Table 1). Fortunately there are
la
n-
guages, middleware and frameworks useful in both cases. First we used
Java
as a
basis, in particular
for mobile code.
Mobile code enables algorithms to move from
device to device. At low
-
level, communication through networks use Ethernet, Wifi,
Blu
e
toot
h and X10. At a higher level, spontaneous networking is managed by Jini.
Finally Ep
i
talk is used for distributed plan recognition and to structure distri
b
uted
reasoning and assistance for adequate deployment on this
distributed

infrastructure. In
the near
f
u
ture, we will integrate relevant parts of E
-
mate
[3]

and MORE
[4]
. E
-
mate
lies over Jini and supplies generic solutions for the d
e
ployment of mobile personalized
geo
-
referenced services. E
-
mate also supports the core of the personalization pro
c
ess.
MORE
dynamically generates user interfaces on fat, thin or web
-
based clients.

Table
1
.

Some d
ifferences between indoors and outdoors hardware and
systems
.


Outdoors

Indoors

Devices

PDAs cellular phones…

pma汬 numbe爠of dev楣is

pen獯牳Ⱐ䑩a楴a
l TV…

䡩eh numbe爠of dev楣is

Network

Not available or low
-
bandwidth

Ethe
r-
net
;Bluetooth,WiFi,
RF
;
X10;

Localiz
a
tion

GPS;

geographic information sy
s
tems

Motion detectors;

Localiz
a
tion systems;

7

Conclusion

We presented three prototypes that sketched how c
ognitive assistance

and telem
o
n-
itoring can assist
people

suffering from cognitive impairments and lessen the burden
on caregivers. Apposite interventions are generated using a model describing the pe
r-
son, the ADLs and the environment. Indoors a distributed

architecture is pro
c
essing
Fig. 5

The
PDA interface for
gathering ecological data for
schiz
o
phreni
c people
.

data acquired from sensors spread around the house. Data are percolating up to the
distributed
cognitive assis
tant
. Outdoors cognitive assistance relies on small portable
device and GPS
-
based localization
. Pervasive computing an
d mobile compu
t
ing pr
o-
vided the basis of the architecture of the information systems.

8

References

1.

Arnold, K., ed.,
The Jini Specifications
,
Addison
-
Wesley, 2nd Edition, 2000
.

2.

Boone, K,
Using X10 for Home Automation
,
http://www.kevinboone.com/x10.html

3.

Carboni, D. et al.
,

e
-
MATE: An open architecture to support mobility of users
,

Fifth Intn’l
Baltic Conf. on Databases and Information Systems, June 3
-
6
th
, 2002 Ta
l
linn, Estonia.

4.

Carboni,
D.
,
et al.,
Interactions Model and Code Generation for J2ME App
lications
,
4th Int.
Symp. on Human Computer Interaction with Mobile Devices

Mobile
,

HCI 2002
,
It
aly.

5.

Kautz, H. et al.
An Overview of the Assisted Cognition Project
, AAAI 2002, wor
k
shop.

6.

Paquette, G., et al.,
EpiTalk, generating advisor agents for exi
sting information sy
s
tems
, J
.

of
AI

in Education, AIED Society, AACE, vol. 7, no 3
-
4, 1996.

7.

Pigot H., et al.
,

The role of intelligent habitats in upholding elders in residence
.
5th Intl
Conf. on Simulations in Biomedicine
, 2003, Slovenia.
WIT Press.

8.

H. Pigot, et al.
,

The intelligent habitat and everyday life activity support
.
5th intern
a
tional
conference on Simulations in Biomedicine
, Slovenia, April 2003.

9.

Reisberg B,
et al.

The Global Deterioration Scale for Assessment of Primary Degenerative
Dem
entia.
Am J Psychiatry

1982; 139: 1136

1139

10.
Rialle .V, et al.
An experimental Health Smart Home and its distributed Internet
-
based
I
n
formation and Communication System
.
MEDINFO 2001,

IOPress. pp. 1479
-
1483.

11.
Schmidt, C. F.;
et al
..
The plan recogni
tion problem: an intersection of psychology and
artif
i
cial intelligence
.
Artificial Intelligence

1978
.
11:45

83.


12.
Weiser, Mark,
The Computer for the 21
st

Century
, Scientific American, 1991.