Technology for the Aging

Technology for the Aging

A Collaborative Effort


Edward Riseman

Allen Hanson

Roderic Grupen

Erik Learned
-
Miller

Phebe Sessions

Julie Abramson

Mary Olson

Candy Sidner

UMassAmherst

Supported by the National Science Foundation

MERL

Collaborative Effort


Collaborative Research Effort


Social scientists and computer scientists


Elderly representatives and their stakeholders


caregivers


family


local and state agencies, etc.


Two Part Talk: Social Science and Technology

Enhancing Control and
Empowerment for the Elderly
through Assistive Technology

Social Science Components

November 4, 2005

Washington, D.C.

Dr. Phebe Sessions

Smith College School for Social Work

Outline


Issues in collaboration


Theoretical framework: ecosystemic and


social constructionist


Gerontological literature and traditions


Research plan


Issues in Collaboration

UMass Computer Sciences/Smith SSW



Why social work?


Pragmatism


Commitment to underserved populations


Community
-
building skills


Challenges of collaboration


Novelty and not knowing


Different languages


Theoretical framework


Ecosystemic and Social Constructionist


Strengths perspective


Collaboration, dialogue


Qualitative methods in research

Gerontological literature and traditions


Aging and the deficit perspective: “declinism”


Positive aging and core significance of control


Learning issues and aging


Social networks and aging


Ethical considerations


Research plan


Qualitative methods


Year One: Focus groups with elderly,


families, and service providers


Year Two: Implementation of technology


in common room


Year Three: Revision of technology based


on Year Two findings

Technology Components


Umass Vision and Robotics groups


Professor Emeritus Edward Riseman (PI)


Professor Allen Hanson (co
-
PI)


Professor Roderic Grupen (co
-
PI)


Mitsubishi Electronic Research Laboratory


Dr. Candace Sidner

Technology Overview

ISSUES:


privacy


access


client control

Graphics


Scripts


Vision


Speech

Speech

Synthesis

Tracking

Speech

Understanding

Lifestyle/Event

Coordination

Activity

Modeling

Conversational

Interaction

Assisted
-
Living Clientele

Family

Medical Structure

Interface

Family/Community

Interface

Human Services

Interface

MERL

UMASS

SMITH

Interaction:

Appointment Management

IA
:

Hi,

Maisie
.

Senior
:

Hi
.

IA
:

later

this

morning

at

11

am

you

have

a

dentist

appt
.

Senior
:

Ok

IA
:

You

will

have

to

take

the

bus

so

you

need

to

leave

here

an

hour

before

and

walk

to

the

bus

stop
.

That

means

leaving

about

10

am
.

Senior
:

I

want

to

leave

later
.


IA
:

Well,

the

bus

takes

20

minutes
.

How

long

do

you

need

to

get

to

the

bus

stop?

Senior
:

15

minutes

IA
:

The

bus

comes

every

15

minutes,

so

when

do

you

think

you

should

leave?

Senior
:

10
:
15

IA
:

Okay
.

How

about

I

remind

you

at

10

am

to

get

ready?

Senior
:

Good
.

IA
:

Good

bye

for

now
.

Technical Goals


An effective API to the client’s environment:


Independent Living


Social/family


Medical


Financial


Social services


Legal


…..


Plug and play for rapid development and deployment


Interface driven user interactions


System interfaces


Nimble, reconfigurable, robust, minimally intrusive,
controllable

The Client
-
Care Sandbox

Nimble & Reconfigurable

Component Technologies



Fault tolerant script based interfaces: Collagen plus GUI


Minimal intrusion


Robust


Speech and speech understanding


Command and control


Voice Analysis
-

e.g. stress


Visual interfaces


Virtual Reality


Tracking


Change Detection


Environment


Object locations


Motion/expected tracks, deviation from normal activity


People


Gait and postural models


Speech patterns


Collagen: Collaborative Agents


Principals: Candace L. Sidner and Charles Rich




Java middleware for collaborative interface agents


User and agent collaborate to complete predefined tasks


Speech or text
-
based discourse with agent



mouse, touch screen, others


Provides friendlier, more familiar interface for


inexperienced users


Many existing applications


http://www.merl.com/projects/collagen




Mitsubishi Electric Research Laboratory

Collaborative Interface Agent

* SharedPlans per Grosz, Sidner, Lochbaum, Kraus, et al.

communicate

interact

interact

observe

observe

plan tree

focus stack

*

Collagen

Collagen Example:

Calendar Application Dialog Fragment

1.
User: “Let’s create a new appointment.”

2.
Agent: activates Create Appointment window.

3.
User: enters the appointment’s date and description.

4.
User: “What’s Next?”

5.
Agent: “What time is the appointment?”

6.
User: “11 AM.”

7.
Agent: sets the time of the appointment to 11 AM.

8.
Agent: “I’m going to save this appointment.”

9.
User: “OK.”

10.
Agent: “We have created a new appointment!”

Speech Recognition


Available engines


IBM Viavoice


Microsoft Speech SDK


CMU Sphinx Engine 2 and 4


Mode of application


Continuous speech recognition mode


Speaker dependent


Low accuracy


Voice commands mode


Speaker independent


Higher Accuracy


Still susceptible to noise

Managing Speech Recognition
Technology


Reduce noise


Pressure
-
gradient/Noise
-
canceling Microphone


Sound card/USB pod


Continuous recognition


Training is essential


Do not expect high accuracy


Approach


Design dialog script carefully


Build in a fault tolerant mechanism


Always have a “take me to an operator” option


Constrained uses:


voice command mode


voice stress analysis


Local Experimental Environment


“Smart” Room


Distributed sensor network


Human and robotic residents


Complex environment


: camera locations

Tracking


Distributed sensor network


Sensor management


Selection, reasoning across sensors


Real
-
Time fault
-
tolerant approach


Fault containment units


Hardware, Software


Hierarchical


Wrappers around a resource policy


Augmented by fault and context


reporting mechanisms


Dynamic restructuring


Supports


Activity modeling


Identification


Unusual event detection

Tracking Supports Virtual Worlds

Immersive Virtual Worlds


Provide several levels of interaction styles


Client control over interaction style and level of
access of remote visitors (privacy)


Avatar plus virtual world


Avatar plus real world


Person plus virtual world


Person plus real world


(Registered)

Transparent

Objects

Rapid Viewpoint Changes


Camera context switches e.g. surveillance


Disconcerting, difficult for user


“Snap” Transitions

“VR Smoothed” Transitions

Mixing Virtual and Real Worlds


Events in real world mapped to virtual world


Temporal events synchronized to virtual world


Transparency
-

look through objects


Form of access control


Conclusions


Starting up


Conjectures on the future


Empowering the elderly: they decide


Social etiquette enforced through virtual worlds


Client controls level of access and degree of privacy


Built on


Modern social science theory


Existing technology base


Client
-
care Sandbox model for rapid prototyping


Particulars determined through focus groups