TAUCHI – Tampere Unit for Computer-Human Interaction

slipperhangingAI and Robotics

Nov 14, 2013 (3 years and 6 months ago)

68 views

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Ambient Intelligence

(Ubiquitous Computing)


Kari
-
Jouko Räihä

Roope Raisamo


Department of Computer Sciences

University of Tampere, Finland

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Contents


Definitions


Prototypes and use scenarios


Influential past examples


Research challenges

TAUCHI


Tampere Unit for Computer
-
Human Interaction

What is Ubiquitous Computing?

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Ubiquitous?


American Heritage Dictionary:

being or seeming to be everywhere at the same
time; omnipresent.


Merriam
-
Webster Dictionary:

existing or being everywhere at the same time;
constantly encountered; widespread


Ubiquitous computing?


In Finnish: jokapaikan tietotekniikka,

läsnäoleva tietotekniikka


TAUCHI


Tampere Unit for Computer
-
Human Interaction

Ubiquitous Computing

Mark Weiser, Xerox PARC 1988


“Ubiquitous computing enhances computer use by
making many computers available throughout the
physical environment, but making them
effectively invisible to the user.”

Source: [Weiser, 1993a]

TAUCHI


Tampere Unit for Computer
-
Human Interaction

What Ubiquitous Computing Is NOT


Ubiquitous computing is not virtual reality, it is
not a Personal Digital Assistant (PDA) such as
Apple’s Newton, it is not a personal or intimate
computer with agents doing your bidding.

Not virtual reality?


Unlike virtual reality, ubiquitous computing
endeavors to integrate information displays into
the everyday physical world. It considers the
nuances of the real world to be wonderful, and
aims only to augment them.

Source: [Weiser, 1993b]

TAUCHI


Tampere Unit for Computer
-
Human Interaction

What Ubiquitous Computing Is NOT


Ubiquitous computing is not virtual reality, it is
not a Personal Digital Assistant (PDA) such as
Apple’s Newton, it is not a personal or intimate
computer with agents doing your bidding.

Not a PDA?


Unlike PDAs, ubiquitous computing envisions a
world of fully connected devices, with cheap
wireless networks everywhere; unlike PDAs, it
postulates that you need not carry anything with
you, since information will be accessible
everywhere.

Source: [Weiser, 1993b]

TAUCHI


Tampere Unit for Computer
-
Human Interaction

What Ubiquitous Computing Is NOT


Ubiquitous computing is not virtual reality, it is not a
Personal Digital Assistant (PDA) such as Apple’s Newton, it
is not a personal or intimate computer with agents doing
your bidding.

Not personal agents?


Unlike the intimate agent computer that responds to
one’s voice and is a personal friend and assistant,
ubiquitous computing envisions computation primarily in
the background where it may not even be noticed.
Whereas the intimate computer does your bidding, the
ubiquitous computer leaves you feeling as though you did
it yourself.

Source: [Weiser, 1993b]

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Visions of Weiser


Idea of personal computer is misplaced


Vision of laptop machines, dynabooks and
knowledge navigators is only a transitional step:
these machines cannot make computing an
integral, invisible part of life


Creating computers which vanish into the natural
human environment

“Most profound technologies are those that
disappear.”

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Ubiquitous Computing


Phase I


The Mainframe Era




Phase II


The PC Era





Transition: Internet and distributed computing


Phase III


The UC Era

Computer

User

Computer

User

Computer

User

TAUCHI


Tampere Unit for Computer
-
Human Interaction

“Moore’s Law of User Interfaces”

“The number of computers per user will double
every two years.”

Source: [Vertegaal, 2003]

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Major Trends in Computing

Source: Weiser and Brown, 1998

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Situation Today?


How many personal computing devices do you
regularly use?

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Characteristics of Ubiquitous Computing


Calm technology


Suggests but does not interrupt


As natural as thermostats or street lights


From interacting to “living with” computers


Depends on our peripheral attention


Easy change between focus and periphery


User takes control by focusing


Example: car engine, we note problems from strange
sounds

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Pervasive Computing


Closely related, almost synonymous term


Pervasive: “that pervades or tends to pervade”


To pervade: “to become diffused throughout
every part of”


Mobility, not just access to invisible computers


seamless mobility


session mobility


adaptation to local capabilities


environment senses instead of explicit user interaction


from small dumb devices to PCs

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Still Some Other Related Concepts


Proactive Computing


Augmented Reality


Mobile Computing


Intelligent Environments


Ambient Intelligence



We will use the short term “Ubicomp”

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Information Appliances


Information Appliances are
computers that are specialized for a
certain task


Can work together


Can be based on a well
-
developed
underlying infrastructure

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Ubicomp vs. Information Appliances


Ubicomp


Computers invisible, embedded in the environment


Peripheral


Integrated


Information Appliances


Computers embedded in specialized tools


Concrete, simple


Functional


In practice, the difference is smaller: appliances
become cognitively unnoticeable in time


e.g., a wrist watch

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Example Ubicomp Projects


Ambient Devices


EU IST Disappearing Computer


Project Aura, CMU


Office of real soon, UNC


Augmented surfaces, Rekimoto


Easy Living, Microsoft


Oxygen, MIT


Portolano, Univ. of Washington


Endeavour, Berkeley


CoolTown, HP Labs




TAUCHI


Tampere Unit for Computer
-
Human Interaction

Scenario


Smart room: EasyLiving
(Microsoft)



Critical issues?

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Critical Issues: EasyLiving


Privacy


Others intervening even if you don’t want to


Anyone can access your data


Possibility to turn tracking off


Robustness


People working closely together


Maintaining the object mapping


Handling break
-
downs


Flexibility


Too restricted model, e.g. various lighting options


Accuracy


Does it anticipate the intentions correctly?


Scaling to new possibilities

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Why Is Take
-
Off of Ubicomp Taking So Long?


Working systems a big challenge


No killer
-
app? Not a good reason!


Compare with the mouse:


Mouse cheap; ubicomp components cheap but a full
system may be expensive


Mouse used to control all applications, not a killer
-
app;
ubicomp environments need to become more common
in general


Even the mouse took 20 years!

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Past, Present and Future of

Ubiquitous Computing

G.D. Abowd and B.D. Mynath, Charting past, present, and
future research in ubiquitous computing.
ACM Transactions
on Computer
-
Human Interaction (TOCHI) 7
, 1 (March 2000),
29
-
58.

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Weiser vs. Georgia Tech


Weiser


Augmentation of physical environment


Invisibility


Infrastructure + variety of devices


Abowd & Mynatt: focus (here) on applications and
interaction


Natural interfaces


Context
-
aware computing


Capture and access

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Additional Themes Covered


Scaling


Terminal size


Number of users


Time


“Everyday Computing”


Social implications


TAUCHI


Tampere Unit for Computer
-
Human Interaction

Natural Interfaces


Goal: make use of natural human actions


Speech


Pen (and keyboard!) input


Perceptual interfaces


Gestures


Gaze
-
based interfaces


Tangible interfaces


Makes applications accessible also for the
disabled

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Example of Natural Gestures: DreamSpace

Source:

http://www.research.ibm.com/natural/dreamspace/

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Natural Interfaces: Challenges


First class data
-
types


Gaze data: gaze points


fixations


events


Freeform ink


characters (OR: clusters of strokes)


Error
-
prone interaction


Error reduction


Error discovery


Error correction


Reusable infrastructure

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Context
-
Aware Computing


Early systems


Active Badge (Olivetti Research)


Xerox ParcTab


Context = mostly just location


Even the location tells a lot!


A variety of technologies


Infrared sensors


Ultrasound detectors


Bar codes


RF
-
ID tags


Vision
-
based techniques

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Active Badge System

Badge sensors

located on the ceiling,

in corridors, ...


buttons

Infra
-
red

receiver

Infra
-
red emitter

light led signs

Mr Som E Body

Firma Ltd

beeper

Size:

60x60x8 mm

Weight:

40 gr

0,1 seconds

every 10 seconds


Locating people (and devices)


Room
-
level accuracy


Badges worn by people periodically
transmit infrared messages


Sensors with known location

Source: [Want et al, 1992]

TAUCHI


Tampere Unit for Computer
-
Human Interaction

One Possible Use


Location
awareness
used to
increase
group
awareness

TAUCHI


Tampere Unit for Computer
-
Human Interaction

The Bat Ultrasonic Location System


Highly accurate
indoor positioning:
95% of readings
within 3cm


Bat device emits
short pulse of
ultrasound


Ceiling mounted
sensor array


Trilateration to
compute position


TAUCHI


Tampere Unit for Computer
-
Human Interaction

Sentient Computing


Use sensors to construct model of the environment


Shared view of the world between system and user

The world as seen by the users

The world as seen by the system

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Object Identification: Augmented Interaction

Jun Rekimoto: Navicam


Objects tagged with bar
codes


Objects identified with
portable bar code readers


Physical world augmented
with additional information


Observe:


Static id, dynamic content


User tracking object


need
to carry along a special
device

Source: http://www.csl.sony.co.jp/person/rekimoto/navi.html

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Natural + Context
-
Aware


Augmented Reality

Source: [Feiner et al., 1993]

TAUCHI


Tampere Unit for Computer
-
Human Interaction

What is Context?


Who


But who else?


What


E.g., walking or standing


but more is needed


Where


Needs to be coupled with other contextual elements


When


Typically used for discrete index information


Pattern detection underutilized


Why


Needs more advanced sensors


and still a challenge

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Sensor Fusion


Helps to create a more accurate

representation of context


Example: sensor fusion for

identity detection


EasyLiving: Vision +

Static Binary Pressure Sensors


Other options?


Vision + Speech


Vision + Speech + Footfall


Research at TAUCHI


Identity detection

based on footfall


Ambient group awareness

using sounds of footsteps


Physically interactive

games

Magic Carpet

Source for video: http://web.media.mit.edu/~joep/SpectrumWeb/captions/Carpet.html

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Past, Present and Future


Applications and interaction


Natural interfaces


Context
-
aware computing


Capture and access


Everyday Computing


Social implications


Challenges in evaluation

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Capture and Access


Capturing a public experience


Early systems: audio capture


Boom in the 1990’s: electronic whiteboards


Capture for individual use


Example: Audio Notebook


Why is this ubicomp?


Multiple devices, each suited for the task


Technology embedded in the environment

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Classroom 2000 (


eClass), Georgia Tech

Source: http://www.cc.gatech.edu/fce/c2000/overview/

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Ubicomp Lecture Room


Remote sensing


Turn lights on automatically


Highlight items on screen just by pointing


Physical alarms to wake up sleeping students


People identification


Identify students from speech / bioidentification / button press on
a personalized device


Identify teacher from fingerprint / login


Allow students unlock the door using id cards / RFID tags


Show names of students to teacher


Context
-
awareness


Automatically put mobile phones in silent mode

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Information Access 1/2


Desks with embedded computers (or with wireless
laptops)

+
Duplicate screen info on local monitor for better
visibility

+
Take personal notes

+
Send questions to teacher

+
Support for collecting ideas produced in groups


Distraction factor


Polling devices for obtaining class opinion


Desks with embedded microphones


Activated by student activity or gestures

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Information Access 2/2


Show transcribed questions on the second screen


Provide automatic translations


Electronic whiteboard for annotations


Smart display outside the room


Show vacant seats


Show changes in lecture schedule


Remote attendance


Video conference facility


Automatic capturing of the lecture

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Adaptation


Lighting adjusts


to the activity (less light during videos)


to the seating of students


Sound volume adjusts to seat location, surrounding noise,
and level of sound input (from teacher or video)


Air conditioning adapts to the size of the group


Set settings automatically after teacher identification


The door


stays unlocked while there are people in the room


front door automatically locks 15 minutes after start of the lecture


Seats adjusting to the size of the student

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Capture & Access Challenges


Compelling applications


Family events


Life capture


Scale: quality capture without expensive
configuration


Segmentation of streams


Synchronization of multiple streams


Finding the suitable form of interaction for direct
access


Summarization


Annotations and revisions

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Past, Present and Future


Applications and interaction


Natural interfaces


Context
-
aware computing


Capture and access


Everyday computing


Social implications

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Synergy Amongst Themes


Designing for activities implicit in ubiquitous
availability of computing



Natural:


Tangible interfaces


Roomware


Context
-
aware: Audio Aura, Nomadic radio


Capture and access: Remembrance agent


TAUCHI


Tampere Unit for Computer
-
Human Interaction

Audio Aura


Goal: to provide serendipitous information, via
background auditory cues, that is tied to people’s
physical actions in the workplace.


Scenario 1: email junkies.


Provide information on amount of new email, who is it
from, what is it about.


Information provided when the user enters a cafeteria.


Scenario 2: office encounters.


Provide subtle information on whether someone is
present in the office (compare to seeing the bag of
someone).

Source: [Mynatt et al., 1998]

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Type of Information

Music

Voice

Nothing new

A little (<5 new)

Some (5
-
15 new)

A lot (>15 new)

Rich

Message

“You have no
mail.”

“You have
n

new messages.”

“You have
n

new messages.”

“You have
n

new messages.”

High, short bell

melody.

High, longer

melody.

Lower, longer

melody.

Longest melody.

A single gull cry.

A gull calling

a few times.

A few gulls

calling.

Gulls squabbling.

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Remembrance Agent

Source: Rhodes & Maes, 2000

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Research Directions in Everyday Computing


Designing a continuously present interface


Appliances?


Agents?


Wearable computers?


Presenting information at different levels of the
periphery of human attention


Connecting events in the physical and virtual
worlds

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Social Issues


Access rights


Secure storage


Conquering invisibility


Users in control


Reciprocity


Changing history? (editing notes or records
afterwards)


Qualitative, overview information vs. raw data

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Security, Privacy, Trust


What data do I wish to expose? To whom?


Who can presently access my data?


How can I retract data exposed?


Who am I communicating with?


How do can the privacy of my communication and
communication patterns?


Who do I trust as a source of information?


How do I convince others that I am trustworthy?



How to make systems simultaneously secure and usable?

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Research Challenges for Ubicomp 1/4


From homogenous to heterogeneous networks


Network structure changes dynamically


Need protocols for network nodes to communicate


Need mechanisms for service discovery


From static to mobile devices


Possibilities become limited (some)


Requirements are higher (some)


Centralization vs. distribution


Decision making (“instant control”) needs to be
distributed in the network


Configuration (“pattern control”) and functioning need
to be monitored centrally

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Research Challenges for Ubicomp 2/4


Location tracking


Many technologies available: IR, RF
-
ID, Bluetooth, GMS,
GPS, …


Becoming more and more accurate


Applications must adapt to failures in location
information


Time synchronization


Events from various sources must be merged properly


Communication speed


Depends on technology used


How long can the system wait for a reply?

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Research Challenges for Ubicomp 3/4


Energy consumption


Infrastructure should be able to generate its own
energy


Use of natural human communication


Speech, pointing, gestures, mimics, body language, …


Needs advanced sensor technology


Sensor fusion for increased accuracy


Scalability


Must allow proliferation of users in same space

TAUCHI


Tampere Unit for Computer
-
Human Interaction

Research Challenges for Ubicomp 4/4


Awareness


What information is desirable?


How should it be represented?


Privacy


Avoiding Big Brother effect


Context


What does context consist of?


Geographic location, civil location, categorical, behavioral, …


How can it be modelled?


How is the information obtained?