Cognitive User Interfaces:

cabbageswerveΤεχνίτη Νοημοσύνη και Ρομποτική

7 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

76 εμφανίσεις






Cognitive User Interfaces:

An Engineering Approach

Machine Intelligence Laboratory

Information Engineering Division

Cambridge University Engineering Department

Cambridge, UK

Steve Young

2

ICASSP Plenary April 2009
©

Steve Young

Outline of Talk


Introduction: what is a cognitive user interface?


Example: a simple gesture
-
driven interface.


Human decision
-
making and planning.


Partially Observable MDPs


an intractable solution?


Scaling up: statistical spoken dialogue systems.


Conclusions and future work.

3

ICASSP Plenary April 2009
©

Steve Young

What is a cognitive user interface?


Capable of reasoning and inference


Able to optimize communicative goals


Able to adapt to changing environments


Able to learn from experience

An interface which supports intelligent, efficient and robust
interaction between a human and a machine.

4

ICASSP Plenary April 2009
©

Steve Young

Example: A Simple Gesture
-
Driven User Interface

Swipe

Scroll

Forward

Scroll

Backward

Delete

Photo

Swipe

A photo sorter

5

ICASSP Plenary April 2009
©

Steve Young

Interpreting the Input

Backwards

Delete

Forwards

Backwards

Delete

Forwards

angle

P(angle)

Decision Boundaries

6

ICASSP Plenary April 2009
©

Steve Young

Pattern Classification

angle

P(angle)

G=forwards

G=delete

G=backwards

Conf(G=backwards)

7

ICASSP Plenary April 2009
©

Steve Young

Flowchart
-
based Decision Making

Confidence
?

Gesture
?

backwards

Move back

>= Threshold

Do Nothing

< Threshold

8

ICASSP Plenary April 2009
©

Steve Young

What is missing?


No modeling of uncertainty


No tracking of belief in the user’s required goal


No quantifiable objectives hence sub
-
optimal decision making

9

ICASSP Plenary April 2009
©

Steve Young

Modeling Uncertainty and Inference


Bayes’ Rule

Reverend Thomas Bayes

(1702
-
1761)

)

data

(
)

belief

(

)

belief
|
data

(
)

data
|
belief

(
P
P
P
P

new belief

data


s
t

1

s
t

o
t

a
t

1
old belief

action

Bayesian

Network

b(s)

s

move

back

?

b’(s)

s

Inference via

Bayes Rule

10

ICASSP Plenary April 2009
©

Steve Young

Optimizing Decisions


Bellman’s Equation


V
*
(
b
)

max
a
r
(
b
,
a
)

P
(

o

o

|
b
,
a
)
V
*
(

b
)






Richard E Bellman

(1920
-
1984)

Reward=

+

+

+

+




r
(
b
1
,
a
1
)

r
(
b
2
,
a
2
)

r
(
b
T

1
a
T

1
)

r
(
b
T
,
a
T
)


(
b
1
)

a
1


(
b
2
)

a
2


(
b
T

1
)

a
T

1


(
b
T
)

a
T
Policy

s
1

s
2

s
T
-
1

s
T

a
1

a
2

a
T
-
1

a
T

o
1

o
2

o
T
-
1

o
T

b
1

b
2

b
T
-
1

b
T

Reinforcement Learning

11

ICASSP Plenary April 2009
©

Steve Young

Optimizing the Photo
-
Sorter

Swipe

Scroll

Forward

Scroll

Backward

Delete

Photo

Swipe

{ scroll
-
forward, scroll
-
backward, delete
-
photo }

User’s Goal

(states)

{ go
-
forward, go
-
back, do
-
delete, do
-
nothing }

System

Action

+1

+1

+5

0

Rewards

-
20

All other:
-
1

Iteratively optimize policy to maximize rewards …

12

ICASSP Plenary April 2009
©

Steve Young

-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
3
Performance on the Photo
-
Sorting Task

10%

20%

30%

40%

50%

0%

Reward

Effective Error Rate

Flow
-
charted Policy

Fixed Policy

and Model

Adapted Policy

and Model

Training

Point

13

ICASSP Plenary April 2009
©

Steve Young

Is Human Decision Making Bayesian?

Humans have brains so that they can move.

So how do humans plan movement? ….

14

ICASSP Plenary April 2009
©

Steve Young

A Simple Planning Task

Prior

Observation

Kording and Wolpert (Nature,
427
, 2004)

15

ICASSP Plenary April 2009
©

Steve Young

Models for Estimating Target Location

0

1

2

Lateral shift (cm)

Probability

Prior

Posterior

Observation

Kording and Wolpert (Nature,
427
, 2004)

0

1

-
1

0

1

2

0

1

-
1

0

1

2

0

1

-
1

0

1

2

Deviation from

Target

True lateral shift

Prior ignored

Bayesian

Min Error Mapping

16

ICASSP Plenary April 2009
©

Steve Young

Practice makes perfect

17

ICASSP Plenary April 2009
©

Steve Young

Bayesian Model Selection in Human Vision

Inventory

Test

“Which is more familiar?”

Train

“Watch these!”

Orban, Fiser, Aslin, Lengyel (Proc Nat. Academy Science,
105
, 2008)

Not visible to

subjects

18

ICASSP Plenary April 2009
©

Steve Young

Partially Observable Markov Decision Processes


Belief represented by distributions over states and

updated from observations by Bayesian inference


Objectives defined by the accumulation of rewards


Policy which maps beliefs into actions and which

can be optimized by reinforcement learning


s


s


o

a

r
(
b
,
a
)


(
b
)

b

Principled approach to handling uncertainty and planning


Humans appear to use similar mechanisms

So what is the problem ?

19

ICASSP Plenary April 2009
©

Steve Young


The state and action sets are often very large.


Real
-
time belief update is intractable.


The mapping is extremely complex


Exact policy optimization is intractable.

Scaling
-
up



(
b
)

a
Applying the POMDP framework in real world user interfaces

is not straightforward:

20

ICASSP Plenary April 2009
©

Steve Young

Spoken Dialog Systems (SDS)


Database

Recognizer

Semantic

Decoder

Dialog

Control

Synthesizer

Message

Generator

User

Waveforms

Words

Dialog

Acts

Is that near the

tower?

confirm(near=tower)

negate(near=castle)

No, it is near

the castle.

21

ICASSP Plenary April 2009
©

Steve Young

Architecture of the Hidden Information State System

Belief

Update


b
Speech

Understanding

Speech

Generation

User


o

g

a
Dialog

Policy

Williams and Young (CSL 2007)

Young et al (ICASSP 2007)

Two key ideas:


States are grouped into equivalence classes called partitions

and belief updating is applied to partitions rather than states


Belief space is mapped into a much simpler summary space

for policy implementation and optimization

Summary

Space


ˆ
b
Heuristic

Mapping


ˆ
a
b(s)

s

POMDP

22

ICASSP Plenary April 2009
©

Steve Young

The HIS Belief Space


s





g
,

u
,

h

Each state is composed of three factors:

User Goal

User Act

Dialog History

Young et al (CSL 2009)

find(venue(hotel,area=east))

find(venue(bar,area=east))

find(venue(hotel,area=west))

….

find(venue)


˜
u
1
˜
u
2
...
˜
u
N












User

Request

User

Informed

System

Informed

Grounded

Denied

Queried

Initial

×

×

User goals are grouped into partitions

HIS Belief Space

Beliefs update is limited to the most likely members of this set.

23

ICASSP Plenary April 2009
©

Steve Young

Master <
-
> Summary State Mapping

Master space is mapped into a reduced summary space:

find(venue(hotel,area=east,near=Museum))

find(venue(bar,area=east,near=Museum))

find(venue(hotel,area=east)

find(venue(hotel,area=west)

find(venue(hotel)

....etc

b
P(top)

P(Nxt)

T12Same

TPStatus

THStatus

TUserAct

LastSA

b
ˆ

a
Heuristic

Mapping

act type


ˆ
a
Policy


Greet

Bold Request

Tentative Request

Confirm

Offer

Inform

.... etc

VQ

confirm( )

confirm(area=east)

24

ICASSP Plenary April 2009
©

Steve Young

Learning with a simulated User

Learning by interaction with real users is expensive/impractical.

A solution is to use a simulated user, trained on real data.

User

Simulator


includes

ASR error

model

Dialog

Corpus


o

a
Belief

Update

Heuristic

Mapping


b
Summary

Space


ˆ
b
Dialog

Policy


ˆ
a
Q
-
Learning

Random action



)
(
random
P
Schatzmann et al (Knowledge Eng Review 2006)

25

ICASSP Plenary April 2009
©

Steve Young

HIS System Demo

26

ICASSP Plenary April 2009
©

Steve Young

HIS Performance in Noise

Success Rate (%)

Error Rate (%)

0

5

10

15

20

25

30

35

40

45

95

90

85

80

75

70

65

60

55

HIS

MDP

Hand
-

crafted

(HDC)

Simulated User

27

ICASSP Plenary April 2009
©

Steve Young

Representing beliefs

An alternative is to model beliefs directly using dynamic Bayesian nets …

Beliefs in a spoken dialog system entail a large number of so
-
called

slot variables. Eg for tourist information:

Cardinality is huge and we cannot handle the full joint distribution.

P(venue, location, pricerange, foodtype, music, …)

In the HIS system, we threshold the joint distribution and just record the high

probability values. The partitions marginalize out all the unknowns.

But this is approximate, and belief update now depends on the

assumption that the underlying user goal does not change.

P(venue=bar, location=central, music=jazz) = 0.32

P(venue=bar, location=central, music=blues) = 0.27

P(venue=bar, location=east, music=jazz) = 0.11

etc

28

ICASSP Plenary April 2009
©

Steve Young

Modeling Belief with Dynamic Bayesian Networks (DBNs)

Decompose state into DBN, retaining only essential conditional dependencies

g
type

g
food

u
type

u
food

h
type

h
food

a

u

o

g’
type

g’
food

u’
type

u’
food

h’
type

h’
food

a’

u’

o’

g

u

h

Time t

Time t+1

Eg restaurant

Eg chinese

Thomson et al (ICASSP, 2008)

29

ICASSP Plenary April 2009
©

Steve Young

Factor Graph for the Full Tourist Information System

Factor graphs are very large, even with minimal dependency modeling.

Hence


need very efficient belief updating


need to define policies directly on full belief networks

30

ICASSP Plenary April 2009
©

Steve Young

Bayesian Update of Dialog State (BUDS) System

Thomson et al (CSL 2009)

Belief update depends on message passing

x
1

x
M

….

x

f



x

f


f

x


f

x
(
x
)

f
(
x
)

x

f
x
m

x

(
x
m
)
sum over all combinations

of variable values

P(food)

food

Z
1

Z
2

Z
3

Grouping possible values

into partitions greatly

simplifies these summations

Fr

It

….

31

ICASSP Plenary April 2009
©

Steve Young

Belief Propagation Times

Network Branching Factor

Time

Standard

LBP

LBP with

Grouping

LBP with

Grouping &

Const Prob

of Change

32

ICASSP Plenary April 2009
©

Steve Young

Policy Optimization in the BUDS System

Thomson et al (ICASSP, 2008)

Summary space now depends on forming a simple characterization of

each individual slot.

Define policy as a parametric

function and optimize wrt
θ

using Natural Actor Critic

algorithm.



'
)
(
.
)
(
.
'
)
,
|
(
a
b
b
a
a
e
e
b
a






]
)
(
,

,...
)
(

,
)
(
[
)
(
,*
,
,
T
T
T
T
b
b
b
b
a
location
a
food
a
a





Each action dependent basis function is separated out into slot
-
based

components, eg

0

1

0

0

0

1.0 0.0 0.0

0.8 0.2 0.0

0.6 0.4 0.0

0.4 0.4 0.2

0.3 0.3 0.4

1
st

2
nd

Rest

slot belief

quantization

action

indicator

function

33

ICASSP Plenary April 2009
©

Steve Young

BUDS Performance in Noise

Error Rate (%)

Simulated User

BUDS

MDP

Average Reward

34

ICASSP Plenary April 2009
©

Steve Young

Conclusions


Future generations of intelligent systems and agents will need

robust, adaptive, cognitive human
-
computer interfaces


Bayesian belief tracking and automatic strategy optimization

provide the mathematical foundations


Human evolution seems to have come to the same conclusion


Early results are promising but research is needed

a)
to develop scalable solutions which can handle very large

networks in real time

b)
to incorporate more detailed linguistic capabilities

c)
to understand how to integrate different modalities: speech,

gesture, emotion, etc

d)
to understand how to migrate these approaches into industrial

systems.

35

ICASSP Plenary April 2009
©

Steve Young

Credits

EU FP7 Project: Computational Learning in
Adaptive Systems for Spoken Conversation

Spoken Dialogue Management using Partially
Observable Markov Decision Processes

Past and Present Members of the CUED Dialogue Systems Group

Milica Gasic, Filip Jurcicek, Simon Keizer, Fabrice Lefevre,

Francois Mairesse, Jost Schatzmann, Matt Stuttle, Blaise Thomson,

Karl Weilhammer, Jason Williams, Hui Ye, Kai Yu

Colleagues in the CUED Information Engineering Division

Bill Byrne, Mark Gales, Zoubin Ghahramani, Mate Lengyel,

Daniel Wolpert, Phil Woodland