Always the Right Answer

heavyweightuttermostΜηχανική

5 Νοε 2013 (πριν από 3 χρόνια και 9 μήνες)

138 εμφανίσεις

Implications of Adaptive vs. Adaptable
UIs on Decision Making:

Why “Automated Adaptiveness” is Not
Always the Right Answer

Christopher A. Miller

Harry B. Funk

Robert P. Goldman

John Meisner

Peggy Wu

Originally presented at

1
st

Annual Conference on Augmented Cognition

Las Vegas, NV


July 24
-
28, 2005

AAAI
-
SS on Interaction Challenges for Automated Assistants

26
-
28 March, 2007

6 November 2013

2

Adaptive vs. Adaptable

Opperman, 1994

“Adaptive Systems” are those that automatically
adapt to user needs

“Adaptable Systems” are those in which the user
adapts system behavior to his/her needs.

“Learning” vs. “Instructing”


6 November 2013

3

Adaptive Automation as “Meta
-
Automation”

World

State

Displays,

Controls &

Systems

State,

Needs

Adaptive Automation removes a human from an aspect
of the control loop … with what results?

State,

Needs

Adaptive

Approaches

6 November 2013

4

Benefits of Adaptive Automation

Greater performance speed

Greater performance consistency

Improved safety/survivability

Reduced training time

6 November 2013

5

Disadvantages to Adaptive Automation

High levels of automation (which remove operators from
portions of the task/control loop) have been shown to …

Increase

human workload at critical times if not properly
designed [1,2]

Decrease human situation awareness [3,4]

Produce human complacency (~ unwarranted trust) [5]

Result in human skill degradation [6]

Decrease human acceptance [7]; increase human
dissatisfaction [8]

Provide inferior human + machine system performance
(relative to lower levels of automation) [9]



1.
Parasuraman and Riley, 1997

2.
Wiener, 1988

3.
Endsley and Kiris, 1995

4.
Sarter and Woods, 1995

5.
Parasuraman, Molloy and Singh, 1993





6.
Rose, 1989

7.
Miller and Hannen, 1999

8.
Vicente, 1999

9.
Layton, Smith, McCovy, 1994

6 November 2013

In any human + automation system,
a given level of Competency will
entail a tradeoff between


Human Workload


(Un
-
)Predictability to User


Competency

Automation Control

(aka Adaptive)

‘Competency’ is appropriate behavior
in context.

Characterizing the Tradeoff

Competency gets harder with
increases in


Precision of context divisions


Flexibility of behavior



Increasing adaptivenss means either




increasing human workload or



increasing unpredictability,




or both.


Competency

Human Control

(aka Adaptable)

Competency

Competency

Competency

Adaptable

Adaptive

A spectrum of alternatives

6 November 2013

7

So maybe taking the human out of the
loop, even to do the human’s
bidding, isn’t the best idea …


For user acceptance

For overall human + machine system
performance



6 November 2013

8

How do we want to interact with automation?

Would you want/trust an assistant who always handed
you ostensibly right information


with no
input/instruction/clarification from you?

Wouldn’t trust, coordination, SA and maybe performance
be improved by explicit, meta
-
level intent coordination?


All right,
dear, I’ll pick
up some milk.

Turn left
now.

In order to go to the
store, turn left now.

OK, then turn
left now

OK, Car, I want to
go to the store.

6 November 2013

9

Advantages of (more) Adaptable Approaches

Increased awareness of the situation and of system
performance

Better tuning of trust and better automation reliance
decisions (since operators instruct automation how to
behave)

Better skill retention (through allowing the user to
perform tasks when needed or desired)

Better balance of mental workload (if humans are better
assessors of their needs than AugCog systems)

Better human + machine performance (if/when users
make better judgments than automation about what is
needed to complement their abilities).

Leaving the user in charge yields greater automation
acceptance, AND greater sense of being primarily
responsible

in turn yielding greater attention and
concern for the situation and all aspects of system
performance.

6 November 2013

10

It’s not Either/Or …

Adaptable or Adaptive

It’s a SPECTRUM…



Find the right point on it!

6 November 2013

11

A Tale of Two Associates

Pilot’s Associate (1985
-
1991)


Single Pilot


Direct pilot interaction
with associate meant
added workload


Design philosophy
minimized direct pilot
interaction with associate


Moderate user acceptance

The Pilot is

ALWAYS in

charge.

The Effort
required of the
pilot to control
the associate
must be less
than the effort
saved by the
associate

Rotorcraft Pilot’s Associate
(1994
-
1999)


Two Pilots


1/3 of human activity is
crew coordination


Design philosophy
included some direct pilot
interaction with associate


Improved User Acceptance

6 November 2013

12

Crew Coordination & Task Awareness Display


Four buttons to convey major, associate
-
inferred task contexts


Single press overrides = “No, you’re wrong. That’s not what
we’re doing”



Associate ‘gets out of the way’


Press and Hold scrolls through tasks at same level of hierarchy



E.g., Area Recon, Zone Recon, Attack in Force, Hasty Attack, Delay,
Evade, Ingress, Egress, etc.

MISSION

PILOT

ASSOCIATE

COPILOT

AREA

HOVER

MANUAL

6 November 2013

13

Subjective Workload (TLX) Ratings

TLX
subscale
AMEP
mean
CDAS
mean
F-Value
(
df: 1,6)
Mental Demand
61.77
46.25
10.487*
Physical Demand
54.48
40.31
12.042*
Temporal Demand
62.08
45.73
14.061**
Perceived Performance
35.00
42.08
2.429
Effort
62.60
48.54
20.470**
Frustration
52.81
45.63
4.961
*
p<.05 ** p<.01
Workload levels consistently higher for Baseline (AMEP)
than for RPA (CDAS)

Significant differences for 4 of 6 TLX subscales (and
close for the 5th)


6 November 2013

14

CIM Utility and Overrides

1

Of no

Use

Not

Very

Useful

2

3

4

5

Extrem
-

ely

Useful

Of

Use

Of Con
-

siderable

Use

Page Selection

Symbol Selection

Window Location

Pan & Zoom

3.68

3.50

3.00

3.56

Usefulness Ratings

Most crews said CIM Behaviors
were ‘Of Use’ or “Of
Considerable Use’

1

Always

Fre
-

quently

2

3

4

5

Never

Now &

Then

Sel
-

dom

Page Selection

Symbol Selection

Window Location

Pan & Zoom

4.50

2.63

3.25

Pilot
-
reported Frequency of

Overrides/Corrections

3.00

Crews ‘Seldom’ overrode
CIM’s symbol selections, but
‘Now & Then’ overrode other
behaviors

6 November 2013

15

CIM was seen as useful and provided perceived
performance and workload advantages


in spite of


‘Now and Then’ or ‘Frequently’ providing the wrong
information.




Why?

6 November 2013

16

Crew Coordination & Task Awareness Display

1

None

Not

Very

2

3

4

5

Ext
-

remely

Of

Use

Con
-

siderable

Mission Task

Pilot Task

Co
-
Pilot Task

Associate Task

4.4

4.0

Usefulness Ratings

4.3

4.3


Perceived accuracy of LED
Task displays was very high



Comments (and other
ratings) indicated these were
very useful to pilots

MISSION

PILOT

ASSOCIATE

COPILOT

AREA

HOVER

MANUAL

6 November 2013

17

“Playbook” Interfaces for Plan Delegation

NUGGET
: Composable ‘plays’
derived from Task Model
shared between human and
automation software.

APPLICATION:


Plan specification to any level
of specificity (‘calling plays’)

Automatic plan completion
within

human intent

Feedback on plan feasibility

STATUS:
Prototypes for:

UCAV pre
-
mission planning

Tactical Robot Control

Work on:

Unmanned Vehicle Teams

Heterogeneous Urban RSTA

UCAR

Sim scenario specification


HTNP

Analysis

Event

Handling

Control

Algorithms


Instructions

Feedback

Provably correct plans

Provably safe reactions

System control

Shared Task Model

Playbook
GUI

Over

-

watch

Obtain

Aircraft

Scan

Ingress

Egress

X

X

X

Destage

Overwatch

means to

provide continuous relayed

imagery of a fixed or

moving point or area.

Overwatch

Sortie

X

Maneuver

X

Fly

-

to

X

Aircraft:_____________

Target Area:_________

Imagery Start:________

Imagery End:_________

6 November 2013

18

Conclusions

Adaptation is good

Not everything has to be (or SHOULD be) done
by automation

Who does the adapting is important


He who does the adapting is


Better aware of what’s been adapted


Better aware of why it’s been adapted


Probably better served by the adaptation

Humans don’t like to be managed … though
there are some times when they should be

Adaptable and Adaptive techniques can be
complementary, not competing