Action, Complexity and Cognition

blabbedharborAI and Robotics

Feb 23, 2014 (2 years and 9 months ago)

58 views

Action, Complexity and Cognition

Matthias Rauterberg

Industrial Design

Technical University Eindhoven

2003

© M. Rauterberg, 2003

2
/35

Possible Interpretations of 'Information'

1.) 'Information' as a message (syntax)

2.) 'Information' as the meaning of a message (semantic)

3.) 'Information' as the effect of a message (pragmatic)

4.) 'Information' as a process

5.) 'Information' as knowledge

6.) 'Information' as an entity of the world

Ref
: Folberth, O. & Hackl, C. (1986, eds.) Der Informationsbegriff in Technik und Wissenschaft. München: Oldenbourg.

© M. Rauterberg, 2003

3
/35

“Information” for Learning Systems

before reception after reception Author



dof of the decision content of the decision HARTLEY 1928

uncertainty certainty
SHANNON 1949

uncertainty information
BRILLOUIN 1964

potential information actual information
ZUCKER 1974

entropy amount of information
TOPSØE 1974

© M. Rauterberg, 2003

4
/35

context

human

mental model

situation
-
1

situation
-
2

com
-


plexity

positive

incongruity

negative

incongruity

Incongruity and Learning

Incongruity = Complexity

context




Complexity

human

learning

Ref
: Rauterberg, M. (1995). About a framework for information and information processing of learning systems. In: E. Falkenberg,

W.

Hesse &
A. Olive (eds.), Information System Concepts
--
Towards a consolidation of views (IFIP Working Group 8.1, pp. 54
-
69). London: Chap
man&Hall.

© M. Rauterberg, 2003

5
/35

physical operation

feedback

control of

action

goal
-
, subgoal
-
setting

mental operation

task(s)

planning of execution



selection of means

The Complete Action Cycle

synchronisation in time

synchronisation

in space

© M. Rauterberg, 2003

6
/35

s0
d
s1
h
s0
a
s2
F3
s3
CR
s3
F9
s1
_
s3
TAB
s3
F2
s3
_
s3
TAB
s3
_
s3
The Idea

Any human task solving process can be described in a finite
state
-
transition chain, if the task can be described in an ‘action
space’, specified by a
finite

set of states ( ) and transitions [ ].

State description:





s0 : main menu

s1 : modul "data"

s2 : routine "browse"

s3 : "wrong input" state

Action description:





_ : ascii key "BLANK"

a : ascii key "a"

d : ascii key "d"

h : ascii key "h"

CR: carriage return

F2: function key "2"

F9: function key "9"

TAB:tabulator key

© M. Rauterberg, 2003

7
/35

s0
d
s1
s1
h
s0
s1
a
s2
s2
F3
s3
s3
CR
s3
s3
F9
s1
s3
_
s3
s3
TAB
s3
s3
F2
s3
elementary processes
Petri-Net
s0
d
h
s1
a
s2
s3
F3
F9
CR
_
TAB
F2
folding
The Folding Operation in Petri Nets

© M. Rauterberg, 2003

8
/35

Task Description

In the experiment all users had to play the role of a camping
place manager. This manager uses a database

system with a data base consisting of three data files: PLACE,
GROUP, and ADDRESS. All users had to

solve the following four different tasks operating the database
system:

Task 1: "How many data records are in the file ADDRESS, in the
file PLACE, and in the file

GROUP? Find out, please."

The user has to activate a specific menu option ("Datafile" in
module "Info" of the menu interface) and to read

the file size (solutions: PLACE = 17 data records, GROUP = 27
data records, ADDRESS = 280 data records).

Task 2: "Delete only the last data record of the file ADDRESS,
the file PLACE, and the file

GROUP (sorted by the attribute 'namekey')."

The user has to open (sorted according to the given attribute),
select and delete the last data record (file:

PLACE, GROUP, ADDRESS).

Task 3: "Search and select the data record with the namekey
'D..8000C O M' in the file

ADDRESS, and show the content of all attributes of this data
record on the screen. Correct

this data record for the following attributes:

State: Germany

Place number: 07

Remarks: Database system dealer can give a demonstration."

The user must select a certain data record (file: ADDRESS),
update the data record with regard to the three

attributes: State, Place number, Remarks.

Task 4: "Define a filter for the file PLACE with the following
condition: all holidaymakers arrived

on date 02/07/87. Apply this filter to the file PLACE, and show
the content of all

selected data records in the mask browsing mode on the
screen."

The user must define a filter for the attribute "arrival date", apply
the filter to the data file PLACE, and display

the content of each data record found on the screen.

In the experiment all 12 users had to play the role of a camping place manager. This manager uses a database
system with a data base consisting of three data files: PLACE, GROUP, and ADDRESS. All users had to solve the
following four different tasks operating the database system:

Task 1: "How many data records are in the file ADDRESS, in the file PLACE, and in the file GROUP? Find
out, please."

The user has to activate a specific menu option ("Datafile" in module "Info" of the menu interface) and to read the
file size (solutions: PLACE = 17 data records, GROUP = 27 data records, ADDRESS = 280 data records).

Task 2: "Delete only the last data record of the file ADDRESS, the file PLACE, and the file GROUP (sorted
by the attribute 'namekey')."

The user has to open (sorted according to the given attribute), select and delete the last data record (file: PLACE,
GROUP, ADDRESS).

Task 3: "Search and select the data record with the namekey 'D..8000C O M' in the file ADDRESS, and
show the content of all attributes of this data record on the screen. Correct this data record for the
following attributes: State: Germany, Place number: 07. Remarks: Database system dealer can give a
demonstration."

The user must select a certain data record (file: ADDRESS), update the data record with regard to the three
attributes: State, Place number, Remarks.

Task 4: "Define a filter for the file PLACE with the following condition: all holidaymakers arrived on date
02/07/87. Apply this filter to the file PLACE, and show the content of all selected data records in the mask
browsing mode on the screen."

The user must define a filter for the attribute "arrival date", apply the filter to the data file PLACE, and display the
content of each data record found on the screen.

© M. Rauterberg, 2003

9
/35

System Description

The dialog system was the relational data base system ADIMENS version 2.21 with a character
oriented user interface (CUI) running on standard IBM PC's with standard keyboard.

The whole dialog structure is strictly hierarchical organized with three levels:

(1)
the main menu has 7 dialog operations (ordinary ASCII characters chosen from a menu)
to go down to 7 different modules, and 5 function keys with specific semantics;

(2)
at the module level each module has exactly 4 different dialog operations to change to
routines and on average 4.1 (
±
1.7; range: 0
-
5) function keys with specific semantics;

(3)
at the routine level the user has only on average 3.7 (
±
2.9; range: 0
-
10) different function
keys to control the dialog (additionally all ASCII keys and the 4 cursor keys are usable).

The number of all ordinary dialog contexts (main menu, modules, routines) is 1+7*4=29.

But to describe the complete dialog structure with all help, error and additional dialog states we
need at least 144 different system states.

To change from one state to the other the system offers overall 358 different dialog operations
(transitions).

© M. Rauterberg, 2003

10
/35

physical execution

evaluation

and control

goal
-
, subgoal setting

mental execution

task

description

action planning



selection of means

goal

system

state

selected

action

result

Observable Data

© M. Rauterberg, 2003

11
/35

G_2
Start menu
M_3
Mai n menu
Mai n menu.
F fi l e
F_3
User
key press
1
User
key press
Mai n menu
i
User
key press
Info
Info.fi l e
Info.screen1
Info.screen2
d
M_22
M_22
M_22
DB content
di splay
User
key press
Info.screen3
Info
Info.screen1
stopped
Info
Mai n menu
Start menu
Start menu
Start menu
M_11
d
Automati c
transi tion
User
key press
M_22
M_22
M_22
M_11
d
User
key press
M_22
BL
F_10
M_22
M_22
M_11
h
h
User
key press
User
key press
G_2
F_10
User
key press
... continues
... continues
... continues
... continues
Automati c
transi tion
Automati c
transi tion
Automati c
transi tion
Automati c
transi tion
Automati c
transi tion
Mai n menu
marked
Info.fi l e
Info.screen1
Info.screen2
Info.screen3
Info
Info.screen1
Info.fi l e
Info.screen3
Info.screen2
Info.screen1
DB content
di splay
DB content
di splay
DB content
di splay
DB content
di splay
DB content
di splay
User
key press
User
key press
DB content
di splay
DB content
di splay
DB content
di splay
Example of a task solving process

© M. Rauterberg, 2003

12
/35

s0
d
s1
h
s0
b
s2
F3
s3
CR
s3
F9
s1
_
s3
TAB
s3
F2
s3
_
s3
TAB
s3
_
s3
structure
as a
Petri net
s0
d
h
s1
b
s2
s3
F3
F9
CR
_
TAB
F2
FOLDING
observable
process
unknown structure
(e.g., mental model)

?
main menu level
module level
routine level
How to Extract the User’s Mental Model?

© M. Rauterberg, 2003

13
/35

How to measure complexity?





In Computer Science...





algorithmic information

(Solomonoff
-
Kolmogorov
-
Chaitin)





computational universality



computational time/space



according McCabe in graph theory





In Physics...







thermodynamics potentials



long
-
range order



long
-
range mutual information



self
-
similar structures



thermodynamic depth



logical depth



In Psychology...





properties of objects (e.g. valence)



properties of attributes (e.g. ordinality)



properties of cognitive structure (e.g. centrality)

© M. Rauterberg, 2003

14
/35

Net Complexity Metrics

McCabe (1976):

Kornwachs (1987):

Stevens, Myers and Constantine (1974):

[with P=1]

Validation study
:

C
cycle

from McCabe outperforms all other metrics!

state
-
1

state
-
2

transition
-
2

transition
-
1

Simple Petri Net:

Ref
: Rauterberg, M. (1992). A method of a quantitative measurement of cognitive complexity. In: G. van der Veer, M. Tauber, S. B
agn
ara
& M. Antalovits (eds.), Human
-
Computer Interaction: Tasks and Organisation
--
ECCE'92 (pp. 295
-
307). Roma: CUD.

T = number of transitions

S = number of states

© M. Rauterberg, 2003

15
/35

state transition net

interactive

dialog

system

automatic

recorded

process



system

description

adjacency matrix

frequency matrix


the analyzing program AMME

ascii text

outputfile with

quantitative

measures

graphic

outputfile in

PostScript

format


0 1 2 3

0 0 1 0 0

1 3 0 1 0

2 0 0 0 1

3 0 2 0 7


0 1 2 3

0 0 1 0 0

1 1 0 1 0

2 0 0 0 1

3 0 1 0 1

• simulation

• task
-
subtask

• similarity

• learning

• MDS

• distances

• personal styles

• MDS

• complexity

• routine

• interface


design

• deadlocks

USER

"defaultp.ps"

Petri net simulator

PACE

"*.str"

"*.log"

Path finder

KNOT

Markov analyzer

SEQUENZ

transformation

to a syntactical

correct logfile

v2132 13

S'initial_state' 15@
-
12


390@330 S 0 0

Tnil nil 480@420


S 1 2 nil 0

cS CSCS cSsS SSRS rS

5 tftt 10 ft

"*.net"

"*.ptf"

"*.mkv"

"*.pro"

"*.ps"

any Postscript

interpreter

any text

processor

The AMME Program Structure

© M. Rauterberg, 2003

16
/35





BC

cycle


= T


S + 1

BC
cycle
Box Plot
Box Plot
1
2
0
10
20
30
40
*
*
*
beginners - experts
1
2
3
4
0
10
20
30
40
*
*
task no.

SOURCE SUM-OF-SQUARES DF MEAN-SQUARE F-RATIO P

experience 275.521 1 275.521 10.337 0.003
tasks 259.563 3 86.521 3.246 0.032
exp. x tasks 25.729 3 8.576 0.322 0.810
ERROR 1066.167 40 26.654
Behavioral Complexity (BC) àla McCabe (1976)

Experiment:

N=6 novices; N=6 experts

4 tasks with a database

Metric BC=C
cycle

Ref
: Rauterberg, M. (1993). AMME: an
Automatic Mental Model Evaluation to
analyze user behaviour traced in a finite,
discrete state space.
Ergonomics,
vol.
36(11), pp. 1369
-
1380.

© M. Rauterberg, 2003

17
/35

Val i dati on of the functi onal equi val ence,
computed by the si mi l ari ty rati o (SR)
Addi ng goal
setti ng
structure
Reconstructed
mental task model
Human mental model









?
Observati on
of human
behavi our
Fol di ng
Addi ng sequenti al
and temporal
i nformati on
Model
executi on
ori gi nal behavi oural sequence
si mul ated behavi oural sequence
Devi ce model
2
G_2
Start menu
M_3
Main menu
F_3
User
key press
Automatic
transition
Automatic
transition
MsDOS
G_2
Main menu
F_3
Start menu
M_3
Automatic
transition
Automatic
transition
User
key press
MsDOS
Mai n
men u
Mai n me nu
F-f il e
St art
men u
F_10
M_3
h
F_3
1
i
h
G_2
MsDOS
Mai n me nu
Mai n me nu
F-f il e
St art menu
F_10
M_3
h
F_3
1
i
h
G_2
MsDOS
1
3
4
5
6
i
Validation of extracted Mental Models

© M. Rauterberg, 2003

18
/35








S
R

1

o
r
g
,
t
R

s
i
m
,
t
R

m
a
x
o
r
g
R


s
i
m

1
N
o
r
g
N

t

1
s
i
m
N












o
r
g
2
N






*
1
0
0
%
The Similarity Ratio SR

Legend
:
R

is the absolute rank position in the
orig
inal or
sim
ulated process

Ref
: Rauterberg, M. (1995). From novice to expert decision behaviour: a qualitative modelling approach with Petri nets. In: Y. A
nza
i, K.
Ogawa & H. Mori (eds.), Symbiosis of Human and Artifact: Human and Social Aspects of Human
-
Computer Interaction
--
HCI'95
(Advances in Human Factors/Ergonomics, Vol. 20B, pp. 449
-
454). Amsterdam: Elsevier.

© M. Rauterberg, 2003

19
/35

d
a
F3
space
space
TAB
F2
TAB
CR
space
F9
h
original
d
a
F3
TAB
F9
h
d
a
F3
TAB
space
F2
F2
CR
F9
h
d
a
F3
space
CR
space
F9
a
F3
TAB
F9
a
...
d
h
d
h
d
h
d
h
d
a
F3
TAB
CR
CR
TAB
space
F2
F9
h
d
a
F3
space
space
CR
space
F9
h
d
a
F3
F9
a
F3
TAB
CR
space
space
F9
h
40%
77%
76%
10%
10%
10%
67%
79%
10%
83%
Simulated logfiles with Model-1
43%
d
h
10%
Simulation Results: Model
-
1

SR

© M. Rauterberg, 2003

20
/35

d
a
F3
space
space
TAB
F2
TAB
CR
space
F9
h
original
d
a
F3
space
TAB
space
TAB
space
F2
CR
F9
h
d
a
F3
space
space
TAB
TAB
space
F2
CR
F9
h
d
a
F3
space
space
TAB
space
TAB
F2
CR
F9
h
d
a
F3
space
TAB
space
TAB
space
F2
CR
F9
h
d
a
F3
space
space
space
TAB
TAB
F2
CR
F9
h
d
a
F3
space
TAB
space
space
TAB
F2
CR
F9
h
d
a
F3
space
TAB
space
TAB
space
F2
CR
F9
h
d
a
F3
space
space
TAB
space
TAB
F2
CR
F9
h
d
a
F3
space
space
TAB
TAB
space
F2
CR
F9
h
d
a
F3
space
TAB
space
TAB
space
F2
CR
F9
h
d
a
F3
space
space
space
TAB
TAB
F2
CR
F9
h
d
a
F3
space
TAB
space
space
TAB
F2
CR
F9
h
94%
94%
96%
96%
94%
94%
94%
96%
94%
94%
94%
96%
Simulated logfiles with Model-4
95%
Simulation Results: Model
-
4

SR

© M. Rauterberg, 2003

21
/35

Model-2 (first part): Event-driven goal setting strategy
Main menu
Main menu
F-file
Start menu
F_10
M_3
h
F_3
1
i
h
G_2
MsDOS
System level
Cognitive level
Goal
instanciation
level
Action level
Model
-
2: event
-
driven goal setting

Ref
: Rauterberg, M., Fjeld, M. & Schluep S. (1997). Parallel or event
-
driven goal setting mechanism in Petri net based models of ex
pert
decision behaviour. In: S. Bagnara, E. Hollnagel, M. Mariani & L. Norros (eds.), Time and Space in Process Control
--
CSAPC'97 (Si
xth
European Conference on Cognitive Science Approaches to Process Control, pp. 98
-
102). Roma: CNR.

© M. Rauterberg, 2003

22
/35

Model
-
3: parallel goal setting without feedback

System level
Cognitive level
Goal
instanciation
level
Action level
Model-3 (first part): Parallel goal setting strategy
M_3
h
F_3
1
i
h
G_2
F_10
Main menu
Main menu
F-file
Start menu
MsDOS
© M. Rauterberg, 2003

23
/35

Goal
instanciation
level
Action level
M_3
h
F_3
1
i
h
G_2
F_10
Main menu
Main menu
F-file
Start menu
MsDOS
System level
Cognitive level
Feedback
level
Model-4 (first part): Parallel goal setting with feedback
Model
-
4: parallel goal setting with feedback

© M. Rauterberg, 2003

24
/35








S
R

1

o
r
g
,
t
R

s
i
m
,
t
R

m
a
x
o
r
g
R


s
i
m

1
N
o
r
g
N

t

1
s
i
m
N












o
r
g
2
N






*
1
0
0
%
T
ab
l
e

1
:

The
m
ode
l

c
o
m
pl
e
xi
t
y

(
C
c
y
cle
)

and

s
imil
a
ri
ty

r
a
ti
o

(S
R
)

o
f

t
he

m
ode
li
ng
app
r
oache
s-
1,

-2
,

-
3
a
nd

-
4
[
s
t
d
:
=
st
anda
r
d dev
i
a
ti
on
]
.
m
o
d
el
i
n
g
a
p
p
r
o
a
ch
n
o.

1
m
o
d
el
i
n
g
a
p
p
r
o
a
ch
n
o.

2
m
o
d
el
i
n
g
a
p
p
r
o
a
ch
n
o.

3
m
o
d
el
i
n
g
a
p
p
r
o
a
ch
n
o.

4
C
c
y
cle
:

(
m
e
a
n

± s
t
d)
:
1
3
± 5
4
3
± 1
7
5
7
± 2
5
1
0
1
±

4
3
C
c
y
cle
:

(
mi
n…
m
a
x
.
)
:
6
…1
8
2
2…
6
8
3
0…
9
7
5
5…
1
7
0
S
R
(m
ea
n
%
± st
d
)
:
4
1
± 2
8
6
6
± 2
1
8
8
± 1
1
1
0
0
±

0
S
R
(m
i
n

m
ax
.
%
)
:
3
…7
9
3
6…
9
8
6
7…
1
0
0
1
0
0
…1
0
0
#

s
i
m
u
l
at
e
d

se
q
u
enc
es
5
*
6
=3
0
5
*
6
=3
0
5
*
6
=3
0
5
*
6
=3
0
The Similarity Ratio SR

Legend
:
R

is the absolute rank position in the
orig
inal or
sim
ulated process

© M. Rauterberg, 2003

25
/35

#MTT = #TST / #DS

Measuring 'Personality Styles'

#R = #AT / #DT

Measuring 'Routinization'

C
cycle

= #T


#S+P





with #S =< #T and P=1

C'
cycle

= #F

(#T + #S)+P

with #S > #T and P=1

Measuring Complexity

Overview over different measures

T = number of transitions

S = number of states

F = number of connectors

TST = task solving time

AT = all used transitions

DT = all different transitions

DS = all different states

© M. Rauterberg, 2003

26
/35

The common AI assumption

observable
behaviour
mental
model
learning

Ref
: Rauterberg, M. (1996). About faults, errors, and other dangerous things. In: C. Ntuen & E. Park (eds.), Human Interaction w
ith

Complex Systems: Conceptual Principles and Design Practice (pp. 291
-
305). Norwell: Kluwer.


© M. Rauterberg, 2003

27
/35

complex
system
operator
beginner
advanced
expert
learning
time
interaction
SC

BC

CC

We found a
negative

correlation between

Behavior
-
Complexity BC and [assumed] Cognitive
-
Complexity CC

Experiment:

N=6 novices; N=6 experts

4 tasks with a database

Metric BC=C
cycle

The reality: what we found!

© M. Rauterberg, 2003

28
/35

observable
behaviour
mental
model
The reality: how to interpret?

learning

© M. Rauterberg, 2003

29
/35

Mental Knowledge Structures: a Metaphor

s0

d

h

s1

b

s2

s3

F3

F9

CR

_

TAB

F2

"wall“: knowledge about
unsuccessful behavior

"dales“: knowledge about successful behavior

This conclusion would have major
impact e.g. on training procedures
of operators of complex systems!

© M. Rauterberg, 2003

30
/35

Learning: the traditional understanding

Before learning phase

After learning phase

© M. Rauterberg, 2003

31
/35

Mental decision making for concrete actions is like rolling a ball between hills

Decision and Action: a new View

© M. Rauterberg, 2003

32
/35

Learning and experience

task complexity
time-1
time-2
time-3
time
task-1
task-1'
task-1''
Ref
: Rauterberg, M. & Aeppli, R. (1995). Learning in man
-
machine systems: the measurement of behavioural and cognitive complexity.
In: Proceedings of IEEE International Conference on Systems, Man and Cybernetics
--
SMC'95 (Vol. 5, IEEE Catalog Number
95CH3576
-
7, pp. 4685
-
4690). Piscataway: Institute of Electrical and Electronics Engineers.


© M. Rauterberg, 2003

33
/35

The Learning Experiment

Time structure and knowledge structure are different!

Task solving time

Behavioral complexity

N=6 men (average age of 25
±

3 years)

© M. Rauterberg, 2003

34
/35

Conclusions


A valid metric for task complexity based on
task structure allows an objective comparison


Automatic analysis for unconstrained task
solving behavior allows analysis with applied
statistics


A new analysis and modeling approach leads
to new insights…

© M. Rauterberg, 2003

35
/35

Thank you for your attention.