From Cognitive Models to Cognitive Systems

bouncerarcheryAI and Robotics

Nov 14, 2013 (3 years and 9 months ago)

182 views

0

Sandia

National

Laboratories


From Cognitive Models to Cognitive
Systems

Chris Forsythe

Computational Initiatives

Sandia National Laboratories

jcforsy@sandia.gov

http://www.sandia.gov/cog.systems/Index.html

1

Sandia

National

Laboratories

What Are “Cognitive Systems?”


A “Cognitive System” is one that utilizes
psychologically plausible

computational
representations of human cognitive
processes as a basis for system designs
that seek to engage the underlying
mechanisms of human cognition and
augment the cognitive capacities of
human users, not unlike a “cognitive
prosthesis.”

2

Sandia

National

Laboratories

Our Vision for Cognitive Systems



Reverse current trends so that the machine
conforms to the human, as opposed to the human
conforming to the machine.




Embed within machines
highly realistic and
individualized

computational representations of
cognitive processes vital to human communication,
cooperation and collaboration.




The machine becomes an augmented human
cognitive entity that knows you like your best friend.

3

Sandia

National

Laboratories

Psychologically Plausible Model of
Human Cognitive Processes

Computational model inspired by naturalistic
decision making and oscillating systems concepts

Mismatch

Sensor


Sensor


Data


Percep
Agent

Percep
Agent

Percep
Agent

Semantic
Knowledge

Pattern
Recognition

Situation/Contextual
Knowledge


Situation




Situation

Episodic
Memory



T 1 T 2
T 3

Comparator

Selective
Attention

Action
Generation

Drive
Mechanism

Emotional
Processes





Perceptua
l
Synthesis

Federation
Perceptual
Agents

Perceptual Processes

Mismatch

4

Sandia

National

Laboratories

Cognitive Framework Composed of
Oscillating Systems

Mismatch

Semantic
Knowledge

Pattern
Recognition

Situation/Contextual
Knowledge


Situation




Situation

Episodic
Memory



T 1 T 2
T 3

Comparator

Selective
Attention

Action
Generation

Drive
Mechanism

Emotional
Processes

Mismatch

0
5
10
15
20
25
30
35
0
1
2
3
4
Time (sec)
Total
Electrical
Potential
Event
-
Related
Activation

SA12_Count
SA1_Count
0.60
0.80
1.00
1.20
1.40
1.60
1.80
0
5
10
SA12_Count
SA1_Count
1.00
1.50
2.00
0
5
10
Entrainment with
Pacemaker

Selective Phase
-
Locking with Stimulus

Initial Recognition

Action Execution

Sensor


Sensor


Data


Percep
Agent

Percep
Agent

Percep
Agent





Perceptua
l
Synthesis

Federation
Perceptual
Agents

Perceptual Processes

5

Sandia

National

Laboratories

Standardized Knowledge Components

B

C

D

E

A

0

4

0

0

B

-

1

2

1

C

1

-

0

0

D

2

0

-

4

E

1

0

4

-

Knowledge
Component


Knowledge components provide standardized format
for representing knowledge to enable automatic
generation of cognitive model components



Knowledge
Elicitation

Automated
Knowledge

Capture

Text and
Related
Sources

Situation/Contextual
Knowledge


Situation




Situation

Cognitive Model
Components

Comparator

Pattern
Recognition

Episodic Memory



T 1 T 2 T 3

Semantic
Knowledge

6

Sandia

National

Laboratories

Flexible Modular Construction of
Cognitive Systems


Cognitive model components may be combined
and linked to inputs and outputs to construct
integrated systems



Situation/Contextual
Knowledge


Situation




Situation

Pattern
Recognition

Episodic Memory



T 1 T 2 T 3

Semantic
Knowledge

8

7

Sandia

National

Laboratories

Knowledge for Adaptive Systems


Some components provide standardized representations
of knowledge for adaptive systems


e.g., Schema
-
Based Prioritization


External Environment

Machine Situation

Model (i.e., current schema)

Schema
-
Based
Prioritization

Adaptive
Interface

Controller

Component indicates the situation
-
based prioritization of
information or tasks for systems employing information
filtering, task delegation, interruption mediation, etc.

8

Sandia

National

Laboratories

Cognitive Systems for Perceptual
Representation


Mechanisms for utilizing perceptual representations to
augment cognition


e.g., Counter
-
bias Transformation for
Illusionary Correlation

Semantic
Knowledge

Expectation

Expectation

No Association

Concepts
Present

Concepts
Absent

Expected
Concepts that
Are/Were

Present

Expected
Concepts that
Are/Were

Absent

Human
-
Computer Interface

9

Sandia

National

Laboratories

Cognitive Collective of Expert Models

SON
(Internet)
SRN
SCN
Proxy
Public Server
SRN Server
Insider Threat
Collective
IBM Compatible

Cognitive
Collective

End User

Police Records

Financial/Credit
Records

Legal Records
(lawsuits, etc)

HR Database

Need
-
To
-
Know App

Network Logs

SNL Phone Records

SNL Cell Phone Records

Email Records

Foreign Nat'l DB

Security DB

Employee/Labor Rel'n DB

Access Control System

Expense Reports

Foreign Travel Req. DB

Audits

Patents and Licensing

LDRD Project DB

WFO DB

A cognitive collective utilizes multiple embedded
experts to attain a collective situation recognition

10

Sandia

National

Laboratories

Making Expert Knowledge Explicit

Progress marker



Number of cues
observed by cognitive
models each day

Dots on the perimeter
correspond to different
cues

Flag indicating a person has been flagged by one expert
model
--
the height showing degree of suspicion.

Can select one or more experts as well
as the collective for further analysis

Wedges of Pie Chart correspond to different themes

Blue dot indicates a situation
has been recognized

Lines indicate one cue has
primed another cue

Lines indicate cues that
contribute evidence to
recognition of a specific situation

The person being analyzed.

Can scale the amount of
information being displayed

11

Sandia

National

Laboratories

Real
-
Time Inference of Operator
Cognition



Cognitive model for an operator
observes data and events and
interprets situations based on the
operator’s cognitive model.


Compared to a reference stating
the operator’s true interpretation
of data and events, the model
interpretation was 87% accurate
overall, and 91% accurate in
recognizing the occurrence of
situations.

AWACS simulator
presented complex
cognitive task
involving management
of multiple assets and
threats

Comparison of model to reference.
Green and gray indicate accurate
inferences, red false positives and
yellow false negatives.

12

Sandia

National

Laboratories

Preliminary Results Indicate Importance
of Conforming to Operator

Individualized Cognitive Models


Utilized knowledge elicitation to
develop individualized cognitive
models that reflected the unique
knowledge of each operator.


As illustrated in the
accompanying figure,
operators
trained to equivalent levels of
expertise may possess different
cognitive models

of a task. Here,
the
blue

and
red

connectors
distinguish the two operators.

Cues

Situations

Actions

Designing the machine to adapt to the individualized human
cognitive model is critical,
one size does not fit all

13

Sandia

National

Laboratories

Cognitive Systems for Interaction


Embedded experts engage user in interaction exposing
user to alternative perspectives and expanding their
interpretative powers


e.g. Critique Generation

Expert

Model

Inferred User

Model

Discrepancy
Detection

Critique
Generation

Mouse/Joystick, Visual
Fixation, Communication

“Have you considered the
relationship between
these two antibodies?”

14

Sandia

National

Laboratories

Part 1 of Our Vision: Aide

Aide



Goes where you go



Knows what you do



Knows what you know



Knows your priorities, interests, etc.



Co
-
evolves with you



Self
-
aware , meta
-
cognition



Serves as mediator, shield



Mentor / tutor or student/trainee



Trusted

15

Sandia

National

Laboratories

Part 2 of Our Vision: Council

Aide



Goes where you go



Knows what you do



Knows what you know



Knows priorities, interests, etc.



Co
-
evolves with you



Meta
-
cognitive, self
-
aware



Serves as mediator, shield



Mentor / tutor or student/trainee



Trusted

Council



Virtual meeting with synthetic experts



Flexible mixture humans and agents



Agents possess unique domain
knowledge



Dialogue
-
based interaction with agents



Agent
-
agent synthesis / contrast of
knowledge/perspective



Incorporation of supporting artifacts



Multiple interaction paradigms

16

Sandia

National

Laboratories

Part 3 of Our Vision: Oracle

Aide



Goes where you go



Knows what you do



Knows what you know



Knows priorities, interests, etc.



Co
-
evolves with you



Meta
-
cognitive, self
-
aware



Serves as mediator, shield



Mentor / tutor or student/trainee



Trusted

Council



Virtual meeting with synthetic experts



Flexible mixture humans and agents



Agents possess unique domain knowledge



Dialogue
-
based interaction with agents



Agent
-
agent synthesis / contrast of opinion



Incorporation of supporting artifacts



Multiple interaction paradigms

Oracle



Rapidly configurable simulation with
highly realistic synthetic humans



Flexible mixed human, agent, robot,
sensor and electronic systems



Automatic generation populations of
unique cognitive entities



Specific political decision makers



Reusable entity libraries



Empirically validated models

17

Sandia

National

Laboratories

Anticipated Challenges


Automated knowledge capture


Multi
-
modal memory representation


Emergent perception


Cognitive
-
affective interplay


Meta
-
cognitive self
-
awareness

18

Sandia

National

Laboratories

Alternative Data Sources for Automated
Knowledge Capture

There are many different data sources that may be used to
infer individual knowledge and ongoing cognitive processes.

Semantic
Knowledge

Pattern
Recognition

Situation/Contextual
Knowledge


Situation




Situation

Individualized Cognitive Model

Instrumentation of
Software

User
-
Generated Text

Eye
-
tracking

User
-
Generated
Speech

Autonomic
Response

Brain
Activation

User
-
Read Text

User Actions













Known Cognitive
Models

Power Spectrum
0
10
20
30
40
50
60
70
80
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
cycles per minute
Power (AU)
Facial
Expressions

19

Sandia

National

Laboratories

Multi
-
Modal Memory Representation

Episodic


Spatial World Model

Action

Perceptual

axon

nerve

neuron

pons

hypothalamus

Brain Stem

cerebellum

Cerebral cortex

Cell body

dendrite

Corpus callosum

Occipital lobe

hippocampus

synapse

Semantic

Lexica
l

20

Sandia

National

Laboratories

Emergent Perception

Semantic
Knowledge

Pattern
Recognition

Situation/Contextual
Knowledge


Situation




Situation

Business Meeting

Perceptual Processes as an Emergent Phenomenon,
as Opposed to a Bottom
-
Up or Top
-
Down Process

Emotional
Processes

21

Sandia

National

Laboratories

Cognitive
-
Affective Interplay

Semantic
Knowledge

Pattern
Recognition

Situation/Contextual
Knowledge


Situation




Situation

Pleasure

Dysphoria

Anxiety

-

Fear

Surprise

Frustration

-

Anger

Disgust

Cognitive and Emotional Representations and
Processes Inseparably Linked

Real
-
time input

Cognitive
Processes

Affective
Processes

22

Sandia

National

Laboratories

Meta
-
Cognitive Self
-
Awareness

Level 1: Knows of Its Cognitive Limitations

-

What effects performance, e.g., arousal

-

Limits of knowledge and skills

-

Expectations/bases

Level 2: Learns Its Cognitive Limitations

-

Monitors own cognition

-

Diagnoses own cognition

Level 3: Overcomes Its Cognitive Limitations

-

Differential accessibility of knowledge

-

Optimization of processes

-

Mechanisms for self
-
perturbation

23

Sandia

National

Laboratories

VxInsight Analysis of Relevant Scientific
and Technical Literature



Cognitive models (white dots
-
frame on left)



Knowledge elicitation (green dots)



Knowledge representation (magenta dots)



Decision making (orange dots)



Human
-
computer and user interfaces



Customization (blue dots).



Papers retrieved by two or more



queries (white dots
-
frame on right)




There is modest overlap between technical concepts and fields being bridged
by this program, but the lack of more significant overlap indicates an
opportunity for the laboratories to create a unique capability.


24

Sandia

National

Laboratories

Conclusion


Arguably, many of the technologies in which
we’ve invested the past few decades may have
reached the point of diminishing returns


New approaches are needed that have a breadth
of application ranging from cell phones/PDA’s to
massive systems
-
of
-
systems


A transformation in human
-
machine systems
comparable to the 80’s transition from command
line interfaces to GUI’s is sorely needed