Brain related semantics - DAI

siennaredwoodIA et Robotique

23 févr. 2014 (il y a 3 années et 1 mois)

74 vue(s)

CSCTR


Session 8

Dana
Retov
á


Rejects the standard view that
amodal

symbols
represent knowledge in semantic memory


Cognition shares the same mechanisms with
perception, action and introspection


Simulation


A core form of computation in the brain


Reenactment of perceptual, motor and introspective states
acquired during experience



As experience occurs, the brain captures the states across
modalities and integrates them with a
multimodal
representation
stored in memory



Modal representation and imagery
representing knowledge


Epicurus, Kant, Reid



Behaviorists


Imagery not sufficiently scientific


Cognitivists


Amodal

representation (feature lists, semantic
networks, frames)


Elegant and powerful formalisms for representing
knowledge


Could be implemented in AI


No evidence supports the presence of
amodal

symbols in cognition


Grounding problem


Traditional theories fail to explain how cognition
interfaces with perception and action


Problem where the brain stores
amodal

symbols


How is it consistent with neural principles of
computation?


Simulation


Situated action


Bodily states



Modal representations are central to knowledge


Cognitive Linguistics Theories


Lakoff

& Johnson (1980, 1999)


Abstract concepts are grounded metaphorically in
embodied and situated knowledge


Theories of situated action


Gibson (1979)


Role of environment in shaping cognitive mechanisms


Coupling of perception and action during goal achievement


Social interaction


Research in robotics


Dynamic systems as preferred architecture


Fixed representations do not exist


Memory theories


Glenberg

(1997)


Memory is not just passive storage of information


Perception of relevant objects triggers affordances for
action stored in memory


Reasoning about future actions relies on remembering
affordances while
suppressing

perception of the
environment


Social simulation theories


How we represent the mental states of other people


We use simulations of our own minds


To feel someone else’s pain we simulate our own pain


Mirror neurons


Empathy, imitation, social coordination


Perceptual Symbol Systems


Synthetic approach


Implements standard symbolic functions


Type
-
token binding, inference, productivity, recursion, propositions


A single multimodal representation system in the brain supports
diverse forms of simulation across different cognitive processes


High
-
level perception


Working memory


long
-
term memory


conceptual knowledge


Convergence zone architecture (
Damasio

1989, Simmons &
Barsalou

2003)


Single representation system controlled by multiple simulation mechanisms



ensemble of neurons within which many
feedforward
/feedback loops make contact.


It 1) receives forward projections from
cortical regions located in the connectional
level immediately below


2) Sends reciprocal backward projections to
the originating cortices


3) Sends forward projections to cortical
regions in the next connectional level; and


4) Receives projections from
heterarchically

placed cortices and from
subcortical

nuclei in
thalamus, basal forebrain, and brainstem.


Perceptual Inference


Perception
-
action coordination


Perception of space


Memory


Implicit memory


Explicit memory


Working memory


Conceptual processing


Vision and motion


Goldstone (1995)


Association between shape and color


Hansen et al. (2006)


Object’s natural color distort achromatic perception of
the object toward the opponent color


Motion (
Freyd

1987,
Shiffrar

&
Freyd

1990,1993)


Subjects simulate the visual trajectory
beyond

its actual
trajectory


Also during apparent motion, simulation of possible
action shapes the perception of motion


Speech (Warren 1970) :
http://www.youtube.com/watch?v=UlJs24j3i8E


Lexical knowledge produces simulation in speech
perception


missing phoneme simulation




Simulations of potential actions


Viewing an object grasped with a precision or power grip (grape
vs. hammer) produces a simulation of the appropriate action
(Tucker & Ellis 1998)


This is affected by


object’s orientation (
Symes

et al. 2007)


Size (Glover et al. 2004)


Simulations of both grasping and functional actions (
Bub

et al
2007)


Also name triggers simulation (Tucker & Ellis 2004)


Hearing a word activates the
articulatory

action associated with
producing it (
Pulvermuller

2006)


Perceived effort affects visual perception (
Proffitt

2006)


Being tired from a run makes a hill look steeper


Carrying a heavy pack makes a path look longer



Motor simulations


Motor system constructs a feed
-
forward simulation
of the action to guide and correct it (
Grush

2004,
Wolpert

et al. 1999)


Generating visual inferences about the anticipated
actions of perceived agents (Wilson &
Knoblich

2005)


The perception of space is shaped by the
body and it’s relation to the environment


Locating objects has various difficulty along
different axes


Vertical


easiest


Front
-
back


Left
-
right


Most difficult


bodily cues are lacking


Perception of near space extends with arm length
(Longo &
Laurenco

2007)


Results form simulation of perceptual
memories


Repetition priming is strongest when the
modalities of the memory and stimulus match
(e.g. auditory) (
Kirsner

et al., 1989)


Repetition priming is strongest when
perceptual details of the memory and
stimulus match (e.g. orientation, size,…)
(
Jacoby&Hayman
, 1987)


Imagining produces repetition priming similar
to actual perception (
Roediger&Blaxton
,
1987)


Multimodal simulations of previous episodes


Important for constructing future events


The retrieval of a word stimulates the modal
operations performed at encoding (Wheeler et
al. 2000)


Visual areas become active during retrieval
following visual study while auditory areas become
active following auditory study


Greater activation in modal areas when
remembering something that really occurred
than false memories (
Slotnick

&
Schacter

2004)


Absent stimulus is stored in working memory (Levy
&
GoldmanRakic

2000)


To maintain working memory, neurons in the frontal lobes
maintain a simulation of the absent stimulus in the modal
system that processed it originally.


Some frontal regions maintain working memories of
objects, other spatial locations, motion, textures, etc.


They are highly selective for the specific features


Visual imagery in working memory simulates visual
processing (Finke 1989,
Kosslyn

1980,…)


Analogously, motor imagery, auditory imagery, etc.


Mental rotation of visual objects
-
> motor simulations of
making them turn (Richter et al. 2000)


Behavioral evidence


When asked whether an property belongs to an objects
subjects simulate properties to verify them (Solomon &
Barsalou

2004)


Lesion evidence


Lesions in one modality


losing categories that rely on it
for processing (
Damasio

1994, …)


E.g. damage to visual areas


losing of ability to
categorize animals (visual processing is dominant)


Damage to motor areas


categorization of tools


Neuroimaging

evidence (Martin 2001, 2007)


When processing conceptual knowledge, brain areas
representing properties are active



Perceptual simulation


Motor simulation


Affective simulation


Situation models


Evidence of modal representations in language
comprehension


Spatial representation (Bower & Morrow 1990)


People confused pictures with text (
Intraub

& Hoffman
1992)


Replacing words with pictures did not disrupt sentence
processing (Potter 1986)


Subjects read a sentence and then processed
a picture that either matched or mismatched
something implied by the sentence


“The ranger saw the eagle in the sky”


Picture of an eagle


wings outstretched or folded


Visual irrelevant information interferes with
spatial inferences (Fincher
-
Keifer

2001)


Verbs for head, arm and leg actions
produce
head,
arm and leg simulation in the
respective areas of the motor system
(
Pulvermuller

2005)


When action to make a response is consistent
with text meaning, the response is quicker
(
Glenberg

&
Kaschak

2003)


Subjects simulate corresponding motion
through space (Richardson et al. 2003)


Positive/negative valence (Meier & Robinson)


High/low power (Schubert 2005)


Subjects’ faces configured according to
sentences with emotional content (
Havas

2007)


When facial emotion matched the content
comprehension was better


Gesture


Producing gestures helps speakers retrieve words
whose meaning are related to the gestures (Krauss
1998)


Also help listeners comprehend what speaker says


Children can gesture before speaking


Physical reasoning


Gear, pulleys


Driven by spatial simulation


Sketchy, not holistic and detailed


Abstract reasoning


Content effects


Reasoning about time using space domain
-

metaphors


Embodiment effects


Activating elderly stereotype causes people to walk
slowly and to perform lexical decision slowly
(
Dijksterhuis

&
Bargh

2001)


Engaging the smiling musculature produces
positive affect (
Strack

et al. 1988)


Social mirroring


Individual differences in the ability to simulate
other people’s mental states correlate with rated
empathy (Jackson et al. 2005)


Development


Mirroring, object permanence…