From place cells in rats to human syntax: the construction of a cognitive map of grammar

grassquantityΤεχνίτη Νοημοσύνη και Ρομποτική

15 Νοε 2013 (πριν από 3 χρόνια και 10 μήνες)

111 εμφανίσεις

Gideon
Borensztajn

gideonbor@gmail.com


Institute of Phonetic Sciences

University of Amsterdam



CLS
-
workshop on Learnability and Computational Models of
Language Acquisition, ILLC

March 11, 2013

From place cells in rats to human
syntax: the construction of a cognitive
map of grammar

What does rat navigation have to do

with the human ability to talk?

rat brain

human brain

Well, actually quite a lot.

Place cells and cognitive map theory


Certain cells in rat hippocampus
fire only when rat is in specific
location in the maze.


Together these
place cells

encode a
mental, or

cognitive map

of the
surrounding environment
[O'Keefe and Nadel, 1979].


Map is constructed gradually
from spatial
episodic

experiences,
by linking overlapping spatial
cues.

spatial sensitivity of
different place cells

Function of the cognitive map

Cognitive map
of structured
episodic memories

allows
the brain to


combine memories flexibly
and productively
, by

making associative jumps
between linked memories.



Mentally explore
new routes

in the maze.



Make
transitive inferences



(if A > B
Λ

B > C then A > C)



Similar abilities needed for productive language use



novel sentences build by reusing stored fragments


Claim
: humans construct a
cognitive map
of syntactic relations
from episodic linguistic experiences

Item
-
based learning


Child language acquisition may shed light on transition from
episodic to semantic memory:


According to Usage Based Grammar [
Tomasello
, 2003] children
initially memorize and
imitate

complete utterances (holophrastic
stage).


Then follows a stage of item
-
based speech, often organized
around so
-
called verb
-
islands.


Subsequently, children start breaking down the item
-
based
constructions, introducing variables in slots, as in “
Where's the
X?”, “I
wanna

X”
,
etc



Local scope of children’s categories (e.g., verb
-
islands)
gradually expands to system
-
wide scope, while converging to
an
abstract

and adult
-
like language.



Claim
: grammar acquisition reflects a process of memory
consolidation from episodic to semantic memory


Grammaticalization

=
semantization


Language processing & memory


Semantic memory

is a person’s
general world knowledge,
including language, in the form of
concepts that are systematically
related to each other.

Two kinds of declarative memory,
semantic and episodic

[Tulving, 1972]:

“bread”


Episodic memory

is a person’s
memory of personally
experienced events or
episodes
,
embedded in a temporal and
spatial
context
.

Me lining up in front of the bakery


Relation
: episodic memories bind


together sequences of items stored in semantic memory


[e.g.,
Shastri
, 2002;
Eichenbaum
, 2004]


Claim
: duality rules versus exemplars arises from
s
emantic
-
episodic memory interaction during language
processing


Episodic memory


item
-
based nature of language
(with a role for concrete constructions (sentence
fragments) larger than rules


Semantic memory


abstract, rule
-
based grammar
(Context Free grammar)


The
construct
-
i
-
con

is an instantiation of episodic
-
semantic memory system for language



How to model this?

Semantic
-
episodic :: Rules vs. exemplars


In Data Oriented Parsing (DOP) [e.g., Bod, 1993, 1998] primitive
elements of the grammar are
subtrees

of
arbitrary size


They vary in size, form and level of abstraction from complete
sentences to abstract rules


Derivation of a sentence in DOP is a sequence of
subtrees
,
combined by substitution operation.

Data oriented parsing

VP

CD

NNS

NP

cars

VBD

sold

PP

DT

NN

NP

The

company

NP

S

VP

JJ

NN

last

year

NP

1,214

CD

DT

NNP

NP

the

U.S.

IN

in

PP

Towards a neural model of language
processing and acquisition


Next goal: explain the transition from concrete
(imitative) to abstract (productive) language use from a
neural perspective, in terms of changes in the
representations and organization of the memory system



Propose an explicit model of episodic
-
semantic
interaction in language, where


language processing is modeled as
retrieval

from memory
, both
episodic (fragments) and semantic (productive rules)


language acquisition is modeled as transition from episodic to
semantic linguistic memory (


increasing abstraction of
children’s language)





A model of episodic memory must take into account that



All attended episodic experiences leave physical
memory
traces

in the brain.


Sequentiality
: episodes are construed as temporal
sequences that bind together static semantic elements,
within a certain context [e.g.,
Eichenbaum
, 2004].


Episodic memories are
content addressable
: their retrieval
can be
primed

by cues from semantic memory.


We will use these properties of the human memory system

in designing a computational model of language processing,
conceived as
retrieval from episodic memory
.

Properties of episodic memory


Suppose that the primitive elements of a


grammar,

corresponding to context
-
free


rewrite rules and words, are stored within


treelets

in a structured network.


A network of such treelets constitutes a
semantic memory
,
corresponding to a
context
-
free

grammar.


(later we will get rid of the labels, and situate treelets in
a continuous “substitution space”)

S

NP

VP

Episodic grammar


Derivation of a sentence is sequence of visits to treelets;
describes a path through the network


In the process treelets serially bind to each other, while
leaving
traces

in the local treelet memories.


Proposal
: episodic memory of a sentence consists of
physical memory traces distributed across the treelets
visited in the derivation.







Distributed episodic memory of a sentence

NP

boy

START
*

boy

S*

eats

VT

eats

S

NP

VP

VP

VT

NP

VP
*

mango

NP

mango

VP

VT

NP

S

NP

VP

NP

girl

VP
*

apple

NP

apple

PP

IN

NP

NP

pear

semantic memory

(grammar)

NP

boy

START
*

boy

S*

eats

VT

eats

S

NP

VP

VP

VT

NP

VP
*

mango

NP

mango

VP

VT

NP

S

NP

VP

NP

girl

VP
*

apple

NP

apple

PP

IN

NP

NP

pear

semantic memory with

integrated episodic memory

Episodic

traces

1
-
1

START
*

boy

VP
*

mango

NP

boy

1
-
2

1
-
7

NP

mango

1
-
8

2
-
1

START
*

girl

NP

girl

2
-
2

VP
*

apple

2
-
7

NP

apple

2
-
8

S*

eats

1
-
4

2
-
4

VT

eats

1
-
5

2
-
5

S

NP

VP

VP

VT

NP

1
-
6

1
-
3

2
-
3

2
-
6

VP

VT

NP

1
-
9

2
-
9

S

NP

VP

1
-
10

2
-
10

Treelets after processing
“boy eats mango”
(orange),

and “
girl eats apple
” (blue).

Traces are encoded as
x
-
y



x

= #sentence in corpus;


y

= #position in derivation (top
-
down or left
-
corner)




implements pointers.

Parsing as a priming effect

eats

S*

1
-
4

2
-
6

VT

eats

1
-
5

2
-
7

S

NP

VP

1
-
3

VP

VT

NP

1
-
6

2
-
8

NP

bear

VP
*

apple

VP*

mango

2
-
9

1
-
7

CH=1

Activation


When parsing a novel sentence, the traces in a visited
treelet are
primed
, and trigger memories of stored
sentences (
content addressability
).


The traces (
e
x
) receive an
activation

value (
A
), of which the
strength depends on the
common history

(
CH
) of the
pending derivation (
d
) with the stored derivation (x)





CH

is given as #derivation steps shared between
d

and x.




Based on this one can define a probabilistic model

CH=2

CH=1

CH=3

CH=2

CH=4

CH=3

Episodic left corner parsing


Based on probabilistic left corner chart parser of [van
Uytsel, 2001] and [Stolcke, 1995]


Episodic probabilities

for
P
shift
, P
proj

and P
att

are no longer
estimated in advance, but

computed
on
-
the
-
fly

from
trace
activations


Rules


Treelets

(=rules containing traces)


States


Treelet states

q, which are of the form



q = {G; X


j
λ


i
μ

; E
q
}



E
q
is set of
traces

(stored in treelet) with
activations



The activation, or
CH
, of a trace e
j

in state q is updated
using dynamic programming.



Borensztajn, G. & Zuidema, W. (2011),

Episodic grammar: a computational model of
the interaction between episodic and semantic memory in language processing
. Proc.
CogSci 2011

Results of episodic left corner parser on
syntactic parsing task

Parsing model

LR

LP

F

PCFG

78.5

All fragments grammar [Bansal
and Klein, 2011]

86,9

Left corner [van Uytsel, 2001]**

79,0

Non
-
episodic LCP*

70,4

76,7

73,4

Episodic Left Corner Prob.*

84,5

84,6

84,6

Episodic Left Corner SD*

82,3

81,1

81,7

* For this work, results are on section 22 of WSJ, sentences <=20

** No smoothing, results do not generalize to entire section

From supervised to unsupervised


In episodic grammar (labeled)
treelets

were given
innately (copied from
treebank
, supervised).


What we really want is to learn abstract rules of
grammar from (episodic) linguistic experience alone
(unsupervised).


Suppose we had a large repository of
blank

treelets

(without labels)


The labels of the
treelets

are replaced by
vectors

within a high
-
dimensional `substitution space’


Grammar acquisition then amounts to construction of
cognitive map

of syntactic relations, through
topological self
-
organization.

Borensztajn, G., Zuidema, W. & Bod, R. (2009),

The hierarchical prediction network:
towards a neural theory of grammar acquisition
. Proc. CogSci 2009


Features
:

The
Hierarchical Prediction Network (HPN)

From theory to model

Hierarchical temporal compression: simplified model of MPF

N

NP

Det

Adj

No labels, no fixed
associations with
non
-
terminals

Continuous category space

NP

VP

Prototypical,

graded categories

NP

VP

Dynamic, serial binding

Pointers stored in
local memories


N

NP

Det

Adj

S

NP

VP

A cognitive map of syntactic relations:


the HPN “substitution space”

X1

X2

X3

under

VP, verb

PP, prep

NP, noun

eat

A node’s position in a continuous
substitution space

defines

its graded membership to one or more syntactic categories.

Substitutability

is given as the topological distance in
substitution
space,
and it is
learned

gradually.

Conventional syntactic categories correspond to
regions

in this space.

the

tomato

complex units

Simple, lexical units

happy


Work in progress:


Learning a grammar in HPN amounts to self
-
organization of the network topology,
driven by episodic
experience (i.e., the shortest derivation)
:

1.
Fast, one
-
shot learning of exemplars by storing traces
(episodic memory).

2.
Slow learning of a topology, by adjusting vectors of
treelets

bound within derivation (semantic memory).


Episodic component prefers parses that are compatible
with seen exemplars, which in turn affects the
topological organization of the semantic memory.




Grammar acquisition is construed as
memory
consolidation

from episodic to semantic memory.


Demonstrates gradual `
decontextualization


of episodic
memories (traces) into abstract grammar (topology).



Learning a grammar from episodes in HPN

Dual role of the hippocampus


‘Cognitive maps’ (place cells)
situated in hippocampus

1.
Hippocampus functions as a
`gateway‘ for episodic memories

2.
Involved in flexible and
productive use of memory, needed
for novel problem solving
(
Eichenbaum

et al., 1990)


Also involved in binding in language (e.g., Opitz, 2010)


Common explanation? both
episodic reconstruction and on
-
line
processing involve flexible,
dynamic binding


Allows efficient storage of episodes (w/o massive binding)


Allows systematic and productive use of language


Hippocampus implements a
switchboard

function

The hippocampus implements a
“switchboard” function

switchboard

boy

feeds

the

serial transmission

of stored address

lexical units

complex units

(‘treelets’)

subunits

buffer

who

walks

Borensztajn, G., Zuidema, W. & Bechtel, W. (in press).
Systematicity and the need for encapsulated
representations.
In Calvo, P. and Symons, J., (eds.)

Systematicity and Cognitive Architecture

(MIT Press).

Conclusions


Episodic grammar is a promising framework that can
bridge the fields of computational linguistics and
cognitive science.


Explanatory value: gives neural perspective on the trade
-
off between rule
-
based and exemplar
-
based models of
language processing.


Offers quantitative evaluation of an original hypothesis
about the relation between semantic and episodic
memory, and offers account of memory consolidation.


New, cognitively inspired computational approach to
unsupervised grammar induction as gradual transition
between episodic and semantic memory
(consolidation)
,
but still much work to be done.

Thank you!


Questions?












email:
gideonbor@gmail.com


webpage: staff.science.uva.nl/~gideon