Psychology 209: Models of Cognitive Processes Winter, 2012-2013 Syllabus

lovethreewayΤεχνίτη Νοημοσύνη και Ρομποτική

20 Οκτ 2013 (πριν από 3 χρόνια και 9 μήνες)

73 εμφανίσεις

Psychology 209: Models of Cognitive Processes

Winter, 2012
-
2013

Syllabus

Key:




Foundational material you should know


read if you don’t already know this

*

Essential readings for homework

or in
-
class discussion

~

Readings very closely related to
lecture
, adding more detail, for reference

_

Other readings, sometimes historic
al

+

Models we won’t cover, possible starting places for projects

!

Quantitatively challenging
materi
als, sometimes very long, for those interested


Jan

7
: Introduction to PD
P


~

McClelland, J. L., Rumelhart, D. E. & Hinton, G. E. (1986). The Appeal of Parallel
Distributed Processing. In
D. E. Rumelhart, J. L. McClelland, and the PDP Research
Group

(Eds.),

Parallel Distributed Processing: Explorations in the Microstructure of

Cognition.

Vol I.
Cambridge, MA MIT Press.

Chapter 1
.


~

McClelland, J. L. (2009). The place of modeling in cognitive science.
Topics in
Cognitive Science, 1(1),

11
-
38.



Marr, D. (2001). The Philosophy and the Approach. Chapter 1 of
Vision
.

San
Francisco: Freeman.


Jan 9
: Information integration in neurons and behavior


*

McClelland, J. L. (2013).
Bayesian inference, generative models, and probabilistic
computations in interactive neural networks.
Draft, Jan. 6. 2013, Department of
Psychology, S
tanford University.
Pages 1
-
28.


~

Saltzman, C. Daniel and Newsome, William T. (1984). Neural Mechanisms for
Forming a Perceptual Decision.
Science,
264
,
231
-
7.


† Kolb, B. and Whishaw, I. Q. (1980). Physiological organization of the nervous system.
Ch
apter 2 of
Fundamentals of Human Neuropsychology

(pp. 31
-
42). San Francisco:
Freeman. (General background on cellular physiology for those with no prior exposure.
Stop at EEG Recording).


Jan

11
:

Introduction to
the
PDPtool s
oftware
:

the IAC model of
representation of
general and specific information


*
McClelland, J
. L
.

(in prep
).
Explorations in Parallel
-
Distributed Processing: A
Hand
book of models, programs, and ex
ercises.

Second Edition.


Chapter 1:
Introduction.


*
McClelland, J. L (in prep
).
Explorations in Parallel
-
Distributed Processing: A
Hand
book of models, programs, and ex
ercises.

Second Edition.

Chapter 2:

Interactive
Activation and Competition.

Sections 2.1 & 2.2.

Exercises will be specified in
Homework.


Jan 14
:

Discussion of IAC
Model, Relation to Exemplar Models

HOMEWORK #1 DUE


* McClelland, J. L. (1981). Retrieving general and specific information from stored
knowledge of specifics.
Proceedings of the Third Annual Conference of the Cognitive
Science Society

170
-
172.


~ Nosofsky
, R. M. (1984). Choice, similarity, and the context theory of

classification.
Journal of Experimental Psychology: Learning, Memory,

and Cognition, 10
, 104

114. doi:10.1037/0278
-
7393.10.1.104


~

Kumaran, D. & McClelland, J. L. (2012). Generalization through

the recurrent
interaction of episodic memories: A model of the hippocampal system.
Psychological
Review, 119
, 573
-
616.


Jan 16
: Collective computation and optimal perceptual inference


* McClelland, J. L (in progress).
Explorations in Parallel
-
Distribute
d Processing: A
Hand
book of models, programs, and ex
ercises.

Second Edition.

Chapter 3:
Constraint
Satisfaction Models.

Sections 3.1, 3.2, and 3.6. Exercises will be specified in
Homework.


* J. J. Hopfield, "Neural networks and physical systems with

emergent collective
computational abilities",
Proceedings of the National Academy of Sciences of the USA
,
vol. 79 no. 8 pp. 2554
-
2558, April 1982.


Jan 18:
Interactive Activation
: Behavioral and Brain Evidence

and the IA Model


*

McClelland, J. L. and Rumelhart, D. E. (1981). An interactive activation model of
context effects in letter perception: Part 1. An account
of basic f
indings.
Psychological
Review, 88,

375
-
407.


_

Marr, D. (1982). From images to surfaces. In
Vision

(pp. 99
-
111). San Francisco:
Freeman.

An alternative perspective on interactivity


~

Lee, T. S., & Nguyen, M. (2001). Dynamics of subjective contour formation in the
early visual cortex.
Proceedings of the National Academy of Sciences, 98(4)
, 1907
-
11.


~

McClelland, J. L. Mirman, D., and Holt, L. L. (2006). Are there interactive processes in
speech perception?
Trends in Cognitive Sciences, 10(8)
, pp. 363
-
369. Reply by
McQueen, Norris and Cutler. Response to reply by Mirman, McClelland and Holt.

Current s
tate of debate on whether there are interactive processes in speech perception.
Note:

There are three separate files, all fairly short, with TiCS in their titles for this item.


Jan 21
:
Martin Luther King Day, Jr., Day (holiday, no classes)


Jan

23
:
Inter
active Activation and Bayesian Inference

Homework #2 Due


*
McClelland, J. L. (2013).
Bayesian inference, generative models, and probabilistic
computations in interactive neural networks.
Draft, Jan. 6. 2013, Department of
Psychology, Stanford University.
Pages
28
-
end.

Note: Find pdf in 01_09 directory.


!

Movellan, J., and McClelland, J. L. (2001). The Morton
-
Massaro Law of Information
Integration: Implications for Models of Perception.
Psychological Review
,
108
, 113
-
148.
Computational analysis extending
the response to Massaro to address broader
foundational issues.


_ Dean, T. (2005). A computational Model of Cerebral Cortex. Preprint of a paper in the
Proceedings of the AAAI
. Contemporary take on computational analysis of brain and
perceptual inferenc
e from a Bayesian perspective.


Jan
25
: Dynamics of Decision M
aking


*
Bogacz,R., Usher, M., Zhang, J. & McClelland, J. L. (2012). Extending a biologically
inspired model of choice: multi
-
alternatives, nonlinearity and value
-
based
multidimensional choice.

In A. K. Seth, T. J. Prescott and J. J. Bryson (Eds.),
Modelling
Natural Action Selection
. pp 91
-
119. Cambridge, UK: Cambridge University Press.


~

Gold, J. I. and Shadlen, M. N. The Neural Basis of Decision Making.
Annual Review
of Neuroscience, 30,

535
-
574


!

Wong, K.F. and Wang, X.J. A recurrent network mechanism of time integration in
perceptual decisions.
Journal of Neuroscience, 26
, 1314
-
1328.
Very interesting because
it links from detailed neuronal models to behavior.
The on
-
line supplement is
also
available in the class readings directory for those who are interested.


!

Bogacz, R., Brown, E., Moehlis, J., Holmes, P., &

Cohen, J.D. (2006). The physics of
optimal decision making: A formal analysis of models of performance in two
-
alternative
forced choice tasks.
Psychological Review
,

113,

700
-
65. This is a very technical paper
and is provided only for those interested.


Jan 2
8
: Hebbian
and Competitive
Learning


*

McClelland, J. L. (2006). How far can you go with Hebbian learning, and when does it
lead you astray? In Munakata, Y. and Johnson, M. H.
Processes of Change in Brain and
Cognitive Development: Attention and Perf
ormance XXI.

pp. 33
-
69. O
xford: Oxford
University Press.




Gazzaniga, M. S, Ivry, R. B., & Mangan, G. R. (1998). Cellular Basis of Memory. In
Cognitive Neuroscience: The Biology of Mind
(pp 283, 285
-
288). New Your: Norton.
(
This article is background

for those with no prior exposure to synapses and the biology
of synaptic modification.
)


~ Wikipedia article on Kohonen’s Self
-
Organizing Map:

http://en.wikipedia.org/wiki/Self
-
organizing_map


!

Miller, K.D. (1990a).
Correlation
-
based models of neural dev
elopment,

in
Neuroscience and Connectionist Theory, M.A. Gluck and D.E. Rumelhart, Eds.
(Lawrence Erlbaum Associates, Hillsdale NJ), pp. 267
-
353.
(
This
is
a harder
-
than
-
average article that explains in depth how ocular dominance columns may arise from a
s
imple learning process. Another Miller paper (Miller, 1995) covering a wider range of
models is available in the archive. Its first two figures are missing, but are the same as
Fig 1a and 1b of the above paper.
)


+

McClelland, J. L (in prep).
Explorations

in Parallel
-
Distributed Processing: A
Hand
book of models, programs, and ex
ercises.

Second Edition.

Chapter 6: Competitive
Learning
. Read from on
-
line handbook if interested.



+

Vallabha, G. K., McClelland, J. L., Pons, F., Werker, J. and Amano, S. (2
007).
Unsupervised learning of vowel categories from infant
-
directed speech.
Proceedings of
the National Academy of Science, 104,

13273
-
13278.


Jan 30
: Pattern Association: One
-
layer networks and learning rules


*

M
cClelland, J. L. (in prep
).
Exploration
s in Parallel
-
Distributed Processing: A
Hand
book of models, programs, and ex
ercises.

Second Edition
.
Chapter 4: Learning in
PDP

Models: The Pattern Associator
.


_

Rosenblatt, F.
(1958).
The Perceptron: A probabilistic model for information storage
and o
rganization in the brain.
Psychological Review, 65
,386
-
408.


_
Willshaw, D
.

J
.

(1981)

Holography, associative memory and inductive generalisation.
In
Parallel models of

associative memory
. G E Hinton & J A Anderson (eds), New
Jersey, Erlbaum, 83
-
104.


_
Kohonen, T. Oja, E. and Lehtio, P. (1981). Storage and processing of information in
distributed associative memory systems.
In
Parallel models of

associative memory
. G E
Hinton & J A Anderson (eds), New Jersey
, Erlbaum, 105
-
143
.


Feb 1
: Distributed Repres
entations in Memory and Knowledge Representation


* McClelland, J. L. and Rumelhart, D. E. (1985). Distributed memory and the
representation of general and specific information.
Journal of Experimental Psychology:
General, 114,

159
-
197.


_ Hinton, G. E., M
cClelland, J. L., & Rumelhart, D. E. Distributed Representations. In
Rumelhart, D. E., McClelland, J. L., and the PDP research group. (1986).
Parallel
distributed processing: Explorations in the microstructure of cognition. Volume I.

Cambridge, MA: MIT Pre
ss.


_
Bowers, J. S. (2009
)
. On the biological plausibility of grandmother cells: Implications
for neural network theories in psychology and neuroscience.
Psychological Review, 116
,
220
-
251.


_
Plaut, D. C., & McClelland, J. L. (2010). Locating object kno
wledge in the brain: A
critique of Bowers' (2009) attempt to revive the grandmother cell hypothesis.
Psychological Review, 117
, 284
-
288.


Feb 4
:
The Past Tense Debate

HWK
#
3 Due


*

McClelland, J. L., Patterson, K., Pinker, S. and Ullman, M. (2002). The
Past Tense
Debate: Papers and replies by S. Pinker and M. Ullman and by J. McClelland and K.
Patterson.

Trends in Cognitive Sciences, 6
, 456
-
474.


~ Rumelhart, D. E., & McClelland, J. L. On learning the past tenses of English verbs. In
McClelland, J. L., R
umelhart, D. E., and the PDP research group. (1986).
Parallel
distributed processing: Explorations in the microstructure of cognition. Volume II.

Cambridge, MA: MIT Press.


Feb 6
:

Back Propagation Learning Algorithm


* McClelland, J.L. (in preparation).
Ex
plorations in Parallel
-
Distributed Processing: A
Handbook of models, programs, and exercises, Second Edition
.
Chapter 5: Training
Hidden Units.



~

Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal
representations by error propag
ation. In Rumelhart, D. E., McClelland, J. L., and the
PDP research group. (1986).
Parallel distributed processing: Explorations in the
microstructure of cognition. Volume I.

Chapter 8, pp. 318
-
362. Cambridge, MA: MIT
Press.


Feb 8
:
Workshop: Applying B
ackprop to Semantic Cognition

Jay out: Cynthia to lead class


* Rumelhart, D.E. and Todd, P.M. (1993).
Learning and Connectionist Representations
.
Attention and Performance XIV, Synergies in Experimental Psychology, Artificial
Intelligence & Cognitive Neur
oscience, D.E. Meyer and Sylvan Kornblum, Eds.
Cambridge, MA MIT Press, pp. 3
-
30.


_

Rogers, T.T. and McClelland, J.L. (2005). A PDP approach to semantic cognition:
Applications to conceptual development. In L. Gershkoff
-
Stowe and D. Rakison (Eds),
Buildi
ng Object Categories in Developmental Time.
Mahwah, NJ: LEA.


Feb 11
:
Complementary Learning Systems

HWK #4 Due


* Read one of these papers (the second is a condensed version of the key points in the
first).


McClelland, J. L., McNaughton, B. L., and O'Re
illy, R. C. (1995). Why there are
complementary learning systems in the hippocampus and neocortex: Insights from the
successes and failures of connectionist models of learning and memory.
Psychological
Review, 102
, 419
-
457.


McClelland, J. L. (1996). Role
of the hippocampus in learning and memory: A
computational analysis. In T. Ono, B. L. McNaughton, S. Molitchnikoff, E. T. Rolls and
H. Nichijo (Eds.),
Perception Memory, and Emotion: Frontier in Neuroscience.

Oxford:
Elsevier Science, Ltd. 601
-
613.


NOTE:
Rest of Syllabus is Subject to Change


Feb 13
: Simple Recurrent Networks:
Finding Structure in Time


* Elman, J.L. (1990). Finding Structure in Time.
Cognitive Science 14
, 179
-
211. A
seminal article introducing an alternative approach to time and syntax.


~ Servan
-
Schreiber, D., Cleeremans, A., and McClelland, J.L. (1991). Graded State
Machines: The Representation of Temporal Contingencies in Simple Recurrent
Networks.
Machine Learning 7
, 161
-
193. Builds on Elman (1990) and provides the basis
for the exerc
ises for the SRN


+

McClelland, J.L. (in preparation).
Explorations in Parallel
-
Distributed Processing: A
Handbook of models, programs, and exercises, Second Edition.

Chapter 7
:
Simple
Recurrent

Networks
.


_ Elman, J.L. (1993). Learning and development in neural networks: the importance of
starting small.
Cognition, 48
, 71
-
99. This and the next article address an interesting issue
but come to very different conclusions. A side topic for those interested.


_

Rohde, D.L.T. and Plaut, D.C. (1999). Language acquisition in the absence of explicit
negative evidence: how important is starting small?
Cognition, 72
, 67
-
109.


Feb 15
: Connectionist Approaches to Language Processing

Meet with Jay or Cynthia by this Date

to Plan

Class Project


* Elman, J. L. (2009
). On the meaning of words and dinosaur bones: Lexical knowledge
without a lexicon.
Cognitive Science.

33
(4)
,

547

582.


~

McClelland, J.L., St. John, M., and Taraban, R. (1989). Sentence Comprehension: A
Paralle
l Distributed Processing Approach.
Language and Cognitive Processes, 4
, 287
-
335.


_ Rohde, D.L.T. (1999).
A Connectionist Model of Sentence Comprehension and
Production
. Unpublished PhD thesis proposal, School of Computer Science, Carnegie
Mellon Universi
ty, Pittsburgh, PA.


_ Tabor, W., Juliano, C., & Tanenhaus, M. K. (1997). Parsing in a dynamical system: An
attractor
-
based account of the interaction of lexical and structural constraints in sentence
processing.
Language and Cognitive Processes
, 12, 211
-
2
71.


Feb 18
:
President’s Day: No Class


Feb 20:
Learning in Recurrent Networks


*
Williams, R. J. and Zipser
, D. (1995). Gradient
-
based learning algorithms for recurrent
networks and their computational complexity. In: Y. Chauvin and D. E. Rumelhart (Eds.)
Back
-
propagation: Theory, Architectures and Applications,

Hillsdale, NJ: Erlbaum.


~

Movellan, J. R., & McC
lelland, J. L. (1993). Learning continuous probability
distributions with symmetric diffusion networks.
Cognitive Science, 17,

463
-
496.


~

O'Reilly, R.C. (1996). Biologically Plausible Error
-
driven Learning using Local
Activation Differences: The Generaliz
ed Recirculation Algorithm.
Neural Computation,
8,

895
-
938.


+ McClelland, J.L. (in preparation).
Explorations in Parallel
-
Distributed Processing: A
Handbook of models, programs, and exercises, Second Edition.

Chapter 8: Recurrent
Backpropagation.


Feb 22
: Disorders of Lexical and Semantic Processing

One
-
Page Project Proposals Due


*
Dilkina, K., McClelland, J. L. & Plaut, D. C. (2008). A single
-
system account of
semantic and lexical deficits in five semantic dem
e
ntia patients.
Cognitive
Neuropsychology, 2
5(2)
, 136
-
164.


_

Rogers, T. T., Lambon Ralph, M. A., Garrard, P., Bozeat, S., McClelland, J. L.,
Hodges, J. R., and Patterson, K. (2004). The structure and deterioration of semantic
memory: A neuropsychological and computational investigation.
Psychological Review,
111
, 205
-
235.


_
Plaut, D. C.

and Shallice, T. (1993). Deep dyslexia: A case study of connectionist
neuropsychology.
Cognitive Neuropsychology
,
10
, 377
-
500.


_
Plaut, D. C., McClelland, J. L., Seidenberg, M. S., and Patterson, K. (19
96).
Understanding normal and impaired word reading: Computational principles in quasi
-
regular domains.
Psychological Review
,
103
, 56
-
115.


Feb
25
:
Executive functions and attention


* Cohen, J. D., Dunbar, K. and McClelland, J. L. (1990). On the Control o
f Automatic
Processes: A Parallel
-
Distributed Processing Account of the Stroop Effect.
Psychological
Review, 97,

332
-
361.


~

Botvinick, M. and Plaut, D. C. (2004). Doing without schema hierarchies: A recurrent
connectionist approach to normal and impaired
routine sequential action.
Psychological
Review, 111,

395
-
429.


~

Moody, S. L., Wise, S. P., di Pellegrino, G., & Zipser, D. (1998). A model that
accounts for activity in primate frontal cortex during a delayed matching
-
to
-
sample task.
The
Journal of Neu
roscience, 18
(1), 399
-
410.


_ Cohen, J. D., Servan
-
Schreiber, D., & McClelland, J. L. (1992). A parallel distributed
processing approach to automaticity.
American Journal of Psychology, 105,

239
-
269.


_ Miller, E. K. and Cohen, J. D. (2001). An Integrative Theory of Prefrontal Cortex
Function.
Annual Review of Neuroscience, 24,

167
-
202.


Feb 27
: Deep Belief Networks


*
Hinton,

G. E. and Salakhutdinov, R. R.
(2006).

Reducing the dimensionalit
y of data
with neural networks.
Science,

Vol. 313. no. 5
786, pp. 504
-

507
.


_
Ranzato, M. Boureau, Y
-
L, and LeCun, Y.
(2007).
Sparse feature learning for deep
belief networks, Advances in Neural Information Proc
essing Systems (NIPS 2007).


_
Bengio, Y., Lamblin
, P., Popovici, D. & Larochelle, H. (2006) Greedy layer
-
wise

training of deep networks.

Technical Report, 1282, Univ of Montreal.



Mar 1
: Reinforcement Learning


*/+

McClelland, J.L. (in preparation).
Explorations in Parallel
-
Distributed Processing: A
H
andbook of models, programs, and exercises, Second Edition.

Chapter 9
:
Temporal
Difference Learning.


Mar 4: Efficient Sensory Coding


* Olshausen, B. A. and Field, D. (2004). Sparse coding of sensory inputs.
Current
Opinion in Neurobiology, 14
:481
-
487.


~ Smith, E. C. & Lewicki, M. J. (2006). Efficient auditory coding.
Nature, 439
, 978
-
982.


~ Karklin, Y., and Lewicki, M. S.
Emergence of complex cell properties by learning to
generalize in natural scenes
.
Nature, 457,

83
-
86.


March 6
: The Binding Problem

and Complementary Object processing systems in
Dorsal and Ventral Cortex

Lecture by Cynthia Henderson


Readings to be d
etermined


March 8
:
Bayesian and Connectionist Approaches to Learning in Neural Networks


Readings to be determined


Mar

11, 13
: Project

Presentations



March 15
: Wrap
-
up in class discussion


Mar 20
, 5:00 pm. Final Project Due