Model of familiarity discrimination in the brain efficiency, speed and robustness

sciencediscussionAI and Robotics

Oct 20, 2013 (3 years and 10 months ago)

82 views

Model of familiarity discrimination in the brain


efficiency, speed and robustness



Rafal Bogacz

Malcolm W. Brown

Christophe Giraud
-
Carrier


Dept. of Computer Science

Dept. of Anatomy

Dept. of Computer Science


University of Bristol

University of Bristol

University of Bristol


Bristol BS8 1UB, U.K.

Bristol BS8 1TD, U.K.

Bristol BS8 1UB, U.K.


R.Bogacz@bristol.ac.uk

M.W.Brown@bristol.ac.uk

cgc@cs.bris.ac.uk

Abstract

Results of psychological and neurological experiments suggest that familiarity discriminati
on
(recognising whether a stimulus is familiar or novel) is distinct from recall (recognising the
stimulus itself). The perirhinal cortex, part of the temporal lobe, has been shown to be
essential in familiarity discrimination. Here, we describe a neural n
etwork model of the
perirhinal cortex. The spike trains of the simulated neurons of the spike
-
response version of
the model are virtually indistinguishable from those recorded from monkeys’ brains. The
network is very efficient, it achieves such high compr
ession of information that it may
explain how the comparatively small perirhinal cortex can discriminate familiarity for a
much larger number of stimuli than can be recalled by the remaining areas of the cortex. It is
also very fast


although so high capa
city, it may compute familiarity within 10 ms. The
network is very robust to damage and noise as well.

1

Introducti on

The perirhinal cortex, part of the temporal lobe, has been shown to be essential in familiarity discrimination.
Removal of the perirhinal co
rtex in monkeys [10] and damage including this area in amnesic patients [1]
result in impairments in tasks that rely on discrimination of the relative familiarity of objects or pictures.
Within the perirhinal cortex, a proportion (~25%) of neurons respond
strongly to the sight of novel objects but
respond only weakly or briefly when these objects are seen again [6]. This is illustrated in Figure 1a.

We have shown that checking the value of the energy function of a Hopfield network is a very efficient
method

of familiarity discrimination [3]. By sacrificing the ability to extract patterns


not strictly needed for
familiarity detection


an amazing jump from the normal capacity for retrieval of 0.145
N

to a capacity for
novelty discrimination of 0.023
N
2

is ach
ieved. In this paper, we present a neural network model of the
perirhinal cortex, which achieves great compression of information and performs familiarity discrimination
efficiently. The network’s implementation is based on the spiking neurons framework, a
nd the behaviour of
the simulated neurons (shown in Figure 1b) is very similar to that recorded from monkeys’ perirhinal cortices.










Figure 1. Response to first and repeat presentations of ten novel pictures of: a, perirhinal neurons recorded
from

monkey’s brain; b, simulated neurons in the model

Stimulus

First

Repeat

s

30 spikes/s

Stimulus

a Real neuron

b

Simulation



Since the principles of familiarity discrimination used in the model of the perirhinal cortex are similar to those
introduced in [3], we first give a brief overview of the method, which is based on checki
ng the value of an
appropriate energy function. We then show how this method may be implemented in a neural network and
describe our model of the perirhinal cortex. Finally, we report on the surprisingly high storage capacity of the
network, its speed and
robustness.

2

Fami l i ari ty di scri mi nati on i n Hopfi el d nets

A Hopfield network is a simple model of associative memory [8]. It is a fully connected recurrent neural net
consisting of
N

units, whose activation is denoted by
x
i
. The active state of a neuron is
represented by 1, and
the inactive state by

1. The patterns stored by the network are denoted by



and the number of these patterns
by
P
. The weight of the connection between units
j

and
i

is denoted by
w
ij

and computed according to the
Hebb rule [8]:



(1)

The energy of the Hopfield network is defined by [8]:



(2)

It is easy to show that the average value of the energy for stored patterns is

N
/2, while for novel patterns it is
0 [3]. Therefore, by taking as

a threshold the middle value

N
/4, we can define a familiarity discrimination
criterion, namely, if
E

<
-
N
/4, then the pattern is considered familiar, otherwise it is novel [3]. We can prove
that assuming random patterns, the variance of the energy functi
on is equal to 2
P

[3]. Therefore, the signal
-
to
-
noise analysis shows that, accepting a probability of error of 1%, the capacity of the Hopfield network for
familiarity discrimination is equal to 0.023
N
2

[3]. This capacity is much greater than the standard
capacity of
the Hopfield network for retrieval, namely 0.145
N

[8].

3

Does the brai n compute the energy of a Hopfi eld net?

The higher capacity of the neural network for familiarity discrimination than for retrieval is consistent with
the fact that we can ofte
n tell if a stimulus is familiar to us, even if we cannot recognise the stimulus itself.
This suggests that our brain may use a similar principle for familiarity discrimination. However, the energy of
the Hopfield network is an artificial function, whose v
alue is not computed by the neural network. But in the
equation for the energy there are summations, which are typical operations for neurons, and after some
modification this equation may be computed by a neural network. In fact, there is a lot of network
s which may
calculate this energy. During our research we have tried many architectures and finally we found one which
generates spikes very similar to those recorded from monkey’s brain (see Figure 1).

In this section, we describe how the checking of the
energy function may be implemented in the neural
network and as a result we present the architecture which generates behaviour shown in Figure 1. Let us
define a decision function, which is equal to 1 for familiar patterns and to

1 for novel ones. From se
ction 2,
the simplest such function is given by:



(3)

We showed in [4] that this function may be approximated by:



(4)

The inner part of equation
(4)

is a typical function of a neuron, hence let us denote it
by
y
i
. Function
d

may
thus be implemented by a single neuron, as follows:




where:

(5)

Hence, the familiarity discrimination may be performed by a neural network with two processing layers, as
shown in Figur
e 2. The output unit of the network computing
d

is called the decision neuron and the hidden
units computing
y
i

are called Familiarity Discrimination Neurons (FDNs) [4]. The network includes the same
number of FDNs as input units. The weights of FDNs are
set up according to the Hebb rule


analogously
like the weights of the Hopfield network. In equation
(5)
, the summing is done only over the activation of
those FDN
i

for which
x
i

= 1. In the network, this is implemented by privileged connections between in
put
units and corresponding FDNs (denoted by double lines in Figure 2). The privileged connections have the
highest synaptic weights (not changed during learning), which in practice ensures that to activate a FDN, the
corresponding input neuron must also b
e active.










Figure 2. Architecture of the network in the model of perirhinal cortex

4

Model of the peri rhi nal cortex

The model operates in two phases. During a brief initial period, the familiarity of a pattern is discriminated
using computations sim
ilar to those of section 3. This initial period corresponds to the short interval in which
the neurons are active for both novel and familiar patterns (see Figure 1a). In the subsequent longer period,
information storage is effected. If, in the initial per
iod, the network classifies the stimulus as novel, the
neurons continue to fire, thus allowing the modification of synaptic strengths and the subsequent
memorisation of the stimulus. If, on the other hand, the network recognises the stimulus as familiar, t
he
neurons have no need to continue firing (see Figure 1a). Hence, in the proposed model, only the first spike(s)
in an action potential train is used for computation and the remaining spikes are used only to cause high
frequency activity resulting in the
modification of weights.

The decision neuron is the output of the network and, after the initial period of familiarity discrimination, it
should govern the subsequent activity of the network (i.e., high frequency activity for novel patterns, no
activity fo
r familiar ones). Many neuronal circuits might be used to achieve such behaviour. Here, for
simplicity, we use an inhibitory circuit. FDNs receive inputs from inhibitory neurons (dotted lines in Figure
2), which are triggered by the decision neuron for nov
el patterns causing FDNs to cease firing [4].

In the model, an active state of a neuron is denoted by 1, while an inactive state by 0 (in contrast to the
Hopfield model, where it was denoted by

1). Additionally, both the decision neuron and the FDNs have

positive weights (as the pyramidal neurons whose recordings are shown in Figure 1a), which in the case of the
FDNs is achieved by the weights’ initiation with a positive constant. Details of the model may be found in [5].

In the capacity estimates of sect
ion 2, we assumed an acceptable probability of error. One type of error the
network may make is to classify a novel pattern as familiar and hence experience a kind of déjà vu. The
patterns for which this error may be made correspond to spurious attractors
in the Hopfield network


states
having lower values of the energy function than the stored patterns [8]. By decreasing the number of stored
pattern
P
, the probability of such errors may be decreased, but the spurious attractors exist even for very small
P

[8]. Furthermore, the erroneous activation of a sufficient number of FDNs will also lead to déjà vu. This
property is consistent with observations that stimulation or epileptic attacks involving the temporal lobe may
result in déjà vu [2].

trigger

x
j

y
i

d

w
ij


5

Capaci ty, speed

and robustness of the peri rhi nal network model

It is easy to show that the capacity for familiarity discrimination of the network used in the model of the
perirhinal cortex is half that of the capacity of the Hopfield network namely 0.012
N
2

(assuming a pr
obability
of error of 1%) [5]. This estimation is consonant with results of numerical experiments [5].

Using the above prediction of capacity, we can estimate the theoretical capacity of the human perirhinal
cortex. The average volume of the human perirhin
al cortex is about 5 cm
3
, the size of a playing dice [9].
Hence, there are ~10
7

pyramidal neurons in the perirhinal cortex, each having ~10
4

synapses. Assume, as
suggested by experiments, that 25% of these neurons are novelty responsive [6]. If the probabi
lity of error is a
mere 10
-
6
, the network may store about 10
8

patterns, and since there are 2.5 * 10
6

novelty responsive neurons,
each pattern may consists of 2.5 * 10
6

bits. If these many patterns were to be written into books and these
books placed one o
n top of another, then the pile would have a height of 7000 km, which is approximately the
radius of the Earth. The speed of searching this database is also impressive. It would take 20 ms for light to go
from one end of the pile to the other, while discri
minating familiarity in the perirhinal network model takes
only about 10 ms.

The network is also very resistant to damage


the model will work after the removal of any number of FDNs’
input connections and even after the removal of FDNs with the capacity

per synapse remaining the same. The
network demonstrates generalisation to disruption by noise


a pattern will still be classified as familiar even
if it differs in a substantial proportion of bits from its previous representation.

6

Concl usi on

This paper
presents a model of familiarity discrimination in the perirhinal cortex, which exhibits behaviour
virtually indistinguishable to that recorded from monkeys’ brains. The proposed network has very high
storage capacity for familiarity discrimination, which i
s much greater than the capacity of neural networks for
retrieval. This difference in capacities is so large, that it is possible for a relatively small number of perirhinal
neurons to store occurrences of more patterns than could be retrieved using the fa
r greater number of neurons
available in other parts of the association cortex. The model suggests that there may be two separate processes
in the brain for familiarity discrimination and for storage/retrieval.

Acknowl edgment s

This work is supported in par
t by an ORS grant held by the first author.

Ref erences

[1]

Aggleton, J.P. & Brown, M.W. (1999) Episodic memory, amnesia and the hippocampal
-
anterior thalamic
axis.
Behav. Brain Sci.

In press.

[2]

Bancaud, J., BrunetBourgin, F., Chauvel, P. & Halgren, E. (1
994) Anatomical origin of déjà vu and vivid
'memories' in human temporal lobe epilepsy.
Brain

117
:71
-
90.

[3]

Bogacz, R., Brown, M.W. & Giraud
-
Carrier C. (1999) High capacity neural networks for familiarity
discrimination. In
Proceedings of ICANN’99
. To app
ear.

[4]

Bogacz, R., Brown, M.W. & Giraud
-
Carrier C. (1999) How the brain may solve familiarity
discrimination? Submitted to
Nature Neuroscience
.

[5]

Bogacz, R., Brown, M.W. & Giraud
-
Carrier C. (1999) Model of familiarity discrimination in the brain
and d
éjà vu. Submitted to
NIPS
.

[6]

Brown, M.W. & Xiang, J.Z. (1998) Recognition memory: Neuronal substrates of the judgement of prior
occurrence.
Prog. Neurobiol.

55
:149
-
189.

[7]

Fohlmeister, C., Gerstner, W., Ritz, R. & van Hemmen, J. (1995) Spontaneous excit
ation in the visual
cortex: stripes, spirals, rings and collective bursts.
Neural Comp.

7
:905
-
914.

[8]

Herz, J., Krogh, A. & Palmer R. G. (1991)
Introduction to the theory of neural computations
, Addison
-
Wesley

[9]

Insautri, R. et al. (1998) MR volumetric

analysis of the human entorhinal, perirhinal and temporopolar
cortices.
Am. J. Neuroradiology

19
:659
-
671.

[10]

Murray, E.A. What have ablation studies told us about the neural substrates of stimulus memory?
Semin.
Neurosci.

8
, 13
-
22 (1996).