Does the Conventional Leaky Integrate-and-Fire Neuron Synchronize Spikes in Multiple Firing Mode?

foulchilianAI and Robotics

Oct 20, 2013 (3 years and 1 month ago)

78 views


1

Does t
he Conventional Leaky Integrate
-
and
-
Fire Neuron
Synchronize Spikes in Multiple Firing Mode?



(
Synchronization/ Integrate and Fire Neuron/ Spike time Distribution/ Refractory Period/ Neural Network)




Baktash Babadi, Ehsan Arabzadeh, Arash Yazda
nbakhsh, Shahin Rouhani












Abstract


The importance of synchronization of firing in a neuron pool is widely emphasized in many
studies. In most of the computational studies concerned with the synchronization mechanism in
feed
-
forward neural poo
ls, only a single spike (first spike) of each neuron is analyzed. In this
paper we argue that a more realistic setting is when each neuron fires multiply. It is shown that
unlike the simple case of single firings for each neuron, in the more realistic cond
ition, the
synchronization of firings in successive pools of leaky integrate and fire neurons is impossible.
We have confirmed this through the results of our simulation as well as analytic results. Finally we
present two possibilities in order to explain
the physiological experiments where presence of
synchronization is evident.



1. Introduction


Considerable evidence indicates that neurons in different cortical areas are capable of producing
synchronous action potentials on the time scale of millis
econd (Bair 1994, Bair 1996, Marsalek
1997) and the synchronization may play functional roles in neural processing (Abeles 1993, Prut
1998, Riehle 1997, Singer 1993).


Apart from the physiological evidence, synchronous firing in a group of neurons is of gr
eat
importance from computational viewpoint. A neuron that receives many simultaneous inputs, is
more likely to generate an action potential than one which receives the same inputs distributed
over a wider time range. Thus, it is believed that synchronous
activity provides an efficient mean
to increase the reliability of responses and to eliminate noise in neural assemblies (Diesmann et
al 1999). Besides, synchronization can be assumed as a mechanism by which spatially separated
neurons, responding to the s
ame stimulus, bind together to make up a functional group (Usher
1993, Engel 1991).



2

A number of studies have addressed the mechanism of synchronization in a feed
-
forward neural
network (Hermann et al 1995, Diesmann 1999, Marsalek 1997, Feng 1997, Bu
rkitt 1999). In
some of these studies, the problem is considered in the neuron pool level (Hermann 1995,
Diesmann 1999), while the others approach the problem in a single neuron level (Marsalek 1997,
Feng 1997, Burkitt 1999), which is then generalized to t
he pool level (Diesmann 1999).


The most commonly used pulse generating neuron in the studies is the Integrate
-
and
-
Fire (I&F)
neuron model. Although the (I&F) model is one of the oldest proposed neuron models (Lapicqe
1907), it mimics the most important pr
operties of the real neuron, such as temporal summation of
inputs and firing due to reaching a threshold (Koch 1999), and has been widely used in modeling
the neural structures (Tuckwell 1988). Despite the simplicity of the I&F model in comparison with
the

detailed descriptive Hodgkin
-
Huxley neuron model (Hodgkin, Huxley 1952), it has been
shown that

many biophysically detailed and biologically plausible Hodgkin
-
Huxley
-
type neural
networks can be transformed into I&F form by a piece
-
wise continuous change o
f variables
(Hoppensteadt , Izhikevich 1997).


The method common in the cited studies (Hermann et al 1995, Marsalek 1997, Feng 1997,
Burkitt 1999, Diesmann 1999), is to present a number of spikes with a known temporal
distribution (a pulse packet) as an i
nput to a pulse generating neuron (or neuron pool) and
investigating the spike response of the neuron (or neuron pool). It showed that the temporal
variance of the spike response is less than the temporal variance of the input pulse packet in
time, i.e. th
e output pulse packet is more synchronized than the input pulse packet.


Some of these studies (Marsalek 1997, Feng 1997, Diesmann 1999) have assumed that a
neuron generates a single spike, and analyzed the variance of this solitary spike in the recipien
t
neuron. The other studies, on the other hand have inferred the distribution of spikes through
evaluating the variance of the first spike times, and assumed that this variance is a good
representative of the general distribution of spikes (Burkitt 1999).
Thus the common trend in the
cited studies is single spike analysis. But in the more realistic case where the neuron generates
multiple spikes in a short time interval, what will be the out put spike time distribution as a whole?




This article addresses

this question. In the next section, the time distribution of output spikes in a
single neuron in response to an incoming pulse packet is evaluated analytically with minimal
approximations. In the third section, the analytical results are confirmed through

a computer
simulation. Finally, the efficiency of the conventional I&F feed forward network, in issuing the
synchronization phenomena is discussed.




















3



2. Analysis of Output Spikes Distribution


Here, we consider a leaky I&F neuron with

a large number of input connections (Fig. 1).
























Fig.1, A pool of
N

Integrate and fire Neurons, feeds its
output to a neuron. The received input pulse packet has
a normal distribution in time. In response, the recipient
neuron genera
tes an output pulse packet.






The membrane potential of the leaky integrate
-
and
-
fire neuron in its sub
-
threshold regime is
governed by:


, (1)


where
is the sum of excitatory and inhi
bitory spikes arriving to the neuron at time
t
and

is the membrane time constant. Whenever the changing membrane potential reaches a
constant value (threshold), a spike is generated as:


,

(2)


where,

is the impulse function and

is the time in which the neuron has reached the
threshold. Immediately after reaching the threshold, the membrane potential is rendered to its
resting
potential, which for the sake of simplicity is set to zero. The resting potential is also
maintained for a time window named its refractory period (
r
).

The recipient neuron receives its input from a pool of firing neurons (Fig. 1), so its input can be
r
epresented by:



4

, (3)


where

(
s=1,2,..
) are the spike times of the feeding neuron
j
and
N

is the total number of
neurons in the feeding pool.




For a moment, s
uppose that the neuron is not allowed to fire (Fig.2). Substituting equation (3) in
(1) and solving the resultant differential equation in terms of
t

yields:


. (4)


Suppose that the neurons in the feeding pool have p
roduced an overall number of
spikes with
a normal time distribution of:



, (5)


where,

is the standard deviation of the spike times in the input layer (F
ig.1).

Due to a large enough
, one can change the above discrete summation (equation 4) into
integration:


. (6)


As we have a narrow distribution for connecting weights, we could assume

independent of
x
,
hence

(mean weight) is put out of integral as the representative of
s. Fig. 2 plots the
membrane potential
versus time.




5



F
ig. 2, The membrane voltage of an Integrate
-
and
-
Fire neuron in response to its feeding inputs, which are
normally distributed in time. This neuron is not allowed to fire, hence, the voltage dynamic is the result of
incoming input and membrane leakage witho
ut resetting potential to zero after firing. This can be considered
as the graphical presentation of equation (4).



Now we consider the case where the threshold is present and the neuron fires consecutively in
times
, in respons
e to the incoming pulse packet (Fig.3).

is the total number of
generated spikes
.

Note that here, the neuron is allowed to generate multiple spikes in response
to its incoming pulse packet. Hermann 1995, Feng 1997, Marsalek 1997, Bur
kitt 1999 and
Diesmann 1999 have analyzed the problem in case the neuron generates only a single spike.







Fig. 3, The neuron fires whenever reaches the threshold, then the membrane potential resets to zero. Thereafter, the
neuron ga
ins the remaining pulse packet to produce the next spike. This procedure continues until the potential cannot
reach the threshold; and decay is the case after weak peak value.





6


If each neuron generates multiple spikes in response to its input pulse pac
ket, let us consider the
time distribution of spikes. After each spike is generated, the membrane potential is set to zero
and remains there for its refractory period,
r
. So in each inter
-
spike interval
, the
membrane potential is ze
ro for

and the integration takes place over the time
interval
, so in order to show the membrane potential in the latter interval we
rewrite equation (6) as:


,


(7)


where,

represents the membrane potential in
. Considering equation (6)
we can rewrite equation (7) as (see Appendix A):


, (8)


where,
.


Note again that

represents the membrane potential if firing is not allowed, while

is
the potential in the actual case where the neuron could have generated spikes.


Obviously, the m
embrane potential reaches the threshold
th
, in the firing times
.

So, for each
,
=
th
. Using equation (8), we have:



. (9)


By iterating equation (9) on
itself and taking into account that
, it can be written as:


. (10)


Assuming that the number of generated spikes (
) is considerably large and the spikes are
close together, we rewrite the above summation in terms of integration:


, (11)


where

is the time density distribution of the
generated spikes of the neuron
.



represents the ratio of generated spikes in
. So, we have to multiply it by
to
reach the number of spikes in each integration interval (
).


Comparing Equation (11) and (6) yields:



7

.





(12)



Now, it is possible to obtain the time distribution of the generated spikes in terms of the input
spikes parameters:


1





(13)




The varian
ce of the output spikes is (
see Appendix B
):



, (14)


where
K
is positive.


Equation (14) shows that
. The order of the refractory period is small (
sec,
Patton 1989), so, at best (when refractory period is negligible)
. This result is
interesting, because, in any case there remains no route for such a model to compress its output
spike packet compared to its input pulse packet.




3. Simulation


We assess the above discussion through a computer simulation on a PC. In the simulation we
studied the time dependent behavior of a single leaky integrate and fire neuron. The model
neuron receives a packet of
1000

spikes as input, which

are normally distributed in time around
t=0
with a standard deviation of 10
msec
. To be a close approximation to the biological reality, the
membrane time constant is set to
20 msec
, the threshold is set to
20 mV
above the resting
potential (as the restin
g potential is assumed zero for simplicity, the threshold is equal to 20
mV

here) and the refractory period is assumed equal to 1.75
msec
(McCormik et al 1985). Regarding
equation (6), each spike raises the EPSP by the value of
. So,

the mean input connection
weight is set to 20
mV.msec

to mach the intracellular recordings which revealed a nearly 1
mV of
EPSP rise per a single input spike (Mason et al 1991). The membrane potential change of the
model neuron was approximated by piece w
ise linear solution of the differential equation (1).


The simulation results can be seen in Fig. 4. Each input is presented by a vertical bar, and
obviously, the inter spike intervals (ISI) are the distance between the bars. Fig. 4a shows the
input spikes

to a neuron. Regarding the normal distribution of input spikes in time, the vertical
bars are denser in the center.


Fig. 4b,c shows the output spikes generated by the neuron in response to the input pulse (spike)
packet shown in Fig. 4a. As mentioned by
Mason et al 1991, approximately 20 spikes are needed
to trigger the neuron’s action potential. So, it was somehow predictable that the output firing
pattern should be sparser than its corresponding input.


Fig. 4b illustrates the output spiking pattern
in the case where no refractory period is
implemented. The time window of firings is trimmed from both sides, and the generated spikes
are sparser in time.



8

Now consider the case in Fig. 4c, where the refractory period is added and compare it with Fig.
4b
. In the presence of refractory period the firing pattern is sparser because in the refractory
period, a number of incoming spikes are neglected, so the ‘effective’ input to the neuron will be
smaller, resulting in a sparser output spike pattern.


Note th
at the narrowing of the output time band does not necessarily imply a decrease in the
standard deviation of spikes in time, which is usually considered the criteria of synchronization in
the literature. There is a tradeoff between the time window narrowing

and the sparseness of the
generated spikes, in contributing to synchronization, i.e. standard deviation in time. In other
words, in a constant time band when spikes become sparser, the inter
-
spike intervals become
greater, so the standard deviation increa
ses, which is more striking in the presence of refractory
period, as is shown analytically by equation (14).






a




b




c



Fig. 4,
a
) The input that a single model neuron receives from its connections. A vertical bar presents each inp
ut spike
which are normally distributed in time. Input bars are denser in the center and sparser in the periphery. The neuron sums
them up temporarily according to its dynamic.
b
) Generated spikes by the recipient neuron with no refractory period.
Because

~20 input spikes are needed to trigger an out put spike the trimming of the time window from both side occurs.
For the same reason compared with 4a the spikes are sparser and the standard deviation is larger.
c
) Output pulse
packet when refractory period
is present. It is sparser and its standard deviation in time is even greater than the case in
4b (see equation 14).



4. Discussion


This study was aimed at evaluating the capability of the leaky integrate
-
and
-
fire neuron to
synchronize its output spikes

in comparison with its input spikes, when it generates multiple
spikes. Our results show that in such a neuron, the output spikes variance is equal or greater
than the input spikes variance, which means that it fails to synchronize its output spikes. This

result can be generalized to the recipient neuron pool level, consisting of identical independent
neurons i.e. the time density distribution of the total generated spikes in the pool is equal to that
of single neurons (See Appendix C).


But as mentioned
before, synchronization is of great importance in the neural assemblies.
Given
that the cortical neurons operate in a noisy environment, in the absence of a synchronizing
mechanism, there will be a permanent tendency for desynchronizing the spikes in corti
cal neural

9

assemblies. To put it another way, even in case of
, one will face a progressive
asynchrony, because of an inevitable noise.
Some of the sources of this noise, which tend to
desynchronize the generated
spikes of a neuron po
ol, could be listed as:



1
-

The differences between the axonal and dendritic lengths and diameters in different
neurons of a pool (
Manor et al, 1991).

(Geometrical noise)

2
-

Variation of the delay between pre
-
synaptic spike arrival and post
-
synaptic channel
ope
ning, in different synapses. (Synaptic noise)

3
-

The noise due to spontaneous firings of the neurons, which is often treated as a Poisson
process. (Spontaneous noise)


Thus, if the cortical neural groups are assumed to be arranged in a feed
-
forward manner (Ab
eles
1991), in the absence of a synchronizing mechanism in the single neuron level, the activity
(spiking) pattern of successive pools desynchronizes or rounds off through the hierarchy.


Yet, this is not the case in the cortex; it has been shown that in
visual cortical hierarchy, the
frequency rise time of the neurons in successive neuron pools remains relatively constant and
accurate in time (Marsalek 1997).


Taking into account the above biological fact, a minimal degree of synchronization is needed at
least to oppose the noise disturbance to prevent the rounding off of the packet of the spikes.





As a conclusion, the above mismatch between the modeling results and the experimental data
can be solved considering either or both of the following assump
tions:


a)

The leaky integrate
-
and
-
fire neuron is an oversimplified model to present the
synchronization phenomena in the single neuron level.


b)

The interconnections between the neurons in a pool (intra
-
pool connections) are
responsible for synchronizing the
firing activity of the neurons in the pool, so the feed
-
forward structure appears to be inappropriate as a model for cortical neural arrangement.




Appendix A


Equation (7) can be written as:


,

(15)


Assuming that the refractory period
r
is considerably short, equation (15)

can be approximated by
Taylor expansion as:


.
(16)


Considering equation (6)

yields:



10

,


which is exactly equation (8).





Appendix B


By replacing helping constants

and
in equation (13) one can
obtain:


. (17)


By dif
ferentiation in terms of
t
, one reaches to:


, (18)



which yields:



, (19)


where
.



By defi
nition of the variance:

, (20)


and substituting equation (19) in (20):

. (21)


The left phrase of the right side is zero, so:


. (22)

Given that
is a density distribution function,
, so:



(23)


11


is a positive bell shaped function, because its derivative,
, is zero at t=0,
positive for
t < 0
, negative for
t > 0
, and near zero at

and

. So
is positive too,
which we will show it by
K
. Substituting the values of
l

and
m
in the above equation (23)
,
Results
in:


,

which is exactly the equation (14).



Appendix C


Let us suppose that
the recipient pool consists of
N
neurons. As the neurons are identical and the
distribution of the connecting weights is narrow, the output spikes time distribution for all the
neurons in this
pool are identical and equal to
. So, in
a small time interval
dt
, the total
number of generated spikes by the pool is
. On the other hand, if we consider
the time density distribution of the pool spikes
-
as a whole
-

as
, in the small time interval
dt
, the total number of generated spikes by the pool will be
, where

is the
total generated spikes by the pool. Given that
, yields:
.



References


Abeles M., Bergm
an H., Margalit E., Vaadia E., Spatiotemporal Firing Patterns in the Frontal
Cortex of Behaving Monkeys, J. Neurophysiol., 70, 1629
-
1638, 1993.


Bair W, Koch C, Newsome WT, Britten KH, Reliable temporal modulation in cortical spike trains in
awake monkey.
Society for Neuroscience, Abstracts 20(2):1279 (#525.7), 1994.


Bair W, Koch C., Temporal precision of spike trains in extrastriate cortex, Neural Computation,
8:1185
-
1202, 1996.


Burkitt, A. N., and Clark, G. M., Analysis of integrate
-
and
-
fire neurons: sy
nchronization of synaptic
input and spike output, Neural Computation 11(4), 871
-
901, 1999.


Diesmann M., Gewaltig M., Aertsen A., Stable Propagation of Synchronous Spiking in Cortical
Neural Networks, Nature, Vol. 402, 2 Dec. 1999.


Engel, A. K., Konig, P,

Kreiter, A. K., & Singer, W., Interhemispheric synchronization of oscillatory
neuronal responses in cat visual cortex, Science, 252: 1177
-
79, 1991


Feng J., Behaviors of Spike Output Jitter in the Integrate
-
and
-
Fire Model, Physical Review
Letters, Vol. 79
, Number 22, Dec. 1997.



12

Herrmann M., Hertz J. A., Prugel
-
Bennett A., Analysis of Synfire Chain ,
Network, 6:403
--
414,
1995.


Izhikevich E.M., Weakly Pulse
-
Coupled Oscillators, FM Interactions, Synchronization, and
Oscillatory Associative Memory, IEEE Trans
actions On Neural Networks, 10:508
-
526, 1999


Koch C.,
Simplified Models of Individual Neurons, in
Biophysics of Computation: Information
Processing in Single Neurons, Oxford University Press, 1999


Lapique, L. Recherches quantitatives sur l’excitation e
lectrique des nerfs traitee comme une
polarization. J. Physiol. Paris 9:620
-
635, 1907.


Manor Y., Koch C., Segev I., Effect of geometrical irregularities on propagation delay in axonal
trees, Biophys. J., 60: 1424
-
1437, 1991.


Marsalek P., Koch C., Maunsel
l J., On the relationship between synaptic input and spike output
jitter in individual neurons, Proc. Natl. Acad. Sci., Vol 94, pp 735
-
740, Jan. 1997


Mason A., Nicoll A. and Stratford K. Synaptic Transmission between Individual Pyramidal
Neurons of the
Rat Visual Cortex
in vitro
. The Journal of Neuroscience, 11:72
-
84., 1991.


McCormick D. A., Connors B. W., Lighthall J. W., and Prince D. A., Comparative
electrophysiology of pyramidal and sparsely spiny stellate neurons of the neocortex. J.
Neurophysiol
., 54:782
-
805, 1985.


Patton, H. D. Fuchs, A. F., Textbook of Physiology: Vol. 1 Nervous System, 1989.


Prut Y. et al Spatiotemporal structure of cortical activity: Properties and behavioral relevance. J.
Neurophysiol., 79, 2857
-
2874, 1998.


Riehle A, Grun

S, Diesmann M and Aertsen A (1997): Spike synchronization and rate modulation
differentially involved in motor cortical function. Science 278, 1950
-
1953.


Singer W., Synchronization of cortical activity and its putative role in information processing and
learning, Annu. Rev. Physiol. 55: 349
-
374, 1993.


Usher M., Schuster H. G., Neibur E., Dynamics of populations of integrate
-
and
-
fire neurons,
partial synchronization and memory, Neural Comput. 5: 570
-
586, 1993.