New measures for estimating neural network structures only from multi-spike sequences

companyscourgeΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 11 μήνες)

80 εμφανίσεις

2007 International Symposium on Nonlinear Theory and its
Applications
NOLTA'07, Vancouver, Canada, September 16-19, 2007
New measures for estimating neural network structures
only from multi-spike sequences
yz y y
Tohru Ashizawa , Daisuke Haraki , and Tohru Ikeguchi
yGraduate School of Science and Engineering, Saitama University,
255 Shimo-Ohkubo, Sakura-ku, Saitama 338-8570, Japan
zE-mail: ashizawa@nls.ics.saitama-u.ac.jp
Abstract In neural systems, a fundamental element of spike time metric is based on spike timings and measure a
the systems, a neuron, interacts with other neurons, then distance or a similarity between two spike sequences. The
they often produce very complicated behavior. To model, partial spike time metric coe cient is based on partializa-
analyze, and predict such complicated behavior, it is impor- tion analysis[3] applied to the spike time metric coe cient.
tant to understand interactions between neurons, namely, The spike time metric coe cient evaluates a static correla-
a neural network structure. In the present paper, to esti- tion between two spike sequences, while the partial spike
mate such a neural network structure by using only ob- time metric coe cient can reveal unbiased correlation be-
served multi-spike sequences, we propose two new mea- tween these spike sequences by removing any spurious cor-
sures, which are based on spike time metric and partial- relations. Using two measures, we can nd hidden relations
ization analysis. To evaluate the validity of our proposed between neurons and reveal the neural network structure.
measures, we apply the proposed measures to multi-spike
Although our nal target is to analyze real neural sys-
sequences which are produced by an electrotonic coupling
tems, before applying the proposed measures to real multi-
of -models. As a result, the proposed measures can iden-
spike sequences, rst we examined the validity of the pro-
tify regular and random neural network structures in high
posed measures with a mathematical model. Our funda-
performance.
mental opinion is that even if we have a good measure, the
measure would be a castle in the air without evaluating its
potential ability under a situation that simulates real exper-
1. Introduction
imental data. Then, in this paper, we assumed that we can
only observe multi-spike sequences but the true network
Neural systems often show very complex behavior due
structure is unknown.
to interaction among many neurons depending on a neural
In the present paper, we used an electrotonic coupling
network topology. Such a neural network topology, or a
of -models[4] to produce internal states of neurons. For
neural network structure, is described by synapses or gap
the -models, we used a ring topology and a random topol-
junctions between neurons. To model, analyze, and predict
ogy. In addition, we produced multi-spike sequences from
the behavior of neural systems, it is important to consider
the internal states. Then, we estimated the neural network
not only the dynamics of each neuron, but also the neural
structure of the electrotonic coupling of -models by using
network structure. However, it is not so easy to investigate
the proposed measures. As a result, if we use both mea-
the neural network structure, which produces multi-spike
sures simultaneously, we can estimate the neural network
sequences, by dissecting a brain cyclopaedically.
with high e ciency.
Meanwhile, it is possible to observe these multi-spike
sequences simultaneously due to recent improvement of
measurement technologies. It is very natural to expect that
2. Spike time metric
these observed spike sequences re ect essential informa-
tion about the neural network structure. Thus, it is an im-
To estimate the neural network structures, we introduced
portant issue to extract such a neural network structure to
a spike time metric[2] which are based on spike timings.
study a neural system, not only from an anatomical point
The spike time metric quanti es a distance which means a
of view, but also a functional point of view. If we can esti-
similarity between two spike sequences. In the spike time
mate a functional connectivity between neurons, we might
metric, the rst rule is that a cost of deleting or inserting
understand how the information is coded and processed in
a spike becomes 1. The second rule is that a cost of mov-
our brain[1].
ing a single spike in time is proportional to the amount of
In the present paper, to estimate the neural network struc- time by which the single spike is moved. For example, if
0
ture only by these observed spike sequences, we proposed two spike sequences Z and Z are identical except for a sin-
0 0
0
two new measures: a spike time metric coe cient and a gle spike that res at t in Z and t in Z , the cost c(Z; Z )
z z
partial spike time metric coe cient. The spike time met- equals to qjt t 0j in the second rule, where q is a cost per
z z
ric coe cient uses a concept of spike time metric[2]. The unit time. The parameter q is important parameter that de-
- 417 -termines deleting and inserting, or moving a single spike. 4.1. Neural network model
In these rules, a metric distance between two spike se-
To evaluate the validity of our proposed measures, we
0
quences Z and Z is de ned as
used an electrotonic coupling of -models[4] to produce
8 9
internal states of neurons:
N 1
>X >
> >
< =
spike 0
8
D [q](Z; Z )= min c(V; V ) ;
> k k+1>
> > > dx 3
i
: ; > 2
>
> = y x (x )+ I+ J
k=0 i i i
> i
>
> dt 2
>
>
> dy
i
> 2
<
= y+ x
wherefV; V;:::; V g is an elementary step from Z to i
0 1 N i
>
dt
>
0 >
n
>
X
Z [2]. Thus, the distance between the two spike sequences
>
>
>
>
> J = g (x x );
i j i
is the minimum total cost of a set of elementary steps that >
>
:
j
transforms one spike sequence into another sequence.
where x and y are the internal states of the i-th neuron,
i i
is a parameter, I is an injected background current, J is a
i
3. Two proposed measures
total current induced by electrotonic couplings to the i-th
neuron, g is a coupling strength, and n is the number of
In the present paper, we proposed two new measures for
couplings to the i-th neuron. In the present paper, we set
estimating the neural network structures. The rst measure
= 1:65; I = 0:005; g= 0:05, and dt= 0:02[ms]. We
is based on the spike time metric. We call it a spike time
used two types of network structure as a neural network:
metric coe cient (STMC). The STMC between two spike
the rst one has a regular ring topology with 30 neurons,
sequences X and X is de ned as
i j
and the second has a random topology produced from the
spike
30 ring topology (Fig.1). In Fig.2, we show the typical
D [q](X; X )
i j
S [q](X; X )= 1 :
T i j
outputs of internal states time series of -models, and its
spike
maxfD [q](X; X )g
i j
i; j
spike sequences in the case of the regular topology.
Then, we de ne S [q] as a matrix consisting of
T
S [q](X; X ). If two neurons, which produce the two spike
T i j
sequences X and X , are coupled, S [q](X; X ) might be-
i j T i j
come larger than the case of the two neurons are uncou-
spike
pled. The reason is that D [q](X; X ) becomes smaller
i j
than the uncoupled case, if the two neurons interact with
each other through the coupling.
However, the STMC is spuriously biased if the two neu-
rons are driven by a common input from other neurons.
(a) (b)
Then, we proposed the second measure which is based
on partialization analysis[3] applied to the STMC. We call
Figure 1: The network structures which are used to pro-
it a partial spike time metric coe cient (PSTMC). The
duce the internal states of neurons. (a) Regular ring topol-
PSTMC between X and X is de ned as
i j
ogy which consists of 30 neurons, and (b) random topol-
ogy produced from the 30 ring topology. To obtain the ran-


(i; j)
dom topology, we rewired all the connections with rewiring

P [q](X; X )= ;
T i j

(i; i) ( j; j)
probability 1.
where (i; j) is the (i; j)-th element of inverse matrix of
S [q]. The PSTMC estimates a partial correlation between
T
the two spike sequences, X and X , or a correlation by re- 4.2. Estimation method
i j
moving spurious correlations.
We observed the internal states of all neurons in the elec-
trotonic coupling of -models. Then, we produced multi-
spike sequences from the internal states with thresholding,
4. Experiments
and calculated the STMC and the PSTMC between two
We cannot evaluate the validity of the proposed measures spike sequences, X and X (i; j= 1; 2;:::; 30).
i j
only by using real spike sequences, because we do not have For our measures, it is important to decide q appropri-
any information of the true neural network structure pro- ately, because it determines a relative sensitivity of delet-
ducing the spike sequences. Then, we used a neural net- ing and inserting, and moving a single spike. Then, we rst
work model to check the validity of the proposed measures checked S [q](X; X ) and P [q](X; X ) in the case of the
T i j T i j
because we can have the information of the true neural net- regular topology by changing q as shown in Fig.3. From
work structure. these results, we set q= 500 because the disparity between
- 418 - 0.8

0.7
U-C are a ratio of the estimated number of coupled neuron
0.6
0.5
pairs to the truly coupled number of neuron pairs, and to the
0.4
0.3

0.2 number of truly uncoupled neuron pairs, respectively. C-U,
0.1

0
and U-U are a ratio of the estimated number of uncoupled
-0.1
5000 5020 5040 5060 5080 5100 5120 5140 5160 5180 5200
time[msec]
neuron pairs to the truly coupled number of neuron-pairs,
(a)
and to the number of truly uncoupled neuron-pairs, respec-
30

tively. The more C-C and U-U approach one, or C-U and
25
20

C-U approach zero, the more the estimation accuracy be-
15
10 comes high.
5
0
5000 5020 5040 5060 5080 5100 5120 5140 5160 5180 5200
time[msec]
(b)
5. Results
Figure 2: (a) Time series of internal states, and (b) a raster
plot of output spikes from the electrotonic couplings of
For the case of the 30 ring topology, we show an example
-models in the case of the regular topology. To obtain
of frequency histograms of S [q] and P [q] (Fig.4). Blue
T T
the output sequences, we used thresholding at the value of
lines show a threshold which divides coupled or uncoupled
0.65[mV].
classes. If S [q] and P [q] are less than the threshold, we
T T
decided that corresponding neurons are uncoupled. On the
other hand, if S [q] and P [q] are more than the threshold,
T T
we decided that they coupled. Table 1 shows the estimation
coupled and uncoupled neurons is relatively larger than the
accuracy of the neural network structure.
other q cases. The random topology has the same tendency
As shown in Fig.4(a), if we use the STMC, the frequency
as the regular topology.
distribution shows that coupled or uncoupled is classi ed,
1 1
even if the discriminant analysis does not work well. How-
0.8 0.8
ever, if we use the PSTMC, the frequency distribution is not
0.6 0.6
classi ed (Fig.4(b)). The reason is that even if the neuron
0.4 0.4
A is coupled to the neuron C as shown in Fig.5, a coupling
0.2 0.2
between the neurons A and C is estimated as uncoupled,
0 0
and the P [q] between these neurons becomes low.
T
0.1 1 10 100 1000 10000 0.1 1 10 100 1000 10000
q q
Figure 6 and Table 2 show the results in the case of the
(a) (b)
random topology. In Fig.6(a), if we use the STMC, the fre-
quency distribution does not show clear classi cation, then
Figure 3: Relation between q and (a) S [q], and (b) P [q]
T T
the discriminant analysis does not work well. However, as
in the case of the regular topology. Green lines show S [q]
T
shown in Fig.6(b), if we use the PSTMC, the frequency dis-
and P [q] between coupled neurons. Red lines show S [q]
T T
tribution shows clearer classi cation of the coupled and the
and P [q] between uncoupled neurons. Error bars with 30
T
uncoupled pairs, and the discriminant analysis works well.
trials are the range of S [q] and P [q], respectively.
T T
The results in the case of the 30 ring topology indicate
that we have to use the STMC and the PSTMC simultane-
If the corresponding two neurons are coupled, the two ously to discriminate the coupled and the uncoupled pairs
spike sequences must interact with each other. In such a as shown in Fig.7. If we use the STMC and the PSTMC si-
case, S [q](X; X ) and P [q](X; X ) might become large. multaneously, we can discriminate the coupled and the un-
T i j T i j
On the other hand, if these neurons are not coupled, coupled pairs with much higher accuracy. In addition, we
S [q](X; X ) and P [q](X; X ) might become small. Thus, have already con rmed that the proposed measures work
T i j T i j
to nd coupled neurons pairs, we have extracted large well for another model, such as Izhikevich’s simple neuron
S [q](X; X ) and P [q](X; X ). Then, we calculated a model[6, 7]. From these results, we found that the proposed
T i j T i j
threshold which divides the coupled or the uncoupled pairs. measures exhibit high performance.
The threshold was decided by the Otsu thresholding[5]
which is based on a linear discriminant analysis.
Table 1: Estimation accuracy for the 30 ring topology.

4.3. Evaluation measure C-C U-U C-U U-C
S [q] 1.000 0.920 0.000 0.080
T
In order to evaluate an estimation accuracy, we have to
P [q] 0.500 1.000 0.500 0.000
T
compare an estimated neural network structure with the
true neural network structure. For this end, we used four

evaluation measures, C-C, C-U, U-U and U-C. C-C, and
- 419 -
S [q]
T
neuron index x
P [q]
T 100 100 1 1
80 80 0.8 0.8
60 60 0.6 0.6
40 40 0.4 0.4
20 20 0.2 0.2
0 0 0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
S [q] P [q] S [q] S [q]
T T T T
(a) (b) (a) (b)
Figure 4: Frequency histograms of (a) S [q] and (b) P [q]. Figure 7: An example of discriminating coupled and un-
T T
Histograms of all of S [q] and P [q] are shown in red, coupled pairs by using both measures. (a) The 30 ring
T T
and histograms of S [q] and P [q] corresponding to the topology, and (b) the random topology. Green dots show
T T
coupled neurons are shown in green. Blue lines show a S [q] and P [q] between coupled neurons. Red dots show
T T
threshold decided by the Otsu thresholding. The network S [q] and P [q] between uncoupled neurons. Blue lines
T T
structure is the 30 ring topology. divide coupled and uncoupled pairs.
sures simultaneously, we could estimate the neural network
structures more completely.
As future works, we apply our measures to di erent neu-
A D
ron models and di erent neural structures such as complex
networks. In addition, we have to optimize how to decide q
B C
only from the spike sequences. Moreover, we improve the
discriminant analysis to decide the threshold which divides
into coupled or uncoupled. Our goal is that to evaluate the
Figure 5: A partial diagram of the ring topology with 30
validity of our framework to simultaneously observed real
neurons.
multi-spike sequences, and to analysis brain systems.
100 100
80 80 7. Acknowledgement
60 60
The authors would like to thank Prof. K. Aihara, Dr. Y.
40 40
Hirata, Dr. T. Suzuki and Dr. Y. Katori for their valuable
20 20
comments. The research of TI was partially supported by
0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
Grant-in-Aids for Scienti c Research (C) (No.17500136)
S [q] P [q]
T T
from JSPS.
(a) (b)
Figure 6: The same as Fig.4, but using the random topol-
References
ogy.
[1] H. Fujii et al., Neural Networks, 9(8), 1303, 1996.
[2] J. D. Victor and K. P. Purpura, Network: Comput.
Table 2: The same as Table 1, but using the random topol-
Neural Syst., 8, 127, 1997.
ogy.

[3] B. Schelter et al., PRL, 96(208103), 1, 2006.
measure C-C U-U C-U U-C
S [q] 0.783 0.774 0.217 0.226
T
[4] I. Tsuda et al., J. of Integrative Neuroscience, 3(2),
P [q] 0.833 0.985 0.167 0.015
T
159, 2004.
[5] N. Otsu, IEEE Trans. on Syst., Man and Cybernetics,
SMC-9(1), 62, 1979.
6. Conclusions
[6] T. Ashizawa et al., Technical Reports of IEICE,
107(21), 7, 2007.
In the present paper, we proposed two new measures
in order to solve an important issue of estimating neural
[7] E. M. Izhikevich, IEEE Trans. on Neural Networks,
network structures only from the information of observed
14(6), 1569, 2003.
multi-spike sequences. We applied our proposed measures
to two types of network structure, the 30 ring topology
and the random topology. As a result, we used both mea-
- 420 -
frequency frequency
frequency frequency
P [q]
T
P [q]
T