2007 International Symposium on Nonlinear Theory and its

Applications

NOLTA'07, Vancouver, Canada, September 16-19, 2007

New measures for estimating neural network structures

only from multi-spike sequences

yz y y

Tohru Ashizawa , Daisuke Haraki , and Tohru Ikeguchi

yGraduate School of Science and Engineering, Saitama University,

255 Shimo-Ohkubo, Sakura-ku, Saitama 338-8570, Japan

zE-mail: ashizawa@nls.ics.saitama-u.ac.jp

Abstract In neural systems, a fundamental element of spike time metric is based on spike timings and measure a

the systems, a neuron, interacts with other neurons, then distance or a similarity between two spike sequences. The

they often produce very complicated behavior. To model, partial spike time metric coe cient is based on partializa-

analyze, and predict such complicated behavior, it is impor- tion analysis[3] applied to the spike time metric coe cient.

tant to understand interactions between neurons, namely, The spike time metric coe cient evaluates a static correla-

a neural network structure. In the present paper, to esti- tion between two spike sequences, while the partial spike

mate such a neural network structure by using only ob- time metric coe cient can reveal unbiased correlation be-

served multi-spike sequences, we propose two new mea- tween these spike sequences by removing any spurious cor-

sures, which are based on spike time metric and partial- relations. Using two measures, we can nd hidden relations

ization analysis. To evaluate the validity of our proposed between neurons and reveal the neural network structure.

measures, we apply the proposed measures to multi-spike

Although our nal target is to analyze real neural sys-

sequences which are produced by an electrotonic coupling

tems, before applying the proposed measures to real multi-

of -models. As a result, the proposed measures can iden-

spike sequences, rst we examined the validity of the pro-

tify regular and random neural network structures in high

posed measures with a mathematical model. Our funda-

performance.

mental opinion is that even if we have a good measure, the

measure would be a castle in the air without evaluating its

potential ability under a situation that simulates real exper-

1. Introduction

imental data. Then, in this paper, we assumed that we can

only observe multi-spike sequences but the true network

Neural systems often show very complex behavior due

structure is unknown.

to interaction among many neurons depending on a neural

In the present paper, we used an electrotonic coupling

network topology. Such a neural network topology, or a

of -models[4] to produce internal states of neurons. For

neural network structure, is described by synapses or gap

the -models, we used a ring topology and a random topol-

junctions between neurons. To model, analyze, and predict

ogy. In addition, we produced multi-spike sequences from

the behavior of neural systems, it is important to consider

the internal states. Then, we estimated the neural network

not only the dynamics of each neuron, but also the neural

structure of the electrotonic coupling of -models by using

network structure. However, it is not so easy to investigate

the proposed measures. As a result, if we use both mea-

the neural network structure, which produces multi-spike

sures simultaneously, we can estimate the neural network

sequences, by dissecting a brain cyclopaedically.

with high e ciency.

Meanwhile, it is possible to observe these multi-spike

sequences simultaneously due to recent improvement of

measurement technologies. It is very natural to expect that

2. Spike time metric

these observed spike sequences re ect essential informa-

tion about the neural network structure. Thus, it is an im-

To estimate the neural network structures, we introduced

portant issue to extract such a neural network structure to

a spike time metric[2] which are based on spike timings.

study a neural system, not only from an anatomical point

The spike time metric quanti es a distance which means a

of view, but also a functional point of view. If we can esti-

similarity between two spike sequences. In the spike time

mate a functional connectivity between neurons, we might

metric, the rst rule is that a cost of deleting or inserting

understand how the information is coded and processed in

a spike becomes 1. The second rule is that a cost of mov-

our brain[1].

ing a single spike in time is proportional to the amount of

In the present paper, to estimate the neural network struc- time by which the single spike is moved. For example, if

0

ture only by these observed spike sequences, we proposed two spike sequences Z and Z are identical except for a sin-

0 0

0

two new measures: a spike time metric coe cient and a gle spike that res at t in Z and t in Z , the cost c(Z; Z )

z z

partial spike time metric coe cient. The spike time met- equals to qjt t 0j in the second rule, where q is a cost per

z z

ric coe cient uses a concept of spike time metric[2]. The unit time. The parameter q is important parameter that de-

- 417 -termines deleting and inserting, or moving a single spike. 4.1. Neural network model

In these rules, a metric distance between two spike se-

To evaluate the validity of our proposed measures, we

0

quences Z and Z is de ned as

used an electrotonic coupling of -models[4] to produce

8 9

internal states of neurons:

N 1

>X >

> >

< =

spike 0

8

D [q](Z; Z )= min c(V; V ) ;

> k k+1>

> > > dx 3

i

: ; > 2

>

> = y x (x )+ I+ J

k=0 i i i

> i

>

> dt 2

>

>

> dy

i

> 2

<

= y+ x

wherefV; V;:::; V g is an elementary step from Z to i

0 1 N i

>

dt

>

0 >

n

>

X

Z [2]. Thus, the distance between the two spike sequences

>

>

>

>

> J = g (x x );

i j i

is the minimum total cost of a set of elementary steps that >

>

:

j

transforms one spike sequence into another sequence.

where x and y are the internal states of the i-th neuron,

i i

is a parameter, I is an injected background current, J is a

i

3. Two proposed measures

total current induced by electrotonic couplings to the i-th

neuron, g is a coupling strength, and n is the number of

In the present paper, we proposed two new measures for

couplings to the i-th neuron. In the present paper, we set

estimating the neural network structures. The rst measure

= 1:65; I = 0:005; g= 0:05, and dt= 0:02[ms]. We

is based on the spike time metric. We call it a spike time

used two types of network structure as a neural network:

metric coe cient (STMC). The STMC between two spike

the rst one has a regular ring topology with 30 neurons,

sequences X and X is de ned as

i j

and the second has a random topology produced from the

spike

30 ring topology (Fig.1). In Fig.2, we show the typical

D [q](X; X )

i j

S [q](X; X )= 1 :

T i j

outputs of internal states time series of -models, and its

spike

maxfD [q](X; X )g

i j

i; j

spike sequences in the case of the regular topology.

Then, we de ne S [q] as a matrix consisting of

T

S [q](X; X ). If two neurons, which produce the two spike

T i j

sequences X and X , are coupled, S [q](X; X ) might be-

i j T i j

come larger than the case of the two neurons are uncou-

spike

pled. The reason is that D [q](X; X ) becomes smaller

i j

than the uncoupled case, if the two neurons interact with

each other through the coupling.

However, the STMC is spuriously biased if the two neu-

rons are driven by a common input from other neurons.

(a) (b)

Then, we proposed the second measure which is based

on partialization analysis[3] applied to the STMC. We call

Figure 1: The network structures which are used to pro-

it a partial spike time metric coe cient (PSTMC). The

duce the internal states of neurons. (a) Regular ring topol-

PSTMC between X and X is de ned as

i j

ogy which consists of 30 neurons, and (b) random topol-

ogy produced from the 30 ring topology. To obtain the ran-

(i; j)

dom topology, we rewired all the connections with rewiring

P [q](X; X )= ;

T i j

(i; i) ( j; j)

probability 1.

where (i; j) is the (i; j)-th element of inverse matrix of

S [q]. The PSTMC estimates a partial correlation between

T

the two spike sequences, X and X , or a correlation by re- 4.2. Estimation method

i j

moving spurious correlations.

We observed the internal states of all neurons in the elec-

trotonic coupling of -models. Then, we produced multi-

spike sequences from the internal states with thresholding,

4. Experiments

and calculated the STMC and the PSTMC between two

We cannot evaluate the validity of the proposed measures spike sequences, X and X (i; j= 1; 2;:::; 30).

i j

only by using real spike sequences, because we do not have For our measures, it is important to decide q appropri-

any information of the true neural network structure pro- ately, because it determines a relative sensitivity of delet-

ducing the spike sequences. Then, we used a neural net- ing and inserting, and moving a single spike. Then, we rst

work model to check the validity of the proposed measures checked S [q](X; X ) and P [q](X; X ) in the case of the

T i j T i j

because we can have the information of the true neural net- regular topology by changing q as shown in Fig.3. From

work structure. these results, we set q= 500 because the disparity between

- 418 - 0.8

0.7

U-C are a ratio of the estimated number of coupled neuron

0.6

0.5

pairs to the truly coupled number of neuron pairs, and to the

0.4

0.3

0.2 number of truly uncoupled neuron pairs, respectively. C-U,

0.1

0

and U-U are a ratio of the estimated number of uncoupled

-0.1

5000 5020 5040 5060 5080 5100 5120 5140 5160 5180 5200

time[msec]

neuron pairs to the truly coupled number of neuron-pairs,

(a)

and to the number of truly uncoupled neuron-pairs, respec-

30

tively. The more C-C and U-U approach one, or C-U and

25

20

C-U approach zero, the more the estimation accuracy be-

15

10 comes high.

5

0

5000 5020 5040 5060 5080 5100 5120 5140 5160 5180 5200

time[msec]

(b)

5. Results

Figure 2: (a) Time series of internal states, and (b) a raster

plot of output spikes from the electrotonic couplings of

For the case of the 30 ring topology, we show an example

-models in the case of the regular topology. To obtain

of frequency histograms of S [q] and P [q] (Fig.4). Blue

T T

the output sequences, we used thresholding at the value of

lines show a threshold which divides coupled or uncoupled

0.65[mV].

classes. If S [q] and P [q] are less than the threshold, we

T T

decided that corresponding neurons are uncoupled. On the

other hand, if S [q] and P [q] are more than the threshold,

T T

we decided that they coupled. Table 1 shows the estimation

coupled and uncoupled neurons is relatively larger than the

accuracy of the neural network structure.

other q cases. The random topology has the same tendency

As shown in Fig.4(a), if we use the STMC, the frequency

as the regular topology.

distribution shows that coupled or uncoupled is classi ed,

1 1

even if the discriminant analysis does not work well. How-

0.8 0.8

ever, if we use the PSTMC, the frequency distribution is not

0.6 0.6

classi ed (Fig.4(b)). The reason is that even if the neuron

0.4 0.4

A is coupled to the neuron C as shown in Fig.5, a coupling

0.2 0.2

between the neurons A and C is estimated as uncoupled,

0 0

and the P [q] between these neurons becomes low.

T

0.1 1 10 100 1000 10000 0.1 1 10 100 1000 10000

q q

Figure 6 and Table 2 show the results in the case of the

(a) (b)

random topology. In Fig.6(a), if we use the STMC, the fre-

quency distribution does not show clear classi cation, then

Figure 3: Relation between q and (a) S [q], and (b) P [q]

T T

the discriminant analysis does not work well. However, as

in the case of the regular topology. Green lines show S [q]

T

shown in Fig.6(b), if we use the PSTMC, the frequency dis-

and P [q] between coupled neurons. Red lines show S [q]

T T

tribution shows clearer classi cation of the coupled and the

and P [q] between uncoupled neurons. Error bars with 30

T

uncoupled pairs, and the discriminant analysis works well.

trials are the range of S [q] and P [q], respectively.

T T

The results in the case of the 30 ring topology indicate

that we have to use the STMC and the PSTMC simultane-

If the corresponding two neurons are coupled, the two ously to discriminate the coupled and the uncoupled pairs

spike sequences must interact with each other. In such a as shown in Fig.7. If we use the STMC and the PSTMC si-

case, S [q](X; X ) and P [q](X; X ) might become large. multaneously, we can discriminate the coupled and the un-

T i j T i j

On the other hand, if these neurons are not coupled, coupled pairs with much higher accuracy. In addition, we

S [q](X; X ) and P [q](X; X ) might become small. Thus, have already con rmed that the proposed measures work

T i j T i j

to nd coupled neurons pairs, we have extracted large well for another model, such as Izhikevich’s simple neuron

S [q](X; X ) and P [q](X; X ). Then, we calculated a model[6, 7]. From these results, we found that the proposed

T i j T i j

threshold which divides the coupled or the uncoupled pairs. measures exhibit high performance.

The threshold was decided by the Otsu thresholding[5]

which is based on a linear discriminant analysis.

Table 1: Estimation accuracy for the 30 ring topology.

4.3. Evaluation measure C-C U-U C-U U-C

S [q] 1.000 0.920 0.000 0.080

T

In order to evaluate an estimation accuracy, we have to

P [q] 0.500 1.000 0.500 0.000

T

compare an estimated neural network structure with the

true neural network structure. For this end, we used four

evaluation measures, C-C, C-U, U-U and U-C. C-C, and

- 419 -

S [q]

T

neuron index x

P [q]

T 100 100 1 1

80 80 0.8 0.8

60 60 0.6 0.6

40 40 0.4 0.4

20 20 0.2 0.2

0 0 0 0

0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

S [q] P [q] S [q] S [q]

T T T T

(a) (b) (a) (b)

Figure 4: Frequency histograms of (a) S [q] and (b) P [q]. Figure 7: An example of discriminating coupled and un-

T T

Histograms of all of S [q] and P [q] are shown in red, coupled pairs by using both measures. (a) The 30 ring

T T

and histograms of S [q] and P [q] corresponding to the topology, and (b) the random topology. Green dots show

T T

coupled neurons are shown in green. Blue lines show a S [q] and P [q] between coupled neurons. Red dots show

T T

threshold decided by the Otsu thresholding. The network S [q] and P [q] between uncoupled neurons. Blue lines

T T

structure is the 30 ring topology. divide coupled and uncoupled pairs.

sures simultaneously, we could estimate the neural network

structures more completely.

As future works, we apply our measures to di erent neu-

A D

ron models and di erent neural structures such as complex

networks. In addition, we have to optimize how to decide q

B C

only from the spike sequences. Moreover, we improve the

discriminant analysis to decide the threshold which divides

into coupled or uncoupled. Our goal is that to evaluate the

Figure 5: A partial diagram of the ring topology with 30

validity of our framework to simultaneously observed real

neurons.

multi-spike sequences, and to analysis brain systems.

100 100

80 80 7. Acknowledgement

60 60

The authors would like to thank Prof. K. Aihara, Dr. Y.

40 40

Hirata, Dr. T. Suzuki and Dr. Y. Katori for their valuable

20 20

comments. The research of TI was partially supported by

0 0

0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Grant-in-Aids for Scienti c Research (C) (No.17500136)

S [q] P [q]

T T

from JSPS.

(a) (b)

Figure 6: The same as Fig.4, but using the random topol-

References

ogy.

[1] H. Fujii et al., Neural Networks, 9(8), 1303, 1996.

[2] J. D. Victor and K. P. Purpura, Network: Comput.

Table 2: The same as Table 1, but using the random topol-

Neural Syst., 8, 127, 1997.

ogy.

[3] B. Schelter et al., PRL, 96(208103), 1, 2006.

measure C-C U-U C-U U-C

S [q] 0.783 0.774 0.217 0.226

T

[4] I. Tsuda et al., J. of Integrative Neuroscience, 3(2),

P [q] 0.833 0.985 0.167 0.015

T

159, 2004.

[5] N. Otsu, IEEE Trans. on Syst., Man and Cybernetics,

SMC-9(1), 62, 1979.

6. Conclusions

[6] T. Ashizawa et al., Technical Reports of IEICE,

107(21), 7, 2007.

In the present paper, we proposed two new measures

in order to solve an important issue of estimating neural

[7] E. M. Izhikevich, IEEE Trans. on Neural Networks,

network structures only from the information of observed

14(6), 1569, 2003.

multi-spike sequences. We applied our proposed measures

to two types of network structure, the 30 ring topology

and the random topology. As a result, we used both mea-

- 420 -

frequency frequency

frequency frequency

P [q]

T

P [q]

T

## Comments 0

Log in to post a comment