Spiking Neural Networks: The New Generation of Artificial Neural ...

chickenchairwomanΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 9 μήνες)

86 εμφανίσεις


Clustering using

Spiking Neural Networks


Biological Neuron:

The Elementary Processing Unit of the Brain

Biological Neuron:

A Generic Structure

Dendrite

Soma

Synapse

Axon

Axon Terminal

Biological Neuron:

Nerve Impulse Transiting

Action Potential

(Spike)

Postsynaptic
Potential

Membrane
Potential

Action Potential

(Spike)

Spike
-
After
Potential

Biological Neuron:

Soma Firing Behavior

Synchrony

is the main
factor of soma firing

Biological Neuron:

Information Coding

Neurons communicate via
exact
spike timing

Firing rate
alone
does not carry

all
the relevant
information

Neuroscience Models of Neuron:

The Hodgkin
-
Huxley Model

Alan Lloyd Hodgkin and Andrew
Huxley received
the Nobel Prize
in
Physiology and Medicine in 1963

The Hodgkin
-
Huxley model is
too
complicated

model of neuron to be
used in artificial neural networks

Neuroscience Models of Neuron:

Leaky Integrate
-
And
-
Fire Model

or

Leaky Integrate
-
And
-
Fire model
disregards
the refractory capability

of neuron

Neuroscience Models of Neuron:

Spike
-
Response Model

Spike
-
Response model captures
the major
elements

of a biological neuron behavior

Biological Neuron


Computational
Intelligence Approach:

The First Generation

The first artificial neuron
was
proposed by W.
McCulloch &
W. Pitts in 1943

Biological Neuron


Computational
Intelligence Approach:

The Second Generation

Multilayered Perception is
a
universal approximator

Biological Neuron


Computational
Intelligence Approach:

Artificial Neurons


Too Artificial?

spike occurrence

spike absence

From neurophysiology point of view,
y

is
existence of an output spike

Number of spikes

Time frame

From neurophysiology point of view,
y

is
firing rate

Spike timing is
not considered
at all!

Biological Neuron


Computational
Intelligence Approach:

The Third Generation

Spiking neuron model was introduced
by
J. Hopfield
in 1995

Spiking neural networks are


-

biologically
more plausible
,


-

computationally
more powerful
,


-

considerably
faster


than networks of the second
generation

Spiking Neural Network:

Overall Architecture

RN
is a receptive neuron

MS
is a multiple synapse

SN
is a spiking neuron

Spiking neural network is a
heterogeneous two
-
layered feed
-
forward network with lateral
connections in the second hidden layer

Spiking Neural Network:

Population Coding

Pool of
Receptive
Neurons

Input spike:

Spiking Neural Network:

Multiple Synapse

Delayed postsynaptic potential:

Spike
-
response function:

Total postsynaptic potential
:

Membrane potential:

Spiking Neural Network:

Hebbian Learning


WTA and WTM

Winner
-
Takes
-
All:

Winner
-
Takes
-
More*:

*Proposed for the first time on
the 11
th

International
Conference on Science and Technology “System Analysis
and Information Technologies” (Kyiv, Ukraine, 2009)
by
Ye. Bodyanskiy and A. Dolotov

Spiking Neural Network:

Image Processing*

Original Image

SOM at 50 epoch

SNN at 4 epoch

*
In
Bionics of Intelligence: 2007, 2 (67), pp. 21
-
26
by Ye. Bodyanskiy and A. Dolotov

Spiking Neuron:

The Laplace Transform Basis

Thus,

transformation

of

action

potential

to

postsynaptic

potential

taken

into

synapse

is

nothing

other

than

pulse
-
position



continuous
-
time

transformation
,

and

soma

transformation

is

just

reverse

one,

continuous
-
time



pulse
-
position

transformation

From

control

theory

point

of

view,

action

potential

(spike)

is

a

signal

in

pulse
-
position

form
:

Spiking Neuron Synapse:

A 2
nd

order critically damped response unit *

*Proposed for the first time on
the 6
th

International
Conference “Information Research and Applications”

(Varna, Bulgaria, 2009)

by Ye. Bodyanskiy, A. Dolotov,
and I. Pliss

Spiking Neuron:

Technically Plausible Description*

Incoming Spike:

Time Delay:

Spike
-
Response Function:

Membrane Potential:

Relay:

Outgoing Spike:

*Proposed for the first time on
the 6
th

International
Conference “Information Research and Applications”

(Varna, Bulgaria, 2009)

by Ye. Bodyanskiy, A. Dolotov, and I. Pliss

Spiking Neuron:

Analog
-
Digital Architecture*

*

Proposed for the first time in
Image
Processing / Ed. Yung
-
Sheng Chen: In
-
Teh, Vukovar, Croatia, pp. 357
-
380

by Ye.
Bodyanskiy and A. Dolotov,

Analog
-
digital spiking neurons corresponds
to spike
-
response model entirely

Fuzzy Receptive Neurons*:


*Proposed for the first time in
In
formation Technologies and Computer Engineering: 2009, 2(15), pp. 51
-
55
by Ye. Bodyanskiy and A. Dolotov

Pool of receptive neurons is a
linguistic
variable
, and a receptive neuron within a
pool is a
linguistic term
.

Fuzzy Spiking Neural Network:

Fuzzy Probabilistic Clustering*


*Proposed for the first time in
Sci. Proc. of Riga Technical University: 2008, 36, P. 27
-
33
by Ye. Bodyanskiy
and A. Dolotov

There is no need to calculate cluster centers!

Fuzzy Spiking Neural Network:

Fuzzy Possibilistic Clustering*


*Proposed for the first time on
the 15
th

Zittau East
-
West Fuzzy Colloquium (Zittau, Germany, 2008)

by
Ye. Bodyanskiy, A. Dolotov, I. Pliss, and Ye. Viktorov

Fuzzy Spiking Neural Network:

Image Processing*

*
In
Proceeding of the 4
th

International School
-
Seminar “Theory of Decision Making“
(
Uzhhorod, Ukraine, 2008
)
by Ye. Bodyanskiy, A. Dolotov, and I. Pliss

Original
image

Training set

FSNN at 4
th

epoch

SOM at 40
th

epoch

Fuzzy Spiking Neural Network:

Image Processing*

*
In
Proceeding of the 11
th

International Biennial Baltic Electronics Conference "BEC 2008“
(
Tallinn/Laulasmaa, Estonia, 2008
)
by Ye. Bodyanskiy and A. Dolotov

Original
image

Training set

FSNN at 3
rd

epoch

FCM at 29
th

epoch

Fuzzy Spiking Neural Network:

Image Processing*

*
In
Image Processing / Ed. Yung
-
Sheng Chen: In
-
Teh, Vukovar, Croatia, pp. 357
-
380

by Ye. Bodyanskiy
and A. Dolotov


Original
image

Training set

FSNN at 1
st

epoch

FSNN at 3
rd

epoch

FCM at 3
rd

epoch

FCM at 30
th

epoch