# Cellular Neural Networks

AI and Robotics

Dec 1, 2013 (4 years and 7 months ago)

104 views

Polynomial Discrete Time
Cellular Neural Networks

Eduardo Gomez
-
Ramirez †

Giovanni Egidio Pazienza‡

México, D.F.

Department d’Electronica, EALS

Universitat “Ramon Llull”

Barcelona, Spain

Outline

Cellular Neural Networks (CNN)

Introduction and Objective

Genetic Algorithms (GA)

Polynomial Discrete Time CNNs (PDTCNNs)

XOR Problem

Game of Life

Learning vs Design

Conclusions and future work

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

CNN: Introduction

CNN for complex task (linearly nonseparable
data)

Multilayer CNNs

Include more degrees of freedom for the
output state of each layer

Search in a finite set of templates

Single layer: Polynomial CNNs

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

Improve the representation power of a single
layer CNN including a simple nonlinear term
to solve problems with linearly nonseparable
data (XOR)

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

Objective

CNN: mathematical model

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

The simplified mathematical model is:

where x
c

is the state of the cell, u
c

the input
and y
c

the output

i
u
b
t
y
a
t
x
dt
dx
d
c
d
d
c
d
c
c

)
(
)
(
)
1
)
(
1
)
(
(
2
1

t
x
t
x
y
c
c
c
CNN: Activation Function

-1.5
-1
-0.5
0
0.5
1
1.5
-3
-2
-1
0
1
2
3
x
y
Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

CNN: Block Diagram

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

CNN: Discrete Model

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

Computing x(∞), the model can be
represented as

i
u
b
n
y
a
n
x
d
c
d
d
c
d
c

)
(
)
(

0
)
(
,
1
0
)
(
,
1
)
(
n
x
n
x
n
y
c
c
c
using the following activation function

Steps:

Crossover
C(Fg)

Mutation
M(*)

Ag
(

)

GA: main steps proposed

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

d
c
b
c
d
a
b
a
F
C
d
c
M
b
a
M
M
M
F
g
g
2
1
2
1
,
,
GA: Crossover

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

125
.
0
125
.
0
0
0
0
0
0
A

125
.
125
.
0
0
0
0
0
0
0
B

I=0

125
.
0
0
0
0
0
0
0
0
A

125
.
125
.
125
.
0
0
0
0
0
0
B

I=0

2
1
2
1
2
2
1
1
2
2
2
2
2
1
2
1
2
2
1
1
1
1
1
1
1
c
c
c
c
b
b
b
b
a
a
a
a
c
c
c
c
b
b
b
b
a
a
a
a
P

Individual 1

Individual 2

GA: Crossover

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

a
1

b
1

c
1

a
2

b
2

c
2

GA: Mutation

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

m
ij
m
ij
m
ij
P
r
F
P
r
F
P
F
M
)
(
)
(
)
,
(

where
r

U(0,1)

is a random variable with
uniform distribution defined on a probability
space (

,

,P),

GA: Mutation (resolution)

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

'
2
'
1
'
2
'
1
'
2
'
2
'
1
'
1
'
2
'
2
'
2
'
2
'
2
'
1
'
2
'
1
'
2
'
2
'
1
'
1
'
1
'
1
'
1
'
1
'
1
c
c
c
c
b
b
b
b
a
a
a
a
c
c
c
c
b
b
b
b
a
a
a
a
P

125
.
0
125
.
0
0
0
0
0
0
A

125
.
125
.
0
0
0
0
0
0
0
B

I=0

125
.
0
0
0
0
0
0
0
0
A

125
.
125
.
125
.
0
0
0
0
0
0
B

I=0

Individual 1

Individual 2

a
1

b
1

c
1

a
2

b
2

c
2

Population=sons + sons mutated

g
g
g
C F
A
M C F
 
 

 
 

g
g
n
p
g
g
A
O
n
A
S
p
min
,

GA: Selecting Parents

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

)
(
1
w
F
F
F
g
g
g

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

THEOREM 1: (Weierstrass’s Approximation Theorem)

Let g be a continuous real valued function defined on
a closed interval [a,b]. Then, given any

positive,
there exists a polynomial y (which may depend on

)
with real coefficients such that:

For every x

[a,b].

Polynomial Discrete Time
Cellular Neural Network

)
(
)
(
x
y
x
g
Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

THEOREM 2 *:

Any Boolean Function of n
-
variables
can be realized using a Polynomial Threshold gates
of order s

n.

The quadratic threshold gate can be defined:

And s is the number of inputs and T is the
threshold constant.

Polynomial Discrete Time
Cellular Neural Network

otherwise
T
x
x
w
x
w
if
y
n
i
n
i
j
i
ij
i
i
0
1
1
1
Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

*
N. J. Nilsson. The Mathematical Foundations of Learning
Machines. McGraw Hill, New York, 1990.

PDTCNN: the model (I)

c
d
d
c
N
d
d
c
d
c
N
d
d
c
d
c
i
y
u
g
u
B
k
y
A
k
x
r
r

)
,
(
)
(
)
(
)
(
)
(

0
)
1
(
1
0
)
1
(
1
))
1
(
(
)
(
k
x
f
k
x
if
k
x
f
k
y
c
c
c
c

)
0
(
)
0
(
)
0
(
)
,
(
)
(
)
(
)
(
c
d
c
N
d
c
d
c
d
c
N
d
c
d
d
d
c
N
d
c
d
d
d
y
u
P
u
y
P
y
u
P
y
u
g
r
r
r

)
(
)
(
)
(
)
,
(
)
(
)
(
)
(
k
y
u
P
u
k
y
P
k
y
u
P
y
u
g
c
d
c
N
d
c
d
c
d
c
N
d
c
d
d
d
c
N
d
c
d
d
d
r
r
r
Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

PDTCNN: the model (II)

)
(
)
(
)
,
(
)
(
)
(
k
u
Q
k
y
P
y
u
g
d
c
N
d
c
d
d
c
N
d
c
d
d
d
r
r

c
d
d
c
N
d
d
c
d
c
N
d
d
c
d
c
i
y
u
g
u
B
k
y
A
k
x
r
r

)
,
(
)
(
)
(
)
(
)
(
Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

PDTCNN: Solving XOR problem

Some papers:

Z. Yang, Y. Nishio, A. Ushida,
Templates and algorithms for two
-
layer
cellular neural networks.
IJCNN’02, 2002.

F. Chen, G. He, G. Chen & X. Xu,
Implementation of Arbitrary Boolean
Functions via CNN.
CNNA’06, 2006.

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

PDTCNN:Solving XOR problem

M. Balsi, Generalized CNN: Potentials of a
CNN with Non
-
Uniform Weights.
CNNA
-
92,
2002

.

E. Bilgili, I. C. Göknar and O. N. Ucan,
Cellular neural network with trapezoidal
activation function.
Int. J. Circ. Theor. Appl
.,
2005

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

Learning parameters

Initialpop=20000

Number of fathers=7

Maximum number of random parents to

Kpro=0.8

Increment=1

Mutation Probability=0.15

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

PDTCNN:
First Scheme U:u
ij
=x
ij

x
ij+1

1
4
0
1
1
5
0
1
4
0

i
P
B
A

0
3
0
0
0
1
0
0
3
0

i
P
B
A

0
1
0
0
0
1
0
0
1
0

i
P
B
A

)
(
)
(
)
(
1
k
y
u
u
k
y
k
x
c
c
c
c
c

)
(
)
(
)
(
)
,
(
)
(
)
(
)
(
k
y
u
P
u
k
y
P
k
y
u
P
y
u
g
c
d
c
N
d
c
d
c
d
c
N
d
c
d
d
d
c
N
d
c
d
d
d
r
r
r
Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

PDTCNN:
Second Scheme U:u
ij
=x
ij

y
ij

2
2
8
1
0
0
0
0
0
0

i
P
B
A
b)

1
1
5
1
0
0
0
0
0
1

i
P
B
A
c)

0 1 0 0 0 0
0 1 0 0
A B
P i
  
  
)
(
)
(
)
(
k
y
u
k
y
k
x
c
c
c
c

)
0
(
)
0
(
)
0
(
)
,
(
)
(
)
(
)
(
c
d
c
N
d
c
d
c
d
c
N
d
c
d
d
d
c
N
d
c
d
d
d
y
u
P
u
y
P
y
u
P
y
u
g
r
r
r
Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

The Game of Life (I)

The

Game

of

Life

(GoL)

is

a

totalistic

cellular

automaton

consisting

in

a

two
-
dimensional

grid

cells,

that

may

be

either

alive

(black)

or

(white)
.

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

The Game of Life (II)

The

state

of

each

cell

varies

according

to

the

following

rules
:

Birth
: a cell that is dead at time t becomes alive at
time t + 1 only if exactly 3 of its neighbors were alive
at time t;

Survival
: a cell that was living at time t will remain
alive at t + 1 if and only if it had exactly 2 or 3 alive
neighbors at time t.

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

The Game of Life (III)

Every

sufficient

well
-
stated

mathematical

problem

can

be

reduced

to

a

question

Life
;

It

is

possible

to

make

a

life

computer

(logic

gates,

storage

etc
.
)
;

Life

is

universal
:

it

can

be

programmed

to

perform

any

desired

calculation
;

Given

a

large

enough

Life

space

and

enough

time,

self
-
reproducing

animals

will

emerge
...

The

whole

universe

is

a

CA!

(E
.
Fredkin,

MIT)
.

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

The Game of Life

NOT gate

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

A

CNN & GoL

Multilayer CNN (Chua, Roska)

1990

Activation function (Chua, Roska)

1990

CNN
-
UM (Roska,Chua)
-
1990

CNN Universal Cells (Dogaru, Chua)

1999

Simplicity vs. Computational power

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

Polynomial CNN (I)

)
,
(
)
(
)
(
d
d
e
d
e
e
d
e
d
y
u
g
i
u
b
n
y
a
n
x

0
)
(
,
1
0
)
(
,
1
)
(
n
x
n
x
n
y
d
d
d
What’s g(u
d
,y
d
)?

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

Polynomial CNN (II)

In the simplest case g(u
d
, y
d
) is a second
degree polynomial, whose general form is

2
0
2
)
)
(
)
(
)
(
)
((
)
,
(
i
i
e
d
e
i
i
e
d
e
i
d
d
y
q
u
p
y
u
g
2
0
0
)
(
)
(
)
1
(
)
(
e
d
e
e
d
e
y
q
p

)
(
)
(
)
(
)
(
1
1
e
d
e
e
d
e
y
q
u
p

)
1
(
)
(
)
(
)
(
2
2
2
e
d
e
e
d
e
q
u
p

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

Polynomial CNN (III)

Thanks to some considerations we find that

)
(
)
(
)
(
)
(
)
(
)
(
)
(
1
2
0
d
e
d
e
e
d
e
e
d
e
e
d
e
d
y
u
p
y
q
i
u
b
n
y
a
n
x

0
0
0
0
0
0
0
0
c
a
A

p
p
p
p
c
p
p
p
p
b
b
b
b
b
b
b
b
b
B

0
0
0
0
0
0
0
0
0
0
c
q
Q

p
p
p
p
c
p
p
p
p
p
p
p
p
p
p
p
p
p
P
1
1
1
1
1
1
1
1
1
1
Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

Polynomial CNN (IV)

p
c
p
p
c
c
c
c
d
u
y
p
b
u
y
p
b
n
x
)
(
)
(
)
(
i
y
q
y
a
c
c
c
c

2

u
c

and appear in the state equation

direct link with totalistic Cellular Automata

p
u
Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

GoL: Rules (I)

GoL: Rules (II)

Rule 1: a cell will be alive at least 3 of the 9 cells
in its 3
×

3 neigh. are alive

Black pixel= +1

White pixel=
-
1

pixel centr. = 1 (black)

Σ neigh. =
-
2 (5 w, 2 b)

next state =
-
1 (white)

GoL: Rules (III)

Rule 2: a cell will be alive if at most 3 of its 8
neighbors are alive

Black pixel= +1

White pixel=
-
1

pixel centr. = 1 (black)

Σ neigh. =
-
2 (5 w, 2 b)

next state = 1 (black)

Design algorithm (I)

First iteration: we try to perform the first rule (a cell
will be alive at least 3 of the 9 cells in its 3
×

3 neigh.
are alive)

If Y(0)=0

b
c
=1 b
p
=1 i=3

p
c
p
p
c
c
c
c
d
u
y
p
b
u
y
p
b
n
x
)
(
)
(
)
(
Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

Design algorithm (II)

Second iteration: we try to accomplish the second
rule (
a cell will be alive if at most 3 of its 8 neighbors
are alive
)

p
c
p
c
c
c
d
u
y
p
u
y
p
n
x
)
1
(
)
1
(
)
(
3
2

c
c
c
c
y
q
y
a
Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

Design algorithm (III)

Hyp: p
c
=0

Templates found using learning

Coming soon...

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

Conclusions (I)

In

general
:

In

some

cases

it

is

possible

to

reduce

a

multilayer

DTCNN

to

a

single

layer

PDTCNN

Thanks

to

the

GoL

we

can

explore

the

capacity

of

PDTCNNs

for

Universal

Machine

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

Conclusions (II)

learning
:

The

resolution

used

reduces

the

search

space

The

step

random

parent”

improves

the

behavior

to

avoid

local

minimas

design

We

give

a

simple

algorithm

to

design

templates

for

the

Polynomial

CNN

Future Work

Implementations of mathematical morphology
functions with PDTCNNs

Intro

CNN & GA

Polyn. CNN

XOR

GoL

Conclusions

Polynomial Discrete Time
Cellular Neural Networks

Eduardo Gomez
-
Ramirez

Giovanni Egidio Pazienza

egr@ci.ulsa.mx

gpazienza@salle.url.edu