Cellular Neural Networks Reported by:

maltwormjetmoreAI and Robotics

Oct 19, 2013 (4 years and 6 months ago)


Cellular Neural Networks

Reported by:
Hamada Ahmed Khadragy

A novel class of information processing system called cellular
neural network is proposed. Like a neural network, it is a large-
scale non linear analog circuit which processes signals in real
time. Like cellular automata, it is made of a massive aggregate of
regularly spaced circuit clones called cells, which communicate with
each other directly only through its nearest neighbors. Each cell is
made of a linear capacitor, a nonlinear voltage-controlled current
source, and a few resistive linear circuit elements. Cellular neural
networks share the best features of both worlds; its continuous time
feature allows real-time signal processing found wanting in the
digital domain and its local interconnection feature makes it tailor
made for VLSI implementation. Cellular neural networks are uniquely
suited for high speed parallel signal processing.[1]

After the rebirth of the interest for artificial Neural Networks in 1982
different implementations for this concept have been developed. The differences
between these implementations affect the overall characteristics of the network and
thus the applicability for a certain purpose like, for example, real time image
processing. Here, an important feature of neural networks is touched upon; these
circuits can operate very fast since the 'signal processing' is done simultaneously in
parallel neurons, rather than in sequential time steps.
The major disadvantage of most neural network implementations however is the
number of interconnections between neurons.
order to reduce the number of
interconnections but keep the advantages of parallel processing, Chua and Yang
proposed a so-called Cellular Neural Network (CNN) in 1988 where neurons were
only connected to other neurons within a certain neighborhood.
1992, Rodriguez-Vazquez proposed a modified model for neurons; the so-called
Full Range (PR) model and an implementation for a CNN in the current domain based
on this FR-model.
1996 P. Bruin suggested an implementation for a FR-CNN in the voltage domain.
However, after realization of the implementation, it was found to display instabilities.
Every neuron in a cellular neural network can be described by a certain state-
equation. If this network consists of two interconnected cells, the behavior that is
displayed by those cells can be plotted in a so-called state space.
is interesting to be
able to 'predict' to what final state the CNN will converge. For this, the state-space
will be partitioned into 'basins of attraction', separated by certain boundaries. In this
report a method to find the boundaries of the basins of attraction is proposed.
A Cellular Neural Network
A Cellular Neural Network (CNN) is an artificial neural network consisting of
separate neurons or cells. It has several properties that may be advantageous
compared to other neural networks. The dynamic range of a CNN for example is
bounded. Also a CNN can easily be extended without having to re-adjust the entire
network because a cell is not connected to every other cell in the network but rather to
cells within a certain neighborhood. Albeit its cellular structure it still displays the
complex dynamic behavior as seen with other neural networks. Due to this complex
behavior it can be used in image processing (e.g. noise-removal, connected
component detection (CCD), 'thinning' etc.) It can also be used to simulate certain
equations or be used as an associative memory. Another advantage is that although
the factor with which the output of a cell affects the behavior of other cells (template
parameter) may be different for spatially different neighboring cells, the template thus
formed is translational invariant ('cloning templates'). As stated above, these cells can
interact with other cells within a certain neighborhood. A rectangular two-
dimensional CNN consisting of 16 cells where every cell interacts with only its
directly neighboring cells would look like the circuit shown in Figure 1.

If C(i,j) is the cell on the ilh row and jth column then cell C(2,2) is connected to
C(O,I), C(O,2), C(O,3), C(2,I), C(2,2), C(2,3), C(3,l), C(3,2) and C(3,3).

Architecture of Cellular Neural Networks
Any cell in a CNN is connected only to its neighbor cells. The adjacent cells can
interact directly with each other. Cells not directly connected together may affect each
other indirectly because of the propagation effects of the dynamics of CNNs. An
example of a two-dimensional CNN is shown below.

Every cell is influenced by a limited number of cells in its environment. This locality
of connections between the units is the main difference between CNNs and other
neural networks. Large CNN chips can be implemented using VLSI techniques.
The figure above shows the emphasized cell (black) connected to the nearest
neighbors (gray). The cells marked in gray represent the neighborhood cells of the
black cell. The neighborhood includes the black cell itself. This is called a "3*3-
Similarly, we could define a "5*5-neighborhood", a "7*7-neigborhood" and so on.
The basic circuit unit of CNNs is called a cell. It contains linear and nonlinear
circuit elements, which typically are linear capacitors, linear resistors, linear and
nonlinear controlled sources, and independent sources. All the cells of a CNN have
the same circuit structure and element values. A typical circuit of a single cell is
shown in the figure below.


Each cell contains one independent voltage source E
(Input), one independent
current source I (Bias), several voltage controlled current sources I
u ij
, I
y ij
, and
one voltage controlled voltage source E
y ij
(Output). The controlled current
sources I
u ij
are coupled to neighbor cells via the control input voltage of each
neighbor cell. Similarly, the controlled current sources I
y ij
are coupled to their
neighbor cells via the feedback from the output voltage of each neighbor cell.
The cell C(i,j) has direct connections to its neighbors through two kinds of weights:
the feedback weights a(k,l;i,j) and the control weights b(k,l;i,j), where the index pair
(k,l;i,j) represents the direction of signal from C(i,j) to C(k,l). The coefficients
a(k,l;i,j) are arranged in the feedback-Template or A-Template. The coefficients
b(k,l;i,j) are arranged in the control-Template or B-Template.
The A-Template and the B-Template are assumed to be the same for all the cells in
the network. The global behavior of a CNN is characterized by a Template Set
containing the A-Template, the B-Template, and the Bias I.
If we assume a "3*3-neighborhood", the Template Set consists of 19 coefficients.
The external input to the cell is typically assumed to be constant over a certain
operation interval. Therefore, the total input current to the cell is given by the
weighted sum of control inputs and weighted sum of feedback outputs. In addition, a
constant bias term (I) is added to the cell. Due to the capacitance C and resistance R,
the state voltage x(i,j) satisfies the following differential equation:

k denotes the neighborhood of the specific cell

Without loss of generality, the time constant T = R*C can be set to 1.
The only nonlinear element in each cell is a piecewise-linear voltage controlled
voltage source with characteristic
y(i,j) = f(x(i,j)).
A widely used nonlinearity is the piecewise-linear function as given by:
y(i,j) = f(x(i,j)) = 0.5*(|x + 1| - |x - 1|)

The block diagram of a cell C(i,j) is shown in the figure below.

Modeling and Simulation of the CNN
-Simulation plays an important role in the design of the CNN cloning templates.
-Therefore, it has to be fast enough to allow the design phase of various templates
be accomplished in reasonable time.
-At the same time, the simulation has to be accurate enough, to reflect the
behavior of the analog circuitry correctly.
-In practice, the simulation of the CNN involves a trade-off between accuracy and
computation time
-The true processing capabilities of CNNs for high-speed parallel processing are
only fully exploited by dedicated VLSI hardware realizations.
-Typical CNN chips may contain up to 200 transistors per pixel.
-At the same time, industrial applications require large enough grid sizes (around
100 x 100).
-Thus, CNN chip designers must confront complexity levels larger than 106
transistors, most of them operating in analogue mode
-On the one hand, high-level simulation, which is focused on emulating the
functional behaviour, cannot reflect realistically the underlying electronic
circuitry. Their lack of detail makes them ill-suited for reliable IC simulation.
-On the other hand, the SPICE-type transistor-level simulators, although very
accurate, are barely capable of handling more than about 105 transistors and may
take several days of CPU time for circuit netlists containing about 106 transistors.
-Hence, these low-level tools are ill-suited for simulating large CNN chips.
-Therefore, it would be necessary to bridge the gap between these approaches,
which would give very accurate results in reasonable (but not real-) time.
-However, our main concern now is fast simulation
Global behavior of Cellular Neural Networks
In image processing, n-by-m rectangular grid arrays are often used. n and m are the
numbers of rows and columns, respectively. Each cell in a CNN corresponds to an
element of the array.
Assuming that each cell is connected to its nearest neighbors only ("3*3-
neighborhood") and that the local connections of a cell do not depend on the cell's
position, the Template set contains 19 coefficients (A-Template: a1 .. a9, B-Template:
b1 .. b9, Bias I). The behavior of the CNN is completely determined by this Template

New CNN-Templates for arbitrary tasks may be found using a training algorithm, or
by defining local rules for a given global task. The local rules describe a cell's
equilibrium state depending on the inputs and outputs of the neighbor cells. The
inputs and the outputs of the neighbor cells are assumed the be constant. The
dynamics of the cell is not specified.
If Template values for the local rules are found, simulations are very helpful to test
the dynamic global behavior of the entire clone of cells.

Optimal coefficient calculation leads to solutions which converge after short time.
This means that the output of every cell reaches its final output y=+1 or y=-1 after
short time.
Characteristics of a simple CNN Template

Template set
We assume "3*3-neighborhood". A simple Template set for edge extraction is

Global task: Binary edge detection
If the input image is a binary image (black and white), the output of the CNN
will be a binary image showing the edges of the input image only. If the input
image has intermediate (gray) values, the operation of the CNN with this
simple Template set is not well defined..
Input: U(t) = static binary image
Initial state: X(0) = arbitrary (reason: Feedback Template = 0)
Output: Y(t) converges toward a binary image showing all edges of the input

Local rules
Regarding the global tasks, we can define local rules for a single cell. The
rules are deduced from the following reflections:
1. A white pixel never turns black.
2. A black pixel turns white if it is surrounded by black pixels
3. A black pixel never turns white, when at least one neighbor cell is white. In
this case, this cell belongs to the edge of the object

Input u(i, j) Output y(i, j) (t = infinity)
1. white pixel white, independent of neighbors
2. black pixel white, if all nearest neighbors are black
3. black pixel black, if at least one nearest neighbor is white


Global behavior
The screenshots below show the correct behavior of the Template set.

The input picture above is the input picture of the CNN. It does not change in
this small time period. The four pictures below show the dynamcal behavior of
the entire grid. Starting with the initial state on the left side, the pictures show
the state of the CNN cluster at t = 0.2, t = 0.5 and t = 2.

After two time units (t = 2), the output of the CNN shows the edges of the
input image. The Template set produces the desired output.

The philosophy, interests, and methodologies of CNN researchers are varied. Due to
the potential of the CNN architecture, this platform has attracted people from a variety
of backgrounds and disciplines. Some are exploring practical implementations of
CNN processors, others are using CNN processors to model physical phenomena, and
there are even researchers exploring theoretical mathematical, computational, and
philosophical ideas through CNN processors. This chapter briefly explores what are
CNN processors and what they are being used for. Some applications are engineering
related, where some known, understood behavior of CNN processors is exploited to
perform a specific task, and some are scientific, where CNN processors are used to
explore new and different phenomenon. Hopefully this section will illuminate that
CNN processors are versatile platforms that are being used for a variety of
CNN processors were designed to perform image processing; specifically, the original
application of CNN processors was to perform real-time ultra-high frame-rate
(>10,000 frame/s) processing unachievable by digital processors needed for
applications like particle detection in jet engine fluids and spark-plug detection.
Currently, CNN processors can achieve up to 50,000 frames per second, and for
certain applications such as missile tracking, flash detection, and spark-plug
diagnostics these microprocessors have outperformed a conventional supercomputer.
CNN processors lend themselves to local, low-level, processor intensive operations
and have been used in feature extraction, level and gain adjustments, color constancy
detection, contrast enhancement, deconvolution, image compression, motion
estimation, image encoding, image decoding, image segmentation, orientation
preference maps, pattern learning/recognition, multi-target tracking, image
stabilization, resolution enhancement, image deformations and mapping, image
inpainting, optical flow, contouring, moving object detection, axis of symmetry
detection, and image fusion.
Due to their processing capabilities and flexibility, CNN processors have been used or
has been prototyped for novel field applications such as flame analysis for monitoring
combustion at a waste incinerator, mine-detection using infrared imagery, calorimeter
cluster peak for high energy physics, anomaly detection in potential field maps for
geophysics, laser dot detection, metal inspection for detecting manufacturing defects,
and seismic horizon picking. They have also been used to perform biometric functions
such as fingerprint recognition, vein feature extraction, face tracking, and generating
visual stimuli via emergent patterns to gauge perceptual resonances. CNN processors
have been used for medical and biological research in performing automated
nucleated cell counting for detecting hyperplasia, segment images into anatomically
and pathologically meaningful regions, measuring and quantifying cardiac function,
measure the timing of neurons, detecting brain abnormalities that would lead to
seizers. One potential future application of CNN microprocessors is to combine them
with the DNA microarrays to allow for a near-real time DNA analysis of hundreds of
thousands of different DNA sequences. Currently, the major bottleneck of this DNA
microarray analysis is the amount of time needed to process data in the form of
images, and using a CNN microprocessor, researchers have reduced the amount of
time needed to perform this calculation to 7ms.
CNN processors have also been used to generate and analyze patterns and textures.
One motivation was to use CNN processors to understand pattern generation in
natural systems. They were used to generate Turing patterns in order to understand the
situations in which they form, the different types of patterns which can emerge, and
the presence of defects or asymmetries. Also, CNN processors were used to
approximate pattern generation systems that create stationary fronts, spatio-temporal
patterns oscillating in time, hysteresis, memory, and hetergenity. Furthermore, pattern
generation was used to aid high-performance image generation and compression via
real-time generation of stochastic and coarse-grained biological patterns, texture
boundary detection, and pattern and texture recognition and classification. Control
and Actuator Systems
There is an ongoing effort to incorporate, CNN processors into sensory-computing-
actuating machines as part of the emerging field of Cellular Machines. The basic
premise is to create an integrated system that uses CNN processors for the sensory
signal processing and potentially the decision making and control. The reason is that
CNN processors can provide a low power, small size, and eventually low cost
computing and actuating system suited for that type of system. These Cellular
Machine will eventually create a Sensor-Actuator Network (SAN), a type of Mobile
Ad Hoc Networks (MANET) which can be used for military intelligence gathering,
surveillance of inhospitable environments, maintenance of large areas, planetary
exploration, etc.
CNN processors have been proven versatile enough for some control functions. The
have been used as associative memories, optimize function via genetic algorithm,
measuring distances, optimal path finding in a complex, dynamic environment, and
theoretically be used to learn and associate complex stimuli. They have also been used
to create antonymous gaits and low-level motor for robotic nematodes, spiders, and
lamprey gaits using a Central Pattern Generator (CPG). They were able to function
using only feedback from the environment, allowing for a robust, flexible,
biologically inspired robot motor system. CNN-based systems were able to operate in
different environments and still function if some of the processing units were
The variety of dynamical behavior seen in CNN processors make them intriguing for
communication systems. Chaotic communications using CNN processors is being
researched due their potential low power consumption, robustness and spread
spectrum features. The premise behind chaotic communication is to used a chaotic
signal for the carrier way and to use chaotic phase synchronization to reconstruct the
message. CNN processors can be used on both the transmitter and receiver end to
encode and decode the message. They can also be used for data encryption and
decryption, source authentication through watermarking, detecting of complex
patterns in spectrogram images (sound processing), and transient spectral signals
CNN processors are neuromorphic processors, meaning that they emulate certain
aspects of biological neural networks. The original CNN processors were based on
mammalian retinas, which consist of a layer of photodetectors connected to several
layers of locally coupled neurons. This makes CNN processors part of an
interdisciplinary research area whose goal is to design systems that leverage
knowledge and ideas from neuroscience and contribute back via real-world validation
of theories. CNN processors have implemented a real-time system that replicates
mammalian retinas, validating that the original CNN architecture modeled the correct
aspects of biological neural networks used to perform. However, CNN processors are
not only limited to verifying biological neural networks associated with vision
processing; they have been used to simulate dynamic activity seen in mammalian
neural networks found in the olfactory bulb and locust antennal lobe, responsible for
pre-processing sensory information to detect differences in repeating patterns.
CNN processors are being used to understand systems that can be modeled using
simple, coupled units, such as living cells, biological networks, physiological systems,
and ecosystems. The CNN architecture captures some of the dynamics often seen in
nature and is simple enough to analyze and conduct experiments. They are also being
for stochastic simulation techniques, which allows scientists to explore spin problems,
population dynamics modes, lattice gas models, percolation, etc. Other simulation
includes heat transfer, mechanical vibrating systems, protein production, Josephson
Transmission Line (JTL), seismic wave propagation, and geothermal structures. The
3D (Three Dimensional) CNN have been used to prove as complex shapes are
emergent phenomena establishing a link between art, dynamical systems and VLSI
technology. CNN processors have been used to research a variety of mathematical
concepts, such as researching non-equilibrium systems, constructing non-linear
systems of arbitrary complexity using a collection of simple, well-understood
dynamic systems, studying emergent chaotic dynamics, generating chaotic signals,
and in general discovering new dynamic behavior. They are often used in researching
systemics, a trandisiplinary, scientific field that researches natural systems. Their goal
is to develop a conceptual and mathematical framework necessary to analyze, model,
and understand systems, including, but are not limited to, atomic, mechanical,
molecular, chemical, biological, ecological, social and economic systems. Topics
explore are emergences, collective behavior, local activity and its impact on global
behavior, and quantifying the complexity of an approximately spatially and
topologically invariant system. Although another measure of complexity many not
make some people enthusiastic (Seth Lloyd, a professor from Massachusetts Institute
of Technology (MIT), has identified 32 different definitions of complexity), it can be
potentially be mathematically analyze systems such as economic and social systems.

[1] L.O. Chuo and L.Yang, "Cellular neural networks: Applications," IEEE Trans.
Circuits Syst., pp. 1273-1290, this issue
[2] J.J Hopfield, "Neural networks and physical system with emergent computational
abilities," proc. Natl. Acad. Sci. USA.
[3] D. Balya, G, Tímar, G. Cserey, and T. Roska, "A New Computational Model for
CNN-Ums and its Computational Complexity", Int’l Workshop on Cellular Neural
Networks and Their Applications, 2004
[4] L. Chua, T. Roska, Cellular Neural Networks and Visual Computing: Foundations
and Applications, 2005.
[5] V. Cimagalli, M. Balsi, "Cellular Neural Networks: A Review", Neural Nets
WIRN Vierti, 1993.
[6] H. Harrer and J.Nossek, "Discrete-Time Cellular Neural Networks", Int'l Journal of
Circuit Theory and Applications, 20:453-467, 1992.

[7] S. Majorana and L. Chua, "A Unified Framework for Multilayer High Order CNN",
Int’l Journal of Circuit Theory and Applications, 26:567-592, 1998.

[8] T. Roska, L. Chua, "The CNN Universal Machine: An Analogic Array Computer",
IEEE Trans. on Circuits and Systems-II, 40(3): 163-172, 1993