8. Cellular Automata - Natural Computing Group, LIACS, Leiden ...

backporcupineAI and Robotics

Dec 1, 2013 (3 years and 11 months ago)

98 views

Cellular Automata
Simulation of discrete
spatio-temporal
systems
Systems with many variables
￿Iterative function systems describe systems with a
single variable
￿A iterative system with two variableswas given by
the Julia set
￿Systems in economy, meteorology, ecology,
sociology, etc., consists often of a large number of
variables, interacting with each other
￿Besides chaos many new phenomena occur in such
many variable systems
￿These are for example evolution, self-organization,
emergence, self-reproduction, phase-transitions
Complexity, Chaos, and Anti-Chaos
￿The study of spatio-temporal systems will
reveal that complexity and chaos are not the
same
￿Chaotic processes can produce simple
patterns called anti-chaos
￿The emergence of order out of chaos can be
observed (Cohen, Stewart 1994)
S.A. Kauffman (1991) Antichaosand adaptation, Scientific American,
265 (2), 64-70
J. Cohen and I. Stewart 1994: The collapse of chaos, Penguin, NY
Simple models of complex systems
￿A realistic treatment of complex systems is computationally
expensive and often intractable
￿Scientists are looking for simple models of complex systems
￿Often the predictions made with these simple models are
surprisingly realistic or provide deep insights in the dynamics of
real world systems (e.g. explaining phase-transitions).
￿However, the science of complexity is in its very beginning and
a new frontier in nonlinear systems dynamics; definitions are
evolving and scientists have not yet discovered unifying
theories;
T. Bohr et al. (1998): Dynamical systems approach to turbulence,Cambridge
University Press, New York
Locally interacting cell arrays
￿One of the simplest models involve a spatial
array of cells
￿The cells interact with nearby cells by simple
rules
￿These systems often exhibit spatio-temporal
chaos
–Spatial patterns in time are aperiodicand difficult
to predict
–Complex, often self-similar,patternsevolve in time
Cellular automata
￿Von Neumann introduced cellular automata
1966, Wolfram studied them extensively and
classified them (“A new kind of science”)
￿CA are perhaps the most simple models of
spatio-temporal systems, but their behavioral
spectrum is wide and interesting to study
Wolfram, S. (1986) Theory and application of cellular automata, World
Scientific, Singapore
Von Neumann (1966) Theory of self-reproducing cellular automata, Univ.
Of Illinois Press,Urbana, Il.
Wolfram S. (2002) A new kind of science, Wolfram Press
A motivating example –the XOR 1D
Automaton
￿Consider a ring of people
￿Each one is wearing a cap with the bill forwards,
except one who is wearing the bill to the back
￿Now, each one is looking at his/her left and right
neighbor, and adapts using these rules:
–Left and right neighbor have bill forwards ￿wear bill
forwards
–Left and right neighbor have bill backwards ￿wear bill
forwards
–If only one neighbor has bill forwards ￿wear bill backwards
￿Evolve system over a number of generations for a
large ring of people
Simulation for 15 people, 7 time-steps
BBBB
BBBB
BB
BBBB
BB
BB
B
time
space
Periodic boundary condition
BBBB
BBBB
BBBB
BBBB
BB
BB
BBBBBBBB
Sierpinskicones and maximal-speed of
information
￿Long term behavior shows self-
similar cone-like structures
￿They resemble Sierpinski
triangles
￿The maximal speed-of-
information gives rise to the
boundary of the cones, similar
to the speed-of-light giving rise
to Minkowski’sspace-time
cones
￿Indeed, in CA literature the set
of possible states that
influenced the system in some
past are called ‘light cones’
￿Complex organization, but no
chaos is evident
SierpinskiTriangle in Nature
Formal expression
￿The given game is an example of a
1-D cellular automaton
￿Several ways to express a cellular
automata rule

X(t+1,i)=(X(t,i-1)+X(t,i+1)) modulo 2

X(t+1,i)=X(t,i-1) XOR X(t,i+1)
￿The XOR statement is a logical
function
￿24
= 16 logical functions could be
tried instead of XOR
x411
x301
x210
x100
t+1,it,i+1t,i-1
Rule of a cellular automaton
Initial state
￿The evolution of a cellular
automaton is defined also by its
initial state
￿The left figure shows the
evolution of a cellular
automaton with random initial
conditionusing the XOR
function
￿The behavior is not chaotic, but
propagate the initial conditions
in an ordered way forward in
time
￿Other rules may give rise to
chaotic behavior from ordered
starting conditions
The size of the rule space
￿The discussed XOR automaton is an example of an
1-D Cellular automaton
￿The size of the neighborhood and the number of
possible states determine the number of possible
rules for a cellular automaton
￿If we consider N nearest neighbors to each side, the
number of possible rules would grow to:
Why?
S(N)=2
22N
Four dynamical classes of Cellular
automata
￿Cellular automata were classified by Wolfram (2002), into four
classes based on their dynamics
1.Class 1 reach a homogeneous state with all cells the same for
all initial conditions
2.Class 2 reach a non-uniform state that is either constant or
periodic in time, with a pattern depending on initial conditions
3.Class 3 have somewhat random patterns, are sensitive to
initial conditions, and small scale local structure
4.Class 4 have relatively simple localized structures that
propagate and interact in very complicated ways
￿The four classes correspond roughly to fixed points,
periodicity, and chaos in dynamical systems ￿examples will
follow
Langton’s λ
λ λ λquantity
￿Langton’squantity λis the
number of state
configurations that map to 1
divided by the total number
of state configurations
￿For instance in the left figure
λ=3/8
￿As the numbers of 0 equals
1-λ, only the range from 0
to 0.5 is of particular interest
0111
0011
0101
1001
0110
0010
1100
1000
t+1,it,i+1t,it,i-1
Langton, C. (1986) Studying artificial
life with cellular automata, PhysicaD 22, 120-49
Langton’s λ λ λ λquantity and dynamic
behaviour
Solid at zero temperature
Melting Fluid
Solid at finite temperature
Turbulent Fluid
Melting fluid
Solid at finite temperature
λ
By increasing λfrom 0 to 0.5 (1 downto0.5) roughly
the system goes through the same states than the
logistic map for different values of the constant a
￿
Assignment
Higher dimensional Cellular automata
￿Cellular automata can be defined not only for
1-D arrays but also on higher dimensional
arrays
￿Some mathematical notation:
–1-D arrays are called chains
–2-D arrays are called grids
–Arrays of any dimension d are called d-
dimensional lattices
Cellular automata in 2-D
￿A classical cellular automaton
was defined by Conway –
Conway’s game of life
￿Consider a ‘game’played on a
rectangular grid, each grid cell
can have two states –dead or
alive
￿The neighbors of a center cell
are the nearest neighbors to the
north, south, east, west, north-
west, south-west, south-east,
north-east
￿This is termed the Moore
Neighborhood
SE
S
SW
E
C
W
NE
N
NW
Cellular automata in 2-D
￿Rule
–A cell that is alive, stays
alive, if it has two or three
living neighbors
–A dead cell becomes alive,
when it has exactly three
living neighbors
–For all other cases a cell
dies or remains dead
￿Example of outer totalistic
rule, i.e. a rule that involves
only the sum of neighbor
states
SE
S
SW
E
C
W
NE
N
NW
Evolution of the game of life
Starting from
an initially random
Configuration
Colonies of cells
emerge, some
of them periodic
some of them fixed
or moving through
space, shooting
pixels (glider guns*)
etc.
*Berlekampet al. (1982) Winning ways for your mathematical plays, Academic
Press, New York
The glider gun
￿Conway offered 50$ for everyone, who
could find an endlessly growing
configuration or prove that none exists
￿William Gosper and 5 other MIT students
discovered the glider gun and won the price
–The glider gun shoots a copy of itself
–On an infinite grid it would grow and evolve
without limit
Other possible configuration spaces
￿Regular tilingsof the 2-
D plane (there are three
possibilities)
￿More than 3-
dimensional
configuration spaces
￿Most generally:
–Configuration spaces
represented by Caley
graphs of some group
All possible regular tilingsof
the 2-D plane,i.e. tilingsconsisting
only of the same objects
hexagonal
grid
General CA definition via Caleygraph
￿Groupsdescribe symmetric
structures
￿(M,+)is a group, iff

a,b,c

M:

a+b

M and a+(b+c)=(a+b)+c
–There exists
e

M
with
e+a=a
–each a

Mhas an inverse called
(-a),
such that
a+(-a)=e
.
￿We define a group Mvia a set of
generators
X

M
, such that for
every element
a

M
and generator
x
,
both
a+x

M
and
a+(-x)∈M
; Moreover,
all elements belong to the group that
can be obtained by concatenated
application of generators.
￿Given a generator
{x
1,…,x
m}
we can
define a Caleygraph
C=(V,E)
of the
group:

vertex set: V=M

edges Eare given by (v1,v2)

E, iffv1
= v2+a, or v1=v2+(-a)for some ain X.
￿The fundamental neighborhoodN(C)
of the Caleygraph is defined by the
union of the set of generators and the
set of its inverse elements.
￿For each element in the graph we can
get its neighbors by using the generator
elements in N(C).
￿A cellular automaton
(C, N(C),A, T)
is
defined as a tupleof a Caleygraph with
labeled vertices, its fundamental
neighborhood, an finite alphabet, and a
transition function T
￿Node labels are chosen from a finite
local state spaceA
￿Transition rule
T: A
|N(C)+1|
￿
A
assigns
each element of a cell a new value
based on the neighbors in the Caley
graph, obtained by applying the
generator.
Fd
=￿x1
,...,x
m
|￿Freegroup
Zd
=￿x1
,...,x
d
|∀m,n:xm
+xn
=xn
+xm
￿
Zm
×Zn
=￿a,...,b|ab=ba,ma=nb￿Torus
Neighborhood types and sizes
￿Von Neumann neighborhood and Moore
neighborhood are most commonly used in 2-D grids
￿The radius of these neighborhoods can be
increased, e.g. by applying group generators twice
Examples of groups and their Caley
graphs
Fd
=￿x1
,...,x
m
|￿Freegroup
Zd
=￿x1
,...,x
d
|∀m,n:xm
+xn
=xn
+xm
￿
Questions
￿How many transition rules can we define on
a cellular automaton (C, N(C), A, T)?
￿What could be the set of generators for the -
2D integer lattice with Moore neigborhood?
Extensions of Cellular automata
￿A multidimensional state space
–In Lattice gas models each cell is assigned a
vector (velocity of the fluid flow)
￿Memory of states in t-1, t-2, etc.
￿Dynamic rule sets, dynamic neighborhoods,
etc.
Finite cellular automata and chaos
￿Finite CA cannot be truly chaotic because the
number of states is finite, and thus the
system will eventually return to some
previous state and be trapped in a circle from
then on
￿To obtain maximal periods, prime numbers
are chosen as cell array sizes
Self-organization
￿Simple rules such as the game of lifecan cause an
initially chaotic state to evolve into a highly ordered
one ￿Self-organization
￿This somehow contradicts the third law of
thermodynamics (3LT), that the entropy is always
increasing
￿Hakenattributed the self-organizing behavior to
cooperative effects of the systems components
(synergetics)
￿The 3LT is motivated by deterministic systems, but in
fact also stochastic systems can self-organize
Forest simulation model by Sprott
￿Consider a forest with trees
placed on grid cells, 0=fur,
1=oak
￿We choose a random tree
that dies
￿We replace this tree with a
new tree
￿Five trees in the
neighborhood are chosen
randomly
￿If the vast majority (s=4,5) is
oak, the new tree gets an
oak
￿If the vast majority is fur
(s=0,1), the new tree gets a
fur
￿Otherwise (s=2,3), the same
tree than before will grow
￿Connected patterns emerge
from a random starting set
Broken symmetry
￿It is surprising, that despite the highly symmetrical starting
conditonthe emerging system does not converge to a
symmetrical object
￿This phenomenon is called spontaneous symmetry breaking
and can be observed in highly ordered systems, deterministic
systems (Wolfram 2002)
Self-organized critically
￿So-called dissipative structures
will emerge
￿Connected regions with a
strange but not necessarily
fractal boundary (fat fractals)
￿The size distribution of the
clusters follows a power laws
￿Dissipative patterns are
observed in many spatio-
temporal processes
–Animal migration
–Spread of diseases
–Vegetation patterns
–Clouds and mud
Prigogine, I. (1997) The end of certainty: time, chaos, and the new laws of
nature, the free press, new york
Self-organized critically
￿Systems like the forest
converge to a pattern for which
there is no characteristic scale
size
￿Size distributions of objects
often obey power laws (this
they share with the fractals), i.e.
the distribution can be fitted to a
function
￿Recently, power laws are
applied in all kind of
applications
–Gene regulatory networks
–DNA pattern
–Stock prices
–City distributions
–Letter frequency in human/ape
generated random strings
(Zipf)
￿Not always SOC is the
explanation for the Power law
￿In case of city size distribution it
related to a least effort principle
(Zipf).
d

1
/
f
α
Diffusion
￿Diffusion can be modelledvia:
￿Note, that there is a conservation rule fulfilled
￿Task: Implement diffusion system in 2-D in
MATLAB and visualize its behaviourover
time
a
j
(
t
+
1
)−
a
j
(
t
)=
a
j
+
1
(
t
)−
2
a
j
(
t
)+
a
j

1
(
t
)
Sand Pile –the prototype of a SOC
system
￿Consider a pile of sand to which we add
sand continuously
￿The sand-pile steepensuntil it reaches an
angle of repose, whereupon avalanches
keep the sandpileclose to this angle
￿The avalanches obey a power law scaling in
their size distribution and in their duration
A pile of cheese
Bak’sCA simulation of a pile of sand
￿Baksimulated a pile of sand using the
following CA model
￿The pile is represented by a N ×N matrix
of integers
￿Initially all cells are chose between 1 and
3
￿At each time step choose a random cell
i,jand set Z(t+1,i,j)=Z(t,i,j)
￿Cells outside the boundary are kept as 0
￿All other cells which exceed Z=3, and
their von-Neumann neighbors are
updated with:
Z(t+1,i,j)=Z(t,i,j)-4
Z(t+1,i±1,j)=Z(t,i±1,j)+1
Z(t+1,I,j±1)=Z(t,I,j±1)+1
￿Strictly speaking, this is not a cellular
automaton, as it evolves not
autonomously
d=2
Power spectrum of sum(Z(i,j)) over t
Dropping sand on the central point
Emergence vs. Reductionism
￿Reductionism assumes simple laws that govern
natural processes and that these simple laws help to
understand/explain global behaviour
￿Emergence holds that high level structure is
generally unpredictable from low level processes,
and does not even depend very much on its
properties
￿Due to Sprott(2006), systems are complex, if they
exhibit emergent behaviour
How to measure degree of
(self-)organization
￿The term of self-organization
is used since 1947, but up to
know there is no standard
definition except “I know it
when I see it”
￿Thermodynamic entropy
measures the degree of a
system’s “mixedupedness”
(to use Gibbs’s word), or
how far it departs from being
in a pure state
￿Organisms are essentially
never in pure states, and are
highly mixed up at the
molecular level, but are the
paradigmatic examples of
organization.
￿Furthermore, there are many
different kinds of
organization, and entropy
ignores all the distinctions
and gradations between
them
W. R. Ashby, “Principles of the self-organizing dynamic system,”Journal of
General Psychology 37, pp. 125–128, 1947.
How to measure a degree of self-
organization and complexity?
￿Another school of thought has
been put forward by
Kolmogorovand Solomonoff
“A complex phenomena is one
which does not admit of
descriptions which are both
short and accurate”
￿Problem exactness: Coin
tossing, produces sequences of
maximal Kolmogorov
complexity, though dynamics
are simple to describe.
￿Grassbergergave a more
general definition: ‘The
complexity of a process as the
minimal amount of information
about its state needed for
maximally accurate prediction’
￿Crutchfield and Young gave
operational definitions of
“maximally accurate prediction”
and “state”
￿The Crutchfield-Young
“statistical complexity”, C
µ, of a
dynamical process is the
Shannon entropy (information
content) of the minimal
sufficient statistic for predicting
the process’s future.
￿Shaliziand Shaliziused this
measure recently to quantify
self-organization in CA
practically
￿They used cyclic CA to assess
their method
Cyclic CA
￿Cyclic cellular automata
(CCA) are simple models of
chemical oscillators.
￿Started from random initial
conditions, they produce
several kinds of spatial
structure, depending on their
control parameters.
￿They were introduced by
David Griffeath, and
extensively studied by Fisch
￿Transition rule

Each site in a square
two-
dimensional lattice is in one
of κcolors.

A cell of colork will change
its colorto k + 1 mod κif
there are already at least T
cells of that colorin its
neighborhood

Otherwise, the cell retains
its current color
Fisch, R. (1990a). "The one-dimensional cyclic cellular automaton: A system
with deterministic dynamics that emulates an interacting particle system with
stochastic dynamics". Journal of Theoretical Probability3(2): 311–338.
Cyclic CA
￿The CCA has three generic
forms of long-term behavior,
depending on the size of the
threshold relative to the range.
￿At high thresholds, the CCA
forms homogeneous blocks of
solid colors, which are
completely static —so-called
fixation behavior.
￿At very low thresholds, the
entire lattice eventually
oscillates periodically;
–sometimes the oscillation takes
the form of large spiral waves
which grow to engulf the entire
lattice.
￿There is an intermediate range
of thresholds where incoherent
travelingwaves form,
propagate for a while, and then
disperse;
–this is called “turbulence”, but
whether it has any connection
to actual fluid turbulence is
unknown.
Spiralling
waves
Turbulent behavior
Spirals engulfing the space for Moore
neighborhood
￿Cyclic CA for a Moore
neighborhood and T=2
￿For Moore neighborhood the
following transitions can be
found:
–T=1: local oscillations
–T=2: spiraling waves
–T=3: turbulence, often
metastablein very long run
(then spirals can take over)
Cellular Automata and beyond
￿Statistical complexity of cyclic CA over time
CosmaRohillaShaliziand Kristina Lisa Shalizi: Quantifying Self-Organization
in Cyclic Cellular Automata, http://arxiv.org/abs/nlin/0507067v1
Cellular automata and beyond
Partial Differential
equations
ContinousContinuousContinuous
DiscreteContinuousContinuous
ContinuousDiscreteContinuous
DiscreteDiscreteContinuous
Coupled Flow
Lattices
ContinuousContinuousDiscrete
DiscreteContinuousDiscrete
Coupled Map
Lattices
ContinuousDiscreteDiscrete
Cellular AutomatonDiscreteDiscreteDiscrete
ModelStateTimeSpace
Summary (1)
￿Cellular automata are defined on a Caleygraph (with
state labels) with a neighbourhoodand transition
rule mapping the state of a center cell to a new state
based on its neighbor states.
￿The number of possible transition rules grows
exponentially with the size of the local state space
and neighborhood
￿Common neighborhood types are von Neumann and
Moore neighborhood, and the k-neighbors in 1-D
arrays (with periodic boundary conditions)
Summary (2)
￿CA are simple models of natural systems
￿Despite their simplicity the behavior of CA can be
extremely complex and difficult to predict
￿CA serve as models for studying emergent
phenomena and self-organization
￿Self organized systems are often at the boundary of
chaotic and ordered states; many open questions
remain, and definitions are not yet clarified
￿An interesting question if the type of global behavior
can be predicted from properties of the rules (e.g.
Langtonslambda)
Summary (3)
￿As simulators CA models are easy to
implemented (also in parallel) and can be
used to model phenomena such as diffusion,
cell systems, flow, pattern formation, etc.
￿CA can be seen as discretecounterparts of
partial differential equations
￿As such they belong to the class of spatio-
temporal models