The Evolutionary Review
Neural Edelmanism and Neural Darwinism
Chrisantha Fernando
For many years
Gerald Edelman
’s theory of Neural Darwinism
confused me
[1]
.
After winning a
Nobel Prize for discovering the
structure
of antibodies,
few
people were in a better position to appreciate that the adaptive immune system
works by a kind of natural selection called somatic selection.
In somatic
selection,
B
-
cells
that produce antibodies that bind a foreign
antigen
mor
e
tightly
can
outcompete
(
replicate faster
than
)
other
B
-
cells that produce antibodies that
don’t bind
a foreign
antigen as well.
Edelman
went on to propose that a similar
process takes place in the brain for
neuronal groups
(neurons conne
cted to each
other by synapses). He
argued
that
like antibodies,
n
euronal g
roups
also
compete with each other,
binding not antigens
as antibodies
d
o,
but
binding
stimuli
.
N
euronal groups that bind stimuli
best can
obtain more
reward
, with
r
e
-
entrant
(reciprocal)
connections between groups allowing
the winning groups
to
remould the losing groups.
But i
s this natural selection
?
Is Edelman’
s theory entitled to its title of
“Neural
Darwinism”?
It depends which great evolutionary
biologist
you ask.
Acco
rding
to one definition given by John Maynard Smith
(JMS)
[2]
,
Edelman’s
neuronal
groups would only be units of evolution if
re
-
entrant connections between
groups really allowed the replication of informa
tion between groups so that the
losing group cam
e to resemble the winning group.
I
have not been able to find
evidence of such a mechanism
in Edelman’
s theory. Edelman has
not
shown how
a neuronal group could really
transmit information to another
neuronal
group
.
But according to a broader definition of natural selection given by
the
theoretical
biologist
George
Price
[3]
, one does not require explicit
multiplication of
neuronal groups, just a redistribution of resources
(competition)
between
groups for
a
process
to be called natural selection
. Michod has pointed out that
Edelman’s neuronal groups are Darwinian entities in this somewhat
boarder
Pric
e
ian
sense
[4]
.
The Evolutionary Review
This paper
describes how
Eörs Szathmáry and I
solved the problem of
how
neuronal groups cou
ld really be units of evolution
in the
JMS
sense
and not just
in the weaker
Priceian
sense
.
The principles involved
point to a fundamental
link
between the origin of life
and
the origin of human
cognition
[5]
.
Our
debugging
of Edelman’s theory
may permit
the
unification of two fields of human
endeavour,
evolut
i
onary biology and neuroscience. Both these fields
aim to
explain open
-
ended adaptation, but
up to now have done
s
o in relative isolation.
Along with
Gerald
Edelman
and William Calvin
[6]
we
propose
that an
underlying process of natural sel
e
ction takes place in the brain. But we differ in
claiming
that
p
opulations of neuronal groups
(replicators)
can
undergo natu
ral
selection as defined by JMS, i.e. with true
replication
of information, and not just
in the broader sense described by Price.
In our formulation, n
euronal replicators
are
patterns
of connecti
vity or
patterns of
activity
in the brain tha
t can make
copies of themselves to nearby brain regions
with generation times of seconds to
minutes
[7,8]
. We believe
neuronal units of evolution
can
undergo natural
selection in the brain
itself
,
to
contribute to
adaptive thought and action
[9,10]
;
populations of
good ideas evolve
overnight.
Their fitness is determined by the
same
Dopamine
-
based
reward
s that have been proposed in other
neural
theories
of reinforcement learning,
including Edelman’s.
Unlike Edelman
and Calvin
however, we propose
several viable mechanisms for replication of neuronal
units of evolution.
But why is replication so important for natural selection?
And
what
makes JMS’s formulation
of natural selection
more powerful than Price’s?
L
ets
examine both d
efinitions of
natural selection
more closely.
Definitions are
never
right or w
rong, only helpful or unhelpful
.
JMS
defined a unit of evolution is
any entity that has the following properties
[2]
.
The firs
t property is
m
ultiplication;
the entity
produces copies of itself
that can make
further
copies
of itself, o
ne
entity
produces two, two
entities
produce
four, four
entities
produce
eight, in a process
known as autocatalytic growth.
Most
living things
are
capable of
autocatalytic growth
, but there are some
exceptions;
for example,
sterile worker ants and
mules do not multiply and so whilst being alive,
they
are
not units of evolution.
The second requirement is v
ariation, i.e. there m
ust
be
The Evolutionary Review
multiple
pos
sible kinds of entity.
Some things are capable of autocatalytic growth
and yet do not vary, for example
fire can grow exponentially for
it
is the
macroscopic
phenomena
arising
from
an
autocatalytic
reaction,
yet
fire does not
accumulate adaptations by natu
ral selection.
The third requirement is that there
must be heredity, i.e. like begets like, so that offspring resemble their parents.
Doron Lancet
proposed that
prior to nucleotides and gene based heredity
,
clumps of lipid molecules called composomes
could
be
capable
of
undergoing
natural selection
[11]
. B
ut
recently we
have shown
that
whilst
composomes
can
multiply
and
possess
variation,
they
do not
have stable heredity, i.e. like
occasionally produces
very much unlike
(the mutation bias being too strong in
certain directions),
and so
unfortunately
such
systems
cannot
after all
evolve by
natural selection
[12]
.
Later we will
see that Edelman’s neuronal groups may fall
into this final category.
If units of evolution of different types have different
probabilities of producing offspring, i.e. if they have differential fitness,
and
if
these probabilities are independent of the fr
equencies of other entities, the
average fitness of the population will be maxi
mised,
and
there will be survival of
the fittest.
George Price
gave a
more general
and more inclusive
definition of natural
selection
[3]
. He said that
a trait (any measurable value)
would
increase in
frequency to the extent that the
probability of that
trait
being present
in
the next
generation
was
positively correlated with
the
trait
itself,
counterbalanced by that
trait’s
variability (the
tendency of that trait to
change
between generations for
any reason, e.g.
due to
mutation)
.
Notice that
JMS’s definition is
algorithmic,
it
tells you
roughly
how to m
ake the natural selection c
ake.
Price’s
definition is
st
atistical,
it tells you whether somethi
ng is
a natural selection cake or not, i.e.
whether
it is the kind of
cake
that
can undergo
the accumulation of
adaptation,
or
survival
of the fittest.
It is important to note that both these definitions were
intended for use in the debate that raged over group selection
[13]
because there
it was essential to formally define what
a legitimate
evolvable
substrate
was
.
Here we use them to understand
neuronal
group selection.
The Evolutionary Review
Lets take some
search
algorithms and see if they satisfy these
two very different
definitions of natural selection.
From a computer science perspective, n
atural
selection
is
a search algorith
m that generates and selects
entities
for solving
a
desired
p
roblem,
if t
he
quality of that
entities’
desired
solution
is
correlated with
the probability of transmission of
that
entity, i.e. with its fitness.
G
enetic
algorithms
work in this way.
They
are computer programs that
implement
natural selection as defined by JMS and Price
[14]
.
Many other algorithms exist
such as hill
-
climbing,
simulated annealing,
t
emporal difference learning, an
d
random search, for findin
g adaptations.
We ask, do these
satisfy either of the
definitions of natural selection?
A c
lassification of
search
algorithms show
s
that
natural selection
as defined by
JMS
really does have some special properties that are
often overlooked because
we take its implementatio
n in the biosphere for granted and because we have
erroneously
come to equate the models of natural selection t
hat evolutionary
biologists use,
with natural selection
itself
.
The table
below shows my
classi
fication. Systems under
g
oing natural selection appear on the right.
Solitary
Search
Parallel Search
Parallel Search with
Competition
(Price)
Parallel Search with
Competition and
Information Transmission
(JMS)
(Stochastic)
hill climbing
Independent
hill
climbers
1.
Competitive Learning
2.
Reinforcement Learning
3.
Synaptic Selectionism
4.
Neural Edelmanism
1.
Genetic Natural Selection
2.
Adaptive Immune System
3.
Genetic Algorithms
4. Didactic receptive fields
5
.
Neuronal Replicators
Table 1.
A classification of search (generate
-
and
-
test) algorithms.
On the
left
hand column
of Table 1
is
shown the
simplest
class
of search
algorithm,
solitary search
.
In solitary search
at most two
c
a
ndidate
unit
s are
maintained at one
time.
An algorithm known as
Hill
-
climbing is an example of a
solitary search algorithm in which
a variant of the unit
(candidate solution)
is
produced and
tested at each ‘generation’.
If
the offspring solution’s
quality
exceeds that of
its
parent, then the offsp
ring
replaces the parent.
If it does not,
then the offspring is destroyed and the parent produces another correlated
The Evolutionary Review
offspring.
Such an a
lgorithm can get stuck on local
optima. Figure 1 shows this
algorithm implemented by a robot on a
n actual
hill
y
landscape
.
The robot carries
a windmill, and its aim is to get to the highest peak.
Lets assume wind
speed
increases with altitude for now.
It moves randomly to a point on a radius a few
meters away, measures the
wind
speed and stays there if this wind
spe
ed is
higher than the previous wind
speed it measured.
If it is not higher, it goes back
to its previous location.
To do this, it must have some memory of the previous
location.
Figure 1.
Imagine a robot on a mountainous landscape whose task it is to
reach
the
highest peak.
One can imagine for example that
it holds a windmill which it
wishes to rotate at the highest speed possible, and the higher up it is the faster
its windmill will rotate.
If it behaves according to hill
-
climbing it
starts from a
ra
ndom position (1)
moves to a nearby location
(2)
and tests whether that
location is hi
gher than its original location by measuring the speed of its
windmill.
If
the wind speed is faster
, it
remains there, but if the wind speed is not
faster
(shown in the unnumbered circles)
the robot
move
s back to the
previous
The Evolutionary Review
location.
The robot may get stuck on a peak
that
is not the highest peak (a local
optimum
)
.
A
robot
(not shown)
behaving according to stochastic hill
-
climbing
does the same, except tha
t it accepts the new position with a certain probability
even if it is slightly lower than the orig
inal position. By this method stochastic
hill
-
climbing
can sometimes avoid gett
ing stuck on the local optimum, but it can
also oc
casionally lo
se the peak it
is
on because memory is only kept of the
immediately preceding position.
Stochastic hill
-
climbing and simulated annealing are examples
of solitary search
where there is a certain probability of accept
ing a worse quality offspring. This
balances
exploration and exploitation and can
reduce
the chances of
getting
stuck on local optima, however,
the
cost
is
potentially lo
sing
the currently
optimal
peak.
But one can ask, isn’t
solitary search
actually
a kind of
natural selection
according to Price
and according to JMS
? Is it
not
natural selection with a
population size of
two
in which
one individual replaces the othe
r based on which
is the fitter
?
Or can the fact that it can be implemented
without explicit
multiplication by use of pointers and memor
y
, or by a robot moving on a hillside
mean that it is not an example of natural selection
according to JMS
? S
ee Figure 2
which shows
two other implementations of hill
-
climbing, this time not on a
hillside but
in a
system of
physical
discrete registers
that can be in binary states.
The Evolutionary Review
Figure 2.
Two implementations of hill
-
climbers
that are
both
trying to maximize
the number of 0’s in the string.
The hill
-
climber on the left stores a solution, here
represented as a binary string, it modifies the solution
at some position in the
string, and stores this modification. After assessing the new solution and
comparing its quality with the original solution it either keeps the modification,
or erases the modification.
The hill
-
climber on the right
replicates
the
entire
solution to a separate location in memory.
Only the hill
-
climber on the right has
true multiplication as defined by JMS.
The
search dynamics
shown
by
both
machines
in Figure 2
are identical and
would be capable of accumulation of adaptations according to
Price’s
formulation of nat
ural selection
a
s
there
wa
s covariance between a trait and
fitness.
According to JMS’s definition,
the implementation
on the right
involving
the explic
it multiplication (replication) of a unit would constitute natural
The Evolutionary Review
selection but the implement
ation
on the left
using pointers and memory
would
not.
However, notice that this distinction is between implem
entations of the
same algorithm
both of which
are in
distinguishable in terms of
search
performance (although
the system on the left
uses
fewer resources).
What about
the robot on the hillside?
Here the brain of the robot may store merely the path
back to the
previous
position,
and
so
the
implementation of h
illclimbing in that
spatially
embodied case may require no replication of an explicitly stored
entity
(
i.e. a
position representation)
at all.
Therefore, we can say that
the
phenomenon
of hill
-
climbing can be implemented either with or without explicit replication,
and therefore may or may not involve natural selection as defined by JMS.
However,
in all cases, the
phenomenon
of hill
-
climbing must
accord with the
principle
of
natural sel
ection according to
Price in order to accumulate
adaptations.
N
otice that
in
Figure 2
(right)
only two memory slots
at most are available that
can contain a maximum of two candidate solutions at any one time.
A slot is
simply a material organization or
substance that can be reconfigured into the
form of a unit or candidate solution, for example a piece of memory in a
computer or the organic molecules constituting an organism.
What happens if
many more slots are availab
le
?
How should one best use them?
In
terms of
Figure 1, this is equivalent to a
hillside now inhabited by many robots,
rather han
just one.
Now our aim is that at least one robot finds the highest peak.
Notice
that
a slightly different aim would have been
to maximize the total wind collected by
t
he windmills of all the robots.
The simplest algorithm for these robots to follow would be that each one
behaves completely independently of the others and does not communicate with
the others at all. Each of the
m behaves exactly like the robot in Figure 1.
In terms
of the implementations shown in Figure 2,
this multiple robot version of search
(simple parallel search)
could be achieved by simply having mult
iple instances of
the hill
-
climbing machinery, either of
the replicating kind,
or the pointer and
memory kind, it doesn’t matter.
The Evolutionary Review
So, if we have many slots or robots available, it is possible just to let many of the
solitary searches run at the same time, i.e. in parallel.
However,
can you see
this
would be wa
ste
ful whatever the implementation?
If one pair of slots
(or a robot)
became stuck on a local optimum then
there would be no way of reusing
this pair
of
slots
(or the robot)
.
Whereas, i
f being stuck on a local optimum could be
detected, then random rei
niti
alization of the stuck slot pair
would
be a
possibility
(or in the robot example, moving
the stuck
robot randomly
to a new
position)
. Even so, one c
ould expect only a
linear speed up in the time taken to
find a global optimum
(the highest peak)
.
It is
difficult to imagine why anyone
would
want to do something l
ike this given all those slots, and all those robots.
This is
the
parallel search described in the second column of Table 1, and it is not
surprising that not many algorithms fall into this class.
A cleverer way to use the extra slots
would be
to
allow competition between
slots for search resources,
and
by
resources
I mean
the
generate
-
and
-
test step of
producing a variant and assessing its quality.
In the case of robots
a step is
moving a robot t
o a new position and reading the wind
-
speed
there.
Such an
assessment step is often the constraining fact
o
r in time and processing costs.
If
such steps were biased
so that
the
currently higher quality solutions
(robots) did
proportionally more
of the
search,
then
there would be a biased
search by
higher
quality solutions. This is known as competitive
learning because candidate
solutions com
pete with each other for reward and exploration
opportunities.
I
f
the robots
are programmed such that the amount o
f exploration they do is
greater
as the altitude increases, then those at
higher
altitude
s
do more
exploration
and this may allow a faster discovery of the global optimum
.
No
robot com
municates with any other robot.
If robots utilize a common power
supply
then the robots are competing with each other for exploration resources.
This is an example of parallel search with resource competition, shown in
column 3 of Table 1.
It requires no
natur
al selection as defined by JMS, i.e.
it
requires no explicit multiplication of information.
Several algorithms fall into the above category.
Reinforcement learning
algorithms
are
examples of
parallel search with competition
[1
5]
.
Such
The Evolutionary Review
a
lgorithms have been proposed as an explanation for learning in the brain, and
work in the following way. If the
response produced by the
firing of a synapse is
positively correlated with reward
and if this reward strengthens this synapse,
whic
h increases its subsequent
probability of firing
, then the conditions of
Price’s definition of natural selection have been fulfilled. This is because there is
a positive correlation between the trait (
i.e. the
response produced by firing
the
synapse) and t
he
subsequent
probability of that response occurring again.
Similarly, a negative correlation between synaptic firing and reward reduces the
subsequent probability of firing.
Sebastian Seun
g calls
these
h
edon
istic synapses
[16]
.
A single
hedonistic
synapse is equivalent to a single allele in genetic terms.
If there is an array of such synapses
emanating from the same neuron and there
is competition for chemical resources from the cell body of the neuron, then
these synapses are equivalent to multiple genetic alleles
competing
for resou
rces
from the cell body
, and the situation is
almost
mathemat
ically
equivalent
to
the
Nobel Prize winner
Manfr
ed Eigan’s replicator equations
[17]
in which the total
population size
of replicators
is kept fixed and there is a
well defined
number of
possible
distinct variants
[9]
.
Eigen’s equations are a popular model used by
evolutionary biologist
s to model evolution.
Notice that there is a subtle difference between the competition between
synapses described above and robot example given for parallel search with
competition.
To use the robot analogy, e
ach synapse is like a robot
stuck in
particular location on the hillsi
de
, unable to move.
Those with higher
wind
speeds are allowed to build
larger windmills.
In this simple kind of synaptic
selectionism the system only exploits the variation that exists at the beginning.
It is not even as
powerful as the case we first considered as parallel search with
competition in which the robots with the greatest wind
speeds
are able to do
more exploration because the variation is limited to the variation that was
produced at the very beginning, robots
cannot move up hills, just
increase
(or
decrease) the size of their windmills.
So d
o such systems
of parallel search
with competition
between synaptic slots
really
exhibit natural selection? Not according to the definition of JMS
because
The Evolutionary Review
there is no rep
licator; there is
no
copying
of solutions from
one
slot
to another
slot
, there is
no
information
that
is transmitted between synapses
.
Resources are
simply redistributed between
synapses (i.e.
synapses
are strengthened or
weakened
in the same way that
the
stationary
robots
increase
or
decrease
the
size of their windmills)
.
Traits
(responses)
are not copied between slots.
Instead
,
adaptation arises by
the mechanism proposed by Price
because there is
covari
ance between traits (here reward obtained by a partic
ular synaptic
response
,
or the
amount of wind collected by a
particular
robot
windmill
) and
their probability of
subsequent
activation
determined by changing the
synaptic
weight
,
or changing the
size of a robot windmill
. According to Price if this
covariance is maintained
there is survival of the fittest synapse or
windmill.
Such
a process of synaptic selectionism has been proposed by the neur
oscientist Jean
-
Piere Changeux
[18]
.
A surprising
consequence is that
Eigen’s replicator equatio
ns
can be run without
any system
having to undergo
natural selection as defined by JMS
. That is, they
can model JMS type natural selection without t
here being any real replicators in
implementing them
.
B
ut they do
always
exhibit natural selection
as descri
bed by
Price, and could of
-
course serve as
models
of systems undergoing natural
selection as defined by JMS.
Nothing
needs to multiply
when
Eigen
’s equations
are run
, but
they emulate the
consequences of multiplication
that would occur in
a system
undergoing natural selection according to JMS.
In short,
synaptic selection
algorithms can best be
understood
as competitive
learning
between
synapse
s
(slots)
that satisfy Price
’s
criteria
for
adaptati
on by
natural selection but use
a different recipe to
achieve this to that
proposed by
JMS.
Instead of explicit multiplication of replicators,
i.e. a process where matter
at one site reconfigures matter at another site
(i.e. where traits are explicitly
copied)
, both Hebbian learning and Eigen’s replicator eq
uations model the
effects of multiplication.
The recipe
i
n the case of synapses emanating from a
single
neuron
involves encoding
the information
(trait)
by
the
location
of the
synapse, and
allowing
matter
to
be redistributed
(fitness)
between synapses.
This is a very different recipe
compared to
how JMS pictured natural selection
The Evolutionary Review
working at the genetic, organismal and group levels in the biosphere.
Synapses
compete for
growth
resources, but it is their connections that encode
information.
Thus the synapt
ic se
lectionism of Changeux
[18]
is a sound form of
Darwini
an dynamic
s as defined by Price and Eigen, but is not
the same class of
implementation of
natural sele
c
tion as defined by JMS. In fact,
no modification of
the responses encoded by each
individual synapse
is possible
.
Each
synapse is a
slot that signifies one
fixed
solution and the relativ
e probability of a slot being
active is
modified by competition
.
Notice
,
there is no transmission of inform
ation
be
tween sl
ots,
in fact no
communication
between slots
at all
,
in other words, the
response arising from activating synapse A does not
become
the response arising
from activating synapse B.
In terms of the robot analogy, a robot that is doing
well does not call other robots to join it.
So,
selec
tionism is an example of parallel
search with competition. It is natural selection in the Price sense, but not in the
JMS sense.
It is at this stage
that I think
Edelman
took
Changeux
(and later Seung’s)
ideas
of
natural selection acting at the level of
the synapse
a step
too far
. The
third
Nobel
Prize
winner in our story,
Francis Crick who worked down the corridor to Gerald
Edelman
at the Salk institute
disliked Edelman’s Neural Darwinism so much that
he called it Neural Edelmanism
[19]
. The reason
wa
s
that
Edelman
had
identified
no replicator
s in the brain
,
and so there wa
s
no unit of evolution
as required by
JMS.
However,
Edelman had sati
sfied the definitions of
natural selection defined
by
Price and Eigen.
Edelman
proposed
c
ompetition between neuronal groups
(
a
neuronal group is Edelman’s implementation of a slot
)
for
synaptic resources,
but he failed
to explain how the particular pattern of synaptic weights that
constitute
the function of
one group c
ould
be copied
from
one group to another.
This leaves no mechanism by which
a
synaptic
-
pattern
-
dependent trait
could
be
inherited between neuronal groups.
In the best paper to formulate Edelman’
s theory of neuronal group selection
,
Izhikevich
shows that there is
no mechanism by which
functional
variations in
synaptic connectivity patterns can be inherited
(transmitted)
between neuronal
groups
[20]
.
Edelman does satisfy Price if a neuronal group is doing no more
The Evolutionary Review
than a sing
le synapse in Chan
geux’s theory, i.e. encoding a particular response.
However, it does not satisfy Price if Edelman wishes to claim that the
trait in
question
is a
transmissible
pattern of
synaptic strengths
within a neuronal group
because
Edelman cannot show there is cova
riance between
such a trait
and the
number of groups in which such a trait is found across generations
.
There is no
communication of solutions between group
-
based
slots, no information transfer
as there is no information transfer between synapses.
Edel
man’
s mechanism
appears to be a
mechanism of competitive learning betwee
n neuronal group
slots which only has a Darwinian interpretation according to Price
if a neuronal
group is
nothing more than
a synapse in Changeux’s model
, i.e. with a fixed
response funct
ion, without copying of response functions between groups.
Therefore,
Francis Crick
was right in a sense.
N
eural Edelmanism falls into the
third column of my classification of search algo
r
ithms
as competitive learning,
but
, which
if interpreted as the same theory as Changeux and Seung’s
can
satisfy
Price’s
phenomenological
but
never
JMS
’s
definition of natural selection.
This is
because natural selection as defined by JMS requires informa
tion transmission
between slots, i.
e. multi
plication (replication), and Price’s definition requires
only covariance between a response encoded by a neuronal group and the
probability of a change in frequency of that response
.
This leads us to the
final
column in Table 1.
Here is a radically di
fferent way of
utilizing multiple slots that extends the algorithmic capacity of
the competitive
learning algorithms above.
In this case
I
allow
not only the competition of slots
for
generate and test cycles, but
I
allow
slots to pass
information
(traits/responses)
be
tween each other, see Figure 3.
The Evolutionary Review
Figure
3
shows
the robots on the hillside again but this time, those robots in the
higher altitudes can recruit robots in lower altitudes to come and join them.
This
is equivalent to replication of
robot locations.
The currently best location can be
copied to other slots. There is transmission of information between slots.
Note,
replication is always of information (patterns), i.e. reconfiguration by matter of
The Evolutionary Review
other matter.
This is
one of the reasons
the cyberneticist anthropologist
Gregory
Bateson called evolution a mental process
[21,22]
.
This means that the currently higher quality slots have not only a greater chance
of being var
ied and tested, but that they can
copy their trait
s
to other slots that
do not have such good quality traits. This permits the redistribution of
information between material slots. Notice that the
synaptic slot system did not
have this capability. If one s
ynapse location produced response A, then it was not
possible for other synaptic locations t
o come
to produce response A
, even
if
response A was associated with higher reward.
We will see shortly a real case in
the brain where such copying of response prop
e
rties is possible between slots,
and is therefore clear evidence for natural selection in the brain not just of the
Prician type but
of
the
JMS
type.
Crucially, s
uch a system
of parallel search,
competition
and
information
transmission between slots doe
s satisfy JMS
’
definition of natural selection. The
configuration of a unit of evolution (s
lot) can
reconfigure other material slots. It
also satisfies Price’s definition.
Some
eponymists might wish to say that
this is a
full Darwinian population
.
But it
is better to show that there are some
algorithmic advantages compared to a competitive learning system w
ithout
information transmission
that satisfies only Price’s formulation of natural
selection.
The critical advantage
of JMS’s definition over Price’s
definition
is that multiple
search points can be recruited to the region of the search space that is currently
the best.
In terms of evolutionary theory, a solution can r
each fixation and then
utilize all the search resources available for further explorat
ion.
This allows the
entire population
(of robots)
to acquire the response characteristics
(locations)
of the
currently
best unit
(robot)
, and therefore, allows the accumulation of
adaptations.
Once one peak has been reached by
all the robots, they can the
n all
be in a position to do further exploration to find even higher peaks. Adaptations
can accumulate.
In many real world problems there is never a global optimum,
rather further mountain ranges remain to be explored after a plateau has been
The Evolutionary Review
reached.
For
example, there is no end to science.
Not
every system
that satisfies
Price’s definition of natural selection
can have these special properties.
It may come as a surpr
ise that there are already well
-
recognised processes in the
brain that
are known to
a
limited
extent to implement natural select
ion as
defined by JMS
(and Price)
, and that have the same
algorithmic characteristics
as
Figure
3
.
Figure
4
shows a recent experiment by
Young et al that
showed that
receptive fields in the primary visual cortex o
f cats
can
replicate
to adjacent
neurons, a process they called “didactic transfer”.
Figure
4
. Adapted from Young, the
orientation selectivity
of simple cells in the
visual cortex can be copied between cells.
Simple cells in the visual cortex
have
orientation selectivity which means they respond optimally to bars presented to
the visual field of a particular orientation.
The arrows in each cell show the
direction of a bar
that maximally stimulates that cell.
The orient
ation selectivity
can be
copied between cells. In fact, t
he fitness of an orientation
selective
response
is the extent to
which
stimulation at
the
retina activates the cell with
The Evolutionary Review
such a response
.
If the retina supplying the cells in the inner circle is
cut out
then those cells rec
eive no inputs, and the increase
their s
ensitivity to activation
by horizontal connections
the adjacent cells.
This process can copy the
orientiation selectivity of adjacent cells
onto the central cells.
If a region of the retina is removed then
the cor
tical neurons that normally
receive input from that region are less active and so they become more sensitive
to being activated by their adjacent active neighbours.
With a special type of
plasticity called spike
-
time
-
dependent plasticity
(STDP)
, nearly all
the neurons
in the silenced region take on the orientation selecti
vity of a neuron adjacent to
that
region.
In this case the trait is
the orientation selectivity, and the fitness is
the change in the proportion of cells with that orientation selectivity.
The unit of
evolution is the receptive field
, which multiplies, and has hereditary variation.
However,
an important term in Price’s
formulation
is the bias due to
transmission
, e.g. mutation
(the rate of which is too large in the GARD model),
but this includes any other factor that alters the trait or the fitness of the trait.
It
seems that the
capacity of
STDP in horizontal connections to continue to
make
copy after copy with
sufficient f
idelity may be low in this case, alt
hough this has
yet
to be tested, but if it is so then
covariance between fitness and orientation
selectivity cannot b
e
maintained across many neurons.
Another limitation of Young’s system of didactic
receptive fields
is that they are
capable of only
limited
heredity, which
means that all possible
orientations could
be
exhaustively
encoded.
The situation is analogous to
pre
-
genetic inheritance in
the origin of life.
Prior to the origin of nucleotides and template replication, e.g.
DNA and RNA replication,
natural select
ion
may have utilized attractor
-
based
heredity in the form of autocatalytic chemical reaction networks
[23]
.
These
were capable of only limited information transmission.
However, the origin of
symbolic information in the form of strings of nucleotides permitted unlimited
heredity. To see thi
s, imagine how long it would take to generate
all possible
strings of DNA of only 100 nucleotides in
length. There are 4 nucleotides, A, C, G,
and T, so
this gives 4
100
possibilities
.
If each
string
could be made in one
second
it
The Evolutionary Review
would still take 5 x 10
46
years to make th
em all. The universe is only 433 x 10
15
seconds old according to Wikipedia.
The capacity to encode
symbolic (digital)
information allows
a far greater number of states and therefore strategies and
responses to be encoded.
This brings us
back then to
what
Edelman
failed to explain in his
neuronal
group
selection.
Is there a way in which
a neuronal pattern could transmit an unlimited
amount of informatio
n to another neuronal pattern? The
neuronal replicator
hypothesis
Eors
Szathmary
and I
proposed in 2008 claims
that the origin of
human
language and unlimited open
-
ended thought
and problem solving
in
humans
arose
because of a major transition in evolution that
bears
a
resemblance to the origin of nucleotides in the origin of life and the o
rigin
of the
adaptive immune system.
We propose that t
he capacity
of
the brain to evolve
unlimited heredity neuronal replicators by
neuronal
natural selection allows
truly
open
-
ended
creative
thought.
Penn and Povinelli have given a politically
incorrect
b
ut
convincing argument that human cognition is indeed qualitatively
distinct from all other animals in that we can reason about unobserved hidden
causes, and the abstract relations between such entities, whereas no other
animal can
[24]
.
We propose that t
his
cognitive sophistication
involved the
evolution at the genetic level of the neuronal capacity for not only competitive
learning
and Price
ian evolution
as described by Changuex,
Edelman, and
Seung,
but for
information transmission between
higher
-
order units of neuronal
evolution, and thus natural selection as described by JMS.
W
e proposed
a plausible
neuronal basis for
the replication of
higher or
der units
of neuronal evolution above the synaptic level
(ne
uronal groups)
.
The method
allows a
pattern of synaptic connections
to
be copied fr
om one such unit
to
another
as
shown in Figure
5
[7]
.
The Evolutionary Review
Figure 5. Our proposed mechanism for copying patterns of synaptic connection
s
between neuronal groups. The pattern of connectivity from the lower layer is
copied to the upper layer.
See text.
The Evolutionary Review
In the brain there are many topographic maps. These are pathways of parallel
connections that preserve
adjacency relationships and they ca
n act to establish a
one
-
to
-
one (or at least a few
-
to
-
few) transformation between
neurons in
distinct
regions of the brain.
In addition there is
a
kind of synaptic plasticity called spike
-
time
-
dependent plasticity
(STDP)
, the same kind of plasticity that Y
oung used to
explain the copying of receptive fields.
It works rather like Hebbian learning.
Donald Hebb said that neurons that fire together wire together, which means
that the synapse connecting neuron A to neuron B gets stronger if A and B fire at
the s
ame time
[25]
.
However, recently
it has been discovered that there is an
asymmetric
form of Hebbian learning (STDP) where if the pre
-
synaptic neuron A
fires before the post
-
synaptic neuron B,
the synapse is strengthened, but if pre
-
synaptic neuron A fires af
ter post
-
synaptic neuron B then the synapse is
weakened.
Thus STDP in an unsupervised manner
, i.e. without an explicit
external teacher,
reinforces potential causal relationships. It
is able to guess
which synapses were causally implicated in a pattern of activation.
If a neuronal circuit exists in layer A in Figure 7, and is externally stimulated
randomly
to make it’s neurons spike
,
then due to the topographic map from layer
A to
layer B, neurons in layer B will experience
similar
spike pattern statistics
as
in layer A (due to the topographic map).
I
f there is STDP in layer B between
weakly connected neurons
then
this layer becomes a kind of causal inference
machine that observes t
he spike
input from layer A and tries to produce a circuit
with the same
connectivity, or at least that is capable of
generating
th
e same
pattern of correlations.
One problem with this mechanism is that there are
many
possible patterns of connectivity
that
gen
erate the same spike statistics when a
circuit is randomly externally stimulated to spike.
As the circuit
size gets larger
,
due
to the many possible paths that activity can take through a circuit
within a
layer, the number of possible equivalent circui
ts grows. This can be prevented by
limiting the amount of horizontal spread of activity permissible within a layer. If
this is done,
and some simple error correction neurons were added,
we found it
was possible to evolve
a
fairly large network to obtain a particular desired
pattern of connectivity.
The network with connectivity closest to the desired
connectivity was allowed to replicate itself to other circuits.
The Evolutionary Review
At the moment,
neurophysiologists
would
struggle to
observe
the connectivity
patterns in
microcircuits of this size and to
undertake a similar experiment in
slices or neuronal
cultures;
however, I
think t
he
day is not far when
it becomes
possible
to identify the mechanisms we propose.
These are not the only kinds o
f
neurona
l replication that are possible
. But this is the closest to turning Edelman’s
theory into something that is truly Darwinian
as defined by John Maynard
Smith.
Acknowledgements:
Thanks to Eors Szathmary, Phil Husbands, Simon
McGregor, and Yasha H
artberg for discussions about the manuscript.
1. Edelman GM (1987) Neural Darwinism. The Theory of Neuronal Group
Selection New York: Basic Books.
2. Maynard Smith J (1986) The problems of biology. Oxford, UK
: Oxford University Press.
3. Price GR (1970) Selection and covariance. Nature 227: 520
-
521.
4. Michod RE (1988) Darwinian Selection in the Brain. Evolution 43: 694
-
696.
5. Maynard Smith J, Szathmáry E (1995) The Major Transitions in Evolution.
Oxford: Oxford University Press.
6. C
alvin WH (1996) The cerebral code. Cambridge, MA.: MIT Press.
7. Fernando C, Karishma KK, Szathmáry E (2008) Copying and Evolution of
Neuronal Topology. PLoS ONE 3: e3775.
8. Fernando C, Goldstein R, Szathmáry E (2010) The Neuronal Replicator
Hypothesis. N
eural Computation 22: 2809
–
2857.
9. Fernando C, Szathmáry E (2009) Chemical, neuronal and linguistic replicators.
In: Pigliucci M, Müller G, editors. Towards an Extended Evolutionary
Synthesis Cambridge, Ma.: MIT Press. pp. 209
-
249.
10. Fernando C, Szathmá
ry E (2009) Natural selection in the brain. In: Glatzeder
B, Goel V, von Müller A, editors. Toward a Theory of Thinking. Berlin.:
Springer. pp. 291
-
340.
11. Segrè D, Lancet D, Kedem O, Pilpel Y (1998) Graded Autocatalysis Replication
Domain (GARD): kinetic
analysis of self
-
replication in mutually catalytic
sets. Origins Life Evol Biosphere 28: 501
-
514.
12.
Vasas V
,
Szathmáry E
,
Santos M
(
2010
)
Lack of evolvability in self
-
sustaining autocatalytic networks constraints metabolism
-
first
scenarios for the origi
n of life.
.
Proc Natl Acad Sci U S A
.
13. Okasha S (2006) Evolution and the levels of selection. Oxford: Oxford
University Press.
14. Holland JH (1975) Adaptation in Natural and Artificial Systems. Ann Arbor:
University of Michigan Press.
15. Sutton SR, B
arto AG (1998) Reinforcement Learning: An Introduction.
Cambridge, MA: MIT Press.
The Evolutionary Review
16. Seung SH (2003) Learning in Spiking Neural Networks by Reinforcement of
Stochastic Synaptic Transmission. Neuron 40: 1063
-
1973.
17. Eigen M (1971) Selforganization of mat
ter and the evolution of biological
macromolecules. Naturwissenschaften 58: 465
-
523.
18. Changeux JP (1985) Neuronal Man: The Biology of Mind: Princeton
University Press.
19. Crick FHC (1989) Neuronal Edelmanism. Trends Neurosci 12: 240
-
248.
20. Izhikevich
EM, Gally JA, Edelman GM (2004) Spike
-
timing dynamics of
neuronal groups. Cereb Cortex 14: 933
-
944.
21. Bateson G (1979) Mind and Nature: A Necessary Unity: Bantam Books.
22. Bateson G (1972) Steps to an Ecology of Mind: Collected Essays in
Anthropology,
Psychiatry, Evolution, and Epistemology: University Of
Chicago Press.
23. Szathmáry E (2006) The origin of replicators and reproducers. Philos Trans R
Soc London B Biol Sci 361: 1761
-
1776.
24. Penn DC, Holyoak KJ, Povinelli DJ (2008) Darwin's Mistake: Expl
aining the
Discontinuity Between Human and Nonhuman Minds. Behavioral and
Brain Sciences 31: 109
-
130.
25. Hebb DO (1949) The Organization of Behaviour: John Wiley & Sons.
Enter the password to open this PDF file:
File name:
-
File size:
-
Title:
-
Author:
-
Subject:
-
Keywords:
-
Creation Date:
-
Modification Date:
-
Creator:
-
PDF Producer:
-
PDF Version:
-
Page Count:
-
Preparing document for printing…
0%
Comments 0
Log in to post a comment