PowerPoint Presentation - Evolutionary Robotics - Psychology

oregontrimmingAI and Robotics

Nov 2, 2013 (5 years and 3 months ago)


Evolutionary Robotics

Why Robotics, Artificial Intelligence, or
any Engineering
based approach?

Psychobiology is in the business of reverse
engineering cognitive systems

down vs. Bottom
up approaches

Emergentism raises problems for


Encourages specificity

Epistemological Barriers

Braitenberg’s Vehicles

Why Evolutionary?

Functional Decomposition

Optimization vs. “Good

Exaptation (Evolution is a miser)

Natural Selection provides a weakly
constraining fitness function



Cognitive Systems deal rules operating over
symbols, specifically propositions, e.g. “the
black dog”

based Systems

One module contains a warehouse of
knowledge, stored propositionally

Another module queries and searches the
knowledge base for relevant material


Production Systems (ACT

Operate via condition
action rules

If (predator), then (fear activation)

If (fear activation), then (run)

If (run), then (increase heart rate)

Multiple modules for domain specific tasks

Working memory buffer

Perception module

Action module

Global behavior emerges from interactions of

Symbolic systems are very powerful


based Robotics

Global behavior of robot emerges through the
interaction between basic behaviors and the

Basis behaviors implemented in separate parts
of the control system and a coordination
mechanism determines relative strength of each
behavior in the particular situation


Production Systems and Behavior Based Robotics
are not wrong, just incomplete

Behaviorism capitalized on stimulus

Need an architecture sensitive to the structural and
statistical properties of its inputs

Need an architecture that has the capacity for self
organization (removing the burden of explicit
design from the experimenter)


Often, behavior is built into the system
rather than emerging from the system, thus
limiting its explanatory value

Working memory buffer is constrained to only
hold 7+
2 items

What is the propositional representation of a

Functional Decomposition

Neural Networks

McCulloch and Pitts


We know that the neuron is the fundamental
computational unit in the brain

We know that cognitive abilities arise from
collections of neurons acting together (lesions)

How do collections of interconnected neurons
produce coherent behavior?

Computer simulations of neurons and
collections of neurons may tell us

Logical Operators


Sally went to the beach and drank a coke.





Sally will either live or die.




Properties of Neural Networks


Networks are inherently associative. Activity
in one node spreads to neighboring nodes,
activating their respective representations.

Because of this associativity, and the properties
of distributed representations, similar
representations cluster.

Example: Hypercolumns & Retinotopy

Pattern Completion

Networks can complete known patterns on the basis of
partial information. If several units from a particular
known pattern are activated, but a few are not, the
activation reverberates through the network, causing the
missing information to be completed in the manner
most consistent with the stimulus information given.

This property also allows them to compensate for noisy

. Graceful Degredation

If you destroy a piece of the CPU of a computer, it will
crash. Or, if you delete a few lines of code from a program, it
will also crash.

Brains are not like this.

Neurons die all the time, don’t radically disrupt functioning.

Lesions cause focal deficits

Because any given neuron or piece of cortex is only one of
many players in a representation of cognitive function,
damage has limited effects. Deficits increase with increased
damage, but its *not catastrophic*


1949 Donald Hebb

Networks can learn by altering the synaptic
strength, or the weights of connections between

However, prior to the 80s, connection weights
had to be hand
set by trial
error to get the
network to perform a task.

The trick is to devise a rule that specifies how
to adjust the weights as a function of past
performance so that improvement takes place.

Supervised Learning

A “teacher” provides feedback to a network on its

The teacher may be a set of nodes with the correct
output for a given problem. A network then tries
to reach that output given a set of inputs.

An error signal is computed which calculates the
difference between the actual output (teacher) and
the arrived at solution (learner).

Aspects of motor system do this: actual vs.
intended output


An algorithm which assigns “blame” to
nodes for the amount of error.

Determines which connection weights
contributed most to the error and sdjusts

Process iterated until no or minimal error

Reinforcement Learning

Network (learner) is not told the correct
output, only whether the arrived at solution
is good or bad.

Example: The dopaminergic, reward

Unsupervised Learning


No teacher or reinforcement.

The local, causal dynamics of the network
shape its behavior.

Hebbian Learning

The Power of Learning in NNs

In production systems and engineering, a
problem is solved in advance, then

Neural networks only receive inputs and
desired outputs, and finds solutions on its

Often, the derived solution is very

Emergenesis in a Neural Network





Cued Trial

Cued Trial












Response Unit



Input Unit

Spatial Units

Khepera Robot

Genetic Algorithms

Operates on a population of artificial
chromosomes by selectively reproducing
chromosomes of individuals with higher
performance and applying random changes

Applied for many generations until fitness
function stops increasing, or a satisfactory
individual is found

Artificial Chromosome

An artificial chromosome is a string that encodes the
characteristics of an individual

String may encode the value of a variable of a function that
must be optimized, may encode connection weights of a
neural network, or network architecture with learning rules
for network development, etc.

Most of the subsequent experiments encode synaptic
weights, so that in essence, multiple networks are explored

How and what to encode in the chromosome is the subject
of intense research

Fitness Function

A performance criterion that evaluates the performance of
each individual phenotype. Higher is better.

Examples: object location, the closest robots to an object
are selected; maze navigation, those successfully
navigating maze fastest are selected

Choice of fitness function has consequence for artificial

The more detailed and constrained a fitness function is,
the closer artificial evolution becomes to a supervised
learning technique and less is left to emergence and
autonomy of the evolving system

Selective Reproduction

Copies are made of the best individuals in the

the probability of a given individual being reproduced
equals its fitness divided by the sum fitness of the

Tournament Selection


Copies are are then subjected to crossover with a
random partner of the same generation, and

Evolution of Simple Navigation

Fitness function has three components to be maximized: Speed, Straightness,
Obstacle Avoidance

Robot reaches peak speed of 48 mm/s on straight
(max. speed is 80 mm/s)

In terms of fitness function, there is no advantage for
robots that move forward or backward. All robots move in
direction of side with more sensors (front) thus
maximizing information to deal with upcoming walls

Both of these emerge from robot
environment interaction

When compared to hand
coded robots, evolved robots
performed better or comparable (as measured by fitness

When the selection criterion changes (either a
change in environment or fitness function), some
individuals that previously were not among the
best may be selected for reproduction and pull the
population toward a new area of genetic space

Thus, evolving systems are continuously adaptive

Adaptation as displacement of a partially
converged population in genetic space (Harvey
1992, 1993)

Reactive Intelligence

Sensors and motors are directly linked

Agents react to the same sensory state with
the same motor action

Active Perception

What about cases where a robot must react
differently to similar looking sensory
patterns? (Perceptual aliasing problem)

Overcoming Perceptual Aliasing

Agents partially determine the sensory
patterns they receive from the environment
by executing actions that modify the
position of the agent with respect to the
external environment or by altering the
environment itself

Power and Limits of Reactive

Robots exploit the dimensions of the environment

Performance decreases with increasingly radical
changes of environment dimensions

Reactive agents are able to exploit sensory
interactions and the environment to solve complex
tasks and disambiguate similar percepts

However, the agents limits are reached as the
environment becomes increasingly variable and


Modularity is an integral part of traditional
functional decomposition approaches

Specific behaviors allocated to different modules

No mandate for modularity in evolutionary

Is modularity beneficial for some tasks?

Assuming we evolve modular controllers but let
the evolutionary process determine the
functionality of each module, will architectural
modules correspond to basic behaviors?

Modules do aid performance

They do not map onto easily discernible functions
from a distal perspective (outside looking in)

Subsequent analysis shows that from a proximate
perspective (from within), modules aid in
producing different motor behaviors to similar
sensory states

Modules then are a straight
forward extension of
the behavior of purely reactive agents

Hidden Layers

Allow a re
representation of the input layer

These re
representations may combine
inputs from previous layer, like battery level
and floor brightness, allowing behaviors to
be based upon these new higher

Spontaneous emergence of internal
representations (crude topography map)


Increasing internal dynamics, e.g. modules or
hidden layers, allows increased behavioral

Behavioral flexibility allows for increased
robustness in the face of environmental changes

Environmental Generalization vs. Environmental

Abstraction results from overlapping domain
representations at higher levels

Evolution and Learning


Learning allows individuals to adapt to changes
in the environment that occur in the lifespan of
an individual

It can help and guide evolution


Entails a delay in the ability to acquire fitness

Increased unreliability

Perhaps delayed reproduction

Lamarkian Evolution

Baldwin effect

Evolution tends to select individuals who have
already at birth those useful features which
would otherwise be learned

Indirect genetic assimilation, canalization

Evolution can select for a predisposition to
learn in a given domain. This
predisposition may consist of:

The presence of starting conditions at birth, e.g.
a particular architecture suitable for learning a
certain task

An inherited tendency to behave in such a way
that the individual is exposed to the
appropriately learning experiences

Competitive Co

evolution of competing populations (e.g.
predator and prey) may produce increasingly
complex evolving challenges

May reciprocally drive one another to increasing
levels of behavioral complexity by producing an
evolutionary arms race

May also result in cycling

evolving populations may cycle between alternative
classes of strategies that do not represent progress in the
long run, but are temporarily effective against the co
evolving population

evolution will result in increased
behavioral complexity only if a general
enough solution is found that is effective in
a variety of environmental circumstances in
order to avoid cycling

This solution must


Be accessible to the agent on the genetic


Agents are embodied

Agents are situated within an environment

Agents often settle on solutions that are
unintuitive, raising doubts about the efficacy of
functional decomposition

As the field develops, more interesting results will
arise, for example, when more realistic
implementations of genetic code are discovered

Thank You!

Happy Thanksgiving