PowerPoint Presentation - Evolutionary Robotics - Psychology

oregontrimmingAI and Robotics

Nov 2, 2013 (4 years and 5 days ago)

87 views

Evolutionary Robotics



Why Robotics, Artificial Intelligence, or
any Engineering
-
based approach?

Psychobiology is in the business of reverse
-
engineering cognitive systems


Top
-
down vs. Bottom
-
up approaches

Emergentism raises problems for
reductionism


Constructionism

Encourages specificity

Epistemological Barriers

Braitenberg’s Vehicles

Why Evolutionary?

Functional Decomposition

Optimization vs. “Good
-
Enough”


Exaptation (Evolution is a miser)

Natural Selection provides a weakly
constraining fitness function

Architectures

Representationalism


Cognitive Systems deal rules operating over
symbols, specifically propositions, e.g. “the
black dog”

Knowledge
-
based Systems


One module contains a warehouse of
knowledge, stored propositionally


Another module queries and searches the
knowledge base for relevant material

Architectures

Production Systems (ACT
-
r)


Operate via condition
-
action rules


If (predator), then (fear activation)


If (fear activation), then (run)


If (run), then (increase heart rate)


Multiple modules for domain specific tasks


Working memory buffer


Perception module


Action module


Global behavior emerges from interactions of
components


Symbolic systems are very powerful

Architectures

Behavior
-
based Robotics


Global behavior of robot emerges through the
interaction between basic behaviors and the
environment


Basis behaviors implemented in separate parts
of the control system and a coordination
mechanism determines relative strength of each
behavior in the particular situation

Problems

Production Systems and Behavior Based Robotics
are not wrong, just incomplete


Behaviorism capitalized on stimulus
-
response
associations

Need an architecture sensitive to the structural and
statistical properties of its inputs

Need an architecture that has the capacity for self
-
organization (removing the burden of explicit
design from the experimenter)

Problems

Often, behavior is built into the system
rather than emerging from the system, thus
limiting its explanatory value


Working memory buffer is constrained to only
hold 7+
-
2 items

What is the propositional representation of a
picture?

Functional Decomposition

Neural Networks

McCulloch and Pitts
-

1943


We know that the neuron is the fundamental
computational unit in the brain


We know that cognitive abilities arise from
collections of neurons acting together (lesions)


How do collections of interconnected neurons
produce coherent behavior?

Computer simulations of neurons and
collections of neurons may tell us

Logical Operators



AND:


Sally went to the beach and drank a coke.

T=2

+1

+1

OR:


Sally will either live or die.

T=1

+1

+1

Properties of Neural Networks

Associative


Networks are inherently associative. Activity
in one node spreads to neighboring nodes,
activating their respective representations.


Because of this associativity, and the properties
of distributed representations, similar
representations cluster.


Example: Hypercolumns & Retinotopy




Pattern Completion


Networks can complete known patterns on the basis of
partial information. If several units from a particular
well
-
known pattern are activated, but a few are not, the
activation reverberates through the network, causing the
missing information to be completed in the manner
most consistent with the stimulus information given.


This property also allows them to compensate for noisy
input


. Graceful Degredation


If you destroy a piece of the CPU of a computer, it will
crash. Or, if you delete a few lines of code from a program, it
will also crash.


Brains are not like this.


Neurons die all the time, don’t radically disrupt functioning.


Lesions cause focal deficits

Because any given neuron or piece of cortex is only one of
many players in a representation of cognitive function,
damage has limited effects. Deficits increase with increased
damage, but its *not catastrophic*


Learning


1949 Donald Hebb


Networks can learn by altering the synaptic
strength, or the weights of connections between
units.


However, prior to the 80s, connection weights
had to be hand
-
set by trial
-
and
-
error to get the
network to perform a task.


The trick is to devise a rule that specifies how
to adjust the weights as a function of past
performance so that improvement takes place.


Supervised Learning

A “teacher” provides feedback to a network on its
performance.

The teacher may be a set of nodes with the correct
output for a given problem. A network then tries
to reach that output given a set of inputs.

An error signal is computed which calculates the
difference between the actual output (teacher) and
the arrived at solution (learner).

Aspects of motor system do this: actual vs.
intended output

Backpropogation

An algorithm which assigns “blame” to
nodes for the amount of error.

Determines which connection weights
contributed most to the error and sdjusts
them.

Process iterated until no or minimal error
remains.


Reinforcement Learning

Network (learner) is not told the correct
output, only whether the arrived at solution
is good or bad.

Example: The dopaminergic, reward
system.


Unsupervised Learning

Self
-
Organization

No teacher or reinforcement.

The local, causal dynamics of the network
shape its behavior.

Hebbian Learning


The Power of Learning in NNs

In production systems and engineering, a
problem is solved in advance, then
implemented.

Neural networks only receive inputs and
desired outputs, and finds solutions on its
own.

Often, the derived solution is very
unintuitive.

Emergenesis in a Neural Network

x

x

cue

target

Validly
-
Cued Trial

Invalidly
-
Cued Trial

neutral

valid

invalid

RT

Interrupt

Localize

Alert

Disengage

Move

Engage

Inhibit

Response Unit

Object

Unit

Input Unit

Spatial Units

Khepera Robot

Genetic Algorithms

Operates on a population of artificial
chromosomes by selectively reproducing
chromosomes of individuals with higher
performance and applying random changes

Applied for many generations until fitness
function stops increasing, or a satisfactory
individual is found

Artificial Chromosome

An artificial chromosome is a string that encodes the
characteristics of an individual

String may encode the value of a variable of a function that
must be optimized, may encode connection weights of a
neural network, or network architecture with learning rules
for network development, etc.

Most of the subsequent experiments encode synaptic
weights, so that in essence, multiple networks are explored

How and what to encode in the chromosome is the subject
of intense research

Fitness Function

A performance criterion that evaluates the performance of
each individual phenotype. Higher is better.

Examples: object location, the closest robots to an object
are selected; maze navigation, those successfully
navigating maze fastest are selected

Choice of fitness function has consequence for artificial
evolution


The more detailed and constrained a fitness function is,
the closer artificial evolution becomes to a supervised
learning technique and less is left to emergence and
autonomy of the evolving system

Selective Reproduction

Copies are made of the best individuals in the
population


the probability of a given individual being reproduced
equals its fitness divided by the sum fitness of the
population


Tournament Selection


Elitism

Copies are are then subjected to crossover with a
random partner of the same generation, and
mutation

Evolution of Simple Navigation

Fitness function has three components to be maximized: Speed, Straightness,
Obstacle Avoidance

Robot reaches peak speed of 48 mm/s on straight
-
aways
(max. speed is 80 mm/s)

In terms of fitness function, there is no advantage for
robots that move forward or backward. All robots move in
direction of side with more sensors (front) thus
maximizing information to deal with upcoming walls

Both of these emerge from robot
-
environment interaction

When compared to hand
-
coded robots, evolved robots
performed better or comparable (as measured by fitness
function)

When the selection criterion changes (either a
change in environment or fitness function), some
individuals that previously were not among the
best may be selected for reproduction and pull the
population toward a new area of genetic space

Thus, evolving systems are continuously adaptive

Adaptation as displacement of a partially
converged population in genetic space (Harvey
1992, 1993)

Reactive Intelligence

Sensors and motors are directly linked

Agents react to the same sensory state with
the same motor action

Active Perception

What about cases where a robot must react
differently to similar looking sensory
patterns? (Perceptual aliasing problem)

Overcoming Perceptual Aliasing

Agents partially determine the sensory
patterns they receive from the environment
by executing actions that modify the
position of the agent with respect to the
external environment or by altering the
environment itself

Power and Limits of Reactive
Agents

Robots exploit the dimensions of the environment

Performance decreases with increasingly radical
changes of environment dimensions

Reactive agents are able to exploit sensory
-
motor
interactions and the environment to solve complex
tasks and disambiguate similar percepts

However, the agents limits are reached as the
environment becomes increasingly variable and
indeterministic

Modularity

Modularity is an integral part of traditional
functional decomposition approaches


Specific behaviors allocated to different modules

No mandate for modularity in evolutionary
systems

Is modularity beneficial for some tasks?

Assuming we evolve modular controllers but let
the evolutionary process determine the
functionality of each module, will architectural
modules correspond to basic behaviors?

Modules do aid performance

They do not map onto easily discernible functions
from a distal perspective (outside looking in)

Subsequent analysis shows that from a proximate
perspective (from within), modules aid in
producing different motor behaviors to similar
sensory states

Modules then are a straight
-
forward extension of
the behavior of purely reactive agents

Hidden Layers

Allow a re
-
representation of the input layer

These re
-
representations may combine
inputs from previous layer, like battery level
and floor brightness, allowing behaviors to
be based upon these new higher
-
level
representations

Spontaneous emergence of internal
representations (crude topography map)

Lessons

Increasing internal dynamics, e.g. modules or
hidden layers, allows increased behavioral
flexibility

Behavioral flexibility allows for increased
robustness in the face of environmental changes

Environmental Generalization vs. Environmental
Independence


Abstraction results from overlapping domain
representations at higher levels

Evolution and Learning

Pro:


Learning allows individuals to adapt to changes
in the environment that occur in the lifespan of
an individual


It can help and guide evolution

Con:


Entails a delay in the ability to acquire fitness


Increased unreliability


Perhaps delayed reproduction

Lamarkian Evolution

Baldwin effect


Evolution tends to select individuals who have
already at birth those useful features which
would otherwise be learned


Indirect genetic assimilation, canalization



Evolution can select for a predisposition to
learn in a given domain. This
predisposition may consist of:


The presence of starting conditions at birth, e.g.
a particular architecture suitable for learning a
certain task


An inherited tendency to behave in such a way
that the individual is exposed to the
appropriately learning experiences

Competitive Co
-
evolution

Co
-
evolution of competing populations (e.g.
predator and prey) may produce increasingly
complex evolving challenges

May reciprocally drive one another to increasing
levels of behavioral complexity by producing an
evolutionary arms race

May also result in cycling


Co
-
evolving populations may cycle between alternative
classes of strategies that do not represent progress in the
long run, but are temporarily effective against the co
-
evolving population



Co
-
evolution will result in increased
behavioral complexity only if a general
enough solution is found that is effective in
a variety of environmental circumstances in
order to avoid cycling

This solution must


Exist


Be accessible to the agent on the genetic
landscape

Conclusions

Agents are embodied

Agents are situated within an environment

Agents often settle on solutions that are
unintuitive, raising doubts about the efficacy of
functional decomposition

As the field develops, more interesting results will
arise, for example, when more realistic
implementations of genetic code are discovered

Thank You!

Happy Thanksgiving