Eco-robotics: The Evolutionary Intentional Dynamics of Adaptive Systems

cottonseedbotanistAI and Robotics

Oct 24, 2013 (3 years and 7 months ago)



Workshop on the Challenges and Promises of an Ecological Approach to Robotics

robotics: The Evolutionary Intentional Dynamics

of Adaptive Systems

Robert Shaw

and William Mace

Department of Psychology

University of Connecticut


CT 0626


Department of Psychology

Trinity College

300 Summit Street

Hartford, CT

CT 06106


Version 1.0


The paper has three major aims: Section I provides a brief overview of the state
art i
n robotics.
Recent assessments by experts are briefly discussed. A consensus seems to have developed in the field that the
traditional approach to robotics using GOFAI (good old fashion AI) has failed, so something new needs to be tried.

Section II re
views some of the most promising new techniques that have come to prominence in recent

such as, behavior
based robotics (e.g., Brooks), genetics algorithms, and reconfigurable hardware (e.g.,
Thompson, Higuchi). The technique which many experts beli
eve holds the most promise is evolvable hardware

a wedding of genetic algorithms with reconfigurable hardware. This technique has become central to the
current development of evolutionary robotics (ER), the generic name for the field.

Finally, Sec
tion III shows how ER using EHW can be given a natural interpretation within the methods of
ecological psychology, particularly that branch called intentional dynamics (ID). By combining intentional dynamics
with evolutionary techniques, we have evolution
ary intentional dynamics (EID); and when EID is applied to
robotics, evolutionary ecological robotics is the result.


For convenience, here is a summary of the abbreviations to be found in the paper,


= good old fashion artificial int


= genetic algorithm


= fitness function



= ordinary (fixed) hardware


= reconfigurable hardware


= a field programmed gate array, a type of RHW


= evolutionary hardware


= evolutionary robotics


= intentional dynamics


= evolutionary intentional dynamics


= evolutionary ecological robotics

To anticipate, here is a general sketch of how the different methods to be discussed relate to define EID, namely

EID = EHW + ID = (RHW + EHW) + ID:

NOTE: A system is
able in the formal sense
if a set of equations in closed form describing its behavior can
be given , By contrast, it is
tractable in a practical sense

if either an approximation to its equations can be given
(say by a Monte Carlo routine) or if a scheme i
s given by which the system can be brought into some desired state
in some reasonable amount of time

say, by a GA applied to a population of systems EHW).




I. Current State
Art in Robotics

Humans are by nature inventive artificers, typic
ally remaining undaunted even when faced by the most highly
complicated and subtle engineering problems. Recently, many experts in the robotics community, however, have
concluded that traditional methods offer little hope for building robots that can lear
n to perform practical tasks in
diverse real
world environments. Instead, roboticists seek new methods by which to evolve autonomous agents
that can function effectively in ecologically valid situations.

A situation is
ecologically valid

if it is suffi
ciently rich in resources (e.g., affordances and information) to
allow an evolutionarily attuned, autonomous agent adequate intrinsic means to perform tasks effectively. Hence an
alternative to traditional GOFAI or typical cognitive neuroscience methods
is needed. In later sections, we will
consider some of these alternatives.

Challenges to Traditional Robotics

Practical robotics aims to develop agents capable of performing significant goal
directed tasks in diverse
natural or man
made environments.
It hopes to replace humans in tasks that are too dangerous, too difficult, or too
boring for human agents to perform effectively, as well as to produce agents that can perform tasks impossible or
inconvenient for humans.

Roboticists have proposed a var
iety of real
world tasks, any of which if accomplished would indicate
significant progress in the field. They include:

• Miniature medical robots for endoscopic

inspection and surgical treatment

• Agricultural robots for tilling, harvesting, and pest


• Industrial robots for tool manipulation, toxic waste handling, and distribution of sensitive materials

• Caretaker robots to aid the dependent, treat the sick, and protect the helpless

• Transportation robots to navigate vehicles, direct traffic, r
epair drainage, and maintain roadways

• Architectural robots to help in the construction and maintenance of homes and buildings

• Hazardous duty robots capable of working in places too small or too dangerous for humans to
navigate or manipulate tools.

• Do
mestic help robots to help cook, do housekeeping, make repairs, do gardening, entertain or
teach children.


Progress toward such goals has been disappointing to many contemporary roboticists. For instance, a
group of roboticists at Brandeis University
recently made the following pessimistic assessment:

"The field of Robotics today faces a practical problem: most problems in the physical world are too difficult for the
current state of the art. The difficulties associated with designing, building and c
ontrolling robots have led to a stasis
(Moravec, 1999) and robots in industry are only applied to simple and highly repetitive tasks" (Pollack et al.,
www//paper, p. 1).

The problem is not so much the lack of engineering know
how, nor even the lack of ap
preciation of what
world behaviors demand

but, unfortunately, runs much deeper. For there has been a failure to identify
fundamental principles for explaining how goal
directed behaviors might be programmed into or learned by
machines that are suffic
iently competent to be of practical value. This lack has impeded the development of
autonomous robots in all areas where significant real
world tasks are to be performed.

To be capable of performing real
world tasks, robots must possess complex hardware

to support exquisite
control, and yet, to be practical in that they must operate by principles sufficiently tractable to be generalized over
environmental contexts distinguished by variable task demands, as well as over robots of different design and
lication. Current robots, however, cannot perform any real
world tasks adequately because they fall short in
several fundamental ways.

Major deficiencies in current robotics technology were identified by a blue ribbon committee convened in
April 2002 b
y Information Society Technologies (IST), a European Community (EC) funding agency, whose aim is
to initiate projects addressing problems of Future and Emerging Technologies (FET). Some of the problems
identified were:

(1) Robotic principles must exhibit

scalability over real
world environments.

Systems are needed that exhibit robust operation in spite of changes in task and environment. Ones that
seem to work in simulated or artificially simple environments fail when placed in real environments fraught

with noisy

"None of the artifacts that have been designed this far have demonstrated capability to open ended domains or truly
complex task environments. . . For operation in natural domains it is necessary for these systems to be able to cope
with scalability that are several magnitudes beyond those available today" (the FET Beyond Robotics Work

Likewise, the principles governing the coordination of subsystems responsible for learning, information detection,
and control of action must

be scalable over robots of different functional complexity.

(2) Criteria are needed to evaluate a robot's degree of success.

Although the performance of robots today is nearly always evaluated in application settings, criteria are too
often qualitativ
e, lacking in scientific rigor (IST report 2002). Unfortunately, psychologists have offered little help in
this regard because there is no consensus on how to evaluate human performance on similar tasks over diverse

(3) Robot subsystems requ
ire coordination and integration.

Many subsystems (e.g., sensors, effectors, controllers, electrical circuits, mechanical linkages,
computational algorithms, environmental physics) must be integrated or coordinated in order for a robot to perform
al tasks in real
world situations. Since no one person can be expert in all facets of this problem, a team of
experts from diverse fields is required. The primary difficulty is finding a common ground for such diverse

a general systems theory o
f autonomous agents.


"The need for large interdisciplinary teams or an operational theory makes this a hard problem. So far robots have
almost exclusively been successful in areas where it has been possible to identify the 'ecological niche' for the
em, i.e., where all relevant parameters can be identified or characterised sufficiently well to allow design of
situated system

this is the case of 'insect robots' . . " (IST report, 2002).

This calls for identification of environmental invariants to unde
rwrite robust performance in the present of task and
environmental changes

a clear task for ecological psychology and the special branch of intentional dynamics (See
Section III).

(4) Robots require robust perception of environmental information in chang
ing environments if their task
relevant actions are to be robust as well.

For a robot to be capable of robust performance on a wide variety of tasks in changing environments, it
must also be capable of robust perception to direct those actions. (In Secti
on III, we shall construe this problem of
robust perception and robust action in ecological psychological terms: namely, a robot must be able to detect
information specific to environmental affordances. Likewise, to act upon these affordances, a robot mus
t have
skillful control, or in ecological terms, it must act according to "rules" for the perceptual control of the relevant

(5) Robot learning must extend beyond fixed task domains.

Some of the most interesting work on robotic learning has emer
ged in the last decade, but, perhaps
surprising, not from the popular areas such as artificial intelligence, artificial neural nets, humanoid simulation, or
artificial life. These are all relevant but insufficient because, to be effective, they must restri
ct learning to fixed task
domains (FET, 2002). In the next section, we consider three recent methods that have shown promise.

II. The Promise of a New Robotics

Traditional robots acquire their designs by human intervention (e.g., programming); they
reach a degree of
success only when they are designed to operate in closed task domains; but when placed in new task domains, their
programs fail to generalize. If traditional methods fail, is there an alternative approach by which robots might
Perhaps, if they can design themselves, then human intervention could be avoided.

Fortunately, there is a continuum of ways in which robots might have their designs evolved in lieu of human
intervention. At one end are techniques where evolutionary algo
rithms are used to design simulated versions of
robots performing simulated tasks in simulated environments; at the other end are populations of real robots whose
hardware architectures evolve new configurations from repeated interactions with real environ
ments, under the
guidance of an fitness function which determines the 'selection pressure' for a given task function. Intermediate
methods exist by which a robot's design is evolved by simulating a genetic algorithm for a simulated environment
and then by

placing the robot in the corresponding real environment for training in the specific exigencies of the

characteristics too subtle to be explicitly described and too complicated to be handled by programming.

A major goal that remains elusive is how t
o make the evolutionary process free of human intervention, such
as supervised learning (connectionism), selection criteria (evolutionary grammars), fitness functions (genetic
algorithms), rejection criteria (Monte Carlo), or any other source of constraint
s on learning or evolution not
intrinsically derived from interaction with the environmental situation itself.

"Evolvable hardware (EHW) refers to hardware that modifies its architecture and behaviour dramatically and
autonomously by interacting with th
e environment. At present, almost all EHW uses an evolutionary algorithm (EA)
as their main adaptive mechanism. One of the key motivations behind EHW is to learn from Nature since she has
done so well in evolving wonders such as ourselves (i.e., human be
ings) without external forces" (Yao and Kiguchi,

The techniques of EHW have the following virtues:


(1) Evolutionary design approach, it seems, can explore a much wider range of design alternatives for robots
than those that can be programmed by h

(2) Evolutionary design does not assume a priori knowledge of any particular design domain.

(3) Evolutionary design can work with varying degrees of constraints and special requirements, if necessary, by
incorporating them in "chromosome" represen
tations and fitness functions.

Weirdness in Evolutionary Circuits

Much of the power of EHW arises from several unusual properties of the FPGA's used:

Mysterious Couplings
. The functional parts of the circuit consists of those logic modules which beco
connected (hardwired) and, surprisingly, also of some of the modules that are not even connected to the other
modules (i.e., are isolates). Thompson (1998) reports that if some of these isolated modules are clamped, the
performance of the circuit does
not noticeably suffer. On the other hand, some isolated modules if clamped do
cause the performance of the circuit to be noticeably degraded. How might this happen?

No one knows for sure but several hypotheses have been offered: The isolated modules m
ay be coupled to
the rest of the circuit magnetically, or may be influencing it through the shared power supply wiring.

Digital or Analog Circuit?
Although the final input/output behavior of an evolved FPGA is, by design,
digital, at intermediate stages

of evolution complex analog waveforms may be detected at the output. This suggests
that the internal processes of the chip may be exploiting a rich continuous time, continuous value dynamics

behavior that cannot be controlled from the outside by progr

specific Adaptation
. A practical difficulty is encountered with these chips. Their free dynamics allow
them to explore and exploit the subtle physics of the device or its context (e.g., occupying different positions on a
thermal gradient).

Thus chips designed to do the same task but which evolved that competence indifferent
situations, may achieve specific adaptation to the details of their distinct real
world context. Hence they may cease
to function properly when moved to another contex
t in which these details vary. Fortunately this fault can be

"Evolving circuits will potentially come to depend upon

properties that are sufficiently stable during evolution for
at least the number of generations it takes to exploit them"

(Thompson, 1998, p. 87).

Notice, this statement applies equally well to different devices evolving to do the same task in the same
environment. The evolving circuits will likewise come to depend so much on the most stable properties of each
distinct dev
ice, while adapting to the same task situation, that their circuit designs will be quite dissimilar. Hence
they perform the same task in the same environment but do so differently in ways unique to the physical properties
of their bodies.

Thompson points

out that this statement of the problem suggests a solution: If we subject the ". . . evolving circuits
to the range of conditions in which they will be required to operate . . . The intrinsic evaluation of an individual
circuit's fitness will be the measu
rement of its ability to perform under

of these conditions, with the circuit being
instantiated on all the chips" [where on chip is adapted to each condition] (Thompson, 1998, p. 87).

The circuits for different devices, evolving a given task compete
nce in distinct contexts (whether the contexts in
question are distinct environments or distinct robot bodies) will exhibit a competence specific to the given context
(situation, body). Hence competent performance on the given task may show degradation if

generalized over either
situations or devices.
The most general solution to the autonomous agent problem dictates that the chips
designed for a given range of tasks be exposed to the widest possible range of stable physical properties
characteristic of
environments in which the agent must operate and of the robot bodies with which they
must operate.


In the next section, we shall interpret the three properties for EER, and use the same argument to generalize
over situations, tasks, and robot embodiments
in order to obtain an autonomous agent with the most general

Section III. Is an Eco
robotics Theory Possible?

As argued earlier, it is generally recognized that the most serious impediment to progress in robotics is how
to design autonomous

agents that, not only can perform given tasks competently in specific environments but, more
importantly, can learn or develop a broader competence for effective performance on many tasks across diverse
world environments. And, as argued above, beca
use each robot body will differ slightly in structural design
and functional detail, the exact duplication of evolutionary contexts is impossible. Ecological robotics, or, more
simply, eco
robotics (EER), is suggested as the name of the hybrid field which

brings ER together with EP. Indeed,
an effort has already begun in this regard (See Efffken & Shaw, 199?).

In 2002 an evolutionary robotics conference was held in Fukui, Japan with the stated goal of furthering this
effort. Progress reports on on
ng ER projects were presented: They ranged from the evolution of real
avoiding flying or rolling robots, and robots that walk over uneven terrain, or simulated robots that adapt
to dynamic environments, such as RoboCup soccer, or highway tra
ffic. Nearly all of the projects were concerned
with ER adapted to real or simulated dynamical environments, and thus could profit from the development of EER.

The natural question to ask is can ecological psychology furnish a general account of autonom
ous agents to
complement ER? And if so, what are the major issues to be resolved? We explore some of these issues in this last
section of the paper.

Before doing so, let's clear up one point that has sometimes been a source of confusion. We shall expec
EER to provide methods and concepts for understanding autonomous agents that exhibit effective performance in
world task environments. Note, however,

performance does not mean
performance, but
only that it be tolerant, that is, su
ccessful to a limited but practical degree. The degree of effective performance is
defined relative to some practical criterion for success

thus in the case of an EHW device, it either learns or
evolves so as to satisfy a kind of FF. Indeed, finding a pr
oper ecological interpretation of this FF will prove to be
the key by which we can formulate a theory of EER relevant to our problem.

Evolving Ecological Robots with Affordance
effectivity Fits

Ecological psychology argues that effective learning is shap
ed by selective evolutionary pressures which
guarantee a commensurability between an animal's action capabilities and what the environment demands of
adaptive acts. That is to say, the agent's

relevant task
constraints) must match t

(invariant environmental properties specific that provide causal and informational
support for a potential goal
directed activity).

It should be clear from the definitions given above that a measure of an agent's competence i
n reaching a
goal is synonymous with how well the task affordances are matched by the agent's effectivities. The evolution of
such competence can be construed as a GA
driven EHW process, where the degree of match required plays the
role of the FF. The gr
eater the affordance
effectivity match in a given generation, the better the FF score.

More specifically, imagine a population of robots whose members are endowed with the same type of
EHW chips and which share the same FF for a given task. Further, assu
me that this population is partitioned into
several sub
populations, with each sub
population being assigned to a different environmental situation. The
environmental situations belong to a range of task
situations over which one would like an autonomous
agent to be

Consider an ecologized version of Thompson's suggestion for how to evolve autonomous agents with broad
competencies: Let each situation
specific sub
population of robots evolve according to the same FF. Because
they share the sa
me FF, over many generations, the emerging selection pressure evolves robots which are all
competent to exploit the same affordances, namely, do the same task; but do so in different ways since their
effectivities were attuned to different situations. Thu
s the different strains of selection pressure will produce a class
of circuit designs that support different effectivities but realize the same task
specific affordance. The common


affordance defines an
invariant structure

over all the robots and reflects

those properties most stable over change
in situation. The distinct effectivities, by contrast, provide the
perspective structure

specific to properties that are
unstable over change in situation.

Thus a robot whose FPGA circuit evolved for one task s
ituation may not work properly when transferred
to a new situation. To construct a robot that can solve the same task in different situations, the trick will be to
couple all the different circuits into a single super
FPGA endowed with all the situation
pecific, embodiment
specific competences.

A New Tractability

Critics of the ecological approach have often complained that the affordance concept is unclear, primarily
because the notion of invariant structure has not been explicitly defined. They als
o have complained that
effectivities, here construed as the situation
specific perspective structure, would not be scientifically acceptable until

a mechanism can be given for their design (Kubovy & Pomerantz). We now reply to those critics: Any equation

set adequate to express the defining characteristics of either of these real
world constructs would have too many
dimensions to be solved in closed form. Their context
sensitivity makes them truly intractable in the mathematical

But, here, by i
nterpreting affordances and their matching effectivities in terms of ER, we have a constructive
argument for their scientific tractability; they can be reliably evolved even if not formally or mechanistically
described. It may be that the robotics field h
as run up against what might be called the von Neumann barrier to
mathematical tractability.

A half
century ago, John von Neumann, the great Hungarian
American mathematician, and a father of the
computer era, conjectured that there may exist a
barrier t
o tractability

that a theorist encounters in trying to state
explicitly how systems of even moderate complexity work. By complexity, he was not referring to how
complicated a system is as determined by counting parts; rather he was referring to how compet
ent the system was
in a variety of real
world contexts, that is, how many

directed behaviors) it was able to perform.
When the complexity barrier is reached, he argued, the best model of such systems may not be mathematical or
verbal b
the system itself
. This prompts the question: Is the deep level of physical detail that EHW modules
engage and exploit with their rich digital
analog dynamics already on the wrong side of this tractability barrier?

It may be frustrating to the posit
ivist to admit that possibility, but, by its very nature, the conjecture cannot
be formally proven since it tries to relate a mathematical domain to a nonmathematical domain

a semantic rather
than formal problem. Still, the example of EHW may be informal
evidence of its validity. If so, then EER methods
demonstrating that a complex system can, in principle, be evolved to have certain competencies (specific
effectivity matches) may offer another kind of tractability.

The Case for History Machi

Here are three significant ways that robots with EHW may be an improvement over those with OHW:

First, consider Thompson's aphorism: "Implementation

design" (Thompson, 1998) reminds us that by
implementing EHW we immediately endow the system's ar
chitecture with the means to design (or redesign) itself
repeatedly over time.

Second, analog hardware

free digital hardware, obtained by exploiting the transients of the pulses
which are permitted when the constraints that render them "al
nothing" are removed.

Third, where ordinary hardware is characterized by states and state transitions, EHW, instead, is characterized by
configurations and their histories (a

is defined as a succession of epochs, where a historical epoch is
segment of history preserved between a pair of generations).

The first and second points help explain the improvement of ER with EHW over robots with OHW. The third point
is the basic thesis of EER, or any other evolutionary system for that matter; it

asserts that any evolutionary systems


is best described by their unfolding histories rather than by transitions over states. Indeed, given the tractability
barrier, there may be no other choice. States, however, may be used where they specify directly t
he history to
which the states, like symbols, refer (and thus are semantically

The developing EHW strategy (see, Higuchi, et al., 1995; Thompson, 1998; Nolfi & Floreano, 2000),
when merged with ecological psychology yields the basic ingredient
s for an EER. Earlier, a case was made that
EER is both valid and (practically) tractable. To the extent that this is so, then, to that extent, it counters the Minsky
(1967) and Wells (2002) claim that history machines are useless because they are too il
defined and cumbersome
to use.

In addition, it also redeems the Shaw & Todd (1981) claim that, contrary to Minsky/Wells claim, history
driven machines not only are tractable (again, in a practical if not formal sense), but in many important ways can
urpass state
controlled systems, especially, in modeling how systems (e.g., robots) evolve the competence for
learning to solve real
world tasks. For instance, Thompson (1998) shows how systems based on evolutionary
principles (e.g., EHW) can explore circ
uit designs that transcend the scope of conventional ones. These circuits,
being less constrained in spatial structure than ordinary silicon chips, exhibit considerably richer dynamics than usual.

For having more freedom, they achieve greater sensitivity

to the properties of the physical medium in which the
circuit is implemented. This results in a circuit that is better tailored to exploit all of the characteristics available in

implementation medium.

The late Robert Rosen in his seminal book,
, makes a strong case for the logic and design of
biological systems being quite different from that of programmable machines (Rosen, 1991). Until the advent of
evolvable hardware, machines could be endowed with artificial intelligence only through
a human's intervention as
designer and programmer. Now, it seems, we have systems that can evolve their own intelligence but which may
not be so much artificial as ecological. If so, then evidence to support Rosen's case is evolving.


Brooks, R. (1991). New approaches to robotics.
, 253:1227

Future Emerging Technologies (FET Robotics Work Group 2002. Information Society Technologies (IST). (see
IST/FET website) (no authors given)

Harvey, I., Husbands, P., Cliff, D., Th
ompson, A., & Jacobi, N. (1997). Evolutionary robotics: The Sussex
Robotics and Autonomous Systems
, 20:205

Higuchi, T., Niwa, T., Iba, H., Hirao, H., Furuya, T, & Mandrick, B. (1995). Evolving hardware with genetic
learning: A first step

toward a Darwin machine. In J
A, Meyer, H. L. Roitblat, and S.W. Wilson (Eds.).
Animals to Animats 2. Proceedings of the International Conference on Simulation of Adaptive Behavior.

Cambridge, MA: MIT Press.

Minsky, M. L.
Computation: Finite and I
nfinite Machines
, Englewood Cliffs, N.J., Prentice
Hall, 1967.

Moravec, H. (1999). Rise of robots.
Scientific American
. pp. 124

Pollack, J., Lipson, H., Ficici, S., Funes, P., Hornby, G., & Watson, R.(2000). Evolutionary techniques in physical

Rosen, R. (1991).
Life itself: A comprehensive inquiry into the nature, origin. and fabrication of life.

York: Columbia University Press


Thompson, A. (1998).
Hardware evolution: Automatic desi
gn of electronic circuits in reconfigurable
hardware by artificial evolution.

London: Springer

Tuci, E., Quinn, M., and Harvey, I. (undated WWW paper). An evolutionary ecological approach to the study of
learning behaviour using a robot based mo

Yao, X. and Higuchi, T. (1996). Promises and challenges of evolvable hardware.
First International
Conference on Evolvable Systems: from Biology to Hardware
. Tsukuba, Japan, 7
8 October 1996.