CO-EVOLUTION OF GENOMES WITH THEIR GENOMIC PARASITES IN GENETIC ALGORITHMS:

chatventriloquistAI and Robotics

Dec 1, 2013 (3 years and 6 months ago)

99 views

CO
-
EVOLUTION OF GENOMES WITH THEIR GENOMIC
PARASITES
IN GENETIC ALGORITHMS
:

APPLICATION TO

THE

ANT PROBLEM

Alexander V. Spirov
1, 2, *
, Alexander B. Kazansky
1
, Alexey S. Kadyrov
1
,
Juan J. Merelo
3

and Vladimir F.
Levchenko
1

1
The Sechenov Institute of Evolut
ionary Physiology and Biochemistry, 44 Thorez Ave., St. Petersburg, 194223,
Russia

2
Dept. of Applied Mathematics and Statistics, The State University of New York at Stony Brook, Stony Brook NY
11794
-
3600, USA

3
Departamento de Arquitectura y Tecnología de C
omputadores
, University of Granada, Granada, Spain

*
Corresponding author:
Dept. of Applied Math and Statistics, The University at Stony Brook, Stony Brook NY
11794
-
3600


Email: spirov@kruppel.ams.sunysb.edu; v:631
-
632
-
8370; f:631
-
632
-
8490


ABSTRACT

Genetic

Algorithms (GA) and Genetic Programming were inspired by classic Darwinian ideas

of evolution
.
However modern evolutionary biology is far beyond classic Darwinism. The application of

other algorithms and
modern biological ideas may substantially improve t
he

performance of Evolutionary Computations (EC).
Macroevolutionary mechanisms based on t
he mobile selfish

genetic elements
-

transposons are good candidates
for this

breakthrough. The processes in the world of

transposons, living on a substratum of genome
s of whole
biological

communities, are thought to be the main source of evolution creativity.


In this communication we propose a strategy of construction of a new scheme

for EC exploiting the
most essential aspects of co
-
evolution of the

genomes

with thei
r

genom
ic parasites. We named this strategy as

construction of

the Two
-
layered Co
-
evolving Worlds.
Our approach exploits implicitly hybrid character of
evolut
ionary mechanisms discovered b
y modern evolutionary biology, where global search combined with loc
al
one, as well as

random mutagenesis combined with targeted one
.

We apply our approach

to one of known benchmark

tests

-

the ant problem
. We found that our
enhancement of GA technique by the artificial

transposons obviously increase the efficacy of searc
hing of the
ant's

navigation algorithm. We in
vestigate in details
the transposons

activity as example of
sophisticated

local
search
.

I
NTRODUCTION

Many areas of evolutionary computation, especially genetic algorithms (GA), and genetic programmi
ng (GP),
wer
e inspired by ideas from

evolutionary biology. However

modern evolutionary biology has since advanced
considerably, revealing that
Darwinian evolution (microevolution) apparently is a particular case of essentially
more complicated mechanisms of macroevolu
tion
. Many branches of modern evolution
ary computation research
realize
evolution of mechanisms, such as functional programs, neural networks, decision trees, cellular automata,
L
-
systems, finite state automata. For these d
omains,
recent achievements

in

ge
nomic
seems more appropria
te as
an inspirational model tha
n classic set of Darwinian algorithms [Cf. Altenberg, 1994
;

Luke et al., 1999
; Lee and
Antonsson, 2001; Lones and Tyrrell, 2001
].

This stimulates us to select, formalize and apply to EC some new evo
lutionary
mechanisms that
could
help to
simulate the creative, heuristic and self
-
organizing character of (biological) evolution [Spirov, 1996a;
1996b; Spirov and Samsonova, 1997; Spirov and Kadyrov, 1998; Spirov et al., 1998; Spirov and Kazansky,
1999
; 20
02a; 2002b; 2002c].


We concentrate on mechanisms of
Natural Genet
ic Engineering, whose key acto
rs are
mobile selfish
genetic elements
. The basic idea
can be outlined as follows:




The

genome
of every organism has got

very special mecha
nisms of genomic rear
rangements;



These mechanisms

are

activate
d in the period

of evolutionary crisis;



2

2



The mechanisms cause multiple systemic rearrangement of gen
omes in a few generations
.

The mobile selfish genetic elements (synonymous or related terms are jumpin
g genes, selfi
sh DNA
,
transposons and retroelements

[Makalowski, 1995; 2200; Lozovskaya, 1995; Hurst

&
Werren, 2001]
) are good
candidates for this breakthrough
. Many biologists speculate that proces
ses in the world of transposons
, living on
a substratum of genomes of th
e whole biological communities, are the main source of macroevolution creativity
[Brosius, 1991
;
King, 199?;
Shapiro, 2000
; 2002
].

T
here are several gro
up
s
of genomic parasites.
Transposons form the most sophisticated one.
“…
it is
now recognized that a si
gnificant portion of the genome of any eukaryote is composed of 'selfish' or 'parasitic'
genetic elements, which gain a transmission advantage relative to other components of an individual's genome,
but are either neutral or detrimental to the organism's f
itness
” [Hurst

&
Werren, 2001].
Arguments have been
made that the structure of eukaryotic genomes, including the abundance of transposons, repetit
ive DNA and
introns, provide high

evolvablity

[Shapiro, 2000; 2002]
.


It has been estimated that typically 98%

of the DNA in higher organisms is neither translated into
proteins nor involved in gene regulation, that it is simply ``junk'' DNA.

The bulk of the junk DNA is
intermediate
-
repeats comprised of DNA elements that are able to move (or transpose) throughout
the genome.
These mobile DNA elements are sometimes termed "selfish DNA," since they do not appear to directly benefit
the host. Th
eir

behavior i
s

purely parasitic
: they

jump

between different parts of a genome in order to propagate
themselves
,

and
, from
the first sight,

this is usually to the detriment of their host.

But t
hey do appear to benefit
the host organism
by providing genomic rearrangement
s

that p
ermit the evolution of new genetic networks
.

Transposons are ubiquitous and may comprise up to 45% of

an organism's genome

[Lozovskaya, 1995;
Makalowski, 1995; 2200; Hurst

&
Werren, 2001]
.

DNA of transposon origin can be recognized by their
palindrome endings flanked by short non
-
reversed repeated sequences resulting from insertion after staggered
cuts.

M
any transposons have a unique DNA site, which acts as a forwarding address, directing the transposon to
a complementary DNA site in its host genome.

There are usually multiple copies of any given DNA site in the
host genome and

transposon will attach the p
roper site in
random

manner
.

Transposons may grow be acquiring
more sequences
; one

such

mechanism involves the placement of two transposons into close proximity so that
they act as a single large transposon incorporating the intervening code.

Some data was

obtained recently which evidence that transposable elements are a major source of
genetic change, including the creation of novel genes, the alteration of gene functions, and the genesis of major
genomic rearrangements [
Fadool et al., 1998; Hurst

&
Werren
, 2001;
Lozovskaya, 1995; Makalowski, 1995;
2200
]. The long coexistence of transposable elements in the genome is expected to be accompanied by host
-

transposon

co
-
evolution
. In this connection, speci
al interest is attracted by
known examples of both compe
titive
and cooperative strategies

in populations of transposons. That is why trans
p
osons are

treat
ed
as higher
-
level and
intelligent

mutators. Besides, transposon
s can be considered as evolutionary “SOS
-
crew”, cooperatively acting
“in emergency”, at host’s

genomic stress [McClintock, 1984; Lozovskaya, 1995].

Modern evolutionary biology and evolutionary genetics (
and
genomics) compile

a lot of knowledge

about very complicated mechanisms of genome rearrangements governed by transposons [
Agrawal

et al., 1998;
Brosius, 1991; Hurst

&
Werren, 2001; King, 199?; Makalowski, 1995; 2200; Lozovskaya, 1995; Shapiro, 2000;
2002]. From EC point of view the most sophisticated of these mechanisms can be treated as local evolutionary
search. First of all
,

we mean
mutational
/ recombinational hot spots
caused by transposons.

It is very promising
to include the world of selfish elements in the basic frames of Evolutionary Computations (EC) and to use the
objects/procedures for maintaining and manipulation of the parasites as a
new prospective branch of EC.
In this
article we study in details co
-
evolution of
artificial transposons with their hosts for GA version of the ant
problem.

State
-
of
-
t
he
-
Art

Biologically Inspired EC

There is a feeling that the field of EC is getting more
inspired with the latest achievements in biology, trying to
make the evolutionary algorithms more effective. These attempts have been termed
Genomic algorithms
,
bacterial

algorithms or even “genetic” algorithms (quotes intended). Many other techniques take
n from biology,
such as transposition, host
-
parasite interaction and gene
-
regulatory networks have also been applied to
evolutionary computation. There is no uniform nomenclature, however, and sometimes the same terms are
applied to different methods, or t
he other way round.

These techniques can be divided broadly into three groups:


Host
-
parasite methods
.

These methods are based on the co
-
evolution of two different populations, one
of them acting as “parasite”, and the other acting as “host”; the parasites

usually encode a version the problem


3

3

domain, and the hosts

encode

the solution to the problem [Hillis, 1990; Potter and De Jong, 1994; 1995; De Jong
and Potter, 1995; Olsson, 1996; 2001]. They have been used mainly to evolve sorting networks; but similar
approaches have been used, for instance, for evolving players of the
Othello

game. In this case, the approach is
not significantly different from other co
-
evolutionary approaches, in which solutions and instances of a problem
are co
-
evolved.


Transposition

operators
. They

are sometimes known as “bacterial” algorithms [Harvey, 1996, Nawa et
al., 1996; Simoes and Costa, 2001]. The basic idea of both approaches is to make intra
-
chromosome crossovers,
that is, crossover of a chromosome with another part of itse
lf, or else asymmetric crossover, in which a donor
chromosome transfers part of its genetic material to an acceptor chromosome. In some cases, these operators
seem to be better than classical genetic algorithms for combinatorial optimization problems, and
also in the case
of a fuzzy rule base. Similar transposition operators have also been used in other contexts, mainly in
combinatorial optimization problems.


Other biological approaches
.

Luke et al. [Luke et al., 1999] use a method similar to genetic regul
atory
networks to evolve finite state automata that represent a language grammar; this kind of objects cannot be easily
represented using serial, bitstring, genetic algorithms. Burke et al. [Burke et al., 1998] try to make the
evolutionary process closer t
o its real molecular genetics base, by having a 4
-
letter genetic alphabet, transduction
processes, and variable
-
length genetic algorithms. In both cases, it is not a general
-
purpose techniques; they only
draw some elements from biology to solve problems th
at would be difficult to solve in other cases.

So far, it has not been found in the literature a technique that is general enough to be applied to a wide
range of problems, and that, in some cases, is able to yield as good or better results than evolutiona
ry algorithms.
In this paper we try to outline such an approach, and apply it to one of known benchmark problems


John Muir
ant’s trail test [
Jefferson et al., 1991; Koza, 1992
].

In this article we propose general strategy of construction of a new scheme
of
EC

exploiting t
he most
essential sides of genomes
-

genom
ic parasites co
-
evolution. We named this strategy as

construction of

the
Two
-
level Evolving World
. In the next part
s

we give the definition of the strategy.

One of the key features of the
approach

is very special local search performed by populations of artificial transposons. After description of the
local search

we demonstrate the efficacy of a new approach on the example of John Muir ant’s trail test.

Key Characters of Natura
l Transposons as In
spirational M
odel

Mobile (transposable) elements encode the ability to move to new positions in the genome and can therefore
be
accumulate
d

in genomes.

These genomic parasites li
ve on a substratum of gene pool of the whole biological
community
.

They are th
e autonomous selfish or parasitic codes
, which

co
-
evolve with their hosts.

They are
transmissible horizontally (from one host to another) and vertically (from the host
-
ancestor to the descendants).


Transposons

act as intelligent and sophisticated mutator
s for the host codes

(genes)
.
They
cause
different types of mutations when they transpose themselves. These mechanisms

are

based both on enzymes
encoded

both
by

transposon own genes and host genes.

Class I transposable elements transpose through an RNA in
termediate. After the element is transcribed,
the RNA copy is converted into DNA, frequently as a result of reverse transcriptase activity encoded in the
element itself. This DNA copy now reinserts in the genome at another location. Transposition of class
II
elements involves DNA excision and homologous repair. Each class II element encodes a transposase, which
excises the element from the chromosome. The point of excision is repaired using the sequence of homologue or
sister chromatid as a template, which
creates a duplicate of the transposable element. The excised copy is free to
insert at another site within the genome.


Different mechanisms of transposon activity caus
e different types of mutations:

1)

Non
-
random insertions. The insertion of transposons (
by above sketched mechanisms) is
generally non
-
random but targeted. Certain regions of the host DNA are favored for insertion; these are referred
to as hot spots and tend to be specific to a transposon or its family. The insertional specificity based on fa
ctors
encoded by transposon genes.

2)

Recombinational hot
-
spots. Transposable elements serve as recombinational hot
-
spots
allowing the exchange of genetic material between unrelated chromosome
sequences [Makalowski, 1995]
.
Because they are so abundant, th
ey can mediate large scale chromosome restructurings by means of homologous
recombination between similar transposable elements at distant locations.

3)

Mutational hot
-
spots. For instance, two enzymes key to generating the immune system's
inexhaustible va
riety of antibodies are relics of an ancient transposon. A great deal of this variability is due to the


4

4

way antibody genes are assembled: by joining three separate sequences (denoted V, J, and D), each of which
comes in many variants, into a single antibod
y gene. The enzymes that do this assembly, Rag1 and Rag2, work
just like enzymes transpos
ases that mobilize transposons [Agrawal

et al., 1998]
. The transposases recognize the
ends of the transposon and cut it out of a chromosome. That freed
-
up piece of DNA
, with the transposase still
attached, can loop around itself and move to a new place in the genome. That's just what Rag1 and
-
2, which act
together in a complex, do.

4)

Rearrangements of host genetic networks. In addition to moving themselves, all type
s of
transposable elements occasionally move or rearrange neighboring DNA sequences of the host genome.
Transposons “strew” the genome different regulatory sequences. By so doing, transposons can re
arrange hosts
genetic networks [
Makalowski
,

1995
; 2000]
. T
ransposons may capture genes and move them wholesale to new
parts of the genome. They make possible DNA shuffling that can place genes in new regulatory contexts and,
possibly, new roles. But what is more, when two transposable elements recognized by the
same transposase
integrate into neighboring chromosomal sites, the DNA between them can become subject to transposition.
Because this provides a particularly effective pathway for the duplication and movement of exons, these
elements can help to

create new

gene
tic ensembles

[Alberts
,

1994]
.


It has been estimated that 80% of spontaneous mutations are caused by transposons. Repeated sequences,
resulting from the activity of mobile elements, range from dozens to millions in numbers of copies per genome.


Tran
sposons co
-
evolved with their host genome as result of selection, favoring transposons, which
introduce useful variation through gene rearrangement. In this manner, "smart" genetic operators evolved.


It is likely, that transposons to r
epresent the highly
coevolved “genetic operators”
. The possession of
smart genetic operators must have contributed to the explosive diversification of higher organisms providing
them with the capacity for natural genetic engineering.

In designing artificial evolution, it woul
d be wo
rthwhile to introduce such genom
ic parasites, in order to
facilitate the sharing of the code that they bring about.

The artificial transposons act as intelligent and sophisticated mutators. They can generate arbitrary
procedures of manipulations wit
h hosts’ chromosomes. In general, these operators can be the unitary, binary or
plural ones. Each host has got the mutators of its own. In the simplest case the transposons are the only source of
the host’s mutations.

If artificial transposon founds hopefu
l mutation strategy, then both host and parasite will get chance for
reproduction. Virtually we have co
-
evolution of hosts and their intelligent mutators
-
parasites.

The Two
-
level Evolving World

We propose a new approach exploiting some aspects of co
-
evolut
ion of the hosts and their genomic parasites.

The key feature of the approach is the usage of artificial transposons. Two
-
level Evolving World consists of

basic level of hosts and upper level of transposons multiplying on a substratum of the host genomes.
The
artificial transposons are codes served by their own operators of reproduction, mutation and transmission.

Parasites and parasite ensembles always accompany biological evolution. Tom Ray simulated this
process in his
Tierra

[
Ray, 1991
]. Parasites tend
with time to reduce their organization for the safe of host’s
resources. In a result, they acquire the possibility of fast (essentially faster then hosts) multiplication and
horizontal and vertical transmission. As a general result, they can evolve drasti
cally faster then hosts (influenza
virus is a sound example).

There are examples of evolvable virtual worlds such as Swarm, Creatures, Network Tierra [Daniels,
1999; Cliff and Grand, 1999; Ray, 2001]. In the course of evolution the worlds of
that type can
split over
parasite

and host co
-
evolving worlds, i.e. they can become the two
-
leveled. It is the question of time and such
worlds’ complexity. In less complex virtual worlds similar splitting could be realized “by hand”, as in the case of
developing world
of computer viruses.


Parasites are so common that hosts soon co
-
evolve immunity to them. Then eventually the parasites co
-
evolve strategies to circumvent that immunity. Then eventually the hosts co
-
evolve defenses to repel them again,
and so on. This is
evolutionary Arms Race.


A special kind of parasite is genomic parasite living in the host genome. This special kind of co
-
evolution is the arms race between genomic parasites and their hosts.



5

5


Genomic parasites are the main source of host mutagenesis. If
the parasite founds a good strategy for
desirable host mutations, both the host and the parasite will get a chance for reproduction. The parasite rides a
new turn of evolution on the transformed host.

We assume that the simplest realization of the two
-
laye
r evolving worlds would be as follows:

the hosts
-
world is GA
-
like system (standard GA in the simplest case). The manifold of hosts’
chromosomes
-
st
rings is the environment for parasitic codes
. In the simplest case these GAs don’t have any
mutation operators

of their own;

the p
arasites are the LISP
-
like programs, manipulating with the hosts’ strings. (For our applications
these
programs must include
the search

function

performing the search of
sequence
s in the host strings).

the parasites

live in hosts, they

are transmitted vertically (when host reproduces) and horizontally (from
one host to another, as infection or computer virus
, by
the transposition operator
)
.

Artificial Transposon
s

for GA

Analogously with their biological prototype, we can define artific
ial transposon as marked block of code in
host's chromosome. The code block marked as artificial transposon still functionally belongs to host. It is still the
part of host code. However this code block is transmittable now from host to host and able to mu
tate and growth
according different rules than remaining parts of host’s code.

The way to distinguish transposon code from other code depends from problem. In case of ant problem
we use strict definition. Special routine scan each chromosome in population

to find appropriate code blocks. In
the case of Royal Road functions we don’t use any
definitions or

search routines, but we introduce in initial
population initial transposons

[
Spirov

and Kazansky
, 2002a; 2002b
]
.


To mark the beginning and the end of th
e transposon's code block, we could use special ''signaling``
sequences or signaling symbols in the host's chromosome. But we found more convenient to use double
-
string
chromosomes. The main string (binary and non
-
binary) is used for codes, while additiona
l one (binary)
-

for
marks. The possibility to use such chromosomes there is, for instance, in C++ library
Evolving Objects (EO)

[
Keijzer

et al., 2001
]
.

Let mark will be 1 and the absence of it
-

0. The block (cluster) of marks (.
.
.1111111...) distinguis
hes
transposon elements from the host's ones. In other word, instead to mark the beginning and the end of a
transposon, we mark all its elements in a sequence.


The main string contain
ing the host's and transposon's code blocks can be binary, symbolic, fl
oating
-
point or other. In the case of binary main string transposon looks as follows:

The additional string 1111111100000000...

The main string 11111111********...


--------



transposon


where * is either 0
or 1.


We used such scheme for evaluations the performance of our artificial transposons approach relative to
standard GA on a class of Royal Road fitness functions
[
Spirov

and Kazansky
, 2002a; 2002b
]
.

In the case of ant problem treated in this article w
e can represent our artificial transposon as follows:


The additional string 1 1 1 . . . 1 1 1 0 0 0 0

The main string s
1

s
2

s
3

. . .s
n
-
2

s
n
-
1

s
n


* * * *


-------------------
-
--



trans
poson




6

6

Here the main string is symbolic one; S
i

is one of symbols (R, L, F, N, #); R, L, F, N are the symbols of
navigational commands for animate
-

ant and # is turning
point referring at one of previous positions

of the
string. R means turn left, L
-

t
urn left, F means one step forward, N
-
is NOP.

Mutagenesis Caused by Artificial Transposons

As in the case of biological prototype, artificial transposons act as sophisticated high
-
level mutators acting
dependently on the transposon own code and on the tr
anspo
son “context”

(surrounding host's code).
Owing from
different examples of known mechanisms of transposon
-
caused mutagenesis we can test several schemes of
ar
tificial transposons activity
. Some of these schemes have been testing by other authors also i
nspired by modern
evolutionary ideas [
Burke et al., 1998; Luke et al., 1999; Simoes and Costa, 2001;

Pereira and Costa, 200?;


].
Our approach is different from others because we exploit co
-
evolution of hosts
-
transposons based on the
genomic parasites popu
lational dynamics.

Mutational Hot Spots

One of the most general and simple
schemes

of transposon action as mutators is
random point mutagenesis of the
sequence covered by transposon, coupled with transmission of the transposon. Apparently it would be an
e
xample of local search distributed through host population via transmission. We tested this general scheme on
two benchmark problem


Royal Road functions and the ant problem and found substantial improvement of
evolutionary search comparing with standard
GA. These results will be published else
wh
ere.
Apart from this
scheme of mutagenesis we investigated some more, which we describe here.

Let we define as array
S

the transposon own code with it's nearest surrounding. For instance, the
S
for
the Royal Roads

test is:


S: 111...1110


111...111*



where * is either 0 or 1.


Hence, in the case of our Royal Roads tests, the context includes only one host’s element, following the
most right element of transposon

[
Spirov

and Kazansky
, 2002a; 2002b
]
.

Genera
l scheme of transposon action is:
S' = F (S).

Concrete view of the
F

function depends on
the concrete problem.

In case of Royal Road tests

described in [
Spirov

and Kazansky
, 2002a; 2002b
]

we used such simple
definition of F function:


if S: 111...111
0


111...1110


then S': 111...1110


111...1111



if S: 111...1110


111...1111


then S': 111...1110


111...1110



7

7



Hence this
F

function gives flip
-
flop mutation of the first host's element
following the most right
element of transposon.
In other words,
transposon conditions

mutational hot spot.

In the case of ant problem,
one of
the

tested

F

functions
is defined as follows:


If S:

1 1 1 . . . 1 1 1


s
1

s
2

s
3

. . . s
n
-
2

s
n
-
1

N


then




S': 1 1 1 . . . 1 1


s
1

s
2

s
3

. . . s
n
-
1

s
n




If S:
1 1 1 . . .


1


1


s
1

s
2

s
3

. . . s
n
-
1


#


then




S': 1 1 1 . . . 1 1


s
1

s
2

s
3

. . . s
n
-
1

s
n
-
5



Here
s
i

means i
th

symbol in the transposon's code;

N is NOP and # is internal cycle reference (turning point).
The F function substitutes the NOP element of the transposon code by
, for instance,

the fifth element of the
transposon, counted in order, while if the transposon ends on the internal cycle refer
ence, this element is
substituted by the fifth element of the transposon, counted backward from the end.

The Growth of Artificial Transposon

It is known that transposons tend to duplicate themselves and form clusters in host's chromosomes. We simulate
thi
s feature as outgrowth of our transposon by one element in a time. Of course, it is not the only way to simulate
multiplication and spreading transposons in host's chromosomes, but this concrete

scheme works well in our
applications. More precisely, it mim
ics known feature of real transposons to grow by including more genes.

Namely, time by time, an artificial transposon adds one more element to it's edge:


time T
1
:


1111...1111000


****...*******



time T
2
:


1111...11111
00


****...*******

where * is appropriate element of the main string.



8

8

Depending on application, the new
-
recruited transposon element may be arbitrary one (say 0 or 1 in our Royal
Road tests

[
Spirov

and Kazansky
, 2002a; 2002b
]
) or it must have pred
etermined values, as in the case of our
version of ant problem.

The

Ant Problem

The artificial ant problem is the simulation of an ant navigation aimed at
passing through the labeled trail
(nicknamed “The John Muir Trail” in the UCLA experiment [Jefferson
et al., 1991]).
The problem has been
repeatedly used as a benchmark problem [Jefferson et al. 1992; Koza, 1992; Lee and Wong
,
1995; Chellapilla,
1997; Harries and Smith, 1997; Luke and Spector, 1997; Ito et al., 1998, Kuscu, 1998].

In [
Jefferson et al.,
1
991
], the artificial ant has to path the John Muir trail placed in a grid world. The ant’s task is to pass through the
labeled cells one by one (the more the better) for the limited time period.
The ants are simple finite
-
state automata
or an artificial ne
ural network

which can move along the grid world and test their immediate surroundings. The trail
starts off quite easy to follow, and gradually gets more difficult, as the turns become more unpredictable and gaps
appear (Fig.1).
Each black (labeled) cell
is numbered sequentially, from the 1
st

which is settled directly next to
the starting cell, through to the 89
th
, the last cell. The ant’s task is to follow this trail and move across each black
cell in sequence.
Therefore, the successful ant’s program must

be quite sophisticated.

<FIGURE 1>

Artificial ant problem for two variants of trail (the “Santa Fe” and the “Los Altos”) is well tested and
discussed in publications. In both cases the trail has been carefully designed in such a way that it would be easy

to navigate at start but as ant proceeds through the trail, gaps and unexpected turns emerges progressively. By
the end of trail there are more gaps then labeled cells. But at the beginning, the successful tactic is very simple
-

only to pick up trail and

to move forward. A little further, the ant will need to learn the trick of turning to the
right with the trail. Some next moment they will need to learn the secret of turning occasionally left, and so on.
So, ant is progressively self
-
learning and perfect
ing algorithm of search in a course of passing it through.

The ant stands on a single cell and directed to the north, south, east, or west. It is capable of sensing the
state of the cell directly in front of it. In each time step, the ant must take one of

four actions. It may turn left,
turn right, move forward one step, or stand still. (When an ant steps on to a black cell, the cell turns white.) After
the ant performs its action, it shifts to a new state. Only the internal state of the ant and the state
of the cell in
front of the ant are used as inputs to the decision table which determines (a) the ant’s action and (b) the state it
will assume in the next time step. The decision table itself is coded as the ant’s genotype (Fig. 1B). The ant’s
score is th
e number of cells passed by the ant for the fixed time period.

METHODS AND APPROACH

While the ant test was implemented at least in two different C++ l
ibraries [Zongker and Punch, 199
5],
we gave preference to the Peter Brennan’s version [Brennan, 1994].
T
his “ANT program” was designed in such a
way that
to isolate, as far as possible, the components of the genetic algorithm from the trail
-
following
experiment and the ant representation. This is a deliberate decision intended to foster an easy transition fo
r the
GA code itself, out of the ANT application, and into whatever application may find a use for the GA. The
development of this program as

example of the MGE
-
approach was the aim of magisterial thesis of one of us
(A.S.K).

ANT P
rogram

The program consi
sts of three major subroutines:
Expose
,
Select
, and
Reproduce
[Brennan, 1994].

In the
Expose

subroutine, the each ant in the population is run against the John Muir Trail. Each ant’s
score is recorded.

In the
Select

subroutine, statistics are generated for

the previous Expose run. Each ant’s score is
compared to the maximum score attained in the population. One of two selection strategies is employed to
choose a given ant for reproduction. If the user has selected a
truncation

strategy, the fraction of ants

only with
the highest scores are marked for reproduction. If the user has selected a
roulette
-
wheel

strategy, an ant with a
higher score has a greater chance of being marked, although there is a chance that even the highest
-
scoring ants
may not be marked.

In the
Reproduce

subroutine, the genes of ants which are not marked for reproduction are overwritten
by copies of those which are marked, and then crossover and mutation are applied.



9

9

Technique o
f Mobile Genetic Elements

(Transposons)

Mobile genetic eleme
nts (MGEs) are akin to computer viruses. They are the autonomous programs, which are
transmissible horizontally (viz., from one site to another one on the same or another chromosome) or vertically
(from the ancestor to the descendants in the reproduction p
rocess). These autonomous parasitic programs
cooperate with the host genetic programs, thus realizing process of self
-
replication
-

the only aim, which can be
associated with that activity. We developed some new operators which are the computer program pro
cedures,
performing processes of replication, mutation and invasion of
transposon
s into specific sites on chromosomes, as
well as interactions of
transposons

with the chromosome (interrelations of parasite
-

host type).

It is appropriate here to make some

notes,
concerning the terminology. MGE
-
technique comprises the
procedures for initialization of mobile genetic elements and procedures for operating with the
se elements.
Hereinafter in remaining

section
s

mobile elements will be referred to as “
transposons
”, whereas the procedures,
operating w
ith them will be termed as “MGE
-
operators”. There are only two types of operators. The one
-
place
operator is an analogue of point mutation and the two
-
place (binary) operator realizing the procedure of
transmission of
transposon

from one chromosome (host) to another chromosome (another host).

Artificial
Transposon
s

Let us recall (see the previous sections) that the ant binary string
-

chromosome is coding a state transition table
of finite state automation. Altogether
there are 32 finite states of automation, ranging f
rom
state
#0 up to
state
#31
(See

Fig. 1B). All operators start reading and interpreting the table beginning from the
state
#0. For example,
state
#0 determines one of the four actions or instructions (FWD
-


forward”, RGT
-

“to the right”, LFT
-

“to the
left” or NOP
-

“do
-
nothing”) and the number of the next state, depending on binary input value (0 or 1). This
finite state automation can be represented as a state transition diagram and interpreted as a decisi
on tree but, as
far as references to already passed by states are permissible, that tree can have loops.

We will take into consideration that half of the decision tree which corresponds zero binary input value,
i.e. when ant see white (empty) square in fr
ont of it. Namely this part of ant’s transition state governs ant’s
search of black

squares. In so doing, we reduce ant’s genome to symbolic string

and substantially simplify
following treatment of the problem. This is an example of such symbolic
string
:

L
,R,R,F,R,L,L,F,
#0

w
here L is LFT, R is RGT, F is FWD and #0 is
turning
point referring at the first element of the
string
. The
navigation program encoded by this
string

is optimal for classic
Santa Fe
trail.


It is remarkable, that
these string
s are really

variable length ones, but the maximum length is 32
symbolic elemen
ts. For instance, this string


R,L,L,F,#0

is shorter but equivalent to the previous one.


For further consideration it is essential that two abovementioned

examples of symbolic string
s are

good
as navigation program for our trail also, but up to 64
th

element only
. Hence we can treat these programs as
building blocks for search of more sophisticated ant’s navigation algorithms.


If we use two
-
string implementation, then it seems perspective

for following search to mark this
sequence of symbols as a transposon.
Namely we could imagine such

artificial transposon
:


The

additional string 1 1 1 1 1
0 0 0 0

The main string
R L L F #0

* * * *



----------




transposon


Generally speaking, in accordance with all what we know about transposons activity in rearrangement
of host genomes, we treat it as mutational / recombinational hot spots. Besides, one of the very special features
of transposon
s is that they are jumping hot spots. Apparently there are
many w
ays to
implement these features in
artificial transpo
so
ns. In this article

we
pay attention to that transposon features which allow us to treat it as kind
of generator and multiplicator / dis
tributor of building blocks. In such a way artificial transposon activity is
keen
example of local search in limits of global GA search. Namely, initial population of artificial transposons


10

10

disseminate and mutate short code blocks till them find the first
building block. The transposon caring and
covered this b
uilding block (for example as it is

the case of abovementioned
R,L,L,F,#0

string
)
have
go
t
selective advantage, spread

and become dominant in host population.
In the long run another tran
sposon
will
f
ind one more
,

or large
,

or better building block and will get selective advantage, etc.

Obviously the simplest way to perform such local search would be the
general scheme o random point
mutagenesis of transposon sequence coupled with the transposon horiz
ontal transmission, as mentioned in

Mutagenesis Caused by Artificial Transposons
” section.

However
, t
o

st
udy in details

populational dynamics of
artificial transposons
as sophisticated mutators we decide

to restrict

maximally

the
transposon variability. T
o
achieve this we use

a

definition of transposon
.
For all tests discussed in this article w
e
use this concrete
definition
.
Namely, t
ransposon

is the sequence of symbols
, having the following properties:

the

sequence should include symbol
s which number lie
in the range between minimum and maximum
values;

the
sequence

should not contain NOP elements and internal circles;

the
sequence

should be finished up with command NOP or with a reference to the initial state. The
transitions cycle will be executed until o
nly white squares remain ahead of the ant.

If the fitting
sequence

is a cycle, it will be referred to as a
mature

transposon
. But, if the fitting
sequence

finishes up with NOP, we will name it the
immature
transposon
. The exampl
es of mature and
immature t
ransposon
s are given on Fig. 2
.

The
usage of such sort of transposon definition drastically restricts transposon’s search space and
essentially facilitates

monitoring of transposon population dynamics.

By usage of the definition we exploit very
small part

of transposon sequences manifold, but this small part contains solutions we looking for.

Apparently
this restriction is one of some (or many) others.
We tested it and found appropriate for our purposes. What’s
more, the usage of the tran
sposon definition
makes unnecessary

to mark the transposon section
s, because
transposon have definition.


MGE
-

operators

MGE
-

operators scan the predetermined quota of chromosomes in population. Successively decoding
chromosome record, this operator is seeking for

seque
nces, which are identified as
transposon

(mature or
immature).

Two
-
place MGE
-
operator provides the transmission of the
transposon

from an ant to another one, thus
realizing the reproduction procedure of this
transposon

in gene pool of the host (ant) popul
ation. This procedure
performs the following operations.

First, a pair of ants is chosen at random. Then, the chromosome of any of them is scanned in search of
the
transposon
. If the
transposon

is found, it is replicated in the partner chromosome, irrespec
tively of initial
record character in that chromosome. The chromosome scanning starts from the zero line (state#0) and goes on
as far as the first
transposon

is met. If no
transposon

is met, scanning finishes up only when the chromosome
record ends. So, sc
anning ceases irrespectively of the remaining chromosome un
-
scanned part content.

One
-
place MGE
-
operator is a sort of point mutation, realized under particular conditions

(Cf. the
previous section)
. This is what we call an
intelligent mutator
. In detail,
the operator acts in such a way. If it finds
a
sequence

in the predetermined length range, and the action NOP completes this
sequence
, then this instruction
is substituted for the one of the three other actions (FWD, RGT or LFT). Specifically, this NOP is
substituted for
the action from the fifth element of the
sequence
, counted in order. But, if the found
sequence

is completed by
the reference to the one of the elements inside
sequence

(internal cycle), then we have the following. The action
of this elemen
t is substituted for the action of the fifth element, counted backward from the end, the reference
being substituted for found at random reference to the element outside of the
sequence
.

The following peculiarity should be marked here.
The reference from t
he last symbol

to the first one
(i.e., by definition, a mature
transposon

proper) is processing with the mutator using the same rules. In other
words, the mutator breaks the cycle, thus destroying
transposons
!

So, the action of an intelligent mutator resul
ts in accumulation in the population of large length
immature
transposon
s. Point mutations are needed for their maturation. In a result, these new mature
transposon
s
get chance of spreading in host population.

We should emphasize, that the two
-
place MGE
-
o
perator operate only with mature
transposons
!



11

11

RESULTS

The test trail, used in this work is illustrated in Fig. 3. It can be seen that up to the 64
th

element our trail coincide
with the
Santa Fe
one, but the next part of the trail includes chaotically scatt
ered elements of high complexity.
Being trained on much simpler preceding trail part, the ant is not prepared to surmount the subsequent,
complicated sector (biologists would say that the ant is not pre
-
adapted to new conditions it faced with in this
secto
r). More specifically, problems arise at attempts to get over gaps between the 64
th

and the 65
th
, or the 67
th

and the 68
th

cells.

<FIGURE 3>

Transposon
s Really Accelerates
the

Evolutionary Search

The preliminary computer experiments showed that it is not
possible to construct trail, which ants could not
come through for the real feasible computer time. The accelerating effect of
transposons

is especially noticeable
for small populations, when the probability of the effective navigation algorithm finding by

applying standard
crossover and mutation operators is low.

On this basis, the following experiments were carried out on populations of 100 ants. The choice of
such a small population is also explained by our aim to carry out a comprehensive analysis of
tr
ansposons

dynamics. Such an analysis is not feasible for large populations of ants because of great number of
transposon
s.
For example, thousands of
transposons

appear in a population of only 100 ants for 1000 generations.

Besides limiting value of ant po
pulation abundance, the following parameters were fixed in all test and
control runs: the maximum number of generations (330) and the part of population to reproduce (15%). The
truncation strategy

of reproduction was used when the copies of chromosomes wit
h score exceeding the average
value replace all chromosomes having score less then average. The anal
ysis of preliminary runs showed

that the
experimental results concerned and the inferences made are not sensitive to the variation of these three
parameters
, i.e. the parameters are not critical in this case.

With the aim of demonstrating of the MGE
-
technique efficiency we performed 100 independent runs of
the program, 5000 generations each. The results of test and control runs (population with
transposons

an
d
without
transposons

correspondingly) were compared in several series with the different values of standard
mutation parameters. Everywhere in this section we will accept that the effective navigation algorithm should
overcome the level of maximum score i
n 64 for 330 time steps. Different populations reached different
maximum scores (65, 67, 69, 72, 74, 76, 78, 79, 80, 81) for 5000 generations. But the best result was the score
81, reached for 330 generations.

The resul
ts of program runs with the MGE
-
opera
tor and without it are illustrated in Fig. 4. It ca
n be
seen
, that MGE
-
technique obviously increases the probability of finding of effective navigation algorithm for
small populations and for a little number of generations.

<FIGURE 4>

As it is evident from

the graphs on Fig. 4, the mean and the

best
-
of
-
generation score

in experiment and
in control are growing, to a first approximation, linear in time. But the increment of growth in experiment with
transposons

is substantially higher, than in control.

It may

be suggested that MGE
-
operators raise ant variability mainly in nonspecific manner thus
supplementing mutation effect of standard operators. But, this suggestion is not substantiated by the detailed
analysis of mutation process (see below). We carried out

control runs with the frequency values of standard
mutations, which are two and ten times higher than in control series given in Fig. 4. Hence, the high level of
standard mutation
does not raise the efficiency

of the navigation algorithm
search;

moreover,

it decreases this
efficiency

(the results are not shown graphically).

No values of mutations and crossover parameters taken from the wide spectrum of combinations had a
pronounced effect on evoluti
onary search efficiency, if MGE
-
operators were disabled.

(Specifically, the
following combinations of crossover and mutation rates were tested: 0.0001 and 0.01, 0.0 and 0.04, 0.0001 and
0.08, 0.001 and 0.04, correspondingly; the results are not shown graphically).

Analysis of Transposon

Populations D
ynamics

Th
e
transposons

population dynamics analysis was studied for host population of 100 ants, the number of
generations to the end of experiment being limited by 500. These limitations were non
-
critical for the


12

12

experiment. At the same time, they made possible th
e graphical representation of the results, because the number
of new abundant
transposons

forms in every run did not exceed the value of 200.

Every generation of computer population in 100 ants produces new
transposons

offspring, consisting of
tens of new

mutant forms. We accounted for only forms with abundance, exceeding minimal threshold in 10
individuals. So, for the first 500 steps of artificial evolution we had the order of 100
-

150 new forms.

We performed a comparative analysis of dynamics of the 2
5 ant populations which were succeeded to
found for the determined number of generations an effective navigation algorithm with the 25 unlucky ant
populations, which could not elaborate such an algorithm. Some regularities in the dynamics of ant and
transp
osons

populations were revealed. These regularities are illustrated in Fig. 5 and summarized as follows.

1) As a rule, only one form of
transposons

is dominating in the population at a moment; tens of new
forms appear in every host’s generation and disapp
ear as maximum, in a several generations.


2) From time to time, the current dominating form is displaced by the other one. In a large number of
generations the former dominating form can rehabilitate itself (at least, some cases were observed).


3) As a
rule, 2
-
3 (less often,

5
-
6, sometimes, up to 10) dominant forms have an opportunity to change
each other in population for the chosen period of time (the first 500 generations).


4) From time to time, an explosion of abundance of subdominant forms does oc
cur, which can last for
5
-
10 generations. Eventually, these subdominant forms can give 2
-
3 population explosions. One of such
explosions can finish up with the conversion of subdominant form into a dominating one.

<FIGURE 5>

These regularities are inherent

equally to the dynamics of stagnant ant populations and to the high
-
evolvable populations, which found an effective navigation algorithm for the period being studied. It turned out,
that dynamics of stagnant
transposons

populations demonstrate the same fo
ur properties as the populations,
succeeded in finding an effective navigation algorithm. The moments of finding of that algorithm correlates with
dominant forms changes.

Comparison of the dynamics of learning of ant population with the dynamics of
transpo
sons

populations revealed apparent “coincidence” of the moment of change of
transposon

dominant form with the
moment of finding of the first effective navigation algorithm (Fig. 5). Just after this moment, mean population
score steeply rises up to the next

plateau. The process lasts for several generations. This local S
-
shape function
of the mean score growth correlate with curves of population size dynamics of dominant
transposon

forms (first,
the old form, then the new one) (Fig.5). It should be emphasize
d that this is only a correlation, not a functional
relationship. However, the existence of direct functional relationship between the code of a new
transposon

dominant form and the code of an effective navigation algorithm is still should be proved.

Large
r
Transposon
s Have More Pronounced Effect on the Rate of Ant
Population Training

It was mentioned in the section “Methods and
Approach”, that by default, MGE
-
operator is working only with
sequence
s of length, which lie in the range from 5 to 11. In all exp
eriments, illustrated in the Figs. 3
-
5, the
admissible sizes of
transposons

lie in the range from 5 to 11. Analyzing
transposons

population dynamics, we
revealed, that very soon, over some tens generations, all
transposons

reach the upper length threshold
(= 11).
Moreover,
sequence
s of length 12 and more appear in the population. MGE
-
operator recognizes these large
sequence
s but leave them intact, i.e. it does not apply to them the mutation procedure. This is explained by the
fact, that the one
-
pla
ce MGE
-
op
erator causes mutation not only in the
sequence
s, close to
transposons by the
definition (See “MGE
-
operators” section), but in the mature
transposons

as well. Selection pressure cause
surprisingly fast elimination of all
transposons

of length less than 11
(irrespectively, mature or immature it is)
and an accumulation of
sequence
s of length 12 and more (MGE
-
operator does not act on the latter).

We investigated also the behavior of an ant population, having
transposon
s of lager lengths. It became
clear, that
revealed regularity viz., domination of
transposons

of maximum length is valid for all its sizes up to
32 (this is a physical limit).

Moreover, it turned out, that
transposons

with higher upper size threshold have more pronounced effect
on the rate of ant
training. The averaged score dynamics, being taken from the previous experiments (see Fig. 3)
was compared with the score dynamics, obtained in the experiment with the same parameters but one
-

the upper
transposons

length threshold which was made equal to

32 (Fig. 6).

<FIGURE 6>



13

13

The analysis of
transposons

population dynamics has shown that observed general features of dynamics
sequence
s are the same for both cases
-

transposons

with high upper size threshold and
transposons

with low
one. In a several gene
rations,
transposons

of maximum possible length are coming to domination. As may be
inferred from the diagram (Fig. 6),
transposons

of length 32 are more effective mutators, than
transposons

of
length 11. So, the larger
transposons
, the “wiser” is it as a
mutator. We cannot give any simple explanation of
this effect so far.

Horizontal
Transmission of Transposon
s Is Necessary For Their
Effective Mutation Effect

As far
transposon
s are transmitted vertically (from ancestors to descendants),
transposons

of the
host, that have a
superiority in reproduction success is rapidly spreading in the population and gives new forms. But this process
per se is insufficient for the effective acceleration

of ant learning. Two
-
place MGE
-
operator, performing
horizontal distribu
tion of
transposons

from one ant to another is a necessary for rising of ant training ability. In
Fig. 7 we illustrate the results of comparing of the test, presented in Fig.3, with the similar test, in which
frequen
cy of applying of two
-
place MGE
-
operator

was reduced by the factor of 10 and accounted 5%. This
parameter determines the proportion of population, which is subjected
to the action the two
-
place MGE
-
operator
in a generation. In all previous experiments, presented in this article, this quota accou
nted 50%.

<FIGURE 7>

The obvious lowering of ant learning abilities with the decreasing of frequency of the two
-
placed
operator application is evident from the diagram. Disabling of the operator lowers the efficacy further and makes
it almost equal to the
control (case without
transposons
).

Ant
Popu
lations Adapts to the Selective

Pressure

of Transposons

In contrast to external, rigidly predetermined operator of mutation for classical GA, wise mutators co
-
evolve
with their host. It turned out, that the maxim
um length of
transposons

is the critical parameter, controlling fitting
of the host to parasite.

Small sized ant populations are predisposed to noticeable fluctuations, what makes them inappropriate
for our analysis. That is why, in this section we presen
t the results of computer experiments with the more
abundant ant population of 1000 individuals. The large population is more stable, so, the fine peculiarities of
dynamics are obvious (Fig. 8).

<FIGURE 8>

The analysis of all computer experiments, carried
out and presented in previous sections gave us
possible to propose the following model of the
transposons

role in the ant population dynamics.

In accordance to our model,
transposons subject

the population of hosts (i.e., ants) to a selective
pressure only

for short period of evolutionary time (as a maximum
-

some tens of generations). So, as can be
seen from the diagram 8, the quotas of population, subjected to the action

of one
-
place and two
-
place MGE
-
operator accounts 17% and 48% correspondingly, over 40

-

50 generations. For this short period of time ant
population is able to adapt to the
transposons

selection pressure at the expense of loosing of short
transposon
s
with length of less than 11.

DISCUSSION

The problem of programming an artificial ant to fo
llow the Santa Fe trail has been repeatedly used as a
benchmark problem in GP [Koza, 1992; Lee and Wong
,
1995; Chellapilla, 1997; Harries and Smith, 1997; Luke
and Spector, 1997; Ito et al., 1998, Kuscu, 1998]. Recently Langdon and Poli have shown that per
formance of
several techniques is not much better than the best performance obtainable using uniform random search
[Langdon and Poli, 1998].
According

to these authors, the search space is large and forms a Karst landscape
containing many false peaks and m
any plateaus riven with deep valleys. The problem fitness landscape is
difficult for hill climbers and the problem is also difficult for Genetic Algorithms as it contains multiple levels of
deception.

There are many techniques capable of finding solutions

to the ant problem (GA, GP, simulated
annealing, hill
climbing
) and although these have different performance the best typically only do marginally
better than the best performance that could be obtained with random search [Langdon and Poli, 1998]. That i
s
why the ant problem may be indicative of real
optimization

problem spaces and so be worthy of further study.



14

14

Dominant Transposon

Forms Are the Components of the Effective
Navigation Algorithms

The results of careful analysis of organization of several t
ens of dominant
transposon
s
(
11
-
13 elements in
length
)
, taken from those ant populations, which coped with the navigation task, can be summarized as follows.

1) By the definition, the
transposon
-
program begins and ends with the zero state, i.e., it is a lo
op,
executed over and over until the ant will meet the black cell. Typical examples of
transposon
-
programs and
corresponding ant movement trajectories are given in Table 1.

2) Four
-
fold execution of the
transposon
-
program produces in most cases the closed
ant trajectories,
i.e., the ant will return to the starting position. The examples of these closed trajectories are given in Table 1. As
a rule, the closed contour is located in domains the size of 4

4 or 5

5 cells. Some cases of
transposon
-
programs
are en
countered which provide ant movement back and forth along the polygonal path (See Table 1).

3) As a rule, the
transposon
-
program is beginning to work not from the zero state but from the Nth state,
which is specific to every
transposon
, not beginning with
the initial, zero state. This transition into the Nth state
takes place as soon as the ant (host of the
transposon
) runs against the white cell.

4) Start the
transposon
-
program from the Nth state provides the execution of the simplest navigation
algorithm,

necessary for overcoming the simplest gaps, arranged in the first half of the trail (“looking around”,
then one step ahead, “looking around” again and so forth). This algorithm provides the successful passage of trail
up to the 64
th

cell inclusive.

5) The

majority of program
-
transposon
s guarantee overcoming of the element of high complexity
between the 64
th

and the 65
th

cells (See Table 1).

6) Some
transposon
s are not suitable for the navigation programs. In that case the chromosome
elements, arranged in
transposon
-
free domain take control over navigation.

<<TABLE 1>>

The detailed analysis of the organization of dominant
transposon

forms in populations, which are
succeeded in finding of the effective navigation programs, showed, that the
transposons

themse
lves become the
components of these programs. Namely, the case in point is about the part of navigation program that is used for
effective “snuffing around” in situation, when ant faces with a wide gap.


In other words, transposons in our experiments reall
y found, cover and transmit host’s building blocks.

W
ise
Mutators Have a Search Space Confining Effect

The Muir’s Trail search space has rugged geometry due to specific and discrete character of the problem. That is
why, the gradient methods are not effec
tive here. Moreover, this ant navigation problem is classified as a GA
hard problem, especially if trail is not designed specially for ant population training. The efficiency of
transposons

in the role of intelligent mutators can be measured by their searc
h space domain confining ability.
Therefore, the selec
tion criteria inserted into MGE
-
operators had to increase the probability of the effective
navigation algorithm finding on the element of high complexity. We proposed, that quasi
-
periodical state
succes
sions without NOP (“no operation” instruction) and without internal cycles (in conditions, when an ant
have white elements ahead of it) will be picked up by selection and used for universal navigation algorithms
development.

A comparison of mutation frequ
encies in experiment and control with the according learning rates
confirm multiple reduction of evaluation numbers, needed for reaching of the same required learning in
experiments with
transposons
. Mutation frequencies for basic experiments (Fig.3) in co
ntrol accounts: crossover
rate + mutation rate = 0.0001+0.04 P/bit/generation; MGE1 and MGE2 operators add in average 0.0027 and
0.0075 P/bit/generation accordingly. In other words,
transposons in average add

to value 0.041 about 0.012
P/bit/generation. Th
is addition brings to multiple acceleration of ant population learning! Hence, according to
fig. 5, up to the end of the experiment (4622 time
-
step) the control set gives max score 6.47, whereas in the test
set this value is attained already on the 451 tim
e
-
step, i.e. 10 times sooner.

And what is more, though the ant test is now accepted as a classical one, its implementation in the form
of finite state automata is far from optimum. Firstly, the 32 states are obviously too much for the classical trails.
Thi
s redundancy brings to enormous extension of search space dimensionality. Secondly, the possibility to
generate various action sequences in the situation, when the ant faces with the black cell is redundant as well. It
is clear that the “step forward” is t
he only adequate action in this situation. Thirdly, multiple internal cycles,
essentially increasing the size of search space are hardly needed. At last, the action NOP is absolutely useless.



15

15

Apparently, the most simple and effective method of this navigat
ion problem solving would be using
the symbol strings, composed of combinations of three possible actions (FWD, RGT, and LFT) in the situation,
when the ant faces with t
he white cell. Applying the MGE
-
approach to the ant problem, we just perform such a
sea
rch space size reduction. In so doing, we deal only with the “white half” of search space which is formed by
all sequences of actions, executed while the ant has white cells in front of it. Besides we escape internal loops,
and exclude the NOP action. We s
uppose,
that mechanism explains the MGE
-
approach effect i.e., ten
-
fold
increase of evolutionary search rate. The additional
contributory factor for the MGE
-
technique efficiency raising
is inclusion of transposition procedure, application frequency of which

being made high in compare to the other
mutation operators.


Targeted

Mut
agenesis
Coupled

With
Horizontal Transmission

It is become common place that hybrid optimization approaches are more effective instrument for heuristic
search then solo techniques. I
n this regard it is remarkable that complicated mechanisms are thought to be the
main power of macroevolution has evident hybrid character.
In this paper we demonstrated that the targeted
mutagenesis of transposon sequence coupled with the transposon horiz
ontal transmission qualitatively improve
the GA search performance. High rate of the transmission is crucial for such improvement. As we mentioned
before, recent years some authors have been using several new algorithms for EC improvement owing from fresh
findings and ideas of modern evolutionary biology. For our approach is essential “populational ecological” as
well as co
-
evolutionary aspects of interactions jumping selfish mutators with chromosomes
-
solutions.
Co
-
evolutionary formation of the world of gen
omic parasites with own competition and cooperation and own
communication we suppose have very promising perspectives. In this regard, our long term goal is to find
conditions and rules of genomes


genomic parasites co
-
evolution. It seems quite reasonable

that in the long run
of evolution genomes


parasites interaction forms kind of natural

operation system
” facilitating

manipulations
with genomes
. The usage of this meta
-
language for genome rearrangements could explain fascinating efficacy of
biological
evolution that still beyond complete rational explanation.


REFERENCES

1.

Agrawal, Eastman & Schatz, Nature 394, 744


751, 1998
.

2.

Altenberg, L. The evolution of evolvability in genetic programming. P. 47
-
74 in K. E. Kinnear, ed.
Advances in Genetic Programm
ing. MIT Press, Cambridge, (1994).

3.

Brennan, P., (1994), ANT: Simulated Evolution on a PC, manuscript.

4.

Brosius J., 1991. Retroposons
-

Seeds of evolution.
Science 251, 753.

5.

Burke, D.S.; De Jong, K.A.; Grefenstette, J.J.; Ramsey, C.L.
and Wu, A. S.

(1998): P
utting more genetics
into genetic algorithms, Evolutionary Computation, 6:4, 387
-
410.

6.

Charlesworth B, Lapid A, Canada D, The distribution of transposable elements within and between
chromosomes in a population of Drosophila melanogaster. I. Element frequen
cies distribution.
Genet Res, 60: 103
-
114 (1992).

7.

Chellapilla K., Evolutionary programming with tree mutations: Evolving computer programs without
crossover. In John R. Koza, Kalyanmoy Deb, Marco Dorigo, David B. Fogel, Max Garzon,
Hitoshi Iba, and Rick L.

Riolo, editors,
Genetic Programming 1997: Proceedings of the Second
Annual Conference
, pages 431
--
438, Stanford University, CA, USA, 13
-
16 July 1997.
Morgan
Kaufmann.

8.

Cliff D. and Grand S. The
Creatures

Global Digital Ecosystem.
Artificial Life 5(1): 77
-
9
3 (1999)


9.

Daniels M., Integrating Simulation Technologies with Swarm, Agent Simulation: Applications, Models
and Tools, October 1999, Argonne National Laboratory, University of Chicago.

10.

Doolittle, W. F. and Sapienza C., 1980. Selfish genes, the phenotype
paradigm and genome evolution.
Nature 284: 601
-
603.

11.

Fadool, J. M., Hartl, D. L., and Dowling, J. E. (1998) Transposition of the mariner element from
Drosophila mauritiana in zebrafish.
Proc. Natl. Acad. Sci. USA 95, 5182
-
5186.

12.

Fogel L.J., Owens A.J., Walsh

M.J. Artificial Intelligence through simulated Evolution. J.Walley & Sons,
Inc., London, New York, Sidney, (1966).



16

16

13.

Harries K. and Smith P., Exploring alternative operators and search strategies in genetic programming. In
John R. Koza, Kalyanmoy Deb, Marc
o Dorigo, David B. Fogel, Max Garzon, Hitoshi Iba, and
Rick L. Riolo, editors,
Genetic Programming 1997: Proceedings of the Second Annual
Conference
, pages 147
--
155, Stanford University, CA, USA, 13
-
16 July 1997.
Morgan
Kaufmann.

14.

Harvey I., The microbial g
enetic algorithm, unpublished work (1996),
available at
ftp://ftp.cogs.susx.ac.uk/pub/users/inmanh/Microbe.ps.gz

15.

Hillis W.D., Co
-
evolving parasites improve simulated evolution as an optimization procedure, Physica D,
42:228
-
234, 1990.

16.

Hurst

&
Werren, 2001
,
Nat
ure Reviews Genetics 2, 597
-
606.

17.

Ito T., Iba I., and Sato S., Non
-
destructive depth
-
dependent crossover for genetic programming. In W.
Banzhaf, R. Poli, M. Schoenauer, and T. C. Fogarty, editors,
Proceedings of the First European
Workshop on Genetic
Programming
, LNCS, Paris, 14
-
15 April 1998.
Springer
-
Verlag.
Forthcoming.

18.

Jefferson, D.,
Collins R., Cooper C., Dyer M., Flowers M., Korf R., Taylor C. and Wang A.

Evolution as a
Theme in Artificial Life: The Genesys/Tracker System. In:
Artificial Life II,

SFI Studies in the
Sciences of Complexity, vol. X
, edited by C.G. Langton, C. Taylor, J.D. Farmer, and S.
Rasmussen. Addison
-
Wesley, 1991.

19.

De Jong K.A. and Potter M.A., Evolving complex structures via cooperative coevolution, In: Forth Annual
Conference o
n Evolutionary Computation, San Diego, CA, 1
-
3 March 1995.

20.

Keijzer M., Merelo J.J., Romero G., and Schoenauer M. (2001) Evolving Objects: A General Purpose
Evolutionary Computation Library. In: P. Collet, C. Fonlupt, J.
-
K. Hao, E. Lutton, M.
Schoenauer, e
ditors, Artificial Evolution: Springer Verlag Lecture Notes in Computer Science,
Number 2310, pp 231
-
244, Springer Verlag, http://geneura.ugr.es/~jmerelo/eo

21.

King, K. 199?

22.

Koza, J.R. Genetic Programming: on the Programming of Computers by Means of Natural S
election.
-

MIT Press, Cambridge, Mass.
-

1992.

23.

Kuscu I., Evolving a generalised behavior: Artificial ant problem revisited. In V. William Porto, editor,
Seventh Annual Conference on Evolutionary Programming
, Mission Valley Marriott, San Diego,
California,
USA, 25
-
27 March 1998.
Springer
-
Verlag.to be presented.

24.

Langdon W. B. and Poli R., Why ants are hard. In John R. Koza, Wolfgang Banzhaf, Kumar Chellapilla,
Kalyanmoy Deb, Marco Dorigo, David B. Fogel, Max H. Garzon, David E. Goldberg, Hitoshi
Iba, and Ric
k Riolo, editors,
Genetic Programming 1998: Proceedings of the Third Annual
Conference
, University of Wisconsin, Madison, Wisconsin, USA, 22
-
25 July 1998.
Morgan
Kaufmann. To be presented at GP
-
98.

25.

Lee J. Y. B. and Wong P. C., The effect of function noise

on GP efficiency. In X. Yao, editor,
Progress in
Evolutionary Computation
, volume 956 of
Lecture Notes in Artificial Intelligence
, pages 1
--
16.
Springer
-
Verlag, Heidelberg, Germany, 1995.

26.

Lozovskaya E.R., Hartl D.L. and Petrov D.A. Genomic regulation of t
ransposable elements in Drosophila.
Curr Opin genet Dev 5:768
-
73 (1995).

27.

Luke, S., Hamahashi, S.; Kitano, H. (1999), “Genetic” programming, GECCO
-
99: Proceedings of the
Genetic and Evolutionary Computation Conference, Banzhaf, W. et al, eds. San Fransisco:

Morgan Kaufmann.

28.

Luke S. and Spector L., A comparison of crossover and mutation in genetic programming. In John R.
Koza, Kalyanmoy Deb, Marco Dorigo, David B. Fogel, Max Garzon, Hitoshi Iba, and Rick L.
Riolo, editors,
Genetic Programming 1997: Proceedi
ngs of the Second Annual Conference
,
pages 240
--
248, Stanford University, CA, USA, 13
-
16 July 1997.
Morgan Kaufmann.

29.

Makalowski, W. SINEs as a Genomic Scrap Yard. Chap. 5 in
The Impact of Short Interspersed Elements
(SINEs) on the Host Genome
, edited by Ri
chard J. Maraia.
Austin: R.G. Landes Company, 1995.

30.

Makalowski, W.

2000. Gene, 259: 61
-
67.

31.

McClintock B, The significance of responses of the genome to challenge. Science, 226: 792
-
801 (1984).



17

17

32.

Potter M.A. and De Jong K.A., A cooperative co
-
evolutionary app
roach to function optimization, In: Third
Parallel Problem Solving from Nature, Jerusalem, Israel, 1994.

33.

Nawa, N. E.; Furuhashi, T.; Hashiyama, T.; Uchikawa, Y.: A study on the discovery of relevant fuzzy rules
using pseudo
-
bacterial genetic algorithms, IE
EE Transactions on Industrial Electronics, vol. 7,
no. 5, pp.608
-
616, October 1999.

34.

Orgel, L. E. and Crick F. H. C., 1980. Selfish DNA: The ultimate parasite.
Nature 284: 604
-
607.

35.

Olsson, B. (1996): Optimization using a host
-
parasite model with variable
-
si
ze distributed populations. In
Proceedings of the 1996 IEEE 3
rd

International Conference on Evolutionary Computation, 295
-
299. IEEE Press.

36.

Olsson, B. (2001): Co
-
evolutionary search in asymmetric spaces, In Wang, P.P., ed., Proceedings of The
Fifth Joint Co
nference on Information Sciences, 1040
-
1043.
Association for Intelligent
Machinery.

37.

Potter M.A. and De Jong K.A., Evolving neural networks with collaborative species, In: Proc. of the 1995
Summer Computer Simulation Conference, Ottawa, Ontario, Canada, 24
-
26 July 1995.

38.

Ray, T. S. 1991. An approach to the synthesis of life. In: Langton, C., C. Taylor, J. D. Farmer, & S.
Rasmussen [eds], Artificial Life II, Santa Fe Institute Studies in the Sciences of Complexity, vol.
XI, 371
-
408. Redwood City, CA: Addison
-
Wesley.

39.

Ray, T. S. 2001. Overview of Tierra at ATR. In: «Technical Information, No.15, Technologies for
Software Evolutionary Systems».
ATR
-
HIP. Kyoto, Japan.


40.

Shapiro
, J.A. 1999.
Transposable elements as the key to a 21st century view of evolution
, Gene
tica 107:
171

179.


41.

Shapiro
,
2002
,
J. Biol.
Phys.


42.

Simoes, A. and Costa, E. An evolutionary approach to the Zero/One knapsack problem: testing ideas from
biology; Procs. 5
th

Int. Conference on Artificial Neural Networks and Genetic Algorithms
(ICANNGA 2001
), Prague, Czech Republic, 22
-
25 April 2001.

43.

Spirov A.V. Self
-
Assemblage of gene Networks in Evolution via Recruiting of New Netters. Lecture Notes
in Computer Sciences 1141: 91
-
100 (1996a).

44.

Spirov A.V. Self
-
organisation of gene networks in evolution via r
ecruiting of new netters. Pp 399
-
405, In
Proceedings of the First International Conference on Evolutionary Computations and Its
Applications, Moscow, Russia, (1996b).

45.

Spirov A.V. and Samsonova M.G., Strategy of Co
-
evolution of Transposons and Host Genome:
Application to Evolutionary Computations. Proceedings of the Third Nordic Workshop on
Genetic Algorithms and their Applications (3NWGA), 20
-

22 August 1997, Helsinki, Finland,
Ed. Jarmo T. Alander, Finnish Artificial Intelligence Society, 71
-
82, (1997).

46.

S
pirov A.V., Kadyrov A.S. Transposon Element Technique Applied to GA
-
based John Muir’s Trail Test,
In: High
-
Performance Computing and Networking, (1998), 925
-
928.

47.

Spirov A.V., Kazansky A.B. and

Kadyrov A.S. Utilizing of “Parasitic” Mobile Genetic Elements
in
Genetic Algorithms. In: International Conference on Soft Computing and Measurments, St.
-
Petersburg, (1998), 266
-
269.

48.

Spirov A.V. and Kazansky A.B., Evolutionary Biology and Evolutionary Computations: Parasitic Mobile
Genetic Elements in
Artificial

Evolu
tion, In: 2
nd

Int. Conf. on Soft Computing and
Measurements
, St.
-
Petersburg, 1999.

49.

Spirov A.V. and Kazansky A.B., The usage of artificial transposons for the

tests with royal road functions,
5th Int. Conf. on Soft Computing and

Measurments, SCM2002, June,
2002
a
, St.Petersburg,
Russia

50.

Spirov A.V. and Kazansky A.B.,

2002
d
.

Jumping genes
-
mutators can
raise

efficacy of

evolutionary search,
Genetic and Evolutionary Computation Conference,

GECCO2002, July 9
-
13, New York.

51.

Spirov A.V. and Kazansky A.B.
,

2002
c
.

The
usage of artificial transposons for the

protection of already
found building blocks: the tests with royal road

functions, The 6th World Multiconference on
Systemics, Cybernetics and

Informatics, SCI2002, July 14
-
18, Orlando, Florida.



18

18

52.

Zongker, D. and Punch,

B., (1955), lil
-
gp 1.0,
http://isl.cps.msu.edu/GA/software/lil
-
gp





19

19

A.






20

20


B.


State

Input=0

Input=1

00

FWD/0A

NOP/09

01

RGT/0E

FWD/03

02

NOP/08

NOP/0E

03

FWD/13

RGT/18

04

NOP/17

NOP/0B

0
5

RGT/17

RGT/0A

06

FWD/04

NOP/09

07

LFT/0A

LFT/17

08

LFT/12

FWD/1E

09

RGT/1E

RGT/16

0A

RGT/16

FWD/06

0B

RGT/13

NOP/16

0C

LFT/03

LFT/0B

0D

LFT/0E

NOP/14

0E

LFT/0C

NOP/12

0F

RGT/15

FWD/1F

10

NOP/0E

LFT/17

11

FWD/12

FWD/0F

12

LFT/11

NOP/0C

13

RG
T/02

NOP/1D

14

RGT/0C

LFT/0E

15

FWD/18

FWD/09

16

FWD/01

NOP/08

17

NOP/0B

LFT/1A

18

LFT/13

NOP/11

19

NOP/0D

RGT/01

1A

NOP/1E

LFT/1B

1B

FWD/03

FWD/10

1C

RGT/0A

NOP/00

1D

RGT/06

LFT/0A

1E

RGT/0C

NOP/18

1F

RGT/10

FWD/04


Figure 1. (A). The John Mu
ir Trail (According to Brennan, 1994). Some “milestone” squares are numbered. The trail itself is
a series of black squares on a 32x32 white toroidal (i.e., wraparound) grid. Each black cell is numbered sequentially, from t
he
1
st

to the 89
th
.

(B). An exam
ple of organization of an ant as finite state automata. There are 32 states at all. Each state determines
two alternate actions, depending on input signal. The input signal is what an ant sees before him. If the cell before him is
black then the input is 1
, in opposite case the input is 0. Each of alternative actions includes one of four possible movements
(FWD, RGT, LFT or NOP) and transition to the next state. (See text for further details.)



21

21

A.


The

additional string 1 1 1 1 1 1
0 0 0 0

The main str
ing
L F F L L #0
* * * *



------------


transposon



B.

State

Input=0

0

LFT/#17

17

FWD/#13

13

FWD/#21

21

LFT/#9

9

LFT/#0



C.

The

additional string 1 1 1 1 1
0 0 0 0

The main string
L

F F L N
* * * *



---------


transposon




D
.

State

Input=0

0

LFT/#17

17

FWD/#13

13

FWD/#21

21

LFT/#9

9

NOP/#
3
0




Figure 2
.

Mature and immature transposons.

A. Here is an example of mature
transposon
.
The
transposon

is a closed five
-
element cycle of states transitions (0, 17, 13, 21,
9, and again, 0).

B. The state table corresponding to the transposon (A).

C
. Here is an example of immature
transposon
. It dif
fers from the previous one
by the presence o
f procedure NOP in the last,
fifth line

and

this sequence is not

cycle
.

D. The state table corresponding to the transposon (C).











22

22




Figure 3.
Ant trail used in our computer experiments. The trail itself is a series of squares on a 32x32 white
toro
idal grid. Each cell is numbered sequentially, from the 1
st

to the 89
th
.
The first two gaps of the higher
complexity are between 64
th

and 65
th

and 67
th

and 68
th
.




23

23




Figure 4. Numerical experiments, demonstrating statistically certain increasing of the G
A efficiency due to the effect of
MGE
-
operators. A comparison of the mean and the best
-
of
-
generation score dynamics (MGE
-
operator being activated) with
the

control (MGE
-
operator is disabled). The score values are averaged over 100 runs in both cases.

The s
ize of population = 100; the number of generations = 5000; the
sequence

size varies from 5 to 11; crossover
rate (P/bit)/generation = 0.0001; mutation rate (P/bit)/generation = 0.04;
I

are the best
-
of
-
generation scores and
II

are the
mean scores for the te
st runs;
III

are the best
-
of
-
generation scores and
IIII

are the mean scores for the control runs.





24

24





Figure 5. An example of mobile genetic elements size population dynamics inside gene pool of the host (ant) population (the
first 500 generations). T
he curve of maximum score dynamics for ant population is given at the top of the diagram (marked
off with squares; the additional score scale is given on the left of the diagram). The moment of finding of the first effecti
ve
navigation algorithm and the mo
ment of changing of dominant form of
transposon

are marked off with the edges of arrows
(see details in the text).



25

25




Figure 6. Comparison of ant populations learning abilities in dependence on
transposon

length. The curves of
maximum and mean score dyna
mics (the parameters of
sequence

size lies in the range of 5
-

32), averaged over
100 computer runs are compared with the corresponding curves, presented on fig.3, where parameter of
transposon

sequence

size lies between 5 and 11. The values of all other p
arameters are the same as in the
caption to Fig. 3.
I

are the best
-
of
-
generation scores and
II

are the mean scores for the runs for large
transposon
s;
III

are the best
-
of
-
generation scores and
IIII

are the mean scores for the runs from the Fig. 3.



26

26




Fig
ure 7. The influence of decreasing of frequen
cy of applying of two
-
place MGE
-
operator on the ant learning
abilities. The other parameters are the same as in the previous experiments (see caption to Fig. 3).
I

are the best
-
of
-
generation scores and
II

are th
e mean scores for the test runs from the Fig. 3;
III

are the best
-
of
-
generation
scores and
IIII

are the mean scores for the runs with disable MGE
-
algorithm.



27

27




Figure 8. The dynamics of quotas (in %) of individuals in ant population, subjected to the act
ion

of one
-
place
and two
-
place MGE
-
operators (MGE 1 and MGE 2, correspondingly).



28

28

Table 1. Examples of ant’ paths determined by
transposon
s.


R

is turn right,
L

is turn left,
>

is step
forward
.