Basics of Genetic Programming

cathamAI and Robotics

Oct 23, 2013 (3 years and 10 months ago)

83 views


Introduction


What is Genetic Programming?


Genetic Operators


Mutation


Crossover


How does it work?


What is Genetic Algorithm?


Genetic Algorithm


Sequence of steps


When to stop?


Mathematical Tools


Crossover Operation


Mutation


Symbolic Regression Problem


Advantages of GP


Advantages of GA


Application Fields of GP


Application Fields of GA


Our Example


Conclusion




A

genetic

programming

system

is

not

only

a

problem

solving

system

but

also

an

automatic

programming

system
.

We

provide

the

system

with

information

that

describes

what

we

want

a

program

to

do,

and

successful

run

of

the

system

produces

a

program

that

meets

the

desired

specification
.

We

must

also

provide

values

for

a

few

other

parameters,

but

most

of

this

is

straightforward

and

requires

little

expertise

about

the

problem

we

are

using

the

system

to

solve
.




In

this

study,

the

aim

is

to

using

Genetic

Programming

makes

easy

to

solve

problem

which

is

the

hardest
.

Also,

we

can

use

Genetic

Algorithm

which

is

the

kind

of

Genetic

Programming

for

solving

these

problems
.



Genetic

Programming

(GP)

is

a

method

of

solving

problems

using

computers

through

an

analogue

of

natural

selection
.

It

is

a

way

to

have

computers

automatically

solve

problems,

without

having

to

define

or

even

know

the

form

or

structure

of

the

optimal

solution

ahead

of

time
.


You

simply

provide

genetic

programming

with

basic

building

blocks

of

the

solution

and

some

method

of

analyzing

how

well

a

proposed

solution

solves

the

problem

and

GP

does

the

rest
.


Mutation



Mutation

affects

an

individual

in

the

population
.

It

can

replace

a

whole

node

in

the

selected

individual,

or

it

can

replace

just

the

node's

information
.

To

maintain

integrity,

operations

must

be

fail
-
safe

or

the

type

of

information

the

node

holds

must

be

taken

into

account
.

For

example,

mutation

must

be

aware

of

binary

operation

nodes,

or

the

operator

must

be

able

to

handle

missing

values
.


Crossover



Crossover

is

applied

on

an

individual

by

simply

switching

one

of

its

nodes

with

another

node

from

another

individual

in

the

population
.

With

a

tree
-
based

representation,

replacing

a

node

means

replacing

the

whole

branch
.

This

adds

greater

effectiveness

to

the

crossover

operator
.

The

expressions

resulting

from

crossover

are

very

much

different

from

their

initial

parents
.



Genetic

algorithms

are

a

very

effective

way

of

quickly

finding

reasonable

solution

to

a

complex

problem
.

Granted

they

aren't

instantaneous,

or

even

close,

but

they

do

an

excellent

job

of

searching

through

a

large

and

complex

search

space
.

Genetic

algorithms

are

most

effective

in

a

search

space

for

which

little

is

known
.

You

may

know

exactly

what

you

want

a

solution

to

do

but

have

no

idea

how

you

want

it

to

go

about

doing

it
.

This

is

where

genetic

algorithms

thrive
.

They

produce

solutions

that

solve

the

problem

in

ways

you

may

never

have

even

considered
.


Then

again
,

they

can

also

produce

solutions

that

only

work

within

the

test

environment

and

flounder

once

you

try

to

use

them

in

the

real

world
.

Put

simply
:

use

genetic

algorithms

for

everything

you

cannot

easily

do

with

another

algorithm
.


Sequence of
steps


1.
Input
: A generation of individuals.

2.
Calculation
of misfit and selection.

3.
Reproduction
(cross
-
over + mutation) = New population

4.
Go
to
(1)
with a new
generation




When
to stop?



Misfit

(a minimum
criteria
)



Number of generations (
maximum

reached
)


Allocated

budget reached (time,
money,
etc
.)



Solution

reached

a
plateau


Combinations

of
previous

criteria



Crossover Operation




Two

primary

operations

exist

for

modifying

structures

in

genetic

programming
.

The

most

important

one

is

the

crossover

operation
.

In

the

crossover

operation,

two

solutions

are

sexually

combined

to

form

two

new

solutions

or

offspring
.

The

parents

are

chosen

from

the

population

by

a

function

of

the

fitness

of

the

solutions
.

Three

methods

exist

for

selecting

the

solutions

for

the

crossover

operation
.


The

first

method

uses

probability

based

on

the

fitness

of

the

solution
.

If

is

the

fitness

of

the

solution

Si

and


Crossover Operation



is
the

total
sum

of
all

the

members

of
the

population
,
then

the

probability

that

the

solution

Si
will

be
copied

to

the

next

generation

is:











Another method for selecting the solution to be copied is
tournament selection. Finally, the last method is done by rank. In
rank selection, selection is based on the rank, of the fitness
values of the solutions of the population.


Mutation


Mutation is another
important feature of
genetic programming.
Two types of mutations
are possible. In the first
kind a function can only
replace a function or a
terminal can only
replace a terminal. In the
second kind an entire
subtree can replace
another subtree.


GP does not impose any fixed length of the solution.
In principle, the maximal length can be extended up
to the hardware limits.


GP does not require as much knowledge about the
problem and the possible solutions as do GAs. These
decisions are crucial for the power of GAs in solving a
problem and they usually require a lot of domain
knowledge.


Using GP, you can theoretically evolve any series of
actions a computer can possibly do, provided that
you give the GP algorithm a set of commands to
choose from that can describe all possible actions.

Advantages of Genetic Algorithm are the following:



They efficiently search the model space, so they are
more likely to converge toward a global minima.


There is no need of linearization of the problem.


There is no need to compute partial derivatives.


More probable models are sampled more frequently
than less probable ones.


Symbolic Regression and
Function
Synthesis


(Performing
symbolic
regression)


Grammar Induction


Data Mining and Data
Analysis


(
Data mining and
classification)


Electrical Engineering and
Circuit Design


(
Designing analog circuits)


Medicine


Economics and Finance


Geometry and Physics


Cellular Automata and Finite
State Machines




Automated Programming



Robotics


(Optimising robot control)



Networking and
Communication


(Evolving
sorting

networks
)



Evolving Behaviors for Agents
or


Game Players



Pattern Recognition



Biochemistry



Machine Learning



Discovery of quantum
algorithms



Image processing



Creating security protocols


Automotive Design


Engineering Design


Robotics


Evolvable Hardware


Optimized
Telecommunications Routing


Joke and Pun Generatio



Trip,Traffic and Shipment
Routing


Finance and Investment
Strategies




Biomimetic

Invention


Computer

Gaming


Encryption

and
Code

Breaking


Computer
-
Aided

Molecular

Design


Gene
Expression

Profiling


Optimizing

Chemical
Kinetic

Analysis


Marketing and Merchandising





Genetic programming is much more powerful than
genetic algorithms. The output of the genetic algorithm
is a quantity, while the output of the genetic
programming is a another computer program. In
actuality, this is the beginning of computer programs
that program themselves.



Genetic programming works best for several types of
problems. The first type is where there is no ideal solution,
(for example, a program that drives a car). In this case
genetic programming will find a solution that attempts to
compromise and be the most efficient solution from a
large list of variables.




Furthermore, genetic programming is useful in finding
solutions where the variables are constantly changing.


What is Genetic Programming,
http://www.geneticprogramming.us/What_is_Genetic_Programming.ht
ml


Lee Spector, 2004. Automatic Quantum Computer Programming,Kluwer
Academic


Publishers


Mutation, http://
en.wikipedia.org/wiki/Genetic_programming#mutation


Crossover,
http://
en.wikipedia.org/wiki/Genetic_programming#Crossover


Applications, http://
geneticalgorithms.ai
-
depot.com/Tutorial/Overview.html


Sequence of Steps,
http
://www.colorado.edu/geolsci/courses/GEOL6670/monsalve_GenAl
g.pdf


When to stop?,
http
://
www.colorado.edu/geolsci/courses/GEOL6670/monsalve_GenAl
g.pdf


Thomas Weise, 2009. Global Optimization Algorithms


Theory and
Application


David
R. White & Simon Poulding, 2009. A Rigorous Evaluation of
Crossover and


Mutation
in Genetic Programming, University of York,
UK


15 Real World Uses of Genetic Algorithms, http://brainz.org/15
-
real
-
world
-
applications
-
genetic
-
algorithms/



Koza, John R. 1992. Genetic Programming: On the Programming of
Computers by Means of Natural Selection. Cambridge, MA: The MIT
Press


Cramer, Nichael Lynn: "A Representation for the Adaptive
Generation of Simple Sequential Programs", Proceedings,
International Conference on Genetic Algorithms and their
Applications, July 1985 [CMU], pp183
-
187


Tree
-
based Genetic Programming,
http://en.wikipedia.org/wiki/Genetic_programming#Crossover


Genetic Programming,
http://www.ooop.org/publications/thesis/node8.html


Genetic Algorithm,


http://www.colorado.edu/geolsci/courses/GEOL6670/monsalve_Ge
nAlg.pdf