Genetic Algorithms, Search Algorithms
Jae C. Oh
Overview
Search Algorithms
Learning Algorithms
GA
Example
Brief History
Evolutionary Programming
Fogel in 1960s
Individuals are encoded to be finite state
machines
Intellgent Behavior
Evolutionary Strategies
Rechenberg, Schwefel in 1960s
Real

valued parameter optmization
Genetic Algorithms
Holland in 1960s
Adaptive Systems
Crossover Operators
Current Status
Wide variety of evolutionary algorithms
No one seriously tries to distinguish them
except for some cases and purposes.
We will call all Evolutionary Algorithms
And I will call them Genetic Algorithms or
Evolutionary Algorithms for generic terms
Search
Search
Search
Notion of Search Space
Real world problem
Search space
Abstraction

> State Space
Exploring the state space for given
problem
Search Algorithms
The Peak
Search Space
Learning Algorithms
Finding (through search) a suitable
program, algorithm, function for a given
problem
Learning Algorithm
Training Data
(Experience)
Program
Learning Algorithms (function
Optimizations)
Problem instance
Set of Hypothesis
Hypothesis Space
Program Space
Function Space
The One??
Learning Algorithms (Digression)
How do we know the found hypothesis,
program, function, etc. are
the one
we
are looking for?
We don’t know for sure
Is there any mathematical way of telling
how good hypothesis is?
I.e., h(x)
–
f(x) = ?
Computational Learning Theory can tell us
this
Valiant (1984)
What are Genetic Algorithms?
Find solutions for a problem with the idea of
evolution.
Search
and
optimization
techniques
based on Darwin’s Principle of
Natural
Selection
.
Randomized
search and optimization algorithms
guided by the principle of Darwin’s natural selection:
Survival of fittest.
Evolve potential solutions
Step

wise refinement?
Mutations? Randomized, parallel search
Models natural selection
Population based
Uses fitness to guide search
Evolution is a search process
From the Tree of the Life Website,
University of Arizona
Orangutan
Gorilla
Chimpanzee
Human
Evolution is parallel search
AAGACTT
TG
GACTT
AAG
G
C
C
T
A
G
GGC
A
T
T
AG
C
CCT
A
G
C
ACTT
TAGCCC
A
TAG
A
C
T
T
AGC
G
CTT
AGCAC
AA
AGGGCAT
AGGACTA
T
GGACTA
AG
T
GAC
C
A
AG
G
G
CA
A
CAGCA
CCA
A
G
C
ACTA
TC
GCCCA
T
AG
GC
C
T
A
AG
TG
CTA
AG
T
AC
A
A
A
A
GGCAA
Genetic Algorithm Overview
1.
Starting with a subset of
n
randomly chosen
solutions ( )from the search space (i.e.
chromosomes). This is the
population
2.
This population is used to produce a next
generation
of individuals by reproduction
3.
Individuals with a higher
fitness
(

)have
more chance to reproduce (i.e. natural selection)
GA in Pseudo code
0 START
: Create random population of
n
chromosomes
1 FITNESS :
Evaluate fitness
f(x)
of each chromosome in
the population
2 NEW POPULATION
0 SELECTION :
Based on
f(x)
1 RECOMBINATION
:
Cross

over chromosomes
2 MUTATION :
Mutate
chromosomes
3 ACCEPTATION :
Reject or accept new one
3 REPLACE :
Replace old with new population: the new
generation
4 TEST :
Test problem criterium
5 LOOP :
Continue step 1
–
4 until criterium is
satisfied
GA vs. Specialized Alg.
Genetic Algorithms (GAs)
Problems
P
Specialized Algo.
GA
Specialized algorithms
–
best performance for special problems
Genetic algorithms
–
good performance over a wide range of problems
Randomized Algorithms
Guided random search technique
Uses the payoff function to guide
search
Global optima
local
optima
Hill Climbing
Evolutionary Algorithms?
Search Algorithms?
Learning Algorithms?
Function Optimization Algorithms?
They are fundamentally the same!!
Things needed for GAs
How do we represent individuals? Domain
Dependent
How do we interpret individuals?
Domain Dependent
What is the fitness function?
Domain Dependent
How are individual chosen for reproduction?
Choose better individuals (probabilistic)
How do individuals reproduce?
Crossover, Mutation, etc.
How is the next generation generated?
Replace badly performing individuals
Encoding Methods
11111110000000011111
Chromosome
B
10110010110011100101
Chromosome A
8
5
6
7
2
3
1
4
9
Chromosome B
1
5
3
2
6
4
7
9
8
Chromosome A
(left), (back), (left), (right), (forward)
Chromosome
1.235
5.323
0.454
2.321
2.454
Chromosome
Binary Encoding/Ternary Encoding
Permutation Encoding (TSP)
Real numbers, etc. Specialized
Fitness Function
A fitness function quantifies the optimality of a
solution (chromosome) so that that particular
solution may be ranked against all the other
solutions.
A fitness value is assigned to each solution depending
on how close it actually is to solving the problem.
Ideal fitness function correlates closely to goal +
quickly computable.
Example. In TSP, f(x) is sum of distances between
the cities in solution. The lesser the value, the fitter
the solution is
Producing Offspring
The process that determines which
solutions are to be preserved and allowed
to reproduce and which ones deserve to
die out.
The primary objective of the recombination
operator is to
emphasize the good solutions
and
eliminate the bad solutions
in a population,
while
keeping the population size constant.
“Selects The Best, Discards The Rest”.
Roulette Wheel Selection
1
2
3
4
Spin
Chromosome #
Fitness
1
15.3089
2
15.4091
3
4.8363
4
12.3975
Strings that are fitter are assigned a larger slot and hence
have a better chance of appearing in the new population.
GA in Action for 8

Queen
Fitness for 8

Queen?
Minimum conflict fitness function.
Theory (Schema Theorem)
Schema
Substring where some positions left
undecided
246*****
Instance of this schema: 24613587
Theorem: if the average of the instances
the schema is above the mean fitness of
the population, the number of instances of
the schema will increase over time.
Applications
Many many…
VLSI, TSP, Function Optimization, Data
mining, security, etc.
Comments 0
Log in to post a comment