Exploiting
Randomness
Evolution and Genetic Algorithms
Genetic Algorithms
Building blocks and recombination
A Biological Metaphor
Genetic Algorithms
Operations
Crossover
Exchange genetic material between two individuals
Mutation
Randomly change part of the genetic material
Selection
The fittest individuals have the best chance of reproducing
ACTGCCGTCGTCGAAACGCGTAATTTCCG
Operations
Selection (reproduction): favors fittest strings
Crossover: provides way to simultaneously explore and exploit
Mutation: helps prevent development of uniform population that stagnates at less than
optimum fitness
Strengths of Evolutionary Approaches
to Problem Solving
massive parallelism
adaptability
innovation
skirting complex algorithms
balancing exploitation and exploration
(cf. Mitchell)
Luger: Artificial Intelligence, 5
th
edition. © Pearson Education Limited, 2005
Genetic algorithms visualized as parallel hill climbing, adapted from Holland (1986).
A Simple GA
Initialize population with
n
randomly generated
chromosomes of size
m
.
Repeat until fitness
a specified value or a
specified # of generations has been obtained
Compute fitness of each member of the population
Repeat until
n
offspring have been created
Probabilistically select two parents from current
population (based on fitness)
Probabilistically perform cross

over at random locations
(generating two new individuals)
Probabilistically mutate each offspring at random
locations
Replace current population with new population
Fitness

Proportionate Selection
An individual with fitness
z
should have a
reproductive probability of
z / t,
where
t
is the
total fitness of all members of the current
population.
Can implement via roulette

wheel sampling.
Compute normalized fitness (FN
i
) for each member of population and put
results in fitness array.
FN
i
= F
i
/SF (where SF is the sum of all fitness values)
Compute cumulative normalized fitness (FC
i
) for each member of
population and put results in fitness array.
FC
0
= FN
0
FC
i
= FC
i

1
+ FN
i
(for
i
> 0)
Select members to be reproduced, proportional to their fitness
Generate P random numbers R
k
For each random number R
k
, find the member
i
such that FC
i

1
< R
k
≤ FC
i
cf. Kennedy and Eberhart, 2001
GA Example
An airline company has a certain number of items
to fly from Jakarta to Singapore. Each item has a
specific weight and value (corresponding to the
profit they will bring when they reach their
destination). Unfortunately, the airplane being
used can only carry a limited amount of cargo (by
weight). The airline must, therefore, determine
which items to carry so as not to exceed the
airplane’s capacity but also so as to maximize its
profit. Design and implement a genetic algorithm
solution to this problem.
Airline
Problem
Consider a collection of items from which to
choose (for shipment not to exceed
W
pounds):
Item
Weight
Value
1
5.0
15.0
2
10.0
20.0
3
8.0
12.0
4
7.0
13.0
5
12.0
8.0
6
15.0
17.0
7
4.0
10.0
8
11.0
5.0
9
3.0
8.0
10
9.0
9.0
Total
84.0
117.0
The 0

1 Knapsack
Problem
(a naïve solution: select items on basis of value/weight)
http://www

cse.uta.edu/~holder/courses/cse2320/lectures/l15/node11.html
http://xkcd.com/287/
What factors must be considered when
designing a GA approach to this problem?
Some factors to consider
Data representation (encoding: what, how, binary or decimal, etc.)
How to do cross

over
Whether to do mutation (and if so, how to do it)
Fitness evaluation
Population size and limit
Number of births per generation
How to determine when to quit (# generations; fitness ceiling; etc.)
Control parameters
Processing logic
What factors must be considered when
designing a GA approach to this problem?
Airline Problem via Genetic Algorithm
Encode a possible shipment of
N
items as
an
N

bit string
Bit position
i
represents the selection (1) or
omission (0) of item
i
.
Example:
For 10 items, the string 0110000010 indicates
that the second, third, and ninth items are
being considered
for
shipment
.
DEMO
The Traveling Salesman Problem
What is the shortest route a salesman can
take to visit all of the cities in his territory
and return home?
The Traveling Salesman Problem
What is the shortest route a salesman can
take to visit all of the cities in his territory
and return home?
The (Real) Traveling Salesman “Problem”
As the number of cities increases, the time it takes
to find an exact solution increases exponentially.
Example:
Number of cities
# paths
Time to solve (on a fast PC)
8
2520
almost instantaneously
10
181,440
1 second
12
20 million
20 seconds
20
60,800,000,000,000,000
?
100
4.67 x 10
157
?
Applying a Genetic Algorithm to the Traveling
Salesman Problem
Encode a population of paths
AFDHBGEJCI
fitness=.8
EBDGAFJICH
fitness=.4
…
DGEIAFBJHC
fitness=.6
Select (based on fitness)
CAGBJEIDHF
IDJGEBFHAC
Crossover
CA
GBJE
IDHF
→
CA
JGEB
IDHF
ID
JGEB
FHAC
→
ID
GBJE
FHAC
Mutate
I
D
GBJEF
H
AC
→
I
H
GBJEF
D
AC
Repeat for many
“generations”
Applying a Genetic Algorithm to the
Traveling Salesman Problem
Solution is the most fit individual after some specified
number of generations: e.g., ACDEFIJGHB
(i.e., GA prediction of the best route)
Key: The solution is obtained relatively quickly.
Demo
Determinism and randomness can interact
productively
Genetic algorithms
illustrate interplay between
determinism and randomness
during search (exploration
and exploitation)
Insights from the TSP
We used a biological metaphor (natural
selection) to construct the algorithm to
solve this problem.
Determinism and randomness can
interact productively
The TSP is itself a metaphor for many
other problems of interest.
Insights from
the TSP
(continued)
Insights from the TSP (continued)
A non

rooted phylogenetic tree
Multiple Sequence Alignment
A
B
C
D
A
0
2
1
3
B
2
0
5
3
C
1
5
0
1
D
3
3
1
0
Inter

sequence
distances
(one approach)
Initialization
1. create G
0
Evaluation
2. evaluate the population of generation n (G
n
)
3. if the population is stabilized then END
4. select the individuals to replace
5. evaluate the expected offspring (EO)
Breeding
6. select the parent(s) from G
n
7. select the operator
8. generate the new child
9. keep or discard the new child in G
n+1
10. goto 6 until all the children have been successfully put into G
n+1
11. n = n+1
12. goto EVALUATION
End
13. end
SAGA: Sequence Alignment by Genetic Algorithm
(Notredame and Higgins, 1996)
http://www.tcoffee.org/Publications/Ps_pdf/saga_paper.pdf
SAGA
(continued)
Notredame and Higgins
backwards
Mount
SAGA (continued)
Notredame and Higgins
SAGA (continued)
Notredame and Higgins
SAGA
(continued)
Notredame and Higgins
SAGA
(continued)
Notredame and Higgins
SAGA
(continued)
Notredame and Higgins
SAGA
(continued)
Notredame and Higgins
Comments 0
Log in to post a comment