# Genetic Algorithms

AI and Robotics

Oct 24, 2013 (4 years and 6 months ago)

165 views

Exploiting
Randomness

Evolution and Genetic Algorithms

Genetic Algorithms

Building blocks and recombination

A Biological Metaphor

Genetic Algorithms

Operations

Crossover

Exchange genetic material between two individuals

Mutation

Randomly change part of the genetic material

Selection

The fittest individuals have the best chance of reproducing

ACTGCCGTCGTCGAAACGCGTAATTTCCG

Operations

Selection (reproduction): favors fittest strings

Crossover: provides way to simultaneously explore and exploit

Mutation: helps prevent development of uniform population that stagnates at less than
optimum fitness

Strengths of Evolutionary Approaches
to Problem Solving

massive parallelism

innovation

skirting complex algorithms

balancing exploitation and exploration

(cf. Mitchell)

Luger: Artificial Intelligence, 5
th

edition. © Pearson Education Limited, 2005

Genetic algorithms visualized as parallel hill climbing, adapted from Holland (1986).

A Simple GA

Initialize population with
n

randomly generated
chromosomes of size
m
.

Repeat until fitness

a specified value or a
specified # of generations has been obtained

Compute fitness of each member of the population

Repeat until
n

offspring have been created

Probabilistically select two parents from current
population (based on fitness)

Probabilistically perform cross
-
over at random locations
(generating two new individuals)

Probabilistically mutate each offspring at random
locations

Replace current population with new population

Fitness
-
Proportionate Selection

An individual with fitness
z

should have a
reproductive probability of
z / t,

where
t

is the
total fitness of all members of the current
population.

Can implement via roulette
-
wheel sampling.

Compute normalized fitness (FN
i
) for each member of population and put
results in fitness array.

FN
i

= F
i
/SF (where SF is the sum of all fitness values)

Compute cumulative normalized fitness (FC
i
) for each member of
population and put results in fitness array.

FC
0

= FN
0

FC
i

= FC
i
-
1

+ FN
i

(for
i

> 0)

Select members to be reproduced, proportional to their fitness

Generate P random numbers R
k

For each random number R
k
, find the member
i

such that FC
i
-
1

< R
k

≤ FC
i

cf. Kennedy and Eberhart, 2001

GA Example

An airline company has a certain number of items
to fly from Jakarta to Singapore. Each item has a
specific weight and value (corresponding to the
profit they will bring when they reach their
destination). Unfortunately, the airplane being
used can only carry a limited amount of cargo (by
weight). The airline must, therefore, determine
which items to carry so as not to exceed the
airplane’s capacity but also so as to maximize its
profit. Design and implement a genetic algorithm
solution to this problem.

Airline
Problem

Consider a collection of items from which to
choose (for shipment not to exceed
W

pounds):

Item

Weight

Value

1

5.0

15.0

2

10.0

20.0

3

8.0

12.0

4

7.0

13.0

5

12.0

8.0

6

15.0

17.0

7

4.0

10.0

8

11.0

5.0

9

3.0

8.0

10

9.0

9.0

Total

84.0

117.0

The 0
-
1 Knapsack
Problem

(a naïve solution: select items on basis of value/weight)

http://www
-
cse.uta.edu/~holder/courses/cse2320/lectures/l15/node11.html

http://xkcd.com/287/

What factors must be considered when
designing a GA approach to this problem?

Some factors to consider

Data representation (encoding: what, how, binary or decimal, etc.)

How to do cross
-
over

Whether to do mutation (and if so, how to do it)

Fitness evaluation

Population size and limit

Number of births per generation

How to determine when to quit (# generations; fitness ceiling; etc.)

Control parameters

Processing logic

What factors must be considered when
designing a GA approach to this problem?

Airline Problem via Genetic Algorithm

Encode a possible shipment of
N

items as
an
N
-
bit string

Bit position
i

represents the selection (1) or
omission (0) of item
i
.

Example:

For 10 items, the string 0110000010 indicates
that the second, third, and ninth items are
being considered
for
shipment
.

DEMO

The Traveling Salesman Problem

What is the shortest route a salesman can
take to visit all of the cities in his territory
and return home?

The Traveling Salesman Problem

What is the shortest route a salesman can
take to visit all of the cities in his territory
and return home?

The (Real) Traveling Salesman “Problem”

As the number of cities increases, the time it takes
to find an exact solution increases exponentially.

Example:

Number of cities

# paths

Time to solve (on a fast PC)

8

2520

almost instantaneously

10

181,440

1 second

12

20 million

20 seconds

20

60,800,000,000,000,000

?

100

4.67 x 10
157

?

Applying a Genetic Algorithm to the Traveling
Salesman Problem

Encode a population of paths

AFDHBGEJCI

fitness=.8

EBDGAFJICH

fitness=.4

DGEIAFBJHC

fitness=.6

Select (based on fitness)

CAGBJEIDHF

IDJGEBFHAC

Crossover

CA
GBJE
IDHF

CA
JGEB
IDHF

ID
JGEB
FHAC

ID
GBJE
FHAC

Mutate

I
D
GBJEF
H
AC

I
H
GBJEF
D
AC

Repeat for many
“generations”

Applying a Genetic Algorithm to the
Traveling Salesman Problem

Solution is the most fit individual after some specified
number of generations: e.g., ACDEFIJGHB

(i.e., GA prediction of the best route)

Key: The solution is obtained relatively quickly.

Demo

Determinism and randomness can interact
productively

Genetic algorithms

illustrate interplay between

determinism and randomness

during search (exploration

and exploitation)

Insights from the TSP

We used a biological metaphor (natural
selection) to construct the algorithm to
solve this problem.

Determinism and randomness can
interact productively

The TSP is itself a metaphor for many
other problems of interest.

Insights from
the TSP
(continued)

Insights from the TSP (continued)

A non
-
rooted phylogenetic tree

Multiple Sequence Alignment

A

B

C

D

A

0

2

1

3

B

2

0

5

3

C

1

5

0

1

D

3

3

1

0

Inter
-
sequence
distances

(one approach)

Initialization

1. create G
0

Evaluation

2. evaluate the population of generation n (G
n
)

3. if the population is stabilized then END

4. select the individuals to replace

5. evaluate the expected offspring (EO)

Breeding

6. select the parent(s) from G
n

7. select the operator

8. generate the new child

9. keep or discard the new child in G
n+1

10. goto 6 until all the children have been successfully put into G
n+1

11. n = n+1

12. goto EVALUATION

End

13. end

SAGA: Sequence Alignment by Genetic Algorithm

(Notredame and Higgins, 1996)

http://www.tcoffee.org/Publications/Ps_pdf/saga_paper.pdf

SAGA

(continued)

Notredame and Higgins

backwards

Mount

SAGA (continued)

Notredame and Higgins

SAGA (continued)

Notredame and Higgins

SAGA

(continued)

Notredame and Higgins

SAGA

(continued)

Notredame and Higgins

SAGA

(continued)

Notredame and Higgins

SAGA

(continued)

Notredame and Higgins