Parallel Simulated Annealing with Adaptive Neighborhood determined by GA

blessinghomoeopathΤεχνίτη Νοημοσύνη και Ρομποτική

30 Νοε 2013 (πριν από 4 χρόνια και 1 μήνα)

105 εμφανίσεις

2003 IEEE International Conference on Systems, Man & Cybernetics

Doshisha University, Kyoto, Japan

Parallel Simulated Annealing with Adaptive
Neighborhood determined by GA



Mitsunori MIKI



Tomoyuki HIROYASU



Toshihiko FUSHIMI

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Introduction



Optimization problems become more complicated


and larger.


Simulated Annealing (SA)

based on the simulation of the physical process “annealing”.

Important matters

1.
Parallelization

2.
Adaptive parameter tuning



Heuristic search


GA, CA, NN etc.

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Algorithm of Simulated Annealing

Algorithm

Energy

low

1. Generation

2. Judge


Transition

3. Cooling

Design space

high

(

E = Enext
-

Enow)

good acceptance

bad acceptance

1

Exp(




)

-

E

Temperature

Metropolis probability

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Neighborhood range


Can’t search optimum effectively.


Often trapped in a local minimum.


Too large neighborhood range


Too small neighborhood range

The neighborhood range in the continuous Euclid space is the
extent for generating next solution.

Global optimum


The range has to be small.


The range has to be large.

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Background

For the control of the neighborhood range, some method are
proposed.

These methods control the neighborhood range using
an appropriate acceptance ratio.



The adaptive neighborhood mechanism. [Corana 1987]



The advanced adaptive neighborhood mechanism. [Miki 2002]

This type of adaptive neighborhood method is very effective and useful,
but the target acceptance ratio should be determined experimentally.

Propose a new adaptive neighborhood mechanism

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Purpose

Parallel Simulated Annealing with Adaptive Neighborhood

determined by Genetic Algorithm (PSA/ANGA)

Controlling the neighborhood range adaptively during
the search.


This method is Parallel model.


This method parallels neighborhood
ranges on each processes.


This neighborhood range is controlled
by GA.

Characteristics

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Effect of Neighborhood Ranges

Neighborhood range

large

small

Fixed neighborhood range

Search space


The neighborhood range has a significant effect on the
accuracy of the solution.


In order to verify this effect, some numerical experiments
were carried out with various fixed neighborhood ranges.

Compare the qualities of
the solutions.

Obtain the effect of the
neighborhood ranges.

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Test problems

Rastrigin

Griewangk

Rosenbrock

Mathematical

test functions

Griewangk function

Rastrigin function

Rosenbrock function

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Appropriate neighborhood range

Appropriate

neighborhood range

Rastrigin

The neighborhood range has a significant effect on
the performance of SA.

Good solution

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Appropriate neighborhood range

Appropriate

neighborhood range

Griewangk

The neighborhood range has a significant effect on
the performance of SA.

Good solution

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Appropriate neighborhood range

Appropriate

neighborhood range

Rosenbrock

The neighborhood range has a significant effect on
the performance of SA.

Good solution

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Concept of PSA/ANGA

The neighborhood range determined adaptively by GA.

PSA searches the solution with various neighborhood range.


The appropriate neighborhood ranges depend on problems.


It is difficult to find the appropriate neighborhood ranges in
advance.

There are the appropriate neighborhood ranges in SA when
solving the continuous optimization problems.

Parallel Simulated Annealing with Adaptive Neighborhood

determined by Genetic Algorithm (PSA/ANGA)

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Algorithms of PSA/ANGA

Neighborhood range

large

small

Multiple SA processes searches the solution with various
neighborhood range.

GA operators are applied on neighborhood ranges.

Fitness =

1

Energy

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Abstract of numerical experiments

Parallel SA with Fixed Neighborhood (PSA/FN)

Optimum fixed neighborhood range

Comparative method

Use the optimum fixed neighborhood range
determined by preliminary numerical experiments.

PSA/ANGA is compared with a parallel SA with optimum fixed
neighborhood range, PSA/FN.

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Parameters used

Functions

Rastrigin

Griewangk

Rosenbrock

Max temperature

10

20

1

Min temperature

0.01

0.001

0.001

Markov length

102400

307200

3072

No. of variable

3

3

3

Cooling rate

0.8

0.7

0.8

Optimum fixed
neighborhood range

1.0

5.5

0.3

No. of processes

32

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Performance of the proposed method

Proposed method

The proposed method, PSA/ANGA, provides better
performance than PSA/FN in all problems.

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

History of Neighborhood range

Rastrigin


History of the neighborhood ranges in 32 SA processes.


The appropriate neighborhood range varies dynamically during
the search.

The appropriate neighborhood range is automatically
determined using GA.

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

History of energy (Rastrigin)


The proposed method,
PSA/ANGA
, shows fast convergence
of the energy and obtains lower energy than PSA/FN.


Accuracy of the solution improves because the neighborhood
ranges were changed adaptively.

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

Conclusions

A new Parallel Simulated Annealing method with adaptive
neighborhood range mechanism is proposed.

Parallel SA with Adaptive Neighborhood


determined by Genetic Algorithm (PSA/ANGA)

PSA/ANGA shows good performance on the some test functions.

The appropriate neighborhood range varies
according to
the condition of the search.

The proposed method adapts to these appropriate
neighborhood ranges.

The method is effective in SA for continuous optimization
problems.

2003 IEEE International Conference on Systems, Man & Cybernetics

2003.10.06

questions and answers

Thank you for your kind attention.