i

NCARNA

NCAR TECHNICAL NOTE

h

AN INTR ODUCTION GENETIC ALGORITHMS

F OR NUMERICAL OPTIMIZA

P Charb onneau

HIGH AL TITUDE OBSER V A Y

NA CENTER F OR A TMOSPHERIC RESEAR

BOULDER COLORADO

CH TIONAL

TOR

aul

TION

TO

MarciiT

List of Figures v

List of T ables

Preface

In tro duction Optimization

Optimization and hill

The simplex metho d

Iterated simplex

A set of test problems

P erformance of the a iterated simplex m etho ds

Ev olution optimization and genetic algorithms

Biological ev olution

The po w er of cum ulativ e election s

A basic genetic algorithm

Information transfer in genetic algorithms

PIKAIA A genetic algorithm for n umerical optimization

Ov erview and problem deition

Minimal algorithmic comp onen

Additional comp onen ts

A case study GA P

Hamming w alls and creep m utation

P erformance on problems

A real application orbital elemen ts of binary

Binary stars

Radial v elo cities and Keplerian orbits

A genetic algorithm solution using PIKAIA

Final though ts and further readings

T o cross o v er or to o v er

Hybrid metho ds

cross not

stars

test

on

ts

nd simplex

bing clim

ix

vii

CONTENTS OF ABLE

iii When should y ou use genetic algorithms

F urther readings

Bibliograph y

ivv

LIST FIGURES

Op eration of a generic hill metho d

A hard maximization problem

An iterated hill clim bing sc heme

Absolute p erformance of the simplex metho d

T est problem P

T est problem P

Accelerated learning b y means of cum ulativ e selection

Con v ergence curv es for tence learning h problem

Breeding in genetic algorithms

Con v ergence curv es for GA on P

Ev olution of the p pulation o in parameter space

Global con v ergence probabilit y

Radial v elo cit yv ariations in Bo

Ev olution of a t ypical solution to the binary ting problem

iso con tours in four h yp erplanes of parameter space

orbit

otis

searc sen the

Norsk

bing clim

OFviLIST OF T ABLES

I Simplex p erformance measures on test problems

II P erformance on test problems vs iterated simplex

PIKAIA

viiviiiA CE

In I w in vited to presen t a lecture on genetic algorithms at a Mini

W orkshop on Numerical Metho ds Astroph ysics held June at Institute

for Theoretical Astroph ysics Oslo Norw a y I subsequen prepared a written

v ersion of the lecture n i the form of a tutorial n i tro uction d to genetic algorithms for

n umerical optimization Ho w ev er for reasons bey ond the organizers con trol the

planned Pro ceedings of the W orkshop w ere nev er published the written

v ersion a v ailable through the W eb P age since septem ber con tin ues

pro v e p opular with users of the PIKAIA soft w I decided to ublish the

pap er in the form of the presen t NCAR T hnical

The pap er is organized as follo ws Section establishes the distinction be

t w lo cal and global optimization and the meaning of p erformance

in con text of global optimization Section tro uces d the general idea of a

genetic algorithm as nspired i from the biological pro cess of ev olution b y means of

natural selection Section pro vides a detailed comparison of the p erformance of

three genetic algorithmased optimization sc hemes against iterated clim

using the simplex metho d Section describ es in full detail the use of a genetic

algorithm t o olv s e a real data mo deling problem namely the determination of or

bital elemen ts a binary star system from observ ed radial v elo cities The

closes in section rections on matters of a somewhat more philosophical

nature and includes a list of suggested further readings

I ended up making v ery few mo diations to the text originally prepared in

ev en though if I w ere to rewrite it w things undoubtedly w ould

turn out diren t The of test functions I no w use to test mo diations

PIKAIA has ev olv ed signian from that presen ted x herein V ersion of

PIKAIA publicly released in April w compare en v orably

the iterated simplex metho d against whic h x herein I

up dated expanded the of further reading x to b etter rect curren t

topic and trends the genetic algorithm literature addition to some minor

rew ording here and there throughout the text I restored a Figure x and

a al subsection to x b oth originally eliminated to within the age limit

of the ab o v een tioned illated W orkshop Pro ceedings

Bac k in I c hose e pap er the v or of a tutorial h section

ends with a ummary s of imp ortan t p oin ts to remem b r e rom f that section Y ou are

Eac this giv to

to also

In in

list and

in pitted is PIKAIA

to fa more ev ould

in tly

to suite

some no

with

er pap of

bing hill

in the

measures een

Note ec

are to

PIKAIA

Because

tly in

the in

as

PREF

ixx

of course encouraged to ber more than whatev er listed there Y ou

also d at the end of eac h section a s eries of Exercises are

some require programming on y our part These designed to be done

using PIKAIA a public omain d selfon tained genetic algorithmased optimization

subroutine The source co de PIKAIA s w ell as answ ers most

a v ailable on the tutorial W eb P age from whic hy ou can also access he t PIKAIA

W eb P age

httpwwaocar du publ ic r es earc h i ikai a to u ria lt ml

The T utorial P age also includes v arious animations for of solutions dis

cussed in the text The PIKAIA W eb P age con tains links to HA O ftp arc hiv e

from whic h y ou can obtain in addition to the source co de PIKAIA a User

Guide as w ell as source for the v arious examples discussed therein The

idea b ehind all this is that b y the y ou are reading through this

and doing the Exercises y should be go o d shap e to solv e global n

optimization problems y ou t encoun y our o wn researc h

The writing of this preface ors a e opp ortunit y to thank m y f riends and

colleagues Viggo Hansteen and ats M Carlsson for their in vitation and ancial sup

potr to attend their Mini orkshop on Numerical Metho ds in Astroph ysics

as w ell as for their y during m y extended y Norw a y The

CrB ata d and some source co des or f the orbital elemen t tting problem of x ere

pro vided b y Tim Bro wn who w as also generous with his ime t n i explaining to me

some of the subtleties of orbital elemen t determinations Thorough readings of

the draft of his t p ap er b y S andy and Gene Arnn Tim Bro wn Sarah Gibson

Barry K napp Hardi P eter are also gratefully ac kno wledged

Throughout m yt w e y ears w orking at NCAR High Altitude O bserv atory

it been m y privilege in teract with a large n um ber of brigh t en th usi

astic studen and p ostdo s c fora ys to algorithms ha v e particularly

b eneed from suc h collab orators Since Iha v e had k eep in turn with

T ed Kennelly arah S Gibson Hardi P eter Scott McIn tosh T ra vis Metcalfe I

thank them all for k eeping me on m ytoes all this time

P aul Charb onneau

h Boulder

Marc

and

up to

genetic in My ts

and to has

elv

and

in sta hospitalit kind

in ter migh

umerical in ou

er pap done time

des co

for

the

the some

are

exercises to for

are and so

less others easy Some

will is remem INTR ODUCTION OPTIMIZA TION

Optimization and hill clim

Optimization is something that readers of this tutorial ha v e

faced a long time ago in their st c alculus class one is giv en an analytic function

f x and presen ted of ding the v alue of x at whic h the function

reac its maxim um v alue pro cedure taugh tto w this end i s diren

tiate the function with resp ect to x set the resulting expression to zero

solv e for x call the result x and there y ou v e Ev en though most of

max

w ould no longer think t wice ab out it this is actually a prett y tric k

F or the eader r trained in ph ysics the limitation of this analytical metho d w as

encoun tered p erhaps st in optics hen w studying the diaction pattern of a sin

gle v ertical slit Jenkins White c Y ou migh t recall that the

in tensit y of the diaction pattern v aries as sin x ehre x is directly prop or

tional to the distance along the direction p erp endicular to the slit on the screen on

whic h the diaction pattern pro ected j lo of the in y minima

readily found be x n with n n is tric kier Ho w

min

ev er calculating the lo cations of the n i tensit y maxima b y t he analytical p ro cedure

describ ed ab o v e leads a nast y nonlinear transcenden tal equation whic h

be solv ed algebraically for x turn to iterativ e or graphical means

n the course of whic h the kier n of the minima is also resolv

This diult y with the diaction problem symptomatic of the fact that it is

usually harder often much harder d the zeros of functions than

extrema the more so the higher the dimensionalit y of f

et al x for a concise y et lucid discussion of this matter The nescapable i

conclusion is that once one mo v b ey ond high sc ho ol calculus minax problems

optimization is b est carried out n umerically

Up on op ening a t ypical in tro ductory textb o ok on n analysis one is

almost guaran teed to d therein a optimization metho ds describ ed in some

In fact y ou also v e to diren tiate the result of step once again and

v erify that the resulting expression is negativ e when ev aluated at x but

max

subtlet y migh tha v e b een elab orated up on in the lecture

next only

this

ha

few

umerical

es

Press see unctions said the

their to ery

is

ed case tric

to has One

cannot to

to are

tensit cation The is

hap

neat

us it ha

ard The hes

task the with

st will most

bing

detail In nearly all cases those m etho ds will fall under the b road category of hil l

climbing schemes op eration of a generic clim sc heme is illustrated

on Figure in the con text of maximizing a f unction of t w ov ariables i ding

the maxim um ltitude in a D andscap e bing b e gins b y c ho

a starting lo cation in parameter space anels One then determines the

lo cal steep est uphill direction mo v es a certain distance in direction anel

rev aluates the lo cal uphill direction and so on un til a lo cation in parameter

space is arriv ed at where all surrounding directions do wnhill This marks

the successful completion of the maximization task anel Most textb o ok

optimization metho ds basically op erate i n this w a y and simply dir in ho wthye

go ab out etermining d the steep est uphill direction c ho osing ho w a a is to

be akt en in that direction nd a whether or not in doing so use is made of gradien t

information accum ulated the course of previous steps

Hill clim bing metho ds w ork great if faced unimo al d landscap es suc h as

the one w ards whic h the rabid paratro op er of Fig A ab out dep osit

lo w er bac kside Unfortunately is not alw a ys that simple Consider instead

the D landscap e sho wn on Figure the um is the narro w cen tral spik e

indicated b y the arro w and is surrounded b y concen tric rings of secondary maxima

The only w a y that hill clim bing can d the true maxim um in case is if

paratro op r e happ ens to somewhere on the slop es of the cen maxim

hill clim bing rom f an y other landing site will lead to of the rings The cen tral

p eak co v ers a fractional surface area of ab out of the full parameter space

x y Unlik e on the landscap e of Fig A here the starting p oin t is

critical if hill clim bing is to w Hill bing is a lo c al optimization strategy

Figure ors a glob optimization problem

Of course if the sp eci optimization problem y ou are w orking on happ ens

be h that y ou can alw a ys come up with a go o d enough starting guess then

all y ou need is lo cal hilllim bing and y ou can pro ceed merrily ev after But

what if y ou are in the situation most p eople d themselv es in when dealing with

a ard h global optimization problem namely not bneiginapsiontoti o llag pu ood

starting guess out of y our hat

I kno wwhaty oue thinking If the cen tral p e ak co v ers ab out of parameter

space it means that y ou v e ab out one c hance in a h undred a random drop

land close enough for hill to w ork So the question y ou ha v e to

y ourself is do I f eel luc ky Y our answ er to this q uestion is em b o died in the First

W ell do y punk

ou

ask bing clim to

for ha

er

suc to

al

clim ork

one

um tral land

our this

maxim

life

his to is to

with

in

step big

are

that

osing clim Hill

bing hill The

Figure Op eration a generic hill clim bing d llegory F rom a ran

domly c hosen starting p oin t panel the direction of maxim um slop e is f ollo w ed

anel un til one reac hes a p oin t where all s urrounding directions are do wnhill

anel Landing anel not problematic from the computational p oin t

of view

is

metho of

Figure Tw o dimensional surface f x y x y deing a hard

maximization problem The global maxim um is f x y at x y

and is indicated b y the arro w

Rule of Global Optimization also kno wn as

THE HARR Y R ULE

ou should nev er feel luc ky

F aced with the landscap e of Figure the straigh ard solution lies with

a hnique called iter d l climbing This is a fancy name for something v

simple as illustrated on Figure Y ou run y our fa v orite lo cal hill clim

metho d rep eatedly eac h time from a diren t randomly c hosen starting p oin t

While doing so y ou k eep trac k of the v arious maxima lo cated and once y ou

satisd that all maxima v e b een found y ou pic k the tallest y ou

done with y our global optimization roblem p As y ou migh t imagine deciding

when to stop is the crux of otherwise straigh tforw pro cedure

ard this

are

and one ha are

so

bing just

ery hil ate tec

tforw most

TY DIR

with

Figure An iterated hilllim sc After landing eac h trial pro eeds c as

on Fig

heme bing

With a fractional co v erage of for the cen tral p eak of Figure y ou migh t

exp ect to ha v e to on a v erage something of the order of iterated

clim bing trials b fore e ding the cen tral p e ak As is faced with

problems of increasing parameter space dimensionalit y andr situations where

the global maxim um spans a tin y fraction of parameter space iterated

clim bing can add up to a lot of w This leads us naturally to the Rule

of Global Optimization also kno wn as

THE NO FREE R ULE

f y ou really w an t the global optim um y ou h a v e to w ork it

These considerations also lead us to distinguish bet w een three distinct asp ects of

p erformance when dealing a global optimization problem

Absolute p erformance Ho wn umerically accurate is the solution returned

b y m y adopted d

Global p erformance Ho w certain can I be that solution returned b y

m y metho d is the true global maxim um in parameter space

Relativ e p erformance Ho w m uc h computational w ork is b y m y

metho d to return a solution

Most fancy optimization m etho ds y ou migh t read ab out in textb o oks are designed

to do as w ell as p ossible on and s im ultaneously Suc h metho ds will do w ell

on only if pro vided with a uitable s starting guess If suc h a guess i s consisten tly

a v ailable for the roblems p y w orking on y ou need not ead r an y further But

assured that Dirt y h up y ou one of da ys

The simplex metho d

The distinction bet w lo cal and global optimization as w ell as related

p erformance issues are p erhaps best appreciated b y lo oking in some detail at the

beah vior of a l o al c hill clim bing metho d on a global optimization problem T o w ard

this end w e retain D landscap e of Figure as a b ed and attempt

maximize it using the Simplex Metho d

A d iren t terminology ma yw be used in optimization textb o oks but y ou

can be assured that they do discuss something equiv alen t

ell

to test the

the een

these with catc will Harry rest

are ou

required

the

metho

with

for will

LUNCH

Second ork

hill only

optimization one

hill run

The Simplex d of Nelder actually a v robust

clim bing sc heme A brief y et clear in tro duction to the metho d can be found in

Press et al x A simplex is a geometrical ure with n v ertices that

liv es in a parameter space of dimension n In D pace s a simplex is a triangle in

D space a t etrahedron and so on Giv en the function v alue ere he t ltitude

f x y at eac h of t he simplex v ertices ere an x y p t t he w orst v ertex is

displaced b y ha ving the simplex undergo one of p ossible t yp es of o v es

namely con traction expansion or rection ee Fig Press al The

mo v e is executed n i a manner suc h that the f unction v alue of the d isplaced v

is increased b y the mo v e in the con text of a aximization m problem

undergo es successiv e suc h mo v es un no mo v e can b e found that leads to further

impro v emen t bey ond some preset tolerance W atc the simplex con tract and

expand and squirt around the l andscap e of ig F is go o d visual fun and justis

w ell name giv en b y Press al to simplex subroutine amoeba This is

the implemen tation u sed here

By the standards of lo cal optimization metho ds simplex passes for a

lo w metho d The accuracy the solution increases ximately

linearly with the n um b r e of simplex mo v es Ho w ev er the simplex can pull itself out

of situations t hat w ould defeat or seriously mp i ede faster marter gradien tased

lo cal metho ds it can eien tly cra t v alleys and squeeze through

saddle p oin ts In this sense it can be said to exhibit pseudolobal capabilities

Eviden tly the simplex metho d requires that one pro vide initial co

x y for the simplex three v ertices Despite he t simplex metho d pseudolobal

abilities on a m ultimo dal global problem the c hoice of initial lo for the

simplex often determines whether the global maxim um is ultimately found Figure

ho ws a series of con v ergence curv es for the test problem of Figure Eac h curv e

corresp onds to a iren d t random initial simplex conuration

ds the cen tral p eak it do es so rather quic kly requiring ab out mo v es for

accuracy The p roblem of course is hat t the simplex often d o es not con v erge to the

cen tral p eak Rep eated rev eal that the metho d ac es global con v ergence

for only or so of trials

Iterated Simplex

The prosp ects of the simplex metho d for global p erformance are greatly en

hanced if one of starting v ertices lies high enough on slop es of cen tral

p eak This suggests that iterated hill clim bing using the simplex metho d ere

after iterated simplex should ac hiev e global p erformance within a few h undred

An animation of the simplex at w ork on the landscap e of Figure can b e

view ed on the tutorial W eb P age Chec k it

out

the the

hiev trials

simplex the When

cation

ordinates

long up wl

appro of absolute

the

their et the

hing

til

simplex The

ertex

et in

three

oin

hill ery is Mead Metho

Figure Absolute p erformance of the simplex d on the test problem of

Figure Eac h curv e corresp onds to a diren t starting simplex F ailure of the

simplex to lo cate the p eak to the con v ergence curv es eling o at

relativ ely high v alues of f x y With con v erged runs out of trials his t plot

is not represen tativ e of the simplex m etho d global p rformance e on this problem

whic h is in fact signian tly p o namely ab out

iterations And indeed it do es rep eatedly running the simplex times on the

problem of Figure to the cen tral p eak b e ing lo cated in of tri

als price to pa y of course is in n b er of function ev aluations required

ac hiev e this el of global p erformance nearly function ev aluations per

iterated simplex run on a v erage W elcome bac k to the No F ree Lunc h Rule

It is recommended practice when the simplex in singleun mo de

carry a r andom r estart once the simplex has v erged this en tails reinitial

izing randomly one of con v simplex v ertices letting the

simplex recon v erge again What is describ ed here as iterated simplex

in reinitializing al l v ertices randomly so as to mak e eac h successiv e trial

indep enden t from all others

A single simplex mo v e y than one function ev aluation F or

more tail en ma

fully

consists

and erged the but all

con out

to using

lev to

um the The

leads test

orer

lev leads tral cen

metho

A set of test problems

One should righ tfully susp ect that the simplex metho d p e rformance on the

problem Figure migh t not be represen e of its p erformance on

problems This v ery legitimate concern will eviden o v er the v arious

genetic algorithmased optimization sc hemes discussed further belo w It

therefore pro v e useful to ha v ea v ailable not just one but a set of est t roblems p The

four test problems describ ed b elo w a very hard global optimization problems

on whic h most con v tional lo cal optimization algorithms w fail miserably

Also k eep in mind that it is always p ossible design a test problem

defeat any global optimization metho d

P maximizing a function of t w o v ariables parameters

Our st test problem ereafter lab eled is our no w familiar D e

of Figure Mathematically t is deed as

f x y cos n r xp r a

r x y x y b

where n and are constan ts The global maxim is lo at

x y with f x y This global maxim um is surrounded b y

concen tric rings of secondary maxima cen tered on the maxim um at

distances

r f g

Bet w een these a re lo cated another series of concen tric rings c orresp onding to min

ima

m

r m

n

The error asso ciated with a en solution x y can be deed

f x y

example if the trial mo v e do es not lead an increase in f the mo v e migh t be

rep eated with a halv ed or doubled displacemen t ength l r a d iren tt yp e of mo v e

migh t be attempted dep ending on implemen tation On the maximization prob

lem of Figure one simplex mo v e r equires function ev aluations on a v erage

The high n high D v ersion of the fractal function discussed in x of B ac k

is a prett y go o d candidate for the ultimate killer test problem

to

as giv

min

max

radial global

cated um

landscap

will that to

ould en

all re

will

to carry tly

other tativ of test

Note that the eak corresp onding to the global maxim um co v ers a surface

n in parameter space If a illh clim bing sc heme w ere used the robabilit p yof

a andomly r c hosen s tarting p oin t landing close enough to this p e ak for the metho d

lo cate the true global um is only for n

P maximizing a function of t w o v ariables parameters

T est function P sho on Figure is again a D landscap e to be

It is deed b y

f x y exp r exp r a

r x y b

r x y c

The maxim um f x y is at x y and corresp onds to the p eak of

the s econd narro w er Gaussian P is ab out as hard a global optimization problem

as P he simplex succeeds times out of trials but f or a iren d t reason

There are no w only t w o lo cal maxima with the global maxim um again co v

ab out of arameter p space Unlik e P where m o ving to w ard successiv

secondary extrema actually brings one closer to the true m axim um ith w P mo v

ing to secondary maxim um pulls solutions away from the global maxim

Problems exhibiting this c haracteristics sometimes eceptiv e in the

optimization literature

P maximizing a function of v ariables parameters

T est problem P a direct generalization of P to indep e nden t v ariables

w x y z

f w x y z cos n r xp r a

r w x y z w x y z b

again with n and Comparing p erformance on P P

pro vide a measure of sc alability of the metho d under consideration ho w

p erformance degrades as parameter space dimensionalit y is increased ev

else b eing equal P a d global optimization problem the

metho d manages to d the global maxim um of trials

P Minimizing a least squares residual parameters

Our fourth and al test problem is deed as a eal nonlinear least

ting problem Consider a function of v ariable x eed d as sum of t w o

Gaussians

X

x x

j

y x A

j

j

j

exp

the one

squares

out times only

simplex har very is

erything

namely

will and

four is

four

called are

um the

higher ely

ering

maximized wn

again

maxim to

area

Figure T est problem P The problem consist maximizing a function of

t w o v ariables deed b y t w o Gaussians ee The global maxim um is

f x y at x y and is indicated b y the arro w

Dee no w a ataset b y ev aluating function a of K equidistan t

v alues of x in the in terv al i y y x x x x

k k k k

k

v alues of A etc Giv en dataset and the functional form used to generate

it eq the optimization problem then to v er the parameter v

for A originally used to pro uce d the dataset This is b y minimizing

the square residual

K

X

R A y y x A

k

k

with resp ect to the parameters deing the t w o Gaussians If one a

priori that t w o aussians G re a to b e to the data then this esidual r minimization

problem is ob viously equiv alen t a D function maximization problem for

a y whic h simply dees a function D space Figure ws the

generated using the parameter set

A

dataset sho in

to

told is

done etc

alues reco is

that

set some for

set for this

eqs

in

Figure T est problem P This is a parameter problem whic h consists in

ting t w o Gaussians a ataset of p oin ts Note ho w the second Gaussian

is p o orly sampled b y the discretization in x The thin line is

Gaussian function deed b y eq

and K discretization poin x Once the resulting

problem is not an easy one giv en the discretization in x the minimization is

largely dominated b y the need accurately the broader

Gaussian the second Gaussian s i not only of m uc hlo w er it is p o

sampled in x Fitting only the st Gaussian leads to a reasonably lo w residual

R global accuracy requires the second Gaussian be

and in whic h case only R

The simplex succeeds in prop erly ting b oth Gaussians out of

What are the econdary minima on whic h the simplex remains k They

can be divided in to t w o broad classes of mo Gaussians s the

broad higher amplitude comp onen t and the other is en to zero either b y

ha ving A or the metho d a t w o Gaussians solution where

x x and A A The D parameter space

con tains long t alleys and lains of w but sub optimal r esidual v alues in

whic h the simplex grinds to a halt

lo

returns

driv

del the one

stuc

trials

es do

etected also to

orly also amplitude

st amplitude high to

minimization again in ts

underlying the solid

to

T I

Simplex p erformance measures on problems

T est Problem P erformance Simplex Iterated simplex

P h f i

p

G

h N i

f

N

t

P h f i

p

G

h N i

f

N

t

P h f i

p

G

h N i

f

N

t

P h R i

p

G

h N i

f

N

t

P erformance of the simplex and iterated metho ds

T able I ummarizes s the p erformance of the implex s and iterated simplex meth

o d s on he t four test problems hen try represen ts an a v erage o v er at least

indep enden t runs p to for P and so should b e fairly r epresen e of the

metho d s beah vior on eac h test problem F or eac h problem the T able giv es the

absolute p erformance deed here as the a v erage o v er all runs of either f x

for P P and P or the r esidual R f eq f or P The global p erformance is

deed in terms of a p robabilit y easure m p as the fraction of all runs for whic h

G

the true global extrem um has been lo cated f for P P and P or the

second smaller Gaussian been prop erly R for P As amuasree of

relativ e p erformance the table simply the a v erage n um ber of functiono del

ev aluations h N i required b yeac h metho d The astl en try for eac h roblem p is the

f

n ber of trials N executed b y iterated simplex his n ber b y deition

t

for the basic simplex metho d without restart

is um um

lists

has

tativ

Eac

simplex

test

able

A t this stage only a commen ts need be made on the basis of T I

The st is that as adv ertised all four test problems are hard global optimization

problems as can b e judged from the p o or global p erformance of the basic simplex

metho d on eac h T urning to iterated simplex leads sp ectacular impro v t

in global p e rformance cases but of course the n um ber of required function

ev aluations go es up b ya orders of magnitude In fact the global p e rformance

of iterated simplex can be predicted on the of the singleun simplex The

global p e rformance on the later can b e view ed as a p robabilit y p of l o ating c the

global maxim the omplemen tary probabilit y of a giv en run to do so is

p the probabilit yof al l iterated simplex runs not ding the global maxim um

N

t

is then p that the probabilit y of any one of N iterations lo the

t

global maxim um is

N

t

p p terated hill clim bing

G

On the basis of eq one w ould predict global p erformances

on P through P giv en the n b er of h c bing iterations listed i n he t

righ tmost column of T able whic h compares quite w ell w ith he t actual measured

global p erformance One can also rewrite eq as

log p

G

N terated hill clim bing

t

log p

predict the exp ected n ber of hill clim bing iterations required to ac hiev e a

global p erformance lev el p p for P requiring p w ould

G G

demand n a v erage N hill clim bing trials adding up to a grand total of

t

ab out function ev aluations since a single simplex run on P carries out

on a v erage function ev aluations f T able Iterated hill clim

w orks there really is no h thing as a free lunc h

It is easy to predict the global p erformance of iterated simplex be

cause eac h trial pro ceeds completely indep enden tly impro v emen t in

p erformance simply rects the b etter initial sampling of parameter space asso ci

ated with the initial distribution of simplex v ertices Ev erything else b e

as problem dimensionalit y n increases the n um ber of trials N required can be

t

n

exp ected to scale as N a where a is n um ber c haracterizing in this case

t

the fraction of parameter space co v ered b y the global maxim um

is not only demanding in terms of function ev aluations but addition it

not scale w ell on a giv en problem dimensionalit y is increased in

fact is the cen tral problem facing iterated hill clim bing general not just its

simplexased incarnation

in

This as all at

es do in

simplex Iterated

some

equal ing

global The

ected exp

suc but

certainly bing

with

um to

lim ill um

cating so

not um

basis

few

all in

emen to

able few

The poor scalabilit y iterated stems from e

trial pr o c e e ds indep endently The c hallenge dev eloping global metho ds t hat are

outp erform iterated hill clim bing consists in tro ucing d a transfer of informa

tion bet w een trial solutions in a manner that con tin uously roadcasts eac h

paratro op r e in the squadron top ographical information garnered b y h in

dividual paratro p o e r in the course of hiser lo cal clim b The c hallenge of

course is to ac hiev e this without o v biasing the ensem ble of trials

A relativ w ellno metho d that often ac hiev es reliably is simulate d

anne aling etrop olis et see also Press et al x ulated

annealing is inspired b y he t global transfer of energynformation ac hiev ed b y col

liding constituen t p articles of a co oling liquid metal hic w hallo ws the ubstance s to

ac hiev e the crystallineetallic conuration that minimizes the total energy of

the hole w system The a lgorithmic implemen tation of the ec t hnique for n

optimization requires the sp eciation of a c o oling sche dule chi h i s ar f from triv

ial fast co oling is computationally eien otl w h N i but can lead to con v ergence

f

on a secondary extrem um o w p while slo w co oling impro v es global con v ergence

G

igh p but at the exp ense of a h N i No F h remem ber

G f

Genetic A lgorithms ac hiev e the are inspired b y the exc hange

of genetic information o c curring a breeding p opulation to

selection They can b e used to form the core of v ery robust global n umerical opti

mization metho ds as detailed Section beol w The follo wing Section pro

a brief in tro duction to genetic algorithms in a general sense

The least y ou should ber Section

Global optimization is a totally diren t game from lo cal optimization

Y ou should nev er feel luc ky

There is no suc h thing as a free h

Y ou can alw a ys design a problem that defeat an y global

metho d

Exercises for Section

Lo ok bac k at Figure Whenev er the simplex to ac hiev e global con v er

gence f seems remain stuc k at a discrete set of f

v alues What do these v alues corresp ond to

Consider again the use iterated simplex on the test problem of Figure

calculate the fractional surface of the of the cen p eak lies

higher the innermost of secondary maxima On this basis what

ring that

that tral part area

of

to it

fails

optimization will

lunc

from remem

more

vides in

natural jected sub in

but goal same

Lunc ree high

umerical

Sim al

this wn ely

erly

hill

eac the

to

in to

in

ach that fact the bing clim hill of

w ould y ou predict the required n um b er of s to b e on a v erage or f

iterated simplex to lo cate cen tral p

Rep eat the same analysis as for Exercise ab o v e but in the con text of the

P test p roblem Are y our r esults in basic agreemen twtih T able Ho w can

y ou explain the dirences f an y

eak the

trials implex

EV OLUTION OPTIMIZA TION AND GENETIC ALGORITHMS

Biological ev olution

The general ideas of evolution and adaptation predate Charles Darwin On the

Origin of Sp ecies b y Means of Natural Selection but it is Darwin nd

more or less sim ultaneously Alfred Russell W allace who rst den i w is

considered to be primary d riving mec hanism of ev olution natur al sele ction

Nature is v m uc ho v ersubscrib ed A t a an ytiem in an y cosystem e far

more individuals are born than can p o surviv e en ecosystem a v ail

able resources This implies that man y mem besr of a en sp will die from

attrition or predation b e fore ha v e a c hance to d The of

natural selection states that individuals b e adapted to their en vironmen t i

for whatev er reason b etter at obtaining lunc h a v oiding b ecoming lunc h and nd

ingttractingomp eting for mates will on a v erage lea v e b more opring

than their less apt colleagues

F or natural selection to lead evolution t w o more essen tial ingredien ts are

required inheritanc e opring m ust retain at least some of the features that

made their paren ter than a v erage otherwise ev olution ectiv ely reset at

ev ery generation variability at an ygiv en time individuals of v arying nesses

m ust co exist in the p opulation otherwise natural selection h as nothing to op erate

on

Both these additional requiremen ts w ere plainly ob to Darwin and

con temp oraries but their underlying mec hanisms remained unexplained in

lifetime Ho w er this situation c hanged rapidly i n the early decades of the t w en

tieth tury and the p through whic h heredit y mediated and

v ariation main tained are n o w basically understo o d Inan the information

determining the gro wth and dev elopmen t of individuals is enco ded as linear se

quences of genes that can eac h assume a ite set of alues In sexual sp ecies

when t w o ndividuals i breed complemen tary p ortions of their genetic material are

passed on to their opring and com bined to dee that opring full genetic

mak That the inheritance part In the course of repro cessing the genetic

material to b e later passed on to opring cop y m istak es and t ruly random alter

ation of some gene v alues also o ccur o c casionally m utation ev en ts coupled

These

eup

utshell

is cesses ro primary cen

ev

their

his vious

is ts

to

ehind

tter

principle uce repro they

ecies giv

the giv ssibly

lmost ery

the

still hat tid

to the fact that an opring receiv es complemen tary genes from t w o arenp ts hic h

is true of most animals pro vides the needed source of v ariabilit y

The individual that mo v feeds and mates in real space can b e lo ok ed at as

an outer manifestation of its deing genes Think t hen of an i ndividual ness

as a function of the v alues assumed b y its genes What ev olution do es s i to riv d ea

gradual increase n i a v erage ness v alues o v er the course of an m y generations

is what Darwin called adaptation No w that b eginning to sound lik e illh limc bing

do esn it In fact ev olution do es not optimize at least not in the mathematical

sense of the w ord Ev olution blind Ev olution do es not e a ab out

globally maximal ness n d eplaise a T eilhard de Chardin Ev en if it did

ev olution m ust accommo date ysical constrain ts asso ciated with dev elopmen t

and gro wth so that not all paths p ossible genetic arameter space

ev olution do es is pro duce individuals of ab o v e v erage ness Nonetheless the

basic ideas of natural selection and inheritance with v ariation can be used

construct v ery robust algorithms or f global n umerical optimization

The po w ulativ e selection

The idea that natural selection lead to a of hill clim in ness

space y b ecome in tuitiv ely ob vious after thinking ab out it for a while

remains less ob vious is the degree to h cum ulativ e selection i selection

op erating on successiv e generations can accelerate w ould in its absence be

a random searc h of genetic parameter space The follo wing p opularized

in Ric hard Da wkins Blind W atc hmak er a book w ell w orth reading

inciden tally mak es for suc h a nice demonstration of this v ery poin t it

b y no w found its w a y in to at least one textb o ok on ev olutionary a y

nard Smith an excellen t d the topic Consider the

sen tence

JEG SNAKKER BARE LITT NORSK

This sen tence is c haracters including blank spaces and is up of

an alphab et of letters a blank c haracter included lease note that I am

taking in to accoun t the famous Scandina vian letters A and Consider no w

This is said without at all den ying that a large part of what mak es us who w e

are arises rom f learning and other in teractions with the en vironmen t in the course

of elopmen t and gro wth what genes de is some sort of b eha

Bauplan from whic h these higher lev el pro cesses tak e o

The original sen tence b y Da METHINK IT IS A WEASEL

whic h of course is tak en from Shak e Hamlet

are esp

LIKE is wkins used

vioral basic enco dev

is if

made long

wing follo to uction tro in

genetics

has that

The

example

what

whic

What ma

bing form can

cum of er

to

All in are

ph

damn giv is

This

es

the ro p cess of ro p ducing haracterong sen tences b y r

from the a v ailable c haracters of the alphab et Here an example

GE YT AUMNBGH JH QMWCXNES

Do esn lo ok m h lik e the original sen although careful comparison

sho wtath t w o letters actually coincide n um b er of d istinct haracter

long sen tences that can be made of a haracter alphab et

This is a very large n um b er ev en b y astronomical standards The corre

sp onding probabilit y of generating our st target sen tence b y this random pro cess

on the st trial is then suc h a n um ber that in

v oking the Dirt y Harry Rule at this poni t w be mo ot Instead consider the

follo wing pro cedure

Generate sen tences of randomly c hosen c haracters

Select the sen tence that has the most correct letters

Duplicate this b est sen tence ten times

F or eac h suc h duplicate randomly eplace r a few letters

Rep eat steps through un til the target sen tence been matc

This searc h algorithm incorp orates the three ingredien ts men tioned previously as

essen tial to the ev olutionary pro cess Step is natural selection in a

deterministic and rather extreme form the b est and only the best acts as

progenitor to the next eneration Step inheritance again of a

extreme form as opring start o as exact replicas of the ingle progenitor

Step is a sto c hastic pro cess whic h pro vides the v ariabilit y Note

that the algorithm op erates with minimal ness i nformation all it has a v ailable

w man y correct etters l a sen tence con tains but not which letters are correct or

incorrect What is still missing is exc hange of information b et w een trial solutions

but be patien t this will in due time

Figure illustrates the v olution of the b estf sen tence starting from

an initial ten random en s tences as describ ed ab o v e The m utation ate r w as set at

p meaning that an y giv letter a probabilit y of bengi sub jected

to random replacemen t Iteration coun t s i l isted i n t he leftmost column and error

in righ tmost column is deed here simply as n um ber of

letters in the bets sen tence generated the course of the curren t iteration Note

ho w the error decreases rather rapidly at st m uc h more wly later on it

es ab out as man y iterations to get the st letters righ t as it tak es to get

the last one The target sen tence is found after only iterations in the course

More precisely dee a mutation r as probabilit y p that a

giv en constituen t letter be replaced

randomly

the ate

tak

slo but

in

incorrect the Error the

has en

come

ho is

also required

rather is

since

fact in

hed has

ould

small is This

is out

total The

will tence uc

letters selecting andomly

Figure Accelerated Norsk l earning b y means of cum ulativ e selection Iteration

coun t is listed in the left column and the error deed as the n um b e r of ncorrect i

letters in he t righ tmost column The target sen tence is found after iterations

of whic h trial sen tences w ere generated and v aluated against the

This is almost initely than the of en umerativ e or random

searc h

Figure sho ws con v ergence curv es for three runs starting w ith the same initial

random sen tence ev under diren t m utation rates The solid is

the solution of Figure Note ho w the solution with the highest m utation rate

con v erges more rapidly at but ev en tually lev o at a ite nonzero

lev el What is happ ening here is that m utations are pro ducing the n eeded

letters as fast as they are destro ying curren tly correct letters en an et

size and sen tence length will alw a exist a critical m utation rate ab o v e

whic h this will happ en

There t w o imp ortan t things to remem ber at this poin t First m utation

This is in fact a notion cen to our understanding of the emergence of

life Among a v ariet y of selfeplicating olecules m of diren t l engths omp eting

for c hemical constituen ts limited supply in the primaev al soup those

closest to the critical m utation rate can adapt the fastest to an ev olving c

hemical

lying in

tral

are

ys there

alphab Giv

correct

error els st

line olving but

purely less

target

Figure Con v ergence curv es for he t sen tence searc h roblem p or f three diren t

m utation rates The curv es sho w the error c the best sen tence

pro d uced at h iteration The solid line corresp onds to the solution sho wn on

Figure

is a mixed blessing It is clearly needed as a source of v ariabilit y to o m uc h

of it is deitely deleterious Second the general s hap e of the con v ergence curv es

in Figure is w orth noting Con v ergence is rather swift at st but lev els

o This is a b eha vior w e again again in what follo ws Time no w

mo v e on ally o genetic algorithms

en vironmen t without selfestructing and so rapidly tak eo v er the soup see Eigen

for a comprehensiv e though somewhat dated review This is conjectured

be the explanation b ehind the ersalit yof the genetic co de among v

all living organisms

nearly ery univ

to

to

and meet will

then

but

eac

with iated asso

A basic genetic algorithm

F undamen tally genetic are a class of searc h tec hniques that use sim

plid forms of the b iological pro cesses of selectionnheritance ariation

sp eaking they are not optimization ds p but can be used to form the

core of a class of robust and xible ds kno wn as genetic algorithm ase d

optimizers

Let go ac b k to a generic optimization problem One s i giv en a mo del that

dep ends on a of parameters u and a functional relation f u that returns

a measure of qualit y or ness asso ciated with the corresp onding mo his

could be a yp e goodesns of measure the mo del is compared to data

example The optimization task usually consists ding the oin t u in

parameter space corresp onding to the mo that maximizes the ness function

f u Dee no w a p opulation as a set of N realizations of the parameters u A

p

topev el view of a basic genetic algorithm is then as follo ws

Randomly initialize p opulation and ev aluate ness of besr

Breed selected mem b ers of curren t p opulation to pro duce opring p opulation

election based on ness

Replace curren t p opulation b y opring p opulation

Ev aluate ness of new p opulation b

Rep eat steps through un til the test mem b r e of the curren t p opulation

is deemed enough

W ere not that what it b eing cycled through the iteration is a p opulation of

solutions rather than a s ingle trial olution s this w ould v ery m uc h smell of iterated

hill clim bing It should also giv e y ou that uncann y feeling of d ej a unless y our

memory is really shot or on y ou y ou ha v e ed o v er the

section The crucial v elt y lies with Br e e ding is in the course of

breeding that information is passed and hanged across p opulation mem b

Ho w this information transfer t ak es place s i r ather p eculiar and merits discussion

in some detail and only b ecause this genetic algorithms justify the

enetic in their name

Figure illustrates the breeding pro cess the con of a simple D max

imization problem suc h as the P or P test problems In an individual

is a x y p oin t and so is eed b yt w o oating p oin tn um b e rs The st step

is o de the t w o ating p oin t n um bers deing h individual selected

breeding Here this is done b y dropping the decimal p oin t and concate

nating the resulting set of simple decimal n i tegers i n a c hromosomelik e

ines on Figure Breeding prop er a t w o step pro cess The st step

is

string to

simply

for eac enc to

case this

text in

where is not

ers exc

It step no

preceding skipp shame

vu

it

ers mem

mem its

del

in

for if

del

set

metho

se er metho

Strictly

algorithms

is ossover t w o strings generated b y the enco ding pro cess are laid side b y

side and a cutting p oin t is randomly selected along the length of the

strings string fragmen lo cated righ t of the cutting p oin t are then in ter

c hanged and spliced on to the fragmen ts originally lo cated left of the cutting p oin t

ines for a cutting poin t lo cated bet w een the third fourth

digit The second breeding step is mutation F or eac h string pro ucedd b y the

crosso v pro cess a few randomly selected digits r enes are b y a

new randomly selected digit v alue ines for am utation hitting the ten th

digit of the second opring string The resulting fragmen ts then deco ded

in t w o x y pairs whose ness then ev aluated here simply b y

the function v alue f x y

Some additional commen ts are in order First n ote that opring incorp orate

in tact h unks of genetic material coming from b oth paren ts that the needed

inheritance as w ell as the promised exc hange of information bet w trial solu

tions Ho w ev er both the crosso v er m utation op erations also in v olv e purely

sto c hastic comp onen suc h as the c hoice of cutting poni t of m utation and

new v alue of m utated digit This is w e get the v ariabilit y to sus

tain the ev olutionary p ro cess as discussed earlier Se c ond d

pro cess illustrated on Figure is just of man y possible suc h sc hemes T ra

ditionally genetic algorithms ha v e use of binary enco ding this often

not particularly adv an tageous for n umerical optimization use of a

genetic lphab et is no more artiial than a inary b represen tation ev en so

giv en that v ery nearly all kno wn living organisms nco e de their genetic information

in a base alphab et fact in terms of enco ding ating oin t n um b e rs b oth

binary and decimal alphab ets sur from signian t shortcomings can

the p erformance of the resulting optimization algorithms Thir d the crosso v

and m utation op erators op erating conjunction the d

pro cesses as illustrated on Figure preserv e total range parameter space

That is if the ating oin t parameters deing paren t solutions are restricted to

the range then the opring solution parameters will be restricted

to This is a v ery imp ortan tpropter y through whic h one can rtlessly e

hardwire constrain suc h p ositivit y F ourth ving the m utation op erator

act on the enco ded form of paren t solution in teresting consequence

that opring can dir v m uc horv ery little from their p aren ts dep ending on

whether digits acted b y m utation eco d de in to one of the leading or

digits of the corresp onding ating oin tn um ber This means that rom f the p oin t

of view of parameter space exploration a genetic algorithm can carry out b oth

wide exploration and e tuning in parallel Fifth it es t w o paren ts to pro

duce im ultaneously t w o opring One can of course devise orgiastic

hemes that in v olv e more t w o paren ts yield an y n um ber of opring

and than sc

breeding

tak

trailing the

ery

the has the

ha as ts

also

in the

ding ingeco enco with in

er

act that

In

more

decimal The

is but made

one

ding ingeco enco the

needed where

site ts

and

een

computing is to

are

replaced er

decimal and

ts The

deing

The cr

Figure Breeding in genetic algorithms Here the pro cess illustrated in the

con text of a D maximization problem uc h asPrPf x An individual is

an x y p oin t t w oscu h t individuals are needed for breeding enoted

P and P here one oint cr ossover and one oint mutation op erators

act on string represen tations of the paren ts and S to pro duce opring

strings S and S whic h are ally deco ded i n t w o opring x y p oin

P and P

Exp erience sho that this rarely impro v es the p erformance of the resulting algo

rithms Sixth f u m ob viously be computable for all u but necessarily

not ust

ws

ts to

The

paren and

is

diren tiable since deriv ativ es of the ness function with resp ect to its i nput pa

rameters not required for the algorithm to op erate F rom a practical p oin t of

view this can be a great adv an tage

Information transfer in genetic algorithms

Time to step bac k and revisit the issue of information pro cessing Genetic

algorithms ac hiev e transfer of information through the breeding of trial solutions

selected on the basis of their ness whic his wh y he t crosso v er op erator is usually

deemed to be the deing feature of algorithms as compared to

classes of ev olutionary algorithms ee e B ac k

The join t action of crosso v er and nessased selection on a p opulation of

strings enco ding trial solutions is to increase the o currence c frequency of sub

strings that con v ey their trial solution ab o v e v erage ness at a rate

prop ortional to dirence bet w een the a v erage ness of trial solutions includ

ing that substring in their enot yp e the stringnco ded v ersion of

deing parameter set and a v erage ness of the whole p opulation The

mathematical expression of the preceding mouthful adequately expanded tak e

in accoun t the p o ssibilit y of substring disruption b y crosso v er or m utation is

kno wn as the Schema or and is originally due Holland see

Goldb erg As the p opulation ev olv es resp onse to breeding ness

based selection adv an tageous substrings are con uously sorted and om c bined b y

crosso v in to individuals leading to an inexorable ness increase in the

p opulation as a whole Because this in v olv es the concurren t c of a great

man y distinct substrings Holland ed this prop ert y intrinsic p ar al lelism and

argues that therein fundamen tally ies l he t exploratory p o w er of genetic algorithms

The least y ou should ber Section

Natural selection alone cannot lead to ev olution inheritance and v ariation are

also needed

Cum ulativ e s election can accelerate an otherwise random searc h pro cess b ya

factor that astronomically enormous

Genetic Algorithms searc h tec hniques that e use of simplid forms

of the biological selectionnheritance ariation triad

Exercises for Section

All exercises for hist part of the tutorial aim at etting l y ou explore quan titativ

the probabilistic asp ects of sen tence searc h example of x

the

ely

mak are

is

from remem

dubb

essing pro

single er

tin

and in

also to em The

to

to

the

their

all

ded deco

other genetic

are

First some asic b probabilit y calculations to w arm up what is the proba

bilit y of getting l of the l etters w rong on an initial r andom trial getting

at least one etter l n y etter l righ t getting exactly one l etter n y letter

righ t

In the run of Figure it to ok iterations to get the poni t of ha

correct letters What is no w the probabilit y of obtaining a

fully correct tence in one of the ten m utated copies after the subsequen t

iteration What is the probabilit y of al l m utated copies ha ving regressed

only correct letters

Giv the sen tence S alphab et size A and a m utation

rate p obtain an estimate not a formal calculation the n ber of

iterations required on a v erage to reac h zero error Ho w do es y estimate

compare Figure Do y ou think that Figure a t ypical solution

Giv again a sen tence length S an alphab et size A and a m utation rate p

calculate the error lev at whic h the sen tence searc h algorithm saturate

ik eteh dotted line on Figure Use this result o t estimate an optimal m u

tation rate as a function of S and A that will on a v erage lead to con v ergence

in the smallest p ossible n um ber of iterations

In terms of an analogy biological ev olution what do y ou think are the

most signian t failings of the sen tence searc h example

for

will el

en

is to

our

um for

length en

to

sen

of out

ving to

al

PIKAIA A GENETIC ALGORITHM

F NUMERICAL OPTIMIZA TION

Ov erview problem deition

In this section w e will be primarily concerned with the comparison of genetic

algorithmased optimizers with other global optimization sc hemes sp ecially

iterated hill clim bing using simplex metho d x T o w e st need

settle on a sp eci implemen tation a genetic algorithm

PIKAIA is a p ublic omain d general purp ose genetic algorithmased optimiza

tion subroutine is written in F TRAN is completely selfon tained and

is designed to be as easy to use as the optimization subroutines found in

et al Numeric R e cip or example their simplex routine amoeba It comes

with limited I capabilities and no fancy graphics The w is describ ed in

great detail in the User Guide to PIKAIA harb onneau Knapp

hereafter PUG whic h n umerous references made in what follo ws The

soft w are and User can both be obtained from the PIKAIA W eb P age

httpwwaocar du publ ic r es earc h i ikai a to u ria lt ml

This section op ens a brief o v erview of the op erators and tec hniques

included in PIKAIA In ternally seeks to maximize a usereed function

f x in a b ounded n imensional space i

x x x x k

n k

The restriction of parameter v alues in the range allo xibilit y

and p ortabilit y across problem domains This ho w ev er mplies i that the u ser m ust

adequately normalize the input parameters of the function to be maximized with

to those b ounds

The maximization s i carried out on a p opulation up of N individuals

p

rial solutions This p opulation size remains ed throughout ev

Rather than olving p opulation un some tolerance criterion is satisd

PIKAIA carries the ev olution o v er a u sereed preset n um b r e o f generations N

g

til the ev

olution the

made

ect resp

greater ws

PIKAIA

with

Guide

are to

are soft

es al

Press

OR It

of

to so do the

and

OR

PIKAIA ors the ser u the xibilit y to sp ecify a n um b r e of other input param

eters that con trol the b eha vior of the underlying genetic algorithm The subroutine

do es include builtn default settings that ha v e pro v en across problem do

mains All suc h input parameters passed to in the imensional

con trol v ector Section of the PUG for allo w ed default v

of those con parameters

The topev el structure of PIKAIA the same as the sequence of algorithmic

steps listed in x an outer lo op trolling generational iteration and an

inner lo op con trolling breeding Since breeding in v olv the d of

opring the inner lo op executes N times p er generational iteration where N

p p

is the p pulation o size N is the default v alue

p

All parameter v alues deing the individual mem b ers of the initial p opulation

assigned a random n ber the range extracted from a uniform

distribution of random deviates ee x of the PUG This ensur es no

bias whatso ever is intr o duc e d the initialization

Minimal algorithmic comp onen ts

Selection UG x

PIKAIA uses astoc hastic selection pro cess to assign to eac h individual n i the p op

ulation a pr ob ability of being selected for breeding Sp ecially probabilit y

is made linearly prop ortional the nessased r ank of eac h individual within

the curren t p o pulation This is carried using a heme kno wn as the R oulette

Whe el A lgorithm as detailed x of the PUG ee also Da vis c hap

Note that general it is a go o d idea to e selection probabilit y

prop ortional to ness value as often leads to a of selection pressure

late in the ev olutionary once p opulation b ers v e ound the

global optim um some it can also lead early on to a up e

b eing selected so frequen tly that the p opulation b ecomes degenerate through the

computational equiv alen t of breeding The prop ortionalit y constan t bet w een

nessased rank and selection probabilit y is sp ecid as an input parameter

PIKAIA The default v alue is

Breeding UG xx and

Once t w o individuals ha v e b een selected breeding pro ceeds exactly as in Figure

The enco ding pro cess requires one to sp ecify the n ber of digits to be retained

in the enco ing d pro ess c a userp ecid quan tit y whic h is set to in

calculations rep orted up on here his also the default v alue in PIKAIA Tw o

additional quan tities need to be sp ecid the crosso v er rate whic h sets the

is

all is this

um

to

in

rindividual cases In

ha mem most run

loss this

directly mak not in

in

sc out

to

that

by

initial that

in um are

two uction pro es

the con

is

trol

alues and the See ctrl

PIKAIA are

robust

probabilit y that the crosso v er op eration actually tak es place efault

the m utation rate whic h s ets the probabilit y for e m up

string of an opring that a m utation tak es place at that lo cation

is

P opulation replacemen t UG x

Under PIKAIA default settings t he opring p opulation is accum ulated in to tem

poaryr storage and once n um ber of suc h opring equals of the curren t

breeding p pulation o the latter is deleted and replaced b y the opring p opulation

This is the default strategy used b y PIKAIA although it is p ossible for the

sp ecify other p opulation replacemen t tec hniques ee PUG xx

Additional comp onen ts

The comp onen ts listed ab o v e a minimal genetic algorithm Suc h an

algorithm can be used for n umerical optimization but as w e so on see

out to be far from optimal what follo ws w e refer to this algorithm as GA

The ollo f wing t w o imple s additions o t A G lead to an algorithm to b e referred to

as GA that ac hiev es far b etter p erformance on n umerical optimization problems

So m uc h b etter in fact that use of these t w o additional comp onen ts is the

default c hoice in PIKAIA

Elitism UG x

This simply consists in storing a w a y the parameters deing the test mem ber

of curren t p opulation and later cop ying tact the opring p

This represen ts a s afeguard gainst a the p ossibilit y that crosso v er andr m utation

destro y he t curren t b est solution hic w hw ould a h v e a go o d c hance of unnecessarily

wing do wn the optimization pro cess Elitism in b comes e essen up on

in tro ucing d our second impro v emen t to GA

V ariable m utation rate UG x

This one is p erhaps the single most i mp ortan t mproi v emen t t hat can nd should

be made to GA As discussed x m utation is v ery m uc h a mixed blessing

F or those of y ou t w an t to run PIKAIA to repro duce the

b elo w GA is pro d uced b y explicitly setting the follo elemen of PIKAIA

con trol v ector ctrl ctrl ctrl and all other e lemen ts of ctrl

negativ e v alues to activ ate default options

This means initializing al l elemen of the trol v ector negativ e

v alues Note that this sets a p opulation size equal to ia ctrl and a

n ber of generations equal to ia ctrl

um

to ctrl con ts

to

ts wing

results migh who

in

vital

tial fact slo

opulation in in it the

the

In

turns will

dee

to user

that the

efault digit

deing the aking digit ach

is

it vides the m uc h needed source of v ariabilit y through h no v el parameter

v alues are injected i n to the p opulation Ho w ev er it also leads to the destruction of

go o d solutions This w as precisely the p oin t of igure F dotted line Finding the

exact v alue for the m utation ate r t hat ac es optimal balance b t e w een those t w o

ects to maximize t he former while minimizing the latter is of course p ossible

Ho w ev er in doing ds that the optimal parameter settings often end up

b eing highly problem dep enden t

One po w erful solution to this problem to dynamically adjust the m utation

rate The k ey to this strategy lies with recognizing that as long as the p opulation

is broadly distributed in parameter space the crosso v er op erator leads t o a prett y

eien t earc h as it recom bines fragmen ts of existing solutions Ho w er once

the p opulation has con v hether on a secondary or absolute optim um

crosso v no longer ac hiev es m uc h as it leads to hange of fragmen ts that

are early n iden tical since all paren ts ha v e n early iden tical parameter v alues

ob viously is where a high m utation rate is needed to reinject v ariabilit y in to the

p opulation

Consider then the follo wing pro cedure A t an y giv en time k eep trac k of the

ness v alue of t he test p opulation mem b er and of t he median rank ed mem ber

The ness dirence f bet w een those t w o individuals is clearly a measure of

p opulation con v ergence if f large the p opulation is presumably distributed

more broadly in parameter space than if f is v Therefore if f

b ecomes to o small increase the m utation rate if it besomec to o large decrease

the m utation rate again This ho w PIKAIA dynamically adjusts m utation

rate during runime This strategy represen a s imple of selfdaptation of

a arameter p con trolling the b eha vior of the underlying genetic algorithm F

details and implemen tation issues are discussed x of the PUG

A case study GA on P

It will pro v e u seful to st tak e a etailed d lo ok at the beha vior of the genetic

algorithm in the con text of a s imple problem Figure sho ws ten con v ergence

curv es for GA w orking on P with N What plotted is min us the

p

In fact this often done b y letting the m utation rate nd other con

parameters of the algorithm ev e under trol of a second higher lev

genetic algorithm with ness b eing t hen deed as the p erformance of the genetic

algorithm deed b y t hose parameters on the p roblem under consideration Prett y

cute but as y ou migh t i magine ather r time consuming

Just as the optimal m utation y ou op efully w ork out in Problem

of x is rather sensitiv ely d ep enden t on the sen tence length and alphab e t size

ed rate

el con the olv

trolling is

one is

in

urther

form ts

its is

small ery

is

This

exc the er

erged

ev

is

one so

hiev

whic pro

ness v alue of the test individual v ersus generation coun t for separate

of GA Figure should be compared to Figure the con v ergence

of the simplex on the same problem Early on the curv es ha v e qualitativ

similar shap es either con v ergence o ccurs relativ ely uicq kly h more quic

for simplex when it do es con v erge or solutions remain k on one of the

rings of secondary extrema f Fig whic h leads the error lev o at a

ed v alue Unlike simplex however GA is able to pul l itself o the se c ondary

extr ema rings It do es so primarily through m utation although crosso v er b et w een

t w o paren ts prop erly p ositioned in parameter space can ac e the ect

Mutation b eing a fundamen sto c hastic pro cess is then surprising

see diren t runs requiring diren t generation b e needed

fa v orable m utation tak place

Clearly m utation pla ys a c ritical role here Figure sho ws the nesses of

the b est olid line and medianank ed ashed ine l individuals in the p opulation

as a function of generational t for the GA run plotted a thic k er line

on panel The otted d ine l sho ws the v ariation of the m utation rate Figure

sho ws the distribution of the p opulation in D parameter space at the ep o c hs

indicated b y solid dots Fig

T o start with note on that no individual in the initial random

p opulation has landed an ywhere close enough to cen tral p eak for hill clim

w ork The rst few generational iterations see the p opulation cluster itself closer

and loser c to en c ter ig but the ness dirence b t e w een b est and median

is still quite large The m utation decreases sligh tly rom f i ts initial lo w v

th

then remains constan t By the generation Fig ost m of the p opulation

has con v erged somewhere inner ring of secondary extrema f

that the nesses of the bets and median are no w comparable This leads to a

th

sharp increase of the m utation rate et w een the and generations The

high m utation rate results in opring begin kno c k ed all o v er parameter space

in the course of breeding ig some m utan t individuals do land

regularly on the slop e of cen p it is only b y the generation that

one suc h m utan t is catapulted high enough b ecome the test of the curren t

p opulation ig F urther breeding during subsequen t generations

more and more individuals to the cen tral p eak and further increases ness of

Y ou migh t notice that GA already starts o doing signian tly b e than

the simplex metho d this merely results from the initial random p opulation of

GA ha ving ampled poni ts parameter space compared only for the

simplex

An animation of the ev olving p opulation for this solution can be ed on

the T utorial W eb P age

view

to in

tter

in

brings

to

eak tral the

th

While

th

so

the on

but alue

to

bing the

Fig

on

with coun

es

the fore ts coun GA

to not it tally

same hiev

eling to

tuc

kly uc

ely

wing sho

runs

Figure P anel sho ws con v ergence es for distinct of GA on

P As b efore the error is deed as f x y P anel B sho ws for the single run

plotted with a thic k er line on panel t he v ariations with generation coun tof hte

bets individual of the p opulation olid line medianank ed individual ashed

line m utation rate otted line

and

runs curv

Figure Ev olution of the p opulation of trial s olutions parameter space

the GA run sho wn as a thic k er line on Fig The concen tric circles indicate the

rings of secondary maxima and the l arger solid blac k d ot is the test solution of

the curren t generation

for in

the curren t b est via b oth crosso v er and m utation ig Note ho w litism e is

essen tial here otherwise the utan t ha ving landed on the slop es of the cen tral

p eak w ould ha v e a w lik ielhood of replicating itself tact in to the subsequen t

generation in view of the high m utation rate

GA basically b eha v es in exactly the same w a y w ith the imp ortan t exception

that man y more generations needed for the v orable m utation to sho w up

this is b e cause GA op rates e w ith a ed lo wm utation rate G A

rate v ary dep ending on the degree of con v ergence of the p opulation f x

Hamming w alls creep m utation

W e are doing prett yw ell ith w GA but w e still need to correct a fundamen tal

shortcoming of the one oin tm utation op erator arising from the ecimal d enco ding

heme of Fig Consider a problem where sough tfter optimal solution

requires the follo wing substring to be pro duced b y the ev olutionary pro cess

deco ding in to the ating p oin tn b er no w early n i the ev olutionary run

an individual ha ving sa y

will lik ely b e ter than a v erage and so this genetic material ill w spread through

out the p opulation After a while follo wing v orable m utations or crosso v

recom binations the substring migh t lo ok lik e sa y

whic h is admittedly quite close to Ho w ev er t w o v w ell co ordinated

m utations are needed o t p ush t his to w ards the target ust m utate

a the st o a Note hat t either m utation o ccurring in isolation

andr m utating to a diren t v alue tak es us farther from the t arget ating

p oin t n b er Mutation benig a slo w pro cess probabilit y of the needed pair

of m utations o ccurring sim ultaneously will in general b e quite s mall meaning that

the ev olution w ould ha v e t o b e pushed o v er y generations for it to app h en The

p opulation is getting iled at in ternal b oundaries of the enco ding system

These b oundaries are called Hamming wal ls They can b e b ypassed b yc ho os

ing n a enco ding sc heme suc h that successiv esiglne m utations can alw a ys to a

con uous v ariation in the deco ded p arameter This is wh y the soalled Gra y bi

nary co ding Press et al x is no w used almost univ ersally in genetic

algorithms based on binary enco ding Another p ossibilit y is to devise m utation

op erators that can jump o v er Hamming w alls

tin

lead

up

man

the um

digit

and to

the

ery

er fa

um

the sc

and

this lets while

fa are

in lo

Cr e ep mutation do es precisely this Once a digit on the enco ding string

b een targeted for m utation instead of replacing the existing digit b y a

randomly c hosen one just add either or ith equal probabilit y and if

the resulting digit ecause a has b een with or bauecse

a has b een hit carry the one o v er to the next digit on the

Just lik e in grade sc ho ol So example creep m utation hitting the middle

with in the last substring ab o v e w ould to

whic h ac hiev es the desired ect of umping the w all

The one thing creep m utation do es not wis to akt e l arge jumps in p arame

ter pace s As argued b fore e j umping is actually a n eeded apabilit c y consequen

in practice for eac h opring individual a probabilit y est t will decide whether one

p oin t or creep m utation i s to be used ith equal probabilities

Creep m utation i s included the original release of PIKAIA o w kno wn

as PIKAIA although it in v ersion h b een released in

ee the PIKAIA W eb P age and the Release PIKAIA NCAR

T ec hnical Note STR results describ ed follo ws w ere obtained

using a mo did v ersion of PIKAIA G A whic h includes m

is otherwise iden tical GA

P erformance on test problems

Time w turn lo ose our algorithms on the suite of problems x

W e ha v e three v ersions of genetic algorithmased optimizers GA whic h rep

resen ts a minimal algorithm a nd GA whic h tical to GA includes in

addition elitism and dynamic adjustmen t of the m utation rate GA includ

ing creep m utation otherwise iden tical to GA As a comparison algorithm

w e r etain iterated hill clim bing using the simplex metho d as describ d e in x As

will b ecome eviden t GA is actually not a v ery go o d optimizer so that the

more in teresting comparison will b e among GA GA and iterated simplex

Before getting t o o carried a w a y l et ause p and rect on what w e to

ac hiev e here Ideally ne w an ts a metho d t hat ac es con v ergence to

optim um with high probabilit y p sa y while requiring smallest

G

p ossible n um ber of mo del r function ev aluations doing so This latter p oin t

can b ecome a dominan t constrain t dealing with a application where

ev aluating the tness of a en trial solution computationally in tensiv e

Consider the helioseismic in v ersions describ ed in Charb onneau et al

using a genetic algorithm giv en a set of parameters deing a trial solution ness

is giv

real when

in

the

global the hiev

trying are

on so

but

and

but iden is

of test to no

to

but utation creep

what in The

for Notes

April has whic is

in not

tly

allo

lead

for

left with

hit is

simply

has

Suc h considerations are easily quan tid Let N and N be the p opulation

p g

size and generation length a the required n ber of function ev

N is ob viously

f

N N N A GA GA a

f p g

while for iterated simplex N is the n um ber of hill clim bing trials N times the

f t

a v erage n ber fo function aluations r equired b y a single simplex run N

s

quan tit y is run and problemep enden t

N N N terated simplex b

f t s

So w e pla y the follo wing game w e run iterated simplex and GA increas

ing n um b ers of generationsterations and c hec k whether global con v ergence is

ac hiev ed to get statistically meaningful results w e times for eac h

metho d and eac h generationteration coun t allo ws to empirically estab

lish the probabilit y of global con v ergence p as a function of genera

G

tionteration coun t In doing so to decide whether or not a giv en h as

con v erged w e use again the f for P P P and R

P The results of his t p ro cedure applied to eac h t est problem is s ho wn in Figure

It should be easy to con vince y ourself of the follo wing on P and P

b oth iterated implex s and GA p erform equally w ell on all asp cts e of p e rformance

when pushed long enough to ha v e p P is a and

G

tec hnique p erforms satisfactorily on it Still GA outp erforms iterated

simplex on global p erformance P GA iterated simplex do

w ell up to p but then GAs p erformance starts to lag b ehind as the

G

solutions are pushed to p

G

An ob vious conclusion to be dra wn at this juncture is that iterated

clim bing using the simplex metho d es for a prett y decen t global optimiza

tion sc heme Not quite y ou w ere exp e cting as a sales pitc h for genetic

algorithmased optimization righ t This is in a consequence of the rel

ativ ely lo w dimensionalit y of our test problems Recall x that iterated

simplex leads impro v ed p erformance ith resp e ct to single run simplex pri

marily as a consequence of the b etter sampling of arameter p space asso ciated with

the initial andom distribution of simplex v ertices giv en enough trials one is

almost guaran teed v e one initial simplex v ertex landing close enough to the

ev aluation in v olv es the construction of a rotation curv e a l arge matrix

v ector m ultiplication the calculation of a against some data p oin

This adds up to ab out half a PUecond C on a Cra y J All test p roblems of x

require v little computation in comparison

ery

ts

ha to

to

from

part

what

mak

hill

equally and On

largely

neither problem hard

for and criteria

globally run

us This

this do

for

this ev um

aluations um run of

Figure Global con v ergence robabilit p y as a function of the n um b er of unction f

ev aluations N required b y iterated simplex iamonds and GA olid dots on

f

the four test problem dotted line P dashed line P solid line P dash

dotted line The probabilities w ere estimated from distinct trials and in he t

case of iterated simplex N is an a v erage o v er the trials

f

global maxim um to ensure subsequen t global con v ergence w dimensional

searc h spaces iterated simplex th us ends up bengi quite comp e titiv e Figure

already indicates that dge do es not carry o v er to higher dimensionalit y

ompare results for P and P

GAs p erformance on P is actually a delicate matter T ak e another lo ok

at Figure consider what happ ens the p opulation has v erged to the

broad secondary maxim s do es early in the run for nearly ev ery single

trial for m utation to prop el a olution s from x y to the n arro wpake

x y t w o v w co ordinated m utations m ust e place sim ul

taneously otherwise m utan t solutions up regions of rather lo w elev ations

and do not con tribute m uc h the next generation a w probabilit y

o ccurrence ev en at relativ ely high m utation rates the pro c ess tak es

GAs global p erformance on P then results from an in terpla ybet w een one oin t

Notice on Figure ho w f ew solutions sho w up n i the corners of he t

time so

lo is This to

in end

tak ell ery

it um

con once and

this

lo In

m utation the rather direct relationship that exists bet w een a solution

deing parameters and its string represen tation on hicw hm utation and c rosso v

op erate If the narro w Gaussian is cen tered on x y then a single

m utation can prop el a solution from the broad cen Gaussian to the narro w

one Not surprisingly on this mo problem GA outp erforms iterated sim

plex to a signian t degree p with only N i faster than

G f

iterated simplex b y a factor of Enco ding is a tric ky business with p

fareac hing consequences p erformance

Iterated simplex sup erior p erformance on P is certainly notew y y et

rects in part the p eculiar structure of parameter space deed b y the Gaussian

ting problem whic h is relativ w accommo dated b y the simplex metho d

pseudolobal capabilities Other lo optimization metho ds do not

as w ell F or a detailed comparison of genetic algorithms and other metho d s on

ting Gaussian proes to real thetic data see McIn tosh et al

It is really only with v ery hard problems suc h as P that GA starts sho

w orth By an y s tandards P s i a v ery hard global optimization roblem p While

on its D v ersion GA nd a iterated simplex do ab out as w ell as dimensionalit yis

increased the global p erformance of iterated simplex egrades d m uc h

than GA This is in fact the po w er of algorithmased optimizers

lies although for searc h spaces of high dimensionalit y n s a y the one oin t

crosso v m utation op erators describ ed in x usually sub optimal and

m be impro v ed up on

In some sense a fairer comparison of the resp ectiv e exploratory capabilities

of iterated can be carried b y setting n um ber of trials

in iterated simplex so that original distribution simplex v ertices samples

parameter space with same densit y as GAs initial random p opulation in

other w ords using the notation of w e

N N n

t p

where n is the dimensionalit yof parameter space and compare results of the

resulting iterated simplex to some tandard GA and GA runs Suc h a

comparison is presen in T able II in a format essen tially tical to T able I

P erformance measures also listed for a set of GA runs extending o v er the

domain

PIKAIA to be released in April includes a t w o oin t crosso v er op

erator whic h generally impro v es p erformance for problems in v olving man y pa

rameters See e Section fthe Release Notes for harb onneau

PIKAIA

are

iden ted

runs

the

set eqs

the

of the

the out simplex and GA

ust

are and er

genetic where

rapidly more

its

wing

syn and

nearly fare cal

ell ely

orth

on

tially oten

did

tral

er

here and

T able II

P erformance on test problems ith eq enforced

T est Problem P erformance Iter Simplex G A GA GA

P h f i

p

G

h N i

f

N N

t g

P h f i

p

G

h N i

f

N N

t g

P h f i

p

G

h N i

f

N N

t g

P h R i

p

G

h N i

f

N N

t g

same n um ber of generations as the GA and GA Once again p e rformance

measures are established on the basis of distinct runs for eac h metho d

Eviden tly GA and GA outp erform GA on all asp e cts of p rformance e to a

staggering degree GA is not m uc h of a global n umerical optimization algorithm

Comparison with T able I sho ws that its global p erformance exceeds somewhat that

of the s implex metho d n i singleun o m de but the n um b er of unction f v e aluations

required b y GA to ac hiev e orders magnitude l arger

What is also lainly p eviden ton T able II i s the degree to whic h GA GA

outp erform iterated simplex for a given level of initial sampling of p ar ameter sp ac e

Although the n ber of function ev aluations is t ypically an order of

magnitude larger b oth algorithms are far b etter than iterated simplex at activ

exploring parameter space This is plain evidenc efro the p ositive e cts of tr ansfer

of information b etwe en trial solutions in c ourse of the se ar ch pr o c ess

The w orth of creep m utation can be ascertained b y comparing

global the

the

ely

required um

and

of is this

runs

or

or

or

or

p erformance of the GA and GA solutions results are not clearut

do es b etter than GA on P alttewPil orse on P and signian w orse

on P The usefulness of creep m utation is con tingen t on there actually b

Hamming w alls in vicinit y of the global solution if there are creep m utation

helps sometimes uite q a bit Otherwise ectiv ely d ecreases probabilit yof

taking large jumps parameter space and can be deleterious cases

This is what is happ ening here P where mo ving a w a y the secondary

maxim um requires a large in parameter space to tak e place x y

to

A tan y rate the ab o v e iscussion d amply llustrates i the degree to whic h glob al

p erformanc e oblemep endent This cannot be o v eremphasized Y ou should

certainly b ew are of an y empirical comparisons b et w een v arious global optimization

metho d s that rely on a small of test problems esp cially e of w dimensional

it y Y ou should also k eep in mind that GA one sp eci instance of a genetic

algorithmased optimizer and that other incarnations ma y beha v e diren tly

ither betert or w orse the same test problems

The least y ou should ber Section

Through random initialization of the p opulation genetic algorithms in

no initial bias whatso ev er in the h pro cess

F or n umerical optimization elitism and an adjustable m utation rate are t w o

crucial additions a genetic algorithm

Iterated hill clim bing using t he simplex metho d m ak es a prett y ecend t

optimization t ec hnique esp ecially for lo wimensionalit y problems

P erformance measures of an y global optimization metho d are ighly h problem

dep enden t

Exercises for Section

Lo ok bac k at Figure The dynamically adjusting m utation rate lev els o

at a v alue of ab out One could ha v e predicted a v erage v alue b efore

running the co de Ho w in t reead x

Co de up a D D and D v ersion of P in its GA form

efault settings except for generation coun t in v estigate ho w global p erfor

mance degrades with problem dimensionalit y Keep the generation oun c etxd

at ctrl Ho w do es compare to iterated simplex

this

PIKAIA Using

this

global

basic to

searc

duce tro

from remem

on

is

lo set

pr is

from jump

from with

some in so in

the it

the

eing

tly and

GA The

A REAL APPLICA TION

ORBIT BINAR Y ST ARS

Binary stars

More than half all stars observ ed in the solar neigh b orho o d are comp onen

of binary systems This is presumed to be a consequence of angular momen tum

conserv ation leading to fragmen tation in the later tages s of collapse of protostellar

clouds Some binary stars can b e resolv ed optically with ev en a s mall telescop e the

st suc h visual binary w as disco v ered b y G Riccioli Binarit y

can also be established sp ectroscopically y measuring the small Doppler shift in

narro w sp ectral lines caused b y the comp onen t V along linefigh t of the

orbital v elo cit y ab out the system ter of mass F or nonelativistic

sp eeds the w a v elength shift is

V

c

where c is the sp eed of ligh t The st suc h e ctr w as disco v ered

in b y E Pic k ering Curren t hardw are and analysis tec hniques no w allo w

us to measure tellar s radial v c y w ith useful accuracy do wn to p r e

second This lev el of accuracy is what has made p ossible the t sp ectacular

disco v ery of extrasolar planets Figure sho ws radial v c it y measuremen ts of

the s tar Bo otis a lassical sp ectroscopic b inary star F rom these data one can

determine the orbital parameters of the system

Am usingly sp ectroscopic binary the brigh ter comp onen t of the

visual binary disco v ered b y Riccioli the star Mizar A in the constellation

Majoris Ev en b tter e w as later realized that Mizar B is a sp ectroscopic

binary

This ure is for bac k in this er w as originally

it w as giv en as m s Prett y remark able impro v emen t a little o v

three y ears

er just in

written pap when

also it

Ursa

st is this

elo

recen

meters it elo

binary opic osc sp

orbital cen

the

in system

ts of

OF ELEMENTS AL

Figure Radial v c y v ariation observ ed in the sp ectroscopically visible

comp onen t of the binary star Bo axis is giv units of Julian

Date ne JD one solar y are from Bertiau with one lone

datum at JD on this plot solid is the b estt solution

obtained later section The asymmetrical shap e of curv e is due to the

eccen tricit y of the orbit a circular orbit w to a purely sin usoidal

v elo cit yv ariation

Radial v elo cities and Keplerian orbits

If the shap e and size of an orbit are kno wn as w as orien tation of its semi

ma jor axis with resp ect to the l ine of s igh t the exp ected radial v cit yv ariations

can b e computed and compared to observ ations There are actually a few subtleties

in v olv Determining the radial v c y v ariation asso ciated the motion of

a b inary comp onen t in an arbitrarily p ositioned elliptical orbit ab out the common

cen of mass of the s ystem i s a straigh tforw ard b ut somewhat messy problem in

spherical trigonometry The pro cedure is in great gory details in man y

astronomical monographs ee e mart S W e shall simply write d o wn the

resulting expression here

V t V K os v t e cos

out laid

ter

with it elo ed

elo

the ell

radial lead ould

the this in

line The missing

Data da

in en time The otis

it elo

The quan tit y V is the radial v c y of the binary system cen of mass and

eq only holds once the Earths orbital motion ab the Sun as been sub

## Comments 0

Log in to post a comment