Design and Analysis of Algorithms

courageouscellistAI and Robotics

Oct 29, 2013 (3 years and 7 months ago)

54 views

Design and Analysis of Algorithms

Branch and Bound Algorithms


Chuhua

Xian

Email: chhxian@scut.edu.cn

School of Computer Science and Engineering


Course Information


Website


http://222.201.132.37/algorithms
/


Textbook



Design and Analysis of Algorithms

Branch and Bound Algorithms

Topics:


General Method


Least Cost Search

Searching in S
tate Space Tree


Basic Concepts


Alive node
: a generated node whose sons have not all
been generated.


E
-
node
: the node whose son is generating currently.


Dead node
: a node that all his sons have been
generated or there is no need to generate his sons.


Generating problem states in BFS way


A
E
-
node remains E
-
node until it becomes a dead
node.

E
-
node

E
-
node

Branch & Bound Paradigm

D
-
Search

This method differs from BFS in that the next node to
explore is the node most recently added

Alive node table


Branch and Bound needs a table which holds
the alive nodes.


A FIFO(First In First Out) table for BFS


Queue


A LIFO(Last In First Out) table for D
-
search


Stack

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30


8

2

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

3

4

5

2

7

8

3

11

3

4

12

12

4

5

15

16

7

7

20

8

11

22

22

11

12

25

25

15

15

16

20

20

22

25


:alive node


: E
-
node


:dead node

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30


8

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

2

3

3

4

12

4

5

7

7

8

11

22

11

12

25

15

15

16

20

20

22

25

If our goal is to find all feasible
solutions, the number of nodes
generated is same as that of a back
-
tracking algorithm.

But if only
one feasible solution

is
required, then we find node
31

later
than a back tracking algorithm.


:alive node


: E
-
node


:dead node

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30


8

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

2

3

3

4

12

4

5

7

7

8

11

22

11

12

25

15

15

16

20

20

22

25

In LIFO or FIFO branch
-
bound
algorithm, we are quite hidebound
when selecting the next E
-
node. Our
choice is not helpful to find an answer
more quickly.


:alive node


: E
-
node


:dead node

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30


8

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

2

3

3

4

12

4

5

7

7

8

11

22

11

12

25

15

15

16

20

20

22

25

Improve FIFO or LIFO rule: use a
“intelligent” sorting function c(.) to select
next E
-
node from the alive nodes, so as to
find an answer more quickly.


:alive node


: E
-
node


:dead node

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30


8

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

2

3

3

4

12

4

5

7

7

8

11

22

11

12

25

15

15

16

20

20

22

25

Measurement

to alive node X:


瑨e number o映nodes in 瑨e 瑲ee roo瑥d by X
need 瑯 genera瑥 be景re 晩nding an answer


瑨e leng瑨 o映瑨e pa瑨 晲om X 瑯 瑨e neares琠
answer node in 瑨e 瑲ee roo瑥d by X.

The value calculated on X under a specific
measurement is defined as the
cost

of X.



4



3



3



2



1



1


:the cost of the nodes which can not
result in answer should be bigger than
that of the nodes which can result in
answer (maybe




:alive node


: E
-
node


:dead node

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30


8

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

2

3

3

4

12

4

5

7

7

8

11

22

11

12

25

15

15

16

20

20

22

25

Measurement

to alive node X:


瑨e number o映nodes in 瑨e 瑲ee roo瑥d by X
need 瑯 genera瑥 be景re 晩nding an answer


瑨e leng瑨 o映瑨e pa瑨 晲om X 瑯 瑨e neares琠
answer node in 瑨e 瑲ee roo瑥d by X.

The value calculated on X under a specific
measurement is defined as the
cost

of X.



4



3



3



2



1



1


:alive node


: E
-
node


:dead node

If we use measurement

, 瑨en a branch
-
bound algori瑨m is expec瑥d 瑯 genera瑥
as 晥w nodes as possible.

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30


8

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

2

3

3

4

12

4

5

7

7

8

11

22

11

12

25

15

15

16

20

20

22

25



4



3



3



2



1



1


:alive node


: E
-
node


:dead node

Define the “
intelligent
” sorting function (
cost
function
) as
c(.)
.

The value of c(X):latter
-
wit

It is unpractical to get accurate cost function.

We just sort the alive nodes using an
approximate cost estimating function
ĝ(.).


General Method
——
LC Search


Define ĝ(X) as an estimating function of the cost
from node X to an answer node. The order of
checking the alive nodes if determined by ĝ(X).


Is it suitable to use only ĝ(X) to sort alive nodes?

X

Y

E
-
node(the node
whose estimated cost
is the least)

Usually

ĝ(Y)

ő;yͿ

E
-
node

X

Y

E
-
node



The nodes excluded in the sub
-
tree
rooted by X will not be checked
until all this sub
-
tree is checked.

Using only ĝ(X) will lead to “depth
first search”.

If ĝ(X) is exactly c(X), it is OK.

But, ĝ(X) is only an
estimation

of
the accurate cost, so do depth first
search may waste our time.

General Method
——
LC Search


To avoid fulsome depth first search, the cost estimating
function should be revised, so as to take into account the
cost from the root to X
h(x)

(besides the cost from X to an
answer node).


Define a new estimating function
ĉ(X)
:
ĉ(X)

f(h(X))+ĝ(X)


To make sense,
f(.)

should be a non
-
desending

funciton
. Use of
f(x) would be helpful to avoid fulsome depth first search, making
the algorithm to select the node nearer to an answer (estimated)
and nearer to the root.


An strategy using the estimating function
ĉ(X)

f(h(X))+ĝ(X)

will always select the node whose
ĉ(.)

is minimum as the
next E
-
node. We define this kind of strategy
Least Cost
Search
, and
LC
-
Search

in short.

General Method
——
LC Search


If bound function is used in LC
-
Search, we call
it
LC
-
Branch
-
Bound
-
Search
.

15
-
Puzzle


There are
15

numbered block on a
16
-
grid
chessboard,
given an initial arrange, you are required to change it
to the target arrange shown in Figure (b) through a
series of valid moving (only movings of a block abutting
on the blank space to the blank space are valid).

1

3

4

15

2

5

12

7

6

11

14

8

9

10

13

(a) Initial arrange

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

(b) Target arrange

15
-
Puzzle


State
: each valid moving results in a new arrange, which is called a state of the
problem.


Initial state

and
target state
: the initial arrange and the target arrange.


A state is
reachable

iff there are a series of valid movings from the initial state to it.


The state space of an initial state consists all the states reachable from it.


There
16!
≈20.9
×
10
12

different arranges.


Any initial state can reach about half of the arranges. The state space is quite large!

1

3

4

15

2

5

12

7

6

11

14

8

9

10

13

(a) Initial arrange

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

(b) Target arrange

15
-
Puzzle


It is necessary to know whether the target state is reachable before moving.


Number the grid by the block in the target arrange, and number the blank space grid
16
.


POSITION(i)

is the position of block
i
in the initial arrange.


LESS(i)

is the number of such block
j

which is less than
i

and
POSITION(j)>POSITION(i)
.


X
: If in the initial arrange, the blank space is in a shadowed place shown in Figure(c),
X=1
;
otherwise
X

0
.

(c)

1

3

4

15

2

5

12

7

6

11

14

8

9

10

13

(a) Initial arrange

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

(b) Target arrange

15
-
Puzzle

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

Moving the block
equals to moving
the blank space

Generate sons in up

right

down

le晴f
order.

Search in DFS way.

Abandon s瑡瑥s seen be景re.

1

2



4

5

6

3

8

9

10

7

11

13

14

15

12

up

1

2

4



5

6

3

8

9

10

7

11

13

14

15

12

right

1

2

4

8

5

6

3



9

10

7

11

13

14

15

12

down

1

2

4

8

5

6

3

11

9

10

7



13

14

15

12

down

1

2

4

8

5

6

3

11

9

10

7

12

13

14

15



down

1

2

4

8

5

6

3

11

9

10

12

13

14


7


15

left

1

2

4

8

5

6



11

9

10

3

12

13

14

7

15

up

1

2



8

5

6

4

11

9

10

3

12

13

14

7

15

up

1

2

4

8

5

6

3

11

9

10

7

12

13

14




15

up

1

2

8



5

6

4

11

9

10

3

12

13

14

7

15

right

1

2

8

11

5

6

4



9

10

3

12

13

14

7

15

down

Searching is hidebound.

We are far and far away from the
target state!!!



15
-
Puzzle

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

7

8

9

10



11

13

14

15

12

1

2

3

4

5



6

8

9

10

7

11

13

14

15

12

up

right

down

left

1

2

4



5

6

3

8

9

10

7

11

13

14

15

12

1



2

4

5

6

3

8

9

10

7

11

13

14

15

12

right

left

1

2

3

5

6

8

4

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

11

9

10

7

13

14

15

12

up

down

1

2

3

4

5

6

7

8

9

10


11



13

14

15

12

1

2

3

4

5

6

7

8

9

10


15

11

13

14



12

1

2

3

4

5

6

7

8

9



10

11

13

14

15

12

right

down

left

up

down

left



1

2

4

8

5

6

3



9

10

7

11

13

14

15

12

down

1

6

2

4

5



3

8

9

10

7

11

13

14

15

12



1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

down

left

1

2



3

5

6

8

4

9

10

7

11

13

14

15

12

left

1

2

3

4

5

6

8

11

9

10

7

12

13

14

15



1

2

3

4

5

6

8

11

9

10



7

13

14

15

12

down

left

1

2

3

4

5

6

7



9

10


11

8

13

14

15

12

1

2

3

4

5

6

7

8

9

10


11

12

13

14

15



down

up

Searching in BFS way

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

15
-
Puzzle

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

7

8

9

10



11

13

14

15

12

1

2

3

4

5



6

8

9

10

7

11

13

14

15

12

up

right

down

left

1

2

4



5

6

3

8

9

10

7

11

13

14

15

12

1



2

4

5

6

3

8

9

10

7

11

13

14

15

12

right

left

1

2

3

5

6

8

4

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

11

9

10

7

13

14

15

12

up

down

1

2

3

4

5

6

7

8

9

10


11



13

14

15

12

1

2

3

4

5

6

7

8

9

10


15

11

13

14



12

1

2

3

4

5

6

7

8

9



10

11

13

14

15

12

right

down

left

up

down

left



1

2

4

8

5

6

3



9

10

7

11

13

14

15

12

down

1

6

2

4

5



3

8

9

10

7

11

13

14

15

12



1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

down

left

1

2



3

5

6

8

4

9

10

7

11

13

14

15

12

left

1

2

3

4

5

6

8

11

9

10

7

12

13

14

15



1

2

3

4

5

6

8

11

9

10



7

13

14

15

12

down

left

1

2

3

4

5

6

7



9

10


11

8

13

14

15

12

1

2

3

4

5

6

7

8

9

10


11

12

13

14

15



down

up

Searching in BFS way

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

target

The answer state found is the
one nearest to the root.

Is there an “intelligent” way?

Assign a cost c(X) to each node in
the state space tree.

Searching in a fixed way,
whatever the initial state is.

Hidebound again!

up

15
-
Puzzle

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

7

8

9

10



11

13

14

15

12

1

2

3

4

5



6

8

9

10

7

11

13

14

15

12

1

2

4



5

6

3

8

9

10

7

11

13

14

15

12

1



2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

3

5

6

8

4

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

11

9

10

7

13

14

15

12

1

2

3

4

5

6

7

8

9

10


11



13

14

15

12

1

2

3

4

5

6

7

8

9

10


15

11

13

14



12

1

2

3

4

5

6

7

8

9



10

11

13

14

15

12



1

2

4

8

5

6

3



9

10

7

11

13

14

15

12

1

6

2

4

5



3

8

9

10

7

11

13

14

15

12



1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2



3

5

6

8

4

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

11

9

10

7

12

13

14

15



1

2

3

4

5

6

8

11

9

10



7

13

14

15

12

1

2

3

4

5

6

7



9

10


11

8

13

14

15

12

1

2

3

4

5

6

7

8

9

10


11

12

13

14

15



c(.): set the cost of the nodes on the
path from the root to the nearest
target state, to the distance between
this target state and it; set the cost of
all other states to

.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

c(1)=3;c(4)=2;c(10)=1;c(23)=0

Others are



I琠is unprac瑩cal 瑯 calcula瑥 瑨e
accura瑥 c.

right

down

left

right

left

up

down

right

down

left

down

left

down

down

left

left

down

left

down

up

15
-
Puzzle


A practical way is to define a estimating function
ĉ(X)

f(X)+ĝ(X)

which is convenient to calculate, where


f(X)

is the length of the path from the root to X


ĝ(X)

is the minimum number of
movings

from X to the
target state.


A possible
ĝ(X)

is:






ĝ(X)

the number of blocks which are not in its
target position (except the blank space)


The real number of
movings

may be greater than ĝ(X), so
ĝ(X) is a low
-
bound of c(X).

1

2

3

4

5

6



8

9

10

11

12

13

14

15

7

ĝ(X)=1, but we need
more than one movings
to reach the target state.

15
-
Puzzle

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

7

8

9

10



11

13

14

15

12

1

2

3

4

5



6

8

9

10

7

11

13

14

15

12

up

right

down

left

1

2

3

4

5

6

7

8

9

10


11



13

14

15

12

1

2

3

4

5

6

7

8

9

10


15

11

13

14



12

1

2

3

4

5

6

7

8

9



10

11

13

14

15

12

right

down

left

down

up

1

2

3

4

5

6

7



9

10


11

8

13

14

15

12

1

2

3

4

5

6

7

8

9

10


11

12

13

14

15



1

2

3

4

5

10

11

12

22

23

ĉ(X)

I ; ƣ ;

f(X)=length of the path from the root to X

ĝ(X)

the number of blocks in wrong position

ĉ(2)

1+4=5

ĉ(3)

1+4=5

ĉ(4)

1+2=3

ĉ(5)

1+4=5

ĉ(10)

2+1=3

ĉ(11)

2+3=5

ĉ(12)

2+3=5

target

LC
-
Search is as efficient as having
the accurate cost function c(.).

With a proper ĉ(.), LC
-
Search can
be very efficient!

Shortest Path

1

2

5

4

3

10

20

50

100

30

10

60

Graph G

初始时,将源
[1, 0, 0]
放入
Open

Closed
为空。

Open table

[1, 0, 0]

Closed table

取出
[1, 0, 0]
放入
Closed

生成其后继
[2, 10, 1]


[4, 30, 1]

[5, 100, 1]
,并依序放入
Open


[1, 0, 0]

[5, 100, 1]

[4, 30, 1]

[2, 10, 1]

取出
[2, 10, 1]
放入
Closed

生成其后继
[3, 60, 2]

并依序插入
Open


[2, 10, 1]

[3, 60, 2]

[4, 30, 1]

取出
[4, 30, 1]
放入
Closed

生成其后继
[3, 50, 4]

[5, 90, 4]
,修订
Open
中已有的两个结点并依
序排列


[4, 30, 1]

[5, 90, 4]

[3, 50, 4]

取出
[3, 50, 4]
放入
Closed

生成其后继
[2, 100, 3]

[5, 60, 3]
,前者因劣于
Closed
中的
[2, 10, 1]

被抛弃,后者修订了
Open
中的
[5, 90, 4]


[3, 50, 4]

[5, 60, 3]

取出
[5, 60, 3]
,因其已经是目标结点,算法成
功并终止。

依据逆向指针可得最短路径为
1→4→3→5


Branch
-
Bound Algorithms


What else if we are trying to write down the
algorithm?


Alive nodes table


Queue

Stack

Heap…


Re
-
construct the answer


Store the tree


……