# Design and Analysis of Algorithms

Τεχνίτη Νοημοσύνη και Ρομποτική

29 Οκτ 2013 (πριν από 4 χρόνια και 8 μήνες)

75 εμφανίσεις

Design and Analysis of Algorithms

Branch and Bound Algorithms

Chuhua

Xian

Email: chhxian@scut.edu.cn

School of Computer Science and Engineering

Course Information

Website

http://222.201.132.37/algorithms
/

Textbook

Design and Analysis of Algorithms

Branch and Bound Algorithms

Topics:

General Method

Least Cost Search

Searching in S
tate Space Tree

Basic Concepts

Alive node
: a generated node whose sons have not all
been generated.

E
-
node
: the node whose son is generating currently.

: a node that all his sons have been
generated or there is no need to generate his sons.

Generating problem states in BFS way

A
E
-
node remains E
-
node until it becomes a dead
node.

E
-
node

E
-
node

D
-
Search

This method differs from BFS in that the next node to
explore is the node most recently added

Alive node table

Branch and Bound needs a table which holds
the alive nodes.

A FIFO(First In First Out) table for BFS

Queue

A LIFO(Last In First Out) table for D
-
search

Stack

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30

8

2

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

3

4

5

2

7

8

3

11

3

4

12

12

4

5

15

16

7

7

20

8

11

22

22

11

12

25

25

15

15

16

20

20

22

25

:alive node

: E
-
node

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30

8

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

2

3

3

4

12

4

5

7

7

8

11

22

11

12

25

15

15

16

20

20

22

25

If our goal is to find all feasible
solutions, the number of nodes
generated is same as that of a back
-
tracking algorithm.

But if only
one feasible solution

is
required, then we find node
31

later
than a back tracking algorithm.

:alive node

: E
-
node

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30

8

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

2

3

3

4

12

4

5

7

7

8

11

22

11

12

25

15

15

16

20

20

22

25

In LIFO or FIFO branch
-
bound
algorithm, we are quite hidebound
when selecting the next E
-
node. Our
more quickly.

:alive node

: E
-
node

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30

8

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

2

3

3

4

12

4

5

7

7

8

11

22

11

12

25

15

15

16

20

20

22

25

Improve FIFO or LIFO rule: use a
“intelligent” sorting function c(.) to select
next E
-
node from the alive nodes, so as to

:alive node

: E
-
node

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30

8

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

2

3

3

4

12

4

5

7

7

8

11

22

11

12

25

15

15

16

20

20

22

25

Measurement

to alive node X:

need 瑯 genera瑥 be景re 晩nding an answer

answer node in 瑨e 瑲ee roo瑥d by X.

The value calculated on X under a specific
measurement is defined as the
cost

of X.

4

3

3

2

1

1

:the cost of the nodes which can not
result in answer should be bigger than
that of the nodes which can result in

:alive node

: E
-
node

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30

8

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

2

3

3

4

12

4

5

7

7

8

11

22

11

12

25

15

15

16

20

20

22

25

Measurement

to alive node X:

need 瑯 genera瑥 be景re 晩nding an answer

answer node in 瑨e 瑲ee roo瑥d by X.

The value calculated on X under a specific
measurement is defined as the
cost

of X.

4

3

3

2

1

1

:alive node

: E
-
node

If we use measurement

, 瑨en a branch
-
bound algori瑨m is expec瑥d 瑯 genera瑥
as 晥w nodes as possible.

x
1

1

2

x
1

2

3

x
1

3

4

x
1

4

5

4
-
Queen problem

1

1

1

18

x
3

2

19

x
3

4

18

19

2

20

x
3

2

21

x
3

3

21

x
2

2

6

7

x
2

3

x
2

4

8

30

x
4

3

30

8

9

x
2

1

10

x
2

3

11

x
2

4

22

x
3

1

23

x
3

3

31

x
4

3

12

x
2

1

13

x
2

2

14

x
2

4

24

x
3

2

25

x
3

4

32

x
4

2

26

x
3

3

27

x
3

2

26

27

5

17

x
2

1

15

x
2

2

16

x
2

3

16

16

28

x
3

1

29

x
3

3

28

29

28

16

1

2

3

3

4

12

4

5

7

7

8

11

22

11

12

25

15

15

16

20

20

22

25

4

3

3

2

1

1

:alive node

: E
-
node

Define the “
intelligent
” sorting function (
cost
function
) as
c(.)
.

The value of c(X):latter
-
wit

It is unpractical to get accurate cost function.

We just sort the alive nodes using an
approximate cost estimating function
ĝ(.).

General Method
——
LC Search

Define ĝ(X) as an estimating function of the cost
from node X to an answer node. The order of
checking the alive nodes if determined by ĝ(X).

Is it suitable to use only ĝ(X) to sort alive nodes?

X

Y

E
-
node(the node
whose estimated cost
is the least)

Usually

ĝ(Y)

ő;yͿ

E
-
node

X

Y

E
-
node

The nodes excluded in the sub
-
tree
rooted by X will not be checked
until all this sub
-
tree is checked.

Using only ĝ(X) will lead to “depth
first search”.

If ĝ(X) is exactly c(X), it is OK.

But, ĝ(X) is only an
estimation

of
the accurate cost, so do depth first
search may waste our time.

General Method
——
LC Search

To avoid fulsome depth first search, the cost estimating
function should be revised, so as to take into account the
cost from the root to X
h(x)

(besides the cost from X to an

Define a new estimating function
ĉ(X)
:
ĉ(X)

f(h(X))+ĝ(X)

To make sense,
f(.)

should be a non
-
desending

funciton
. Use of
f(x) would be helpful to avoid fulsome depth first search, making
the algorithm to select the node nearer to an answer (estimated)
and nearer to the root.

An strategy using the estimating function
ĉ(X)

f(h(X))+ĝ(X)

will always select the node whose
ĉ(.)

is minimum as the
next E
-
node. We define this kind of strategy
Least Cost
Search
, and
LC
-
Search

in short.

General Method
——
LC Search

If bound function is used in LC
-
Search, we call
it
LC
-
Branch
-
Bound
-
Search
.

15
-
Puzzle

There are
15

numbered block on a
16
-
grid
chessboard,
given an initial arrange, you are required to change it
to the target arrange shown in Figure (b) through a
series of valid moving (only movings of a block abutting
on the blank space to the blank space are valid).

1

3

4

15

2

5

12

7

6

11

14

8

9

10

13

(a) Initial arrange

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

(b) Target arrange

15
-
Puzzle

State
: each valid moving results in a new arrange, which is called a state of the
problem.

Initial state

and
target state
: the initial arrange and the target arrange.

A state is
reachable

iff there are a series of valid movings from the initial state to it.

The state space of an initial state consists all the states reachable from it.

There
16!
≈20.9
×
10
12

different arranges.

Any initial state can reach about half of the arranges. The state space is quite large!

1

3

4

15

2

5

12

7

6

11

14

8

9

10

13

(a) Initial arrange

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

(b) Target arrange

15
-
Puzzle

It is necessary to know whether the target state is reachable before moving.

Number the grid by the block in the target arrange, and number the blank space grid
16
.

POSITION(i)

is the position of block
i
in the initial arrange.

LESS(i)

is the number of such block
j

which is less than
i

and
POSITION(j)>POSITION(i)
.

X
: If in the initial arrange, the blank space is in a shadowed place shown in Figure(c),
X=1
;
otherwise
X

0
.

(c)

1

3

4

15

2

5

12

7

6

11

14

8

9

10

13

(a) Initial arrange

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

(b) Target arrange

15
-
Puzzle

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

Moving the block
equals to moving
the blank space

Generate sons in up

right

down

le晴f
order.

Search in DFS way.

Abandon s瑡瑥s seen be景re.

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

up

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

right

1

2

4

8

5

6

3

9

10

7

11

13

14

15

12

down

1

2

4

8

5

6

3

11

9

10

7

13

14

15

12

down

1

2

4

8

5

6

3

11

9

10

7

12

13

14

15

down

1

2

4

8

5

6

3

11

9

10

12

13

14

7

15

left

1

2

4

8

5

6

11

9

10

3

12

13

14

7

15

up

1

2

8

5

6

4

11

9

10

3

12

13

14

7

15

up

1

2

4

8

5

6

3

11

9

10

7

12

13

14

15

up

1

2

8

5

6

4

11

9

10

3

12

13

14

7

15

right

1

2

8

11

5

6

4

9

10

3

12

13

14

7

15

down

Searching is hidebound.

We are far and far away from the
target state!!!

15
-
Puzzle

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

7

8

9

10

11

13

14

15

12

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

up

right

down

left

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

right

left

1

2

3

5

6

8

4

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

11

9

10

7

13

14

15

12

up

down

1

2

3

4

5

6

7

8

9

10

11

13

14

15

12

1

2

3

4

5

6

7

8

9

10

15

11

13

14

12

1

2

3

4

5

6

7

8

9

10

11

13

14

15

12

right

down

left

up

down

left

1

2

4

8

5

6

3

9

10

7

11

13

14

15

12

down

1

6

2

4

5

3

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

down

left

1

2

3

5

6

8

4

9

10

7

11

13

14

15

12

left

1

2

3

4

5

6

8

11

9

10

7

12

13

14

15

1

2

3

4

5

6

8

11

9

10

7

13

14

15

12

down

left

1

2

3

4

5

6

7

9

10

11

8

13

14

15

12

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

down

up

Searching in BFS way

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

15
-
Puzzle

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

7

8

9

10

11

13

14

15

12

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

up

right

down

left

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

right

left

1

2

3

5

6

8

4

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

11

9

10

7

13

14

15

12

up

down

1

2

3

4

5

6

7

8

9

10

11

13

14

15

12

1

2

3

4

5

6

7

8

9

10

15

11

13

14

12

1

2

3

4

5

6

7

8

9

10

11

13

14

15

12

right

down

left

up

down

left

1

2

4

8

5

6

3

9

10

7

11

13

14

15

12

down

1

6

2

4

5

3

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

down

left

1

2

3

5

6

8

4

9

10

7

11

13

14

15

12

left

1

2

3

4

5

6

8

11

9

10

7

12

13

14

15

1

2

3

4

5

6

8

11

9

10

7

13

14

15

12

down

left

1

2

3

4

5

6

7

9

10

11

8

13

14

15

12

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

down

up

Searching in BFS way

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

target

The answer state found is the
one nearest to the root.

Is there an “intelligent” way?

Assign a cost c(X) to each node in
the state space tree.

Searching in a fixed way,
whatever the initial state is.

Hidebound again!

up

15
-
Puzzle

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

7

8

9

10

11

13

14

15

12

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

3

5

6

8

4

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

11

9

10

7

13

14

15

12

1

2

3

4

5

6

7

8

9

10

11

13

14

15

12

1

2

3

4

5

6

7

8

9

10

15

11

13

14

12

1

2

3

4

5

6

7

8

9

10

11

13

14

15

12

1

2

4

8

5

6

3

9

10

7

11

13

14

15

12

1

6

2

4

5

3

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

3

5

6

8

4

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

11

9

10

7

12

13

14

15

1

2

3

4

5

6

8

11

9

10

7

13

14

15

12

1

2

3

4

5

6

7

9

10

11

8

13

14

15

12

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

c(.): set the cost of the nodes on the
path from the root to the nearest
target state, to the distance between
this target state and it; set the cost of
all other states to

.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

c(1)=3;c(4)=2;c(10)=1;c(23)=0

Others are

I琠is unprac瑩cal 瑯 calcula瑥 瑨e
accura瑥 c.

right

down

left

right

left

up

down

right

down

left

down

left

down

down

left

left

down

left

down

up

15
-
Puzzle

A practical way is to define a estimating function
ĉ(X)

f(X)+ĝ(X)

which is convenient to calculate, where

f(X)

is the length of the path from the root to X

ĝ(X)

is the minimum number of
movings

from X to the
target state.

A possible
ĝ(X)

is:

ĝ(X)

the number of blocks which are not in its
target position (except the blank space)

The real number of
movings

may be greater than ĝ(X), so
ĝ(X) is a low
-
bound of c(X).

1

2

3

4

5

6

8

9

10

11

12

13

14

15

7

ĝ(X)=1, but we need
more than one movings
to reach the target state.

15
-
Puzzle

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

4

5

6

3

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

1

2

3

4

5

6

7

8

9

10

11

13

14

15

12

1

2

3

4

5

6

8

9

10

7

11

13

14

15

12

up

right

down

left

1

2

3

4

5

6

7

8

9

10

11

13

14

15

12

1

2

3

4

5

6

7

8

9

10

15

11

13

14

12

1

2

3

4

5

6

7

8

9

10

11

13

14

15

12

right

down

left

down

up

1

2

3

4

5

6

7

9

10

11

8

13

14

15

12

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

1

2

3

4

5

10

11

12

22

23

ĉ(X)

I ; ƣ ;

f(X)=length of the path from the root to X

ĝ(X)

the number of blocks in wrong position

ĉ(2)

1+4=5

ĉ(3)

1+4=5

ĉ(4)

1+2=3

ĉ(5)

1+4=5

ĉ(10)

2+1=3

ĉ(11)

2+3=5

ĉ(12)

2+3=5

target

LC
-
Search is as efficient as having
the accurate cost function c(.).

With a proper ĉ(.), LC
-
Search can
be very efficient!

Shortest Path

1

2

5

4

3

10

20

50

100

30

10

60

Graph G

[1, 0, 0]

Open

Closed

Open table

[1, 0, 0]

Closed table

[1, 0, 0]

Closed

[2, 10, 1]

[4, 30, 1]

[5, 100, 1]
，并依序放入
Open

[1, 0, 0]

[5, 100, 1]

[4, 30, 1]

[2, 10, 1]

[2, 10, 1]

Closed

[3, 60, 2]

Open

[2, 10, 1]

[3, 60, 2]

[4, 30, 1]

[4, 30, 1]

Closed

[3, 50, 4]

[5, 90, 4]
，修订
Open

[4, 30, 1]

[5, 90, 4]

[3, 50, 4]

[3, 50, 4]

Closed

[2, 100, 3]

[5, 60, 3]
，前者因劣于
Closed

[2, 10, 1]

Open

[5, 90, 4]

[3, 50, 4]

[5, 60, 3]

[5, 60, 3]
，因其已经是目标结点，算法成

1→4→3→5

Branch
-
Bound Algorithms

What else if we are trying to write down the
algorithm?

Alive nodes table

Queue

Stack

Heap…

Re
-