# PPTX

Ηλεκτρονική - Συσκευές

27 Νοε 2013 (πριν από 4 χρόνια και 5 μήνες)

163 εμφανίσεις

Automated Planning
and Decision Making

Prof. Ronen Brafman

Automated Planning and Decision Making 2007

1

Search

Automated Planning and Decision Making

2

Introduction

In the past, most problems you encountered
were such that the way to solve them was very

clear

and straight forward:

Solving linear equation.

Matching input to a regular expression

Most of the problems in AI are of different kind.

NP
-
Hard problems.

Require searching in a large solution space.

Automated Planning and Decision Making

3

Search Problem

A set S of possible states

the search space.

Initial State: s
I

in S.

Operators
-

which take us from one state to another
(S→S functions)

Goal Conditions.

4

The Search Space

A graph G(V,E) where:

V={v :
v
in S}

E={<
v,u
> :
exists o
in Operators such that
o(v)=u}

4

5

Find a path in the search space from s
I

to a
state s where the Goal Conditions are valid.

Sometimes we only care about s, and not the
path

Sometimes we only care whether s exists.

Sometimes value is associated with states, and
we want the best s.

Sometimes cost is associated with operators,
and we want the cheapest path

Automated Planning and Decision Making

4

Search Problem Example

Finding a solution for Rubik's Cube.

S
I
: Some state of the cube.

Operators: 90
o

rotation of any of the 9 plates.

Goal Conditions: same color for each of the
cube's sides.

The Search Space will include:

A node for any possible state of the cube.

An edge between two nodes if you can reach from one
to the other by a 90
o

rotation of some plate.

Automated Planning and Decision Making

5

Another Example

States: Locations of Tiles.

Operators: Move blank right/left/down/up.

Goal Conditions: As in the picture.

Note: optimal solution of n
-
Puzzle family is

NP
-
hard…

Initial State

Goal State

Example:
n

queens problem

(
a constraint satisfaction problem
)

8

States: legal placements
of
k≤n

queens

Goal condition: n queens
placed with no conflicts
(no constraint violated)

Example:
n

queens problem

An alternative formulation

9

States: placements of n
queens

one per column

Operators: move one of
the queens

Goal condition: n queens
placed with no conflicts
(no constraint violated)

Automated Planning and Decision Making

6

Solving Search Problems

Apply the Operators on States in order to expand
(produce) new States, until we find one that
satisfies the Goal Conditions.

At any moment we may have different options to
proceed (multiple Operators may apply)

Q: In what order should we expand the States?

Our choice affects:

Computation time

Space used

Whether we reach an optimal solution

Whether we are guaranteed to find a solution if one
exists (completeness)

Automated Planning and Decision Making

7

Search Tree

You can think of the search as spanning a tree:

Root: The initial state.

Children of node v are all states reachable from v by applying
an operator.

We can reach the same state via different paths.

In such a case, we can ignore this duplicate state, and treat it
as a leaf node

Requires that we
remember all
states we visited!

Important parameters affecting performance:

b

Average branching factor.

d

Depth of the closest solution.

m

Maximal depth of the tree.

Fringe: set of current leaf (unexpanded) nodes

Automated Planning and Decision Making

8

Search Methods

Different search methods differ in the order in which they
visit/expand the nodes.

It is important to distinguish between:

The search space

The order by which we scan that
space

These are orthogonal issues!

Some algorithms are described abstractly
by describing
the
space they
generate w
/o specifying how it is
searched

You can implement them in different ways by choosing different
search algorithms

Blind Search

Informed Search

Additional information used to improve search efficiency

Automated Planning and Decision Making

9

Relation to Planning

Simplest
(and currently most popular) way
to solve planning problem is to search
from the initial state to a goal

Search states are states of the world

Operators = actions

There are other formulations!

For instance: search states = plans

Search is a very general technique

very
important for many
applications beyond
planning

Automated Planning and Decision Making

10

Blind Search

Automated Planning and Decision Making

11

Blind Search

Main algorithms:

DFS

Depth First Search

Expand the deepest unexpanded node.

Fringe is a LIFO.

BFS

Breath First Search

Expand the shallowest unexpanded node.

Fringe is a FIFO.

IDS

Iterative Deepening Search

Combines the advantages of both methods.

Avoids the disadvantages of each method.

There
many are
other
variants (e.g., optimizing
disk access, parallel search, etc.)

Automated Planning and Decision Making

12

1

4

2

3

Automated Planning and Decision Making

13

Properties of BFS

Complete?

Yes (if b is finite).

Time?

1+b+b
2
+b
3
+…+b(b
d
-
1)=O(b
d+1
(

Space?

O(b
d
) (keeps every node on the fringe).

Optimal?

Yes (if cost is 1 per step).

Space is a
BIG

problem…

18

18

19

Automated Planning and Decision Making

14

Depth First Search

5

6

7

8

10

9

11

12

1

2

3

4

Automated Planning and Decision Making

15

Properties of DFS

Complete?

No if given infinite branches (e.g., when we have loops)

Easy to modify: check for repeat states along path

Complete in finite spaces!

May require large space to maintain list of visited states

Time?

Worst case: O(b
m
).

Terrible if m is much larger then d.

But if solutions occur often or in the

left

part of the tree, may be much
faster then BFS.

Space?

O(m) Linear Space!

Optimal?

No.

Automated Planning and Decision Making

16

Depth Limited Search

DFS with a depth limit ℓ, i.e., nodes at
depth ℓ have no successors.

Automated Planning and Decision Making

17

Iterative Deepening Search

Increase the depth limit of the DLS with
each Iteration:

Iterative Deepening Search

24

Automated Planning and Decision Making

19

Complexity of IDS

Number of nodes generated in a depth
-
limited search to
depth d with branching factor b:

N
DLS

= b
0

+ b
1

+ b
2

+ … + b
d
-
2

+ b
d
-
1

+
b
d

Number of nodes generated in an iterative deepening
search to depth d with branching factor b:

N
IDS

= (d+1)b
0

+ db
1

+ (d
-
1)b
2

+ … + 3b
d
-
2

+2b
d
-
1

+ 1b
d

For b = 10, d = 5
:

N
DLS
= 1 + 10 + 100 + 1,000 + 10,000 + 100,000 =
111,111

N
IDS

= 6 + 50 + 400 + 3,000 + 20,000 + 100,000 =
123,456

= (123,456
-

111,111)/111,111 = 11%

Automated Planning and Decision Making

20

Properties of IDS

Complete?

Yes.

Time?

(d+1)b
0

+ db
1

+ (d
-
1)b
2

+ … + b
d

= O(b
d
)

Space?

O(d)

Optimal?

Yes. If step cost is 1.

Automated Planning and Decision Making

21

Comparison of the Algorithms

Algorithm

Criterion

BFS

Uniform Cost

DFS

DLS

IDS

Complete

Yes

Yes

No

No

Yes

Time

O(b
d+1
)

O(b
Ciel(C*/ε)
)

O(b
m
)

O(b
l
)

O(b
d
)

Space

O(b
d+1
)

O(b
Ciel(C*/ε)
)

O(m)

O(l)

O(d)

Optimal

Yes

Yes

No

No

Yes

28

General Search Scheme

Solve(Nodes)

if empty Nodes
-
> return Failure

else

let Node = Select
-
Node(Nodes)

let Rest = Nodes
-

Node

if Node is Goal
-
> return Solution

else

let Children = Expand
-
Node(Node)

let New
-
-
Nodes(Children, Nodes)

Solve(New
-
Nodes)

Different algorithms obtained by suitable instantiation of Select
-
-
Nodes

Nodes are data structures that contain state and bookkeeping info

Initially Nodes = {root}

29

Some Blind Instances of GSS

DFS expands

deepest

node first

Select
-
Node
: select first
Node
in
Nodes

-
Nodes
: Puts
Children
before
Nodes

Implementation:
Nodes
is a stack (LIFO)

-
First Search expands

shallowest

nodes first

Select
-
Node
: select first
Node
in
Nodes

-
Nodes
: Puts
Children
after
Nodes

Implementation:
Nodes
is a queue (FIFO)

30

Bounded GSS

Solve(
Nodes,Bound
)

if empty Nodes
-
> return Failure

else

let Node = Select
-
Node(Nodes)

let Rest = Nodes
-

Node

if f(Node) > Bound

Solve(Rest, Bound) ;;; PRUNE Node

else if Node is Goal
-
> return
ProcessSolution
(
Node,Rest
)

else

»
let Children = Expand
-
Node(Node)

»
let New
-
-
Nodes(Children, Nodes)

»
Solve(New
-
Nodes,Bound
)

IDS calls Bounded GSS with bound 1,2,3,...

ProcessSolution
(
Node,Rest
) returns Node as solution

Automated Planning and Decision Making

22

Bidirectional Search

Assumptions:

There is a small number of states satisfying the goal
conditions.

It is possible to reverse the operators.

Simultaneously run BFS from both directions.

Stop when reaching a state from both directions.

Under the assumption that validating this overlap
takes constant time, we get:

Time: O(b
d/2
)

Space: O(b
d/2
)

Automated Planning and Decision Making

23

Informed Search

Automated Planning and Decision Making

24

Informed/Heuristic Search

Heuristic Function
h:S→R

For every state s, h(s) is an estimation of the minimal
distance/cost from s to a solution.

Distance is only one way to set a price.

How to produce h? later on…

Cost Function
g:S→R

For every state s, g(s) is the minimal cost to s from the initial
state.

f=
g+h
, is an estimation of the cost from the initial state
to a solution
.

Properties of Heuristics

h

is
perfect

if
h(s) = shortest distance to goal
from s (and

if goal is unreachable from s)

The perfect heuristic is denoted by
h*

h is
safe

if h(s)=

implies h*(s)=

h is
goal
-
aware

if h(s)=0 for every goal state s

h is

if h(s)

h*(s) for all s

h is
consistent

if h(s)

h(s’)+1 whenever s’ is a
child of s

34

Automated Planning and Decision Making

25

Best First Search

Greedy on h values.

Fringe stored in a queue ordered by h
values.

In every step, expand the

best

node so
far, i.e., the one with the best h value.

36

Reaching Bucharest with BFS

37

1
2
3
4
38

38

Automated Planning and Decision Making

28

Properties of Best First

Complete?

No, can get into
loops

Yes, with duplicate detection and safe h

Time?

O(
b
m
). But a good heuristic can give dramatic
improvement.

Space?

O(
b
m
). Keeps all nodes in memory.

Optimal

No.

Automated Planning and Decision Making

29

A* Search

Idea: Avoid expanding paths that are already
expensive.

Greedy on f values.

Fringe is stored in a queue ordered by f
values.

Recall, f(n)=g(n)+h(n), where:

g(n): cost so far to reach n.

h(n): estimated cost from n to goal.

f(n): estimated total cost of path through n to
goal.

41

Slides 5
-
12

41

Bucharest via A*

42

Bucharest via A*

43

Automated Planning and Decision Making

32

A heuristic h(n) is

if for every node n, h(n) ≤ h
*
(n),
where h
*
(n) is the
real

cost to reach the goal state from n.

never overestimates

the cost to reach
the goal, i.e., it is
optimistic

Theorem:

If h(n) is
, A
*

using Tree
-
Search is optimal.

Tree search

nodes encountered more than once are not treated
as leaf nodes

Automated Planning and Decision Making

33

Optimality of A* (Proof)

Suppose some suboptimal goal G
2

has
been generated and is in the fringe. Let n
be an unexpanded node in the fringe such
that n is on a shortest path to an optimal
goal G.

Automated Planning and Decision Making

34

Optimality of A* (Proof)

1.
f(G
2
) = g(G
2
)

since h(G
2
) = 0.

2.
f(G) = g(G)

since h(G) = 0.

3.
g(G
2
) > g(G)

since G
2

is suboptimal.

4.
f(G
2
) > f(G)

by 1,2,3.

5.
h(n) ≤ h
*
(n)

6.
g(n)+h(n) ≤ g(n)+h
*
(n)

by 5.

7.
f(n) ≤ f(G)

by definition of f.

8.
f(G
2
) > f(n)

by 4,7.

Hence A
*

will never select G
2

for expansion.

47

A
u
t
o
m
a
t
e
d

P
l
a
n
n
i
n
g

a
n
d

D
e
c
i
s
i
o
n

M
a
k
i
n
g

3
5
C
o
n
s
i
s
t
e
n
t

H
e
u
r
i
s
t
i
c
s
§
A

h
e
u
r
i
s
t
i
c

i
s

c
o
n
s
i
s
t
e
n
t

(
m
o
n
o
t
o
n
i
c
)

i
f

f
o
r

e
v
e
r
y

n
o
d
e

n
,

e
v
e
r
y

s
u
c
c
e
s
s
o
r

n
'

o
f

n

g
e
n
e
r
a
t
e
d

b
y

a
n
y

a
c
t
i
o
n

a
,

h
(
n
)

c
(
n
,
a
,
n
'
)

+

h
(
n
'
)
§
I
f

h

i
s

c
o
n
s
i
s
t
e
n
t
,

w
e

h
a
v
e
:
f
(
n
'
)

=

g
(
n
'
)

+

h
(
n
'
)

=

g
(
n
)

+

c
(
n
,
a
,
n
'
)

+

h
(
n
'
)

g
(
n
)

+

h
(
n
)

=

f
(
n
)
i
.
e
.
,

f
(
n
)

i
s

n
o
n
-
d
e
c
r
e
a
s
i
n
g

a
l
o
n
g

a
n
y

p
a
t
h
.
§
T
h
e
o
r
e
m
I
f

h
(
n
)

i
s

c
o
n
s
i
s
t
e
n
t
,

A
*

u
s
i
n
g

G
r
a
p
h
-
S
e
a
r
c
h

i
s

o
p
t
i
m
a
l
.

G
r
a
p
h
-
s
e
a
r
c
h
:

n
o
d
e
s

e
n
c
o
u
n
t
e
r
e
d

a

s
e
c
o
n
d

t
i
m
e

c
a
n

b
e

t
r
e
a
t
e
d

a
s

l
e
a
f

n
o
d
e
s
.
Automated Planning and Decision Making

37

Properties of A*

Complete?

Yes, unless there are infinitely many nodes with f ≤ f(G).

Time?

Exponential.

Optimal?

Yes.

Space?

Keeps all nodes in memory.

A serious problem in many cases!

How can we overcome it?

Automated Planning and Decision Making

38

Iterative Deepening A* (IDA*)

Combine Iterative Deepening with A* in
order to overcome the space problem.

A simple idea:

Replace upper bound on depth with upper
bound on f values.

If
f(n)>U, n
is not
enqueued
.

If no solution is found given U, increase U.

Another method
--

Weighted A* (soon)

Automated Planning and Decision Making

39

Branch & Bound

Used for finding an optimal solution (i.e., when there are
multiple possible solutions, but some are better than
others)

Attempts to prune entire sub
-
trees

Applicable in the context of different search methods (BFS,
DFS, …).

Assumes we can generate upper bound U and lower
bound L on the value of solutions reachable from n

If for some node n there exists a node n

such that
U(n)<I(n

), we can prune n.

Often used as follows:

First, quickly find any solution with value v

Prune any node n s.t. f(n) > v

Automated Planning and Decision Making

40

Local search algorithms

In many problems, the
path

to the goal is
irrelevant; the goal state itself is all we

Example: n
-
queens problems

More generally: constraint
-
satisfaction
problems

Local search algorithms keep a single
"current" state, and try to improve it.

Automated Planning and Decision Making

41

Hill
-
climbing search

Like climbing mount Everest in thick fog with
amnesia…

Improve while you can, i.e. stop when
reaching a maxima (minima
)

Many variants: tie
-
breaking rules, restarts

Automated Planning and Decision Making

42

Hill
-
climbing search

Problem: depending on initial state, can
get stuck in local maxima
.

Automated Planning and Decision Making

44

Solution by Hill
-
climbing

h = number of pairs of
queens that are attacking
each other, either directly or
indirectly.

Moves: change position of a
single queen

On A, h=17.

On B, h=1. local minimum.

B

A

56

57

Automated Planning and Decision Making

45

Simulated annealing search

Idea: escape local maxima by allowing
decrease

their frequency.

Automated Planning and Decision Making

46

Properties of SAS

One can prove: If T decreases slowly
enough, then SAS will find a global
optimum with probability approaching 1.

Widely used in VLSI layout, airline
scheduling, etc.

Automated Planning and Decision Making

47

Local beam search

Keep track of k states rather than just one.

At each iteration, all the successors of all
k states are generated.

If any one is a goal state, stop; else select
the k best successors from the complete
list and repeat.

61

Weighted A*

Makes A* more greedy

f(n) = g(n) + W*h(n) for W>1

Gives more weight to the heuristic value,
i.e. to getting closer to the goal

Solution is at most factor W optimal

Usually reaches the goal faster and
requires less memory, but IDA* uses less
memory
.

62

63

Automated Planning and Decision Making

48

Generating a Heuristic Function

Heuristic functions are usually obtained by
finding a simple approximation to the
problem

Simple heuristics often work well (but not
very well)

Examples:

N
-
puzzle: Manhattan distance

Simplification: assumes that we can move each tile
independently. Ignores interactions between tiles

A very general idea

ignore certain interactions