Automated Planning
and Decision Making
Prof. Ronen Brafman
Automated Planning and Decision Making 2007
1
Search
Automated Planning and Decision Making
2
Introduction
In the past, most problems you encountered
were such that the way to solve them was very
“
clear
”
and straight forward:
○
Solving quadratic equation.
○
Solving linear equation.
○
Matching input to a regular expression
Most of the problems in AI are of different kind.
○
NP

Hard problems.
○
Require searching in a large solution space.
Automated Planning and Decision Making
3
Search Problem
A set S of possible states
–
the search space.
Initial State: s
I
in S.
Operators

which take us from one state to another
(S→S functions)
Goal Conditions.
4
The Search Space
A graph G(V,E) where:
○
V={v :
v
in S}
○
E={<
v,u
> :
exists o
in Operators such that
o(v)=u}
4
5
The Task
Find a path in the search space from s
I
to a
state s where the Goal Conditions are valid.
○
Sometimes we only care about s, and not the
path
○
Sometimes we only care whether s exists.
○
Sometimes value is associated with states, and
we want the best s.
○
Sometimes cost is associated with operators,
and we want the cheapest path
Automated Planning and Decision Making
4
Search Problem Example
Finding a solution for Rubik's Cube.
S
I
: Some state of the cube.
Operators: 90
o
rotation of any of the 9 plates.
Goal Conditions: same color for each of the
cube's sides.
The Search Space will include:
○
A node for any possible state of the cube.
○
An edge between two nodes if you can reach from one
to the other by a 90
o
rotation of some plate.
Automated Planning and Decision Making
5
Another Example
States: Locations of Tiles.
Operators: Move blank right/left/down/up.
Goal Conditions: As in the picture.
Note: optimal solution of n

Puzzle family is
NP

hard…
Initial State
Goal State
Example:
n
queens problem
(
a constraint satisfaction problem
)
8
States: legal placements
of
k≤n
queens
Operators: add a queen
Goal condition: n queens
placed with no conflicts
(no constraint violated)
Example:
n
queens problem
An alternative formulation
9
States: placements of n
queens
–
one per column
Operators: move one of
the queens
Goal condition: n queens
placed with no conflicts
(no constraint violated)
Automated Planning and Decision Making
6
Solving Search Problems
Apply the Operators on States in order to expand
(produce) new States, until we find one that
satisfies the Goal Conditions.
At any moment we may have different options to
proceed (multiple Operators may apply)
Q: In what order should we expand the States?
Our choice affects:
○
Computation time
○
Space used
○
Whether we reach an optimal solution
○
Whether we are guaranteed to find a solution if one
exists (completeness)
Automated Planning and Decision Making
7
Search Tree
You can think of the search as spanning a tree:
○
Root: The initial state.
○
Children of node v are all states reachable from v by applying
an operator.
We can reach the same state via different paths.
○
In such a case, we can ignore this duplicate state, and treat it
as a leaf node
•
Requires that we
remember all
states we visited!
Important parameters affecting performance:
○
b
–
Average branching factor.
○
d
–
Depth of the closest solution.
○
m
–
Maximal depth of the tree.
Fringe: set of current leaf (unexpanded) nodes
Automated Planning and Decision Making
8
Search Methods
Different search methods differ in the order in which they
visit/expand the nodes.
It is important to distinguish between:
○
The search space
○
The order by which we scan that
space
○
These are orthogonal issues!
Some algorithms are described abstractly
by describing
the
space they
generate w
/o specifying how it is
searched
○
You can implement them in different ways by choosing different
search algorithms
Blind Search
○
No additional information about search states is used.
Informed Search
○
Additional information used to improve search efficiency
Automated Planning and Decision Making
9
Relation to Planning
Simplest
(and currently most popular) way
to solve planning problem is to search
from the initial state to a goal
Search states are states of the world
Operators = actions
There are other formulations!
○
For instance: search states = plans
Search is a very general technique
–
very
important for many
applications beyond
planning
Automated Planning and Decision Making
10
Blind Search
Automated Planning and Decision Making
11
Blind Search
Main algorithms:
○
DFS
–
Depth First Search
•
Expand the deepest unexpanded node.
•
Fringe is a LIFO.
○
BFS
–
Breath First Search
•
Expand the shallowest unexpanded node.
•
Fringe is a FIFO.
○
IDS
–
Iterative Deepening Search
•
Combines the advantages of both methods.
•
Avoids the disadvantages of each method.
There
many are
other
variants (e.g., optimizing
disk access, parallel search, etc.)
Automated Planning and Decision Making
12
Breadth First Search
1
4
2
3
Automated Planning and Decision Making
13
Properties of BFS
Complete?
Yes (if b is finite).
Time?
1+b+b
2
+b
3
+…+b(b
d

1)=O(b
d+1
(
Space?
O(b
d
) (keeps every node on the fringe).
Optimal?
Yes (if cost is 1 per step).
Space is a
BIG
problem…
18
18
19
Automated Planning and Decision Making
14
Depth First Search
5
6
7
8
10
9
11
12
1
2
3
4
Automated Planning and Decision Making
15
Properties of DFS
Complete?
○
No if given infinite branches (e.g., when we have loops)
○
Easy to modify: check for repeat states along path
•
Complete in finite spaces!
•
May require large space to maintain list of visited states
Time?
○
Worst case: O(b
m
).
○
Terrible if m is much larger then d.
○
But if solutions occur often or in the
“
left
”
part of the tree, may be much
faster then BFS.
Space?
○
O(m) Linear Space!
Optimal?
○
No.
Automated Planning and Decision Making
16
Depth Limited Search
DFS with a depth limit ℓ, i.e., nodes at
depth ℓ have no successors.
Automated Planning and Decision Making
17
Iterative Deepening Search
Increase the depth limit of the DLS with
each Iteration:
Iterative Deepening Search
24
Automated Planning and Decision Making
19
Complexity of IDS
Number of nodes generated in a depth

limited search to
depth d with branching factor b:
N
DLS
= b
0
+ b
1
+ b
2
+ … + b
d

2
+ b
d

1
+
b
d
Number of nodes generated in an iterative deepening
search to depth d with branching factor b:
N
IDS
= (d+1)b
0
+ db
1
+ (d

1)b
2
+ … + 3b
d

2
+2b
d

1
+ 1b
d
For b = 10, d = 5
:
○
N
DLS
= 1 + 10 + 100 + 1,000 + 10,000 + 100,000 =
111,111
○
N
IDS
= 6 + 50 + 400 + 3,000 + 20,000 + 100,000 =
123,456
Overhead
= (123,456

111,111)/111,111 = 11%
Automated Planning and Decision Making
20
Properties of IDS
Complete?
Yes.
Time?
(d+1)b
0
+ db
1
+ (d

1)b
2
+ … + b
d
= O(b
d
)
Space?
O(d)
Optimal?
Yes. If step cost is 1.
Automated Planning and Decision Making
21
Comparison of the Algorithms
Algorithm
Criterion
BFS
Uniform Cost
DFS
DLS
IDS
Complete
Yes
Yes
No
No
Yes
Time
O(b
d+1
)
O(b
Ciel(C*/ε)
)
O(b
m
)
O(b
l
)
O(b
d
)
Space
O(b
d+1
)
O(b
Ciel(C*/ε)
)
O(m)
O(l)
O(d)
Optimal
Yes
Yes
No
No
Yes
28
General Search Scheme
Solve(Nodes)
○
if empty Nodes

> return Failure
○
else
•
let Node = Select

Node(Nodes)
•
let Rest = Nodes

Node
○
if Node is Goal

> return Solution
○
else
•
let Children = Expand

Node(Node)
•
let New

Nodes = Add

Nodes(Children, Nodes)
•
Solve(New

Nodes)
Different algorithms obtained by suitable instantiation of Select

Node and Add

Nodes
Nodes are data structures that contain state and bookkeeping info
Initially Nodes = {root}
29
Some Blind Instances of GSS
DFS expands
“
deepest
”
node first
○
Select

Node
: select first
Node
in
Nodes
○
Add

Nodes
: Puts
Children
before
Nodes
○
Implementation:
Nodes
is a stack (LIFO)
Breadth

First Search expands
’
shallowest
’
nodes first
○
Select

Node
: select first
Node
in
Nodes
○
Add

Nodes
: Puts
Children
after
Nodes
○
Implementation:
Nodes
is a queue (FIFO)
30
Bounded GSS
Solve(
Nodes,Bound
)
○
if empty Nodes

> return Failure
○
else
•
let Node = Select

Node(Nodes)
•
let Rest = Nodes

Node
•
if f(Node) > Bound
–
Solve(Rest, Bound) ;;; PRUNE Node
•
else if Node is Goal

> return
ProcessSolution
(
Node,Rest
)
–
else
»
let Children = Expand

Node(Node)
»
let New

Nodes = Add

Nodes(Children, Nodes)
»
Solve(New

Nodes,Bound
)
IDS calls Bounded GSS with bound 1,2,3,...
ProcessSolution
(
Node,Rest
) returns Node as solution
Automated Planning and Decision Making
22
Bidirectional Search
Assumptions:
○
There is a small number of states satisfying the goal
conditions.
○
It is possible to reverse the operators.
Simultaneously run BFS from both directions.
Stop when reaching a state from both directions.
Under the assumption that validating this overlap
takes constant time, we get:
○
Time: O(b
d/2
)
○
Space: O(b
d/2
)
Automated Planning and Decision Making
23
Informed Search
Automated Planning and Decision Making
24
Informed/Heuristic Search
Heuristic Function
h:S→R
○
For every state s, h(s) is an estimation of the minimal
distance/cost from s to a solution.
•
Distance is only one way to set a price.
•
How to produce h? later on…
Cost Function
g:S→R
○
For every state s, g(s) is the minimal cost to s from the initial
state.
f=
g+h
, is an estimation of the cost from the initial state
to a solution
.
Properties of Heuristics
h
is
perfect
if
h(s) = shortest distance to goal
from s (and
∞
if goal is unreachable from s)
The perfect heuristic is denoted by
h*
h is
safe
if h(s)=
∞
implies h*(s)=
∞
h is
goal

aware
if h(s)=0 for every goal state s
h is
admissible
if h(s)
≤
h*(s) for all s
h is
consistent
if h(s)
≤
h(s’)+1 whenever s’ is a
child of s
34
Automated Planning and Decision Making
25
Best First Search
Greedy on h values.
Fringe stored in a queue ordered by h
values.
In every step, expand the
“
best
”
node so
far, i.e., the one with the best h value.
36
Reaching Bucharest with BFS
37
1
2
3
4
38
38
Automated Planning and Decision Making
28
Properties of Best First
Complete?
○
No, can get into
loops
○
Yes, with duplicate detection and safe h
Time?
○
O(
b
m
). But a good heuristic can give dramatic
improvement.
Space?
○
O(
b
m
). Keeps all nodes in memory.
Optimal
○
No.
Automated Planning and Decision Making
29
A* Search
Idea: Avoid expanding paths that are already
expensive.
Greedy on f values.
Fringe is stored in a queue ordered by f
values.
Recall, f(n)=g(n)+h(n), where:
○
g(n): cost so far to reach n.
○
h(n): estimated cost from n to goal.
○
f(n): estimated total cost of path through n to
goal.
41
Slides 5

12
41
Bucharest via A*
42
Bucharest via A*
43
Automated Planning and Decision Making
32
Admissible Heuristics
A heuristic h(n) is
admissible
if for every node n, h(n) ≤ h
*
(n),
where h
*
(n) is the
real
cost to reach the goal state from n.
An admissible heuristic
never overestimates
the cost to reach
the goal, i.e., it is
optimistic
Theorem:
If h(n) is
admissible
, A
*
using Tree

Search is optimal.
○
Tree search
–
nodes encountered more than once are not treated
as leaf nodes
Automated Planning and Decision Making
33
Optimality of A* (Proof)
Suppose some suboptimal goal G
2
has
been generated and is in the fringe. Let n
be an unexpanded node in the fringe such
that n is on a shortest path to an optimal
goal G.
Automated Planning and Decision Making
34
Optimality of A* (Proof)
1.
f(G
2
) = g(G
2
)
since h(G
2
) = 0.
2.
f(G) = g(G)
since h(G) = 0.
3.
g(G
2
) > g(G)
since G
2
is suboptimal.
4.
f(G
2
) > f(G)
by 1,2,3.
5.
h(n) ≤ h
*
(n)
since h is admissible.
6.
g(n)+h(n) ≤ g(n)+h
*
(n)
by 5.
7.
f(n) ≤ f(G)
by definition of f.
8.
f(G
2
) > f(n)
by 4,7.
Hence A
*
will never select G
2
for expansion.
47
A
u
t
o
m
a
t
e
d
P
l
a
n
n
i
n
g
a
n
d
D
e
c
i
s
i
o
n
M
a
k
i
n
g
3
5
C
o
n
s
i
s
t
e
n
t
H
e
u
r
i
s
t
i
c
s
§
A
h
e
u
r
i
s
t
i
c
i
s
c
o
n
s
i
s
t
e
n
t
(
m
o
n
o
t
o
n
i
c
)
i
f
f
o
r
e
v
e
r
y
n
o
d
e
n
,
e
v
e
r
y
s
u
c
c
e
s
s
o
r
n
'
o
f
n
g
e
n
e
r
a
t
e
d
b
y
a
n
y
a
c
t
i
o
n
a
,
h
(
n
)
≤
c
(
n
,
a
,
n
'
)
+
h
(
n
'
)
§
I
f
h
i
s
c
o
n
s
i
s
t
e
n
t
,
w
e
h
a
v
e
:
f
(
n
'
)
=
g
(
n
'
)
+
h
(
n
'
)
=
g
(
n
)
+
c
(
n
,
a
,
n
'
)
+
h
(
n
'
)
≥
g
(
n
)
+
h
(
n
)
=
f
(
n
)
i
.
e
.
,
f
(
n
)
i
s
n
o
n

d
e
c
r
e
a
s
i
n
g
a
l
o
n
g
a
n
y
p
a
t
h
.
§
T
h
e
o
r
e
m
I
f
h
(
n
)
i
s
c
o
n
s
i
s
t
e
n
t
,
A
*
u
s
i
n
g
G
r
a
p
h

S
e
a
r
c
h
i
s
o
p
t
i
m
a
l
.
○
G
r
a
p
h

s
e
a
r
c
h
:
n
o
d
e
s
e
n
c
o
u
n
t
e
r
e
d
a
s
e
c
o
n
d
t
i
m
e
c
a
n
b
e
t
r
e
a
t
e
d
a
s
l
e
a
f
n
o
d
e
s
.
Automated Planning and Decision Making
37
Properties of A*
Complete?
○
Yes, unless there are infinitely many nodes with f ≤ f(G).
Time?
○
Exponential.
Optimal?
○
Yes.
Space?
○
Keeps all nodes in memory.
○
A serious problem in many cases!
•
How can we overcome it?
Automated Planning and Decision Making
38
Iterative Deepening A* (IDA*)
Combine Iterative Deepening with A* in
order to overcome the space problem.
A simple idea:
○
Replace upper bound on depth with upper
bound on f values.
○
If
f(n)>U, n
is not
enqueued
.
○
If no solution is found given U, increase U.
Another method

Weighted A* (soon)
Automated Planning and Decision Making
39
Branch & Bound
Used for finding an optimal solution (i.e., when there are
multiple possible solutions, but some are better than
others)
Attempts to prune entire sub

trees
Applicable in the context of different search methods (BFS,
DFS, …).
Assumes we can generate upper bound U and lower
bound L on the value of solutions reachable from n
If for some node n there exists a node n
’
such that
U(n)<I(n
’
), we can prune n.
Often used as follows:
○
First, quickly find any solution with value v
’
○
Prune any node n s.t. f(n) > v
’
Automated Planning and Decision Making
40
Local search algorithms
In many problems, the
path
to the goal is
irrelevant; the goal state itself is all we
care about.
Example: n

queens problems
More generally: constraint

satisfaction
problems
Local search algorithms keep a single
"current" state, and try to improve it.
Automated Planning and Decision Making
41
Hill

climbing search
“
Like climbing mount Everest in thick fog with
amnesia…
”
Improve while you can, i.e. stop when
reaching a maxima (minima
)
Many variants: tie

breaking rules, restarts
Automated Planning and Decision Making
42
Hill

climbing search
Problem: depending on initial state, can
get stuck in local maxima
.
Automated Planning and Decision Making
44
Solution by Hill

climbing
h = number of pairs of
queens that are attacking
each other, either directly or
indirectly.
Moves: change position of a
single queen
On A, h=17.
On B, h=1. local minimum.
B
A
56
57
Automated Planning and Decision Making
45
Simulated annealing search
Idea: escape local maxima by allowing
some "bad" moves, but
gradually
decrease
their frequency.
Automated Planning and Decision Making
46
Properties of SAS
One can prove: If T decreases slowly
enough, then SAS will find a global
optimum with probability approaching 1.
Widely used in VLSI layout, airline
scheduling, etc.
Automated Planning and Decision Making
47
Local beam search
Keep track of k states rather than just one.
Start with k randomly generated states.
At each iteration, all the successors of all
k states are generated.
If any one is a goal state, stop; else select
the k best successors from the complete
list and repeat.
61
Weighted A*
Makes A* more greedy
f(n) = g(n) + W*h(n) for W>1
Gives more weight to the heuristic value,
i.e. to getting closer to the goal
Solution is at most factor W optimal
Usually reaches the goal faster and
requires less memory, but IDA* uses less
memory
.
62
63
Automated Planning and Decision Making
48
Generating a Heuristic Function
Heuristic functions are usually obtained by
finding a simple approximation to the
problem
Simple heuristics often work well (but not
very well)
Examples:
○
N

puzzle: Manhattan distance
•
Simplification: assumes that we can move each tile
independently. Ignores interactions between tiles
•
A very general idea
–
ignore certain interactions
Σχόλια 0
Συνδεθείτε για να κοινοποιήσετε σχόλιο