PPTX

greatgodlyΗλεκτρονική - Συσκευές

27 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

141 εμφανίσεις

Automated Planning
and Decision Making



Prof. Ronen Brafman

Automated Planning and Decision Making 2007

1

Search

Automated Planning and Decision Making

2

Introduction


In the past, most problems you encountered
were such that the way to solve them was very

clear


and straight forward:


Solving quadratic equation.


Solving linear equation.


Matching input to a regular expression


Most of the problems in AI are of different kind.


NP
-
Hard problems.


Require searching in a large solution space.

Automated Planning and Decision Making

3

Search Problem


A set S of possible states


the search space.


Initial State: s
I

in S.


Operators
-

which take us from one state to another
(S→S functions)


Goal Conditions.

4

The Search Space


A graph G(V,E) where:


V={v :
v
in S}


E={<
v,u
> :
exists o
in Operators such that
o(v)=u}

4

5

The Task


Find a path in the search space from s
I

to a
state s where the Goal Conditions are valid.


Sometimes we only care about s, and not the
path


Sometimes we only care whether s exists.


Sometimes value is associated with states, and
we want the best s.


Sometimes cost is associated with operators,
and we want the cheapest path

Automated Planning and Decision Making

4

Search Problem Example


Finding a solution for Rubik's Cube.


S
I
: Some state of the cube.


Operators: 90
o

rotation of any of the 9 plates.


Goal Conditions: same color for each of the
cube's sides.


The Search Space will include:


A node for any possible state of the cube.


An edge between two nodes if you can reach from one
to the other by a 90
o

rotation of some plate.

Automated Planning and Decision Making

5

Another Example


States: Locations of Tiles.


Operators: Move blank right/left/down/up.


Goal Conditions: As in the picture.


Note: optimal solution of n
-
Puzzle family is

NP
-
hard…

Initial State

Goal State

Example:
n

queens problem

(
a constraint satisfaction problem
)

8

States: legal placements
of
k≤n

queens

Operators: add a queen

Goal condition: n queens
placed with no conflicts
(no constraint violated)

Example:
n

queens problem

An alternative formulation

9

States: placements of n
queens


one per column

Operators: move one of
the queens

Goal condition: n queens
placed with no conflicts
(no constraint violated)

Automated Planning and Decision Making

6

Solving Search Problems


Apply the Operators on States in order to expand
(produce) new States, until we find one that
satisfies the Goal Conditions.


At any moment we may have different options to
proceed (multiple Operators may apply)


Q: In what order should we expand the States?


Our choice affects:


Computation time


Space used


Whether we reach an optimal solution


Whether we are guaranteed to find a solution if one
exists (completeness)

Automated Planning and Decision Making

7

Search Tree


You can think of the search as spanning a tree:


Root: The initial state.


Children of node v are all states reachable from v by applying
an operator.


We can reach the same state via different paths.


In such a case, we can ignore this duplicate state, and treat it
as a leaf node


Requires that we
remember all
states we visited!


Important parameters affecting performance:


b


Average branching factor.


d


Depth of the closest solution.


m


Maximal depth of the tree.


Fringe: set of current leaf (unexpanded) nodes

Automated Planning and Decision Making

8

Search Methods


Different search methods differ in the order in which they
visit/expand the nodes.


It is important to distinguish between:


The search space


The order by which we scan that
space


These are orthogonal issues!


Some algorithms are described abstractly
by describing
the
space they
generate w
/o specifying how it is
searched


You can implement them in different ways by choosing different
search algorithms


Blind Search


No additional information about search states is used.


Informed Search


Additional information used to improve search efficiency

Automated Planning and Decision Making

9

Relation to Planning



Simplest
(and currently most popular) way
to solve planning problem is to search
from the initial state to a goal


Search states are states of the world


Operators = actions


There are other formulations!


For instance: search states = plans


Search is a very general technique


very
important for many
applications beyond
planning

Automated Planning and Decision Making

10

Blind Search

Automated Planning and Decision Making

11

Blind Search


Main algorithms:


DFS


Depth First Search


Expand the deepest unexpanded node.


Fringe is a LIFO.


BFS


Breath First Search


Expand the shallowest unexpanded node.


Fringe is a FIFO.


IDS


Iterative Deepening Search


Combines the advantages of both methods.


Avoids the disadvantages of each method.


There
many are
other
variants (e.g., optimizing
disk access, parallel search, etc.)

Automated Planning and Decision Making

12

Breadth First Search

1

4

2

3

Automated Planning and Decision Making

13

Properties of BFS


Complete?

Yes (if b is finite).


Time?

1+b+b
2
+b
3
+…+b(b
d
-
1)=O(b
d+1
(


Space?


O(b
d
) (keeps every node on the fringe).


Optimal?

Yes (if cost is 1 per step).



Space is a
BIG

problem…

18

18

19

Automated Planning and Decision Making

14

Depth First Search

5

6

7

8

10

9

11

12

1

2

3

4

Automated Planning and Decision Making

15

Properties of DFS


Complete?


No if given infinite branches (e.g., when we have loops)


Easy to modify: check for repeat states along path


Complete in finite spaces!


May require large space to maintain list of visited states


Time?


Worst case: O(b
m
).


Terrible if m is much larger then d.


But if solutions occur often or in the

left


part of the tree, may be much
faster then BFS.


Space?


O(m) Linear Space!


Optimal?


No.

Automated Planning and Decision Making

16

Depth Limited Search


DFS with a depth limit ℓ, i.e., nodes at
depth ℓ have no successors.

Automated Planning and Decision Making

17

Iterative Deepening Search


Increase the depth limit of the DLS with
each Iteration:

Iterative Deepening Search

24

Automated Planning and Decision Making

19

Complexity of IDS


Number of nodes generated in a depth
-
limited search to
depth d with branching factor b:


N
DLS

= b
0

+ b
1

+ b
2

+ … + b
d
-
2

+ b
d
-
1

+
b
d



Number of nodes generated in an iterative deepening
search to depth d with branching factor b:

N
IDS

= (d+1)b
0

+ db
1

+ (d
-
1)b
2

+ … + 3b
d
-
2

+2b
d
-
1

+ 1b
d



For b = 10, d = 5
:


N
DLS
= 1 + 10 + 100 + 1,000 + 10,000 + 100,000 =
111,111


N
IDS

= 6 + 50 + 400 + 3,000 + 20,000 + 100,000 =
123,456


Overhead
= (123,456
-

111,111)/111,111 = 11%

Automated Planning and Decision Making

20

Properties of IDS


Complete?

Yes.


Time?

(d+1)b
0

+ db
1

+ (d
-
1)b
2

+ … + b
d

= O(b
d
)


Space?

O(d)


Optimal?

Yes. If step cost is 1.

Automated Planning and Decision Making

21

Comparison of the Algorithms


Algorithm

Criterion

BFS

Uniform Cost

DFS

DLS

IDS

Complete

Yes

Yes

No

No

Yes

Time

O(b
d+1
)

O(b
Ciel(C*/ε)
)

O(b
m
)

O(b
l
)

O(b
d
)

Space

O(b
d+1
)

O(b
Ciel(C*/ε)
)

O(m)

O(l)

O(d)

Optimal

Yes

Yes

No

No

Yes

28

General Search Scheme


Solve(Nodes)


if empty Nodes
-
> return Failure


else


let Node = Select
-
Node(Nodes)


let Rest = Nodes
-

Node


if Node is Goal
-
> return Solution


else


let Children = Expand
-
Node(Node)


let New
-
Nodes = Add
-
Nodes(Children, Nodes)


Solve(New
-
Nodes)



Different algorithms obtained by suitable instantiation of Select
-
Node and Add
-
Nodes


Nodes are data structures that contain state and bookkeeping info


Initially Nodes = {root}

29

Some Blind Instances of GSS


DFS expands

deepest


node first


Select
-
Node
: select first
Node
in
Nodes


Add
-
Nodes
: Puts
Children
before
Nodes


Implementation:
Nodes
is a stack (LIFO)


Breadth
-
First Search expands

shallowest


nodes first


Select
-
Node
: select first
Node
in
Nodes


Add
-
Nodes
: Puts
Children
after
Nodes


Implementation:
Nodes
is a queue (FIFO)


30

Bounded GSS


Solve(
Nodes,Bound
)


if empty Nodes
-
> return Failure


else


let Node = Select
-
Node(Nodes)


let Rest = Nodes
-

Node


if f(Node) > Bound


Solve(Rest, Bound) ;;; PRUNE Node


else if Node is Goal
-
> return
ProcessSolution
(
Node,Rest
)


else

»
let Children = Expand
-
Node(Node)

»
let New
-
Nodes = Add
-
Nodes(Children, Nodes)

»
Solve(New
-
Nodes,Bound
)



IDS calls Bounded GSS with bound 1,2,3,...


ProcessSolution
(
Node,Rest
) returns Node as solution

Automated Planning and Decision Making

22

Bidirectional Search


Assumptions:


There is a small number of states satisfying the goal
conditions.


It is possible to reverse the operators.


Simultaneously run BFS from both directions.


Stop when reaching a state from both directions.


Under the assumption that validating this overlap
takes constant time, we get:


Time: O(b
d/2
)


Space: O(b
d/2
)

Automated Planning and Decision Making

23

Informed Search

Automated Planning and Decision Making

24

Informed/Heuristic Search


Heuristic Function
h:S→R


For every state s, h(s) is an estimation of the minimal
distance/cost from s to a solution.


Distance is only one way to set a price.


How to produce h? later on…


Cost Function
g:S→R


For every state s, g(s) is the minimal cost to s from the initial
state.


f=
g+h
, is an estimation of the cost from the initial state
to a solution
.

Properties of Heuristics


h

is
perfect

if
h(s) = shortest distance to goal
from s (and


if goal is unreachable from s)


The perfect heuristic is denoted by
h*


h is
safe

if h(s)=

implies h*(s)=



h is
goal
-
aware

if h(s)=0 for every goal state s


h is
admissible

if h(s)

h*(s) for all s


h is
consistent

if h(s)

h(s’)+1 whenever s’ is a
child of s

34

Automated Planning and Decision Making

25

Best First Search


Greedy on h values.


Fringe stored in a queue ordered by h
values.


In every step, expand the

best


node so
far, i.e., the one with the best h value.

36

Reaching Bucharest with BFS

37

1
2
3
4
38

38

Automated Planning and Decision Making

28

Properties of Best First


Complete?


No, can get into
loops


Yes, with duplicate detection and safe h


Time?


O(
b
m
). But a good heuristic can give dramatic
improvement.


Space?


O(
b
m
). Keeps all nodes in memory.


Optimal


No.

Automated Planning and Decision Making

29

A* Search


Idea: Avoid expanding paths that are already
expensive.


Greedy on f values.


Fringe is stored in a queue ordered by f
values.


Recall, f(n)=g(n)+h(n), where:


g(n): cost so far to reach n.


h(n): estimated cost from n to goal.


f(n): estimated total cost of path through n to
goal.

41

Slides 5
-
12

41

Bucharest via A*

42

Bucharest via A*

43

Automated Planning and Decision Making

32

Admissible Heuristics


A heuristic h(n) is
admissible

if for every node n, h(n) ≤ h
*
(n),
where h
*
(n) is the
real

cost to reach the goal state from n.


An admissible heuristic
never overestimates

the cost to reach
the goal, i.e., it is
optimistic


Theorem:

If h(n) is
admissible
, A
*

using Tree
-
Search is optimal.


Tree search


nodes encountered more than once are not treated
as leaf nodes

Automated Planning and Decision Making

33

Optimality of A* (Proof)


Suppose some suboptimal goal G
2

has
been generated and is in the fringe. Let n
be an unexpanded node in the fringe such
that n is on a shortest path to an optimal
goal G.

Automated Planning and Decision Making

34

Optimality of A* (Proof)

1.
f(G
2
) = g(G
2
)



since h(G
2
) = 0.

2.
f(G) = g(G)



since h(G) = 0.

3.
g(G
2
) > g(G)



since G
2

is suboptimal.

4.
f(G
2
) > f(G)



by 1,2,3.


5.
h(n) ≤ h
*
(n)



since h is admissible.

6.
g(n)+h(n) ≤ g(n)+h
*
(n)


by 5.

7.
f(n) ≤ f(G)



by definition of f.


8.
f(G
2
) > f(n)



by 4,7.


Hence A
*

will never select G
2

for expansion.

47

A
u
t
o
m
a
t
e
d

P
l
a
n
n
i
n
g

a
n
d

D
e
c
i
s
i
o
n

M
a
k
i
n
g

3
5
C
o
n
s
i
s
t
e
n
t

H
e
u
r
i
s
t
i
c
s
§
A

h
e
u
r
i
s
t
i
c

i
s

c
o
n
s
i
s
t
e
n
t

(
m
o
n
o
t
o
n
i
c
)

i
f

f
o
r

e
v
e
r
y

n
o
d
e

n
,

e
v
e
r
y

s
u
c
c
e
s
s
o
r

n
'

o
f

n

g
e
n
e
r
a
t
e
d

b
y

a
n
y

a
c
t
i
o
n

a
,


h
(
n
)



c
(
n
,
a
,
n
'
)

+

h
(
n
'
)
§
I
f

h

i
s

c
o
n
s
i
s
t
e
n
t
,

w
e

h
a
v
e
:
f
(
n
'
)


=

g
(
n
'
)

+

h
(
n
'
)

=

g
(
n
)

+

c
(
n
,
a
,
n
'
)

+

h
(
n
'
)










g
(
n
)

+

h
(
n
)

=

f
(
n
)
i
.
e
.
,

f
(
n
)

i
s

n
o
n
-
d
e
c
r
e
a
s
i
n
g

a
l
o
n
g

a
n
y

p
a
t
h
.
§
T
h
e
o
r
e
m
I
f

h
(
n
)

i
s

c
o
n
s
i
s
t
e
n
t
,

A
*

u
s
i
n
g

G
r
a
p
h
-
S
e
a
r
c
h

i
s

o
p
t
i
m
a
l
.

G
r
a
p
h
-
s
e
a
r
c
h
:

n
o
d
e
s

e
n
c
o
u
n
t
e
r
e
d

a

s
e
c
o
n
d

t
i
m
e

c
a
n

b
e

t
r
e
a
t
e
d

a
s

l
e
a
f

n
o
d
e
s
.
Automated Planning and Decision Making

37

Properties of A*


Complete?


Yes, unless there are infinitely many nodes with f ≤ f(G).


Time?


Exponential.


Optimal?


Yes.


Space?


Keeps all nodes in memory.


A serious problem in many cases!


How can we overcome it?

Automated Planning and Decision Making

38

Iterative Deepening A* (IDA*)


Combine Iterative Deepening with A* in
order to overcome the space problem.


A simple idea:


Replace upper bound on depth with upper
bound on f values.


If
f(n)>U, n
is not
enqueued
.


If no solution is found given U, increase U.


Another method
--

Weighted A* (soon)

Automated Planning and Decision Making

39

Branch & Bound


Used for finding an optimal solution (i.e., when there are
multiple possible solutions, but some are better than
others)


Attempts to prune entire sub
-
trees


Applicable in the context of different search methods (BFS,
DFS, …).


Assumes we can generate upper bound U and lower
bound L on the value of solutions reachable from n


If for some node n there exists a node n


such that
U(n)<I(n

), we can prune n.


Often used as follows:


First, quickly find any solution with value v



Prune any node n s.t. f(n) > v


Automated Planning and Decision Making

40

Local search algorithms


In many problems, the
path

to the goal is
irrelevant; the goal state itself is all we
care about.


Example: n
-
queens problems


More generally: constraint
-
satisfaction
problems


Local search algorithms keep a single
"current" state, and try to improve it.

Automated Planning and Decision Making

41

Hill
-
climbing search



Like climbing mount Everest in thick fog with
amnesia…



Improve while you can, i.e. stop when
reaching a maxima (minima
)


Many variants: tie
-
breaking rules, restarts


Automated Planning and Decision Making

42

Hill
-
climbing search


Problem: depending on initial state, can
get stuck in local maxima
.


Automated Planning and Decision Making

44

Solution by Hill
-
climbing


h = number of pairs of
queens that are attacking
each other, either directly or
indirectly.


Moves: change position of a
single queen


On A, h=17.


On B, h=1. local minimum.

B

A

56

57

Automated Planning and Decision Making

45

Simulated annealing search


Idea: escape local maxima by allowing
some "bad" moves, but
gradually
decrease

their frequency.

Automated Planning and Decision Making

46

Properties of SAS


One can prove: If T decreases slowly
enough, then SAS will find a global
optimum with probability approaching 1.


Widely used in VLSI layout, airline
scheduling, etc.

Automated Planning and Decision Making

47

Local beam search


Keep track of k states rather than just one.


Start with k randomly generated states.


At each iteration, all the successors of all
k states are generated.


If any one is a goal state, stop; else select
the k best successors from the complete
list and repeat.

61

Weighted A*


Makes A* more greedy


f(n) = g(n) + W*h(n) for W>1


Gives more weight to the heuristic value,
i.e. to getting closer to the goal


Solution is at most factor W optimal


Usually reaches the goal faster and
requires less memory, but IDA* uses less
memory
.

62

63

Automated Planning and Decision Making

48

Generating a Heuristic Function


Heuristic functions are usually obtained by
finding a simple approximation to the
problem


Simple heuristics often work well (but not
very well)


Examples:


N
-
puzzle: Manhattan distance


Simplification: assumes that we can move each tile
independently. Ignores interactions between tiles


A very general idea


ignore certain interactions