Constraint Satisfaction Problems

boorishadamantΤεχνίτη Νοημοσύνη και Ρομποτική

29 Οκτ 2013 (πριν από 4 χρόνια και 2 μήνες)

82 εμφανίσεις

1

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Constraint Satisfaction Problems


Basic Algorithms




My Thanks to
Roman Bartak

(for “stealing” some of his slides)

2

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Search Algorithms for

CSPs



Simple or Chronological Backtracking (BT)



Backjumping (BJ) and Conflict
-
Based Backjumping



Forward Checking (FC)



Maintaining Arc Consistency (MAC)


We will study variations of

DFS especially for

CSPs.



These algorithms are based on backtracking search

Also two variations of hill climbing



Min
-
conflicts



Min
-
conflicts with Random Walk

3

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Intelligent Backtracking


ΒΤ
suffers from

thrashing



it visits again and again the same regions of the search tree because

it
has a very local view of the problem



One way to get rid of the problem is using
intelligent backtracking

algorithms



BJ, CBJ, DB, Graph
-
based BJ, Learning


Backjumping
(
BJ
) is different from

ΒΤ
in the following
:



When BJ reaches a dead
-
end it does not backtrack to the immediately
preceding variables
.
It backtracks to the deepest variable in the search
tree which is in conflict with the current variable

4

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

BJ vs. BT






We want to color each area in the map with a different color



We have three colors


red
,
green
,
blue

5

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

BJ vs. BT


Let’s consider what

ΒΤ
does in the map coloring problem



Assume that variables are assigned in the order

Q
,
NSW
,
V
,
T
,
SA
,
WA
,
NT



Assume that we have reached the partial assignment




Q

=
red
,
NSW

=
green
,
V

=
blue
,
T

=
red




When we try to give a value to the next variable

SA
, we find out that all
possible values violate constraints



Dead end
!



BT will backtrack to try a new value for variable

Τ
!



Not a good idea!

6

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

BJ vs. BT


BJ has a smarter approach to backtracking



It tells us to go back to one of the variables which are responsible for the
dead
-
end



The set of these variables is called a
conflict set




The conflict set for

SA is

{
Q, NSW, V}



BJ backjumps to the deepest variable in the

conflict set of the variable
where the dead
-
end occurred



deepest

=
the one we visited most recently



CBJ
,
DB
,
Graph
-
based BJ
,
Learning, Backmarking

7

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Conflict
-
based Backjumping (CBJ)


Conflict
-
based Backjumping

is a

look
-
back algorithm that performs
intelligent backtracking at dead
-
ends


In contrast to

BJ which

backjumps only from leaf dead
-
ends
,
CBJ can
also backjump from dead
-
ends at inner nodes




for each variable

x

we have a conflict set



when an assignment

(
x,a
) fails because of a constraint violation with a
previous variable

y
,
y

is added to the conflict set of
x




if there are no values left in the domain of the current variable
x
,

CBJ
backjumps to the deepest variable
w

in the conflict set of
x
(as BJ)



and the conflict set of
x

is added to the conflict set of
w



then a further

backjump can occur from

w

8

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Forward Checking


Forward Checking

(FC) belongs to the family of backtracking
algorithms called

lookahead

algorithms



The basic idea of

lookahead is that when you assign a value to a variable

the problem is reduced through

constraint propagation



constraint propagation is defined in a different way for each look
-
ahead
algorithm



FC does the following
:



When a variable

x

takes a value

v
,
for each future variabe

y

which
appears in a constraint with

x

we remove from

D
x

all the values that are
not consistent with

v

9

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Forward Checking


If the domain of some variable becomes empty then

value

v

is rejected
and we try the next value of

x



The operation of

FC means that the following holds for each step of
the search
:



All values of each

future variable

are compatible with all the values that
have been assigned to past variables




FC maintains a restricted form of

arc consistency

10

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Forward Checking

procedure
FORWARD_CHECKING (
vars
,
doms
,
cons
)


solution



FC (
vars
,
Ø
,
doms
,
cons
)


function

FC (
unlabelled,compound_label,doms,cons
)

returns
a solution or NIL


if

unlabelled

=
Ø

then return
compound_label



else

pick a variable

x
from
unlabelled




repeat




pick a value

v
from

D
x
;
delete

v
from

D
x




doms’


UPDATE(
unlabelled
-
{x},doms,cons,
compound_label
+ {(
x,v
)})


if
no domain in

doms’
is empty
then





result



FC(
unlabelled
-

{
x
},
compound_label

+








{(
x,v
)},
doms’
,
cons
)





if
result



NIL

then return
result




end



until
D
x

=
Ø



return

NIL


end

11

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Forward Checking

function

UPDATE (
unlab_vars,doms,cons,compound_label
)

returns
an updated set of domains


for
each variable
y

in
unlab_vars

do



for
each value

v
in
D
y

do



if
(
y
,
v
) is incompatible with
compound_label
with respect



to the constraints between

y
and the variables of




compound_label



then
D
y



D
y



{
v
}



end


end


return

doms


12

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

FC in operation



R
G
B

R
G
B

R
G
B

R
G
B

R
G
B

R
G
B

R
G
B

R

G
B

R
G
B

R
G
B

R
G
B

G
B

R
G
B

R

B

G

R
B

R
G
B

B

R
G
B

R

B

G

R

B

R
G
B

WA

NT

Q

NSW

V

T

SA

initial domains

after WA=
R

after Q=
G

after V=
B

13

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Consistency Techniques


removing inconsistent values from variables’ domains


graph representation of the CSP


binary and unary constraints only (relatively easy)


nodes = variables


edges = constraints


node consistency (NC)


arc consistency (AC)


path consistency (PC)


(strong) k
-
consistency

A

B

C

A>5

A

B

A
<C

B=C

14

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Node Consistency


A variable

X

is

node consistent

iff each value a of
X

satisfies all
the unary constraints on
X



Node consistency can be applied as a preprocessing step before
starting search to remove all the node inconsistent values

A

B

C

A>5

A

B

A
<C

B=C


If D(A)={0,…,9} node consistency


will remove values 0,…,5


15

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Arc Consistency


Definition
:



A variable

X

is

arc consistent

iff for each other variable

Y

the
following holds
:
For each value

a

of

Χ

there is at least one value

b

of

Υ

such that

a

and

b

are compatible



Then we say that

a
supports

b



An algorithm that applies

arc consistency deletes values from
the domain

of a variable when they are not supported by any
value in the domain of another variable

16

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Arc Consistency (AC)


the most widely used consistency technique
(good
simplification/performance ratio)


deals with individual binary constraints








repeated revisions of arcs


Directional (one pass) AC

a

b

c

a

b

c

X

Y

a

b

c

Z

17

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

AC
-

Example


Problem:


X::{1,2}, Y::{1,2}, Z::{1,2}



X = Y,

X


Z, Y > Z

1

2

1

2

1

2

1 2

1 2

1 2

X

Y

Z

X

Y

Z

18

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Arc Consistency propagation:

Crossword Puzzle example

1

2

3

4

5

X1

X2

X4

astar

happy

hello

hoses

live

load

peal

peel

save

talk

live

load

peal

peel

save

talk

….No more changes!

19

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Arc Consistency


We apply arc consistency
:



As a

(
preprocessing
) step before we start search



in that way we can reduce the size of the search tree



and in some cases discover inconsistent problems




While searching after an assignment of a value to a variable



constraint propagation


fast discovery of dead ends




The search algorithm which applies

arc consistency is called

MAC

(maintaining arc consistency)

20

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

MAC

procedure
Maintaining Arc Consistency (
vars
,
doms
,
cons
)


solution



MAC (
vars
,
Ø
,
doms
,
cons
)


function

MAC (
unlabelled,compound_label,doms,cons
)

returns
a solution or NIL


if

unlabelled

=
Ø

then return
compound_label



else

pick a variable

x
from
unlabelled




repeat




pick a value

v
from

D
x
;
delete

v
from

D
x




doms’


AC(
unlabelled
-
{x},doms,cons,
compound_label
+ {(
x,v
)})


if
no domain in

doms’
is empty
then





result



MAC(
unlabelled
-

{
x
},
compound_label

+








{(
x,v
)},
doms’
,
cons
)





if
result



NIL

then return
result




end



until
D
x

=
Ø



return

NIL


end

21

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Algorithms for

Arc Consistency


Arc consistency can be enforced with

Ο(
ed
2
)
optimal worst case time
complexity



AC
-
4, AC
-
6, AC
-
7,
AC
-
2001



AC
-
3
:

non
-
optimal,

but simple

AC algorithm



AC
-
3
and

AC
-
2001 use:




a queue (or stack) where the variables that are checked for arc
consistency are inserted



a routine

Revise which deletes values that are not supported



AC
-
4, AC
-
6, AC
-
7 use more complex data structures



support lists

22

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Achieving Arc Consistency


From Mackworth (1977a):









procedure

AC
-
3
(G)

Let Q be the set of (directed) arcs of G (not self
-
cyclic)

while

Q not empty
do


select and remove any arc (x,y) from Q;


REVISE(x,y)


if

REVISE(x,y) changed the domain of x
then


add
to Q
the set of all arcs of G
(z,x)
that go into x;

procedure

REVISE

(x,y)


for

each
value a

in domain of x
do



if

there is no
value b

in the domain of y such that (
a
,
b
) is consistent



then

delete
a

from the domain of x

23

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Achieving Arc Consistency


AC
-
2001/3.1

achieves

the optimal
Ο(
ed
2
)
complexity by using a set
of pointers Last
x,a,y



For each value
a

of a variable, Last
x,a,y

points to the most recently
discovered value in the domain of
y

that supports
a


Runtime

of
AC
-
3
: O(
e
d
3
) for graph
e

binary constraints, and
maximum
domain size of

d



For one constraint, function Revise costs O(
d
2
) and it can be called
d

times



there are
e

constraints, so the complexity is
O(
e
d
3
)

procedure

REVISE
-
2001/3.1

(x,y)


for

each
value a

in domain of x
do



if

there is no
value b

in the domain of y such that

b> Last
x,a,y
and

(
a
,
b
) is consistent



then

delete
a

from the domain of x



else

Last
x,a,y
= first such value

24

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Algorithms for Arc Consistency


In some cases we can exploit the semantics of certain binary
constraints to achieve an even better complexity



functional, anti
-
functional, monotonic, piecewise functional, etc.



algorithm AC
-
5



What the complexity of AC processing for a constraint of the
following types?



x = y



x
≠ y



x < y



x > y

25

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Directional Arc Consistency (DAC)


Observation:

AC has to repeat arc revisions; the total

number of
revisions depends on the number of arcs but

also on the size of
domains (while cycle)


Is it possible to weaken AC in such a way that every arc is

revised just
once?



Definition
:
A
CSP is
directional arc consistent

using a given

order of
variables iff every arc (i,j) such that i<j is arc

consistent



Again, every arc has to be revised, but revision in one

direction is
enough now


26

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Arc Consistency as a Solution Method


Question:



Are there cases where we can guarantee that solubility (or insolubility)
will be determined by applying arc consistency?



Answer

(Freuder 1982):



When the constraint graph of the problem is a tree



In this case, a solution can be found (if one exists) in a backtrack
-
free
manner by first applying directional arc consistency



A case of polynomially solvable CSPs




Many other such cases exist depending on the structure of the
constraint graph and the nature of the constraints

27

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Is AC enough?


empty domain => no solution


cardinality of all domains is 1 => solution


Problem:


X::{1,2}, Y::{1,2}, Z::{1,2}



X


Y,

X


Z, Y


Z








1

2

1

2

1

2

X

Y

Z

28

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Stronger Levels of

Consistency


Beyond

arc consistency there are numerous other levels of

consistency



path consistency



singleton arc consistency



neighborhood inverse consistency







These are stronger than

arc consistency

(
i.e. they delete more
inconsistent values when they are applied
)



But they are more expensive

(
higher time complexity
)




We will review some of them in the next lecture

29

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Constraint Propagation


systematic search only => no efficient


consistency only => no complete


combination of search (backtracking) with consistency techniques


methods:



look back (restoring from conflicts)



look ahead (preventing conflicts)

look back

Labelling order

look ahead

30

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Look Back Methods


intelligent backtracking


consistency checks among instantiated variables



backjumping


backtracks to the conflicting variable



backchecking and backmarking


avoids redundant constraint checking

by remembering conflicting level

for each value

jump
here

a

b

b

b

conflict

still conflict

31

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Look Ahead Methods


preventing future conflicts via consistency checks among not yet
instantiated variables



forward checking

(FC)


AC to direct neighbourhood


partial look ahead

(PLA)


DAC


(full) look ahead

(LA)


Arc Consistency


Path Consistency

labelling order

instantiated
variable

32

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Look Ahead
-

Example


Problem:


X::{1,2}, Y::{1,2}, Z::{1,2}



X = Y,

X


Z, Y > Z









generate & test
-

7 steps

backtracking
-

5 steps

propagation
-

2 steps

33

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

4
-
queen problem

Place 4 queens so that no two queens are
in attack.

1

2

3

4

Q
1

Q
2

Q
3

Q
4

Q
i
: line number of queen in column i, for 1

i

4

Q
1
, Q
2
, Q
3
, Q
4



Q
1

Q
2
, Q
1

Q
3
, Q
1

Q
4
,

Q
2

Q
3
, Q
2

Q
4
,

Q
3

Q
4
,

Q
1

Q
2
-
1, Q
1

Q
2
+1, Q
1

Q
3
-
2, Q
1

Q
3
+2,

Q
1

Q
4
-
3, Q
1

Q
4
+3,

Q
2

Q
3
-
1, Q
2

Q
3
+1, Q
2

Q
4
-
2, Q
2

Q
4
+2,

Q
3

Q
4
-
1, Q
3

Q
4
+1

34

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

4
-
queen problem first solution

1

2

3

4

Q
1

Q
2

Q
3

Q
4

There is a total of 256 valuations

GT algorithm will generate



64

valuations with Q
1
=1;


+

48

valuations with Q
1
=2, 1

Q
2

3;


+

3

valuations with Q
1
=2, Q
2
=4, Q
3
=1;

=

115

valuations to find first solution

35

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

4
-
queen problem, BT algorithm

1

2

3

4

Q
1

Q
2

Q
3

Q
4

1

2

3

4

Q
1

Q
2

Q
3

Q
4

1

2

3

4

Q
1

Q
2

Q
3

Q
4

1

2

3

4

Q
1

Q
2

Q
3

Q
4

1

2

3

4

Q
1

Q
2

Q
3

Q
4

36

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

4
-
queen problem, FC algorithm

1

2

3

4

Q
1

Q
2

Q
3

Q
4

1

2

3

4

Q
1

Q
2

Q
3

Q
4

1

2

3

4

Q
1

Q
2

Q
3

Q
4

1

2

3

4

Q
1

Q
2

Q
3

Q
4

1

2

3

4

Q
1

Q
2

Q
3

Q
4

1

2

3

4

Q
1

Q
2

Q
3

Q
4

1

2

3

4

Q
1

Q
2

Q
3

Q
4

37

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

4
-
queen problem, MAC algorithm

Value 3 of Q
2
is unsupported in Q
3
,

Value 4 of Q
3
is unsupported in Q
2
,

Value 2 of Q
3
is unsupported in Q
4
,

1

2

3

4

Q
1

Q
2

Q
3

Q
4

1

2

3

4

Q
1

Q
2

Q
3

Q
4

Q
1

Q
2

Q
3

Q
4

1

2

3

4

x

1

2

3

4

Q
1

Q
2

Q
3

Q
4

x

x

38

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Hybrid Algorithms


We can combine the operations of various backtracking algorithms to
design

hybrid algorithms



For example

we can combine the

lookahead function of

forward
checking and the

lookback function of

BJ



FC
-
BJ



FC
-
CBJ



MAC
-
BJ



MAC
-
CBJ






39

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

FC
-
CBJ


Forward Checking with Conflict
-
based Backjumping




FC
-
CBJ combines the look
-
ahead of FC and the intelligent backjumping
of CBJ



each variable is associated with a conflict set



when the forward checking of an assignment (
x,a
) results in a value
deletion from the domain of a variable

y
,
x

is added to the conflict set of
y




if after the forward checking of an assignment (
x,a
) the domain of a
variable
y

is wiped out, the variables in the conflict set of
y

are added to the
conflict set of
x



why is this done?



if there are no more values left in the domain of the current variable
x
, FC
-
CBJ backjumps to the deepest variable
w

in the conflict set of
x



the conflict set of
x

is added to the conflict set of
w

40

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Evaluation of Backtracking Algorithms


How can we compare backtracking algorithms for

CSPs ?



Time

/
Space Complexity



not very useful
.
They all have exponential time complexity
!




cpu times




number of nodes they visit in the search tree




amount of

consistency checks they perform




amount of backtracks they perform

41

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Evaluation of Backtracking Algorithms


Some theoretical results
:





Search tree nodes visited




FC
-
CBJ


FC
-
BJ


FC


BJ


BT




CBJ


BJ




Number of consistency checks




CBJ


BJ



ΒΤ




FC
-
CBJ


FC
-
BJ


FC




CPU times

?

We always need

experiments
!!!

42

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Heuristic Methods for

CSPs


Search algorithms must take decisions
:

1)

Which will be the next variable to assign

?

2)

Which value should I give it

?

3)

Which constraint should I check

?



The decisions that the algorithm takes at each step have a drastic
effect on the search space

(
and the efficiency of the algorithm
)



Especially decision

(1)



Heuristics help the algorithms take correct decisions


fail first principle

43

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Heuristic Methods for

CSPs


Variable ordering heuristics



static heuristics



MaxDegree, Bandwidth, …



dynamic heuristics



MRV, Brelaz, dom/deg, dom/wdeg…



Value ordering heuristics



Geelen’s promise, least
-
constraining…



Heuristics for constraint ordering



based on the cost of propagation

44

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Variable Ordering Heuristics


Minimum Width



The width of a variable
x

is the number of variables that are before
x
,
according to a given ordering, and are constrained with
x



The width of an ordering is the maximum width of all the variables
under that ordering



The width of a constraint graph is the minimum width of all possible
orderings



Variables are ordered in descending width



useful when the degree of the nodes varies significantly




Problem:

How many are the possible orderings?

45

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Variable Ordering Heuristics


Maximum Degree



Variables are ordered in decreasing order of their degree in the
constraint graph



degree is the number of adjacent variables in the graph



Heuristic to find a minimum width ordering



Maximum Cardinality



Selects the first variable arbitrarily



Then, at each stage, selects the variable that is adjacent to the largest set
of already selected variables.

46

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Variable Ordering Heuristics


Minimum Bandwidth



The bandwidth of a variable
x
, according to a given ordering, is the
maximum distance between
x

and any other variable which is adjacent to
x



The bandwidth of an ordering is the maximum bandwidth of all the
variables under that ordering



The bandwidth of a constraint graph is the minimum bandwidth of all
possible orderings



Idea:

The closer the variables involved in a constraint are placed to each
other the less backtracking will be required



Problem:

Computing the minimum bandwidth is NP
-
complete


47

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Dynamic Variable Ordering Heuristics


Minimum Remaining Values (MRV) or Smallest Domain (SD)



At each stage of search select the variable with the smallest domain size





How do we break ties?



Select a variable randomly



Select the variable with the highest degree in the original graph



Select the variable with the highest future degree (i.e. the one involved
in the maximum number of constraints with future variables). This is
called the
Brelaz

heuristic



Many variations have been proposed



dom/deg, dom/fdeg

48

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

State
-
of
-
the
-
art Dynamic Variable Ordering Heuristics


Weighted degree heuristics



each constraint is associated with a weight initially set to 1



each time a constraint
c

removes the last value from a domain (i.e.
causes a domain wipeout
-

DWO) its weight is incremented by 1



the
weighted degree

of a variable
x

is the sum of the weights of the
constraints that include x



wdeg heuristic



selects the variable with maximum weighted degree



dom/wdeg heuristic



selects the variable with minimum ration of domain size to weighted degree


What is the rationale behind these heuristics?



they use information gathered throughout search


not just from the
current search state like
dom/fdeg


49

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Value Ordering Heuristics


Min
-
Conflicts



Associate with each value
a

the total number of values in future
variables that are incompatible with
a



Select the value with lowest sum



Alternative:

Divide the number of incompatible values of future variable
x with the domain size of x



Geelen’s Promise



For each value
a

count the total number of values in each future variable
that are compatible with
a



Take the product of the counts. This is called the
promise

of value
a



Select the value with the maximum promise

50

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Constraint Ordering Heuristics


Is this issue important?



not very much when maintaining arc consistency



but there exist heuristics for ordering the constraints in the propagation
queue. Can you think of such a heuristic?



but very important in modern advanced solvers that use propagators
for the various (global) constraints



the idea here is to propagate the less expensive constraints first

51

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Stochastic and Local Search Methods


local search
-

chooses best neighbouring configuration


hill climbing


neighbourhood = value of one variable changed


min
-
conflicts


neighbourhood = value of selected conflicting variable


changed


can we avoid local optima?


restarts

if at a local optimum, start procedure from scratch



random
-
walk


sometimes picks neighbouring configuration randomly


tabu search


few last configurations are forbidden for next step


local search does not guarantee completeness

52

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

The Min
-
Conflicts Algorithm


Start with a random assignment of values to variables



or a seemingly good one according to some heuristic



some constraints will be violated



Try to repair it



change the value assignment that resolves the greatest numbers of
constraints



local optima can be escaped using
random restarts


Otherwise:



Simulated annealing



Tabu search



Random walk

53

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Min
-
Conflicts (version 1)

procedure

Min_Conflicts(
P
,
maxTries
,
maxChanges
)


for

i :=1 to
maxTries

do



A

:= initial complete assignment of the variables in
P



for

j:=1 to
maxChanges
do




if

A

satisfies
P

then

return (
A
)




else






x

:= randomly chosen variable whose assignment is in conflict





(x,a)

:= alternative assignment of
x

which satisfies





the maximum number of constraints under the current



assignment
A





if

by making assignment
(x,a)

you get a cost
≤ current cost
then






make the assignment




endif



endfor


endfor


return

(“No solution found”)


54

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Min
-
Conflicts (version 2)

procedure

Min_Conflicts(
P
,
maxTries
,
maxChanges
)


for

i :=1 to
maxTries

do



A

:= initial complete assignment of the variables in
P



for

j:=1 to
maxChanges
do




if

A

satisfies
P

then

return (
A
)




else







(x,a)

:= the alternative assignment of a variable
x

which minimizes




the number of constraint violations under the current




assignment
A





if

by making assignment
(x,a)

you get a cost
≤ current cost
then






make the assignment





else break




endif



endfor


endfor


return

(“No solution found”)



55

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Min
-
Conflicts with Random Walk


How
can we

leave the local optimum without a restart




(i.e. via a local step)?



By adding some “noise” to the algorithm!


Random walk



a state from the neighbourhood is selected randomly

(e.g., the value is
chosen randomly)



such technique can hardly find a solution



so it needs some guide


Random walk can be combined with the heuristic guiding

the
search via probability distribution:



p

-

probability of using the random walk



(1
-
p)

-

probability of using the heuristic guide

56

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Min
-
Conflicts with Random Walk (version 1)

procedure

Min_Conflicts(
P
,
maxChanges,p
)


A

:= initial complete assignment of the variables in
P


for

j:=1 to
maxChanges
do



if

A

satisfies
P

then

return (
A
)



else





if

probability
p

verified






x

:= randomly chosen variable whose assignment is in conflict





(x,a)

:= randomly chosen alternative assignment of
x





else





(x,a)

:= the alternative assignment of a variable
x

which minimizes



the number of constraint violations under the current




assignment
A




make the assignment
(x,a)




endif


endfor


return

(“No solution found”)


57

ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ
-

Lecture 1

Min
-
Conflicts with Random Walk (version 2)

procedure

Min_Conflicts(
P
,
maxChanges,p
)


A

:= initial complete assignment of the variables in
P


for

j:=1 to
maxChanges
do



if

A

satisfies
P

then

return (
A
)



else




x

:= randomly chosen variable whose assignment is in conflict




if

probability
p

verified






(x,a)

:= randomly chosen alternative assignment of
x





else





(x,a)

:= the alternative assignment of
x

which satisfies




the maximum number of constraints under the current assignment
A




make the assignment
(x,a)




endif


endfor


return

(“No solution found”)