Bayesian networks

AIMA2e Chapter 14

AIMA2e Chapter 14 1

Outline

You have 3 lectures that should be read in this order:

} chapter14a-BBN.pdf:intro

} chapter14b-BBN.pdf:inference (skip slides:8,9,11,12,[23-38])

} chapter14c-MSBNx.ppt:intro with MS Belief Network software

} Bayesian Belief Net.doc (extensive/detailed tutorial use only as tuto-

rial/reference if you don't understand the above slides)

AIMA2e Chapter 14 2

Outline

} Syntax

} Semantics

} Constructing Bayesian Networks

AIMA2e Chapter 14 3

Bayesian networks

A Bayesian Belief Network is a directed acyclic graph where:

a set of nodes,one per random variable

a directed edges forming an acyclic graph (link

¼

\directly in°uences")

a conditional distribution for each node given its parents:

P(X

i

jParents(X

i

))

The arcs can be thought as causal relationships between variables,

but speci¯cally an arc from X to Y means that

X has direct in°uence

on our belief in Y.

Weather

Cavity

Toothache

Catch

AIMA2e Chapter 14 4

Bayesian networks

Notice that a BBN is a simple graphical notation for conditional indepen-

dence assertions and hence for compact speci¯cation of full joint distributions

Conditional distributions are represented as a

conditional probability table

AIMA2e Chapter 14 5

(CPT) giving the distribution over

X

i

for each combination of parent values

.001

P(B)

.002

P(E)

Alarm

Earthquake

MaryCalls

JohnCalls

Burglary

A

P(J)

t

f

.90

.05

A P(M)

t

f

.70

.01

B

t

t

f

f

E

t

f

t

f

P(A)

.95

.29

.001

.94

AIMA2e Chapter 14 6

Example

Topology of network encodes conditional independence assertions:

Weather

Cavity

Toothache

Catch

Weather

is independent of the other variables

Toothache

and

Catch

are conditionally independent given

Cavity

AIMA2e Chapter 14 7

Example

I'm at work,neighbor John calls to say my alarm is ringing,but neighbor

Mary doesn't call.Sometimes it's set o® by minor earthquakes.Is there a

burglar?

Variables:

Burglar

,

Earthquake

,

Alarm

,

JohnCalls

,

MaryCalls

Network topology re°ects\causal"knowledge:

{ A burglar can set the alarm o®

{ An earthquake can set the alarm o®

{ The alarm can cause Mary to call

{ The alarm can cause John to call

AIMA2e Chapter 14 8

Example contd.

AIMA2e Chapter 14 9

.001

P(B)

.002

P(E)

Alarm

Earthquake

MaryCalls

JohnCalls

Burglary

A

P(J)

t

f

.90

.05

A P(M)

t

f

.70

.01

B

t

t

f

f

E

t

f

t

f

P(A)

.95

.29

.001

.94

To be more exact,we should say in the tables

P(ajB;E)

(i.e.use

a

instead

AIMA2e Chapter 14 10

of

A

because we are listing the case for

A=True

.

AIMA2e Chapter 14 11

Global semantics of Bayesian Networks

A Bayesian network de¯nes the full joint distribution

as the product of the local conditional distributions:

P(X

1

;:::;X

n

) = ¦

n

i =1

P(X

i

jParents(X

i

))

AIMA2e Chapter 14 12

Constructing Bayesian networks

Need a method such that a series of locally testable assertions of

conditional independence guarantees the required global semantics

1.Choose an ordering of variables

X

1

;:::;X

n

2.For

i

= 1 to

n

add

X

i

to the network

select parents from

X

1

;:::;X

i¡1

such that

P(X

i

jParents(X

i

)) = P(X

i

jX

1

;:::;X

i¡1

)

This choice of parents guarantees the global semantics:

P(X

1

;:::;X

n

) = ¦

n

i =1

P(X

i

jX

1

;:::;X

i¡1

)

(chain rule)

= ¦

n

i =1

P(X

i

jParents(X

i

))

(by construction)

AIMA2e Chapter 14 13

Example

Suppose we choose the ordering

M

,

J

,

A

,

B

,

E

P(JjM) = P(J)

?

AIMA2e Chapter 14 14

Example

Suppose we choose the ordering

M

,

J

,

A

,

B

,

E

P(JjM) = P(J)

?No

P(AjJ;M) = P(AjJ)

?

P(AjJ;M) = P(A)

?

AIMA2e Chapter 14 15

Example

Suppose we choose the ordering

M

,

J

,

A

,

B

,

E

P(JjM) = P(J)

?No

P(AjJ;M) = P(AjJ)

?

P(AjJ;M) = P(A)

?No

P(BjA;J;M) = P(BjA)

?

P(BjA;J;M) = P(B)

?

AIMA2e Chapter 14 16

Example

Suppose we choose the ordering

M

,

J

,

A

,

B

,

E

P(JjM) = P(J)

?No

P(AjJ;M) = P(AjJ)

?

P(AjJ;M) = P(A)

?No

P(BjA;J;M) = P(BjA)

?Yes

P(BjA;J;M) = P(B)

?No

P(EjB;A;J;M) = P(EjA)

?

P(EjB;A;J;M) = P(EjA;B)

?

AIMA2e Chapter 14 17

Example

Suppose we choose the ordering

M

,

J

,

A

,

B

,

E

P(JjM) = P(J)

?No

P(AjJ;M) = P(AjJ)

?

P(AjJ;M) = P(A)

?No

P(BjA;J;M) = P(BjA)

?Yes

P(BjA;J;M) = P(B)

?No

P(EjB;A;J;M) = P(EjA)

?No

P(EjB;A;J;M) = P(EjA;B)

?Yes

AIMA2e Chapter 14 18

Example contd.

Deciding conditional independence is hard in noncausal directions

(Causal models and conditional independence seem hardwired for humans!)

Assessing conditional probabilities is hard in noncausal directions

AIMA2e Chapter 14 19

Compactness

A CPTfor Boolean

X

i

with

k

Boolean parents has

2

k

rows for the combinations of parent values

Each row requires one number

p

for

X

i

=true

(the number for

X

i

=false

is just

1 ¡p

)

If each variable has no more than

k

parents,

the complete network requires

O(n ¢ 2

k

)

numbers

I.e.,grows linearly with

n

,vs.

O(2

n

)

for the full joint distribution

For the small burglary net,

1 +1 +4 +2 +2 =10

numbers (vs.

2

5

¡1 = 31

)

For the large burglary net,the network is less compact:

1+2+4+2+4 =13

numbers needed

AIMA2e Chapter 14 20

Global semantics of Bayesian Networks

A Bayesian network de¯nes the full joint distribution

as the product of the local conditional distributions:

P(X

1

;:::;X

n

) = ¦

n

i =1

P(X

i

jParents(X

i

))

e.g.,

P(j ^m^a ^:b ^:e)

=

AIMA2e Chapter 14 21

Global semantics of Bayesian Networks

A Bayesian network de¯nes the full joint distribution

as the product of the local conditional distributions:

P(X

1

;:::;X

n

) = ¦

n

i =1

P(X

i

jParents(X

i

))

e.g.,

P(j ^m^a ^:b ^:e)

= P(jja)P(mja)P(aj:b;:e)P(:b)P(:e)

AIMA2e Chapter 14 22

Local semantics

Local

semantics:each node is conditionally independent

of its nondescendants given its parents

Theorem:

Local semantics

,

global semantics

AIMA2e Chapter 14 23

Markov blanket

Each node is conditionally independent of all others given its

Markov blanket

:parents + children + children's parents

AIMA2e Chapter 14 24

Example:Car diagnosis

Initial evidence:car won't start

Testable variables (green),\broken,so ¯x it"variables (orange)

Hidden variables (gray) ensure sparse structure,reduce parameters

AIMA2e Chapter 14 25

Example:Car insurance

SocioEcon

Age

GoodStudent

ExtraCar

Mileage

VehicleYear

RiskAversion

SeniorTrain

DrivingSkill

MakeModel

DrivingHist

DrivQuality

Antilock

Airbag

CarValue

HomeBase

AntiTheft

Theft

OwnDamage

PropertyCost

LiabilityCost

MedicalCost

Cushioning

Ruggedness

Accident

OtherCost

OwnCost

AIMA2e Chapter 14 26

Summary

Bayes nets provide a natural representation for (causally induced)

conditional independence

Topology + CPTs = compact representation of joint distribution

Generally easy for (non)experts to construct

SOME OF THE TOPIC (more examples and how belief update is done) IS

COVERED IN Bayesian Belief Net.doc!

AIMA2e Chapter 14 27

## Comments 0

Log in to post a comment