Bayesian Networks

brewerobstructionAI and Robotics

Nov 7, 2013 (3 years and 11 months ago)

63 views

Bayesian Networks

Quiz: Probabilistic Reasoning

1.
What is P(F), the probability that some creature can
fly?

2.
Creature b is a bumble bee. What’s P(F|B), the
probability that b can fly given that it’s a bumble bee?

3.
b

has unfortunately met a malicious child, who has
torn off b’s wings. What is P(F|B,N), the probability
that b can fly given that it has no wings?

4.
b

somehow makes its way onto a jumbo jet, where it
survives by drinking juice spilled by passengers. What
is P(F|B, N, L=j), the probability that b can fly given
that it has no wings and its location is a jet?

Example

BN = (V, E, P)


V = a set of random variables


E = directed edges between
them (cycles not allowed)


P = for every node in the
network, a conditional
probability distribution for
that random variable, given
its parents in the graph

Has
diabetes?

(D or

D)

Test was
positive?

(+ or
-
)

Observable
node

Unobservable
node

Diab
?

P(D)

D

0.01


D

〮㤹

Diab
?

Test?

P(T|D)


D

+

0.9


D

-

0.1


D

+

〮0


D

-

〮0

Simple probabilistic reasoning

You already know how to figure out:


P(D)


stored in the Bayes Net


P(+|D)


stored in the Bayes Net


P(D,+)


multiply P(D)P(+|D)


P(+)


apply marginalization to P(D, +)


P(D|+)


apply Bayes’ Rule


Purpose behind Bayes Networks

Bayes Nets help figure out more difficult
cases:

What’s

P(Battery Dead |


Car won’t start, Battery is 5 years old)?

or

P(Alternator broken |


Car won’t start, oil light is on, lights are dim)?

Battery

dead

Battery

age

Fan belt

broken

Battery

meter

No

oil

Battery

flat

Alternator

broken

Not

charging

No

gas

Starter

broken

Fuel line

blocked

Lights

Gas

gauge

Oil

light

Car won’t

start

dipstick

Types of Bayes Net Queries

Bayes Nets let you solve “queries”, or probabilistic questions.


There are different types of queries for a Bayes Net with
random variables X1, …, XN:

1.
Joint queries: What is P(car starts, oil light on)?

2.
Conditional queries: What is P(alternator broken, battery
light dim | oil light off, lights dim)?

3.
Maximum a posteriori (MAP):


what values (true or false) for “Will Car Start?” makes
this probability the biggest:


P(Will Car Start? | battery is 5 years old, lights dim)

The Bayes Net Equation

A BN specifies the joint distribution over all
random variables in the graph, using this
eqn
:


𝑃
𝑋
1
,

,
𝑋
𝑁
=

𝑃
𝑋
𝑖
|
𝑝𝑎 𝑒𝑛
(
𝑋
𝑖
)
𝑋
𝑖

Example

P(
Diab
, Test) =


P(
Diab|parents
(
Diab
))


*P(
Test|parents
(Test))


=

P(
Diab
)


*P(
Test|Diab
)

Has
diabetes?

(D or

D)

Test was
positive?

(+ or
-
)

Quiz: Two
-
test Diabetes

1.
What is

P(Test1=+|D)?


2. What is

P(Test1=+|D,Test2=+)?


3. What is

P(D|Test1=+,Test2=+)?


4.
What is

P(D|Test1=+,Test2
=
-
)?


Has
diabetes?

(D or

D)

Test 1 was
positive?

(+ or
-
)

Test 2 was
positive?

(+ or
-
)

Diab
?

P(D)

D

0.01


D

〮㤹

Diab
?

Test1?

P(T1|D)


D

+

0.9


D

-

0.1


D

+

〮0


D

-

〮0

Diab
?

Test2?

P(T2|D)


D

+

0.9


D

-

0.1


D

+

〮0


D

-

〮0

Conditional Independence in a BN

In this BN,

T1


T2 | D


This means, e.g.:

P(T1=+|D, T2=+)

is the same as

P(T1=+|D)

Has
diabetes?

(D or

D)

Test 1 was
positive?

(+ or
-
)

Test 2 was
positive?

(+ or
-
)

Quiz: Two
-
test Diabetes

What is P(T1=+|T2=+)?

Has
diabetes?

(D or

D)

Test 1 was
positive?

(+ or
-
)

Test 2 was
positive?

(+ or
-
)

Absolute vs. Conditional Independence

Remember:

T1


T2 | D


Does this mean
that
T1


T2
?


In other words,

P(T1) =? P(T1 | T2)

Has
diabetes?

(D or

D)

Test 1 was
positive?

(+ or
-
)

Test 2 was
positive?

(+ or
-
)

Confounding Cause

1.
What is P(R | S)?



2.
What is P(R | H, S)?



3.
What is P(R | H,

S
)?



4.
What is P(R | H)?

Happy?

(H or

H)

Sunny?

(S or

S)

Raise?

(R or

R)

S
?

P(D)

S

0.7


S

〮0

R
?

P(R)

R

0.01


R

〮㤹

Happy?

Sunny?

Raise?

P(H|S,R)?

H

S

R

1.0

H

S


R

〮0

H


S

R

〮0

H


S


R

〮0

Absolute vs. Conditional Independence

Remember:

R


S


Does this mean
that

R



S | H ?



In other words,

P(R | H) =? P(R | H, S)

Happy?

(H or

H)

Sunny?

(S or

S)

Raise?

(R or

R)

D
-
Separation

D
-
separation is the technical method for
determining conditional independence in a BN.

Active Triplets

Inactive Triplets







D
-
Separation

Node A is
d
-
separated

(short for
directional
-
separated
)
from node B if

all paths from A to B contain at least one inactive triplet.



A


B |
K
1
, …, K
m






nodes
A and B are d
-
separated when nodes K
1
, …, K
m

are
known


D
-
Separation Quiz 1

C


A?


C


A | B?


C


D?


C


D | A?


E


C | D?


D

A

B

C

E

D
-
Separation
Quiz
2

A



E
?


A


E | B?


A


E | C?


A


B
?


A



B | C?


C

A

B

D

E

D
-
Separation Quiz

F


A?


F


A

| D?


F


A | G?


F



A | H?



B

A

C

D

F

E

G

H

Counting BN Parameters

A complete joint
distribution over 5
binary variables would
require 31 = 2
5
-
1
parameters.


This BN requires

10 =
1+1+4+2+2
parameters.

C

A

B

D

E

Quiz

A full joint over 6
binary variables
requires 2
6
-
1 = 63
parameters.


How many parameters
does this network
require?

C

A

B

D

E

F

Quiz

A full joint distribution
over 7 binary variables
requires 2
7
-
1 = 127
parameters.


How many parameters
does this network
require?

D

A

C

E

G

F

B

Quiz

A full joint distribution over
16 binary variables requires
2
16
-
1 = 65,535 parameters.


How many parameters does
this network require?

Battery

dead

Battery

age

Fan belt

broken

Battery

meter

No

oil

Battery

flat

Alternator

broken

Not

charging

No

gas

Starter

broken

Fuel line

blocked

Lights

Gas

gauge

Oil

light

Car won’t

start

dipstick