Bayesian Networks
Quiz: Probabilistic Reasoning
1.
What is P(F), the probability that some creature can
fly?
2.
Creature b is a bumble bee. What’s P(FB), the
probability that b can fly given that it’s a bumble bee?
3.
b
has unfortunately met a malicious child, who has
torn off b’s wings. What is P(FB,N), the probability
that b can fly given that it has no wings?
4.
b
somehow makes its way onto a jumbo jet, where it
survives by drinking juice spilled by passengers. What
is P(FB, N, L=j), the probability that b can fly given
that it has no wings and its location is a jet?
Example
BN = (V, E, P)
V = a set of random variables
E = directed edges between
them (cycles not allowed)
P = for every node in the
network, a conditional
probability distribution for
that random variable, given
its parents in the graph
Has
diabetes?
(D or
D)
Test was
positive?
(+ or

)
Observable
node
Unobservable
node
Diab
?
P(D)
D
0.01
D
〮㤹
Diab
?
Test?
P(TD)
D
+
0.9
D

0.1
D
+
〮0
D

〮0
Simple probabilistic reasoning
You already know how to figure out:
•
P(D)
stored in the Bayes Net
•
P(+D)
stored in the Bayes Net
•
P(D,+)
multiply P(D)P(+D)
•
P(+)
apply marginalization to P(D, +)
•
P(D+)
apply Bayes’ Rule
Purpose behind Bayes Networks
Bayes Nets help figure out more difficult
cases:
What’s
P(Battery Dead 
Car won’t start, Battery is 5 years old)?
or
P(Alternator broken 
Car won’t start, oil light is on, lights are dim)?
Battery
dead
Battery
age
Fan belt
broken
Battery
meter
No
oil
Battery
flat
Alternator
broken
Not
charging
No
gas
Starter
broken
Fuel line
blocked
Lights
Gas
gauge
Oil
light
Car won’t
start
dipstick
Types of Bayes Net Queries
Bayes Nets let you solve “queries”, or probabilistic questions.
There are different types of queries for a Bayes Net with
random variables X1, …, XN:
1.
Joint queries: What is P(car starts, oil light on)?
2.
Conditional queries: What is P(alternator broken, battery
light dim  oil light off, lights dim)?
3.
Maximum a posteriori (MAP):
what values (true or false) for “Will Car Start?” makes
this probability the biggest:
P(Will Car Start?  battery is 5 years old, lights dim)
The Bayes Net Equation
A BN specifies the joint distribution over all
random variables in the graph, using this
eqn
:
𝑃
𝑋
1
,
…
,
𝑋
𝑁
=
𝑃
𝑋
𝑖

𝑝𝑎 𝑒𝑛
(
𝑋
𝑖
)
𝑋
𝑖
Example
P(
Diab
, Test) =
P(
Diabparents
(
Diab
))
*P(
Testparents
(Test))
=
P(
Diab
)
*P(
TestDiab
)
Has
diabetes?
(D or
D)
Test was
positive?
(+ or

)
Quiz: Two

test Diabetes
1.
What is
P(Test1=+D)?
2. What is
P(Test1=+D,Test2=+)?
3. What is
P(DTest1=+,Test2=+)?
4.
What is
P(DTest1=+,Test2
=

)?
Has
diabetes?
(D or
D)
Test 1 was
positive?
(+ or

)
Test 2 was
positive?
(+ or

)
Diab
?
P(D)
D
0.01
D
〮㤹
Diab
?
Test1?
P(T1D)
D
+
0.9
D

0.1
D
+
〮0
D

〮0
Diab
?
Test2?
P(T2D)
D
+
0.9
D

0.1
D
+
〮0
D

〮0
Conditional Independence in a BN
In this BN,
T1
T2  D
This means, e.g.:
P(T1=+D, T2=+)
is the same as
P(T1=+D)
Has
diabetes?
(D or
D)
Test 1 was
positive?
(+ or

)
Test 2 was
positive?
(+ or

)
Quiz: Two

test Diabetes
What is P(T1=+T2=+)?
Has
diabetes?
(D or
D)
Test 1 was
positive?
(+ or

)
Test 2 was
positive?
(+ or

)
Absolute vs. Conditional Independence
Remember:
T1
T2  D
Does this mean
that
T1
T2
?
In other words,
P(T1) =? P(T1  T2)
Has
diabetes?
(D or
D)
Test 1 was
positive?
(+ or

)
Test 2 was
positive?
(+ or

)
Confounding Cause
1.
What is P(R  S)?
2.
What is P(R  H, S)?
3.
What is P(R  H,
S
)?
4.
What is P(R  H)?
Happy?
(H or
H)
Sunny?
(S or
S)
Raise?
(R or
R)
S
?
P(D)
S
0.7
S
〮0
R
?
P(R)
R
0.01
R
〮㤹
Happy?
Sunny?
Raise?
P(HS,R)?
H
S
R
1.0
H
S
R
〮0
H
S
R
〮0
H
S
R
〮0
Absolute vs. Conditional Independence
Remember:
R
S
Does this mean
that
R
S  H ?
In other words,
P(R  H) =? P(R  H, S)
Happy?
(H or
H)
Sunny?
(S or
S)
Raise?
(R or
R)
D

Separation
D

separation is the technical method for
determining conditional independence in a BN.
Active Triplets
Inactive Triplets
…
D

Separation
Node A is
d

separated
(short for
directional

separated
)
from node B if
all paths from A to B contain at least one inactive triplet.
A
B 
K
1
, …, K
m
nodes
A and B are d

separated when nodes K
1
, …, K
m
are
known
D

Separation Quiz 1
C
A?
C
A  B?
C
D?
C
D  A?
E
C  D?
D
A
B
C
E
D

Separation
Quiz
2
A
E
?
A
E  B?
A
E  C?
A
B
?
A
B  C?
C
A
B
D
E
D

Separation Quiz
F
A?
F
A
 D?
F
A  G?
F
A  H?
B
A
C
D
F
E
G
H
Counting BN Parameters
A complete joint
distribution over 5
binary variables would
require 31 = 2
5

1
parameters.
This BN requires
10 =
1+1+4+2+2
parameters.
C
A
B
D
E
Quiz
A full joint over 6
binary variables
requires 2
6

1 = 63
parameters.
How many parameters
does this network
require?
C
A
B
D
E
F
Quiz
A full joint distribution
over 7 binary variables
requires 2
7

1 = 127
parameters.
How many parameters
does this network
require?
D
A
C
E
G
F
B
Quiz
A full joint distribution over
16 binary variables requires
2
16

1 = 65,535 parameters.
How many parameters does
this network require?
Battery
dead
Battery
age
Fan belt
broken
Battery
meter
No
oil
Battery
flat
Alternator
broken
Not
charging
No
gas
Starter
broken
Fuel line
blocked
Lights
Gas
gauge
Oil
light
Car won’t
start
dipstick
Comments 0
Log in to post a comment