# Bayesian Networks - Computer Science - University of San Francisco

Τεχνίτη Νοημοσύνη και Ρομποτική

7 Νοε 2013 (πριν από 4 χρόνια και 7 μήνες)

88 εμφανίσεις

ArtiﬁcialIntelligenceProgramming
BayesianNetworks
ChrisBrooks
DepartmentofComputerScience
UniversityofSanFrancisco
DepartmentofComputerScience—UniversityofSanFrancisco–p.1/23
19-0:
ProbabilisticInference
drawlogicalinferences.




















 

 




 
 

Wewouldliketoextendthissortofoperationtoworlds
withuncertainty.

 













 










 







 



DepartmentofComputerScience—UniversityofSanFrancisco–p.2/23
19-1:
ProbabilisticInference
Ingeneral,wecanuseBayes’Ruletodothis.
Theproblemisinworkingwiththejointprobability
distribution.
Exponentiallylargetable.
variablesdonotinﬂuenceeachother.
Forexample,thecolorofBart’shatdoesnotinﬂuence
whetherHomerishungry.
WecallthisstructureaBayesiannetwork(orabelief
network)
DepartmentofComputerScience—UniversityofSanFrancisco–p.3/23
19-2:
BayesianNetwork
nodeisannotatedwithprobabilityinformation.Anetwork
has:
Asetofrandomvariablesthatcorrespondtothenodes
inthenetwork.
Asetofdirectededgesorarrowsconnectingnodes.
Theserepresentinﬂuence.InthereisanarrowfromX
toY,thenXistheparentofY.
Eachnodekeepsaconditionalprobabilitydistribution
indicatingtheprobabilityofeachvalueitcantake,
conditionalonitsparentsvalues.
Thetopologyofthenetworkspeciﬁeswhatvariables
directlyinﬂuenceothervariables.(conditional
independencerelationships).
DepartmentofComputerScience—UniversityofSanFrancisco–p.4/23
19-3:
Burglaryexample
B
T
T
F
F
E
T
F
T
F
P(A)
.95
.29
.001
.001
P(B)
.002
P(E)
Alarm
Earthquake
MaryCalls
JohnCalls
Burglary
A
P(J)
T
F
.90
.05
AP(M)
T
F
.70
.01
.94
Twoneighborswillcall
whentheyhearyour
alarm.
Johnsometimes
overreacts
Marysometimes
missesthealarm.
Twothingscansetoffthe
alarm
Earthquake
Burglary
Givenwhohascalled,
what’stheprobabilityofa
burglary?
DepartmentofComputerScience—UniversityofSanFrancisco–p.5/23
19-4:
Networkstructure
Eachnodehasaconditionalprobabilitytable.
Thisgivestheprobabilityofeachvalueofthatnode,given
itsparents’values.
Thesesumto1.
Nodeswithnoparentsjustcontainpriors.
DepartmentofComputerScience—UniversityofSanFrancisco–p.6/23
19-5:
Summarizinguncertainty
Noticethatwedon’tneedtohavenodesforallthe
reasonswhyMarymightnotcall.
Aprobabilisticapproachletsussummarizethis
informationin

Thisallowsasmallagenttodealwithlargeworldsthat
havealargenumberofpossiblyuncertainoutcomes.
Howwouldwehandlethiswithlogic?
DepartmentofComputerScience—UniversityofSanFrancisco–p.7/23
19-6:
Implicitlyrepresentingthefull
jointdistribution
Recallthatthefulljointdistributionallowsustocalculate
theprobabilityofanyvariable,givenalltheothers.
Independenteventscanbeseparatedintoseparate
tables.
ThesearetheCPTsseenintheBayesiannetwork.
Therefore,wecanusethisinfotoperformcomputations.



 





 

 

 













 








DepartmentofComputerScience—UniversityofSanFrancisco–p.8/23
19-7:
ConstructingaBayesian
network
ThereareoftenseveralwaystoconstructaBayesian
network.
Theknowledgeengineerneedstodiscoverconditional
independencerelationships.
Parentsofanodeshouldbethosevariablesthatdirectly
inﬂuenceitsvalue.
JohnCallsisinﬂuencedbyEarthquake,butnotdirectly.
JohnandMarycallingdon’tinﬂuenceeachother.
Formally,webelievethat:

 

 

 

 

 



 

 

 

DepartmentofComputerScience—UniversityofSanFrancisco–p.9/23
19-8:
CompactnessandNode
Ordering
Bayesiannetworksallowforamorecompact
representationofthedomain
Redundantinformationisremoved.
Example:Saywehave30nodes,eachwith5parents.
EachCPTwillcontain

 

numbersTotal:960.
Joint:


entries,nearlyallredundant.
DepartmentofComputerScience—UniversityofSanFrancisco–p.10/23
19-9:
Buildinganetwork
Beginwithrootcauses
Thisiscalledacausalmodel
Reasoningintermsofcauseandeffect
Estimatesaremucheasiertocomeupwiththisway.
Wecouldtrytobuildfromeffecttocause,butthenetwork
wouldbemorecomplex,andtheCPTshardtoestimate.

 

DepartmentofComputerScience—UniversityofSanFrancisco–p.11/23
19-10:
ConditionalIndependencein
Bayesiannetworks
Recallthatconditionalindependencemeansthattwo
variablesareindependentofeachother,giventhe
observationofathirdvariable.

 

 

Anodeisconditionallyindependentofits
nondescendants,givenitsparents.
GivenAlarm,JohnCallsisindependentofBurglaryand
Earthquake.
Anodeisconditionallyindependentofallothernodes,
givenitsparents,children,andsiblings(thechildren’s
otherparents).
BurglaryisindependentofJohnCallsgivenAlarmand
Earthquake.
DepartmentofComputerScience—UniversityofSanFrancisco–p.12/23
19-11:
ConditionalIndependencein
Bayesiannetworks
. . .
. . .
U
1
X
U
m
Y
n
Z
nj
Y
1
Z1j
. . .
. . .
U
1
U
m
Yn
Z
nj
Y
1
Z
1j
X
DepartmentofComputerScience—UniversityofSanFrancisco–p.13/23
19-12:
InferenceinBayesiannetworks
Inmostcases,we’llwanttouseaBayesiannetworktotell
usposteriorprobabilities.
Weobserveavariable-howdoothervariableschange?
Wecandistinguishbetweenqueryvariables(thingswe
canobserve).
Therearealsohiddenvariables,whichinﬂuencequery
variablesbutarenotdirectlyobservable.
JohnCallsandMaryCallsareevidence,Earthquakeand
Burglaryarequeries,andAlarmishidden.
DepartmentofComputerScience—UniversityofSanFrancisco–p.14/23
19-13:
Enumeration
Recallthattheprobability

 

 

whereyarehidden
variables.
Thisisequivalenttosummingwithinthejointprobability
distribution.
Consider

 

 

 

 

 

 

 

Thisis

For
 

,wemust
calculate:

 

DepartmentofComputerScience—UniversityofSanFrancisco–p.15/23
19-14:
Enumeration
Problem-anetworkwith

nodeswillrequire

computations.
Wecansimplifybybringingthepriorsoutsidethe
summation.

 

Stillrequires

calculations
DepartmentofComputerScience—UniversityofSanFrancisco–p.16/23
19-15:
Enumeration
P(j|a)
.90
P(m|a)
.70.01
P(m|¬a)
.05
P( j|¬a)
P( j|a)
.90
P(m|a)
.70.01
P(m|¬a)
.05
P( j|¬a)
P(b)
.001
P(e)
.002
P(¬e)
.998
P(a|b,e)
.95
.06
P(¬a|b,¬e)
.05
P(¬a|b,e)
.94
P(a|b,¬e)
Thistreeshowsthe
computationsneededto
determine

Noticethattherearelots
ofrepeatedcomputations
inhere.
Byeliminatingrepeated
computations,wecan
speedthingsup.
DepartmentofComputerScience—UniversityofSanFrancisco–p.17/23
19-16:
VariableElimination
Wecanreducethenumberofcalculationsbycaching
resultsandreusingthem.
Evaluatetheexpressionfromright-to-left.
Summationsaredoneonlyfornodesinﬂuencedbythe
variablebeingsummed.
Consider

 



 



 







Theequationhasbeenseparatedintofactors.
Oncewecompute

and



,wecachethose
resultsinamatrix

.
Similarlyfor

and



storedin

DepartmentofComputerScience—UniversityofSanFrancisco–p.18/23
19-17:
VariableElimination

 

willresultina2x2x2matrix,storedin

.
Wethenneedtocomputethesumoverthedifferent
possiblevaluesof

Thissumis

 

 

 

Wecanprocess

thesameway.
Inessence,we’redoingdynamicprogramminghere,and
exploitingthesamememoizationprocess.
Complexity
Polytrees:(oneundirectedpathbetweenanytwo
nodes)-linearinthenumberofnodes.
Multiply-connectednodes:Exponential.
DepartmentofComputerScience—UniversityofSanFrancisco–p.19/23
19-18:
ScalingUp
ManytechniqueshavebeendevelopedtoallowBayesian
networkstoscaletohundredsofnodes.
Clustering
Nodesarejoinedtogethertomakethenetworkintoa
polytree.
CPTwithinthenodegrows,butnetworkstructureus
simpliﬁed.
Approximatinginference
MonteCarlosamplingisusedtoestimateconditional
probabilities.
DepartmentofComputerScience—UniversityofSanFrancisco–p.20/23
19-19:
MonteCarlosamplingexample
P(C)=.5
CP(R)
t
f
.80
.20
CP(S)
t
f
.10
.50
SR
tt
tf
ft
ff
.90
.90
.00
.99
Cloudy
Rain
Sprinkler
Wet
Grass
P(W)
Startatthetopofthenetworkand
selectarandomsample.(sayit’s

).
Drawarandomsamplefromits
children.conditionedon

.

 



 




=
<0.1,0.9>.Sayweselect



 

 
 



=<0.8,
0.2>.Sayweselect






=<0.9,0.1>.
Sayweselect

.
Thisgivesusasamplefor<cloudy,

Sprinkler,Rain,WetGrass>
Asweincreasethenumberof
samples,thisprovidesanestimateof


 









.
Inthisway,wechoosethequeryweare
interestedinandthensamplethenet-
work“enough”timestodeterminethe
probabilityofthateventoccurring,
DepartmentofComputerScience—UniversityofSanFrancisco–p.21/23
19-20:
OtherApproachesto
Uncertainty
Defaultreasoning
Logical
“Unlessweknowotherwise,AfollowsfromB”
Dempster-Shafer