Bayesian Networks - Computer Science - University of San Francisco

reverandrunΤεχνίτη Νοημοσύνη και Ρομποτική

7 Νοε 2013 (πριν από 3 χρόνια και 7 μήνες)

59 εμφανίσεις

ArtificialIntelligenceProgramming
BayesianNetworks
ChrisBrooks
DepartmentofComputerScience
UniversityofSanFrancisco
DepartmentofComputerScience—UniversityofSanFrancisco–p.1/23
19-0:
ProbabilisticInference
Previously,wetalkedaboutrule-basedsystemsthatcan
drawlogicalinferences.







































 

 





 
 









Wewouldliketoextendthissortofoperationtoworlds
withuncertainty.


 


























 
















 

















 











DepartmentofComputerScience—UniversityofSanFrancisco–p.2/23
19-1:
ProbabilisticInference
Ingeneral,wecanuseBayes’Ruletodothis.
Theproblemisinworkingwiththejointprobability
distribution.
Exponentiallylargetable.
Weneedadatastructurethatcapturesthefactthatmany
variablesdonotinfluenceeachother.
Forexample,thecolorofBart’shatdoesnotinfluence
whetherHomerishungry.
WecallthisstructureaBayesiannetwork(orabelief
network)
DepartmentofComputerScience—UniversityofSanFrancisco–p.3/23
19-2:
BayesianNetwork
ABayesiannetworkisadirectedgraphinwhicheach
nodeisannotatedwithprobabilityinformation.Anetwork
has:
Asetofrandomvariablesthatcorrespondtothenodes
inthenetwork.
Asetofdirectededgesorarrowsconnectingnodes.
Theserepresentinfluence.InthereisanarrowfromX
toY,thenXistheparentofY.
Eachnodekeepsaconditionalprobabilitydistribution
indicatingtheprobabilityofeachvalueitcantake,
conditionalonitsparentsvalues.
Nocycles.(itisadirectedacyclicgraph)
Thetopologyofthenetworkspecifieswhatvariables
directlyinfluenceothervariables.(conditional
independencerelationships).
DepartmentofComputerScience—UniversityofSanFrancisco–p.4/23
19-3:
Burglaryexample
B
T
T
F
F
E
T
F
T
F
P(A)
.95
.29
.001
.001
P(B)
.002
P(E)
Alarm
Earthquake
MaryCalls
JohnCalls
Burglary
A
P(J)
T
F
.90
.05
AP(M)
T
F
.70
.01
.94
Twoneighborswillcall
whentheyhearyour
alarm.
Johnsometimes
overreacts
Marysometimes
missesthealarm.
Twothingscansetoffthe
alarm
Earthquake
Burglary
Givenwhohascalled,
what’stheprobabilityofa
burglary?
DepartmentofComputerScience—UniversityofSanFrancisco–p.5/23
19-4:
Networkstructure
Eachnodehasaconditionalprobabilitytable.
Thisgivestheprobabilityofeachvalueofthatnode,given
itsparents’values.
Thesesumto1.
Nodeswithnoparentsjustcontainpriors.
DepartmentofComputerScience—UniversityofSanFrancisco–p.6/23
19-5:
Summarizinguncertainty
Noticethatwedon’tneedtohavenodesforallthe
reasonswhyMarymightnotcall.
Aprobabilisticapproachletsussummarizethis
informationin

Thisallowsasmallagenttodealwithlargeworldsthat
havealargenumberofpossiblyuncertainoutcomes.
Howwouldwehandlethiswithlogic?
DepartmentofComputerScience—UniversityofSanFrancisco–p.7/23
19-6:
Implicitlyrepresentingthefull
jointdistribution
Recallthatthefulljointdistributionallowsustocalculate
theprobabilityofanyvariable,givenalltheothers.
Independenteventscanbeseparatedintoseparate
tables.
ThesearetheCPTsseenintheBayesiannetwork.
Therefore,wecanusethisinfotoperformcomputations.

















 













 









 












 































 









DepartmentofComputerScience—UniversityofSanFrancisco–p.8/23
19-7:
ConstructingaBayesian
network
ThereareoftenseveralwaystoconstructaBayesian
network.
Theknowledgeengineerneedstodiscoverconditional
independencerelationships.
Parentsofanodeshouldbethosevariablesthatdirectly
influenceitsvalue.
JohnCallsisinfluencedbyEarthquake,butnotdirectly.
JohnandMarycallingdon’tinfluenceeachother.
Formally,webelievethat:





 




 


 





 



 





 





 









 










DepartmentofComputerScience—UniversityofSanFrancisco–p.9/23
19-8:
CompactnessandNode
Ordering
Bayesiannetworksallowforamorecompact
representationofthedomain
Redundantinformationisremoved.
Example:Saywehave30nodes,eachwith5parents.
EachCPTwillcontain

 


numbersTotal:960.
Joint:


entries,nearlyallredundant.
DepartmentofComputerScience—UniversityofSanFrancisco–p.10/23
19-9:
Buildinganetwork
Beginwithrootcauses
Thenaddthevariablestheydirectlyinfluence.
Thenaddtheirdirectchildren.
Thisiscalledacausalmodel
Reasoningintermsofcauseandeffect
Estimatesaremucheasiertocomeupwiththisway.
Wecouldtrytobuildfromeffecttocause,butthenetwork
wouldbemorecomplex,andtheCPTshardtoestimate.





 

DepartmentofComputerScience—UniversityofSanFrancisco–p.11/23
19-10:
ConditionalIndependencein
Bayesiannetworks
Recallthatconditionalindependencemeansthattwo
variablesareindependentofeachother,giventhe
observationofathirdvariable.







 

 









Anodeisconditionallyindependentofits
nondescendants,givenitsparents.
GivenAlarm,JohnCallsisindependentofBurglaryand
Earthquake.
Anodeisconditionallyindependentofallothernodes,
givenitsparents,children,andsiblings(thechildren’s
otherparents).
BurglaryisindependentofJohnCallsgivenAlarmand
Earthquake.
DepartmentofComputerScience—UniversityofSanFrancisco–p.12/23
19-11:
ConditionalIndependencein
Bayesiannetworks
. . .
. . .
U
1
X
U
m
Y
n
Z
nj
Y
1
Z1j
. . .
. . .
U
1
U
m
Yn
Z
nj
Y
1
Z
1j
X
DepartmentofComputerScience—UniversityofSanFrancisco–p.13/23
19-12:
InferenceinBayesiannetworks
Inmostcases,we’llwanttouseaBayesiannetworktotell
usposteriorprobabilities.
Weobserveavariable-howdoothervariableschange?
Wecandistinguishbetweenqueryvariables(thingswe
wanttoknowabout)andevidencevariables(thingswe
canobserve).
Therearealsohiddenvariables,whichinfluencequery
variablesbutarenotdirectlyobservable.
JohnCallsandMaryCallsareevidence,Earthquakeand
Burglaryarequeries,andAlarmishidden.
DepartmentofComputerScience—UniversityofSanFrancisco–p.14/23
19-13:
Enumeration
Recallthattheprobability





 






 









whereyarehidden
variables.
Thisisequivalenttosummingwithinthejointprobability
distribution.
Consider






 



 


 







 



 
















 







 




Thisis














For
 




,wemust
calculate:




















 
















DepartmentofComputerScience—UniversityofSanFrancisco–p.15/23
19-14:
Enumeration
Problem-anetworkwith

nodeswillrequire






computations.
Wecansimplifybybringingthepriorsoutsidethe
summation.












 
















Stillrequires





calculations
DepartmentofComputerScience—UniversityofSanFrancisco–p.16/23
19-15:
Enumeration
P(j|a)
.90
P(m|a)
.70.01
P(m|¬a)
.05
P( j|¬a)
P( j|a)
.90
P(m|a)
.70.01
P(m|¬a)
.05
P( j|¬a)
P(b)
.001
P(e)
.002
P(¬e)
.998
P(a|b,e)
.95
.06
P(¬a|b,¬e)
.05
P(¬a|b,e)
.94
P(a|b,¬e)
Thistreeshowsthe
computationsneededto
determine







Noticethattherearelots
ofrepeatedcomputations
inhere.
Byeliminatingrepeated
computations,wecan
speedthingsup.
DepartmentofComputerScience—UniversityofSanFrancisco–p.17/23
19-16:
VariableElimination
Wecanreducethenumberofcalculationsbycaching
resultsandreusingthem.
Evaluatetheexpressionfromright-to-left.
Summationsaredoneonlyfornodesinfluencedbythe
variablebeingsummed.
Consider



 















 







 





























Theequationhasbeenseparatedintofactors.
Oncewecompute





and





,wecachethose
resultsinamatrix


.
Similarlyfor






and






storedin


DepartmentofComputerScience—UniversityofSanFrancisco–p.18/23
19-17:
VariableElimination

 





willresultina2x2x2matrix,storedin


.
Wethenneedtocomputethesumoverthedifferent
possiblevaluesof

Thissumis



 







 



 

Wecanprocess

thesameway.
Inessence,we’redoingdynamicprogramminghere,and
exploitingthesamememoizationprocess.
Complexity
Polytrees:(oneundirectedpathbetweenanytwo
nodes)-linearinthenumberofnodes.
Multiply-connectednodes:Exponential.
DepartmentofComputerScience—UniversityofSanFrancisco–p.19/23
19-18:
ScalingUp
ManytechniqueshavebeendevelopedtoallowBayesian
networkstoscaletohundredsofnodes.
Clustering
Nodesarejoinedtogethertomakethenetworkintoa
polytree.
CPTwithinthenodegrows,butnetworkstructureus
simplified.
Approximatinginference
MonteCarlosamplingisusedtoestimateconditional
probabilities.
DepartmentofComputerScience—UniversityofSanFrancisco–p.20/23
19-19:
MonteCarlosamplingexample
P(C)=.5
CP(R)
t
f
.80
.20
CP(S)
t
f
.10
.50
SR
tt
tf
ft
ff
.90
.90
.00
.99
Cloudy
Rain
Sprinkler
Wet
Grass
P(W)
Startatthetopofthenetworkand
selectarandomsample.(sayit’s




).
Drawarandomsamplefromits
children.conditionedon




.

 







 









=
<0.1,0.9>.Sayweselect




 



 
 








=<0.8,
0.2>.Sayweselect





































=<0.9,0.1>.
Sayweselect




.
Thisgivesusasamplefor<cloudy,

Sprinkler,Rain,WetGrass>
Asweincreasethenumberof
samples,thisprovidesanestimateof


 

























.
Inthisway,wechoosethequeryweare
interestedinandthensamplethenet-
work“enough”timestodeterminethe
probabilityofthateventoccurring,
DepartmentofComputerScience—UniversityofSanFrancisco–p.21/23
19-20:
OtherApproachesto
Uncertainty
Defaultreasoning
Logical
“Unlessweknowotherwise,AfollowsfromB”
Dempster-Shafer
Incorporatesuncertaintyaboutprobabilities
Fuzzylogic
Allows“partiallytrue”statements.
DepartmentofComputerScience—UniversityofSanFrancisco–p.22/23
19-21:
ApplicationsofBayesian
Networks
Diagnosis(widelyusedinMicrosoft’sproducts)
Medicaldiagnosis
Spamfiltering
Expertsystemsapplications(plantcontrol,monitoring)
Roboticcontrol
DepartmentofComputerScience—UniversityofSanFrancisco–p.23/23