First Order Bayesian Networkx - cse655probreasoning

farmpaintlickInternet and Web Development

Oct 21, 2013 (3 years and 5 months ago)

65 views

Literature Review




Presented By:

Syeda Saleha Raza

Faculty of Computer Science

IBA, Karachi


Reasoning is ability to do inferences.



Automated reasoning is computing system that
helps in doing this.



Reasoning is one those several characteristics
that distinguish humans from machines.



AI progresses with the vision of having machines
either mimicking human behavior OR assisting
them in exhibiting those behaviors





Logic programs


Deterministic



Bayesian networks
-

Probabilistic


Ability to create multiple instances of same
node


Effective mechanism to specify frequency
distribution for a node having unbounded
number of parents


Ability to quantify (existentially & universally)
over unbounded and potentially infinite
number of parents for a given node


Ability to consider size of population that is
not part of domain but is known to exist


Davis (1990) defines


“Logic is a schema for defining languages to describe and reason about
entities in different domains of application.”


This is expressed in form of sequence of facts and rules.



Logic is propositional if it talks about particular instances of
entities and their relationships.

Person(John)

Person(Jim)

Person(Jack)

Father(Jim, John), Father(Jack, Jim)
-
>
GrandFather

(Jack, John)



Predicate Logic, also called First Order Logic, can reason about
general properties and relationships that apply to collection of
individual.

Person(X)

Person(X), Person(Y), Person(Y), Father(X,Y), Father(Z, X)
-
>
GrandFather

(Z, Y)





Systems based on FOL have the ability to
represent entities of different types
interacting with each other in varied ways



However systems based on FOL lack
theoretically principled, widely accepted,
logically coherent methodology for reasoning
under uncertainty


Bayesian Networks are probabilistic in nature but
represents one particular instance of problem.



First Order logic is generalization for all
instances but deterministic in nature



The strength of one is the weakness of other and
vice versa.



Integration of both these system can produce
systems that are generalized yet probabilistic in
nature.




Probabilistic Relational Model


Object Oriented Bayesian Network


Bayesian Program Logic


Probabilistic Ontology




PRM/PER has its roots in relational model



Perceives data in the form of relation ( tabular structure)



PER model attempted to integrate multiple instance
capability of relational model with probabilistic support of
Bayesian network



DAPER is an ER model with directed (solid) arcs among the
attribute classes that represent probabilistic dependencies
among corresponding attributes, and local distribution
classes that define local distributions for attributes.



DAPER, Plate and PRM are examples of relational
probabilistic models



Ref: Probabilistic Entity
-
Relationship Models, PRMs and Plate Models, David Heckerman,
Christopher Meek, and Daphne
Koller
, 2007


OOBN allows knowledge representation in the form of
classes.


Classes represent complex objects that in turn are
composed of other simple/complex objects.


Set of classes typically forms a is
-
a or part
-
of hierarchy
in the system.


Each class contains several properties that are
categorized as Input/Hidden/Output properties. Input
and Output properties define interface of class.


Each object of class is considered a stochastic function
that transforms particular values of Input attributes to
output attributes.



BLP has its roots in First order logic



Instead of having predicates, Bayesian logic
program comprises of Bayesian clauses.



Bayesian clause can have different states with
varying degree of belief.



Bayesian clause has an associated conditional
probability table and each Bayesian predicate has
an associated combining rule that maps finite set
of multiple probability distributions onto one
probability distribution.



[Ref: Bayesian logic programming: Theory & Tool
Kristian

Kersting

and Luc De
Raedt
]

Propositional Representation:


A(s1)


CT (c1)


Q (t1)


G (s1, c1)

A(s1)


C (c2)


Q (t2)


G (s1, c2)

A(s1)


C (c3)


Q (t3)


G (s1, c3)

G (s1,c1) . G (s1,c2). G (s1,c3)


P (s1)


Predicate
-
based Representation:


A(X)


C (Y)


Q (Z)


G (X,Y)


r

| r


RC, G(
X,c
)


P (X)



A(X)


C (Y)


Q (Z
)


G (X,Y)


r

| r


RC
, G(
X,c
)


P (X)


v
| v


B, P(v)



P (B)




[ A = Aptitude, CT = Type of Course, Q = Quality of Teaching, G = Grade, P = Performance,

RC=Registered Courses, B= Batch]






Ontology is a formal representation of a set of concepts
within a domain and the relationship between these
concepts. Its is used define particulars of a domain and to
reason about the properties of it.



Probabilistic ontologies expand the possibilities of
standard ontologies by introducing the requirement of a
proper representation of the statistical regularities and the
uncertain evidence about entities in a domain of
application and also allowing for reasoning upon what now
can be represented via probabilistic ontologies.



Multi
-
Entity Bayesian Network (MEBN) has been used a
framework for defining probabilistic ontologies and
making use of them in inferences.



MEBN is not a computer language such as Java or
C++, or an application such as
Netica

or
Hugin
.
Rather, it is formal system that combines
expressive power of FOL with logically consistent
treatment of uncertainty.



MEBN provides syntax, a set of model
construction and inference processes, and
semantics that together provide a means of
defining probability distributions over
unbounded and possibly infinite numbers of
interrelated hypotheses



MEBN is to Bayesian networks as algebra is to arithmetic”



Each MEBN model, called
MTheory
, represents a particular domain of discourse.



Different
Subjects

of that domain are represented by smaller components known
as
MFrag
.
MFrag

provides grouping of entities and their relationships pertinent to
that particular subject.



Each node in
MFrag

is parameterized hence providing support for the multiple
instances of it. An
MFrag

node can be of any of three types:



Context Node

that is evaluated to either true/false when substituted with constant
values in place of parameters.



Resident Nodes

are local nodes of
MFrag

and form the real core of it. There can be
multiple resident nodes in an
MFrag

and each resident node defines its own
probability distribution. Semantically,
MFrag

is a representation of group of its
resident nodes. Resident nodes can in turn be dependent on the other instance of
their own hence providing support for recursive
MFrags
.



Input Nodes

serve as input to derive probability distribution of resident nodes.
Input nodes are at the interface of local
MFrag

and are resident nodes of some
other
MFrag

where their own probability distribution is defined. Hence Input
nodes provide mechanism to connect multiple
MFrags
.


Ref: Costa, P. C. G. 2005. Bayesian Semantics for the Semantic Web. PhD Diss. Department of Systems
Engineering and Operations Research, George Mason University. 315p, July 2005, Fairfax, VA, USA




Ref: Costa, P. C. G. 2005. Bayesian Semantics for the Semantic Web. PhD Diss. Department of Systems
Engineering and Operations Research, George Mason University. 315p, July 2005, Fairfax, VA, USA




This
table shows Danger
To Self
MFrag

Probability
Distribution



Relevant Starships Nearby Danger Level Dist.

At least 1
Cardassian

[0.925, 0.024, 0.006, 0]

At least 2
Cardassians

[0.99, 0.008, 0.002, 0]

At least 3
Cardassians

[0.975, 0.2, 0.05, 0]

More than 4
Cardassians

[1, 0, 0, 0]

No
Cardassians

but at least 1
Romulan

[.73, .162, .081,
.027]

No
Cardassians

but at least 1
Romulans

[.76, .144, .072,
.024]

… … (see formula)

No
Cardassians

but 10 or more
Romulans

[1, 0, 0, 0]

No
Cardassians

or
Romulans
, one Unknown [.02, .48, .48,
.02]

… … (see formula)

No
Cardassians

or
Romulans
, 10+ Unknown [.20, .30, .30,
.20]

… …(see formula)

Ref: Costa, P. C. G. 2005. Bayesian Semantics for the Semantic Web. PhD Diss. Department of
Systems Engineering and Operations Research, George Mason University. 315p, July 2005,
Fairfax, VA, USA




Finding:


In our example, the finding
MFrags

will convey
information that we have five starships (!
ST0 through
!
ST4) and that the first is our own starship. For the
sake of illustration, let’s assume that our Finding set
also includes data regarding the nature of the space
zone we are in (!
Z0), its magnetic disturbance for the
first time step (!
T0), and sensor reports for starships
!
SR1 to !SR4 for the first two time steps.

Targets:


We assume that the Target set for our illustrative
query includes an assessment of the level of danger
experienced by the
Enterprise and the best decision
to
take given this level of danger.

Ref: Costa, P. C. G. 2005. Bayesian Semantics for the Semantic Web. PhD Diss. Department of
Systems Engineering and Operations Research, George Mason University. 315p, July 2005,
Fairfax, VA, USA




Full integration of first
-
order logic with
Bayesian Network can enable us to:


Provide a true representation of domain of
discourse that can dynamically generate multiple
instances depending upon the situation in hand.



Capture statistical regularities of that domain



Make inferences or diagnose causes given certain
evidences




[1] Bayesian logic programming: Theory & Tool,
Kristian

Kersting

and Luc De
Raedt
.


[2] A Dynamic Approach to Probabilistic Inference using Bayesian Networks, Michael C.
Horsch

and David
Poole, Department of Computer Science, University of British Columbia, Canada


[3] First
-
order probabilistic inference, David Poole in Proceedings IJCAI 2003. Acapulco, Mexico, August
2003, pages 985
-
991.


[4] Probabilistic Entity
-
Relationship Models, PRMs and Plate Models, David Heckerman, Christopher Meek,
and Daphne
Koller
, 2007


[5] Bayesian networks and influence diagrams, A guide to Construction and Analysis,
Uffe

B.
Kjærulff


Anders L. Madsen


[6]
Koller
, D., &
Pfeffer
, A. (1997). Object
-
Oriented Bayesian Networks. Paper presented at the Thirteenth
Conference on Uncertainty in Artificial Intelligence (UAI
-
97). San Francisco, CA, USA.



[7]
Laskey
, K.B., MEBN: A Language for First
-
Order Bayesian Knowledge Bases, Artificial Intelligence,
172(2
-
3), 2007.



[8] Costa, P. C. G. 2005. Bayesian Semantics for the Semantic Web. PhD Diss. Department of Systems
Engineering and Operations Research, George Mason University. 315p, July 2005, Fairfax, VA, USA