M M M

huntcopywriterAI and Robotics

Oct 24, 2013 (3 years and 9 months ago)

78 views

Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
Three compact closed categories for natural
language processing
Anne Preller
LIRMM,Montpellier,France
Oxford,2010
The categorical flow of information in quantum physics and
linguistics
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
Two semantics
Text generates two kinds of semantics
based on functional logic (sentence logic)
set of individuals
E
and truth
based on quantum logic and probability (concept logic)
set of concepts
P
and truth
they cooperate in the semantics of
pregroup grammars (Lambek)
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
common mathematical structure:compact 2-categories
syntax:free compact 2-category
semantics:symmetric compact 2-category
= compact closed category
1-cell = object,2-cell = morphism
syntax
2-sorted
functions
semi-modules
Hilbert spaces
1-cell type set space
composition of 1-cells concatenation
× ⊗
I
,
u n i t o f 1 - c e l l s e m p t y s t r i n g
{ ∗ }
[0
,
1]

,
field
C
2-cell cut-free proof 2-function linear map
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
three compact closed categories with biproducts
semantics adds two distinguished 1-cells
for the logic
0 zero-object
,
I
￿
I
I
=
{∗}
2
F
,category of two-sorted functions
0 =

,
I
￿
I
=
S
=
{ ￿
,
⊥}
,(truth)
I
= [0
,
1 ]
M
[ 0
,
1]

,category of semi-modules over [0
,
1]

R
0-space,
I
￿
I
=
V
{
p
￿
,
p

}
(concept)
I
=
C
H
C
,category of complex Hilbert spaces
0-space,
I
￿
I
=
H
{
q

,
q

}
(qubit )
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
meaning of sentences in functional semantics
M(
Some boys like books
)
like
(
some
(
boy
)
,
book
) =
r e d u c t i o n

(
s o m e
×
b o y
×
l i k e
×
b o o k
)
￿
￿
some
￿
￿
￿
￿
S
￿
￿
I
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
I
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
like
boy
book
(
E
×
E

)
×
(
E
)
×
(
E

×
S
×
E

)
×
(
E
)
some
×
boy
×
like
×
book
reduction
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
2-sorted functions,definition
2FOL two-sorted first order logic
two sorts:elements and sets
a
,
A
,
a

A
,
a
=
b
,
A
=
B
,

x
,

x
,

X
,

X
function
g
:
A

B
is
two-sorted
if
x

A
=

g
(
x
)

B
or
g
(
x
)

B
X

A
=

g
(
X
)

B
or
g
(
X
)

B
g
(

) =

g
(
{
a
}
) =
g
(
a
) for
a

A
g
(
X

Y
) =
g
(
X
)

g
(
Y
) for
X
,
Y

A
O x f o r d 2 0 1 0,F l o w i n ’ c a t p r e l l e r @ l i r m m.f r
C o m p a c t c l o s e d c a t e g o r i e s a n d N L P
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
2-sorted functions,closure conditions
sets are closed under product and union
f
:
A

B
,
f

:
B

A
,
￿
f
￿
:
I

A
×
B
,
￿
f
￿
:
A
×
B

I
f

(
b
) =
{
a
:
b
=
f
(
a
) or
b

f
(
a
)
}
￿
f
￿
(

) =
{
<
a
,
b
>
:
b
=
f
(
a
) or
b

f
(
a
)
}
￿
f
￿
(
a
,
b
) =
￿

if
b
=
f
(
a
) or
b

f
(
a
)

else
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
predicates
Definiton:
A two-sorted function
f
:
A

S
=
{￿
,
⊥}
is a
predicate
if it maps elements to elements
c
=

Fundamental property
For every predicate
g
:
A

S
,
b

S
,
X

A
g
(
X
) =
b

X
￿
=

and
g
(
x
) =
b
,
for all
x

X
c
=

For non-empty set
X
g
(
X
) =
￿
or
g
(
X
) =

or
g
(
X
) =
{ ￿
,
⊥}
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
semi-modules over [0
,
1]

I
= [0
,
1 ] i s a n i m p l i c a t i o n c o m p l e m e n t e d,d i s t r i b u t i v e l a t t i c e
with 0
,
1 and negation.In particular,it is a semi-ring
V
semi-module over
I
= [0
,
1 ]

t h e o r d e r o n
I
d e fi n e s a p a r t i a l o r d e r o n t h e v e c t o r s o f
V
by
comparing them coordinate by coordinate
vectors of
V
form a distributive lattice with
−→
0,
−→
1andconnectives

,

,

,
¬
defined coordinatewise
a vector and its negation are
orthogonal
a normal vector
v
t h a t g e n e r a t e s a
1 - d i m e n s i o n a l
subspace
I
v
=
{
α
v
:
α

I
}
is a sum of basis vectors
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
semi-modules over [0
,
1]

I
= [0
,
1 ] i s a n i m p l i c a t i o n c o m p l e m e n t e d,d i s t r i b u t i v e l a t t i c e
with 0
,
1 and negation.In particular,it is a semi-ring
V
semi-module over
I
= [0
,
1 ]

t h e o r d e r o n
I
d e fi n e s a p a r t i a l o r d e r o n t h e v e c t o r s o f
V
by
comparing them coordinate by coordinate
vectors of
V
form a distributive lattice with
−→
0,
−→
1andconnectives

,

,

,
¬
defined coordinatewise
a vector and its negation are
orthogonal
a normal vector
v
t h a t g e n e r a t e s a
1 - d i m e n s i o n a l
subspace
I
v
=
{
α
v
:
α

I
}
is a sum of basis vectors
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
lattice of
vectors

lattice of
subspaces
logical connectives given by
quantum logic
on
subspaces
¬
E
=
E

,
E

F
=
E

F
,
E

F
=
E

F
,...
vectors
internalise
closed subspaces of
V
E
v
=
{
w
:
w

v
}
v
￿→
E
v
is an isomorphic embedding of
the lattice of vectors into the lattice of subspaces
E
¬
u
=
E

u
,
E
u
+
v
=
E
u

E
v
,
E
u

v
=
E
u

E
v
I
v
￿→
E
v
is an isomorphism for normal vectors
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
geometry of [0,1]-spaces
E
¬
u
=
E

u
,
E
u
+
v
=
E
u

E
v
,
E
u

v
=
E
u

E
v
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
u
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
v
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
u
+
v
￿∨⊥
=
−→
1

=(0
,
1)
￿
=(1
,
0)
−→
0
distinctbasisvectorsareorthogonal,theirconjunctionis
−→
0
moreover,in2-dimensionalspace:
¬
(1
,
0) = (0
,
1)
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
V
A
finite dimensional semi-module over the semiring [0
,
1]

generated
by the basis set
A
H
A
finite dimensional Hilbert space over the field of complex numbers
generated by the basis set
A
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
embeddings
subsets of
B
identify with vectors in
V
B
J
B
(
{
b
i
1
,...
b
i
m
}
) =
￿
m
k
=1
b
i
k

V
B
J
B
(
b
) =
b
D e fi n i t i o n o f f u n c t o r
J
:2
F →M
[ 0
,
1]

A
,
B
sets,
g
:
A

B
two-sorted function
J
(
A
) =
V
A
J
(
g
)(
a
) =
J
B
(
g
(
a
))
,
a

A
Definition of functor
E
:
M
[0
,
1]

→H
C
V
A
,
f
:
V
A

V
B
∈ M
[ 0
,
1]

E
(
V
A
) =
H
A
E
(
f
)(
a
) =
f
(
a
)
,
a

A
O x f o r d 2 0 1 0,F l o w i n ’ c a t p r e l l e r @ l i r mm.f r
C o mp a c t c l o s e d c a t e g o r i e s a n d N L P
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
embeddings
subsets of
B
identify with vectors in
V
B
J
B
(
{
b
i
1
,...
b
i
m
}
) =
￿
m
k
=1
b
i
k

V
B
J
B
(
b
) =
b
D e fi n i t i o n o f f u n c t o r
J
:2
F →M
[ 0
,
1]

A
,
B
sets,
g
:
A

B
two-sorted function
J
(
A
) =
V
A
J
(
g
)(
a
) =
J
B
(
g
(
a
))
,
a

A
Definition of functor
E
:
M
[0
,
1]

→H
C
V
A
,
f
:
V
A

V
B
∈ M
[ 0
,
1]

E
(
V
A
) =
H
A
E
(
f
)(
a
) =
f
(
a
)
,
a

A
O x f o r d 2 0 1 0,F l o w i n ’ c a t p r e l l e r @ l i r mm.f r
C o mp a c t c l o s e d c a t e g o r i e s a n d N L P
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
properties of
J
,
E
J
,
E
preserve compact closed structure and dimension
J
,
E ◦ J
t r a n s f o r m t h e B o o l e a n s e t - o p e r a t o r s i n t o l o g i c a l c o n n e c t i v e s
J
B
i s a n i s o m o r p h i s m f r o m t h e a l g e b r a o f s u b s e t s o f
B
o n t o t h e o n e - d i m e n s i o n a l s u b s p a c e s g e n e r a t e d b y n o r m a l v e c t o r s
In particular,
n
￿
i
=1
{
p
i
￿
,
p
i

} ￿→
V
{
p
1
￿
,
p
1

}

...

V
{
p
n
￿
,
p
n

}
H
{
p
1
￿
,
p
1

}

...

H
{
p
n
￿
,
p
n

}
{ ￿
,
⊥} ⊆
S
￿→
−→
1

V
{￿
,
⊥}
￿→
−→
1

H
{￿
,
⊥}
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
quantum logic and concept space
P
=
{
p
1
,...,
p
n
}
(
words
)
qubit,state spin up
p
i

p
i
￿
basic concept,present
qubit,state spin down
p
i

p
i

basic concept,absent
H
{
p
i

,
p
i

}
=
C
(
p
i
) =
V
{
p
i
￿
,
p
i

}
q u a n t u m/c o n c e p t s p a c e
C
(
P
) =
C
(
p
1
)

...

C
(
p
n
)
state
concept
where
state = 1-dimensional subspace in quantum space
concept = Boolean combination of basic concepts
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
concept space
’words’ live on in tensor products as concepts and states
−→
p
i
=
−→
1

...

−→
1

p
i
￿

−→
1

...

−→
1
¬
−→
p
i
=
−→
1

...

−→
1

p
i


−→
1

...

−→
1
=

−→
p
i
=
￿
q
,
q
(
i
)=
p
i
￿
b
q
¬
−→
p
i
=
￿
q
,
q
(
i
)=
p
i

b
q
basis vector
b
q
=
q
(1)

...

q
(
n
) i s g i v e n by t he
c ho i c e
q
(
i
)
∈ {
p
i
￿
,
p
i

}
,
1

i

n
-
Hence,
−→
p
i
,
¬
−→
p
i
are states
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
concept logic in
C
(
P
)
Theorem
The following are isomorphic
-
the free Boolean algebra generated by
P
=
{
p
1
,...,
p
n
}
.
-
t he l a t t i c e o f C
(
P
)
g e ne r a t e d by
￿
−→
p
1
,
¬
−→
p
1
,...,
−→
p
n
,
¬
−→
p
n
￿
-
thelatticeofsubspacesgeneratedbysetsofbasisvectors
c
=

C
(
P
)includespropositionallogic
Oxford2010,Flowin’cat preller@lirmm.fr
CompactclosedcategoriesandNLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
trace of sentence logic in
C
(
P
)
g
predicate
define a
state
in
C
(
P
),namely the vector
g
(1)

...

g
(
n
)
where
g
(
i
) =





p
i
￿
,
if
g
(
p
i
) =
￿
p
i

,
if
g
(
p
i
) =

−→
1
,
if
g
(
p
i
) =
{ ￿
,
⊥}

C
(
p
i
)
example:
g
=
like
,
p
1
,...,
p
n
= e nume r a t i o n o f pa i r s o f c o unt - no uns
Ox f or d 2010,Fl owi n’ c at pr el l er @l i r mm.f r
Compac t c l os ed c at egor i es and NLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
geometry of concept spaces
C
(
P
) =
p
1

p
2

p
3

...
p
1
=
hate
,
p
2
=
love
,
p
3
=
like
,...
−→
0
￿
￿
love
￿

hate

￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
love
￿

hate
￿
￿
￿
love


hate
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
−→
love
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
￿
−→
love

−→
hate
−→
hate
−−→
hate
￿⊥
−−→
love
,
¬
−−→
hate
￿
=
−−→
love
,
−−→
hate

−−→
love
=
hate
￿

love
￿
￿
= 0
O x f o r d 2 0 1 0,F l o w i n ’ c a t p r e l l e r @ l i r mm.f r
C o mp a c t c l o s e d c a t e g o r i e s a n d N L P
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
similarity class
experimental data from the BNC =

opposites are similar
probability distribution
φ
on finite strings of words
s
,
s
￿
similar words,
x
=
p
1
...
p
i

1
,
y
=
p
i
+1
...
p
n
s t r i ng s o f wor ds
φ
(
xsy
) =
φ
(
xs
￿
y
)
the distribution is defined on the similarity class
c
of
s
,
s
￿
,...
φ
(
xcy
):=
φ
(
xsy
)
‘interpersonal emotions’= similarity class
c
c
:=
−−→
hate

−−→
love

−−→
like
∨ ∙ ∙ ∙ ∈
C
(
P
)
¬
c
=
¬
−−→
hate
∧ ¬
−−→
love
∧ ¬
−−→
like
∧ ∙ ∙ ∙ ∈
C
(
P
)
Ox f o r d 2 0 1 0,F l o wi n ’ c a t p r e l l e r @l i r mm.f r
Co mp a c t c l o s e d c a t e g o r i e s a n d NL P
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
similarity class
experimental data from the BNC =

opposites are similar
probability distribution
φ
on finite strings of words
s
,
s
￿
similar words,
x
=
p
1
...
p
i

1
,
y
=
p
i
+1
...
p
n
s t r i ng s o f wor ds
φ
(
xsy
) =
φ
(
xs
￿
y
)
the distribution is defined on the similarity class
c
of
s
,
s
￿
,...
φ
(
xcy
):=
φ
(
xsy
)
‘interpersonal emotions’= similarity class
c
c
:=
−−→
hate

−−→
love

−−→
like
∨ ∙ ∙ ∙ ∈
C
(
P
)
¬
c
=
¬
−−→
hate
∧ ¬
−−→
love
∧ ¬
−−→
like
∧ ∙ ∙ ∙ ∈
C
(
P
)
Ox f o r d 2 0 1 0,F l o wi n ’ c a t p r e l l e r @l i r mm.f r
Co mp a c t c l o s e d c a t e g o r i e s a n d NL P
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
similarity space
H
{
c
￿
,
c

}
but my feelings can go through
hate
,
dislike
,...,
like
,
love
c
c
￿
=
c
s
= (cos
α
+sin
α
i
)
c
￿
c

=
¬
c
c
c
c
Ox f o r d 2 0 1 0,F l o wi n ’ c a t p r e l l e r @l i r mm.f r
Co mp a c t c l o s e d c a t e g o r i e s a n d NL P
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
similarity space
H
{
c
1
￿
,
c
1

}

...

H
{
c
m
￿
,
c
m

}
View
text as an experiment
context as self-adjoint linear map
measure the probability that after the experiment
E
with the right words
(
−−→
eva
⊗¬
−−→
like

−−→
pete
) =
−−→
eva

−−→
love

−−→
pete
thesystemgoesfromonestatetotheother
Oxford2010,Flowin’cat preller@lirmm.fr
Compact closedcategoriesandNLP
Outline
Compact 2-categories
Sentence semantics
Concept semantics
Interaction of both semantics
conclusion
THANK YOU!
Joachim Lambek (1993,1999),Compact bilinear logic,
Pregroups
C.van Rijsbergen (2004),Quantum logic in Information
Retrieval
Samson Abramsky,Bob Coecke (2004),Categorical semantics
of quantum protocols
Anne Preller (2005,2007),Categorical semantics for pregroup
grammars,functional semantics for pregroup grammars
Daoud Clarke (2007),Context-theoretic Semantics for Natural
Language
Stephen Clark,Bob Coecke,Mehrnoosh Sadrzadeh (2008),
Vector semantics for pregroup grammars
Oxford 2010,Flowin’cat preller@lirmm.fr
Compact closed categories and NLP