1 Wide Coverage Natural Language Processing using Kernel ...

huntcopywriterΤεχνίτη Νοημοσύνη και Ρομποτική

24 Οκτ 2013 (πριν από 3 χρόνια και 9 μήνες)

93 εμφανίσεις


                                                         
                              
             
a
 
           
a
 
   

      
a

           

    
b
a

                                   
 
          

     

            
  
  

     
 
   
                                       

      
                  

         
 
  
b

                         
 
                     

          

       
    
  
                        

     
 
                                                                                             
                                                                                               
                                                                                                 
                                                                                       
                                                             
                                                                                      
             
                                     
                      

                   
                                 

     
                           

     

      
                                          
                   

                  
            

                              
                                          
                                         

                                         
                
                                       
                                       
                                         
         
 
       
 
                 
                              
 
         

        

           

          

      
  

   

                             
     
 
                                    
             

                           



                                   
                                     
                           

    

    
                                        
             

                                        
                                         

                                   
                                        
                               

         
          

                            
     

                                   
                  

     

            
                    

    

                
                                      
        

                           
                                       
                                        
                                        
          

                              
                                        
            

                           
           

                           
                                         
                                       
                                      

 

           


                           

                                         

                                         
                                       
                                        
    

                      
             

                
                          

   

    
                                         
                                  
Φ
    
  

                                
                                        
                                       
              
Φ
                      
                                        
                   

                  
                                        
            

         

                       
                                           
                                           
                                        
                         

                
                                  
 



  
                                         
                                        
                                    
                                        
                                         
                                      
                                       
                                         
                                         
                                      
                                        

          

                              
                  
         
  
x
                           
X

y
                      
Y
             
X

D = {(x
i
,y
i
)}
m
i=1
                        
   
X ×Y
  
m
              

        
      

  
D = {(x
i1
,y
i1
),...,(x
ik
i
,y
ik
i
)}
m
i=1
         

    
(x
i1
,...,x
ik
i
)
    
i
     

                        

x
ij
∈ X

y
ij



          
x
ij
             

x
ij
       
x
ik
       
x
ij
￿ x
ik
)
 
y
ij
< y
ik


      
            

  
D = {(x
i1
,...,x
ik
i
),y
i
}
m
i=1
         

    
y
i


                
       
                        
G

  
V
    
            
v
               
[v]
    
                                          
     
v

 
i
[v]
  
i
         
v
  
c
v
  
                 
v
   
I
       
N
     
                                  
I(v)
  
                    
v

                   
         

                         
                                         
                                       
                                        
                                        
                      

  
φ:X ￿→ H
 
                       
X
          
        

          
H


          

  
           
H = {τ
i
}
n
i=1
               
             

                        
                                        
        

               
x
            
   
n
                 
φ(x) = {φ
i
(x)}
n
i=1
                  
i
              
φ
i
(x)
                                
i
      
       
τ
i
 
x


                         
                                         
                                      
                                     
                                       
K(x
1
,x
2
) = φ(x
1
)
T
φ(x
2
)
               
                  

                       
                      
K(x
1
,x
2
) =
￿
v
1
∈V
1
￿
v
2
∈V
2
T(v
1
,v
2
)
  
    
T(v
1
,v
2
)
                      
              
x
1
  
x
2
              
   
v
1
  
v
2

T(v
1
,v
2
)
                  
         
T(v
1
,v
2
) = 0
                 
v
1
  
v
2
          

     
v
1
  
v
2
      
         
T(v
1
,v
2
) = 1
        
T(v
1
,v
2
) =
cv
1
￿
i=1
(1 +T(
 
i
[v
1
],
 
i
[v
2
]))
  
1
                                        

                                    
                                     
         

                         


                                        
                                        
                                       
                                          
                     

               
                                        
                    

                
                                       
                  

                     
                
K
λ
(x
1
,x
2
) =
n
￿
i=1
λ
size
i
φ
i
(x
1

i
(x
2
)
  
    
size
i
                            
i
                 
0 < λ ≤ 1

                  

                                      
                                     
                                     
                                       

                                            
                               

       
       

                               
                               

      
                                          

            
10
8
                    


                    


                                     
                                         






                               
                                       
        
          

                 
                                   

 
                     




             
            

                      
                        

               
                          
D = {(x
i
,y
i
)}
m
i=1

    
x
i
∈ X
  
y
i
∈ {+1,−1}
      
   
k



                        
x
i
  
                           
w
k


       
          
ˆy
i
                         
y
i

            
w
k
        

x
i
      
      
L
                                
                     

            
c
k
 
                                   
w
k
    
        

            
E
              
    
W = {(w
k
,c
k
)}
E
k=1
 



         

    
        

    
L = {(x
q
k
,c
k
)}
E
k=1
            
                                         
                                       
                    
ˆy =
   
[f(x)]
    
f(x) =
E
￿
k=1
c
k
Γ[w
k
T
φ(x)]
=
E
￿
k=1
c
k
Γ[
k
￿
j=1
y
q
j
K(x
q
j
,x)]
  
  
Γ
                               
 
                        

            
      
O(dm
2
u)

    
d
            
        
u
                          
 

 

        



                      
    
                        
                                     
                                      

                                        
              

                        
                  

             
                                     
                                      
                                         
                                    

 
                               

 
   
 

   


  

                       

    
                                      
                                       
         
v

                     
                                    
                 
v


                 
                      

            

  
           

                       
t
   
   
ϕ(v) = t(ϕ(
 
[v]),I(v))
    
ϕ(v) ∈


n
                                      
v
  
ϕ(
 
[v]) ∈


c
v
∙n
                      
                                       
             
c
         
v
             
       
t:


c
v
∙n
×I ￿→


n
           
v
 
                    
v
                   
v
              
ϕ
0
= 0
                  

                                      
                               
t
        
           
a
j
(v) = ω
j0
+
N
￿
h=1
ω
jh
z
h
(I(v))+
+
c
￿
k=1
n
￿
￿=1
w
jk￿
ϕ
￿
(
 
k
[v])
ϕ
j
(v) = tanh(a
j
(v)) h = 1,...,n



    
ϕ
j
(v)
         
j
                
                  
v

z
h
(I(v))
            
                  
v
  
ω
jh
,w
jk￿
     
               

                     

  
          
Φ
ω
(x) = ϕ(r)
  
        
r
                           
                            

           
                
f(x)
                  
           
f(x) = o
T
Φ
ω
(x)
    
o
     
                         

            
       

                                
 
E(D) =
m
￿
i=1
(f(x
i
) −y
i
)
2
  
                                       
                                      
                                           
                                         
                                  

                             
                                   
                                  

     

                                        
                                        
                    

                     

                     

                  
                                        
                   

                              

      
                   
f:X ￿→ Y
        
                                         
                                    
Y
                                 
Y
    
               

                        
        
Y
             

   

           
      

                               
         

                               
                                     
                                           
               
Y
                    
Y
                   

                     
   

                                 
       

                             

  
                                          

                                      
                                  

      
                                         
                                         
                        

                

               
f
RANK
:X

￿→ P

         

          
                             

  
X

    
         

                  
X
  
P

=

k
P
k
    
P
k
                            
k
        

       
f
RANK
(x
i1
,...,x
ik
i
) =

i1
,...,π
ik
i
)
         
π
ij
           
                 
x
ij
                  
                                        
                                     
       

                             
       

                               
                    

    

        
π
i
=
  
min
j=1,...,k
i

ij
}


               
              

                
L
r
(f
RANK
(x
i1
,...,x
ik
i
),y
i
) =
=
￿
1
 
π
y
i
> r
0
 
π
y
i
≤ r



    
r = 1,...,k
i
                        
                          

             
       

          

L
1
                
   

                                
                   

                   
                                       
                
f
RANK

                             
                                   
                           
U:X ￿→

  
        
x,z ∈ X
       
x ￿ z

   

U(x) > U(z)
 
       

             
                                      
                  

       

        
                                 
U
   
     
x
ij
 
π
ij
=
￿
k
i
r=1
θ(U(x
ir
) − U(x
ij
))
    
θ:


￿→ {1,0}
    

            
           
θ(x) = 1
 
x ≥ 0
 
     
    

                                    
     

                                  
π
i
= arg max
j=1,...,k
i
U(x
ij
)

                                 
        
          
U(x)
                   
                                       
             
Y
i
                        
                        

     

     
                      
Y
i
= j

       
x
ij
 
                                        
                          
ω

          
                                
P(Y
i
= j|x
i1
,...,x
iki
,ω) =
e
U(x
ij
)
￿
k
i
r=1
e
U(x
ir
)
  

            

                      
￿
m
i=1
P(Y
i
= y
i
|x
i1
,...,x
ik
i
,ω)
            
                                         
E
PREF
(D) = −
m
￿
i=1
log
e
U(x
iy
i
)
￿
k
i
￿=1
e
U(x
i￿
)
=
m
￿
i=1
log
k
i
￿
￿=1
e
U(x
i￿
)−U(x
iy
i
)




                              
ω
      
                                         
                                       
                                
E
PREF
(D)
            
U(x
i￿
)

                        

                     
 
          
        
 
                            
                      
U
               
      
w
       
U(x) = w
T
x


     
                 

w
                    
             
w
T
(x
ij
−x
ik
) > 0
i = 1,...,m
  
j,k = 1,...,k
i
:y
ij
< y
ik
  

             

                       
w
T
(x
iy
i
−x
ij
) > 0
i = 1,...,m
  
j = 1,...,k
i
,j ￿= y
i
 
 

       

                              
                                         
                         

          

 
             
f
PAIR
:X ×X ￿→{−1,+1}
   
           

             
L
￿
(f
PAIR
(x
ij
,x
i￿
),z
ij￿
) =
=
1 −f
PAIR
(x
ij
,x
i￿
)z
ij￿
2


 
    
z
ij￿
= 1
 
x
ij
￿ x
i￿
  
z
ij￿
= −1
     
    

                                     
               
E
PAIR
(D) =
=
m
￿
i=1
k
i
￿
j=1,j￿=y
i
L
￿
(f
PAIR
(x
iy
i
,x
ij
),z
iy
i
j
)

 

                                           
    
E
PREF
(D) =
=
m
￿
i=1
L
1
(f
RANK
(x
i1
,...,x
ik
i
),y
i
)
 


           

                          
                                          
                                   
                                     
k
i
    

                              
                                 
w
T
[φ(x
ij
) −φ(x
ik
)] =
=
￿
t
α
s
t
p
t
q
t
[K(x
s
t
p
t
,x
ij
) −K(x
s
t
p
t
,x
ik
)+
−K(x
s
t
q
t
,x
ij
) +K(x
s
t
q
t
,x
ik
)]
                       
α
s
t
p
t
q
t
         
s
t

p
t
  
q
t


                             
             

Γ
                        
                       
2
                                               


       

                        

            
      

                              
                                      
                                        
                                      
                                        
               
                         
                                        
                                       
                                        
                                        
      

        

                  

                    
who
                   
   
                                          
                                       
            
 
               
           
                                       
                                          
                                      
                                 

    
                         

            

                                        
                                          
                       

                
                                        
                      

                   
        

                                  

                                        
                                        
              

                      
                                    
                        
T
k
               
k
                                      
                                         
      

    

                         
                  
T
k+1
               
  

                                 






                         
T
k
    
                                       

                    

             

             

                         
                              

          
 

      

       

       

                
   
T
k


                            

 
The ofservant the actress who
ND P D N WH
WHNP
NP
PP
NP
NP
S'
alternative
anchors
foot
incremental tree connection path

    



                                
            
                           
    

                               

       

       

                     
       

           
                                    
              
T
k
         

          
                          



    
                                            
                                     
                                           
          

                           
                                        
                    
                                   
                                   
                     

                   
                                        
             
 
                          
                                         
                                           
                                        

 
                              

       
                                       
                                        
     
                            

         
                                         
                                   
                                       
                                  

    

                                          
                                         
                   
                   
            

                      



                                             

                                          

                                         
                         

        

   
                                         
                                         
                                         
                                          
                                         
                                      
                         

               
 

                                    
                                  

       
                                     
                                     
                                     
                                      
     

                                
                                        
                                      
                                        
                           

         
                                            
                                          
                  



                  
                                        
                                       


                                    
    

                          
                                   
                              

    

F
1
                                     
                                         
                                    
                      

              

                 

       

         

               


                                           
                 
   
                 
                                     
                                         
                                        
                         

               
                         
                        

                                   
   

                                
                                         
    
 
 
 

          

                 
                                         
                                      
    

    
 
          

       
 
      
    

         

      

           
 
     
    

      

                                      
           
100

500

2,000

10,000
  
40,000
        

                            
           

                
40,000
   
                               
24
     
               
120
                     
                                  
2
           
2,000
            

        
           
10
8
     

                                     
              

                        
                            

   

  
                        

     
  
      
                                        
λ
 


                          
λ = 0.5


                                   
25
                                     
[−0.01,+0.01]
                             
             
                        
     



                           
                           

             
     

          

                      
                                         
                                         
        

                               
                                         
  

         

   
15
       

   
0.3%
   
                 

                                   

                        

         
                                       
        

       

                  
      
1,000
                             
24
   

                       
10
5
   
                                         
                         

                                 

 
  


                               
L
1
     


 

                             
                                  


              

                      


    


                                                                              
 
 
             

         

         

                     

            
       


 
 
 
 
 

 
   


    
 
 

 


          

 
              
  
    



    

      
   
 

     
     




 




  

 
    


  
   




  
      
  
 
   


      



     

 
    

    

     

 



     







    
               

  

    



    

   
  

 
           

        



     


 
 

   
   
   



 
 






 




 



 




 

 


   





   

  

 
 









 
 




 

     

















 


        

   
 



    

     

 


  










   


 
 

          

                    




                                           
                                        
           

    

                  
    

                                 
             

                      
                    

                  
       

                                     
       

                             
                        

 

    
                
 
            

       
           

            

          
                                         
                                           
                                          
          
                          
                           

     
                                      

    


                                   
        

                   

 
         
     

  

      








 







  



 
±



                



    

±
 


        

      

                   
                              

        

 
                            



 
           

                   
                                   

              

   
 
            

          
 
        

             
                                        
                 



                
                                    
                 

 
 
         
                                      

                                        
   
   
                             
   

                                 
            

                             
                       

             
        

                            
     

                                 
              
                  

                                 
               




       

           
                                     
                                      
               

                        
             

                  

      
                                         
                                        
                               
       
                         

         
                            

            
                                      


         
 
                         

         
K
λ,β
(x
1
,x
2
) = K
λ
(x
1
,x
2
) +
+ β log p(x
1
) log p(x
2
)


 
    
K
λ
(x
1
,x
2
)
                   

  

p(x)
                             
β ≥
0
                                      
                  
λ
  
β
       
0.3
  
0.2
                                  
         

       

                          
     
alog p(x) +b
                        
                                     
a
  
b
                                
                   
25
    

            
η = 0.0001
           
α = 0.5

         
                             
[−0.01,+0.01]
                           
15


 
                                     
                       
1820


                                      
                                         
                             

           
                                        
          
     

        

             
                           
     


                        



        

         
       

  

      

 
 


  


 

 
 
 
  

  

 
 


   
 
 


 
      
  
         
       







  



  
 
 


     
 
 
  
 
 
 


 

  
 
 
                            
                                       
φ(x)
                        
Φ(x)
    
                          

  
 

 
        

                           
      
Φ(x
i
)
                         


      

                                    
                                     
                                     
                                        
                                    
         

                            
           

                

          
                                          

          

                       
                                     
                                         
                                        
                  

   



                                
              

                   

 
   
      
     


         


   

   
 

    
      

 

    


       
         
±
      
±

 
                                       
                 

               

 
                                           
               

                        
                   

                  
                                      
         


2
                         
                                   
                     



     

        
                                         
                                       
                                          
            
                                   
            

                                   

 
                     
L
r
  

 

      
       
L
￿
  

  
                     
                            

  
      


  

                                  
            

     
5
        
100
   
                                      
                                        


-
-3
-2.5
-2
-1.5
-1
-0.5
0
4.5
-4 -3.5
-3
-2.5
-2
-1.5
-1

      

                            

   

                                        
                  

   

     


 
   
±

     
±





    
±
 

   
±

 


 
 
±

 

 
    

±
 

 
    

                               
                                        
                                     
                                        
                                         
                                 

  
                                     
           

   

                    
                                      
                                       
                   
            

                                   
                             

  
                                      


         

                                 
                                         
                           
                                     
                                         
    

          

                       
                                      
                                          
             

         

                
        
 
 

                               
                                        
                                       
             
              
                                    
                             

       
                             

         

   

     

                         
                     
         





      

          

       

  
              


       

          

  
                               
      
   

      
 


      

             
              

                      

  

         

         

            
   


 
 
 
  


          

 

  

          

        
              

  

         

      
   

               


 
  


 
 
 

     
 
            

            

        

                           

     

  

         

         

  
             


 





   
  


 


 





      

     

         

    
   

                    

          

           
 
   
                 
         

 

 

 


 




        




   

        


      
   

            

          

  
                
 
               

            
 
 

 





 

     
 
     

         
 


   
   

       

    

                   
   

             

             
                         

        
 
    
 
         

         

 

  

              
        

                

         

                

          

     
 
   

  

        


 
 
    
 

                    

   
                       

        
    

  

              


 
 


  

    

           

        
                       

             
    

                            
   
          

 

      

            
        

                            
                                     
   

  

  

                       
             

   

       

 
           

           

       

              

    
 
             
   

                  




    

   



 

      
 
        

               

  
         

                       

 
                                

   
      



 


 
   

 

 


        
 
      

       

 

   
    


                    

   

      

            

          
                                    


       

                          




 

 

  
 



          
 
     
 
               
                        

           
  
 
           

                 
              

        

            

                     
 
           

 
     
  

     
 


      

        
 
     

     

                            

         

                      


                                 
                    
          

          

              


                             

 
        

  

      

            
          

 

    

   




  

      


       
 
         

              

                
            

         
 


      


         
 
           

   

  
                            





   


  


 



      


                 

     
                     

  

  

  
             



               
        

             

            
      


 
 