A Quadratic Loss Multi-Class SVM

zoomzurichΤεχνίτη Νοημοσύνη και Ρομποτική

16 Οκτ 2013 (πριν από 3 χρόνια και 7 μήνες)

90 εμφανίσεις

A Quadratic Loss Multi-Class SVM
Emmanuel Monfrini —Yann Guermeur
April 30,2008
    
 

  

      
               
    C            
             
               
              
              
                 
         2     
        
2
       2
              

         

 

 
     
              
              C  
                
             
              
            
               
               
               2   
        
2
     
    2          
   
           
     
  
                 
    C            
             
             
                 
               
                  
               
                 
                
               
                   
        2       

         2      
 
2
            
  ￿
1
              
                  
2              
             
                
  
               
            
                
2
               
                 
       2       
              
         
   
  
     
           
   Q    3 ≤ Q < ∞     
            x ∈ X   
Y    y             [[ 1,Q]]
                  
  P      X×Y       
     G   g = (g
k
)
1≤k≤Q
 X  R
Q
  
                g
 x ∈ X    l     g
l
(x) > max
k￿=l
g
k
(x)     
x         ∗  f       X
 Y
￿
{∗}    g          
        P (f (X) ￿= Y )     
               
 (X,Y ) ∈ X × Y    P       m
D
m
= ((X
i
,Y
i
))
1≤i≤m
     (X,Y )
                
 G         g

     D
m
 
               
              
   
    
                 
                
      
          κ   
X       κ:X
2
→R 
∀n ∈ N

,∀(x
i
)
1≤i≤n
∈ X
n
,∀(a
i
)
1≤i≤n
∈ R
n
,
n
￿
i=1
n
￿
j=1
a
i
a
j
κ(x
i
,x
j
) ≥ 0.
          (H,￿∙,∙￿
H
)    
   X  H⊂ R
X
   κ:X
2
→R       H  
 
 ∀x ∈ X,κ
x
= κ(x,∙) ∈ H
 ∀x ∈ X,∀h ∈ H,￿h,κ
x
￿
H
= h(x)  
     
               
    
    (H
κ
,￿∙,∙￿
H
κ
)       X    κ
     Φ  X    
￿
E
Φ(X)
,￿∙,∙￿
￿
 
∀(x,x
￿
) ∈ X
2
,κ(x,x
￿
) = ￿Φ(x),Φ(x
￿
)￿.
Φ        E
Φ(X)
   
            
    κ       X     
(H,￿∙,∙￿
H
)    X   κ      H
 κ         X   (H
κ
,￿∙,∙￿
H
κ
)     
 κ 
¯
H = (H
κ
,￿∙,∙￿
H
κ
)
Q
  H = ((H
κ
,￿∙,∙￿
H
κ
) +{1})
Q
   H 
     h = (h
k
)
1≤k≤Q
 X  
h(∙) =
￿
m
k
￿
i=1
β
ik
κ(x
ik
,∙) +b
k
￿
1≤k≤Q
  x
ik
   X           
{x
ik
:1 ≤ i ≤ m
k
}     X           
      H            Φ(X)
 h      
h(∙) = (￿w
k
,∙￿ +b
k
)
1≤k≤Q
   w
k
   E
Φ(X)
         (w,b)
 w = (w
k
)
1≤k≤Q
∈ E
Q
Φ(X)
 b = (b
k
)
1≤k≤Q
∈ R
Q
   
¯
H    
       Φ(X)     ￿.￿
¯
H
 

¯
h ∈
¯
H,
￿
￿¯
h
￿
￿
¯
H
=
￿
￿
￿
￿
Q
￿
k=1
￿w
k
￿
2
= ￿w￿,
 ￿w
k
￿ =
￿
￿w
k
,w
k
￿          
         
        ((x
i
,y
i
))
1≤i≤m
∈ (X ×[[ 1,Q]])
m

λ ∈ R

+
  Q          
  
￿
Q
k=1
h
k
= 0  H    J

  
J

(h) =
m
￿
i=1
￿

(y
i
,h(x
i
)) +λ
￿
￿¯
h
￿
￿
2
¯
H
         ￿

  
   
                  
            ￿

 
￿

(y,h(x)) =
￿
k￿=y
(1 −h
y
(x) +h
k
(x))
+
,
     (∙)
+
   max(0,∙)      
           ￿

 
￿

(y,
¯
h(x)) =
￿
1 −
¯
h
y
(x) +max
k￿=y
¯
h
k
(x)
￿
+
.
                   
 ￿

 
￿

(y,h(x)) =
￿
k￿=y
￿
h
k
(x) +
1
Q−1
￿
+
.
                 
          
       
      ￿

       ￿


             
               
     
    
min
w,b
J

(w,b)
s.t.





￿w
k
,Φ(x
i
)￿ +b
k
≤ −
1
Q−1
,(1 ≤ i ≤ m),(1 ≤ k ￿= y
i
≤ Q)
￿
Q
k=1
w
k
= 0
￿
Q
k=1
b
k
= 0

J

(w,b) =
1
2
Q
￿
k=1
￿w
k
￿
2
.
    
min
w,b
J

(w,b)
     
s.t.









￿w
k
,Φ(x
i
)￿ +b
k
≤ −
1
Q−1

ik
,(1 ≤ i ≤ m),(1 ≤ k ￿= y
i
≤ Q)
ξ
ik
≥ 0,(1 ≤ i ≤ m),(1 ≤ k ￿= y
i
≤ Q)
￿
Q
k=1
w
k
= 0
￿
Q
k=1
b
k
= 0

J

(w,b) =
1
2
Q
￿
k=1
￿w
k
￿
2
+C
m
￿
i=1
￿
k￿=y
i
ξ
ik
.
    ξ
ik
           
     C       
                  
   λ   C = (2λ)
−1
        
               
               
              
 α = (α
ik
)
1≤i≤m,1≤k≤Q
∈ R
Qm
+
        
                
           α
iy
i
    0
    δ ∈ E
Φ(X)
         
￿
Q
k=1
w
k
= 0  β ∈ R       
￿
Q
k=1
b
k
= 0
        
L(w,b,α,β,δ) =
1
2
Q
￿
k=1
￿w
k
￿
2
−￿δ,
Q
￿
k=1
w
k
￿ −β
Q
￿
k=1
b
k
+
m
￿
i=1
Q
￿
k=1
α
ik
￿
￿w
k
,Φ(x
i
)￿ +b
k
+
1
Q−1
￿
.
           w
k
    
   Q         δ
δ

= w

k
+
m
￿
i=1
α

ik
Φ(x
i
),(1 ≤ k ≤ Q).
   
￿
Q
k=1
w

k
= 0     k     
 δ

      
δ

=
1
Q
m
￿
i=1
Q
￿
k=1
α

ik
Φ(x
i
).
   
           w
k
  
w

k
=
1
Q
m
￿
i=1
Q
￿
l=1
α

il
Φ(x
i
) −
m
￿
i=1
α

ik
Φ(x
i
),(1 ≤ k ≤ Q)
      
w

k
=
m
￿
i=1
Q
￿
l=1
α

il
￿
1
Q
−δ
k,l
￿
Φ(x
i
),(1 ≤ k ≤ Q) 
 δ     
            b       
β

=
m
￿
i=1
α

ik
,(1 ≤ k ≤ Q)
 
m
￿
i=1
Q
￿
l=1
α

il
￿
1
Q
−δ
k,l
￿
= 0,(1 ≤ k ≤ Q).
  
￿
Q
k=1
b
k
= 0   
m
￿
i=1
Q
￿
k=1
α

ik
b

k
= β

Q
￿
k=1
b

k
= 0.
   
Q
￿
k=1
￿w

k
￿
2
=
Q
￿
k=1
￿
m
￿
i=1
Q
￿
l=1
α

il
￿
1
Q
−δ
k,l
￿
Φ(x
i
),
m
￿
j=1
Q
￿
n=1
α

jn
￿
1
Q
−δ
k,n
￿
Φ(x
j
)￿
=
m
￿
i=1
m
￿
j=1
Q
￿
l=1
Q
￿
n=1
α

il
α

jn
￿Φ(x
i
),Φ(x
j
)￿
Q
￿
k=1
￿
1
Q
−δ
k,l
￿￿
1
Q
−δ
k,n
￿
=
m
￿
i=1
m
￿
j=1
Q
￿
l=1
Q
￿
n=1
α

il
α

jn
￿
δ
l,n

1
Q
￿
κ(x
i
,x
j
).
    
m
￿
i=1
Q
￿
k=1
α

ik
￿w

k
,Φ(x
i
)￿ =
m
￿
i=1
Q
￿
k=1
α

ik
￿
m
￿
j=1
Q
￿
l=1
α

jl
￿
1
Q
−δ
k,l
￿
Φ(x
j
),Φ(x
i
)￿
     
=
m
￿
i=1
m
￿
j=1
Q
￿
k=1
Q
￿
l=1
α

ik
α

jl
￿
1
Q
−δ
k,l
￿
κ(x
i
,x
j
).
    
1
2
Q
￿
k=1
￿w

k
￿
2
+
m
￿
i=1
Q
￿
k=1
α

ik
￿w

k
,Φ(x
i
)￿ = −
1
2
Q
￿
k=1
￿w

k
￿
2
= −
1
2
m
￿
i=1
m
￿
j=1
Q
￿
k=1
Q
￿
l=1
α

ik
α

jl
￿
δ
k,l

1
Q
￿
κ(x
i
,x
j
).
       e
n
     R
n
   
     e  H      M
Qm,Qm
(R)   
h
ik,jl
=
￿
δ
k,l

1
Q
￿
κ(x
i
,x
j
).
                
       
L(α

) = −
1
2
α

T


+
1
Q−1
1
T
Qm
α

.
           
      
max
α
J

(α)
s.t.
￿
α
ik
≥ 0,(1 ≤ i ≤ m),(1 ≤ k ￿= y
i
≤ Q)
￿
m
i=1
￿
Q
l=1
α
il
￿
1
Q
−δ
k,l
￿
= 0,(1 ≤ k ≤ Q)

J

(α) = −
1
2
α
T
Hα +
1
Q−1
1
T
Qm
α,
        H 
h
ik,jl
=
￿
δ
k,l

1
Q
￿
κ(x
i
,x
j
).
  
￿
w
0
,b
0
￿
         
α
0
=
￿
α
0
ik
￿
1≤i≤m,1≤k≤Q
∈ R
Qm
+
           
  w
0
k
 
w
0
k
=
m
￿
i=1
Q
￿
l=1
α
0
il
￿
1
Q
−δ
k,l
￿
Φ(x
i
).
   
  
                 
   {(w
k
,b
k
):1 ≤ k ≤ Q}     C
2
Q
   
             
   
            Q
    H        {(x
i
,y
i
):1 ≤ i ≤ m}
  γ
kl
      k  l       
     k  l         
d

= min
1≤k<l≤Q
￿
min
￿
min
i:y
i
=k
(h
k
(x
i
) −h
l
(x
i
)),min
j:y
j
=l
(h
l
(x
j
) −h
k
(x
j
))
￿￿
  1 ≤ k < l ≤ Q  d
,kl

d
,kl
=
1
d

min
￿
min
i:y
i
=k
(h
k
(x
i
) −h
l
(x
i
) −d

),min
j:y
j
=l
(h
l
(x
j
) −h
k
(x
j
) −d

)
￿
.
  
γ
kl
= d

1 +d
,kl
￿w
k
−w
l
￿
.
         d

    
     
d

=
Q
Q−1
.
       d
,kl
 d
,kl
     
     
￿
w
0
,b
0
￿
 
            J

 
   
￿
k<l
￿w
k
−w
l
￿
2
= Q
Q
￿
k=1
￿w
k
￿
2
,
                     
            
              
Q
(Q−1)
2
￿
k<l
￿
1 +d
,kl
γ
kl
￿
2
=
Q
￿
k=1
￿w
0
k
￿
2
= α
0
T

0
=
1
Q−1
1
T
Qm
α
0
.
     
 

Q
(Q−1)
2
￿
k<l
￿
1+d
,kl
γ
kl
￿
2
=
￿
Q
k=1
￿w
0
k
￿
2
           

￿
Q
k=1
￿w
0
k
￿
2
= α
0
T

0
             H
 α
0
T

0
=
1
Q−1
1
T
Qm
α
0
      
α
0
ik
￿
￿w
0
k
,Φ(x
i
)￿ +b
0
k
+
1
Q−1
￿
= 0,(1 ≤ i ≤ m),(1 ≤ k ￿= y
i
≤ Q),
 
m
￿
i=1
Q
￿
k=1
α
0
ik
￿
￿w
0
k
,Φ(x
i
)￿ +b
0
k
+
1
Q−1
￿
= 0.
      
m
￿
i=1
Q
￿
k=1
α
0
ik
￿w
0
k
,Φ(x
i
)￿ +
1
Q−1
1
T
Qm
α
0
= 0.

m
￿
i=1
Q
￿
k=1
α
0
ik
￿w
0
k
,Φ(x
i
)￿ = −α
0
T

0
          
   
  
2
       
             
              
 ξ                 
       ξ = (ξ
ik
)
1≤i≤m,1≤k≤Q
 (ξ
iy
i
)
1≤i≤m
= 0
m
   
            ξ = (ξ
i
)
1≤i≤m
     
       C￿ξ￿
1

                
2             
 C￿ξ￿
2
2
               
              
                
       
      2      
             ￿ξ￿
1
 ￿ξ￿
2
2

                
             
 2               
              


T
Mξ = C
m
￿
i=1
m
￿
j=1
Q
￿
k=1
Q
￿
l=1
m
ik,jl
ξ
ik
ξ
jl
 M = (m
ik,jl
)
1≤i,j≤m,1≤k,l≤Q
      
  
2
      2 
                 
                 
     M  m
ik,jl
=
￿
δ
k,l

1
Q
￿
δ
i,j
     

2
   2          
  
2

min
w,b
J

2(w,b)
s.t.





￿w
k
,Φ(x
i
)￿ +b
k
≤ −
1
Q−1

ik
,(1 ≤ i ≤ m),(1 ≤ k ￿= y
i
≤ Q)
￿
Q
k=1
w
k
= 0
￿
Q
k=1
b
k
= 0
     

J

2(w,b) =
1
2
Q
￿
k=1
￿w
k
￿
2
+C
m
￿
i=1
m
￿
j=1
Q
￿
k=1
Q
￿
l=1
￿
δ
k,l

1
Q
￿
δ
i,j
ξ
ik
ξ
jl
.
                
           
L(w,b,ξ,α,β,δ) =
1
2
Q
￿
k=1
￿w
k
￿
2
+Cξ
T
Mξ −￿δ,
Q
￿
k=1
w
k
￿ −β
Q
￿
k=1
b
k
+
m
￿
i=1
Q
￿
k=1
α
ik
￿
￿w
k
,Φ(x
i
)￿ +b
k
+
1
Q−1
−ξ
ik
￿
.
    L     ξ      
2CMξ

= α


     


T


−α

T
ξ

= −Cξ

T


.
                
      
L(ξ



) = −
1
2
α

T


−Cξ

T


+
1
Q−1
1
T
Qm
α

.
  
α

in
α

ip
= 4C
2
Q
￿
k=1
￿
δ
k,n

1
Q
￿
ξ

ik
Q
￿
l=1
￿
δ
l,p

1
Q
￿
ξ

il
 
α

in
α

ip
= 4C
2
Q
￿
k=1
Q
￿
l=1
￿
δ
k,n
δ
l,p
−(δ
k,n

l,p
)
1
Q
+
1
Q
2
￿
ξ

ik
ξ

il
.
     n  p  
Q
￿
n=1
Q
￿
p=1
α

in
α

ip
￿
δ
n,p

1
Q
￿
= 4C
2
Q
￿
k=1
Q
￿
l=1
ξ

ik
ξ

il
Q
￿
n=1
Q
￿
p=1
￿
δ
k,n
δ
l,p
−(δ
k,n

l,p
)
1
Q
+
1
Q
2
￿￿
δ
n,p

1
Q
￿
.
   

Q
￿
n=1
Q
￿
p=1
￿
δ
k,n
δ
l,p
−(δ
k,n

l,p
)
1
Q
+
1
Q
2
￿￿
δ
n,p

1
Q
￿
= δ
k,l

1
Q
,
  
Q
￿
n=1
Q
￿
p=1
α

in
α

ip
￿
δ
n,p

1
Q
￿
= 4C
2
Q
￿
k=1
Q
￿
l=1
￿
δ
k,l

1
Q
￿
ξ

ik
ξ

il
.
     i  j  
α

T


= 4C
2
ξ

T


.
      
L(α

) = −
1
2
α

T
￿
H +
1
2C
M
￿
α

+
1
Q−1
1
T
Qm
α

.
                  
       b      
m
￿
i=1
Q
￿
l=1
α

il
￿
1
Q
−δ
k,l
￿
= 0,(1 ≤ k ≤ Q).
             
 
  
2
  
max
α
J

2
,d
(α)
s.t.
￿
α
ik
≥ 0,(1 ≤ i ≤ m),(1 ≤ k ￿= y
i
≤ Q)
￿
m
i=1
￿
Q
l=1
α
il
￿
1
Q
−δ
k,l
￿
= 0,(1 ≤ k ≤ Q)

J

2
,d
(α) = −
1
2
α
T
￿
H +
1
2C
M
￿
α +
1
Q−1
1
T
Qm
α.
       H  M       
 κ     κ
￿
 
κ
￿
(x
i
,x
j
) = κ(x
i
,x
j
) +
1
2C
δ
i,j
,(1 ≤ i,j ≤ m).
 Q = 2              
              
ξ
T
Mξ =
1
2
￿ξ￿
2
2
  
2
     2 
     
      
    
2
               
               
    
               
 X         d
m
= {(x
i
,y
i
):1 ≤ i ≤ m}  m  
X ×{−1,1}             
   d
m
\{(x
p
,y
p
)}       (x
p
,y
p
)   
α
0
p

1
D
2
m
  D
m
             
        
       γ      
          d
m
   L
m
  
           
 
L
m

D
2
m
γ
2
.
                
                  
           
   
         Q   
        X  d
m
= {(x
i
,y
i
):1 ≤ i ≤ m}    
       d
m
\{(x
p
,y
p
)}       (x
p
,y
p
)
  
max
k∈[[ 1,Q ]]
α
0
pk

1
Q(Q−1)D
2
m
  D
m
             
 {Φ(x
i
):1 ≤ i ≤ m}
   (w
p
,b
p
)            
   d
m
\{(x
p
,y
p
)} 
α
p
= (α
p
11
,...,α
p
(p−1)Q
,0,...,0,α
p
(p+1)1
,...,α
p
mQ
)
T
   
         α
p
   R
Qm
+
 
￿
α
p
pk
￿
1≤k≤Q
= 0
Q

              
   α
p
          

pk
)
1≤k≤Q
= 0
Q
        R
Qm
+
 λ
p
= (λ
p
ik
)
1≤i≤m,1≤k≤Q

µ
p
= (µ
p
ik
)
1≤i≤m,1≤k≤Q
 λ
p
        α
0
− λ
p
 
         
￿
α
0
pk
−λ
p
pk
￿
1≤k≤Q
= 0
Q

 α
0
−λ
p
     α
p
  
∀i ￿= p,∀k ￿= y
i

0
ik
−λ
p
ik
≥ 0 ⇐⇒λ
p
ik
≤ α
0
ik
.
         
∀k,
m
￿
i=1
Q
￿
l=1
￿
α
0
il
−λ
p
il
￿
￿
1
Q
−δ
k,l
￿
= 0 ⇐⇒
m
￿
i=1
Q
￿
l=1
λ
p
il
￿
1
Q
−δ
k,l
￿
= 0.
    λ
p
   







∀k,λ
p
pk
= α
0
pk
∀i ￿= p,∀k,0 ≤ λ
p
ik
≤ α
0
ik
￿
m
i=1
￿
Q
l=1
λ
p
il
￿
1
Q
−δ
k,l
￿
= 0,(1 ≤ k ≤ Q)
.
     µ
p
   α
p
+ K
1
µ
p
     
  K
1
                 
 
∀i,α
p
iy
i
+K
1
µ
p
iy
i
= 0 ⇐⇒µ
p
iy
i
= 0.
  
∀i,∀k ￿= y
i

p
ik
≥ 0 =⇒α
p
ik
+K
1
µ
p
ik
≥ 0.

m
￿
i=1
Q
￿
l=1

p
il
+cµ
p
il
)
￿
1
Q
−δ
k,l
￿
= 0 ⇐⇒
m
￿
i=1
Q
￿
l=1
µ
p
il
￿
1
Q
−δ
k,l
￿
= 0.
    µ
p
   







∀i,µ
p
iy
i
= 0
∀i,∀k ￿= y
i

p
ik
≥ 0
￿
m
i=1
￿
Q
l=1
µ
p
il
￿
1
Q
−δ
k,l
￿
= 0,(1 ≤ k ≤ Q)
.
          J    J

   
 λ
p
 µ
p
   J(α
0
−λ
p
) ≤ J(α
p
)  J (α
p
+K
1
µ
p
) ≤ J(α
0
)    

J(α
0
) −J(α
0
−λ
p
) ≥ J(α
0
) −J(α
p
) ≥ J (α
p
+K
1
µ
p
) −J(α
p
).
     
      
J(α
0
) −J(α
0
−λ
p
) =
1
2
λ
p
T

p
+
￿
−Hα
0
+
1
Q−1
1
Qm
￿
T
λ
p
.
       H
￿
−Hα
0
+
1
Q−1
1
Qm
￿
T
λ
p
=
m
￿
i=1
￿
k￿=y
i
￿
￿w
0
k
,Φ(x
i
)￿ +
1
Q−1
￿
λ
p
ik
=
m
￿
i=1
￿
k￿=y
i
￿
h
0
k
(x
i
) +
1
Q−1
￿
λ
p
ik

m
￿
i=1
￿
k￿=y
i
b
0
k
λ
p
ik
.
              
 λ
p
              
        
￿
Q
k=1
b
0
k
= 0 
m
￿
i=1
Q
￿
k=1
b
0
k
λ
p
ik
=
Q
￿
k=1
b
0
k
m
￿
i=1
λ
p
ik
=
￿
Q
￿
k=1
b
0
k
￿￿
m
￿
i=1
Q
￿
l=1
1
Q
λ
p
il
￿
= 0.

￿
−Hα
0
+
1
Q−1
1
Qm
￿
T
λ
p
≤ 0.
              J(α
0
) −J(α
0
−λ
p
)
J(α
0
) −J(α
0
−λ
p
) ≤
1
2
λ
pT

p
,
     H
J(α
0
) −J(α
0
−λ
p
) ≤
1
2
Q
￿
k=1
￿
￿
￿
￿
￿
m
￿
i=1
Q
￿
l=1
λ
p
il
￿
1
Q
−δ
k,l
￿
Φ(x
i
)
￿
￿
￿
￿
￿
2
.
                
  
J (α
p
+K
1
µ
p
) −J(α
p
) =
K
1
￿
−Hα
p
+
1
Q−1
1
Qm
￿
T
µ
p

K
2
1
2
Q
￿
k=1
￿
￿
￿
￿
￿
m
￿
i=1
Q
￿
l=1
µ
p
il
￿
1
Q
−δ
k,l
￿
Φ(x
i
)
￿
￿
￿
￿
￿
2


￿
−Hα
p
+
1
Q−1
1
Qm
￿
T
µ
p
=
m
￿
i=1
￿
k￿=y
i
￿
h
p
k
(x
i
) +
1
Q−1
￿
µ
p
ik
.
   
       d
m
\{(x
p
,y
p
)}     x
p
 
    n ∈ [[ 1,Q]]\{y
p
}   h
p
n
(x
p
) ≥ 0  I     
[[ 1,Q]]\{n}  [[ 1,m]]\{p}  
∀k ∈ [[ 1,Q]]\{n},α
p
I(k)n
> 0.
              
 α
p
         K
2
∈ R

+
  µ
p
     R
Qm

         
￿
µ
p
pn
= K
2
∀k ∈ [[ 1,Q]]\{n},µ
p
I(k)k
= K
2
.
         
1
Q
￿
m
i=1
￿
Q
k=1
µ
p
ik
=
K
2

￿
m
i=1
µ
p
ik
= K
2
 (1 ≤ k ≤ Q)      µ
p
   
   
K
2


h
p
n
(x
p
) +
￿
k￿=n
h
p
k
￿
x
I(k)
￿
+
Q
Q−1


.
 µ
p
                  
       n h
p
n
(x
p
) ≥ 0   
 
α
p
ik
￿
￿w
p
k
,Φ(x
i
)￿ +b
p
k
+
1
Q−1
￿
= 0,(1 ≤ i ￿= p ≤ m),(1 ≤ k ￿= y
i
≤ Q)
 
￿
h
p
k
￿
x
I(k)
￿￿
1≤k￿=n≤Q
= −
1
Q−1
1
Q−1
         
      
m
￿
i=1
￿
k￿=y
i
￿
h
p
k
(x
i
) +
1
Q−1
￿
µ
p
ik

K
2
Q−1
.
        
J (α
p
+K
1
µ
p
) −J(α
p
) ≥
K
1
K
2
Q−1

K
2
1
2
Q
￿
k=1
￿
￿
￿
￿
￿
m
￿
i=1
Q
￿
l=1
µ
p
il
￿
1
Q
−δ
k,l
￿
Φ(x
i
)
￿
￿
￿
￿
￿
2
.
      
1
2
Q
￿
k=1
￿
￿
￿
￿
￿
m
￿
i=1
Q
￿
l=1
λ
p
il
￿
1
Q
−δ
k,l
￿
Φ(x
i
)
￿
￿
￿
￿
￿
2

     
K
1
K
2
Q−1

K
2
1
2
Q
￿
k=1
￿
￿
￿
￿
￿
m
￿
i=1
Q
￿
l=1
µ
p
il
￿
1
Q
−δ
k,l
￿
Φ(x
i
)
￿
￿
￿
￿
￿
2
.
 ν
p
= (ν
p
ik
)
1≤i≤m,1≤k≤Q
     R
Qm
+
  µ
p
= K
2
ν
p
    
 K
3
= K
1
K
2
      
K

3
=
1
Q−1
￿
Q
k=1
￿
￿
￿
￿
m
i=1
￿
Q
l=1
ν
p
il
￿
1
Q
−δ
k,l
￿
Φ(x
i
)
￿
￿
￿
2
.
      
(Q−1)
2
Q
￿
k=1
￿
￿
￿
￿
￿
m
￿
i=1
Q
￿
l=1
λ
p
il
￿
1
Q
−δ
k,l
￿
Φ(x
i
)
￿
￿
￿
￿
￿
2
Q
￿
k=1
￿
￿
￿
￿
￿
m
￿
i=1
Q
￿
l=1
ν
p
il
￿
1
Q
−δ
k,l
￿
Φ(x
i
)
￿
￿
￿
￿
￿
2
≥ 1.
 η  R
Qm
  K(η) =
1
Q
￿
m
i=1
￿
Q
k=1
η
p
ik
  
￿
￿
￿
￿
￿
1
Q
m
￿
i=1
Q
￿
l=1
λ
p
il
Φ(x
i
) −
m
￿
i=1
λ
p
ik
Φ(x
i
)
￿
￿
￿
￿
￿
2
= K(λ
p
)
2
￿
1
(Φ(x
i
)) −
2
(Φ(x
i
))￿
2
 
1
(Φ(x
i
))  
2
(Φ(x
i
))       Φ(x
i
)  
 ￿
1
(Φ(x
i
)) −
2
(Φ(x
i
))￿
2
         D
2
m
  
    ν
p
  
(Q−1)
2
Q
2
K(λ
p
)
2
K(ν
p
)
2
D
4
m
≥ 1.
  K(ν
p
) = 1      λ
p
   
 K            
∀k ∈ [[ 1,Q]],
m
￿
i=1
λ
p
ik
=
1
Q
m
￿
i=1
Q
￿
l=1
λ
p
il
.
  
∀(k,l) ∈ [[ 1,Q]]
2
,
m
￿
i=1
λ
p
ik
=
m
￿
i=1
λ
p
il
.
  
∀k ∈ [[ 1,Q]],
m
￿
i=1
λ
p
ik
≥ max
l∈[[ 1,Q ]]
α
0
pl
.
   
             K      
     λ
p
ik
       λ
p

 
∀k ∈ [[ 1,Q]],
m
￿
i=1
λ
p
ik

= max
l∈[[ 1,Q ]]
α
0
pl
.
     K(λ
p

) = max
k∈[[ 1,Q ]]
α
0
pk
       K(ν
p
)
 K(λ
p

)     
￿
max
k∈[[ 1,Q ]]
α
0
pk
￿
2

1
(Q−1)
2
Q
2
D
4
m
.
               
    
          Q  
          X  d
m
= {(x
i
,y
i
):1 ≤ i ≤ m}  
  L
m
         
     D
m
          
   {Φ(x
i
):1 ≤ i ≤ m}        
L
m
≤ Q
2
D
2
m
￿
k<l
￿
1 +d
,kl
γ
kl
￿
2
.
            max
k∈[[ 1,Q ]]
α
0
pk
  
    d
m
\{(x
p
,y
p
)}     (x
p
,y
p
)   (x
p
,y
p
) 
 L
m
   
1
T
Qm
α
0

m
￿
i=1
max
k∈[[ 1,Q ]]
α
0
ik

L
m
Q(Q−1)D
2
m
.
     1
T
Qm
α
0
=
Q
Q−1
￿
k<l
￿
1+d
,kl
γ
kl
￿
2
     
     
     
    
                  
       2      
 
2
           
            Q
2
      
                  
           

               
        
   

   
   
                            
                            
                               
                                   
  
2

                   
  
2
      2         
         

2

                                
                                   
                              
     
     

           
       
             
      
              
          
            
 
            
          
             
 
            
    
            
    
              
        
              
           
       
              
         
             
       
            
   
             
       
              
   
                
    
              
            
            
 
               
            
      
              
         

           
         