Structured Sparse Principal Component Analysis

parathyroidsanchovyΤεχνίτη Νοημοσύνη και Ρομποτική

17 Νοε 2013 (πριν από 3 χρόνια και 11 μήνες)

110 εμφανίσεις

Structured Sparse Principal Component Analysis

Reading Group Presenter:

Peng

Zhang

Cognitive Radio Institute

Friday, October 01, 2010

Authors:
Rodolphe

Jenatton
, Guillaume
Obozinski
, Francis Bach

Outline


Introduction (in Imaging Sense)


Principal Component Analysis (PCA)


Sparse PCA (SPCA)


Structured Sparse PCA (SSPCA)


Problem Statement


The SSPCA Algorithm


Experiments


Conclusion and Other Thoughts

Introduction (Imaging Sense)


The face recognition problem


A database includes a huge amount of faces


How to let computer to recognize different faces with database


The challenge


Huge amount of data


Computation complexity


The trick


Represent the face using a weighted “face dictionary”


Similar to code book in data compression


Example: An 200 X 200 pixel face can be represented by 100
coefficients using the “face dictionary”


The solution


Principal component analysis (PCA)


PCA


PCA


A compression method


Given a large amount of sample vectors {x}


2
nd

moment statistics of the sample vectors


Eigen
-
decomposition finds the “dictionary” and “energy” of the
dictionary codes







Eigen
-
vectors {v} form the “dictionary”


Eigen
-
values {d} give the “energy” of “dictionary” elements









1 2 1 2
1
1 2 1 2
1
,,...,,,...,
,,...,...,,...,
T
N N
T
N N
N
x x x x x x R
N
v v v v v v



 
 

 
 
 
PCA


Original signal can be represented using only part of the
dictionary




Data is compressed with fewer elements


Meaning of “dictionary” v:


It is the weights of each elements in x


The problem for PCA for face recognition: No physical
meaning for “dictionary”



,
1
M M N
i i n
n
x y n v




PCA


Face recognition

The Face Samples

PCA

The “dictionary”,
eigen
-
faces

These
eigen
-
faces can reconstruct original faces perfectly, but make no sense in real life

Structured SPCA


The SPCA goal:


Make dictionary more interpretable


The “sparse” solution: Limit the number of
nonzeros

Non
-
sparse Eigen
-
faces from PCA

Sparse Eigen
-
faces from SPCA

But the
eigen
-
faces are still meaningless most of time

Structured SPCA


The new idea, SSPCA


Eigen
-
faces will be meaningful when some structured constraints
are set


Meaningful areas in faces are constrained in “grids”

Eigen
-
faces from SSPCA

Structured SPCA


This paper’s contribution


Add the “structure” constraint to make the dictionary more
meaningful


How the constraint works


Meaningful dictionary is more close to “true” dictionary


Meaningful dictionary is more robust against noise


Meaningful dictionary is more accurate in face recognition


Outline


Introduction


Principal Component Analysis (PCA)


Sparse PCA (SPCA)


Structured Sparse PCA (SSPCA)


Problem Statement


The SSPCA Algorithm


Experiments


Conclusion and Other Thoughts

Problem Statement


From SPCA to SSPCA


The optimization problem






X is sample matrix, U is coefficient matrix, V is dictionary


||.|| and


are different types of norms


The trick in SPCA



L1 norm force the dictionary to be a sparse solution

Problem Statement


Structured SPCA, however, deal with a mixed l1/l2
minimization:





Right now it’s hard for me to understand the G and d

Problem Statement


In short, the norm constraints have the following effects


Dictionary has some structures


All non
-
zeros in the dictionary will be confined inside a grid

Outline


Introduction


Principal Component Analysis (PCA)


Sparse PCA (SPCA)


Structured Sparse PCA (SSPCA)


Problem Statement


The SSPCA Algorithm


Experiments


Conclusion and Other Thoughts

The SSPCA Algorithm


Making the dictionary sparser


The


norm,






The new SSPCA problem:

The SSPCA Algorithm


Methods to solve a sequence of convex problems

Excerpt from Author’s slide


Excerpt from author’s slide:

Excerpt from Author’s slide

Excerpt from Author’s slide

Excerpt from Author’s slide

Excerpt from Author’s slide

Excerpt from Author’s slide

Excerpt from Author’s slide

Excerpt from Author’s slide

Excerpt from Author’s slide

Outline


Introduction


Principal Component Analysis (PCA)


Sparse PCA (SPCA)


Structured Sparse PCA (SSPCA)


Problem Statement


The SSPCA Algorithm


Experiments


Conclusion and Other Thoughts

Conclusion and Other Thoughts


Conclusion


This paper shows how to use SSPCA


SSPCA gets better performance in
denoising
, face recognition
and classification


Other thoughts


Usually, the meaningful dictionary in communication signals is
Fourier dictionary


But Fourier dictionary may not fit some transient signals or time
-
variant signals


How to manipulate the G, d and norms to set constraints for our
needs?

THANK
YOU
!