Structured Sparse Principal Component Analysis
Reading Group Presenter:
Peng
Zhang
Cognitive Radio Institute
Friday, October 01, 2010
Authors:
Rodolphe
Jenatton
, Guillaume
Obozinski
, Francis Bach
Outline
■
Introduction (in Imaging Sense)
□
Principal Component Analysis (PCA)
□
Sparse PCA (SPCA)
□
Structured Sparse PCA (SSPCA)
■
Problem Statement
■
The SSPCA Algorithm
■
Experiments
■
Conclusion and Other Thoughts
Introduction (Imaging Sense)
■
The face recognition problem
□
A database includes a huge amount of faces
□
How to let computer to recognize different faces with database
■
The challenge
□
Huge amount of data
□
Computation complexity
■
The trick
□
Represent the face using a weighted “face dictionary”
▪
Similar to code book in data compression
▪
Example: An 200 X 200 pixel face can be represented by 100
coefficients using the “face dictionary”
■
The solution
□
Principal component analysis (PCA)
PCA
■
PCA
□
A compression method
□
Given a large amount of sample vectors {x}
□
2
nd
moment statistics of the sample vectors
□
Eigen

decomposition finds the “dictionary” and “energy” of the
dictionary codes
▪
Eigen

vectors {v} form the “dictionary”
▪
Eigen

values {d} give the “energy” of “dictionary” elements
1 2 1 2
1
1 2 1 2
1
,,...,,,...,
,,...,...,,...,
T
N N
T
N N
N
x x x x x x R
N
v v v v v v
PCA
■
Original signal can be represented using only part of the
dictionary
▪
Data is compressed with fewer elements
■
Meaning of “dictionary” v:
□
It is the weights of each elements in x
■
The problem for PCA for face recognition: No physical
meaning for “dictionary”
,
1
M M N
i i n
n
x y n v
PCA
■
Face recognition
The Face Samples
PCA
The “dictionary”,
eigen

faces
These
eigen

faces can reconstruct original faces perfectly, but make no sense in real life
Structured SPCA
■
The SPCA goal:
□
Make dictionary more interpretable
□
The “sparse” solution: Limit the number of
nonzeros
Non

sparse Eigen

faces from PCA
Sparse Eigen

faces from SPCA
But the
eigen

faces are still meaningless most of time
Structured SPCA
■
The new idea, SSPCA
□
Eigen

faces will be meaningful when some structured constraints
are set
□
Meaningful areas in faces are constrained in “grids”
Eigen

faces from SSPCA
Structured SPCA
■
This paper’s contribution
□
Add the “structure” constraint to make the dictionary more
meaningful
□
How the constraint works
□
Meaningful dictionary is more close to “true” dictionary
□
Meaningful dictionary is more robust against noise
□
Meaningful dictionary is more accurate in face recognition
Outline
■
Introduction
□
Principal Component Analysis (PCA)
□
Sparse PCA (SPCA)
□
Structured Sparse PCA (SSPCA)
■
Problem Statement
■
The SSPCA Algorithm
■
Experiments
■
Conclusion and Other Thoughts
Problem Statement
■
From SPCA to SSPCA
□
The optimization problem
□
X is sample matrix, U is coefficient matrix, V is dictionary
□
. and
are different types of norms
□
The trick in SPCA
▪
L1 norm force the dictionary to be a sparse solution
Problem Statement
■
Structured SPCA, however, deal with a mixed l1/l2
minimization:
■
Right now it’s hard for me to understand the G and d
Problem Statement
■
In short, the norm constraints have the following effects
□
Dictionary has some structures
□
All non

zeros in the dictionary will be confined inside a grid
Outline
■
Introduction
□
Principal Component Analysis (PCA)
□
Sparse PCA (SPCA)
□
Structured Sparse PCA (SSPCA)
■
Problem Statement
■
The SSPCA Algorithm
■
Experiments
■
Conclusion and Other Thoughts
The SSPCA Algorithm
■
Making the dictionary sparser
□
The
norm,
□
The new SSPCA problem:
The SSPCA Algorithm
■
Methods to solve a sequence of convex problems
Excerpt from Author’s slide
■
Excerpt from author’s slide:
Excerpt from Author’s slide
Excerpt from Author’s slide
Excerpt from Author’s slide
Excerpt from Author’s slide
Excerpt from Author’s slide
Excerpt from Author’s slide
Excerpt from Author’s slide
Excerpt from Author’s slide
Outline
■
Introduction
□
Principal Component Analysis (PCA)
□
Sparse PCA (SPCA)
□
Structured Sparse PCA (SSPCA)
■
Problem Statement
■
The SSPCA Algorithm
■
Experiments
■
Conclusion and Other Thoughts
Conclusion and Other Thoughts
■
Conclusion
□
This paper shows how to use SSPCA
□
SSPCA gets better performance in
denoising
, face recognition
and classification
■
Other thoughts
□
Usually, the meaningful dictionary in communication signals is
Fourier dictionary
□
But Fourier dictionary may not fit some transient signals or time

variant signals
□
How to manipulate the G, d and norms to set constraints for our
needs?
THANK
YOU
!
Comments 0
Log in to post a comment