# Structured Sparse Principal Component Analysis

Τεχνίτη Νοημοσύνη και Ρομποτική

17 Νοε 2013 (πριν από 4 χρόνια και 6 μήνες)

128 εμφανίσεις

Structured Sparse Principal Component Analysis

Reading Group Presenter:

Peng

Zhang

Cognitive Radio Institute

Friday, October 01, 2010

Authors:
Rodolphe

Jenatton
, Guillaume
Obozinski
, Francis Bach

Outline

Introduction (in Imaging Sense)

Principal Component Analysis (PCA)

Sparse PCA (SPCA)

Structured Sparse PCA (SSPCA)

Problem Statement

The SSPCA Algorithm

Experiments

Conclusion and Other Thoughts

Introduction (Imaging Sense)

The face recognition problem

A database includes a huge amount of faces

How to let computer to recognize different faces with database

The challenge

Huge amount of data

Computation complexity

The trick

Represent the face using a weighted “face dictionary”

Similar to code book in data compression

Example: An 200 X 200 pixel face can be represented by 100
coefficients using the “face dictionary”

The solution

Principal component analysis (PCA)

PCA

PCA

A compression method

Given a large amount of sample vectors {x}

2
nd

moment statistics of the sample vectors

Eigen
-
decomposition finds the “dictionary” and “energy” of the
dictionary codes

Eigen
-
vectors {v} form the “dictionary”

Eigen
-
values {d} give the “energy” of “dictionary” elements

1 2 1 2
1
1 2 1 2
1
,,...,,,...,
,,...,...,,...,
T
N N
T
N N
N
x x x x x x R
N
v v v v v v

 
 

 
 
 
PCA

Original signal can be represented using only part of the
dictionary

Data is compressed with fewer elements

Meaning of “dictionary” v:

It is the weights of each elements in x

The problem for PCA for face recognition: No physical
meaning for “dictionary”

,
1
M M N
i i n
n
x y n v

PCA

Face recognition

The Face Samples

PCA

The “dictionary”,
eigen
-
faces

These
eigen
-
faces can reconstruct original faces perfectly, but make no sense in real life

Structured SPCA

The SPCA goal:

Make dictionary more interpretable

The “sparse” solution: Limit the number of
nonzeros

Non
-
sparse Eigen
-
faces from PCA

Sparse Eigen
-
faces from SPCA

But the
eigen
-
faces are still meaningless most of time

Structured SPCA

The new idea, SSPCA

Eigen
-
faces will be meaningful when some structured constraints
are set

Meaningful areas in faces are constrained in “grids”

Eigen
-
faces from SSPCA

Structured SPCA

This paper’s contribution

Add the “structure” constraint to make the dictionary more
meaningful

How the constraint works

Meaningful dictionary is more close to “true” dictionary

Meaningful dictionary is more robust against noise

Meaningful dictionary is more accurate in face recognition

Outline

Introduction

Principal Component Analysis (PCA)

Sparse PCA (SPCA)

Structured Sparse PCA (SSPCA)

Problem Statement

The SSPCA Algorithm

Experiments

Conclusion and Other Thoughts

Problem Statement

From SPCA to SSPCA

The optimization problem

X is sample matrix, U is coefficient matrix, V is dictionary

||.|| and

are different types of norms

The trick in SPCA

L1 norm force the dictionary to be a sparse solution

Problem Statement

Structured SPCA, however, deal with a mixed l1/l2
minimization:

Right now it’s hard for me to understand the G and d

Problem Statement

In short, the norm constraints have the following effects

Dictionary has some structures

All non
-
zeros in the dictionary will be confined inside a grid

Outline

Introduction

Principal Component Analysis (PCA)

Sparse PCA (SPCA)

Structured Sparse PCA (SSPCA)

Problem Statement

The SSPCA Algorithm

Experiments

Conclusion and Other Thoughts

The SSPCA Algorithm

Making the dictionary sparser

The

norm,

The new SSPCA problem:

The SSPCA Algorithm

Methods to solve a sequence of convex problems

Excerpt from Author’s slide

Excerpt from author’s slide:

Excerpt from Author’s slide

Excerpt from Author’s slide

Excerpt from Author’s slide

Excerpt from Author’s slide

Excerpt from Author’s slide

Excerpt from Author’s slide

Excerpt from Author’s slide

Excerpt from Author’s slide

Outline

Introduction

Principal Component Analysis (PCA)

Sparse PCA (SPCA)

Structured Sparse PCA (SSPCA)

Problem Statement

The SSPCA Algorithm

Experiments

Conclusion and Other Thoughts

Conclusion and Other Thoughts

Conclusion

This paper shows how to use SSPCA

SSPCA gets better performance in
denoising
, face recognition
and classification

Other thoughts

Usually, the meaningful dictionary in communication signals is
Fourier dictionary

But Fourier dictionary may not fit some transient signals or time
-
variant signals

How to manipulate the G, d and norms to set constraints for our
needs?

THANK
YOU
!