PCA-FX: Supervised Feature Extraction scheme based on PCA

connectionviewAI and Robotics

Nov 17, 2013 (3 years and 11 months ago)

65 views

PCA
-
FX:
S
upervised
F
eature
E
xtraction
scheme
based on PCA


Myoung Soo Park, Jin Hee Na, Jin Young Choi

School of Electrical Engineering and Computer Science,

Seoul National University,

E
-
mail: {mspark, jhna, jychoi}@neuro.snu.ac.kr


Recently, feature selec
tion or feature extraction methods are becoming important for classification of
data with large input dimensions, such as face recognition. The main purpose of these methods is to
reduce the input dimension, that is, the number of input variables by genera
ting feature which has less
dimension than original input. Reduction of input dimension is important issues in many learning
problem and data mining, since the success of input dimension reduction can be a way to avoid the curse
of dimensionality. It also
can improve the classification performance by removing non
-
relevant inputs.

The famous methods of feature extractions are PCA, ICA, LDA, and the method using MLP, PCA and
ICA are statistical way to extract features without losing the variance (PCA) or the

mutual information
(ICA) in input data, which are classified as unsupervised methods since they do not use the output value
-

the class information of input. In this case, the extracted features may not be fitted for classifications. To
overcome this limi
tation, the supervised
-
type methods, such as LDA, MLP are developed. In recent years,
ICA
-
FX is proposed to overcome limits of the previous algorithms. ICA
-
FX which uses both input and
class information to extract features which results in the better class
ification performance than PCA, ICA.
Also, it can be applied for more general cases than LDA. Though, there are cases in which it seems to be
necessary for PCA to be used as preprocessing for reducing the input dimension before using ICA
-
FX as
like LDA. In

such cases, PCA is used to reduce the input dimension at first, then for input in the reduced
dimension, ICA
-
FX or LDA are used to extract the features. PCA seems to have the very large portions in
reduction of input dimension. From this setting, we can o
bserve two problems. First, there is the
possib
i
lity of side effect in classification performance, since the criterion for extracting features are
different in ICA
-
FX(or LDA) and PCA. Second, we should implement two algorithms, for example, LDA
+ PCA or IC
A
-
FX + PCA instead of one algorithm LDA or ICA
-
FX.

PCA
-
FX
algorithm
is feature extraction scheme based on PCA. It can be
implemented

by conventional
PCA without additional implementation of
other algorithms

and since it is based on PCA
,

the possibility
of
side effect in PCA
-
FX is less than that in ICA
-
FX or LDA. We apply our scheme for face recognition
problem using Yale database to show the feature extraction performance and compare the performance of
PCA
-
FX with that of other algorithms.