PCA-FX: Supervised Feature Extraction scheme based on PCA

connectionviewAI and Robotics

Nov 17, 2013 (4 years and 8 months ago)


based on PCA

Myoung Soo Park, Jin Hee Na, Jin Young Choi

School of Electrical Engineering and Computer Science,

Seoul National University,

mail: {mspark, jhna, jychoi}@neuro.snu.ac.kr

Recently, feature selec
tion or feature extraction methods are becoming important for classification of
data with large input dimensions, such as face recognition. The main purpose of these methods is to
reduce the input dimension, that is, the number of input variables by genera
ting feature which has less
dimension than original input. Reduction of input dimension is important issues in many learning
problem and data mining, since the success of input dimension reduction can be a way to avoid the curse
of dimensionality. It also
can improve the classification performance by removing non
relevant inputs.

The famous methods of feature extractions are PCA, ICA, LDA, and the method using MLP, PCA and
ICA are statistical way to extract features without losing the variance (PCA) or the

mutual information
(ICA) in input data, which are classified as unsupervised methods since they do not use the output value

the class information of input. In this case, the extracted features may not be fitted for classifications. To
overcome this limi
tation, the supervised
type methods, such as LDA, MLP are developed. In recent years,
FX is proposed to overcome limits of the previous algorithms. ICA
FX which uses both input and
class information to extract features which results in the better class
ification performance than PCA, ICA.
Also, it can be applied for more general cases than LDA. Though, there are cases in which it seems to be
necessary for PCA to be used as preprocessing for reducing the input dimension before using ICA
FX as
like LDA. In

such cases, PCA is used to reduce the input dimension at first, then for input in the reduced
dimension, ICA
FX or LDA are used to extract the features. PCA seems to have the very large portions in
reduction of input dimension. From this setting, we can o
bserve two problems. First, there is the
lity of side effect in classification performance, since the criterion for extracting features are
different in ICA
FX(or LDA) and PCA. Second, we should implement two algorithms, for example, LDA
+ PCA or IC
FX + PCA instead of one algorithm LDA or ICA

is feature extraction scheme based on PCA. It can be

by conventional
PCA without additional implementation of
other algorithms

and since it is based on PCA

the possibility
side effect in PCA
FX is less than that in ICA
FX or LDA. We apply our scheme for face recognition
problem using Yale database to show the feature extraction performance and compare the performance of
FX with that of other algorithms.