gaybayberryAI and Robotics

Nov 17, 2013 (3 years and 6 months ago)


Malardalen University, Vasteras, Sweden.
Face recognition can be applied for a
wide variety of problems like image and film
processing, human-computer interaction, criminal
identification etc. This has motivated researchers
to develop computational models to identify the
faces, which are relatively simple and easy to
implement. The existing system represents some
face space with higher dimensionality and it is not
effective too. The important fact which is
considered is that although these face images have
high dimensionality, in reality they span very low
dimensional space. So instead of considering whole
face space with high dimensionality, it is better to
consider only a subspace with lower dimensionality
to represent this face space.
The goal is to implement the system
(model) for a particular face and distinguish it
from a large number of stored faces with some
real-time variations as well.The Eigenface
approach uses Principal Component Analysis
(PCA) algorithm for the recognition of the images.
It gives us efficient way to find the lower
dimensional space.
Further this algorithm can be extended to
recognize the gender of a person or to interpret the
facial expression of a person.Recognition could be
carried out under widely varying conditions like
frontal view, a 45° view, scaled frontal view,
subjects with spectacles etc. are tried, while the
training data set covers limited views.The
algorithm models the real-time varying lighting
conditions as well. But this is out of scope of the
current implementation.
Key Words:Eigen faces, Eigen Vectors, Principal
Component Analysis (PCA), Linear Discriminant
Analysis (LDA).

Malardalen University, Vasteras, Sweden.
This paper is a step towards developing a face
recognition system which can recognize static
images. It can be modified to work with dynamic
images. In that case the dynamic images received
from the camera can first be converted in to the
static ones and then the same procedure can be
applied on them.The scheme is based on an
information theory approach that decomposes face
images into a small set of characteristic feature
images called ‘Eigenfaces’, which are actually the
principal components of the initial training set of
face images. Recognition is performed by
projecting a new image into the subspace spanned
by the Eigenfaces (‘face space’) and then
classifying the face by comparing its position in the
face space with the positions of the known
individuals.The Eigenface approach gives us
efficient way to find this lower dimensional space.
Eigenfaces are the Eigenvectors which are
representative of each of the dimensions of this
face space and they can be considered as various
face features. Any face can be expressed as linear
combinations of the singular vectors of the set of
faces, and these singular vectors are eigenvectors of
the covariance matrices.
In general, face recognition techniques can be
divided into two groups based on the face
representation they use:
1. Appearance-based, which uses holistic texture
features and is applied to either whole-face or
specific regions in a face image.
2. Feature-based, which uses geometric facial
features (mouth, eyes, brows, cheeks etc) and
geometric relationships between them?
Among many approaches to the problem of
face recognition, appearance-based subspace
analysis is one of the oldest,still gives the most
promising results.Subspace analysis is done by
projecting an image into a lower dimensional space
(subspace) and after that recognition is performed
by measuring the distances between known images
and the image to be recognized. The most
challenging part of such a system is finding an
adequate subspace.
When using appearance-based methods, we
usually represent an image of size n x m pixels by a
vector in an n x m dimensional space.These (n x m
dimensional) spaces are too large to allow robust
and fast object recognition. A common way to
attempt to resolve this problem is to use
dimensionality reduction techniques. PCA & LDA
are the two most popular techniques usually used
for dimensionality reduction. In this paper we make
use of PCA dimensionality reduction method.
The organization of the paper can be
summarized as: The Eigenface approach is briefly
explained in Section-2.The distinguishing
characteristics of PCA & LDA are discussed in
Section-3. The working of Face Recognition
system using Eigen values is briefly explained in
Section-4.The Significance of Eigen face approach
is explained in Section-5.Summary is briefly
explained in Section-6.Conclusions & Future work
explained in Section-7
2. Eigenface Approach:
2.1 Eigen Values and Eigen Vectors:
In linear algebra, the eigenvectors of a
linear operator are non-zero vectors which, when
operated on by the operator, result in a scalar
multiple of them. The scalar is then called the
eigenvalue (λ) [3] associated with the
eigenvector(X). Eigen vector is a vector that is
scaled by a linear transformation. It is a property of
a matrix. When a matrix acts on it, only the vector
magnitude is changed not the direction.
AX=λX (2.1)
Where Ais a Vector function.
2.2 Calculations of Eigen Values and Eigen
By using (2.1), we have the equation,

(A-λI)X=0 (2.2)

Where I is the n x n Identity matrix. This is a
homogeneous system of equations, and from
fundamental linear algebra, we know that a
nontrivial solution exists if and only if
det (A-λI)=0 (2.3)
Where det() denotes determinant. When evaluated,
becomes a polynomial of degree n. This is known
as the characteristic equation of A, and the
corresponding polynomial is the characteristic
polynomial. The characteristic polynomial is of
degree n. If A is n x n, then there are n solutions or
n roots of the characteristic polynomial. Thus there
are n eigenvalues of A satisfying the equation,
Where i=1, 2, 3….n
If the eigenvalues are all distinct, there are n
associated linearly independent eigenvectors,
whose directions are unique, which span an n
dimensional Euclidean space.
Repeated Eigenvalues:
In the case where there are r repeated eigenvalues,
then a linearly independent set of n eigenvectors
exist, provided the rank of the matrix
(A-λI) (2.5)
is rank n-r . Then, the directions of the r
eigenvectors associated with the repeated
eigenvalues are not unique.
3.1 Introduction to PCA approach:
Principal Component Analysis [8] is a
widely used technique in the fields of signal
processing, communications, control theory, image
processing, etc. In the PCA approach the
component matching relies on good data to build
eigenfaces. In other words, it builds Meigenvectors
for an N x Mmatrix. They are ordered from largest
to lowest where the largest eigenvalue if associated
with the vector that finds the most variance in the
image. An advantage of PCA to other methods is
that the 90% of the total variance is contained in 5-
10% of the dimensions. To classify an image we
want to find the eigenface with smallest Euclidean
distance from the input face. To do so we transform
the input image to a lower dimension M’ by
[v_1 v_2 … v_M]^T
Where each v_i = w_i * e_i ^T,v_i is the ith
coordinate of the facial image in the new space,
which came to be the principal component. The
vectors e_i, also called eigenimages, can also be
represented as images and look like faces. These
vectors represent the classes of faces to which we
can classify a new instance of a face. Now that we
have our transformed M we can find the vector k to
which the image is the closest.
Let Omega represent the contribution of each
eigenface to the representation of an image in a
basis constructed from the eigenvectors. Then
find k such that

Epsion_k = || Omega - Omega_k|| < Theta
Where Omega_k is the vector describing
the kth face class. When Epsilon_k is less than
some threshold value Theta the new face is
classified to belong to class k.
3.2 Introduction to LDA approach:
Linear Discriminant Analysis (LDA) is a
well-known scheme for feature extraction and
dimension reduction. It has been used widely in
any applications involving high-dimensional data,
such as face recognition and image retrieval. An
intrinsic limitation of classical LDA is the so-called
Singularity problem, that is,it fails when all scatter
matrices are singular. LDA finds the vectors in the
underlying space that best discriminate among
classes.For all samples of all classes the between-
class scatter matrix SB and the within-class scatter
matrix SW are defined by:
Where Mi is the number of training samples in
class i, c is the number of distinct classes,
is the
mean vector of samples belonging to class i and Xi
represents the set of samples belonging to class i
With xk being the k-th image of that class. SW
represents the scatter of features around the mean
of each face class and SB represents the scatter of
features around the overall mean for all face classes
3.3 Comparison of PCA and LDA:

Distinguishing characteristics of these two
techniques are as follows:
• LDA and PCA are the two popular independent
feature extraction methods. Both of them extract
features by projecting the original parameter
vectors into a new feature space through a linear
transformation matrix.
• They optimize the transformation matrix with
different intentions. PCA optimizes the
transformation matrix by finding the largest
variations in the original feature space.
• LDA pursues the largest ratio of between-class
variation and within-class variation when
projecting the original feature to a subspace.
• PCA has been used in face recognition, handprint
recognition, human-made object recognition,
industrial robotics, and mobile robotics.
• LDA has been used in face recognition and
mobile robotics. LDA has also been proposed for
generic object recognition.

The main differences between these two
methods are as follows;
 LDA is based on a single face image as
input [1], [2]. That means LDA can
perform face recognition for a single input
image; hence it does not consider multiple
input images.
 Where as PCA is based on multiple face
images as input. Hence it considers
multiple input images.
 PCA is less sensitive where as LDA is
more sensitive.
 PCA takes very less computational time
where as LDA takes more computational
Fig.1 [10] describes the decision thresholds yielded
by the nearest-neighbor approach for the two cases
are marked DPCA and DLDA. As can be seen by
the manner in which the decision thresholds
intersect the ellipses corresponding to the class
distributions, PCA yields superior results.
Fig 1
Where X- axis : Class A and Y axis : Class B
Suppose there are two different classes
(Class A and Class B) embedded in two
different”Gaussian-like” distributions.However,
only two samples per class are supplied to the
learning procedure (PCA or LDA).The
classification result of the PCA procedure (using
only the first eigenvector) is more desirable than
the result of the LDA. DPCA and DLDA represent
the decision thresholds obtained by using the
nearest-neighbor classification.
Principle component analysis aims to
catch the total variation in the set of training faces,
and to explain the variation by a few variables. In
fact, observation described by a few variables is
easier to understand than one defined by a huge
amount of variables and when many faces have to
be recognized the dimensionality reduction is
important.The other main advantage of PCA is that
once you have found these patterns in the data, and
you compress the data, ie. By reducing the number
of dimensions,without much loss of information.
This technique is also used in image compression.
4. Face Recognition System:
4.1 Face space creation:
The accurate reconstruction of the
face is not required, so we can now reduce the
dimensionality to M’ instead of M. This is
done by selecting the M’ Eigenfaces which
have the largest associated Eigenvalues. These
Eigenfaces now span a M’-dimensional which
reduces computational time.

Eigen Faces
Face Space (sum of Eigen Faces)
Figure 2
Fig-2 describes about Eigen faces. [9]
In order to reconstruct the original image from
the eigenfaces, one has to build a kind of
weighted sum of all eigenfaces (Face Space).
That is, the reconstructed original image is
equal to a sum of all eigenfaces, with each
eigenface having a certain weight. This weight
specifies, to what degree the specific feature
(eigenface) is present in the original image. If
one uses all the eigenfaces extracted from
original images, one can reconstruct the
original images from the eigenfaces exactly.
But one can also use only a part of the
eigenfaces. Then the reconstructed image is an
approximation of the original image. However,
one can ensure that losses due to omitting
some of the eigenfaces can be minimized. This
happens by choosing only the most important
features (eigenfaces).
4.2 Calculation of Eigen Values:
Two algorithms called TRED2 () & QL
algorithm are used for calculating Eigen
Values. In TRED2 algorithm, Covariance
Matrix is given as a input. Here Covariance
Matrix is converted into Tri-diagonalised form
except Upper, Lower & main diagonal
elements all other elements are made zero.
Consider an example:
Where e1, e2,...en are the Lower diagonal
elements and e1’,e2’,…en’ are the Upper
diagonal elements. This algorithm uses House-
Holder Vector form which is n x m matrix
(This is a way of Tri-diagonalising the
Matrix.). It accepts only symmetric matrix as

Working of QL algorithm:
According to the largest value it shifts
the elements in a matrix among themselves.
(That means Upper and Lower diagonal
elements are shifted with respect to main
diagonal elements).Hence the number of shifts
gives the EigenValues. Higher the Eigen
values higher the property of that image and
diagonal elements will be in a sorted order.
Normalization: It takes max Eigen value of the
Eigen Vector and divides each Eigen value by
max Eigen value.
4.3 Face Recognition system:
A new image T is transformed into its
eigenface components (projected into ‘face
space’) by a simple operation[11],
k =
(T - ψ)
Here k = 1, 2….M’. The weights obtained as
above form a vector Ω
= [w
that describes the contribution of each
eigenface in representing the input face image.
The vector may then be used in a standard
pattern recognition algorithm to find out which
of a number of predefined face class, if any,
best describes the face. The face class can be
calculated by averaging the weight vectors for
the images of one individual. The face classes
to be made depend on the classification to be
made like a face class can be made of all the
images where subject has the spectacles. With
this face class, classification can be made if the
subject has spectacles or not. The Euclidean
distance of the weight vector of the new image
from the face class weight vector can be
calculated as follows,

= || Ω – Ω

Where Ω
is a vector describing the k
class.Euclidean distance formula can be found
in [2]. The face is classified as belonging to
class k when the distance ε
is below some
threshold value θε. Otherwise the face is
classified as unknown. Also it can be found
whether an image is a face image or not by
simply finding the squared distance between
the mean adjusted input image and its
projection onto the face space.
= || Ф - Ф
Where Ф
is the face space and Ф = T
– Ψ is
the mean adjusted input.With this we can
classify the image as known face image,
unknown face image and not a face image.
4.3.1 Section explains the Figure 3
Figure 3
4.3.1 Procedure of Recognition system:
 Eigenfaces Initialization
1. Acquire an initial set of face images (the
training set)
2. Calculate the Eigenfaces from the training set,
keeping only the M images that correspond to the
highest eigenvalues [4], [5], [6], and [7]. These
M images define the face space. As new faces
are experienced, the Eigenfaces can be updated
or recalculated
3. Calculate the corresponding distribution in M-
dimensional weight space for each known
individual, by projecting their face images onto
the “face space.”
 Eigenfaces Recognition
1. Calculate a set of weights based on the input
image and the M Eigenfaces by projecting the
input image onto each of the Eigenfaces.
2. Determine if the image is a face at all by
checking to see if the image is sufficiently close to
“face space.”
3. (Optional) Update the Eigenfaces and/or weight
4. If it is a face, classify the weight pattern as
either a known person or as unknown
To summarize whole process of Face
Recognition using Eigenface approach from the
above diagram, the training set of images are
given as input to find eigenspace. Using these
images, the average face image is computed. The
difference of these images is represented by
covariance matrix. This is used to calculate
Eigenvectors and Eigenvalues. These are the
Eigenfaces which represent various face features.
Sort the eigenvalues, and consider higher of them
since they represent maximum variations. This
becomes eigenspace spanned by the eigenfaces,
Which has lower dimension than original
images?Now given two test images are projected
onto this eigenspace to give the weight vector
also known as Face key for that image. The
Euclidean distance between these two face key
vectors is calculated. If this is below some
threshold value, then two images are said to be
matching that means they belong to same person.
Depending on this result, False
Acceptation Rate (FAR) and False Rejection
Rate (FRR) are found. These are used to change
value of Threshold. In this way Face Recognition
is carried out using Eigenface Approach.
4.3.2 Overview of the PCA algorithm:
The algorithm for the facial recognition using
eigenfaces is basically described in figure 4[7].
First, the original images of the training set are
transformed into a set of eigenfaces E.
Afterwards; the weights are calculated for each
image of the training set and stored in the set W.
Upon observing an unknown image X, the
weights are calculated for that particular image
and stored in the vector W
. Afterwards,W
compared with the weights of images, of which
one knows for certain that they are facing (the
weights of the training set W). One way to do it
would be to regard each weight vector as a point
in space and calculate an average
distance D between the weight vectors
and the weight vector of the unknown
image W
.If this average distance exceeds some
threshold value, then the weight vector of the
unknown image W
lies too ”far apart” from the
weights of the faces. In this case, the
unknown X is considered to not a face. Otherwise
(if X is actually a face),its weight vector W
stored for later classification. The optimal
threshold value has to be determined
By means of PCA one can transform
each original image of the training set into a
corresponding eigenface. An important feature of
PCA is that one can reconstruct any original
image from the training set by combining the
eigenfaces. Remember that eigenfaces are
nothing less than characteristic features of the
faces. Therefore one could say that the original
face image can be reconstructed from eigenfaces
if one adds up all the eigenfaces (features) in the
right proportion. Each eigenface represents only
certain features of the face, which may or may not
be present in the original image. If the feature is
present in the original image to a higher degree, the
share of the corresponding eigenface in the” sum”
of the eigenfaces should be greater. If, contrary, the
particular feature is not (or almost not) present in
the original image, then the corresponding
eigenface should contribute a smaller (or not at all)
part to the sum of eigenfaces. So, in order to
reconstruct the original image from the eigenfaces,
one has to build a kind of weighted sum of all
eigenfaces. That is, the reconstructed original
image is equal to a sum of all eigenfaces, with each
eigenface having a certain weight. This weight
specifies, to what degree the specific feature
(eigenface) is present in the original image.
Figure 4
5. Significance of Eigenface Approach:
We have seen the standard Eigenface
Approach for Face Recognition. Here we find
out M Eigenvectors for representing training set
images. Now it is important to choose only M'
Eigenvectors from these M Eigenvectors, such
that M' is less than M, to represent face space
spanned by images. This will reduce the face
space dimensionality and enhance speed for face
recognition.Here we are reducing the
dimensionality of face images.
We can choose only M' Eigenvectors
with highest Eigenvalues. Now as the higher
Eigenvalues represent maximum face variation in
the corresponding Eigenvector direction, it is
important to consider this Eigenvector for face
space representation.Since, the lower
Eigenvalues does not provide much information
about face variations in corresponding
Eigenvector direction, such small Eigenvalues
can be neglected to further reduce the dimension
of face space.This does not affect the success
rate much and is acceptable depending on the
application of face recognition. Here only M'
Eigenvectors with higher Eigenvalues are chosen
for defining face space. The results of choosing
various values for M' are given in Results
section. The most important point here is to
choose value of M'(Maximum Eigen values
chosen from Co-variance Matrix) so that it does
not result in high error rates for Face recognition
process. The M' should be chosen such that error
rates does not increase much and are acceptable
depending on application for face recognition.
6. Summary
In this paper, the dimensionality
problems are solved for face recognition. The
approach using Eigenfaces and PCA is quite
robust in the treatment of face images with
varied facial expressions as well as the
directions. It is also quite efficient and simple in
the training and recognition stages, dispensing
low level processing to verify the facial geometry
or the distances between the facial organs and
their dimensions. However, this approach is
sensitive to images with uncontrolled
illumination conditions. One of the limitations
for eigenface approach is in the treatment of face
images with varied facial expressions and with
7. Conclusion and Future works:
The Eigenface approach for Face Recognition
process is fast and simple which works well
under constrained environment. It is one of the
best practical solutions for the problem of face
recognition. Many applications which require
face recognition do not require perfect
identification but just low error rate. So instead of
searching large database of faces, it is better to
give small set of likely matches. By using
Eigenface approach, this small set of likely
matches for given images can be easily obtained.

For given set of images, due to high
dimensionality of images, the space spanned is
very large. But in reality, all these images are
closely related and actually span a lower
dimensional space. By using eigenface approach,
we try to reduce this dimensionality. The
eigenfaces are the eigenvectors of covariance
matrix representing the image space. The lower the
dimensionality of this image space, the easier it
would be for face recognition. Any new image can
be expressed as linear combination of these
eigenfaces. This makes it easier to match any two
images and thus face recognition. We have also
seen that taking eigenvectors with higher M'
eigenvalues instead of all M eigenvectors, does not
affect performance much. So even taking lower
dimensional eigenspace for the images is
acceptable as error rates in such cases are very low.
The important part is making this choice of M'
which will be crucial depending on type of
application and error rate acceptable. More
research needs to be done on choosing the best
value of M'. This value of M' may vary depending
on the application of Face Recognition. So various
methods for making best choice of M' needs to be
studied.Further, this approach can be extended to
detect age, facial expressions of a person.
8. References
[1]”Discriminative learning and Recognition of
image set classes using Canonical correlations” by
Tae-Kyum Kim, Josef Kittler, Roberto Cipolla
(IEEE members).
[2] “Intelligent Face Recognition techniques: A
Comparative Study” by S.T Gandhe, K T Talele, A
G Keskar.
[4] “Recognition of Non-symmetric Faces Using
Principal Component Analysis” by N Krishnan,
GR Jothi, K Pulloor, GL Aaron
[5] M.A.Turk and A.P.Pentland, “Face
Recognition Using Eigenfaces”, Proc. Of IEEE
Conf. on Computer Vision and Pattern
Recognition, pp. 586-591, June 1991.
[6] Matthew Turk and Alex Pentland, “Eigenfaces
for Recognition”, Vision and Modeling Group,
The Media Laboratory, Massachusetts Institute of
Technology, In J. of cognitive
neuroscience,1991, vol.3,no.1, pages 71 to 86.
[7] “Eigenface-based facial recognition” by
[8] “Enhanced Principal Component Analysis
Recognition Performance” by Sezin Kaymak
[9] Nick Pears Thomas Heseltine and Jim
Austin. “ Evaluation of image preprocessing
techniques for eigenface based face
recognition”. ACA Group, Dept. of Computer
Science, University of York, 2002
[10] A.M. Martinez and A.C. Kak, “PCA versus
LDA,” IEEE Trans. Pattern Analysis and
Machine Intelligence, vol. 23, no. 2, pp. 228–
233, Feb. 2001
[11] Turk and A.Pent land. Eigenfaces for
recognition.Journal of Cognitive Neuroscience,