FACE RECOGNITION USING EIGENFACE APPROACH

VINAY HIREMATH

Malardalen University, Vasteras, Sweden.

vhh09001@student.mdh.se

ABSTRACT:

Face recognition can be applied for a

wide variety of problems like image and film

processing, human-computer interaction, criminal

identification etc. This has motivated researchers

to develop computational models to identify the

faces, which are relatively simple and easy to

implement. The existing system represents some

face space with higher dimensionality and it is not

effective too. The important fact which is

considered is that although these face images have

high dimensionality, in reality they span very low

dimensional space. So instead of considering whole

face space with high dimensionality, it is better to

consider only a subspace with lower dimensionality

to represent this face space.

The goal is to implement the system

(model) for a particular face and distinguish it

from a large number of stored faces with some

real-time variations as well.The Eigenface

approach uses Principal Component Analysis

(PCA) algorithm for the recognition of the images.

It gives us efficient way to find the lower

dimensional space.

Further this algorithm can be extended to

recognize the gender of a person or to interpret the

facial expression of a person.Recognition could be

carried out under widely varying conditions like

frontal view, a 45° view, scaled frontal view,

subjects with spectacles etc. are tried, while the

training data set covers limited views.The

algorithm models the real-time varying lighting

conditions as well. But this is out of scope of the

current implementation.

Key Words:Eigen faces, Eigen Vectors, Principal

Component Analysis (PCA), Linear Discriminant

Analysis (LDA).

ASHWINI MAYAKAR

Malardalen University, Vasteras, Sweden.

amr09002@student.mdh.se

1.INTRODUCTION:

This paper is a step towards developing a face

recognition system which can recognize static

images. It can be modified to work with dynamic

images. In that case the dynamic images received

from the camera can first be converted in to the

static ones and then the same procedure can be

applied on them.The scheme is based on an

information theory approach that decomposes face

images into a small set of characteristic feature

images called ‘Eigenfaces’, which are actually the

principal components of the initial training set of

face images. Recognition is performed by

projecting a new image into the subspace spanned

by the Eigenfaces (‘face space’) and then

classifying the face by comparing its position in the

face space with the positions of the known

individuals.The Eigenface approach gives us

efficient way to find this lower dimensional space.

Eigenfaces are the Eigenvectors which are

representative of each of the dimensions of this

face space and they can be considered as various

face features. Any face can be expressed as linear

combinations of the singular vectors of the set of

faces, and these singular vectors are eigenvectors of

the covariance matrices.

In general, face recognition techniques can be

divided into two groups based on the face

representation they use:

1. Appearance-based, which uses holistic texture

features and is applied to either whole-face or

specific regions in a face image.

2. Feature-based, which uses geometric facial

features (mouth, eyes, brows, cheeks etc) and

geometric relationships between them?

Among many approaches to the problem of

face recognition, appearance-based subspace

analysis is one of the oldest,still gives the most

promising results.Subspace analysis is done by

projecting an image into a lower dimensional space

(subspace) and after that recognition is performed

by measuring the distances between known images

and the image to be recognized. The most

challenging part of such a system is finding an

adequate subspace.

When using appearance-based methods, we

usually represent an image of size n x m pixels by a

vector in an n x m dimensional space.These (n x m

dimensional) spaces are too large to allow robust

and fast object recognition. A common way to

attempt to resolve this problem is to use

dimensionality reduction techniques. PCA & LDA

are the two most popular techniques usually used

for dimensionality reduction. In this paper we make

use of PCA dimensionality reduction method.

The organization of the paper can be

summarized as: The Eigenface approach is briefly

explained in Section-2.The distinguishing

characteristics of PCA & LDA are discussed in

Section-3. The working of Face Recognition

system using Eigen values is briefly explained in

Section-4.The Significance of Eigen face approach

is explained in Section-5.Summary is briefly

explained in Section-6.Conclusions & Future work

explained in Section-7

2. Eigenface Approach:

2.1 Eigen Values and Eigen Vectors:

In linear algebra, the eigenvectors of a

linear operator are non-zero vectors which, when

operated on by the operator, result in a scalar

multiple of them. The scalar is then called the

eigenvalue (λ) [3] associated with the

eigenvector(X). Eigen vector is a vector that is

scaled by a linear transformation. It is a property of

a matrix. When a matrix acts on it, only the vector

magnitude is changed not the direction.

AX=λX (2.1)

Where Ais a Vector function.

2.2 Calculations of Eigen Values and Eigen

Vectors:

By using (2.1), we have the equation,

(A-λI)X=0 (2.2)

Where I is the n x n Identity matrix. This is a

homogeneous system of equations, and from

fundamental linear algebra, we know that a

nontrivial solution exists if and only if

det (A-λI)=0 (2.3)

Where det() denotes determinant. When evaluated,

becomes a polynomial of degree n. This is known

as the characteristic equation of A, and the

corresponding polynomial is the characteristic

polynomial. The characteristic polynomial is of

degree n. If A is n x n, then there are n solutions or

n roots of the characteristic polynomial. Thus there

are n eigenvalues of A satisfying the equation,

AX

i

=λX

i

(2.4)

Where i=1, 2, 3….n

If the eigenvalues are all distinct, there are n

associated linearly independent eigenvectors,

whose directions are unique, which span an n

dimensional Euclidean space.

Repeated Eigenvalues:

In the case where there are r repeated eigenvalues,

then a linearly independent set of n eigenvectors

exist, provided the rank of the matrix

(A-λI) (2.5)

is rank n-r . Then, the directions of the r

eigenvectors associated with the repeated

eigenvalues are not unique.

3.PCA Vs LDA:

3.1 Introduction to PCA approach:

Principal Component Analysis [8] is a

widely used technique in the fields of signal

processing, communications, control theory, image

processing, etc. In the PCA approach the

component matching relies on good data to build

eigenfaces. In other words, it builds Meigenvectors

for an N x Mmatrix. They are ordered from largest

to lowest where the largest eigenvalue if associated

with the vector that finds the most variance in the

image. An advantage of PCA to other methods is

that the 90% of the total variance is contained in 5-

10% of the dimensions. To classify an image we

want to find the eigenface with smallest Euclidean

distance from the input face. To do so we transform

the input image to a lower dimension M’ by

computing

[v_1 v_2 … v_M]^T

Where each v_i = w_i * e_i ^T,v_i is the ith

coordinate of the facial image in the new space,

which came to be the principal component. The

vectors e_i, also called eigenimages, can also be

represented as images and look like faces. These

vectors represent the classes of faces to which we

can classify a new instance of a face. Now that we

have our transformed M we can find the vector k to

which the image is the closest.

Let Omega represent the contribution of each

eigenface to the representation of an image in a

basis constructed from the eigenvectors. Then

find k such that

Epsion_k = || Omega - Omega_k|| < Theta

Where Omega_k is the vector describing

the kth face class. When Epsilon_k is less than

some threshold value Theta the new face is

classified to belong to class k.

3.2 Introduction to LDA approach:

Linear Discriminant Analysis (LDA) is a

well-known scheme for feature extraction and

dimension reduction. It has been used widely in

any applications involving high-dimensional data,

such as face recognition and image retrieval. An

intrinsic limitation of classical LDA is the so-called

Singularity problem, that is,it fails when all scatter

matrices are singular. LDA finds the vectors in the

underlying space that best discriminate among

classes.For all samples of all classes the between-

class scatter matrix SB and the within-class scatter

matrix SW are defined by:

Where Mi is the number of training samples in

class i, c is the number of distinct classes,

μ

i

is the

mean vector of samples belonging to class i and Xi

represents the set of samples belonging to class i

With xk being the k-th image of that class. SW

represents the scatter of features around the mean

of each face class and SB represents the scatter of

features around the overall mean for all face classes

3.3 Comparison of PCA and LDA:

Distinguishing characteristics of these two

techniques are as follows:

• LDA and PCA are the two popular independent

feature extraction methods. Both of them extract

features by projecting the original parameter

vectors into a new feature space through a linear

transformation matrix.

• They optimize the transformation matrix with

different intentions. PCA optimizes the

transformation matrix by finding the largest

variations in the original feature space.

• LDA pursues the largest ratio of between-class

variation and within-class variation when

projecting the original feature to a subspace.

• PCA has been used in face recognition, handprint

recognition, human-made object recognition,

industrial robotics, and mobile robotics.

• LDA has been used in face recognition and

mobile robotics. LDA has also been proposed for

generic object recognition.

The main differences between these two

methods are as follows;

LDA is based on a single face image as

input [1], [2]. That means LDA can

perform face recognition for a single input

image; hence it does not consider multiple

input images.

Where as PCA is based on multiple face

images as input. Hence it considers

multiple input images.

PCA is less sensitive where as LDA is

more sensitive.

PCA takes very less computational time

where as LDA takes more computational

time.

Fig.1 [10] describes the decision thresholds yielded

by the nearest-neighbor approach for the two cases

are marked DPCA and DLDA. As can be seen by

the manner in which the decision thresholds

intersect the ellipses corresponding to the class

distributions, PCA yields superior results.

Fig 1

Where X- axis : Class A and Y axis : Class B

Suppose there are two different classes

(Class A and Class B) embedded in two

different”Gaussian-like” distributions.However,

only two samples per class are supplied to the

learning procedure (PCA or LDA).The

classification result of the PCA procedure (using

only the first eigenvector) is more desirable than

the result of the LDA. DPCA and DLDA represent

the decision thresholds obtained by using the

nearest-neighbor classification.

Principle component analysis aims to

catch the total variation in the set of training faces,

and to explain the variation by a few variables. In

fact, observation described by a few variables is

easier to understand than one defined by a huge

amount of variables and when many faces have to

be recognized the dimensionality reduction is

important.The other main advantage of PCA is that

once you have found these patterns in the data, and

you compress the data, ie. By reducing the number

of dimensions,without much loss of information.

This technique is also used in image compression.

4. Face Recognition System:

4.1 Face space creation:

The accurate reconstruction of the

face is not required, so we can now reduce the

dimensionality to M’ instead of M. This is

done by selecting the M’ Eigenfaces which

have the largest associated Eigenvalues. These

Eigenfaces now span a M’-dimensional which

reduces computational time.

Eigen Faces

Face Space (sum of Eigen Faces)

Figure 2

Fig-2 describes about Eigen faces. [9]

In order to reconstruct the original image from

the eigenfaces, one has to build a kind of

weighted sum of all eigenfaces (Face Space).

That is, the reconstructed original image is

equal to a sum of all eigenfaces, with each

eigenface having a certain weight. This weight

specifies, to what degree the specific feature

(eigenface) is present in the original image. If

one uses all the eigenfaces extracted from

original images, one can reconstruct the

original images from the eigenfaces exactly.

But one can also use only a part of the

eigenfaces. Then the reconstructed image is an

approximation of the original image. However,

one can ensure that losses due to omitting

some of the eigenfaces can be minimized. This

happens by choosing only the most important

features (eigenfaces).

4.2 Calculation of Eigen Values:

Two algorithms called TRED2 () & QL

algorithm are used for calculating Eigen

Values. In TRED2 algorithm, Covariance

Matrix is given as a input. Here Covariance

Matrix is converted into Tri-diagonalised form

except Upper, Lower & main diagonal

elements all other elements are made zero.

Consider an example:

Where e1, e2,...en are the Lower diagonal

elements and e1’,e2’,…en’ are the Upper

diagonal elements. This algorithm uses House-

Holder Vector form which is n x m matrix

(This is a way of Tri-diagonalising the

Matrix.). It accepts only symmetric matrix as

input.

Working of QL algorithm:

According to the largest value it shifts

the elements in a matrix among themselves.

(That means Upper and Lower diagonal

elements are shifted with respect to main

diagonal elements).Hence the number of shifts

gives the EigenValues. Higher the Eigen

values higher the property of that image and

diagonal elements will be in a sorted order.

Normalization: It takes max Eigen value of the

Eigen Vector and divides each Eigen value by

max Eigen value.

4.3 Face Recognition system:

A new image T is transformed into its

eigenface components (projected into ‘face

space’) by a simple operation[11],

w

k =

u

k

T

(T - ψ)

Here k = 1, 2….M’. The weights obtained as

above form a vector Ω

T

= [w

1,

w

2,

w

3…

w

M’

]

that describes the contribution of each

eigenface in representing the input face image.

The vector may then be used in a standard

pattern recognition algorithm to find out which

of a number of predefined face class, if any,

best describes the face. The face class can be

calculated by averaging the weight vectors for

the images of one individual. The face classes

to be made depend on the classification to be

made like a face class can be made of all the

images where subject has the spectacles. With

this face class, classification can be made if the

subject has spectacles or not. The Euclidean

distance of the weight vector of the new image

from the face class weight vector can be

calculated as follows,

ε

k

= || Ω – Ω

k

||

Where Ω

k

is a vector describing the k

th

face

class.Euclidean distance formula can be found

in [2]. The face is classified as belonging to

class k when the distance ε

k

is below some

threshold value θε. Otherwise the face is

classified as unknown. Also it can be found

whether an image is a face image or not by

simply finding the squared distance between

the mean adjusted input image and its

projection onto the face space.

ε

2

= || Ф - Ф

f

||

Where Ф

f

is the face space and Ф = T

i

– Ψ is

the mean adjusted input.With this we can

classify the image as known face image,

unknown face image and not a face image.

4.3.1 Section explains the Figure 3

Reference:[9]

Figure 3

4.3.1 Procedure of Recognition system:

Eigenfaces Initialization

1. Acquire an initial set of face images (the

training set)

2. Calculate the Eigenfaces from the training set,

keeping only the M images that correspond to the

highest eigenvalues [4], [5], [6], and [7]. These

M images define the face space. As new faces

are experienced, the Eigenfaces can be updated

or recalculated

3. Calculate the corresponding distribution in M-

dimensional weight space for each known

individual, by projecting their face images onto

the “face space.”

Eigenfaces Recognition

1. Calculate a set of weights based on the input

image and the M Eigenfaces by projecting the

input image onto each of the Eigenfaces.

2. Determine if the image is a face at all by

checking to see if the image is sufficiently close to

“face space.”

3. (Optional) Update the Eigenfaces and/or weight

patterns.

4. If it is a face, classify the weight pattern as

either a known person or as unknown

To summarize whole process of Face

Recognition using Eigenface approach from the

above diagram, the training set of images are

given as input to find eigenspace. Using these

images, the average face image is computed. The

difference of these images is represented by

covariance matrix. This is used to calculate

Eigenvectors and Eigenvalues. These are the

Eigenfaces which represent various face features.

Sort the eigenvalues, and consider higher of them

since they represent maximum variations. This

becomes eigenspace spanned by the eigenfaces,

Which has lower dimension than original

images?Now given two test images are projected

onto this eigenspace to give the weight vector

also known as Face key for that image. The

Euclidean distance between these two face key

vectors is calculated. If this is below some

threshold value, then two images are said to be

matching that means they belong to same person.

Depending on this result, False

Acceptation Rate (FAR) and False Rejection

Rate (FRR) are found. These are used to change

value of Threshold. In this way Face Recognition

is carried out using Eigenface Approach.

4.3.2 Overview of the PCA algorithm:

The algorithm for the facial recognition using

eigenfaces is basically described in figure 4[7].

First, the original images of the training set are

transformed into a set of eigenfaces E.

Afterwards; the weights are calculated for each

image of the training set and stored in the set W.

Upon observing an unknown image X, the

weights are calculated for that particular image

and stored in the vector W

X

. Afterwards,W

X

is

compared with the weights of images, of which

one knows for certain that they are facing (the

weights of the training set W). One way to do it

would be to regard each weight vector as a point

in space and calculate an average

distance D between the weight vectors

fromW

X

and the weight vector of the unknown

image W

X

.If this average distance exceeds some

threshold value, then the weight vector of the

unknown image W

X

lies too ”far apart” from the

weights of the faces. In this case, the

unknown X is considered to not a face. Otherwise

(if X is actually a face),its weight vector W

X

is

stored for later classification. The optimal

threshold value has to be determined

empirically.

By means of PCA one can transform

each original image of the training set into a

corresponding eigenface. An important feature of

PCA is that one can reconstruct any original

image from the training set by combining the

eigenfaces. Remember that eigenfaces are

nothing less than characteristic features of the

faces. Therefore one could say that the original

face image can be reconstructed from eigenfaces

if one adds up all the eigenfaces (features) in the

right proportion. Each eigenface represents only

certain features of the face, which may or may not

be present in the original image. If the feature is

present in the original image to a higher degree, the

share of the corresponding eigenface in the” sum”

of the eigenfaces should be greater. If, contrary, the

particular feature is not (or almost not) present in

the original image, then the corresponding

eigenface should contribute a smaller (or not at all)

part to the sum of eigenfaces. So, in order to

reconstruct the original image from the eigenfaces,

one has to build a kind of weighted sum of all

eigenfaces. That is, the reconstructed original

image is equal to a sum of all eigenfaces, with each

eigenface having a certain weight. This weight

specifies, to what degree the specific feature

(eigenface) is present in the original image.

Figure 4

5. Significance of Eigenface Approach:

We have seen the standard Eigenface

Approach for Face Recognition. Here we find

out M Eigenvectors for representing training set

images. Now it is important to choose only M'

Eigenvectors from these M Eigenvectors, such

that M' is less than M, to represent face space

spanned by images. This will reduce the face

space dimensionality and enhance speed for face

recognition.Here we are reducing the

dimensionality of face images.

We can choose only M' Eigenvectors

with highest Eigenvalues. Now as the higher

Eigenvalues represent maximum face variation in

the corresponding Eigenvector direction, it is

important to consider this Eigenvector for face

space representation.Since, the lower

Eigenvalues does not provide much information

about face variations in corresponding

Eigenvector direction, such small Eigenvalues

can be neglected to further reduce the dimension

of face space.This does not affect the success

rate much and is acceptable depending on the

application of face recognition. Here only M'

Eigenvectors with higher Eigenvalues are chosen

for defining face space. The results of choosing

various values for M' are given in Results

section. The most important point here is to

choose value of M'(Maximum Eigen values

chosen from Co-variance Matrix) so that it does

not result in high error rates for Face recognition

process. The M' should be chosen such that error

rates does not increase much and are acceptable

depending on application for face recognition.

6. Summary

In this paper, the dimensionality

problems are solved for face recognition. The

approach using Eigenfaces and PCA is quite

robust in the treatment of face images with

varied facial expressions as well as the

directions. It is also quite efficient and simple in

the training and recognition stages, dispensing

low level processing to verify the facial geometry

or the distances between the facial organs and

their dimensions. However, this approach is

sensitive to images with uncontrolled

illumination conditions. One of the limitations

for eigenface approach is in the treatment of face

images with varied facial expressions and with

glasses.

7. Conclusion and Future works:

The Eigenface approach for Face Recognition

process is fast and simple which works well

under constrained environment. It is one of the

best practical solutions for the problem of face

recognition. Many applications which require

face recognition do not require perfect

identification but just low error rate. So instead of

searching large database of faces, it is better to

give small set of likely matches. By using

Eigenface approach, this small set of likely

matches for given images can be easily obtained.

For given set of images, due to high

dimensionality of images, the space spanned is

very large. But in reality, all these images are

closely related and actually span a lower

dimensional space. By using eigenface approach,

we try to reduce this dimensionality. The

eigenfaces are the eigenvectors of covariance

matrix representing the image space. The lower the

dimensionality of this image space, the easier it

would be for face recognition. Any new image can

be expressed as linear combination of these

eigenfaces. This makes it easier to match any two

images and thus face recognition. We have also

seen that taking eigenvectors with higher M'

eigenvalues instead of all M eigenvectors, does not

affect performance much. So even taking lower

dimensional eigenspace for the images is

acceptable as error rates in such cases are very low.

The important part is making this choice of M'

which will be crucial depending on type of

application and error rate acceptable. More

research needs to be done on choosing the best

value of M'. This value of M' may vary depending

on the application of Face Recognition. So various

methods for making best choice of M' needs to be

studied.Further, this approach can be extended to

detect age, facial expressions of a person.

8. References

[1]”Discriminative learning and Recognition of

image set classes using Canonical correlations” by

Tae-Kyum Kim, Josef Kittler, Roberto Cipolla

(IEEE members).

[2] “Intelligent Face Recognition techniques: A

Comparative Study” by S.T Gandhe, K T Talele, A

G Keskar.

[3]Wikipedia

http://en.wikipedia.org/wiki/Eigenvalue,_eigenvect

or_and_eigenspace

[4] “Recognition of Non-symmetric Faces Using

Principal Component Analysis” by N Krishnan,

GR Jothi, K Pulloor, GL Aaron

[5] M.A.Turk and A.P.Pentland, “Face

Recognition Using Eigenfaces”, Proc. Of IEEE

Conf. on Computer Vision and Pattern

Recognition, pp. 586-591, June 1991.

[6] Matthew Turk and Alex Pentland, “Eigenfaces

for Recognition”, Vision and Modeling Group,

The Media Laboratory, Massachusetts Institute of

Technology, In J. of cognitive

neuroscience,1991, vol.3,no.1, pages 71 to 86.

[7] “Eigenface-based facial recognition” by

Dimitri PISSARENKO

[8] “Enhanced Principal Component Analysis

Recognition Performance” by Sezin Kaymak

[9] Nick Pears Thomas Heseltine and Jim

Austin. “ Evaluation of image preprocessing

techniques for eigenface based face

recognition”. ACA Group, Dept. of Computer

Science, University of York, 2002

.

[10] A.M. Martinez and A.C. Kak, “PCA versus

LDA,” IEEE Trans. Pattern Analysis and

Machine Intelligence, vol. 23, no. 2, pp. 228–

233, Feb. 2001

[11] Turk and A.Pent land. Eigenfaces for

recognition.Journal of Cognitive Neuroscience,

3(1)1991a.http://www.cs.ucsb.edu/mturk/Paper

s/jcn.pdf.

## Comments 0

Log in to post a comment