Face Recognition using Backpropagation Neural Network on Unbalanced Priors

crumcasteAI and Robotics

Nov 17, 2013 (3 years and 4 months ago)

105 views

Face Recognition using
Backpropagation Neural Network on
Unbalanced Priors


Submitted By:

ADEEL RAZA

MS (IT)

Session 2006
-
08









A thesis submitted in partial fulfillment of the requirement for

The degree of Master of Science in information technol
ogy

Artificial Intelligence

Department of Computer Science

Institute of Management Sciences

Peshawar, Pakistan

January 24, 2011

Face Recognition using Backpropagation Neural
Network on Unbalanced Priors





This research dissertation is submitted in parti
al fulfillment of requirements for the
degree of MS (IT) in Computer Science to the Department of Computer Science,
Institute of Management Sciences as on
January 24, 2011
.





Research Scholar:


















A
deel Raza

Supervisor:



















S
ir
A
wais

A
dnan


Head of Computer Science Department:
















Dr. Ali

Face Recognition using Backpropagation neural network
on Unbalanced Priors



It is certified that we have studied this final dissertation by Adeel Raza in detail. We
conclude that

this dissertation is of sufficient standard to warrant its acceptance by the
Institute of Management Sciences for the award of the degree of

MS (IT) in Computer Science.



External Examiner 1


Signature:










Name:











Designation:










Institute:


















External Examiner 2


Signature:










Name:











Designation:










Institute:










Internal Examiner


Signature:










Name:











Designation:










Institute:




















I dedicate
this thesis to my great inspirer and mentor, Sir
Awais

Adnan, Who always encourage us to
complete our thesis and give us ideas and was always there for us when we needed his help. May
GOD
give me the same unrelenting attitude to help others in need.






















Acknowledgements

I acknowledge the help of Greg Helth, who helped me in coding. He helped me in
understanding how those neural networks mechanism works. And giving me time to
time advice on what should I purse and what should I leave.

I ac
knowledge the time and time again help of Sir
Awais

Adnan, who helped me in
whenever I got stuck in programming. Plus there always positive attitude and a deep
insight help me (and countless other) in pursuing and completing their thesis. With
out there he
lp it would be very hard for me to complete my thesis
Table of Contents


Face Recognition using Backpropagation Neural Network on Unbalanced Priors


vi


Table of Contents

Table of Contents

................................
................................
................................
..........

vi

List of Tables

................................
................................
................................
..............
viii

List of Figures

................................
................................
................................
...............

ix

Abbreviations

................................
................................
................................
...............

10

Chapter 1

................................
................................
................................
........................

1

Introduction

................................
................................
................................
....................

1

Chapter 2

................................
................................
................................
......................

11

Background

................................
................................
................................
..................

11

2.1.1

Digital image processing (DIP)

................................
................................
.

13

2.1.2

Signal gathering and preprocessing

................................
...........................

15

2.1.3

Processing

................................
................................
................................
..

15

2.1.4

Actuators

................................
................................
................................
....

17

2.2

Biometric

................................
................................
................................
.......

17

2.2.1

Identification

................................
................................
..............................

19

2.2.2

Verification

................................
................................
................................

19

2.3

Approaches

................................
................................
................................
....

20

2.3.1

Holistic based face recognition

................................
..............................

20

2.3.2

Local based face recognition

................................
................................
.

20

Chapter 3

................................
................................
................................
......................

22

Literature Review
................................
................................
................................
.........

22

3.1.

Neural network

................................
................................
..............................

29

Table of Contents


Face Recognition using Backpropagation Neural Network on Unbalanced Priors


vii

Chapter 4

................................
................................
................................
......................

32

Problem Definition
................................
................................
................................
.......

32

Chapter 5

................................
................................
................................
......................

35

Face Recognition

................................
................................
................................
.........

35

5.1.

Preprocessing

................................
................................
................................

38

5.2.

Network configuration

................................
................................
..................

40

Testing
................................
................................
................................
..................

41

Chapter 6

................................
................................
................................
......................

42

Data and Results

................................
................................
................................
..........

42

6.1.

Data

................................
................................
................................
...............

42

6.2.

Results

................................
................................
................................
...........

46

Chapter 7

................................
................................
................................
......................

47

Conclusions

................................
................................
................................
..................

47

References

................................
................................
................................
....................

52

Annexure A

................................
................................
................................
..................

57

FERET

................................
................................
................................
.....................

57

FRVT

................................
................................
................................
.......................

58

FRGC

................................
................................
................................
.......................

58

The Color FERET Database

................................
................................
....................

59

Annexure B

................................
................................
................................
..................

60

Annexure C

................................
................................
................................
..................

62

Computational complexity

................................
................................
.......................

62

Snapshots of Operating system using resources during Training

........................

63

Copyrights Notice

................................
................................
................................
........

66



List of Tables


Face Recognition using Backpropagation Neural Network on Unbalanced Priors


viii


List of Tables

Table 6.1

................................
................................
................................
......................

46



List of Figures


Face Recognition using Backpropagation Neural Network on Unbalanced Priors


ix


List of
F
igures

Figure 6.1

................................
................................
................................
.....................

43

Figure 6.2

................................
................................
................................
.....................

44


Figure B.1

................................
................................
................................
....................

60

Figure B.2

................................
................................
................................
....................

60

Figure B.3

................................
................................
................................
....................

61

Figure B.4

................................
................................
....

Error! Bookmark not defined.

Figure B.5

................................
................................
................................
....................

61


Figure C.1

................................
................................
................................
....................

64

Figure C.2

................................
................................
................................
....................

64

Figure C.3

................................
................................
................................
....................

65

Figure C.4

................................
................................
................................
....................

65




Abbreviations


AI

Artificial intelligence



BMP

Windows Bitmap



BioID

Biometric technology that uses

face,
voice and lip movement to identify a
person



DIP

Digital image processing



JPG, JPEG

Joint Photographic Experts Group



NN

Neural Network



PCA

Principal Component Analysis



ICA

Independent Component Analysis



TIF, TIFF

Tagged Image
File Format



FERET

Face Recognition Technology



FRVT

Face Recognition Vendor Test



FRGC

Face Recognition Grand Challenge



Face database

A collection of face pictures.






Glossary

Nondeterministic polynomial

A problem is called NP (nondetermi
nistic polynomial) if its solution can be guessed
and verified in polynomial time; nondeterministic means that no particular rule is
followed to make the guess
1
.

NP
-
complete

If a problem is NP and all other NP problems are polynomial
-
time reducible to it,

the
problem is
NP
-
complete
. Thus, finding an efficient algorithm for any
NP
-
complete

problem implies that an efficient algorithm can be found for all such problems, since
any problem belonging to this class can be recast into any other member of the class
1
.





1

Definition
taken from Britannica encyclopedia.








Abstract


Biometric is increasing becoming the norm as the security requirement in the
organizations
. Previously methods used for
authorization
s are slowly and gradually
becoming obsolete i.e. keeping manual track of authorized personnel,
usern
ame/password etc.
In a nutshell any person can be uniquely identified by his/her
face.
As with all the technologies that had not had matured yet, face recognition had
still had a long way to go, but the benefit can still be felt. This evolution process is
usually derived by people trying to break in the system and developer constantly
improving the system against all the apparent and hidden dangers. My thesis on face
recognition is also a trip in to biometric authentication and trying to analyze the pros
an
d cons of it by developing the system from the ground up.



Introduction

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

1



Chapter 1

Introduction

Real life phenomena involve usually multidimensional facets of data. And they almost
always grow exponential in size as the numbers of dimensions are increased. All thos
e
problems whose input size grows exponentially and whose solution cannot be find in
polynomial time are called NPC
[1]

problems.
S
ecurity systems success directly
depends on the need to identify a person correctly and biometric technology is used
for it.
B
iometrics technologies
are
no
w
-
a
-
days
not just used
for security reasons
,

but
are also
used to better facilitate the customers
, for example;

banks. Biometric include
finger print (dactylogram), iris identification or iris
-
scan, retina identification, face
recognition, voice recognitio
n, DNA sequence matching, signature recognition,
psychological profile, palm recognition.

In a nutshell the theory behind face recognition is that we can uniquely identify any
person by his/her face. This concept although would seem absurd at first, just l
ike
when the hypothesis was presented that we can uniquely identify any person through
his/her finger print. Later research proved that point. The same is proved for face
recognition
[2]
.

Face recognition problem
is a subcategory of pattern classification. In pattern
classification problem we extract features from the sample and then tries to match
against the templates records already stored with us. Face recognition is basically a
pattern classification problem o
f the extracted feature points to classes i.e. persons
[3]
.
This can be done by either of the two approaches i.e. holistically or locally. In holistic
approach the whole face is used for face recogniti
on where as in local base
d face

recognition some distinct numbers of features on the face are recorded (manually or
through feature extraction module(s)). This data is then processes usually either
through the neural network or some statistical method
[3]

is used for the recognition
process.

Introduction

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

2


Face recognition is a biometric authentication
method
[4]
. Authentication methods
either involve identification or verification. In identification we had to search for a
pattern in whole of the patterns, where as in verification we had a sample with a
claim
of been same to a pattern already in the repository and we had to verify that (as
already explained in detailed in the biometric chapter of this thesis). So we can define
the face recognition problem as to correctly classify a face from a scene, a mo
vie
(sequence of scenes), or from a number of still pictures. The most applied biometric is
face recognition
[5]
.

Real life facial recognition, even if we takes only fixed number of still mug
2

shot,
with a constant un clustered backgr
ound. We still end up with a infinite number of
permutation of mug shot that differ in pose, expression, illumination and translated
shot (in comparison to each others). Formal languages can
not

handle them because
even a minute amount of change would jeopa
rdize the result, non formal languages
i.e. Artificial intelligence (AI) can give us an approximated result at best. (Well if we
eliminate pose, expression, illumination and translation changes e.g. Visa Security
system we can get 90% correct result). In o
ther circumstances where we have such
restrictions, we had to find an approximation algorithm for it. And this approximation
algorithm could be inspired by none other than how human recognizes faces i.e. how
ours’ own neural network mechanism works. Althou
gh a lot of people now studies
face recognition from various domains e.g. psychologist and neuroscientists studies it
from human cognitive perspective where as a computer engineers studies it from
computational perspective
[3]
. Because this problem is NPC, we knows that we would
not be able to find a optimal solution to this problem any time soon
[1]
, so it is better
to search for an appr
oximation algorithm, that would give satisfactory results.

Face recognition covers both the authentication methods i.e. identification and the
verification. In identification we had to search for a pattern in whole of the patterns,
where as in verificatio
n we had a sample with a claim of been same to a pattern
already in the repository and we had to verify that (as already explained in detailed in
the biometric chapter of this thesis). So we can define the face recognition problem as
to correctly classify
a face from a scene, a movie (sequence of scenes), or from a
number of still pictures.

Human had no problem in authenticating a person which he / she knows already or
even if he / she had seen that person once or twice. Human are excellent in identifying
a

person in various situations, involving varying degree of fluctuations in parameters
i.e. pose variation, illumination variations, aging variation, translation variations and
rotation variations. Plus we seem to have no difficulty in identifying the face
area i.e.



2

Pictures of Face

Introduction

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

3


where is the face. So the best approach for a computer engineer is to copy or follow
the already present best workable model (which by the way is flawless if we compare
it to all the computerized face recognition algorithms). It should be remembe
red that
while copying the human physiological mechanism, computer engineers
3

don’t
always obey all the details how a human physiology i.e. neural network does the work
done
[6]
, instead they only follows the general mechanism (although following
exactly should give bette
r results, but because of time and resources limitations
compromises had to be made).

Human recognizes faces from not just the 2D image but they use a whole array of
senses to correctly identify a person for example; we use auditory, tangoreception
(touch
reception), olfactory etc senses, plus we also use the environment to memorize
and recall. To implement this in a computer system, the software will become gigantic
and any meaningful result couldn’t than be gained through that software.

Face recognition c
annot be done through procedural programming on the still, raw
data
[7]

because of the immense amount of data involved. Plus mostly face
recognition done for the securi
ty system includes processing on the real time data.
And processing on that real time raw data is even an impossible task for a grid of
super computers. So mostly some form of features must be first be extracted from the
images and then recognition process

can take place.




3

Computer engineers could be software engineers or microcontroller engineers

Background

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

11



Chapter
2

Background

Any computer program / system can be build either in declaratively or imperatively
(Procedurally)
[
8, 9]
. Imperative program does not had any feedback and therefore
they never learn (from experience), declarative algorithm could learn but it is mostly
impossible of encode the real life phenomenon in the form of rules and facts (although
we could make a
n approximation of the environment and its’ contents). But it fails
when we had to encode people identification through there biometric features. Mostly
in those cases we can use a different approach in which we had to create a statistical
model (
[10]

even put neural network in statistical learning methods) of the system.
Examples are neural networks or statistical models i.e. PCA or ICA models
.

In all the programming (whether it is declarative of any order i.e. first, second or
higher order) or is imperative of any degree asymptotically i.e. first, second or higher
degree complexity or statistical approach of neural network i.e. one layer, two
layer
etc), we change the variables according to some logic. Of course these approaches
differ tremendously, In imperative approach and declarative approach the numbers of
variables are not fixed where as in statistical approach the numbers of variables ar
e
fixed. Also the problem solving logic used in imperative approach is designed
specifically to that problem (or that set of problem). Declarative approach is a little
better because inferencing is a general approach and can guarantee the solution (if the
all the necessary fact are given
[9]
). But the imperative approaches and in some sense
the declarative approach had one major problem i.e. generality. We cannot fit one
solution to any

problem; the real life can through at us. Whereas neural network can
Background

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

12


be used to solve any kind of problem(s). Of course we are talking theoretically here
Turing machine
4
.



If we consider hardware programming
5
, formal programming
6
, logical programming
7

and neural network
8
, and then if we plot on one axis the time and on the other the
complexity
9

. We would get a curve. Just like in natural world some creatures like
herbivorous can stand and start running in just a few hours after their birth. But on the

other side the carnivorous had to take a long time before they start running
[12]
. And
still yet human ha
d to take the longest time and resources to start. But on the other
hand human can then solve the most extreme of the complex problem. In the same
way neural network program takes a lot of resources and time to start producing
useful output but on the othe
r hand they can give meaningful output even in those
situations where input is incomplete and/or partly incorrect. It is proven that formal
language program can be create far more earlier than neural network
[6]

but they can
only tackle a set of situations and would give
a incorrect results in all the other. While
neural network would take longer to train but would generally give correct output.

The basic flaw in procedural approach is that we had to be very specific and should
cater every situation and even a very small c
hange would cause it to malfunction
while a declarative approach will function in quiet a lot of noise
[9]
. The disadvantage
of declarative approach is that because it work on symbols,

it would be very slow for
real life applications because of large number of symbols. Statistical learning method
has

the

disadvantage

that

they

takes

a

lot

of

time

in

training

(identifying/




4

“Hypothetical computing device introduced in 1936 by the English mathematician and logician Alan
M. Turing. Turing origin
ally conceived the machine as a mathematical tool that could infallibly
recognize undecidable propositions

i.e., those mathematical statements that, within a given formal
axiom system, cannot be shown to be either true or false. (The mathematician Kurt Göd
el had
demonstrated that such undecidable propositions exist in any system powerful enough to contain
arithmetic.) Turing instead proved that there can never exist any universal algorithmic method for
determining whether a proposition is undecidable.”
11.

Encyclopædia

Britannica
, in
Encyclopædia Britannica 2006 Ultimate Reference Suite DVD
, P. Ginnis, Editor. 2006.

5

IC based, with or with the help of hardware programming language.

6

All the languages based on context free grammer i.e. c, c++, java etc.

7

Based on log
ic i.e. prepositional, first order higher order logic i.e. prolog, lisp etc.

8

Using a statistical model to change the values of variables gradually so that it reflect the value of that
variable for that instance by changing it accordingly to the whole pop
ulation or to the samples.

9

Complexity mean both the level of complexity (
algorithmically
) and the number of variables
involved.

Background

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

13


remembering) but once training is complete,

the testing can be done in real time.
Artificial intelligence in general and neural network in particular works in this respect
way. We as human are very efficient in recognizing complex patterns in visually or
acoustically etc, but behind this amazing ac
complishment there are a lot of trails and
error process had gone into training, so that we got to this stage. To comprehend this
process and then to implement the same on a computer is a monumental task. But
fortunately decade of research had gone in this

process and now we had working
model to go on
[6, 7]
. This research had two prompt benefits first we can recognize
and appreciate the human anatomically process more precisely and secondly by
implementing those on antonymous machines we can elevat
e the human from those
tedious jobs involving those fields i.e. pattern recognition.

2.1.1

Digital image processing (DIP)

“The field of digital image processing refers to processing digital images by means of
a digital computer. …. One useful paradigm is to c
onsider three types of
computerized processes in this continuum: low
-
,mid
-
, and high
-
level processes”
[13]
.
Low level characteristic is that the input and output both are images, mid
-
level
processes characteri
stics is that the inputs are images but the outputs are attributes
extracted from the images while high level processes characteristics is that while the
input can be images but the output is making sense of the them
[14]
. Thus, in face
recognition we want to use DIP to extract objects (features vector (record set)) from
the raw pixel data. Mostly high level DIP processes is an amalgam of various fields
(domains) i.e. com
puter vision and artificial intelligence.
[14]
. The goal of DIP is
enhancement, restoration, compression, segmentation, feature extraction and object
recognition.
[14]

Object recognition as such doesn’t totally lies in the DIP domain but
is usually dominated by other artificial intelligence field, mostly using DIP to get the
extract dat
a from the raw data.

Basically DIP usually operates in two domains i.e. spatial domain or frequency
domain.
[14]
. Or more generally, it can be divided into time domain

(i.e. in case of
inclusion of movies) or frequency domain. Both those domain had been used in face
Background

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

14


recognition applications. Both the frequency and spatial domain is used for
enhancement, restoration, compression and segmentation of faces. For example “T
he
low spatial frequency components are used to clarify the sex information of the
individual whereas high frequency components are used to identify the individual”
[3]
.
Restoration, histogram equalization and segmentation are all used just to extract the
feature data from images. Image enhancement is not very specific (scientifically)
because enhancement is h
ighly depended on the user preferences and they change
from one user perceptive to another user. These include either smoothing or
sharpening. Restoration involves deduction of noise from the image(s) by using a
noise model. Image compression is based on i
nformation theory, and can be either
error
-
free or lossy. Morphological and segmentation belongs to the mid
-
level
processes. Morphological image processing mean to extract image component that are
useful in the representation and description of region shap
e, such as boundaries,
skeletons, and convex hull
[14]
. Image segmentation concerns with either edge or
region extraction through using either sharpening or smoothing fi
lters.


There is a growing interest in the commercial and governmental intuitions for
the identification i.e. persons or things etc for security, allowing access to resource
etc. usually the implementation of those system mean some sort of camera e.g. web
cam for gathering the audio
-
visual signal, some sort of processing system usually a
computer system and some sort of actuator system usually a computer GUI displaying
the results. So naturally therefore we can divide this process is divided broadly into
th
ree modules. They are as followed.

1.

Signal gathering and preprocessing

2.

Processing the information

3.

Actuators


Background

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

15


2.1.2

Signal gathering and preprocessing

The preprocessing stage
10

of the system in face recognition involves somehow
extracting the features from the bi
nary pixel data. There are a lot of digital image
processing method are used to this end. Usually this involves extracting the edges in
the picture, through the first or second order differentiation of th
e pixel intensity
surface i.e. G
arssion and Laplace
functions. And alternative is to use the similarity
based region growing extraction. Then the discontinuities in the edges smoothen
through the hafman (edge) filter. Then those edges are compared to and selected
against the required edges. Then all those
edges are grouped together into objects.
Usually morphological filters are used on the objects and then features from those
objects are extracted.

2.1.3

Processing

Processing is the heart of all the system. The simplest processing would be to
compare the record

i.e. features vector with the records in the database. But this
approach would not work in real life face recognition applications because for this to
work we had accord for all settle and minute difference that could occur because of
illumination, transl
ation, scale and rotation. Which is an impossible task because there
are infinite number of those changes that could happen. But suppose if we could
consider that some how this could be possible (by setting a limit on changes to be
incorporated) then it wo
uld be an impossible task from an computational standpoint,
for any computer to compared an input record with the infiniate number of records in
the database. This concludes that any formal language i.e. imperative language system
could never be able to so
lve this problem.

So the only other option is to use an declarative language system for the solution of
this problem. Declarative language or symbolic system is very good at catering the
infinite subtle differences dilemma but the dark side is how can we c
reate an symbolic
rule or fact that would incorporate a class (in our case the person face identifying
features).




10

I will explain from now on those stages with respect to face recognition.

Background

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

16


So perhaps the solution is surprisingly in front of our eyes i.e. how do we identify a
person. So if we some how first identify and then cop
y the same system on the
computer. Thus I used neural network specifically the Backpropagation neural
network.

In central nerves system, neurons got its’ inputs from (usually) many other neurons
when those nerves signal comes to the junction of those neur
on i.e. from the inputs
neuron to the neuron we are considering, its changes from electromagnetic signal to
chemical signal. And then back to electromagnetic signal in the consider neuron. This
conversation of signal depends on how efficiently converts fro
m one form to the other
and back the previous one. If the interaction between those neuron are frequent
enough then this transfer became more efficient i.e. some changes in the chemical
make of the input neuron or the receiving neuron or both changes, such

that the
transfer become more efficient. This same mechanism is copied in the entire
computerized neuron network. The receiving neuron receives all the input from all the
input neuron this way and they all add up in strength in the received neuron and if
the
strength in big enough (to some threshold value) then this neuron fires up and send
signal down to the neurons that it is attached with. This activation mechanism is
different in different part of the brain. The same is copied in computerized neural
ne
twork and the activation function are also different and depends upon the process /
problem that this network is constructed for.


Neural network is implemented by applying the input to a bunch of neurons and they
are multiplied with weights individually
. Then all those multiplicands are summed up
and applied to the activation function. Then as according to the function its fires up
i.e. produces and high output if the summed values is above some threshold value.

Backpropagation neural network is differen
t from others neural network in some
majors ways. These are

1.

In Backpropagation network (as the name suggests) an error value is calculated. i.e.
the neural network output and the actual target output is calculated and this error is
propagated to the layer

of neuron back the neuron form which this output was
produced, those neuron then trie4s to corrects its calculated outputs values and if there
is a layer of neuron behind this layer those are also propagated the error, but this error
Background

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

17


is not the error prop
agated from the final output layer but instead had been calculated
on that layer (one layer behind the final output layer).

2.

Backpropagation neural network work on supervised learning mode.

3.

Backpropagation neural network is a general purpose algorithm and i
t can be used for
any sort of problem.


2.1.4

Actuators

Actuators can do a lot of activities; depending upon the nature of the application it
usually includes giving access to resources such as computer or an organizational
building facility. In our case the act
uator simple mean a GUI showing the results i.e. a
couples of plots and a message box showing the percentage of correct matches in
record set.

2.2

Biometric

Biometric is the field through in which we can authenticate a person through his/her
biometric features

i.e. these mostly include fingerprint, iris
-
scan, voice recognition
and face recognition. “When a user is authenticated the person concerned lets the
system know his identity. There are two modes for authentication; verification and
identification. In ad
dition D.M. Blackburn adds another task to a biometric system;
the watchlist”
[4]
. Thi
s authentication includes the person psychological and
physiological features of that person, though which we could uniquely identify a
person. Biometrics included a lot of disciplines, these includes the following



Iris identification or Iris
-
scan



Retina i
dentification



Face recognition



Finger print (dactylogram)

Background

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

18




Voice recognition



DNA sequence matching



Signature recognition



Psychological profile



Palm recognition

Although if used alone none of these
fields could guarantee to

correctly authenticate a
person (b
ecause of the noise incorporated by the scanner device, medium and formats
used),
especially if we are talking about real time. T
he odd are much better if we use
more than one biometric feature
[4]
. In all of those fields of biometric domains
progress had been made but none give full satisfactory results in high security
environment and com
promises had to be made in between correct authentication and
resources (processor time, memory). For example finger print can be copied, they
could be cut or bruised
[4]
, face recognition softwares cannot distinguished between
twin (although
[15]

had claim that there 3d authentication software can distinguished
between twin). Even when external human supervision is implemented, Human face
recognition is a function of hair
[4]
, this mean that if hair are altered, for example
head hairs are removed, eye brows are plucked or the bread is shaved; woul
d
drastically change the human face recognition rate.

Instead if we use two or more biometric features data than the deficiencies/ drawback
of one is compensated by the other. For example face recognition and finger printing
can be used, so that those per
sons whom had no hands or had their finger print
smudged (cut and bruised) because of some reason, than face recognition can some
-
what compensate that. Or face recognition can be used to narrow down the
possibilities and then on those selected one finger p
rinting be used for final
identification
[4]
.


Authentication can be done in three wa
ys
[3]
.

1.

Identification

2.

Verification

Background

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

19


Although
[16]

included a third category besides those two, the
watchlist
.
But I
think that

if we look closely this third category is actually also a

different
version of the first.
[4]

said that in “The watchlist task: in the watchlis
t task the
person does not claim any identity. The biometric sample of the individual is
compared with the stored samples in a watchlist to see if the individual
concerned is present in the watchlist”

2.2.1

Identification

Identification is defined as to search f
or a person in a repository. Mostly in
identification we had one picture of the person and we want to know who he is
(one
-
to
-
many search). For example to search for a person through his/her
picture in the NIC (national identity card) database. Missing pers
on (john doe)
[4]

identification in hospitals and police departments.

In identificati
on we usually had no time limitation on the execution time of the
algorithm i.e. face recognition software. We don’t mind if the results came out
after an hour or after a day, in fact after a week or if matched results are one or
more (although one result
in more appreciated). Because usually in those
situations, we had only one picture in the database per person and we had a lot
of them to be matched against our given input picture, so we gave a threshold
value and the algorithm returns all those records t
hat are above that threshold,
which means all those person are good enough match compared to our input
image.

2.2.2

Verification

In verification, a person claims that he/she is a authenticated person and we
had to match his/her data against the acclaimed authent
icated person data
already stored in the repository (e.g. a database or a weight matrix). Which
mean that we had search one
-
to
-
one search
[4]

situation.

Background

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

20


2.3

Approaches

There are broadly two approaches used in face recognition.
[3]

2.3.1

Holistic based face recognition

In holistic based face recognition, instead of taking only some local data, the
whole face is used for face recognition. This may include a simple template
matching or more sophisticated

method such as PCA based method to extract
the principle components for the huge set (cloud) and reduced it to a small set
i.e. principle components which are the maximum deviations from normality.
In the template matching we just match the pixel data (in
tensity values) of the
template with the input picture for face recognition. Although template
matching can give the highest accuracy if it is implemented in highly
controlled environment but it is highly susceptible to even the smallest
fluctuation in any

of the various parameter i.e. pose, illumination, rotation,
resolution, aging, expression etc.

Human recognizes faces through both holistic and feature approach. We
cannot remember faces that are common i.e. they are neither attractive nor
very unattracti
ve. Plus we cannot remember those faces whose intensities are
inverted or those faces who are illuminated from bottom up. We remember
mostly those faces that are illuminated from top
[3]
. Plus any change in the
hair change in our face recognition
[4]
.

2.3.2

Local based face recognition

In local based faced recognition some number of distinct feature on the face
are recorded (manually or through feature extraction module(s)). This data is
then proces
ses usually either through the neural network or some statistical
method
[3]

is used for the recognition process. The same is true for other form
of recognitions i.e. voice recognition, hand signal, gesture r
ecognition etc.

Those features used in local based face recognition are the nose position,
nostril position, eye brows positions, eyes position, mouth position and chin
position etc. Those features are then given to a statistical or neural network
(usually

Backpropagation) for training and testing.

Background

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

21


My thesis focuses on this method. An instance of those features i.e. a feature
record of a person can be thought of as a point in an n
-
dimensional
11

space.
And if the features are selected properly then all those
features for a specific
class will occupy a region in that n
-
dimensional space that would not overlap
with any other class region. And their region boundaries could be extracted
(though some algorithm, in our case (Backpropagation) neural network), for
dec
isional proposes (identification or verification).

Both those method had to deal with one major problem, which actually occurs
for all those systems that had to deal with natural occurring signal, especially
for audio and visual signals that we capture in
real time through some
capturing device, gives us a huge amount of data. This raw form could not be
used for any meaningful application (except when the input is very small i.e.
either the picture is very low resolution and size or when we are dealing with

toy problem
12
). So we had to implement some sort of feature extraction
module, in ours’ case recording the local features or extracting the principle
components. I had done my thesis by using feature points on the Bioid
13

face
database.




11

n means the number of inputs

12

Toy problem are t
hose problems with are very specialized and high simplified form of problems.
And they does not occur in real life.

13

Bioid is freely available for download. You can download it from
http://www.bioid.com/downloads/facedb/downloads/BioID
-
FaceDatabase
-
V1.2.z
ip

Literature Review

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

22



Chapter
3

Litera
ture Review

Decades of research had now gone into face recognition. There are now even
commercial products available in the market for face recognition. There are a lot of
researches done on the 2D face recognition algorithm. A lot of different
methodologi
es had been used to find an optimal solution. The easiest of them include
memorizing pixel data and comparing against it as a model. Obviously even a
smallest of
fluctuation

or change would cause that system to malfunction. There a
neural network been used

(which I had used also), there are than the statistical method
used e.g. Eigen function.

Human face recognition is a function of hair. This was pro
ven by the research done by
[4]

in which he proves by taking a sample pictures. First he removes the eyes from the
picture and in the second he removes the eye brows from the eye. We humans can
recognize the face without the eyes but in case of hair removed (i.e. eye brows) we
could not recognize the faces.

Although computerized face recogniti
on softwares are not affected by

hairs. But in
those there are a lot others problems that could jeopardi
ze the accuracy.
E.g.

if we are
assuming the simple comparing of faces. Even a small light intensities change would
throw the accuracy. (Although usually we had to counter act by taking the histogram
equalization of the pictures to guard against this conce
rn.

But obviously this pixel by pixel comparison is impossible if the number of searches
would increase even slightly. Well because this algorithm is actually a brute force
Literature Review

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

23


algorithm and would increase exponentially as the number of samples increase. So
only if the samples a very small. I.e. in ten to 30 it might work if the face did
not
had
any occlusion, expressions, scale, and translation and intensities changes. (An
impossible task).

As you would had guessed by now, as with all the brute force algorit
hms we need
heuristics to get a solution. And all the successful and commercial product does use
them (heuristics). Instead of straight forwards searching for a comparison.

Well for starter, we first need to somehow isolate the face region and then somehow

need to get face features.

This make sense, think about it, we know where face is in the pictures, then we are
had better odds of comparing against/for the required face. Even if w
e would had to
perform brute fo
r
c
e algorithm. The second thing is about t
he face features, what are
face features , there are those points on the face that are invariant to changes,

as
explained by
[2]
.

Because those features do not change it makes perfect sense then to use only those

points in the search. E.g. the location of the eyes, the tip of the nose, the location of
the mouth etc. well first we had reduced thousand to points (pi
xels) to just 20 to 30
points pe
r picture. As
[2]

had done
in
his

research.

If we re
strict ourselves only to the features set
[2]

had done. Plus there are actually
now from US government official website to help the face recognition research and
for the commercial develo
pers, where they had put face databases (a collection of face
pictures), plus not only had they given face pictures that are pictures against a
constant background as well as the scale of them, the light intensities of those pictures
are all done in a prof
essional manner. I.e. they are a taken almost at a constant (with
minor variations to account for the natural variations that occur in reality). This at first
does not look much help but when you are talking about hundreds of different person
with 30 to 40

samples par person. It is actually a great help by two aspects.

First aspect that is obvious is that we ourselves does not had to take all those pictures
and had to account for all the restrictions i.e. light intensities, constant one color
Literature Review

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

24


background (t
hat is in different color than the face or hair color, usually taken white
color), the scale, the position of the face i.e. centered

in the picture,

without
expression. And obviously concerned about the
occlusion

of the faces. And last but
not the least co
ncern about the ration of face.

Second aspect that is although a little settle but is actually more important is that if the
face database is constant i.e. one than it is than possible to compare the results of the
different algorithms.

In the past there

were many vendors that had given face databases
[17, 18]

with
different number of person par database and

with different number of samples par
persons. And many good algorithms and researches had been done on those face
databases. I.e. like the research done by Steve Lawrence, C. Lee Giles, Ah Chung
Tsoi, Andrew D. Back as Face Recognition: A Convolutional Ne
ural Network
Approach. They had used the ORL database which contains the set of faces taken
between April 1992 and April 1994 at the Olivetti Research Laboratory in Cambridge,
UK.

Holistic face recognition by dimension

reduction by Ahmad bahtiyar GUL

in
S
eptember 2003 uses the ORL face database.

But the problem lie that although that those researchers had used one or the other face
database we cannot in the end compare their result because they had used different
face databases.

Then secondly

there are di
fferent face database available that only had faces but also
had their features set extracted. Either manually or implicitly by the algorithm. E.g. as
I had done in my research i.e. I had used the Bioid
14

face database. Bioid contains
pictures of 23 persons
15
, with each person having the following number of pictures, in
the following order 121, 60, 59, 2, 78, 88, 98, 90, 61, 71, 106, 99, 150, 94, 51, 35, 40,
40, 6, 46, 51, 25, 50 from the site
16
.




14

Bioid is freely available for download.

15

You can download the picture from
http://www.bioid.com/downloads/facedb/downloads/BioID
-
FaceDatabase
-
V1.2.zip

16

http://www.bioid.com

Literature Review

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

25


Because the Bioid face database had its feature extracted I did
not had to spend time is
creating the algorithm to extract those features set.

There is a lot of work still remaining if we start from feature set. Well for first again
there can be a brute force algorithm that would search those features point against an

array of those features sets. Although you would had thought that if we were using
only features points but those calculation still got out of hand i.e.
manageable
. Take
for instance Bioid contains 23 people with 30
feature point

picture pe
r person. This
makes 690 sets. If we divide them 50
-
50 (train and test sets). This makes 345 sets.
And every set contains 20 points (each point again had x and y locations). Now image
if an algorithm had to search how many of those 345 set are correctly located in those
test sets. Each set had to be search in for 345 times which make 119025 calculations.
That for brute force which would give wrong results because of RTS (rotation,
translation and scale) variations. A better approach would be to use neural net as
Holistic
face recognition by dimension

reduction by Ahmad bahtiyar GUL
[3]

had
done. Than if we had train just only

for 100 times on the trains sets this make (119025
*2) 100 = 23,805,000 times calculations. Not include the inter layer calculation. That
is if the first layer contains 40 neurons (20 points each having x, y).if the middle layer
30 neurons. This makes 120
0 (40 * 30) calculations and if the output layer contains 23
neurons (because of 23 person) this makes 27600 (1200 * 23) calculations par
iteration of one set. If you consider for 100 iterations for 23 person having 30 samples
sets this makes 1,904,400,000

(27600 * 23 * 30 * 100). No wonder why my memory
got extracted when I executed the training of the neural net in Matlab. Although I had
2 GB ram). I had to purchase 64 bit OS and 64 bit hardware because OS that want to
utilize more than 2 GB ram had to be

64 bit).

The

research done by Zhao et al.
[19
-
21]
, li and lu
[22]
, li and Jain
[23]
, and kong
et
al
[24]

and provided the survey on face recognition. Well the proces
s of face
recognition include location the face region, extracting the face feature set, then
matching those set with the search one, and in the end deciding the what which of the
person face was given.

Almost all of the face recognition algorithm use eith
er

1.

Eigenfaces

Literature Review

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

26


2.

Local feature analysis

3.

Elastic graph matching

Eigenfaces
[25
-
27]

uses principle component analysis PCA to extract the facial
feature to recognize the faces. It is a pure statistical method (mathema
tical approach)
for recognition and that proven to give responsible results if the environment is
controlled i.e. constant background, only frontal face images and having constant
lamination. Successful but not applicable in real time face recognition appl
ications

Local feature analysis is the most popular form of face recognition used. In local
feature analysis, facial feature point are extracted are used for representation and
comparison. Those features are usually the eyes, jaw line, tip of the nose etc
and the
distances between are measured those distances and location of feature points
identifies that person.

In Elastic graph matching each person face picture is taken from different angles. And
then features points are extracted. Those points are then r
epresented on the graph as
nodes. Because some point would not move and thus are more important, thus
weights are introduced. Thus each person had a graphical representation as weighted
graph
[28, 29]
. Advancement on this simple algorithm is to have more than point
values, taken by others pictures of the same person. Thus elasti
c graph matching is
proven
[29]

to be more invariant to affine transformation (rota
tion, translation and
scaling) and facial expressions.

FRVT (face recognition vendor test) 2002 is provided by US government to have
some uniformity in this field. US government had provided a face database of 2d still
images, so that it, 3rd parties or
any interested parties can compare the vendor
software (algorithm) up against other competitors.

But now FRGC (face recognition grand challenge) 2005 is provided. In which a new
face database is provided. This face database is in high resolution and it al
so
challenge 3D vendor to test their product up against each other. For obviously simple
reason 3D is better t
han 2D and is proven
[30]

to had less error rate than 2D. Plus this
time a larger face database is provided both in the persons, as well as their images as
Literature Review

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

27


w
ell as 3D scans. One 3D vendor

Aurora
[30]

there
[31]

could even identify identical
twins.

Geometric feature had been use for face recognition since the start of this field.
Kanade
[32]

reported that 45% to 75% accuracy can be achieved by deploying a
algorithm that automatically extract features from the face of 20 peoples. R. Brunelli,
T. Poggio.
[33]

Repo
rted that if distances of features points i.e. nose width, chin size
etc could achieve 90% accuracy on a face database of 47 people. But they argue that
100% accuracy would be achieved if simple pixel by pixel ma
t
ching would have been
done on the same face

database. Ingemar J. Cox, Joumana Ghosn, and Peter N.
Yianilos.
[34]

Showed that if we extract man
ually mixtures of distances than accuracy
rate of 95% could be achieved on a 685 fa
c
es (sets), i.e. each set contains 30 feature
points.

Eigen faces is the method in which the test images are projected on the principle
components
17
of the tr
a
ining images. T
hus the resultant images of faces are called
Eigen faces and a comparison is then made by comparison in between them e.g. M.
A. Turk and A. P. Pentland
[25]
.

The data we have the accurate would be the comparison. But there would always
be
that inverse relationship between the accurate identification i.e. the more processing
power/memory used and the fast or real time identification. One way would be (the
expensive more CPU time/memory consumption wise) to convert the 2d images from
diffe
rent viewpoints into surfaces and then into 3D model and then use them for
recognition as D. Marr, W. H. Freeman, San Francisco did in
[35]
.

A. Pentland, B. Moghaddam and T. Starner.
[36]

Used Eigenfaces and achieved 95%
recognition on 200 people on a face database of 3000. B. Moghaddam and A.
Pentland. Used FERET
18
face database
[37]
. Although it used extensive preprocessing



17

Principle component analysis is a statistical method by which such indepen
dent component are tried
to be extracted from original data domain, such that they can than be used for accurate
identification/extraction/separation purposes.

18

FERET

face database is given by Department of defense of US of make standardization in between

the different vendor of face recognition. i.e. because there is the same database used we can compare
the effectiveness of the algorithm used.

Literature Review

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

28


of the images to make the affine transformation normalized and only make one
incorrect result among 150 frontal images test.


Template matching is the brute force technique that uses pixel by pixel compar
ison
between the training and testing images. Only effective if the affine transformation
(scale, rotation and translation) are equal between the two (i.e. the training and test
image)...

Graph matching as discussed above is another methodology used for fa
ce recognition.



Literature Review

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

29


3.1.

Neural network

The development of computer science and the neural network started at the same time
[7]
. Neural network was developed as an attempt t
o create a model the human brain.
Most of the earlier models of networks are prove of that. Besides the nomenclatures
or terminologies of the architecture is derived from the medical terminology.

There are a lot of neural network and we can classify them
into different categories.
For example into supervised based or unsupervised based learning neural network.
(Although usually most of the neural networks that work in supervised based, also had
a version that work in unsupervised based environment.) Neural

network can also be
classified into pattern classification, pattern association, based on competition.

Backpropagation neural network trend to have general applicability
[38]
. So
Backpropagation had been applied to a lot of different appl
ications in many
disciplines. “We can identify at least two broad categories of face recognition system:

1.

We want to find a person within a large database of faces (e.g. in a police
database). These systems typically return a list of most likely people in t
he
database. Often only one image is available per person. It is usually not
necessary for recognition to be done in real time.

2.

We want to identify particular people in real time (e.g. in a security
monitoring system, location tracking system, etc.), or we

want to allow access
to a group of people and deny access to all others (e.g. access to a building,
computer, etc.). Multiple images per person are often available for training and
real
-
time recognition is required.”
[2]


My thesis is concerned with only the second scenario. I think Backpropagation
is the solution for face recognition because it uses gradient descent based
algorithm that usually converges i.e. if we had the input person pictures (or
features vector
to be more precise) and we had already made classes
accordingly to person i.e. targets then Backpropagation net will find the
association i.e. weight matrix between the two. Backpropagation is better
Literature Review

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

30


suited for this problem than hobbnet, perceptron, adalin
e, madaline, BAM.
Competition based neural network does not apply for this problem.


A lot of work had been done on face recognition. At first some people had
tried to use memorization but failed because there a infinite number of
variation in the face of
any person that memorization would never work plus if
somehow one can get all the variation in the rotation and illumination, scale
and translation etc of any person picture the task of searching in the face
database for any particular person is impossible

for any computer.

Many researcher had tried recognition based on the distance measure i.e.
geometric features attribute of the face. Kanade
[39]

had showed that

by using
a ratio of distance mesure, he got a 45


75% accuracy on a database of 20
people. Brunelli and Poggio
[33]

research is based on the nose width and
length, mouth position and chin shape, and their acc
uracy rate was 90% on a
database of 47 people. Cox et al
[34]

used 30 manually extracted and got
ac
curacy rate 95% on a database of 685 people, using only 95 person query at
a time.

Memorization or template matching technique
[33]

used exact matching based
on the pixel values and can work only whe
n the database is very small and the
train and test images are free from scale, orientation and illumination
variations. Graph matching is another face recognition method. Graph
matching method are computationally very expensive because they search for
an

object in the database (e.g. face) with all the other object. For example in
[28]

uses an distortion invariant object recognition based o
n elastic graph
matching. They used a dynamic link architecture of find the best match. The
vertices in the graph of the object, were labeled in terms of its local power
spectrum and the edges were labeled with geometrical distances between
them. They pre
sent good results with a database of 87 people and test images
composed of different expressions and faces turned 15 degrees
[2]
.

Wiskott et al
[29]

achieved an accuracy rate of 97.3% on 300
persons taken at
a time. Statistical methods such as principle component was used by Turk and
Pentland
[40]
. They extracted the principle components

of the training set of
the faces and then used simple comparison to the testing set. Marr et al
[35]

Literature Review

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

31


used a many stage process i.e. Marr paradigm, from image of surface to 3d
model to match model. Turk and Pentland
[40]

had showd that 2d face
recognition is also possible. “Their argument is based on the early
development and extreme rapidity of face recognition in humans, and on
physiological experiments in mo
nkey cortex which claim to have isolated
neurons that respond selectively to faces “
[41]
. Eigenfaces is also a method
for face

recognition but is highly susceptible to the image pixel intensities of
the train and test datasets. To overcome this fatal flaw usually extensive
preprocessing is done both on the training and test dataset. Pentland et al. in
[42]

and
[43]

show a 95% accuracy rate of a test set of 200 persons, selected
from a database of 3000 persons.



Literature Review

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

32



Chapter
4

Problem
D
efinition

Well technological challenges asi
de, it is more of a mathematical and logical
challenge. So that why not only software companies involved in created facial
recognition are interested in it, but also the academic are also interested in it. Well the
reason that it had became a good practica
l real life problem, more formally it a NP
hard problem. Recognition by Britannica definition “
a form of remembering
characterized by a feeling of familiarity when something previously experienced is
again encountered; in such situations a correct response

can be identified when
presented but may not be reproduced in the absence of such a stimulus.

[11]

The problem in a nut shell is that we want to associate faces to identities (either names
of ID numbers of peoples). As anybody can guess the problem does not have a linear
solution. Because the data are in totally different doma
ins i.e. input data in binary
form whereas output belong in a set. And no such association can be possible through
formal languages. Even if we try we had to use three layers of nested loops. This is
fine when the input/ output are small. But as the number

of inputs (either row images
or even key feature points) increased those loops takes exponential amount of time as
well as resources (memory and processing time). Here we had not even taken into
account the noise. Not even images taken with extreme care a
re free of noise. We had
backgrounds, illumination fluctuations, pose variation, expressions, translation,
rotation even the dust in the air end up noise in the final image. If we had to take
account of all those noise than that software not even start, an
d if we hypothetically
consider that it did than it would never stop

computing
. We
also
cannot use
languages using logic. Because the data in our case is not in form of facts, instead it is
Literature Review

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

33


multidimensional array and that entire array had to be taken to r
epresent a fact
(individual).

The data we capture i.e. images or frames of the interested persons are in
multidimensional array (in RGB etc or uncolored). But to extract those features from
that data is a complex problem in itself. (Actually it is also a
research area). Today’s
professional facial recognition softwares can extract the key features from those
frames (or images) with in an acceptable degree of error. (This error incorporated in
this stage hugely effects all the remaining stages. In simple wo
rds even if rest of
stages perform at 100% correct results and this stage gave it data that had errors, those
remaining stages will suffer because of it).

Because there is no efficient solution (polynomial time) found for face recognition but
if we some
how know the solution (through nondeterministic approach), we can
validate tha
t solution in polynomial time.
But to conform that face recognition belong
to the class of NP
-
complete might be incorrect, because we cannot be sure that we
could ever find a pol
ynomial time solution for NP problem. If we could find that
polynomial time solution than all the other NP problem could also be reduced.

One of the solutions to face recognition problem is to use neural network. Other
workable solution includes PCA. Or w
e could use a combination of both. In a way
neural network is also a statistical method because it trains (change the weights of) the
network to that the samples (of the person) best represent the population (of that
person).

In a way the training of the
neural network is a like finding a polynomial time
solution. So when the neural network is trained the end weights represent a
polynomial time solution. Although it gives results in polynomial time in testing
phase but takes
exponential
time in the trainin
g phase.

Well like all the other NP problem when we need a solution in a polynomial time, we
usually approximate. It depend on the problem how much approximation is
acceptable. For example in a high security system we would want a high percent of
approxi
mation than for example in recognizing handwriting.

Literature Review

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

34



To better our odds, heuristics are used. And the better the heuristic the better would be
the approximation. In our case where we had to recognize a face from the pixel data,
it is just not possible to t
ake into account all the pixels, because of the limitation of
memory and processing power. Even the super computers, grid computer or clusters
could process that much data. So what usually is done is to extract the key features
points from the pixel data a
nd the rest of the recognition is done through those
features.

One thing is for curtain that we cannot use procedural algorithms for the solution.
Because even that we now had to deal with only the key features point, still the
combinations of those featu
res are limit less. (Unless the data set is very small). So
the only way is to use artificial intelligence algorithm. Usually some preprocessing is
necessary before using those algorithms. First problem is the identify the face area.
Then there are the pro
blems of excluding the background from the face. It gets
extremely difficult when the background is moving as well as the face. Things got
easier when we only consider constant background, preferably in one color.
Secondly

the face area must greater than a

certain size, so that the key features could be
extracted. Key features could be extracted either manually or automatically. If we are
considering the automatically
(i.e.

t
h
rough programmatically
), w
e must first find the
contours of the face, the eyes, no
se etc. but because of lighting irregularities, noise in
the mediums i.e., air and the recording medium. Mostly the edges that are found are
only small curves but are disconnected from each other. Than we had to use some
algorithm to change those curves in

to a continuous edge (this is also a research area
in itself). Then there is the problem of which I call RTS (rotation, translation and
scaling). Occlusion is a problem than cannot be solve in 2D face images (, but 3D face
images can tolerate to some degr
ee).


Problem Solution and Methodology

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

35



Chapter
5

Face Recognition

I had used the Bioid
19

face database.

It contains image and
feature

point
s

already
extracted from it.
I used these features points to train the neural network. But first I
had done preprocessing on that feature dataset. I

first divided the features dataset in to
60
-
40% partitions. 60% dataset was used
for training the neural network
, while 40%
was used for testing the neural net.

Because the features were taken from pictures,
that were of different in scale
,
translation a
nd rotation I had to first preprocess the data
20
. Because the face database
had different number of pictures per persons, thus the prior of some persons’ picture
are far greater than the prior of other persons’ pictures. This causes the convergence
for the

weights of the neural network an impossible task i.e. when the dataset is as
large.

Thus preprocessing had to be done to equalize the priors of all the classes. In my
opinion there are three ways to equalize the dataset priors.

1.

Only take 35 pictures fr
om all those persons pictures i.e. who had 35 pictures
and discards all the other (pictures) data.

2.

Simulate new data i.e. simulate new records that had the same mean and
standard deviation as that of the population.




19

Bioid is freely available for download.

20

An alternative would be to translate, use the on
-
center
-
off
-
surround

(algorithm) to normalize the
dataset and then rotation correction.

Problem Solution and Methodology

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

36


3.

Copy the records i.e. to make all the re
cords per person equal to the maximal
person’s pictures.

Permutation of the records was done before training the neural net as a
precaution that the net would not tries to learn one class i.e. person and then
when the other class records starts, will forgo
t the already learn class(es)
[6,
7]
.

To get the program a little more efficient, I saved the records once I read them
from their individual files and saved them in to Matlab workspace variables files.
The
program checks

for those files and if the
y are not present then it read the all
the individual files and saves all the records i.e. after 60
-
40% di
vision into
workspace file(s). T
he same is true for the neural net, if the net not already created
it is created
otherwise

the net is read from the wo
rkspace file. After training there
is an option given for saving the trained net i.e. into the workspace file.

Testing is for most part same as the training program file i.e. loading the
appropriate records and the neural net from their workspa
ce file. A
ft
er loading the
testing records are applied to the neural net and on the results of the neural net a
binary threshold is applied. Which is the maximal value of the records. Thus only
one out of twenty three 1
-
of
-
23 output of the records remains on while all

the
others became off. This result is matched against the target records (which are
already of one of the class 1
-
of
-
c type). The percentage of the matched records are
calculated and this percentage and the plot of the neural net results ant the actual
ta
rgets are plotted.

I used the Matlab neural network toolbox i.e. I created the Backpropagation neural
network using newff function.

Flowchart of the algorithm is given below
21
.

Code
could be downloaded
from




21

Annexure C. Elementwise code.

Problem Solution and Methodology

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

37


http://sourceforge.net/projects/trainbpelementw/

When one executes the program, for training the neural network, the first thing it
(program) ask you to change the present working directory to the directory were
that program is lying
22
. Select the change directory to continue. Executes starts




22

See a
nnexure

for diagram

Problem Solution and Methodology

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

38


5.1.

Preprocessi
ng

The program asks to select the dataset upon which the neural network is trained
on
23
.

These three options are either to select only first 30+ records (and discards all the
other records for that person), or select an equal records for all, by compensati
ng
the remaining records by simulated records or finally to select equal records for
all, by compensating the remaining records by replicating the records. You had
to select 1,2 or 3 for selection.

At least 30 records restriction is implemented becaus
e statistical theory suggests
that that to get a proper mean and variance for a population the input(s) should be
at least 30. And because the neural network tries to get an approximation of the
population i.e. to identify the person though this features d
ataset records, the
records should at least be 30 plus. This means that all those person whose records
are less than 30 are eliminated. The algorithm only takes 35 records because the
minimum number of records that are more than 30 is 35 for a person. All
the other
records if they are present are discarded.

In the simulated approach I had used the statistical approach that when we got
records that are less that the required number we can simulate new records. This
involves estimation of the population mean
and variance (by population here I
mean the person). Once the mean and variance is calculated, random records are
simulated with the same mean and variance as that of the population.

In the replicated approach, if the person had less that maximum number of

records
for a person in the database, than his/her records are replicated to make it equal to
that maximum number.

For program efficiency once the datasets had be fetched from the files and divided
in 60
-
40 i.e. 60% for training and 40% for testing, thos
e matrices are saved in a



23

See annexure B for snapshot

Problem Solution and Methodology

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

39


files so that those exactly same process can be avoided and only that matrix had to
be loaded again.

If you take the default argument i.e. 30+ (or you simply entered), than the
program asks for

Enter the number of records per
per
son:

[35]

By default it takes 35 records out of all those who had 35 records, this includes 20
out of 23 i.e. three persons who had less than 35 (2,6 and 25) are excluded. The
number of records of the persons, in the database are121, 60, 59, 2, 78, 88, 98,

90,
61, 71, 106, 99, 150, 94, 51, 35, 40, 40, 6, 46, 51, 25 and 50. If sort by the number
of records they became 2, 6, 25, 35, 40, 40, 46, 50, 51, 51, 59, 60, 61, 71, 78, 88,
90, 94, 98, 99, 106, 121 and 150.

All the records (
in
face database) had to
be
d
ivided into 60% and 40%. And all those
division had to be done for all the persons (classes). After the division we either had
to insert new records (simulate or replicate). Affine transformation is done after
division. For affine transformation we get th
e center of the input array (face). I take
the extreme feature points on the face like the left and right temples, tip of the chin,
left and right inner end of the eye brows to draw four lines (both pairs parallel to each
others). These four lines creates
a square. Those four intersection points of those lines
thus, gives the center points of the face (left upper and right lower point, right upper
and left lower point). The slope of the face is calculating the slope of the line from the
tip of the nose to t
he line in between the inner end of right and left eye brow. The
correction in rotation is calculated for each point by the formula

t = [cosd(ang)
-
sind(ang); sind(ang) cosd(ang
)] * [x(i) y(i)] ;

Where ang

is the slope of the face and i is the feature
po
ints of the face. After that
translation and scaling of the input array to a unified format is done by the formula

x = ((x + cent(1))
-

min(x)) * 1/max(x);

Where x is

the feature point(s).

Problem Solution and Methodology

Face Recognition using Backpropagation Neural Network on Unbalanced Priors

40


5.2.

Network configuration

Once the preprocessing is done. The program a
sks the parameters to correctly
configure the neural network. Those involves the following

Enter the maximum number of Epochs: 200 | [100=oneHour]

Because the program saves after every completion of the program its best to set