C C C1 1 1

crumcasteAI and Robotics

Nov 17, 2013 (3 years and 7 months ago)

125 views

Neural Network
-
Based Face Recognition, using ARENA algorithm.


1

A
BSTRACT

................................
................................
................................
................................
.................

4




C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R
1
1
1



1.1

INTRODUCTION

................................
................................
................................
...........................

5

1.2

AIMS

................................
................................
................................
................................
................

6

C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R
2
2
2



2.1

FACE RECOGNITION

................................
................................
................................
..................

7

2.2

WHY FACE RECOGNITION
?

................................
................................
................................
.....

7

2.3

HISTORY OF FACE RECO
GNITION

................................
................................
........................

8

2.4

THE PRESENT

................................
................................
................................
.............................

10

2.5

FACE RECOGNITION AND

FACE DETECTION.

................................
................................

11




C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R
3
3
3



3.1

NEURAL NETWORKS

................................
................................
................................
...............

14

3.2

HOW ARTIFICIAL NEURA
L NETWORKS WORK.

................................
............................

15

3.2.1

I
NPUT
-

O
UTPUT

................................
................................
................................
......................

16

3.2.2

F
UNCTIONS

................................
................................
................................
.............................

17

3.2.2.1

A
CTIVATION
F
UNCTION
…………..…………….……………………………………….
18

3.2.2.2


T
HRESHOLD
F
UNCTIONS
…………………..…………………………………………….
19

3.2.3

A
SSOCIATED
M
EMORY

................................
................................
................................
...........

20

3.3

LEARNING AND TRAININ
G

................................
................................
................................
....

21

3.3.1

L
EARNING
M
ODELS

................................
................................
................................
................

21

3.3.1.1


S
UPERVISED
L
EARNING
…………………..…………………………………………….
21

3.3.1.2


U
NSUPERVISED
L
EARNING
………………………...……………………………………
22

3.3.1.3


L
EARNING
R
ULES
……...……………..
……………………………………………
……
23

3.3.2

L
EARNING
R
ATES AND
T
RAINING

................................
................................
..........................

23

3.4

BACK PROPAGATION

................................
................................
................................
..............

23

3.4.1

A
RCHITECTURE

................................
................................
................................
......................

24

3.4.1.1

N
EURON
M
ODEL
……………………...……………………………………………
……
24

3.4.2

F
EEDFORWARD
N
ETWORK
................................
................................
................................
.....

25




C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R



4
4
4



4.1

DATABASES

................................
................................
................................
................................
.

27

4.2

TEST AND TRAIN SETS

................................
................................
................................
............

28




C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R



5
5
5



5.1

ALGORITHMS FOR FACE
RECOGNITION

................................
................................
.........

30

5.1.1

P
RINCIPAL
C
OMPONENT
A
NALYSIS

................................
................................
.......................

30

5.1.1.1

A
LGORITHM
..…………………………………………………………………………...
31

Neural Network
-
Based Face Recognition, using ARENA algorithm.


2

5.1.2

E
IGENFACES

................................
................................
................................
...........................

32

5.1.3

E
UCLIDEAN DISTA
NCE

................................
................................
................................
............

33

5.2

ARENA ALGORITHM

................................
................................
................................
................

33

5.2.1

T
HE ALGORITHM

................................
................................
................................
....................

34




C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R
6
6
6



6.1

CODING

................................
................................
................................
................................
........

36

6.2

MATLAB

................................
................................
................................
................................
.......

36

6.3

IMPLEMENTIN
G ARENA ALGORITHM

................................
................................
..............

37

6.4

IMPLEMENTATION OF TH
E NEURAL NETWORKS

................................
.........................

40

6.4.1

T
RAINING

................................
................................
................................
................................

40

6.4.2

T
ESTING

................................
................................
................................
................................
..

44




C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R



7
7
7



7.1

RESULTS

................................
................................
................................
................................
......

46

7.2

ARENA

................................
................................
................................
................................
..........

46

7.2.1

T
RAIN ARENA

................................
................................
................................
..........................

46

7.2.2

T
EST ARENA

................................
................................
................................
............................

50


7.3

NEURAL NETWORKS

................................
................................
................................
...............

57

7.3.1

T
RAIN NEURAL NETWORK

................................
................................
................................
......

57

7.3.2

T
ESTING
N
EURAL
N
ETWORK


F
INAL
R
ESULTS
.

................................
................................
..

66

7.4

ACCURACY

................................
................................
................................
................................
..

71

7.5

COMPLEXITY AND STORA
GE

................................
................................
...............................

73




C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R



8
8
8



8.1

FURTHER WORK

................................
................................
................................
.......................

75

8.1.2

F
URTHER WORK

................................
................................
................................
.....................

75

8.1.2.1

A
LGORITHM
…………………………………………………………………………….

75

8.1.
2.2

I
MPLEMENTATION
……………………………………………………………………….76

8.1.2.3

WEB………..…………………………………………………………………………...76

8.1.3

F
UTURE OF
F
ACE
R
ECOGNITION

................................
................................
...........................

76

CONCLUSION

................................
................................
................................
................................
.........

78

REFERENCES

................................
................................
................................
................................
..........

79

B
IBLIOGRAPHY

................................
................................
................................
................................
.......

79

PAPERS

................................
................................
................................
................................
.....................

79

WEB

................................
................................
................................
................................
...........................

80

OTHER DOCUMENTATION

................................
................................
................................
................

80

Neural Network
-
Based Face Recognition, using ARENA algorithm.


3

APPENDIX I


APPENDIX II

Neural Network
-
Based Face Recognition, using ARENA algorithm.


4


Abstract

T
he purpose of that project is to implement a face recognition
algorithm. The face recognition algorithm that is presented here is a
memory based face recognition system. We show that an extremely simple,
memory
-
based technique for view
-
based frontal face r
ecognition can
outperform more than sophisticated algorithms that use Principal
Components Analysis (PCA) and neural networks. The goal of this report
is to write about the most common methods that have been used till now
for face recognition. Analyse thes
e methods and give a general idea of the
background of the algorithm, ARENA, which we are going. The capability
of the face recognition is to find the exact mach of a face image from an
image database project. The algorithm that is used in order to achieve

that
is called ARENA.

Neural Network
-
Based Face Recognition, using ARENA algorithm.


5

C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R



1
1
1



1.1

Introduction


The objective of our system is to recognise and identify faces, not previously
presented to or in some way processed by the system. There are many datasets involved
in this project. Some of them are the
ORL, MIT database which consisting of a large set
of images of different people. The database has many variations in pose, scale, facial
expression and details. Some of the images are used for training the system and some for
testing. The test set is not i
nvolved in any part of training or configuration of the system,
except for the weighted committees as described in a section later on.

The algorithm used for the face recognition, from the project is known as
ARENA. Similar to several other approaches to
face recognition and identification,
which use Principal Component Analysis (PCA) as pre
-
processing, dimensionality
reduction and feature extraction, of the input images. One of the main parts of the
project is a neural network. The use of a neural networ
k makes the algorithm perform
better.


In chapters two and three we are going to analyse the background of the project.
A literature background of face recognition and neural networks discussing also the
methods that were used for the project. In the next
chapter, four there is a description of
the datasets that where used in order to test and train the algorithm and the neural
network, the implementation of which is in chapter six. The code also is included in
appendix 1. After that, in chapter seven there

is a detailed analysis of the outputs we get
from the programs and a comparison of the ARENA algorithm with other methods that
have been used for face recognition, the theory of which is analysed in chapter five.
Finally in chapter eight there is a discus
sion about the work that had been done and
further improvement that could be done.

Neural Network
-
Based Face Recognition, using ARENA algorithm.


6

1.2

Aims


The purpose of face recognition algorithm is to examine a set of images and try
to find the exact match of a given image. An advanced system would be a neural
netw
ork face recognition algorithm. The system examines small windows of the image
in order to calculate the distances of given points. That would be done from any
algorithm but in a system where we use neural networks the system arbitrates between
multiple ne
tworks in order to improve performance over a single network.

The goal of this ongoing project is to formulate paradigms for detection and
recognition of human faces, and especially develop an algorithm, which is going to have
high performance in complex

backgrounds. One of the applications would be towards
adding face
-
oriented queries to our image database project.

The fundamental principle, which we are exploiting for our face recognition
algorithm, is Principal Component Analysis. Thought the algorith
m is much simpler.
One of the aims is to run tests in order to compare the algorithm with two PCA
algorithm and also show that the calculation between two given point with the ARENA
algorithm is efficient as much as if we had use the Euclidean distance.

Neural Network
-
Based Face Recognition, using ARENA algorithm.


7

C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R



2
2
2


2.1

Face recognition


Face recognition is a part of a wide area of pattern recognition technology.
Recognition and especially face recognition covers a range of activities from many
walks of life. Face recognition is something that humans are pa
rticularly good at and
science and technology have brought many similar tasks to us. Face recognition in
general and the recognition of moving people in natural scenes in particular, require a set
of visual tasks to be performed robustly. That process incl
udes mainly three
-
task
a
cquisition, normalisation and recognition. By the term acquisition we mean
the
detection and tracking of face
-
like image patches in a dynamic scene.
Normalisation is
the segmentation, alignment and normalisation of the face images,
and finally
r
ecognition that is

the representation and modelling of face images as identities, and the
association of novel face images with known models.



2.2

Why Face Recognition?


Given the requirement for determining people's identity, the obvious qu
estion is
what technology is best suited to supply this information? The are many ways that
humans can identify each other, and so is for machines. There are many different
identification technologies available, many of which have been in commercial use fo
r
years. The most common person verification and identification methods today are
Password/PIN known as Personal Identification Number, systems. The problem with
that or other similar techniques is that they are not unique, and is possible for somebody
to
forget loose or even have it stolen for somebody else. In order to overcome these
problems there has developed considerable interest in “biometrics” identification
Neural Network
-
Based Face Recognition, using ARENA algorithm.


8

systems, which use pattern recognition techniques to identify people using their
characteris
tics. Some of those methods are fingerprints and retina and iris recognition.
Though these techniques are not easy to use. For example in bank transactions and entry
into secure areas, such technologies have the disadvantage that they are intrusive both
p
hysically and socially. The user must position the body relative to the sensor, and then
pause for a second to declare himself or herself. That doesn’t mean that face recognition
doesn’t need specific positioning. As we are going to analyse later on the po
ses and the
appearance of the image taken is very important.

While the pause and present interaction are useful in high
-
security, they are
exactly the opposite of what is required when building a store that recognise its best
customers, or an information
kiosk that remembers you, or a house that knows the
people who live there. Face recognition from video and voice recognition have a natural
place in these next generation smart environments, they are unobtrusive, are usually
passive, do not restrict user m
ovement, and are now both low power and inexpensive.
Perhaps most important, however, is that humans identify other people by their face and
voice, therefore are likely to be comfortable with systems that use face and voice
recognition. [I2, I5]



2.3

Hist
ory of Face Recognition


Face recognition rises from the moment that machine started to become more
and more ”intelligent” and had the advance of fill in, correct or help the lack of human
abilities and senses.

The subject of face recognition is as old
as computer vision and both because of
the practical importance of the topic and theoretical interest from cognitive science. Face
recognition is not the only method of recognising other people. Even humans between
each other use senses in order to recogn
ise others. Machines have a wider range for
recognition purposes, which use thinks such as fingerprints, or iris scans. Despite the
fact that these methods of identification can be more accurate, face recognition has
Neural Network
-
Based Face Recognition, using ARENA algorithm.


9

always remains a major focus of researc
h because of its non
-
invasive nature and because
it is people's primary method of person identification.

Since the start of that field of technology there were two main approaches. The
two main approaches to face recognition are



geometrical approach



pi
ctorial approach.

The geometrical approach uses the spatial configuration of facial features. That
means that the main geometrical features of the face such as the eyes, nose and mouth
are first located and then faces are classified on the basis of variou
s geometrical
distances and angles between features. On the other hand, the pictorial approach uses
templates of the facial features. That method is using the templates of the major facial
features and entire face to perform recognition on frontal views of

faces. Many of the
projects that where based on those two approaches have some common extensions that
handle different poses backgrounds. Apart from these two techniques we have other
recent template
-
based approaches, which form templates from the image
gradient, and
the principal component analysis approach, which can be read as a sub
-
optimal template
approach. Finally we have the deformable template approach that combines elements of
both the pictorial and feature geometry approaches and has been applie
d to faces at
varying pose and expression.




Since the early start of face recognition there is a strong relation and connection
with the science of neural networks. Neural networks are going to be analyses in more
detail later on. The most famous early

example of a face recognition “system”, using
Figure 1:

Face recognition using
face geometry.

Neural Network
-
Based Face Recognition, using ARENA algorithm.


10

neural networks is the Kohonen model. That system was a simple neural network that
was able to perform face recognition for aligned and normalised face images. The type
of network he employed computed a face d
escription by approximating the eigenvectors
of the face image's auto
-
correlation matrix; these eigenvectors are now known as
“eigenfaces”. [O1]

After that there were more other methods that where developed based on older
techniques. If we want to summaris
e the methods that the “idea” of face recognition is
based we have a geometrical approach or pictorial approach, and after that we have
methods like eigenfaces, Principal Component Analysis, or other methods that process
images in combination with neural n
etworks or other expert systems.

In the recognition stage, the input is compared against all selected model views
of each person. To compare the input against a particular model view, the face is first
geometrically aligned with the model view. An affine t
ransform is applied to the input to
bring the facial features automatically located by the system into correspondence with
the same features on the model. A technique based on the optical flow between the
transformed input and the model is used to compensa
te for any remaining small
transformation between the two. Templates from the model are then compared with the
image using normalised correlation. Both the model and input images are pre
-
processed
with a differential operator such the ones mentioned just a
bove. In the future, we plan to
address the problem of recognising faces when only one view of the face is available.
The key to making this work will be an example
-
based learning system that uses
multiple images of prototype faces undergoing changes in po
se to learn, with the help of
neural networks. The system will apply this knowledge to synthesise new virtual views
of the person's face.



2.4

The present


With the rapid evolution of the technology and the commercialisation of
technological achievements
, face recognition became more and more popular, not only
for research but also for the use of security systems.

Neural Network
-
Based Face Recognition, using ARENA algorithm.


11

That gave the motive to many researchers, and also companies in order to
develop techniques for automatically recognising faces that would find

many
applications, including security and human
-
computer interaction. For instance, a face
recognising machine could allow automated access control for buildings or enable a
computer to recognise the person sitting at the console. Most existing face recog
nition
systems, however, work only for frontal or nearly frontal images of faces. By
recognising faces under varying pose, one makes the conditions under which face
recognition systems operate less rigid.



2.5

Face recognition and Face detection.


As men
tioned above face recognition is a technique of recognising faces but it is
not necessary to “freeze” the user in order to take a picture. Though there is a problem
with recognising faces when the pose of the face is different, but in particular, there is
a
limit on face rotations in depth, which include left and right and up and down rotations.
Face recognition itself is difficult because it is a fine discrimination task among similar
objects, once we can find faces, which are quite similar. Adding pose va
riation naturally
makes the problem more difficult. This is because the appearance of a person's face
changes under rotation since the face has a complex 3D structure.

At this point we have to distinguish face recognition for face detection. Many
people t
hing that these two terms are the same. Thought even they have many similar
techniques, are based on the same idea and algorithms, are two different, systems. The
main difference is the fact that face recognition is detecting faces and search through a
dat
aset in order to find an exact match but on the other have face detection is looking for
any match and as soon as one is found then the search stops.


This task of face recognition seems to be sequential and have traditionally often
been treated as such.
However, it is both computationally and psychophysically more
appropriate to consider them as a set of co
-
operative visual modules with closed
-
loop
feedback. In order to realise such a system, an integrated approach has been adopted
Neural Network
-
Based Face Recognition, using ARENA algorithm.


12

which will perform acqu
isition, normalisation and recognition in a coherent way. Images
of a dynamic scene are processed in real
-
time to acquire normalise and aligned face
sequences. In essence, this process is a closed
-
loop module that includes the
computation and fusion of thr
ee different visual cues: motion, colour and face
appearance models. Face tracking based upon motion and a face appearance model has
been addressed in greater detail elsewhere.

The visual cue of colour is important but most of the face recognition algorit
hms,
and also the one used for the project, are not really concerned about it. This project
focuses upon person identification within such a framework. Complementary to
recognition, appearance
-
based mechanisms for real
-
time face pose estimation have been
d
eveloped which can be used to improve the robustness of detection and alignment.

In general much research effort has been concentrated on face recognition tasks
in which only a single image or at most a few images of each person are available. A
major conc
ern has been scalability to large databases containing thousands of people.
However, large intra
-
subject variability casts doubt upon the possibility of scaling face
recognition, at least in this form, to very large. Databases are mentioned in detail in
an
other part of the report.

The tasks of face recognition mostly require recognition to be performed using
sequences acquired and normalise automatically in poorly constrained dynamic scenes.
These are characterised by low resolution, large
-
scale changes, v
ariable illumination and
occasionally inaccurate cropping and alignment. Recognition based upon isolated
images of this kind is highly inconsistent and unreliable. However, accumulating
recognition scores over time can compensate the poor quality of the da
ta.

Face recognition is an active research area involving different fields such as
physics, psychology, biology, mathematics, computer science and several others. A wide
range of problems has been approached, resulting in many interesting applications. In

the research presented here, we look at one of the core problems in a face identification
system.




Neural Network
-
Based Face Recognition, using ARENA algorithm.


13




Neural Network
-
Based Face Recognition, using ARENA algorithm.


14

C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R



3
3
3



3.1

Neural Networks


Face recognition as an idea comes from the real life term “face recognition”. Part
of the human “face recognition” i
s the “memory” of the humans, which means the brain.
The connection between this term of a biological organisation, and the computer is the
neural networks. The neuron has a biological meaning even thought the neural networks
are just inspired from that bi
ological term of neurone and neuron systems, “network”.
Even thought that is not that positively true, as long as the aim of neural networking is to
create a computer system which actually works in a similar way to how we think the
neurones in the human br
ain works.[O1]. Neural networks can be defined as:


A neural network is a system composed of many simple processing elements operating in
parallel whose function is determined by network structure, connection strengths, and
the processing performed at comp
uting element or nodes. Neural network architecture is
inspired by the architecture of biological nervous systems, which use many simple
processing elements operating in parallel to obtain high computation rates

[5].


Neural networks are a form of micropro
cessor computer system with simple processing
elements, a high degree of interconnection, simple scalar messages and adaptive
interaction between elements

[6].


The neural networks resemble the brain mainly in two respects



Knowledge is acquired by the netw
ork through a learning process



Interneuron connection strengths known as synaptic weights are used to store the
knowledge.

Neural Network
-
Based Face Recognition, using ARENA algorithm.


15


That means to construct a machine that is able to think. Somehow, not really
known yet, the brain is capable to think and perform so
me operations and computations,
much faster sometimes from a computer even the “memory” is much less. How the brain
is managed to do that is a hardware parallelism. The computing elements are arranged so
that very many of them are working on a problem at t
he same time. Since there is a huge
number of neurones, somehow the weak computing powers of these many slow elements
are combined together to form a powerful result.

A Neural Network is an interconnected assembly of simple processing elements,
units

or
no
des
, whose functionality is loosely based on the animal neuron. The
processing ability of the network is stored in the inter
-
unit connection strengths, or
weights
, obtained by a process of adaptation to, or
learning

from, a set of training
patterns.

In or
der to see how different this is from the processing done by conventional
computers it is worth examining the underlying principles that lie at the heart of all such
machines.


Neural network algorithms for face recognition work by applying one or more
ne
tworks directly to portions of the input image and arbitrating their results. Each
network is trained to output the presence or absence of a face. Training now a neural
network for face recognition is a challenging task because of the difficulty in
charact
erising prototypical “non
-
face” images. The structure of the images that are used
is going to be analysed in the dataset chapter. [7]

Finally we have to mention the why that a neural network system operates.
Mainly face recognition systems operate in two s
tages. The first stage is to apply a set of
neural network based filters to an image and then use an arbitrator to combine the
outputs. Then the arbitrator merges detection from individual filters and eliminates
overlapping detection.


3.2

How Artificia
l Neural networks Work.



Neural Network
-
Based Face Recognition, using ARENA algorithm.


16

Artificial neural networks can modify their behaviour in the response to their
environment. This factor, more than any other, is responsible for the interest they have
received. Shown a set of inputs, which perhaps have specific d
esired output, they self
adjust to produce consistent responses. A wide variety of training algorithms has been
developed for that reason. Each of the algorithms has it’s own strength and weaknesses.




Figure 2: How neural network operate


As mentioned
before the shape, the structure and “use” of neural networks is not
the same. There is a big variation. But despite that differences the functions and the way
they operate we can say is similar, if not the same. All the neural networks behave, react,
organ
ise and generalise data in the same way, learn and forget in the same way. The
functions and operations are the same for all neural networks. In the next few pages we
are going to come through some of the basic functions and processes of the neural
network
s.



3.2.1

Input
-

Output

Once more the fact that the artificial neural network is similar and operates in a
similar way to the biological neural networks is really important in order to understand
the input and output of a neural network. As in biologi
cal neurons, there are many
Neural Network
-
Based Face Recognition, using ARENA algorithm.


17

different inputs, even in the same time, is also to the artificial neural networks. So from
the moment that we have stimulation or a signal for ANN’s as input, there is a series of
functions, which take place till the moment we
have some output.











Figure 3: Neurons Input/ Output




It is possible some of the input to be influenced and effected from other things
than inputs, as neurons in real life. These influences are called weighting factors. Each
of the inpu
ts will be given relative weighting. The why that the weights are influencing
the inputs is simple. The output after the influence of some weight is given by the
function


Output = Sum of Inputs * Weight


If the sum of the inputs is greater than the thresh
old value, then the neural
network generates a signal. If the sum of the inputs is less than the specified threshold
level no signal will occur.


3.2.2

Functions

Some of the most important functions that taking place from the moment of the
given input,
signal, till we have the output are the Activation function, Associated
Memory and the Threshold functions.


Output

Input 1

Input 2

Input n

Neural Network
-
Based Face Recognition, using ARENA algorithm.


18

3.2.2.1

Activation Functions

When a neuron gets a signal before it passed in the transfer function is processed
by an activation function F. That
function produces the neuron’s output signal, OUT.
The activation functions are a likely component of the network as a whole rather than
just a artificial neural network component. That function can be presented as follows:

OUT = K(INPUT)

where K is a con
stant, a threshold function ( discussed later).

OUT = 1 if INPUT > T

where T is a constant threshold value

OUT = 0 otherwise


The signal output is accepted from a block F that produces the output signal
OUT. If F processing block compresses the range of
net so that the output signal, OUT
never exceeds some low limits regardless of the value of the input. F is called a
Squashing function. That squashing function is a logistic function or sigmoid. So
mathematically the function can be written:

OUT = 1/(1 +

e
-
input

) = F(INPUT)

and the graph is



















Figure 4: Sigmoid Logistic Function


0

5

OUT

Neural Network
-
Based Face Recognition, using ARENA algorithm.


19

Activation function is defining a non
-
linear gain for the artificial neuron. This
gain is calculated by finding the ratio of the change in output, OUT,
to a small change in
input. In 1973 Grossberg found that this non
-
linear gain characteristic solves the noise
saturation dilemma. That dilemma is how can the same network handle the different
small or large signals.


3.2.2.2

Threshold Functions

The thresh
old function as the activation function is non
-
linear. Some times the
threshold function can also be called transfer function. Different values can be taken for
the function. Depending on the output value we have two types of functions. That can be
more cl
ear if we take a look to the Input Output function (Output = Sum of Inputs *
Weight) and the mathematical equation of the activation function.


In the case that the threshold function is 0, then the output depends on the result
of the summation. The summat
ion can be either positive or negative.

So the two cases we have are:



1.

Hard limited or else step function. In that case the output is 1 or
-
1.




Figure 5: Step function













1

-
1

Input

Output

Neural Network
-
Based Face Recognition, using ARENA algorithm.


20

2. Ramping function, which mirrors the input within a specified ran
ge. The output this
time is 1 and 0.










Figure 6: Ramping Function




3.2.3

Associated Memory

The neural networks have an associated memory, which can store the results of
the computations of the functions mentioned before and subsequen
tly alter the weights.
That is important because the ability to change the weights allows to the network to
modify its behaviour in response to its inputs. Human memory operates in an associative
manner. That means that a portion of a recollection can pro
duce a larger related
memory. An example for that is the fact that if someone hears few bars of music may
recall a complete sensory of “experience”, like sounds, songs, scenes.


Like human memory the artificial neural networks have associative memory. The

inputs as mentioned before are influencing the “reactions” of the neural networks. A
positive input will promote firing of the system, and on the other hand a negative input
keeps the system from firing.


Once the weights are determined, the network may b
e used to produce the
desired output vector. Even if we give as input a vector that may be incorrect or
incomplete we are going to have the desired output. In order to achieve that the outputs
of the system are forced to the values of that input vector and

then the input is removed
and the network is allowed to “relax”. That is because the network has the desired output
1

1

Input

Output

Input > 1

Output = 1

Input < 1

Output = 0

Neural Network
-
Based Face Recognition, using ARENA algorithm.


21

or the closest. When weight adjustments are made in preceding layers of feed forward
networks by “backing up” from output, the term “back p
ropagation” is used. 8




3.3

Learning and Training


Learning is a really important theme in the theory of neural networks. Mainly is
connected to handle the data stream. Learning aims at programming the synaptic
efficacious in such a way that a maximal am
ount of information can be retrieved. There
are three main topics that we have to concern about, for learning. The training
techniques the time of learning and the rules that are used in order to learn.

The But before we start analyse the three these area
s, it would be really helpful
to understand what learning means. As said many times before the artificial neural
networks are strongly connected with the biological neural networks. Learning is the
knowledge and the experience that is gained by studying an
d training.


3.3.1

Learning Models

We can separate the learning techniques of a neural network in to main
categories. These two types of learning are supervised and unsupervised. The neural
networks categorisation which involve supervised learning means t
hat certain output
nodes are trained to respond to certain “exemplar” patterns and the changes in
connection weights due to learning cause those same nodes to respond to more general
classes of patterns. Other models involve unsupervised learning; that is,

input patterns
are presented in some sequence and the network discovers through self
-
organisation a
natural categorisation of the sensory world.


3.3.1.1

Supervised Learning

The idea behind that model is to compare the output with the desired output. Th
e
use of neural network for supervised learning dates back to the early work of Rosenblatt.
The perceptron theorem he states is: “
Given an

-
perceptron, a stimulus world W, and
any classification C(W) for which a solution exists, let all stimuli in W occur

in any
Neural Network
-
Based Face Recognition, using ARENA algorithm.


22

sequence, provided that each stimulus must reoccur in finite time. Then beginning from
arbitrary initial state an error correction procedure will always yield a solution to C(W)
in finite time


What the model is trying to do is to produce a close m
ach as possible, to the
desired output. Training procedures to minimise the error between the current output and
the desired does that. The training of the network is done by giving to the neural network
a data set. So with every input we give the correspo
nding desired output, is given as well
so the network can figure out the error, if any between the “calculated” output and the
desired one. The training of the network is complete when for every input, we give we
get the same output as the desired one. Som
etimes the supervised categorisation is called
classification. Some of the important neural network supervised models are


3.3.1.2

Unsupervised Learning

The types of neural networks that don’t use any external human influence, or any
data set, in order to
adjust their weights are called unsupervised. Unsupervised learning
models are sometimes called self
-
supervised learning or clustering.

As mentioned before the supervised models give a result as well as on the correct
output. In an unsupervised learning pr
ocedure no correct output is provided to the
network. Presentation of an output is accompanied by depend only on the actual state
that the system is arrived and not on any externally provided correct output. The network
is looking for patterns in the input

signals and then makes matching according to the
function of the network. The network is self
-
organising it self by getting information if
the result is correct but even if is wrong. An unsupervised learning algorithm might
emphasise co
-
operation among cl
usters of processing elements. In such a scheme, the
clusters would work together and try to stimulate each other. O1

Even there are more, neural networks modelled on unsupervised classification,
these days, unsupervised learning still not well understood

and more new models still on
research. Supervised learning procedures have achieved a reputation for producing good
results in practical applications, which are gaining in popularity.



Neural Network
-
Based Face Recognition, using ARENA algorithm.


23

3.3.1.3

Learning Rules

We can categorise the rules for learning in tw
o parts. Rules for associative
learning and rules related to changes in the node activities. Associative rules started in
the late 1960’s. Early efforts in this regard include some non
-
linear models and some
models, which started out being linear and late
r added some non
-
linearity.



3.3.2

Learning Rates and Training

The fact of how fast is the neural network on learning depends on many different
things. Is not always good to have fast learning systems. That is because sometimes a
neural network, which is
learning in fast rates might not make the fine distributions as a
slower learning neural network. The rate of learning depends on the training techniques,
the learning rules, but also the complexity and the size of the system. The learning rule
and the mod
ification of the weights play a smaller but sometimes lengthy role in the
training effort.

The training techniques of a neural network are mainly different that the
common programmed system. The differences is that in common systems the
programmer gives t
he output and by having the correct output try to develop the
procedures and functions in order to achieve the desired result. But when neural
networks are trained to determine the relationship between input and output by looking
at examples of many input
-
output pairs. Pairs of input and outputs are applied to the
neural network. These pairs of data are used to train the network. Knowing what output
is expected, from each input the network learns by automatically adjusting or adapting
the strengths of th
e connections between artificial neurons.


3.4

Back Propagation


Generalising the Widrow
-
Hoff learning rule to multiple
-
layer networks and non
-
linear differentiable transfer functions created back propagation. Input vectors and the
corresponding output ve
ctors are used to train a network until it can approximate a
function, associate input vectors with specific output vectors, or classify input vectors in
an appropriate way as defined by you.

Neural Network
-
Based Face Recognition, using ARENA algorithm.


24

Networks with biases, a sigmoid layer, and a linear output layer

are capable of
approximating any function with a finite number of discontinuities. Standard back
propagation is a gradient descent algorithm, as is the Widrow
-
Hoff learning rule. There
are a number of variations on the basic algorithm, which are based on
other standard
optimisation techniques, such as conjugate gradient and Newton methods.

The term back propagation refers to the process by which derivatives of network
error, with respect to network weights and biases, can be computed can compute
derivativ
es of network error. This process can be used with a number of different
optimisation strategies. The architecture of a multilayer network is not completely
constrained by the problem to be solved. The number of inputs to the network is
constrained by the
problem, and the number of neurons in the output layer is constrained
by the number of outputs required by the problem. However, the number of layers
between network inputs and the output layer and the sizes of the layers are up to the
user.

The two
-
layer
that is used for the project sigmoid/linear network can represent
any functional relationship between inputs and outputs if the sigmoid layer has enough
neurons.



3.4.1

Architecture


In this section we want to present the architecture of the network which is mo
st
commonly used with the back propagation algorithm, the multilayer feedforward
network.


3.4.1.1

Neuron Model

Each of the inputs is weighted with an appropriate weight. The sum of the
weighted inputs and the bias forms the input to the transfer function
f. Neurons may use
any differentiable transfer function f

to generate their output. Most of the multilayer
networks are using the log
-
sigmoid transfer function or the tan
-
sigmoid transfer function
or the linear transfer function.



Neural Network
-
Based Face Recognition, using ARENA algorithm.


25



Figure 7: Two layer n
eural network


The log
-
sigmoid function generates outputs between 0 and 1 as the neuron’s net
input goes from negative to positive infinity. Alternatively, multilayer networks may use
the tan
-
sigmoid transfer function. Occasionally, the linear transfer fun
ction is used in
back propagation networks at the output layer. That is what we are going to use for our
neural network. That is because if we had sigmoid output neurons, then the outputs of
the network would be limited to a small range. If linear output n
eurons are used the
network outputs can take on any value.

The three transfer functions described here are the most commonly used transfer
functions for back propagation, but other differentiable transfer functions can be created
and used with back propaga
tion if desired.



3.4.2

Feedforward Network


Feedforward networks often have one or more hidden layers of sigmoid neurons
followed by an output layer of linear neurons. Multiple layers of neurons with non
-
linear
transfer functions allow the network to learn non
-
linear and linear relationships between
input and output vectors. The linear output layer lets the network produce values outside
the range

1 to +1. On the other hand, if it is desirable to constrain the outputs of a
network then the output layer should

use a sigmoid transfer function such as logsig.

Neural Network
-
Based Face Recognition, using ARENA algorithm.


26

Furthermore in the case of multiple
-
layer networks we use the number of the
layers to determine the superscript on the weight matrices. The appropriate notation is
used in the two
-
layer tansig or purelin net
work shown next.




Figure 8: Feedforward neural network.


A feed forward network can be used as a general function approximation. It can
approximate any function with a finite number of discontinuities, arbitrarily well, given
sufficient neurons in the h
idden layer.

In order to create a feedforward neuron network we have to follow a specific
procedure. The first step in training a feedforward network is to create the network
object. Then we have to i
nitialise the weights and the bias. Then the network is

ready to
be trained. A feedforward network takes as said, before an object as input and returns a
network object with all weights and biases initialised. There is a more detailed analysis
of the network and the process we follow in the coding part of the
report.



Neural Network
-
Based Face Recognition, using ARENA algorithm.


27

C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R



4
4
4



4.1

Databases


The databases, which are used for the project, are standard databases of the
University of Surrey, Olivetti
-
Oracle Research Lab and FERET. Thought is possible to
test the algorithm with other databases as well.

The d
atabases consist of more than 400 images, each. All the databases contain
images of different people, but in sets. That means that there are a number of images of
the same people in each of them. Though each image is different from each other. For
example
, in the ORL database we have ten different images of each of 40 distinct
subjects. For some people, the images were taken at different times, with different
lighting, where we might have facial expressions, with open or closed eyes, where the
people are s
miling or not and facial details, glasses or with out no glasses. Many images
of a person can be acquired in a few seconds. Given sufficient data, it becomes possible
to model class
-
conditional structure, i.e. to estimate probability densities for each
per
son.


Figure 9: Images from ORL

Two images of the same person in
different pose, and facial
orientation, with very simple
back
ground.



The above images with reduced
resolution (16x16).

Neural Network
-
Based Face Recognition, using ARENA algorithm.


28





Figure 10: Images from FERET .




Apart from that, in all the databases images were taken against a dark
homogeneous background with the subjects in an upright, frontal position, but also we
have images with more comp
lex backgrounds. On of the most important aspects of the
databases is the variation of the pose. There is a limitation of

20


at the posing angle. If
the person’s pose, in the image is more than then is nearly impossible to be detected
from mainly any of
the existing face recognition algorithms. Thought in the databases
we have posing angle variations but with in the limits.

The files of the images that are used are in TIFF format, and can conveniently be
viewed on UNIX, TM systems using the xv program. M
ost of the images have size of
92x112 pixels, with 256 grey levels per pixel.



4.2

Test and Train Sets



The data sets that have been used for the particular project are divided to two
sub sets. The first is the training set that contains the images that we
re used in order to
train the algorithm and the neural network. Training sets are used from the two training
Two images of the same person in
different pose, and facial
orientation, different background.



The above images with reduced
resolution (16x16).

Neural Network
-
Based Face Recognition, using ARENA algorithm.


29

programs, arntrn and nntrn. Samples of the set can be found in appendix II. The other set
is the other subset is the testing database, which contai
ns different images than the
training set but of the same people.

In MATLAB we use the commands
imread

and
imresize

in order to read the
images and reduce the resolution. More detailed description of the commands and their
properties is given in the code
implementation chapter.

Neural Network
-
Based Face Recognition, using ARENA algorithm.


30

C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R



5
5
5



5.1

Algorithms for face recognition


As mentioned in the introduction but also in other parts of the report, there are
many algorithms that can be used for face recognition. Most of them are based on the
same techniques

and methods. Some of the most popular are Principal component
analysis and the use of eigenfaces.


5.1.1

Principal Component Analysis

On the field of face recognition most of the common methods employ Principal
Component Analysis. Principal Component Anal
ysis is based on the Karhunen
-
Loeve
(K
-
L), or Hostelling Transform, which is the optimal linear method for reducing
redundancy, in the least mean squared reconstruction error sense. 1. PCA became
popular for face recognition with the success of eigenfaces
.

The idea of principal component analysis is based on the identification of linear
transformation of the co
-
ordinates of a system. “The three axes of the new co
-
ordinate
system coincide with the directions of the three largest spreads of the point
distri
butions.”

In the new co
-
ordinate system that we have now the data is uncorrected with the data we
had in the first co
-
ordinate system. [2]

For face recognition, given dataset of
N

training images, we create
N

d
-
dimensional vectors, where each pixel is a un
ique dimension. The principal components
of this set of vectors is computed in order to obtain a
d x m

projection matrix,
W
. The
image of the
i
th

vector may be represented as weights:

T
im
i
i
i
)
,...,
2
,
1
(






(1)

Neural Network
-
Based Face Recognition, using ARENA algorithm.


31


Such that



W
i
x



(2)

Approximates the original image where


is the mean, of the

i

and the reconstruction is
perfect when
m = d
. P1

As mentioned before the ARENA algorithm is going to be tested and its
performance is going to be compa
red with other algorithms. For the comparison we are
going to use two different PCA algorithms. The first algorithm is computing and storing
the weight of vectors for each person’s image in the training set, so the actual training
data is not necessary. In

the second algorithm each weight of each image is stored
individually, is a memory
-
based algorithm. For that we need more storing space but the
performance is better.

In order to implement the Principal component analysis in MATLAB we simply
have to use t
he command
prepca.

The syntax of the command is

ptrans,transMat = prepca(P,min_frac)

Prepca pre
-
processes the network input training set by applying a principal
component analysis. This analysis transforms the input data so that the elements of the
input
vector set will be uncorrected. In addition, the size of the input vectors may be
reduced by retaining only those components, which contribute more than a specified
fraction (
min_frac
) of the total variation in the data set.

Prepca takes these inputs the
matrix of centred input (column) vectors, the
minimum fraction variance component to keep and as result returns the transformed data
set and the transformation matrix.


5.1.1.1

Algorithm

Principal component analysis uses singular value decomposition to c
ompute the
principal components. A matrix whose rows consist of the eigenvectors of the input
covariance matrix multiplies the input vectors. This produces transformed input vectors
whose components are uncorrected and ordered according to the magnitude of

their
variance.

Neural Network
-
Based Face Recognition, using ARENA algorithm.


32

Those components, which contribute only a small amount to the total variance in
the data set, are eliminated. It is assumed that the input data set has already been
normalised so that it has a zero mean.


In our test we are going to use
two different “versions’ of PCA. In the first one
the centroid of the weight vectors for each person’s images in the training set is
computed and stored. On the other hand in PCA
-
2 a memory based variant of PCA, each
of the weight vectors in individually c
omputed and stored.


5.1.2

Eigenfaces

Human face recognition is a very difficult and practical problem in the field of
pattern recognition. On the foundation of the analysis of the present methods on human
face recognition, a new technique of image feature

extraction is presented. And
combined with the artificial neural network, a new method on human face recognition is
brought up. By extraction the sample pattern's algebraic feature, the human face image's
eigenvalues, the neural network classifier is trai
ned for recognition. The Kohonen
network we adopted can adaptively modify its bottom up weights in the course of
learning. Experimental results show that this method not only utilises the feature aspect
of eigenvalues but also has the learning ability of n
eural network. It has better
discriminate ability compared with the nearest classifier. The method this paper focused
on has wide application area. The adaptive neural network classifier can be used in other
tasks of pattern recognition.

In order to calcul
ate the eigenfaces and eigenvalues in MATLAB we have to use
the command eig
.

The syntax of the command is

d = eig(A)

V,D = eig(A)

V,D = eig(A,'nobalance')

d = eig(A,B)

V,D = eig(A,B)


d = eig(A)
returns a vector of the eigenvalues of matrix
A
.
V,D = eig(A
)

produces
matrices of eigenvalues (
D
) and eigenvectors (
V
) of matrix
A
, so that
A
*
V

=

V
*
D
.
Neural Network
-
Based Face Recognition, using ARENA algorithm.


33

Matrix
D

is the
canonical form

of
A,
a diagonal matrix with
A
's eigenvalues on the main
diagonal. Matrix
V

is the
modal matrix
, its columns are the eigenvectors of
A
. The
eigenvectors are scaled so that the norm of each is 1.0. Then we use
W,D = eig(A'); W =
W'

in order to compute the
left eigenvectors
, which satisfy
W
*
A

=

D
*
W
.

V,D = eig(A,'nobalance')

finds eigenvalues and eigenvectors without a preliminary
balanc
ing step. Ordinarily, balancing improves the conditioning of the input matrix,
enabling more accurate computation of the eigenvectors and eigenvalues. However, if a
matrix contains small elements that are really due to round
-
off error, balancing may
scale
them up to make them as significant as the other elements of the original matrix,
leading to incorrect eigenvectors. We can use the
no balance

option in this event.

d = eig(A,B)

returns a vector containing the generalised eigenvalues, if
A

and
B

are
squar
e matrices.
V,D = eig(A,B)

produces a diagonal matrix
D

of generalised
eigenvalues and a full matrix
V

whose columns are the corresponding eigenvectors so
that
A
*
V

=

B
*
V
*
D
. The eigenvectors are scaled so that the norm of each is 1.0.



5.1.3

Euclidean dis
tance

One of the ideas on which face recognition is based is the distance measures,
between to points. The problem of finding the distance between two or more point of a
set is defined as the Euclidean distance. The Euclidean distance is usually referred
to the
closest distance between two or more points. So we can define the Euclidean distance d
ij
between points x x
ik

and x
jk

as :


)
(
2
1
x
x
d
jk
ik
p
k
ij





(3)

[3]


5.2

Arena Algorithm


As mentioned before in the introduction, the algorithm that is used in
the project
is called ARENA. What we want to develop with that algorithm is a technique that is
Neural Network
-
Based Face Recognition, using ARENA algorithm.


34

extremely simple. Is a memory
-
based technique for view
-
based frontal face recognition
that can outperform more sophisticated algorithms that use Principal Compo
nents
Analysis and neural networks. This method does not perform any complex feature
extraction, nor does it incorporate any face
-
specific information. The ARENA algorithm
technique is closely related to correlation templates. However, we show that the use

of
novel distance metrics greatly improves the performance. We show that augmenting the
memory base with additional, synthesised face images results in further improvements in
performance.

The technique is going to be tested on standard face recognition d
atabases, and
direct comparisons with other techniques will show that our algorithm achieves
comparable or superior results.

Arena algorithm has also a good asymptotic computation and storage behaviour,
and is ideal for incremental training. The system ha
s been integrated with a neural
-
network based face detection system into a real
-
word visitor identification system that
has been operating successfully in an outdoor environment with uncontrolled lighting for
several months.


5.2.1

The algorithm

Arena is
, as mentioned, a memory based algorithm that employs reduced
resolution images, like in Principal Component Analysis, 16x16 and the a parameter of
similarity measure L
0
*
. One of the most important parts of the system is to reduce the
resolution of the ima
ge. That is achieved by averaging over non
-
overlapping rectangular
regions in the image. The aim of the system is to find the exact mach of an image from
the given datasets, so the distance from the query image to each of the datasets stored
images is comp
uted and the best much is returned.



The key point of the algorithm for its good performance is the L
p
*

similarity
measure. The measure that is used has a better performance than the Euclidean distance.
Lp
*

is defined as

p
p
a
i
a
Lp
/
1
)
(
)
(
|
|



(4)

The E
uclidean is defined for p=2 so that we have:

Neural Network
-
Based Face Recognition, using ARENA algorithm.


35

).
(
2
y
x
L


(5)

Because we are not interested in the actual distances, but only in the ordering we can say
that equation 3 becomes:

p
p
a
i
a
Lp
/
1
)
(
)
(
|
|



(6)

For each reduced resolution image w
e have is converted to a vector,
x
, where each pixel
in the image is represented as a component of the vector. So since the individual pixel
intensities are noisy, we can define the similarity measure for p=0 as:

1
)
(
|
|
0






yi
xi
y
x
L

(7)

where


is a threshold value, such that pixels whose intensities differ by less than


are
considered equivalent. [P1]



Neural Network
-
Based Face Recognition, using ARENA algorithm.


36

C
C
C
H
H
H
A
A
A
P
P
P
T
T
T
E
E
E
R
R
R



6
6
6



6.1

Coding


The main part of the project is the implementation of the algorithm. The coding
process of the p
roject has mainly two parts. First the implementation of the arena
algorithm and secondly the implementation of a neural network which is using the
outputs of ARENA. The coding of ARENA algorithm is the implementation of the
mathematical forms that calcula
te the distance between two or more images.
Furthermore we have to develop two programs for the neural network one for the
training process and one for testing. Each of those programs is using as input the output
of ARENA, the distance measures of the imag
es, from the training and testing set.


Before starting the analysis of the implementation is good to give s quick
preview of the programming language that was used in order to code the algorithm and
the neural network, which is MATLAB.


6.2

MATLAB


MATL
AB is an integrated technical computing environment that combines
numeric computation, advanced graphics and visualisation, and a high
-
level
programming language. MATLAB includes hundreds of functions for:

• Data analysis and visualisation

• numeric and
symbolic computation

• engineering and scientific graphics

• Modelling, simulation, and prototyping

• programming, application development, and GUI design


Neural Network
-
Based Face Recognition, using ARENA algorithm.


37

“MATLAB is used in a variety of application areas including signal and image
processing, control

system design, financial engineering, and medical research. The
open architecture makes it easy to use MATLAB and companion products to explore
data and create custom tools that provide early insights and competitive advantages. An
overview of The MathWor
ks product family is available”.I6


6.3

Implementing ARENA algorithm


As was mentioned in the introduction of the chapter first we have to implement
the ARENA algorithm in order to use its results as input to the neural network. The fact
that we have to tr
ain and test the neural network is making the development of the
algorithm more complicated. That is because the inputs of the training and testing
“programs” are different even thought we use ARENA in both cases in order to produce
them. That is because f
or training the neural network we take the distances of each
image from its self and the rest of the images. For testing the neural network we use the
distances of one selected image, from the test set, from the images in the training
dataset.

In order to
produce the input for training we use the program
arntrn.m

and for
testing the program
arntst.m

. Below we can see the code of those two programs, which
is very similar. In both cases the user is asked to insert values for the three most
important paramet
ers, p, the size of the databases (test and train) and the resolution.




power = input(
'insert
a value for p: '
);

data = input(
'insert number of images in training and testing set: '
);

rez = input(
'insert resolution: '
);


Neural Network
-
Based Face Recognition, using ARENA algorithm.


38



Figure 11:

User inputs for p, number of images in sets and resolution.


After that the program is reading the images from the specific directories, train or
test. The command t
hat we use for that is
imread
. Imread re
ads an image from a
graphics file a
nd returns a matrix with the gray value of each pixel.


The
for

loop is used in order to read all the images in the data directory.

Then the resolution of each image is reduced to t
he scale that the user entered above. We
can do that by using the command
imresize
. Imresize is reducing the resolution to
rez by
rez

by using,
'nearest'
, nearest neighbour interpolation.




for
j = 1:data

string =
'train
\
'
num2str(j)
'.tif'
;
% or'test
\
' num2str(j) '.tif';



Train_Aimages{j} = imread(string);

end
;


for
j = 1:data


Train_RAimages{j}= imresize(Train_Aimages{j},rez,rez,
'nearest'
);

end
;


Neural Network
-
Based Face Recognition, using ARENA algorithm.


39




Figure 12:
Reduced resolutions to 16x16, 32x32 and 256x256


Now is time to calculate the distance L between the images. As described in the
analysis of the algorithm if the value of p is 0 then we get L equal with 1. If not then we
have to calculate it using the formula (7). So we have



First we calculate the d
ifference between two images by substituting the two
matrixes we generated from imread, and we produce a matrix x by x, where x is the
resolution value. We have to double the range of the matrix values from [0 256] to [
-
256
256].



Then we take the absolu
te value of the matrix elements and we take it to the
power of p. after that we sum the columns of the matrix and we get a 1 by X vector of
which the sum is the distance L
p
.




if
power==0;

Train_FSum{j,i}= 1;

else


Train_X{j,i}=double(Train_RAimages{j})
-

double(Train_RBimages{i});


Neural Network
-
Based Face Recognition, using ARENA algorithm.


40



The outputs we get are of the form n by n for
arntrn

and 1 by n for
arntst
,

where
n is the number of images that we use. For example if we run arntrn for p=1 using five
images from the training dataset and resolution 16x16 we have the following 5x5 output.



Figure13: 5x5 output matrix


6.4

Implementation of the Neural Networks


Now is time to run the neural networks programs. First the training and then the
testing.


6.4.1

Training


The training program of the neural network is
nntrn.m
. The first thing when we
run the program is to input the number of the neurons in the output l
ayer of the network.

Train_Y{j,i} = abs(Train_X{j,i});


Train_Matrix_Sum{j,i} = sum((Train_Y{j,i})^power);



Train_Vector_Sum{j,i} = sum(Train_Matrix_Sum{j,i});



Train_FSum{j,i}

=Train_Vector_Sum{j,i};

end
;


Neural Network
-
Based Face Recognition, using ARENA algorithm.


41


Create a feed
-
forward back propagation network by using the command
newff
. In
newff we have to define an

n

x
2

matrix, where n is the number of images, of min=
-
1,
and max=1, values for n input elements. Also the size of hidden layer

that we use and
the number of output neurons that are user defined. Further more we define the
t
ransfer
function of the hidden layer, which is '
tansig
' and ' purelin ' for the last output layer.
Finally we define the network’s training function. What we g
et from that is an
N

layer
feed
-
foward back
-
propagation network.


There is a possibility to have as input to the network or output for the neural
network, five or ten output neurons. That means that we have the to define to the neuron
network’s input ran
ge each time we change the number of inputs or outputs




Before train or test the network we have to initialise the weights and the bias of each
layer. The initial values are chosen randomly.





classes = input(
'insert number of classes: '
);


net=newff([
-
1 1;
-
1 1;
-
1 1;
-
1 1;
-
1 1;
-
1 1;
-
1 1;
-
1 1;
-
1 1;
-
1
1;],[
10,classes],{
'tansig'
,
'purelin'
},
'traingd'
);

net.layers{1}.initFcn =
'initwb'
;

net.inputWeights{1,1}.initFcn =
'rands'
;

net.biases{1,1}.initFcn =
'rands'
;

net.biases{2,1}.initFcn =
'rands'
;

net=init(net);


if
data==5


net = newff([
-
1 1;
-
1 1;
-
1 1;
-
1 1;
-
1
1;],[10,classes],{
'tansig'
,
'purelin'
},
'traingd'
);

elseif
data==10



net = newff([
-
1 1;
-
1 1;
-
1 1;
-
1 1;
-
1 1;
-
1 1;
-
1 1;
-
1 1;
-
1 1;
-
1
1;],[10,classes],{
'tansig'
,
'purelin'
},
'traingd'
);

end
;


Neural Network
-
Based Face Recognition, using ARENA algorithm.


42


The next step is to define the learning functions tha
t the weights and the bias are
going to use in order to update their values.




After initialising the weights we give to the network the output distances of the
training set, that we have from the
arntrn.m

program in order to train the network.




Befor
e training the network we have to define a target for the network which is
also going to be a matrix but of dimensions n by m, with n equal to the images we want
train and m the number of the output neurons of the network.

net.biases{1,1}.learnFcn =
'learngd'
;

net.biases{2,1}.learnFcn =
'learngd'
;

net.layerWeights{2,1}.learnFcn =
'learngd'
;

net.layerWeights{1,1}.learnFcn =
'learngd'
;

n
et.adaptParam.passes = 500;


for
q=1:data


for
w=1:data


p(q,w)=Train_FSum{q,w};


end
;

end
;


for
y=1:classes


for
u=1:classes


if
y==u


t(y,u)=0;


else


t(y,u)=2000000;


end
;


end
;

end
;

if
classes<data

for
y=1:classes


for
u=classes+1:data


if
u==y+5


t(y,u)=0;


else


t(y,u)=2000;


end
;


end
;

end
;

end
;


Neural Network
-
Based Face Recognition, using ARENA algorithm.


43




The input matrix and the tar
get of the neural network have to be normalised and
then trained. In order to normalise the to matrixes we use
premnmx

and the values that
are produced from that are used for training the network and update the weights and
bias.
Given an input sequence wit
h t

steps the network is updated as follows. Each step
in the sequence of inputs is presented to the network one at a time. The network's weight
and bias values are updated after each step, before the next step in the sequence is
presented. Thus the networ
k is updated t times.





The two output values we get from the training procedure is the networks output
and the error of the inputs and the target.

Output a for example, for p =1, by using five images with resolution 16x16 is

-
0.7430

0.8394

1.4159

0.64
31

0.8089

0.7951

-
0.7758

0.9397

1.0016

1.0809

0.9969

1.0437

-
0.7726

0.8577

0.9106

0.8782

1.1213

0.5370

-
0.4840

0.8249

0.9293

0.9874

1.1417

0.9298

-
1.0342



And the error

-
0.2570

0.1606

-
0.4159

0.3569

0.1911

0.2049

-
0.2242

0.0603

-
0.0016

-
0.0809

0.00
31

-
0.0437

-
0.2274

0.1423

0.0894

0.1218

-
0.1213

0.4630

-
0.5160

0.1751

0.0707

0.0126

-
0.1417

0.0702

0.0342




From these two matrixes we have to notice the diagonal values and especially the
values of output matrix. In the diagonal of the output matrix,
we have the distances of
each image from its self. That means that, each of these values must be the smallest on
each of the lines. For the error we have to mention that is the output of the substitution
pn,minp,maxp,tn,mint,maxt = premnmx(p,t);

net,a,e = adapt(net,pn,tn);


Neural Network
-
Based Face Recognition, using ARENA algorithm.


44

of the output and the target table of which the dia
gonal is

1, and the rest of the elements
1.



But in order to make sure that the network is well trained and the weights and
bias values are set correctly we have to simulate it with the network with the same input
we trained it. If the result is going to

be the same means that the network is well trained.
So we have




and the output is indeed the same, and with correct results.

-
0.7430

0.8394

1.4159

0.6431

0.8089

0.7951

-
0.7758

0.9397

1.0016

1.0809

0.9969

1.0437

-
0.7726

0.8577

0.9106

0.8782

1.1213

0
.5370

-
0.4840

0.8249

0.9293

0.9874

1.1417

0.9298

-
1.0342


Better analysis of the results and the expected outcomes of the network are made
in the next chapter.


6.4.2

Testing


The testing program is the shortest and is very simple to implement. What we
have

to do is to take the output that we generate from the ARENA testing program. That
would be a vector with n elements where n is the number of images that we use from the
training set. The problem is that the network needs an m by n matrix input, with n equ
al
to the classes we have and m the number of images. In order to do that, we repeat the
output vector n times.


b= sim(net,pn)

for
q=1:classes


for
w=1:data


tp(q,w)=Test_FSum{w};


end
;

end
;


Neural Network
-
Based Face Recognition, using ARENA algorithm.


45

So if our vector was


9299


9357


10377


10068

10796


The input for the neuron testing would be

9299


9357


10377


10
068

10796

9299


9357


10377


10068

10796

9299


9357


10377


10068

10796

9299


9357


10377


10068

10796

9299


9357


10377


10068

10796




Then we normalise again the input of the network and the t
arget and we test it by
using the simulation command and the testing input.








The full implementation code, for all four programs, is in Appendix I.








tpn,minp,maxp,tn,mint,maxt = premnmx(tp,t);