Vision-Based Biometric Authentication

connectionviewAI and Robotics

Nov 17, 2013 (3 years and 4 months ago)

190 views




Vision
-
Based Biometric Authentication
System






Padraic o hIarnain


B.E. Electronic and Computer Engineering

Project Report





Project Supervisor: Mr. Frank Callaly



March 2007
Vision
-
Based Biometric Authentication System


ii

Final Year Project, Dept. of Electronic Engineering, NUI Galway

D
eclaration of Originality

I hereby declare
that this thesis is my

orig
inal work except where stated
.

______________ ____________________







Date Signature


Vision
-
Based Biometric Authentication System


iii

Final Year Project, Dept. of Electronic Engineering, NUI Galway

Abstract

PC security is a big issue in the world today. Password and PIN authenticat
ion systems are outdated and
insecure. Recent advances in Biometric technology pave the way for a new vision
-
based type of
authentication system

that is more secure and easier to use.


The aim of this project was to
design and develop a vision based biomet
ric authentication system for PCs.
This system would enable users to sit down at a PC and be authenticated by their facial features. This face
recognition authentication system involves capturing images using a webcam, processing the captured
images and de
tecting the presence of a face.
This face is then analysed using Principal Component
Analysis (PCA) in order to
compare

the face

to known users
.

Verifying a face as a user allows that user to
be logged on to the PC
. This is achieved by integrating the face

recognition program with the PC’s
authentication system.

Vision
-
Based Biometric Authentication System


iv

Final Year Project, Dept. of Electronic Engineering, NUI Galway

Acknowledgements

I would firstly like to thank my supervisor, Mr Frank Callaly for his help throughout the course of the
project. I would also like to thank Mr. Martin Burke for his help and thanks
to all my friends and
classmates for their assistance.

Vision
-
Based Biometric Authentication System


v

Final Year Project, Dept. of Electronic Engineering, NUI Galway

Table of Contents

Declaration of Originality

................................
................................
................................
.............................

ii

Abstract

................................
................................
................................
................................
........................

iii

Acknowledgements

................................
................................
................................
................................
......

iv

Table of Contents

................................
................................
................................
................................
..........

v

List of Figures

................................
................................
................................
................................
.............

vii

Introduction

................................
................................
................................
................................
...................

1

Chapter 1 Software and Tools

................................
................................
................................
.......................

7

1.1 Linux

................................
................................
................................
................................
...................

7

1.1.1 Overview

................................
................................
................................
................................
......

7

1.1.2 Why Linux?

................................
................................
................................
...............................

10

1.1.3 Red Hat


Fedora Core
................................
................................
................................
...............

10

1.2 GNOME

................................
................................
................................
................................
............

10

1.3 GCC (GNU Compiler Collection)

................................
................................
................................
....

12

1.4 C Programming Language

................................
................................
................................
................

13

1.4.1 Overview

................................
................................
................................
................................
....

13

1.4.2 Characteristics of C

................................
................................
................................
....................

13

1.5 OpenCV

................................
................................
................................
................................
............

14

1.5.1 Overview

................................
................................
................................
................................
....

14

1.5.2 OpenCV contents

................................
................................
................................
.......................

15

1.6 Web camera

................................
................................
................................
................................
......

16

Chapter 2 Face Detection

................................
................................
................................
............................

18

2.1 Method of Face Detection

................................
................................
................................
.................

18

2.1.1 Viol
a

Jones Algorithm for Face Detection

................................
................................
................

18

2.1.2 Face Detection Program

................................
................................
................................
.............

20

2.1.3 Face Detection with a USB camera

................................
................................
...........................

22

2.2 Testing and implementation

................................
................................
................................
..............

23

Chapter 3 Authentication

................................
................................
................................
............................

26

3.1 PAM

................................
................................
................................
................................
..................

26

3.1.1 Configuring an application for PAM

................................
................................
.........................

26

3.1.2 Creating a PAM module

................................
................................
................................
............

28

3.2 Implem
enting Biometric programs with Authentication

................................
................................
..

30

3.3 Integrating PAM with Biometric system

................................
................................
..........................

30

3.4 Testing and Implementation
................................
................................
................................
..............

31

Vision
-
Based Biometric Authentication System


vi

Final Year Project, Dept. of Electronic Engineering, NUI Galway

Chapter 4 Face recognition

................................
................................
................................
.........................

34

4.1 PCA

................................
................................
................................
................................
...................

34

4.1.1 Why use PCA?

................................
................................
................................
...........................

34

4.1.2 Theory of PCA

................................
................................
................................
...........................

35

4.1.3 Mathematics of PCA

................................
................................
................................
..................

36

4.2 PCA using OpenCV

................................
................................
................................
..........................

38

4.2.1 Eigenvector computation

................................
................................
................................
...........

38

4.2.2 Calculating decomposition coefficients

................................
................................
.....................

40

4.2.3 Calculating the Euclidean distance

................................
................................
............................

40

4.3 Testing and I
mplementation
................................
................................
................................
..............

41

Discussion

................................
................................
................................
................................
...................

45

Conclusion

................................
................................
................................
................................
..................

48

Appendix

................................
................................
................................
................................
.....................

49

References

................................
................................
................................
................................
...................

50


Vision
-
Based Biometric Authentication System


vii

Final Year Project, Dept. of Electronic Engineering, NUI Galway

List of Figures

Figure 0
-
1 : Example of face recognition [4]

................................
................................
................................

2

Figure 0
-
2 : Biometric Market Technology [7]

................................
................................
............................

3

Figure 0
-
3 : Overall Project Structure

................................
................................
................................
...........

5

Figure 1
-
1 : Linux Logo

................................
................................
................................
................................

7

Figure 1
-
2 : CLI being used from a terminal

................................
................................
................................

8

Figure 1
-
3 : Fedora logo

................................
................................
................................
.............................

10

Figure 1
-
4 : GNOME Logo

................................
................................
................................
.........................

11

Figure 1
-
5 : GNOME Desktop environmen
t

................................
................................
...............................

11

Figure 1
-
6 : GNOME login screen

................................
................................
................................
..............

12

Figure 1
-
7 : The C programming specification

................................
................................
...........................

13

Figure 1
-
8 : Labtec Quickcam

................................
................................
................................
....................

17

Figure 2
-
1 : 14
different Haar
-
like features [18]

................................
................................
.........................

19

Figure 2
-
2 : Formula for Summed Area Table [18]

................................
................................
....................

19

Figure 2
-
3 : Two node classifier [18]

................................
................................
................................
..........

19

Figure 2
-
4 : Three node classifier [1
8]

................................
................................
................................
........

20

Figure 2
-
5 : Result of face detection in an image

................................
................................
.......................

23

Figure 2
-
6 : Result of face detection from a camera

................................
................................
...................

24

Figure 2
-
7 : Improved face detection result

................................
................................
................................

25

Figure 4
-
1 : Results of face recognition

................................
................................
................................
......

42

Figure 4
-
2 : Another result of face recognition

................................
................................
...........................

43

Figure 4
-
3 : Result of face recognition using a camera

................................
................................
..............

43

Figure 4
-
4 : Result of face recognition using a camera

................................
................................
..............

44

Vision
-
Based Biometric Authentication System


1


Final Year Project, Dept. of Electronic Engineering, NUI Galway

Introduction


Biometrics consist of technologies that support automatic identification or verification of identity based
on behavioral or physical traits


[3].
Biometrics
validates

identities
by calculating

unique individual
characteristics.
The most predominant areas of biometrics involve

fingerprints,

iris and facial
characteristics, hand geometry, retina, voice and touch.


Over the last twenty years or s
o there has not
been a significant market for Biometric technologies.

They have been consigned to infrequent use in films
and in some high
-
security government or military installations
.
Nowadays, Biometrics is increasing it’s
stronghold in many aspects of
both public and private life
.
For example, in some cases in the computer
industry, Biometrics is replacing the more conventional personal identification numbers (PIN) and
passwords
.
Although Password/Pin systems and Token systems are still the most common
person
verification and identification methods, trouble with forgery, theft and lapses in users’ memory pose a
very real threat to high security environments which are now turning to biometric technologies to
alleviate this potentially dangerous threat. Th
e areas in which Biometrics are gaining the most support is
in the protection of restricted areas, both commercial and domestic.


This project is based on the face recognition method of Biometric authentication.
Although other
Biometric methods such as fi
ngerprints and iris scans are more accurate methods of identification
;

facial
recognition provides an ingrained human backup because we instinctively recognize one another.

Face
recognition is the process in which the facial features of someone are recogni
zed and then matched to one
of the many faces in a database. “The intuitive way to do face recognition is to look at the major features
of the face and compare them to the same features on other faces.”[2].


The history of face recognition dates back to
the 1960’s when a semi
-
automated method was used to
compare facial features. First the key features in the photograph were marked by hand; key features
included eyes, ears, nose and mouth. Then the distances and ratios between these marks and a common
re
ference point were computed and these values were then compared to reference data of other faces. In
the early 1970’s Goldstein, Harmon and Lesk [2] created a face recognition system using 21 particular
markers e.g. hair color and lip thickness. This meth
od was less computerized then the previous method
because many of the measurements had to be made entirely by hand.


The next step in face recognition was made by Fisher and Elshlagerb [2] in the early 1970’s. They
measured the key features in a face using

templates of the features of the different parts of the face. They
Vision
-
Based Biometric Authentication System


2

Final Year Project, Dept. of Electronic Engineering, NUI Galway

then plotted all the pieces on to a general template. Even though this method was more automated then
the previous it proved to be too inconclusive as the features used did not include eno
ugh distinctive data
to represent a face.


Kirby and Sirovich pioneered the eigenface approach in 1988 at Brown University. It was the first
genuinely successful system for automatic recognition of human faces. The system functions by
projecting face image
s onto a feature space that projects the significant variations among known face
images. These significant features are known as “Eigenfaces”. This eigenface approach is used in this
project.


Fingerprints, hand geometry, iris scans, DNA analysis and to s
ome extent personal signatures are all
biometric identifiers. But the only one that does not delay or interfere with access is face recognition
.

H
umans identify others by their face and voice
and therefore
are likely to be more comfortable with a
system th
at uses face and voice recognition.




Figure
0
-
1

:
Example of face recognition [4]


This makes face recogniti
on ideal for high traffic areas which are open to the general public for e.g.
airports and railway stations, ATM’s, public transportation and businesses of all kinds.
Face recognition
gives a record of who was there.

Since the record is stored in a databa
se, known persons can be detected
automatically and unknown persons checked quickly
.



Vision
-
Based Biometric Authentication System


3

Final Year Project, Dept. of Electronic Engineering, NUI Galway

C
oncerns following recent terror attacks and constant threats to safet
y
have created a
pressing

need for
advanced security.
As a result, merchants
of sophisticated face
recognition biometrics solutions

have
found their products very much in demand.


According to the
World Face Recognition Biometrics Market,
the face recognition

market earned
revenues of $186 million in 2005 and is likely to grow at a compound annual grow
th rate (CAGR) of 27.5
percent to reach $1021.1 million in 2012

[
6
]
.




Figure
0
-
2

: Biometric Market Technology [7]


However,

to compete effectively with alternative biometric technologies
, price reduction
of face
biometric solutions is critical. A major competitor that matches the face biometric system in terms of
accuracy and performance is the
non
-
automated fingerprint identification systems (non
-
AFIS)
. A
reluctance to lower prices may

hamper the adoption

of face recognition biometrics.



Vision
-
Based Biometric Authentication System


4

Final Year Project, Dept. of Electronic Engineering, NUI Galway

The purpose of this project is to create a vision based biometric authentication system for PCs. Nowadays,
most PCs use a password based authentication system to determine access rights. The reason for creating
a vision
based authentication system is because the password based system:




Less secure


Anybody may enter anybody else’s password



Remembering passwords


This may become a problem when a user is accessing a number of
different systems



Tedious


A user has to ente
r his/her passwords every time the workstation needs to be locked


Because of the advances in image processing techniques, particularly in the areas of face detection and
face recognition, coupled with the low cost of digital imaging hardware, make a visio
n based
authentication system quite practical.


The aim of this project is to build a
face recognition authentication
system that:




Retrieves images from a camera in real
-
time



Detects the presence of a face in the image



Identifies the face against some enr
olled images



Through integration with the PCs authentication system, logs on the user corresponding to the
identified face



Vision
-
Based Biometric Authentication System


5

Final Year Project, Dept. of Electronic Engineering, NUI Galway

Camera
Capture Images
Transfer to PC
Face Detection
Detect face in image
Face recognition
Analyse face in image
Recognise the face
Authentication
Authenticate the user
whose face is in the
image

Figure
0
-
3

: Overall Project Structure


The initial plan for this project was:




G
et a face detection program up and running



Implement that program with the authentication system to create an initial working demo



Get a face recognition program working



Detect a users face with the face recognition program and log the user on



Add a utilit
y to add new users to the database of
authorized

faces



Integrate the above programs into a fully functional face recognition authentication system


During the course of the year this plan was updated to include additional goals with more specific
requireme
nts e.g. as well as writing the code to detect a face
,

the face detection system required the
installation of Linux, OpenCV and a webcam driver.


In the first chapter, a background and detailed description will be provided on the various software,
hardwar
e and tools used to implement the many steps of the project. In chapter two, the face detection
methodology will be presented, and excerpts of the face detection code will be explained. We will
Vision
-
Based Biometric Authentication System


6

Final Year Project, Dept. of Electronic Engineering, NUI Galway

conclude chapter two with a brief summary on the testing and
implementation of the face detection
process. Chapter three is concerned with Authentication. A brief discussion on PAM is followed by
biometric implementation with PAM. Chapter three will conclude with a summary on the testing and
implementation of the
authentication process. In Chapter four, the Principal Component Analysis (PCA)
method of face recognition is introduced and we will discuss why the PCA method was chosen, the
theory of face recognition using the PCA method and how it is used in this syst
em. Also we will describe
the mathematics behind face recognition and outline the relevant formulae.

This chapter will close with a
description of the tests and results.
This project will finish with a discussion on the results and a
conclusion driven from

these results.

Vision
-
Based Biometric Authentication System


7

Final Year Project, Dept. of Electronic Engineering, NUI Galway

Chapter 1

Software and Tools

1.1

Linux

1.1.1

Overview

Linux is a Unix
-
like computer operating system. It is an
exceptional

example of open source development
and of free software. Linux’s
core

source code is available for anyone to use,
alter

and redistribut
e freely.
The first Linux systems were created in 1992 when system utilities and libraries from the GNU project
were
joined

with the Linux kernel. A few years later Linux was gaining the support of big corporations
such as IBM, Hewle
tt
-
Packard and Sun Micr
osystems

[8]
.

N
ow

Linux is

used as an operating system for
a wider variety of computer hardware than any other operating system.
Primarily

known for its use in
servers, it is also used in such hardware as desktop computers, mainframes, supercomputers and
e
mbedded devices such as cell phones.



Figure
1
-
1

: Linux Logo


A Linux distribution is a package containing the Linux kernel and a variety of other Linux based software
packages. They initially
make possibl
e

the installation and configuration of the Linux operating system. A
distribution is also responsible for system security and the
assimilation

of thousands of different software
packages into a coherent whole. A general purpose distribution usually includ
es: a boot loader (e.g.
GRUB), the Linux kernel, GNU Compiler Collection (GCC), GNU C library, Desktop Environment (e.g.
GNOME), X window system display and networking protocol, a GNU bash shell and thousands of other
software applications such as text edi
ting, media players, office suites, scientific programs and web
servers

[
8
]
. There are many distributions available for download; the most commonly know ones are:




Gentoo



Ubuntu

Vision
-
Based Biometric Authentication System


8

Final Year Project, Dept. of Electronic Engineering, NUI Galway



Susé



Fedora Core


One of the main
components

of a Linux system is its command
line interface (CLI). It is possible to
completely run a Linux system from the CLI as it provides accessibility to all of the underlying source
code. Some distributions use the CLI as their only interface, thus increasing security and reducing system
resou
rce consumption. The CLI is universal in the Linux world, it does not change from machine to
machine or from distribution to distribution, which is why it is the preferred interface among
programmers and Linux power users.
The CLI was used continually thro
ughout the project because it was
the easiest way to:




Creat
e
, mov
e

and copy files and folders.



Install applications.



Execute

programs.



Chang
e

file/folder permissions


The CLI also helped a lot in the debugging of program code. The standard output of a pro
gram could be
redirected to a text file for later observation
. It was especially useful in the debugging of the face
detection program. One terminal was used to run the program and another terminal was used to
continually concatenate the output file
.

S
o
wh
ile the program was running it was possible to check if it
was working.



Figure
1
-
2

: CLI being used from a terminal

Vision
-
Based Biometric Authentication System


9

Final Year Project, Dept. of Electronic Engineering, NUI Galway



Nowadays most Linux distributions come with a desktop environment
;

“They combine a wind
ow
manager with a suite of standard applications that adhere to human interface guidelines” [
8
].

The two
most predominant desktop environments are KDE and GNOME which were released in 1996 and 1997
respectively. Later in the report the GNOME desktop enviro
nment will be described in detail.


The number of Linux users is greatly increasing every year and because of that the number of high quality
applications being developed for the Linux operating system is also increasing. Here are some of the
major softwar
e applications available:




Mozilla Firefox


Internet



VLC player


Multimedia



OpenOffice


Office suite



The GIMP


Graphics


Linux still lacks in the kind of application support provided by the Windows operating system but
projects such as WINE

[
9
]

are try
ing to remedy this by providing a way to run Windows compatible
applications on a Linux system.


Most Linux distributions support a lot of different programming languages. Here are some of the
languages Linux supports:




C



C++



Java



Python



Fortran



Perl



Ruby


The bulk of the programming utilities are found within the GNU tool chain, including the GNU Compiler
Collection (GCC) and the GNU Build System. GCC provides compilers for C, C++, Java, FORTRAN
and other less
common

languages. Core system software such as

libraries and utilities are usually written
Vision
-
Based Biometric Authentication System


10

Final Year Project, Dept. of Electronic Engineering, NUI Galway

in C and compiled with GCC. The Linux kernel itself is built the same way. Later in the report, the C
language and GCC are described in greater detail as they are a major part in this project.

1.1.2

Why Linux?

In orde
r for this project to work, it required the underlying source code to be accessible and editable to the
user. The programs included in this project needed to be integrating with the system core which is not
possible using the Windows operating system but i
s possible with Linux because it is an open source
development. Linux also provided the necessary programming language compiler and application
support. For those reasons Linux was the best choice of operating system for this project.

1.1.3

Red Hat


Fedora Core

Red Hat Linux was a popular Linux distribution first released in 1994. It was the first Linux distribution
to use RPM as its packaging format. Since 2003 it has been discontinued in
favor

of its new Red Hat
Enterprise Linux for enterprise environments and

Fedora Core for the free version
.



Figure
1
-
3

: Fedora logo


Fedora Core is a complete, general purpose operating system that contains only free and open source
software. It is available for distribution

[
10]

in a five
-
CD set or on a single DVD. That includes the
requirements for basic installation and a lot of optional applications, such as office, programming and
graphics applications. There are also several software repositories available online containi
ng thousands
of additional software packages

[11]
.


Fedora Core 5 was the release used for this project. It includes the GNOME desktop environment, the
GRUB boot loader (gives the option to boot different operating systems) and GCC; all of which are
requir
ed for this project.


1.2

GNOME

The GNOME project is an easy to use computing platform built entirely from free software. It is a
desktop environment that includes programs necessary for application launching, window management,
Vision
-
Based Biometric Authentication System


11

Final Year Project, Dept. of Electronic Engineering, NUI Galway

file handling and task manageme
nt. According to the GNOME website

[
12
]
: “The GNOME project
provides two things: The GNOME desktop environment, an intuitive and attractive desktop for users, and
the GNOME development platform, an extensive framework for building applications that integra
te into
the rest of the desktop.”



Figure
1
-
4

: GNOME Logo


GNOME also includes a lot of useful applications such as:



Nautilus


a file manager



Totem


A music player



GNOME Terminal


a terminal emulator



ge
dit


a text editor



Figure
1
-
5

: GNOME Desktop environment

Vision
-
Based Biometric Authentication System


12

Final Year Project, Dept. of Electronic Engineering, NUI Galway


Included in the GNOME desktop environment are the GNOME login screen and GNOME screensaver;
both of which are adapted to work in the authenticati
on part of this project.



Figure
1
-
6

: GNOME login screen


1.3

GCC (GNU Compiler Collection)

The GCC is a set of free software, programming language compiler
s

produced by the GNU project. It is
the standard com
piler for all Linux operating systems. It was first released in 1987 when it only handled
the C programming language. Now it handles many programming languages including: C, C++, Java,
Pascal, and FORTRAN. Today, GCC has been ported to more kinds of operat
ing systems and processors
than any other compiler.


In a Linux system,

users
invoke a driver program named
gcc
, which interprets command arguments,
decides which
language
compi
lers to use for each input file,
runs the assembler
on their output, and then
possibly runs the linker to produce a complete executable binary” [
13
].

Each of the programming
language compilers is a separate program that is called using the ‘gcc’ command.


In this project the GCC compiler was used to compile all
the
source code

for t
he C programs which
included the face detection, face recognition and PAM programs
. The face detection/recognition
programs were compiled from the command line using the ‘g++’
command

(pretty much the same thing
as the
gcc

command).

The PAM
modules were co
mpiled with the
gcc

command but were

implemented
using a
m
akefile.


Vision
-
Based Biometric Authentication System


13

Final Year Project, Dept. of Electronic Engineering, NUI Galway

1.4

C Programming Language

1.4.1

Overview


C is a general purpose, procedural, imperative computer programming language developed in 1972

by
Dennis Ritchie at the Bell Telephone Laboratories

for use

with the UNIX operating system
” [
1
4]
. Now it
has
extended

to
a lot of

other platforms and is one of the most
commonly

used programming languages. C
is the
programming language used most in the

writing of system software

and is also widely used in the
writ
ing of applications.



Figure
1
-
7

: The C programming specification

1.4.2

Characteristics of C

Here are some of the more important characteristics of the C programming language:




Has facilities for structured prog
ramming



Allows lexical variable scope and recursion



Static type system prevents many unintended operations



C has many keywords:

o


Declaring data types using ‘struct’, ‘union’ or ‘enum’.

o


Declaring variables using ‘char’, ‘int’ or ‘double’.

o

Control flow sta
tements used for conditional or iterative execution such as ‘if’, ‘else’,
‘do’, ‘while’ and ‘for’.

o

Arbitrary jumps are possible with ‘goto’.



Parameters of C functions are passed by value

Vision
-
Based Biometric Authentication System


14

Final Year Project, Dept. of Electronic Engineering, NUI Galway



Pass
-
by
-
reference is achieved by explicitly passing pointer values



Th
e struct data type allows
associated

data elements to be
linked

and
controlled

as a unit



Low level
admission

to computer memory via machine addresses and typed pointers



Intricate

functionality such as mathematical and I/O functions included in library rout
ines



Pointers allows an object or function to be referenced and recorded in memory by their address or
location



Source text is free
-
format, using semicolon as a statement
stopper


All the programs in this project were written in the C programming. In order

for the face
detection/recognition programs to compile they needed to be written in C because the OpenCV libraries
were compiled in C. The PAM modules written for this project were also written in C because PAM
modules, implementing the PAM libraries, wil
l not compile in any other language.

1.5

OpenCV

1.5.1

Overview

OpenCV stands for an open source computer vision library. Originally developed by Intel, it runs on both
Windows and Linux and is focused mainly towards real
-
time image processing. Other areas that the
l
ibrary covers are:




Object Identification



Face recognition



Segmentation and recognition



Motion understanding and tracking



Mobile robotics



Ego
-
motion



Gesture recognition



Structure from motion (SFM)



Human
-
computer interface (HCI)


OpenCV was downloaded from
the sourceforge
O
pen
CV

project
page [
15
]
.

Vision
-
Based Biometric Authentication System


15

Final Year Project, Dept. of Electronic Engineering, NUI Galway

1.5.2

OpenCV contents

In order to deal with all sorts of image processing applications, there are many functions available in the
OpenCV library. Here are some of the basic functions




Basic Structures



Operations on array
s


copying, filing, arithmetic, linear algebra, math functions, discrete
transforms



Dynamic structures


memory storages, sets, trees



Drawing functions


Curves and shapes, text, points and contours



Data persistence


file storage, writing data, reading d
ata



Error handling and system functions


All of these basic functions are used through out the face detection
/recognition

code.


As well as the basic functions listed above, OpenCV has many high level functions that deal with specific
applications in image

processing
.
Here are some of the most commonly used
high level functions:




Image processing

o

Gradients, edges and corners

o

Filters and colour conversion

o

Special image transforms

o

Histograms

o

Matching

o

Sampling, interpolation and geometric transforms



Structural

analysis

o

Contour processing

o

Computational geometry

o

Planar subdivisions



Pattern recognition

o

Object detection



Camera Calibration


Vision
-
Based Biometric Authentication System


16

Final Year Project, Dept. of Electronic Engineering, NUI Galway

The process of face detection in this project is carried out by a combination of functions. The keys one
s

are
related to patter
n
recognition and image processing
.


Then there are experimental functions also included in the OpenCV libraries:



3D tracking functions

o

3D tracker calibrate camera

o

3D tracker locate objects



Eigen objects (PCA) functions

o

Calculate covariance matrix

o

Calculat
e eigen objects and values

o

Calculate decomposite coefficients

o

Projection of eigen vectors in eigen space



Embedded hidden markov models functions


The functions used in this project’s face recognition are the Eigen objects

or PCA

functions.


1.6


Web camera

The

face detection process in this biometric authentication system required the use of a USB web camera.
When a camera was being selected for this project there were a few
details

that had to be taken into
consideration

first:




Camera resolution


H
ow accurat
e must it be for face recognition?



Size


If this camera is to be used in a biometric authentication system for a PC then it must not
be
obstructive or awkward
.



Price


If this authentication system were to be implemented on a large scale system with many
PCs then the price must be considered
.

The system would be more practical if the camera costs
were low.


Taking these details into account, the camera obtained for this project was a Labtec Quickcam.


Vision
-
Based Biometric Authentication System


17

Final Year Project, Dept. of Electronic Engineering, NUI Galway


Figure
1
-
8

: Labtec Quickcam


The reasons for selecting this particular camera were:




Resolution: Considering that the camera will be close to the user’s face, the resolution would not
need to be excellent in order for face detection or face recognition to work
, so this camera was
picked because it has good resolution.



Size: This camera is very small and can easily be placed on top of the PC monitor.



Price:
The normal retail price for this model of camera is €20. Since the rest of the tools used in
this project,

software and hardware, are free; the total cost of implementing this improved
authentication system would only be €20 per PC.



Availabilit
y: Frank Callaly, project

supervisor, had one of these cameras already in his possession
so it was borrowed
from

him f
or the duration of the project.


After obtaining the camera, a driver had to be installed in order for the Linux system to recognise it as
USB camera device. A universal camera driver, called the g
spca5xx
driver, was recommended by the
project supervisor.
This driver, specifically designed for Linux, is able to work on a large number of USB
cameras and was downloaded from the spca5xx site [
1
6
].

There was a complication
with the gspca.c
source
file when the driver was being installed. The source file was try
ing to locate a higher version of the
Linux kernel then what was installed on the PC.

The gspca.c

file was edited to include the kernel

version

2.6.15
-
1.2054_FC5

.


Vision
-
Based Biometric Authentication System


18

Final Year Project, Dept. of Electronic Engineering, NUI Galway


Chapter 2

Face Detection

Face detection is a process that determines the location of a human face

in a digital image.

In this project,
face detection is the first step. It involves capturing images in real
-
time from a camera and then
determining whether or not the image contains facial features.

If a face is detected in an image it is then
highlighted

in the image and passed on to face recognition
.



2.1

Method of Face Detection

In this project, The OpenCV library is used in the detection of faces. This library uses the statistical
approach originally developed by Paul Viola and Michael Jones

[
17
]

for face

object detection.

This
algorithm is implemented in this project using a modified face detection program contained in the library.

2.1.1

Viola

Jones Algorithm for Face Detection

The Viola
-
Jones method of face detection uses simple Haar
-
like features and a cascad
e of boosted tree
classifiers. The Haar
-
like features are computed for the input image and then these features are passed
through a cascade of classifiers in order to determine if they are facial features.


The
features are called
Haar
-
like features

becaus
e they are calculated similar to the coefficients in the
Haar wavelet transform. The Haar wavelet is the first ever wavelet and is also the simplest.

The detected
Haar
-
like features are described by their shape, their coordinates and their size. The shape
of the feature is
described by a certain template. Figure below shows a set of fourteen Haar
-
like templates.


Vision
-
Based Biometric Authentication System


19

Final Year Project, Dept. of Electronic Engineering, NUI Galway


Figure
2
-
1

: 14 different Haar
-
like features

[18]


But a face, because of its complexity, cannot

be classified by such a small number of templates. In real
classifiers, hundreds of features are used. Using hundreds of features is indeed more accurate

but the time

taken to compute the pixel sums of a lot small rectangles would slow down detection. Pau
l Viola
introduced a method to speed up these computations.

An Integral Image, Summed Area Table (SAT), is
first
computed for the whole image, where



Figure
2
-
2

: Formula for Summed Area Table

[18]


Using
SAT a given rectangle’s pixel sum can be computed by using only its corners.



The computed feature value is then passed through a simple classifier that might have two or three nodes.


Figure
2
-
3

: Two node

classifier

[18]

Vision
-
Based Biometric Authentication System


20

Final Year Project, Dept. of Electronic Engineering, NUI Galway


Figure
2
-
4

: Three node classifier

[18]

The classifier will respond with either a +1, meaning the feature passed

or with a
-
1 meaning the feature
does not satisfy the criteria of the classi
fier.

Such a classifier is known as a weak classifier, it is not used
to detect a face but rather to distinguish if the image contains a feature that may relate to a face.

E.g. the
Haar
-
like feature 3(a) displayed in figure
2
-
1
will respond to an eye once
properly positioned and scaled.


The next step is to chain a number of weak classifiers together into a more complex classifier know as a
boosted classifier.

In this chain of weak classifiers, the ones with the smallest return of error are given
more weigh
t in the boosted classifier. But for this boosted classifier to give a high success rate and small
false alarm rate, it would require a very large number of weak classifiers, resulting in slow execution.
Viola has instead built a cascade of boosted classif
iers. The simpler boosted classifiers are first in line
because they are more effective at eliminating non
-
face candidates, therefore most of the detection time is
spent on real faces
. A face is only detected when a candidate passes through all classifiers

without being
rejected.

The
cascade of boosted classifiers used in this project was

trained using hundred of frontal face
images and is supplied with the OpenCV library.


2.1.2

Face Detection Program

Apart from installing software and tools, face detection was
the first step in this project. The aim was to
write a program to process images from a camera in real
-
time and then detect if any face objects are
present in that image.

This program was written using the OpenCV libraries.


The face detection program is a
ble to read in from several different outputs


an image, a file containing
the names of images, an
AVI

file or from a USB camera.
The initial goal was to get the face detection
prog
ram running with a simple image containing a face.

The important parts
of
the face detection code is
displayed and explained below.


When running the program, the user must pass the classifier location and the input image. An example of
the command used to run the program
from the command line is
:


Vision
-
Based Biometric Authentication System


21

Final Year Project, Dept. of Electronic Engineering, NUI Galway

./facedetect
--
cascade=

"/usr/
src/redhat/SOURCES/opencv
-
1.0.0/data/haarcascades/haarcascade_frontalface_alt2.xml" input.jpg


The following code in facedetect.c reads in the classifier name
and the image name
from the input and
loads it:


if( argc > 1 && strncmp( argv[1], "
--
cascade=",
optlen ) == 0 )


{


cascade_name = argv[1] + optlen;


input_name = argc > 2 ? argv[2] : 0;


}

cascade = (CvHaarClassifierCascade*)cvLoad( cascade_name, 0, 0, 0 );


The input image is then passed into the face detection function,
convert
ed from a colour image to a
greyscale image and then resized as a smaller image. These steps are taken to make the Haar
-
like features
easier to
identify.


cvCvtColor( img, gray, CV_BGR2GRAY );

cvResize( gray, small_img, CV_INTER_LINEAR );

cvEqualizeHist( s
mall_img, small_img );


The new image is then checked for face objects; if a face object is detected it is
stored as a rectangular
object, the coordinates for which are then used to draw a circle around the detected face. I
t is then
displayed in a window:


CvSeq* faces = cvHaarDetectObjects( small_img, cascade, storage,


1.2, 2, 0/*CV_HAAR_DO_CANNY_PRUNING*/, cvSize(40, 40));

for( i = 0; i < (faces ? faces
-
>total : 0); i++ )


{


CvRect* r = (CvRect*)cvGetSeqElem( faces, i

);



CvPoint center;


int radius;


center.x = cvRound((r
-
>x + r
-
>width*0.5)*scale);


center.y = cvRound((r
-
>y + r
-
>height*0.5)*scale);


radius = cvRound((r
-
>width + r
-
>height)*0.25*scale);


cvCircl
e( img, center, radius, colors[i%8], 3, 8, 0 );

Vision
-
Based Biometric Authentication System


22

Final Year Project, Dept. of Electronic Engineering, NUI Galway


}

cvNamedWindow( "result", 0 );

cvShowImage( "result", img );


The
cvHaarDetectObjects

function

is the most important function in the face detection code. It
basically does all the work explained earlier in
Viola
-
Jones algorithm

[17].

Here are the function
arguments details in order to better understand what it is doing. Arguments listed in order:


1.

small_img : this is the reduced grayscale image used as the input to the function
.

2.

cascade : the cascade of boos
ted classifiers is located at this location
.

3.

storage : temporary memory storage
.

4.

1.2 : scaling the cascade by 20% each pass
.

5.

2 : groups of at least 3 (2+1) neighbor face rectangles are grouped into a single face, smaller
groups are rejected.

6.

0 : passing no

arguments here. Passing
CV_HAAR_DO_CANNY_PRUNING

into this argument
will speed up detection by reducing number of false alarms but at the cost of accurate detection.

7.

cvSize(40, 40) : minimum face size allowed by classifier.



2.1.3

Face Detection with a USB cam
era

The face detection program is also designed to read in images from a camera. If the user doesn’t specify
an input to the program then the program checks if a USB camera device is connected.


if( !input_name || (isdigit(input_name[0]) && input_name[1] =
= '
\
0') )


capture = cvCaptureFromCAM( !input_name ? 0 : input_name[0]
-

'0' );


The feed from the camera is stored in the
CvCapture

variable
capture
. Using OpenCV’s frame
manipulation, a frame is captured from the camera an
d

transferred to an image
.

The code then checks if
that image’s origin (top left corner) is in the right place (and corrects it if its not) and copies the image to
new image. This new
image is then used as the input for the
detect_and_draw

function
.


if( !cvGrabFrame( capture ))


break;

frame = cvRetrieveFrame( capture );

Vision
-
Based Biometric Authentication System


23

Final Year Project, Dept. of Electronic Engineering, NUI Galway

if( !frame )


break;

if( !frame_copy )


frame_copy = cvCreateImage( cvSize(frame
-
>width,frame
-
>height),


IPL_DEPTH_8U, frame
-
>nChannels );

if( frame
-
>origin == IPL_ORIGIN
_TL )


cvCopy( frame, frame_copy, 0 );

else


cvFlip( frame, frame_copy, 0 );



detect_and_draw( frame_copy );


So the program captures frames from the camera in real
-
time and if there is a face object in the image
then it is circled

and displaye
d
.

The average detection time is a
round 20ms which is
more than

fast
enough.


2.2

Testing and implementation

The usual rate of false
alarms

(detecting a face in an image where there is none)
is
very
low
.
Therefore
t
he testing was not
very rigorous
.
First t
he p
rogram was

tested with

different images

e.g. jpegs
. Below is
an example
of one of these images:




Figure
2
-
5

: Result of face detection in an image



Vision
-
Based Biometric Authentication System


24

Final Year Project, Dept. of Electronic Engineering, NUI Galway

The program was tested using other faces and even images

with no faces.
In each case the face detection
program was successfully i.e. detected faces in facial images and rejected non
-
facial images.
The results
of which are in
the
Appendix.


The program was then tested with the input from the camera. The program

was run for 10 minutes at a
rate of 1 frame every 20ms without detecting any other object than the face of the user. So the program
processed approximately 30000 frames without a single error. A sample of the output is displayed below.



Figure
2
-
6

: Result of face detection from a camera


When the program was first run, there were issues with the quality of the camera input. Since the
attributes (brightness, contrast, hue, etc) of the camera input were not
set in the program or
in
the
OpenCV configuration files the image was way too bright and blurry. The first approach to solve this
problem was to edit the configuration fi
les in the OpenCV source folder. This proved to be a problem as
changing the values in

the configuration also changed the size of the image, making it smaller. So instead,
the attribute values were passed manually from the program.


cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT, 240);

cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME
_WIDTH, 320);

cvSetCaptureProperty(capture, CV_CAP_PROP_BRIGHTNESS, 0.4);

cvSetCaptureProperty(capture, CV_CAP_PROP_CONTRAST, 0.
08
);

cvSetCaptureProperty(capture, CV_CAP_PROP_HUE, 0);

cvSetCaptureProperty(capture, CV_CAP_PROP_SATURATION, 0.2);

Vision
-
Based Biometric Authentication System


25

Final Year Project, Dept. of Electronic Engineering, NUI Galway


This greatl
y improved the quality of picture
which in turn returned better results

when the face was
passed onto face recognition
.



Figure
2
-
7

: Improved face detection result


In conclusion, the face detection progra
m was a complete success. The program detected faces in images
where there were faces with a high degree of accuracy.

Vision
-
Based Biometric Authentication System


26

Final Year Project, Dept. of Electronic Engineering, NUI Galway

Chapter 3

Authentication

The chapter describes the authentication process, precisely how it

i
s done and how it is implemented with
the vision base
d biometric system.

The first aim was to create a basic authentication system
.

The next step
was

to gradually integrate it with the face detection

program and then later with

face recognition.


3.1

PAM

PAM or P
luggable

Authentication Module
s are a

method

to as
similate multiple low
-
level authentication
systems into high
-
level applications.

PAM
is based on a set of library modules located in the /lib/security
directory. Applications that rely on PAM have their PAM configuration files stored in /etc/pam.d/.

There
are many applications that depend on PAM
. I
n this project the ones that
are

looked at

are the GNOME
login and screensaver applications. These applications request authentication from the PAM modules
listed in their PAM configuration files.

In this chapter

we discuss how to create and compile PAM
modules, how to configure application PAM settings and then how to integrate the biometric system with
PAM.


3.1.1

Configuring an application for PAM

Each application dependant on PAM has their matching configurati
on file

located in /etc/pam.d/. In order
for that application to be authenticated it must call the modules
listed

in this file. An example of a PAM
config
uration file

is shown below.


#%PAM
-
1.0


# Fedora Core

auth include

system
-
auth

account include

syst
em
-
auth

password include

system
-
auth

session include

system
-
auth


This configuration file is for the GNOME screensaver.
The file is pointing to another configuration file
containing more lists of PAM modules.
All config
uration

files come in the format
:

Vision
-
Based Biometric Authentication System


27

Final Year Project, Dept. of Electronic Engineering, NUI Galway



Module
-
type control
-
flag

module
-
path

arguments


Where,




Module
-
type refers to the type of PAM module

o

auth


authenticates users, usually asking a password and then checking it.

o

account


checks if account details are correct for authentication. E.g.

has the account
expired or has the system resources reached maximum users.

o

password


used to set passwords.

o

session


makes it possible for the user to use their account once authenticated



Control flag describes the importance of module

o

r
equired


in ord
er for the module
-
type to succeed this PAM module must return a
success.

o

r
equisite


if the module fails, control is r
estored to the application.

o

s
ufficient


if no previous required module has failed and this module succeeds, then no
more of this module
-
t
ype is invoked.

o

o
ptional


this module is not crucial to the success or failure of the application’s access.
This module will be invoked if all other modules have not returned successes.



Module path is the name of the module to be invoked.



Arguments


only

some modules use arguments. E.g. passing ‘shadow’ to a password module
-
type will invoke the use of shadow passwords

[19]
.


As mentioned above, the two applications that had to be altered so they depend on face
detection/recognition are the login
and scree
nsaver applications.
In order to do this their PAM
configuration files needed to be changed so they pointed at PAM modules designed specifically to rely on
the biometric programs.

Here is what
the new configuration files look like:


#%PAM
-
1.0

GNOME screens
aver


auth required

pam_face_screensaver.so

account required

pam_face_screensaver.so

password required

pam_face_screensaver.so

session required

pam_face_screensaver.so


Vision
-
Based Biometric Authentication System


28

Final Year Project, Dept. of Electronic Engineering, NUI Galway


#%PAM
-
1.0

GNOME Desktop Manager login

auth required

pam_env.so

aut
h


required pam_face_login.so

account required

pam_nologin.so

account required pam_face_login.so

password required pam_face_login.so

session required pam_face_login.so

session required pam_loginuid.so

session optional

pam_console.so


Now when GNOME runs the login application or brings up the screensaver, they will only be
authenticated using the biometric system.


3.1.2

Creating a PAM module

A PAM module needed to be created that relied on the output being produced by th
e biometric programs.
The solution for this was easy. Create a PAM module that read a user name in from a file. Since one of
the initial aims of this project was to authenticate a user by just detecting a face,
therefore

the face
detection program could be

modified to write a default user’s name to a file
every time a face was
detected

and

as a r
e
sult

authenticating that user.

Reading a user name from a file would also eventually
work with face recognition because the recognition program could output the us
er name of a face that has
been recognised to be in the user database.


The two applications, login and screensaver, need two different types of authentication:




Login requires the module to read in a user name from a file and then log that user on. If the
re is
no name in the file, it will fail. If the user name is wrong, it will fail.



Screensaver requires that the module reads a user name in from a file, checks if that name is the
same as the current user, then grants access if it is and returns a failure
if it doesn’t.



For each of these tasks a separate PAM module was made although they are very similar
.


Vision
-
Based Biometric Authentication System


29

Final Year Project, Dept. of Electronic Engineering, NUI Galway

Here are excerpts from the
pam_face_screensaver.so

module. First a

PAM function gets the current user
name. If there is an error retrieving it then it

will return an error. This should not happen because
the user
is already logged on.


retval = pam_get_user(pamh, &user, NULL);

if (retval != PAM_SUCCESS) {


D(("get user returned error: %s", pam_strerror(retval)));


return retval;


The next step is to tra
nsfer that user name to a temporary string. Then read in a string from the output file
of the face detection/recognition code, compare the two and return a success if they’re equal or return a
fail if they’re not.


sscanf(user, "%s", temp);

ifp = fopen("/l
ib/security/user.txt", "r");

fscanf(ifp, "%s", name);

if( *name == *temp )

{


return PAM_SUCCESS;

}

else

{


return PAM_AUTH_ERR;

}


For the login module,
pam_face_login.so
, only a few lines are changed from the screensaver module.
Instead of comparin
g two strings, the module tries to log the user read in from the output file of the
biometric programs.


ifp = fopen("/lib/security/user.txt", "r");



fscanf(ifp, "%s", name);


pam_set_item(pamh, PAM_USER, (const void *) name);


return PAM_SUCCESS;


Vision
-
Based Biometric Authentication System


30

Final Year Project, Dept. of Electronic Engineering, NUI Galway

The

code
returns

PAM_SUCCESS no matter what the outcome is because if the
inputted
user name
doesn’t exist
,

the authentication fails

anyway
.

According to the GNOME login PAM configuration file
the application is not authenticated by
pam_face_login.so

alone. S
o it must also pass the other modules,
e.g.
pam_loginuid.so

will fail if the user name set to PAM doesn’t exist in the database.


3.2

Implementing Biometric programs with Authentication

As mentioned above in previous chapter, the PAM modules depend on the outp
ut of the
face
detection/recognition system. So the first step was to get the face detection program to write a default user
name to a file every time a face is detected (any face) and get that much working with the PAM modules.
The next step would be to w
rite the user name retrieved from the face recognition function to the same
file to see if that would work with the PAM modules.


Writing a default user name to a file each time a face is detected was implemented by the following code:


CvRect* r = (CvRect
*)cvGetSeqElem( faces, i );

if(r)

{



rewind(ofp);




fprintf(ofp, "pohiarnain");

}


The face recognition would print out the user name instead of “pohiarnain” after it has finished with its
face recognition
.


fprintf(ofp, "
%s
"
, user_name
);


3.3

Integrati
ng PAM with Biometric system

Every time the GNOME screensaver or the login application runs its corresponding PAM module, the
output from the biometric

program is read in.
Because of this the biometric programs must commence on
start up

and continue to run

so the input to the PAM modules is constantly updated. In order to do this a
script was written to call the program in run level 5. This script was written and moved to /etc/inti.d/
. In
the run level 5 directory, /etc/rc.d/rc5.d/ there is a list of all pr
ograms to be run at start up.

The script to
Vision
-
Based Biometric Authentication System


31

Final Year Project, Dept. of Electronic Engineering, NUI Galway

run the biometric programs is included in here

(see appendix A for script)
.

The key statement in the script
is when it calls the program; the whole path name must be included:


start() {


echo
-
n $"Starting
F
ace Detection
"


/root/Desktop/OpenCV/facedetect
--
cascade=”/usr/src/redhat/SOURCES/
opencv
-
1.0.0/data/haarcascades/haarcascade_frontalface_alt2.xml"

&


echo_success


echo

}


Now the program is able to run indefinitely as long as the PC is on.


3.4

Test
ing and Implementation

While implementing and testing with PAM it was established that working with the login screen and
screensaver would be a problem because if the program failed then the user would have to restart the
computer, thus making it very time

consuming to debug the program.

So instead of experimenting with
the login or screensaver it was more efficient to use the switch user application. This application is run
solely from the command line

so any errors occurring with the PAM module would just

return an error to
the command line and not freeze the system.

Once the module was completely debugged then porting it to
the other applications was easy.


Compiling the module was another issue because it had to be compiled in a way so it was recognised
as a
PAM shared
library

(.so file)
.

So first step was trying to get a standard PAM module to compile and run.
The module pam_permit.so was selected for this testing because it

s an easy to understand program with
a
very simple output i.e.

it returns a PAM_
SUCCESS no matter what.

The source

code

for all the standard
PAM modules was downloaded from the fedora rpm repository [
11
].

After researching
alternative PAM
modules in hope to find a suitable Makefile to compile the module, a

project

was found called
pam
_chroot

which was downloaded from the pam_chroot sourceforge site [
20
]
. This module is not
included in the standard PAM libraries so the source code is included
with the download as well as
the
Makefile.

After editing the Makefile to compile pam_permit ins
tead, it compiled the module without
error. Then after including the new pam_permit

module

in the switch user configuration file and moving
Vision
-
Based Biometric Authentication System


32

Final Year Project, Dept. of Electronic Engineering, NUI Galway

the module to where the modules are stored, /lib/security/, the switch user application ran without error
and always

returned PAM_SUCCESS.


At this stage t
he appropriate

application and Makefile were
selected for testing the pam_face modules.

The main issues with debugging the pam_face modules were:




Comparing the constant value ‘user’ with the string value ‘name’ was r
eturning segmentation
faults

(segmentation faults occur at run
-
time, the error does not show up when compiling. These
faults are usually due to the program trying to access a memory location that it is not allowed to
access or when it is trying to access a

memory location

in the wrong way)
. The
solution

was to
copy the ‘user’ value to a temporary string and then compare that to the ‘name’.



At first, the program was comparing the two pointer values and returning a segmentation fault.

This happened because of

a simple error with the code syntax, the code was comparing the
pointer values not the values the pointers were referencing i.e. the strings.



The authentication module at first was written to return PAM_SUCCESS if the right conditions
were met. What if th
e wrong conditions occur? If there are conditions set in a PAM module, then
each condition should return a value. Due to this the module was failing at first because it didn’t
return anything when the conditions were not met. Adding in another value, PAM_A
UTH_ERR,
which

would be returned if the conditions were not
met,

solved this issue.


The final stage in testing the Authentication process was to
test if the biometric system would
intialise

on
start
-
up and then authenticate the user

every time either the
screensaver or login applications are run
.

The
first attem
pt to get the program running on start
-
up was done by adding it to the run level 5 script,
rc.local. This is the final script that is
called

during start
-
up
; it basically runs any programs listed in

the
file rc.local. This didn’t work because there were errors due to permission issues. The next
attempt

in
volved writing a

new start
-
up script to initiate the program
. This was carried out by using the simple
‘readahead’ script as a template and added th
e path name of the face detection/recognition program. The
first attempt to get this to
operate

worked but with one problem. The face detection/recognition program
was called during start
-
up but was not told to run in the background and since this program
is designed to
run indefinitely, the PC never left the start
-
up stage.

The Linux rescue CD had to be used to boot the PC
directly to the
command line so the script could be changed to run the program in the background. This is
done by adding a ‘&’ at the e
nd of the program path.


Vision
-
Based Biometric Authentication System


33

Final Year Project, Dept. of Electronic Engineering, NUI Galway

So now the biometric system was constantly running in the background and continuously updating the
outp
ut text file ‘user.txt’. The PAM module for login was checking this file
every time

the login
application
was

implemented. At fi
rst the login application worked perfectly, logging in the user listed in
‘user.txt’. So did the screensaver application
, unlocking the screen when the current user’s name was the
same as the name listed in ‘user.txt’.

But after 2 minutes the face detectio
n/recognition program would
stop running. Since it was running in the background it was not returning any error data either. In order to
find out the error details the start
-
up script was edited to print out the output of the program to a
temporary text fi
le as shown below.


cascade=”/usr/src/redhat/SOURCES/
opencv
-
1.0.0/data/haarcascades/haarcascade_frontalface_alt2.xml"

> /tmp/output.txt &


Having done this the program was now printing out the standard output information to this text file.
According to the

file a
fter two minutes of running the biometric program would shut down because there
was too much data being stored in some of the image variables. In order to remedy
this, the

image
variables had to be released after they
’ve
been processed.
Adding in th
is function,
cvReleaseImage(image_name)
, solved the problem. The biometric system was then run for around 5
hours straight to see if it could
operate

for very long periods of time.
The

results were
good;

the
face
detection

authentication system ran for hou
rs with fail.

I next tested it by putting different objects (book,
hand, bag
, etc.
) in front of the camera and see if it would log on. The system didn’t detect any faces in
those cases so it didn’t authenticate.


So
in conclusion,

the authentication system

worked perfectly with the face detection system and would
eventually work with face recognition.


Vision
-
Based Biometric Authentication System


34

Final Year Project, Dept. of Electronic Engineering, NUI Galway

Chapter 4

Face recognition

The process of face recognition involves the examination of facial features in an image, recognizing those
features and matching them to o
ne of the many faces in the database.

There are many algorithms capable
of performing face recognition; such as:




Principal Component Analysis



Discrete Cosine Transform



3D recognition methods



Gabor Wavelets method



Hidden Markov Models



Kernel methods


The P
rincipal Component Analysis (PCA) method of face recognition is used in this biometric
authentication system.

In this chapter we will discuss

why the PCA method was chosen,

the theory of face
recognition using the PCA method and how it is used in this syst
em.


4.1

PCA

4.1.1

Why use PCA?

There were many issues to consider when choosing a face recognition method. The keys ones were:




Accuracy



Time limitations



Process speed



Availability


With these in mind the PCA method of face recognition was selected for this project

because:




Simplest and easiest method to implement


due to project deadlines this method seemed the
most practical

Vision
-
Based Biometric Authentication System


35

Final Year Project, Dept. of Electronic Engineering, NUI Galway



Very fast computation time



Accurate


this method is definitely not the most accurate of face recognition algorithms but
considering the re
quirements of this project it was judged to be accurate enough



PCA is supported within the OpenCV library


this was key because it made integration with the
face detection program very easy


There are disadvantages with the PCA method but they were deemed

to be inconsequential. PCA is:




Translation variant


if the image is shifted or tilted then it will not recognise the face. This flaw
doesn’t affect this system as the camera will always be located upright and in front of the user.



Scale variant


scalin
g the images will affect the performance of face recognition. This was not a
big problem in this pro
ject as new users are added using the camera at the PC so the face to be
recognised and the faces in the database are captured in the same way; making them
very similar
in size. Also t
he user will alw
ays be relatively close to the camera eliminating the possibility of a
blurry, out of focus face.



Background variant


recognising a face with a different background is difficult. The face
detection program is al
tered so it not only detects a face but extracts it to a new image, thus
cutting out almost all the background. The
residual sections

of background
are insignificant since
the camera’s resolution will only keep the face in focus because of the distance.



Li
ghting variant


if the light intensity changes then the accuracy of face recognition drops.
This
authentication system involves capturing images in front of a PC and usually the room is lighted
when a user is using a PC.


Taking all this into consideratio
n,
the PCA method of face recognition was deemed to be the most
practical for this project.


4.1.2

Theory of PCA

Principal Component Analysis

is

a process that extracts the most relevant information contained in a face
and then tries to build a computational mod
el that best describes it. The basic theory of PCA can be
described in the following steps:


1.

E
igenvectors

or
eigen
faces of the covariance matrix are found. This is done by training a set of
face images
.

Vision
-
Based Biometric Authentication System


36

Final Year Project, Dept. of Electronic Engineering, NUI Galway

2.

These
e
igenvectors

become the
eigenspace

(a multi
-
dim
ensional sub
space

comprised of the
Eigenvectors
) in which every face is projected on.

3.

Recognition is performed by comparing the location of a face in the
eigenspace

with the location
of known users.

In other
words
calculating the
Euclidean

distance.


This
is only the basic theory of PCA. In order to fully understand PCA it needs to be explained
mathematically which is done
below
.



4.1.3

Mathematics of PCA

The mathematics of PCA [
21
]

describes

how an image is broken down, how the
eigenvectors

are
calculated and e
ventually how images can be compared to each other by calculating the
Euclidean

distance.


First a 2D
face image is broken down into a 1D vector by concatenating each row into one long vector.

Say
M

equals the number of input images there are and
N

(
= rows

of image x columns of image) equals
the size of the images. Then an image can be represented by the vector:


[
21
]

w
here the
p
j
’s represent an individual pixel value. Before analysis these image vectors have to be mean
centred by
subtracting the mean imag
e from the image vector. Let
m

be the mean image and
w
i

be the
mean centred image:


[
21
]

The next step is to find a set of
eigenvectors

that best describe the principal information in each of the