Biometric Based User Authentication through

ovenforksqueeΑσφάλεια

3 Νοε 2013 (πριν από 3 χρόνια και 9 μήνες)

616 εμφανίσεις

1


Biometric Based User Authentication through
Facial Expression

A

Dissertation

Submitted in the partial fulfillment

of the requirement

For the Award of Degree

Of

Master of Technology

In Computer Science and Engineering






Supervised By:

Submitted By:

Dr. Chander Kant

Annu

Assistant Professor

M. Tech (Final
Year)



Roll No. 49035



Department of Computer Science & Applications

Kurukshetra University, Kurukshetra

20
13



2


DEPARTMENT OF COMPUTER SCIENCE &APPLICATIONS

KURUKSHETRA UNIVERSITY KURUKSHETRA

HARYANA (INDIA)


No: ____________




Dated: __________



DECLARATION


I, Annu, a student of Master of Technology (Computer Science & Engineering), in the
De
partment of Computer Science and Applications, Kurukshetra University,
Kurukshetra, Roll no. 49035, for the session 2011
-
2013, hereby, declare that the
dissertation entitled

Biometric Based User Authentication through Facial
Expression
” has been completed

after the theory examination of 3
rd

semester.


The matter embodied in this Dissertation has not been submitted to any other institute or
university for the award of any degree to the best of my knowledge and belief.





Date:


Annu















3


DEPARTMENT OF COMPUTER SCIENCE & APPLICATIONS

KURUKSHETRA UNIVERSITY KURUKSHETRA

HARYANA (INDIA)




Dr. Chan
der Kant
No:

____________

(Assistant Professor)


Dated: __________



CERTIFICATE


Th
is is to certify that the dissertation

entitled

Biometric Based User Authentication
through Facial Expression”
submitted by Ms. Annu

for the award of

degree of Master
of Technology

in C
omputer Science and Engineering Roll No. 49035, Kurukshetra
Univer
sity, Kurukshetra is a record of bonafide research work carried by her, under my
supervision and guidance. The dissertation, in my opinion

worthy for

consi
deration for
the award of Degree of Master of Technology

in accordance with the

regulations of
Kurukshetra University, K
urukshetra. The matter entitled

in this

dissertation has

not been
submitted to any other institute or university for the award of

any degree to the best of
my knowledge and belief
.



(Dr. Chander Kant)

Assistant Professor,

Department of Computer Science & Applications

Kurukshetra University, Kurukshetra








4


DEPARTMENT OF COMPUTER SCIENCE & APPLICATIONS

KURUKSHETRA UNIVERSITY KURUKSHETRA

HARYANA (INDIA)



Prof. Suchita Upadh
yaya

No:

____________

(Chairperson
)

Dated: __________



CERTIFICATE


It is certified that
Ms. Annu is a bonafide student of
Master of Technology in Computer
Science and Engineering, Roll no.

49035
,

Kurukshetra University,

Kurukshetra.

H
e has
undertaken the dissertation entitled
“Biometric Based User Authentication through
Facial Expression

und
er the supervision of
Dr
.

Chander Kant.




(Prof.
Suchita Upadh
yaya)

Chairperson

Department of Computer Science & Applications

Kurukshetra University, Kurukshetra











5


Acknowledgement



No one can do everything all without someone’s help and guidance.

Here, I would like to
take a moment to thank those who have helped and inspired me throughout this journey
and without their guidance and support I might not have been able to complete this
research work.

First of all I would like to express my sincere gratitude to my supervisor
Dr. Chander Kant
, Assistant Professor, Department of Computer Science &
Applications, Kurukshetra University, Kurukshetra, for his constant support, inspiratio
n
and guidance in both academic and personal life. I am extremely grateful to him for
being an excellent advisor and a wonderful teacher. His attention to detail and
completeness to every aspect of the work, including research, presentation and technical
w
riting has helped me tremendously to improve my skills. In the course of my master
study, he has helped me with helpful ideas, guidance, comments and criticisms. His
constant motivation, support and infectious enthusiasm have guided me throughout my
disser
tation work. I have been privileged to work under his supervision, and I truly
appreciate this help. His encouraging words have often pushed me to put in my best
possible efforts.

I express my sincere thanks to
Dr. Suchita Upadhyaya
, Chairperson, Departmen
t of
Computer Science & Application, Kurukshetra University, Kurukshetra and other staff
members for their support and encouragement.

Last, but not least, I would like to express my gratitude and appreciation to all of my
family for their support, encourag
ement, faith, understanding and help. Without them,
this work would not have been possible.






Annu



6


List of P
ublicati
ons


Publications in Conference:



-


Challenges and Scope of Face Recognition

published in the proceedings of All
India Seminar on Information Security on February 25
-
26, 2013 by DCRUST,
Murthal, Sonepat, under the aegis of Computer Engineering
Division Board, IEI.


Publications in the International Journal:


-


Liveness Detection in Face recognition using Euclidean Distances

published
in “International Journal for advance Research in Engineering and Technology

ISSN: 2320
-
6802
” Vol. 1, Issue IV, May 2013, pp 1
-
5.


-

“A Novel Approach for Facial Expression R
ecognition Using Euclidean
Dist
a
n
ces”

is accepted in “International Journal of Engineering and Advanced
Technology (IJEAT)
ISSN: 2249
-
8958”,

Vol. 2, Issue V, June 2013.











7


Table of Contents


List of Figures
…………
…………………………………………………….........10


List of Tables
…………………………………………………………………….11

Abstract………………………………………………………..
.............................12

Dissertation Outline

……………………………………………….………..……13


Chapter1 Introduction

to B
iometrics………………………………………………….14

1.1 Biometric Techn
ology……………………………………………………………….
1
4


1.2 Biometric Tech
niques………………………………………………………………..18


1.2.1 Physical Characteristics B
ased Techniques………………………………………..18


1.2.2 Behavioral Characteristics B
ased Techn
iques……………………………………..23


1.3 Comparison of Various Biomet
ric Technologies……………………………………25

1.4 Advantages of using B
iometrics…………………………………………………….27


1.5 Challenges of Biometr
ic System…………………………………………………….27


1.6 Multimodal Biom
etrics……………………………………………………………
…28


1.7 Soft Biometrics……………………………………………
…………………………29



Chapter 2 Face Recognition System
…………………………………………………..31

2.1. Motivation
…………………………………………………………………………...32

2.2 Why face Recognition is

difficult……………………………………………………33


2.3 Automatic Face De
tection……………
………………………………………………35

2.4 Face Detection App
roaches………………………………………………………….35

2.4.1 Face Detection by Temp
late Matching……………………………………….……36

2.4.2 Skin Distribution

Model………………………………………………….………..36

2.4.3 Face Detection by Neur
al Network………………………………………………..36

2.4.4 Face Detection by Eigen Face Method………
………………………………….…37

2.5 Image Processing
………………………………………………………………….....38

8


2.6
Fusion

of Face with Other Biometric Traits……………………………………...
.....
40


2. 7 Various Biometric Fusion Techniques……………………………………………
…41

2.7.1
Fusion at the Imag
e Level………………………………………………………….42

2.7.2 Fusion at the Feature Extr
action Level…………………………………………….43

2.7.3 Fusion at the Ran
k Level………………………………………………………….44

2.7.4 Fusion at the Matching
Score Level……………………………………………….45

2.7.5 Fusion at the De
cisi
on Level……………………………………………………….47

2.8 Summary……………………………………………………………………………..
48


Chapter 3

Literature Survey
………………………………………………………….

49

3.1 Security and Privacy Issues in Biomet
ric Authentication…………………………..68

3.1.2 Biometric System Concerns…………
…………………………………………….68

3.1.2 Vulnerability of Biometric Auth
entication System……………………………….69


Chapt
er 4 Conceptual Models for Face
Recognition
…………………………………74

4.1 Spoofing in Biometri
c system……………………………………………………….76


4.1.1 Spoofing in face Recognition System…
……………………………………….….76

4.2

Face Image Acquisition
……………………………………………………………...77

4.3 Proposed face Recognition Algorithm
for Image Acquisition………………………78

4.3.1 Architecture of the propo
sed approach…………………………………………….78

4.3.2 Euclidean Distance Test……………………………………

……………………80

4.3.2.1 Euclidean Di
stance………………………………………………….……………81

4.3.2.2 Algorithm of the prop
osed approach………………………….…………………81

4.3.2.3 Compari
son………………………………………………………………………82

4.4 Summary
………………………………………………………………….………….83


Chapter 5
F
acial Expressio
n
Recognition……………………………………

…….
84

5.1 Face Detection / Loca
lization…………………………………………………….….84

5.1.1 Color models for skin color
classification…………………………………………85

9


5.1.2 Algorithm for detectin
g a face……………………………………………………..88

5.2 Facial Expression
Recognition………………………………
………………………90

5.2.1 Facial Action Coding

System……………………………………………………...90

5.2.2 Proposed W
ork…………………………………………………………………….93

5.2.2.1 Algorithm of the propo
sed approach…………………………………………….93

5.2.2.2 Eigen Face
Method………………………………………………………………9
5

5.2.2.3 Calculating Eige
n Faces………………………………………………………….97

5.3 Summary…
…………………………………………………………………………..98


Chapter 6

Face Recognition Applications
…………………………………………….99

6.1 Government use appli
cations……………………………………………….………..99

6.1.1 Law enforcem
ent…………………………………………………………………..99

6.1.2 Security / counterter
rorism……………………………………………………….100

6.1.3 Immigratio
n……………………………………………………………………….101

6.1.4 Correctional institution
s/prisons …………………………………………………101

6.1.5 Legislature
………………………………………………………………………..101

6
.2 Commercial use………………………………………………………
…………….102

6.2.1 Bankin
g……………………………………………………………………...……102

6.2.2 Pervasive com
puting…………………………………………………………...…101

6.2.3 Voter verifica
tion…………………………………………………………………103

6.2.4 Healthcar
e………………………………………………………………...………104

6.2.5 Day car
e…………………………………………………………………..………104

6.2.6 Missing children/r
unaways……………………………………………………….104

6.2.7 Residential secu
rity……………………………………………………………….104

6.2.8 Internet, E
-
co
mmerce…………………………………………………..…………104

6.2.9 Benefit pay
ment……………………………………………………
..……………105

6.3 Other areas where face recognition use
d………………………………...…………105


Chapter 7

Conclusion and Future Scope
…………………………………………….106


Bibliog
raphy…………………………………………...…………………………
……
108

10


List of Figures


Figure 1.1 Enrollment Process in Biometric System

Figure 1.2 Verification and Identification Process

Figure 1.3 Minutiae point of a fingerprint

Figure 1.4 Stricture of hand geometry

Figure 1.5 Structure of iris

Figure 1.6 Image of a face

Figure 1.7 Image of retina

Figure 1.8 Image of voice rhythm

Figure 1
.9 Keystroke pattern

Figure 1.10 Example of a signature

Figure 2.1
Steps for face recognition system applications

Figure 2.2
Methods for face detection

Figure 2.3 Biometric fusion systems

Figure 2.4

Authentication Process Flow

Figure 2.5

Fusion at the Image level

Figure 2.6

Fusion at the Feature Extraction level

Figure 2.7 Fusion at the Matching Score level

Figure 2.8 Fusions at the Decision Level

Figure 3.1 Block diagram of a typical biometric authentication system

Figure 4.1 Techniques

used in face recognition

Figure

4.2

Block Diagram of image acquisition using CCD camera

Figure 4.3 Architecture
of the proposed approach

Figure 5.1 RGB color space

Figure 5
.
2

The RGB color space within the YCbCr color space

Figure 5.3 HSI color space

Figu
re 5.4

Steps of a face detection algorithm

Figure 5.5

Facial action coding system


Figure 5.6

Flow Chart

of the proposed approach


11


List of Tables


Table 1.1 Performance Comparison of various Biometric Traits

Table 5
.1 FACS Action Units

Table 5.2
Miscellaneous Actions Units



























12


Abstract


Biometric Based U
ser

Authentication through Facial
Expression



The dissertation addresses certain special concept
s

of face recognition system. Compared
with other biometric technologies, such as iris, speech, fingerprint recognition, face
recognition can be easily considered as the most reliable biometric trait in all. Face
recognition is a particular type of biometri
c system that can be used to reliably identify a
person by analyzing the facial features of a person’s image. A biometric system verifies
user identity through comparison of a certain behavioral or psychological characteris
tic
possessed by the user. Face

recognition is used worldwide for the security purposes.
User authentication systems that are based on knowledge such as password or

physical
tokens such as ID card are not able to meet strict security performance

requirements of a
number of modern compute
r applications. These applications

generally use computer
networks (e.g., Internet), affect a large portion of population,

and control financially
valuable tasks (e.g., e
-
commerce). Biometrics
-
based

authentication systems are good
alternatives to the tradi
tional methods. These

systems are more reliable as biometric data
cannot be lost, forgotten, or guessed and

more user
-
friendly because we don’t need to
remember or carry anything
. The main functional component of the existing face
recognition system consis
ts of image capturing, face detection, and then features
extraction and finally matching. If it matches correctly then accept the user otherwise
rejected. This dissertation gives a general presentation of the face recognition technology
and also the facial

expression recognition with its advantages as well as challenges.






13



Dissertation Outline


Chapter 1

Chapter
1

gives the introduction part of the biometric authentication system, parameter
characteristics, advantages of biometric system, and challenges

of biometric system,
multimodal biometric system, soft biometrics and image processing.

Chapter 2

Chapter 2

gives the introduction of the face recognition and detection, face detection
approaches, difficulties in face recognition and fusion of face with
other biometric traits.

Chapter 3

Chapter 3 gives the literature survey, security and privacy issues related to the use of
biometrics. Solutions and previous work in this field are presents as well.

Chapter 4

Chapter 4 will discussed about existing face re
cognition techniques and

the proposed
work which includes a method to check the liveness of a person.

Chapter 5

Chapter 5 will discuss about the facial expression recognition, Facial action coding
system, and

the proposed work to recognize a person’s expr
ession and then
authenticating the user.

Chapter 6

Chapter 6 presents current applications of face recognition, face recognition survey and
some areas where face recognition is used.

Chapter 7

Chapter 7 gives the conclusion and future scope of face recogni
tion technology.


14


Chapter 1


Introduction

to Biometrics

Now
-
a
-
days, biometric recognition
is a common and reliable way to authenticate any
human being based on his physiological or behavioral biometrics

[1]. A physiological
biometric traits is stable in
their biometric like fingerprint, iris pattern, facial feature,
hand geometry, gait pattern etc. whereas behavioral biometric traits is related to the
behavior of person such as signature, spe
ech pattern, keystroke pattern.
Facial recognition
system is a c
omputer application for automatically identifying or verifying a person from
a digital

image or a video frame from a video source
.

Face recognition is not a new idea but
i
t has received

substantial attention over the last
three decades due to

its value
both in understanding how FR process works

in humans as
well as in addressing many challenging

real
-
world applications, including de
-
duplication
of

identity documents (e.g. passport, driver license), access

control and vid
eo
surveillance.

While face recogn
ition in controlled conditions

(frontal face of cooperative
users and controlled indoor

illumination) has already achieved impressive

performance
over lar
ge
-
scale galleries [2],
there still exist many challenges

for face recognition in
uncontrolled environ
ments,

such as partial occlusions, large pose variations, and

ext
reme
ambient illumination.

Local

facial features have played an important role in forensic
applications for matching face images. These features include any salient skin region that
appears o
n the face. Scars, moles, and freckles are representative examples of the local
facial features

[3]
.


1.1
Biometric Technology

Biometric refers to the automatic identification of a person based on his or her physical or
behavioral characteristics. This identification method is preferred over traditional
methods involving passwords and PINs (personal identification numbers) for sev
eral
15


reasons, including the person to be identified is required to be physically present at the
point of identification and identification based on biometric techniques avoids the need to
remember a password or carry a token. Now
-
a
-
days, with the increased

use of computers
as the means of transportation of information technology, restrict access to sensitive or
personal data is essential. Biometric authentication has seen considerable progress in
reliability and accuracy, with some of the traits offering go
od performance.

The terms “Biometrics” and “Biometry” have been used since the early 20
th

century
to
refer to the field of improvement of arithmetical and mathematical methods applicable to
data analysis problems in the biological sciences [4]. The need fo
r biometrics can be
found in central, state and local governments, in the military and in commercial
applications. Now
-
a
-
days, worldwide network security infrastructures, government IDs,
secure electronic banking, investing and other financial communicatio
n, retail sales, law
enforcement and health and social services are already benefiting from these
technologies. Biometric based authentication applications include workstation, network
and domain access, single sign in, application login, data protection f
rom illegal access,
remote access to resources, transaction security
and web security. Trust in this

type of

electronic communication is the necessary to the successful growth of the global financial
system. Utilizing biometrics for personal authentication

is appropriate, well
-
situated and
considerably more accurate than current methods such as utilization of passwords or
PINs.

A biometric system may operate either in an ‘identification’ system or a ‘verification’
(authentication) system. Before the system
can be put into verification or identification
mode, a system database consisting of biometric

te
mp
lates must be created through the

process of enrollment
.


Enrollment
-

is the process where a user’s initial biometric sample(s) are collected,
assessed, processed, and stored for ongoing use in a biometric system as shown in Figure
1.1.
Basically,
user enrollment is a process that is responsible for registering individuals
i
n the biometric system storage. During the enrollment process, the biometric
characteristics of a person are

first captured by a biometric scanner to produce a sample.
Some systems collect multiple samples of a user and then either select the best image or

16


fuse multiple images or create a composite template.

If users are experiencing problems
with a biometric system then they have to re
-
enroll to gather higher quality data.


Figure1.1 Enrollment Process in Biometrics System

Biometric system provides two ma
in functionalities viz. verification and identification.
Figure 1.2 shows the flow of information in verification and identification systems.


Identification
-

One
-
to
-
Many Correspondence: Biometrics can be used to determine a
person’s identity even without
his knowledge or permission.
The user’s input is
compared with the templates of all the persons enrolled in the database and the identity of
the person whose template has the highest degree of similarity with the user’s input is
output by the biometric
system. Typically, if the highest similarity between the input and
all the templates is less than a fixed minimum threshold, the system rejects the input,
which implies that

the user presenting the input is not one among the enrolled users.
For
example, sc
anning a crowd with a camera and using biometric recognition technology,
one can determine matches against a known database. Identification is the initial stage to
identify the user through his biometric trait. The data of user stored for ongoing use in a
biometric system permanently.


Verification
-

One
-
to
-
One correspondence: Biometrics can also be used to verify a
person’s identi
ty

and the system verifies whether the claim is genuine. If the user’s input
and the template of the claimed identity have a high

degree of similarity, then the claim is
accepted as “genuine”. Otherwise, the claim is rejected and the user is considered as
“fraud”
. For example, one can grant physical access to a secure area in a building by
17


using finger scans or can grant access to a

bank account at an ATM by using retinal scan.
Figure 1.2 shows the flow of information in verification and identification systems.


Figure1.2 Verification and Identification Process


Biometric is necessary due to these characteristics:



Links the event to

a particular user ( a password or symbol may be used by
somebody other than the approved user)



Is convenient (nothing to carry or memorize)



Accurate ( it provides for positive authentication)



Fast and scalable



Easy to use and easily understandable



Is beco
ming socially acceptable and



Cost effective




18


1.2
Biometric Techniques

There are many different techniques available to identify/verify a person based on

their

biometric characteristics
as suggested by U.K. Biometric Working Group [UKBWG,
2003]. These

techniques can be divided into physical characteristics and behavioral
characteristics

based techniques.


1.2.1
Physical characteristics based Techniques

Biometrics techniques based on physical characteristics of human being such as

finger
print, hand
geometry; palm print etc are called physical characteristics based

techniques.
Following are examples of biometric techniques based on physical characteristics.


Fingerprint Recognition:

Among all the biometric techniques
, fingerprints based
identification is the oldest method which has been successfully used in several
applications. The fingerprint itself consists of patterns originate on the tip of the finger,
thus making it a physical biometric. Fingerprints are known to be unique and absol
ute for
each person and the b
asic characteristics of fingerprints do not change with time. The
distinctiveness of a fingerprint can be determined by the patterns of ridges and minutiae
points on the surface of the finger. These unique patterns of lines can

either be in a loop,
whorl or arch pattern. The most common method involves recording and comparing the
fingerprint’s “minutiae points”. Minutiae points can be considered the uniqueness of an
individual’s fingerprint. The major Minutia points in fingerpri
nt are: ridge ending,
bifurcation, and short ridge or dot as shown in Figure 1.3.




(a) Ridges Ending (b) Ridges Bifurcation (c) Dot


Figure 1.3 Minutiae points in fingerprint

19


The ridge ending is the point at wh
ich a ridge terminates (see Figure 1.3 a). Bifurcations
are points at which a single ridge splits into two ridges (see Figure 1.3 b). Short ridges or
dots are ridges which are significantly shorter than the average ridge length on the
fingerprint (see Figu
re 1.3 c). Some examples of the use of fingerprint devices in general
areas are:



Fight the abuse of social services like social security.



Permitting logins based on fingerprints.



Fight against criminal immigration.



Attendance management system in industry,

colleges or companies.


Hand Geometry:

This biometric approach uses the geometric form of the hand for
confirming the user’s identity.

Specific features of a hand must be combined to assure
dynamic verification, since human hands are not unique.

Individua
l hand features are not
descriptive enough for identification. Characteristics such as finger curves, thickness and
length, the height and width of the back of the hand, the distances between joints and the
overall bone structure are usually extracted. Tho
se characteristics are pretty much
determined and mostly do not change in a range of years. The basic structure of hand
geometry is shown in figure 1.4.



Figur
e 1.4
Structure of hand geometry


As the hand geometry readers are big, rugged, quick and handy, they are well suited to
use in warehouses, manufacturing facilities and other industrial locations that have the
space to comfortably house them. Hand geometry readers are good for time
-
and
-
att
endance applications (replacing punch clocks) where their simplicity and rapid cycle
times are big assets and their lackluster accuracy rates are not major liabilities. The list
20


showing below following examples in real world areas where hand scan identific
ation is
or was used:



Users at the Olympic Games 1996 were identified with hand scans.



In a lot of cases access to military plants is granted upon successful hand scan
identification.



Airport personnel at the San Francisco Airport

are identified by hand sc
ans.


Iris Recognition:

Biometrics is the science
of measuring human characteristics that are
stable and unique among users. Iris recognition is the process of verify a human being by
the pattern of iris. The iris is the area of the eye where the pigmented

or colored circle,
usually black, brown or blue rings the dark pupil of the eye. As compared to other
biometric traits the iris is more secured and protected. Iris recognition is an approach to
biometric based verification and identification of people [5]
. The future of the iris
recognition system is better in fields that demand rapid identification of the users in a
dynamic environment

[6]
. Iris patterns are extremely complex. The basic structure of iris
is shown below in figure 1.5
.


Figure 1.5 S
tructure of iris


In
this technique, the user places
him

so that he can see his own eye's reflection in the
device. The user may be able to do this from up to 2 feet away or may need to be as close
as a couple of inches depending on the device. Verificatio
n time is generally less than 5
seconds, though the user will only need to look into the device for a couple of moments.
To prevent a fake eye from being used to fool the system, these devices may vary the
light shone into the eye and watch for pupil dilat
ion.


21


Face Recognition:
Face recognition (FR) is the problem of verifying or identifying a
face from its image. User face detection plays an important role in applications such as
video observation, human computer interfaces, face recognition and face imag
e databases
[7]. Local facial features have played an important role in forensic applications for
matching face images. The use of local features has become more popular due to the
development of higher resolution sensors, an increase in face image databas
e size, and
improvements in image processing and computer vision algorithms. Local features
provide a unique capability to investigate, annotate, and exploit face images in forensic
applications by improving both the accuracy and the speed of face
-
recognit
ion systems.
This information is also necessary for forensic experts to give testimony in courts of law
where they are expected to conclusively identify suspects [8]. The image of face is shown
in figure 1.6.



Figure 1.6 F
ace

Recognition

To enable this
biometric technology it requires having at least one video camera, PC
camera or a single
-
image camera. On the other hand, this biometric technique still has to
deal with a lot of problems. Finding a face in a picture where the location, the direction,
the
background and the size of a face is variable is a very hard task and many algorithms
have been worked on to solve this problem.


Retina Recognition:

Along with iris recognition technology, retina scan is possibly the
most accurate and reliable biometric t
echnology. It is also among the most difficult to use
and requires well trained and is supposed as being quite to highly invasive. The users
have to be cooperative and patient to achieve an accurate performance.

22



Figure 1.7

Retina

Recognition


Basically
the retina is a thin curve on the back of the eye which senses light and transmits
impulses through the optic nerve to the brain as shown in figure 1.7.
Retinal scanning
analyses the layer of blood vessels at the back of the eye.

Blood vessels used for
biometric identification are located along the neural retina which is the outermost of the
retina’s four cell layers. Research has proven that the patterns of blood vessels on the
back of the human eye were unique from person to per
son. It even been proven that these
patterns, even between twins, were indeed unique. This pattern also does not change over
the course of a lifetime. The retinal
scanner

requires the user to place their eye into some
sort of device and then asks the user
to look at a particular mark so that the retina can be
clearly imaged. Scanning involves using a low
-
intensity light source and an optical
coupler and can read the patterns at a great level of accuracy.

This process takes about 10
to 15 seconds in total. T
here is no known way to replicate a retina, and a retina from a
dead person would deteriorate too fast to be useful, so no extra precautions have been
taken with retinal scans to be sure the user is a living

human being.


Vein pattern recognition
:
Vascular

patterns are best described as a picture of the

veins
in a person's hand or face. The thickness and location of these veins are

believed to be
unique enough to an individual to be used to verify a person's identity.

The most common
form

of vascular patter
n readers is

hand
-
based, requiring the

user to place their hand on a
curved reader that takes an infrared scan. This scan

creates a picture that can then be
compared to a database to verify the user's stated

identity
.




23


1.2.2
Behavioral Characteristics ba
sed Techniques:


Those Biometrics techniques which are based on the behavior of human being such as
voice, signature, gait, keystroke etc. are called behavioral characteristics based
techniques. Following are examples of biometric techniques based on
behavioral
characteristics.


Voice Recognition:

Mostly,

the voice biometric solutions can be used through a typical
telephone or microphone equipment to the computer. In order to identify or authenticate
users, most voice biometric solution creates a voice

print of the user, a template of the
person’s unique voice characteristics created when the user enrolls with the system.
During enrollment the user has to select a passphrase or repeat a sequence of numbers or
words. The passphrase should be in the lengt
h of 1 to 5 seconds. The problem with short
passphrases is that they have not enough data for identification. Longer passphrases have
too much information and takes too much time. The user has to repeat the passphrase or
the sequence of numbers several tim
es. This makes the enrollment process long lasting
than with other biometric technologies. All successive attempts to access the system
require the user to speak, so that their live voice sample may be compared against the
pre
-
recorded template. Voice rhyt
hm is shown below in figure 1.8.


Figure 1.8 V
oice rhythm

pattern


A voice biometric sample is a numerical model of the sound, pattern and rhythm of a
user’
s voice. The main

problem occurring
in the voice is that the user’s voice changes
over time along
with the growth of a user or when someone has got a cold or another
disease. Background noise can also be a disturbing factor which does not gives the
accurate result.


24


Keystroke Dynamics:

Keystroke dynamics uses the manner and rhythm in which a user
types

characters/
password or phrase on the keyboard

or keypad
. The

system then records
the timing of the typing and compares the password itself and

the timing to its database.
Here, verification takes less than 5 seconds.

Keystroke dynamics is the process of
a
nalyzing the way a user types at a terminal by monitoring the keyboard inputs thousands
of times per second in an attempt to identify users based on normal typing rhythm
patterns. The keystroke pattern shown as below in figure 1.9.


Figure 1.9 K
eystroke p
attern


Signature Recognition:

Signature verification is the process that is used to recognize a
user’s handwritten signature. Dynamic signature verification uses behavioral biometrics
of a handwritten signature to validate the identity of a person. This c
an be achieved by
analyzing the shape, speed, stroke,
and pen

pressure and timing information during the
act of signing. The example of signature is shown below in figure 1.10.


Figure 1.10 S
ignature

Recognition


On the other hand, there is the simple
signature comparison which only takes into
account what the signature looks like. So with dynamic signature verification, it is not the
shape or look of the signature that is meaningful, there are the changes in the speed,
pressure and timing that occur du
ring the act of signing, thus making it virtually
impossible to duplicate those features. The main difficulty with this technology is to
distinguish between the reliable par
t of a
signature, these are the characteristics of the
25


static image, and the behavi
oral parts of a signature
, which vary with each signing
process. Comparing many signatures made by one user reveals the fact that a user’s
signature is never completely the same and can vary considerably over a user lifetime.
Allowing these variations in t
he system, while providing the best protection against fake
is a big problem faced by this biometric technology. The financial industry sometimes
uses signature verification for money transactions.


1.3
Comparison of Various Biometric Technologies:

Performance of a biometric measure is usually referred to in terms of the false accept

rate
(FAR), the false non match or reject rate (FRR), and the failure to enroll rate

(FTE or
FER). The FAR measures the percent of invalid users who are incorrectly

acce
pted as
genuine users, while the FRR measures the percent of valid users who

are rejected as
impostors.

A number of biometric characteristics may be captured in the first phase of
processing. However, automated capturing and automated comparison with

previ
ously
stored data requires that the biometric characteristics satisfy the following

characteristics:

1.
Universal:
Every person
must possess the characteristic or
attribute. The

attribute must
be
one that is universal and rarely

lost to accident or disease.


2.

Uniqueness
/singularity
:

Each expression of the attribute must be unique to the

individual.
The characteristics should have sufficient unique properties to

distinguish one
person from any other. Height, weight, hair and eye

color are

all attributes that are unique
assuming a particularly precise measure, but do

not offer enough points of differentiation
to be useful for more than categorizing.


3.
Permanence
/Invariance of Properties
: They should be constant over a long
period of
time. The

attribute should not be subject to significant differences based on age either

episodic or chronic disease.


4.
Collectability/Measurability:
The properties should be suitable for captur
ing

without
waiting time and must be easy to gathe
r the attri
bute data inactively
.

26


5.
Performance:
it is the measurement of accuracy, speed, and robustness of

technology
used.


6.
Acceptability:
The capturing should be possible in a way acceptable to a

large
percentage of the population. Excluded are part
icularly invasive

technologies, i.e.
technologies which require a part of the human body to be

ta
ken or which apparently
damage

the human body.


7. Circumvention:

Ease of

use of a substitute.


There are also some other parameters which are very important
during the

analysi
s of a
biometric trait. T
hese are:

Reducibility:
The captured data should be capable of being reduced to a file

which is
easy to handle.


Reliability and Tamper
-
resistance:
The attribute should be impractical to mask

or
manipulate. The pr
ocess should ensure high reliability and reproducibility.


Privacy:
The process should not violate the privacy of the person.


Comparable:
Should be able to reduce the attribute to a state that makes it

digitally
comparable to others. The less
probabilistic the matching involved, the

more authoritative
the identification.


Inimitable:
The attribute must be irreproducible by other means. The less

reproducible
the attribute, the more

likely it will be reliable
.


Table 1
.1 below shows a comparison
of various biometric systems in terms of above
mentioned parameters. A. K. Jain ranks each biometric based on the
categories as being
low, medium

or high. A low ranking indicates poor performance in the evaluation
criterion whereas a high ranking indicates

a very good performance.

27


Table 1
.1 Performance Comparison of various Biometric Traits



1.4

Advantages of Using Biometrics:



Easier
fraud detection



Better than password/PIN or smart cards



No need to memorize passwords



Requires physical presence of the
person to be identified



Unique Physical or behavioral characteristic



Cannot be borrowed, stolen or forgotten



Cannot leave it at home


1.5

Challenges with Biometric System

The human face is not a unique rigid object. There are

billions of different faces and

each
of them can assume a variety of deformations. These variations can be classified as

follows:

a) Inter
-
personal variations due to

1
.
Race
:

It signifies the complexion of the person according to which a person can be
di
stinguished from other.

28


2.

Identity:

It tells about the name, address, phone no. of the person which
distinguishes
him
from

the other person
.

3. Genetics:


Genetic code of each person is different. Because of the genes every
person has different face and
gender.

b) Intra
-
personal variations due to

1. Deformations
:

These are the result of injury or accident on the face.

2. Expression
s:

This shows the mood of the person. From expressions it is easy to
determine whether the person is happy or sad.

3. Aging
:

With age wrinkles appears on the face. The wrinkles change the formation of
the face to a great extent
.

4. Facial hair
s:

Man have moustaches and beard wh
ich change the look of the face
when
shaven.

5. Cosmetics
:

Cosmetic surgery has become one
of the widely used
techniques

to

enhance your facial features.

Each of these variations can give rise to a new field of research. Among each of these
facial expressions is a most widely studied technique from past many years.



1.6 Multimodal Biometrics

To enhance the security of the system Biometric follow a new techniques called as
Multimodal biometric. Multimodal Biometric means Biometric authentication by
scanning more number of characteristics.

A multimodal biometric system uses multiple modalities t
o capture different types of
biometrics. This allows the integration of two or more types of biometric recognition and
verification systems in order to meet stringent performance requirements or combine
several weak biometric measurements to engineer a str
ong optionally continuous
biometric system.

The multimodal system could be, for instance, a combination of
fingerprint verification, face recognition, voice verification and keystroke dynamics or
any other combination of biometrics. This enhanced structure

takes advantage of the
29


proficiency of each individual biometric and can be used to overcome some of the
limitations of a single biometric. A multimodal system can combine any number of
independent biometrics and overcome some of the limitations presented
by using just one
biometric as the verification tool. This is important if the quality scores from individual
systems are not very high. A multimodal system, which combines the conclusions made
by a number of unrelated biometrics indicators, can overcome m
any of these limitations.
Also it is more difficult to forge multiple biometric characteristics than to forge a single
biometric characteristic.


1.7 Soft Biometrics

Soft biometric traits are the

characteristics of human being that provide some

information
about the user
, but lack of the distinctiveness and permanence to

sufficiently differentiate
any two
users
.

Soft biometric traits embedded in a face (e.g., gender and facial marks) are
ancillary information and are not fully di
stinctive by them
selves in the facial
recognition
tasks. However, this information can
be explicitly combined with facial

matching sc
ore to
improve the overall facial
recognition accuracy and efficiency and helps the forensic
investigators.

The characteristics

of human bei
ng

like gender, height, weight and age can
also be used for

the

identification purpose. Although these characteristic
s

are not unique
and reliable, yet

they provide

some

useful information about the user. These
characteristics

are

known as soft

biometric

traits and
these
can be integrated with the
primary biometric identifiers like

fingerprint, face, iris, signature for

the identification
purpose
.

Uni
modal biometric systems make
just
use of a

single biometric trait for
individual
recognition. It is difficu
lt to achieve very high recognition rates using
unimodal

systems due to problems like noisy sensor data and non
-
universality or lack of

distinctiveness of the selected

biometric trait. Multimodal biometric systems address

some of these problems by combinin
g evidence obtained from multiple sources. A
multimodal biometric system that utilizes a number of different

biometric identifiers like
face, fingerprint, hand
-
geometry, and iris can be more

robust to noise and minimize the
problem of non
-
universality and
lack of

distinctiveness.

However, the problem with
multimodal system is that it will require a longer

verification time thereby causing
some
inconvenience to the users. A possible solution to the

problem of designing a trustworthy

30


and user
-
friendly biometr
ic

system is to use additional information about the user like
height, weight, age,

gender, ethnicity, and eye color to improve the performance of the
primary biometric

system.



The rest of the disse
rtation is divided into the six

Chapters.


Chapter 2 dis
cusses the face recognition and detection system, face detection approaches,
difficulties of face recognition system, fusion of face with other biometric traits.


Chapter 3

discusses the literature survey and analyses the existing work in the field

of
face recognition and detection in general
.


Chapter 4

will discuss about the proposed works, existing face recognition and detection
techniques and
to check the liveness of a user.


Chapter 5

will discuss about facial expression recognition system,
facial action coding
system and the proposed method for recognizing the facial expression and the Eigen face
method.


Chapter 6

presents the current applications of face recognition, face recognition survey
and some areas where face recognition is used.


A
t l
ast, Chapter 7

p
resents the conclusion of dissertation

and gives the scope of future

work.





31


Chapter 2


Face Recognition System

Face recognition systems are part of facial image processing applications and their
significance as a research ar
ea is

increasing recently.

These systems

use biometric
information of the humans and are applicable easily instead of fingerprint, iris, signature
etc., because these types of biometrics are not much suitable for non
-
collab
orative
people.
Face detection is ess
ential front end for a face recognition sy
stem. Face detection
locates and

segments face regions from cluttered images, either obtained from video or
still image.
Detection application is used to find position of the faces in a given image

[9
]
.
Numerous tec
hniques have been developed to
detect faces in a single image[10][11
].
It
has numerous applications in areas like surveillance and security

control systems, content
based
image retrieval, video conferencing and intelligent human computer interfaces.
Most of

the current face recognition systems presume that faces are readily available for
processing. However, we do not typically get images with just faces. We need a system
that will segment faces in cluttered images. With a portable system, we can sometimes
a
sk the user to pose for the face identification task.


Figure 2.1

Steps for face recognition system applications

The first step for face recognition system is to acquire an image from a camera. Second
step is face detection from the acquired image. As a

third step, face recognition that takes
the face images from output of detection part. Final step is person identity as a result of
recognition part. An illustration of the steps for the face recognition system is shown in
figure 2.1.

32


In addition to
creating a more cooperative target, we can interact with the system in order
to improve and monitor its detection. With a portable system, detection seems easier. The
task of face detection is seemingly trivial for the human brain, yet it still remains a
c
hallenging and difficult problem to enable a computer /mobile phone/PDA to do face
detection. This is because the human face changes with respect to internal factors like
facial expression, beard, mustache glasses etc and it is also affected by external fa
ctors
like scale, lightning conditions, and contrast between face, background and orientation of
face.

Face detection remains an open problem. Many researchers have proposed different
methods addressing the problem of face detection. In a recent survey fac
e detection
technique is classified in to feature based and image based. The feature based techniques
use edge information, skin color, motion and symmetry measures, feature analysis,
snakes, deformable templates and point distribution. Image based techniq
ues include
neural networks, linear subspace method like Eigen faces, fisher faces etc. The problem
of face detection in still images is more challenging and difficult when compared to the
problem of face detection in video since emotion information can le
ad to probable
regions where face could be located.


2.1

Motivation

Face detection plays an important role in today’s world. They have many real world
applications like human/computer interface, surveillance, authentication and video
indexing. However res
earch in this field is still young. Face recognition depends heavily
on the particular choice of features used by the classifier One usually starts with a given
set of features and then attempts to derive an optimal subset (under some criteria) of
features

leading to high classification performance with the expectation that similar
performance can also be displayed on future trials using novel (unseen) test data.

Interactive Face Recognition
is divided in to several phases. I
t includes
:




Creating drivers fo
r the handheld device that link with the application with the
captured image.

33




A face detection program is run inside the handheld device which detects the face
from the image.



The obtained face is transmitted through wireless network.



The server performs

the face recognition and is transmitted back.


2.2

Why Face Recognition

is Difficult?

The

greatest diffi
culty of face recognition, compared to other biometrics,

stems from the
immense variability of the human face. The facial appearance depends h
eavily on

environmental factors

.
for example, the

lighting conditions, background scene and head
pose. It also depends

on facial hair, the use of cosmetics, jewe
l
l
e
ry and piercing.




Pose
: Variation due to the relative camera
-
face pose (frontal, 45 degree, profile,
upside down), and

some facial features such as Facial

Expression or the nose may
become partially or wholly occluded.



Presence or absence of structural components
: Facial features such as beards,
mustaches, and glasses may or may not be present, and there
is a great deal of
variability amongst these components including shape, color, and size.

Local
facial mark features such scars, moles, and freckles play an important role for
matching face images in f
orensic applications [12
].

Local facial mark features
provide a unique capability to investigate, annotate, and exploit face images in
forensic applications by improving both the accuracy and the matching speed of
face
-
recognition systems. This information is also necessary for forensic experts
to give testim
ony in courts of law where they are expected to co
nclusively
identify suspects
.



Facial expression
: The
appearance of faces is

directly affected by a person's
facial expression.



Occlusion
: Faces may be partially occluded by other objects. In an image with a

group of people, some faces may partially occlude other faces.

34




Image orientation
: Face images directly vary for different rotations about the
camera's optical axis.



Imaging conditions
: When the image is formed, factors such as lighting (spectra,
source di
stribution and intensity) and camera characteristics (sensor response,
lenses) affect the appearance of a face.



Facial Aging
:
Many face recognition scenarios exhibit a significant age
difference between the probe and gallery images of a subject. Facial agi
ng is a
complex process that affects both the shape and texture (e.g., skin tone or
wrinkles) of a face. This aging process also appears in different manifestations in
different age groups.

In addition to facial aging, there are other factors that
influenc
e facial appearance as well (e.g. pose, lighting, expression, occlusion)
which makes it difficult to study the aging pattern using these two public domain
longitudinal face databases.



Forensic Sketch Recognition:

When no photograph of a suspect is availabl
e, a
forensic sketch is often generated. Forensic sketches are an artist rendition of a
person’s facial appearance that is derived from an eye witness description.
Forensic sketches have a long history of use, where traditionally they have been
disseminate
d to media outlets and law enforcement agencies in the hopes that
someone will recognize the person in the sketch. Forensic sketches can be
misleading due to errors in witness memory recall that cause inaccuracies in the
sketch drawn by a forensic artist.
Because a significant amount of time is needed
to generate a single forensic sketch, they generally represent culprits who
committed the most heinous crimes (e.g. murder and sexual assault). Thus, the
ability to match forensic sketches to mug shot database
s is of great importance.



Face Recognition in Video:

Face recognition in video has gained importance due
to the widespread deployment of surveillance cameras. The ability to
automatically recognize faces in video streams will facilitate a method of human
i
dentification using the existing networks of surveillance cameras. However, face
images in video often contain non
-
frontal poses of the face and may undergo
severe ligh
ting changes.


35


2.
3

Automatic Face Detection

a)
Facial measures have historically been of

interest to a small group of scientists and
clinicians. The main reason is that pain assessment by facial expression is burdensome.
The coding still consumes many multiples of the real times involved in the behavior.

b)
The other limitation include facial

coding technique: Anyone who has performed facial
measurement realizes that there are elements of subjectivity in deciding when an action
has occurred or when it meets the minimum requirements for coding. In addition, the
human observer has inherent limit
ations in his or her capacity to make precise
discriminations at the margins. Alternatives exist or are in development.

c)
Some faces are often falsely read as expressing some emotion, even when they are
neutral, because their proportions naturally resembl
e those another face would
temporarily assume when emoting.

Automatic face detection

is a compl
ex problem in image processing. Many
methods exist
to solve this problem such as te
mplate matching, Fisher Linear
Discriminant, Neural
Networks, EIGENFACE, and M
RC. Success has been achieved with each method to
varying degrees and complexities.


2.
4

Face Detection Approaches


Face detection is the first step of face recognition system. Output of the detection can be
location of face region as a whole, and location

of face region with facial features (i.e.
eyes, mou
th, eyebrow, nose etc.).
Mainly, detection can be classified into two groups as
Knowledge
-
Based Methods and Image
-
Based Methods. The methods for

de
tection are
given in Figure 2.2
.


Knowledge
-
Based method
s use information about Facial Features, Skin Color or
Template Matching. Facial Features are used to find eyes, mouth, nose or other facial
features to detect the human faces. Skin color is different from other colors and unique,
and its characteristics d
o not change with respect to changes in pose and occlusion. Skin
color is modeled in each color spaces like RGB (Red
-
Green
-
Blue), YCbCr (Luminance
-
36


Blue Difference Chroma
-
Red Difference Chroma), HSV (Hue
-

Saturation
-
Value), YUV
(Luminance
-
Blue Luminance Dif
ference
-
Red Luminance Difference), and in statistical
models. Face has a un
ique pattern to differentiate
from

other objects and hence a template
can be generated to scan and detect faces.


Figure
2.2

Methods for face detection


2.
4
.1

Face detection by

Template Matching

Once individual candidate face images are separated, template matching is used as not
only a final detection scheme for faces, but also for locating the centroid of the face. The
idea of templa
te matching is to perform cross
co
-
variances

with the given image and a
template that is representative of the image. Therefore, in application to face detection,
the template should be a representative face
-

being either an average face of all the faces

in the training images, or an E
igen

face. In

our case, both templates were created. The
first step was to crop out the faces from each training image posted on the website. Using
these faces as our set of training images was justified since the same people would be
present in the actual test image
-

otherwise a larger and more diverse set of training faces
images.


37


2.
4
.2 Skin color distribution model

In conventional methods, all visible colors are divided into two groups: One is the “skin
color” and the other is not. However, consider two colors near the boundary of the skin
part. Although the difference between them is almost unnoticeable by a human v
iewer,
one is regarded as “skin color” and the other is not. This is unnatural, and is considered as
one of the reasons of instability in conventional methods for skin color detection.

SCDM is a fuzzy set of
skin color
. We use a large image set containing

to investigate the
distribution of color of the human skin region in order to build the SCDM.

The procedure to build the SCDM is as follows:


1.

Manually select skin regions in each image.


2.

Prepare a table of 92
×
140 entries to record the 2
-
di
mensional chromatic
histogram of skin regions, and initialize all the entries with zero.


3.

Convert the chromaticity value of each pixel in the skin regions
UCS
,
and then
increase the entry of the chromatic histogram corresponding to it by one.


4.

Normalize the table by dividing all entries with the greatest entry in the table.

2.
4
.3

Face detection by

Neural Network


Neural Nets are essentially networks of simple neural processors, arranged and
interconnected in parallel. Neural Networks are based on our current level of knowledge
of the human brain, and attract interest from both engineers, who can use Neural Nets to
solve a wide range of problems, and scientists who can use them to help further our
understanding of the human brain. Since the early stages of development in the 1970’s,
interest in neural networks has spread through many fields, due to the speed of proce
ssing
and ability to solve complex problems. As with all techniques though, there are
limitations. They can be slow for complex problems, are often susceptible to noise, and
can be too dependent on the training set used,
but these effects can be minimiz
ed
through
careful design. Neural Nets can be used to construct systems that are able to classify data
into a given set or class, in the case of face detection, a set of images containing one or
38


more face, and a set of images that contains no faces. Neural Ne
tworks consist of parallel
interconnections of simple neural processors. Neurons have many weighted inputs, that is
to say each input (p1, p2, p3… pm) has a related weighting (w1, w2, w3… wm)
according to its importance. Each of these inputs is a scalar, r
epresenting the data. In the
case of face detection, the shade of GRAY of each pixel could be presented to the neuron
in parallel (thus for a 10x10 pixel image, there would be 100 input lines p1 to p100, with
respective weightings w1 to w100, corresponding

to the 100 pixels in the input image).

2.
4
.
4 Face detection by Eigen Face M
ethod

The motivation behind Eigen faces is that it reduces the dimensionality of the training set,
leaving only those features that are critical for face recognition.

Definition

1.

The Eigen faces method looks at the face as a whole.

2.

In this method, a collection of face images is used to generate a 2
-
D gray
-
scale image
to produce the biometric template.

3.

Here, first the face images are processed by the face detector. Then we calculate

the
Eigen faces from the training set, keeping only the highest Eigen values.

4.

Finally we calculate the corresponding location in weight space for each known
individual, by projecting their face images onto the “face space”.


2.5

Image Processing

Image processing

is any form of

signal processing

for which the input is an image, such
as a photograph

or

video frame; the

output

of image processing may be either an image
or, a set of characteristics or

parameters

related to the image. Most image
-
proces
sing
techniques involve treating the image as a

two
-
dimensional

signal

and applying standard
signal
-
processing techniques to it. Image processing usually refers to

digital image
processing, but optical

and

analog image processing

also are possible. The

acq
uisition

of
images (producing the input image in the first place) is referred to as

imaging. Image
processing is a physical process used to convert an image signal into a physical image.
39


The image signal can be either digital or analog. The actual output i
tself can be an actual
physical image or the characteristics of an image. The most common type of image
processing is photography. In this process, an image is captured using a camera to create
a digital or analog image. In order to produce a physical pict
ure, the image is processed
using the appropriate technology based on the input source type. In digital photography,
the image is stored as a computer file. This file is translated using photographic software
to generate an actual image. The colors, shadin
g, and nuances are all captured at the time
the photograph is taken the software translates this information into an image. When
creating images using analog photography, the image is burned into a film using a
chemical reaction triggered by controlled exp
osure to light. The image is processed in a
darkroom
, using special chemicals to create the actual image. This process is decreasing
in popularity due to the advent of digital photography, whic
h requires less effort and
special training to product images. In addition to photography, there are a wide range of
other image processing operations. The field of digital imaging has created a whole range
of new applications and tools that were previousl
y impossible. Face recognition
software,

medical image processing

and remote sensing are all possible due to the
development of digital image processing. Specialized computer prog
rams are used to
enhance and correct images. These programs apply

algorithms

to the actual data and are
able to reduce signal distortion, clarify fuzzy images and add light to an underexposed

image.

Animations are series of single images put together into a movie. The images
might be a

volume view, a projection, a slice, a time point. The animation is done by just
playing the

sequential data set, or by rotating 3D models or volume
representations, by
zoom
-
in & fly

through

motions, changing of surfaces and transparencies, etc.

Today
’s
computer allows
for
calculating and representing animated sequences reasonably

fast.
Movie files can be published i.e
.
in

power point or on the web. Al
so interactive

file
formats are possible.

If one or more of the images in a data set is taken through a filter
that allows radiation that lies outside the human vision span to pass


i.e. it records
radiation invisible to us
-

it is of course not
possible
to make a natural colo
r image. But it
i
s still possible to make a colo
r image that shows important information about the object.
This type of image is called a representati
ve colo
r image.

Normally one would assign
colo
rs to these exposures in chromatic ord
er with blue assigned to the shortest
40


wavelength, and red to the longest. In this
way it is possible to make colo
r images from
electromagnetic radiation far from the human vision area, for example x
-
rays. Most often
it is either infrared or ultraviolet rad
iation that is used.


2.6

Fusion of Face with Other Biometric Traits

In recent years, biometric authentication has seen considerable improvement in reliability
and accuracy, with some of the biometric traits offering
practically good performance
[13
] for
a comparative survey of up to date biometric authentication technology. On the
other hand, even the best biometrics up to date is at rest facing several problems, some of
them natural to the technology itself. Biometric systems that use a single trait are
called
unimodal systems, whereas those that integrate two or more traits are referred to as
multimodal biometric systems. A multimodal biometric system requires an integration
scheme to fuse the information obtained from the individual modalities. The

mult
imodal
biometric systems [14
] are found to be extremely useful and exhibit

robust performance
over the unimodal biometric systems in terms of several

constraints. The
aim of any
multimodal system

is to acquire multiple sources of

information from different

modalities
and minimize the error prone effect of

mono

modal systems.

In particular, biometric
authentication systems normally suffer from enrollment problems due to non
-
universal
biometric traits, vulnerability to biometric spoofing or unsatisfactory acc
uracy caused by
noisy data acquisition in certain environments. Although some biometrics has gained
more popularity than other biometrics, but each such biometric has its own strengths and
limitations, and no single biometric is expected to meet the desire
d performance of the
authentication applications. Multi biometrics is relatively a new approach to overcome all
these problems. Driven by lower hardware costs, a multi biometric system uses multiple
sensors for data achievement. In 2002 the research paper

“Multi Modal Technology
makes Biometrics Work” states the title of a recent press

release from Aurora Defense
[15
]. Certainly, multi biometric systems ensure significant improvements over single
biometric systems, for example, higher accuracy and increase
d resistance to spoofing.
They also claim to be more universal by enabling a user who does not have a particular
biometric identifier to still enroll and authenticate using other biometric traits, thus
eliminating enrollment problems. But can multi biometr
ics live up to the promotion? At a
41


first quick look, incorporating multiple biometrics into one system appears to be a very
sensitive and understandable concept. There are many different ways to actually merge
various sources of information to make a fina
l authentication conclusion. Information
fusion strategies range from simple Boolean combination to complicated numerical
modeling.


Figure 2.3 Biometric fusion systems

The goal of information fusion, therefore, is to determine the best set of experts in
a given
problem domain and devise an appropriate function that can optimally combine the
decisions rendered by the individual experts. The recognition process

itself may be
viewed as the reconciliation of evidence pertaining to these multiple

modalities. E
ach
modality on its own cannot always be reliably used to

perform recognition. However, the
consolidation of information presented by

these multiple experts can result in the accurate
determination or verification

of identity.

Several approaches can be ado
pted for
combining the different modalities [16]. Biometric fusion system is shown in figure 2.3.


2
.7

Various biometric fusion techniques

In a multi modal biometric system, the information fusion can occur at any of the module
of feature extraction,
matcher and decision. Multi biometric systems can be categorized
into five system architectures according to the strategies used for information fusion:



Fusion at image level



Fusion at Feature Extraction Level



Fusion at the Rank Level



Fusion at the Matchin
g score Level

42




Fusion at the Decision Level


That is we classify the systems depending on how early in the authentication process the
information from the different sensors is combined. Biometric authentication is a chain
process, as depicted in Figure 2.4
for a more detailed explanation:


Figure
2
.4

Authentication Process Flow


Fusion at the feature extraction level stands for immediate data mixing at the beginning
of the processing chain, while fusion at the decision level represents late integration at t
he
end of the process. The following section describes each of these architectures in detail:


2.
7
.
1 Fusion at the Image Level

In practice, it is possible to combine only “compatible images”. Therefore, in the context
of facial images, image level combination is used only to combine multiple images of the
same face. Facial images first need to be registered with each other before
fusion. In case
facial images from multiple sensors are combined, the sensors must be pre
-
registered.
The objective of such a fusion is to improve the quality of acquired facial images.
The
basis of this improvement is that the useful signal in the facial
images captured with
different sensing technologies will be independent because different imaging conditions
capture different surface and sub
-
surface properties of the skin. Another reason to
conduct the image level fusion of facial images is to increase
the acquired fingerprint area
wherein each individual image has captured only a portion of the face.

43



Figure 2.5 Fusion at the Image level

2.
7
.
2 Fusion at the Feature Extraction Level

In this architecture, the information extracted from the different sens
ors is encoded into a
joint feature vector, which is then compared to an enrollment template which itself is a
joint feature via stored in a database and assigned a matching score as in a single
biometric system.



Figure 2.6 Fusion at the Feature Extract
ion level


However, feature
-
level fusion may be preferred because features are more compact than
image, leading to efficient fusion algorithms. But fusion at the feature extraction level is
much less preferable than the other next strategies. Two main prob
lems with this
approach are used to identify the problems:



The feature vectors to be joined might be incompatible due to numerical
problems, or some of them might even be unavailable in cases where the users
possess all biometric identifiers. While the first issue might be resolved by
44


careful system design, leadin
g to a very tightly coupled system, the second one
will cause the enrollment problems we already know from single biometric
systems.



Score generation is problematic: even in a single biometric system, it is difficult
to find a good classifier, i.e. to gene
rate a representative score based on the
matching of feature vector and enrollment template. But for the high
-
dimensional
joint feature vectors in a multi biometric system, it is even more complic
ated. As
pointed out in [17
], the relationship between the d
ifferent components of the joint
feature vector may not be linear.

2.
7
.
3 Fusion at the Rank Level

Rank
-
level fusion is used only in identification systems and is applicable when the
matcher output is a ranking of the “candidates” in the template database.
The system is
expected to assign a higher rank to a template that is more similar to the query. Most
identification system actually provides the matching score associated with the candidates.
Therefore, the rank level fusion is widely used in other fields
such as pattern recognition
and data mining. This method involves combining identification ranks obtained from
multiple unimodal biometrics. It consolidates a rank that is used for making final
decision. In multimodal biometric system, rank level fusion is

the method of
consolidating ranks from different biometric modalities (fingerprints, facial features, iris,
retina etc.) to recognize a user. Ho et al., 1994 describe three methods to

combine the
ranks assigned by different matchers. In the highest rank m
ethod,

each possible identity is
assigned the best (minimum) of all ranks computed by

different systems. Ties are broken
randomly to arrive at a strict ranking order

and the final decision is made based on the
consolidated ranks. The Borda

count method use
s the sum of the ranks assigned by the
individual systems to a

particular identity in order to calculate the fused rank. The logistic
regression

method is a generalization of the Borda count method where a weighted sum

of the individual ranks is used. The
weights are determined using logistic

regression.




45


2.
7
.
4 Fusion at the Matching Score Level

Each system

provides a matching score indicating the

proximity of the feature vector
with the template

vector. These scores can be combined

to assert the veracity

of the
claimed identity.

Techniques such as logistic regression may be

used to combine the
scores reported by the

two sensors.

Figure 2.7 Fusion at the Matching Score level

In a multi biometric system built on this architecture, feature vectors are creat
ed in
parallel for each sensor and then compare it with the enrollment templates, which are
stored independently for each biometric sample. Based on the closeness of feature vector
and template, each subsystem now computes its own matching score. These use
rs’ scores
are finally combined into a total score, which is handed over to the decision module. The
complete process is shown in figure 2.7.

Score level fusion is widely recognized to offer the best tradeoff between the
effectiveness of fusion and the ea
se of fusion. While the information contained in
matching scores is not as rich as in images or features, it is much richer than ranks and
decisions. Further, while score
-
level fusion is not as easy or intuitive as rank
-
level or
decision
-
level fusion, it i
s easier to study and implement than image
-
level and feature
-
level fusion. Scores are typically more accessible and available than images or features
but it does not contain rich information than images or features. Also, fusion at the score
level requires

some care. But the main difficulties in the score level fusion may emanate
46


from the non
-
homogeneity of scores from different matchers, differences in the
distributions of the scores, correlation among the scores, and differences in the accuracies
of diffe
rent matchers
. All scores were mapped to the range

[0; 100].

A very well
-
designed example for this fusion strategy has recently been presented by Ross and

Jain in
two research papers [18
]. They include facial scan, iris verification and hand geometry
scan into a common authentication system, and using well known methods for each
identifier (Eigen faces for the facial scan, patterns for the iris system, and commonly used
hand geometry feat
ures).
A score vector
-

(x1; x2; x3)
-

represents the scores of multiple
matchers,

with x1, x2 and x3 corresponding to the scores obtained from the 3 modalities.

Matching scores for the three different modalities are then normalized and combined
using one
of the following strategies.


1. Sum Rule:

The sum rule method of integration takes t
he weighted average of the
indi
vidual score values. This strategy was applied to all possible combinations

of the
three modalities. Equal weights were assigned to each mod
ality as

the bias of each
matcher was not available.


2. Decision Tree:

The C5:0 programs was used to generate a decision tree from the
training set

of genuine and impostor score vectors. The training set consisted of 11; 025

impostor score vectors and 225

genuine score vectors. The test set consisted

of the same
number of independent impostor and genuine score vectors.

This strategy uses a sequence
of threshold comparisons on the different scores to make an authentication decision.


3. Linear Discriminant
Function:

Linear discriminant analysis of the training set helps
in transforming the 3
-
dimensional score vectors into a new subspace
in which the
separation between the classes of genuine user scores and imposter scores is maximized.

The test set vectors a
re classifi
ed by using the minimum

Mahalanobis distance rule (with
the assumption that the two classes have

unequal covariance matrices).

The optimal
parameters for this transformation are calculated in advance based on a training set. The
output score is
defined as the minimum distance to the centroids of the two classes, using
a special metric, the Mahalanobis distance.

47



Based on the experimental results, the authors make the observation that the sum rule
achieves the best performance. Most importantly, t
hey further extend the sum rule using a
really new approach: they suggest applying user
-
specific weights to the user biometric
samples to be combined as well as using user
-
specific threshold levels for making the
final authentication decision.


2.
7
.5

Fusion at the Decision Level

In this fusion strategy, a separate authentication decision is made for each biometric trait.
These decisions are then combined into a final vote, as shown below in fi
gure 2.8.

Decision
-
level fusion is not as popular as score
-
level or rank
-
level fusion. Still, it may be
the only feasible approach if the commercial biometric systems involved in fusion
provide only the
final match decision. Similar to the rank
-
level and score