U of H
COSC 6397
–
Lecture 10
#
1
U of H
COSC 6397
Face Recognition in the Infrared Spectrum
Prof. Ioannis Pavlidis
U of H
COSC 6397
–
Lecture 10
#
2
Primary Applications
•
Biometric Identification
–
Passwords/PINs.
–
Tokens (like ID cards).
–
You can be your own password.
•
Surveillance
–
Off
-
the
-
shelf facial recognition
system that identifies humans as they
pass through a camera’s field of
view.
U of H
COSC 6397
–
Lecture 10
#
3
Novel Applications
•
Wearable Recognition Systems
–
Adapt to a specific user and be more
intimately and actively involved in the
user's activities.
–
Face recognition software can help
you remember the name of the
person you are looking at.
•
Useful for Alzheimer's patients.
•
Smart Systems
–
Key goal is to give machines perceptual abilities that allow them to
function naturally with people.
–
Critical for a variety of human
-
machine interfaces.
U of H
COSC 6397
–
Lecture 10
#
4
Why Infrared?
•
Visible light has no effect on images
taken in the thermal infrared
spectrum.
•
Even images taken in total darkness
are clear in the thermal infrared.
U of H
COSC 6397
–
Lecture 10
#
5
Why Infrared? (Contd..)
•
Illumination Invariance
–
Major problem in visible domain.
•
Uniqueness and Repeatability
–
Sense thermal patterns of blood vessels under the skin,
which transport warm blood throughout the body.
–
Remain relatively unaffected by aging.
–
Even identical twins have different thermograms.
•
Immune from Forgery
–
Disguises can be easily detected.
U of H
COSC 6397
–
Lecture 10
#
6
Previous Work
•
Lot of research was done in the visible band but little attention was
given in the infrared spectrum.
•
Recent reduction in the cost of infrared cameras and availability of
large data sets encouraged active research in infrared face recognition.
•
Low
-
Level Models
–
Directly analyze the image pixels and impose probabilities on the features.
–
Examples are PCA, ICA, and FDA.
–
Not good in challenging conditions.
•
High
-
Level Models
–
Synthesize images from 3D templates of known objects and impose probabilities
on transformations.
–
Template matching approaches.
–
Computationally expensive.
•
Our Proposal
–
Intermediate model which takes advantage of both Low
-
Level and High
-
Level
models.
U of H
COSC 6397
–
Lecture 10
#
7
Principal Component Analysis
•
A
D = H x W
pixel image of a face,
represented as a vector occupies a
single point in
D
2
-
dimensional image
space.
•
Images of faces being similar in overall
configuration, will not be randomly
distributed in this huge image space.
•
Therefore, they can be described by a
low dimensional subspace.
•
Main idea of PCA (cutler96):
–
To find vectors that best account for
variation of face images in entire
image space.
–
These vectors are called eigen
vectors.
–
Construct a face space and project the
images into this face space
(eigenfaces).
U of H
COSC 6397
–
Lecture 10
#
8
Eigenfaces Approach
-
Training
•
Training set of images represented by
1
,
2
,
3
,…,
M
•
The average training set is defined by
Ψ = (1/M) ∑
M
i=1
i
•
Each face differs from the average by
vector Φ
i
= Γ
i
–
Ψ
•
A covariance matrix is constructed as:
C = AA
T
, where A=[Φ
1
,…,Φ
M
]
•
Finding eigenvectors of
N
2
x
N
2
matrix
is intractable. Hence, find only
M
meaningful eigenvectors.
M
is typically
the size of the database.
U of H
COSC 6397
–
Lecture 10
#
9
Eigenfaces Approach
-
Training
•
Consider eigenvectors
v
i
of
A
T
A
such that
A
T
A
v
i
=
μ
i
v
i
•
Pre
-
multiplying by
A
,
A
A
T
(
A
v
i
) =
μ
i
(
A
v
i
)
•
The eigenfaces are
u
i
=
A
v
i
•
A face image can be projected into this face space by
Ω
k
= U
T
(Γ
k
–
Ψ); k=1,…,M
U of H
COSC 6397
–
Lecture 10
#
10
Eigenfaces Approach
-
Testing
•
The test image, Γ, is projected into the face space to obtain a vector, Ω:
Ω = U
T
(Γ
–
Ψ)
•
The distance of Ω to each face class is defined by
Є
k
2
= ||Ω
-
Ω
k
||
2
; k = 1,…,M
•
A distance threshold,Ө
c
, is half the largest distance between any two
face classes:
Ө
c
= ½ max
j,k
{||Ω
j
-
Ω
k
||}; j,k = 1,…,M
U of H
COSC 6397
–
Lecture 10
#
11
Eigenfaces Approach
-
Testing
•
Find the distance, Є , between the original image, Γ, and its
reconstructed image from the eigenface space, Γ
f
,
Є
2
= || Γ
–
Γ
f
||
2
, where Γ
f
= U * Ω + Ψ
•
Recognition process:
–
IF Є≥Ө
c
then input image is not a face image;
–
IF Є<Ө
c
AND Є
k
≥Ө
c
for all k
then input image contains an unknown face;
–
IF Є<Ө
c
AND Є
k
*=min
k
{ Є
k
} < Ө
c
then input image contains the face of individual k*
U of H
COSC 6397
–
Lecture 10
#
12
Limitations of Eigenfaces Approach
•
Variations in lighting conditions
–
Different lighting conditions for enrolment and query.
–
Bright light causing image saturation.
•
Differences in pose
–
Head orientation
–
2D feature distances appear to distort.
•
Expression
–
Change in feature location and shape.
U of H
COSC 6397
–
Lecture 10
#
13
IR Face Recognition
–
Training Phase
U of H
COSC 6397
–
Lecture 10
#
14
IR Face Recognition
–
Test Phase
U of H
COSC 6397
–
Lecture 10
#
15
Segmentation
•
Noise in the background may effect
the performance of a face
recognition system.
•
Remove the background.
•
Use thermal information on face to
compute the features.
•
Adaptive Fuzzy Segmentation (kakadiaris02)
–
Fuzzy affinity is assigned to spels w.r.t. target object spel.
–
Affinity is computed as weighted sum of the temperature and the
temperature gradient in the neighborhood of the target spel.
–
Minimal user interaction because of dynamically assigned weights.
U of H
COSC 6397
–
Lecture 10
#
16
Segmentation (Contd..)
•
Fuzzy affinity is calculated by:
–
Spatial Adjacency:
U of H
COSC 6397
–
Lecture 10
#
17
Segmentation (Contd..)
–
Temperature homogeneity & gradient:
–
Weights:
-
Temperature of seed c
-
Temperature of seed d
-
Mean Temperature
-
Standard deviation of temperature
U of H
COSC 6397
–
Lecture 10
#
18
Problem with Single Seed
•
Temperatures on face are
different at different regions.
•
If a single seed is chosen in a particular region,
then the connectivity stretches only along this
region and the segmentation goes wrong.
U of H
COSC 6397
–
Lecture 10
#
19
Multiple Seeds
•
Solution to this problem is to choose
multiple seeds in different regions
on face and merge the resulting
segmented parts .
•
Choose a seed pixel on face wherever
there is sharp change in gradient.
•
Works well even when the subject is
wearing glasses.
•
Robust to variation of
poses.
U of H
COSC 6397
–
Lecture 10
#
20
Choosing Multiple Seeds
U of H
COSC 6397
–
Lecture 10
#
21
Assumptions
•
Merge all resultant segmented regions to
form final image.
ASSUMPTIONS
•
The center of the image contains the pixel from facial region.
•
The temperatures at all pixels are mapped between 0 and 255.
–
If this mapped temperature at a pixel is between
175
-
200
, it is classified
to be in
blue
region.
–
If this mapped temperature at a pixel is between
200
-
225
, it is classified
to be in
pink
region.
–
If this mapped temperature at a pixel is between
225
-
255
, it is classified
to be in
cyan
region.
U of H
COSC 6397
–
Lecture 10
#
22
Feature Extraction
•
The Gabor filter bank is given by:
•
The segmented facial image is divided into its spectral components
using Gabor filters
.
•
The resultant Gabor filtered images are modeled using Bessel models.
U of H
COSC 6397
–
Lecture 10
#
23
Gabor Filter Bank
•
Example Gabor filter bank with 3 scale values and 4
orientation values:
U of H
COSC 6397
–
Lecture 10
#
24
Spectral Components
U of H
COSC 6397
–
Lecture 10
#
25
Bessel Parameters
•
The filtered images are modeled using Bessel parameters:
SK
–
Sample Kurtosis
SV
–
Sample Variance
•
Each segmented image in training set is convolved with the filters in
Gabor filter bank to obtain Gabor filtered images.
U of H
COSC 6397
–
Lecture 10
#
26
Sample Variance and Kurtosis
•
Sample Variance
is the measure of the “
spread”
of the
distribution
.
•
Sample Kurtosis
is the measure of the “
peakedness”
or
“
flatness”.
Sample Kurtosis,
U of H
COSC 6397
–
Lecture 10
#
27
Bessel Model
•
Using the bessel parameters
p
and
c
, t
he filtered image I
(j
)
(x,y)
is modeled as:
(p)
is gamma function
I
v
(z)
is modified bessel function of
first kind given by:
U of H
COSC 6397
–
Lecture 10
#
28
Bessel Model
U of H
COSC 6397
–
Lecture 10
#
29
Performance of Bessel K Forms
•
Kullback
-
Leiber divergence:
KL div=0.0013 KL div=0.0027 KL div=0.0055
KL div=0.0058
–
observed marginal density
–
Estimated Bessel Form
U of H
COSC 6397
–
Lecture 10
#
30
Comparing IR Images
•
Images modeled into Bessel parameters can be compared by:
•
L
2
-
metric between two Bessel forms f(x;p
1
,c
1
) and f(x;p
2
,c
2
) in
D
:
U of H
COSC 6397
–
Lecture 10
#
31
Hypothesis Pruning
•
Applying a high
-
level classifier on entire database is
computationally very expensive.
•
Pruning of hypotheses can be achieved by using Bessel
parameters (anuj01).
•
Helps in short listing best matches.
•
Bessel parameters for images in database can be computed
offline which helps in saving a lot of computation time.
U of H
COSC 6397
–
Lecture 10
#
32
Hypothesis Pruning (Contd..)
•
Define a probability mass function on the database
A:
(p
(j)
obs
,c
(j)
obs
)
–
observed Bessel parameters for test image I
(j)
(p
(j)
,s
,c
(j)
,s
)
–
estimated Bessel parameters which can be computed offline
•
Images in database
A
with
P
1
(
|I
)
greater than a specific
threshold value are short listed as best matches.
(D=0.3 for Equinox dataset)
U of H
COSC 6397
–
Lecture 10
#
33
Hypothesis Pruning (Contd..)
•
Shortlist the subjects of
A
with P
1
(
/I) greater than a specific
threshold:
U of H
COSC 6397
–
Lecture 10
#
34
Pruning Algorithm
U of H
COSC 6397
–
Lecture 10
#
35
Classification
•
Bayesian target recognition (anuj00) searches for the target
hypothesis with largest
posterior probability
given by:
–
Likelihood
:
–
Apriori
is same for all images in database (for database of
n images, it is 1/n for each image).
: Variance of test image
d : dimension of image (2 in this case)
U of H
COSC 6397
–
Lecture 10
#
36
Experiments
•
Equinox Database:
www.equinoxsensors.com
•
Image frame sequences were
acquired at 10 frames/sec while
the subject was reciting the
vowels ‘a’,’e’,’i’,’o’,’u’.
U of H
COSC 6397
–
Lecture 10
#
37
Results
–
ROC Curves
Correct Positive
: Test image is in the
database and is correctly recognized.
False Positive
: Test image is not in the
database, but is recognized to be an image
of the database
Negatives
: Test images that are not in the
database.
U of H
COSC 6397
–
Lecture 10
#
38
Results
–
Precision & Recall
U of H
COSC 6397
–
Lecture 10
#
39
Conclusion
•
We came up with a face recognition approach which is
computationally inexpensive and at the same time good
in challenging conditions.
•
The features of all images in database can be computed
offline and stored for future use. This saves lot of
computation time.
•
We improved the performance of classifier by removing
background noise of pruned hypothesis using adaptive
fuzzy connectedness based image segmentation.
U of H
COSC 6397
–
Lecture 10
#
40
References
•
[anuj01] A. Srivastava, X. W. Liu, B. Thomasson, and C. Hesher,
"Spectral Probability Models for IR Images with Applications to IR Face
Recognition," in
Proceedings 2001 IEEE Workshop on Computer Vision
Beyond the Visible Spectrum: Methods and Applications,
Kauai, HI, Dec
14
.
•
[cutler96] R. Cutler, “Face recognition using infrared images and
eigenfaces”,
website, http://www.cs.umd.edu/rgc/face/face.htm
, 1996.
•
[anuj00] A. Srivastava, M. I. Miller, and U. Grenander, “Bayesian
automated target recognition,"
Handbook of Image and Video
Processing
, Academic Press, pp. 869
-
881, 2000.
•
[kakadiaris02] A. Pednekar, I.A. Kakadiaris, U. Kurkure. Adaptive fuzzy
connectedness
-
based medical image segmentation. In
Proc. of the Indian
Conf. on Computer Vision, Graphics, and Image Processing (ICVGIP
2002)
, pp.457
-
462, Ahmedabad, India, December 16
-
18 2002.
Enter the password to open this PDF file:
File name:
-
File size:
-
Title:
-
Author:
-
Subject:
-
Keywords:
-
Creation Date:
-
Modification Date:
-
Creator:
-
PDF Producer:
-
PDF Version:
-
Page Count:
-
Preparing document for printing…
0%
Σχόλια 0
Συνδεθείτε για να κοινοποιήσετε σχόλιο