Prof. Ioannis Pavlidis

parathyroidsanchovyΤεχνίτη Νοημοσύνη και Ρομποτική

17 Νοε 2013 (πριν από 3 χρόνια και 7 μήνες)

90 εμφανίσεις

U of H

COSC 6397


Lecture 10

#
1

U of H

COSC 6397

Face Recognition in the Infrared Spectrum

Prof. Ioannis Pavlidis

U of H

COSC 6397


Lecture 10

#
2

Primary Applications



Biometric Identification


Passwords/PINs.


Tokens (like ID cards).


You can be your own password.



Surveillance


Off
-
the
-
shelf facial recognition
system that identifies humans as they
pass through a camera’s field of
view.




U of H

COSC 6397


Lecture 10

#
3

Novel Applications


Wearable Recognition Systems



Adapt to a specific user and be more
intimately and actively involved in the
user's activities.


Face recognition software can help
you remember the name of the
person you are looking at.


Useful for Alzheimer's patients.




Smart Systems



Key goal is to give machines perceptual abilities that allow them to
function naturally with people.


Critical for a variety of human
-
machine interfaces.


U of H

COSC 6397


Lecture 10

#
4

Why Infrared?


Visible light has no effect on images
taken in the thermal infrared
spectrum.


Even images taken in total darkness
are clear in the thermal infrared.



U of H

COSC 6397


Lecture 10

#
5

Why Infrared? (Contd..)



Illumination Invariance


Major problem in visible domain.



Uniqueness and Repeatability


Sense thermal patterns of blood vessels under the skin,
which transport warm blood throughout the body.


Remain relatively unaffected by aging.


Even identical twins have different thermograms.



Immune from Forgery



Disguises can be easily detected.




U of H

COSC 6397


Lecture 10

#
6

Previous Work


Lot of research was done in the visible band but little attention was
given in the infrared spectrum.


Recent reduction in the cost of infrared cameras and availability of
large data sets encouraged active research in infrared face recognition.


Low
-
Level Models


Directly analyze the image pixels and impose probabilities on the features.


Examples are PCA, ICA, and FDA.


Not good in challenging conditions.


High
-
Level Models


Synthesize images from 3D templates of known objects and impose probabilities
on transformations.


Template matching approaches.


Computationally expensive.


Our Proposal


Intermediate model which takes advantage of both Low
-
Level and High
-
Level
models.



U of H

COSC 6397


Lecture 10

#
7

Principal Component Analysis


A
D = H x W

pixel image of a face,
represented as a vector occupies a
single point in
D
2
-
dimensional image
space.


Images of faces being similar in overall
configuration, will not be randomly
distributed in this huge image space.


Therefore, they can be described by a
low dimensional subspace.


Main idea of PCA (cutler96):


To find vectors that best account for
variation of face images in entire
image space.


These vectors are called eigen
vectors.


Construct a face space and project the
images into this face space
(eigenfaces).


U of H

COSC 6397


Lecture 10

#
8

Eigenfaces Approach
-

Training


Training set of images represented by





1
,

2
,

3
,…,

M




The average training set is defined by



Ψ = (1/M) ∑
M
i=1


i



Each face differs from the average by
vector Φ
i

= Γ
i



Ψ




A covariance matrix is constructed as:


C = AA
T
, where A=[Φ
1
,…,Φ
M
]



Finding eigenvectors of
N
2

x
N
2

matrix
is intractable. Hence, find only
M

meaningful eigenvectors.
M

is typically
the size of the database.






U of H

COSC 6397


Lecture 10

#
9

Eigenfaces Approach
-

Training


Consider eigenvectors
v
i

of
A
T
A

such that





A
T
A
v
i

=
μ
i
v
i




Pre
-
multiplying by
A
,
A
A
T
(
A
v
i
) =
μ
i
(
A
v
i
)



The eigenfaces are


u
i

=
A
v
i





A face image can be projected into this face space by




Ω
k

= U
T

k



Ψ); k=1,…,M








U of H

COSC 6397


Lecture 10

#
10

Eigenfaces Approach
-

Testing


The test image, Γ, is projected into the face space to obtain a vector, Ω:


Ω = U
T



Ψ)



The distance of Ω to each face class is defined by



Є
k
2

= ||Ω
-
Ω
k
||
2
; k = 1,…,M



A distance threshold,Ө
c
, is half the largest distance between any two
face classes:




Ө
c

= ½ max
j,k

{||Ω
j
-
Ω
k
||}; j,k = 1,…,M


U of H

COSC 6397


Lecture 10

#
11

Eigenfaces Approach
-

Testing



Find the distance, Є , between the original image, Γ, and its
reconstructed image from the eigenface space, Γ
f
,



Є
2

= || Γ


Γ
f

||
2
, where Γ
f
= U * Ω + Ψ



Recognition process:


IF Є≥Ө
c

then input image is not a face image;


IF Є<Ө
c

AND Є
k
≥Ө
c

for all k

then input image contains an unknown face;


IF Є<Ө
c

AND Є
k
*=min
k
{ Є
k
} < Ө
c


then input image contains the face of individual k*


U of H

COSC 6397


Lecture 10

#
12

Limitations of Eigenfaces Approach



Variations in lighting conditions


Different lighting conditions for enrolment and query.


Bright light causing image saturation.



Differences in pose


Head orientation


2D feature distances appear to distort.



Expression



Change in feature location and shape.


U of H

COSC 6397


Lecture 10

#
13

IR Face Recognition


Training Phase

U of H

COSC 6397


Lecture 10

#
14

IR Face Recognition


Test Phase

U of H

COSC 6397


Lecture 10

#
15

Segmentation


Noise in the background may effect
the performance of a face
recognition system.



Remove the background.



Use thermal information on face to
compute the features.



Adaptive Fuzzy Segmentation (kakadiaris02)


Fuzzy affinity is assigned to spels w.r.t. target object spel.


Affinity is computed as weighted sum of the temperature and the
temperature gradient in the neighborhood of the target spel.


Minimal user interaction because of dynamically assigned weights.


U of H

COSC 6397


Lecture 10

#
16

Segmentation (Contd..)


Fuzzy affinity is calculated by:



Spatial Adjacency:


U of H

COSC 6397


Lecture 10

#
17

Segmentation (Contd..)


Temperature homogeneity & gradient:



Weights:


-

Temperature of seed c


-

Temperature of seed d


-

Mean Temperature

-

Standard deviation of temperature


U of H

COSC 6397


Lecture 10

#
18

Problem with Single Seed


Temperatures on face are
different at different regions.


If a single seed is chosen in a particular region,
then the connectivity stretches only along this
region and the segmentation goes wrong.

U of H

COSC 6397


Lecture 10

#
19

Multiple Seeds


Solution to this problem is to choose
multiple seeds in different regions
on face and merge the resulting
segmented parts .


Choose a seed pixel on face wherever
there is sharp change in gradient.


Works well even when the subject is
wearing glasses.


Robust to variation of
poses.

U of H

COSC 6397


Lecture 10

#
20

Choosing Multiple Seeds

U of H

COSC 6397


Lecture 10

#
21

Assumptions


Merge all resultant segmented regions to
form final image.

ASSUMPTIONS


The center of the image contains the pixel from facial region.


The temperatures at all pixels are mapped between 0 and 255.


If this mapped temperature at a pixel is between
175
-

200
, it is classified
to be in
blue

region.


If this mapped temperature at a pixel is between
200
-

225
, it is classified
to be in
pink

region.


If this mapped temperature at a pixel is between
225
-

255
, it is classified
to be in
cyan

region.

U of H

COSC 6397


Lecture 10

#
22

Feature Extraction


The Gabor filter bank is given by:


The segmented facial image is divided into its spectral components
using Gabor filters
.


The resultant Gabor filtered images are modeled using Bessel models.


U of H

COSC 6397


Lecture 10

#
23

Gabor Filter Bank


Example Gabor filter bank with 3 scale values and 4
orientation values:


U of H

COSC 6397


Lecture 10

#
24

Spectral Components

U of H

COSC 6397


Lecture 10

#
25

Bessel Parameters


The filtered images are modeled using Bessel parameters:





SK


Sample Kurtosis


SV


Sample Variance






Each segmented image in training set is convolved with the filters in
Gabor filter bank to obtain Gabor filtered images.


U of H

COSC 6397


Lecture 10

#
26

Sample Variance and Kurtosis


Sample Variance

is the measure of the “
spread”
of the
distribution
.



Sample Kurtosis

is the measure of the “
peakedness”

or

flatness”.


Sample Kurtosis,

U of H

COSC 6397


Lecture 10

#
27

Bessel Model



Using the bessel parameters
p

and
c
, t
he filtered image I
(j
)
(x,y)

is modeled as:




(p)

is gamma function

I
v
(z)

is modified bessel function of
first kind given by:

U of H

COSC 6397


Lecture 10

#
28

Bessel Model

U of H

COSC 6397


Lecture 10

#
29

Performance of Bessel K Forms


Kullback
-
Leiber divergence:









KL div=0.0013 KL div=0.0027 KL div=0.0055

KL div=0.0058











observed marginal density




Estimated Bessel Form

U of H

COSC 6397


Lecture 10

#
30

Comparing IR Images


Images modeled into Bessel parameters can be compared by:








L
2
-
metric between two Bessel forms f(x;p
1
,c
1
) and f(x;p
2
,c
2
) in
D
:




U of H

COSC 6397


Lecture 10

#
31

Hypothesis Pruning


Applying a high
-
level classifier on entire database is
computationally very expensive.



Pruning of hypotheses can be achieved by using Bessel
parameters (anuj01).



Helps in short listing best matches.



Bessel parameters for images in database can be computed
offline which helps in saving a lot of computation time.

U of H

COSC 6397


Lecture 10

#
32

Hypothesis Pruning (Contd..)


Define a probability mass function on the database
A:

(p
(j)
obs
,c
(j)
obs
)


observed Bessel parameters for test image I
(j)

(p
(j)

,s
,c
(j)

,s
)


estimated Bessel parameters which can be computed offline


Images in database
A

with
P
1
(

|I
)

greater than a specific
threshold value are short listed as best matches.

(D=0.3 for Equinox dataset)

U of H

COSC 6397


Lecture 10

#
33

Hypothesis Pruning (Contd..)


Shortlist the subjects of

A

with P
1
(

/I) greater than a specific
threshold:


U of H

COSC 6397


Lecture 10

#
34

Pruning Algorithm

U of H

COSC 6397


Lecture 10

#
35

Classification


Bayesian target recognition (anuj00) searches for the target
hypothesis with largest
posterior probability

given by:


Likelihood
:


Apriori
is same for all images in database (for database of


n images, it is 1/n for each image).



: Variance of test image

d : dimension of image (2 in this case)

U of H

COSC 6397


Lecture 10

#
36

Experiments



Equinox Database:

www.equinoxsensors.com




Image frame sequences were
acquired at 10 frames/sec while
the subject was reciting the
vowels ‘a’,’e’,’i’,’o’,’u’.


U of H

COSC 6397


Lecture 10

#
37

Results


ROC Curves

Correct Positive

: Test image is in the
database and is correctly recognized.

False Positive

: Test image is not in the
database, but is recognized to be an image
of the database

Negatives

: Test images that are not in the
database.

U of H

COSC 6397


Lecture 10

#
38

Results


Precision & Recall

U of H

COSC 6397


Lecture 10

#
39

Conclusion



We came up with a face recognition approach which is
computationally inexpensive and at the same time good
in challenging conditions.



The features of all images in database can be computed
offline and stored for future use. This saves lot of
computation time.



We improved the performance of classifier by removing
background noise of pruned hypothesis using adaptive
fuzzy connectedness based image segmentation.



U of H

COSC 6397


Lecture 10

#
40

References



[anuj01] A. Srivastava, X. W. Liu, B. Thomasson, and C. Hesher,
"Spectral Probability Models for IR Images with Applications to IR Face
Recognition," in
Proceedings 2001 IEEE Workshop on Computer Vision
Beyond the Visible Spectrum: Methods and Applications,
Kauai, HI, Dec
14
.



[cutler96] R. Cutler, “Face recognition using infrared images and
eigenfaces”,
website, http://www.cs.umd.edu/rgc/face/face.htm
, 1996.


[anuj00] A. Srivastava, M. I. Miller, and U. Grenander, “Bayesian
automated target recognition,"
Handbook of Image and Video
Processing
, Academic Press, pp. 869
-
881, 2000.


[kakadiaris02] A. Pednekar, I.A. Kakadiaris, U. Kurkure. Adaptive fuzzy
connectedness
-
based medical image segmentation. In
Proc. of the Indian
Conf. on Computer Vision, Graphics, and Image Processing (ICVGIP
2002)
, pp.457
-
462, Ahmedabad, India, December 16
-
18 2002.