Application of a Post

processing Algorithm
for Improved Human Face Recognition
Metin Artiklar and Mohamad H. Hassoun
Paul Watta
Dept. Electrical and Computer Engineering
Wayne State University
Detroit, MI 48202
Dept. Electrical and Computer Engineering
University of Michigan

Dearborn
Dearborn, MI 48128
metin@tarek.eng.wayne.edu
watta@umich.edu
Abstract
This paper presents a shifting algorithm which
can be used in a pattern recognition system to
improve the system’s performance in the
presence of shif
ted input patterns. The algorithm
is outlined and simulation results are presented
for some human face recognition experiments. It
is shown that the shifting algorithm improves
recognition performance for several different
face recognition algorithms.
1. I
ntroduction
The problems of shift, rotation, and scaling are
troublesome for image processing applications,
such as automatic recognition systems. This
paper presents an algorithm which can be used to
improve the classification performance of such a
system
in the presence of shifted images.
Figure 1(a) shows an example of an 82
x
115

dimensional face image. To facilitate the shifting
process, the image is cropped to a size of 72
x
72,
as shown in (b). The image is shown in gray
scale, but it can be made binary
by simply
thresholding the gray levels at 127.
Figure 2 shows average Hamming distances (on
the binary images) computed among 100 face
images such as (b). The Hamming distance is
computed between the image and itself, but with
various amounts of shift an
d various directions
of the shift. In the middle of the Figure, since the
left/right and top/bottom shifts are 0, the
Hamming distance is 0 (no shift is applied). As
the image is moved 1 pixel to the right, the
Hamming distance is 10.4% (on average). If it
is
moved 2 pixels up, the average Hamming
distance is 14.6%.
Figure 2 shows that even a 1

pixel shift in the
image can yield a large change in Hamming
distance.
(a)
(b)
Figure 1.
(a) An 82
x
115 face image and (b)
the same image cropped to 72
x
72.
% Ave
Hamming distance
Up/down shift
Left/right shift
Figure 2.
The average Hamming distance (in
percent and averaged over 100 faces) between a
face image and shifted versions of itself. The 0 in
the center represents the case where no shift is
applied.
Note that all the images discussed here were
ob
tained in a laboratory setting using an
apparatus which constrains the amount of shift,
rotation, scale, and tilt of the face. In particular,
the experimental setup consists of a frame
attached to a tripod. The subject puts his or her
head in the frame, an
d the picture is snapped.
This setup eliminates the need for segmenting
the face from the rest of the image.
Even with this constrained method of snapping
the images, there can still be an appreciable
amount of shift present in (different) images of
the s
ame person. Figure 3 shows two different
pictures of the same individual. The images were
snapped within minutes of each other. Figure 4
shows the resulting Hamming distances as image
(b) is shifted and compared to image (a).
(a)
(b)
Figure 3.
Two different 7
2
x
72 images of the
same individual.
29.4
28.5
27.5
27.3
27.3
28.5
29.4
30.5
31.5
27.9
27.0
25.8
25.4
25.6
26.4
27.8
29.1
30.6
27.0
25.4
24.2
23.5
23.7
25.0
25.9
27.6
29.9
25.8
23.7
22.0
21.4
21.4
22.8
24.5
26.8
29.2
25.0
22.3
19.6
18.0
17.7
20.1
22.5
25.4
27.6
24.0
21.2
17.2
14.2
13.9
17.0
21.0
24.4
27.1
23.7
19.7
15.7
11.3
10.4
15.8
20.4
23.2
26.2
24.2
20.8
16.9
12.9
12.8
16.6
20.4
23.5
26.9
25.4
22.8
19.8
17.5
17.0
19.2
22.0
25.2
27.9
Figure 4.
This diagram shows the Hamming
distance (in percent) between
image 3(a) and
shifted versions of image 3(b).
At the center value in the table (where neither
image is shifted), the Hamming distance is
17.7%. Starting from this position, a path can be
outlined (shown underlined) which seeks to
minimize the Hamming dis
tance between the
two images. This is accomplished by examining
the Hamming distance in the four nearest
neighbors: north, south, east, and west. In this
case, the Hamming distance is smallest in the
south direction, hence the path moves in that
direction.
This process is continued until no
further improvements can be made, as shown in
the final Hamming distance value of 10.4%.
Here, by applying this shifting process, the
distance between the two images decreases by
41%. Increasing the match between a test
image
of a person and a stored prototype is desirable in
many face recognition systems.
Unfortunately, this process can also improve the
Hamming distance for images of different
people. For example, Figure 5 shows two images
of different people, and Figure
6 shows the
corresponding Hamming distances as image (b)
is shifted and compared to image (a). Again, a
path can be taken which lowers the Hamming
distance, but notice that the improvement in
Hamming distance here (28%) is smaller than the
case of the sam
e individual.
(a)
(b)
Figure 5.
Images of two different people.
25.3
26.4
28.2
30.7
31.8
32.8
33.7
35.4
37.5
23.7
25.3
27.3
29.5
31.3
32.5
33.6
35.2
37.4
23.1
24.5
27.1
29.2
31.2
32.9
33.7
35.2
37.5
23.6
25.5
27.3
29.8
31.9
33.3
34.2
35.7
38.0
24.3
26.2
28.3
30.5
32.6
34.0
34.8
36.3
38.5
25.6
27.1
29.3
31.4
33.5
34.6
35.1
36.6
38.7
26.8
28.7
30.9
32.7
34.3
35.2
35.7
37.0
38.9
28.0
29.5
32.1
33.6
34.9
36.1
36.7
37.8
39.5
30.0
31.8
33.5
34.9
35.8
37.2
38.0
39.1
40.8
Figure 6.
This diagram shows the Hamming
dista
nce (in percentage) between the two images
shown in Figure 5 with various amounts of shift.
Of course, for recognition purposes, it is desired
that the distance between images of different
people be as large as possible. The conjecture
here is that even th
ough the shifting process
decreases the distance between different people,
it tends to do so by a lesser amount than the
distance improvement for images of the same
person. Hence, there is an overall increase in
the separation ability of the classifier.
The process of shifting the input image to obtain
a better match between images (of the same
individual taken at different times) is the basis
for the post

processing algorithm which is
described in the next section.
2. The Shifting Algorithm
This method r
equires the use of a pattern
recognition scheme which can produce an
ordered list of outputs ranked according to
similarity to an input pattern. In our case, we use
a simple nearest neighbor classification scheme
and rank the outputs on the basis of Hammin
g
distance (in the case of binary images) and city
block distance (in the case of gray

scale images).
In this post

processing algorithm, we select two
parameters:
k
, the number of patterns from the
ordered list which will participate in the shifting
proces
s (i.e., the
k
patterns at the top of the list),
and
s
, the maximum path length in the shifting
process. To minimize computational time,
k
should be chosen as small as possible, but
sufficiently large so that there is a high
probability that the correct im
age is contained in
this subset of patterns.
As illustrated in the previous section, each of the
k
candidate output patterns is shifted by 1 pixel
in 4 possible directions: north, east, south, and
west. After computing a similarity measure for
each of thes
e directions, we select the direction
which yields the lowest distance and move the
k
th pattern in that direction. This process is
repeated at most
s
times, or until there is no
improvement in the distance measure in any
direction. The final distance is re
corded for each
of the
k
patterns, and the image which has the
least distance after this shifting process is taken
as the output of the system (the best match).
Note that when computing the optimal path, it is
not necessary to generate an entire grid of al
l
possibilities, as shown in Figure 5. Rather, we
need only compute those Hamming distances
along the path (and those distances surrounding
the optimal path).
In addition, from a computational point of view,
there is no need to actually shift the pixels o
f the
image at each step. Rather, one need only store
the current corner point of the shifted image and
use that as an offset index when computing the
required Hamming distance. Hence, this
algorithm can be implemented very efficiently.
In fact, the comput
ational burden of computing
the optimal path consists of computing 4
Hamming distances at each step (on the 72
x
72
images), along with finding the minimum of
those 4 values.
Many recognition system use several prototypes
of patterns to be recognized. For e
xample, in the
context of face recognition, it is possible to store
several images of each person. We have
constructed such a data set, containing 400
images of 100 different people. Each person has
4 images showing different facial expressions: a
blank ex
pression, smile, angry, and surprised.
In this case, it is possible to select the top 10
winners (
k
= 10) and then also include all 4
expressions of each person as part of the shifting
process. Hence, 40 images participate in the
shifting post

processing a
lgorithm. In the results
below, this method is called Shift

10.
Alternatively, we can simply select the top 40
images (
k
= 40) and apply the shift to these
images. This method is referred to as Shift

40.
3. Results
We tested the performance of this post

pr
ocessing algorithm on a data set consisting of
400 72
x
72

dimensional face images (100
different people), as described in the previous
section. Note that an additional test image was
taken of each person in the data set (a blank
expression).
Figure 7 shows
the results of the simulations
before and after the post

processing algorithm on
binary face images using two different
classification methods: a Hamming distance
method and a two

level decoupled Hamming
network (Watta, Akkal, and Hassoun, 1997;
Ikeda, Wa
tta, and Hassoun, 1998). The numbers
in column 1 indicate the percentage that the
correct person was ranked first in the list, and
columns 2 and 3 indicate what percentage of the
time the correct person was ranked second and
third, respectively.
For these
results,
s
was set at 5 and
k
was set at
40 (using the Shift

10 and Shift

40 approaches
as described in the previous section).
Post
Processing
Algorithm
1
2
3
None
Hamming Classifier
94
73
45
2

Level Hamming
79
58
41
Shift

10
Hamming Classifier
97
87
47
2

Level Hamming
97
88
53
Shift

40
Hamming Classifier
97
87
46
2

Level Hamming
98
87
46
Figure 7.
Results of the post

processing
algorithms for binary face images.
Figure 8 shows the results of the post

processing
algorithms when the gray

scale images are
used.
Here, 3 different recognition schemes are used: a
nearest neighbor classifier (using the city block
distance as the similarity measure), the two

level
decoupled Hamming network, and a wavelet face
recognition algorithm
(Stollnitz, DeRose, and
Sales
in, 1995; Jacobs, Finkelstein, and Salesin,
1995)
.
The results indicate that the proposed post

processing shifting algorithm improves overall
system performance. Most notable is the
improvement for the patterns which appear
second and third in the ordered
list. For example,
for binary images using the Hamming distance,
the correct person was present second on the list
only 76% of the time, but after applying the
shifting algorithm, the correct person appeared
90% of the time. Hence this post

processing
alg
orithm could be used in the context of a
sensor fusion scheme whereby final
classification is made on the basis of the
information present in several of the top
matching patterns, rather than just the best
matching pattern.
Future publications will explore
the use of this
shifting algorithm in a more practical recognition
system, which includes a mechanism for
rejecting an image when there is an insufficient
match between the input and one of the memory
patterns.
Post
Processing
Algorithm
1
2
3
None
Hammi
ng Classifier
94
82
52
2

Level Hamming
94
75
52
Wavelet Classifier
97
80
62
Shift

10
Hamming Classifier
97
89
65
2

Level Hamming
95
91
70
Wavelet Classifier
98
93
76
Shift

40
Hamming Classifier
98
89
65
2

Level Hamming
95
85
61
Wavelet Clas
sifier
97
90
75
Figure 8.
Results of the post

processing
algorithms for gray

scale images.
Acknowledgments
This work was supported by the National
Science Foundation (NSF) under contract ECS

9618597.
References
1.
Ikeda, N., Watta, P., and Hassoun, M. (1998).
“Capacit
y Analysis of the Two

Level Decoupled
Hamming Associative Memory”
Proceedings of
the
IEEE International Conference on Neural
Networks
, ICNN’98, May 4

9, 1998, Anchorage,
Alaska, pp. 486

491.
2.
Jacobs, C., Finkelstein, A., and Salesin, D.
(1995). "Fast Mult
iresolution Image Querying,"
Proceedings of SIGGAPH 95, in
Computer
Graphics Proceedings
, Annual Conference
Series, pp. 277

286, August 1995, Los Angeles,
CA.
3.
Stollnitz, E., DeRose, T., and Salesin, D. (1995).
"Wavelets for computer graphics: A primer, Par
t
1,"
IEEE Transactions on Computer Graphics
and Applications
,
15
(3), pp. 76

84.
4.
Watta, P., Akkal, M., and Hassoun, M. (1997).
“Decoupled Voting Hamming Associative
Memory Networks”
Proceedings of the
IEEE
International Conference on Neural Networks
,
ICNN’
97, June 9

12, 1997, Houston, Texas, pp.
1188

1193.
Comments 0
Log in to post a comment