Application of a Post-processing Algorithm for Improved Human Face Recognition

breezebongAI and Robotics

Nov 6, 2013 (3 years and 10 months ago)

105 views

Application of a Post
-
processing Algorithm

for Improved Human Face Recognition

Metin Artiklar and Mohamad H. Hassoun

Paul Watta

Dept. Electrical and Computer Engineering

Wayne State University

Detroit, MI 48202


Dept. Electrical and Computer Engineering

University of Michigan
-
Dearborn

Dearborn, MI 48128

metin@tarek.eng.wayne.edu

watta@umich.edu


Abstract

This paper presents a shifting algorithm which
can be used in a pattern recognition system to
improve the system’s performance in the
presence of shif
ted input patterns. The algorithm
is outlined and simulation results are presented
for some human face recognition experiments. It
is shown that the shifting algorithm improves
recognition performance for several different
face recognition algorithms.

1. I
ntroduction

The problems of shift, rotation, and scaling are
troublesome for image processing applications,
such as automatic recognition systems. This
paper presents an algorithm which can be used to
improve the classification performance of such a
system

in the presence of shifted images.

Figure 1(a) shows an example of an 82
x
115
-
dimensional face image. To facilitate the shifting
process, the image is cropped to a size of 72
x
72,
as shown in (b). The image is shown in gray
scale, but it can be made binary

by simply
thresholding the gray levels at 127.

Figure 2 shows average Hamming distances (on
the binary images) computed among 100 face
images such as (b). The Hamming distance is
computed between the image and itself, but with
various amounts of shift an
d various directions
of the shift. In the middle of the Figure, since the
left/right and top/bottom shifts are 0, the
Hamming distance is 0 (no shift is applied). As
the image is moved 1 pixel to the right, the
Hamming distance is 10.4% (on average). If it

is
moved 2 pixels up, the average Hamming
distance is 14.6%.

Figure 2 shows that even a 1
-
pixel shift in the
image can yield a large change in Hamming
distance.



(a)





(b)

Figure 1.

(a) An 82
x
115 face image and (b)
the same image cropped to 72
x
72.

% Ave
Hamming distance
Up/down shift
Left/right shift

Figure 2.

The average Hamming distance (in
percent and averaged over 100 faces) between a
face image and shifted versions of itself. The 0 in
the center represents the case where no shift is
applied.

Note that all the images discussed here were
ob
tained in a laboratory setting using an
apparatus which constrains the amount of shift,
rotation, scale, and tilt of the face. In particular,
the experimental setup consists of a frame
attached to a tripod. The subject puts his or her
head in the frame, an
d the picture is snapped.
This setup eliminates the need for segmenting
the face from the rest of the image.

Even with this constrained method of snapping
the images, there can still be an appreciable
amount of shift present in (different) images of
the s
ame person. Figure 3 shows two different
pictures of the same individual. The images were
snapped within minutes of each other. Figure 4
shows the resulting Hamming distances as image
(b) is shifted and compared to image (a).



(a)


(b)

Figure 3.

Two different 7
2
x
72 images of the
same individual.


29.4

28.5

27.5

27.3

27.3

28.5

29.4

30.5

31.5

27.9

27.0

25.8

25.4

25.6

26.4

27.8

29.1

30.6

27.0

25.4

24.2

23.5

23.7

25.0

25.9

27.6

29.9

25.8

23.7

22.0

21.4

21.4

22.8

24.5

26.8

29.2

25.0

22.3

19.6

18.0

17.7

20.1

22.5

25.4

27.6

24.0

21.2

17.2

14.2

13.9

17.0

21.0

24.4

27.1

23.7

19.7

15.7

11.3

10.4

15.8

20.4

23.2

26.2

24.2

20.8

16.9

12.9

12.8

16.6

20.4

23.5

26.9

25.4

22.8

19.8

17.5

17.0

19.2

22.0

25.2

27.9

Figure 4.

This diagram shows the Hamming
distance (in percent) between

image 3(a) and
shifted versions of image 3(b).

At the center value in the table (where neither
image is shifted), the Hamming distance is
17.7%. Starting from this position, a path can be
outlined (shown underlined) which seeks to
minimize the Hamming dis
tance between the
two images. This is accomplished by examining
the Hamming distance in the four nearest
neighbors: north, south, east, and west. In this
case, the Hamming distance is smallest in the
south direction, hence the path moves in that
direction.

This process is continued until no
further improvements can be made, as shown in
the final Hamming distance value of 10.4%.

Here, by applying this shifting process, the
distance between the two images decreases by
41%. Increasing the match between a test
image
of a person and a stored prototype is desirable in
many face recognition systems.

Unfortunately, this process can also improve the
Hamming distance for images of different
people. For example, Figure 5 shows two images
of different people, and Figure

6 shows the
corresponding Hamming distances as image (b)
is shifted and compared to image (a). Again, a
path can be taken which lowers the Hamming
distance, but notice that the improvement in
Hamming distance here (28%) is smaller than the
case of the sam
e individual.



(a)


(b)

Figure 5.

Images of two different people.


25.3

26.4

28.2

30.7

31.8

32.8

33.7

35.4

37.5

23.7

25.3

27.3

29.5

31.3

32.5

33.6

35.2

37.4

23.1

24.5

27.1

29.2

31.2

32.9

33.7

35.2

37.5

23.6

25.5

27.3

29.8

31.9

33.3

34.2

35.7

38.0

24.3

26.2

28.3

30.5

32.6

34.0

34.8

36.3

38.5

25.6

27.1

29.3

31.4

33.5

34.6

35.1

36.6

38.7

26.8

28.7

30.9

32.7

34.3

35.2

35.7

37.0

38.9

28.0

29.5

32.1

33.6

34.9

36.1

36.7

37.8

39.5

30.0

31.8

33.5

34.9

35.8

37.2

38.0

39.1

40.8

Figure 6.

This diagram shows the Hamming
dista
nce (in percentage) between the two images
shown in Figure 5 with various amounts of shift.

Of course, for recognition purposes, it is desired
that the distance between images of different
people be as large as possible. The conjecture
here is that even th
ough the shifting process
decreases the distance between different people,
it tends to do so by a lesser amount than the
distance improvement for images of the same
person. Hence, there is an overall increase in
the separation ability of the classifier.

The process of shifting the input image to obtain
a better match between images (of the same
individual taken at different times) is the basis
for the post
-
processing algorithm which is
described in the next section.

2. The Shifting Algorithm

This method r
equires the use of a pattern
recognition scheme which can produce an
ordered list of outputs ranked according to
similarity to an input pattern. In our case, we use
a simple nearest neighbor classification scheme
and rank the outputs on the basis of Hammin
g
distance (in the case of binary images) and city
block distance (in the case of gray
-
scale images).

In this post
-
processing algorithm, we select two
parameters:
k
, the number of patterns from the
ordered list which will participate in the shifting
proces
s (i.e., the
k

patterns at the top of the list),
and
s
, the maximum path length in the shifting
process. To minimize computational time,
k

should be chosen as small as possible, but
sufficiently large so that there is a high
probability that the correct im
age is contained in
this subset of patterns.

As illustrated in the previous section, each of the
k

candidate output patterns is shifted by 1 pixel
in 4 possible directions: north, east, south, and
west. After computing a similarity measure for
each of thes
e directions, we select the direction
which yields the lowest distance and move the
k
th pattern in that direction. This process is
repeated at most
s

times, or until there is no
improvement in the distance measure in any
direction. The final distance is re
corded for each
of the
k

patterns, and the image which has the
least distance after this shifting process is taken
as the output of the system (the best match).

Note that when computing the optimal path, it is
not necessary to generate an entire grid of al
l
possibilities, as shown in Figure 5. Rather, we
need only compute those Hamming distances
along the path (and those distances surrounding
the optimal path).

In addition, from a computational point of view,
there is no need to actually shift the pixels o
f the
image at each step. Rather, one need only store
the current corner point of the shifted image and
use that as an offset index when computing the
required Hamming distance. Hence, this
algorithm can be implemented very efficiently.
In fact, the comput
ational burden of computing
the optimal path consists of computing 4
Hamming distances at each step (on the 72
x
72
images), along with finding the minimum of
those 4 values.

Many recognition system use several prototypes
of patterns to be recognized. For e
xample, in the
context of face recognition, it is possible to store
several images of each person. We have
constructed such a data set, containing 400
images of 100 different people. Each person has
4 images showing different facial expressions: a
blank ex
pression, smile, angry, and surprised.

In this case, it is possible to select the top 10
winners (
k

= 10) and then also include all 4
expressions of each person as part of the shifting
process. Hence, 40 images participate in the
shifting post
-
processing a
lgorithm. In the results
below, this method is called Shift
-
10.
Alternatively, we can simply select the top 40
images (
k

= 40) and apply the shift to these
images. This method is referred to as Shift
-
40.

3. Results

We tested the performance of this post
-
pr
ocessing algorithm on a data set consisting of
400 72
x
72
-
dimensional face images (100
different people), as described in the previous
section. Note that an additional test image was
taken of each person in the data set (a blank
expression).

Figure 7 shows

the results of the simulations
before and after the post
-
processing algorithm on
binary face images using two different
classification methods: a Hamming distance
method and a two
-
level decoupled Hamming
network (Watta, Akkal, and Hassoun, 1997;
Ikeda, Wa
tta, and Hassoun, 1998). The numbers
in column 1 indicate the percentage that the
correct person was ranked first in the list, and
columns 2 and 3 indicate what percentage of the
time the correct person was ranked second and
third, respectively.

For these
results,
s

was set at 5 and
k

was set at
40 (using the Shift
-
10 and Shift
-
40 approaches
as described in the previous section).


Post
Processing

Algorithm

1

2

3

None

Hamming Classifier

94

73

45

2
-
Level Hamming

79

58

41

Shift
-
10

Hamming Classifier

97

87

47

2
-
Level Hamming

97

88

53

Shift
-
40

Hamming Classifier

97

87

46

2
-
Level Hamming

98

87

46

Figure 7.

Results of the post
-
processing
algorithms for binary face images.

Figure 8 shows the results of the post
-
processing
algorithms when the gray
-
scale images are

used.
Here, 3 different recognition schemes are used: a
nearest neighbor classifier (using the city block
distance as the similarity measure), the two
-
level
decoupled Hamming network, and a wavelet face
recognition algorithm
(Stollnitz, DeRose, and
Sales
in, 1995; Jacobs, Finkelstein, and Salesin,
1995)
.

The results indicate that the proposed post
-
processing shifting algorithm improves overall
system performance. Most notable is the
improvement for the patterns which appear
second and third in the ordered

list. For example,
for binary images using the Hamming distance,
the correct person was present second on the list
only 76% of the time, but after applying the
shifting algorithm, the correct person appeared
90% of the time. Hence this post
-
processing
alg
orithm could be used in the context of a
sensor fusion scheme whereby final
classification is made on the basis of the
information present in several of the top
matching patterns, rather than just the best
matching pattern.

Future publications will explore

the use of this
shifting algorithm in a more practical recognition
system, which includes a mechanism for
rejecting an image when there is an insufficient
match between the input and one of the memory
patterns.


Post
Processing

Algorithm

1

2

3

None

Hammi
ng Classifier

94

82

52

2
-
Level Hamming

94

75

52

Wavelet Classifier

97

80

62

Shift
-
10

Hamming Classifier

97

89

65

2
-
Level Hamming

95

91

70

Wavelet Classifier

98

93

76

Shift
-
40

Hamming Classifier

98

89

65

2
-
Level Hamming

95

85

61

Wavelet Clas
sifier

97

90

75

Figure 8.

Results of the post
-
processing
algorithms for gray
-
scale images.

Acknowledgments

This work was supported by the National
Science Foundation (NSF) under contract ECS
-
9618597.

References

1.

Ikeda, N., Watta, P., and Hassoun, M. (1998).
“Capacit
y Analysis of the Two
-
Level Decoupled
Hamming Associative Memory”
Proceedings of

the

IEEE International Conference on Neural
Networks
, ICNN’98, May 4
-
9, 1998, Anchorage,
Alaska, pp. 486
-
491.

2.

Jacobs, C., Finkelstein, A., and Salesin, D.
(1995). "Fast Mult
iresolution Image Querying,"
Proceedings of SIGGAPH 95, in

Computer
Graphics Proceedings
, Annual Conference
Series, pp. 277
-
286, August 1995, Los Angeles,
CA.

3.

Stollnitz, E., DeRose, T., and Salesin, D. (1995).
"Wavelets for computer graphics: A primer, Par
t
1,"
IEEE Transactions on Computer Graphics
and Applications
,
15
(3), pp. 76
-
84.

4.

Watta, P., Akkal, M., and Hassoun, M. (1997).
“Decoupled Voting Hamming Associative
Memory Networks”
Proceedings of the

IEEE
International Conference on Neural Networks
,
ICNN’
97, June 9
-
12, 1997, Houston, Texas, pp.
1188
-
1193.