Inter-modality Face Sketch

paraderollAI and Robotics

Nov 17, 2013 (3 years and 4 months ago)

84 views

Inter
-
modality Face Sketch
Recognition

Hamed

Kiani

Outline


Overview


Previous Works


Proposed Approach


Results


Summary


Inter
-
modality

Face Sketch Recognition
ICME'12

2

Overview


Face Recognition


Inter
-
modality Face Sketch Recognition
ICME'12

3

Known Face
Images

Face
Recognition
System

Identity

Input Face

Overview


Face sketch recognition


Inter
-
modality Face Sketch Recognition
ICME'12

4

Known Face
Photos

(Mug shot)

Photo
-
Sketch
Matching

Suspect’s identity

Viewing

Verbal
description

Drawing

Eyewitness

Police artist

Sketch

Overview










Modality Gap: the difference of visual cues
between face sketch and photo.

Inter
-
modality Face Sketch Recognition
ICME'12

5

Intra
-
modality approaches

Inter
-
modality approaches

Matching face photos and
sketches in a same modality by
(photo or sketch) Synthesis

Matching photo and sketch
of different modalities
(direct matching).

Tang and Wang

[ECCV’03]


Liu et al. [CVPR’5]

Wang and Tang

[PAMI’09]

Klare

and Jain [SPIE ‘10]

Klare

et al. [PAMI’11]

Zhang et al. [CVPR’11]

Image

synthesis

No modality gap


Modality

gap


No image synthesis

Overview





Visual cues of face come from:


Fine texture (appearance):


low contrast
details, flaws, moles, wrinkles , etc.


Coarse texture (shape):

high contrast
boundaries of facial components
eyes, mouth, etc

Inter
-
modality Face Sketch Recognition
ICME'12

6

Overview


Face textures and modality gap:


Fine textures of a face photo captured by camera (true
pixels)


Fine texture of a sketch is rendered by artist, depending on
drawing style and tools


Fine textures of face photo and sketch are not equivalent:
high amount
of modality gap


Coarse texture (facial component and boundaries) exists in
both sketch and photo


modality gap is
not affected
significantly by coarse texture


Inter
-
modality Face Sketch Recognition
ICME'12

7

Proposed Approach


H
istogram of
A
veraged
O
riented
G
radients
(HAOG):
a modified version of
H
istogram of
O
riented
G
radients (HOG)


HOG for sketch recognition:


Modeling local
appearance

and
shape


Based on
fine

and
coarse

textures.




Fine

texture leads to a high amount of modality
gap”




Inter
-
modality Face Sketch Recognition
ICME'12

8

Proposed Approach


Idea of HAOG:


E
mphasizing
coarse

texture much more than
fine

texture in feature extraction.



How?


By
averaged

gradient

vector (dominant
gradient) instead of pixel’s gradient vector
(orientation and magnitude).




Inter
-
modality Face Sketch Recognition
ICME'12

9

Proposed Approach


But
: Local gradients cannot directly be
averaged, opposite gradient vectors cancel
each other


Solution
: Doubling the angles of the gradient
vectors before averaging: equal to squaring
the length of gradient vectors [
Bazen

and
Grez
, 2002].


Inter
-
modality Face Sketch Recognition
ICME'12

10

Proposed Approach


Thus, we define squared gradient vectors

Inter
-
modality Face Sketch Recognition
ICME'12

11

Proposed Approach


HAOG


Inter
-
modality Face Sketch Recognition
ICME'12

12

x
-
gradient

y
-
gradient

Proposed Approach


HAOG




Inter
-
modality Face Sketch Recognition
ICME'12

13

HAOG

Proposed Approach


Given a query sketch and a gallery of face photos



, face sketch recognition is done by:







Inter
-
modality Face Sketch Recognition
ICME'12

14


: HAOG descriptor , :chi
-
square

Proposed Approach


Inter
-
modality Face Sketch Recognition
ICME'12

15

Figure 1. (a1) Face photo, (a2) Face sketch, (b1,b2) Gradient magnitudes of
(a1,a2), Squared gradient magnitudes of (a1,a2).


Proposed Approach


Inter
-
modality Face Sketch Recognition
ICME'12

16

Figure 2. Face sketch (top), photo (bottom), (
b,c,d
) local patches (first row),
HAOG descriptors (second row) and HOG descriptors (third row).

Results


Results on CUHK dataset with 606 pairs of face
photo/sketch

Inter
-
modality Face Sketch Recognition
ICME'12

17

Summary


Face sketch recognition vs. face recognition


Modality gap


HOG vs. HAOG


Future work



Inter
-
modality Face Sketch Recognition
ICME'12

18

Inter
-
modality Face Sketch Recognition
ICME'12

19