in Face Space

parathyroidsanchovyAI and Robotics

Nov 17, 2013 (3 years and 6 months ago)

62 views

Searching and Browsing Video


in Face Space


Lee Begeja


Zhu Liu


Video and Multimedia Technologies Research

Page
2

Face Oriented Video Browsing


Challenge
-

non linguistic browsing


Browse a video using faces


Anchorpersons in news broadcasts


Main casts in movies


Hosts and guests in talk shows

Face Finding


Face Detection


Find a face


Face Recognition


Find a specific
face


Face Clustering


Find a set of similar faces

Face Clustering

What’s in a Face ?

Feature extraction
-

50

Features


9



3 color moments in Luv space


Moments


mean, variance, skew


Luv


L
-
luminance; u,v

chrominance


24



Gabor textures


3 scales x 4
directions, mean and std dev


17



Edge detection histogram in 16 bins
across the 2∏ polar coordinate space; with
one bin for non
-
edge pixels


Face Clustering


Torso region alone
: The face dissimilarity is defined as
the torso region distance,
TD
.


Torso region and Icon region
: The face dissimilarity is
defined as weighted summation of torso and icon region
distance, α∙
TD

+ (1
-

α)∙
ID
, where α is the weighting
factor.


Torso region and Face region
: The face dissimilarity is
defined as the minimum of the torso region distance and
face distance based on eigenface projection,
min(
TD
,
FD
).



Icon region alone
: The
face dissimilarity is defined
as the icon region distance,
ID
.

Video Browsing
Interface

Page
7

Performance Metrics


Average Cluster Purity

(
ACP
)


perfect
ACP of 1.0 means each cluster only contains
faces from one person.


Average Face (Class) Purity

(
AFP
)


perfect AFP of 1.0 would have all the faces of
one person appearing in one cluster.


Analogous to precision(ACP) vs. recall(AFP)

Results

Face dissimilarity

Video 1
AFP ACP

Video 2
AFP ACP

Icon region alone

0.45

0.61

Torso region alone

0.47

0.82

0.57

0.92

Torso + Face regions
(α=0.5)

0.51

0.83

0.70

0.97

Torso + Face regions
(eigenface)

0.54

0.92

0.72

1

Future Work


Working with Sumit Chopra to incorporate
dimensionality reduction (DrLIM)



Face Search/Clustering across programs


Discussions with Patrick Haffner on using
SVMs for Face Recognition


Do specific face recognition (Obama, Leno)


Search for multiple faces within a frame


Improve Face Detection


Include user generated video in our results

Additional Slides


Thatcher

Effect

Gabor textures


3 scales x 4 directions

Directions

Scales

Eigenface

Eigenface approach is a PCA (Principal Component
Analysis) method, in which a small set of
characteristic pictures are used to describe the
variation between face images.

Recognition is performed by projecting a new image
onto the subspace spanned by the eigenfaces and
then classifying the face by comparing its position
in the face space with the positions of known
individuals.

Informally, eigenfaces are a set of "standardized
face ingredients", derived from analysis of many
pictures of faces. Any human face can be
considered to be a combination of these standard
faces. For example, a face might be composed of
the average face plus 20% from eigenface 1, 35%
from eigenface 2, and
-
12% from eigenface 3.


Eigenfaces