Active Appearance Models for Face Detection

bijoufriesΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 5 μήνες)

139 εμφανίσεις

Active
Appearance

Models

for

Face

Detection

Rocío Cabrera, Guillaume
Lemaître
,
Mojdeh

Rastgoo

Presentation

Outline

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

2


Introduction


Database Used


The IMM Face Database


Models


Statistical Shape Models


Statistical Models of
Appearance


Active Appearance Models


Implementation


Training Stage


Testing Stage


Conclusions

Introduction

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

3


Non
-
trivial Applications in Machine Vision


“Understand” the presented images


Recover image structure


Know what this structure means


Real applications include complex/variable structures


Faces Detection




Model
-
based Methods


Prior knowledge of the problem


Expected Shapes of Structures


Their Spatial Relationship


Greylevel

Appearance

Restrict

Automated

Search

to

Plausible
Interpretations

Introduction

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

4


Generative Models


Are able to generate realistic images of target objects


Deformable Models


Are
able to deal with object variability


Two main desired characteristics


General


capable of generating plausible examples of the class they represent


Specific


capable of generating only legal/valid example


Model
-
based Methods


Top
-
down Strategy


“Measure” if the target is actually present

Find Best Match in Image

Prior Model of Expected Class

The IMM Face Database

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

5


An Annotated Dataset of 240 Face Images


40 different human subjects (
7
females

vs. 33 males
)


All without glasses or accessories


Manual Annotation of 58 landmarks




Six Different

Positions


Full frontal face,
neutral/happy

expression, diffuse light


Face rotated (30
°

right/left
), neutral expression, diffuse light


Full frontal face, neutral expression,
spot light
added at the person's left side.


Full frontal face,
arbitrary
expression, diffuse light.


Eyebrows


Eyes


Nose


Mouth


Jaw

Statistical

Shape

Models

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

6

Shape alignment

Modelling

Shape Variation


Procrustes Analysis :
Aligning the images onto the same reference axes


Translation, Rotation and Scaling Transformations


Procrustes

Analysis

minimizes

the

distance

between

a

reference

shape

and

each

shape

in

the

dataset



Computation of the
mean shape



Computation of the
scatter (covariance) matrix



Sorting the eigenvectors and
keeping the first k eigenvectors

, based on the largest
eigenvalues




Eigen decomposition
of the shapes where ,



Value of
k

is based on


Statistical Shape Models

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

7


Mean Shape and Largest Deformation

Statistical

Shape

Models


1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

8

Statistical

Models

of
Appearance

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

9


Texture mapping is required to generate the photo realistic synthetic
images


Combination of a shape variation model with texture variation model










Configuration of landmarks



Texture is the pattern of intensities or color across the image patch




shape model

Texture model

Statistical Models of Appearance

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

10

Training set of label Images

Computation of statistical shape models
-
PCA

Computation of Free
-
Patch Images


Image
wrapping

Applying PCA on Free
-
Patch Images

Statistical texture Models

Statistical Shape models


Mean shapes

Appearance models

PCA

Statistical Models of Appearance


Image wrapping

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

11


Piece

Wise

Affine



Performing

the

Delaunay

triangulation

on

each

shape

model











Affine

Transformation

which

maps

the

corner

of

the

triangles

to

their

new

positions

in

new

Image



Statistical Models of Appearance


Texture modeling

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

12


Training set of shape
-
free normalized image
patches



Performing PCA


Model of texture:

Mean normalized gray level

Set of orthogonal modes of variations

Set of gray level parameters

Statistical

texture

Models


1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

13


Eigen
-
faces decomposition


Texture model

Statistical Models of Appearance


Combined Image

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

14


Shape parameter vector and texture parameter vector
might have correlation



Performing PCA


Appearance Model:

Controlling both shape and texture

Diagonal matrix of weight for each shape
parameter

Eigenvectors

Shape and texture will be a function of
c

Statistical

Appearance

Models


1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

15


Texture model


Combination of texture model
and shape model


Difference Image

Implementation

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

16



Consists of two main stages


Training Stage


Multi
-
scale implementation to obtain an AAM model


N
scales

implementation



Testing Stage


Searches for the object (face)

in a test
image

Implementation


Training Stage

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

17


Load Training Data


for SCALE = 1:N


Make

Shape

Model


Align shapes with
Procrustes Analysis


Obtain main directions of variations with
PCA


Keep the 98%
most significant eigenvectors


G
rey
-
level

appearance

Model


Transform face image into
mean texture image


Normalize the
greyscale
, to
compensate for illumination


Perform PCA


Keep the
99%

most significant eigenvectors


Co
mbined

Shape
-
Appearance

Model


Addition of the shape and appearance models


Perform PCA


Keep only
99%
of all eigenvectors


S
earch

Model


Find the object location in a test set


Training done by
translation

and
intensity difference
computation (keep position with smallest
difference)


Transform the image to a coarser scale


end

Make Shape Model

Grey
-
level Appearance Model

Combined Shape
-
Appearance Model

Search Model

Transform Image to a Coarser Scale

Implementation


Testing Stage

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

18


Manual Initialization


For

SCALE = 1:N (
start

in
coarser

scale
)


Get

Model

for

Current

Scale


Image

Scaling


Search

Iterations


Sample

Image

Intensities


Compute
difference

between

model

and real
image

intensities


If

Error
old

<
Error
current


Go

to

previous

location


Else


Update

Error
old


End


End


Go

to

next

finer

scale


End


Show
Detection

Results

Search Iterations

Manual Initialization

Show Detection Results

Results

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

19

Lower Scale

Higher Scale

Results

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

20

Highest Scale

Texture map found at this scale

Problems Faced during Implementation

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

21



Memory

issues

during

the

training



Problem

of
the

reconstruction

of
the

appearance



Not

real
-
time
application

Conclusion

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

22


Face

detection

and

face

tracking

are

non
-
trivial

applications

in

machine

vision


Model
-
based

methods


Prior knowledge of the problem


Expected Shapes of Structures


Their Spatial Relationship


Grey
-
level Appearance


Active

Appearance

Models


Are

built

from

a

set

of

training

examples



Should

account

for

class

variabilty


They

heavily

rely

on

Principal

Component/

Eigenvalue

Analysis


Through

a

search

algorithm

we

seek

to

interpret

a

new

target

image

with

the

optimal

model

parameters

which

best

describe

the

target

image


The

extension

to

Face

Detection

was

not

yet

achieved

but

it

is

expected

to

work

for

the

deliverable

due

date


The

use

of

AAM

seem

like

a

promising

method

to

perform

face

detection

and/
or

recognition

References

1/14/2011

3D Digitization
-

Active Appearance Models for Face Detection

23

[
1
]

Ginneken

B
.

et

al
.

"Active

Shape

Model

Segmentation

with

Optimal

Features",

IEEE

Transactions

on

Medical

Imaging

2002
.


[
2
]

T
.
F
.

Cootes
,

G
.
J

Edwards,

and

C,J
.

Taylor

"Active

Appearance

Models",

Proc
.

European

Conference

on

Computer

Vision

1998


[
3
]

T
.
F
.

Cootes
,

G
.
J

Edwards,

and

C,J
.

Taylor

"Active

Appearance

Models",

IEEE

Transactions

on

Pattern

Analysis

and

Machine

Intelligence

2001

[
4
]

Vazeos

Ioannis
.

Active

Appearance

Models

(AAM)
.

MASTER

THESIS

REPORT
.

Master

of

Science

in

Information

Networking
.

Athens

Information

Technology
.

2004

-
2005

[
5
]

T
.
F
.

Gootes

and

C
.
J
.

Taylor

Statistical

Models

of

Appearance

for

Computer

Vision
.

Imaging

Science

and

Biomedical

Engineering
,

University

of

Manchester
.

Technical

Report
.

2004
.

[
6
]

F
.

Dornaika

and

J
.

Ahlberg

.

Efficient

Active

Appearance

Model

for

Real
-
Time

Head

and

Facial

Feature

Tracking

Proceedings

of

the

IEEE

International

Workshop

on

Analysis

and

Modeling

of

Faces

and

Gestures

(AMFG’
03
)
.

2003

[
7
]

Akshay

Asthana
,

Jason

Saragih
,

Michael

Wagner

and

Roland

Goecke
.

Evaluating

AAM

Fitting

Methods

for

Facial

Expression

Recognition
.

2009

IEEE

[
8
]

Mingcai

Zhou,

Yangsheng

Wang,

Xiaoyan

Wang

and

Xuetao

Feng
.

A

Two
-
Stage

Approach

for

AAM

Fitting
.

Eighth

International

Conference

on

Intelligent

Systems

Design

and

Applications
.

2008

IEEE

[
9
]


Fangqi

Tang

and

Benzai

Deng

Facial

Expression

Recognition

using

AAM

and

Local

Facial

Features
.

Third

International

Conference

on

Natural

Computation

(ICNC

2007
)
.

2007
.