Principles of 3D Face

brasscoffeeΤεχνίτη Νοημοσύνη και Ρομποτική

17 Νοε 2013 (πριν από 3 χρόνια και 9 μήνες)

104 εμφανίσεις

Berk G
ö
kberk

Bo
ğaziçi University



Perceptual Intelligence Lab

Turkey

Principles of 3D Face
Recognition

2

Prom
ises and motivations


2D face recognition still requires help


Pose, expression, illumination variations



Possible solutions: other modalities?


Video, infra
-
red, stereo, 3D



Promises of 3D facial recognition


High
-
security applications


3D shape information invariance


Pose and illumination problems can be solved


Better facial feature localization

3

Some scenarios


3D
-
to
-
3D





2D
-
to
-
2D via 3D










2D
-
to
-
3D or 3D
-
to
-
2D

4

How do we get 3D facial data?


Stereo cameras


Quality: low to medium


Speed: fast


Problems: reconstruction


Structured
-
lights


Quality: medium


Speed: fast


Problems: intrusive


Laser scanners


Quality: high


Speed: slow


Problems: intrusive,


Shape from {shading, motion, video}



N/A

5

3D Facial Recognition Pipeline

Features

Point
Clouds

Depth
Images

Pattern
Classifier

3D Face
Detection

Pre
-
proc.

Face Normalization/Alignment

Fine
Alignment

Noise
Removal

Hole Filling

Smoothing

Cropping

Landmark
Finding

Coarse
Alignment

6

3D Face Detection


This problem has not been touched so far!


Simple heuristics such as nose tip


In complex scenes, curvature analysis is generally used


Ref: 3D face detection using curvature analysis, Alessandro Colombo, Claudio Cusano, Raimondo Schettini,

Pattern Recognition 39 (2006) 444


455

7

Pre
-
processing


Artifact removal


Noise removal:
spikes (filters), clutter (manually), noise (median
filter)


Holes
filling
(Gaussian smoothing, linear interpolation, symmetrical
interpolation)



8

Face Normalization/Alignment


Coarse alignment by


C
entre of mass,


P
lane fitted to the data


Facial landmarks (
eyes, nose

tip
)



Fine alignment


ICP


Warping


Elastic deformations

9

3D Facial Features

10

PC

-
ICP gives the distance

-
Hausdorff

-
Too many points

11

PC

SN

-
Enhanced Gaussian Image

-
Too many normals



-
ICP gives the distance

-
Hausdorff

-
Too many points

12

PC

SN

PRO

-
Sparse

-
Easy to compare

-
Not fully descriptive


-
Enhanced Gaussian Image

-
Too many normals



-
ICP gives the distance

-
Hausdorff

-
Too many points

13

PC

SN

PRO

CURV

Mean

Gaussian

Shape Index

Principal
directions

-
Landmark detection

-
Segmentation

-
Sensitive to noise
and the quality of
data



-
Sparse

-
Easy to compare

-
Not fully descriptive


-
Enhanced Gaussian Image

-
Too many normals



-
ICP gives the distance

-
Hausdorff

-
Too many points

14

PC

SN

PRO

CURV

Mean

Gaussian

Shape Index

Principal
directions

DI

PCA

-
Landmark detection

-
Segmentation

-
Sensitive to noise
and the quality of
data



-
Sparse

-
Easy to compare

-
Not fully descriptive


-
Enhanced Gaussian Image

-
Too many normals



-
ICP gives the distance

-
Hausdorff

-
Too many points

-
Benefit from 2D literature

-
Easy to fuse with texture

-
Applicable to 2.5D only



15

PC

SN

PRO

CURV

Mean

Gaussian

Shape Index

Principal
directions

DI

TEX

PCA

PCA

Gabor

-
Landmark detection

-
Segmentation

-
Sensitive to noise
and the quality of
data



-
Sparse

-
Easy to compare

-
Not fully descriptive


-
Enhanced Gaussian Image

-
Too many normals



-
ICP gives the distance

-
Hausdorff

-
Too many points

-
Benefit from 2D literature

-
Easy to fuse with texture

-
Applicable to 2.5D only



16

Baseline algorithms


When you design a new system, which algorithms should
be selected to show your algorithm’s superiority?

PCA

PCA

Texture & Shape

Match score
for texture

Match score
for shape

Use weighted
sum

Perform ICP

Output the surface matching error

Face A

Face B

Baseline Algorithm 1

Baseline Algorithm 2

17

What are the scenarios tested?


The discriminative power of texture and shape?

Taken from: “Three
-
dimensional face recognition”, Bronstein, A.M., and Bronstein, M.M., and Kimmel, R.

International Journal of Computer Vision 2005, Vol.64 No.1 p.5
-
30

18

What are the scenarios tested?


The discriminative power of texture, shape, or
texture+shape?


Pose variations


No disciplined analysis to compare 2D and 3D under pose
variations


Expression variations


Most of the databases do not contain expression variations


No comparison to 2D


Illumination variations


Image relighting


Albedo estimation


19

Open Issues & Challenges


Uncontrolled acquisition


Non
-
cooperative


Different lighting conditions


Texture map + shape map inconsistenies


Real
-
time 3D video data


Computational complexity


Issues related to performance assesment


Publicly available standard face databases


Quality (resolution) of the data?


Artifacts such as eyeglasses



Images taken from: A survey of approaches and challenges in 3D and multi
-
modal 3D + 2D face recognition,
Kevin W. Bowyer, Kyong Chang, Patrick Flynn, Computer Vision and Image Understanding 101 (2006) 1

15

Quick notes on

3D acquisition systems &

3D face databases

21

3D Acquisition Systems


Face specific


Biometrics


A4Vision


Geometrix


Modeling


Cyberware


Genex


Inspeck


Medeim


Breuckmann


General sensors


Minolta V
-
910

22

3D Face Databases


UND


275 subjects, 943 scans


Shape + texture


FRGC


400 subjects, 4007 scans


Shape + texture


3D_RMA


120 subject, 6 scans


Shape only


GavabDB


61 subjects (9 scans)


Shape only


Pose, expression variations


USF database


357 scans


3DFS generator


Custom face databases


12 persons to ~6000 persons (A4Vision)

UND


USF

GavabDB


3DFS

23

Conclusions


3D face recognition systems were proposed to overcome
expression, illumination, and pose challenges


Illumination correction is simpler


Facial landmark localization is better


Baseline recognizers


Combining depth maps with texture channel at the decision level


The core algorithm, ICP, has limited capabilities


Not suitable for non
-
rigid deformations


Lots of research on shape channel representation


Few in combining shape + texture

24

References


B
oğaziçi U
niversity


Perceptual Intelli
gence Lab (PILAB)


Signal and Image Processing Lab (BUSIM)



Relevant Papers:


Comparative analysis of decision
-
fusion methods for 3D face recognition

B. G
ökberk, L. Akarun
, submitted for publication.


Exact 2D
-
3D Facial Landmarking for Registration and Recognition

Salah, A.A., H. Çınar, L. Akarun, B. Sankur, submitted for publication.


3D Shape
-
based Face Representation and Feature Extraction for Face Recognition

B. Gökberk, M. O. İrfanoğlu, L. Akarun, Image and Vision Computing (accepted).


3D Face Recognition by Projection Based Methods

H. Duta
ğ
ac
ı
, B. Sankur, Y. Yemez, in SPIE Conf. on Electronic Imaging, 2006.


2D/3D facial feature extraction

Çınar Akakın, H., A.A. Salah, L. Akarun, B. Sankur, in SPIE Conf. on Electronic Imaging, 2006.



Selection and Extraction of Patch Descriptors for 3D Face Recognition

B. Gökberk, L. Akarun, ISCIS 2005, LNCS, Vol. 3733 Springer 2005, pp. 718
-
727.


3D Face Recognition for Biometric Applications

L. Akarun, B. Gökberk, A. A. Salah, EUSIPCO2005, Antalya, Turkey.


Rank
-
based Decision Fusion for 3D Shape
-
based Face Recognition

B. Gökberk, A. A. Salah, L. Akarun, AVBPA 2005, LNCS, Vol.3546 p.1019
-
1028.


3D Shape
-
based Face Recognition using Automatically Registered Facial Surfaces

M. O. İrfanoğlu, B. Gökberk, L. Akarun, ICPR2004, pp.183
-
186



Contact:


Berk G
ök
berk, e
-
mail: gokberk@boun.edu.tr

Additional Material

26

High Resolution 2D
vs 2D

3D Shape+Texture

3D Texture Only

3D Shape Only

Reference:

Jonathon Phillips,

FRGC Workshop,

CVPR’05

Face Recognition Grand Challenge