以區域二元圖樣與部分比對為基礎之人臉辨識Face Recognition with ...

paraderollAI and Robotics

Nov 17, 2013 (3 years and 7 months ago)

75 views

以區域二元圖樣與部分比對為基礎之
人臉辨識

Face Recognition with Local Binary
Patterns
and

Partial Matching

Presenter
:
施佩汝

Advisor
:
歐陽明

教授

1

Outlines


Motivation


Implementation


Result


Conclusion

2

MOTIVATION

3

Motivation


4

Publication


Che
-
Hua Yeh, Pei
-
Ruu Shih, Kuan
-
Ting Liu, Yin
-
Tzu
Lin, Huang
-
Ming Chang, Ming Ouhyoung.
A
Comparison of Three Methods of Face
Recognition for Home Photos
.
ACM Siggraph,
poster, 2009.

5

Problem Statement


6

Main Contribution


Improve Local Binary Patterns by using Partial
Matching Metric



Better Performance in Home Photos


7

IMPLEMENTATION

8

System Overview


Pre
-
Processing

Build Descriptor

Images

Descriptors

Clustering

Calculate
LBP

Build
Descriptor

9

System Overview


Pre
-
Processing

Build Descriptor

Images

Descriptors

Clustering

Calculate
LBP

Build
Descriptor

10

Pre
-
Processing


11

System Overview


Pre
-
Processing

Build Descriptor

Images

Descriptors

Clustering

Calculate
LBP

Build
Descriptor

12

System Overview


Pre
-
Processing

Build Descriptor

Images

Descriptors

Clustering

Calculate
LBP

Build
Descriptor

13

Local Binary Patterns
[PAMI2006]


An operator to encode the relationship of a
pixel and its neighbors.




Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

58

99

21

54

54

86

67

12

13

1















otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
14

Local Binary Patterns
[PAMI2006]


An operator to encode the relationship of a
pixel and its neighbors.




58

99

21

54

54

86

67

12

13

1

1















otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

15

Local Binary Patterns
[PAMI2006]


An operator to encode the relationship of a
pixel and its neighbors.




58

99

21

54

54

86

67

12

13

1

1

0















otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

16

Local Binary Patterns
[PAMI2006]


An operator to encode the relationship of a
pixel and its neighbors.




58

99

21

54

54

86

67

12

13

1

1

0

1















otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

17

Local Binary Patterns
[PAMI2006]


An operator to encode the relationship of a
pixel and its neighbors.




58

99

21

54

54

86

67

12

13

1

1

0

1

0















otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

18

Local Binary Patterns
[PAMI2006]


An operator to encode the relationship of a
pixel and its neighbors.




58

99

21

54

54

86

67

12

13

1

1

0

1

0

0















otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

19

Local Binary Patterns
[PAMI2006]


An operator to encode the relationship of a
pixel and its neighbors.




58

99

21

54

54

86

67

12

13

1

1

0

1

1

0

0















otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

20

Local Binary Patterns
[PAMI2006]


An operator to encode the relationship of a
pixel and its neighbors.




58

99

21

54

54

86

67

12

13

1

1

0

1

1

1

0

0















otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

21

Local Binary Patterns
[PAMI2006]


An operator to encode the relationship of a
pixel and its neighbors.




58

99

21

54

54

86

67

12

13

1

1

0

1

1

1

0

0















otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

LBP

= 11010011

22

System Overview


Prepared
-
Works

Build Descriptor

Images

Descriptors

Clustering

Calculate
LBP

Build
Descriptor

23

Facial Image Descriptor


They use
Spatially Enhanced Histogram
in
original Local Binary Pattern.
[PAMI2006]

24

Local Patches


We sample a


patch for every
s

pixels.


There are
S

patches for one image.


m
m

s

s

25

Spatial Block
[CVPR2007]


We use three concentric circles to describe a
patch.

26

Descriptor


Build a descriptor for one face.

27















KK
K
K
K
K
f
f
f
f
f
f
f
f
f







2
1
2
22
21
1
12
11
System Overview


Pre
-
Processing

Build Descriptor

Images

Descriptors

Clustering

Calculate
LBP

Build
Descriptor

28

System Overview


Pre
-
Processing

Build Descriptor

Images

Descriptors

Complete
-
Linkage Clustering

Calculate
LBP

Build
Descriptor

Compute all
the similarities

Clustering

29

System Overview


Pre
-
Processing

Build Descriptor

Images

Descriptors

Complete
-
Linkage Clustering

Calculate
LBP

Build
Descriptor

Compute all
the similarities

Clustering

30

Similarity


They use the weighted Chi
-
Square distance in
original Local Binary Pattern.
[PAMI2006]













j
ij
ij
ij
ij
i
i
w
M
S
M
S
w
M
S
2
2
,

31

Partial Matching
[ICCV2009]


Step1:


Compute the similarity of each patch from one
image with the nearby patches in another image.




,


Image 1:
I
(1)

Image 2:
I
(2)





)
2
(
)
1
(
|
|
;
|
:|
,
)
1
(
,
min
kl
ij
r
s
l
s
j
r
s
k
s
i
l
k
ij
f
f
sim
f
d









32

Partial Matching
[ICCV2009]


Step2:


Sort the similarities of all patches.



d
α
S

is the similarity of
I
(1)

to
I
(2)
.










K
j
i
ij
s
S
f
d
Sort
d
d
d
d
1
,
)
1
(
2
1
,...,
,...,
,



S
d
I
I
d



)
(
)
2
(
)
1
(
33

Partial Matching
[ICCV2009]


Step3:


Calculate the similarity of
I
(2)
to
I
(1)

)
(
)
1
(
)
2
(
I
I
d

34

Partial Matching
[ICCV2009]


Step4:


Use the maximum of two similarity



)
(
),
(
max
)
,
(
)
1
(
)
2
(
)
2
(
)
1
(
)
2
(
)
1
(
I
I
d
I
I
d
I
I
D



35

System Overview


Pre
-
Processing

Build Descriptor

Images

Descriptors

Complete
-
Linkage Clustering

Calculate
LBP

Build
Descriptor

Compute all
the similarities

Clustering

36

Hierarchical Clustering


Build a tree
-
based hierarchical taxonomy
(
dendrogram) from a set of documents.



Material Selected from
Introduction to
Information Retrieval
,
Christopher D.
Manning, Prabhakar Raghavan and
Hinrich Schütze, 2008.

37

Hierarchical Clustering


Clustering obtained by cutting the dendrogram
at a desired level: each connected connected
component forms a cluster.



Material Selected from
Introduction to
Information Retrieval
,
Christopher D.
Manning, Prabhakar Raghavan and
Hinrich Schütze, 2008.

38

Hierarchical Complete
-
Linkage
Clustering


Similarity of the “furthest” points.


Makes “tighter,” spherical clusters that are
typically preferable.





Material Selected from
Introduction to
Information Retrieval
,
Christopher D.
Manning, Prabhakar Raghavan and
Hinrich Schütze, 2008.

39

Performance Optimization


4 threads in Quad
-
Core system


Pre
-
Processing

Build Descriptor

Images

Descriptors

Complete
-
Link Clustering

Calculate
LBP

Build
Descriptor

Compute all
the similarities

Clustering

40

Performance Optimization


4 threads in Quad
-
Core system


3 times faster than single thread.


73 minutes to 24 minute for 309 images.


Pre
-
Processing

Build Descriptor

Images

Descriptors

Complete
-
Link Clustering

Calculate
LBP

Build
Descriptor

Compute all
the similarities

Clustering

41

RESULT

42

FERET Result


fa
:

gallery, 994 images


fb
: alternative facial expression, 992 images


dup1: the photos taken after later, 736 images


dup2: the photos taken at least one year after
the gallery, 228 images

43

FERET Result

44

Accuracy

fb

dup1

dup2

LBP
[PAMI2006]

95.67%

59.92%

45.61%

Our Result

98.89%

71.33%

68.42%

Time


Register

fb

dup1

dup2

LBP
[PAMI2006]

46

seconds

102 seconds

67 seconds

21

seconds

Our Result

4

minutes

224

minutes

168

minutes

5

minutes



The time results are computed in multithreads version.

Experiments


Home Photo Dataset I


309 images, 5 subjects




Home Photo Dataset II


838 images, 8 subjects


45

Evaluation


Cluster Number


Unknown Number


Pair
-
wise Precision


Pair
-
wise Rand Index


Executing time

46

Unknown Number


The number of clusters which contain only one
component.

47

Precision/Rand Index

fp
tp
tp
precision


fn
tp
tp
recall


tn
fp
fn
tp
tn
tp
index
rand






48

Assigned

Same
Cluster

Different
Clusters

Ground

Truth

Same

Cluster

tp

fn

Different
Clusters

fp

tn

Dataset
-
I Result

#Clusters

#Unknown

Precision

Rand
Index

Time


Picasa Web

94

73

99.92%

0.829816

10
seconds

Picasa PC

99

75

100%

0.865002

3
minutes

LBP
[PAMI2006]

100

31

90.39%

0.811856

4
seconds

LID_PM
[ICCV2009]

99

37

98.78%

-

11 minutes

Our

Result

100

39

99.46%

0.816290

24
minutes

LID+PM
[Chang2010]

100

43

99.24%

-

25 minutes

LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block



The time results are computed in multithreads version.

49

Dataset
-
I Result


50

LBP

Our Result

Dataset
-
I Result


Wrong clustering result in LBP





The clusters in our result

51

Dataset
-
II Result

#Clusters

#Unknown

Precision

Rand
Index

Time



Picasa Web

195

47

99.49%

0.876229

10
seconds

Picasa PC

253

150

100%

0.888492

10
minutes

LBP
[PAMI2006]

253

45

91.70%

0.70261

17
seconds

LID_PM

[ICCV2009]

253

79

97.88%

-

51 minutes

Our Result

253

64

99.59%

0.871187

163 minutes

LID+PM
[Chang2010]

253

89

99.82%

-

136 minutes

LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block



The time results are computed in multithreads version.

52

Dataset
-
II Result


53

LBP

Our Result

Dataset
-
II Result


Wrong clustering result in LBP





Clusters in our result

54

Demo


Face Recognition with web camera

55

CONCLUSION

56

Conclusion


LBP is an efficient algorithm for face
recognition.



Partial Matching is good for the different facial
expression or different illumination in facial
images.



Our system has better performance than LBP.


57

Future Works


Improve performance by GPU.



Use other extension of LBP combined with
Partial Matching.

58

Thank You for Your
Attention!

59

Result

LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block

0.7
0.75
0.8
0.85
0.9
0.95
1
0
0.05
0.1
0.15
0.2
Precision

Recall

Home Photo Dataset I

LBP
LBP+SB
LBP+PM
LBP+LID+SB
0.7
0.75
0.8
0.85
0.9
0.95
1
0
0.05
0.1
0.15
0.2
Precision

Recall

Home Photo Dataset II

LBP
LBP+SB
LBP+PM
LBP+SB+PM
60