# 以區域二元圖樣與部分比對為基礎之人臉辨識Face Recognition with ...

AI and Robotics

Nov 17, 2013 (4 years and 5 months ago)

102 views

Face Recognition with Local Binary
Patterns
and

Partial Matching

Presenter
:

:

1

Outlines

Motivation

Implementation

Result

Conclusion

2

MOTIVATION

3

Motivation

4

Publication

Che
-
Hua Yeh, Pei
-
Ruu Shih, Kuan
-
Ting Liu, Yin
-
Tzu
Lin, Huang
-
Ming Chang, Ming Ouhyoung.
A
Comparison of Three Methods of Face
Recognition for Home Photos
.
ACM Siggraph,
poster, 2009.

5

Problem Statement

6

Main Contribution

Improve Local Binary Patterns by using Partial
Matching Metric

Better Performance in Home Photos

7

IMPLEMENTATION

8

System Overview

Pre
-
Processing

Build Descriptor

Images

Descriptors

Clustering

Calculate
LBP

Build
Descriptor

9

System Overview

Pre
-
Processing

Build Descriptor

Images

Descriptors

Clustering

Calculate
LBP

Build
Descriptor

10

Pre
-
Processing

11

System Overview

Pre
-
Processing

Build Descriptor

Images

Descriptors

Clustering

Calculate
LBP

Build
Descriptor

12

System Overview

Pre
-
Processing

Build Descriptor

Images

Descriptors

Clustering

Calculate
LBP

Build
Descriptor

13

Local Binary Patterns
[PAMI2006]

An operator to encode the relationship of a
pixel and its neighbors.

Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

58

99

21

54

54

86

67

12

13

1

otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
14

Local Binary Patterns
[PAMI2006]

An operator to encode the relationship of a
pixel and its neighbors.

58

99

21

54

54

86

67

12

13

1

1

otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

15

Local Binary Patterns
[PAMI2006]

An operator to encode the relationship of a
pixel and its neighbors.

58

99

21

54

54

86

67

12

13

1

1

0

otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

16

Local Binary Patterns
[PAMI2006]

An operator to encode the relationship of a
pixel and its neighbors.

58

99

21

54

54

86

67

12

13

1

1

0

1

otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

17

Local Binary Patterns
[PAMI2006]

An operator to encode the relationship of a
pixel and its neighbors.

58

99

21

54

54

86

67

12

13

1

1

0

1

0

otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

18

Local Binary Patterns
[PAMI2006]

An operator to encode the relationship of a
pixel and its neighbors.

58

99

21

54

54

86

67

12

13

1

1

0

1

0

0

otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

19

Local Binary Patterns
[PAMI2006]

An operator to encode the relationship of a
pixel and its neighbors.

58

99

21

54

54

86

67

12

13

1

1

0

1

1

0

0

otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

20

Local Binary Patterns
[PAMI2006]

An operator to encode the relationship of a
pixel and its neighbors.

58

99

21

54

54

86

67

12

13

1

1

0

1

1

1

0

0

otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

21

Local Binary Patterns
[PAMI2006]

An operator to encode the relationship of a
pixel and its neighbors.

58

99

21

54

54

86

67

12

13

1

1

0

1

1

1

0

0

otherwise

,
0
0

if

,
1
2
8
1
0
x
x
s
Z
Z
s
LBP
p
p
p
Z
1

Z
2

Z
3

Z
8

Z
0

Z
4

Z
7

Z
6

Z
5

LBP

= 11010011

22

System Overview

Prepared
-
Works

Build Descriptor

Images

Descriptors

Clustering

Calculate
LBP

Build
Descriptor

23

Facial Image Descriptor

They use
Spatially Enhanced Histogram
in
original Local Binary Pattern.
[PAMI2006]

24

Local Patches

We sample a

patch for every
s

pixels.

There are
S

patches for one image.

m
m

s

s

25

Spatial Block
[CVPR2007]

We use three concentric circles to describe a
patch.

26

Descriptor

Build a descriptor for one face.

27

KK
K
K
K
K
f
f
f
f
f
f
f
f
f

2
1
2
22
21
1
12
11
System Overview

Pre
-
Processing

Build Descriptor

Images

Descriptors

Clustering

Calculate
LBP

Build
Descriptor

28

System Overview

Pre
-
Processing

Build Descriptor

Images

Descriptors

Complete
-

Calculate
LBP

Build
Descriptor

Compute all
the similarities

Clustering

29

System Overview

Pre
-
Processing

Build Descriptor

Images

Descriptors

Complete
-

Calculate
LBP

Build
Descriptor

Compute all
the similarities

Clustering

30

Similarity

They use the weighted Chi
-
Square distance in
original Local Binary Pattern.
[PAMI2006]

j
ij
ij
ij
ij
i
i
w
M
S
M
S
w
M
S
2
2
,

31

Partial Matching
[ICCV2009]

Step1:

Compute the similarity of each patch from one
image with the nearby patches in another image.

,

Image 1:
I
(1)

Image 2:
I
(2)

)
2
(
)
1
(
|
|
;
|
:|
,
)
1
(
,
min
kl
ij
r
s
l
s
j
r
s
k
s
i
l
k
ij
f
f
sim
f
d

32

Partial Matching
[ICCV2009]

Step2:

Sort the similarities of all patches.

d
α
S

is the similarity of
I
(1)

to
I
(2)
.

K
j
i
ij
s
S
f
d
Sort
d
d
d
d
1
,
)
1
(
2
1
,...,
,...,
,

S
d
I
I
d

)
(
)
2
(
)
1
(
33

Partial Matching
[ICCV2009]

Step3:

Calculate the similarity of
I
(2)
to
I
(1)

)
(
)
1
(
)
2
(
I
I
d

34

Partial Matching
[ICCV2009]

Step4:

Use the maximum of two similarity

)
(
),
(
max
)
,
(
)
1
(
)
2
(
)
2
(
)
1
(
)
2
(
)
1
(
I
I
d
I
I
d
I
I
D

35

System Overview

Pre
-
Processing

Build Descriptor

Images

Descriptors

Complete
-

Calculate
LBP

Build
Descriptor

Compute all
the similarities

Clustering

36

Hierarchical Clustering

Build a tree
-
based hierarchical taxonomy
(
dendrogram) from a set of documents.

Material Selected from
Introduction to
Information Retrieval
,
Christopher D.
Manning, Prabhakar Raghavan and
Hinrich Schütze, 2008.

37

Hierarchical Clustering

Clustering obtained by cutting the dendrogram
at a desired level: each connected connected
component forms a cluster.

Material Selected from
Introduction to
Information Retrieval
,
Christopher D.
Manning, Prabhakar Raghavan and
Hinrich Schütze, 2008.

38

Hierarchical Complete
-
Clustering

Similarity of the “furthest” points.

Makes “tighter,” spherical clusters that are
typically preferable.

Material Selected from
Introduction to
Information Retrieval
,
Christopher D.
Manning, Prabhakar Raghavan and
Hinrich Schütze, 2008.

39

Performance Optimization

-
Core system

Pre
-
Processing

Build Descriptor

Images

Descriptors

Complete
-

Calculate
LBP

Build
Descriptor

Compute all
the similarities

Clustering

40

Performance Optimization

-
Core system

3 times faster than single thread.

73 minutes to 24 minute for 309 images.

Pre
-
Processing

Build Descriptor

Images

Descriptors

Complete
-

Calculate
LBP

Build
Descriptor

Compute all
the similarities

Clustering

41

RESULT

42

FERET Result

fa
:

gallery, 994 images

fb
: alternative facial expression, 992 images

dup1: the photos taken after later, 736 images

dup2: the photos taken at least one year after
the gallery, 228 images

43

FERET Result

44

Accuracy

fb

dup1

dup2

LBP
[PAMI2006]

95.67%

59.92%

45.61%

Our Result

98.89%

71.33%

68.42%

Time

Register

fb

dup1

dup2

LBP
[PAMI2006]

46

seconds

102 seconds

67 seconds

21

seconds

Our Result

4

minutes

224

minutes

168

minutes

5

minutes

The time results are computed in multithreads version.

Experiments

Home Photo Dataset I

309 images, 5 subjects

Home Photo Dataset II

838 images, 8 subjects

45

Evaluation

Cluster Number

Unknown Number

Pair
-
wise Precision

Pair
-
wise Rand Index

Executing time

46

Unknown Number

The number of clusters which contain only one
component.

47

Precision/Rand Index

fp
tp
tp
precision

fn
tp
tp
recall

tn
fp
fn
tp
tn
tp
index
rand

48

Assigned

Same
Cluster

Different
Clusters

Ground

Truth

Same

Cluster

tp

fn

Different
Clusters

fp

tn

Dataset
-
I Result

#Clusters

#Unknown

Precision

Rand
Index

Time

Picasa Web

94

73

99.92%

0.829816

10
seconds

Picasa PC

99

75

100%

0.865002

3
minutes

LBP
[PAMI2006]

100

31

90.39%

0.811856

4
seconds

LID_PM
[ICCV2009]

99

37

98.78%

-

11 minutes

Our

Result

100

39

99.46%

0.816290

24
minutes

LID+PM
[Chang2010]

100

43

99.24%

-

25 minutes

LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block

The time results are computed in multithreads version.

49

Dataset
-
I Result

50

LBP

Our Result

Dataset
-
I Result

Wrong clustering result in LBP

The clusters in our result

51

Dataset
-
II Result

#Clusters

#Unknown

Precision

Rand
Index

Time

Picasa Web

195

47

99.49%

0.876229

10
seconds

Picasa PC

253

150

100%

0.888492

10
minutes

LBP
[PAMI2006]

253

45

91.70%

0.70261

17
seconds

LID_PM

[ICCV2009]

253

79

97.88%

-

51 minutes

Our Result

253

64

99.59%

0.871187

163 minutes

LID+PM
[Chang2010]

253

89

99.82%

-

136 minutes

LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block

The time results are computed in multithreads version.

52

Dataset
-
II Result

53

LBP

Our Result

Dataset
-
II Result

Wrong clustering result in LBP

Clusters in our result

54

Demo

Face Recognition with web camera

55

CONCLUSION

56

Conclusion

LBP is an efficient algorithm for face
recognition.

Partial Matching is good for the different facial
expression or different illumination in facial
images.

Our system has better performance than LBP.

57

Future Works

Improve performance by GPU.

Use other extension of LBP combined with
Partial Matching.

58

Thank You for Your
Attention!

59

Result

LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block

0.7
0.75
0.8
0.85
0.9
0.95
1
0
0.05
0.1
0.15
0.2
Precision

Recall

Home Photo Dataset I

LBP
LBP+SB
LBP+PM
LBP+LID+SB
0.7
0.75
0.8
0.85
0.9
0.95
1
0
0.05
0.1
0.15
0.2
Precision

Recall

Home Photo Dataset II

LBP
LBP+SB
LBP+PM
LBP+SB+PM
60