下載

1

1

Introduction

Recently,
the mental and physical diseases caused by
negative emotions and stress are increasing year by
year.

The physiological problems of 90 percent are related with mental
factor.


Many emotion recognition techniques have been
proposed.

Facial Expression Recognition
:

Using the relationship between the facial features for facial
expression recognition.

Physiological Emotion Recognition:

Using the physiological signals to recognize emotions.

2

2

Introduction
-

facial expression recognition


Most of automatic facial expression recognition systems
have two characteristics:

Extract features in gray scale images.

Recognize from general expression model.

Expression may be expressed differently by different people
.

Therefore, we proposed a personalized facial expression
recognition system combine with
face recognition system

and
facial expression recognition system
.

Expression

model #1

Expression

model #2

Expression

model #n

.

.

.

Personalized facial expression recognition system

Facial expression

recognition

Face

recognition

Facial feature

extraction

3

3

Introduction
-

physiological signals

Physiological reactions are non
-
autonomic nerves in
physiology. The physiological reactions and the
corresponding signals are hardly to control while
emotions are excited.

The physiological reaction of emotion is generated similarly in
different people.


Therefore, we proposed a emotion recognition system,
combine with support vector regression and
physiological signals
.

4

4

Feature extraction:

To extract features of face image. We extract
two kinds of features: facial feature distances
and facial edge features.

Face recognition:

Use the method of Chang [9] to obtain the
personal identity of user.

Face detection:

Use the method of Chang[8] to extract face
image in original image.

Original image:

Use web camera to catch original image.

Expression recognition:

Use radial
-
basis function neural network to
recognize expression by personalized
information.

Pre
-
processing:

Face image normalization, pupil detection and
Gabor transform are involved.

System overview:
Facial Expression
Recognition

Original Image

Face Detection

Face Recognition

Expression

Recognition

Result

Feature Extraction

Facial feature
distances

Facial edge
features

Pre
-
processing

5

5

Methods
-

Face detection

Face detection (
最佳論文獎

TAAI2006
)

Expression recognition depends on robust face detection and
tracking.

We adopt an adaptive color space switching method proposed by
Chang to detect face image.

It can detect multiple faces and mark face regions automatically under
complex background

environment and
variable lighting condition
.

The algorithm was validated under different human behavior and
environmental variations such as
camera motion
,
background change
,
object motion

and
brightness variation
.

Samples of face detection (a) in color image; (b) in gray scale image; (c) in multiple face image

(a)

(b)

(c)

6

6

A Subject
-
dependent Facial
Expression Recognition System

Extracting significant
facial features is important
in the design and implementation of automatic
facial expression systems

7

7

Methods
-

Feature extraction

Facial feature distances

16 descriptive points were
detected using empirical
information in two color
space (YCbCr and HSV).



17 feature distances are
used to describe face
changes. These distances
are denominated
D
1
,
D
2
,...,
D
17
.

8

8

Methods
-

Feature extraction

Facial edge features

To obtain the edge image, the average Gabor image is
convoluted with a Sobel edge detection mask.

On the edge image, 16 blocks are captured according to the
detected facial feature points and compose the ”
inner face
region
”.



where
B
i
(
x,y
) is the
i
-
th

block’s intensity of (
x,y
).
bw

and
bh

are the width
and height of each block.
Blocks

is the number of blocks, we set as 16.













































Blocks
i
bh
y
bw
x
i
bh
y
bw
x
i
i
Blocks
bw
bh
y
x
B
bw
bh
y
x
B
F
1
0
0
0
0
*
*
,
*
,
(1)

9

9

Methods
-

Face recognition

The personalized expression recognition is achieved by
identifying a user’s face before expression recognition.

Chang’s method [5] was adopted. (
佳作論文獎

TAAI2008
)


However, in Chang’s method, a full face image, such as
the extracted face images shown in Fig. (a), was used.


Since the background and

hair significantly affect

recognition,
we used the

inner face to identify the

user
, as show in Fig. (b).

(a)

(b)

10

10

Methods
-

Face recognition

Challenges in face recognition include
illumination variation
,
pose
variation
,
facial expression
, aging, hair, and glasses.


Contributions of the proposed
face recognition

method

We used Gabor filters to obtain
Gabor faces

that have properties of
scale
normalization

and
grayscale equalization
.

An
AdaBoost committee machine

is used to promote the recognition rate.

The Radial Basis Function Neural Network was adopted as the weak
classifier.

The centers of RBFNN were adaptively selected using PCA.

A novel weight updating mechanism was applied to reduce the training time.

The proposed method has a
high recognition rate

and requires a
short training
time
.


The proposed method can attenuate the influences of

illumination
,
facial
expression
, and
pose variations
.


11

11

Methods
-

Face recognition

Introduction

Variations in illumination and facial
expression


Pose variation

(a)

(c)

(e)

(b)

(d)

(f)

Variations in illumination, pose and facial expression
seriously affect the detection of invariant salient features.

12

12

Methods
-

Face recognition

Introduction

There are two scenarios for face recognition

Pure Face Recognition

Identify or verify a person from a digital image or a video
frame.

智慧數位宅

看臉色開門

(2008/05/01/
蘋果日報

)

Integrated the RFID and face recognition for ID
authentication.

電子投票
(
國立雲林科技大學自由軟體研究中心
)

2007/08/02~2007/08/06
台北世貿電腦應用展

U
化餐廳
(
國立雲林科技大學自由軟體研究中心
)

2008/07/30~ 2008/08/02
台北世貿電腦應用展

智慧生活空間人臉辨識門禁系統

2008/11/29 (
國立雲林科技大學校慶記者會
)

13

13

智慧數位宅

看臉色開門

(2008/05/01/
蘋果日報

)

「
數位住宅的概念讓房子有了智慧，屋主進門不須用鑰匙，
只要人臉感應即可
」

最近建築業者耗資
3000
萬元，在接待中心設立未來概念館，將進階導入一般住宅。

愈來愈多新
建案規劃「數位住宅」，提升未來居住品質，除了可利用手機感應在社區內購物，還可以設定
人員追蹤，從網路上寮解家人的所在位置。昨日遠雄建設（
5522
）聯合
ㄲ
家科技廠商，在林口
「大未來」接待中心，成立「遠雄
2015
未來生活概念館」。


12
科技廠打造概念屋


遠雄企業團
董事長趙藤雄說：「
2015
年的住宅科技可以在生活概念館體驗，未來也會逐步導入
遠雄的建案中。」遠雄副理劉純信表示，未來生活概念館由國內外科技大廠聯手打造，包括三
聯科技、立皓科技、永奕科技、台灣國際松下電工、台灣飛利浦照明事業部、資策會、富陽光
電、傳技資訊、精誠資訊、滿景資訊、德凌資訊、韓國
S
-
呥T
（台灣三星物產代理）等，共展
示
ㄷ
項數位生活設施。有興趣的民眾都可以到「大未來」接待中心感受未來住宅科技。

電影場景在日常實現


遠雄
2015
未來生活概念館可提供的實境有
5
大主題，包括智慧型無人商店，住戶在購物時，用
手機感應就能付款；社區數位信箱提供低溫宅配物品，住戶可以使用手機感應付費領取物品；
住戶進入梯廳時，電腦會自動儲存人臉紀錄，可以不用鑰匙，就
進入大門等等
。這些如同在電影場景的科技，都可以運用在生活中。

遠雄建設在林口「大未來」接待中心，成立未來生活概念館，體驗
2015
年
的住宅科技。




14

14

Methods
-

Face recognition

Block diagram of the proposed method

15

15

Gabor wavelet transform

The Gabor filter transforms an image into a fixed
normalized size

and
eliminates the influence of
illumination variance
.

Gabor filter

Methods
-

Face recognition






2
2
2
s
k
Eight orientations
（
q
=
0,

⼸ⰲ

/8,….,7

/8
）


Five scales
（
s‽ 0ⰱⰠ2Ⱐ3ⰴ
）


















2
)
(
2
)
(
2
2
,
2
2
2
2
2
2
2
)
,
(


q

e
e
e
k
q
p
g
q
p
ik
q
p
k
s
where

s

= 0, 1, 2, 3, 4


q

= 0,

/8, 2

/8, ..., 7

/8

16

16

總結

提出了一套結合
人臉辨識
的個人化人臉表情辨識
系統。

藉由人臉偵測技術由輸入影像中擷取出目標人臉，再
以人臉辨識技術來辨識出使用者的身分

擷取使用者臉部的各個特徵

依人臉辨識的結果與取出的臉部特徵組成特徵向量

使用類神經網路辨識
無表情
、
快樂
、
生氣
、
驚訝
及
悲
傷
等五種表情。

實驗結果顯示提出的方法能夠正確地辨識人臉表情。


17

17

Mel
-
Frequency Cepstrum
Coefficient

The output of the filter bank by
Y
(
m
)
, the
MFCCs are calculated as






where
M

is number of band
-
pass filter,
c(k)

is
the
k
th coefficient of MFCC,
P

is number of
MFCC coefficient,
m

is the index of band
-
pass filters.







P
k
M
m
k
m
Y
k
c
M
m
,...,
2
,
1

,
2
1
cos
log
1



















(22)

18

18

Linear Predictive
Cepstrum Coefficient

In the Linear Predictive Cepstrum (LPC)
analysis of audio each sample is predicted
as linear weighted sum of the past
p

samples, where
p

represents the order of
prediction.




where
S(n)

is the present sample;
A
i

is the
i
th
linear combination coefficient.


The difference between the actual and the
predicted sample value is termed as the
prediction error.

Signal

Frame Segmentation

Auto
-
correlation

Durbin algorithm

LPC coefficient

LPCC coefficient










P
i
i
i
n
S
A
n
S
1
(23)

19

19

Linear Predictive
Cepstrum Coefficient

Using auto
-
correlation approach to reduce
prediction error, and defined as






where
x
(
n
)

is the original signal;
R
(
k
)

is the
auto
-
correlation function;
p

is the order of
prediction.


Using the Durbin algorithm to acquire LPC
coefficients, and using equation (25) to
acquire LPCC coefficients.





where
a
m

is the LPC coefficient;
c
m

is the
LPCC coefficient.







1
,...,
1
,
0

,
1
0







P
k
k
n
x
n
x
k
R
N
n
































otherwise
1
1
1
1
1
1
P
k
k
m
k
m
k
k
m
k
m
m
c
m
k
P
m
c
m
k
c
a
a
a
(24)

(25)

20

20

The sequential floating forward selection
algorithm is utilized to find discriminative
features, and
is a revised algorithm based on
“sub
-
optimal” feature subset selection.

Sequential Floating
Forward Selection

Accuracy

Network