A Neural Network Based Cascaded Classifier for Face ... - PRIP

cracklegulleyΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 9 μήνες)

73 εμφανίσεις

Project 10

Facial Emotion Recognition

Based On Mouth Analysis

SSIP 08, Vienna

1

http://www.we
-
hope
-
project10
-
will
-
win.info

The Project


Objective : To recognize emotional state / expression
using mouth information



Input: Mouth images (no make
-
up)



Output: Emotional State/ Expression


Happy, Neutral, Sad



2

http://www.we
-
hope
-
project10
-
will
-
win.info

The Team

3

Kornél

programmer

Péter

Web

programmer


Kamal

programmer

Naiem

researcher

Sofia

programmer

http://www.we
-
hope
-
project10
-
will
-
win.info

The Tasks


Create facial expressions photographic database



Segment the mouth in the input image



Use suitable features for expression characterization



Design a reliable classifier to distinguish between
different mouth expressions




4

http://www.we
-
hope
-
project10
-
will
-
win.info

SSIP Lips database


Happy, Neutral and Sad Photos of SSIP students and
lecturers







Thank you all!!!



Happy



Neutral



Sad

5

http://www.we
-
hope
-
project10
-
will
-
win.info

Mouth Segmentation




6

Input Image

HSV Space
-

Hue

Thresholding

Morphological
Operations

http://www.we
-
hope
-
project10
-
will
-
win.info

Segmentation Results…

And Segmentation Problems…

7

http://www.we
-
hope
-
project10
-
will
-
win.info

Lips Features Extraction



Detect the leftmost and rightmost lip points



Normalize images (rotation, translation and scaling)



Calculate features


Eccentricity


Convex Area


Minor Axis


Ratio of Upper to Lower Lip

8

http://www.we
-
hope
-
project10
-
will
-
win.info

Expression Classification


SVM Classifier



Two Stage Classification




Mouth
Features





9

http://www.we
-
hope
-
project10
-
will
-
win.info

Results 1


Differences between different classes were found
to be statistically significant (p<0.01)



Classification Accuracy


Stage 1 (Sad / Not Sad)


88%


Stage 2 (Happy/ Neutral)

62%


10

http://www.we
-
hope
-
project10
-
will
-
win.info

Results 2

11

http://www.we
-
hope
-
project10
-
will
-
win.info

Future Work


Acquire larger database for training and testing



Test different facial expressions (such as anger and
disgust)


Other classifiers: NN, FIS

Conclusion


Mouth information is often insufficient for recognizing
facial expression / emotional state



Other face features such as eyes and eyebrows can
contribute in emotional state recognition

12

http://www.we
-
hope
-
project10
-
will
-
win.info

GUI

13

http://www.we
-
hope
-
project10
-
will
-
win.info

References


M. Gordan, C. Kotropoulos, I. Pitas, “
Pseudoautomatic Lip Contour Detection Based on Edge
Direction Patterns”


J. Kim, S. Na, R. Cole, “
Lip Detection Using Confidence
-
Based Adaptive Thresholding”


F. Tang, “
Facial Expression Recognition using AAM and Local Facial Features”


M. Pantic, M. Tomc, L. Rothkrantz , “
A Hybrid Approcah to Mouth Features Detection


14

http://www.we
-
hope
-
project10
-
will
-
win.info


Thank you for your attention!!!


15

http://www.we
-
hope
-
project10
-
will
-
win.info