Facial Feature Extraction

parathyroidsanchovyΤεχνίτη Νοημοσύνη και Ρομποτική

17 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

95 εμφανίσεις

Facial Feature Extraction

Yuri Vanzine C490/B657

Computer Vision

Potentially Bigger Project

Future Goal: Application Component



Receive Frontal and Profile Images of User’s Head


Localize face, locate position of eyes, mouth to determine face
geometry



Use face geometry to locate secondary features, eg. nose, facial hair, ears.


Extract face geometry and features and transform them into quantative
feature vectors.


Apply features and geometry to 3
-
D model in a standard 3
-
D model format,
eg. 3DS, OpenNURBS, DXF, DWG, etc.


Need: Provide Ability to import Avatar appearance from web cam

Application needs to perform the following:

Part of the Project for C490/B657

Face Localization, Locating Eyes and Mouth



Face Localization


Mouth Location


Left and Right Eye Location



Process phases:

Face Localization

Existing Techniques:


RGB, YCbCr, HSV Late Fusion


Several weak, fast classifiers instead of
‘expensive’ ones


Environment is controlled: sufficient
lighting, head position, subject is told
to place head a certain way, but our
classifier is rotation, scale and
translation invariant!




Skin
-
tone techniques in various colorspaces


Template
-
based


Eigen
-
faces


Other

Technique used for Face Detection:

RGB, YCbCr, HSV Late Fusion

RGB Colorspace

Matlab Code:


for xc=1:x


for yc = 1:y


iR = (int)R(xc,yc);


iG = (int)G(xc,yc);


iB = B(xc,yc);


mxRGB = max(max(iR, iG), iB);


mnRGB = min(min(iR, iG), iB);


if iR>95 && iG>40 && iB>20 && mxRGB
-
mnRGB>15 && abs(iR
-
iG) > 15 && iR > iG &&
iR > iB




else


F(xc,yc,1)=0;


F(xc,yc,2)=0;


F(xc,yc,3)=0;


end


end

end



HSV Colorspace

Bounding planes, defining skin
-
color, determined in [3]:

Matlab Code:

for xc=1:x


for yc = 1:y


iH = double(H(xc,yc))*255;


iS = double(S(xc,yc))*255;


iV = double(V(xc,yc))*255;


if iH>=0 && iS>=15 && iS>=(0.75*iH+0.3*iV
-
30) || iS<=
-
iH
-
0.1*iV+110 && iH<=
-
0.4*iV+75 && iS<=0.08 * (100
-
iV) *iH +0.6* iV


else


F(xc,yc,1)=0;


F(xc,yc,2)=0;


F(xc,yc,3)=0;


end


end

end


YCbCr Colorspace

“YCbCr simple” from [3] (could not get it to work):

Late Fusion of Colorspaces

RGB

HSV

YCbCr

To be implemented…

Bounding Box

RGB and HSV in this implementation is luminance
-
dependent.

Frequent false positives are likely.

This implementation should be replaced by YCbCr from [4].

More examples of Bounded Images

Mouth Location (YCbCr)


See [4] for further detail.


Mouth Location (continued)

Mouth Mask

Convex Face

Bounding Box

Eye Location (YCbCr)

Eye Map

Eye Location (YCbCr)

Complexity Map

5

6

5

7

150

6

7

150

6

Average of Differences

between pixels in the

8
-
neighborhood,

followed by dilation

using a disk of 5 pixels

Eye Location (YCbCr)

Eye Map and Complexity Map Combined

AND


Notice problems
because bounding box

needs to be replaced
by Connected
Component Mask

AND


Matlab Demo of Bounding Box, EyeMap and MouthMap

Current Problems and Future Work



Problem:

Current skin
-
tone clustering is not robust.



Solution:

YCbCr luma
-
independent detection needs to be implemented.




Problem:

Manual thresholding is implemented.



Solution:
Automatic thresholding is desired.




Problem:
Bounding Box instead of Skin
-
Tone Connected component



Solution:

Improve algorithm extracting just face
-
mask



YCbCr Colorspace (continued)

YCbCr Colorspace (continued)

This model alone is sufficient. Taken from [4] (to be implemented in the future).
YCbCr is luminance
-
dependent. Direct use of YCbCr results in false positives.
Therefore YCbCr must be made luma
-
independent.

YCbCr Colorspace (continued)

YCbCr Colorspace (continued)

References



1.
A Survey on Pixel
-
Based Skin Color Detection Techniques, Vladimir Vezhnevets,

Vassili Sazonov,
Alla Andreeva, http://www.graphicon.ru/2003/Proceedings/Technical/paper509.pdf

2.
Sanjay Kr. Singh, D. S. Chauhan, Mayank Vatsa, Richa Singh, A Robust Skin Color Based Face
Detection Algorithm,
http://www2.tku.edu.tw/~tkjse/6
-
4/6
-
4
-
6.pdf

3.
Face Detection in Color Images using Wavelet Packet Analysis, C. Garcia, G. Zikos, G. Tziritas,
http://www.csd.uch.gr/~tziritas/papers/ICMCS
-
99.pdf

4.

Face Detection in Color Images, Rein
-
Lien Hsu, Mohamed Abdel
-
Mottaleb, Anil K. Jain,
http://www.csd.uch.gr/~hy471/papers/FaceDetectionColor.pdf