# Computer Vision (CS558) Final Project Report Eye Detection

Τεχνίτη Νοημοσύνη και Ρομποτική

18 Οκτ 2013 (πριν από 4 χρόνια και 8 μήνες)

62 εμφανίσεις

Computer Vision (CS558) Final Project Report

Eye Detection

Student Name: Liefei (Lucy) Xu

This report is organized as below:

1.

Project Description

2.

Algorithm

3.

Matlab Result

4.

Discussion

5.

How to run my code

1.
Project Description:

This project is to detect eyes

in given image
s
. There are 2 parts in this project.
Part 1 is

training an
d testing detector.
Part 2 is

us
ing

the detector to
detect eyes in given image
s
.

1196 left eye images, 1196 right eye images and 3839 non
-
eye images are provide
d
. We separate each ki
nd of
images into 2 sets. The 1
st

set is used to train the detector while the 2
nd

set is used to test the detector.

We will
detect eyes in
14
provided ppm images
.

2. Algorithm

Part1
:

1)

Divide the eye images and the non
-
eye images up into 2 halves of eq
ual size.

2)

Use the 1
st

set of eye/non
-
eye images as the training
set;

come up with an eye detector

2.1 Use matlab function prestd() to subtract out the mean brightness and rescale all images.

2.2 Use function runpca() to do the principle components
analysis, select the first 8
components(or use matlab function prepca() )

2.3 Plot the 1
st

and 2
nd

principle components of eye/non
-
eye set to get a sense about the
distribution

2.4 Calculate the Euclidian distance of the principle components of each eye
to the training eye
set and each non
-
eye to the training eye set.
Assign different weights to different components.
Usually the 1
st

component gets the biggest weight.
The distance is the measurement I selected.
(It’s different with the method suggested by
the professor. I will talk about it more in the
discussion section.)

2.5 Plot the measurement and select a threshold. Here comes my detector.

3)

Use the 2
nd

set to test the eye detector

3.1 Use prestd() to subtract out the mean brightness and rescale al
l images.

3.2 Use the eigenvectors got in 2.2 to get the values of principle components.

3.
3

Calculate the Euclidian distance of the principle components
of each

eye to the training eye
set and each non
-
eye to the training eye set.

Same as 2.4.

3.4 Calc
ulate and plot the missed eye rate and the false positive rate for different selection of the
detector. Select a best detector for part2.

Part2
:

1)

2)

Use function face() to segment the skin area.

3)

On different scales
(use matlab function imres
ize())
, on the skin area,
create a window, calculate the
distance to the training set. And slide the window pixel by pixel on the skin area. If the distance is
lower than the detector, record it as a candidate eye.

4)

On all the candidate eyes we found, selec
t the minimum one as the location of one eye.

5)

Search the other eye in an appropriate distance. If not found 2 eyes on this scale, search on next scale.

6)

If found 2 eyes on a scale, draw a rectangle on each eye. Otherwise, report ‘Did not find eyes’.

3
.
M
atlab R
esult:

-

Result of part1:

Figure1 plots the 1
st

and 2
nd

principle components value of right eyes
vs.

non
-
eyes. Blue dots mean eyes, red dots
non
-
eyes. We could see that by the 1
st

principle component we could separate the eyes and non
-
eyes well.

An
observation is that sometimes the figure is reversed, i.e. the blue part is at the right side of the red part with positive
values.

Figure
1
.

the 1
st

and 2
nd

principle components value, right eyes
vs.

non
-
eyes

Figure2 plots the
1
st

and 2
nd

principle comp
onents value of left eyes
vs.

non
-
eyes. Blue dots mean eyes, red dots
non
-
eyes. We could see that by the 1
st

principle component we could separate the eyes and non
-
eyes well.

Same as
above, sometimes the figure is reversed, i.e. the blue part is at the lef
t side of the red part with negative values.

Figure2.
the 1
st

and 2
nd

principle components value, left eyes
vs.

non
-
eyes

Figure
3

plots the measurements of eyes and non
-
eyes
.
X axis is the index of the training eye image in the training
set. Y axis is t
he measured value. Blue dots mean

eyes.
Red dots mean

non
-
eyes.

We could see it clearly that the
most majority of eye
-
dots are below 0.25e5 while most non
-
eye
-
dots are above 0.25e5. Here comes the detector.

Figure3
.
Result

of training

Figu
re4

shows the

change of missed eyes rate along with the change of the detector, i.e. if we set the detector to
different value of measurement how missed eyes rate changes. We could see that if we set the detector to 0.1e5, all
eyes in the testing set will be missed. If

we set the detector to 0.3e5, 90% of the eyes in the testing set will be
detected. And if we set the detector to 2e5, all eyes in the testing set will be detected.

Figure
4
.
Miss

rate

Figure5

shows the false positive rate on the non
-
eye training set. We

could see that if we set detector to 0.1e5, none
non
-
eye

image

will be detected as eye.
If we set detector to 0.3e5, less than 10% non
-
eye image will be detected as
eye. And if we set the detector to 3e5, all the non
-
eye image
s

will be detected as eye.

Figure5
.
False

positive rate

Figure6

shows the tradeoff: setting the parameter to miss fewer eyes also means that you’ll incorrectly identify
more non
-
eyes as eyes. We still set the detector from 0.1e5 to 3.5e5, step 0.1e5. We could see that the false
pos
itive rate arises along with the detection rate of eyes.

vs.

false positive rate

-

Result of part2:

Detected eyes in image
fDec0010.ppm (
Succeeded)
:

Detected eyes in image
fDec0060.ppm (
Succeeded):

Detected eyes i
n
fDec0110.ppm (
Succeeded):

Detected eyes in
fDec0160.ppm (
Succeeded):

Detected eyes in
fDec0210.ppm (
Failed because the eyes are covered by the hand):

Detected eyes in
fDec0260.ppm (
Succeeded):

Detected eyes in
fDec0310.ppm (
Succeeded):

Detecte
d eyes in
fDec0360.ppm (
Half succeeded. Missed the right eye because the eye is not clear.)
:

Detected eyes in
fDec0410.ppm (
Succeeded):

Detected eyes in
fDec0460.ppm (
Succeeded):

Detected eyes in
fDec0510.ppm (
Succeeded):

Detected eyes in
fDec0560
.ppm (
Succeeded):

Detected eyes in
fDec0610.ppm (
Succeeded):

Detected eyes in
fDec0660.ppm (
Succeeded):

4. Discussion

1)

Randomly dividing eye images.

To divide the eye and non
-
eye images into 2 halves of equal size. I have tried 3 ways.

The f
irst

method, generating

a random number between 1
-
1
196

(the size of the image set). Check if the random
number is already in the training set. If yes, discard it and generate a new number. If not, add the number into the
set. When the training set gets to bigge
r, the confliction occurs more often. So it takes some time to generate a real
random set. Sometimes it needs about 10 seconds.

The second method, assigning the image from 1 to 598 to the training set, and the remaining part to the testing set.
This is the

quickest method. Almost no wait.

The third method, selecting the first 598 good images, i.e. if the image is marked as good image in regEyes, then
add it into the training set, otherwise discard it. In this way, all the images in training set are good ima
ges.

The result of the third method doesn’t have obvious improvement than the second one.

2) PCA

http://visl.technion.ac.il/projects/2003w25/matlab/eeglab/runpca.m
. I also tried the matlab function prepca(), there
is no big difference between this 2.

3) How to measure

There are several ways to design the measurements. I have
tried the way the professor suggested, i.e. use as one
parameter the average error between the original image and its approximation by the principle components, another
parameter be the size of the 1
st

principle component. It could get good result on train
ing and testing set, but when it
comes to the ppm images, it just could not find the correct location of eyes.

So I used the method in this link:
http://people.cs.uchicago.edu/~qingj/ThesisHt
ml/
, i.e. calculating the Euclidian
distance of the principle components between the image and the training set.
Theoretically, this will be a little bit
slower then the above method, but it can find the correct location of eyes. So I selected this one.

4) Different weight to principle components

I tried to assign equal weights, different weights to the principle components when calculating the distance. It turns
out that the result is better when
assigning

more weight to the 1
st

component.

5
) In part2,
I first find the face area then detect eyes in this area. This could speed up the program. The function
face() and skin() are d
htt
.

6
) Find the best eye location

I tried 2 methods. The first method is as soon as I detected a pair of eyes I stop the algorithm and mark them out.
In this way, t
he algorithm is quick. But the problem is that there are many candidate eyes in the image, the first is
not the best match.

I used the second method: search all around the face, record the candidate eyes, then select the minimum value to
be the eye locatio
n. This is slower than the first method but the location is more accurate.

7) The result

The detection result is that in 14 ppm images, it correctly detected 12 pair of eyes. It missed both eyes in
fDec0210.ppm

because the eyes are covered by the hand. An
d it missed the right eye in fDec0360
.ppm
, I guess it’s
because the eye is closed and not frontal. The total rate is 25/28 ~= 90%.

8) Below images from left to right are the first left eye’s original image, recovered image by top 100 principle
components
and recovered image by top 8 principle components. Although by using 8 components we could not
recover the original image well, it does

not

matter because we can still separate the eye and non
-
eye by only using
8 components.

9)

Below left image is th
e 1
-
30 eigeneyes and the right image is the 461
-
490 eigeneyes.

We could see that the left
image is more relevant to eyes than the right image. Some sub
-
image in the left image have white pupils, I guess
this may caused by negative principle components.

5. How to run my code

The program need below files:

eyes.mat

: to load the eye images

nonEyes.mat

-
eye images

part1.m

: to train and test the detector

part2.m

: to use the detector detect eyes in
given images

runpca.m

: provides runpca() function

face.m

: provides face() function to get the face area.

skin.m

: provides skin() function called by face(), to get the skin area by the color information

In the beginning of part1.m, I have code:

'D:
\
Courses2nd
\
CS558
\
homework
\
hw4
\
eyes.mat');

\
Courses2nd
\
CS558
\
homework
\
hw4
\
nonEyes.mat');

The directory is on my own machine
.

P
lease replace them with the directory on your machine.