Thesis proposal for MS ECE program Written for Dr.Lyon by

connectionviewAI and Robotics

Nov 17, 2013 (3 years and 6 months ago)







Written f


Nishanth Vincent

Electrical and Compute
r Engineering Department

Fairfield University

This thesis describes the design and construction of a test
bed for the prototyping of
embedded face detection and recognition algorithms. The test bed makes

use of hand
held devices. Called the PITS (Portable Interactive Terroist Identification System), it is
our intention push embedded computer vision beyond the experimental research setting.
The embedded device has a processor, camera, color display, and wi
reless networking.
This system is different from existing systems because of its state
the novel image
processing algorithms, and embedded information technologies. The device provides an
interactive face detection which upon detection
integrates with a

server using a
wireless LAN module for uploading recorded faces

and associated data (time, location,
interviewer notes, etc.)


ace detection is essential front end for a face re
cognition system. Face
detection locates and segments face regions from cluttered images, either obtained from
video or still image. It has numerous applications in areas like surveillance and security
control systems, content based image retrieval, video
conferencing and intelligent human
computer interfaces. Most of the current face recognition systems presume that faces are
readily available for processing. However, we do not typically get images with just faces.
We need a system that will segment faces
in cluttered images [2]. With a portable system,
we can sometimes ask the user to pose for the face identification task. In addition to
creating a more coorperative target, we can interact with the system in order to improve
and monitor its detection. With

a portable system, detection seems easier. The task of
face detection is seemingly trivial for the human brain, yet it still remains a challenging
and difficult problem to enable a computer /mobile phone/PDA to do face detection. This
is because the human

face changes with respect to internal factors like facial expression,
beard, mustache glasses etc and it is also affected by external factors like scale, lightning
conditions, and contrast between face, background and orientation of face.

detection remains an open problem. Many researchers have proposed different
methods addressing the problem of face detection. In a recent survey face detection
technique is classified in to feature based and image based. The feature based techniques
use ed
ge information, skin color, motion and symmetry measures, feature analysis,
snakes, deformable templates and point distribution. Image based techniques include
neural networks, linear subspace method like Eigen faces [1], fisher faces etc. The
problem of f
ace detection in still images is more challenging and difficult when
compared to the problem of face detection in video since emotion information can lead to
probable regions where face could be located.

Problem definition:

We are given an input scene and a suspect database, the goal is to
find a set of possible candidates. We are subject to the constraint that we are able to
match the faces from the scene in an interactive time and that our algorithm is able to run
n the given embedded hardware.


The basic algorithm starts with a pre
processing step, consisting of digitization and
segmentation. The next step is called face segmentation. We define the face segmentation
problem as: given a scene that may c
ontain one or more faces, create sub
images that
crop out individual faces. There are several algorithms available in the literature that can
solve this problem. A survey on face detection with more than 150 references appears in
[4]. Face segmentation mak
es use of facial features in order to identify the face [4]. Some
algorithms for tracking face contours are known to be effective. Tuning them to our
embedded system will be a challenge [5]. After face segmentation, the device enters into
face identifi
cation mode
, as shown in Figure 2.1

Suspect database
Data Base
Feat ure
Data Base
Segment ation
Feat ure
Extract ion
mat ches
Displays possible
candidat es for selection

Face Identification System

Human skin is relatively easy to detect in controlled environments, but detection in
uncontrolled settings is still an open problem [6.]. Many
approaches to face detection are
only applicable to static images assumed to contain a single face in a particular part of the
image. Additional assumptions are placed on pose, lighting, and facial expression. When
confronted with a scene containing an unk
nown number of faces, at unknown locations,
they are prone to high false detection rates and computational inefficiency. Real
images have many sources of corruption (noise, background activity, and lighting
variation) where objects of interest, such
as people, may only appear at low resolution.
The problem of reliably and efficiently detecting human faces is attracting considerable
interest. An earlier generation of such a system has already been used for the purpose of
flower identification by [7, 8]


Face detection plays an important role in today’s world. They have many real world
applications like human/computer interface, surveillance, authentication and video
indexing. However research in this field is still young. Face recogni
tion depends heavily
on the particular choice of features used by the classifier One usually starts with a given
set of features and then attempts to derive an optimal subset (under some criteria) of
features leading to high classification performance with

the expectation that similar
performance can also be displayed on future trials using novel (unseen) test data

Interactive Face Recognition is divided in to several phases; it includes

Creating drivers for the handheld device that link with the applic
ation with the
captured image.

A face detection program is run inside the handheld device which detects the face
from the image [9], [10], [11], [12], [13] and [18].

The obtained face is transmitted through wireless network

The server performs the face r
ecognition and is transmitted back [14], [15], [16],
[17], [18], [19], [20] and [21].

Each work is assigned three weeks of time.

The Interactive Face Recognition can benefit the areas of: Law Enforcement,
Airport Security, Access Contr
ol, Driver's Licenses & Passports, Homeland Defense,
Customs & Immigration and Scene Analysis. The following paragraphs detail each of
these topics, in turn

Law Enforcement
: Today's law enforcement agencies are looking for innovative
technologies to help
them stay one step ahead of the world's ever
advancing terrorists.

Airport Security: The Interactive Face Recognition device can enhance security efforts
already underway at most airports and other major transportation hubs (seaports, train
stations, etc.
). This includes the identification of known terrorists before they get onto an
airplane or into a secure location.

Access Control
: The Interactive Face Recognition device can enhance security efforts
considerably. Biometric identification ensures that a
person is who they claim to be,
eliminating any worry of someone using illicitly obtained keys or access cards.

Driver's Licenses & Passports
: The Interactive Face Recognition device can leverage the
existing identification infrastructure. This includes,
using existing photo databases and
the existing enrollment technology (e.g. cameras and capture stations); and integrate with
terrorist watch lists, including regional, national, and international "most

Homeland Defense
: The Interactiv
e Face Recognition device can help in the war on
terrorism, enhancing security efforts. This includes scanning passengers at ports of entry;
integrating with CCTV cameras for "out
ordinary" surveillance of buildings and
facilities; and more.

s & Immigration
: New laws require advanced submission of manifests from
planes and ships arriving from abroad; this should enable the system to assist in
identification of individuals who should, and should not be there.

The Interactive F
ace Recognition device is a test bed for embedded face
recognition research. As such, it contributes toward building a general infrastructure for
research into embedded vision, further benefiting society.

iterature Cited

Turk and Pentland, Face

Reconstruction and Recognition Using Eigen Face


K.Sandeep and A.N.Rajagopalan, Human Face Recognition in cluttered color

Images using skin color and edge information

[3] Kim Topley, J2ME IN A NUTSHELL, A Desktop Quick Reference, Oreilly

Publications Jonathan Knudsen, Wireless Java Developing With J2ME
, a!


Boehme, H.
J., Brakensiek, A., Braumann, U.
D., Krabbes, M., and

Gross, H.
M. Visually
Based Human
MachineInteraction in a Neural Architecture.


Selbstorganisation von adaptivem Verhalten, pages 166
175. VDI

Verlag, 1997.

[5] J. Sobottka and I. Pittas.
Segmentation and tracking of faces in color images.

In Proceedings of the Second International Conference on Automatic Face and
Gesture Recognition,
pages 236

241, 1996.

] Wang, C., and

Brandstein, M.S., “A hybrid real
time face tracking system”
ICASSP 1998,
Seattle, WA, May, 1998, pps. 3636

] G. Nagy, J. Zou, “Interactive Visual Pattern Recognition”,
Proc. Int. Conf.
Pattern Recognition XIV
, vol. 2, pp. 478
481, Quebec City
, 2002.

[8] Arthur Evans, John Sikorski, Patricia Thomas, Sung
Hyuk Cha, Charles Tappert,

Computer Assisted Visual Interactive Recognition (
) Technology

Renaud S´eguier
, A vary fast adaptive face detection system,


onics and Telecommunications of Rennes

Sup´elec, Avenue de la Boulaie, 35511
S´evign´e, France

[10] Peng Wang, Qiang Ji,
View Face Detection under Complex Scene based on
Combined SVMs
Department of Electrical, Computer and System

Engineering, Rensselaer Polytechnic Institute, Troy, NY

[11] Minsick Park, Chang Wo
o Park, and Mignon Park], Algorithms for Detecting
Human Faces Based on Convex hull, Yonsei University, Seoul, Korea

.] H.
Y. S. Li, Y. Qiao, and D. Psaltis, "Optical network for real time face
recognition," Applied Optics 32(26), pp. 5026
5035, 1993.

Lien Hsu
, Student Member, IEEE, Mohamed Abdel
, Member,
IEEE, Anil K. Jain, Fellow, IEEE

Face Detection in Color Images


Jianke Zhu, Mang Vai and
Peng Un Mak,
Face Recognition, a Kernel PCA Approach
Department of Electrical and

Electronics Engineering, F
aculty of Science & Technology,

University of Macau, Macau SAR, China

[15] [Terence, Rahul, Mathew,

Shumeet] Terence Sim, Rahul Sukthankar, Mathew
Mulin, Shumeet Baluja, Memory Based Face Recognition for Visitor Identification,

Robotics Institute, Carnegie Mellon Univ., Pittsburgh, PA

[16] FernandoDe La Torre, Michael J.Black 2003 .Principal Component Analysis For
Computer Vision.

[17] Turk and Pentland, Face Reconstruction and Recognition Using Eigen Face Method

] Dou
glas Lyon. Image Processing in Java, Prentice Hall, Upper Saddle River,
NJ. 1998.


Creed F Jones III 2003, Color Face Recognition Using Quaterionic Gabor Filters

[20] Ilker A

Face R

Using E
s, Istanbul Technical U

[21] Liang Wang, Tieniu Tan, Weiming Hu,
Face Tracking Using Motion

Dynamic Template Matching
National Laboratory of Pattern Recognition

Institute of Automation, Chinese Academy of Sciences, Beijing, P. R. China, 100080

] Douglas Lyon, The

DocJava Home Page, <>.

] “The Imperion Threading System” by Douglas A. Lyon,
Journal of Object


publication pending

] “Project Imperion: New Semantics, Facade and Command Design Patterns for
Swing”, by Douglas A. L
Journal of Object Technology,
vol. 3, no. 5, May
June 2004, pp. 51

] Asynchronous RMI for CentiJ”, by Douglas A. Lyon,
Journal of Object

vol. 3, no. 3, March
April 2004, pp. 49

] “On the use of a Visual Cortical Sub
Model for Interactive Heuristic
Edge Detection”, by Douglas A. Lyon,
International Journal of Pattern
Recognition & Artificial Intelligence


publication pending.

Java for Programmers
, Prentice Hall, Englewood Cliffs, NJ, 2004.

] “Buildin
g Bridges: Legacy Code Reuse in the Modern Enterprise”, By
Douglas A. Lyon and Christopher L. Huntley,
, May, 2002, pp. 102


McKenna, S.J.[Stephen J.], Jabri, S.[Sumer], Duric, Z.[Zoran], Rosenfeld,
A.[Azriel], Wechsler, H.[Harry], Tracking

Groups of People,CVIU(80), No. 1,
October 2000, pp. 42

.] Jeffrey M. Gilbert and Woody Yang. A real
time face recognition system using
custom VLSI hardware. In IEEE Workshop on Computer Architectures for Machine
Perception, pages 58
66, December

.] A. Pentland, B. Moghaddam, T. Starner, O. Oliyide, and M. Turk. View
Based and Modular Eigenspaces for Face Recognition. Technical Report 245,
M.I.T Media Lab, 1993.

] H. Schneiderman, “Learning Statistical Structure for Object Detection”,
Computer Analysis of Images and Patterns

(CAIP), 2003, Springer
August, 2003.


Friedrich Ludwig Bauer; J H Wilkinson; C Reinsch,
for automatic



v.: IL. Springer, 1971



Viola and Michael Jones. Rapid object detection using a boosted
cascade of simple features. In CVPR, 2001.