Department of Electronic Engineering

lynxherringΤεχνίτη Νοημοσύνη και Ρομποτική

18 Οκτ 2013 (πριν από 3 χρόνια και 2 μήνες)

53 εμφανίσεις





Department of Electronic Engineering

4
th

B.Eng Electronic & Computer 2007


Final Year Project

Progress Report

Due: 08/01/2007







Supervisor: Fergal O’Malley

Student: Dallan Byrne

FYP Progress Report


2

Contents


1

Introduction

................................
................................
.................

3

1.1

Project
Abstract

................................
................................
.....................

3

1.2

Project Goals
................................
................................
..........................

4

1.3

P
roject Layout
................................
................................
........................

5

1.4 Software Design

................................
................................
....................

6


2 Research and Development

................................
........................

7

2.1 IP Camera

................................
................................
..............................

7

2.2 Home Operation

................................
................................
....................

9

2.3 Software Environment

................................
................................
.........

10

2.4 Development Environments

................................
................................

12

2.5 Libraries/Plug
-
ins

................................
................................
................

12

2.6 Java Components

................................
................................
.................

14

2. 7 Miscellaneous Tools

................................
................................
...........

14


3 Progress to Date

................................
................................
........

15

3.1 Camera Setup

................................
................................
......................

15

3.2 Webpage

................................
................................
..............................

15

3.3 Software

................................
................................
...............................

16

3.4 Mobile Phone Software

................................
................................
.......

18


4 Remaining Work

................................
................................
.......

20

4.1 Camera

................................
................................
................................
.

20

4.2 Face Detection and Recognition

................................
.........................

20

4.3 Java Server Software

................................
................................
...........

20

4.4 Socket Application

................................
................................
..............

21

4.4 Java Mobile Phone Application

................................
..........................

21

4.5 Remaining Project Timetable

................................
..............................

22


5 References

................................
................................
..................

23

5.1 Books & Doc
umentation

................................
................................
.....

23

5.2 Websites

................................
................................
..............................

23

5.3 Table of Figures

................................
................................
...................

25


FYP Progress Report


3

1 Introduction

__________________________
___________________________________


1.1

Project
Abstract


The primary goal of this project is to develop a fully operational home security system
utilizing a wireless network
/IP

camera.
A network camera has a built in web server
allowing the owner to access i
ts video stream by simply typing an address into the URL
bar, i.e.
http://192.168.0.198

.
Once the camera is installed
,

the user can view images or
video from anywhere in the world with an internet connection.

The home

owner will be able to install software on a server and a small
application on their mobile phone. The software on the Server will allow the camera to
record pictures of an intruder and detect their faces.

This server will be fixed but not
necessarily at t
he camera location, as the camera can be accessed from any internet
connection.

The finished system should allow the us
er/owner to view a number of intruder
faces or stream a short video
on their mobile application once
the

SMS message is
received alerting

them to a break in.

The system must be able to operate with an always on internet connection for a

typical

Irish home owner using a commercially available broadband connection.



Figure
1
.1
: Camera and Subject

FYP Progress Report


4

1.2

Project Goal
s


To c
reate a fully operational security system which will:



Use a network/IP camera to monitor an area and capture imag
es once an intruder
is detected.



Perform Image processing on these images of the intruder.

o

This will locate the face of the
person present.

o

So
ftware must determine if the person present in the image is an intruder
or a recognised member of the household.

o

Images will be saved on the server
.



Allow home owner to receive notification of the break in on their cell phone

o

Transfer a text (SMS) message
from the server to the home owners mobile
phone.

o

This will be achieved through a GSM Modem or through the internet
through a carrier service.



Enable home owner to launch a
J2ME mobile application/MIDlet

on phone to
view images
.

o

Images transferred over GPRS

from server to phone

through a variation of
a Client
-
Server Socket connection

o

Home owner should be able to view images of intruders fac
e
.



Enable system to operate on a standard home broadband connection in Ireland.

o

A wireless network will be required.

o

It
must be compatible with the home owners ISP (Internet Servi
ce
Provider). Issues regarding d
ynamically allocated IP addresses will need
to be resolved for the user to access the server from outside the network
(i.e. The Internet).




See Figure 1
.1

for illust
ration

of Physical setup.
Figure 1.2 describes the desired
software operation of the system.



FYP Progress Report


5

1.3

Project Layout



Figure
1.
2
:

System Overview


FYP Progress Report


6

1.4
Software Design



Figure 1.
3
: Software Flow Diagram

of Project








FYP Progress Report


7

2
Research and Development

____________
_________________________________________________


2.1
IP Camera


This is the main and only hardware component of the project.

A Network

or IP

camera was chosen due to having a built in Web Server. This
means the camera can operate as a standalone produc
t with its own IP address,
therefore
not req
uiring a PC/workstation

to function
.
This provides an economical method to
monitor a household from a remote location.

The live video stream from the camera is obtained using st
andard http URL’s and
arguments,

i
.e.
http://140.203.8.243/

.

Various arguments can be passed to the camera
web server as long as a http connection exists.

IP Cameras range in price from €160
-


3,360. Higher end cameras have
pan/tilt/zoom functionali
ty, variable frame rate (due to bandwidth) and superior Charge
Coupled Device
. They

can operate in extremely low light conditions (down to .18lux).
Current market leaders are Sony, Axis and Panasonic.


Http

protocol support

These cameras use the http proto
col to issue and receive instructions or data to/from an
outside source
, i
.e.
t
o
view

a Motion JPEG stream from the camera
:
http://140.203.8.30/axis
-
cgi/mjpg/video.cgi?resolutio
n=352x240

(or a similar address)

can be accessed through a standard http connection in code or from a few lines of java
script on a html page. When an intruder is detected the camera can issue a http request to
a specified web address to alert the owner.






FYP Progress Report


8

Choosing a Camera

The

main requirements

for the camera

were:



W
ireless capabilities.



A built in motion detection feature to determine whether or not an intruder was
present.



It had some class of SDK to interact with software
.




There was good software s
upport.



That it was capable of viewing images in low light conditions, a lens that operated
to a low lux specification.



Capable of recording MJPEG as well as MPEG4 video

o

MJPEG


Motion JPEG is effectively a group of JPEG images transmitted
sequentially in
a group along with some header and footer information

o

This was required as MPEG4 is very difficult to manipulate and extract
images from.



Viewing capability for up to 25 frames per second.


Many products matched this criteria
. As a result the only issue w
as price and reliability.

The
Camera chosen w
as the Axis207W which is packaged with a full http, Visual
Basic.net and C/C++ API.


Motion Detection feature sends a http notification to a
specified address once the Axis 207W detects movement in the viewing a
rea.

Price: €320



Figure
2
.1
: Axis

207W Network Camera

FYP Progress Report


9

2.2
Home Operation


Most standard broadband users in Ireland have dynamically allocated IP
addresses. This

mean
s

each individual machine’s

IP address
is

provided by their ISP and
can change at
any stage.


In order

to access the

camera
outside the home network

it must be able to have a
single URL address (http://mycamera.net) which points to its dynamic IP address. An ISP
will change this dynamic address therefore the URL needs to b
e updated each time this
occurs.

This is easily achieved on a PC with a simple software application.

The
Axis 207W
has support for dynamic DNS Servicing. This means that its URL
(
http://mycamera.net
) is updated periodi
cally with the camera’s current public IP address.


The user’s router must be configured for port forwarding to allow access to the
camera from outside the home network.
This is a simple operation, assigning a port
number (say 8000) for the camera and noti
fying the router to allow access from the
internet on said port number.

The software itself
must be

packaged in a simple manner, allowing it to installed
and operated with minimum effort and complexity. Th
erefore it must be robust and
tested
fully before a
n unknown third party (home owner) attempts to use it.














FYP Progress Report


10

2.3
Software Environment
s


Visual C

Very popular programming language. Still the most commonly used for writing system
software worldwide.
C is used in

the

p
roject for image processing
alo
ng with the
OpenCV library. Used in this case for its efficient handling of complex algorithms.


Java

Sun’s alternative to C++. Robust object orientated language that provides good support
for web services
. Java differs from lower level languages (such as

C and Fortran) by
having its own unique run
-
time environment and Garbage Disposal. The Run
-
time
environment converts given java code to c source code which is then converted into
machine code.

The Garbage disposal deals with any resources allocated or us
ed during the
compilation or execution of code
. I
n C
,

functions such as malloc() and free() must be
used in order for
CPU

and memory to set aside

the desired resources and remove them
afterwards.


The majority
of coding in the project is and will be writt
en in some form of java.
This is due to the authors experience and comfort with the language.


J2SE SDK

Java 2 Standard Edition Software Development Kit. Collection of standard java API’s for
basic java programming
.









FYP Progress Report


11

J2ME

(Java 2 Micro Edition)



This

is

a collection of useful API’s
to enable the creation of Java applications for
mobile phones
, PDA’s and other small handheld devices
.



API differs to that of the J2SE or J2EE SDK’s.



It is
designed for efficiency as it was first
d
eveloped to run

java app
lications

on an
embedded system with limited resources.



Operation:

o

J2ME utilises
Mobile Information Device Profile

(MIDP), a specification
on the application of java on mobile devices or embedded systems.
Applications written for MIDP are called MIDlets.

o

The core API’s are defined on the phone by the Connected Limited Device
Configuration

(CLDC)
.

MIDP and CLDC when combined provide the
java runtime environment on the mobile phone.

o

The LCDUI

(LCD User Interface)

package contains classes for displaying
data

on the device in GUI form.

(See figure 2.1)




Figure
2.2:

Description of J2ME operation




FYP Progress Report


12

2.4
Development Environments


Eclipse

Open Source
Java compiler and coding environment.

Extremely useful due to the number
of add
-
on’s it supports from J2ME devel
opment to creating Servlets without

the

J2EE

run
-
time environment
.


Apache Tomcat Web Container

Freely available software tool which allows users to setup a simple web server on a
PC/Laptop. Supports
J2EE components (Servlets, JSP) that are not available w
ith the
J2SE SDK. Tomcat uses the Catalina Servlet container.

Tomcat is usually used with other
web servers for Servlet and JSP support, it can however operate as a standalone server.


Microsoft Visual Studio 6

Microsoft issued
coding

environment
.
Used in

project for C development.



2.5
Libraries/Plug
-
ins


Intel O
pen
CV Library

Open Computer Vision Library developed by Intel.
Open Source

collection of useful

C
headers and functions

based on image processing and manipulation.


Created originally

for
Intel I
ntegrated Performance Primitives to perform intense
computer vision operations efficiently
.
At present its function is mainly aimed towards
real time computer vision.

Many standard image processing functions are supported
coupled with the option to improv
e existing code.


Library is packaged with simple example executables and code for various
processing functions.

Some current applications are
object identification, motion tracking

and image
segmentation.

OpenCV also supports Python. No Java implementatio
n to date.

FYP Progress Report


13

SimpleWire.com SMS package

Java package which contains a

list of classes to send and receive text messages through
the SimpleWire carrier service. Once the coder has an account with SimpleWire they can
easily write simple java classes to send an
d receive messages. Service mainly used for
SMS but has MMS capabilities. There is still some issues with this tool as it always
includes a company logo on each message and MMS messages
appear as black and white
images on the receivers phone.


Sysdeo Tomca
t Plug
-
in

Useful eclipse plug
-
in which allows coders to start/stop/restart Tomcat from the
Eclipse
development environment. Enables the creation of Tomcat projects and War files in the
environment which are automatically uploaded to the Tomcat container w
hen they are
saved.



Nokia Carbide.j

Plug
-
in

Eclipse plug
-
in. Used to create J2ME applications

in a J2SE environment
.
Enables

user to
easily
create java

MIDlets and supporting classes with imported microedition packages.

This tool also allows for simple
Over the Air (OTA) packaging to create a single .jad or
.jar file to be downloaded to the mobile phone.

Plug
-
in also contains a f
ull debugging tool
to test any application before transferring it to the phone.


Nokia Series 40 SDK

Nokia mobile phone emulato
r

for a PC
. Used to observe working J2ME application on
PC before actually transferring anything to the phone.

Packaged with a resource viewer to
giv
e the coder some idea as to how phone resources are being used during the running of
the application on the

emulator. One of many

mobile phone emulators available, chosen
here for its simplicity.




FYP Progress Report


14

2.6
Java Compone
nts


Servlets

API which enables the use of dynamic content on a web server. They can be embedded
in any Http server and are an effective replaceme
nt for CGI scripts. Servlets typically
handle Http GET and POST requests from a client and generate appropriate responses.
This is an extremely useful tool for building web applications

as it has access to the
complete Java API.
The javax.servlet package p
rovides a number of interfaces and
classes for writing Servlets.

In the project Servlets receive a Http notification from the IP
camera once an intruder is detected. Upon receipt of notice, Server calls methods, classes
and executables on the local machine
/server.


Threading Issues are plentiful with Servlets as numerous clients may access the
Servlet simultaneously. As the Servlet accesses a single resource (PC), this becomes a
problem. This issue rises occasionally with the camera attempting to notify th
e server
repeatedly. The other main problem with this would be if multiple cameras were
accessing the Servlet at a single instance. Extending the SingleThreadModelInterface on
a Servlet is the best known solution for this.



2. 7
Miscellaneous Tools


DynDN
S

Online service
/protocol

allowing users to obtain a Dynamic Name Service (DNS)

host

for
their public IP address. Software available to continuously update DNS host

(
i.e.
http://
byrned.selfip.com
) with current IP
address allocated by the ISP.


This is u
sed as a dynamic address for the Tomcat server currently used from
authors laptop.

IP Camera also supports the DynDNS protocol
, automatically updating through
its internal software
.


FYP Progress Report


15

3

Progress
t
o Date

_____________
________________________________________________


3.1
Camera Setup




Camera is currently hard wired into to the elecom network in the Electronic
Engineering Department. This is due to difficulties with the universities wireless
system

(permissions)

and the
cameras MAC address.



At the Authors home the Axis 207W is accessed through a wireless connection.



In a home environment setup is very simple and straightforward.



Camera is first set up through a wired Ethernet connection with the PC. Then
settings on the

camera server are altered to connect to the wireless network using
its WPA/WEP key and SSID.


Port Mapping



Like a workstation or PC, the camera has a private and public IP address. The
private being viewable only within the wireless network and the publi
c address
viewable from the internet. The wireless router or access point must allow
incoming traffic on a specified IP port to allow users to view the camera from the
internet.



The Authors home router has port forwarding enabled to allow the camera to be

viewed from the internet.

o

This however does not register with the cameras DNS domain name.

Typing the public IP address of the camera (
http://89.134.54.66
) will
allow access, but
currently the

DNS host URL does not re
spond.

3.2 Webpage



Setup and operational. Live Camera link is now accessible through the webpage.
Created using a mix of FrontPage, HTML and a little JavaScript.



Needs to be updated at regular intervals. Not so at the moment.

FYP Progress Report


16

3.
3

Softwa
re


Current Overall
Design



Figure
3
.
1
: Main Server Software Design


StreamVideo



AxisCam.java



First java class to be completed. Streams a live MJPEG stream to a simple java
application using threads. MJPEG file is received and header and footer
in
formation removed while displaying the actual JPEG information in a JFrame
window.



FYP Progress Report


17

MotionServlet



MotionServlet.java



Con
structed to monitor incoming Http alerts from the camera when motion
detection occurs. When an intruder enters the camera’s viewing
scope, it invokes
the Servlet.



Servlet uses the java runtime class to run a C executable file for the Image
Processing and
creates an instance of the
SendMailUsingAuthentication

class.


Send
Mail


SendMailUsingAuthentication.java



Sends an email to a speci
fied list of recipients (unlimited).

Uses the JavaMail
API.


RecordImages

Class
-

Cam.java



Ja
va class which continuously saves and overwrites a number (currently 50) of
JPEG’s from the IP camera to the Server.



Creates a text file, containing a list of image

locations on the server
(C:/Pictures/Camera/p1.jpg). This is used by the Image Pr
ocessing code
(facedetection.c)

to locate images to process.



Face Detection code


faceDetect.c



Created using OpenCV and visual studio 6. The code first locates a text file

containing a list of the image locations.



The first image file is opened. An array inside a function is created to store
instances of human faces and their locations in pixel format on the overall image.



A
n

OpenCV function is called which

applies a Haar
Wavelet classifier and a
number of spatial domain
filters over the image to locate individual faces.



Once the faces are recorded, the software writes each face to a file (,jpg)
.



First Image file is closed and the next is opened following the same cycle
unt
il

all
images have been scanned.



FYP Progress Report


18

3.
4

Mobile Phone Software


MidletClass.java



Launch MIDlet of the mobile phone application. The MIDlet class controls the
overall layout of the software regarding the user display.

The MIDlet creates and
controls all comma
nds available to the user. Essentially an event driven class
which responds to user interaction.



Contains the connect() method which instantiates a Http connection to a given
URL web address. Once a valid connection is made then bytes are read from the
spe
cific URL and converted to a JPEG image.



At present the software is
not

using a Client
-
Server connection as originally
planned.

The Application reads a number of JPEGs posted on a
web server
.



WelcomeScreen.java



A standard J2ME
Form which is the opening
display for the user. It contains a
small image and a Textbox. The user must enter their password or pin number to
progress any further within the application. Class also contains a setter and getter
for the password in order for the MidletClass to allow a

http connection.


ChoiceForm



Small Form created within the MidletClass class. Once a connection is established
to
the
URL address
, the ChoiceForm determines the number of jpegs that are
available and
creates a display to allow the user to view one.


Conne
ct.java



A

simple

Canvas class

which produces the image of the intruders face with a
small caption underneath. Canvas uses the Graphics class to allow the coder to
resize

and manipulate

images for the small mobile display.



FYP Progress Report


19

Mobile Application
Overview


Fi
gure
3
.2
: Mobile Application Flow Diagram



Figure
3
.
3
: J2ME
Mobile Application Software Design

FYP Progress Report


20

4
Remaining Work

_____________________________________________________________


4.1
Camera



Camera DNS service must be setup correctly. Enabling the user to t
ype in the
http://axis75e9fd.axiscam.net/

URL
within a web browser

and interacting with it from
any location on the internet. Different methods will apply for viewing the camera in a
simple home network and t
he NUIG

and ELECOM

wireless network
s
.


4.2
Face Detection and Recognition


The software will detect faces but only under specific conditions.
Camera must be
positioned in an uncluttered environment and the i
n
truder must be facing it

directly
.

Multiple face
s and the actual viewing angle of the face can cause issues with the
software. Requires research into further
development of
comprehensive face detection
algorithms
,

allowing for these discrepancies.

Facial recognition algorithms have not been implemented

to date. This will
require a large amount of research
into detailed image processing techniques.



4.3
Java Server Software


Greatest task at hand will be bundling the software for simple installation and use for a
customer/user. Will require numerous .ja
r files and creating a simple GUI to enter
information from the user.

Another problem is the software’s efficiency. Needs to utilise threading and
daemon code in order for it operate at a satisfactory pace.


FYP Progress Report


21

4.4 Socket Application


A Socket Server is writ
ten but must be integrated into the system to communicate
directly

to

the
user’s

mobile phone over a secure GPRS connection.

Classes
coded

to
transfer a JPEG image over a network connection between two workstations. Mobile
phone application must now look f
or a s
pecific port number instead of
a URL address on
the internet.

This will be straight forward as little code needs to be modified on the Client
(Mobile Phone) side.



4.4
Java Mobile Phone Application


As previously described
,

the software
currently
re
ads a stream of bytes from .jpg files
placed in a folder on a web Server. The MIDlet must establish a secure socket connection
to the phone. Although fairly straight forward between 2 devices on the same network,
this requires

more

research and effort to c
ommunicate over

the internet
.

T
ransferring the images one by one, sequentially over the internet may be
inefficient. Another option would be to transport an object o
r

even a .txt file containing
the byte information for each .jpg file.


Streaming a short v
ideo clip of an intruder is another aspect that must be
addressed. Transferring a group of JPEGs (around 100
-
200) will work if the phone
application is threaded and the images are played back in quick succession. Actually
streaming MJPEG is another
,

provi
ding the phone will support an ActiveX plug
-
in of
some variety. This is doubtful though
,

but still requires clarification.








FYP Progress Report


22

4.5
Remaining Project Timetable


Period

Task(s)

02/01/07 → 30/01/07



Camera
.

o

Sort DNS issue with camera URL at home.

o

Once operational, ensure camera operates in the
same fashion on the Electronic Engineering
wireless device network.

o

Camera must be fully operational.




Socket Connection between Server and
Phone


(J2ME & J2SE).

o

Investigate how to connect securely over the
internet

o

Transfer a number of jpegs

o

Transfer/View a video clip of the intruder

30/01/07 → 27/2/07



Image Processing (In C with OpenCV).

o

Further research and development into Face
Detection
methodology and algorithms.



Handling multiple faces.



Handling faces at non
-
direct viewing
angles to the camera.

o

Face Recognition.



Research conditions required.



Code a functional executable.

27/2/07 → 27/03/07



Link system together.

o

Create a GUI for user se
tup.

o

Finish off loose ends



Report.



Demonstration.


FYP Progress Report


23

5

References

_____________________________________________________________


5.1
Books & Documentation




Java 2 Enterprise Edition™ How to Program, 6
th

e
dition

(2005
,
Dr. Harvey &
Paul Deitel
)



Simply C
, (
20
05
,
Dr. Harvey & Paul Deitel)



Java


2 Platform Standard Edition 5.0
API Specification

(2004, Sun
Microsystems)



Java

Mobile Edition Technology API Documentation (2006, Sun
Microsystems)

o

CLDC API

o

MIDP API

o

Mobile Media API


5.2
Websites




http://www.axiscam.net

o

Axis Camera
customer support page
.



http://www.axis.com/techsup/cam_servers/dev/cam_http_api_2.htm#doc_
history

o

Axis Http A
PI for IP
c
ameras
.



http://www.dyndns.org

o

Dynamic DNS
home page.



http://www.networkcamerareviews.com

o

Support
forum

for IP
camera owners.


FYP Progress Report


24



http://www.brickhousesecurity.com/about
-
ip
-
network
-
video
-
cameras.html

o

Network Camera FAQ



http://java.sun.com/j2se/1
.5.0/docs/api/index.html


o

J2SE
API

Spec
ification.



http://opencvlibrary.sourceforge.net/

o

OpenCV Wiki
s
ite
.



http://tech.groups.yahoo.com/group/
OpenCV/

o

Yahoo
f
orum for OpenCV developers
.



http://tomcat.apache.org/

o

Tomcat homepage
.



www.
sysdeo
.com/eclipse/
tomcatplugin

o

Eclipse plug
-
in for Tomcat
.



http://www.simplewire.com

o

Vendor of Java SDK for
m
obile
p
hone messaging through the internet
.



http://java.sun.com/javame/reference/apis.jsp#api

o

Java

ME APIs
.



http://forum.java.sun.com/category.jspa?categoryID=23

o

Sun’s J2ME
d
eveloper
f
orum
.



http://forum.nokia.com/

o

Nokia Developers
f
orum with
downloadable tools
.









FYP Progress Report


25

5.3
Table of Figures


FIGURE

DESCRIPTION

1.1

Camera and Intruder Illustration




Sony



p乃
J
ow㌰P⁎e瑷潲t⁣a浥牡


㈰〴⁓ony
C潲灯oa瑩潮

1.2

System Overview Description

1.3

Software Flow Diagram of Project

2.1

Axis


㈰Ot⁃a浥ma
䥭f来



Source: 207W Datasheet
©
200
5

Axis Communications AB

2.
2

Java ME Runtime Description

3.1

Server Softw
are Design Description


3.2

J2ME Mobile Phone Application Flow Diagram

3.3

J2ME Mobile Phone Application Software Design Description


All Diagrams created with SmartDraw


2007

unless otherwise stated

SmartDraw
™ 2007
© 2007
SmartDraw.com