Computer Vision Problem to Solve

piloturuguayanΤεχνίτη Νοημοσύνη και Ρομποτική

15 Οκτ 2013 (πριν από 3 χρόνια και 7 μήνες)

120 εμφανίσεις

Ammar Ahmed and James Reetzke

April 7, 2011

ITCS 4152

Final Project

Milestone 1: Project Plan

Computer Vision
Problem to Solve

Automated vehicle license plate
detection and character recognition

of video

Introduction and

Automated license plate detection and recognition is a very powerful application of
Computer Vision in the fields of law enforcement, building security and general
ormation gathering. Although techniques for

license pla
te detection and
nition have

been developed and implemented
previously, we would like to
implement and test license plate detection and recognition ourselves, for the
purpose of gaining more experience in
Computer Vision techniques of image
segmentation and feature det

and extraction

The main areas of difficulty

Computer Vision problem are the following:


Identifying the rectangular area of a moving vehicle as the region in the
image, which contains and displays the license plate information
. Other
gular shapes within the image
may have
similar dimensions and

which make distinguishing the specific license plate region
more difficult.


detected symbols on a
nse plate to known letters and


appear on li
cense plates
. This falls into the realm
of OCR: Optical Character Recognition.


Blurred and distorted images from the detected license plate
. The
images of

license plate from a video will most likely have blurrin
g and
distortion on some frames.

A robust
method for handling this problem would
to gather information fro
m multiple frames and analyze it
, to obtain a
more accurate picture of the information displayed on the license plate.

Proposed Solution

In order to solve the problems described above, we would like to attempt to
implement some of the license plate detection and recognition techniques described
by Donoser et al.[1]. Their solution to the problems of license plate detection and
can be broken down into the following steps:


Create two


segmentation images

of the first

video frame using MSER

(Maximally Stable Extremal Region)
. One

type of

MSER image
bright areas with dark boundaries and the other

type of

MSER image
gments dark areas with bright boundaries. Using these two MSER images
in combination makes detecting the area o
the video frame containing
license plate
, easier to detec
t using the feature descriptors


et al.

in their paper. This type of
segmentation also has the advantage of
segmenting out the individual blobs representing each character on the
license plate.
We plan on using the OpenCV libraries for this project, and one
of the feature detector methods OpenCV provides is MSER, so that sh
reduce the amount of time needed to implement this project.


Use the detected region from the first frame to create an interest region in
following video frames. This reduces the amount of searching the algorithm
must do, to find the license plate and

characters in
every subsequent frame


Use SVM (Support Vector Machines) to match the detected letters on the
license plate to English numbers and letters. This is one area where we may
not follow the procedures listed by Donoser, as SVM seems to be rela
ted to
machine learning, which is an area neither myself or Ammar are familiar
with. So we may use other pattern recognition and training methods
available in OpenCV, to do the
OCR component of the implementation.


Finally, use the detected characters from

each frame, to create a voting
system for the letters detected. This
solves the problem of blurring and
distortion of the plate image, within a video frame. It also takes advantage of
videos providing many frames, and each one can help strengthen the vote

the correctly detected letter or number.

Data Sources

For this project, we intend to record our own videos

of vehicles on the road. These
videos will include ‘dash cam’ recordings, which record traffic from the point of
view of the inside a car, look
ing forward. These will be useful for testing whether
our implementation can detect and track images of vehicle license plates when the
camera is pointin
g directly

at the license plate. This angle

may also have other
things in the image

which look like a
license plate.

These will be daytime shots.

We also intend to produce recordings of vehicles
from outside the car. These
recordings will be hand held
as well as stationary. They will be useful for testing
whether our implementation is robust enough to hand
le blurring and distortion in
the source image.
These recordings will be made from a stationary position. There
will be two types of recordings from this position. One type will follow a car as it
travels in front of the camera. The other type will just po
int the camera at the road,
and cars will pass by it, without the camera moving.


To test our algorithm, we intend to evaluate how successful our implementation is,
with regard to

well it is able to detect license plates, in videos with va
rying levels
of image complexity
. Meaning that, we will use a video clip


contain varying amounts of noise. Noise as described here, means objects in
the image that have a similar

appearance to a
license plate. This will

test the
implementation’s ability to discern between license plates and
areas that
look like license plates

How accurate is the implementation, at reading characters from the
license plate using OCR

or pattern recognition
. In order to test this, we
l have video clips

which have varying levels of vehicle speed and motion
within the frame, and use this to determine
how fast, if at all, the
implementation is able to detect and recognize characters
. We can then
determine ways to modify the implementatio
n, in order to improve
accuracy and speed of recognition.

Evaluation Criteria

Low Target

We believe the low target for this project should be the detection of a single
icense plate, within a straight
orward facing

‘dash cam’ video. This

require the first part of the implementation

and would not require any
character recognition or the ability to find license plates in other
, more noisy
or complicated,

types of video recordings.

Medium Target

The medium target for this project sho
uld be that our implementation is able
to detect and track a license plate in a video recording. It should also be able
to do character recognition of the letters within the
detected license plate.
The character recognition for this target would use simple

pattern matching,
to identify the letters and numbers on the license plate.
For the medium
target, the implementation should be able to
detect and recognize license
plates in all of the recorded videos we provide, within reason of course.

High Target

e high target for this project would be that our implementation is able to
perform all of the tasks require
d by the low and medium targets, but instead
of using pattern matching for the character recognition, we would use SVM
or other machine learning techniques, to do the matching.


Donoser, M., Arth, C., Bischof, H
.: Detecting, Tracking and Recognizing License
Plates. In: Lecture Notes in Computer Science. (2007) 447