License Plate Recognition

madbrainedmudlickΤεχνίτη Νοημοσύνη και Ρομποτική

20 Οκτ 2013 (πριν από 3 χρόνια και 7 μήνες)

71 εμφανίσεις

Image Processing Assignment 2



Koh Chan Loong

1

|
P a g e


License Plate Recognition

1.0

The Program Flow


Figure
1

P
rocess flow chart

The license plate recognition approach used in this assignment are demonstrated as the
flow above where blue colour flow indicated the main program flow,

and the orange
colour flow indicated the neural network flow.

1.1

Colour Segmentation and Hough Line Detection

Generally,
a

c
aptured car image consist

many lines,
by only used of Hough line
detection,

it is not
efficient enough

to recognize
where

the license
plate is
,
and thus the colour
segmentation must be applied first before the Hough line detection.

For colour segmentation
, it is designed once

yellow colour
was

detected
then

set
it
to
white colour
(255)

while other colour will be set to black

(0)

(see
figure

3
a
).

As for
Hough line detection, the contents inside the processed license plate is not necessary,
thus

the canny edge detection technique was apply to obtained the edge of the license plate for
best
result (
see figure

3
b
).

The colour segmentati
on
can be done by determined

the characteristic of yellow colour
was known, as for RGB image, the yellow colour consists of RED and GREEN are
Image Processing Assignment 2



Koh Chan Loong

2

|
P a g e


almost equal and having the parameters range from 100 to 255 and the BLUE must be
lower than 50.

After the colour se
gmentation and the edge obtained, thus the Hough detection now can
detect 2 lines on the top and bottom of the license plate by suitable configuration (see
figure

3
c
). Once the line

was

obtained, the license plate
can be
detected,
by using the
relationship

of the two
lines;

the image can be crop by “
i
mcrop
” function in Matlab

(see
figure
4
a)
.


Figure
2

Original Image


Figure
3

from left, (a
) image after colour segmentation (
b)

after canny edge detection
(b)

Hough line
detection



Figure
4

from left (a)
cropped license plate after the process of colour segmentation and Hough line
detection

(b) binary image with noise reduce and image resize


Once the license plate
obtained, the following process is to convert the image into binary
by the “
im2bw
” Matlab function
.

Due to

t
he image was captured in the real

situation,
thus the noise might be obtained in the
image,

in order
to

improve the process of
character segmentatio
n,
and the

noise must be
reduced
. The image size
was reduced

in
this case to obtaining faster process
ing speed
. The figure
4
b show
n

above was the
successfully converted image with noise reduction.

Image Processing Assignment 2



Koh Chan Loong

3

|
P a g e


1.2

Character Segmentation

To segment the character in the license plate, one of the
ways

is to
labelling the

connected binary to a group.
This can be done by using “
bwlabel


function
In Matlab,
the example of this operation functions are show below,


The binary
’s

image input

After labelling process


1 1 0 0 0 0 0 0 0 0 0

1 1 0 0 1 1 1 0 0 1 1

1 1 0 0 1 1 1 0 0 1 1

0 0 0 0 0 0 0 0 0 1 1



1 1 0 0 0 0 0 0 0 0 0

1 1 0 0 2 2 2 0 0 3 3

1 1 0 0 2 2 2 0 0 3 3

0 0 0 0 0 0 0 0 0 3 3




For each character, the corresponding character
must be connected together
, thus th
e
characters in the license plate were labelled into different
number. By

finding each
number in the image
’s matrix
,
then the

character can be extracted

(see figure 4).



Figure
5

Character Segmentation display in GUI interface

In order to input the character image to the neural network, the image’s matrix must be
reshaping into vector form.


1.3

Neural Network


The feed
-
forward back propagation network was use in this assignment.
The approaches
can be done by the nntool which introduced in the lecturer notes, which
is GUI

tool box
type. But as the training data from 0 to 9 and A to Z
were too big and inconvenient to
load into nntool,

thus the newff function was
used

(code format)
.

1.3.1

Neural network
setup

To
establish

a

feed
-
forward backpropagation
neural n
etwork function in Matlab, it
can be
easily done by the internal function “
newff
”.
The syntax in Matlab
for neural network is
known as

Image Processing Assignment 2



Koh Chan Loong

4

|
P a g e



net = newff(PR,[S1 S2...SNl],{TF1 TF2...TFNl},BTF,BLF,PF)




where,



PR
-

Rx2 matrix of min and max values for R input elements.



Si
-

Size of ith layer, for Nl layers.



TFi
-

Transfer function of ith layer, default = 'tansig'.



BTF
-

Backprop network train
ing function, default = 'trainlm'.



BLF
-

Backprop weight/bias learning function, default = 'learngdm'.



PF
-

Performance function, default = 'mse'.


1.3.1

Training the neural network

To train the network, the input and target are required. For the input, the dat
a are obtained
from the image show in figure
6
. By using the same technique that introduce in charact
er
segmentation, then each character’s matrix
was obtained as the input. The target was
predetermined as the ASCII code

as the reasons of its
universality
.


Figure
6

character’s image use in network training

As the input and target were obtained
, by using
the “
train
” function in matlab, the network can
be train with suitable configuration.

The syntax for network training was shown,

[net,tr] = train(net,Input,Target
)

where ,



Input


All the segmented character from figure 6 in vector form



Target


The entire target (ASCII) corresponded to input


1.3.3

Network Simulation and Character Pattern Recog
nition

After the network
had been successfully

trained, then now the
segmented character in
license plate can n
ow inputted
the neural network to simulation. Ideally, the input
character
s

will compare with the data that trained in neural network, and then outputted
the ASCII code for corresponding input.
I.e.

if
image’s
character “A” was detected, the
results of
[1 0 0 0 0 0 1] obtained.

The 7 bits ASCII
codes were

input to the Ch
aracter
p
attern recognition process to obtaining its corresponding character.




Image Processing Assignment 2



Koh Chan Loong

5

|
P a g e


2.0

Discussion on the approaches

There is many ways to
achieve the in each process, such as
different way of
license
plate
recognition technique, character segmentation etc.



For
license plate recognition, the better way that introduce in reality was the use of
Hough rectangle transform, which

only detected rectangle shape instead of lines that used
for this assignment.
Thus the operations can be
achieved even without the use of co
lour
segmentation. Other then Hough
transforms
,
the more complicated method such as
Eigenspace Approach

(which introduced in face recognition), signature segmentations
and as well as FFT segmentations etc was also been used to recognize license plate
application.

The performance will vary based on the image quality, and image captured
environment and situat
ion.


To
perform

colour segmentations, the
simplest

ways to achieve was believe by using
the
characteristic of yellow colour in RGB format as the assumption made was for most of
the UK car, the back license plate must be yellow. In more practical way, the
colour filter
technique
is highly recommended than the approaches
that
use in this assignment, but the
complexity may increase in return.
The colour filter technique was achieved i
n frequency
domain, the Fourier transform can be applied to image, by applyi
ng the filter technique,
and the inverse Fourier transform after, the required colour can obtained which based on
the corresponded filter applied.


Character segmentation
can be
done by
vertical segmentation algorithm is based on
projection

analysis
. By comparing to the approaches that designed for this assignment
which described., it is believed that the both method can achieved the same tasks but for
more noisy image, the
vertical segmentation

is believed to performed better, but it
required longe
r process time and the designed was complicated.



Figure
7


Vertical segmentation

[1]


The neural network
is one of the best solutions for character recognition, even there
might contains some small errors.

By compare to the template matching

method, if the
image noise obtained is high
, or the detected character was deform slightly, then the
template matching method might not be effective
.

F
or neural network, if
the interference
maintain in acceptable ran
ge
, the results
can
still be achieved.

As the size of license plate was varies via the captured distance and some other factor,
thus after the resize of image, the quality of image was reduced and tends to distorted the
image pixels.
To encounter this pro
blem in real situation, the captured distance between
Image Processing Assignment 2



Koh Chan Loong

6

|
P a g e


the car and the camera should be limited by a
specified

range
,

and the camera
use
to
capture all

the

images

should be the same.

3.0

Conclusion


The GUI interface that designed in the assignments
is

show

below
, which is able to input
image
, train network and also simulation

(see figure 8a).


Figure
8


(a) GUI interface designed (b) simulation result

As the input matrix was not perfect, thus some errors are obtained during the

simulations
(see figure 8b). But in overall, if the image was in same size, the performance was still
acceptable. If the errors obtained was too high, the network can be retrain to get better
performance.

Nevertheless, the design can still consider
succes
ses

at this stage.

References

1.

Yungang Zhang, Changshui Zhang, “A New Algorithm for Character
Segmentation of License Plate
”:

Dept. of Automation, Tsinghua University
,

The
Institute of Information Processing Beijing, China


2.

Prof Nicholas Bowring,
Lecturer notes for
64ET4508. Manchester Metropolitan
University