By
Prof. Dr. Erol UYAR
R.A. Aytac GOREN
Mech. Eng. (M.Sc.) Alper AYBERK
A Combined Vision

Robot Arm
System for Material Assortment
•
Introduction
•
Object Recognition via Classical Moments
•
Control of a Robot Arm System
•
Conclusion
Contents:
•
In
this
project,
an
application
of
computer
image
processing
to
recognize
various
objects
and
the
vision

based
g
uidance
of
a
robot
arm
to
assort
them
are
investigated
.
•
Captured
image
of
the
passing
objects
on
the
conveyor
band
are
digitalized
and
features
of
the
objects
are
calculated
to
identify
them
.
•
After
identification
process,
objects
are
assorted
by
a
built
up
revolute
jointed
robot
manipulator
with
five
degrees
of
freedom
with
a
gripper
,
as
an
end

effector,
is
used
to
assort
the
recognized
objects
to
desi
r
ed
locations
via
a
predefined
trajectory
.
Introduction
Introduction
Image Processing
(Object Recognition
via feature extraction,
Edge Detection to define
the place of the object
to be hold by gripper)
Feedback of the
control of the robot
arm
Control of the robot arm
using the points defined
via image processing
Conveyor
Band and
CCD camera
system
Grabber Card
digitializes the
images without any
software to be fast
•
After
Hu
presented
moment
invarients
in
1962
,
they
are
widely
used
in
many
applications
.
Subsequently,
Resis
revised
and
Li
reviewed
and
re

formulated,
also
produced
higher
order
invariants
.
However,
increasing
complexity
with
increasing
order
and
having
redundant
information
because
of
not
being
derived
from
a
family
of
orthogonal
functions
are
the
two
main
drawbacks
of
Hu
moment
invariants
.
•
Afterwards,
i
n
term
of
noise
sensitivity
,
image
description
capability
and
redundancy
of
information
Legendre
moments,
ortogonal
moments,
geometrical
moments,
Zernike
moments,
pseudo

Zernike
moments
and
Orthogonal
Fourier

Mellin
moments
are
evaluated
.
Image Processing via Classical Moments
Image Processing via Classical Moments
Central Moment of an Area:
)
,
(
)
(
)
(
y
x
f
y
y
x
x
q
p
y
x
pq
00
10
m
m
x
00
01
m
m
y
Whereas;
and
If
we
calculate
the
central
moments
to
the
third
order
via
using
the
equation
(
1
),
we
have
the
following
results
:
(1)
)
,
(
)
(
)
(
0
1
10
y
x
f
y
y
x
x
y
x
)
,
(
)
(
)
(
1
1
11
y
x
f
y
y
x
x
y
x
Image Processing via Classical Moments
)
,
(
)
(
)
(
0
2
20
y
x
f
y
y
x
x
y
x
)
,
(
)
(
)
(
2
0
02
y
x
f
y
y
x
x
y
x
)
,
(
)
(
)
(
2
1
12
y
x
f
y
y
x
x
y
x
)
,
(
)
(
)
(
2
2
22
y
x
f
y
y
x
x
y
x
)
,
(
)
(
)
(
3
0
03
y
x
f
y
y
x
x
y
x
)
,
(
)
(
)
(
0
3
30
y
x
f
y
y
x
x
y
x
00
pq
pq
1
2
q
p
Denoting
and
as in the following formulas,
Hu moments
can be found.
02
20
1
(2)
2
11
2
02
20
2
4
)
(
2
03
21
2
12
30
3
)
3
(
)
3
(
(3)
(4)
Image Processing via Classical Moments
2
03
21
2
12
30
4
)
(
)
(
2
03
21
2
12
30
03
21
03
21
2
03
21
2
12
03
12
30
12
03
5
3
)
)(
3
(
3
)
)(
3
(
03
21
12
30
11
2
03
21
2
12
30
02
20
6
4
)
(
(5)
(6)
(7)
Six
of
these
invariants
are
invariable
if
the
image
is
mirrored
.
But
the
last
changes
its
sign
.
There
were
no
two
patterns
those
are
the
mirror
of
each
other
.
Thus,
the
first
six
moments
are
used
for
object
recognition
.
Image Processing via Classical Moments
Zernike
Moments
:
Zernike
polynomials,
form
a
complete
orthogonal
set
over
the
interior
of
the
unit
circle
x
2
+y
2
=
1
.
The
Zernike
function
of
order
(
p
,
q
)
is
de
fi
ned
in
the
polar
coordinate
(r,
)
as
:
iq
pq
pq
e
r
R
r
W
)
(
)
,
(
where,
p
even
k
p
q
k
k
pqk
q
p
r
B
r
R


,
,
,
)
(
)!
2
/
)
((
)!
2
/
)
((
)!
2
/
)
((
)!
2
/
)
((
)
1
(
2
/
)
(
q
k
k
q
k
p
k
p
B
k
p
pqk
(8)
(9)
(10)
Image Processing via Classical Moments
Zernike
moments
of
order
p
with
repitation
q
for
a
digital
image
is
the
projections
of
the
image
function
onto
defined
orthogonal
basis
functions
(
8
),(
9
),(
10
)
and
can
be
shown
as
:
y
x
r
W
y
x
f
p
Z
x
y
pq
pq
)
,
(
)
,
(
1
*
Note that
,
)
,
(
)
,
(
,
*
r
W
r
W
q
p
pq
and thus
q
p
pq
Z
Z
,
*
(11)
r
is
the
length
of
the
vector
from
origin
to
pixel
(x
,y
)
and
θ
is
the
angle
between
vector
r
and
the
x

axis
in
the
counter

clockwise
direction
.
x
2
+
y
2
=
1
,
x
=
r
cos
θ
y
=
r
sin
θ
.
Image Processing via Classical Moments
Orthogonal Fourier

Mellin Moments:
r
is
the
length
of
the
vector
from
origin
to
pixel
(x
,y
)
and
θ
is
the
angle
between
vector
r
and
the
x

axis
in
the
counter

clockwise
direction
.
x
2
+
y
2
=
1
,
x
=
r
cos
θ
y
=
r
sin
θ
.
rdrd
e
r
f
r
F
iq
p
pq
2
0
0
)
,
(
where q=0,
±
1,
±
2,...
Then
the
discrete
version
of
the
Fourier

Mellin
Moments
and
Orthogonal
Fourier

Mellin
moments
can
be
defined
as
:
x
y
iq
p
pq
y
x
e
r
y
x
f
F
)
,
(
Image Processing via Classical Moments
x
y
iq
p
pq
y
x
e
r
Q
y
x
f
p
O
)
(
)
,
(
1
The
magnitude
of
Hu
moments,
OFMMs
and
ZMs
are
rotation,
translation
and
scaling
invariant
features
of
the
captured
images
.
In
this
respect,
in
application
of
the
feature
extraction,
identifying
and
recognizing
an
object
in
a
class,
normalization
must
be
made
for
scale
invariance
.
Describing
a
circle
and
investiga
ting
if
the
feature
of
an
object
belongs
to
this
circle
or
not
is
a
way
to
have
rotation
invariance
and
to
define
an
object
in
a
similarity
region
.
Image Processing via Classical Moments
Light Shields
(on both sides)
CCD Camera
(connected to the
grabber card on PC)
Conveyor
Drums
Control Panel
Place where
objects are
passing
Application of Hu moments on a image recognition experimental system:
Image Processing via Classical Moments
Trigerring Line
Boundary Lines
Object Direction
Object
Conveyor Band
CCD Camera
Image Processing via Classical Moments
In this project, four types of image processing
techniques are used:
1. Image Data Reduction
2. Segmentation
3. Feature Extraction
4. Object Recognition
Image Processing via Classical Moments
1
Image
Data
Reduction
a
.
Digital
Conversion
:
It
reduces
the
number
of
gray
levels
used
by
the
machine
vision
system
.
For
example,as
we
have
used
in
the
project,
an
8

bit
register
used
for
each
pixel
would
have
256
gray
levels
.
b
.
Windowing
It
involves
using
only
a
portion
of
the
total
image
stored
in
the
frame
buffer
for
image
processing
and
analysis
.
In
this
project,
we
used
windowing
in
choosing
a
smaller
area
for
better
feature
extraction
.
Image Processing via Classical Moments
2. Segmentation
a. Edge Detection
In
Edge Detection
it is considered that the intensity change
which occurs in the pixels at the boundary or edge of a part.
Given that a region of similar attributes has been found, the
boundary shape is unknown, the boundary can be determined
via a simple edge following procedure.
Canny Edge Detection Algorithym
is used in this project in
order to find the coordinates of the recognized object to handle
by the robot arm.
b. Tresholding
Image Processing via Classical Moments
Some examples of
Canny Edge Detection Algorithym
can be seen
below
Image Processing via Classical Moments
2. Segmentation
a. Edge Detection
b. Tresholding
Tresholding
is a binary conversion technique in which each
pixel is converted into a binary value, either black or white.
This is accomplished by utilizing a frequency histogram of the
image and establishing what intensity(gray level) to be the
border between black and white.
In this project,adaptive tresholding is used to define the object
area and the background area before feature extraction.
Image Processing via Classical Moments
2. Segmentation
a. Edge Detection
b. Tresholding
Max{[((k

j)
2
*h[k])] 
(0<=k<255)}
J: First maximum point,
H[k]= k th histogram value
Image Processing via Classical Moments
Images of some of the objects, tresholded images and histograms
of the real object onthe background can be seen in figure.
Image Processing via Classical Moments
3. Feature Extraction
Hu moments, Legendre moments, ortogonal moments, geometrical
moments, Zernike moments, pseudo

Zernike moments and
Orthogonal Fourier

Mellin moments are some methods for feature
extraction of an object.
In this project, Hu moments are used to extract the features of the
Objects. After calculating 6 features from Hu moments, thay are
grouped as three ones in two groups to show 6 dimensional space as
three dimensioned two spaces in the figures given below.
Image Processing via Classical Moments
4. Object Recognition
Object Recognition process can be defined as labeling the
objects which are modelled before.
In this project, 14 object classes are used as seen in the
figure.
Robot Arm System and Control
Motor # 5
Motor # 4
Motor # 3
Motor # 2
Motor # 1
Link # 1
Link # 2
Link # 3
Link # 4
Gripper
A Robot Arm with Five Degree of Freedom
Robot Arm System and Control
F
irst
,
kinematic
analysis
and
“link
table
“
of
the
revolute
jointed
manipulator
with
five
degrees
of
freedom
is
investigated
.
Furthermore
,
a
path
following
and
control
algorithm
software
for
joint
motors
is
developed
to
move
the
manipulator
through
a
given
trajectory
.
The
body
and
the
joints
of
the
manipulator
are
all
made
from
aluminium
due
to
light
weight
and
toughness
of
this
material
.
For
reducing
the
speed
and
also
to
increase
the
torques
of
the
motors
to
desirable
level,
gear
reduction
mechanisms
have
been
used
for
the
actuators
of
the
joints
.
Robot Arm System and Control
Z
1
X
1
y
2
X
2
z
1
X
3
y
3
z
3
X
4
y
4
z
4
z
5
y
5
X
5
y
1
I
i

1
A
i

1
D
i
I
1
0
0
210
1
2
+90
0
0
2
3
0
202
0
3
4
0
202
0
4
5

90
360
0
5
Second
step
in
forward
kinematics
is
to
construct
a
table
to
show
all
link
parameters
Coordinates Located into the links
Table 1
Figure 1
Robot Arm System and Control
1
0
0
0
210
1
0
0
0
0
0
0
1
1
1
1
0
1
c
s
s
c
T
1
0
0
0
0
0
0
1
0
0
0
0
2
2
21
2
1
2
c
s
s
c
T
T
ransformation
matrices
according
to
table
are,
1
0
0
0
0
1
0
0
0
0
202
0
3
3
3
3
2
3
c
s
s
c
T
1
0
0
0
0
1
0
0
0
0
202
0
4
4
4
4
3
4
c
s
s
c
T
1
0
0
0
0
0
0
1
0
0
360
0
5
5
5
5
4
5
c
s
s
c
T
Robot Arm System and Control
In
deriving
the
kinematical
equations,
we
formed
the
product
of
link
transformations
:
Forming
the
product
to
obtain
;
5
0
1
0
2
1
3
2
4
3
5
4
T
T
T
T
T
T
.
.
.
.
1
0
0
0
33
32
"
3
23
22
21
13
12
11
5
0
z
y
x
P
r
r
r
P
r
r
r
P
r
r
r
T
Robot Arm System and Control
P
x
,
P
y
,
P
z
,
can
be
found
with
respect
to
the
angles
θ
1
,
2
,
3
,
4
After
finding
the
solution
matri
c
es,
program
executes
for
choosing
the
best
solution
for
the
manipulator
control
.
The
criteria
of
choosing
the
best
one
are,
searching
the
easiest
and
shortest
movement
of
the
manipulator
.
In
this
study,
the
aim
is
obtaining
a
path
control
algorithm
by
using
inverse
kinematics
method
.
As
told
before,
by
this
method
manipulator’s
joints
angles
are
determined
from
the
required
target
given
in
Cartesian
coordinates
.
And
after
these
steps
the
main
control
program
starts
to
execute
by
using
joint
base
control
algorithms
.
Robot Arm System and Control
Before
the
path
control
applications,
the
position
following
capability
of
the
manipulator
is
tested
for
each
arm
.
First
of
all,
reconsidering
the
mechanical
design
and
its
workspace
;
the
possible
motion
capabilities
of
the
manipulator
are
investigated
.
Joint
base
d
control
is
used
for
each
joint
by
applying
additional
control
algorithms
.
Responses
of
the
joints
for
different
control
algorithms
are
tested
.
Figure 2
Figure 3
Robot Arm System and Control
While
planning
a
trajectory,
first
two
or
more
points
should
be
decided
.
By
using
inverse
kinematics
method
these
points
can
also
be
defined
by
angular
positions
of
the
arms
.
Figure 4
Figure 5
Robot Arm System and Control
After
defining
these
angles,
the
trajectory
of
a
joint
can
be
described
by
a
polynomial
equation
as
given
below
.
Θ
(t)=a
0
+ a
1
t + a
2
t
2
+ a
3
t
3
and
the
angular
velocity
can
be
defined
as,
Θ
’
(t)= a
1
+ 2 a
2
t + 3 a
3
t
2
.
If
the
coefficients
can
be
determined
correctly
then
the
trajectory
following
will
be
possible
.
After
determining
these
coefficients,
we
can
generate
the
control
algorithm
of
the
manipulator
by
applying
the
time
depended
angular
velocity
equation
.
Robot Arm System and Control
.
As
an
example
the
trajectory
and
parameters
of
the
manipulator
are
given
in
table
below
:
J
0
J
1
J
2
J
4
θ
0
=64.6
0
θ
0
=8.5
0
θ
0
=

5
0
θ
0
=57
0
θ
f
=41.1
0
θ
f
=8.8
0
θ
f
=22.8
0
θ
f
=58
0
D
θ
=23.5
0
D
θ
=

0.3
0
D
θ
=27.8
0
D
θ
=1
0
Dt=6.77 s
Dt=0.05 s
Dt=9.62 s
Dt=0.27 s
Here
D
θ
represents
the
amount
of
the
way
that
the
manipulator
takes
among
this
trajectory,
and
also
D
t
represents
the
time
that
the
joint
needs
for
reaching
the
end
point
.
D
t
can
easily
be
computed
by
the
division
of
the
(D
θ
/
w)
where
w
is
in
(
0
/s)
format
.
Table 2
Robot Arm System and Control
.
By
this
table
,
we
can
easily
determine
the
needed
time
of
the
arms
for
whole
trajectory
.
Since
we
want
all
joints
to
start
and
end
their
motions
at
the
same
time
then
we
have
to
choose
each
joint
a
specific
time
value
.
After
th
at,
we
can
determine
the
angular
velocities
of
the
arms
by
using
the
equations
which
are
formed
via
using
Table
2
.
Figure 6
Path for object A
(in XY plane)
Figure 7
Path for object A
(in XZ plane)
In
this
study
there
are
two
objects
with
their
own
placing
points
after
picking
from
the
convey
o
r
band
.
Thus
there
are
two
paths
determined
for
the
manipulator
and
the
path
following
features
are
observed
for
each
of
them
.
Results
can
be
seen
in
figures
below
.
•
After
theoretical
study,
to
test
the
performance
of
the
presented
algorithym,
an
application
mechanism
is
prepared
as
shown
before
.
During
the
tests,
system
classified
14
predefined
objects
in
500
times
random
taken
photos
of
passing
objects
.
In
these
tests,
the
95
%
of
the
system
decisions
were
right
.
And
it
is
determined
that
most
of
the
reasons
of
the
faults
were
that
the
objects
were
far
from
the
center
of
the
conveyor
band
.
•
Even
there
are
a
little
data
loss
for
the
bigger
objects
in
14
classes,
identification
is
realized
whereas
there
were
some
problems
(mentioned
in
5
%
faults)
for
the
smaller
objects
.
•
In
addition,
when
the
noise
in
reduced
data
of
the
objects
which
are
same
in
color
with
the
conveyor
band
is
homogenous,
it
doesn’t
affect
the
identification
algorithym
;
but
if
the
noise
is
in
the
characteristic
of
damaging
the
region
distrubition
of
any
object,
it
causes
errors
for
the
algorithym
.
•
Improving
the
mechanical
system
of
the
conveyor
system,
performance
of
the
software
can
be
higher
.
Furthermore,
with
a
conveyor
band
that
has
homogenous
reflection
coefficient
distribution
and
is
not
swinging
during
moving
the
percentage
of
right
identification
can
be
improved,
Conclusion
•
It
is
observed
that,
with
its
present
design,
manipulator
is
more
favourable
in
pick
and
place
operations
.
However
,
by
adding
a
prismatic
joint,
manipulator
will
be
more
sensitive
and
smooth
for
two

dimensional
motions
.
•
In
addition,
high
reduction
usage
of
the
motion
motors
due
to
the
limited
sources
has
caused
limited
velocity
adjustments,
where
it
is
necessary
for
smooth
path
control
of
the
manipulator
.
Also
the
result
of
the
low
speed
is
observed
when
joints
need
high
moments
for
path
control
.
It
is
clear
that
usage
of
high
capacity
AC
or
DC
servo

motors
can
prevent
this
problem
.
•
Again
due
to
limited
sources,
multi

turn
potentiometers
are
used
for
the
measurement
of
the
joint
angles
.
And
it
is
observed
that
the
hysterisys
of
these
potentiometers
while
the
arms
moves
forward
and
backward
s
affect
the
smoothness
and
the
repeatability
features
of
the
manipulator
in
negative
way
.
Conclusion
Σχόλια 0
Συνδεθείτε για να κοινοποιήσετε σχόλιο