Robotics
Chapter 6
–
Machine Vision
Dr. Amit Goradia
Topics
•
Introduction
–
2 hrs
•
Coordinate transformations
–
6 hrs
•
Forward Kinematics

6 hrs
•
Inverse Kinematics

6 hrs
•
Velocity Kinematics

2 hrs
•
Trajectory Planning

6 hrs
•
Robot Dynamics (Introduction)

2 hrs
•
Force Control (Introduction)

1 hrs
•
Task Planning

6 hrs
•
Machine Vision

6 hrs
Machine Vision Objectives
•
To recover useful information about a
scene from its 2

D projections.
•
To take images as inputs and produce
other types of outputs (object shape,
object contour, etc.)
•
Geometry + Measurement + Interpretation
•
To create a model of the real world from
images.
Related Fields
•
Image processing
–
Transformation of images into other images
–
Image compression, image enhancement
–
Useful in early stages of a machine vision
system
•
Computer graphics
•
Pattern recognition
•
Artificial intelligence
Vision System Hardware
Image Representation
•
Images are represented by a 2D intensity
function f (x,y) where:
–
x,y represent spatial coordinates
–
Value of f is proportional to the brightness (grey level)
of the image.
•
A digital image is represented by a matrix u(m,n)
whole rows and columns represent a point in
space and the element value represents the
grey level of the image.
•
Each point is referred to as a pixel.
Image Representation
•
Typical grey levels stored in powers of 2 (computer
representation)
•
The indices [i, j] of pixels : integer values that specify the
rows and columns in pixel values
–
8

bit image can
represent 256 shades of grey.
Sampling and Quantization
•
Sampling
–
The real image is sampled at a finite number of points.
–
Sampling rate : image resolution
•
how many pixels the digital image will have
•
e.g.) 640 x 480, 320 x 240, etc.
•
Pixel
–
Each image sample
–
At the sample point, an integer value of the image intensity
•
Quantization
–
Each sample is represented with the finite word size of the
computer.
–
How many intensity levels can be used to represent the intensity
value at each sample point.
–
e.g.) 2
8
= 256, 2
5
= 32, etc
Euclidean V/s Projective
Geometry
•
Euclidean geometry describes shapes as they
are
–
Properties of objects are unchanged by rigid motions
–
Preserves: length, angles, parallelism
•
Projective Geomerty describes objects as they
appear
–
Lengths, angles, parallelism become distorted
–
Provides a mathematical model for how objects
appear in 3D.
Projective Geometry
Center of projection (COP)
–
center of the camera lenses
origin of the camera frame
Direction of Projection (DOP)
–
viewing from infinity
Planar geometric projections
Non

planar projections
Classical Views
Orthographic Projections
•
Multi

view orthographic
projection
•
Preserves angles
•
3 orthogonal views
Perspective Projections
•
Vanishing Points
–
One point perspective
–
Two point perspective
–
Three point perspective
Projective Transforms in a Plane
•
Projectivity
–
Mapping of points in a plane to points in a
plane
–
3 aligned points mapped to 3 aligned points
•
Also called
–
Homography
–
Collineation
•
Represented by a 3x3 matrix multiplication
Projective Transformations
•
Definition: A
projectivity
(Homography) is
an invertible mapping h from P2 to itself
such that three points x1,x2,x3 lie on the
same line if and only if
h
(x1),
h
(x2),
h
(x3)
do.
Special Projective Transformations
Perspective Projection
•
Similar triangles approach to camera
modeling
Perspective Projection Equations
•
3d world mapped to 2d projection in image plane
Scene point
Image coordinates
Perspective Projection Matrix
•
Projection is a matrix multiplication using
homogeneous coordinates
Intrinsic Camera Parameters
f
Z
X
O
Intrinsic Camera Parameters
•
Intrinsic
Parameters:
–
Focal Length
f
–
Pixel size
s
x
,
s
y
–
Image center
o
x
,
o
y
–
(Nonlinear radial distortion coefficients
k
1
,
k
2
…)
•
Calibration = Determine the intrinsic
parameters of a camera
Importance of Intrinsic Parameters
Radial Distortion Models
•
Types of distortion
–
Pincushion distortion
–
Barrel distortion
–
Tangential distortion
•
Distortion models
distance from center
Extrinsic Parameters
•
Location of the camera origin with respect
to the world frame.
Camera Calibration
•
Find the intrinsic and extrinsic parameters
•
Using calibration objects of known size
and geometry
•
Can be solved by optimization techniques.
Image Segmentation
•
The purpose of image segmentation is to
partition an image into
meaningful
regions with
respect to a particular application
•
The segmentation is based on measurements
taken from the image and might be
greylevel
,
colour
,
texture
,
depth
or
motion
•
Segmentation Techniques
–
Thresholding
–
Clustering
–
Region based
–
Edge based
–
Model based
–
Watershed approach
Greylevel Histogram

Based
Segmentation
•
We will look at two very simple image
segmentation techniques that are based
on the greylevel histogram of an image
–
Thresholding
–
Clustering
•
We will use a very simple object

background test image
–
We will consider a zero, low and high noise
image
27
28
Noise free
Low noise
High noise
Greylevel Histogram

Based
Segmentation
•
How do we characterise low noise and high
noise?
•
We can consider the histograms of our images
–
For the noise free image, its simply two spikes at
i
=100,
i=150
–
For the low noise image, there are two clear peaks
centred on
i
=100,
i=150
–
For the high noise image, there is a single peak
–
two
greylevel populations corresponding to object and
background have merged
29
Greylevel Histogram

Based
Segmentation
30
Greylevel Histogram

Based
Segmentation
Greylevel Thresholding
•
Clear ‘valley’ present
between to two peaks
•
Picking threshold is
the hard part:
–
Human operator
decided the threshold
–
Use mean gray level
of the image
–
A fixed proportion of
pixels are detected (
set to 1) by the
thresholding operation
–
Analyzing the
histogram of an image
Greylevel Thresholding
•
Simple algorithm
–
If
the
greylevel
of
pixel
p
<=T
then
pixel
p
is
an
object
pixel
–
else
Pixel
p
is
a
background
pixel
Object and Shape Recognition
•
Typically, recognition is applied after
objects have been detected.
•
Detection:
where
is the object?
•
Recognition:
what
is the object?
•
Note that, to detect an object, we already
need to know what
type
of object it is.
•
Recognition further refines the type.
Examples
•
Hands:
–
Where is the hand?
–
What is the shape and orientation of the
hand?
Examples
•
Letters and numbers.
–
Detection: where is the number?
–
Recognition: what number is it?
Recognition Steps
•
Training phase:
–
Build models of each class.
•
Test phase:
–
Find the model that best matches the image.
•
Model: a single structure (but as complicated as
needed) that describes how patterns of a class
are expected to be.
•
Types of models:
–
Geometric moments
–
PCA (principle component analysis) based models
–
Many more…
Image Moments
•
Computed on
binary
images.
–
Pixels are assumed to have values 0 or 1.
•
Raw moments:
•
Interpretation:
–
M
00
is what?
Image Moments
•
Computed on
binary
images.
–
Pixels are assumed to have values 0 or 1.
•
Raw moments:
•
Interpretation:
–
M
00
is the area of the shape.
Image Moments
•
Computed on
binary
images.
–
Pixels are assumed to have values 0 or 1.
•
Raw moments:
•
Interpretation:
–
M
00
is the area of the shape.
–
How can we express the center of the shape
in terms of moments?
Image Moments
•
Computed on
binary
images.
–
Pixels are assumed to have values 0 or 1.
•
Raw moments:
•
Interpretation:
–
M
00
is the area of the shape.
–
Defining the center of the shape using
moments:
•
[yc, xc] = [M
01
/M
00
, M
10
/M
00
].
Central Moments
•
Like raw moments, but computed with
respect to the
centroid
of the shape.
•
What is mu
00
?
Central Moments
•
Like raw moments, but computed with
respect to the
centroid
of the shape.
•
What is mu
00
?
–
mu
00
= the area of the shape.
•
mu
10
= ?
Central Moments
•
Like raw moments, but computed with
respect to the
centroid
of the shape.
•
What is mu
00
?
–
mu
00
= the area of the shape. What is mu
00
?
•
mu
10
is the x coordinate of the centroid of
the shape,
in a coordinate system whose
origin is that centroid.
So, mu
10
= mu
01
= 0.
Central Moments
Image 1
Image 2
•
Will the raw moments be equal?
•
Will the central moments be equal?
Central Moments
Image 1
Image 2
•
Will the raw moments be equal? No.
•
Will the central moments be equal? Yes.
Central Moments
Image 1
Image 2
•
Central moments are
translation invariant.
Central Moments
Image 1
Image 2
•
How can we make these moments
translation and scale invariant
?
Normalized Central Moments
Image 1
Image 2
•
Normalized central moments are
translation
and scale invariant.
For i+j >= 2:
For i = 1, j = 0, or i = 0, j = 1:
Just divide by shape area.
Hu Moments
Image 1
Image 2
Translation, scale, and
rotation invariant.
Using Moments
•
Good for recognizing shapes that have been
segmented very well
.
•
Each image can be represented as a
vector
of moments
. E.g., a vector of 7 Hu moments.
•
Can easily be used as part of:
–
Model

based recognition methods.
•
Boosting, or PCA, can be applied on top of these
vectors.
–
Nearest neighbor classification methods.
•
Using Hu moments, recognition is
translation, scale, and orientation

invariant.
Comments 0
Log in to post a comment