Chapter 6 Machine Vision

jabgoldfishAI and Robotics

Oct 19, 2013 (3 years and 5 months ago)

52 views

Robotics

Chapter 6


Machine Vision



Dr. Amit Goradia

Topics


Introduction







2 hrs


Coordinate transformations




6 hrs


Forward Kinematics
-




6 hrs


Inverse Kinematics
-




6 hrs


Velocity Kinematics
-




2 hrs


Trajectory Planning
-




6 hrs


Robot Dynamics (Introduction)
-


2 hrs


Force Control (Introduction)
-


1 hrs


Task Planning
-





6 hrs


Machine Vision
-





6 hrs

Machine Vision Objectives


To recover useful information about a
scene from its 2
-
D projections.


To take images as inputs and produce
other types of outputs (object shape,
object contour, etc.)


Geometry + Measurement + Interpretation


To create a model of the real world from
images.

Related Fields


Image processing


Transformation of images into other images


Image compression, image enhancement


Useful in early stages of a machine vision
system


Computer graphics


Pattern recognition


Artificial intelligence

Vision System Hardware

Image Representation


Images are represented by a 2D intensity
function f (x,y) where:


x,y represent spatial coordinates


Value of f is proportional to the brightness (grey level)
of the image.


A digital image is represented by a matrix u(m,n)
whole rows and columns represent a point in
space and the element value represents the
grey level of the image.


Each point is referred to as a pixel.

Image Representation


Typical grey levels stored in powers of 2 (computer
representation)


The indices [i, j] of pixels : integer values that specify the
rows and columns in pixel values


8
-
bit image can
represent 256 shades of grey.

Sampling and Quantization


Sampling


The real image is sampled at a finite number of points.


Sampling rate : image resolution


how many pixels the digital image will have


e.g.) 640 x 480, 320 x 240, etc.


Pixel


Each image sample


At the sample point, an integer value of the image intensity



Quantization


Each sample is represented with the finite word size of the
computer.


How many intensity levels can be used to represent the intensity
value at each sample point.


e.g.) 2
8

= 256, 2
5

= 32, etc

Euclidean V/s Projective
Geometry


Euclidean geometry describes shapes as they
are


Properties of objects are unchanged by rigid motions


Preserves: length, angles, parallelism


Projective Geomerty describes objects as they
appear


Lengths, angles, parallelism become distorted


Provides a mathematical model for how objects
appear in 3D.

Projective Geometry

Center of projection (COP)


center of the camera lenses

origin of the camera frame

Direction of Projection (DOP)


viewing from infinity

Planar geometric projections

Non
-
planar projections

Classical Views


Orthographic Projections


Multi
-
view orthographic
projection


Preserves angles


3 orthogonal views

Perspective Projections


Vanishing Points


One point perspective


Two point perspective


Three point perspective

Projective Transforms in a Plane


Projectivity


Mapping of points in a plane to points in a
plane


3 aligned points mapped to 3 aligned points


Also called


Homography


Collineation


Represented by a 3x3 matrix multiplication

Projective Transformations


Definition: A
projectivity

(Homography) is
an invertible mapping h from P2 to itself
such that three points x1,x2,x3 lie on the
same line if and only if
h
(x1),
h
(x2),
h
(x3)
do.


Special Projective Transformations


Perspective Projection


Similar triangles approach to camera
modeling

Perspective Projection Equations


3d world mapped to 2d projection in image plane

Scene point

Image coordinates

Perspective Projection Matrix


Projection is a matrix multiplication using
homogeneous coordinates

Intrinsic Camera Parameters


f

Z

X

O

Intrinsic Camera Parameters


Intrinsic

Parameters:


Focal Length
f


Pixel size
s
x
,

s
y


Image center
o
x
,

o
y


(Nonlinear radial distortion coefficients
k
1
,

k
2
…)


Calibration = Determine the intrinsic
parameters of a camera

Importance of Intrinsic Parameters

Radial Distortion Models


Types of distortion


Pincushion distortion


Barrel distortion


Tangential distortion


Distortion models

distance from center

Extrinsic Parameters


Location of the camera origin with respect
to the world frame.

Camera Calibration


Find the intrinsic and extrinsic parameters


Using calibration objects of known size
and geometry


Can be solved by optimization techniques.

Image Segmentation


The purpose of image segmentation is to
partition an image into
meaningful

regions with
respect to a particular application



The segmentation is based on measurements
taken from the image and might be
greylevel
,
colour
,
texture
,
depth

or
motion



Segmentation Techniques


Thresholding


Clustering



Region based



Edge based



Model based



Watershed approach

Greylevel Histogram
-
Based
Segmentation


We will look at two very simple image
segmentation techniques that are based
on the greylevel histogram of an image


Thresholding


Clustering


We will use a very simple object
-
background test image


We will consider a zero, low and high noise
image

27

28

Noise free

Low noise

High noise

Greylevel Histogram
-
Based
Segmentation


How do we characterise low noise and high
noise?


We can consider the histograms of our images


For the noise free image, its simply two spikes at
i
=100,
i=150


For the low noise image, there are two clear peaks
centred on

i
=100,
i=150


For the high noise image, there is a single peak


two
greylevel populations corresponding to object and
background have merged

29

Greylevel Histogram
-
Based
Segmentation

30

Greylevel Histogram
-
Based
Segmentation

Greylevel Thresholding


Clear ‘valley’ present
between to two peaks




Picking threshold is
the hard part:


Human operator
decided the threshold



Use mean gray level
of the image



A fixed proportion of
pixels are detected (
set to 1) by the
thresholding operation


Analyzing the
histogram of an image

Greylevel Thresholding


Simple algorithm


If

the

greylevel

of

pixel

p

<=T

then

pixel

p

is

an

object

pixel


else

Pixel

p

is

a

background

pixel



Object and Shape Recognition


Typically, recognition is applied after
objects have been detected.


Detection:
where

is the object?


Recognition:
what

is the object?


Note that, to detect an object, we already
need to know what
type

of object it is.


Recognition further refines the type.

Examples


Hands:


Where is the hand?


What is the shape and orientation of the
hand?

Examples


Letters and numbers.


Detection: where is the number?


Recognition: what number is it?

Recognition Steps


Training phase:


Build models of each class.


Test phase:


Find the model that best matches the image.


Model: a single structure (but as complicated as
needed) that describes how patterns of a class
are expected to be.


Types of models:


Geometric moments


PCA (principle component analysis) based models


Many more…

Image Moments


Computed on
binary

images.


Pixels are assumed to have values 0 or 1.


Raw moments:





Interpretation:


M
00

is what?


Image Moments


Computed on
binary

images.


Pixels are assumed to have values 0 or 1.


Raw moments:





Interpretation:


M
00

is the area of the shape.

Image Moments


Computed on
binary

images.


Pixels are assumed to have values 0 or 1.


Raw moments:





Interpretation:


M
00

is the area of the shape.


How can we express the center of the shape
in terms of moments?

Image Moments


Computed on
binary

images.


Pixels are assumed to have values 0 or 1.


Raw moments:




Interpretation:


M
00

is the area of the shape.


Defining the center of the shape using
moments:



[yc, xc] = [M
01
/M
00
, M
10
/M
00
].

Central Moments


Like raw moments, but computed with
respect to the
centroid

of the shape.


What is mu
00
?

Central Moments


Like raw moments, but computed with
respect to the
centroid

of the shape.


What is mu
00
?


mu
00

= the area of the shape.


mu
10

= ?


Central Moments


Like raw moments, but computed with
respect to the
centroid

of the shape.


What is mu
00
?


mu
00

= the area of the shape. What is mu
00
?


mu
10

is the x coordinate of the centroid of
the shape,
in a coordinate system whose
origin is that centroid.

So, mu
10

= mu
01

= 0.

Central Moments

Image 1

Image 2


Will the raw moments be equal?


Will the central moments be equal?

Central Moments

Image 1

Image 2


Will the raw moments be equal? No.


Will the central moments be equal? Yes.

Central Moments

Image 1

Image 2


Central moments are
translation invariant.

Central Moments

Image 1

Image 2


How can we make these moments
translation and scale invariant
?

Normalized Central Moments

Image 1

Image 2


Normalized central moments are
translation
and scale invariant.

For i+j >= 2:

For i = 1, j = 0, or i = 0, j = 1:

Just divide by shape area.

Hu Moments

Image 1

Image 2

Translation, scale, and

rotation invariant.

Using Moments


Good for recognizing shapes that have been
segmented very well
.


Each image can be represented as a
vector
of moments
. E.g., a vector of 7 Hu moments.


Can easily be used as part of:


Model
-
based recognition methods.


Boosting, or PCA, can be applied on top of these
vectors.


Nearest neighbor classification methods.


Using Hu moments, recognition is
translation, scale, and orientation
-
invariant.