# Chapter 6 Machine Vision

Τεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 4 χρόνια και 6 μήνες)

74 εμφανίσεις

Robotics

Chapter 6

Machine Vision

Topics

Introduction

2 hrs

Coordinate transformations

6 hrs

Forward Kinematics
-

6 hrs

Inverse Kinematics
-

6 hrs

Velocity Kinematics
-

2 hrs

Trajectory Planning
-

6 hrs

Robot Dynamics (Introduction)
-

2 hrs

Force Control (Introduction)
-

1 hrs

-

6 hrs

Machine Vision
-

6 hrs

Machine Vision Objectives

To recover useful information about a
scene from its 2
-
D projections.

To take images as inputs and produce
other types of outputs (object shape,
object contour, etc.)

Geometry + Measurement + Interpretation

To create a model of the real world from
images.

Related Fields

Image processing

Transformation of images into other images

Image compression, image enhancement

Useful in early stages of a machine vision
system

Computer graphics

Pattern recognition

Artificial intelligence

Vision System Hardware

Image Representation

Images are represented by a 2D intensity
function f (x,y) where:

x,y represent spatial coordinates

Value of f is proportional to the brightness (grey level)
of the image.

A digital image is represented by a matrix u(m,n)
whole rows and columns represent a point in
space and the element value represents the
grey level of the image.

Each point is referred to as a pixel.

Image Representation

Typical grey levels stored in powers of 2 (computer
representation)

The indices [i, j] of pixels : integer values that specify the
rows and columns in pixel values

8
-
bit image can

Sampling and Quantization

Sampling

The real image is sampled at a finite number of points.

Sampling rate : image resolution

how many pixels the digital image will have

e.g.) 640 x 480, 320 x 240, etc.

Pixel

Each image sample

At the sample point, an integer value of the image intensity

Quantization

Each sample is represented with the finite word size of the
computer.

How many intensity levels can be used to represent the intensity
value at each sample point.

e.g.) 2
8

= 256, 2
5

= 32, etc

Euclidean V/s Projective
Geometry

Euclidean geometry describes shapes as they
are

Properties of objects are unchanged by rigid motions

Preserves: length, angles, parallelism

Projective Geomerty describes objects as they
appear

Lengths, angles, parallelism become distorted

Provides a mathematical model for how objects
appear in 3D.

Projective Geometry

Center of projection (COP)

center of the camera lenses

origin of the camera frame

Direction of Projection (DOP)

viewing from infinity

Planar geometric projections

Non
-
planar projections

Classical Views

Orthographic Projections

Multi
-
view orthographic
projection

Preserves angles

3 orthogonal views

Perspective Projections

Vanishing Points

One point perspective

Two point perspective

Three point perspective

Projective Transforms in a Plane

Projectivity

Mapping of points in a plane to points in a
plane

3 aligned points mapped to 3 aligned points

Also called

Homography

Collineation

Represented by a 3x3 matrix multiplication

Projective Transformations

Definition: A
projectivity

(Homography) is
an invertible mapping h from P2 to itself
such that three points x1,x2,x3 lie on the
same line if and only if
h
(x1),
h
(x2),
h
(x3)
do.

Special Projective Transformations

Perspective Projection

Similar triangles approach to camera
modeling

Perspective Projection Equations

3d world mapped to 2d projection in image plane

Scene point

Image coordinates

Perspective Projection Matrix

Projection is a matrix multiplication using
homogeneous coordinates

Intrinsic Camera Parameters

f

Z

X

O

Intrinsic Camera Parameters

Intrinsic

Parameters:

Focal Length
f

Pixel size
s
x
,

s
y

Image center
o
x
,

o
y

k
1
,

k
2
…)

Calibration = Determine the intrinsic
parameters of a camera

Importance of Intrinsic Parameters

Types of distortion

Pincushion distortion

Barrel distortion

Tangential distortion

Distortion models

distance from center

Extrinsic Parameters

Location of the camera origin with respect
to the world frame.

Camera Calibration

Find the intrinsic and extrinsic parameters

Using calibration objects of known size
and geometry

Can be solved by optimization techniques.

Image Segmentation

The purpose of image segmentation is to
partition an image into
meaningful

regions with
respect to a particular application

The segmentation is based on measurements
taken from the image and might be
greylevel
,
colour
,
texture
,
depth

or
motion

Segmentation Techniques

Thresholding

Clustering

Region based

Edge based

Model based

Watershed approach

Greylevel Histogram
-
Based
Segmentation

We will look at two very simple image
segmentation techniques that are based
on the greylevel histogram of an image

Thresholding

Clustering

We will use a very simple object
-
background test image

We will consider a zero, low and high noise
image

27

28

Noise free

Low noise

High noise

Greylevel Histogram
-
Based
Segmentation

How do we characterise low noise and high
noise?

We can consider the histograms of our images

For the noise free image, its simply two spikes at
i
=100,
i=150

For the low noise image, there are two clear peaks
centred on

i
=100,
i=150

For the high noise image, there is a single peak

two
greylevel populations corresponding to object and
background have merged

29

Greylevel Histogram
-
Based
Segmentation

30

Greylevel Histogram
-
Based
Segmentation

Greylevel Thresholding

Clear ‘valley’ present
between to two peaks

Picking threshold is
the hard part:

Human operator
decided the threshold

Use mean gray level
of the image

A fixed proportion of
pixels are detected (
set to 1) by the
thresholding operation

Analyzing the
histogram of an image

Greylevel Thresholding

Simple algorithm

If

the

greylevel

of

pixel

p

<=T

then

pixel

p

is

an

object

pixel

else

Pixel

p

is

a

background

pixel

Object and Shape Recognition

Typically, recognition is applied after
objects have been detected.

Detection:
where

is the object?

Recognition:
what

is the object?

Note that, to detect an object, we already
need to know what
type

of object it is.

Recognition further refines the type.

Examples

Hands:

Where is the hand?

What is the shape and orientation of the
hand?

Examples

Letters and numbers.

Detection: where is the number?

Recognition: what number is it?

Recognition Steps

Training phase:

Build models of each class.

Test phase:

Find the model that best matches the image.

Model: a single structure (but as complicated as
needed) that describes how patterns of a class
are expected to be.

Types of models:

Geometric moments

PCA (principle component analysis) based models

Many more…

Image Moments

Computed on
binary

images.

Pixels are assumed to have values 0 or 1.

Raw moments:

Interpretation:

M
00

is what?

Image Moments

Computed on
binary

images.

Pixels are assumed to have values 0 or 1.

Raw moments:

Interpretation:

M
00

is the area of the shape.

Image Moments

Computed on
binary

images.

Pixels are assumed to have values 0 or 1.

Raw moments:

Interpretation:

M
00

is the area of the shape.

How can we express the center of the shape
in terms of moments?

Image Moments

Computed on
binary

images.

Pixels are assumed to have values 0 or 1.

Raw moments:

Interpretation:

M
00

is the area of the shape.

Defining the center of the shape using
moments:

[yc, xc] = [M
01
/M
00
, M
10
/M
00
].

Central Moments

Like raw moments, but computed with
respect to the
centroid

of the shape.

What is mu
00
?

Central Moments

Like raw moments, but computed with
respect to the
centroid

of the shape.

What is mu
00
?

mu
00

= the area of the shape.

mu
10

= ?

Central Moments

Like raw moments, but computed with
respect to the
centroid

of the shape.

What is mu
00
?

mu
00

= the area of the shape. What is mu
00
?

mu
10

is the x coordinate of the centroid of
the shape,
in a coordinate system whose
origin is that centroid.

So, mu
10

= mu
01

= 0.

Central Moments

Image 1

Image 2

Will the raw moments be equal?

Will the central moments be equal?

Central Moments

Image 1

Image 2

Will the raw moments be equal? No.

Will the central moments be equal? Yes.

Central Moments

Image 1

Image 2

Central moments are
translation invariant.

Central Moments

Image 1

Image 2

How can we make these moments
translation and scale invariant
?

Normalized Central Moments

Image 1

Image 2

Normalized central moments are
translation
and scale invariant.

For i+j >= 2:

For i = 1, j = 0, or i = 0, j = 1:

Just divide by shape area.

Hu Moments

Image 1

Image 2

Translation, scale, and

rotation invariant.

Using Moments

Good for recognizing shapes that have been
segmented very well
.

Each image can be represented as a
vector
of moments
. E.g., a vector of 7 Hu moments.

Can easily be used as part of:

Model
-
based recognition methods.

Boosting, or PCA, can be applied on top of these
vectors.

Nearest neighbor classification methods.

Using Hu moments, recognition is
translation, scale, and orientation
-
invariant.