The University of
Ontario
CS
4487/9587
Algorithms for Image Analysis
Image
Processing Basics
Lecture 3
Lena Gorelick, substituting for Yuri
Boykov
Acknowledgements: slides from Steven Seitz,
Aleosha
Efros
, David Forsyth, and Gonzalez & Woods
The University of
Ontario
CS
4487/9587
Algorithms for Image Analysis
Image Processing Basics
Image Processing
•
Domain Transformation
–
Resize, Rotate etc.
•
Range Transformation
–
Point
Processing
•
gamma correction
•
window

center correction
•
histogram equalization
)
,
(
)
,
(
))
,
(
),
,
(
(
)
,
(
y
N
x
f
y
x
g
y
x
t
y
x
t
f
y
x
g
y
x
2
/
)
,
(
)
,
(
))
,
(
(
)
,
(
,
y
x
f
y
x
g
y
x
f
t
y
x
g
y
x
Window = 800
Center = 1160
Pixel location
Pixel Intensity
The University of
Ontario
CS
4487/9587
Algorithms for Image Analysis
Image Processing Basics
Image Processing
•
Domain Transformation
–
Resize, Rotate etc.
•
Range Transformation
–
Point
Processing
•
gamma correction
•
window

center correction
•
histogram equalization
–
Neighborhood Processing (Filtering)
)
2
,
2
(
)
,
(
))
,
(
),
,
(
(
)
,
(
y
x
f
y
x
g
y
x
t
y
x
t
f
y
x
g
y
x
2
/
)
,
(
)
,
(
))
,
(
(
)
,
(
,
y
x
f
y
x
g
y
x
f
t
y
x
g
y
x
The University of
Ontario
Neighborhood Processing (or filtering)
What is wrong with this image?
How can we remove noise?
•
Look spatially around each pixel

neighborhood
Can we use Point Processing?
Readings: Forsyth & Ponce, chapters 8.1

8.2
Courtesy of Carlo
Tomasi
Courtesy of Neel Joshi
The University of
Ontario
Neighborhood Processing (or filtering)
Lets reshuffle
all pixels within the
image
Point Processing will have the same effect
•
Gets intensity of a single pixel as an input
•
Unaware of spatial information
Neighborhood Processing
•
Takes “spatial information” into account
•
Images contain “spatial information”
Readings: Forsyth & Ponce, chapters 8.1

8.2
Has Spatial Information
Looks noisy
The University of
Ontario
Neighborhood Processing (filtering)
Linear image transforms
Assume 1D function
]
[
i
f
0
1
0
0
0
0
1
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
1
0
0
0
0
1
0
0
0
0
M
j
j
f
j
i
M
i
g
f
M
g
]
[
]
,
[
]
[
?
The University of
Ontario
Neighborhood Processing (filtering)
Linear image transforms
Assume 1D function
]
[
i
f
1
1
0
0
0
1
1
0
0
0
1
1
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
1
0
0
0
1
1
0
0
0
1
1
0
0
0
2
2
/
1
M
j
j
f
j
i
M
i
g
f
M
g
]
[
]
,
[
]
[
?
The University of
Ontario
A commonly used pattern
Represented by a kernel
(or a mask)
For 2k+1 size kernel
(
i

1)
Neighborhood Processing (filtering)
Linear shift

invariant filters
matrix M
]
[
c
b
a
h
*
*
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
*
*
b
a
c
b
a
c
b
a
c
b
a
c
b
a
c
b
a
M
k
k
u
u
i
f
u
h
i
g
f
M
g
]
[
]
[
]
[
i
(
i
+ 1)
The University of
Ontario
A 2D function
Can be filtered by a 2D filter (kernel)
To get a new image
This is called a
cross

correlation
operation
Or a
sliding dot

product
Neighborhood Processing (filtering)
2D filtering
f
h
g
k
k
u
k
k
v
v
j
u
i
f
v
u
h
j
i
g
]
,
[
]
,
[
]
,
[
]
,
[
j
i
f
]
,
[
v
u
h
The University of
Ontario
Cross

correlation in which the filter is flipped
horizontally and vertically
is called
convolution
Neighborhood Processing (filtering)
2D filtering
f
h
g
*
k
k
u
k
k
v
v
j
u
i
f
v
u
h
]
,
[
]
,
[
k
k
u
k
k
v
v
j
u
i
f
v
u
h
j
i
g
]
,
[
]
,
[
]
,
[
The University of
Ontario
If the kernel is symmetric
convolution = cross

correlation
Neighborhood Processing (filtering)
Convolution vs. Cross

Correlation
)
,
(
)
,
(
v
u
h
v
u
h
The University of
Ontario
Types of noise:
•
Salt
and pepper
noise
•
Impulse noise
•
Gaussian noise
Due to
•
transmission errors
•
dead CCD pixels
•
specks on lens
•
can be specific to a sensor
Neighborhood Processing
2D filtering
Noise
The University of
Ontario
How can we
remove noise?
Replace each pixel with
the average of a
k
x
k
window around it
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
100
130
110
120
110
0
0
0
0
0
110
90
100
90
100
0
0
0
0
0
130
100
90
130
110
0
0
0
0
0
120
100
130
110
120
0
0
0
0
0
90
110
80
120
100
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Neighborhood Processing
Practical Noise Reduction
104
The University of
Ontario
Neighborhood Processing (filtering)
Mean filtering
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
0
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
]
,
[
y
x
f
]
,
[
y
x
g
The University of
Ontario
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
0
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Neighborhood Processing (filtering)
Mean filtering
0
10
20
30
30
30
20
10
0
20
40
60
60
60
40
20
0
30
60
90
90
90
60
30
0
30
50
80
80
90
60
30
0
30
50
80
80
90
60
30
0
20
30
50
50
60
40
20
10
20
30
30
30
30
20
10
10
10
10
0
0
0
0
0
]
,
[
y
x
f
]
,
[
y
x
g
The University of
Ontario
Neighborhood Processing (filtering)
Effect
of
mean
filters
3x3
5x5
7x7
Salt & Pepper noise
Gaussian noise
Side effect

blur
The University of
Ontario
Neighborhood Processing (filtering)
Mean kernel
What’s the kernel for a 3x3
mean
filter?
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
0
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
]
,
[
y
x
f
]
,
[
v
u
h
The University of
Ontario
Neighborhood Processing (filtering)
Mean kernel
What’s the kernel for a 3x3
mean
filter?
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
0
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
]
,
[
y
x
f
]
,
[
v
u
h
The University of
Ontario
Neighborhood Processing (filtering)
Mean kernel
What’s the kernel for a 3x3
mean
filter
?
Equal weight to all pixels
within the neighborhood
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
0
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
1
1
1
1
1
1
1
1
]
,
[
y
x
f
]
,
[
v
u
h
1/9
The University of
Ontario
Neighborhood Processing (filtering)
Gaussian Filtering
A
Gaussian kernel
gives less weight to pixels
further from the center
of the window
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
90
0
90
90
90
0
0
0
0
0
90
90
90
90
90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
is a discrete approximation
of a Gaussian function:
1
2
1
2
4
2
1
2
1
]
,
[
y
x
f
]
,
[
v
u
h
16
1
The University of
Ontario
Neighborhood Processing (filtering)
Mean vs. Gaussian filtering
Input image
Gaussian filtering
Mean Filtering
Gaussian filter, 20x20,
σ
= 5
Mean filter, 20x20
The University of
Ontario
Neighborhood Processing (filtering)
Mean vs. Gaussian filtering
The University of
Ontario
Neighborhood Processing (filtering)
Gaussian Filtering
Low

pass filter
Smooth color variation (low
frequency) is preserved
Sharp edges (high frequency) are
removed
The University of
Ontario
Neighborhood Processing (filtering)
Median filters
A
Median Filter
operates over a window by
selecting the median intensity in the window.
Image credit: Wikipedia
–
page
on Median Filter
The University of
Ontario
Neighborhood Processing (filtering)
Median filters
Is
a median filter a kind of
convolution?
No, median filter is an example of non

linear
filtering
))
(
(
))
(
(
))
(
)
(
(
2
1
2
1
x
f
Median
x
f
Median
x
f
x
f
Median
2
1
1
)
0
1
0
1
2
1
0
1
0
(
)
1
0
1
0
1
0
1
0
1
(
Median
Median
)
0
1
0
1
2
1
0
1
0
1
0
1
0
1
0
1
0
1
(
)
1
1
1
1
3
1
1
1
1
(
1
Median
Median
The University of
Ontario
Salt&Pepper
Noise
The University of
Ontario
Gaussian
noise
The University of
Ontario
Neighborhood Processing (filtering)
Median filters
What
advantage does a
median filter
have
over a
mean filter
?
Better at removing Salt & Pepper noise
Disadvantage:
Slow
The University of
Ontario
Neighborhood Processing (filtering)
Derivatives and Convolution
Image Derivatives and Gradients
•
Used for Edge/Corner Detection
•
Computed with Finite Differences Filters
Laplacian
of Gaussians (
LoG
) Filter
•
Used for Edge/Blob Detection and
Image Enhancement
•
Approximated using Difference of Gaussians
The University of
Ontario
Reading: Forsyth & Ponce, 8.1

8.2
First Derivative
Recall
Sharp changes in gray level of the input
image correspond to “peaks or valleys” of the
first

derivative of the input signal.
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
The University of
Ontario
How can we approximate it
for a discrete function?
Is this operation shift

invariant?
Is it linear?
Reading: Forsyth & Ponce, 8.1

8.2
First Derivative and
convolution
h
y
x
f
y
h
x
f
f
h
x
)
,
(
)
,
(
lim
0
?
?
?
?
?
?
?
?
?
]
,
[
v
u
x
The University of
Ontario
Reading: Forsyth & Ponce, 8.1

8.2
First Derivative
and convolution
Finite Difference
x
x

h
x
x+h
x
x+h
/2
x

h/2
)
(
'
)
(
)
(
0
]
[
x
f
h
x
f
h
x
f
h
h
h
f
)
(
)
(
b
x
f
a
x
f
Slide credit: Wikipedia
The University of
Ontario
Reading: Forsyth & Ponce, 8.1

8.2
First Derivative
and convolution
Finite Difference
–
The order of an error
can be derived using Taylor Theorem
Slide credit: Wikipedia
The University of
Ontario
Reading: Forsyth & Ponce, 8.1

8.2
First Derivative
and convolution
Pixel Size
Using Finite Central Difference
We need a kernel such that
x
y
x
x
f
y
x
x
f
f
x
2
)
,
(
)
,
(
f
f
x
x
0
0
0
1
0

1
0
0
0
]
,
[
v
u
x
x
2
1
h
x
x
The University of
Ontario
Neighborhood Processing (filtering)
Finite differences
–
responds to edges
Dark = negative
White = positive
Gray = 0
f
x
The University of
Ontario
Neighborhood Processing (filtering)
Finite differences

responding to noise
Increasing noise

>
(this is zero mean additive gaussian noise)
f
x
f
x
f
x
The University of
Ontario
Neighborhood Processing (filtering)
Finite
differences and noise
Finite difference filters respond strongly to noise
•
Noisy pixels look
very different from their
neighbours
•
The
larger the noise
the stronger is
the
response
How can we eliminate the response to noise?
•
Most pixels in images look similar to their
neighbours
(even at an edge)
•
Smooth the image (mean/
gaussian
filtering)
The University of
Ontario
Neighborhood Processing (filtering)
Smoothing
and Differentiation
Smoothing
before
differentiation = two convolutions
Convolution
is associative
)
(
H
x
f
H
f
H
x
x
)
(
)
(
)
(
H
y
)
(
f
H
x
The University of
Ontario
The scale of the smoothing filter affects derivative estimates, and also
the semantics of the edges recovered.
1 pixel
3 pixels
7 pixels
f
H
x
)
(
Neighborhood Processing (filtering)
Smoothing and Differentiation
The University of
Ontario
Yet another approximation frequently used
Sobel
kernels
1
0

1
2
0

2
1
0

1
]
,
[
v
u
x
x
8
1
f
x
1
2
1
0
0
0

1

2

1
]
,
[
v
u
y
y
8
1
f
y
Neighborhood Processing (filtering)
The University of
Ontario
Recall for a function of
two variables
The
gradient
at a point (
x,y
)
Gradient Magnitude
Gradient Orientation
•
direction of the “steepest ascend”
•
orthogonal to object boundaries in the image
x
f
y
f
Neighborhood Processing (filtering)
Image Gradients
f
f
y
x
y
f
x
f
f
)
,
(
y
x
f
x
y
small image
gradients in low
textured areas
2
2
2
2
)
(
)
(
)
(
)
(


f
f
f
y
x
y
f
x
f
The University of
Ontario
Neighborhood Processing (filtering)
Image Gradient for Edge Detection
Typical application where image gradients are used
is
image edge
detection
•
find points with large image gradients
Canny edge detector
suppresses
non

extrema
Gradient points
The University of
Ontario
Reading: Forsyth & Ponce, 8.1

8.2
Second derivatives
and convolution
Peaks or valleys of the first

derivative of the input
signal, correspond to “zero

crossings” of the
second

derivative of the input signal.
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
The University of
Ontario
Neighborhood Processing (filtering)
Second derivatives and convolution
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
The University of
Ontario
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
Neighborhood Processing (filtering)
Second derivatives and convolution
Better localized edges
But more sensitive to
noise
The University of
Ontario
Neighborhood Processing (filtering)
Second derivatives and convolution
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
The University of
Ontario
Neighborhood Processing (filtering)
Second Image Derivatives
Laplace operator
f
f
f
2
f
f
y
f
x
f
f
y
x
y
x
]
[
2
2
2
2
0
0
0

1
2

1
0
0
0
0

1
0
0
2
0
0

1
0
0

1
0

1
4

1
0

1
0
“divergence
of gradient”
rotationally invariant
second derivative for 2D functions
Finite Difference
Second Order
Derivative in x
Finite Difference
Second Order
Derivative in y
The University of
Ontario
Neighborhood Processing (filtering)
Second Image Derivatives
Laplacian
Zero Crossing
Used for edge detection
(alternative to computing Gradient
extrema
)
http://homepages.inf.ed.ac.uk/rbf/HIPR2/zeros.htm
magnitude of
image gradient
image
intensity
Laplacian
(2
nd
derivative)
of the image
2
2
2
2
y
f
x
f
f
f


f
f
The University of
Ontario
Neighborhood Processing (filtering)
Laplacian
Filtering

Zero on uniform regions

Positive on one side
of an edge

Negative on the other side

Zero at some point in between
on the edge itself
band

pass filter (
Suppresses both high and low frequencies
)
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
The University of
Ontario
Neighborhood Processing (filtering)
Laplacian
of a Gaussian (
LoG
)
Smooth before differentiation
(remember associative property of convolution)
LoG
G
2
2
2
2
2
2
2
4
2
1
1
)
,
(
y
x
e
y
x
y
x
LoG
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
The University of
Ontario
Neighborhood Processing (filtering)
Laplacian
of a Gaussian (
LoG
)
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
Suppresses both high and low frequencies
The University of
Ontario
Neighborhood Processing (filtering)
Laplacian
of a Gaussian (
LoG
)
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
The University of
Ontario
Neighborhood Processing (filtering)
Laplacian
of a Gaussian (
LoG
)
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
The University of
Ontario
Neighborhood Processing (filtering)
Laplacian
of a Gaussian (
LoG
)
Can be approximated by a difference of two
Gaussians (
DoG
)
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
The University of
Ontario
Neighborhood Processing (filtering)
DoG
vs.
LoG
Separable (product decomposition)
more efficient
Can explain band

pass filter since Gaussian is a low

pass filter.
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
The University of
Ontario
Neighborhood Processing (filtering)
LoG
for Blob Detection
Cross correlation with a filter can be viewed as
comparing a little “picture” of what you want to find
against all local regions in the image.
Scale of blob (size ; radius in pixels)
is determined by the sigma parameter
of the
LoG
filter.
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
The University of
Ontario
Neighborhood Processing (filtering)
LoG
for Blob Detection
http://www.cse.psu.edu/~rcollins/CSE486/lecture11_6pp.pdf
The University of
Ontario
Reading: Forsyth & Ponce ch.7.5
Filters
and
T
emplates
Recall: filtering = cross

correlation = dot

product
Measures the angle between two vectors:
the filter and the image window
Measures visual similarity
f
h
*
(
x,y
)
Filter Mask
Input Image
f(
x,y
)
g(
x,y
)
Output Image
(
x,y
)
Correlation
)
cos(
)
,
(



arccos
)
,
(
y
x
g
f
h
f
h
y
x
f
f
w
w
w
w
The University of
Ontario
Since we divide by the norm
Normalized cross

correlation
Bring the template and the image to a uniform scale:
subtract the mean and divide by the variance
Neighborhood Processing (filtering)
Template Matching



arccos
)
,
(
w
w
w
w
f
h
f
h
y
x
f
f
http://scien.stanford.edu/pages/labsite/2008/psych221/projects/08/MariaJabon/index.htm
)
,
(
y
x
h
)
,
(
y
x
f
The University of
Ontario
Normalized cross

correlation
Extremely time consuming due to sliding window
Neighborhood Processing (filtering)
Template Matching
h
f
1


)
cos(
g
g
Slide
credit:
Matlab
manual



arccos
)
,
(
w
w
w
w
f
h
f
h
y
x
f
f
The University of
Ontario
Neighborhood Processing (filtering)
Template Matching
h
f
)
cos(
g
Slide
credit:
OpenCV
manual
Comments 0
Log in to post a comment