Introduction to Image
Processing
(Signal Processing)
NEU 259
Gina Sosinsky
May 10, 2007
The Electron Microscope
(Example of a Physical System)
H
0
H
1
f
H
n
g
Simian SV40
•
Physical systems can be modeled as input signals that
are transformed by the system, or cause the system
to respond in some way, resulting in other signals,
e.g., all imaging devices.
H
0
f
0
g
g
0
H
1
f
1
g
1
H
n
f
n
g
n
H
0
f
g
H
1
H
0
H
1
H
n
f
0
g
n
f
1
f
n
g
0
g
1
H
f
g
Systems and Signals
What is image processing?
•
The analysis, manipulation, storage, and display
of graphical images from sources such as
photographs, drawings, and video.
•
Any technique either computational or
photographic which alters the information in
an image.
The analysis, manipulation, storage, and
display of signals in a multi

dimensional
space.
•
Image Enhancement (Real and Reciprocal Spaces).
•
Image Segmentation.
•
Feature Recognition and Classification.
•
Three

dimensional Imaging.
•
Image Visualization (2D and 3D).
Common Topics in Image Processing
•
Acquisition of Images.
•
Representation and Storage.
•
Boolean Operations.
•
Morphological Operations.
•
Image Measuring.
•
Correction of Imaging Defects (Image Restoration).
What is image processing?
•
Any technique either computational or photographic
which alters the information in the image.
–
Examples
•
Reversing the contrast of an image (black becomes
white and vice versa).
•
Maximizing the values for color tables (histogram
stretching)
•
Pattern recognition & analysis (correlations).
–
As Misell points out, image processing will not turn a
poor image into a good one, but extracts the maximum
amount of structural information from the original.
Types of operations
an input image a[m,n] ==> an output image b[m,n] (or another representation)
* Point

the output value at a specific coordinate is dependent only on the input
value at that same coordinate.
* Local

the output value at a specific coordinate is dependent on the input
values in the neighborhood of that same coordinate.
* Global

the output value at a specific coordinate is dependent on all the
values in the input image.
Image Enhancement
Objective
: process an image to obtain the
most information from it for a particular
application.
Example:
Common operations to enhance
images depend on the convolution of masks
and the Fourier transform.
How do I know that my
new/corrected/enhanced image is
correct?
•
Does it resemble the original image?
•
Are any unusual features being introduced (e.g.
aliasing)?
•
Is it consistent with other results outside of this
image (e.g. biochemistry, NMR, MRI etc.)?
Two simple operations
•
Reversing the contrast
new_pix = max

old_pix + min
Histogram stretching
(contrast stretching)
Can use histogram to replace out

lying points.
A bit of history
•
For electron micrographs, first applications:
Markham et al. in 1963, Klug and Berger in
1964.
–
Involved the signal from a periodic specimen
was separated from the non

periodic noise
(electron crystallography).
Specific topics
•
Shannon sampling and Nyquist limits
•
Fourier analysis
•
Projection Theorem
•
Convolution theorem
•
Resolution & Filtering
•
Correlation analysis
Shannon Sampling & Nyquist limits
Nyquist Limit
is defined as 2
r/M.
M = magnification; r = step size of scanner or camera
Basically, the Shannon sampling theorem tells you that you
need at least 2 data points to sample a function. But…in
practice, in order to get a given resolution, you need to use
r/(3
M) or r/(4
M)
(e.g. if you want 12 Å resolution, you need to use a pixel
size of 3

4 Å)
This is referred to as
oversampling
your data.
Undersampling
can result in an image processing artifact
known as
aliasing
.
Shannon Sampling & Nyquist Limits
Effect of sampling interval on recovery of information. In this
example, a sampling interval of 32 appears to be just fine enough to
recover the shape of the 1D function without loss of information. At
coarser sampling intervals (4

16), the subtler features in the data are
lost.
In practice, one aims to digitize the data at a fine enough interval
to be certain that no information is lost.
Thus, using the three

times
pixel resolution criteria, in this example one ought to sample the data
96 (= 3 x 32) or greater to be certain to recover all the information
contained in the data.
Raster size versus Nyquist limits
Jean Baptiste Joseph Fourier
(1768

1830)
Fourier Theory using
trigonometrical series
expansion done in ~1807
http://www

groups.dcs.st

and.ac.uk/~history/Mathematicians/Fourier.html
“The profound study of nature
is the most fertile source of
mathematical discoveries.”
Fear not the Fourier Transform, it
is your friend!
Continuous FT
Discrete FT
(what we calculate)
Inverse FT
r
= T

1
(T(
r
))
Inverse Theorem
Terms for Fourier analysis
•
Real space
:
Our coordinate system (x,y,z)
•
Reciprocal, Fourier, inverse, tranform space
:
Coordinate system after Fourier transformation
•
Amplitudes
and
phases
or
real
and
imaginary
parts due to complex number analysis
Reciprocal Space: The Final Frontier
•
Dimensions in the object (REAL SPACE) are
inversely related to dimensions in the transform
(RECIPROCAL SPACE).
•
Small spacings or features in real space are
represented by features spaced far apart in
reciprocal space. Resolution is inversely
proportional to spacings.
•
Outer regions of the transform arise from fine
(high resolution) details in the object. Coarse
object features contribute near the central (low
resolution) region of the transform.
Advantages of using Fourier analysis
•
The recorded diffraction pattern of an object is the square
of the Fourier transform of that object.
•
FT are linear process (like multiplication and division).
Can go backward and forward easily if functions are
known. Advantages for micrographs where the FT is
calculated and we want to do noise reduction, filtering or
averaging.
•
Projection Theorem
(next slides)
•
Convolution Theorem
: Deconvolution is more easily
computed in Fourier space rather than in real space
(slides after Projection Theorem).
The Fourier Duck
Behold the duck.
It does not cluck.
A cluck it lacks.
It quacks.
It is specially fond of a puddle or pond.
When it dines or sups, it bottoms ups.
•
The Fourier Duck originated in a book of optical
transforms (Taylor, C. A. & Lipson, H., Optical
Transforms 1964). An optical transform is a Fourier
transform performed using a simple optical apparatus.
http://www.ysbl.york.ac.uk/~cowtan/fourier/fourier.html
The Fourier Cat and its Transform
FT
FT

1
Resolution in real versus reciprocal
space
The effect of taking only
low angle diffraction to
form the image of a duck
object. A drawing of a
duck is shown, together
with its diffraction pattern.
Also shown are the images
formed (as the diffraction
pattern of the diffraction
pattern) when stops are
used to progressively more
of the high angle
diffraction pattern. (From
Holmes and Blow)
FT
FT
FT

1
FT

1
Projection Theorem
This is the most fundamental principle for 3d
reconstruction from electron micrographs.
Every micrograph we obtain in TEM is a projection (sum)
of everything in the specimen.
The Projection Theorem
Simply stated it says:
The Fourier Transform of the projected structure
of a 3D object is equivalent to a 2D central
section of the 3D Fourier transform of the object.
The central section intersects the origin of the 3d
transform and is perpendicular to the direction of
the projection. The 3d structure is reconstructed
from several independent 2d views by the inverse
Fourier transform of the complete 3d Fourier
transform.
Radon and X

ray transforms
Illustration of Projection
Theorem
Baumeister et al. (1999)
Trends Cell Biol.
,
9
, 81

85.
Projections
Central Sections
The Convolution Theorem
The convolution theorem is one of the most
important relationships in Fourier theory
It forms the basis of X

ray, EM and neutron
crystallography.
•
Holmes and Blow (1965) give a general statement
of the operation of convolution of two functions:
"Set down the origin of the first function in every
possible position of the second, multiply the value
of the first function in each position by the value
of the second at that point and take the sum of all
such possible operations."
c(u) = f(x) * g(x)
Convolution symbol
Properties of Convolution
•
Convolution is commutative.
•
Convolution is associative.
* Convolution is distributive.
where a, b, c, and d are all images, either continuous or
discrete.
A simple example of convolution. One function is a drawing of a
duck, the other is a 2D lattice. The convolution of these functions is
accomplished by putting the duck on every lattice point. (From
Holmes and Blow, p.123)
FT
FT
FT

1
FT

1
Molecule
convoluted with
lattice points
Convolution & Fourier Transforms
•
Fourier transform of the convolution of two functions is
the product of their Fourier transforms.
T(ƒ *g) = F x G
•
The converse of the above also holds, namely that the
Fourier transform of the product of two functions is equal
to the convolution of the transforms of the individual
functions.
T(ƒ x g) = F * G
Computationally, multiplication and fast

Fourier transform
algorithms are speedier processes than deconvolution.
Words to image process by
Convolution is easy, Deconvolution
is hard.
(Next week’s lecture)
Need to know
T(ƒ x g) = F * G
Simple Filtering Operations
•
Low pass filter
•
High pass filter
•
Band pass filter
•
Crystalline masks, Layer line masks
(All the above can have hard or soft edges)
•
Median filter
•
Sobel filter
•
Rotational harmonics filtering (Fourier

Bessel
analysis)
The Fourier Duck
The Fourier Duck: Low pass
filtering
•
If we only have the low resolution terms of
the diffraction pattern, we only get a low
resolution duck:
High pass filtering
•
If we only have the high resolution terms of
the diffraction pattern, we see only the
edges of the duck but see internal features
(missing box function):
Inverse band pass
(missing shell of data)
•
The edges are sharp, but there is smearing around
them from the missing intermediate resolution
terms. The core of the duck is at the correct level,
but the edges are weak.
Hard versus soft edged filters
•
Gaussian falloffs at the edges prevent
aliasing artifacts.
•
Butterworth filters which have soft edges
“programmed” in.
Crystalline masking
Median Filter (Real space filter
based on image statistics)
•
Ranks the pixels in a neighborhood (
kernel
)
according to their brightness value (intensity). The
median value in the ordered list is used as a
brightness value for the central pixel.
•
Excellent rejector of “shot noise” and for
smoothing operations. Outlying pixels are
replaced by a reasonable value

the median value
in the neighborhood.
Sobel Filter (Real Space Filters)
•
Uses the derivitives of the values and filters
based on the square root of the sum of the
squares for the values.
•
Good for edge detection
•
(computationally intensive!)
Edge
Detection
Filters
From Russ’ Book on
Image Processing
Correlation Analysis & Pattern Recognition
Correlation Analysis & Pattern Recognition
Rotational Filtering (Fourier

Bessel)
Friedrich Wilhelm Bessel
1784

1846
Good General References
•
D.L. Misell, "Image Analysis, Enhancement and
Interpretation" (1978) (Practical Methods in
Electron Microscopy series vol. 7, Audrey
Glauert editor) North

Holland publishers
•
J. Frank, "Three

Dimensional Electron
Microscopy of Macromolecular Assemblies"
(2004, 2nd edition) Academic Press publishers
•
J. Russ, "The Image Processing Handbook"
(1995, 2nd edition) CRC Press
•
Image Processing Fundamentals Web Site
http://www.ph.tn.tudelft.nl/Courses/FIP/nofra
mes/fip.html
•
Web course
Comments 0
Log in to post a comment