Intro_IP_2007

peachpuceAI and Robotics

Nov 6, 2013 (3 years and 9 months ago)

71 views

Introduction to Image
Processing

(Signal Processing)


NEU 259

Gina Sosinsky

May 10, 2007

The Electron Microscope

(Example of a Physical System)

H
0

H
1

f

H
n

g

Simian SV40


Physical systems can be modeled as input signals that
are transformed by the system, or cause the system
to respond in some way, resulting in other signals,
e.g., all imaging devices.

H
0

f
0

g

g
0

H
1

f
1

g
1

H
n

f
n

g
n

H
0

f

g

H
1

H
0

H
1

H
n

f
0

g
n

f
1

f
n

g
0

g
1

H

f

g

Systems and Signals

What is image processing?


The analysis, manipulation, storage, and display
of graphical images from sources such as
photographs, drawings, and video.


Any technique either computational or
photographic which alters the information in
an image.


The analysis, manipulation, storage, and
display of signals in a multi
-
dimensional
space.


Image Enhancement (Real and Reciprocal Spaces).


Image Segmentation.


Feature Recognition and Classification.


Three
-
dimensional Imaging.


Image Visualization (2D and 3D).

Common Topics in Image Processing


Acquisition of Images.


Representation and Storage.


Boolean Operations.


Morphological Operations.


Image Measuring.


Correction of Imaging Defects (Image Restoration).

What is image processing?


Any technique either computational or photographic
which alters the information in the image.



Examples



Reversing the contrast of an image (black becomes
white and vice versa).


Maximizing the values for color tables (histogram
stretching)


Pattern recognition & analysis (correlations).



As Misell points out, image processing will not turn a
poor image into a good one, but extracts the maximum
amount of structural information from the original.


Types of operations


an input image a[m,n] ==> an output image b[m,n] (or another representation)


* Point
-

the output value at a specific coordinate is dependent only on the input
value at that same coordinate.



* Local
-

the output value at a specific coordinate is dependent on the input
values in the neighborhood of that same coordinate.



* Global
-

the output value at a specific coordinate is dependent on all the
values in the input image.





Image Enhancement

Objective
: process an image to obtain the
most information from it for a particular
application.


Example:

Common operations to enhance
images depend on the convolution of masks
and the Fourier transform.

How do I know that my
new/corrected/enhanced image is
correct?


Does it resemble the original image?


Are any unusual features being introduced (e.g.
aliasing)?


Is it consistent with other results outside of this
image (e.g. biochemistry, NMR, MRI etc.)?

Two simple operations


Reversing the contrast



new_pix = max
-

old_pix + min




Histogram stretching

(contrast stretching)

Can use histogram to replace out
-
lying points.

A bit of history


For electron micrographs, first applications:
Markham et al. in 1963, Klug and Berger in
1964.


Involved the signal from a periodic specimen
was separated from the non
-
periodic noise
(electron crystallography).

Specific topics


Shannon sampling and Nyquist limits


Fourier analysis


Projection Theorem


Convolution theorem


Resolution & Filtering


Correlation analysis

Shannon Sampling & Nyquist limits

Nyquist Limit

is defined as 2


r/M.

M = magnification; r = step size of scanner or camera

Basically, the Shannon sampling theorem tells you that you
need at least 2 data points to sample a function. But…in
practice, in order to get a given resolution, you need to use


r/(3


M) or r/(4


M)

(e.g. if you want 12 Å resolution, you need to use a pixel
size of 3
-
4 Å)

This is referred to as
oversampling

your data.
Undersampling

can result in an image processing artifact
known as
aliasing
.

Shannon Sampling & Nyquist Limits

Effect of sampling interval on recovery of information. In this
example, a sampling interval of 32 appears to be just fine enough to
recover the shape of the 1D function without loss of information. At
coarser sampling intervals (4
-
16), the subtler features in the data are
lost.
In practice, one aims to digitize the data at a fine enough interval
to be certain that no information is lost.

Thus, using the three
-
times
pixel resolution criteria, in this example one ought to sample the data
96 (= 3 x 32) or greater to be certain to recover all the information
contained in the data.

Raster size versus Nyquist limits

Jean Baptiste Joseph Fourier

(1768
-
1830)

Fourier Theory using
trigonometrical series

expansion done in ~1807

http://www
-
groups.dcs.st
-
and.ac.uk/~history/Mathematicians/Fourier.html

“The profound study of nature
is the most fertile source of
mathematical discoveries.”

Fear not the Fourier Transform, it
is your friend!

Continuous FT

Discrete FT

(what we calculate)

Inverse FT

r

= T
-
1

(T(
r
))

Inverse Theorem

Terms for Fourier analysis


Real space
:
Our coordinate system (x,y,z)


Reciprocal, Fourier, inverse, tranform space
:
Coordinate system after Fourier transformation


Amplitudes

and
phases

or
real

and
imaginary

parts due to complex number analysis

Reciprocal Space: The Final Frontier


Dimensions in the object (REAL SPACE) are
inversely related to dimensions in the transform
(RECIPROCAL SPACE).


Small spacings or features in real space are
represented by features spaced far apart in
reciprocal space. Resolution is inversely
proportional to spacings.


Outer regions of the transform arise from fine
(high resolution) details in the object. Coarse
object features contribute near the central (low
resolution) region of the transform.

Advantages of using Fourier analysis


The recorded diffraction pattern of an object is the square
of the Fourier transform of that object.


FT are linear process (like multiplication and division).
Can go backward and forward easily if functions are
known. Advantages for micrographs where the FT is
calculated and we want to do noise reduction, filtering or
averaging.


Projection Theorem

(next slides)


Convolution Theorem
: Deconvolution is more easily
computed in Fourier space rather than in real space
(slides after Projection Theorem).


The Fourier Duck



Behold the duck.

It does not cluck.

A cluck it lacks.

It quacks.

It is specially fond of a puddle or pond.

When it dines or sups, it bottoms ups.



The Fourier Duck originated in a book of optical
transforms (Taylor, C. A. & Lipson, H., Optical
Transforms 1964). An optical transform is a Fourier
transform performed using a simple optical apparatus.

http://www.ysbl.york.ac.uk/~cowtan/fourier/fourier.html

The Fourier Cat and its Transform

FT

FT
-
1

Resolution in real versus reciprocal
space


The effect of taking only
low angle diffraction to
form the image of a duck
object. A drawing of a
duck is shown, together
with its diffraction pattern.
Also shown are the images
formed (as the diffraction
pattern of the diffraction
pattern) when stops are
used to progressively more
of the high angle
diffraction pattern. (From
Holmes and Blow)

FT

FT

FT
-
1

FT
-
1

Projection Theorem

This is the most fundamental principle for 3d
reconstruction from electron micrographs.

Every micrograph we obtain in TEM is a projection (sum)
of everything in the specimen.


The Projection Theorem

Simply stated it says:

The Fourier Transform of the projected structure
of a 3D object is equivalent to a 2D central
section of the 3D Fourier transform of the object.

The central section intersects the origin of the 3d
transform and is perpendicular to the direction of
the projection. The 3d structure is reconstructed
from several independent 2d views by the inverse
Fourier transform of the complete 3d Fourier
transform.




Radon and X
-
ray transforms

Illustration of Projection
Theorem

Baumeister et al. (1999)
Trends Cell Biol.
,
9
, 81
-
85.

Projections

Central Sections

The Convolution Theorem

The convolution theorem is one of the most
important relationships in Fourier theory


It forms the basis of X
-
ray, EM and neutron
crystallography.



Holmes and Blow (1965) give a general statement
of the operation of convolution of two functions:



"Set down the origin of the first function in every
possible position of the second, multiply the value
of the first function in each position by the value
of the second at that point and take the sum of all
such possible operations."

c(u) = f(x) * g(x)

Convolution symbol

Properties of Convolution



Convolution is commutative.




Convolution is associative.



* Convolution is distributive.




where a, b, c, and d are all images, either continuous or
discrete.

A simple example of convolution. One function is a drawing of a
duck, the other is a 2D lattice. The convolution of these functions is
accomplished by putting the duck on every lattice point. (From
Holmes and Blow, p.123)

FT

FT

FT
-
1

FT
-
1

Molecule
convoluted with
lattice points

Convolution & Fourier Transforms


Fourier transform of the convolution of two functions is
the product of their Fourier transforms.




T(ƒ *g) = F x G



The converse of the above also holds, namely that the
Fourier transform of the product of two functions is equal
to the convolution of the transforms of the individual
functions.


T(ƒ x g) = F * G


Computationally, multiplication and fast
-
Fourier transform
algorithms are speedier processes than deconvolution.

Words to image process by

Convolution is easy, Deconvolution
is hard.


(Next week’s lecture)

Need to know
T(ƒ x g) = F * G



Simple Filtering Operations


Low pass filter


High pass filter


Band pass filter


Crystalline masks, Layer line masks

(All the above can have hard or soft edges)


Median filter


Sobel filter


Rotational harmonics filtering (Fourier
-
Bessel
analysis)

The Fourier Duck



The Fourier Duck: Low pass
filtering


If we only have the low resolution terms of
the diffraction pattern, we only get a low
resolution duck:


High pass filtering


If we only have the high resolution terms of
the diffraction pattern, we see only the
edges of the duck but see internal features
(missing box function):


Inverse band pass

(missing shell of data)


The edges are sharp, but there is smearing around
them from the missing intermediate resolution
terms. The core of the duck is at the correct level,
but the edges are weak.

Hard versus soft edged filters


Gaussian falloffs at the edges prevent
aliasing artifacts.



Butterworth filters which have soft edges
“programmed” in.



Crystalline masking

Median Filter (Real space filter
based on image statistics)


Ranks the pixels in a neighborhood (
kernel
)
according to their brightness value (intensity). The
median value in the ordered list is used as a
brightness value for the central pixel.



Excellent rejector of “shot noise” and for
smoothing operations. Outlying pixels are
replaced by a reasonable value
--

the median value
in the neighborhood.

Sobel Filter (Real Space Filters)


Uses the derivitives of the values and filters
based on the square root of the sum of the
squares for the values.


Good for edge detection



(computationally intensive!)

Edge
Detection
Filters

From Russ’ Book on
Image Processing

Correlation Analysis & Pattern Recognition

Correlation Analysis & Pattern Recognition

Rotational Filtering (Fourier
-
Bessel)

Friedrich Wilhelm Bessel

1784
-

1846

Good General References



D.L. Misell, "Image Analysis, Enhancement and
Interpretation" (1978) (Practical Methods in
Electron Microscopy series vol. 7, Audrey
Glauert editor) North
-
Holland publishers


J. Frank, "Three
-
Dimensional Electron
Microscopy of Macromolecular Assemblies"
(2004, 2nd edition) Academic Press publishers



J. Russ, "The Image Processing Handbook"
(1995, 2nd edition) CRC Press


Image Processing Fundamentals Web Site
http://www.ph.tn.tudelft.nl/Courses/FIP/nofra
mes/fip.html


Web course