# Image Processing/Analysis Fundamentals

Τεχνίτη Νοημοσύνη και Ρομποτική

6 Νοε 2013 (πριν από 4 χρόνια και 6 μήνες)

100 εμφανίσεις

Introduction to Image
Processing

(Signal Processing)

NEU 259

Gina Sosinsky

May 13, 2012

Quantization of images is key….

The Electron Microscope

(Example of a Physical System)

H
0

H
1

f

H
n

g

Simian SV40

Physical systems can be modeled as input signals that
are transformed by the system, or cause the system
to respond in some way, resulting in other signals,
e.g., all imaging devices.

H
0

f
0

g

g
0

H
1

f
1

g
1

H
n

f
n

g
n

H
0

f

g

H
1

H
0

H
1

H
n

f
0

g
n

f
1

f
n

g
0

g
1

H

f

g

Systems and Signals

What is image processing?

The analysis, manipulation, storage, and display
of graphical images from sources such as
photographs, drawings, and video.

Any technique either computational or
photographic which alters the information in
an image.

The analysis, manipulation, storage, and
display of signals in a multi
-
dimensional
space.

Image Enhancement (Real and Reciprocal Spaces).

Image Segmentation.

Feature Recognition and Classification.

Three
-
dimensional Imaging.

Image Visualization (2D and 3D).

Common Topics in Image Processing

Acquisition of Images.

Representation and Storage.

Boolean Operations.

Morphological Operations.

Image Measuring.

Correction of Imaging Defects (Image Restoration).

What is image processing?

Any technique either computational or photographic which alters
the information in the image.

Examples

Reversing the contrast of an image (black becomes white
and vice versa).

Maximizing the values for color tables (histogram
stretching)

Pattern recognition & analysis (correlations).

As Misell points out, image processing will not turn a poor
image into a good one, but extracts the maximum amount of
structural information from the original.

Types of operations

an input image a[m,n] ==> an output image b[m,n] (or another representation)

* Point
-

the output value at a specific coordinate is dependent only on the input
value at that same coordinate.

* Local
-

the output value at a specific coordinate is dependent on the input
values in the neighborhood of that same coordinate.

* Global
-

the output value at a specific coordinate is dependent on all the
values in the input image.

Image Enhancement

Objective: process an image to obtain the
most information from it for a particular
application.

Example: Common operations to enhance
images depend on the convolution of masks
and the Fourier transform.

How do I know that my
new/corrected/enhanced image is
correct?

Does it resemble the original image?

Are any unusual features being introduced (e.g.
aliasing)?

Is it consistent with other results outside of this
image (e.g. biochemistry, NMR, MRI etc.)?

Two simple operations

Reversing the contrast

new_pix = max
-

old_pix + min

Histogram stretching

(contrast stretching)

Can use histogram to replace out
-
lying points.

A bit of history

For electron micrographs, first applications:
Markham et al. in 1963, Klug and Berger in
1964.

Involved the signal from a periodic specimen
was separated from the non
-
periodic noise
(electron crystallography).

Seminal publications:

Crowther, DeRosier, Klug, Nature 1968

Unwin and Henderson, JMB 1976

Specific topics

Shannon sampling and Nyquist limits

Fourier analysis

Projection Theorem

Convolution theorem

Resolution & Filtering

Correlation analysis

Shannon Sampling & Nyquist limits

Nyquist Limit

is defined as 2

r/M.

M = magnification; r = step size of scanner or camera

Basically, the Shannon sampling theorem tells you that you
need at least 2 data points to sample a function. But…in
practice, in order to get a given resolution, you need to use

r/(3

M) or r/(4

M)

(e.g. if you want 12 Å resolution, you need to use a pixel
size of 3
-
4 Å)

This is referred to as
oversampling

Undersampling

can result in an image processing artifact
known as
aliasing
.

Effect of sampling interval on recovery of information. In this example, a
sampling interval of 32 appears to be just fine enough to recover the
shape of the 1D function without loss of information. At coarser
sampling intervals (4
-
16), the subtler features in the data are lost. In
practice, one aims to digitize the data at a fine enough interval to be
certain that no information is lost.
Thus, using the three
-
times pixel
resolution criteria, in this example one ought to sample the data 96 (= 3
x 32) or greater to be certain to recover all the information contained in
the data.

Shannon Sampling & Nyquist Limits

Raster size versus Nyquist limits

Jean Baptiste Joseph Fourier

(1768
-
1830)

Fourier Theory using trigonometrical series expansion done in ~1807

http://www
-
groups.dcs.st
-
and.ac.uk/~history/Mathematicians/Fourier.html

“The profound study of nature
is the most fertile source of
mathematical discoveries.”

Fear not the Fourier

Continuous FT

Discrete FT

(what we calculate)

Inverse FT

r

= T
-
1

(T(
r
))

Inverse Theorem

Terms for Fourier analysis

Real space
:
Our coordinate system (x,y,z)

Reciprocal, Fourier, inverse, tranform space
:
Coordinate system after Fourier transformation

Amplitudes

and
phases

or
real

and
imaginary

parts due
to complex number analysis

r
(xyz)

Fourier transform (F (hkl)exp i

(hkl))

Where F(hkl) is the amplitude and

is the phase and

r
(xyz) is the density function

Need both phase and amplitude data

Original

correct amplitudes

random phases

random amplitudes

correct phases

Dimensions in the object (REAL SPACE) are
inversely related to dimensions in the transform
(RECIPROCAL SPACE).

Small spacings or features in real space are
represented by features spaced far apart in
reciprocal space. Resolution is inversely
proportional to spacings.

Outer regions of the transform arise from fine
(high resolution) details in the object. Coarse
object features contribute near the central (low
resolution) region of the transform.

Reciprocal Space: The
Final Frontier

The recorded diffraction pattern of an object is the square
of the Fourier transform of that object.

FT are linear process (like multiplication and division).
Can go backward and forward easily if functions are
known. Advantages for micrographs where the FT is
calculated and we want to do noise reduction, filtering or
averaging.

Projection Theorem

(next slides)

Convolution Theorem
:
Deconvolution
is more easily
computed in Fourier space rather than in real space
(slides after Projection Theorem).

The Fourier Duck

Behold the duck.

It does not cluck.

A cluck it lacks.

It quacks.

It is specially fond of a puddle or pond.

When it dines or sups, it bottoms ups.

The Fourier Duck originated in a book of optical
transforms (Taylor, C. A. & Lipson, H., Optical
Transforms 1964). An optical transform is a Fourier
transform performed using a simple optical apparatus.

http://www.ysbl.york.ac.uk/~cowtan/fourier/fourier.html

The Fourier Cat and its Transform

FT

FT
-
1

Resolution in real versus
reciprocal space

The effect of taking only
low angle diffraction to
form the image of a duck
object. A drawing of a
duck is shown, together
with its diffraction pattern.
Also shown are the images
formed (as the diffraction
pattern of the diffraction
pattern) when stops are
used to progressively more
of the high angle
diffraction pattern. (From
Holmes and Blow)

FT

FT

FT
-
1

FT
-
1

This is a test

If we take the amplitudes of the Fourier Duck
and the phases of the Fourier Cat and back transform, what
will we get?

Yes, we get the Fourier Cat!

Duck amplitudes, Cat Phases

Projection Theorem

This is the most fundamental principle for 3D
reconstruction from electron micrographs.

Every micrograph we obtain in TEM is a projection (sum)
of everything in the specimen.

When examining 3D objects, 2D images

may not provide the complete picture

The Projection Theorem

Simply stated it says:

The Fourier Transform of the projected structure
of a 3D object is equivalent to a 2D central
section of the 3D Fourier transform of the object.

The central section intersects the origin of the 3d
transform and is perpendicular to the direction of
the projection. The 3d structure is reconstructed
from several independent 2d views by the inverse
Fourier transform of the complete 3d Fourier
transform.

-
ray transforms

Illustration of Projection Theorem

Baumeister et al. (1999)
Trends Cell Biol.
,
9
, 81
-
85.

Projections

Central Sections

The Convolution Theorem

The convolution theorem is one of the most
important relationships in Fourier theory

It forms the basis of X
-
ray, EM and neutron
crystallography.

Holmes and Blow (1965) give a general statement
of the operation of convolution of two functions:

"Set down the origin of the first function in every
possible position of the second, multiply the value
of the first function in each position by the value
of the second at that point and take the sum of all
such possible operations."

c(u) = f(x) * g(x)

Convolution symbol

Also use

Properties of Convolution

Convolution is commutative.

c = a
*

b = b
*

a

Convolution is associative.

c = a
*

(b
*

d) = (a
*

b)
*

d = a
*

b
*

d

Convolution is distributive.

c = a
*

(b + d) = (a
*

b) + (a
*

d)

where a, b, c, and d are all images, either continuous or
discrete.

A simple example of convolution. One function is a drawing of a
duck, the other is a 2D lattice. The convolution of these functions is
accomplished by putting the duck on every lattice point. (From
Holmes and Blow, p.123)

FT

FT

FT
-
1

FT
-
1

Molecule
convoluted with
lattice points

Convolution & Fourier Transforms

Fourier transform of the convolution of two functions is
the product of their Fourier transforms.

T(ƒ *g) = F x G

The converse of the above also holds, namely that the
Fourier transform of the product of two functions is
equal to the convolution of the transforms of the
individual functions.

T(ƒ x g) = F * G

Computationally, multiplication and fast
-
Fourier
transform algorithms are speedier processes than
deconvolution.

Words to image process by

Convolution is easy, Deconvolution
is hard.

(Thursday’s lecture)

Need to know:

T(ƒ x g) = F * G

Simple Filtering Operations

Low pass filter

High pass filter

Band pass filter

(All the above can have hard or soft edges)

Median filter

Sobel filter

Rotational harmonics filtering (Fourier
-
Bessel
analysis)

The Fourier Duck

The Fourier Duck: Low pass filtering

If we only have the low resolution terms of
the diffraction pattern, we only get a low
resolution duck:

High pass filtering

If we only have the high resolution terms of
the diffraction pattern, we see only the
edges of the duck but see internal features
(missing box function):

Inverse band pass

(missing shell of data)

The edges are sharp, but there is smearing
around them from the missing intermediate
resolution terms. The core of the duck is at the
correct level, but the edges are weak.

Hard versus soft edged filters

Gaussian falloffs at the edges prevent
aliasing artifacts.

Butterworth filters also have soft edges

Gaussian

Butterfield

Median Filter (Real space filter
based on image statistics)

Ranks the pixels in a neighborhood (
kernel
)
according to their brightness value (intensity). The
median value in the ordered list is used as a
brightness value for the central pixel.

Excellent rejector of “shot noise” and for
smoothing operations. Outlying pixels are
replaced by a reasonable value
--

the median value
in the neighborhood.

Sobel Filter (Real Space
Filters)

Uses the derivitives of the values and filters
based on the square root of the sum of the
squares for the values.

Good for edge detection

(computationally intensive!)

Edge
Detection
Filters

From Russ’ Book on
Image Processing

Filters within Photoshop

Correlation Analysis & Pattern Recognition

Correlation Analysis & Pattern Recognition

Pearson coefficient as
a measure of
correlation of labeling:

a measure of the
correlation (linear
dependence) between
two variables
X

and
Y
,
giving a value between
+1 and −1 inclusive.

Rotational Filtering (Fourier
-
Bessel)

Friedrich Wilhelm Bessel

1784
-

1846

Good General References

D.L. Misell, "Image Analysis, Enhancement and
Interpretation" (1978) (Practical Methods in
Electron Microscopy series vol. 7, Audrey
Glauert editor) North
-
Holland publishers

J. Frank, "Three
-
Dimensional Electron
Microscopy of Macromolecular Assemblies"
(2004, 2nd edition) Academic Press publishers

J. Russ, "The Image Processing Handbook"
(1995, 2nd edition) CRC Press

Image Processing Fundamentals Web Site
http://www.ph.tn.tudelft.nl/Courses/FIP/nofra
mes/fip.html

Baker lecture notes