Image Image enhancement enhancement

paradepetAI and Robotics

Nov 5, 2013 (4 years and 4 days ago)

70 views

Medical Image Processing and Management
Thomas M. Lehmann
Department of Medical Informatics
Aachen University of Technology (RWTH)
5. Linear and Non-linear Image Enhancement
Image
Image
enhancement
enhancement


What is image enhancement?
What is image enhancement?


What is the difference to
What is the difference to
calibration, correction, registration, optimization
calibration, correction, registration, optimization
?
?


Which methods are used?
Which methods are used?


How to evaluate image enhancement?
How to evaluate image enhancement?
Image
Image
enhancement
enhancement


Improvement of image quality
Improvement of image quality


Image quality
Image quality


depends on context & application
depends on context & application


Improvement
Improvement


depends on quality measure
depends on quality measure
optimization
filtering
registration
calibration
measurement
interpretation
segmentation
classification
feature extraction
display
illumination
surface.reconstr..
shading
communication
archiving
compression
retrieval
digitization
capturing
Pre-
processing
Analysis
Visualization
Management
Acquisition
Image processing
Image processing


Scheme
Scheme
transformation
Image
Image
enhancement
enhancement


Improvement of image quality
Improvement of image quality


Image quality
Image quality


depends on context & application
depends on context & application


Improvement
Improvement


depends on quality measure
depends on quality measure
acquisition
acquisition
visualization
visualization
analysis
analysis
communication
communication
enhancement
enhancement
optimization
filtering
registration
calibration
measurement
interpretation
segmentation
classification
feature extraction
display
illumination
surface.reconstr..
shading
communication
archiving
compression
retrieval
digitization
capturing
Image enhancement
Image enhancement


Scheme
Scheme
transformation
Image enhancement
Image enhancement


Levels of operation
Levels of operation


r
r
aw data
aw data


p
p
ixel
ixel


e
e
dge
dge


t
t
exture
exture


r
r
egion
egion


o
o
bject
bject


s
s
cene
cene


Enhancement involves low
Enhancement involves low
-
-
and
and
high
high
-
-
level processing
level processing


Multi
Multi
-
-
level algorithms
level algorithms


a
a
daptive
daptive
Implementation
Implementation
of image
of image
enhancement
enhancement


Local transforms: point to point mapping
Local transforms: point to point mapping


grid
grid
-
-
fitting (e.g., histogram transform)
fitting (e.g., histogram transform)


non grid
non grid
-
-
fitting (e.g., geometric transforms)
fitting (e.g., geometric transforms)


Regional transforms: region to point mapping
Regional transforms: region to point mapping


linear (e.g., convolution)
linear (e.g., convolution)


non
non
-
-
linear (e.g., sliding median)
linear (e.g., sliding median)


m
m
orphological
orphological
(e.g.,
(e.g.,
erosion
erosion
)
)


Global transforms: image to point mapping
Global transforms: image to point mapping


real value range (e.g., cosine transform)
real value range (e.g., cosine transform)


complex value range (e.g., Fourier transform)
complex value range (e.g., Fourier transform)
Grid
Grid
-
-
fitting local transforms
fitting local transforms


Point to point
Point to point
mapping
mapping
Example
Example


Histogram stretching
Histogram stretching
50
50
100
100
150
150
200
200
255
255
0
0
0
0
50
50
100
100
150
150
200
200
255
255
g
g‘
)('gTg =
)('gTg =
Non
Non
grid
grid
-
-
fitting local transforms
fitting local transforms


Point to point
Point to point
mapping
mapping
Example
Example
:
:
Shift
Shift


changes
changes


location
location


preserves
preserves


orientation
orientation
(direction of lines)
(direction of lines)


dimension
dimension
(size, area)
(size, area)


is determined by
is determined by


x
x
’ =
’ =
x
x
+
+
x
x
0
0
y
y
’ =
’ =
y
y
+
+
y
y
0
0


1 point or
1 point or


2 parameters (
2 parameters (
x
x
0
0
,
,
y
y
0
0
)
)
Example
Example
:
:
Rigid
Rigid


changes
changes


location
location


orientation
orientation


preserves
preserves


dimension
dimension


relative
relative
angulation
angulation


is determined by
is determined by


2 points or
2 points or


3 parameters
3 parameters
Example
Example
:
:
RST
RST


changes
changes


location
location


orientation
orientation


dimension
dimension


preserves
preserves


relative
relative
angulation
angulation


parallelism
parallelism


is determined by
is determined by


2 points or
2 points or


4 parameters
4 parameters
Example
Example
: a
: a
ffine
ffine


changes
changes


location
location


orientation
orientation


dimension
dimension


relative
relative
angulation
angulation


preserves
preserves


parallelism
parallelism


straightness
straightness


is determined by
is determined by


3 points or
3 points or


6 parameters
6 parameters
Example
Example
: p
: p
rojective
rojective


changes
changes


location
location


orientation
orientation


dimension
dimension


relative
relative
angulation
angulation


parallelism
parallelism


preserves
preserves


straightness
straightness


is determined by
is determined by


4 points or
4 points or


8 parameters
8 parameters
Elastic deformation
Elastic deformation


changes
changes


location
location


orientation
orientation


dimension
dimension


relative
relative
angulation
angulation


parallelism
parallelism


straightness
straightness


local deformation
local deformation


no model applies
no model applies


only based on
only based on
corresponding points
corresponding points


use carefully
use carefully
How
How
to
to
implement
implement
?
?


Destination
Destination
driven
driven


interpolation
interpolation
Theory of interpolation
Theory of interpolation


Convolution of the discrete samples
Convolution of the discrete samples
s
s
(
(
m,n
m,n
)
)
with a continuous 2D impulse response
with a continuous 2D impulse response
h
h
(
(
x,y
x,y
)
)
∑∑
−−⋅=
m n
nymxhnmsyxs ),(),(),(
∑∑
−−⋅=
m n
nymxhnmsyxs ),(),(),(
)()(),( yhxhyxh ⋅=
)()(),( yhxhyxh ⋅=
(
(
k,l
k,l
)
)
(
(
x,y
x,y
)
)


Symmetry and
Symmetry and
separability
separability
Determination of the convolution kernel
Determination of the convolution kernel
h
h


Ideal interpolation
Ideal interpolation


derived from the sampling theorem
derived from the sampling theorem
spatial
spatial
domain
domain
frequency
frequency
domain
domain
Fourier
Fourier
transform
transform
)(xs
)(xs
)( fS
)( fS
x
x
f
f
)(xs
)(xs
x
x
f
f
)( fS
)( fS
*
*
*
=
=
=
=
=
=
.
.
.
)(ks
)(ks
)( fH
)( fH
.
.
.
)(xs
)(xs
x
x
f
f
)( fS
)( fS
=
=
=
)(ks
)(ks
=
=
=
)(xh
)(xh
*
*
*
)( fH
)( fH
.
.
.


−−⋅=
m n
nymxhnmsyxs ),(),(),(


−−⋅=
m n
nymxhnmsyxs ),(),(),(
Ideal
Ideal
interpolation
interpolation


x
x
xh
π
π )sin(
)( =
x
x
xh
π
π )sin(
)( =
spatial domain
spatial domain
(
(
plot truncated
plot truncated
)
)
Fourier
Fourier
domain
domain
Fourier
Fourier
domain
domain
(
(
logarithmic plot
logarithmic plot
)
)
Ideal interpolation
Ideal interpolation


Problem
Problem


h
h
(
(
x
x
) is spatially unlimited
) is spatially unlimited


Solution
Solution


similar but spatially limited functions
similar but spatially limited functions


Take care of basic properties
Take care of basic properties


interpolation vs. approximation
interpolation vs. approximation


DC
DC
-
-
constantness
constantness
Interpolation vs. approximation
Interpolation vs. approximation


Image not modified if
Image not modified if
resampled
resampled
on the same grid
on the same grid
h
h
(
(
x
x
)
)
is interpolator
is interpolator


,...2,1||,0)(
1)0(
=





xxh
h
,...2,1||,0)(
1)0(
=





xxh
h


interpolator
interpolator
(Lagrange)
(Lagrange)


approximator
approximator
(notch filter)
(notch filter)
spatial domain
spatial domain
spatial domain
spatial domain
DC
DC
-
-
constant vs. DC
constant vs. DC
-
-
inconstant
inconstant


No direct current amplification, bias is unchanged
No direct current amplification, bias is unchanged
h
h
(
(
x
x
)
)
is DC
is DC
-
-
constant
constant


...
...


where
where
H
H
(
(
f
f
)
)
denotes the Fourier transform of
denotes the Fourier transform of
h
h
(
(
x
x
)
)


−∞=
≡+
k
kdh 1)(


−∞=
≡+
k
kdh 1)(
,...7.12,3.6||,,...2,1||,0)(
1)0(
==





ωffH
H
,...7.12,3.6||,,...2,1||,0)(
1)0(
==





ωffH
H
DC
DC
-
-
inconstantness
inconstantness


−∞=
≠+⇒≠
k
kdhH 1)(1)0(


−∞=
≠+⇒≠
k
kdhH 1)(1)0(


truncated sinc
truncated sinc
spatial domain
spatial domain
Fourier
Fourier
domain
domain
DC
DC
-
-
inconstantness
inconstantness


−∞=
≠+⇒=≠
k
kdhffH 1)(,...2,1||,0)(


−∞=
≠+⇒=≠
k
kdhffH 1)(,...2,1||,0)(


2
2
nd
nd
-
-
order
order
Gaussian
Gaussian
spatial domain
spatial domain
logarithmic Fourier
logarithmic Fourier
The easy
The easy
-
-
to
to
-
-
implement standard
implement standard


Linear interpolation
Linear interpolation
(
(
N
N
=2)
=2)


yields poor results
yields poor results
spatial domain
spatial domain
(plot truncated)
(plot truncated)
Fourier domain
Fourier domain
Fourier domain
Fourier domain
(logarithmic plot)
(logarithmic plot)
Cubic interpolation
Cubic interpolation


Piecewise polynomial kernel
Piecewise polynomial kernel


Boundary conditions
Boundary conditions


interpolator:
interpolator:
h
h
(
(
k
k
) = 1
) = 1
for
for
k
k
= 0
= 0


DC
DC
-
-
constant:
constant:
h
h
(
(
k
k
) = 0
) = 0
for
for
k
k


0
0


C0
C0
-
-
continuity:
continuity:
h
h
(
(
k
k
-
-
) =
) =
h
h
(
(
k
k
+
+
)
)


C1
C1
-
-
continuity:
continuity:


(
(
k
k
-
-
) =
) =


(
(
k
k
+
+
)
)


C2
C2
-
-
continuity:
continuity:
h´´
h´´
(
(
k
k
-
-
) =
) =
h´´
h´´
(
(
k
k
+
+
)
)
elsewhere
2/||...
...||1
1||0
0
||
||
||
...
||
||
||
)(
11
2
2
1
3
3
1
Nx
x
x
D
D
xC
xC
xB
xB
xA
xA
xh
NN
NN
<≤
<≤
<≤







+
+
+
+
+
+
=
Cubic interpolation
Cubic interpolation


N
N
= 6
= 6
spatial domain
spatial domain
(plot truncated)
(plot truncated)
Fourier domain
Fourier domain
Fourier domain
Fourier domain
(logarithmic plot)
(logarithmic plot)
The advanced standard
The advanced standard


B
B
-
-
spline interpolation
spline interpolation
(
(
N
N
=
=


)
)
spatial domain
spatial domain
(
(
plot truncated
plot truncated
)
)
Fourier
Fourier
domain
domain
Fourier
Fourier
domain
domain
(
(
logarithmic plot
logarithmic plot
)
)
How to compare?
How to compare?


Local properties
Local properties


interpolation vs. approximation
interpolation vs. approximation


DC
DC
-
-
constant vs. DC
constant vs. DC
-
-
inconstant
inconstant


Fourier analysis
Fourier analysis


passband
passband
(constant amplification)
(constant amplification)


cut
cut
-
-
off point (large slope)
off point (large slope)


stopband
stopband
(high attenuation, ripples, side lobes)
(high attenuation, ripples, side lobes)


Quantitative error analysis
Quantitative error analysis


subtract a forward
subtract a forward
-
-
backward transformed image
backward transformed image
from the original
from the original


Computational complexity
Computational complexity


Runtime measurements
Runtime measurements
Quality analysis
Quality analysis


E.g. aspect
E.g. aspect
-
-
ratio correction (4/3)
ratio correction (4/3)


interpolation only in
interpolation only in
x
x
direction
direction


every 3
every 3
rd
rd
row remains unchanged
row remains unchanged
Quality analysis
Quality analysis


The more black, the worse the method
The more black, the worse the method


frame indicates B
frame indicates B
-
-
spline
spline
border
border
-
-
effect zone
effect zone


white lines indicate interpolators
white lines indicate interpolators


equal distribution indicate DC
equal distribution indicate DC
-
-
constantness
constantness
truncated
truncated
sinc
sinc
linear
linear
notch filter
notch filter
DC
DC
-
-
inconstant
inconstant
interpolator
interpolator
DC
DC
-
-
constant
constant
approximator
approximator
DC
DC
-
-
constant
constant
interpolator
interpolator
Lagrange
Lagrange
N
N
=5
=5
2
2
nd
nd
Gaussian
Gaussian
N
N
=6
=6
10
10
th
th
Gaussian
Gaussian
N
N
=8
=8
DC
DC
-
-
inconstant
inconstant
approximator
approximator
DC
DC
-
-
inconstant
inconstant
approximator
approximator
DC
DC
-
-
constant
constant
interpolator
interpolator
Quality analysis
Quality analysis


Large
Large
-
-
sized Gaussian kernels
sized Gaussian kernels


Large
Large
-
-
sized Lagrange kernels
sized Lagrange kernels
B
B
-
-
spline
spline
N
N
=
=


Cubic
Cubic
N
N
=6
=6
windowed
windowed
sinc
sinc
N
N
=6
=6
DC
DC
-
-
constant
constant
interpolator
interpolator
DC
DC
-
-
constant
constant
interpolator
interpolator
DC
DC
-
-
constant
constant
interpolator
interpolator
Quality analysis
Quality analysis


Blackman
Blackman
-
-
Harris windowed
Harris windowed
sinc
sinc


C2
C2
-
-
continuous cubic
continuous cubic
Quality analysis quantitatively
Quality analysis quantitatively


Select images
Select images


sufficient
sufficient


representative
representative


Perform cyclic alteration
Perform cyclic alteration


e.g., 15 times rotation of 24 degree
e.g., 15 times rotation of 24 degree


Compute quality analysis
Compute quality analysis


absolute error for each image
absolute error for each image


mean error and variance for all images
mean error and variance for all images


statistical analyses of significance (t
statistical analyses of significance (t
-
-
test)
test)


Results may differ
Results may differ
to visual inspection of one image
to visual inspection of one image
Runtime
Runtime
Nearest Neighbor 0.13 s
Linear 0.15 s
Cubic 0.18 s
Quadratic 0.22 s
Lagrange 0.23 s
Cubic 0.27 s
Lagrange 0.28 s
Gaussian M=2 0.73 s
Truncated sinc 0.63 s
Lagrange 0.34 s
Truncated sinc 0.72 s
Cubic 0.39 s
Lagrange 0.42 s
Gaussian M=2 1.10 s
Gaussian M=6 1.13 s
Truncated sinc 0.78 s
Lagrange 1.50 s
Cubic 1.54 s
Lagrange 0.58 s
Gaussian M=6 1.54 s
Gaussian M=10 1.60 s
B-spline interpolation 0.36 s
1x1
2x2
3x3
4x4
5x5
6x6
7x7
8x8
1.751.51.251.00.750.50.250
Time
in sec.
Interpolation
Rotation
LUT
4x4
LUT
8x8
LUT
6x6
How to compare?
How to compare?
-
-
Summary
Summary


Task dependent
Task dependent


runtime
runtime


content of images
content of images


Testing on only one image is insufficient
Testing on only one image is insufficient


large number (depends on variance)
large number (depends on variance)


representative images
representative images


statistical analyses
statistical analyses
Regional
Regional
transform
transform


Linear
Linear


convolution
convolution
Regional
Regional
transform
transform


Linear
Linear


convolution
convolution
Linear regional filtering
Linear regional filtering


Smoothing
Smoothing


low pass
low pass


Contrast enhancement
Contrast enhancement


high pass
high pass


unsharp
unsharp
masking
masking


edge crisping
edge crisping


reliefs
reliefs


Floating mean
Floating mean


Gaussian respective binomial low pass
Gaussian respective binomial low pass


template coefficients
template coefficients
according to binomial distribution
according to binomial distribution
Smoothing
Smoothing
2 1
2 4 2
1 2 1
1
1 4 6 4 1
4 16 24 16 4
6 24 36 24 6
4 16 24 16 4
1 6 4 14
1/16 1/256
original
original
3x3 template
3x3 template
5x5 template
5x5 template
Example: binomial low pass
Example: binomial low pass


Smoothing and noise suppression
Smoothing and noise suppression
Edge enhancement
Edge enhancement
1/16 1/256
-1
-1
-1
-1
-2
-2 -2
-2
12
-1-1
-1-1
-4
-4-4
-4
-4 -4
-4 -4
-6
-6
-6
-6
-24
-24
-24
-24
-16
-16 -16
-16
220


Positive and negative template pixels
Positive and negative template pixels


Gaussian respective binomial high pass
Gaussian respective binomial high pass


template coefficients
template coefficients
according to binomial distribution
according to binomial distribution
Example: binomial high pass
Example: binomial high pass
original
original
3x3 template
3x3 template
5x5 template
5x5 template


Edge extraction
Edge extraction
Unsharp
Unsharp
masking
masking
original
original
high pass
high pass
unsharp
unsharp
masking
masking


Contrast and edge enhancement
Contrast and edge enhancement
Unsharp masking
Unsharp masking


Conjunction
Conjunction
of original and high pass
of original and high pass
step
step
ramp
ramp
impulse
impulse
original
original
high pass
high pass
unsharp masking
unsharp masking
Relief
Relief
images
images
original
original
single
single
displacement
displacement
double
double
displacement
displacement
0
0
-1
0
0
0
1
0
0
0
0
-1
0
0
0
0
0
1
Relief
Relief
images
images


Displacement
Displacement
and
and
subtraction
subtraction
of an image
of an image
Edge
Edge
crisping
crisping
0 -1 0
-1 5 -1
0 -1 0
-1
-1
-1
-1
-1
-1
-1
-1
9
1
1
1
1
-2
-2
-2
-2
5
Non
Non
-
-
linear regional
linear regional
filtering
filtering


Rank
Rank
ordering filter
ordering filter


median
median


Adaptive
Adaptive
filter
filter


Nagao
Nagao
&
&
Matsuyama
Matsuyama
(1979)
(1979)


Morphological filter
Morphological filter


erosion
erosion


dilation
dilation
original
original
low
low
pass
pass
median
median
impulse
impulse
ramp
ramp
step
step
Rank
Rank
ordering filter
ordering filter


Median
Median
Example
Example
: median
: median
filter
filter
original
original
low
low
pass
pass
(
(
template
template
5x5)
5x5)
median
median
(
(
window
window
size
size
7)
7)


Removal of
Removal of
discrete defects
discrete defects
Adaptive filtering
Adaptive filtering


Selection of filter or filter parameters
Selection of filter or filter parameters
depends on local image properties
depends on local image properties
filter
filter
analysis
analysis
linear
linear
non
non
-
-
linear
linear
Nagao &
Nagao &
Matsuyama
Matsuyama
(1979)
(1979)


Variable computation of mean and variance
Variable computation of mean and variance
Nagao
Nagao
&
&
Matsuyama
Matsuyama
(1979)
(1979)
for m = 1 to M { // walk over columns
for m = 1 to M { // walk over columns
for n = 1 to N { // walk over lines
for n = 1 to N { // walk over lines
for current_mask = 1 to MASKS { // for all masks
for current_mask = 1 to MASKS { // for all masks
current_variance = calc_variance( input_image, m, n,
current_variance = calc_variance( input_image, m, n,
mask[current_mask]);
mask[current_mask]);
if (( current_mask == 1) ||
if (( current_mask == 1) ||
( current_variance < least_variance)) {
( current_variance < least_variance)) {
least_variance = current_variance;
least_variance = current_variance;
result_mean = calc_mean( input_image, m, n,
result_mean = calc_mean( input_image, m, n,
mask[current_mask]);
mask[current_mask]);
}
}
} // for current_mask
} // for current_mask
result_image[ m, n ] = result_mean;
result_image[ m, n ] = result_mean;
} // for n
} // for n
} // for m
} // for m
Nagao
Nagao
&
&
Matsuyama
Matsuyama
(1979)
(1979)


Example
Example
1
1
iteration
iteration
48
48
iterations
iterations
original
original
Morphological Filtering
Morphological Filtering


Logical operations on binary images
Logical operations on binary images


only black and white pixels
only black and white pixels


Basic operations:
Basic operations:


Erosion:
Erosion:
AND
AND


Dilation:
Dilation:
OR
OR


Opening:
Opening:
dilation of the eroded image
dilation of the eroded image


Closing:
Closing:
erosion of the dilated image
erosion of the dilated image


Sceletonizing
Sceletonizing
:
:
computation of the
computation of the
sceleton
sceleton
Definitions
Definitions


Erosion
Erosion
s
s
:
:
I
I
s
s
E
E
=
=
{
{
p
p


Z
Z
²
²
|
|
E
E
p
p


I
I
}
}


Dilation
Dilation
r
r
:
:
I
I
r
r
E
E
=
=
{
{
p
p


Z
Z
²
²
|
|
E
E
p
p


I
I




}
}


Opening
Opening
°
°
:
:
(
(
I
I
°
°
E
E
)
)
n
n
=
=
(
(
(
(
I
I
s
s
E
E
)
)
n
n
r
r
E
E
)
)
n
n


Closing
Closing


:
:
(
(
I
I


E
E
)
)
n
n
=
=
(
(
(
(
I
I
r
r
E
E
)
)
n
n
s
s
E
E
)
)
n
n
Example
Example


Image
Image


Template
Template
Erosion
Erosion


Image
Image


Template
Template
Dilation
Dilation


Image
Image


Template
Template
Opening
Opening


Image
Image


Template
Template
Closing
Closing


Image
Image


Template
Template
Skeletonizing
Skeletonizing


In discrete domains:
In discrete domains:
sceleton
sceleton
is ambiguous
is ambiguous
Global
Global
transform
transform


Fourier
Fourier
transform
transform
Global transform
Global transform


Wavelet transform
Wavelet transform


Fourier transform
Fourier transform


extraction of frequency components
extraction of frequency components
Example: Fourier domain filtering
Example: Fourier domain filtering


Input data
Input data


Unwinded
Unwinded
image
image


Preprocessing
Preprocessing


Segmentation
Segmentation


Classification
Classification
Example
Example
: Fourier
: Fourier
domain
domain
filtering
filtering
original
original
Fourier
Fourier
power
power
spectrum
spectrum
modified
modified
Fourier
Fourier
power
power
spektrum
spektrum
result
result
Summary: image enhancement
Summary: image enhancement


What is image enhancement?
What is image enhancement?


improvement of image quality
improvement of image quality


task dependent
task dependent


image dependent
image dependent


Terminology
Terminology


optimization
optimization
-
-
any improvement
any improvement


enhancement
enhancement
-
-
qualitative improvement
qualitative improvement


correction
correction
-
-
quantitative improvement, relative
quantitative improvement, relative
(addresses systematic
(addresses systematic
artefacts
artefacts
)
)


calibration
calibration
-
-
quantitative improvement, absolute
quantitative improvement, absolute


registration
registration
-
-
correction of differences
correction of differences
(two or more images)
(two or more images)
Summary: image enhancement
Summary: image enhancement


Which methods are used?
Which methods are used?


local
local


regional
regional


global
global


Most frequently used: spatial filtering (regional)
Most frequently used: spatial filtering (regional)


non
non
-
-
linear
linear


context adaptive
context adaptive


How to evaluate image enhancement?
How to evaluate image enhancement?


sufficient number of representative images
sufficient number of representative images


statistical analyses
statistical analyses