Multiresolution image processing

paradepetAI and Robotics

Nov 5, 2013 (3 years and 7 months ago)

49 views

Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 1
Multiresolutionimage processing

Laplacianpyramids

Some applications of Laplacianpyramids

Discrete Wavelet Transform (DWT)

Wavelet theory

Wavelet image compression
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 2
Image pyramids
[Burt, Adelson, 1983]
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 3
Image pyramid example
original image
Gaussian pyramid
Laplacian pyramid
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 4
Overcomplete representation
2
Number of samples in Laplacian or Gaussian pyramid =
1114
1... x number of ori
g
inal ima
g
e samples
4443
P
⎛⎞
++++≤
⎜⎟
⎝⎠
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 5
LoGvs. DoG
Laplacianof Gaussian
Difference of Gaussians
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 6
Image processing with Laplacianpyramid
Analysis
Synthesis
Interpolator
Interpolator
Subsampling
+
+
-
-
Filtering
Filtering
Interpolator
Interpolator
Subsampling
Input
picture
Processed
picture












Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 7
Expanded Laplacianpyramid
Analysis
Synthesis
+
+
-
-
Filtering
Filtering
Input
picture
Processed
picture












Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 8
Mosaicingin the image domain
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 9
Mosaicingby blending Laplacian pyramids
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 10
Expanded Laplacian pyramids
Original: begonia
Original: begonia
Original: dahlia
Original: dahlia
Gaussian level 2
Gaussian level 2
Gaussian level 2
Gaussian level 2
Laplacian level 2
Laplacian level 2
Laplacian level 2
Laplacian level 2
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 11
Blending Laplacian pyramids
Level 0
Level 0
Level 2
Level 2
Level 4
Level 4
Level 6
Level 6
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 12
Image analysis with Laplacianpyramid
Analysis
Interpolator
Interpolator
Subsampling
+
+
-
-
Filtering
Filtering
Subsampling
Input
picture

Recognition/
detection/
segmentation
result
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 13
Multiscaleedge detection
Zero-crossings of Laplacian images of different scales
Spurious edges removed
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 14
Multiscaleface detection
Input
Network
Outpu
t
subsampling
PreprocessingNeural network
pixels
20 by 20
Extracted windowInput image pyramid
(20 by 20 pixels)
Correct lightingHistogram equalizationReceptive fields
Hidden units
[Rowley, Baluja, Kanade, 1995]
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 15
Example: multiresolutionnoise reduction
OriginalNoisyNoise reduction:
3-level Haartransform
no subsampling
w/ soft coring
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 16
1-d Discrete Wavelet Transform

Recursive application of a two-band filter bank to the
lowpassband of the previous stage yields octave band
splitting:
frequency
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 17
Cascaded analysis / synthesis filterbanks
0
h
0
g
1
g
1
h
0
g
1
g
0
h
1
h
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 18
2-d Discrete Wavelet Transform
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
...etc
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 19
2-d Discrete Wavelet Transform example
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 20
2-d Discrete Wavelet Transform example
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 21
2-d Discrete Wavelet Transform example
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 22
2-d Discrete Wavelet Transform example
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 23
2-d Discrete Wavelet Transform example
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 24
Review
: Z-transform and subsampling

Generalization of the discrete-time Fourier transform

Fourier transform on unit circle: substitute

Downsamplingand upsamplingby factor 2
()
[]
; ;
n
n
x
zxnzzrzr


−+
=−∞
=∈<<

C
j
z
e
ω
=
2
2
(
)
x
z
()()
1
2
x
zxz+−




Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 25
Two-channel filterbank

Aliasing cancellation if :
01
10
()()
()()
gzhz
gzhz
=−

=−
[]
[]
[]
[]
()
000111
00110011
11
ˆ
()()()()()()()()()()()
22
11
()()()()()()()()()
22
xz
g
zhzxzhzxz
g
zhzxzhzxz
hz
g
zhz
g
zxzhz
g
zhz
g
zxz
=+−−++−−
=++−+−−
Aliasing
2
2
2
2
0
h
1
h
0
g
1
g
(
)
x
z
(
)
ˆ
x
z
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 26
Example: two-channel filter bank with
perfect reconstruction

Impulse responses, analysis filters:
Lowpass
highpass

Impulse responses, synthesis filters
Lowpass
highpass

Mandatory in JPEG2000

Frequency responses:
|g |
0
|h |
0
1
|h |
1
|g |
π
2
π
1
2
0
0
Frequency
Frequency response
11311
,,,,
42224
−−
⎛⎞
⎜⎟
⎝⎠
111
,,
424

⎛⎞
⎜⎟
⎝⎠
11311
,,,,
42224

⎛⎞
⎜⎟
⎝⎠
111
,,
424
⎛⎞
⎜⎟
⎝⎠
“Biorthogonal5/3 filters”
“LeGallfilters”
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 27
Lifting

Analysis filters

L
“lifting steps”

First step can be interpreted as prediction of odd samples
from the even samples
K0
1
λ
2
λ
1L
λ

L
λ
Σ
Σ
Σ
Σ
K1
[
]
even samples 2
x
n
[]
odd samples
21xn+
0
low band
y
1
hi
g
h band
y
[Sweldens1996]
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 28
Lifting (cont.)

Synthesis filters

Perfect reconstruction (biorthogonality) is directly build into
lifting structure

Powerful for both implementation and filter/wavelet design
1
λ
2
λ
1L
λ

L
λ
Σ
Σ
Σ
Σ
[
]
even samples 2
x
n
[]
odd samples
21xn+
0
low band
y
1
hi
g
h band
y
1
0
K

1
1
K

-
-
-
-
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 29
Example: lifting implementation of 5/3 filters
[
]
even samples 2
x
n
(
)
1
2
z

+
1
1
4
z

+
Σ
Σ
1/2
[]
odd samples
21xn+
0
low band
y
1
high band y
Verify by considering response to unit impulse in even and
odd input channel.
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 30
Conjugate quadraturefilters

Achieve aliasing cancelationby

Impulse responses

With perfect reconstruction: orthonormalsubbandtransform!

Perfect reconstruction: find power complementary prototype filter
(
)
()
()
1
00
11
11
()()
()
hzgzfz
hzgzzfz

−−
=≡
==−
[Smith, Barnwell, 1986]
Prototype filter
[
]
[
]
[
]
[][]
()()
00
1
11
11
k
hkgkfk
hkgkfk
+
=−=
=−=−−+
⎡⎤
⎣⎦
()
()
(
)
2
2
2
j
j
FeFe
ωπ
ω
±
+
=
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 31
Wavelet bases
(
)
(
)
()
()
()
()
()
2
2
2
Consider Hilbert space of finite-energy functions .
Wavelet basis for : family of linearly independent functions
22
that span . Hence any signal
m
mm
n
xt
ttn
ψψ
−−
=
=−

x
x
\
\
\
L
L
L
()
()
[]
()
2
can be written as

mm
n
mn
yn
ψ
∞∞
=−∞=−∞
=
∑∑
x
\L
“mother
wavelet”
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 32
Multi-resolution analysis
()()()()()
()
()
()
()
{}
()
()
21012
2
2
0
Upward compl
Nested subspaces

eteness
Downward complete



ness
Self-similari t fy i
m
m
m
m
VVVVV
V
V
xtV
−−


⊂⊂⊂⊂⊂⊂⊂
=
=

0
……
\
\


Z
Z
L
L
()
()
()
()
()
()
()()()
{}
()
00
0
f 2
iff for all
There exists a "scaling function" with integer translates -
such that forms an orthonormal basis f
Translation i
or
nvaria

nce

m
m
n
n
n
xtV
xtVxtnVn
tttn
V
ϕϕϕ
ϕ



∈−∈∈
=
]
Z

Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 33
MultiresolutionFourier analysis
ω
()
{}
()
span
p
n
p
n
V
ϕ
=
()
{}
()
11
span
pp
n
n
V
ϕ
−−
=
()
{}
()
22
span
pp
n
n
V
ϕ
−−
=
()
{}
()
33
span
pp
n
n
V
ϕ
−−
=
()
{}
()
span
p
n
p
n
W
ψ
=
()
{
}
()
11
span
pp
n
n
W
ψ


=
()
{
}
()
22
span
pp
n
n
W
ψ
−−
=
()
{
}
()
33
span
pp
n
n
W
ψ


=
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 34
Relation to subbandfilters
()()
()
[]
()
()
()
[]
()
[]
()()
1
01
1
00

00
0
linear combination
of scaling functions in
Since , recursive definition of scaling function
22
Orthonormality
n,

n
nn
n
V
VV
tgntgntn
ϕϕϕ
δϕϕ


∞∞

=−∞=−∞

==−
=
∑∑


[]
()
()
[]
()
()
[][]
()()
[][]
[]
0
11
002
11
0000
,
k
unit norm and orthogonal
to its 2-translates: corresponds
to synthesis lowpass filter of
orthonormal subband tra

2,2
ijn
ij
ij
iji
g
gitgjtdt
gigjngigin
ϕϕ
ϕϕ

−−
+
−∞
−−
⎛⎞
=
⎜⎟
⎝⎠
=−=−
∑∑

∑∑
nsform


Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 35
Wavelets from scaling functions
()()()
()()()()()
()
{}
()()
()
[]
()
()
1
1
001
1
1
linear combination
of scaling functions
is orthogonal complement of in
and
Orthonormal wavelet basis for

ppp
ppppp
n
n
n
WVV
WVWVV
WV
tgnt
ψ
ψϕ





=−∞
⊥∪=

=

()
[]
()
[]
()()
()
{}
()
{}
(
)
1
1

1
10
001
in
22
Using conjugate quadrature high-pass synthesis filter

The mutually orthonormal
11
Easy
functions, and , together span .
to e
n
n
n
n
n
nn
V
gntn
gngn
V
ϕ
ϕ
ψ





+

=−
=−
=−−−⎡⎤
⎣⎦



ZZ
()
()
{}
()
2
,
xtend to dilated versions of to construct orthonormal wavelet basis
for .

m
n
nm
t
ψ
ψ

\
Z
L
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 36
Calculating wavelet coefficients for a continuous signal

Signal synthesis by discrete filter bank

Signal analysis by analysis filters h0[k], h1[k]

Discrete wavelet transform
(
)
(
)
(
)
[
]
(
)
(
)
[
]
(
)
(
)
()
()
()()
()
()
()
()
()
[]
()
()
()
()
()
[]
()
()
()
()
11
11
00000
00
1111
01111
01
Suppose continuous signal
Write as superposition of and


n
nn
nn
ij
xtV
wtW
xt
y
ntn
y
nV
xtVwtW
xtyiyj
ϕϕ
ϕψ
∈∈
∈∈


=−=∈
∈∈
=+


∑∑




ZZ
ZZ
()()
[][]
()
[][]
()
[]
0
0
011
0011
2 2
n
nij
yn
yngniyjgni
ϕ
∈∈∈
⎛⎞
=−+−
⎜⎟
⎝⎠
∑∑∑


ZZZ
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 37
0
h
0
g
1
g
1
h
0
g
1
g
0
h
1
h
1-d Discrete Wavelet Transform
()
[]
0
0
y
n
()
[
]
1
0
y
n
(
)
[
]
1
1
y
n
(
)
[
]
2
0
yn
(
)
[
]
2
1
yn
(
)
[
]
1
0
y
n
(
)
[
]
0
0
y
n
Sampling
(
)
xt
()
Interpolation
t
ϕ
()
xt
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 38
Example: Daubechieswavelet, order 2
ϕ
ψ
0
h
0
g
1
h
1
g
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 39
Example: Daubechieswavelet, order 9
ϕ
ψ
0
h
0
g
1
h
1
g
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 40
Comparison JPEG vs. JPEG2000
Lenna, 256x256 RGB
Baseline JPEG: 4572 bytes
Lenna, 256x256 RGB
JPEG-2000: 4572 bytes
Bernd Girod: EE368Digital Image Processing
Multiresolution Image Processingno. 41
Comparison JPEG vs. JPEG2000
JPEG with optimized Huffman tables
8268 bytes
JPEG2000
8192 bytes