Some Blind Deconvolution

molassesitalianΤεχνίτη Νοημοσύνη και Ρομποτική

6 Νοε 2013 (πριν από 3 χρόνια και 5 μήνες)

39 εμφανίσεις

Some Blind Deconvolution
Techniques in Image Processing

Tony Chan

Math Dept., UCLA

Astronomical Data Analysis Software & Systems

Conference Series 2004

Pasadena, CA, October 24
-
27, 2004

Joint work with
Frederick Park and
Andy M. Yip

2

Outline

Part I:

Total Variation Blind Deconvolution


Part II:

Simultaneous TV Image Inpainting and
Blind Deconvolution


Part III:

Automatic Parameter Selection for TV
Blind Deconvolution


Caution: Our work not developed specifically for Astronomical images

3

Blind Deconvolution Problem

=



+

Observed
image

Unknown
true image

Unknown point
spread function

Unknown
noise

Goal:

Given
u
obs
, recover both
u
orig

and
k

obs
u
orig
u

k
4

Typical PSFs

PSFs w/ sharp edges:

PSFs w/ smooth transitions

5

Total Variation Regularization

dx
x
u
u
TV



)
(
)
(

Deconvolution ill
-
posed: need regularization


Total variation Regularization:

dx
x
k
k
TV



)
(
)
(

The characteristic function
of
D

with height
h

(jump):


TV = Length(
∂D
)

h


TV doesn

t penalize jumps


Co
-
area Formula:

D

h

dr
ds
f
dx
u
f
n
R
r
u
)
(
|
|
}
{









6

TV Blind Deconvolution Model

TV
TV
obs
k
u
k
u
u
k
u
k
u
F
2
1
2
,
)
,
(
min







)
,
(
)
,
(
,
1
)
,
(
,
0
,
y
x
k
y
x
k
dxdy
y
x
k
k
u






Subject to:

Objective:

(C. and Wong (IEEE TIP, 1998))



1

determined by signal
-
to
-
noise ratio



2

parameterizes a

family of solutions, corresponds to
different spread of the reconstructed PSF




Alternating Minimization Algorithm:






Globally convergent with H
1

regularization.


)
,
(
min
)
,
(
)
,
(
min

)
,
(
1
1
1
1
k
u
F
k
u
F
k
u
F
k
u
F
n
k
n
n
n
u
n
n






7

Blind v.s. non
-
Blind Deconvolution

Observed Image
noise
-
free


An out
-
of
-
focus blur is recovered automatically


Recovered blind deconvolution images almost as good as
non
-
blind


Edges well
-
recovered in image and PSF

non
-
Blind

Recovered Image

PSF

Blind


1

= 2

10

6
,

2

= 1.5

10

5

Clean image

True PSF

8

Blind v.s. non
-
Blind Deconvolution:
High Noise

Observed Image
SNR=5 dB

non
-
Blind

Clean image

True PSF

Blind


An out
-
of
-
focus blur is recovered automatically


Even in the presence of
high noise level
, recovered
images from blind deconvolution are almost as good as
those recovered with the exact PSF


1

= 2

10

5
,

2

= 1.5

10

5

9



Controlling Focal
-
Length

Recovered Images are 1
-
parameter family w.r.t.

2

Recovered Blurring Functions

(

1

=
2

10

6
)

0

1

1
0

7

1

1
0

5

1

1
0

4


2
:

The parameter

2

controls the focal
-
length

10

Generalizations to Multi
-
Channel Images


Inter
-
Channel Blur Model


Color image (Katsaggelos et al, SPIE 1994):

































B
obs
G
obs
R
obs
B
G
R
k
k
k
k
k
k
k
k
k
u
u
u
u
u
u
H
H
H
H
H
H
H
H
H
noise
1
2
2
2
1
2
2
2
1
7
5
7
1
7
1
7
1
7
5
7
1
7
1
7
1
7
5
obs
k
u
noise
u
H


k
1
: within channel blur

k
2
: between channel blur

m
-
channel TV
-
norm (Color
-
TV)

(C. & Blomgren, IEEE TIP ‘98)




2
2
1
2
)
(
m
i
TV
i
m
k
k
TV



m
i
TV
i
m
u
u
TV
1
2
)
(
11

Original image


Out
-
of
-
focus blurred blind non
-
blind


Gaussian blurred blind non
-
blind

Examples of Multi
-
Channel Blind Deconvolution

(C. and Wong (SPIE, 1997))



Blind is as good as non
-
blind



Gaussian blur is harder to recover (zero
-
crossings in frequency domain)

12

TV Blind Deconvolution Patented!

13

Outline

Part I:

Total Variation Blind Deconvolution


Part II:

Simultaneous TV Image Inpainting and
Blind Deconvolution


Part III:

Automatic Parameter Selection for TV
Blind Deconvolution


14

TV Inpainting Model

(C. & Shen SIAP 2001)








E
D
E
dxdy
u
u
dxdy
u
u
J
,
|
|
2
|
|
]
[
2
0

,
0
)
(
|
|
0
















u
u
u
u
e

.
0
;
,
,
D
z
E
z
e






Graffiti Removal


Scratch Removal

15

Images Degraded by Blurring and
Missing Regions


Blur


Calibration errors of devices


Atmospheric turbulence


Motion of objects/camera


Missing regions


Scratches


Occlusion


Defects in films/sensors

+

16

Problems with Inpaint then Deblur


Inpaint first


reduce plausible solutions


Should pick the solution using more information

Original Signal

Blurring func.



Original Signal

Blurring func.



Blurred Signal

Blurred Signal

=

=

Blurred + Occluded

Blurred + Occluded





=

17

Problems with Deblur then Inpaint


Different BC’s correspond to different image intensities in
inpaint region.


Most local BC’s do not respect global geometric structures

Original

Occluded

Support of PSF

Dirichlet

Neumann

Inpainting

18

The Joint Model


D
o

---

the region where the image is observed


D
i

---

the region to be inpainted


A natural combination of TV deblur + TV inpaint


No BC’s needed for inpaint regions


2 parameters (can incorporate automatic
parameter selection techniques)

Inpainting
take place

Coupling of
inpainting &
deblur

19

Simulation Results (1)

Degraded

Restored

Zoom
-
in



The vertical strip is completed



Not completed



Use higher order inpainting methods



E.g. Euler’s elastica, curvature
driven diffusion

20

Simulation Results (2)

Observed

Restored

Deblur then inpaint

(many artifacts)

Inpaint then deblur

(many ringings)

Original

21

Boundary Conditions for Regular Deblurring

Original image
domain and
artificial boundary
outside the scene

Dirichlet B.C
.

Periodic B.C.

Neumann B.C.

Inpainting B.C.

22

23

Outline

Part I:

Total Variation Blind Deconvolution


Part II:

Simultaneous TV Image Inpainting and
Blind Deconvolution


Part III:

Automatic Parameter Selection for TV
Blind Deconvolution


(Ongoing Research)

24

Automatic Blind Deblurring
(ongoing research)



Recovered images: 1
-
parameter family wrt

2


Consider external info like sharpness to choose optimal

2

Problem: Find

2

automatically to recover best u & k


SNR = 15 dB

20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
Clean
image

observed
image

25

Motivation for Sharpness & Support


Sharpest image has large gradients


Preference for gradients with small support

20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
u

|
|
u

|
|
u

20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
Support of

26

Proposed Sharpness Evaluator


F(u)
small => sharp image with small support


F(u)
=0 for piecewise constant images


F(u) p
enalizes smeared edges


|
|
of
support

of

Area
)
(
u
u
F


20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
u

|
|
u

20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
Support of

27

Planets Example

20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
Rel. errors in u (
blue
) and
k (
red
) v.s.

2


Proposed Objective v.s.

2

Optimal Restored Image

Auto
-
focused Image

The minimum of the sharpness
function agrees with that of the
rel. errors of u and k

(minimizer of
sharpness func.)


(minimizer of rel.
error in u)



1
=0.02 (optimal)

10
5
10
6
10
7
0
0.5
1
1.5
10
5
10
6
10
7
1500
2000
2500
3000
3500
4000
4500
5000
28

Satellite Example

Rel. errors in u (
blue
) and
k (
red
) v.s.

2


Proposed Objective v.s.

2

Optimal Restored Image

Auto
-
focused Image

The minimum of the sharpness
function agrees with that of the
rel. errors of u and k

(minimizer of
sharpness func.)


(minimizer of rel.
error in u)



1
=0.3 (optimal)

10
5
10
6
10
7
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
20
40
60
80
100
120
20
40
60
80
100
120
10
5
10
6
10
7
2000
3000
4000
5000
6000
7000
8000
9000
10000
11000
20
40
60
80
100
120
20
40
60
80
100
120
29

Potential Applications to Astronomical Imaging


TV Blind Deconvolution


TV/Sharp edges useful?


Auto
-
focus: appropriate objective function?


How to incorporate a priori domain knowledge?


TV Blind Deconvolution + Inpainting


Other noise models: e.g. salt
-
and
-
pepper noise

30

References

1.
C. and C. K. Wong,
Total Variation Blind Deconvolution
, IEEE
Transactions on Image Processing, 7(3):370
-
375, 1998.

2.
C. and C. K. Wong,
Multichannel Image Deconvolution by Total
Variation Regularization
, Proc. to the SPIE Symposium on
Advanced Signal Processing: Algorithms, Architectures, and
Implementations, vol. 3162, San Diego, CA, July 1997, Ed.: F.
Luk.


3.
C. and C. K. Wong,
Convergence of the Alternating Minimization
Algorithm for Blind Deconvolution
, UCLA Mathematics
Department CAM Report 99
-
19
.

4.
R. H. Chan, C. and C. K. Wong,

Cosine Transform Based
Preconditioners for Total Variation Deblurring
, IEEE Trans. Image
Proc., 8 (1999), pp. 1472
-
1478

5.
C., A. Yip and F. Park,
Simultaneous Total Variation Image
Inpainting and Blind Deconvolution
, UCLA Mathematics
Department CAM Report 04
-
45
.