# Some Blind Deconvolution

AI and Robotics

Nov 6, 2013 (4 years and 8 months ago)

72 views

Some Blind Deconvolution
Techniques in Image Processing

Tony Chan

Math Dept., UCLA

Astronomical Data Analysis Software & Systems

Conference Series 2004

-
27, 2004

Joint work with
Frederick Park and
Andy M. Yip

2

Outline

Part I:

Total Variation Blind Deconvolution

Part II:

Simultaneous TV Image Inpainting and
Blind Deconvolution

Part III:

Automatic Parameter Selection for TV
Blind Deconvolution

Caution: Our work not developed specifically for Astronomical images

3

Blind Deconvolution Problem

=

+

Observed
image

Unknown
true image

Unknown point

Unknown
noise

Goal:

Given
u
obs
, recover both
u
orig

and
k

obs
u
orig
u

k
4

Typical PSFs

PSFs w/ sharp edges:

PSFs w/ smooth transitions

5

Total Variation Regularization

dx
x
u
u
TV

)
(
)
(

Deconvolution ill
-
posed: need regularization

Total variation Regularization:

dx
x
k
k
TV

)
(
)
(

The characteristic function
of
D

with height
h

(jump):

TV = Length(
∂D
)

h

TV doesn

t penalize jumps

Co
-
area Formula:

D

h

dr
ds
f
dx
u
f
n
R
r
u
)
(
|
|
}
{



6

TV Blind Deconvolution Model

TV
TV
obs
k
u
k
u
u
k
u
k
u
F
2
1
2
,
)
,
(
min

)
,
(
)
,
(
,
1
)
,
(
,
0
,
y
x
k
y
x
k
dxdy
y
x
k
k
u

Subject to:

Objective:

(C. and Wong (IEEE TIP, 1998))

1

determined by signal
-
to
-
noise ratio

2

parameterizes a

family of solutions, corresponds to
different spread of the reconstructed PSF

Alternating Minimization Algorithm:

Globally convergent with H
1

regularization.

)
,
(
min
)
,
(
)
,
(
min

)
,
(
1
1
1
1
k
u
F
k
u
F
k
u
F
k
u
F
n
k
n
n
n
u
n
n

7

Blind v.s. non
-
Blind Deconvolution

Observed Image
noise
-
free

An out
-
of
-
focus blur is recovered automatically

Recovered blind deconvolution images almost as good as
non
-
blind

Edges well
-
recovered in image and PSF

non
-
Blind

Recovered Image

PSF

Blind

1

= 2

10

6
,

2

= 1.5

10

5

Clean image

True PSF

8

Blind v.s. non
-
Blind Deconvolution:
High Noise

Observed Image
SNR=5 dB

non
-
Blind

Clean image

True PSF

Blind

An out
-
of
-
focus blur is recovered automatically

Even in the presence of
high noise level
, recovered
images from blind deconvolution are almost as good as
those recovered with the exact PSF

1

= 2

10

5
,

2

= 1.5

10

5

9

Controlling Focal
-
Length

Recovered Images are 1
-
parameter family w.r.t.

2

Recovered Blurring Functions

(

1

=
2

10

6
)

0

1

1
0

7

1

1
0

5

1

1
0

4

2
:

The parameter

2

controls the focal
-
length

10

Generalizations to Multi
-
Channel Images

Inter
-
Channel Blur Model

Color image (Katsaggelos et al, SPIE 1994):

B
obs
G
obs
R
obs
B
G
R
k
k
k
k
k
k
k
k
k
u
u
u
u
u
u
H
H
H
H
H
H
H
H
H
noise
1
2
2
2
1
2
2
2
1
7
5
7
1
7
1
7
1
7
5
7
1
7
1
7
1
7
5
obs
k
u
noise
u
H

k
1
: within channel blur

k
2
: between channel blur

m
-
channel TV
-
norm (Color
-
TV)

(C. & Blomgren, IEEE TIP ‘98)

2
2
1
2
)
(
m
i
TV
i
m
k
k
TV

m
i
TV
i
m
u
u
TV
1
2
)
(
11

Original image

Out
-
of
-
focus blurred blind non
-
blind

Gaussian blurred blind non
-
blind

Examples of Multi
-
Channel Blind Deconvolution

(C. and Wong (SPIE, 1997))

Blind is as good as non
-
blind

Gaussian blur is harder to recover (zero
-
crossings in frequency domain)

12

TV Blind Deconvolution Patented!

13

Outline

Part I:

Total Variation Blind Deconvolution

Part II:

Simultaneous TV Image Inpainting and
Blind Deconvolution

Part III:

Automatic Parameter Selection for TV
Blind Deconvolution

14

TV Inpainting Model

(C. & Shen SIAP 2001)

E
D
E
dxdy
u
u
dxdy
u
u
J
,
|
|
2
|
|
]
[
2
0

,
0
)
(
|
|
0

u
u
u
u
e

.
0
;
,
,
D
z
E
z
e

Graffiti Removal

Scratch Removal

15

Missing Regions

Blur

Calibration errors of devices

Atmospheric turbulence

Motion of objects/camera

Missing regions

Scratches

Occlusion

Defects in films/sensors

+

16

Problems with Inpaint then Deblur

Inpaint first

reduce plausible solutions

Original Signal

Blurring func.

Original Signal

Blurring func.

Blurred Signal

Blurred Signal

=

=

Blurred + Occluded

Blurred + Occluded

=

17

Problems with Deblur then Inpaint

Different BC’s correspond to different image intensities in
inpaint region.

Most local BC’s do not respect global geometric structures

Original

Occluded

Support of PSF

Dirichlet

Neumann

Inpainting

18

The Joint Model

D
o

---

the region where the image is observed

D
i

---

the region to be inpainted

A natural combination of TV deblur + TV inpaint

No BC’s needed for inpaint regions

2 parameters (can incorporate automatic
parameter selection techniques)

Inpainting
take place

Coupling of
inpainting &
deblur

19

Simulation Results (1)

Restored

Zoom
-
in

The vertical strip is completed

Not completed

Use higher order inpainting methods

E.g. Euler’s elastica, curvature
driven diffusion

20

Simulation Results (2)

Observed

Restored

Deblur then inpaint

(many artifacts)

Inpaint then deblur

(many ringings)

Original

21

Boundary Conditions for Regular Deblurring

Original image
domain and
artificial boundary
outside the scene

Dirichlet B.C
.

Periodic B.C.

Neumann B.C.

Inpainting B.C.

22

23

Outline

Part I:

Total Variation Blind Deconvolution

Part II:

Simultaneous TV Image Inpainting and
Blind Deconvolution

Part III:

Automatic Parameter Selection for TV
Blind Deconvolution

(Ongoing Research)

24

Automatic Blind Deblurring
(ongoing research)

Recovered images: 1
-
parameter family wrt

2

Consider external info like sharpness to choose optimal

2

Problem: Find

2

automatically to recover best u & k

SNR = 15 dB

20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
Clean
image

observed
image

25

Motivation for Sharpness & Support

Preference for gradients with small support

20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
u

|
|
u

|
|
u

20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
Support of

26

Proposed Sharpness Evaluator

F(u)
small => sharp image with small support

F(u)
=0 for piecewise constant images

F(u) p
enalizes smeared edges

|
|
of
support

of

Area
)
(
u
u
F

20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
u

|
|
u

20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
Support of

27

Planets Example

20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
20
40
60
80
100
120
Rel. errors in u (
blue
) and
k (
red
) v.s.

2

Proposed Objective v.s.

2

Optimal Restored Image

Auto
-
focused Image

The minimum of the sharpness
function agrees with that of the
rel. errors of u and k

(minimizer of
sharpness func.)

(minimizer of rel.
error in u)

1
=0.02 (optimal)

10
5
10
6
10
7
0
0.5
1
1.5
10
5
10
6
10
7
1500
2000
2500
3000
3500
4000
4500
5000
28

Satellite Example

Rel. errors in u (
blue
) and
k (
red
) v.s.

2

Proposed Objective v.s.

2

Optimal Restored Image

Auto
-
focused Image

The minimum of the sharpness
function agrees with that of the
rel. errors of u and k

(minimizer of
sharpness func.)

(minimizer of rel.
error in u)

1
=0.3 (optimal)

10
5
10
6
10
7
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
20
40
60
80
100
120
20
40
60
80
100
120
10
5
10
6
10
7
2000
3000
4000
5000
6000
7000
8000
9000
10000
11000
20
40
60
80
100
120
20
40
60
80
100
120
29

Potential Applications to Astronomical Imaging

TV Blind Deconvolution

TV/Sharp edges useful?

Auto
-
focus: appropriate objective function?

How to incorporate a priori domain knowledge?

TV Blind Deconvolution + Inpainting

Other noise models: e.g. salt
-
and
-
pepper noise

30

References

1.
C. and C. K. Wong,
Total Variation Blind Deconvolution
, IEEE
Transactions on Image Processing, 7(3):370
-
375, 1998.

2.
C. and C. K. Wong,
Multichannel Image Deconvolution by Total
Variation Regularization
, Proc. to the SPIE Symposium on
Advanced Signal Processing: Algorithms, Architectures, and
Implementations, vol. 3162, San Diego, CA, July 1997, Ed.: F.
Luk.

3.
C. and C. K. Wong,
Convergence of the Alternating Minimization
Algorithm for Blind Deconvolution
, UCLA Mathematics
Department CAM Report 99
-
19
.

4.
R. H. Chan, C. and C. K. Wong,

Cosine Transform Based
Preconditioners for Total Variation Deblurring
, IEEE Trans. Image
Proc., 8 (1999), pp. 1472
-
1478

5.
C., A. Yip and F. Park,
Simultaneous Total Variation Image
Inpainting and Blind Deconvolution
, UCLA Mathematics
Department CAM Report 04
-
45
.