Image Denoising using

dealerdeputyAI and Robotics

Nov 25, 2013 (3 years and 6 months ago)

62 views

Image Denoising using
Locally Learned Dictionaries

Priyam Chatterjee

Peyman Milanfar


Dept. of Electrical Engineering

University of California, Santa Cruz

Computational Imaging VII


20 Jan, 2009

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

2

Overview


Data Model




Kernel Regression for Denoising




Denoising with Locally Learned Dictionaries (K
-
LLD)




Results




Conclusions

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

3

Data Model


Pointwise data model






Patchwise model

Locally smooth function

to be estimated

Zero
-
mean

I.I.D. noise

Observation

denoted as


20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

4

Steering Kernel Regression (SKR)


Optimization problem




Solution:
Nonlinear

filters


Polynomial
basis

Data
-
dependent
weights

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

5

SKR Weights


Weights based on pixel “self
-
similarity” in
a local patch





Covariance matrix takes into account:
orientation and strength of edges

Gradient
Covariance

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

7

SKR Weights

Note how the weights adapt to the underlying image structure

Noisy

Noise
-
free

H. Takeda, S. Farsiu, and P. Milanfar, “Kernel Regression for Image Processing and Reconstruction”,

IEEE Trans. on Image Processing, vol. 16, no. 2, pp. 349
-
366, February 2007.

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

8

Now we extend it …..



is fixed order, everywhere
--

not
depending on underlying image structure



Lower orders fit flat regions, higher order for texture
and fine details



Global

dictionary does not adapt to
local

image characteristics



Dictionary atoms should capture underlying
local

image structure

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

9

Denoising with
L
ocally
L
earned
D
ictionaries (K
-
LLD
)


Identify dictionary which best captures
underlying geometric structure



Similar structures will have similar dictionary,
similar weights



Cluster image based on
geometric similarity
(
K
-
Means

on the SKR weights)



Learn dictionary and order of regression for
each cluster

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

10

K
-
LLD: Algorithm Outline

Calculate


weights

Learn

dictionaries

Clustering

Iterate

Noisy Image

Kernel
Regression

Denoised
Image

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

11

Class 1

Class K

Clustering Stage

K
-
LLD : Algorithm Outline

Dictionary Selection Stage

Noisy Img

Calculate Steering

Weights

Coefficient

Calculation Stage

Denoised
Img

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

12

Segment

Image

Clustering Stage

K
-
Means

Class 1

Class K

K
-
LLD : Algorithm Outline

Dictionary Selection Stage

Noisy Img

Calculate Steering

Weights

Coefficient

Calculation Stage

Denoised
Img

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

13

Clustering Stage


Objective : Cluster image based on geometric



similarity of underlying data



Feature Selection



What features capture data geometry ?



Distance Metric


What metric captures distance between features ?



Clustering Algorithm


What algorithm segments the image best ?

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

14

K
-
Means for Clustering


Features : normalized steering wts



Distance Metric : L
2



Initialization : Randomly initialize cluster
centers



Run K
-
Means multiple times and select result
that minimizes within
-
cluster distance

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

15

Noise
-
free

Noisy

Clustering the noise
-
free image

Clustering the noisy image

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

16

Segment

Image

Clustering Stage

K
-
Means

Class 1

Class K

K
-
LLD : Algorithm Outline

Dictionary Selection Stage

Noisy Img

Calculate Steering

Weights

Coefficient

Calculation Stage

Denoised
Img

Dictionary Selection Stage

PCA

Form

Dictionary

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

17

Dictionary Selection


Represent each patch in the
k
th
cluster




Variable Proj.

Solved by
PCA

Mean patch of
k
-
th cluster

Enforce

orthonormality

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

18

Dictionary Selection


PCA in each cluster to form a dictionary



Describe data without fitting noise






Number of atoms based on cluster geometry


Clusters with flat regions need fewer atoms, finer
details need more

constant

Singular values

patch size

No. of atoms

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

19

Example : House Image

Dictionary atoms for AWGN of std. dev. 15

Cluster

Atom 1

Atom 2

Atom 3

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

21

Example : Noise
-
free clustering

Cluster

Atom 1

Atom 2

Atom 3

Atom 4

Few of the atoms in the dictionaries for different clusters

Brick
facade

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

23

Algorithm Outline

Segment

Image

Clustering Stage

Class 1

Class K

K
-
Means

Noisy Img

Calculate Steering

Weights

Coefficient

Calculation Stage

Denoised
Img

Dictionary Selection Stage

PCA

Form

Dictionary

Kernel Regression

Denoised
Img

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

24


Kernel Regression




Weighted least squares solution




Final estimate

Coefficient Calculation

center pixel of
patch .

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

25

Kernel Regression

Denoised
Img

Algorithm Outline

Segment

Image

Clustering Stage

Class 1

Class K

K
-
Means

Noisy Img

Calculate Steering

Weights

Dictionary Selection Stage

PCA

Form

Dictionary

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

26

Iteration


Re
-
learn weights (features) from denoised
image



Perform clustering of updated image using
new features



Learn dictionary from updated image



Kernel regression on input noisy image


Preserves edges and finer structures

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

28

K
-
LLD: Algorithm Outline

Calculate


weights

Learn

dictionaries

Clustering

Iterate

Noisy Image

Kernel
Regression

Denoised
Image

Original Noisy
Image

Results


AWG noise (std dev 25)

K
-
LLD, MSE 96.95
SSIM 0.825

BM3D, MSE 88.82
SSIM 0.841

Original Parrot Image

K
-
SVD, MSE 101.54
SSIM 0.826

SKR, MSE 99.96
SSIM 0.826

Noisy Image

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

31

More Results

MSE

SSIM

Results


Real noise & color

ISKR

K
-
LLD

BM3D

Color Results

ISKR, Order 2

BM3D

Original Image

K
-
LLD

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

35

Thank you




P. Chatterjee and P. Milanfar, “
Clustering
-
based Denoising with Locally Learned

Dictionaries
”, Accepted for publication in IEEE Trans. Image Processing


Available at:





http://www.ee.ucsc.edu/~milanfar

20 Jan, 2009

Image Denoising using Locally Learned Dictionaries

36

Iterative Scheme


Why iterate ?



Weights true to underlying structure in
presence of lesser noise



Better weights means better clustering



Dictionary captures underlying data better
when learned on less noisy image