Image Denoising using
Locally Learned Dictionaries
Priyam Chatterjee
Peyman Milanfar
Dept. of Electrical Engineering
University of California, Santa Cruz
Computational Imaging VII
–
20 Jan, 2009
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
2
Overview
Data Model
Kernel Regression for Denoising
Denoising with Locally Learned Dictionaries (K

LLD)
Results
Conclusions
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
3
Data Model
Pointwise data model
Patchwise model
Locally smooth function
to be estimated
Zero

mean
I.I.D. noise
Observation
denoted as
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
4
Steering Kernel Regression (SKR)
Optimization problem
Solution:
Nonlinear
filters
Polynomial
basis
Data

dependent
weights
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
5
SKR Weights
Weights based on pixel “self

similarity” in
a local patch
Covariance matrix takes into account:
orientation and strength of edges
Gradient
Covariance
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
7
SKR Weights
Note how the weights adapt to the underlying image structure
Noisy
Noise

free
H. Takeda, S. Farsiu, and P. Milanfar, “Kernel Regression for Image Processing and Reconstruction”,
IEEE Trans. on Image Processing, vol. 16, no. 2, pp. 349

366, February 2007.
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
8
Now we extend it …..
is fixed order, everywhere

not
depending on underlying image structure
•
Lower orders fit flat regions, higher order for texture
and fine details
Global
dictionary does not adapt to
local
image characteristics
•
Dictionary atoms should capture underlying
local
image structure
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
9
Denoising with
L
ocally
L
earned
D
ictionaries (K

LLD
)
Identify dictionary which best captures
underlying geometric structure
Similar structures will have similar dictionary,
similar weights
Cluster image based on
geometric similarity
(
K

Means
on the SKR weights)
Learn dictionary and order of regression for
each cluster
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
10
K

LLD: Algorithm Outline
Calculate
weights
Learn
dictionaries
Clustering
Iterate
Noisy Image
Kernel
Regression
Denoised
Image
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
11
Class 1
Class K
Clustering Stage
K

LLD : Algorithm Outline
Dictionary Selection Stage
Noisy Img
Calculate Steering
Weights
Coefficient
Calculation Stage
Denoised
Img
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
12
Segment
Image
Clustering Stage
K

Means
Class 1
Class K
K

LLD : Algorithm Outline
Dictionary Selection Stage
Noisy Img
Calculate Steering
Weights
Coefficient
Calculation Stage
Denoised
Img
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
13
Clustering Stage
Objective : Cluster image based on geometric
similarity of underlying data
•
Feature Selection
•
What features capture data geometry ?
•
Distance Metric
•
What metric captures distance between features ?
•
Clustering Algorithm
•
What algorithm segments the image best ?
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
14
K

Means for Clustering
Features : normalized steering wts
Distance Metric : L
2
Initialization : Randomly initialize cluster
centers
Run K

Means multiple times and select result
that minimizes within

cluster distance
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
15
Noise

free
Noisy
Clustering the noise

free image
Clustering the noisy image
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
16
Segment
Image
Clustering Stage
K

Means
Class 1
Class K
K

LLD : Algorithm Outline
Dictionary Selection Stage
Noisy Img
Calculate Steering
Weights
Coefficient
Calculation Stage
Denoised
Img
Dictionary Selection Stage
PCA
Form
Dictionary
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
17
Dictionary Selection
Represent each patch in the
k
th
cluster
Variable Proj.
Solved by
PCA
Mean patch of
k

th cluster
Enforce
orthonormality
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
18
Dictionary Selection
PCA in each cluster to form a dictionary
Describe data without fitting noise
Number of atoms based on cluster geometry
•
Clusters with flat regions need fewer atoms, finer
details need more
constant
Singular values
patch size
No. of atoms
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
19
Example : House Image
Dictionary atoms for AWGN of std. dev. 15
Cluster
Atom 1
Atom 2
Atom 3
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
21
Example : Noise

free clustering
Cluster
Atom 1
Atom 2
Atom 3
Atom 4
Few of the atoms in the dictionaries for different clusters
Brick
facade
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
23
Algorithm Outline
Segment
Image
Clustering Stage
Class 1
Class K
K

Means
Noisy Img
Calculate Steering
Weights
Coefficient
Calculation Stage
Denoised
Img
Dictionary Selection Stage
PCA
Form
Dictionary
Kernel Regression
Denoised
Img
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
24
Kernel Regression
Weighted least squares solution
Final estimate
Coefficient Calculation
center pixel of
patch .
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
25
Kernel Regression
Denoised
Img
Algorithm Outline
Segment
Image
Clustering Stage
Class 1
Class K
K

Means
Noisy Img
Calculate Steering
Weights
Dictionary Selection Stage
PCA
Form
Dictionary
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
26
Iteration
Re

learn weights (features) from denoised
image
Perform clustering of updated image using
new features
Learn dictionary from updated image
Kernel regression on input noisy image
•
Preserves edges and finer structures
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
28
K

LLD: Algorithm Outline
Calculate
weights
Learn
dictionaries
Clustering
Iterate
Noisy Image
Kernel
Regression
Denoised
Image
Original Noisy
Image
Results
–
AWG noise (std dev 25)
K

LLD, MSE 96.95
SSIM 0.825
BM3D, MSE 88.82
SSIM 0.841
Original Parrot Image
K

SVD, MSE 101.54
SSIM 0.826
SKR, MSE 99.96
SSIM 0.826
Noisy Image
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
31
More Results
MSE
SSIM
Results
–
Real noise & color
ISKR
K

LLD
BM3D
Color Results
ISKR, Order 2
BM3D
Original Image
K

LLD
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
35
Thank you
P. Chatterjee and P. Milanfar, “
Clustering

based Denoising with Locally Learned
Dictionaries
”, Accepted for publication in IEEE Trans. Image Processing
Available at:
http://www.ee.ucsc.edu/~milanfar
20 Jan, 2009
Image Denoising using Locally Learned Dictionaries
36
Iterative Scheme
Why iterate ?
•
Weights true to underlying structure in
presence of lesser noise
•
Better weights means better clustering
•
Dictionary captures underlying data better
when learned on less noisy image
Comments 0
Log in to post a comment