ABSTRACT
Image restoration is a basic problemof image processing.Its objective is to restore
an image,blurred by a smoothing operator or contaminated by additive noises,from
its deterioration.Image restoration has been widely applied to medical imaging and
astronomy.
Regularization is a primary method used in image restoration.The variation
functional of a regularized method contains two terms:a ﬁdelity term and a regu
larization term.Therefore,characteristics of a regularized method are determined
by selections of the ﬁdelity and the regularizer.In past decades,a great amount of
studies of regularized methods have focused on selection of the ﬁdelity as an L
p
norm
(p = 1,2).In this thesis,we concentrate on selection of other possible ﬁdelities with
a total variation regularizer.
We characterize a general class of ﬁdelities,which includes existing ﬁdelities in the
literatures as special examples.We prove the wellposedness of the resulting regular
ized minimization problem,composed of a ﬁdelity from the general class of ﬁdelities
and the total variation regularization.We also show convergence of minimizers of the
regularized minimization problem as errors in data and the blurring operator tend to
zero.
We specify a new subclass of ﬁdelities:contentdriven ﬁdelities.Contentdriven
ﬁdelities provide spatial varying measurements of the recovered image to the observed
image,which can accurately adapt to local image contents.They interpolate strengths
ii
of a variety of existing ﬁdelities under diﬀerent image contents.Four gradientbased
algorithms,including the steepest descent algorithm(SD),heavy ball algorithm(HB),
steepest descent algorithm with two point step size (SDTPSS),and conjugate gradi
ent algorithm(CG),are applied to solve the contentdriven minimization composed of
a contentdriven ﬁdelity and the total variation regularization.We also show conver
gence of the steepest gradient algorithm and the conjugate gradient algorithm based
on the contentdriven estimation.
We use numerical studies to assess properties of the L
p
norm estimations,com
posed of L
p
norm ﬁdelities and the total variation regularizer,for 1 ≤ p ≤ 2 under
diﬀerent image contents.We observe the relationship of the performance of the L
p

norm estimation,the selection of p,the noise levels,and the presence of outliers.Fi
nally,practical eﬀectiveness of the contentdriven approach is shown on synthetically
generated data.The superiorities of the contentdriven estimation over the L
p
norm
estimation (1 ≤ p ≤ 2) and other existing estimations are numerically demonstrated.
All rights reserved
INFORMATION TO ALL USERS
The quality of this reproduction is dependent on the quality of the copy submitted.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.
All rights reserved. This edition of the work is protected against
unauthorized copying under Title 17, United States Code.
ProQuest LLC.
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106  1346
UMI 3459386
Copyright 2011 by ProQuest LLC.
UMI Number: 3459386
Copyright
c
⃝2010,Xiaofei Hu
All Rights Reserved
Contents
Acknowledgments viii
Notations x
1 Introduction 1
1.1 Image Restoration and Regularized Methods..............1
1.2 Choices of Regularization Functional..................3
1.3 Choices of DataFidelity Term......................5
1.4 ContentDriven Fidelity Terms......................6
2 ContentDriven Fidelity Terms 10
2.1 The General Class of Fidelity Terms..................11
2.2 The Class of ContentDriven Fidelity Terms..............16
3 Variational Models for Image Restoration 26
3.1 Preliminary on the Bounded Variation Space..............27
3.2 WellPosedness of the General Variational Model............29
v
CONTENTS vi
3.3 Convergence of Minimizers........................36
3.4 EulerLagrange Equations........................41
4 Discrete ContentDriven Variational Models 45
4.1 Steepest Descent Algorithms for the ContentDriven Variational Model 46
4.1.1 Discrete ContentDriven Variational Models in One Dimension 46
4.1.2 Steepest Descent Algorithms...................48
4.1.3 Convergence of Steepest Descent Methods in One Dimension.49
4.1.4 Discrete ContentDriven Variational Models in Two Dimension 55
4.1.5 Convergence of Steepest Descent Methods for an Anisotropic
Total Variation and ContentDriven Variational Model....57
4.1.6 Convergence of Steepest Gradient Methods for an Isotropic To
tal Variation and ContentDriven Variational Model......61
4.2 Conjugated Gradient Algorithms for the ContentDriven Variational
Model...................................64
4.2.1 Conjugated Gradient Algorithms................64
4.2.2 Convergence Analysis of Conjugated Gradient Methods in One
Dimension.............................66
4.2.3 Convergence of Conjugate Gradient Methods for an Anisotropic
Total Variation and ContentDriven Variational Model....71
4.2.4 Convergence of Conjugate Gradient Methods for an Isotropic
Total Variation and ContentDriven Variational Model....74
CONTENTS vii
4.3 Other Gradient Based Algorithms....................78
4.3.1 Heavy Ball Algorithm......................78
4.3.2 Steepest Descent Algorithm with Twopoint Step Sizes....80
5 Numerical Experiments 81
5.1 Numerical Results of Gradient Based Methods for a ContentDriven
Minimization Problem..........................84
5.2 Numerical Results of Representative ContentDriven Minimization Prob
lems....................................92
5.3 Numerical Comparisons of Minimization Problems Based on ϕ from
the General Class C and from ContentDriven Class S (Gaussian Noise
Only)...................................98
5.4 Numerical Comparisons of Minimization Problems Based on ϕ from
the General Class C and from ContentDriven Class S (Combinations
of Gaussian and Impulsive Noises)....................116
6 Conclusions and Future Work 122
Acknowledgments
I came to United States six years ago and continued my mathematical study in the
Department of Mathematical at Syracuse University.Through these years,I own my
gratitude to all those people who helped me realize my goal.
I would like to express my sincere appreciation to my advisor,Professor Yuesheng
Xu,for everything he has done to help me during my graduate study.Without the
insightful discussions,unfailing supports,and warmhearted cares,I could never ﬁnish
this dissertation.I admire his enthusiasm in mathematics,which has deeply impact
on me.
I am deeply grateful to my coadvisor,Professor Lixin Shen.He brought me into
the great ﬁeld of image processing.His excellent guidance,endless patience,con
siderable encouragement,and understanding have helped me grow up in this ﬁeld.I
appreciate himfor inspiring me to work on image precessing and sharing his invaluable
experience with me.
I thank Professor Andrzej Krol of State University of New York Upstate Medical
University.He is the ﬁrst person who introduces me to the ﬁeld of biomedical image
viii
ACKNOWLEDGMENTS ix
processing.He is very generous to share with me his research experience in biomedical
image science,which is crucial to my future career.It is enjoyable to work in his
laboratory.
My appreciation also goes to Professors Uday Banerjee,Dan Coman,Peng Gao,
Eugene Poletsky,and Andrew Vogel.I thank them for serving on my defense com
mittee and spending their time on it.I also thank my friends,Ms.Yan Chan,Dr.
Weixing Cai,Mr.Feishe Chen,Dr.Ying Jiang,Ms.Weilin Li,Dr.Dong Mao,Dr.
Wei Ning,Dr.Hongwei Ye,Dr.Guohui Song,Dr.Levon Vogelsang,Dr.Junfeng
Wu,Dr.Bo Yu,Dr.Haizhang Zhang,and Mr.Liang Zhao.
Thanks go to the staﬀ of Department of Mathematics,Ms.Madaline Argiro,Ms.
Christine Gilmore,Ms.Beckie Moon,Ms.Patricia O’Malley,Ms.Carolyn Sabloski,
and Mr.Bill Vogel,for their continuously and kindly help during my stay in Syracuse.
Special thanks to my parents and my elder brother for their endless support and
love.Finally,I own my gratitude to my husband,Yabing Mai,for everything he did
for me.Because of him,my life is full of merry.
x
NOTATIONS xi
Notations
R:the ﬁeld of real numbers
N:the set of all positive integers
R
+
:the set of all nonnegative numbers
R
d
:the space of all dtuples of real numbers
R
N×N
:the set of all N by Nmatrices
Ω:a bouded convex doman in R
d
with a Lipschitz continous boundary ∂Ω
¯
Ω:Ω∪∂Ω
Ω:the Legesque measure of Ω
∥ · ∥
p
:=
(∫
Ω
 · 
p
dx
)
1/p
for 1 ≤ p < ∞
∥ · ∥
∞
:= inf{C ≥ 0   ·  ≤ C almost everywhre in Ω}
L
p
(Ω):the set of all functions f satisfying ∥f∥
p
< ∞for some p ∈ [1,∞]
C
1
0
(Ω):the set of all continuously diﬀerentiable functions with compact supports in Ω
C
m
(
¯
Ω):the set of all u ∈ C
m
(Ω) satisfying that ∇
α
1
,...,α
d
u is uniformly continuous on
bounded subsets of Ω for all α
1
 +...+α
d
 ≤ m
BV (Ω):the subspace of functions with bounded L
1
norm and total variation
Chapter 1
Introduction
1.1 Image Restoration and Regularized Methods
Images are usually corrupted during acquisition,transmission,and recording
processes.Classically,deterioration of images is the result of two phenomena,blurring
and noise degradations.The ﬁrst one is deterministic and is due to possible defects of
the imaging system,for instance,a blur caused by an incorrect lens adjustment or by
motion.Suppose that we describe an object image by a function u and an observed
blurry image by a function z.A blurring degradation model can be described as
Au = z,(1.1)
where A is a known linear observation operator.The second phenomenon is random
noise,which arises during transmission.The random noise is usually modeled as a
probabilistic distribution,and it contaminates images through adding or multiplying
1
CHAPTER 1.INTRODUCTION 2
it onto original images.The multiplicative noise occurs in certain imaging systems,
such as laser imaging,microscope imaging,and synthetic aperture radar imaging.
The approaches of multiplicative denoising are proposed in [5,29,37,41].In this
thesis,we mainly consider additive noise.Thus,the blurring and noisy model is
formulated by
Au +ϵ = z,(1.2)
where ϵ is random noise.
Image restoration refers to the problem of recovering an image u from its linearly
degraded version (1.2).It is one of the earliest inverse problems in imaging,and has
wide applications ranging fromastronomy to medical imaging.The main challenge in
image restoration is to faithfully recover detailed image structures (i.e,edges in the
case of natural images) from a noisy observation with a minimal risk of amplifying
the noise.Image restoration has been intensively studied by researchers from areas of
mathematics,computer science,and electrical engineering in the past three decades.
Many image restoration methods,such as linear ﬁltering based,e.g.,least square
ﬁlter [3],iteration based,e.g.,Landweber iteration [42],and variation or minimization
based,e.g.,the RudinOsherFatemi algorithm [38,39],were proposed.Among these
methods,regularization is a commonly used technique due to the nature of the ill
posedness of inverse problem (1.2).Regularization aims at incorporating knowledge
of either the original image or the noise into solution algorithms.Some background
on regularization can be found in an excellent review article [9] or the book [18] about
CHAPTER 1.INTRODUCTION 3
iterative image deblurring.
We study the problem of image restoration within the following variational frame
work:
inf
u
{E(u,z) +λR(u)},(1.3)
where E provides a datafaithful estimate,Rdenotes a regularization functional mea
suring the roughness of u,and λ is a regularization parameter balancing the impor
tance between two terms E and R.Speciﬁcation of a particular method in the general
variational formulation (1.3) entails making choices of E,R,and λ.Selections of E,
R,and λ are problemdependent and are usually tangled together.In the following,
we will brieﬂy and separately review various choices of E and R appeared in the
existing literatures.
1.2 Choices of Regularization Functional
Regularization functional R primarily determines the performance of variational
image restoration model (1.3).Many regularization functionals can be deﬁned in
terms of a potential function ψ in the following way
R(u):=
∫
Ω
ψ(∇u) dx,
where Ω ⊂ R
d
is the image domain.A simple choice of the potential function is
ψ(x) = x
2
,which leads to the wellknown Tikhonov regularization [44].With this
regularization functional,unpleasant blurring eﬀects appear in the vicinity of edges
CHAPTER 1.INTRODUCTION 4
of the recovered image from (1.3).A popular choice of the potential function is
ψ(x) = x,which yields the socalled total variation (TV) regularization [38,39].
Edge locations of the recovered images obtained from the TV regularization tend to
be preserved.Total variation has received a great amount of attention in the image
processing community.The theoretical analysis,numerical implementations,and a
variety of applications of TVbased image restoration have been extensively studied
(see,for example,[10,11,13,16,48]).Other examples of the potential function,
which can yield to edgepreserving regularization,are the Lorentzian potential ψ(x) =
ηx
2
/(1 + ηx
2
),the Gaussian potential ψ(x) = 1 − exp(−ηx
2
),the Huber potential
ψ(x) = x
2
if x ≤ η,ψ(x) = η(η + 2x − η) if x > η [30],and the Charbonnier
potential ψ(x) = log(1 +x
2
) [14].
Recently,a sparsity constraint of the underlying image serving as a regularizer was
proved to be eﬃcient for image restoration [16].It is based on the assumption that
the ideal image has a sparse representation with respect to a given system {w
γ
}
γ∈Γ
,
where Γ is an index set.The system could be wavelets,framelets,and curvelets.
Under this circumstance,regularization functional R is formulated as follows:
R(u):=
∑
γ∈Γ
λ
γ
⟨u,w
γ
⟩
p
,
where ⟨u,w
γ
⟩ are coeﬃcients of u with respect to the system {w
γ
}
γ∈Γ
,λ
γ
are non
negative parameters,and p ∈ [1,∞).In particular,for p = 1,R in (1.2) becomes the
ℓ
1
regularization functional,which is a current hot topic in compressive sensing and
image processing.
CHAPTER 1.INTRODUCTION 5
1.3 Choices of DataFidelity Term
For model (1.2),the frequently used dataﬁdelity term is
E(u,z):=
1
2
∥Au −z∥
2
2
,
which is usually called the L
2
norm ﬁdelity term.This term is appropriate when ϵ
is normally distributed.However,when the observed data z contains outliers,such
as saltpepper noise,the variational image restoration model (1.3) with the L
2
norm
ﬁdelity term is very sensitive to such outliers (see,for example,[30,32,33]).
Another popular choice is the L
1
norm ﬁdelity term,which is deﬁned by
E(u,z):= ∥Au −z∥
1
.
This dataﬁdelity term has shown its robustness on suppressing outliers [12,31,35].
However,the lack of diﬀerentiability for the L
1
dataﬁdelity term at the origin cer
tainly increases diﬃculty in solving the corresponding minimization problem (1.3).
A direct generalization of the above L
1
norm or L
2
norm ﬁdelity term is
E(u,z):= ∥ϕ(Au −z)∥
1
,(1.4)
where ϕ is a continuous even function on R with ϕ(0) = 0.In particular,ϕ(·) =
1
p
 · 
p
with p = 1 or p = 2 reduces to the L
1
norm or L
2
norm ﬁdelity term,respectively.
Obviously,it is natural to use ϕ(·) =
1
p
 · 
p
in (1.4) with values of p other than 1 or
2.Actually,the dataﬁdelity term deﬁned by (1.4) with ϕ(·) =
1
p
 · 
p
for 1 ≤ p < ∞
and called the L
p
norm ﬁdelity has been extensively studied in other contexts such
CHAPTER 1.INTRODUCTION 6
as regression analysis and statistical classiﬁcation.It was suggested in [21] to use the
L
p
norm ﬁdelity with 1 ≤ p ≤ 2 for contaminated data.Selecting values of p of the
L
p
norm ﬁdelities depends critically on the distribution of residuals Au −z.Several
empirical schemes for determining p were proposed in [23,24,26].With such schemes,
numerical experiments as reported in [17,33] showed that the L
1
and L
p
(1 < p < 2)
ﬁdelities performed equivalently for larger outliers in z;the L
p
(1 < p < 2) ﬁdelity
worked better than the L
1
ﬁdelity for moderate outliers;some L
p
(1 < p < 2) ﬁdelity
and the L
2
ﬁdelity were essentially the same for small outliers,and both were better
than the L
1
ﬁdelity.
Comparing to the L
1
norm or L
2
norm ﬁdelity,the L
p
norm (1 < p < 2) ﬁdelity
has shown less impact on the community of image processing.It might be due to the
facts that no single p value can faithfully describe the distribution of residuals Au−z
and varying p values are required in the diﬀerent regions of the underlying image.
Recently,an eﬀort was made to design a function ϕ in (1.4),which can mimic the
behaviors of L
1
and L
2
norms and the transition from L
1
norm to L
2
norm and vise
verse [34].It provides a diﬀerent perspective to formulate datafaithful estimation.
1.4 ContentDriven Fidelity Terms
In this thesis,we systemically study the selection of dataﬁdelity terms.The
ﬁdelity term E(u,z) in (1.4) is uniquely determined by the function ϕ.We deﬁne
CHAPTER 1.INTRODUCTION 7
a general class of functions ϕ,which characterizes a general class of ﬁdelities.This
class of ﬁdelities includes most ﬁdelities in the existing literatures,such as L
p
norm
ﬁdelities (1 < p < 2) and the Huber ﬁdelity.
Motivated by [34],we ﬁnd a new subclass of functions ϕ in the above mentioned
general function class,called the contentdriven function class.The L
p
functions
(1 ≤ p ≤ 2) and the Huber function are not in the contentdriven class.The derivative
of a contentdriven function ϕ behaves similarly to the L
1
function as the input of ϕ
is large enough,and similarly to L
2
function as the input of ϕ is small.The derivative
of a contentdriven function ϕ shows L
p
function behavior (1 < p < 2) as the input of
ϕ is moderate.The behavior of a contentdriven function ϕ smoothly transits from
that of the L
2
function to that of the L
1
function.
The ﬁdelity introduced by the contentdriven function ϕ is called the content
driven ﬁdelity.It inherits most strengths of L
p
norm ﬁdelities (1 ≤ p ≤ 2) for image
restoration.The contentdriven ﬁdelity shows the behavior of the L
2
norm ﬁdelity as
the local image content is normally distributed or as the noise level of ϵ is low,and
the behavior of the L
1
norm ﬁdelity as the local image content contains outliers or
as the noise level is high.The contentdriven ﬁdelity can provide a spatially varying
measurement of the reconstructed image to the original image.
Theoretical studies based on the general class of ﬁdelities are presented in this
work.We show the wellposedness of regularized methods,composed of a ﬁdelity
termE chosen fromthe general class of ﬁdelities and the total variation regularization.
CHAPTER 1.INTRODUCTION 8
Convergence of minimizers of regularized estimations based on the general class of
ﬁdelities are established as errors in the data and the blurring kernel diminish to
zero.Since contentdriven ﬁdelities are contained in the general class of ﬁdelity,
theoretical results held for the general ﬁdelity estimations are also held for content
driven estimations.
Four gradient based algorithms,such as the steepest descent algorithm (SD),
heavy ball algorithm (HB),steepest descent algorithm with two point step size (SD
TPSS),and conjugate gradient algorithm (CG),are applied to solve contentdriven
estimations.We show convergence of SD and CG algorithms for solving discrete
contentdriven estimations in the one dimensional case and the two dimensional case.
Numerical comparisons to assess performances of L
p
norm ﬁdelities,the Huber
ﬁdelity,and contentdriven ﬁdelities are performed.The superiorities of the content
driven estimation over L
p
norm estimations and other existing estimations are shown
in our numerical studies.
This thesis consists of seven chapters:In Chapter 2,we deﬁne the general class of
ﬁdelities,and study their properties as preliminary materials for theoretical analysis
in Chapter 3.We specify contentdriven ﬁdelities,which is motivated by seeking an
appropriate measurement of the reconstructed images to the original images other
than the existing ﬁdelity measurements.Special examples of content ﬁdelities are
presented.In Chapter 3,we prove the wellposedness of total variation regularized
estimations with ﬁdelities fromthe general class of ﬁdelities.As errors in the observed
CHAPTER 1.INTRODUCTION 9
image and the blurring kernel tend to zero,convergence of minimizers of corresponding
estimations are proved.We present the EulerLagrange equations of estimations
based on the general class of ﬁdelities.We also provide a suﬃcient and necessary
condition that ensures the existence of the minimizer of the regularized minimization
problem with a ﬁdelity from the general ﬁdelity class.In Chapter 4,we apply four
gradient based methods to solve contentdriven estimations.Convergence of SD and
CG methods based on the discrete contentdriven estimations are presented in both
cases of one dimension and two dimension.In Chapter 5,numerical studies that
compare performances of L
p
norm estimations (1 ≤ p ≤ 2),the Huber estimation,
and contentdriven estimations are presented.The last chapter is about conclusions
and future research plans.
Chapter 2
ContentDriven Fidelity Terms
The ﬁdelity plays a key role in a regularized method.It provides a measurement of
the closeness between the reconstruction image and the original image.The essence
of choosing a ﬁdelity is to ﬁnd the most accurate measurement among all possible
selections.In this thesis,we concentrate on the issue of the ﬁdelity selection by
studying existing ﬁdelities and deﬁning more accurate ﬁdelity measurements,which
can adapt to local image contents.
In the previous chapter,we concluded a general form of dataﬁdelity terms as
(1.4).The ﬁdelity provided by (1.4) is uniquely determined by the deﬁnition of ϕ.
We can not say that (1.4) includes all possible selections of ﬁdelities,but it includes
most of wellknown ﬁdelities by varying the choice of ϕ.In this section,we introduce
a class of functions ϕ which can measure the closeness between the estimation Au and
the observed data z in (1.4).We ﬁrst give a general class of functions which includes
10
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS 11
functions
1
p
 · 
p
(it will be referred to as L
p
functions) with 1 ≤ p ≤ 2 as well as the
Huber function [30].We then narrow this class down to a special class in which each
function will be able to serve our purpose in a way that the corresponding estimator
(1.4) can locally adapt to noise and outliers in the observed data.
2.1 The General Class of Fidelity Terms
The class of functions we will introduce below is denoted by
C:= {ϕ  ϕ satisﬁes Condition H}.(2.1)
We say that a function ϕ satisﬁes Condition H if ϕ meets the following hypotheses
•
H1:ϕ:R →R
+
is a convex,even,nonconstant function with ϕ(0) = 0.
•
H2:There exists a positive number r ∈ [0,1] such that for any x and y in R,
one has
ϕ(x) −ϕ(y) ≤ c
ϕ
ξ
r
x −y,(2.2)
where c
ϕ
is a constant depending only on ϕ and ξ is a number between x and y.
Here R
+
denotes all nonnegative real numbers.The next two results show that L
p
functions and the Huber function are in the set C.
Proposition 2.1.
For 1 ≤ p ≤ 2,the L
p
function ϕ(x) =
1
p
x
p
is in C.
Proof.
It is easy to check that L
p
function satisﬁes H1.Now we check if the L
p
function satisﬁes (2.2).If p = 1,then (2.2) holds with c
ϕ
= 1 and r = 0.If 1 < p ≤ 2,
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS 12
by the mean value theorem,we have that
ϕ(x) −ϕ(y) ≤ pξ
p−1
x −y,
where ξ is a number between x and y.Therefore,the L
p
function is in the class C.
Recall that the Huber function [30] is deﬁned by
H(x):=
1
2
x
2
,0 ≤ x ≤ η,
ηx −
η
2
2
,x ≥ η,
(2.3)
where η is a positive constant.The Huber function is a combination of L
1
and L
2
functions.Here η is a parameter controlling the location of the connection point
between L
1
and L
2
functions.Similar to the L
p
function,we also have
Proposition 2.2.
The Huber function deﬁned by (2.3) is in C.
Proof.
It is wellknown that the Huber function satisﬁes hypothesis H1.A direct
computation shows H
′
(x) ≤ η for all x ∈ R.Hence,the Huber function satisﬁes
hypothesis H2 with r = 0 and c
ϕ
= η.
From the above examples,we know that the general function class C includes
two wellknown ﬁdelities:the L
p
ﬁdelity and the Huber ﬁdelity.We further explore
the properties of the function class C.Properties of functions in C are presented in
the following four lemmas.Prior to stating the lemmas,we recall that ϕ is called a
(monotonically) increasing function if ϕ(a) ≤ ϕ(b) (ϕ(a) < ϕ(b)) for any a < b.
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS 13
Lemma 2.3.
If ϕ is a function satisfying the hypothesis H1,then
lim
x→±∞
ϕ(x) = ∞.(2.4)
Proof.
We ﬁrst show that there exists a positive c > 0,such that ϕ monotonically
increases on [c,∞) and monotonically decreases on (−∞,c].Since ϕ is an even
function,we only need to show that ϕ monotonically increases on [c,∞).We assume
that it is not true,i.e.,for any c > 0 there exists two positive number a and b satisfying
c ≤ a < b < ∞,such that ϕ(a) ≥ ϕ(b).Assuming that ϕ(a) > ϕ(b),we have ϕ(a) > 0
due to the positivity of ϕ on R.By ϕ(0) = 0,ϕ(a) > ϕ(b) ≥ 0 contradicts to the
convexity of ϕ.Therefore,ϕ(a) = ϕ(b).By convexity of ϕ with ϕ(0) = 0,we have
that ϕ(x) = 0 on the interval [0,b].Letting c approach +∞,we have that ϕ(x) = 0
on R.It contradicts that ϕ(x) is a nonconstant function.
Now,we verify (2.4).Since ϕ is an even function,we only showthat lim
x→∞
ϕ(x) =
∞.If this is not true,then there exist a positive number M and an nonnegative
increasing sequence {x
n
}
∞
n=1
with lim
x→∞
x
n
= ∞ such that the sequence {ϕ(x
n
)}
is bounded by M.The monotonicity of ϕ implies that the limit of ϕ(x
n
) exists and
equals to a number,say K,when n runs to inﬁnity.That is lim
x→∞
ϕ(x
n
) = K.By
using the monotonicity of ϕ again,it yields that
lim
x→∞
ϕ(x) = K.(2.5)
Let a and b be two numbers with properties c ≤ a < b and ϕ(a) < ϕ(b).For any
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS 14
x > b,the convexity of ϕ leads to
ϕ(b) ≤ ϕ(a) +
ϕ(x) −ϕ(a)
x −a
(b −a)
Letting x approach to inﬁnity in the above inequality and using (2.5),we have ϕ(b) ≤
ϕ(a).This contradicts to ϕ(a) < ϕ(b).Hence,(2.4) is true.
Lemma 2.4.
If ϕ ∈ C,then,for all x ∈ R,
ϕ(x) ≤ c
ϕ
max{x,1}x,(2.6)
where c
ϕ
is the constant in (2.2).
Proof.
Note that ϕ(0) = 0.By (2.2),one has
ϕ(x) = ϕ(x) −ϕ(0) ≤ c
ϕ
ξ
r
x,
where 0 ≤ r ≤ 1 and 0 ≤ ξ ≤ x.Since ξ
r
≤ max{x,1},we have (2.6).
Let Ω ⊂ R
d
be a bounded convex domain with a Lipschitz continuous boundary
∂Ω.Let denote Ω the Lebesque measure of Ω.A function f is in L
p
(Ω) (1 ≤ p ≤ ∞)
if ∥f∥
p
is ﬁnite.Here,∥f∥
p
:=
(∫
Ω
f(x)
p
dx
)
1/p
< ∞ for 1 ≤ p < ∞ and ∥f∥
∞
:=
inf{C ≥ 0  f ≤ C almost everywhre in Ω} < ∞.
Lemma 2.5.
If ϕ ∈ C,then for any function f in L
2
(Ω),ϕ ◦ f ∈ L
1
(Ω).
Proof.
By Lemma 2.4,we have
∥ϕ ◦ f∥
1
=
∫
Ω
ϕ(f(x)) dx ≤ c
ϕ
∫
Ω
max{f(x),1}f(x) dx.
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS 15
By the H¨older inequality,
∥ϕ ◦ f∥
1
≤
(∫
Ω
max{f(x),1}
2
dx
)
1/2
(∫
Ω
f(x)
2
dx
)
1/2
≤ c
ϕ
(∥f∥
2
+Ω
1/2
∥f∥
2
) < ∞.
Lemma 2.6.
If ϕ ∈ C,then for any two functions f and g in L
2
(Ω),
∥ϕ ◦ f −ϕ ◦ g∥
1
≤ c
ϕ
(∥f∥
2
+∥g∥
2
+Ω
1/2
)∥f −g∥
2
.(2.7)
Proof.
By hypothesis H2,we have that
∥ϕ ◦ f −ϕ ◦ g∥
1
=
∫
Ω
ϕ(f(x)) −ϕ(g(x)) dx ≤ c
ϕ
∫
Ω
ξ(f(x),g(x))
r
f(x) −g(x) dx
(2.8)
where ξ(f(x),g(x)) is a number between f(x) and g(x).
Let Ω
0
:= {x ∈ Ω  f(x) ≤ 1 and g(x) ≤ 1} and Ω
1
:= Ω\Ω
0
.Then,we have
that ξ(f(x),g(x))
r
≤ 1 for x ∈ Ω
0
and ξ(f(x),g(x))
r
≤ f(x) +g(x) for x ∈ Ω
1
.
From (2.8) and the CauchySchwartz inequality,we obtain that
∥ϕ ◦ f −ϕ ◦ g∥
1
≤ c
ϕ
(
∫
Ω
0
f(x) −g(x) dx +
∫
Ω
1
(f(x) +g(x))f(x) −g(x) dx
)
≤ c
ϕ
(
Ω
1/2
∥f −g∥
2
+∥f +g∥
2
∥f −g∥
2
)
.
The above lemma implies that the closeness of two functions f and g in the L
2
sense leads to the closeness of ϕ ◦ f and ϕ ◦ g in the L
1
sense.
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS 16
2.2 The Class of ContentDriven Fidelity Terms
The solution of (1.2) can be approached by solving the variational problem (1.3),
whose ﬁdelity E(u,z) has the form of (1.4) with ϕ ∈ C.As discussed in the previous
chapter,L
p
functions (1 ≤ p ≤ 2) are commonly used in deﬁning the ﬁdelity term,
especially with p = 1,2.The regularization problems (1.3) with L
1
norm and L
2

norm ﬁdelities are well studied.We review their strengths and limitations presented
in the literature as follows.
(a) Noisy and Blurred ‘Lena’ (b) L
2
+ TV
Figure 2.1:Restored image from a blurred and noisy`Lena'by using L
2
norm delity and
TV regularization
The L
2
norm can well keep the ﬁdelity of an image,see [2,6,10,45,46].It has
been shown to have the superiority over the L
1
norm if the noisy data is normally
distributed and the noise level of data is low,see [32,33].However,the superiority of
the L
2
norm will lose if noise level of the normally distributed data is high.Another
limitation of the L
2
norm ﬁdelity is that it is sensitive to outliers.If large values of ϵ
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS 17
appear in the image domain,such as salt and pepper noise,the reconstructed image
may be far away from the original image.Figure 2.1 demonstrates the restored result
of a blurry image with salt pepper noise.The result is produced by the estimation
(1.3),composed of the L
2
normﬁdelity and the total variation regularization.We can
see the obvious artifacts around the presence of large errors.Therefore,if z contains
outliers,the quality of the reconstructed image based on the L
2
estimator is poor.
The L
1
normﬁdelity normis addressed in [12,31,35],It has signiﬁcant superiority
over the L
2
norm ﬁdelity in presence of outliers (see,for example,Figure 2.2).The
L
1
norm estimation more depends on the geometric shape of image features and less
depends on the intensity contrast of images.A related discussion was given in [12].
The L
1
normﬁdelity is not optimal to process blurred images with the Gaussian noise
at small noise levels.Moreover,the L
1
norm is not diﬀerentiable at origin.It may
introduce some computation diﬃculties.
The L
p
norm ﬁdelity for some p ∈ (1,2) could be a better ﬁdelity candidate
other than the L
1
norm or L
2
norm ﬁdelity for some image contents.Some L
p
norm
ﬁdelities (1 < p < 2) show the superiorities in the presence of moderate outliers
or small errors (see numerical results in Chapter 5).However,we have to face a
challenging issue:the selection of the optimal p.This issue was addressed in [24,
23,26] from statistical perspective.None of the existing selection schemes is trivial.
Through our numerical observations we ﬁnd out that for each p between 1 and 2 the
behavior of the L
p
norm ﬁdelity estimation is either similar to the L
1
norm ﬁdelity
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS 18
(a) Noisy and Blurred ‘Lena’ (b) L
1
+ TV
Figure 2.2:Recovered results from a blurred and noisy`Lena'by using L
1
delity and TV
regularization
or the L
2
norm ﬁdelity.There is no single p value which keeps both strengths of the
L
1
norm ﬁdelity and L
2
norm ﬁdelity.
From the above discussion,we notice that L
p
norm estimators for 1 ≤ p ≤ 2 have
their unique strengths for processing diﬀerent noisecontained images.We intend to
ﬁnd a small class of ﬁdelities,which vary their behaviors to adapt to the image content
and inherit all strengths of L
p
ﬁdelities,especially for p = 1,2.We plot the derivatives
of L
1
and L
2
functions in Figure 2.3.We assume that values on the horizontal axis
denote the pointwise errors of Au and z.The absolute value of the derivative of the
L
1
function is a constant on R/{0}.It means that the L
1
function gives an equal
weighted measurement of errors in observed images.Therefore,the L
1
norm ﬁdelity
can suppress outliers.For the L
2
function,its derivative grows linearly with respect
to x.It indicates that the L
2
normﬁdelity gives a linear measurement of errors,which
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS
19
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
L
1
norm
L
2
norm
Content−Driven
φ
’
Figure 2.3:Plots of the derivatives of the desired
ϕ
(red solid line),
L
1
function
(dotted line) and
L
2
function (dashed line)
is proportional to the errors.So the
L
2
norm ﬁdelity is very sensitive to outliers in
z
.The derivative of the desired function
ϕ
should work similarly to the derivative of
the
L
1
function as the value of
x
is large,and behaves similarly to the derivative of
the
L
2
function as
x
is small.Therefore,the graph of the derivative of the desired
ϕ
should have similar shape as the red curve in Figure 2.3.
In summary,we will deﬁne a special subset of
C
by requiring two additional
hypotheses:
•
H3
:
ϕ
is twice continuously diﬀerentiable with bounded ﬁrst and second deriva
tives;
•
H4
:
ϕ
is strictly convex.
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS 20
This special class of functions is deﬁned as follows
S:= {ϕ  ϕ satisﬁes the hypotheses H1,H3,and H4}.(2.9)
By the mean value theorem,the hypothesis H3 implies the hypothesis H2.Therefore,
S is a subset of C.The ﬁdelity term deﬁned as (1.4) with ϕ ∈ S is called the content
driven ﬁdelity.
Lemma 2.7.
If ϕ ∈ S,then ϕ
′
(0) = 0.
Proof.
Since ϕ is a nonnegative strictly convex function with ϕ(0) = 0,we know
that ϕ increases on [0,∞) and decreases on (−∞,0].Hence,
lim
x→0+
ϕ(x) −ϕ(0)
x
≥ 0 and lim
x→0−
ϕ(x) −ϕ(0)
x
≤ 0.
The diﬀerentiability of ϕ implies that ϕ
′
(0) = 0.
We can directly check that the L
p
functions,1 ≤ p ≤ 2,and the Huber function
are not in S.Several examples of functions in S are presented in the following lemmas.
Lemma 2.8.
Let η be a positive number.Deﬁne
ϕ(x):=
x
2
2(1 +x/η)
x ∈ R.(2.10)
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS 21
Then,ϕ ∈ S has the properties
lim
x→0
ϕ(x)
1
2
x
2
= 1,
lim
x→∞
ϕ(x)
x
=
1
2
η,
ϕ
′
(x) ≤
1
2
η,x ∈ R,
ϕ
′′
(x) ≤ 1,x ∈ R,
ϕ
′′
(x) > 0,x ∈ R.
Proof.
It is easy to check that ϕ is a nonnegative even function having
ϕ
′
(x) =
2η
2
x +sign(x)ηx
2
2(η +x)
2
,x ∈ R.
We observe that ϕ(0) = ϕ
′
(0) = 0 and ϕ
′
(x) < η/2.The second derivative of ϕ has
the form of
ϕ
′′
(x) =
η
3
(η +x)
3
,x ∈ R.
It is continuous and bounded by 1.Since the second derivatives of ϕ is positive,
ϕ is strictly convex.Hence,ϕ ∈ S and the last three properties hold.By simple
calculations,we can easily obtain the ﬁrst and second properties.
Lemma 2.9.
Let η be a positive number.Deﬁne
ϕ(x):=
1
2
η
2
log cosh
(
x
η
)
,x ∈ R.(2.11)
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS 22
Then,ϕ ∈ S has the following properties
lim
x→0
ϕ(x)
1
2
x
2
= 1,
lim
x→∞
ϕ(x)
x
=
1
2
η,
ϕ
′
(x) ≤
1
2
η,x ∈ R,
ϕ
′′
(x) ≤
1
2
,x ∈ R,
ϕ
′′
(x) > 0,x ∈ R.
Proof.
It is easy to check that ϕ is a nonnegative even function.The ﬁrst derivative
of ϕ has the form
ϕ
′
(x) =
1
2
η tanh(
x
η
),x ∈ R.
We also notice that ϕ(0) = ϕ
′
(0) = 0.The second derivative of ϕ is given by
ϕ
′′
(x) =
1
2 cosh
2
(
x
η
)
,x ∈ R,
which is continuous and bounded by
1
2
.The ﬁrst derivative of ϕ is bounded by η/2.
Since the second derivatives of ϕ is positive,ϕ is strictly convex.Hence,ϕ ∈ S and
the last three properties hold.By simple calculations,we can easily obtain the ﬁrst
and second properties.
Lemma 2.10.
Let η be a positive number.Deﬁne
ϕ(x):= η
√
η
2
+x
2
−η
2
,x ∈ R.(2.12)
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS 23
Then,ϕ ∈ S has the following properties
lim
x→0
ϕ(x)
1
2
x
2
= 1,
lim
x→∞
ϕ(x)
x
= η,
ϕ
′
(x) ≤ η,x ∈ R,
ϕ
′′
(x) ≤ 1,x ∈ R,
ϕ
′′
(x) > 0,x ∈ R.
Proof.
It is easy to check that ϕ is a nonnegative even function.The ﬁrst derivative
ϕ
′
(x) =
ηx
√
η
2
+x
2
,x ∈ R,
with ϕ(0) = ϕ
′
(0) = 0.We also notice that ϕ
′
(x) < η.The second derivative of ϕ
has the form
ϕ
′′
(x) =
η
3
(η
2
+x
2
)
3/2
,x ∈ R,
which is bounced by 1 and continuous.Since the second derivatives of ϕ is positive,
ϕ is strictly convex.Hence,ϕ ∈ S and the last three properties hold.By simple
calculations,we can easily obtain the ﬁrst and second properties.
Lemma 2.11.
Let η be a positive number.Deﬁne
ϕ(x):= ηxarctan
(
x
η
)
−
1
2
η
2
log
(
1 +
(
x
η
)
2
)
,x ∈ R.(2.13)
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS 24
Then,ϕ ∈ S has the following properties
lim
x→0
ϕ(x)
1
2
x
2
= 1,
lim
x→∞
ϕ(x)
x
=
π
2
η,
ϕ
′
(x) ≤
π
2
η,x ∈ R,
ϕ
′′
(x) ≤ 1,x ∈ R,
ϕ
′′
(x) > 0,x ∈ R.
Proof.
It is easy to check that ϕ is a nonnegative even function.The ﬁrst derivative
of ϕ has the form of
ϕ
′
(x) = η arctan(
x
η
),x ∈ R,
with ϕ(0) = ϕ
′
(0) = 0.We also notice that ϕ
′
(x) <
π
2
η.The second derivative of ϕ
ϕ
′′
(x) =
1
π(1 +(
x
η
)
2
)
,x ∈ R,
is bounded by 1 and continuous.Since the second derivatives of ϕ is positive,ϕ
is strictly convex.Hence,ϕ ∈ S and the last three properties hold.By simple
calculations,we can easily obtain the ﬁrst and second properties.
Let η = 1 and we plot the derivatives of the four examples of ϕ deﬁned as in (2.10)
(2.13) in Figure 2.4.The derivative of each ϕ behaves similarly to the L
1
function as
x is large enough,and similarly to the L
2
function as x is suﬃciently small.Figure
2.4 illustrates that the contentdriven ﬁdelity carries both strengths of the L
1
norm
ﬁdelity and the L
2
norm ﬁdelity.
CHAPTER 2.CONTENTDRIVEN FIDELITY TERMS
25
−1.5
−1
−0.5
0
0.5
1
1.5
2
Example 1
Example 2
Example 3
Example 4
L1 Norm
L2 Norm
Figure 2.4:Plots of the derivatives of speciﬁc examples of
ϕ
with
η
= 1 (Example 1:
ϕ
is deﬁned as in (2.10);Example 2:
ϕ
is deﬁned as in (2.11);Example 3:
ϕ
is deﬁned
as in (2.12);Example 4:
ϕ
is deﬁned as in (2.13).)
Chapter 3
Variational Models for Image
Restoration
We recall that the degradation model is given by
z = Au +ϵ,(3.1)
where z is the observed blurry and noisy image,u is the true image which we intend
to recover,and ϵ is the additive random noise.Let Ω ⊂ R
d
(d = 1,2) be a bounded
convex domain with a Lipschitz continuous boundary ∂Ω.Here A denotes a known
linear compact operator from L
p
(Ω) into L
2
(Ω).Equation (3.1) is a typical illposed
problem.
To remedy the illposedness of (1.2),we propose to seek a solution in the bounded
variation space by minimizing a functional (with respect to u) as follows
F
λ
(u,z):= ∥ϕ(Au −z)∥
1
+λ · TV(u).(3.2)
26
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 27
Here TV(u) stands for the total variation of u and its deﬁnition will be given later.
3.1 Preliminary on the Bounded Variation Space
In this subsection,we review some elementary concepts and results related to the
bounded variation space BV (Ω) that are useful for the thesis.The bounded variation
space BV (Ω) is a good space to study images.
Let C
1
0
(Ω) be the collection of continuously diﬀerentiable functions with compact
supports in Ω.We deﬁne
C
1
0
(Ω)
d
:= C
1
0
(Ω) ×· · · ×C
1
0
(Ω)

{z
}
d folds
.
For a vectorvalued function v ∈ C
1
0
(Ω)
d
,we write v:=
√
v
2
1
+· · · +v
2
d
,where v
i
is
the ith component of v for i = 1,...,d.We let
V:= {v ∈ C
1
0
(Ω)
d
:v(x) ≤ 1 for all x ∈ Ω}.
Note that BV (Ω) is the subspace of functions u ∈ L
1
(Ω) such that the following
quantity is ﬁnite:
TV(u):= sup
v∈V
∫
Ω
−udiv v dx.(3.3)
The space BV (Ω) endowed with the norm ∥u∥
BV
:= ∥u∥
1
+TV(u) is a Banach space.
Note that Ω is a bounded open set.It was shown in [1] that
BV (Ω) ⊂ L
p
(Ω) ⊂ L
1
(Ω) (3.4)
for 1 ≤ p ≤ d/(d −1).
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 28
In order to overcome the discontinuity total variation at zero,a modiﬁcation of
the total variation (see,for example,[1,47]) is given by
TV
β
(u):= sup
v∈V
∫
Ω
(
−udiv v +
√
β(1 −v
2
)
)
dx (3.5)
where β is a positive number.A sequence {u
n
}
n
⊂ L
p
(Ω) converges weakly to u ∈
L
p
(Ω) for 1 ≤ p < ∞,if
lim
n→∞
∫
Ω
u
n
vdx =
∫
Ω
uvdx,∀v ∈ L
q
(Ω),
where
1
p
+
1
q
= 1 (q:= ∞ when p = 1).It was proved in [1] that TV
β
is weakly
lowersemicontinuous with respect to the L
p
(Ω) topology for 1 ≤ p < ∞,i.e.,if u
n
converges weakly to bu in L
p
(Ω),then
TV
β
(bu) ≤ liminf
n→∞
TV
β
(u
n
).(3.6)
Clearly,TV
0
(u) = TV(u).Further,we have
TV
β
(u) =
∫
Ω
√
∇u
2
+β
2
dx (3.7)
provided u ∈ C
1
(Ω).When the regularization term R(u) is chosen as TV
β
(u),the
model in (3.2) becomes
F
λ,β
(u,z):= ∥ϕ(Au −z)∥
1
+λ · TV
β
(u),β > 0,(3.8)
where ϕ is a given function in C.By Lemma 2.6 and (3.6),F
λ,β
is weakly semicontin
uous with respect to the L
p
(Ω) topology,that is,if a sequence {u
n
}
n
weakly converges
to bu ∈ L
p
(Ω),then
F
λ,β
(bu,z) ≤ liminf
n→∞
F
λ,β
(u
n
,z).(3.9)
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 29
3.2 WellPosedness of the General Variational Model
In this section,we consider the wellposedness of the minimization problem
inf
u∈BV(Ω)
F
λ,β
(u,z),(3.10)
where the functional F
λ,β
is deﬁned by (3.8).For problem (3.10) to be wellposedness
in the sense of Hadamard,it should have the following properties:(i) a solution of
(3.10) exists;(ii) its solution is unique;and (iii) its solution continuously depends on
the data z and the operator A.The wellposedness of the estimation,composed of
the L
2
norm ﬁdelity and a modiﬁed total variation,is presented in [1].The following
proofs of wellposedness of the general variational model (3.8) are inspired by the
work in [1].The technical diﬃculties in the proofs are aroused by the unspeciﬁc form
of ϕ.We ﬁrst show that the functional F
λ,β
(u,z) is convex with respect to u.
Lemma 3.1.
Let ϕ ∈ C.Let A be a linear operator from L
p
(Ω) to L
2
(Ω) where
1 ≤ p ≤ d/(d − 1).Then the functional F
λ,β
(·,z) is convex.Furthermore,the
functional F
λ,β
(·,z) is strictly convex if A is injective and ϕ is strictly convex.
Proof.
It is known from [1] that TV
β
is convex.By (3.8),we only need to show
that E(·,z) is convex.This directly follows from the convexity of the function ϕ.
Furthermore,the injectivity of A and the strict convexity of ϕ lead to the strict
convexity of E(·,z).Hence,the functional F
λ,β
(·,z) is strictly convex.
Before we give the proof of the existence and uniqueness of the solutions of problem
(3.10),we recall a useful concept of BVcoercivity and a decomposition of a bounded
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 30
variation function.
A functional F:L
p
(Ω) →R is said to be BV coercive if
F(u) →+∞ whenever ∥u∥
BV
→+∞.(3.11)
An arbitrary function u ∈ BV (Ω) can be decomposed as
u = v +w (3.12)
with
∫
Ω
vdx = 0 and w =
1
Ω
∫
Ω
udx.(3.13)
A direct consequence of this decomposition (see,[1]) is that there exists a positive
number M
1
such that
∥v∥
p
≤ M
1
· TV
0
(u) and ∥u∥
BV
≤ ∥w∥
1
+(M
1
+1) · TV
0
(v).(3.14)
for any p such that 1 ≤ p ≤ d/(d −1).Let 1
Ω
be the constant function on Ω whose
function value is 1.The Jensen inequality in [40] plays an important role in the proof
of BV coercivity.It states that
ϕ(
1
Ω
∫
Ω
fdx) ≤
1
Ω
∫
Ω
(ϕ ◦ f)dx.(3.15)
where f is a real function in L
1
(Ω) and ϕ is a convex function.The Jensen inequality
is a generalization of the statement that the secant line of a convex function lies above
the graph of the function.
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 31
Lemma 3.2.
Let ϕ ∈ C.Let A be a linear and bounded operator from L
p
(Ω) to L
2
(Ω)
where 1 ≤ p ≤ d/(d−1).If ∥A1
Ω
∥
1
> 0,then the functional F
λ,β
(·,z) is BVcoercive.
Proof.
Showing that F
λ,β
(·,z) is BVcoercive is equivalent to proving that if for any
sequence {u
n
} ⊂ BV (Ω)
liminf
n→∞
F
λ,β
(u
n
,z) < +∞,
then
liminf
n→∞
∥u
n
∥
BV
< +∞.
Suppose that {u
n
} is a sequence in BV (Ω) and
liminf
n→∞
F
λ,β
(u
n
,z) = K < +∞.
We now show liminf
n→∞
∥u
n
∥
BV
< +∞.Since liminf
n→∞
F
λ,β
(u
n
,z) is ﬁnite,there
is a subsequence {u
n
k
} of {u
n
} such that
lim
k→∞
F
λ,β
(u
n
k
,z) = liminf
n→∞
F
λ,β
(u
n
,z) = K.
Hence the sequence {F
λ,β
(u
n
k
,z)}
k
is bounded.
To showliminf
n→∞
∥u
n
∥
BV
< +∞,it suﬃces to prove that {∥u
n
k
∥
BV
}
k
is bounded.
By (3.12)(3.13),each function in {u
n
k
}
n
k
can be decomposed as follows
u
n
k
= v
n
k
+w
n
k
with w
n
k
=
1
Ω
∫
Ω
u
n
k
dx.
By (3.14),we only need to show that both sequences {∥w
n
k
∥
1
}
k
and {TV
0
(v
n
k
)}
k
are
bounded.Since
λTV
0
(v
n
k
) = λTV
0
(u
n
k
−w
n
k
) = λTV
0
(u
n
k
) ≤ λTV
β
(u
n
k
) ≤ F
λ,β
(u
n
k
,z),
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 32
the boundedness of the sequence F
λ,β
(u
n
k
,z) implies the boundedness of the sequence
{TV
0
(v
n
k
)}
k
.
Now we prove that {∥w
n
k
∥
1
}
k
is bounded.Since ϕ is convex,by the Jensen
inequality (3.15) we have that
ϕ
(
1
Ω
∫
Ω
Au
n
k
−z dx
)
≤
1
Ω
∫
Ω
ϕ(Au
n
k
−z) dx ≤
1
Ω
F
λ,β
(u
n
k
,z).
Hence,the boundedness of the sequence {ϕ(
1
Ω
∫
Ω
Au
n
k
−z dx)}
k
is derived by the
boundedness of the sequence F
λ,β
(u
n
k
,z).By Lemma 2.3,the boundedness of the
sequence {∥Au
n
k
−z∥
1
}
k
is achieved.
By the decomposition of u
n
k
= w
n
k
+v
n
k
,we have that
∥Au
n
k
−z∥
1
= ∥(Av
n
k
−z) +Aw
n
k
∥
1
≥ ∥Aw
n
k
∥
1
−∥Av
n
k
−z∥
1
.(3.16)
By the H¨older inequality,we observe that
∥Av
n
k
−z∥
1
≤ ∥Av
n
k
∥
1
+∥z∥
1
≤ Ω
1/2
∥Av
n
k
∥
2
+∥z∥
1
.(3.17)
By (3.14),we conclude that
∥Av
n
k
−z∥
1
≤ Ω
1/2
∥A∥∥v
n
k
∥
p
+∥z∥
1
≤ Ω
1/2
M
1
∥A∥TV
0
(v
n
k
) +∥z∥
1
.(3.18)
From (3.16) and (3.18),together with the fact ∥Aw
n
k
∥
1
= Ω
−1
∥A1
Ω
∥
1
∥w
n
k
∥
1
,we
obtain that
∥Au
n
k
−z∥
1
≥ Ω
−1
∥A1
Ω
∥
1
∥w
n
k
∥
1
−(Ω
1/2
M
1
∥A∥TV
0
(v
n
k
) +∥z∥
1
).(3.19)
By∥A1
Ω
∥
1
> 0,the boundedness of {∥w
n
k
∥
1
}
k
is achieved by the boundednesses of
{∥Au
n
k
−z∥
1
}
k
and {TV
0
(v
n
k
)}
k
.
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 33
Theorem 3.3 (Existence and Uniqueness of Minimizers).
Let ϕ ∈ C.Let A be a
linear and bounded operator from L
p
(Ω) to L
2
(Ω) where 1 ≤ p ≤ d/(d − 1).If
∥A1
Ω
∥
1
> 0,then the functional F
λ,β
(·,z) has a minimizer over BV (Ω).The unique
ness of the minimizer hold if A is injective and ϕ is strictly convex.
Proof.
Let {u
n
} be a sequence in BV (Ω) such that
lim
n→∞
F
λ,β
(u
n
,z) = inf
u∈BV (Ω)
F
λ,β
(u,z).(3.20)
By Lemma 3.2,the sequence {u
n
}
n
is bounded in BVnorm.It was pointed in [1]
that there exists a subsequence {u
n
k
}
k
which converges to some bu ∈ L
p
(Ω) if 1 ≤ p <
d/(d −1),or converges weakly if p = d/(d −1) and d = 2.By (3.9) we have that
F
λ,β
(bu,z) ≤ liminf
k→∞
F
λ,β
(u
n
k
,z).
This inequality and (3.20) lead to
F
λ,β
(bu,z) ≤ inf
u∈BV (Ω)
F
λ,β
(u,z).
Therefore,bu is a minimizer of the functional F
λ,β
(·,z) over BV (Ω).Furthermore,if
A is injective and ϕ is strictly convex,by Lemma 3.1,the F
λ,β
(·,z) is strictly convex.
Hence,the minimizer is unique.
Next,we study the stability of the minimization problem(3.10) whose exact mean
ing is deﬁned below.Consider a sequence of perturbed problems of (1.2)
z
n
= A
n
u,(3.21)
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 34
where A
n
,1 ≤ n < ∞,are linear and bounded operators from L
p
(Ω) to L
2
(Ω) and
are injective.For each problem,we consider the minimization problem
inf
u∈BV (Ω)
F
λ,β,n
(u,z
n
),(3.22)
where
F
λ,β,n
(u,z
n
):= ∥ϕ(A
n
u −z
n
)∥
1
+λ · TV
β
(u),β > 0,(3.23)
with ϕ ∈ C being strictly convex.By Theorem 3.3,we can assume that bu and
bu
n
are unique minimizers of (3.10) and (3.22),respectively.The stability of the
minimization problem (3.10) refers to that if A
n
converges to A in the operator
norm and z
n
converges to z in the L
2
(Ω) norm,then bu
n
converges to bu in L
p
(Ω) for
1 ≤ p < d/(d −1) and weakly converges for p = d/(d −1) and d = 2.
According to Theorem3.2 of [1],if the functionals F
λ,β
(·,z) and F
λ,β,n
(·,z
n
) satisfy
the following two conditions,then the minimization problem (3.10) is stable:
(i)
For any sequence {u
n
} in L
p
(Ω),1 ≤ p ≤ d/(d −1),
lim
n→+∞
F
λ,β,n
(u
n
,z
n
) = +∞ whenever lim
n→+∞
∥u
n
∥
BV
= +∞.(3.24)
(ii)
For a given M > 0 and ϵ > 0,there exists N such that
F
λ,β,n
(u,z
n
) −F
λ,β
(u,z) < ϵ whenever n ≥ N,∥u∥
BV
≤ M.(3.25)
Theorem 3.4.
Let ϕ ∈ C be strictly convex.Let A and A
n
deﬁned in (1.1) and
(3.21) be linear bounded and injective operators and map from L
p
(Ω) to L
2
(Ω).Let
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 35
F
λ,β
(·,z) and F
λ,β,n
(·,z
n
) be deﬁned respectively by (3.8) and (3.23).Let bu and bu
n
be unique minimizers of (3.10) and (3.22),respectively.Assume that ∥z
n
−z∥
2
→0
and ∥A
n
−A∥ →0 with
∥A
n
1
Ω
∥
1
≥ γ > 0.(3.26)
If 1 ≤ p < d/(d −1),then
lim
n→+∞
∥bu
n
−bu∥
p
= 0.(3.27)
If p = d/(d −1) and d = 2,then bu
n
converges weakly to bu.
Proof.
Based on the discussion prior to this theorem,we only need to show that
F
λ,β
(·,z) and F
λ,β,n
(·,z
n
) satisfy conditions (i) and (ii).
We begin proving (i).To this end,it suﬃces to show that for any sequence {u
n
} in
L
p
(Ω),if liminf
n→∞
F
λ,β,n
(u
n
,z
n
) = K < ∞,then liminf
n→∞
TV(u
n
) < ∞.Clearly,
if liminf
n→∞
F
λ,β,n
(u
n
,z
n
) = K < ∞,then there exists a subsequence {u
n
k
}
k
of {u
n
}
such that
lim
k→∞
F
λ,β,n
k
(u
n
k
,z
n
k
) = liminf
n→∞
F
λ,β,n
(u
n
,z
n
) = K.
Hence,the sequence {F
λ,β,n
k
(u
n
k
,z
n
k
)}
k
is bounded.
To show liminf
n
∥u
n
∥
BV
< +∞,it suﬃces to prove that {∥u
n
k
∥
BV
}
k
is bounded.
By (3.12)(3.13),each function in {u
n
k
} can be decomposed as follows
u
n
k
= v
n
k
+w
n
k
with w
n
k
=
1
Ω
∫
Ω
u
n
k
dx.
By (3.14),we just need to show that both sequences {∥w
n
k
∥
1
}
k
and {TV
0
(v
n
k
)}
k
are
bounded.Fortunately,the boundednesses of {∥w
n
k
∥
1
}
k
and {TV
0
(v
n
k
)}
k
follow by
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 36
using the same procedure as the one in Lemma 3.2.
Now we turn to proving (ii).Notice that for any u ∈ BV (Ω),
F
λ,β,n
(u,z
n
) −F
λ,β
(u,z) ≤ ∥ϕ(A
n
u −z
n
) −ϕ(Au −z)∥
1
.
By Lemma 2.6,we have that
F
λ,β,n
(u,z
n
)−F
λ,β
(u,z) ≤ c
ϕ
(∥A
n
u−z
n
∥
2
+∥Au−z∥
2
+Ω
1/2
)∥(A
n
u−z
n
)−(Au−z)∥
2
.
Since ∥z
n
−z∥
2
→0 and ∥A
n
−A∥ →0,∥A
n
u −z
n
∥
2
and ∥Au −z∥
2
are uniformly
bounded on the set {uTV(u) ≤ M}.Furthermore,we have
∥(A
n
u−z
n
) −(Au−z)∥
2
≤ ∥A
n
−A∥∥u∥
p
+∥z
n
−z∥
2
≤ ∥A
n
−A∥∥u∥
BV
+∥z
n
−z∥
2
.
Thus,we know that (3.25) holds.
In summary,the minimization problem (3.10) is wellposed by Theorem 3.3 and
Theorem 3.4.
3.3 Convergence of Minimizers
Consider the equation
z = Au,(3.28)
where A is a linear and bounded operator from L
p
(Ω) to L
2
(Ω).We assume that
(3.28) has an exact solution u
e
.A sequence of perturbed equations to (3.28) is given
by (3.21) with A
n
,1 ≤ n < ∞,being linear and bounded operators from L
p
(Ω)
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 37
to L
2
(Ω).Let u
λ
n
,n
be approximate solutions to (3.21) obtained by the following
minimization problems
inf
u∈BV (Ω)
F
λ
n
,β,n
(u,z
n
),(3.29)
where
F
λ
n
,β,n
(u,z
n
):= ∥ϕ(A
n
u −z
n
)∥
1
+λ
n
· TV
β
(u),β > 0.(3.30)
Under certain conditions,we will show that u
λ
n
,n
converges to u
e
as regularization
parameters λ
n
approach to 0.Although the following theorem is motivated by [1],its
proof is distinctive.
Theorem 3.5.
Let ϕ ∈ C be strictly convex.Let A and A
n
deﬁned in (3.28) and
(3.21) be injective and map from L
p
(Ω) to L
2
(Ω).Let F
λ
n
,β,n
(·,z
n
) be deﬁned by
(3.30).Let u
λ
n
,n
be unique minimizers of (3.29).Assume that ∥z
n
− z∥
2
→ 0,
∥A
n
−A∥ →0,and ∥A
n
1
Ω
∥
1
≥ γ > 0.If λ
n
→0 at a rate such that
F
0,β,n
(u
e
,z
n
) −F
0,β,n
(u
0,n
,z
n
) = O(λ
n
),(3.31)
where u
0,n
is the minimizer of F
0,β,n
,then u
λ
n
,n
strongly converges to u
e
in L
p
(Ω) for
1 ≤ p < d/(d −1) and weakly converges for p = d/(d −1) and d = 2.
Proof.
The proof is basically split into three parts.The ﬁrst part is to show the
boundedness of the sequence {F
λ
n
,β,n
(u
λ
n
,n
,z
n
)}
n
;the second part is about the bound
edness of the sequence {∥u
n
∥
BV
}
n
;the last part is the convergence of u
n
to u
e
.
Since ∥z
n
−z∥
2
→0,∥A
n
−A∥ →0,and λ
n
→0,{F
λ
n
,β,n
(u
e
,z
n
)}
n
is bounded
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 38
by using the proof of Lemma 2.5.Since u
λ
n
,n
is the minimizer of (3.29),we have that
F
λ
n
,β,n
(u
λ
n
,n
,z
n
) ≤ F
λ
n
,β,n
(u
e
,z
n
).(3.32)
Thus,{F
λ
n
,β,n
(u
λ
n
,n
,z
n
)}
n
is bounded.This ﬁnishes the proof of the ﬁrst part.
We now show that there exists a constant M such that ∥u
λ
n
,n
∥
BV
≤ M for all n.
By (3.14),it suﬃces to show that both sequences {∥w
λ
n
,n
∥
1
}
n
and {TV
0
(v
λ
n
,n
)}
n
are
bounded,where
u
λ
n
,n
= v
λ
n
,n
+w
λ
n
,n
with w
λ
n
,n
=
1
Ω
∫
Ω
u
λ
n
,n
dx.
The boundedness of {∥w
λ
n
,n
∥
1
}
n
can be obtained by following the same way as we
did in Lemma 3.2.Because TV
β
(u
λ
n
,n
) = TV
β
(v
λ
n
,n
),
F
λ
n
,β,n
(u
λ
n
,n
,z
n
) = F
0,β,n
(u
λ
n
,n
,z
n
) +λ
n
· TV
β
(u
n
),
and
F
λ
n
,β,n
(u
e
,z
n
) = F
0,β,n
(u
e
,z
n
) +λ
n
· TV
β
(u
e
).
we have from (3.32) that
F
0,β,n
(u
λ
n
,n
,z
n
) −F
0,β,n
(u
0,n
,z
n
)
λ
n
+TV
β
(v
λ
n
,n
)
≤
F
0,β,n
(u
e
,z
n
) −F
0,β,n
(u
0,n
,z
n
)
λ
n
+TV
β
(u
e
).(3.33)
Since
F
0,β,n
(u
e
,z
n
)−F
0,β,n
(u
0,n
,z
n
)
λ
n
is bounded and F
0,β,n
(u
λ
n
,n
,z
n
)−F
0,β,n
(u
0,n
,z
n
) is non
negative due to u
0,n
being the minimizer of F
0,β,n
(·,z
n
),the above inequality yields
that the sequence {TV
β
(u
λ
n
,n
)}
n
is bounded.Hence,the sequence {∥u
λ
n
,n
∥
BV
}
n
is
bounded.This ﬁnishes the proof of the second part.
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 39
Now we turn to the last part of the proof.It will be suﬃcient to show that every
subsequence {u
λ
n
k
,n
k
}
k
of the sequence {u
λ
n
,n
}
n
has a subsequence which converges
to u
e
in L
p
(Ω) if 1 ≤ p < d/(d −1),or converges weakly if p = d/(d −1) and d = 2.
Let {u
λ
n
k
,n
k
}
k
be an arbitrary subsequence of {u
λ
n
,n
}
n
.Then {∥u
λ
n
k
,n
k
∥
BV
}
k
is
bounded.There exists a subsequence of {u
λ
n
k
,n
k
}
k
which converges to some bu ∈ L
p
(Ω)
if 1 ≤ p < d/(d −1),or converges weakly if p = d/(d −1) and d = 2.Without loss of
generality,we assume that this subsequence is {u
λ
n
k
,n
k
}
k
.Our goal is to show bu = u
e
.
It is equivalent to showing F
0,β
(bu,z) = F
0,β
(u
e
,z).
Note that
F
0,β
(bu,z) −F
0,β
(u
e
,z)
≤ F
0,β,n
k
(u
λ
n
k
,n
k
,z
n
k
) −F
0,β
(bu,z) +F
0,β,n
k
(u
λ
n
k
,n
k
,z
n
k
) −F
0,β,n
k
(u
0,n
k
,z
n
k
)
+F
0,β,n
k
(u
0,n
k
,z
n
k
) −F
0,β,n
k
(u
e
,z
n
k
) +F
0,β,n
k
(u
e
,z
n
k
) −F
0,β
(u
e
,z).
The four terms in the righthand of the above inequality are denoted by I
1
,I
2
,I
3
,
and I
4
,conclusively.We will show that these terms approximate to zero when k
approaches to inﬁnity.
For I
4
,by Lemma 2.6 we have that
I
4
≤ c
ϕ
(
∥A
n
k
u
e
−z
n
k
∥
2
+∥Au
e
−z∥
2
+Ω
1/2
)
∥(A
n
k
u
e
−z
n
k
) −(Au
e
−z)∥
2
.
The assumptions of ∥z
n
− z∥
2
→ 0 and ∥A
n
− A∥ → 0 imply that {∥A
n
∥}
n
and
{∥z
n
∥
2
}
n
are bounded and ∥(A
n
k
u
e
−z
n
k
) −(Au
e
−z)∥
2
→0.Hence,I
4
goes to zero
when k approaches to inﬁnity.
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 40
By the given condition,I
3
≤ Cλ
n
k
,where C is a constant.Hence,I
3
→ 0 as
λ
n
k
→0.
For I
1
,we have,by Lemma 2.6,
I
1
≤ c
ϕ
(
∥A
n
k
u
λ
n
k
,n
k
−z
n
k
∥
2
+∥Abu −z∥
2
+Ω
1/2
)
∥(A
n
k
u
λ
n
k
,n
k
−z
n
k
) −(Abu −z)∥
2
.
We notice that ∥A
n
k
u
λ
n
k
,n
k
∥
2
≤ ∥A
n
k
∥∥u
λ
n
k
,n
k
∥
p
≤ ∥A
n
k
∥∥u
λ
n
k
,n
k
∥
BV
.Because
∥A
n
k
∥ is uniformly bounded and {∥u
λ
n
k
,n
k
∥
BV
}
k
is bounded,we see that ∥A
n
k
u
λ
n
k
,n
k
∥
2
is bounded.We also have that
∥(A
n
k
u
λ
n
k
,n
k
−z
n
k
) −(Abu −z)∥
2
≤ ∥A
n
k
u
λ
n
k
,n
k
−Au
λ
n
k
,n
k
∥
+ ∥Au
λ
n
k
,n
k
−Abu∥
2
+∥z
n
k
−z∥
2
.
Since ∥A
n
k
−A∥ → 0,∥u
λ
n
k
,n
k
− bu∥
BV
→ 0,and ∥z
n
k
−z∥
2
→ 0,we have that I
1
converges to zero as k →∞.
For I
2
,by (3.33) we have
F
0,β,n
(u
λ
n
,n
,z
n
) −F
0,β,n
(u
0,n
,z
n
) ≤ F
0,β,n
(u
e
,z
n
) −F
0,β,n
(u
0,n
,z
n
)
+λ
n
· (TV
β
(u
e
) −TV
β
(v
λ
n
,n
)).
By (3.31),we have F
0,β,n
(u
e
,z
n
) −F
0,β,n
(u
0,n
,z
n
) →0 as λ
n
k
→0.Since TV
β
(u
e
)
and TV
β
(v
λ
n
,n
) are bounded,we have that I
2
→0 as λ
n
k
→0.The proof is complete.
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 41
3.4 EulerLagrange Equations
In this chapter,we proposed a minimization estimation (3.10) to solve the illposed
problem (1.2).The existence of the minimizer of F
λ,β
(u,z) with ϕ ∈ C over BV
space has been proved.A classical way to ﬁnd a minimizer of F
λ,β
is to solve the
EulerLagrange Equation of (3.10),which is
A
∗
ϕ
′
(Au −z) −λdiv
(
∇u
√
∇u +β
2
)
= 0 (3.34)
provided ϕ
′
exists on R and the Neumann boundary condition is given,
∂u
∂ν
= 0 on ∂Ω (ν is the outward normal to ∂Ω),(3.35)
where ∇u is the gradient of u in the distribution sense.In previous sections,we pro
vide examples of ϕ ∈ C.Their corresponding EulerLagrange Equations are presented
as follows:
i) Given ϕ deﬁned as the L
p
function with 1 < p ≤ 2,
pA
∗
sign(Au −z)Au −z
p−1
−λdiv
(
∇u
√
∇u +β
2
)
= 0.(3.36)
ii) Given ϕ deﬁned as (2.10),
A
∗
2η
2
(Au −z) +sign(Au −z)η(Au −z)
2
2(η +Au −z)
2
−λdiv
(
∇u
√
∇u +β
2
)
= 0.(3.37)
iii) Given ϕ deﬁned as (2.11),
1
2
A
∗
η tanh(
Au −z
η
) −λdiv
(
∇u
√
∇u +β
2
)
= 0.(3.38)
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 42
iv) Given ϕ deﬁned as (2.12),
A
∗
η(Au −z)
√
η
2
+(Au −z)
2
−λdiv
(
∇u
√
∇u +β
2
)
= 0.(3.39)
v) Given ϕ deﬁned as (2.13),
A
∗
η arctan(
Au −z
η
) −λdiv
(
∇u
√
∇u +β
2
)
= 0.(3.40)
The EulerLagrange equations (3.363.40) of ﬁve examples of ϕ have the Neumann
boundary condition.
Recall that C
m
(
¯
Ω):={u ∈ C
m
(Ω)∇
α
1
,...,α
d
u is uniformly continuous on bounded
subsets of Ω,for all α
1
+...+α
d
 ≤ m}.If ϕ
′
exists and continuous,the minimization
functional F
λ,β
deﬁned as (3.8) is diﬀerentiable and its Gˆateaux derivative is
G(u,z) = A
∗
ϕ
′
(Au −z) −λdiv
(
∇u
√
∇u +β
2
)
.
In this thesis,we consider that the operator A of (1.1) is a linear and bounded
operator mapping from L
p
(Ω) into L
2
(Ω).Since C
2
(Ω) ⊂ BV (Ω),we provide the
optimality condition of F
λ,β
presented as follows:
Theorem 3.6.
Assume that ϕ ∈ C is continuously diﬀerentiable.The following
statement holds:u
∗
is a minimizer of F
λ,β
(u,z) deﬁned as (3.8) over a convex set
U ⊂ C
2
(
¯
Ω) if and only if
∫
Ω
G(u
∗
,z)(u −u
∗
)dx ≥ 0,∀u ∈ U.(3.41)
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 43
Proof.
If u
∗
is a minimizer of F
λ,β
over U,we suppose that
∫
Ω
G(u
∗
,z)(u −u
∗
)dx =
−ξ < 0 for some u ∈ U.By the mean value theorem,for every ε > 0 there exist an
s ∈ [0,1] such that
F
λ,β
(u
∗
+ε(u −u
∗
),z) = F
λ,β
(u
∗
,z) +ε
∫
Ω
G(u
∗
+sε(u −u
∗
),z)(u −u
∗
)dx.(3.42)
In order to show that the second term of (3.42) is also smaller than zero,we let
u
ε,s
= u
∗
+sε(u −u
∗
).Then

∫
Ω
[G(u
ε,s
,z) −G(u
∗
,z)] (u −u
∗
)dx
≤
∫
Ω
A
∗
[ϕ
′
(Au
ε,s
−z) −ϕ
′
(Au
∗
−z)] (u −u
∗
) dx
+λ
∫
Ω
div
(
∇u
ε,s
√
∇u
ε,x

2
+β
2
−
∇u
√
∇u
2
+β
2
)
(u −u
∗
)
dx
(3.43)
By the continuity of ϕ
′
and the boundness of A,we ﬁnd that the ﬁrst term of (3.43)
is less than ξ/2 for all suﬃciently small ε.We have that
div
∇u
√
∇u
2
+β
2
=
d
∑
i=1
∇
i
(
∇u
√
∇u
2
+β
2
)
.(3.44)
By u ∈ C
2
(
¯
Ω),we have that (3.44) is continuous.Therefore,for all suﬃciently small
ε,the second term of (3.43) can be less than ξ/(2λ).Then
∫
Ω
G(u
ε,s
,z)(u −u
∗
)dx < 0
for all suﬃciently small ε.Then u
ε
is the minimizer of F
λ,β
,which contradicts the
hypothesis of this theorem.Thus,(3.41) holds.
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 44
On the other hand,by the convexity of F
λ,β
,we have that
F
λ,β
(u,z) ≥ F
λ,β
(u
∗
,z) +
∫
Ω
G(u
∗
,z)(u −u
∗
)dx,∀u ∈ U.(3.45)
If (3.41) holds,we have that u
∗
is a minimizer of F
λ,β
over U.
Chapter 4
Discrete ContentDriven
Variational Models
We recall the contentdriven variational model as follows
inf
u∈BV(Ω)
F
λ,β
(u,z),(4.1)
where Ω ∈ R
d
and
F
λ,β
(u,z):= ∥ϕ(Au −z)∥
1
+λ · TV
β
(u),ϕ ∈ S.(4.2)
Wellposedness of the model (4.1) has been discussed in the previous chapters.In
this chapter,we apply four gradient based algorithms,SD,CG,HB,and SDTPSS
algorithms,to ﬁnd the solution of the regularized problem (4.1) and provide brief
discussions about strengths and limitations of these gradient based algorithms.The
convergence analysis of the SD and CG algorithms is presented in the following sec
tions.
45
CHAPTER 4.DISCRETE CONTENTDRIVEN VARIATIONAL MODELS 46
4.1 Steepest Descent Algorithms for the Content
Driven Variational Model
In this section,discrete models to the problem (4.1) in one dimensional case and two
dimensional case are given.We apply the steepest descent algorithm to the discrete
models and prove the convergence of the steepest algorithm in one dimensional case
and two dimensional case.
4.1.1 Discrete ContentDriven Variational Models in One Di
mension
Let d = 1 and Ω = (0,1) ⊂ R.We partition
¯
Ω:= [0,1] into M equalspaced intervals
as
¯
Ω =
∪
M−1
i=0
[ih,(i +1)h],where h =
1
M
.Let z:(0,1) →R be the observed image
and u:(0,1) →R be the solution of the proposed minimization problem (4.1).The
discretelization of u is presented as
u
i
= u
(
(2i −1)h
2
)
for 0 < i < M +1.
Therefore u is discretized as u = (u
1
,...,u
M
).The discrete description of z is as
same as u.
CHAPTER 4.DISCRETE CONTENTDRIVEN VARIATIONAL MODELS 47
We deﬁne a ‘forwarddiﬀerence’ matrix D ∈ R
M×M
as
D =
1
M
−1 1
.
.
.
.
.
.
−1 1
0
.(4.3)
We notice that Du = (Du
1
,...,Du
M−1
,0)
T
is the forward diﬀerence of u with im
posing Neumann boundary conditions
u
0
= u
1
,and u
M+1
= u
M
.(4.4)
and the ith element of Du has the form Du
i
=
u
i+1
−u
i
h
.The vector Du is regarded as
an approximation of ∇u.We consider a matrix A ∈ R
M×M
as a numerical approxi
mation of the degradation operator A such that Au is an approximation of Au.
We provide the discrete form of the minimization functional F
λ,β
deﬁned as (4.2).
By applying the quadrature rule to (4.2),the discrete functional F
λ,β
(·,z):R
M
→R
has the form
F
λ,β
(u,z) =
1
M
M
∑
i=1
ϕ(Au
i
−z
i
) +λ · ψ
β
(Du
i
),∀u ∈ R
M
,ϕ ∈ S,(4.5)
where λ > 0 and Au
i
denotes the ith element of Au.The function ψ
β
:R → R
+
is deﬁned as ψ
β
(·):=
√
(·)
2
+β
2
.Therefore,the discrete contentdriven variational
model is
inf
u∈R
M
F
λ,β
(u,z) (4.6)
with F
λ,β
(u,z) as (4.5).
CHAPTER 4.DISCRETE CONTENTDRIVEN VARIATIONAL MODELS 48
4.1.2 Steepest Descent Algorithms
We apply the steepest descent algorithm to solve the minimization problem (4.6).
The algorithm is presented as follows:
•
Step 1:Given an initial value u
0
,calculate d
0
= −∇F
λ,β
(u
0
,z).
•
Step 2:If u
n
is calculated for n ≥ 0,then compute
u
n+1
= u
n
+α
n
d
n
,(4.7)
where d
n
= −∇F
λ,β
(u
n
,z).Let n = n +1 and go to Step 2.
We consider u
n+1
as an estimation of the solution of the problem (4.5) if n is large
enough.There are many choices for setting up the step size α
n
.We mainly use two
schemes,namely,the constant step size and the Armijo line search.The constant step
size is the simplest and the most popular selection of the step size.It shows a great
advantage in computational eﬃciency.The scheme is simply presented as:Given a
positive constant S,let
α
n
= S ∀n ∈ N.
However,the ﬁxed step size can not guarantee that the functional F
λ,β
(u
n
,z) is
decreasing during the iterations.The Armijo rule is a classic line search scheme,
mentioned in [8] as:Given positive ﬁxed scalars S,ρ,and δ with 0 < ρ < 1 and
0 < δ < 1,we set α
n
= ρ
m
n
S,where m
n
is the ﬁrst nonnegative integer m for which
F
λ,β
(u
n
,z) −F
λ,β
(u
n
+ρ
m
Sd
n
,z) ≥ −δρ
m
S∇F
λ,β
(u
n
,z)
T
d
n
.(4.8)
CHAPTER 4.DISCRETE CONTENTDRIVEN VARIATIONAL MODELS 49
The reduction factor ρ is usually chosen from [1/10,1/2].The Armijo rule keeps
the functional F
λ,β
decreasing at each iteration.However,it scariﬁes computation
eﬃciency due to the line search.
4.1.3 Convergence of Steepest Descent Methods in One Di
mension
We will prove convergence of the steepest descent algorithm with constant step size
and Armijo line search step size for solving the minimization problem (4.6).First of
all,we recall the following preliminary results in [8],which are convergence results of
an unconstrained minimization problem
inf
u∈R
M
F(u) (4.9)
where F(u):R
M
→R.We refer to a vector u
∗
satisfying the condition ∇F(u
∗
) = 0
as a stationary point of F(u).We say that the direction sequence {d
n
}
n
is gradi
ent related to {u
n
}
n
if the following property can be shown:For any subsequence
{u
n
k
}
n
k
⊂ {u
n
}
n
that converges to a nonstationary point,the corresponding subse
quence {d
n
k
}
n
k
is bounded and satisﬁes
limsup
n
k
→∞
∇F(u
n
k
)
T
d
n
k
< 0.(4.10)
Gradient based algorithms for solving (4.9) have a general form the same as (4.7),
except that the descent direction d
n
and the step size α
n
can be arbitrarily selected.
Diﬀerent selections of d
n
and α
n
give diﬀerent algorithms for solving the problem
CHAPTER 4.DISCRETE CONTENTDRIVEN VARIATIONAL MODELS 50
(4.9).In [8],convergence of gradient based algorithms with a constant step size is
stated as follows:Let {u
n
}
n
be a sequence of vectors generated by (4.7),where d
n
is
gradient related to {u
n
}
n
.Assume that for some constant L > 0,we have that
∥∇F(u) −∇F(v)∥
2
≤ L∥u −v∥
2
,∀u,v ∈ R
M
,(4.11)
and that for all n we have d
n
̸= 0 and ε ≤ α
n
≤ (2 −ε)
α
n
,where
α
n
=
∇F(u
n
)
T
d
n

L∥d
n
∥
2
and ε is a ﬁxed positive scalar.Then every limit point of {u
n
}
n
is a stationary point.
Now we calculate the gradient of F
λ,β
(·,z) in order to study the convergence of
the steepest descent algorithm for solving (4.6).Let g(·,z) = ∇F
λ,β
(·,z).A function
Φ:R
M
→R
M
is deﬁned as
Φ(u):= (ϕ
′
(u
1
),...,ϕ
′
(u
M
))
T
,
and Ψ:R
M
→R
M
is given by
Ψ(u):=
(
ψ
′
β
(u
1
),...,ψ
′
β
(u
M
)
)
T
,(4.12)
where ψ
′
β
(x) =
x
ψ
β
(x)
.By calculation,the gradient of F
λ,β
(u,z) with respect to u has
the form
g(u,z) =
1
M
A
T
Φ(Au −z) +
λ
M
D
T
Ψ(Du).(4.13)
Letting g
1
(u,z) =
1
M
A
T
Φ(Au −z) and g
2
(u) =
1
M
D
T
Ψ(Du),we have that
g(u,z) = g
1
(u,z) +λg
2
(u).(4.14)
In order to show convergence of the steepest descent algorithm for minimizing the
functional (4.5),we show below that {d
n
= −g(u
n
,z)}
n
is gradient related and the
gradient of F
λ,β
(·,z) satisﬁes (4.11).
CHAPTER 4.DISCRETE CONTENTDRIVEN VARIATIONAL MODELS 51
Lemma 4.1.
Let {u
n
}
n
be a sequence of vectors generated by the steepest descent
method (4.7) for minimizing the functional F
λ,β
(u,z) in (4.5).Let g(u,z) be the
gradient of F
λ,β
(u,z) with respect to u,which has the form (4.13).Then,the sequence
of directions {d
n
= −g(u
n
,z)}
n
is gradient related.
Proof.
We need to show that {d
n
= −g(u
n
,z)}
n
is bounded and satisﬁes (4.10).By
(4.14),we have that
g(u
n
,z) = g
1
(u
n
,z) +λg
2
(u
n
).
We need to show that both g
1
and g
2
are bounded.By the hypothesis H3,there
exists a positive number c
ϕ
such that ϕ
′
(x) ≤ c
ϕ
.We have that
∥g
1
(u
n
,z)∥
2
= ∥
1
M
A
T
Φ(Au
n
−z)∥
2
≤
1
M
∥A
T
∥∥Φ(Au
n
−z)∥
2
≤ c
ϕ
∥A
T
∥.
Thus,the sequence {g
1
(u
n
,z)}
n
is bounded.Next,we need to show that g
2
is also
bounded.The boundedness of g
2
(u) is determined by the boundedness of Ψ,which is
composed by the scalar function ψ
′
β
(x).Since the scalar function ψ
′
β
(x) is monotone
increasing and bounded by 1,we have that
∥g
2
(u
n
)∥
2
=
1
M
∥D
T
Ψ(Du)∥
2
≤
1
M
∥D∥∥Ψ(Du)∥
2
≤ ∥D∥.
Therefore,
∥g(u
n
,z)∥
2
≤ ∥g
1
(u
n
,z)∥
2
+λ∥g
2
(u
n
)∥
2
≤ c
ϕ
∥A
T
∥ +λ∥D∥.
Secondly,we will show that (4.10) is true for {d
n
}
n
.Since d
n
= −g(u
n
,z) in the
steepest descent algorithm,we have that
g(u
n
,z)
T
d
n
= −∥g(u
n
,z)∥
2
2
≤ 0,∀n ∈ N.
CHAPTER 4.DISCRETE CONTENTDRIVEN VARIATIONAL MODELS 52
Therefore,we only need to show that if any subsequence {u
n
k
}
n
k
⊂ {u
n
}
n
con
verges to a nonstationary point
ˆ
u,i.e.,limsup
n
k
→∞
u
n
k
=
ˆ
u and g(
ˆ
u,z) ̸= 0,then
the inequality limsup
n
k
→∞
g(u
n
k
,z)
T
d
n
k
̸= 0 holds.We assume that there exists
a subsequence {u
n
k
}
n
k
satisfying limsup
n
k
→∞
g(u
n
k
,z)
T
d
n
k
= 0.Since g(u
n
k
,z) is
bounded,there exists a subsequence {u
n
k
j
}
n
k
j
⊂ {u
n
k
}
n
k
such that
lim
n
k
j
→∞
g(u
n
k
j
,z)
T
d
n
k
j
= lim
n
k
j
→∞
∥g(u
n
k
j
,z)∥
2
2
= ∥g(ˆu,z)∥
2
2
= 0.
Thus,we have g(ˆu,z) = 0,which contradicts to the fact that ˆu is a nonstationary
point.
Lemma 4.2.
The gradient of the functional F
λ,β
(·,z) deﬁned as in (4.5) is Lipschitz
continuous,i.e.,for some constant L > 0,
∥∇F(u,z) −∇F(v,z)∥
2
≤ L∥u −v∥
2
,∀u,v ∈ R
M
.(4.15)
Proof.
Since g(·,z) = ∇F
λ,β
(·,z) having the form (4.14),we have that
∥g(u,z) −g(v,z)∥
2
≤ ∥g
1
(u,z) −g
1
(v,z)∥
2
+λ∥g
2
(u) −g
2
(v)∥
2
,∀u,v ∈ R
M
.
Firstly,we will show that there exists a positive number L
1
such that
∥g
1
(u,z) −g
1
(v,z)∥
2
≤ L
1
∥u −v∥
2
.(4.16)
By Lemma 4.1,the each component of g
1
(u,z) can be characterized by ϕ
′
.Since
ϕ ∈ S,there exists a constant C such that ϕ
′′
(x) < C for all x ∈ R.Therefore,by
the mean value theorem
ϕ
′
(x) −ϕ
′
(y) ≤ ϕ
′′
(ξ)x −y ≤ Cx −y,(4.17)
CHAPTER 4.DISCRETE CONTENTDRIVEN VARIATIONAL MODELS 53
where ξ ∈ [x,y].Replacing x by each component of Au−z and y by the corresponding
component of Av −z,we have that
∥g
1
(u,z) −g
1
(v,z)∥
2
≤
1
M
∥A
T
∥∥Φ(Au −z) −Φ(Av −z)∥
2
≤
C
M
∥A
T
∥∥A∥∥u −v∥
2
.
Thus (4.16) holds with L
1
=
C
M
∥A∥
2
.
Secondly,we will show that there exists a positive number L
2
such that
∥g
2
(u) −g
2
(v)∥
2
≤ L
2
∥u −v∥
2
,∀u,v ∈ R
M
.(4.18)
By the deﬁnition of g
2
,we have that
∥g
2
(u) −g
2
(v)∥
2
=
1
M
∥D
T
Ψ(Du) −D
T
Ψ(Dv)∥
2
≤
1
M
∥D∥∥Ψ(Du) −Ψ(Dv)∥
2
.
We notice that Ψ is deﬁned as in (4.12) and ψ
′′
β
(x) = β
2
(x
2
+β
2
)
−3/2
≤
1
β
.By the
mean value theorem,there exists ξ ∈ (x,y) such that
ψ
′
β
(x) −ψ
′
β
(y) ≤ ψ
′′
β
(ξ)x −y.
We have that
∥g
2
(u) −g
2
(v)∥
2
≤
∥D∥
Mβ
∥Du −Dv∥
2
≤
∥D∥
2
Mβ
∥u −v∥
2
.
Comments 0
Log in to post a comment