ABSTRACT Image restoration is a basic problem of ... - Umi.com

pancakesnightmuteΤεχνίτη Νοημοσύνη και Ρομποτική

5 Νοε 2013 (πριν από 3 χρόνια και 9 μήνες)

194 εμφανίσεις

ABSTRACT
Image restoration is a basic problemof image processing.Its objective is to restore
an image,blurred by a smoothing operator or contaminated by additive noises,from
its deterioration.Image restoration has been widely applied to medical imaging and
astronomy.
Regularization is a primary method used in image restoration.The variation
functional of a regularized method contains two terms:a fidelity term and a regu-
larization term.Therefore,characteristics of a regularized method are determined
by selections of the fidelity and the regularizer.In past decades,a great amount of
studies of regularized methods have focused on selection of the fidelity as an L
p
norm
(p = 1,2).In this thesis,we concentrate on selection of other possible fidelities with
a total variation regularizer.
We characterize a general class of fidelities,which includes existing fidelities in the
literatures as special examples.We prove the well-posedness of the resulting regular-
ized minimization problem,composed of a fidelity from the general class of fidelities
and the total variation regularization.We also show convergence of minimizers of the
regularized minimization problem as errors in data and the blurring operator tend to
zero.
We specify a new subclass of fidelities:content-driven fidelities.Content-driven
fidelities provide spatial varying measurements of the recovered image to the observed
image,which can accurately adapt to local image contents.They interpolate strengths
ii
of a variety of existing fidelities under different image contents.Four gradient-based
algorithms,including the steepest descent algorithm(SD),heavy ball algorithm(HB),
steepest descent algorithm with two point step size (SD-TPSS),and conjugate gradi-
ent algorithm(CG),are applied to solve the content-driven minimization composed of
a content-driven fidelity and the total variation regularization.We also show conver-
gence of the steepest gradient algorithm and the conjugate gradient algorithm based
on the content-driven estimation.
We use numerical studies to assess properties of the L
p
-norm estimations,com-
posed of L
p
-norm fidelities and the total variation regularizer,for 1 ≤ p ≤ 2 under
different image contents.We observe the relationship of the performance of the L
p
-
norm estimation,the selection of p,the noise levels,and the presence of outliers.Fi-
nally,practical effectiveness of the content-driven approach is shown on synthetically
generated data.The superiorities of the content-driven estimation over the L
p
-norm
estimation (1 ≤ p ≤ 2) and other existing estimations are numerically demonstrated.
All rights reserved
INFORMATION TO ALL USERS
The quality of this reproduction is dependent on the quality of the copy submitted.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.
All rights reserved. This edition of the work is protected against
unauthorized copying under Title 17, United States Code.
ProQuest LLC.
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106 - 1346
UMI 3459386
Copyright 2011 by ProQuest LLC.
UMI Number: 3459386
Copyright
c
⃝2010,Xiaofei Hu
All Rights Reserved
Contents
Acknowledgments viii
Notations x
1 Introduction 1
1.1 Image Restoration and Regularized Methods..............1
1.2 Choices of Regularization Functional..................3
1.3 Choices of Data-Fidelity Term......................5
1.4 Content-Driven Fidelity Terms......................6
2 Content-Driven Fidelity Terms 10
2.1 The General Class of Fidelity Terms..................11
2.2 The Class of Content-Driven Fidelity Terms..............16
3 Variational Models for Image Restoration 26
3.1 Preliminary on the Bounded Variation Space..............27
3.2 Well-Posedness of the General Variational Model............29
v
CONTENTS vi
3.3 Convergence of Minimizers........................36
3.4 Euler-Lagrange Equations........................41
4 Discrete Content-Driven Variational Models 45
4.1 Steepest Descent Algorithms for the Content-Driven Variational Model 46
4.1.1 Discrete Content-Driven Variational Models in One Dimension 46
4.1.2 Steepest Descent Algorithms...................48
4.1.3 Convergence of Steepest Descent Methods in One Dimension.49
4.1.4 Discrete Content-Driven Variational Models in Two Dimension 55
4.1.5 Convergence of Steepest Descent Methods for an Anisotropic
Total Variation and Content-Driven Variational Model....57
4.1.6 Convergence of Steepest Gradient Methods for an Isotropic To-
tal Variation and Content-Driven Variational Model......61
4.2 Conjugated Gradient Algorithms for the Content-Driven Variational
Model...................................64
4.2.1 Conjugated Gradient Algorithms................64
4.2.2 Convergence Analysis of Conjugated Gradient Methods in One
Dimension.............................66
4.2.3 Convergence of Conjugate Gradient Methods for an Anisotropic
Total Variation and Content-Driven Variational Model....71
4.2.4 Convergence of Conjugate Gradient Methods for an Isotropic
Total Variation and Content-Driven Variational Model....74
CONTENTS vii
4.3 Other Gradient Based Algorithms....................78
4.3.1 Heavy Ball Algorithm......................78
4.3.2 Steepest Descent Algorithm with Two-point Step Sizes....80
5 Numerical Experiments 81
5.1 Numerical Results of Gradient Based Methods for a Content-Driven
Minimization Problem..........................84
5.2 Numerical Results of Representative Content-Driven Minimization Prob-
lems....................................92
5.3 Numerical Comparisons of Minimization Problems Based on ϕ from
the General Class C and from Content-Driven Class S (Gaussian Noise
Only)...................................98
5.4 Numerical Comparisons of Minimization Problems Based on ϕ from
the General Class C and from Content-Driven Class S (Combinations
of Gaussian and Impulsive Noises)....................116
6 Conclusions and Future Work 122
Acknowledgments
I came to United States six years ago and continued my mathematical study in the
Department of Mathematical at Syracuse University.Through these years,I own my
gratitude to all those people who helped me realize my goal.
I would like to express my sincere appreciation to my advisor,Professor Yuesheng
Xu,for everything he has done to help me during my graduate study.Without the
insightful discussions,unfailing supports,and warmhearted cares,I could never finish
this dissertation.I admire his enthusiasm in mathematics,which has deeply impact
on me.
I am deeply grateful to my co-advisor,Professor Lixin Shen.He brought me into
the great field of image processing.His excellent guidance,endless patience,con-
siderable encouragement,and understanding have helped me grow up in this field.I
appreciate himfor inspiring me to work on image precessing and sharing his invaluable
experience with me.
I thank Professor Andrzej Krol of State University of New York Upstate Medical
University.He is the first person who introduces me to the field of biomedical image
viii
ACKNOWLEDGMENTS ix
processing.He is very generous to share with me his research experience in biomedical
image science,which is crucial to my future career.It is enjoyable to work in his
laboratory.
My appreciation also goes to Professors Uday Banerjee,Dan Coman,Peng Gao,
Eugene Poletsky,and Andrew Vogel.I thank them for serving on my defense com-
mittee and spending their time on it.I also thank my friends,Ms.Yan Chan,Dr.
Weixing Cai,Mr.Feishe Chen,Dr.Ying Jiang,Ms.Weilin Li,Dr.Dong Mao,Dr.
Wei Ning,Dr.Hongwei Ye,Dr.Guohui Song,Dr.Levon Vogelsang,Dr.Junfeng
Wu,Dr.Bo Yu,Dr.Haizhang Zhang,and Mr.Liang Zhao.
Thanks go to the staff of Department of Mathematics,Ms.Madaline Argiro,Ms.
Christine Gilmore,Ms.Beckie Moon,Ms.Patricia O’Malley,Ms.Carolyn Sabloski,
and Mr.Bill Vogel,for their continuously and kindly help during my stay in Syracuse.
Special thanks to my parents and my elder brother for their endless support and
love.Finally,I own my gratitude to my husband,Yabing Mai,for everything he did
for me.Because of him,my life is full of merry.
x
NOTATIONS xi
Notations
R:the field of real numbers
N:the set of all positive integers
R
+
:the set of all nonnegative numbers
R
d
:the space of all d-tuples of real numbers
R
N×N
:the set of all N by Nmatrices
Ω:a bouded convex doman in R
d
with a Lipschitz continous boundary ∂Ω
¯
Ω:Ω∪∂Ω
|Ω|:the Legesque measure of Ω
∥ · ∥
p
:=
(∫

| · |
p
dx
)
1/p
for 1 ≤ p < ∞
∥ · ∥

:= inf{C ≥ 0 | | · | ≤ C almost everywhre in Ω}
L
p
(Ω):the set of all functions f satisfying ∥f∥
p
< ∞for some p ∈ [1,∞]
C
1
0
(Ω):the set of all continuously differentiable functions with compact supports in Ω
C
m
(
¯
Ω):the set of all u ∈ C
m
(Ω) satisfying that ∇
α
1
,...,α
d
u is uniformly continuous on
bounded subsets of Ω for all |α
1
| +...+|α
d
| ≤ m
BV (Ω):the subspace of functions with bounded L
1
norm and total variation
Chapter 1
Introduction
1.1 Image Restoration and Regularized Methods
Images are usually corrupted during acquisition,transmission,and recording
processes.Classically,deterioration of images is the result of two phenomena,blurring
and noise degradations.The first one is deterministic and is due to possible defects of
the imaging system,for instance,a blur caused by an incorrect lens adjustment or by
motion.Suppose that we describe an object image by a function u and an observed
blurry image by a function z.A blurring degradation model can be described as
Au = z,(1.1)
where A is a known linear observation operator.The second phenomenon is random
noise,which arises during transmission.The random noise is usually modeled as a
probabilistic distribution,and it contaminates images through adding or multiplying
1
CHAPTER 1.INTRODUCTION 2
it onto original images.The multiplicative noise occurs in certain imaging systems,
such as laser imaging,microscope imaging,and synthetic aperture radar imaging.
The approaches of multiplicative denoising are proposed in [5,29,37,41].In this
thesis,we mainly consider additive noise.Thus,the blurring and noisy model is
formulated by
Au +ϵ = z,(1.2)
where ϵ is random noise.
Image restoration refers to the problem of recovering an image u from its linearly-
degraded version (1.2).It is one of the earliest inverse problems in imaging,and has
wide applications ranging fromastronomy to medical imaging.The main challenge in
image restoration is to faithfully recover detailed image structures (i.e,edges in the
case of natural images) from a noisy observation with a minimal risk of amplifying
the noise.Image restoration has been intensively studied by researchers from areas of
mathematics,computer science,and electrical engineering in the past three decades.
Many image restoration methods,such as linear filtering based,e.g.,least square
filter [3],iteration based,e.g.,Landweber iteration [42],and variation or minimization
based,e.g.,the Rudin-Osher-Fatemi algorithm [38,39],were proposed.Among these
methods,regularization is a commonly used technique due to the nature of the ill-
posedness of inverse problem (1.2).Regularization aims at incorporating knowledge
of either the original image or the noise into solution algorithms.Some background
on regularization can be found in an excellent review article [9] or the book [18] about
CHAPTER 1.INTRODUCTION 3
iterative image deblurring.
We study the problem of image restoration within the following variational frame-
work:
inf
u
{E(u,z) +λR(u)},(1.3)
where E provides a data-faithful estimate,Rdenotes a regularization functional mea-
suring the roughness of u,and λ is a regularization parameter balancing the impor-
tance between two terms E and R.Specification of a particular method in the general
variational formulation (1.3) entails making choices of E,R,and λ.Selections of E,
R,and λ are problem-dependent and are usually tangled together.In the following,
we will briefly and separately review various choices of E and R appeared in the
existing literatures.
1.2 Choices of Regularization Functional
Regularization functional R primarily determines the performance of variational
image restoration model (1.3).Many regularization functionals can be defined in
terms of a potential function ψ in the following way
R(u):=


ψ(|∇u|) dx,
where Ω ⊂ R
d
is the image domain.A simple choice of the potential function is
ψ(x) = x
2
,which leads to the well-known Tikhonov regularization [44].With this
regularization functional,unpleasant blurring effects appear in the vicinity of edges
CHAPTER 1.INTRODUCTION 4
of the recovered image from (1.3).A popular choice of the potential function is
ψ(x) = |x|,which yields the so-called total variation (TV) regularization [38,39].
Edge locations of the recovered images obtained from the TV regularization tend to
be preserved.Total variation has received a great amount of attention in the image
processing community.The theoretical analysis,numerical implementations,and a
variety of applications of TV-based image restoration have been extensively studied
(see,for example,[10,11,13,16,48]).Other examples of the potential function,
which can yield to edge-preserving regularization,are the Lorentzian potential ψ(x) =
ηx
2
/(1 + ηx
2
),the Gaussian potential ψ(x) = 1 − exp(−ηx
2
),the Huber potential
ψ(x) = x
2
if |x| ≤ η,ψ(x) = η(η + 2|x − η|) if |x| > η [30],and the Charbonnier
potential ψ(x) = log(1 +x
2
) [14].
Recently,a sparsity constraint of the underlying image serving as a regularizer was
proved to be efficient for image restoration [16].It is based on the assumption that
the ideal image has a sparse representation with respect to a given system {w
γ
}
γ∈Γ
,
where Γ is an index set.The system could be wavelets,framelets,and curvelets.
Under this circumstance,regularization functional R is formulated as follows:
R(u):=

γ∈Γ
λ
γ
|⟨u,w
γ
⟩|
p
,
where ⟨u,w
γ
⟩ are coefficients of u with respect to the system {w
γ
}
γ∈Γ

γ
are non-
negative parameters,and p ∈ [1,∞).In particular,for p = 1,R in (1.2) becomes the

1
regularization functional,which is a current hot topic in compressive sensing and
image processing.
CHAPTER 1.INTRODUCTION 5
1.3 Choices of Data-Fidelity Term
For model (1.2),the frequently used data-fidelity term is
E(u,z):=
1
2
∥Au −z∥
2
2
,
which is usually called the L
2
-norm fidelity term.This term is appropriate when ϵ
is normally distributed.However,when the observed data z contains outliers,such
as salt-pepper noise,the variational image restoration model (1.3) with the L
2
-norm
fidelity term is very sensitive to such outliers (see,for example,[30,32,33]).
Another popular choice is the L
1
-norm fidelity term,which is defined by
E(u,z):= ∥Au −z∥
1
.
This data-fidelity term has shown its robustness on suppressing outliers [12,31,35].
However,the lack of differentiability for the L
1
data-fidelity term at the origin cer-
tainly increases difficulty in solving the corresponding minimization problem (1.3).
A direct generalization of the above L
1
-norm or L
2
-norm fidelity term is
E(u,z):= ∥ϕ(Au −z)∥
1
,(1.4)
where ϕ is a continuous even function on R with ϕ(0) = 0.In particular,ϕ(·) =
1
p
| · |
p
with p = 1 or p = 2 reduces to the L
1
-norm or L
2
-norm fidelity term,respectively.
Obviously,it is natural to use ϕ(·) =
1
p
| · |
p
in (1.4) with values of p other than 1 or
2.Actually,the data-fidelity term defined by (1.4) with ϕ(·) =
1
p
| · |
p
for 1 ≤ p < ∞
and called the L
p
-norm fidelity has been extensively studied in other contexts such
CHAPTER 1.INTRODUCTION 6
as regression analysis and statistical classification.It was suggested in [21] to use the
L
p
-norm fidelity with 1 ≤ p ≤ 2 for contaminated data.Selecting values of p of the
L
p
-norm fidelities depends critically on the distribution of residuals Au −z.Several
empirical schemes for determining p were proposed in [23,24,26].With such schemes,
numerical experiments as reported in [17,33] showed that the L
1
and L
p
(1 < p < 2)
fidelities performed equivalently for larger outliers in z;the L
p
(1 < p < 2) fidelity
worked better than the L
1
fidelity for moderate outliers;some L
p
(1 < p < 2) fidelity
and the L
2
fidelity were essentially the same for small outliers,and both were better
than the L
1
fidelity.
Comparing to the L
1
-norm or L
2
-norm fidelity,the L
p
-norm (1 < p < 2) fidelity
has shown less impact on the community of image processing.It might be due to the
facts that no single p value can faithfully describe the distribution of residuals Au−z
and varying p values are required in the different regions of the underlying image.
Recently,an effort was made to design a function ϕ in (1.4),which can mimic the
behaviors of L
1
and L
2
norms and the transition from L
1
norm to L
2
norm and vise
verse [34].It provides a different perspective to formulate data-faithful estimation.
1.4 Content-Driven Fidelity Terms
In this thesis,we systemically study the selection of data-fidelity terms.The
fidelity term E(u,z) in (1.4) is uniquely determined by the function ϕ.We define
CHAPTER 1.INTRODUCTION 7
a general class of functions ϕ,which characterizes a general class of fidelities.This
class of fidelities includes most fidelities in the existing literatures,such as L
p
-norm
fidelities (1 < p < 2) and the Huber fidelity.
Motivated by [34],we find a new subclass of functions ϕ in the above mentioned
general function class,called the content-driven function class.The L
p
functions
(1 ≤ p ≤ 2) and the Huber function are not in the content-driven class.The derivative
of a content-driven function ϕ behaves similarly to the L
1
function as the input of ϕ
is large enough,and similarly to L
2
function as the input of ϕ is small.The derivative
of a content-driven function ϕ shows L
p
function behavior (1 < p < 2) as the input of
ϕ is moderate.The behavior of a content-driven function ϕ smoothly transits from
that of the L
2
function to that of the L
1
function.
The fidelity introduced by the content-driven function ϕ is called the content-
driven fidelity.It inherits most strengths of L
p
-norm fidelities (1 ≤ p ≤ 2) for image
restoration.The content-driven fidelity shows the behavior of the L
2
-norm fidelity as
the local image content is normally distributed or as the noise level of ϵ is low,and
the behavior of the L
1
-norm fidelity as the local image content contains outliers or
as the noise level is high.The content-driven fidelity can provide a spatially varying
measurement of the reconstructed image to the original image.
Theoretical studies based on the general class of fidelities are presented in this
work.We show the well-posedness of regularized methods,composed of a fidelity
termE chosen fromthe general class of fidelities and the total variation regularization.
CHAPTER 1.INTRODUCTION 8
Convergence of minimizers of regularized estimations based on the general class of
fidelities are established as errors in the data and the blurring kernel diminish to
zero.Since content-driven fidelities are contained in the general class of fidelity,
theoretical results held for the general fidelity estimations are also held for content-
driven estimations.
Four gradient based algorithms,such as the steepest descent algorithm (SD),
heavy ball algorithm (HB),steepest descent algorithm with two point step size (SD-
TPSS),and conjugate gradient algorithm (CG),are applied to solve content-driven
estimations.We show convergence of SD and CG algorithms for solving discrete
content-driven estimations in the one dimensional case and the two dimensional case.
Numerical comparisons to assess performances of L
p
-norm fidelities,the Huber
fidelity,and content-driven fidelities are performed.The superiorities of the content-
driven estimation over L
p
-norm estimations and other existing estimations are shown
in our numerical studies.
This thesis consists of seven chapters:In Chapter 2,we define the general class of
fidelities,and study their properties as preliminary materials for theoretical analysis
in Chapter 3.We specify content-driven fidelities,which is motivated by seeking an
appropriate measurement of the reconstructed images to the original images other
than the existing fidelity measurements.Special examples of content fidelities are
presented.In Chapter 3,we prove the well-posedness of total variation regularized
estimations with fidelities fromthe general class of fidelities.As errors in the observed
CHAPTER 1.INTRODUCTION 9
image and the blurring kernel tend to zero,convergence of minimizers of corresponding
estimations are proved.We present the Euler-Lagrange equations of estimations
based on the general class of fidelities.We also provide a sufficient and necessary
condition that ensures the existence of the minimizer of the regularized minimization
problem with a fidelity from the general fidelity class.In Chapter 4,we apply four
gradient based methods to solve content-driven estimations.Convergence of SD and
CG methods based on the discrete content-driven estimations are presented in both
cases of one dimension and two dimension.In Chapter 5,numerical studies that
compare performances of L
p
-norm estimations (1 ≤ p ≤ 2),the Huber estimation,
and content-driven estimations are presented.The last chapter is about conclusions
and future research plans.
Chapter 2
Content-Driven Fidelity Terms
The fidelity plays a key role in a regularized method.It provides a measurement of
the closeness between the reconstruction image and the original image.The essence
of choosing a fidelity is to find the most accurate measurement among all possible
selections.In this thesis,we concentrate on the issue of the fidelity selection by
studying existing fidelities and defining more accurate fidelity measurements,which
can adapt to local image contents.
In the previous chapter,we concluded a general form of data-fidelity terms as
(1.4).The fidelity provided by (1.4) is uniquely determined by the definition of ϕ.
We can not say that (1.4) includes all possible selections of fidelities,but it includes
most of well-known fidelities by varying the choice of ϕ.In this section,we introduce
a class of functions ϕ which can measure the closeness between the estimation Au and
the observed data z in (1.4).We first give a general class of functions which includes
10
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS 11
functions
1
p
| · |
p
(it will be referred to as L
p
functions) with 1 ≤ p ≤ 2 as well as the
Huber function [30].We then narrow this class down to a special class in which each
function will be able to serve our purpose in a way that the corresponding estimator
(1.4) can locally adapt to noise and outliers in the observed data.
2.1 The General Class of Fidelity Terms
The class of functions we will introduce below is denoted by
C:= {ϕ | ϕ satisfies Condition H}.(2.1)
We say that a function ϕ satisfies Condition H if ϕ meets the following hypotheses

H1:ϕ:R →R
+
is a convex,even,non-constant function with ϕ(0) = 0.

H2:There exists a positive number r ∈ [0,1] such that for any x and y in R,
one has
|ϕ(x) −ϕ(y)| ≤ c
ϕ
|ξ|
r
|x −y|,(2.2)
where c
ϕ
is a constant depending only on ϕ and ξ is a number between x and y.
Here R
+
denotes all nonnegative real numbers.The next two results show that L
p
functions and the Huber function are in the set C.
Proposition 2.1.
For 1 ≤ p ≤ 2,the L
p
function ϕ(x) =
1
p
|x|
p
is in C.
Proof.
It is easy to check that L
p
function satisfies H1.Now we check if the L
p
function satisfies (2.2).If p = 1,then (2.2) holds with c
ϕ
= 1 and r = 0.If 1 < p ≤ 2,
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS 12
by the mean value theorem,we have that
|ϕ(x) −ϕ(y)| ≤ p|ξ|
p−1
|x −y|,
where ξ is a number between x and y.Therefore,the L
p
function is in the class C.
Recall that the Huber function [30] is defined by
H(x):=







1
2
x
2
,0 ≤ |x| ≤ η,
η|x| −
η
2
2
,|x| ≥ η,
(2.3)
where η is a positive constant.The Huber function is a combination of L
1
and L
2
functions.Here η is a parameter controlling the location of the connection point
between L
1
and L
2
functions.Similar to the L
p
function,we also have
Proposition 2.2.
The Huber function defined by (2.3) is in C.
Proof.
It is well-known that the Huber function satisfies hypothesis H1.A direct
computation shows |H

(x)| ≤ η for all x ∈ R.Hence,the Huber function satisfies
hypothesis H2 with r = 0 and c
ϕ
= η.
From the above examples,we know that the general function class C includes
two well-known fidelities:the L
p
fidelity and the Huber fidelity.We further explore
the properties of the function class C.Properties of functions in C are presented in
the following four lemmas.Prior to stating the lemmas,we recall that ϕ is called a
(monotonically) increasing function if ϕ(a) ≤ ϕ(b) (ϕ(a) < ϕ(b)) for any a < b.
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS 13
Lemma 2.3.
If ϕ is a function satisfying the hypothesis H1,then
lim
x→±∞
ϕ(x) = ∞.(2.4)
Proof.
We first show that there exists a positive c > 0,such that ϕ monotonically
increases on [c,∞) and monotonically decreases on (−∞,c].Since ϕ is an even
function,we only need to show that ϕ monotonically increases on [c,∞).We assume
that it is not true,i.e.,for any c > 0 there exists two positive number a and b satisfying
c ≤ a < b < ∞,such that ϕ(a) ≥ ϕ(b).Assuming that ϕ(a) > ϕ(b),we have ϕ(a) > 0
due to the positivity of ϕ on R.By ϕ(0) = 0,ϕ(a) > ϕ(b) ≥ 0 contradicts to the
convexity of ϕ.Therefore,ϕ(a) = ϕ(b).By convexity of ϕ with ϕ(0) = 0,we have
that ϕ(x) = 0 on the interval [0,b].Letting c approach +∞,we have that ϕ(x) = 0
on R.It contradicts that ϕ(x) is a non-constant function.
Now,we verify (2.4).Since ϕ is an even function,we only showthat lim
x→∞
ϕ(x) =
∞.If this is not true,then there exist a positive number M and an nonnegative
increasing sequence {x
n
}

n=1
with lim
x→∞
x
n
= ∞ such that the sequence {ϕ(x
n
)}
is bounded by M.The monotonicity of ϕ implies that the limit of ϕ(x
n
) exists and
equals to a number,say K,when n runs to infinity.That is lim
x→∞
ϕ(x
n
) = K.By
using the monotonicity of ϕ again,it yields that
lim
x→∞
ϕ(x) = K.(2.5)
Let a and b be two numbers with properties c ≤ a < b and ϕ(a) < ϕ(b).For any
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS 14
x > b,the convexity of ϕ leads to
ϕ(b) ≤ ϕ(a) +
ϕ(x) −ϕ(a)
x −a
(b −a)
Letting x approach to infinity in the above inequality and using (2.5),we have ϕ(b) ≤
ϕ(a).This contradicts to ϕ(a) < ϕ(b).Hence,(2.4) is true.
Lemma 2.4.
If ϕ ∈ C,then,for all x ∈ R,
ϕ(x) ≤ c
ϕ
max{|x|,1}|x|,(2.6)
where c
ϕ
is the constant in (2.2).
Proof.
Note that ϕ(0) = 0.By (2.2),one has
ϕ(x) = ϕ(x) −ϕ(0) ≤ c
ϕ
|ξ|
r
|x|,
where 0 ≤ r ≤ 1 and 0 ≤ ξ ≤ |x|.Since |ξ|
r
≤ max{|x|,1},we have (2.6).
Let Ω ⊂ R
d
be a bounded convex domain with a Lipschitz continuous boundary
∂Ω.Let denote |Ω| the Lebesque measure of Ω.A function f is in L
p
(Ω) (1 ≤ p ≤ ∞)
if ∥f∥
p
is finite.Here,∥f∥
p
:=
(∫

|f(x)|
p
dx
)
1/p
< ∞ for 1 ≤ p < ∞ and ∥f∥

:=
inf{C ≥ 0 | |f| ≤ C almost everywhre in Ω} < ∞.
Lemma 2.5.
If ϕ ∈ C,then for any function f in L
2
(Ω),ϕ ◦ f ∈ L
1
(Ω).
Proof.
By Lemma 2.4,we have
∥ϕ ◦ f∥
1
=


ϕ(f(x)) dx ≤ c
ϕ


max{|f(x)|,1}|f(x)| dx.
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS 15
By the H¨older inequality,
∥ϕ ◦ f∥
1

(∫

|max{|f(x)|,1}|
2
dx
)
1/2
(∫

|f(x)|
2
dx
)
1/2
≤ c
ϕ
(∥f∥
2
+|Ω|
1/2
∥f∥
2
) < ∞.
Lemma 2.6.
If ϕ ∈ C,then for any two functions f and g in L
2
(Ω),
∥ϕ ◦ f −ϕ ◦ g∥
1
≤ c
ϕ
(∥f∥
2
+∥g∥
2
+|Ω|
1/2
)∥f −g∥
2
.(2.7)
Proof.
By hypothesis H2,we have that
∥ϕ ◦ f −ϕ ◦ g∥
1
=


|ϕ(f(x)) −ϕ(g(x))| dx ≤ c
ϕ


|ξ(f(x),g(x))|
r
|f(x) −g(x)| dx
(2.8)
where ξ(f(x),g(x)) is a number between f(x) and g(x).
Let Ω
0
:= {x ∈ Ω | |f(x)| ≤ 1 and |g(x)| ≤ 1} and Ω
1
:= Ω\Ω
0
.Then,we have
that |ξ(f(x),g(x))|
r
≤ 1 for x ∈ Ω
0
and |ξ(f(x),g(x))|
r
≤ |f(x)| +|g(x)| for x ∈ Ω
1
.
From (2.8) and the Cauchy-Schwartz inequality,we obtain that
∥ϕ ◦ f −ϕ ◦ g∥
1
≤ c
ϕ
(


0
|f(x) −g(x)| dx +


1
(|f(x)| +|g(x)|)|f(x) −g(x)| dx
)
≤ c
ϕ
(
|Ω|
1/2
∥f −g∥
2
+∥|f| +|g|∥
2
∥f −g∥
2
)
.
The above lemma implies that the closeness of two functions f and g in the L
2
sense leads to the closeness of ϕ ◦ f and ϕ ◦ g in the L
1
sense.
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS 16
2.2 The Class of Content-Driven Fidelity Terms
The solution of (1.2) can be approached by solving the variational problem (1.3),
whose fidelity E(u,z) has the form of (1.4) with ϕ ∈ C.As discussed in the previous
chapter,L
p
functions (1 ≤ p ≤ 2) are commonly used in defining the fidelity term,
especially with p = 1,2.The regularization problems (1.3) with L
1
-norm and L
2
-
norm fidelities are well studied.We review their strengths and limitations presented
in the literature as follows.
(a) Noisy and Blurred ‘Lena’ (b) L
2
+ TV
Figure 2.1:Restored image from a blurred and noisy`Lena'by using L
2
-norm delity and
TV regularization
The L
2
norm can well keep the fidelity of an image,see [2,6,10,45,46].It has
been shown to have the superiority over the L
1
norm if the noisy data is normally
distributed and the noise level of data is low,see [32,33].However,the superiority of
the L
2
norm will lose if noise level of the normally distributed data is high.Another
limitation of the L
2
-norm fidelity is that it is sensitive to outliers.If large values of ϵ
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS 17
appear in the image domain,such as salt and pepper noise,the reconstructed image
may be far away from the original image.Figure 2.1 demonstrates the restored result
of a blurry image with salt pepper noise.The result is produced by the estimation
(1.3),composed of the L
2
-normfidelity and the total variation regularization.We can
see the obvious artifacts around the presence of large errors.Therefore,if z contains
outliers,the quality of the reconstructed image based on the L
2
estimator is poor.
The L
1
-normfidelity normis addressed in [12,31,35],It has significant superiority
over the L
2
-norm fidelity in presence of outliers (see,for example,Figure 2.2).The
L
1
-norm estimation more depends on the geometric shape of image features and less
depends on the intensity contrast of images.A related discussion was given in [12].
The L
1
-normfidelity is not optimal to process blurred images with the Gaussian noise
at small noise levels.Moreover,the L
1
norm is not differentiable at origin.It may
introduce some computation difficulties.
The L
p
-norm fidelity for some p ∈ (1,2) could be a better fidelity candidate
other than the L
1
-norm or L
2
-norm fidelity for some image contents.Some L
p
-norm
fidelities (1 < p < 2) show the superiorities in the presence of moderate outliers
or small errors (see numerical results in Chapter 5).However,we have to face a
challenging issue:the selection of the optimal p.This issue was addressed in [24,
23,26] from statistical perspective.None of the existing selection schemes is trivial.
Through our numerical observations we find out that for each p between 1 and 2 the
behavior of the L
p
-norm fidelity estimation is either similar to the L
1
-norm fidelity
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS 18
(a) Noisy and Blurred ‘Lena’ (b) L
1
+ TV
Figure 2.2:Recovered results from a blurred and noisy`Lena'by using L
1
delity and TV
regularization
or the L
2
-norm fidelity.There is no single p value which keeps both strengths of the
L
1
-norm fidelity and L
2
-norm fidelity.
From the above discussion,we notice that L
p
-norm estimators for 1 ≤ p ≤ 2 have
their unique strengths for processing different noise-contained images.We intend to
find a small class of fidelities,which vary their behaviors to adapt to the image content
and inherit all strengths of L
p
fidelities,especially for p = 1,2.We plot the derivatives
of L
1
and L
2
functions in Figure 2.3.We assume that values on the horizontal axis
denote the pointwise errors of Au and z.The absolute value of the derivative of the
L
1
function is a constant on R/{0}.It means that the L
1
function gives an equal
weighted measurement of errors in observed images.Therefore,the L
1
-norm fidelity
can suppress outliers.For the L
2
function,its derivative grows linearly with respect
to x.It indicates that the L
2
-normfidelity gives a linear measurement of errors,which
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS
19
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2


L
1
norm
L
2
norm
Content−Driven
φ

Figure 2.3:Plots of the derivatives of the desired
ϕ
(red solid line),
L
1
function
(dotted line) and
L
2
function (dashed line)
is proportional to the errors.So the
L
2
-norm fidelity is very sensitive to outliers in
z
.The derivative of the desired function
ϕ
should work similarly to the derivative of
the
L
1
function as the value of
x
is large,and behaves similarly to the derivative of
the
L
2
function as
x
is small.Therefore,the graph of the derivative of the desired
ϕ
should have similar shape as the red curve in Figure 2.3.
In summary,we will define a special subset of
C
by requiring two additional
hypotheses:

H3
:
ϕ
is twice continuously differentiable with bounded first and second deriva-
tives;

H4
:
ϕ
is strictly convex.
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS 20
This special class of functions is defined as follows
S:= {ϕ | ϕ satisfies the hypotheses H1,H3,and H4}.(2.9)
By the mean value theorem,the hypothesis H3 implies the hypothesis H2.Therefore,
S is a subset of C.The fidelity term defined as (1.4) with ϕ ∈ S is called the content-
driven fidelity.
Lemma 2.7.
If ϕ ∈ S,then ϕ

(0) = 0.
Proof.
Since ϕ is a nonnegative strictly convex function with ϕ(0) = 0,we know
that ϕ increases on [0,∞) and decreases on (−∞,0].Hence,
lim
x→0+
ϕ(x) −ϕ(0)
x
≥ 0 and lim
x→0−
ϕ(x) −ϕ(0)
x
≤ 0.
The differentiability of ϕ implies that ϕ

(0) = 0.
We can directly check that the L
p
functions,1 ≤ p ≤ 2,and the Huber function
are not in S.Several examples of functions in S are presented in the following lemmas.
Lemma 2.8.
Let η be a positive number.Define
ϕ(x):=
x
2
2(1 +|x|/η)
x ∈ R.(2.10)
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS 21
Then,ϕ ∈ S has the properties
lim
x→0
ϕ(x)
1
2
x
2
= 1,
lim
x→∞
ϕ(x)
|x|
=
1
2
η,


(x)| ≤
1
2
η,x ∈ R,
ϕ
′′
(x) ≤ 1,x ∈ R,
ϕ
′′
(x) > 0,x ∈ R.
Proof.
It is easy to check that ϕ is a nonnegative even function having
ϕ

(x) =

2
x +sign(x)ηx
2
2(η +|x|)
2
,x ∈ R.
We observe that ϕ(0) = ϕ

(0) = 0 and |ϕ

(x)| < η/2.The second derivative of ϕ has
the form of
ϕ
′′
(x) =
η
3
(η +|x|)
3
,x ∈ R.
It is continuous and bounded by 1.Since the second derivatives of ϕ is positive,
ϕ is strictly convex.Hence,ϕ ∈ S and the last three properties hold.By simple
calculations,we can easily obtain the first and second properties.
Lemma 2.9.
Let η be a positive number.Define
ϕ(x):=
1
2
η
2
log cosh
(
x
η
)
,x ∈ R.(2.11)
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS 22
Then,ϕ ∈ S has the following properties
lim
x→0
ϕ(x)
1
2
x
2
= 1,
lim
x→∞
ϕ(x)
|x|
=
1
2
η,


(x)| ≤
1
2
η,x ∈ R,
ϕ
′′
(x) ≤
1
2
,x ∈ R,
ϕ
′′
(x) > 0,x ∈ R.
Proof.
It is easy to check that ϕ is a nonnegative even function.The first derivative
of ϕ has the form
ϕ

(x) =
1
2
η tanh(
x
η
),x ∈ R.
We also notice that ϕ(0) = ϕ

(0) = 0.The second derivative of ϕ is given by
ϕ
′′
(x) =
1
2 cosh
2
(
x
η
)
,x ∈ R,
which is continuous and bounded by
1
2
.The first derivative of ϕ is bounded by η/2.
Since the second derivatives of ϕ is positive,ϕ is strictly convex.Hence,ϕ ∈ S and
the last three properties hold.By simple calculations,we can easily obtain the first
and second properties.
Lemma 2.10.
Let η be a positive number.Define
ϕ(x):= η

η
2
+x
2
−η
2
,x ∈ R.(2.12)
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS 23
Then,ϕ ∈ S has the following properties
lim
x→0
ϕ(x)
1
2
x
2
= 1,
lim
x→∞
ϕ(x)
|x|
= η,


(x)| ≤ η,x ∈ R,
ϕ
′′
(x) ≤ 1,x ∈ R,
ϕ
′′
(x) > 0,x ∈ R.
Proof.
It is easy to check that ϕ is a nonnegative even function.The first derivative
ϕ

(x) =
ηx

η
2
+x
2
,x ∈ R,
with ϕ(0) = ϕ

(0) = 0.We also notice that |ϕ

(x)| < η.The second derivative of ϕ
has the form
ϕ
′′
(x) =
η
3

2
+x
2
)
3/2
,x ∈ R,
which is bounced by 1 and continuous.Since the second derivatives of ϕ is positive,
ϕ is strictly convex.Hence,ϕ ∈ S and the last three properties hold.By simple
calculations,we can easily obtain the first and second properties.
Lemma 2.11.
Let η be a positive number.Define
ϕ(x):= ηxarctan
(
x
η
)

1
2
η
2
log
(
1 +
(
x
η
)
2
)
,x ∈ R.(2.13)
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS 24
Then,ϕ ∈ S has the following properties
lim
x→0
ϕ(x)
1
2
x
2
= 1,
lim
x→∞
ϕ(x)
|x|
=
π
2
η,


(x)| ≤
π
2
η,x ∈ R,
ϕ
′′
(x) ≤ 1,x ∈ R,
ϕ
′′
(x) > 0,x ∈ R.
Proof.
It is easy to check that ϕ is a nonnegative even function.The first derivative
of ϕ has the form of
ϕ

(x) = η arctan(
x
η
),x ∈ R,
with ϕ(0) = ϕ

(0) = 0.We also notice that |ϕ

(x)| <
π
2
η.The second derivative of ϕ
ϕ
′′
(x) =
1
π(1 +(
x
η
)
2
)
,x ∈ R,
is bounded by 1 and continuous.Since the second derivatives of ϕ is positive,ϕ
is strictly convex.Hence,ϕ ∈ S and the last three properties hold.By simple
calculations,we can easily obtain the first and second properties.
Let η = 1 and we plot the derivatives of the four examples of ϕ defined as in (2.10)-
(2.13) in Figure 2.4.The derivative of each ϕ behaves similarly to the L
1
function as
x is large enough,and similarly to the L
2
function as x is sufficiently small.Figure
2.4 illustrates that the content-driven fidelity carries both strengths of the L
1
-norm
fidelity and the L
2
-norm fidelity.
CHAPTER 2.CONTENT-DRIVEN FIDELITY TERMS
25
−1.5
−1
−0.5
0
0.5
1
1.5
2

Example 1
Example 2
Example 3
Example 4
L1 Norm
L2 Norm
Figure 2.4:Plots of the derivatives of specific examples of
ϕ
with
η
= 1 (Example 1:
ϕ
is defined as in (2.10);Example 2:
ϕ
is defined as in (2.11);Example 3:
ϕ
is defined
as in (2.12);Example 4:
ϕ
is defined as in (2.13).)
Chapter 3
Variational Models for Image
Restoration
We recall that the degradation model is given by
z = Au +ϵ,(3.1)
where z is the observed blurry and noisy image,u is the true image which we intend
to recover,and ϵ is the additive random noise.Let Ω ⊂ R
d
(d = 1,2) be a bounded
convex domain with a Lipschitz continuous boundary ∂Ω.Here A denotes a known
linear compact operator from L
p
(Ω) into L
2
(Ω).Equation (3.1) is a typical ill-posed
problem.
To remedy the ill-posedness of (1.2),we propose to seek a solution in the bounded
variation space by minimizing a functional (with respect to u) as follows
F
λ
(u,z):= ∥ϕ(Au −z)∥
1
+λ · TV(u).(3.2)
26
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 27
Here TV(u) stands for the total variation of u and its definition will be given later.
3.1 Preliminary on the Bounded Variation Space
In this subsection,we review some elementary concepts and results related to the
bounded variation space BV (Ω) that are useful for the thesis.The bounded variation
space BV (Ω) is a good space to study images.
Let C
1
0
(Ω) be the collection of continuously differentiable functions with compact
supports in Ω.We define
C
1
0
(Ω)
d
:= C
1
0
(Ω) ×· · · ×C
1
0
(Ω)
|
{z
}
d folds
.
For a vector-valued function v ∈ C
1
0
(Ω)
d
,we write |v|:=

v
2
1
+· · · +v
2
d
,where v
i
is
the ith component of v for i = 1,...,d.We let
V:= {v ∈ C
1
0
(Ω)
d
:|v(x)| ≤ 1 for all x ∈ Ω}.
Note that BV (Ω) is the subspace of functions u ∈ L
1
(Ω) such that the following
quantity is finite:
TV(u):= sup
v∈V


−udiv v dx.(3.3)
The space BV (Ω) endowed with the norm ∥u∥
BV
:= ∥u∥
1
+TV(u) is a Banach space.
Note that Ω is a bounded open set.It was shown in [1] that
BV (Ω) ⊂ L
p
(Ω) ⊂ L
1
(Ω) (3.4)
for 1 ≤ p ≤ d/(d −1).
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 28
In order to overcome the discontinuity total variation at zero,a modification of
the total variation (see,for example,[1,47]) is given by
TV
β
(u):= sup
v∈V


(
−udiv v +

β(1 −|v|
2
)
)
dx (3.5)
where β is a positive number.A sequence {u
n
}
n
⊂ L
p
(Ω) converges weakly to u ∈
L
p
(Ω) for 1 ≤ p < ∞,if
lim
n→∞


u
n
vdx =


uvdx,∀v ∈ L
q
(Ω),
where
1
p
+
1
q
= 1 (q:= ∞ when p = 1).It was proved in [1] that TV
β
is weakly
lower-semicontinuous with respect to the L
p
(Ω) topology for 1 ≤ p < ∞,i.e.,if u
n
converges weakly to bu in L
p
(Ω),then
TV
β
(bu) ≤ liminf
n→∞
TV
β
(u
n
).(3.6)
Clearly,TV
0
(u) = TV(u).Further,we have
TV
β
(u) =



|∇u|
2

2
dx (3.7)
provided u ∈ C
1
(Ω).When the regularization term R(u) is chosen as TV
β
(u),the
model in (3.2) becomes
F
λ,β
(u,z):= ∥ϕ(Au −z)∥
1
+λ · TV
β
(u),β > 0,(3.8)
where ϕ is a given function in C.By Lemma 2.6 and (3.6),F
λ,β
is weakly semicontin-
uous with respect to the L
p
(Ω) topology,that is,if a sequence {u
n
}
n
weakly converges
to bu ∈ L
p
(Ω),then
F
λ,β
(bu,z) ≤ liminf
n→∞
F
λ,β
(u
n
,z).(3.9)
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 29
3.2 Well-Posedness of the General Variational Model
In this section,we consider the well-posedness of the minimization problem
inf
u∈BV(Ω)
F
λ,β
(u,z),(3.10)
where the functional F
λ,β
is defined by (3.8).For problem (3.10) to be well-posedness
in the sense of Hadamard,it should have the following properties:(i) a solution of
(3.10) exists;(ii) its solution is unique;and (iii) its solution continuously depends on
the data z and the operator A.The well-posedness of the estimation,composed of
the L
2
-norm fidelity and a modified total variation,is presented in [1].The following
proofs of well-posedness of the general variational model (3.8) are inspired by the
work in [1].The technical difficulties in the proofs are aroused by the unspecific form
of ϕ.We first show that the functional F
λ,β
(u,z) is convex with respect to u.
Lemma 3.1.
Let ϕ ∈ C.Let A be a linear operator from L
p
(Ω) to L
2
(Ω) where
1 ≤ p ≤ d/(d − 1).Then the functional F
λ,β
(·,z) is convex.Furthermore,the
functional F
λ,β
(·,z) is strictly convex if A is injective and ϕ is strictly convex.
Proof.
It is known from [1] that TV
β
is convex.By (3.8),we only need to show
that E(·,z) is convex.This directly follows from the convexity of the function ϕ.
Furthermore,the injectivity of A and the strict convexity of ϕ lead to the strict
convexity of E(·,z).Hence,the functional F
λ,β
(·,z) is strictly convex.
Before we give the proof of the existence and uniqueness of the solutions of problem
(3.10),we recall a useful concept of BV-coercivity and a decomposition of a bounded
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 30
variation function.
A functional F:L
p
(Ω) →R is said to be BV -coercive if
F(u) →+∞ whenever ∥u∥
BV
→+∞.(3.11)
An arbitrary function u ∈ BV (Ω) can be decomposed as
u = v +w (3.12)
with


vdx = 0 and w =
1
|Ω|


udx.(3.13)
A direct consequence of this decomposition (see,[1]) is that there exists a positive
number M
1
such that
∥v∥
p
≤ M
1
· TV
0
(u) and ∥u∥
BV
≤ ∥w∥
1
+(M
1
+1) · TV
0
(v).(3.14)
for any p such that 1 ≤ p ≤ d/(d −1).Let 1

be the constant function on Ω whose
function value is 1.The Jensen inequality in [40] plays an important role in the proof
of BV -coercivity.It states that
ϕ(
1
|Ω|


fdx) ≤
1
|Ω|


(ϕ ◦ f)dx.(3.15)
where f is a real function in L
1
(Ω) and ϕ is a convex function.The Jensen inequality
is a generalization of the statement that the secant line of a convex function lies above
the graph of the function.
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 31
Lemma 3.2.
Let ϕ ∈ C.Let A be a linear and bounded operator from L
p
(Ω) to L
2
(Ω)
where 1 ≤ p ≤ d/(d−1).If ∥A1


1
> 0,then the functional F
λ,β
(·,z) is BV-coercive.
Proof.
Showing that F
λ,β
(·,z) is BV-coercive is equivalent to proving that if for any
sequence {u
n
} ⊂ BV (Ω)
liminf
n→∞
F
λ,β
(u
n
,z) < +∞,
then
liminf
n→∞
∥u
n

BV
< +∞.
Suppose that {u
n
} is a sequence in BV (Ω) and
liminf
n→∞
F
λ,β
(u
n
,z) = K < +∞.
We now show liminf
n→∞
∥u
n

BV
< +∞.Since liminf
n→∞
F
λ,β
(u
n
,z) is finite,there
is a subsequence {u
n
k
} of {u
n
} such that
lim
k→∞
F
λ,β
(u
n
k
,z) = liminf
n→∞
F
λ,β
(u
n
,z) = K.
Hence the sequence {F
λ,β
(u
n
k
,z)}
k
is bounded.
To showliminf
n→∞
∥u
n

BV
< +∞,it suffices to prove that {∥u
n
k

BV
}
k
is bounded.
By (3.12)-(3.13),each function in {u
n
k
}
n
k
can be decomposed as follows
u
n
k
= v
n
k
+w
n
k
with w
n
k
=
1
|Ω|


u
n
k
dx.
By (3.14),we only need to show that both sequences {∥w
n
k

1
}
k
and {TV
0
(v
n
k
)}
k
are
bounded.Since
λTV
0
(v
n
k
) = λTV
0
(u
n
k
−w
n
k
) = λTV
0
(u
n
k
) ≤ λTV
β
(u
n
k
) ≤ F
λ,β
(u
n
k
,z),
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 32
the boundedness of the sequence F
λ,β
(u
n
k
,z) implies the boundedness of the sequence
{TV
0
(v
n
k
)}
k
.
Now we prove that {∥w
n
k

1
}
k
is bounded.Since ϕ is convex,by the Jensen
inequality (3.15) we have that
ϕ
(
1
|Ω|


|Au
n
k
−z| dx
)

1
|Ω|


ϕ(|Au
n
k
−z|) dx ≤
1
|Ω|
F
λ,β
(u
n
k
,z).
Hence,the boundedness of the sequence {ϕ(
1
|Ω|


|Au
n
k
−z| dx)}
k
is derived by the
boundedness of the sequence F
λ,β
(u
n
k
,z).By Lemma 2.3,the boundedness of the
sequence {∥Au
n
k
−z∥
1
}
k
is achieved.
By the decomposition of u
n
k
= w
n
k
+v
n
k
,we have that
∥Au
n
k
−z∥
1
= ∥(Av
n
k
−z) +Aw
n
k

1
≥ ∥Aw
n
k

1
−∥Av
n
k
−z∥
1
.(3.16)
By the H¨older inequality,we observe that
∥Av
n
k
−z∥
1
≤ ∥Av
n
k

1
+∥z∥
1
≤ |Ω|
1/2
∥Av
n
k

2
+∥z∥
1
.(3.17)
By (3.14),we conclude that
∥Av
n
k
−z∥
1
≤ |Ω|
1/2
∥A∥∥v
n
k

p
+∥z∥
1
≤ |Ω|
1/2
M
1
∥A∥TV
0
(v
n
k
) +∥z∥
1
.(3.18)
From (3.16) and (3.18),together with the fact ∥Aw
n
k

1
= |Ω|
−1
∥A1


1
∥w
n
k

1
,we
obtain that
∥Au
n
k
−z∥
1
≥ |Ω|
−1
∥A1


1
∥w
n
k

1
−(|Ω|
1/2
M
1
∥A∥TV
0
(v
n
k
) +∥z∥
1
).(3.19)
By∥A1


1
> 0,the boundedness of {∥w
n
k

1
}
k
is achieved by the boundednesses of
{∥Au
n
k
−z∥
1
}
k
and {TV
0
(v
n
k
)}
k
.
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 33
Theorem 3.3 (Existence and Uniqueness of Minimizers).
Let ϕ ∈ C.Let A be a
linear and bounded operator from L
p
(Ω) to L
2
(Ω) where 1 ≤ p ≤ d/(d − 1).If
∥A1


1
> 0,then the functional F
λ,β
(·,z) has a minimizer over BV (Ω).The unique-
ness of the minimizer hold if A is injective and ϕ is strictly convex.
Proof.
Let {u
n
} be a sequence in BV (Ω) such that
lim
n→∞
F
λ,β
(u
n
,z) = inf
u∈BV (Ω)
F
λ,β
(u,z).(3.20)
By Lemma 3.2,the sequence {u
n
}
n
is bounded in BV-norm.It was pointed in [1]
that there exists a subsequence {u
n
k
}
k
which converges to some bu ∈ L
p
(Ω) if 1 ≤ p <
d/(d −1),or converges weakly if p = d/(d −1) and d = 2.By (3.9) we have that
F
λ,β
(bu,z) ≤ liminf
k→∞
F
λ,β
(u
n
k
,z).
This inequality and (3.20) lead to
F
λ,β
(bu,z) ≤ inf
u∈BV (Ω)
F
λ,β
(u,z).
Therefore,bu is a minimizer of the functional F
λ,β
(·,z) over BV (Ω).Furthermore,if
A is injective and ϕ is strictly convex,by Lemma 3.1,the F
λ,β
(·,z) is strictly convex.
Hence,the minimizer is unique.
Next,we study the stability of the minimization problem(3.10) whose exact mean-
ing is defined below.Consider a sequence of perturbed problems of (1.2)
z
n
= A
n
u,(3.21)
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 34
where A
n
,1 ≤ n < ∞,are linear and bounded operators from L
p
(Ω) to L
2
(Ω) and
are injective.For each problem,we consider the minimization problem
inf
u∈BV (Ω)
F
λ,β,n
(u,z
n
),(3.22)
where
F
λ,β,n
(u,z
n
):= ∥ϕ(A
n
u −z
n
)∥
1
+λ · TV
β
(u),β > 0,(3.23)
with ϕ ∈ C being strictly convex.By Theorem 3.3,we can assume that bu and
bu
n
are unique minimizers of (3.10) and (3.22),respectively.The stability of the
minimization problem (3.10) refers to that if A
n
converges to A in the operator
norm and z
n
converges to z in the L
2
(Ω) norm,then bu
n
converges to bu in L
p
(Ω) for
1 ≤ p < d/(d −1) and weakly converges for p = d/(d −1) and d = 2.
According to Theorem3.2 of [1],if the functionals F
λ,β
(·,z) and F
λ,β,n
(·,z
n
) satisfy
the following two conditions,then the minimization problem (3.10) is stable:
(i)
For any sequence {u
n
} in L
p
(Ω),1 ≤ p ≤ d/(d −1),
lim
n→+∞
F
λ,β,n
(u
n
,z
n
) = +∞ whenever lim
n→+∞
∥u
n

BV
= +∞.(3.24)
(ii)
For a given M > 0 and ϵ > 0,there exists N such that
|F
λ,β,n
(u,z
n
) −F
λ,β
(u,z)| < ϵ whenever n ≥ N,∥u∥
BV
≤ M.(3.25)
Theorem 3.4.
Let ϕ ∈ C be strictly convex.Let A and A
n
defined in (1.1) and
(3.21) be linear bounded and injective operators and map from L
p
(Ω) to L
2
(Ω).Let
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 35
F
λ,β
(·,z) and F
λ,β,n
(·,z
n
) be defined respectively by (3.8) and (3.23).Let bu and bu
n
be unique minimizers of (3.10) and (3.22),respectively.Assume that ∥z
n
−z∥
2
→0
and ∥A
n
−A∥ →0 with
∥A
n
1


1
≥ γ > 0.(3.26)
If 1 ≤ p < d/(d −1),then
lim
n→+∞
∥bu
n
−bu∥
p
= 0.(3.27)
If p = d/(d −1) and d = 2,then bu
n
converges weakly to bu.
Proof.
Based on the discussion prior to this theorem,we only need to show that
F
λ,β
(·,z) and F
λ,β,n
(·,z
n
) satisfy conditions (i) and (ii).
We begin proving (i).To this end,it suffices to show that for any sequence {u
n
} in
L
p
(Ω),if liminf
n→∞
F
λ,β,n
(u
n
,z
n
) = K < ∞,then liminf
n→∞
TV(u
n
) < ∞.Clearly,
if liminf
n→∞
F
λ,β,n
(u
n
,z
n
) = K < ∞,then there exists a subsequence {u
n
k
}
k
of {u
n
}
such that
lim
k→∞
F
λ,β,n
k
(u
n
k
,z
n
k
) = liminf
n→∞
F
λ,β,n
(u
n
,z
n
) = K.
Hence,the sequence {F
λ,β,n
k
(u
n
k
,z
n
k
)}
k
is bounded.
To show liminf
n
∥u
n

BV
< +∞,it suffices to prove that {∥u
n
k

BV
}
k
is bounded.
By (3.12)-(3.13),each function in {u
n
k
} can be decomposed as follows
u
n
k
= v
n
k
+w
n
k
with w
n
k
=
1
|Ω|


u
n
k
dx.
By (3.14),we just need to show that both sequences {∥w
n
k

1
}
k
and {TV
0
(v
n
k
)}
k
are
bounded.Fortunately,the boundednesses of {∥w
n
k

1
}
k
and {TV
0
(v
n
k
)}
k
follow by
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 36
using the same procedure as the one in Lemma 3.2.
Now we turn to proving (ii).Notice that for any u ∈ BV (Ω),
|F
λ,β,n
(u,z
n
) −F
λ,β
(u,z)| ≤ ∥ϕ(A
n
u −z
n
) −ϕ(Au −z)∥
1
.
By Lemma 2.6,we have that
|F
λ,β,n
(u,z
n
)−F
λ,β
(u,z)| ≤ c
ϕ
(∥A
n
u−z
n

2
+∥Au−z∥
2
+|Ω|
1/2
)∥(A
n
u−z
n
)−(Au−z)∥
2
.
Since ∥z
n
−z∥
2
→0 and ∥A
n
−A∥ →0,∥A
n
u −z
n

2
and ∥Au −z∥
2
are uniformly
bounded on the set {u|TV(u) ≤ M}.Furthermore,we have
∥(A
n
u−z
n
) −(Au−z)∥
2
≤ ∥A
n
−A∥∥u∥
p
+∥z
n
−z∥
2
≤ ∥A
n
−A∥∥u∥
BV
+∥z
n
−z∥
2
.
Thus,we know that (3.25) holds.
In summary,the minimization problem (3.10) is well-posed by Theorem 3.3 and
Theorem 3.4.
3.3 Convergence of Minimizers
Consider the equation
z = Au,(3.28)
where A is a linear and bounded operator from L
p
(Ω) to L
2
(Ω).We assume that
(3.28) has an exact solution u
e
.A sequence of perturbed equations to (3.28) is given
by (3.21) with A
n
,1 ≤ n < ∞,being linear and bounded operators from L
p
(Ω)
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 37
to L
2
(Ω).Let u
λ
n
,n
be approximate solutions to (3.21) obtained by the following
minimization problems
inf
u∈BV (Ω)
F
λ
n
,β,n
(u,z
n
),(3.29)
where
F
λ
n
,β,n
(u,z
n
):= ∥ϕ(A
n
u −z
n
)∥
1

n
· TV
β
(u),β > 0.(3.30)
Under certain conditions,we will show that u
λ
n
,n
converges to u
e
as regularization
parameters λ
n
approach to 0.Although the following theorem is motivated by [1],its
proof is distinctive.
Theorem 3.5.
Let ϕ ∈ C be strictly convex.Let A and A
n
defined in (3.28) and
(3.21) be injective and map from L
p
(Ω) to L
2
(Ω).Let F
λ
n
,β,n
(·,z
n
) be defined by
(3.30).Let u
λ
n
,n
be unique minimizers of (3.29).Assume that ∥z
n
− z∥
2
→ 0,
∥A
n
−A∥ →0,and ∥A
n
1


1
≥ γ > 0.If λ
n
→0 at a rate such that
F
0,β,n
(u
e
,z
n
) −F
0,β,n
(u
0,n
,z
n
) = O(λ
n
),(3.31)
where u
0,n
is the minimizer of F
0,β,n
,then u
λ
n
,n
strongly converges to u
e
in L
p
(Ω) for
1 ≤ p < d/(d −1) and weakly converges for p = d/(d −1) and d = 2.
Proof.
The proof is basically split into three parts.The first part is to show the
boundedness of the sequence {F
λ
n
,β,n
(u
λ
n
,n
,z
n
)}
n
;the second part is about the bound-
edness of the sequence {∥u
n

BV
}
n
;the last part is the convergence of u
n
to u
e
.
Since ∥z
n
−z∥
2
→0,∥A
n
−A∥ →0,and λ
n
→0,{F
λ
n
,β,n
(u
e
,z
n
)}
n
is bounded
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 38
by using the proof of Lemma 2.5.Since u
λ
n
,n
is the minimizer of (3.29),we have that
F
λ
n
,β,n
(u
λ
n
,n
,z
n
) ≤ F
λ
n
,β,n
(u
e
,z
n
).(3.32)
Thus,{F
λ
n
,β,n
(u
λ
n
,n
,z
n
)}
n
is bounded.This finishes the proof of the first part.
We now show that there exists a constant M such that ∥u
λ
n
,n

BV
≤ M for all n.
By (3.14),it suffices to show that both sequences {∥w
λ
n
,n

1
}
n
and {TV
0
(v
λ
n
,n
)}
n
are
bounded,where
u
λ
n
,n
= v
λ
n
,n
+w
λ
n
,n
with w
λ
n
,n
=
1
|Ω|


u
λ
n
,n
dx.
The boundedness of {∥w
λ
n
,n

1
}
n
can be obtained by following the same way as we
did in Lemma 3.2.Because TV
β
(u
λ
n
,n
) = TV
β
(v
λ
n
,n
),
F
λ
n
,β,n
(u
λ
n
,n
,z
n
) = F
0,β,n
(u
λ
n
,n
,z
n
) +λ
n
· TV
β
(u
n
),
and
F
λ
n
,β,n
(u
e
,z
n
) = F
0,β,n
(u
e
,z
n
) +λ
n
· TV
β
(u
e
).
we have from (3.32) that
F
0,β,n
(u
λ
n
,n
,z
n
) −F
0,β,n
(u
0,n
,z
n
)
λ
n
+TV
β
(v
λ
n
,n
)

F
0,β,n
(u
e
,z
n
) −F
0,β,n
(u
0,n
,z
n
)
λ
n
+TV
β
(u
e
).(3.33)
Since
F
0,β,n
(u
e
,z
n
)−F
0,β,n
(u
0,n
,z
n
)
λ
n
is bounded and F
0,β,n
(u
λ
n
,n
,z
n
)−F
0,β,n
(u
0,n
,z
n
) is non-
negative due to u
0,n
being the minimizer of F
0,β,n
(·,z
n
),the above inequality yields
that the sequence {TV
β
(u
λ
n
,n
)}
n
is bounded.Hence,the sequence {∥u
λ
n
,n

BV
}
n
is
bounded.This finishes the proof of the second part.
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 39
Now we turn to the last part of the proof.It will be sufficient to show that every
subsequence {u
λ
n
k
,n
k
}
k
of the sequence {u
λ
n
,n
}
n
has a subsequence which converges
to u
e
in L
p
(Ω) if 1 ≤ p < d/(d −1),or converges weakly if p = d/(d −1) and d = 2.
Let {u
λ
n
k
,n
k
}
k
be an arbitrary subsequence of {u
λ
n
,n
}
n
.Then {∥u
λ
n
k
,n
k

BV
}
k
is
bounded.There exists a subsequence of {u
λ
n
k
,n
k
}
k
which converges to some bu ∈ L
p
(Ω)
if 1 ≤ p < d/(d −1),or converges weakly if p = d/(d −1) and d = 2.Without loss of
generality,we assume that this subsequence is {u
λ
n
k
,n
k
}
k
.Our goal is to show bu = u
e
.
It is equivalent to showing F
0,β
(bu,z) = F
0,β
(u
e
,z).
Note that
|F
0,β
(bu,z) −F
0,β
(u
e
,z)|
≤ |F
0,β,n
k
(u
λ
n
k
,n
k
,z
n
k
) −F
0,β
(bu,z)| +|F
0,β,n
k
(u
λ
n
k
,n
k
,z
n
k
) −F
0,β,n
k
(u
0,n
k
,z
n
k
)|
+|F
0,β,n
k
(u
0,n
k
,z
n
k
) −F
0,β,n
k
(u
e
,z
n
k
)| +|F
0,β,n
k
(u
e
,z
n
k
) −F
0,β
(u
e
,z)|.
The four terms in the right-hand of the above inequality are denoted by I
1
,I
2
,I
3
,
and I
4
,conclusively.We will show that these terms approximate to zero when k
approaches to infinity.
For I
4
,by Lemma 2.6 we have that
I
4
≤ c
ϕ
(
∥A
n
k
u
e
−z
n
k

2
+∥Au
e
−z∥
2
+|Ω|
1/2
)
∥(A
n
k
u
e
−z
n
k
) −(Au
e
−z)∥
2
.
The assumptions of ∥z
n
− z∥
2
→ 0 and ∥A
n
− A∥ → 0 imply that {∥A
n
∥}
n
and
{∥z
n

2
}
n
are bounded and ∥(A
n
k
u
e
−z
n
k
) −(Au
e
−z)∥
2
→0.Hence,I
4
goes to zero
when k approaches to infinity.
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 40
By the given condition,I
3
≤ Cλ
n
k
,where C is a constant.Hence,I
3
→ 0 as
λ
n
k
→0.
For I
1
,we have,by Lemma 2.6,
I
1
≤ c
ϕ
(
∥A
n
k
u
λ
n
k
,n
k
−z
n
k

2
+∥Abu −z∥
2
+|Ω|
1/2
)
∥(A
n
k
u
λ
n
k
,n
k
−z
n
k
) −(Abu −z)∥
2
.
We notice that ∥A
n
k
u
λ
n
k
,n
k

2
≤ ∥A
n
k
∥∥u
λ
n
k
,n
k

p
≤ ∥A
n
k
∥∥u
λ
n
k
,n
k

BV
.Because
∥A
n
k
∥ is uniformly bounded and {∥u
λ
n
k
,n
k

BV
}
k
is bounded,we see that ∥A
n
k
u
λ
n
k
,n
k

2
is bounded.We also have that
∥(A
n
k
u
λ
n
k
,n
k
−z
n
k
) −(Abu −z)∥
2
≤ ∥A
n
k
u
λ
n
k
,n
k
−Au
λ
n
k
,n
k

+ ∥Au
λ
n
k
,n
k
−Abu∥
2
+∥z
n
k
−z∥
2
.
Since ∥A
n
k
−A∥ → 0,∥u
λ
n
k
,n
k
− bu∥
BV
→ 0,and ∥z
n
k
−z∥
2
→ 0,we have that I
1
converges to zero as k →∞.
For I
2
,by (3.33) we have
F
0,β,n
(u
λ
n
,n
,z
n
) −F
0,β,n
(u
0,n
,z
n
) ≤ F
0,β,n
(u
e
,z
n
) −F
0,β,n
(u
0,n
,z
n
)

n
· (TV
β
(u
e
) −TV
β
(v
λ
n
,n
)).
By (3.31),we have |F
0,β,n
(u
e
,z
n
) −F
0,β,n
(u
0,n
,z
n
)| →0 as λ
n
k
→0.Since TV
β
(u
e
)
and TV
β
(v
λ
n
,n
) are bounded,we have that I
2
→0 as λ
n
k
→0.The proof is complete.
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 41
3.4 Euler-Lagrange Equations
In this chapter,we proposed a minimization estimation (3.10) to solve the ill-posed
problem (1.2).The existence of the minimizer of F
λ,β
(u,z) with ϕ ∈ C over BV
space has been proved.A classical way to find a minimizer of F
λ,β
is to solve the
Euler-Lagrange Equation of (3.10),which is
A

ϕ

(Au −z) −λdiv
(
∇u

|∇u| +β
2
)
= 0 (3.34)
provided ϕ

exists on R and the Neumann boundary condition is given,
∂u
∂ν
= 0 on ∂Ω (ν is the outward normal to ∂Ω),(3.35)
where ∇u is the gradient of u in the distribution sense.In previous sections,we pro-
vide examples of ϕ ∈ C.Their corresponding Euler-Lagrange Equations are presented
as follows:
i) Given ϕ defined as the L
p
function with 1 < p ≤ 2,
pA

sign(Au −z)|Au −z|
p−1
−λdiv
(
∇u

|∇u| +β
2
)
= 0.(3.36)
ii) Given ϕ defined as (2.10),
A


2
(Au −z) +sign(Au −z)η(Au −z)
2
2(η +|Au −z|)
2
−λdiv
(
∇u

|∇u| +β
2
)
= 0.(3.37)
iii) Given ϕ defined as (2.11),
1
2
A

η tanh(
Au −z
η
) −λdiv
(
∇u

|∇u| +β
2
)
= 0.(3.38)
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 42
iv) Given ϕ defined as (2.12),
A

η(Au −z)

η
2
+(Au −z)
2
−λdiv
(
∇u

|∇u| +β
2
)
= 0.(3.39)
v) Given ϕ defined as (2.13),
A

η arctan(
Au −z
η
) −λdiv
(
∇u

|∇u| +β
2
)
= 0.(3.40)
The Euler-Lagrange equations (3.36-3.40) of five examples of ϕ have the Neumann
boundary condition.
Recall that C
m
(
¯
Ω):={u ∈ C
m
(Ω)|∇
α
1
,...,α
d
u is uniformly continuous on bounded
subsets of Ω,for all |α
1
|+...+|α
d
| ≤ m}.If ϕ

exists and continuous,the minimization
functional F
λ,β
defined as (3.8) is differentiable and its Gˆateaux derivative is
G(u,z) = A

ϕ

(Au −z) −λdiv
(
∇u

|∇u| +β
2
)
.
In this thesis,we consider that the operator A of (1.1) is a linear and bounded
operator mapping from L
p
(Ω) into L
2
(Ω).Since C
2
(Ω) ⊂ BV (Ω),we provide the
optimality condition of F
λ,β
presented as follows:
Theorem 3.6.
Assume that ϕ ∈ C is continuously differentiable.The following
statement holds:u

is a minimizer of F
λ,β
(u,z) defined as (3.8) over a convex set
U ⊂ C
2
(
¯
Ω) if and only if


G(u

,z)(u −u

)dx ≥ 0,∀u ∈ U.(3.41)
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 43
Proof.
If u

is a minimizer of F
λ,β
over U,we suppose that


G(u

,z)(u −u

)dx =
−ξ < 0 for some u ∈ U.By the mean value theorem,for every ε > 0 there exist an
s ∈ [0,1] such that
F
λ,β
(u

+ε(u −u

),z) = F
λ,β
(u

,z) +ε


G(u

+sε(u −u

),z)(u −u

)dx.(3.42)
In order to show that the second term of (3.42) is also smaller than zero,we let
u
ε,s
= u

+sε(u −u

).Then
|


[G(u
ε,s
,z) −G(u

,z)] (u −u

)dx|



|A



(Au
ε,s
−z) −ϕ

(Au

−z)] (u −u

)| dx








div
(
∇u
ε,s

|∇u
ε,x
|
2

2

∇u

|∇u|
2

2
)
(u −u

)





dx
(3.43)
By the continuity of ϕ

and the boundness of A,we find that the first term of (3.43)
is less than ξ/2 for all sufficiently small ε.We have that
div
∇u

|∇u|
2

2
=
d

i=1

i
(
∇u

|∇u|
2

2
)
.(3.44)
By u ∈ C
2
(
¯
Ω),we have that (3.44) is continuous.Therefore,for all sufficiently small
ε,the second term of (3.43) can be less than ξ/(2λ).Then


G(u
ε,s
,z)(u −u

)dx < 0
for all sufficiently small ε.Then u
ε
is the minimizer of F
λ,β
,which contradicts the
hypothesis of this theorem.Thus,(3.41) holds.
CHAPTER 3.VARIATIONAL MODELS FOR IMAGE RESTORATION 44
On the other hand,by the convexity of F
λ,β
,we have that
F
λ,β
(u,z) ≥ F
λ,β
(u

,z) +


G(u

,z)(u −u

)dx,∀u ∈ U.(3.45)
If (3.41) holds,we have that u

is a minimizer of F
λ,β
over U.
Chapter 4
Discrete Content-Driven
Variational Models
We recall the content-driven variational model as follows
inf
u∈BV(Ω)
F
λ,β
(u,z),(4.1)
where Ω ∈ R
d
and
F
λ,β
(u,z):= ∥ϕ(Au −z)∥
1
+λ · TV
β
(u),ϕ ∈ S.(4.2)
Well-posedness of the model (4.1) has been discussed in the previous chapters.In
this chapter,we apply four gradient based algorithms,SD,CG,HB,and SD-TPSS
algorithms,to find the solution of the regularized problem (4.1) and provide brief
discussions about strengths and limitations of these gradient based algorithms.The
convergence analysis of the SD and CG algorithms is presented in the following sec-
tions.
45
CHAPTER 4.DISCRETE CONTENT-DRIVEN VARIATIONAL MODELS 46
4.1 Steepest Descent Algorithms for the Content-
Driven Variational Model
In this section,discrete models to the problem (4.1) in one dimensional case and two
dimensional case are given.We apply the steepest descent algorithm to the discrete
models and prove the convergence of the steepest algorithm in one dimensional case
and two dimensional case.
4.1.1 Discrete Content-Driven Variational Models in One Di-
mension
Let d = 1 and Ω = (0,1) ⊂ R.We partition
¯
Ω:= [0,1] into M equal-spaced intervals
as
¯
Ω =

M−1
i=0
[ih,(i +1)h],where h =
1
M
.Let z:(0,1) →R be the observed image
and u:(0,1) →R be the solution of the proposed minimization problem (4.1).The
discretelization of u is presented as
u
i
= u
(
(2i −1)h
2
)
for 0 < i < M +1.
Therefore u is discretized as u = (u
1
,...,u
M
).The discrete description of z is as
same as u.
CHAPTER 4.DISCRETE CONTENT-DRIVEN VARIATIONAL MODELS 47
We define a ‘forward-difference’ matrix D ∈ R
M×M
as
D =
1
M












−1 1
.
.
.
.
.
.
−1 1
0












.(4.3)
We notice that Du = (Du
1
,...,Du
M−1
,0)
T
is the forward difference of u with im-
posing Neumann boundary conditions
u
0
= u
1
,and u
M+1
= u
M
.(4.4)
and the ith element of Du has the form Du
i
=
u
i+1
−u
i
h
.The vector Du is regarded as
an approximation of ∇u.We consider a matrix A ∈ R
M×M
as a numerical approxi-
mation of the degradation operator A such that Au is an approximation of Au.
We provide the discrete form of the minimization functional F
λ,β
defined as (4.2).
By applying the quadrature rule to (4.2),the discrete functional F
λ,β
(·,z):R
M
→R
has the form
F
λ,β
(u,z) =
1
M
M

i=1
ϕ(Au
i
−z
i
) +λ · ψ
β
(Du
i
),∀u ∈ R
M
,ϕ ∈ S,(4.5)
where λ > 0 and Au
i
denotes the ith element of Au.The function ψ
β
:R → R
+
is defined as ψ
β
(·):=

(·)
2

2
.Therefore,the discrete content-driven variational
model is
inf
u∈R
M
F
λ,β
(u,z) (4.6)
with F
λ,β
(u,z) as (4.5).
CHAPTER 4.DISCRETE CONTENT-DRIVEN VARIATIONAL MODELS 48
4.1.2 Steepest Descent Algorithms
We apply the steepest descent algorithm to solve the minimization problem (4.6).
The algorithm is presented as follows:

Step 1:Given an initial value u
0
,calculate d
0
= −∇F
λ,β
(u
0
,z).

Step 2:If u
n
is calculated for n ≥ 0,then compute
u
n+1
= u
n

n
d
n
,(4.7)
where d
n
= −∇F
λ,β
(u
n
,z).Let n = n +1 and go to Step 2.
We consider u
n+1
as an estimation of the solution of the problem (4.5) if n is large
enough.There are many choices for setting up the step size α
n
.We mainly use two
schemes,namely,the constant step size and the Armijo line search.The constant step
size is the simplest and the most popular selection of the step size.It shows a great
advantage in computational efficiency.The scheme is simply presented as:Given a
positive constant S,let
α
n
= S ∀n ∈ N.
However,the fixed step size can not guarantee that the functional F
λ,β
(u
n
,z) is
decreasing during the iterations.The Armijo rule is a classic line search scheme,
mentioned in [8] as:Given positive fixed scalars S,ρ,and δ with 0 < ρ < 1 and
0 < δ < 1,we set α
n
= ρ
m
n
S,where m
n
is the first nonnegative integer m for which
F
λ,β
(u
n
,z) −F
λ,β
(u
n

m
Sd
n
,z) ≥ −δρ
m
S∇F
λ,β
(u
n
,z)
T
d
n
.(4.8)
CHAPTER 4.DISCRETE CONTENT-DRIVEN VARIATIONAL MODELS 49
The reduction factor ρ is usually chosen from [1/10,1/2].The Armijo rule keeps
the functional F
λ,β
decreasing at each iteration.However,it scarifies computation
efficiency due to the line search.
4.1.3 Convergence of Steepest Descent Methods in One Di-
mension
We will prove convergence of the steepest descent algorithm with constant step size
and Armijo line search step size for solving the minimization problem (4.6).First of
all,we recall the following preliminary results in [8],which are convergence results of
an unconstrained minimization problem
inf
u∈R
M
F(u) (4.9)
where F(u):R
M
→R.We refer to a vector u

satisfying the condition ∇F(u

) = 0
as a stationary point of F(u).We say that the direction sequence {d
n
}
n
is gradi-
ent related to {u
n
}
n
if the following property can be shown:For any subsequence
{u
n
k
}
n
k
⊂ {u
n
}
n
that converges to a nonstationary point,the corresponding subse-
quence {d
n
k
}
n
k
is bounded and satisfies
limsup
n
k
→∞
∇F(u
n
k
)
T
d
n
k
< 0.(4.10)
Gradient based algorithms for solving (4.9) have a general form the same as (4.7),
except that the descent direction d
n
and the step size α
n
can be arbitrarily selected.
Different selections of d
n
and α
n
give different algorithms for solving the problem
CHAPTER 4.DISCRETE CONTENT-DRIVEN VARIATIONAL MODELS 50
(4.9).In [8],convergence of gradient based algorithms with a constant step size is
stated as follows:Let {u
n
}
n
be a sequence of vectors generated by (4.7),where d
n
is
gradient related to {u
n
}
n
.Assume that for some constant L > 0,we have that
∥∇F(u) −∇F(v)∥
2
≤ L∥u −v∥
2
,∀u,v ∈ R
M
,(4.11)
and that for all n we have d
n
̸= 0 and ε ≤ α
n
≤ (2 −ε)
α
n
,where
α
n
=
|∇F(u
n
)
T
d
n
|
L∥d
n

2
and ε is a fixed positive scalar.Then every limit point of {u
n
}
n
is a stationary point.
Now we calculate the gradient of F
λ,β
(·,z) in order to study the convergence of
the steepest descent algorithm for solving (4.6).Let g(·,z) = ∇F
λ,β
(·,z).A function
Φ:R
M
→R
M
is defined as
Φ(u):= (ϕ

(u
1
),...,ϕ

(u
M
))
T
,
and Ψ:R
M
→R
M
is given by
Ψ(u):=
(
ψ

β
(u
1
),...,ψ

β
(u
M
)
)
T
,(4.12)
where ψ

β
(x) =
x
ψ
β
(x)
.By calculation,the gradient of F
λ,β
(u,z) with respect to u has
the form
g(u,z) =
1
M
A
T
Φ(Au −z) +
λ
M
D
T
Ψ(Du).(4.13)
Letting g
1
(u,z) =
1
M
A
T
Φ(Au −z) and g
2
(u) =
1
M
D
T
Ψ(Du),we have that
g(u,z) = g
1
(u,z) +λg
2
(u).(4.14)
In order to show convergence of the steepest descent algorithm for minimizing the
functional (4.5),we show below that {d
n
= −g(u
n
,z)}
n
is gradient related and the
gradient of F
λ,β
(·,z) satisfies (4.11).
CHAPTER 4.DISCRETE CONTENT-DRIVEN VARIATIONAL MODELS 51
Lemma 4.1.
Let {u
n
}
n
be a sequence of vectors generated by the steepest descent
method (4.7) for minimizing the functional F
λ,β
(u,z) in (4.5).Let g(u,z) be the
gradient of F
λ,β
(u,z) with respect to u,which has the form (4.13).Then,the sequence
of directions {d
n
= −g(u
n
,z)}
n
is gradient related.
Proof.
We need to show that {d
n
= −g(u
n
,z)}
n
is bounded and satisfies (4.10).By
(4.14),we have that
g(u
n
,z) = g
1
(u
n
,z) +λg
2
(u
n
).
We need to show that both g
1
and g
2
are bounded.By the hypothesis H3,there
exists a positive number c
ϕ
such that |ϕ

(x)| ≤ c
ϕ
.We have that
∥g
1
(u
n
,z)∥
2
= ∥
1
M
A
T
Φ(Au
n
−z)∥
2

1
M
∥A
T
∥∥Φ(Au
n
−z)∥
2
≤ c
ϕ
∥A
T
∥.
Thus,the sequence {g
1
(u
n
,z)}
n
is bounded.Next,we need to show that g
2
is also
bounded.The boundedness of g
2
(u) is determined by the boundedness of Ψ,which is
composed by the scalar function ψ

β
(x).Since the scalar function ψ

β
(x) is monotone
increasing and bounded by 1,we have that
∥g
2
(u
n
)∥
2
=
1
M
∥D
T
Ψ(Du)∥
2

1
M
∥D∥∥Ψ(Du)∥
2
≤ ∥D∥.
Therefore,
∥g(u
n
,z)∥
2
≤ ∥g
1
(u
n
,z)∥
2
+λ∥g
2
(u
n
)∥
2
≤ c
ϕ
∥A
T
∥ +λ∥D∥.
Secondly,we will show that (4.10) is true for {d
n
}
n
.Since d
n
= −g(u
n
,z) in the
steepest descent algorithm,we have that
g(u
n
,z)
T
d
n
= −∥g(u
n
,z)∥
2
2
≤ 0,∀n ∈ N.
CHAPTER 4.DISCRETE CONTENT-DRIVEN VARIATIONAL MODELS 52
Therefore,we only need to show that if any subsequence {u
n
k
}
n
k
⊂ {u
n
}
n
con-
verges to a nonstationary point
ˆ
u,i.e.,limsup
n
k
→∞
u
n
k
=
ˆ
u and g(
ˆ
u,z) ̸= 0,then
the inequality limsup
n
k
→∞
g(u
n
k
,z)
T
d
n
k
̸= 0 holds.We assume that there exists
a subsequence {u
n
k
}
n
k
satisfying limsup
n
k
→∞
g(u
n
k
,z)
T
d
n
k
= 0.Since g(u
n
k
,z) is
bounded,there exists a subsequence {u
n
k
j
}
n
k
j
⊂ {u
n
k
}
n
k
such that
lim
n
k
j
→∞
g(u
n
k
j
,z)
T
d
n
k
j
= lim
n
k
j
→∞
∥g(u
n
k
j
,z)∥
2
2
= ∥g(ˆu,z)∥
2
2
= 0.
Thus,we have g(ˆu,z) = 0,which contradicts to the fact that ˆu is a nonstationary
point.
Lemma 4.2.
The gradient of the functional F
λ,β
(·,z) defined as in (4.5) is Lipschitz
continuous,i.e.,for some constant L > 0,
∥∇F(u,z) −∇F(v,z)∥
2
≤ L∥u −v∥
2
,∀u,v ∈ R
M
.(4.15)
Proof.
Since g(·,z) = ∇F
λ,β
(·,z) having the form (4.14),we have that
∥g(u,z) −g(v,z)∥
2
≤ ∥g
1
(u,z) −g
1
(v,z)∥
2
+λ∥g
2
(u) −g
2
(v)∥
2
,∀u,v ∈ R
M
.
Firstly,we will show that there exists a positive number L
1
such that
∥g
1
(u,z) −g
1
(v,z)∥
2
≤ L
1
∥u −v∥
2
.(4.16)
By Lemma 4.1,the each component of g
1
(u,z) can be characterized by ϕ

.Since
ϕ ∈ S,there exists a constant C such that |ϕ
′′
(x)| < C for all x ∈ R.Therefore,by
the mean value theorem


(x) −ϕ

(y)| ≤ |ϕ
′′
(ξ)||x −y| ≤ C|x −y|,(4.17)
CHAPTER 4.DISCRETE CONTENT-DRIVEN VARIATIONAL MODELS 53
where ξ ∈ [x,y].Replacing x by each component of Au−z and y by the corresponding
component of Av −z,we have that
∥g
1
(u,z) −g
1
(v,z)∥
2

1
M
∥A
T
∥∥Φ(Au −z) −Φ(Av −z)∥
2

C
M
∥A
T
∥∥A∥∥u −v∥
2
.
Thus (4.16) holds with L
1
=
C
M
∥A∥
2
.
Secondly,we will show that there exists a positive number L
2
such that
∥g
2
(u) −g
2
(v)∥
2
≤ L
2
∥u −v∥
2
,∀u,v ∈ R
M
.(4.18)
By the definition of g
2
,we have that
∥g
2
(u) −g
2
(v)∥
2
=
1
M
∥D
T
Ψ(Du) −D
T
Ψ(Dv)∥
2

1
M
∥D∥∥Ψ(Du) −Ψ(Dv)∥
2
.
We notice that Ψ is defined as in (4.12) and ψ
′′
β
(x) = β
2
(x
2

2
)
−3/2

1
β
.By the
mean value theorem,there exists ξ ∈ (x,y) such that


β
(x) −ψ

β
(y)| ≤ |ψ
′′
β
(ξ)||x −y|.
We have that
∥g
2
(u) −g
2
(v)∥
2

∥D∥

∥Du −Dv∥
2

∥D∥
2

∥u −v∥
2
.