Total Variation and Euler's Elastica for Supervised Learning

beeuppityΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 7 μήνες)

71 εμφανίσεις

Key Lab. Of Machine Perception, School of EECS, Peking
University, China
Total Variation and Euler's
Elastica

for
Supervised Learning

Tong Lin,
Hanlin

Xue
, Ling Wang,
Hongbin

Zha

Contact: tonglin123@gmail.com

Key Laboratory of Machine Perception, School of EECS, Peking University, Beijing 100871, China

MOTIVATION

PROBLEM DEFINITION

MODELS

IMPLEMENTATION

EXPERIMENTAL RESULTS

Classification results on synthetic data sets:

The proposed TV/EE methods are
compared with
SVM, BPNN, and LR
(Laplacian Regularization) on
benchmark data sets for binary
classification, multi
-
class classification,
and regression.

Supervised

learning

infers

a

function

that

maps

inputs

to

desired

outputs

with

the

guidance

of

training

data
.

The

state
-
of
-
the
-
art

algorithm

is

SVM

based

on

large

margin

and

kernel

trick
.

It

was

observed

that

SVM

is

liable

to

overfitting,

especially

on

small

sample

data

sets
;

sometimes

SVM

can

offer

100
%

accuracies
.


We

argue

that

maximal

margin

should

not

be

the

sole

criterion

for

supervised

learning
;

the

curvature

and

gradients

of

the

output

decision

boundaries

can

play

an

important

role

to

avoid

overfitting
.

Our

method

is

inspired

by

the

great

success

of

Total

variation

(TV)

and

Euler’s

elastica

(EE)

in

image

processing
.

We

extend

TV

and

EE

to

high

dimensional

supervised

learning

settings
.



Curvature of a curve

Mean curvature of a
hypersurface
/decision
boundary

Using the calculus of variations, we get the following Euler
-
Lagrange PDEs:

The PDEs are nonlinear and high dimensional, so we use
function approximations to find the numerical solutions.

Radial Basis Function Approximation

(1) GD: Gradient Descent Time Marching


(2)
LagLE
: Lagged Linear Equation Iteration

Total Variation

TV model:

Euler’s Elastic Energy

EE model:

curvature formula:

Euler’s Elastica (EE) model in image
inpainting
:

Geometric
intuition
of TV/EE regularization:

Total

variation

is

the

measure

of

total

quantity

of

the

changes

of

a

function,

which

has

been

widely

used

in

image

processing,

such

as

the

ROF

denoising
.

The

proposed

TV

model

can

be

seen

as

a

special

case

of

the

following

EE

model
.

where

L

denotes

the

loss

function,

and

S(u)

is

the

regularization/smoothing

term
.

This

discrete

model

can

be

changed

into

the

following

continuous

form

with

squared

loss
:


A

general

framework

for

supervised

learning

is

the

following

regularization

framework
:

Our

purpose

is

to

introduce

two

smoothing

terms

(TV

&

EE)

for

supervised

learning

tasks
.


Given

training

data

,

the

goal

is

to

infer

the

underlying

mapping
:

RBF

function approximations

Energy

functional with TV/EE
regularization terms

Two numerical methods to find
the PDE solutions

Euler
-
Lagrange PDE via the

calculus of variations

FLOW CHART

SVM EE