# ppt

AI and Robotics

Oct 19, 2013 (4 years and 7 months ago)

111 views

Normalized Cuts and
Image Segmentation

Amir Lev
-
Tov

IDC, Herzliya

[
1
] Normalized Cuts and Image
Segmentation, Shi and Malik, IEEE Conf.
Computer Vision and Pattern Recognition,
1997
.

[
2
] Normalized Cuts and Image
Segmentation, Shi and Malik, IEEE
Transactions on pattern analysis and
machine intelligence, Vol
22
, No
8
,
2000

Main References

More References

[
3
] Weiss Y. Segmentation using eigenvectors: a
unifying view. Proceedings IEEE International
Conference on Computer Vision,
1999
.

[
4
] Ng A.Y., Jordan , M.I., and Weiss Y, On
Spectral Clustering: Analysis and an algorithm,
NIPS
2001

[
5
] Rayleigh’s Quotient, Nail Gumerov,
2003

[
6
] Wu and Leahy, an optimal graph theoretic
approach to data clustering, PAMI,
1993

Mathematical Introduction

Definition: is an
Eigen Value
of n x n
matrix A, if there exist a non
-
trivial vector

such that:

That vector is called
Eigen Vector

of A
corresponding to the Eigen Value

All Eigenvectors correspond to different
Eigenvalues, are mutually linearlly
independent (Orthogonal set).

Mathematical Introduction

Matrix A is called
Hermitian

if

Where A* is the
conjugate transpose
of A:

Real matrix is Hermitian

Symmetric

Let A be a Hermitian matrix.

Then

is called the
Rayleigh’s Quotient

of A.

Mathematical Introduction

For real matrices the definition becomes:

Where A is just symmetric.

Mathematical Introduction

Theorem
: Rayleigh’s Quotient gets its
minimum value at A’s minimal eigenvalue,

and the corresponding eigenvector

achieve this minimum.

Moreover: if A has n eigenvalues

then R(A,v) has n stationary points
achieved at their eigenvectors

respectively.

Mathematical Introduction

The
Generalized Rayleigh’s Quotient

is:

where B is hermitian (real
-
>symmetric)
and positive definite matrix

(all its eigenvalues are non
-
zero positives).

Minimum achieved by solving
:

Segmentation Introduction

Segmentation Introduction

Problem: Divide an image into subsets of
pixels (Segments).

Some methods:

Thresholding

Region Growing

K
-
means

Mean
-
Shift

Use of changes in color, texture etc.

Contours

Segmentation Introduction

The problem is not very well defined, for
example, how many groups are in the
picture?
4
? Maybe
3
?
2
? Or even every X

Segmentation Introduction

In order to get good Segmentation:

Low level cues like colors, texutre etc.

High level knowledge as global impression
from the picture (top
-
>down).

Need good similarity function

Number of segments is not known in

The Graph partitioning method

Main Idea:

Model the image by a graph G=(V,E).

Assign similarity values to edges weights.

Find a cut in G of minimal value, which yield
partition of V into two subsets.

Matrix representation of computations.

Using Linear Algebra tools and Spectral Analysis
to solve the new minimization problem.

Recursively repartition the subpartitions.

Graph Modeling

The Graph G=(V,E)

Nodes:

Pixels

Some other higher level features

Edges:

Between every pair of nodes in V

Weights
:

Weight w(i,j) is function of similarity between node

i and j.

Graph Modeling

Objective

Partition the set of vertices into disjoint sets

Number of segments m is not known.

Cut:

Case of m=
2
, Bi
-
partition of V into A and B:

The Cut Value is:

The optimal cut is the one that minimizes its value

Minimun Cut

Wu and Leahy[
1993
]:

Use the mentioned cut criteria

Partition G into k subgraphs recursively

Minimize the maximum cut value

Produce good segmentation on some of the
images

Min Cut
-

The Problem

It is not the best cut !

Favors cutting small sets of isolated nodes:

Normalized Cut [Shi,Malick,
1997
]

Normalize the cut value with the volume of
the partition:

Where

Normalized Cut

Properties:

Sets with weak connections

G
et low Ncut value.

High Association within Sets

G
et low Ncut value.

But
-

small sets are panalized with high Ncut value.

Normalized Association

Normalized Association:

Naturally related criterions:

Computing the Optimal Cut

Given partition of nodes from V into A,B :

Let be an N=|V| dimensional indicator for A,

i.e

if node i is in A, and
-
1
otherwise

Let

be the total connection

from node i to all other nodes.

Rewrite:

Computing the Optimal Cut

Objective: Transform

into Rayleigh’s Quotient
-
like expression:

Matrix Representation

Let D be an N x N diagonal matrix

with d on its diagonal:

Let W be an N x N symmetrical affinity matrix with

Matrix Representation

Let k be the Ratio between degree of A to V:

Let be an N x
1
vector of all ones

Note: (
1
+x)/
2
and (
1
-
x)/
2
are indicators for

and

respectively

Matrix Representation

We can rewrite our expression as:

Ncut(A,B) = Ncut(x) =

Matrix Representation

It can be shown that the previous
expression is equal to:

Where

is ratio between degree of A

to degree of B

Matrix Representation

Setting new indicator y=(
1
+x)
-
b(
1
-
x)

we get

the Constraint:

Matrix Representation

Denominator:

Finding the Minimum

Putting the last two expression together we get

the Rayleigh’s quotient:

With the conditions:

Minimum achieved by finding the minimal Eigenvalue of
the system(
1
) (relaxing y to take on real values)

Corresponding Eigenvector will be in fact indicator vector
for nodes in the segment (A)

Finding the Minimum

But

we have two constraints:

We’ll see that the first one is satisfied:

Replacing y by

we get the standard
eigensystem (
2
):

is an eigenvector of it,with an eigenvalue of
0
.

Since the Laplaician matrix (D
-
W) is symmetric semi
-
positive definite, so that the new system

Finding the Minimum

Thus, z
0
is the smallest eigenvector of (
2
)

Also known: all the eigenvectors of
equation (
2
) are orthogonal to each other

In particular, z
1
, the second smallest
eigenvector is orthogonal to z
0

Finding the Minimum

In terms of our original system
)
1
):

Is the smallest eigenvector with

Where y
1
is the
2
nd

smallest

eigenvector of (
1
)

The
1
st

constraint is automatically satisfied:

Finding the Minimum

In Rayleigh’s Quotient, under the constraint that
z is orthogonal to the j
-
1
smallest eigenvectors,
the quotient is minimized by the next smallest
eigenvector zj and its minimum is the eigen
value

We get:

Consequently:

Finding the Minimum

Conclusion: the
2
nd

smallest eigenvector of (
1
)

is the real solution to our Normalized Cut
problem.

2
nd

constraint that y takes on
discrete values??

Solving the discrete problem is NP
-
Complete

Solution

approximate the continuous solution by
splitting the vector coordinates at different
thresholds, choosing the one that gives the best
NCut value.

Complexity

Wait!

What about the original graph problem ?

MinCut

Has Polynomial
-
Time algorithm

by the MaxFlow algorithm.

Impractical for images

Normalized Cut

NP
-
Complete

Need fast approximations

Complexity

Solving standard eigenvalue problem

Impractical for segmenting large number of
pixels

Special properties of our problem:

The graph often locally connected=>sparse
matrix

Only the top eigenvectors are needed

Low precision requirements

Using Lanczos eigensolver
-

Repartitioning

Recursively apply the above method to each of
the partitions

Subject to some “stability” criteria:

Create sub partitions by varying the splitting point around the
optimal value and check if Ncut value change much

Until certain Ncut threshold exceeded

Another approach: Use high order eigenvectors

Pros: more discriminative information

Cons: according to Shi&Malik, Approximation error
accumulates with every eigenvector taken

Summary of the Algorithm

1.
Given features, construct the graph

2.
Solve

for eigenvectors with the
smallest eigenvalues

3.
Use the eigenvector with the
2
nd

smallest
eigenvalue to bipartition the graph

Find the splitting point that minimizes Ncut

4.
Check stability and Ncut threshold to decide
whether
to divide the current partition.

5.
Recursively repartition the segmented parts if
necessary

Experiments

Pixels as Graph nodes

Weight Function:

X(i)

Spatial location of node I

F(i)

Feature vector based on Intensity, Color

or Texture information at node i

Experiments

Example of F(i):

F(i) =
1
, For point set segmentation

F(i) = I(i), Intensity value, for segmenting
brightness images

Experiments

Point set

Taken from [
1
]

Experiments

Synthetic image of corner

Taken from [
1
]

Experiments

Taken from [
1
]

Experiments

“Color” image

Taken from [
1
]

Experiments

Without well defined boundaries:

Taken from [
1
]

Experiments

Texture segmentation

Different orientation stripes

Taken from [
1
]

A little bit more..

Taken from [
2
]

A little bit more..

High order eigenvectors

Taken from [
2
]

High order eigenvectors

Taken from [
2
]

1
st

Vs
2
nd

Eigenvectors

Taken from [
3
]

Summery

Treat the problem as graph partitioning

The new idea:

Normalized Cut instead of Regular Cut

NCut criteria measures both:

Dissimilarity between groups

Similarity within a group

Global impression extraction of the image

Spectral Analysis in favor of segmenting images

Generalized eigenvalue system gives real
solution=>”segmenting” this data provide
clustering of the original image

Thanks!