ppt

blackeningfourAI and Robotics

Oct 19, 2013 (4 years and 25 days ago)

95 views

Normalized Cuts and
Image Segmentation


Amir Lev
-
Tov

IDC, Herzliya

Advanced Topics in Computer Vision


[
1
] Normalized Cuts and Image
Segmentation, Shi and Malik, IEEE Conf.
Computer Vision and Pattern Recognition,
1997
.


[
2
] Normalized Cuts and Image
Segmentation, Shi and Malik, IEEE
Transactions on pattern analysis and
machine intelligence, Vol
22
, No
8
,
2000


Main References

More References


[
3
] Weiss Y. Segmentation using eigenvectors: a
unifying view. Proceedings IEEE International
Conference on Computer Vision,
1999
.


[
4
] Ng A.Y., Jordan , M.I., and Weiss Y, On
Spectral Clustering: Analysis and an algorithm,
NIPS
2001


[
5
] Rayleigh’s Quotient, Nail Gumerov,
2003


[
6
] Wu and Leahy, an optimal graph theoretic
approach to data clustering, PAMI,
1993



Mathematical Introduction


Definition: is an
Eigen Value
of n x n
matrix A, if there exist a non
-
trivial vector


such that:


That vector is called
Eigen Vector

of A
corresponding to the Eigen Value



All Eigenvectors correspond to different
Eigenvalues, are mutually linearlly
independent (Orthogonal set).

Mathematical Introduction


Matrix A is called
Hermitian

if


Where A* is the
conjugate transpose
of A:



Real matrix is Hermitian


Symmetric


Let A be a Hermitian matrix.


Then








is called the
Rayleigh’s Quotient

of A.

Mathematical Introduction


For real matrices the definition becomes:











Where A is just symmetric.



Mathematical Introduction


Theorem
: Rayleigh’s Quotient gets its
minimum value at A’s minimal eigenvalue,


and the corresponding eigenvector


achieve this minimum.


Moreover: if A has n eigenvalues


then R(A,v) has n stationary points
achieved at their eigenvectors



respectively.


Mathematical Introduction


The
Generalized Rayleigh’s Quotient



is:




where B is hermitian (real
-
>symmetric)
and positive definite matrix


(all its eigenvalues are non
-
zero positives).


Minimum achieved by solving
:

Segmentation Introduction

Segmentation Introduction


Problem: Divide an image into subsets of
pixels (Segments).


Some methods:


Thresholding


Region Growing


K
-
means


Mean
-
Shift


Use of changes in color, texture etc.


Contours

Segmentation Introduction


The problem is not very well defined, for
example, how many groups are in the
picture?
4
? Maybe
3
?
2
? Or even every X

Segmentation Introduction


In order to get good Segmentation:


Low level cues like colors, texutre etc.


High level knowledge as global impression
from the picture (top
-
>down).


Need good similarity function


Number of segments is not known in
advance


The Graph partitioning method


Main Idea:


Model the image by a graph G=(V,E).


Assign similarity values to edges weights.


Find a cut in G of minimal value, which yield
partition of V into two subsets.


Matrix representation of computations.


Using Linear Algebra tools and Spectral Analysis
to solve the new minimization problem.


Recursively repartition the subpartitions.

Graph Modeling


The Graph G=(V,E)


Nodes:


Pixels


Some other higher level features


Edges:


Between every pair of nodes in V


Weights
:


Weight w(i,j) is function of similarity between node


i and j.

Graph Modeling


Objective


Partition the set of vertices into disjoint sets




Number of segments m is not known.


Cut:


Case of m=
2
, Bi
-
partition of V into A and B:


The Cut Value is:



The optimal cut is the one that minimizes its value



Minimun Cut


Wu and Leahy[
1993
]:


Use the mentioned cut criteria


Partition G into k subgraphs recursively


Minimize the maximum cut value


Produce good segmentation on some of the
images


Min Cut
-

The Problem


It is not the best cut !


Favors cutting small sets of isolated nodes:



Normalized Cut [Shi,Malick,
1997
]


Normalize the cut value with the volume of
the partition:




Where




Normalized Cut


Properties:


Sets with weak connections


G
et low Ncut value.


High Association within Sets


G
et low Ncut value.


But
-

small sets are panalized with high Ncut value.


Normalized Association


Normalized Association:




Naturally related criterions:

Computing the Optimal Cut

Given partition of nodes from V into A,B :


Let be an N=|V| dimensional indicator for A,



i.e


if node i is in A, and
-
1
otherwise


Let



be the total connection


from node i to all other nodes.


Rewrite:



Computing the Optimal Cut


Objective: Transform






into Rayleigh’s Quotient
-
like expression:

Matrix Representation


Let D be an N x N diagonal matrix


with d on its diagonal:








Let W be an N x N symmetrical affinity matrix with



Matrix Representation


Let k be the Ratio between degree of A to V:





Let be an N x
1
vector of all ones






Note: (
1
+x)/
2
and (
1
-
x)/
2
are indicators for





and



respectively


Matrix Representation





We can rewrite our expression as:


Ncut(A,B) = Ncut(x) =


Matrix Representation


It can be shown that the previous
expression is equal to:





Where



is ratio between degree of A



to degree of B


Matrix Representation




Setting new indicator y=(
1
+x)
-
b(
1
-
x)


we get


the Constraint:







Matrix Representation




Denominator:



Finding the Minimum


Putting the last two expression together we get


the Rayleigh’s quotient:





With the conditions:


Minimum achieved by finding the minimal Eigenvalue of
the system(
1
) (relaxing y to take on real values)




Corresponding Eigenvector will be in fact indicator vector
for nodes in the segment (A)

Finding the Minimum


But


we have two constraints:








We’ll see that the first one is satisfied:


Replacing y by


we get the standard
eigensystem (
2
):








is an eigenvector of it,with an eigenvalue of
0
.


Since the Laplaician matrix (D
-
W) is symmetric semi
-
positive definite, so that the new system

Finding the Minimum


Thus, z
0
is the smallest eigenvector of (
2
)




Also known: all the eigenvectors of
equation (
2
) are orthogonal to each other


In particular, z
1
, the second smallest
eigenvector is orthogonal to z
0



Finding the Minimum


In terms of our original system
)
1
):






Is the smallest eigenvector with







Where y
1
is the
2
nd

smallest


eigenvector of (
1
)




The
1
st

constraint is automatically satisfied:

Finding the Minimum


In Rayleigh’s Quotient, under the constraint that
z is orthogonal to the j
-
1
smallest eigenvectors,
the quotient is minimized by the next smallest
eigenvector zj and its minimum is the eigen
value




We get:



Consequently:


Finding the Minimum


Conclusion: the
2
nd

smallest eigenvector of (
1
)


is the real solution to our Normalized Cut
problem.


What about the
2
nd

constraint that y takes on
discrete values??


Solving the discrete problem is NP
-
Complete


Solution


approximate the continuous solution by
splitting the vector coordinates at different
thresholds, choosing the one that gives the best
NCut value.

Complexity


Wait!


What about the original graph problem ?


MinCut


Has Polynomial
-
Time algorithm


by the MaxFlow algorithm.


Impractical for images


Normalized Cut


NP
-
Complete


Need fast approximations

Complexity


Solving standard eigenvalue problem


Impractical for segmenting large number of
pixels


Special properties of our problem:


The graph often locally connected=>sparse
matrix


Only the top eigenvectors are needed


Low precision requirements


Using Lanczos eigensolver
-


Repartitioning


Recursively apply the above method to each of
the partitions


Subject to some “stability” criteria:


Create sub partitions by varying the splitting point around the
optimal value and check if Ncut value change much


Until certain Ncut threshold exceeded


Another approach: Use high order eigenvectors


Pros: more discriminative information


Cons: according to Shi&Malik, Approximation error
accumulates with every eigenvector taken



Summary of the Algorithm

1.
Given features, construct the graph

2.
Solve




for eigenvectors with the
smallest eigenvalues

3.
Use the eigenvector with the
2
nd

smallest
eigenvalue to bipartition the graph


Find the splitting point that minimizes Ncut

4.
Check stability and Ncut threshold to decide
whether
to divide the current partition.

5.
Recursively repartition the segmented parts if
necessary


Experiments


Pixels as Graph nodes


Weight Function:




X(i)


Spatial location of node I


F(i)


Feature vector based on Intensity, Color


or Texture information at node i

Experiments


Example of F(i):


F(i) =
1
, For point set segmentation


F(i) = I(i), Intensity value, for segmenting
brightness images



Experiments


Point set

Taken from [
1
]

Experiments


Synthetic image of corner

Taken from [
1
]

Experiments


Taken from [
1
]

Experiments


“Color” image

Taken from [
1
]

Experiments


Without well defined boundaries:

Taken from [
1
]

Experiments


Texture segmentation


Different orientation stripes

Taken from [
1
]

A little bit more..


Taken from [
2
]

A little bit more..


High order eigenvectors

Taken from [
2
]

High order eigenvectors

Taken from [
2
]

1
st

Vs
2
nd

Eigenvectors


Taken from [
3
]

Summery


Treat the problem as graph partitioning


The new idea:


Normalized Cut instead of Regular Cut


NCut criteria measures both:


Dissimilarity between groups


Similarity within a group


Global impression extraction of the image


Spectral Analysis in favor of segmenting images


Generalized eigenvalue system gives real
solution=>”segmenting” this data provide
clustering of the original image

Thanks!