lec04_basic_segmx

courageouscellistAI and Robotics

Oct 29, 2013 (3 years and 5 months ago)

64 views

The University of








Ontario

CS
4487/9587

Algorithms for Image Analysis

Segmentation (2D)


Acknowledgements: Alexei Efros, Steven Seitz

The University of








Ontario

CS 4487/9587
Algorithms for Image Analysis


Segmentation (2D)


Blobs


Need for blobs


Extracting blobs


Thresholding, region growing, mean
-
shift


Minimum spanning trees


Object extraction


Intelligent scissors (also known as
live
-
wire
)


Snakes


Region+boundary methods


Local, greedy, and global processing



Extra Reading: Sonka at.al. Ch. 5


Gonzalez and Woods, Ch. 10

The University of








Ontario

HW assignment 1


2D image segmentation


Live
-
wire (based on
Dijkstra

algorithm for shortest



paths on a graph)


Implement and experiment on
many images


Compare
with local/greedy methods


Some useful code libraries and sample images will
be posted within a couple of days

The University of








Ontario

Goal: Extract “Blobs”


What are “blobs”?


Regions of an image that are somehow coherent


Why?


Object extraction, object removal, compositing, etc.


…but are “blobs” objects?


No, not in general

The University of








Ontario

Blob’s coherence


Simplest way to define blob coherence is as
similarity in brightness or color:

The tools become blobs

The house, grass, and sky make


different blobs

The University of








Ontario

Why is this useful?

AIBO

RoboSoccer

(VelosoLab)

The University of








Ontario

Ideal Segmentation

The University of








Ontario

Result of Segmentation

The University of








Ontario

Thresholding


Basic segmentation operation:



mask(x,y) = 1 if im(x,y) > T



mask(x,y) = 0 if im(x,y) < T


T is threshold


User
-
defined


Or automatic




Same as
histogram
partitioning:


The University of








Ontario

Sometimes works well…

What are potential

Problems?

The University of








Ontario

…but more often not

Adaptive thresholding

The University of








Ontario

Region growing


Is this same as global threshold?


Start with initial set of pixels
K

(initial seed(s))


Add to pixels
p

in
K

their neighbors
q

if
|Ip
-
Iq| < T


Repeat until nothing changes

The University of








Ontario

Region growing

The University of








Ontario

What can go wrong

with region growing ?

Region growth may “leak”
through a single
week spot

in the boundary

The University of








Ontario

Region growing


See region
leaks

into sky
due to a weak boundary
between them

The University of








Ontario

Due to Pedro
Felzenszwalb

and Daniel Huttenlocher

Motivating example

This image has three
perceptually distinct regions


Difference along border between A and
B is less then differences within C

A

C

B

Q: Where would
image thresholding

fail?

Q: Where would
region growing

fail?

A: Region A would be divided in two sets and


region C will be split into a large number


of arbitrary small subsets

A: Either A and B are merged or region C is


split into many small subsets


Also, B and C are merged

The University of








Ontario

Color
-
Based Blob Segmentation


Automatic Histogram Partitioning


Given image with N colors, choose K


Each of the K colors defines a region


not necessarily contiguous


Performed by computing color histogram, looking for
modes

The University of








Ontario

Finding Modes in a Histogram


How Many Modes Are There?


Easy to see, hard to
compute

The University of








Ontario

Mean Shift [
Comaniciu & Meer]

1.
Initialize random seed, and fixed window

2.
Calculate center of gravity ‘
x
’ of the window (
the“mean
”)

3.
Translate the search window to the mean

4.
Repeat Step 2 until convergence

Iterative
Mode Search


x



o



x

x



mode

The University of








Ontario

Mean
-
Shift

The University of








Ontario

Mean
-
shift results

More Examples
:
http://www.caip.rutgers.edu/~comanici/segm_images.html

The University of








Ontario

Issues:


Although often useful, all these approaches
work only some of the time, and are considered
rather “hacky”.



Can’t even handle our tiger:

Problem is that blobs != objects!

The University of








Ontario

Extracting objects


How could this be done?

The University of








Ontario

Extracting objects


Many approaches proposed


color cues


region cues


contour cues


We will consider a few of these


Today:


Intelligent Scissors (contour
-
based)


E. N. Mortensen and W. A. Barrett,
Intelligent Scissors for Image
Composition
, in ACM Computer Graphics (SIGGRAPH `95), pp. 191
-
198,
1995


The University of








Ontario

Intelligent Scissors

The University of








Ontario

Intelligent Scissors


Approach answers a basic question


Q: how to find a path from seed to mouse that follows
object boundary as closely as possible?


A: define a path that stays as close as possible to edges


The University of








Ontario

Intelligent Scissors


Basic Idea


Define edge score for each pixel


edge pixels have low cost


Find lowest cost path from seed to mouse

seed

mouse

Questions


How to define costs?


How to find the path?

The University of








Ontario

Path Search (basic idea)


Graph Search Algorithm


Computes minimum cost path from seed to
all

other

pixels

Note: diagonal “paths”
are scaled by a factor
.



Why?

3 sets of nodes




Free



Active



Done

The University of








Ontario

Segmentation should be

Invariant
to Image Rotation

After object rotation

L

Path’s cost along the top boundary

Path’s cost along the top
-
left boundary

After diagonal links are adjusted by

2

2

2

2

2

2

2

2

2

2

2

2

image gradient scores

The University of








Ontario

Instead of nodes, image gradient scores
can be assigned directly to graph edges


Graph


node for every pixel
p


link between every adjacent pair of pixels
e=(p,q)


cost
w(e)

for each link


Note: each
link

has a cost


this is a little different than the figure before where
each pixel (graph node) had a cost

p

q


Treat the image as a graph

The University of








Ontario

Defining edge costs


Treat the image as a graph


Want to hug image edges:
how to define cost of a link?

p

q


the link should follow the intensity edge


want intensity to change rapidly to the link



the cost of edge should be small when the
difference of intensity to the link is large

T

T

The University of








Ontario

Defining edge costs

p

q


First, estimate image derivative in the direction
orthogonal to the edge



Use finite differences (or cross
-
correlation filters)

1

-
1

-
1

-
1

1

1

The University of








Ontario

Defining edge costs

p

q


Second, use some penalty function
g( )

assigning low
penalty to large directional derivatives and large
penalty to small derivatives

The University of








Ontario

Defining edge costs

p

q


Finally, cost of en edge
e

should be local contrast
score adjusted by the edge length









Why?

The University of








Ontario

Defining edge costs


When computing the shortest path we approximate a
contour
C

minimizing continuous geometrically
meaningful functional

Cost of one edge

Cost of a PATH

Integral of contrast penalty along the contour

The University of








Ontario

(see Cormen et.al. “
Introduction to Algorithms
”, p.595)


Dijkstra’s shortest path algorithm

0

5

3

1

3

3

4

9

2

link cost


1.
init node costs to

, set
p

= seed point, cost(
p
) =
0

2.
expand
p

as follows:

for each of
p
’s neighbors
q

that are not expanded


set cost(
q
) = min( cost(
p
) +
c
pq
, cost(
q
) )

ALGORITHM

3 sets of nodes




Free



Active



Done

The University of








Ontario

Dijkstra’s shortest path algorithm

4

1

0

5

3

3

2

3

9

5

3

1

3

3

4

9

2

1

1


1.
init node costs to

, set
p

= seed point, cost(
p
) =
0

2.
expand
p

as follows:

for each of
p
’s neighbors
q

that are not expanded


set cost(
q
) = min( cost(
p
) +
c
pq
, cost(
q
) )


if
q
’s cost changed, make
q

point back to
p


put
q

on the ACTIVE list (if not already there)

ALGORITHM

3 sets of nodes




Free



Active



Done

The University of








Ontario

Dijkstra’s shortest path algorithm

4

1

0

5

3

3

2

3

9

5

3

1

3

3

4

9

2

1

5

2

3

3

3

2

4


1.
init node costs to

, set
p

= seed point, cost(
p
) =
0

2.
expand
p

as follows:

for each of
p
’s neighbors
q

that are not expanded


set cost(
q
) = min( cost(
p
) +
c
pq
, cost(
q
) )


if
q
’s cost changed, make
q

point back to
p


put
q

on the ACTIVE list (if not already there)

3.
set
r

= node with minimum cost on the ACTIVE list

4.
repeat Step 2 for
p

=
r

ALGORITHM

3 sets of nodes




Free



Active



Done

The University of








Ontario

Dijkstra’s shortest path algorithm

3

1

0

5

3

3

2

3

6

5

3

1

3

3

4

9

2

4

3

1

4

5

2

3

3

3

2

4


1.
init node costs to

, set
p

= seed point, cost(
p
) =
0

2.
expand
p

as follows:

for each of
p
’s neighbors
q

that are not expanded


set cost(
q
) = min( cost(
p
) +
c
pq
, cost(
q
) )


if
q
’s cost changed, make
q

point back to
p


put
q

on the ACTIVE list (if not already there)

3.
set
r

= node with minimum cost on the ACTIVE list

4.
repeat Step 2 for
p

=
r

ALGORITHM

3 sets of nodes




Free



Active



Done

The University of








Ontario

Dijkstra’s shortest path algorithm

3

1

0

5

3

3

2

3

6

5

3

1

3

3

4

9

2

4

3

1

4

5

2

3

3

3

2

4

2


1.
init node costs to

, set
p

= seed point, cost(
p
) =
0

2.
expand
p

as follows:

for each of
p
’s neighbors
q

that are not expanded


set cost(
q
) = min( cost(
p
) +
c
pq
, cost(
q
) )


if
q
’s cost changed, make
q

point back to
p


put
q

on the ACTIVE list (if not already there)

3.
set
r

= node with minimum cost on the ACTIVE list

4.
repeat Step 2 for
p

=
r

ALGORITHM

3 sets of nodes




Free



Active



Done

The University of








Ontario

Path Search (basic idea)

A

B

Dijkstra algorithm

-

processed nodes (distance to A is known)

-

active nodes (front)

-

active node with the smallest distance value

The University of








Ontario

Dijkstra’s shortest path algorithm


Properties


It computes the minimum cost path from the seed to every
node in the graph. This set of minimum paths is represented
as a
tree


Running time, with N pixels:


O(N
2
) time if you use an active list


O(N log N) if you use an active priority queue (heap)


takes < second for a typical (640x480) image


Once this tree is computed once, we can extract the optimal
path from any point to the seed in O(N) time.


it runs in real time as the mouse moves


What happens when the user specifies a new seed?

The University of








Ontario

Livewire extensions


Directed graphs


Restricted search space


Restricted domain (e.g. near
a priori

model)


Restricted backward search


Different edge weight functions


Image
-
Edge strength


Image
-
Edge Curvature


Proximity to known approximate model/boundary


Multi
-
resolution processing

The University of








Ontario

Results

http://www.cs.washington.edu/education/courses/455/03wi/projects/project1/artifacts/index.html