The University of
Ontario
CS
4487/9587
Algorithms for Image Analysis
Segmentation (2D)
Acknowledgements: Alexei Efros, Steven Seitz
The University of
Ontario
CS 4487/9587
Algorithms for Image Analysis
Segmentation (2D)
Blobs
•
Need for blobs
•
Extracting blobs
–
Thresholding, region growing, mean

shift
–
Minimum spanning trees
Object extraction
•
Intelligent scissors (also known as
live

wire
)
•
Snakes
•
Region+boundary methods
Local, greedy, and global processing
Extra Reading: Sonka at.al. Ch. 5
Gonzalez and Woods, Ch. 10
The University of
Ontario
HW assignment 1
2D image segmentation
•
Live

wire (based on
Dijkstra
algorithm for shortest
paths on a graph)
•
Implement and experiment on
many images
•
Compare
with local/greedy methods
•
Some useful code libraries and sample images will
be posted within a couple of days
The University of
Ontario
Goal: Extract “Blobs”
What are “blobs”?
•
Regions of an image that are somehow coherent
Why?
•
Object extraction, object removal, compositing, etc.
•
…but are “blobs” objects?
•
No, not in general
The University of
Ontario
Blob’s coherence
Simplest way to define blob coherence is as
similarity in brightness or color:
The tools become blobs
The house, grass, and sky make
different blobs
The University of
Ontario
Why is this useful?
AIBO
RoboSoccer
(VelosoLab)
The University of
Ontario
Ideal Segmentation
The University of
Ontario
Result of Segmentation
The University of
Ontario
Thresholding
Basic segmentation operation:
mask(x,y) = 1 if im(x,y) > T
mask(x,y) = 0 if im(x,y) < T
T is threshold
•
User

defined
•
Or automatic
Same as
histogram
partitioning:
The University of
Ontario
Sometimes works well…
What are potential
Problems?
The University of
Ontario
…but more often not
Adaptive thresholding
The University of
Ontario
Region growing
Is this same as global threshold?
•
Start with initial set of pixels
K
(initial seed(s))
•
Add to pixels
p
in
K
their neighbors
q
if
Ip

Iq < T
•
Repeat until nothing changes
The University of
Ontario
Region growing
The University of
Ontario
What can go wrong
with region growing ?
Region growth may “leak”
through a single
week spot
in the boundary
The University of
Ontario
Region growing
See region
leaks
into sky
due to a weak boundary
between them
The University of
Ontario
Due to Pedro
Felzenszwalb
and Daniel Huttenlocher
Motivating example
This image has three
perceptually distinct regions
Difference along border between A and
B is less then differences within C
A
C
B
Q: Where would
image thresholding
fail?
Q: Where would
region growing
fail?
A: Region A would be divided in two sets and
region C will be split into a large number
of arbitrary small subsets
A: Either A and B are merged or region C is
split into many small subsets
Also, B and C are merged
The University of
Ontario
Color

Based Blob Segmentation
Automatic Histogram Partitioning
•
Given image with N colors, choose K
•
Each of the K colors defines a region
–
not necessarily contiguous
•
Performed by computing color histogram, looking for
modes
The University of
Ontario
Finding Modes in a Histogram
How Many Modes Are There?
•
Easy to see, hard to
compute
The University of
Ontario
Mean Shift [
Comaniciu & Meer]
1.
Initialize random seed, and fixed window
2.
Calculate center of gravity ‘
x
’ of the window (
the“mean
”)
3.
Translate the search window to the mean
4.
Repeat Step 2 until convergence
Iterative
Mode Search
x
o
x
x
mode
The University of
Ontario
Mean

Shift
The University of
Ontario
Mean

shift results
More Examples
:
http://www.caip.rutgers.edu/~comanici/segm_images.html
The University of
Ontario
Issues:
Although often useful, all these approaches
work only some of the time, and are considered
rather “hacky”.
Can’t even handle our tiger:
Problem is that blobs != objects!
The University of
Ontario
Extracting objects
How could this be done?
The University of
Ontario
Extracting objects
Many approaches proposed
•
color cues
•
region cues
•
contour cues
We will consider a few of these
Today:
•
Intelligent Scissors (contour

based)
–
E. N. Mortensen and W. A. Barrett,
Intelligent Scissors for Image
Composition
, in ACM Computer Graphics (SIGGRAPH `95), pp. 191

198,
1995
The University of
Ontario
Intelligent Scissors
The University of
Ontario
Intelligent Scissors
Approach answers a basic question
•
Q: how to find a path from seed to mouse that follows
object boundary as closely as possible?
•
A: define a path that stays as close as possible to edges
The University of
Ontario
Intelligent Scissors
Basic Idea
•
Define edge score for each pixel
–
edge pixels have low cost
•
Find lowest cost path from seed to mouse
seed
mouse
Questions
•
How to define costs?
•
How to find the path?
The University of
Ontario
Path Search (basic idea)
Graph Search Algorithm
•
Computes minimum cost path from seed to
all
other
pixels
Note: diagonal “paths”
are scaled by a factor
.
…
Why?
3 sets of nodes
•
Free
•
Active
•
Done
The University of
Ontario
Segmentation should be
Invariant
to Image Rotation
After object rotation
L
Path’s cost along the top boundary
Path’s cost along the top

left boundary
After diagonal links are adjusted by
2
2
2
2
2
2
2
2
2
2
2
2
image gradient scores
The University of
Ontario
Instead of nodes, image gradient scores
can be assigned directly to graph edges
Graph
•
node for every pixel
p
•
link between every adjacent pair of pixels
e=(p,q)
•
cost
w(e)
for each link
Note: each
link
has a cost
•
this is a little different than the figure before where
each pixel (graph node) had a cost
p
q
Treat the image as a graph
The University of
Ontario
Defining edge costs
Treat the image as a graph
Want to hug image edges:
how to define cost of a link?
p
q
•
the link should follow the intensity edge
–
want intensity to change rapidly to the link
•
the cost of edge should be small when the
difference of intensity to the link is large
T
T
The University of
Ontario
Defining edge costs
p
q
First, estimate image derivative in the direction
orthogonal to the edge
Use finite differences (or cross

correlation filters)
1

1

1

1
1
1
The University of
Ontario
Defining edge costs
p
q
Second, use some penalty function
g( )
assigning low
penalty to large directional derivatives and large
penalty to small derivatives
The University of
Ontario
Defining edge costs
p
q
Finally, cost of en edge
e
should be local contrast
score adjusted by the edge length
Why?
The University of
Ontario
Defining edge costs
When computing the shortest path we approximate a
contour
C
minimizing continuous geometrically
meaningful functional
Cost of one edge
Cost of a PATH
Integral of contrast penalty along the contour
The University of
Ontario
(see Cormen et.al. “
Introduction to Algorithms
”, p.595)
Dijkstra’s shortest path algorithm
0
5
3
1
3
3
4
9
2
link cost
1.
init node costs to
, set
p
= seed point, cost(
p
) =
0
2.
expand
p
as follows:
for each of
p
’s neighbors
q
that are not expanded
•
set cost(
q
) = min( cost(
p
) +
c
pq
, cost(
q
) )
ALGORITHM
3 sets of nodes
•
Free
•
Active
•
Done
The University of
Ontario
Dijkstra’s shortest path algorithm
4
1
0
5
3
3
2
3
9
5
3
1
3
3
4
9
2
1
1
1.
init node costs to
, set
p
= seed point, cost(
p
) =
0
2.
expand
p
as follows:
for each of
p
’s neighbors
q
that are not expanded
•
set cost(
q
) = min( cost(
p
) +
c
pq
, cost(
q
) )
–
if
q
’s cost changed, make
q
point back to
p
•
put
q
on the ACTIVE list (if not already there)
ALGORITHM
3 sets of nodes
•
Free
•
Active
•
Done
The University of
Ontario
Dijkstra’s shortest path algorithm
4
1
0
5
3
3
2
3
9
5
3
1
3
3
4
9
2
1
5
2
3
3
3
2
4
1.
init node costs to
, set
p
= seed point, cost(
p
) =
0
2.
expand
p
as follows:
for each of
p
’s neighbors
q
that are not expanded
•
set cost(
q
) = min( cost(
p
) +
c
pq
, cost(
q
) )
–
if
q
’s cost changed, make
q
point back to
p
•
put
q
on the ACTIVE list (if not already there)
3.
set
r
= node with minimum cost on the ACTIVE list
4.
repeat Step 2 for
p
=
r
ALGORITHM
3 sets of nodes
•
Free
•
Active
•
Done
The University of
Ontario
Dijkstra’s shortest path algorithm
3
1
0
5
3
3
2
3
6
5
3
1
3
3
4
9
2
4
3
1
4
5
2
3
3
3
2
4
1.
init node costs to
, set
p
= seed point, cost(
p
) =
0
2.
expand
p
as follows:
for each of
p
’s neighbors
q
that are not expanded
•
set cost(
q
) = min( cost(
p
) +
c
pq
, cost(
q
) )
–
if
q
’s cost changed, make
q
point back to
p
•
put
q
on the ACTIVE list (if not already there)
3.
set
r
= node with minimum cost on the ACTIVE list
4.
repeat Step 2 for
p
=
r
ALGORITHM
3 sets of nodes
•
Free
•
Active
•
Done
The University of
Ontario
Dijkstra’s shortest path algorithm
3
1
0
5
3
3
2
3
6
5
3
1
3
3
4
9
2
4
3
1
4
5
2
3
3
3
2
4
2
1.
init node costs to
, set
p
= seed point, cost(
p
) =
0
2.
expand
p
as follows:
for each of
p
’s neighbors
q
that are not expanded
•
set cost(
q
) = min( cost(
p
) +
c
pq
, cost(
q
) )
–
if
q
’s cost changed, make
q
point back to
p
•
put
q
on the ACTIVE list (if not already there)
3.
set
r
= node with minimum cost on the ACTIVE list
4.
repeat Step 2 for
p
=
r
ALGORITHM
3 sets of nodes
•
Free
•
Active
•
Done
The University of
Ontario
Path Search (basic idea)
A
B
Dijkstra algorithm

processed nodes (distance to A is known)

active nodes (front)

active node with the smallest distance value
The University of
Ontario
Dijkstra’s shortest path algorithm
Properties
•
It computes the minimum cost path from the seed to every
node in the graph. This set of minimum paths is represented
as a
tree
•
Running time, with N pixels:
–
O(N
2
) time if you use an active list
–
O(N log N) if you use an active priority queue (heap)
–
takes < second for a typical (640x480) image
•
Once this tree is computed once, we can extract the optimal
path from any point to the seed in O(N) time.
–
it runs in real time as the mouse moves
•
What happens when the user specifies a new seed?
The University of
Ontario
Livewire extensions
Directed graphs
Restricted search space
•
Restricted domain (e.g. near
a priori
model)
•
Restricted backward search
Different edge weight functions
•
Image

Edge strength
•
Image

Edge Curvature
•
Proximity to known approximate model/boundary
Multi

resolution processing
The University of
Ontario
Results
http://www.cs.washington.edu/education/courses/455/03wi/projects/project1/artifacts/index.html
Comments 0
Log in to post a comment