Model comparison and challenges II

beeuppityAI and Robotics

Oct 19, 2013 (3 years and 9 months ago)

64 views

Model comparison and challenges II

Compositional bias of salient object detection benchmarking

Xiaodi

Hou

K
-
Lab, Computation and Neural Systems

California Institute of Technology

for the Crash Course on Visual Saliency Modeling:

Behavioral Findings and Computational Models

CVPR 2013

Schedule

On detecting salient objects


Learning to Detect A Salient Object [Liu et. al., CVPR 07]


Frequency
-
tuned
Salient Region
Detection [
Achanta

et. al., CVPR 09]

The progress!


Some top performers:


[PCA]

What makes a patch distinct [
Margolin

et. al., CVPR 13]


[SF]
Saliency filters [
Perazzi

et. al., CVPR 12]:


F
-
Measure: 0.84


[GC]/[GC
-
seg
]
Global contrast
-
based salient region detection [Cheng et. al.,
CVPR 11]


F
-
Measure: 0.75


[FT]

Frequency Tuned Salient Region Detection [
Achanta

et.
a.l
., CVPR 09] :


0.65 by [
Achanta

et. al., CVPR 09].

Image from [
Perazzi

et.
al., CVPR 2012]

The progress?


Salient objects in PASCAL
VOC?


850 images from VOC 2013
validation set.


I
ntersection of main challenge
and segmentation challenge.


Answers more questions:


Where is your algorithm (in
salient object detection)?


Where is salient object detection
(in computer vision).

The progress


FT: 0.28


GC: 0.39


SF: 0.35


PCA: 0.40


GC
-
seg
: 0.38

The arguments


No
!!

These
objects are not
salient
!



Our algorithm works
on
images with salient
objects

only
!

The paradox of salient object
detection

But hey, what is a
“salient object”?

COMPOSITIONAL BIAS

Before we proceed…


Google Image Search: “science”


Rutherford atomic model (9)


Test tubes (10)


Microscopes (4)


D
ouble helix (3)


Old guys with crazy hair and glasses (3)

How to compose a biased salient
object detection dataset

Decide to build a new
salient object dataset!

So what is saliency?

Searching for unambiguous
examples of saliency…

Found one! Add to
my dataset!

Job done! Let other
people play with my
dataset!

The compositional bias


Compositional bias
: Biases introduced during
the
composition

of a dataset:


Exaggerating on stereotypical attributes.


Limited variability in positive samples.


Lack of negative samples at all.

Unlike datasets in
machine learning
, where
the dataset is the world, computer vision
datasets
are supposed
to be a representation
of the world
.

----

[
Torralba

and
Efros
: Unbiased look at Dataset bias]

Compositional bias: the statistics


Object number

Compositional bias: the statistics


Object eccentricity

Compositional bias: the statistics


Global foreground and background contrast

Compositional bias: the statistics


Local foreground/background contrast (contour strength)

TOWARDS A BETTER SALIENT
OBJECT DATASET

The new project


Build a
salient object detection

dataset from a
good
object detection
dataset (e.g. PASCAL VOC).

Let the eye fixations pick up those salient objects!

Data collection (in process)


SR Research
EyeLink

1000


2
-
sec viewing time.


“Free
-
viewing” instruction (will mention it
later).


3 subjects (more subjects on the way).

What makes an object salient


Unit conversion:


From fixation maps


To object fixation
score


sum
of blurred fixation map intensity within the object
mask.


Object size and saliency


Large objects attract
more fixations.



Small objects receive
denser fixations.

Object size and saliency

Objects, salient objects, and the most
salient objects


Salient objects:


Fixation score higher than
mean (67.3% objects).


Most salient objects:


Fixation score higher than
mean*2 (27.8% objects).

Image with fixation

Object labeling

Salient objects

Most salient object(s)

Salient objects and salient object
detection


Guess how does the algorithms perform on
“salient objects” and “most salient objects”?

On all objects:


FT: 0.28


GC: 0.39


SF: 0.35


PC: 0.38

Testing on salient objects

Salient objects on PASCAL VOC


FT: 0.22


GC: 0.35


SF: 0.31


PCA: 0.38


GC
-
seg
: 0.39

Testing on most salient objects

Most salient objects on PASCAL VOC


FT: 0.10


GC: 0.20


SF: 0.15


PCA: 0.26


GC
-
seg
: 0.23

Something is wrong,
seriously!

DISCUSSIONS

The role of saliency in a visual system


Bad performance because of boundary
detection?


Bad performance because of unpredictability
of human “free will”?

Saliency as an oracle


Oracle
selecting
the best segment


CPMC:
78% from 154 segments


gPB
: 61% from 1286 segments

* coverage = intersect/union

Saliency and tasks


Build a salient object detection dataset from
an
egocentric

object dataset.


Let the eye
-
fixation speaks

Eye Tracker

Forward
-
looking Camera

Learning to recognize daily actions using gaze,
[
Fathi

et. al. ECCV
12]

What makes an object salient?

Task

Object

Saliency


O
bject in egocentric actions


Fixated object ==
Manipulated object?

THANKS

Acknowledgement


Joint work with Yin Li @
Gatech
.





Special thanks to
Nathan
Faivre

for his kind help on
eye tracking.

Open discussions