Context-Aware Saliency Detection

soilflippantAI and Robotics

Nov 17, 2013 (3 years and 10 months ago)

137 views


Introduction


Principles of context


aware saliency


Detection of context


aware saliency


Result


Application


Conclusion


How to describe a figure / picture ?



Description:


What most people think is
Important

or
Salient
.


First glance: human attention.



EX: auto focusing.


Dominant object



EX: object recognition/segmentation.


context of the dominant
objects
:

image classification, summarization of a photo
collection,
thumb nailing,
and
retargeting.



Salient regions are
distinctive with respect to
both their local
and global
surroundings
.



Prioritize
regions close to the foci of
attention.
-
> Maintains the
background
texture.


(Gestalt Law)



Retargeting



Summarization


Principles
for context
-
aware
saliency



Algorithm



Applicability


1. Local low
-
level considerations, including
factors
such as
contrast and color.



2
. Global considerations, which
suppress
frequently occurring features
, while
maintaining features that
deviate from
the
norm
.


3. Visual organization rules, which state that
visual forms may possess one or several
centers of gravity about which the form is
organized.



4
. High
-
level factors, such as human faces.



D.Walther

and C.
Koch.

Modeling
attention to salient
protoobjects
.

[2006]



X
.
Hou

and L.
Zhang.

Saliency
detection: A spectral
residual approach.

[2007]



T
. Liu, J. Sun, N.
Zheng
, X. Tang, and H.
Shum.

Learning to Detect
A Salient Object
.

[2007]


Areas that
have distinctive colors or patterns
should obtain
high saliency.


--

P1


frequently
-
occurring features should be
suppressed
.





--

P2


The
salient pixels should be
grouped together
,
and not spread all over the image
.

--

P3


Consider
a single
patch of
scale r at each pixel.
Thus, a pixel i is considered
salient if
the
appearance of the patch pi centered at pixel i
is
distinctive with
respect to all other image
patches.


Euclidean distance between the
vectorized

patches



and



in CIE L*a*b
color space.


𝑐


,


=
(





)
2
+
(





)
2
+
(





)
2


Euclidean distance between the positions
of
patches



and


.


This dissimilarity
measure is
proportional
to the difference in appearance and
inverse
proportional
to the positional distance.





,


=

𝑐
(


,


)
1
+



  
(


,


)


For
every patch pi, we search for the K most
similar patches
{


}

=
1
𝐾

in
the
image.



𝑆


=
1

exp

{

1
𝐾


(



,



)
𝐾

=
1
}


Background
pixels (patches
) are likely to have
similar patches at
multiple scales.



Incorporating
multiple scales to further
decrease
the saliency
of background pixels,
improving the contrast
between salient
and
non
-
salient regions.


For a patch pi of scale r, we consider as
candidate neighbors all
the patches in the
image.


We choose the K most similar
patchs

to
compute saliency.


𝑆


=
1

exp

{

1



(



,



𝑘
)
𝐾

=
1
}


The saliency at pixel i is taken as the mean of
its saliency at different scales:


𝑆


=
1


𝑆




𝑅


Gestalt laws:



visual
forms may possess one or several
centers
of
gravity about which the form is
organized.



1.

A pixel is
considered attended
if its saliency
value exceeds a certain
threshold.



2.

each pixel outside the attended areas is
weighted
according to
its Euclidean distance
to the closest
attended pixel.


𝑆


=
𝑆


(
1


𝑓𝑐
(
𝑖
)
)


Enhancement factors:


1.Recognized objects

2.Face detection


3 cases:


Images show
a single salient object
over an
uninteresting
background.


Images
where the
immediate surroundings
of the salient object shed light
on
the story
the image tells
.


Images
of complex scenes
.


Compare method:


D.Walther

and C. Koch.

Modeling
attention to salient
protoobjects
.


[2006]


X.
Hou

and L. Zhang.

Saliency
detection: A spectral
residual Approach.

[2007]



Image retargeting



Summarization through collage creation


Resizing
an image by
expanding or
shrinking
the non
-
informative
regions.



Seam carving

M. Rubinstein, A. Shamir, and S.
Avidan
.

Improved seam carving
for video retargeting
.

[2008]


Context
-
aware saliency



The salient objects
as well as informative
pieces of the
background should
be
maintained in summaries.

S.
Goferman
, A. Tal, and L.
Zelnik
-
Manor
.

Puzzle
-
like
collage
.


[2010]


3 stages:



Compute the saliency maps for images.


Extracts regions
-
of
-
interest by considering both saliency and
image edge information.


Assemble non
-
rectangular
ROIs.


Propose a new type of saliency:


context
-
aware saliency



Evaluate in 2 applications:


retargeting

summarization



Thanks for attention!