Context-Aware Saliency Detection

soilflippantΤεχνίτη Νοημοσύνη και Ρομποτική

17 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

118 εμφανίσεις


Introduction


Principles of context


aware saliency


Detection of context


aware saliency


Result


Application


Conclusion


How to describe a figure / picture ?



Description:


What most people think is
Important

or
Salient
.


First glance: human attention.



EX: auto focusing.


Dominant object



EX: object recognition/segmentation.


context of the dominant
objects
:

image classification, summarization of a photo
collection,
thumb nailing,
and
retargeting.



Salient regions are
distinctive with respect to
both their local
and global
surroundings
.



Prioritize
regions close to the foci of
attention.
-
> Maintains the
background
texture.


(Gestalt Law)



Retargeting



Summarization


Principles
for context
-
aware
saliency



Algorithm



Applicability


1. Local low
-
level considerations, including
factors
such as
contrast and color.



2
. Global considerations, which
suppress
frequently occurring features
, while
maintaining features that
deviate from
the
norm
.


3. Visual organization rules, which state that
visual forms may possess one or several
centers of gravity about which the form is
organized.



4
. High
-
level factors, such as human faces.



D.Walther

and C.
Koch.

Modeling
attention to salient
protoobjects
.

[2006]



X
.
Hou

and L.
Zhang.

Saliency
detection: A spectral
residual approach.

[2007]



T
. Liu, J. Sun, N.
Zheng
, X. Tang, and H.
Shum.

Learning to Detect
A Salient Object
.

[2007]


Areas that
have distinctive colors or patterns
should obtain
high saliency.


--

P1


frequently
-
occurring features should be
suppressed
.





--

P2


The
salient pixels should be
grouped together
,
and not spread all over the image
.

--

P3


Consider
a single
patch of
scale r at each pixel.
Thus, a pixel i is considered
salient if
the
appearance of the patch pi centered at pixel i
is
distinctive with
respect to all other image
patches.


Euclidean distance between the
vectorized

patches



and



in CIE L*a*b
color space.


𝑐


,


=
(





)
2
+
(





)
2
+
(





)
2


Euclidean distance between the positions
of
patches



and


.


This dissimilarity
measure is
proportional
to the difference in appearance and
inverse
proportional
to the positional distance.





,


=

𝑐
(


,


)
1
+



  
(


,


)


For
every patch pi, we search for the K most
similar patches
{


}

=
1
𝐾

in
the
image.



𝑆


=
1

exp

{

1
𝐾


(



,



)
𝐾

=
1
}


Background
pixels (patches
) are likely to have
similar patches at
multiple scales.



Incorporating
multiple scales to further
decrease
the saliency
of background pixels,
improving the contrast
between salient
and
non
-
salient regions.


For a patch pi of scale r, we consider as
candidate neighbors all
the patches in the
image.


We choose the K most similar
patchs

to
compute saliency.


𝑆


=
1

exp

{

1



(



,



𝑘
)
𝐾

=
1
}


The saliency at pixel i is taken as the mean of
its saliency at different scales:


𝑆


=
1


𝑆




𝑅


Gestalt laws:



visual
forms may possess one or several
centers
of
gravity about which the form is
organized.



1.

A pixel is
considered attended
if its saliency
value exceeds a certain
threshold.



2.

each pixel outside the attended areas is
weighted
according to
its Euclidean distance
to the closest
attended pixel.


𝑆


=
𝑆


(
1


𝑓𝑐
(
𝑖
)
)


Enhancement factors:


1.Recognized objects

2.Face detection


3 cases:


Images show
a single salient object
over an
uninteresting
background.


Images
where the
immediate surroundings
of the salient object shed light
on
the story
the image tells
.


Images
of complex scenes
.


Compare method:


D.Walther

and C. Koch.

Modeling
attention to salient
protoobjects
.


[2006]


X.
Hou

and L. Zhang.

Saliency
detection: A spectral
residual Approach.

[2007]



Image retargeting



Summarization through collage creation


Resizing
an image by
expanding or
shrinking
the non
-
informative
regions.



Seam carving

M. Rubinstein, A. Shamir, and S.
Avidan
.

Improved seam carving
for video retargeting
.

[2008]


Context
-
aware saliency



The salient objects
as well as informative
pieces of the
background should
be
maintained in summaries.

S.
Goferman
, A. Tal, and L.
Zelnik
-
Manor
.

Puzzle
-
like
collage
.


[2010]


3 stages:



Compute the saliency maps for images.


Extracts regions
-
of
-
interest by considering both saliency and
image edge information.


Assemble non
-
rectangular
ROIs.


Propose a new type of saliency:


context
-
aware saliency



Evaluate in 2 applications:


retargeting

summarization



Thanks for attention!