Joint Histogram Based Cost Aggregation

yakzephyrAI and Robotics

Nov 24, 2013 (3 years and 8 months ago)

87 views

Joint Histogram Based Cost Aggregation

for Stereo Matching
-

TPAMI 2013

M.S. Student,
Hee
-
Jong Hong

Sep 24, 2013

Dongbo Min,
Member, IEEE
,

Jiangbo Lu,
Member, IEEE
,

Minh N. Do,
Senior Member, IEEE


Introduction


Related

Works


Proposed Method


:

Improve Cost

Aggregation


Experimental

Results


Conclusion


Outline

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

2

Introduction

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

3


Goal

Perform efficient cost aggregation.


Solution : Joint

histogram

+

reduce

redundancy



Advantage : Low complexity

but

keep

high
-
quality.





Cost Initialization
Cost Aggregation
Refinement

70~75%


20~25%


5%

Related Works

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

4


Complexity

of

aggregation

O(NBL)



Reduce

complexity

approach


Scale

image

: Multi Scale Approach

D
. Min and K.
Sohn
, “Cost aggregation and occlusion handling with WLS in stereo
matching,” IEEE Trans. on Image Processing, 2008
.



Bilateral

filter

: Bilateral Approximation

C
.
Richardt
, D. Orr, I. P. Davies, A.
Criminisi
, and N. A. Dodgson, “Real
-
time spatiotemporal
stereo matching using the dual
-
cross
-

bilateral grid,” in European Conf. on Computer Vision,
2010

S
. Paris and F. Durand, “A fast approximation of the bilateral filter using a signal processing
approach,” International Journal of Computer Vision, 2009
.



Guided

filter

: Run in constant time =>
O(NL
)

C.Rhemann,A.Hosni,M.Bleyer,C.Rother,andM.
Gelautz
,“Fast cost
-
volume filtering for
visual correspondence and beyond,” in IEEE Conf. on Computer Vision and Pattern
Recognition,
2011





N
: all pixels
(
W*H
)

B
: window size

L
: disparity level

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

5

Proposed
Method


Local Method Algorithm

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

6


Cost
initialization : Truncated
Absolute Difference



=>


Cost
aggregation : Weighted
filter





Disparity
computation : Winner
take all




[4,8]

Improve Cost

Aggregation

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

7


New formulation for aggregation


Remove normalization


Joint histogram
representation



Compact representation for search range


Reduce disparity levels



Spatial sampling of matching window


Regularly sampled neighboring pixels


Pixel
-
independent sampling


New formulation for aggregation

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

8


Remove normalization




=>



Joint histogram
representation


Compact Search Range

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

9


Cost aggregation



=>




M
C
(q)

a subset of disparity levels whose size is
D
c
.



O( NBD )

O( NBD
c
)

N
: all pixels
(
W*H
)

B
: window size

D
: disparity level

Compact Search Range

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

10


Non
-
occluded region of ‘Teddy’ image



D
c

=
60

Final Accuracy =
93.7%


D
c

= 6

Final Accuracy = 94.1
%


D
c

=
5
(
Best
)

Final Accuracy

= 94.2%

Spatial Sampling of Matching Window

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

11


Reason


A large matching window and a well
-
defined weighting function leads to high
complexity.


Pixels should aggregate in the
same object
,
NOT

in the window
.


Solution


Color segmentation =>
Time consuming (Heavy)


Spatial
Sampling
=>
Easy
but
powerful


Regular Sampling => Independent from reference pixel => Reduce Complexity


Spatial Sampling of Matching Window

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

12


Cost aggregation




=>




S

: sampling ratio



O( NBD
c
)

O( NBD
c
/ S
2
)

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

13


Parameter definition

N
: size
of
image

B
: size
of matching
window



N(p)
=
W
×
W

M
D
: disparity levels



size=
D

M
C
:
The subset of
disparity



size=
D
C
<<
D

S
:
Sampling
ratio

Pre
-
procseeing

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

14

Experimental Result


Experimental

Results

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

15


Pre
-
processing


5*5 Box filter


Post
-
processing


Cross
-
checking technique


Weighted

median filter (WMF)


Device

Intel Xeon 2.8
-
GHz CPU (using a single core only)
and a 6
-
GB RAM


Parameter

setting


( ) = (1.5, 1.7, 31*31, 0.11, 13.5, 2.0)


Experimental

Results

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

16


(a)

(b)

(c)

(d)

Experimental

Results

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

17


Using too large box windows (7
×
7, 9
×
9) deteriorates the quality,
and incurs more computational overhead.



Pre
-
filtering can be seen as the
first cost aggregation
step and
serves the
removal of noise
.


Experimental

Results

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

18

Fig. 5. Performance evaluation: average percent (%) of
bad matching pixels for ‘nonocc’, ‘all’ and ‘disc’ region
s according to Dc and S.


2 better than 1

The smaller
S, the better

Experimental

Results

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

19

The smaller
S, the longer

The
bigger
Dc
, the longer

Experimental

Results

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

20



APBP :
A
verage
P
ercentage of
B
ad
P
ixels

Experimental

Results

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

21


Ground truth

Error maps

Results

Original images

Experimental

Results

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

22

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

23

Conclusion


Conclusion

Joint Histogram Based
Cost Aggregation for
Stereo
Matching
-

TPAMI 2013

24


Contribution


Re
-
formulate the problem with

the relaxed joint histogram.


Reduce the complexity of the joint histogram
-
based aggregation.


Achieved both accuracy and efficiency.



Future

work


More

elaborate algorithms for selecting the subset of label


hypotheses.


Estimate the optimal number
Dc

adaptively.


Extend

the

method

to

an

optical

flow

estimation.



25

Thank you!