1
PhD dissertation
Hadi
Aliakbarpour
Faculty of Science and Technology
October 2012, University
of Coimbra
2
This
dissertation investigate the problem of multi

sensor 3D data registration using
a network of IS

camera pairs
.
Target
applications:
Surveillance
, human
behaviour
modelling
, virtual

reality,
smart

room, health

care, games, teleconferencing, human

robot interaction,
medical
industries,
and scene and object understanding
3
Performing
3
D
data
registration
and
scene
reconstruction
using
a
set
of
planar
images
is
still
one
of
the
key
challenges
of
computer
vision
.
A
network
of
cameras,
whose
usage
and
ubiquitousness
have
been
increasing
in
the
last
decade,
can
provide
such
planar
images
from
different
views
of
the
scene
.
Recently,
IS
has
been
becoming
much
cheaper
and
more
available
so
that
nowadays
most
smart

phones
are
equipped
in
both
IS
and
camera
sensors
.
3
D
earth
cardinal
orientation
(North

East

Down)
is
one
of
the
outputs
of
an
IS
.
How
can
we
benefit
from
having
a
network
of
IS
and
camera
couples,
for
the
purpose
of
3
D
data
registration?
4
5
A homographic framework is developed for 3D data registration using a network of
cameras and inertial sensors. Geometric relations among different projective
image planes and Euclidean inertial planes involved in the framework are
explored. [AD12a] [AD11c] [AD10b] [AD11b] [AD10a] [AFKD10] [AFQ+11].
A real

time prototype of the framework is developed which is able to perform fully
reconstruction of human body (and objects) in a large scene. The real

time
characteristic is achieved by using a parallel processing architecture on a CUDA

enabled GP

GPU
[AAMD11].
A two

point

based method to estimate translations among virtual cameras in the
framework is proposed and verified [AD12a] [AD11a] [AD10a] [AFQ+11].
The uncertainties of the
homography
transformations involved in the framework
and their error propagations on the image planes and Euclidean planes have been
modelized
using statistical geometry.
6
Within the context of the proposed framework, a genetic algorithm is developed
to provide an optimal coverage of the camera network to a polygonal object (or a
scene
).
A method to estimate extrinsic parameters among camera and laser range finder is
developed [ANP+09]. A related
SLaRF
;
available to download at
http://isr.uc.pt
/~
hadi
is prepared.
7
8
9
10
A 3D point X is registered on different Euclidean planes using
homographies
11
A network of IS

camera couples
is used
to observe the scene from different views
13
Off

the

plane point Y is subject to parallax and on

the

plane point X with no
parralax
An exemplary case: a person is observed by three
cameras
Top

view of the registration plane. Area in white is the intersection to the
person.
14
Knowing the
heights of
two 3D points (
X
1 and
X
2) is sufficient to recover the translation (
t
)
among two cameras
15
16
17
18
The
uncertainties
of
the
homography
transformations
involved
in
the
framework
and
their
error
propagations
on
the
image
planes
and
Euclidean
planes
have
been
modelized
using
statistical
geometry
.
19
•
Uncertainty of point
μ
’
X , where mapped from
μ
ref
to
μ
’ :
•
Uncertainty of point
μ
(k)
X , where mapped from
μ
k

1
to
μ
k
:
20
The quality of reconstruction using a camera network
depends to mainly three parameters
:
1.
Number of
cameras
2.
The
quality of the applied background subtraction
technique
3.
The
cameras configurations (e.g. positions)
21
22
C
1
C
2
X
Y
{W}
ref
π
e
1
e
2
e
3
e
4
e
5
An exemplary convex polygon with 5 edges are observed by two camera. The problem is how to
arrange cameras to have optimum registration of the polygon with most completeness.
After registering with the present camera configuration: An extra part colored in red is
registered as a part of the object!
23
Solution: To use geometry (e.g. normal of the edges etc. ), define some cost
functions and applying GA.
11
Camera
LRF
Sensitivity to
Illumination
Very
high
NA
Occlusion
handling
Weak
Fair
Sensitivity
to
texture
High
NA
Precision
in
range sensing
Fair
Very good
Color sensing
Very good
NA
24
LRF is an active sensor which can be used as a complementary sensor to the cameras:
Comparison table
1
0
=
3
1
)
(
)
(
)
(
x
L
C
L
C
L
C
t
R
T
Estimation of the rigid transformation,
C
T
L(
α
)
, among a stereo camera and a LRF
25
26
First virtual plane
Second virtual plane
47’th virtual plane
Statue
Setup and scene
27
Empirical analysis the effects of IS noise to the translation estimation method
Input noise in
degrees
(roll, pitch
and yaw of inertial sensor)
Output uncertainty in cm (on three
elements of the estimated
translation vector)
Output
uncertainty
(cm)
Input noise (cm)
28
Output
uncertainty
(cm)
Input noise (pixels)
29
30
The
uncertainties for pixels
of the virtual camera’s image plane
are
demonstrated by
covariance
ellipses
, where they are scaled 1000 times for
clarity.
31
The
uncertainties
for
different
registered
points
on
the
Euclidean
inertial
plane,
demonstrated
by
covariance
ellipses
.
The
blue
and
red
ellipses
stand
for
points
registered
by
the
first
and
second
camera,
respectively
.
For
the
sake
of
clarity
the
covariance
values
are
scaled
500
and
600
times,
respectively
for
the
first
and
second
cameras
32
Uncertainties for an
exemplary
pixel
x = [ 450 450 1 ]
T
where
s = [
π
/2

π
/2
0
]
T
33
1200x1200 cm2
34
1
0
=
3
1
)
(
)
(
)
(
x
L
C
L
C
L
C
t
R
T
Reprojection of LRF data on the image
(blue points)
+
Result
Image
Range data
α
= 2
o
α
= 12
o
α
= 23.2
o
(during 6 months)
35
36
•
We investigated the use of IS for 3D data registration by using a network of cameras and
inertial sensors.
•
A
volumetric data registration algorithm was proposed.
•
Normally the volumetric reconstruction of a scene is time consuming due to the huge
amount of data to be processed. In order to achieve a real

time processing, a prototype
was built using GP

GPU and CUDA
•
A method to estimate the translation among cameras within the network was proposed.
The certainty of the method has been evaluated in the presence of different noise.
•
The issue of sensor configuration, particularly the cameras’ positions in the scene was
investigated and a geometric method to find an optimal configuration was proposed using
genetic algorithm.
•
A method to estimate the extrinsic parameters among camera and LRF was proposed as a
step towards applying range data in the framework.
Integration of range data within the proposed inertial

based data
registration
framework.
To develop a probabilistic algorithm for fusion of heterogeneous data, capable of dealing
with the
uncertainty of each
sensor node.
To investigate
a multi

layer 3D
tracking of human/objects. In
this future
investigation, we
will provide
contribution
to
model and
recognize the
state of
scene and to
analyse
the
behavior of small group
using probabilistic approaches.
37
38
Comments 0
Log in to post a comment