Labannotation - MRL/ISR

natureplaygroundAI and Robotics

Nov 14, 2013 (3 years and 10 months ago)

64 views

1

PhD dissertation


Hadi
Aliakbarpour


Faculty of Science and Technology


October 2012, University
of Coimbra

2


This
dissertation investigate the problem of multi
-
sensor 3D data registration using
a network of IS
-
camera pairs
.




Target
applications:
Surveillance
, human
behaviour

modelling
, virtual
-
reality,
smart
-
room, health
-
care, games, teleconferencing, human
-
robot interaction,
medical
industries,
and scene and object understanding

3


Performing

3
D

data

registration

and

scene

reconstruction

using

a

set

of

planar

images

is

still

one

of

the

key

challenges

of

computer

vision
.



A

network

of

cameras,

whose

usage

and

ubiquitousness

have

been

increasing

in

the

last

decade,

can

provide

such

planar

images

from

different

views

of

the

scene
.



Recently,

IS

has

been

becoming

much

cheaper

and

more

available

so

that

nowadays

most

smart
-
phones

are

equipped

in

both

IS

and

camera

sensors
.

3
D

earth

cardinal

orientation

(North
-
East
-
Down)

is

one

of

the

outputs

of

an

IS
.





How

can

we

benefit

from

having

a

network

of

IS

and

camera

couples,

for

the

purpose

of

3
D

data

registration?

4

5


A homographic framework is developed for 3D data registration using a network of
cameras and inertial sensors. Geometric relations among different projective
image planes and Euclidean inertial planes involved in the framework are
explored. [AD12a] [AD11c] [AD10b] [AD11b] [AD10a] [AFKD10] [AFQ+11].



A real
-
time prototype of the framework is developed which is able to perform fully
reconstruction of human body (and objects) in a large scene. The real
-
time
characteristic is achieved by using a parallel processing architecture on a CUDA
-
enabled GP
-
GPU
[AAMD11].




A two
-
point
-
based method to estimate translations among virtual cameras in the
framework is proposed and verified [AD12a] [AD11a] [AD10a] [AFQ+11].



The uncertainties of the
homography

transformations involved in the framework
and their error propagations on the image planes and Euclidean planes have been
modelized

using statistical geometry.

6


Within the context of the proposed framework, a genetic algorithm is developed
to provide an optimal coverage of the camera network to a polygonal object (or a
scene
).



A method to estimate extrinsic parameters among camera and laser range finder is
developed [ANP+09]. A related
SLaRF
;
available to download at
http://isr.uc.pt
/~
hadi
is prepared.

7

8

9

10

A 3D point X is registered on different Euclidean planes using
homographies

11

A network of IS
-
camera couples
is used
to observe the scene from different views

13

Off
-
the
-
plane point Y is subject to parallax and on
-
the
-
plane point X with no
parralax

An exemplary case: a person is observed by three
cameras

Top
-
view of the registration plane. Area in white is the intersection to the
person.

14

Knowing the
heights of
two 3D points (
X
1 and
X
2) is sufficient to recover the translation (
t
)
among two cameras

15

16

17

18

The

uncertainties

of

the

homography

transformations

involved

in

the

framework

and

their

error

propagations

on

the

image

planes

and

Euclidean

planes

have

been

modelized

using

statistical

geometry
.


19


Uncertainty of point
μ

X , where mapped from
μ
ref

to
μ
’ :


Uncertainty of point
μ
(k)
X , where mapped from
μ
k
-
1

to
μ
k
:

20

The quality of reconstruction using a camera network
depends to mainly three parameters
:


1.
Number of
cameras


2.
The
quality of the applied background subtraction
technique


3.
The
cameras configurations (e.g. positions)


21

22

C
1

C
2

X

Y

{W}

ref
π
e
1

e
2

e
3

e
4

e
5

An exemplary convex polygon with 5 edges are observed by two camera. The problem is how to
arrange cameras to have optimum registration of the polygon with most completeness.

After registering with the present camera configuration: An extra part colored in red is
registered as a part of the object!

23

Solution: To use geometry (e.g. normal of the edges etc. ), define some cost
functions and applying GA.

11

Camera

LRF

Sensitivity to
Illumination

Very

high

NA

Occlusion

handling

Weak

Fair

Sensitivity
to
texture

High

NA

Precision

in
range sensing

Fair

Very good

Color sensing

Very good

NA

24

LRF is an active sensor which can be used as a complementary sensor to the cameras:

Comparison table







1
0
=
3
1
)
(
)
(
)
(
x
L
C
L
C
L
C
t
R
T



Estimation of the rigid transformation,
C
T

L(
α
)
, among a stereo camera and a LRF

25

26

First virtual plane

Second virtual plane

47’th virtual plane

Statue

Setup and scene

27

Empirical analysis the effects of IS noise to the translation estimation method

Input noise in
degrees

(roll, pitch
and yaw of inertial sensor)

Output uncertainty in cm (on three
elements of the estimated
translation vector)

Output
uncertainty
(cm)

Input noise (cm)

28

Output
uncertainty
(cm)

Input noise (pixels)

29


30

The
uncertainties for pixels
of the virtual camera’s image plane
are
demonstrated by
covariance
ellipses
, where they are scaled 1000 times for
clarity.

31

The

uncertainties

for

different

registered

points

on

the

Euclidean

inertial

plane,

demonstrated

by

covariance

ellipses
.

The

blue

and

red

ellipses

stand

for

points

registered

by

the

first

and

second

camera,

respectively
.

For

the

sake

of

clarity

the

covariance

values

are

scaled

500

and

600

times,

respectively

for

the

first

and

second

cameras

32

Uncertainties for an
exemplary
pixel
x = [ 450 450 1 ]
T

where
s = [
π
/2
-
π
/2

0
]
T

33

1200x1200 cm2

34







1
0
=
3
1
)
(
)
(
)
(
x
L
C
L
C
L
C
t
R
T



Reprojection of LRF data on the image
(blue points)

+

Result

Image

Range data

α

= 2
o

α

= 12
o

α

= 23.2
o

(during 6 months)

35

36


We investigated the use of IS for 3D data registration by using a network of cameras and
inertial sensors.



A
volumetric data registration algorithm was proposed.



Normally the volumetric reconstruction of a scene is time consuming due to the huge
amount of data to be processed. In order to achieve a real
-
time processing, a prototype
was built using GP
-
GPU and CUDA



A method to estimate the translation among cameras within the network was proposed.
The certainty of the method has been evaluated in the presence of different noise.



The issue of sensor configuration, particularly the cameras’ positions in the scene was
investigated and a geometric method to find an optimal configuration was proposed using
genetic algorithm.



A method to estimate the extrinsic parameters among camera and LRF was proposed as a
step towards applying range data in the framework.


Integration of range data within the proposed inertial
-
based data
registration
framework.



To develop a probabilistic algorithm for fusion of heterogeneous data, capable of dealing
with the
uncertainty of each
sensor node.



To investigate
a multi
-
layer 3D
tracking of human/objects. In
this future
investigation, we
will provide
contribution
to
model and
recognize the
state of
scene and to
analyse

the
behavior of small group
using probabilistic approaches.

37

38