MB07 AIMS and 2006 PROGRESSES

electricfutureΤεχνίτη Νοημοσύνη και Ρομποτική

14 Νοε 2013 (πριν από 3 χρόνια και 9 μήνες)

75 εμφανίσεις

MB07


AIMS and 2006 PROGRES
SES

Y.Baudoin, Dir MB07/2006

1.

Introduction.



Ce projet comprend deux aspects: la mobilité autonome

(robotique mobile)
, greffée sur le projet NMRS de
l’EDA (
E2/EUROPA/B/0002
) et le proje
t européen VIEW
-
FINDER (
IST
-
045541
-
VIEW
-
F
INDER,

Approved by EC on 12 September 2006)
, tous deux concernant la multi
-
robotique en réseau et les missions
sécuritaires (assistance aux services de protection, type Protection Civile, Incendie, BFAST, IEDD, etc)



et la propulsion avancée, calquée sur
les résultats de recherche discutés depuis 1996 au sein des groupes de
travail OTAN DRG/LTSS et RTO/AVT (en cours AVT106)) ou RTO/IST (
NATO IST
-
058
).


La dynamique des véhicules terrestres non autonomes eu égard à la sécurité routière sera également
couve
rte, par l’intermédiaire de l’étude MRR03.



2.

EDA
-
NMRS
, VIEW
-
FINDER, Land
-
MOBINISS






L’étude baptisée ‘Networked Multirobotics Systems’ a été initialisée par la compagnie DHIEL BGT
Defence, impliquée de longue date dans les p
rojets de la défense allemande relatifs au développement de
mini
-
véhicules et véhicules de support au combat autonomes ou semi
-
autonomes (télé
-
opérés). Le
département a suivi ce type de projets, par une participation active au Panel 8 de la DRG (ancienne
o
rganisation de Recherche au sein des Forces Terrestres


à l’époque, AGARD en était l’équivalent pour la
mobilité aérienne) portant sur le développement d’interfaces Homme
-
Machine (ou Robot, ORI, HMI).

Le projet actuel propose d’abord le développement d’un
e simulation d’un réseau de robots fixes ou
mobiles, équipés de capteurs adéquats , affectés à des missions sécuritaires de type DIRTY (c
hemical,
nuclear, biological hazards, mine clearing
), DANGEROUS (
Dangerous:

reconnaissance of unknown or
hostile areas
)

ou DULL (
surveillance of secured areas
).


Les facteurs suivants nécessiteront une recherche soutenue et des résultats, que nous appli
querons en
priorité , mais non exclusivement au ROBUDEM




Le projet
View
-
Finder

(Sheffiel un
iversity, Univ Sapienza of Roma, DUTH, Galileo Avionica, IES, SAS

Belgium and RMA)
s’étalant sur 3 ans, épouse une philosophie équivalente mais implique un réseau de
communications

entre centre de crise et cellu
le de contrôle sur site opérationnel,

selon
le schéma suivant

:



COC

CRMC

COC/Rs
-
COC/Hs

COC/CRMC

2D

2D, 3D

PDAT



Guidance

Navigation

Control


Sensors


Signal Processing


Mission Planning/

Learning, Adaption


Man/Machine

Interface


Multi
-
System

Coordination


Pla
ttform

Power Supply

Motor and

Drive Train

Payload

Datalink/

Communication


‘The VIEW
-
FINDER system is composed of a minimum of two mobile robotic platforms equipped with
olfaction systems . The platforms can cooperatively search for chemical volatiles released from dangerous
devic
es, sharing information with neighbours inside a partially unknown environment . During the
exploration, each robot sends the gathered information about the environment to a high
-
level module,
which fuses all the collected information and uses the processe
d data to provide feedback to global group
controllers (COC and CRMC. Additionally, this high
-
level module provides a friendly user interface with
real
-
time results of the exploration and allows the user to specify missions for the system.’



Le
s

projet
s N
MRS
-

phase 1


et View
-
Finder
s’étaler
ont

sur

3 ans et reposeront pratiquement

sur les
WPs


de MB07 (
ou inversément
)
. V
-
F introduit aussi une coopération entre les départements
MECA, CISS
et MANAGEMENT


WP
-
MB07

WP


NMRS

WP
-
View
-
Finder

Libellé


respons
abilité

DEAD
-
LINES

T20

1

1

Scenario definition and requirements

Crisis Management Tools

Jan 2007
-
Jun 2007

Dec 2008

T20

2

2

Simulation environments for MRS

Jun 200
7
-
Jan 2010

T23/231
-
3

3

2

Guidance, navigation, and control of
individual robots in a mu
lti
-
robot
environment

Jun 2007
-
Jan 2010

T23/232
-
3

4

6

Surveillance Sensor and signal processing

Dec 2008

T23/233
-
4

5

4

Communications (CISS)

Jan 2010

T23/234

6

5

Human
-
Machine
-
Interface

Jun 2009

T24

7

2

Multi
-
robot coordination

Dec 2010



Sel
on la

destination, les WP sont assurés seuls (MECA) ou en coopé
ration avec les partenaires ERM
et hors ERM.


En annexes 1, 2 ,

3
et 4
figurent les contributions scientifiques majeures 2006.


3.

PUBLICATIONS
-

DISSEMINATION 2006



CoRoBa, a multi mobile robot control and simulation framework
,
Eric Colon
, Ph.D. Thesis,
November 2006, Vrije Universiteit Brussel
-

Koninklijke Militaire School



MoRoS3D, a multi mobile robot 3D simulator
,
Eric Colon
, Hichem Sahli,
Yvan Baudoin
,
Introductory Workshop to the CLAWAR 2006 conference,pp 722
-
728, Brussels, 11 September
2006.



A modular control architecture for semi
-
autonomous navigation
,
Daniela Doroftei
,

Eric Colon
,
Yvan Baudoin
, CLAWAR 2006 conference,pp 712
-
715, Brussels, 11 September 2006.



Development of a control architecture for the ROBUDEM outdoor mobile robot platform
,
Daniela
Doroftei
,

Eric Colon
,
Yvan Baudoin
, IARP Workshop RISE, Brussels, June 2006



Tele
-
robotics:Distributed Training
-
oriented Navigation Framework
, Thomas Geerinck,
Eric
Colon
, Sid Ahmed Berrabah, Kenny Cauwerts,Hanene Bahri, Hichem Sahli, IARP Workshop
RISE, Brussels, June 2006



CoRoBa, a multi mobile robot control and simu
lation framework
,
E. Colon
, Y. Shali,
Y.
Baudoin
, Special Issue on "Software Development and Integration in Robotics" of the
International Journal on Advanced Robotics, pp 73
-
78,Volume 3, Number 1, March 2006.
ARS
Special Issue on Software Development and Integration in Robotics



Organisation of IARP Workshop

RISE’2006 (June 2006, RMA


Y.Baudoin
), Organisation of 9
th

Int Symposium CLAWAR’2006

(S
ep 2006, RMA


Y.Baudoin, D.Lefeber
-
VUB)



OIC
-
R³D² : un curriculum complet relatif à la mécatronique et robotique de déminage humanitaire

a également été achevé en 2006:
www.iit.bme.hu/oicr3d2

(projet financé p
ar EUR DG Formation,
géré sous Patrimoine) avec la coopération du CISS/SIC (X.Neyt)



Participation au groupe OTAN IST058_24

(
Military Applications for Multi
-
Robot Systems):
E.Colon



Annexe 1.

WP T20

CoRoBA, a Framework for Multi
-
Sensor Robotic Systems Int
egration

Cdt Dr Ir Colon

1.
Introduction

Research in robotics has moved from a limited capability single robot to multi
-
robot systems equipped with
a plethora of sensors, leading to de
-
facto multi
-
processor and distributed applications. Because of the lack

of standards, ad hoc solutions are generally developed resulting in difficulties to extend and reuse existing
software. In order to increase efficiency of future developments, tools for integrating multi
-
sensor robotic
systems are required.


This work

pr
oposes a solution to this problem with a framework called CoRoBA (Controlling Robot with
CORBA). CoRoBA is made up of component based execution units. It comes with a 3D simulation
application and utility programs for distributing and managing the live and

run cycle of multi
-
process
applications.


2. Design



The design of the framework proposes an answer to the requirements that have been identified for the
targeted family of applications. Most of them have been derived from Robot Control Patterns, an ori
ginal
contribution of this work, while others have been based on general software development considerations.
Robot Control Patterns provides a clear view of operations performed in typical multi
-
sensor robotics
applications like telecontrol, supervised an
d autonomous control or multi
-
robot systems, to name a few.


3.
Implementation


The implementation of the framework is based on several Design Patterns that make the design flexible,
elegant and ultimately reusable. The elementary brick in CoRoBA is a compo
nent. Components are
independent execution units and have separated interfaces for the configuration and the actual functionality
they provide. According to the classical control theory, components are divided in three categories,
Sensors, Processors and A
ctuators. They form a chain along which information is transferred and like in
classic control schemes, the data flow is unidirectional. Sensors read data from external devices and
transmit them to other components. Processors process received data and for
ward results to components
linked to output devices, which are called Actuators. This division provides a clear view of the
functionality of each component and consequently facilitates their reuse in new applications.







In this framework, communicatio
n between components relies on the industry standard CORBA. CORBA
has been selected because of its language and platform independence. Using such a standard simplifies the
development and improves the interoperability with existing software. CORBA is actua
lly a specification
of the Object Management Group and the TAO (The ACE ORB) implementation has been chosen among
others because it is an open source, efficient and standards
-
compliant real
-
time implementation of CORBA.
The framework offers two different c
ommunication mechanisms; the first one is based on classical
synchronous communication while the second relies on Events. In this mode, components exchange data by
Sensor

Processor

Actuator

pushing Events through Event Channels that can be seen as pipes connecting suppliers and con
sumers of
Events. Event based communication increases the flexibility of an application by decreasing the coupling
between components.


Components are multi
-
threaded and posses different running modes for the transmission of events, namely
PERIODIC, SYNCHR
O and TRIGGER. In the PERIODIC mode, components produce events at regular
time intervals. In the SYNCHRO mode, new output events are produced by the component when an input
event is received and in the TRIGGER mode, an external signal must be received in o
rder to process or
produce an event. The availability of different modes increases the flexibility of the framework, each mode
being actually useful in different contexts.


4.
Simulation


Having a simulator offers many advantages. First of all it is tremend
ously cheaper than real robots and
sensors, particularly when experimenting with multi robots systems. It allows focusing on intelligence and
control and disposing of other, less interesting problems. It makes possible reducing the development time
by tryi
ng different scenarios and algorithms before experimenting them in a real environment. A simulator
also increases safety when developing and testing new control applications. For these reasons a Java based
3D multi robot simulator has been developed. Java

provides comprehensive Application Programming
Interfaces to communicate with CORBA objects and a 3D library (Java3D) for the modelling and rendering
of virtual worlds.






The robot simulator is responsible for the realistic motion of the robot by usin
g geometric, kinematic and
dynamic models, and takes care of the collision with fixed and moving obstacles like other robots. The
simulated sensors produce measurement data that are injected in the application control loop. The software
integrates seamless
ly with the components of CoRoBA because all robots and sensors have a CORBA
interface. The utilisation philosophy is to develop and tune control algorithms in simulation and to simply
replace simulated by real components once satisfying results have been
reached, no further modification of
the Processor components being required. Furthermore, as the simulator is totally independent from
CoRoBA, it can be used without it and in this case, robots can be controlled by programs written in any
language having a

CORBA mapping. On the other way, another simulator with a CORBA interface could
be used with the framework.





5.
Validation and evaluation


The framework has been theoretically and practically validated. The theoretically validation consists in two
part
s: comparaison of the implementation with the definition of a framework and review of the
requirements that have been met by the framework. Several distributed control applications have been
implemented in order to practically validate the framework. Share
d control and autonomous navigation
applications involving different robots have been successfully tested in simulation.


The applications that have been developed are representative of what the framework is good for and
allowed us to validate its functio
nality and modularity. It has been shown that applications can be built
incrementally by recycling existing components. Components developed in simple applications can be
reused without any modifications in similar or more elaborated ones. Different applic
ations illustrate the
Robot Control Patterns: direct control, teleoperation, shared and autonomous navigation with Fuzzy logic
engines and Behaviours arbitration mechanisms. All these applications demonstrate that CoRoBA provides
enough flexibility to deve
lop a large panel of control architectures. Development of multi
-
robots
applications, distributed simulation and real robots and sensors has also been addressed.



A qualitative and quantitative evaluation of the framework demonstrates that the proposed s
olution is
efficient, usable and stable. In the qualitative evaluation it is explained how existing applications can be
improved by using CoRoBA. A comparison between CoRoBA and other frameworks like MCA, DCA,
GeNoM,... shows that CoRoBA combines the advan
tages and capabilities of most of them while avoiding
their drawbacks. The framework also reduces the development time in comparaison with raw CORBA
programming. It has been verified that

the components work as they should and that particularly, o
rder of

events are preserved, all sent events are received and that operations are executed in the right order. On the
other hand, for the qualitative evaluation we have defined and applied evaluation criteria and measures of
effectiveness for evaluating performa
nce of the framework and the simulator.



6.
Conclusions


T
he design, implementation and evaluation of a multi
-
sensor robotic control framework has been
performed and typical components used in robotic applications have been developed to validate the
frame
work.










Annexe 2. WP T23
-
231


A behaviour
-
based control and software architecture for the visually
guided Robudem outdoor mobile robot

Ir
Daniela Dorofteï

1.

Abstract


Humanitarian demining still is a highly labor
-
intensive and high
-
risk operation.

Ad
vanced sensors and mechanical aids can significantly reduce the demining time. In this

context, it is the aim to develop a humanitarian demining mobile robot which is able to

scan a minefield semi
-
automatically. This requires the careful consideration and

integration of multiple aspects: sensors and sensor data fusion, design of a control and

software architecture, design of a path planning algorithm and robot control. This report

describes partial aspects of this research work. It focuses mainly on three m
ain aspects of

the design process: visual sensing using stereo and image motion, design of a behaviourbased control
architecture and implementation of a modular software architecture.

Note
: the Humanitarian demining is a particular case of a Risky Interven
tion. Other sesnors than metal
detector may be used, for instance chemical sensors, as it will be the case in the View
-
Finder Project


2.

Introduction


The application for this research project is to prepare the ROBUDEM robot, as shown on

Figure 1, for an
out
door humanitarian demining task. In this setup, the robot navigates

and searches for mines by moving
and sensing with the metal detector for suspicious

objects in the soil. Once a suspicious object is detected,
the robot stops and invokes its

Cartesian sca
nning mechanism. This will make a 2D scan of the soil,
allowing mine

imaging tools to make a reliable classification of the suspicious object as a mine or not.



Figure 1: ROBUDEM robot with scanning mechanism


The designated prospect for this research pr
oject is to implement an existing cognitive

behaviour
-
based
approach for mobile robot navigation on a mobile robotic platform. This

will allow the robot to scan a
suspected minefield semi
-
autonomously and return a map

with locations of suspected mines. The

development of such an intelligent mobile robotrequires consideration of different side
-
aspects.

Robots use
sensors to perceive the environment. Sensors under consideration for this

research work are ultrasonic
sensors, a laser range scanner, a stereo cam
era system, an

inertial measurement system, a GPS receiver and
of course a metal detector. All but the

last one of these sensors return positional and perceptual information
about the

surroundings. This sensor data has to be fused in a correct way to form
a coherent

“image” of the
environment. Hence the need for an intelligent sensor fusion algorithm to

combine the often erratic,
incomplete and conflicting readings received by the different

sensors, to form a reliable model of the
surroundings. Sensor fusio
n has been subject to a

lot of research [1][4], most of the proposed methods use
Kalman Filtering [17] and

Bayesian reasoning [15]. However, in recent years, there has been a tendency to
make

more and more use of soft computing techniques such as artificia
l neural networks [8]

and fuzzy
logic for dealing with sensor fusion. [11][6].

This report will focalize completely on the visual sensors
(cameras).

An autonomous mobile agent needs to reason with perceptual and positional data in order

to navigate safely
in a complex human
-
centered environment with multiple dynamic

objects. This
translation of sensory data into motor commands is handled by the robot

navigation controller.

3

Its design is closely related to the design of the control architecture which descr
ibes the

general strategy for
combining the different building blocks. The basis for this reasoning

process is often a map, which
represents a model of the environment. These maps can be

simple grid maps, topological maps [7], or
integrated methods [16]. T
he used path

planning technique depends highly upon the type of map chosen
before. A survey of

different methods can be found in [5]. The goal of this research is to use a
behaviourbased

control architecture to navigate while modeling (mapping) the environ
ment in 3

dimensions, using vision as a primary sensing modality.The control architecture has to be translated into a
software architecture which manages

the building blocks on a software level. This software architecture has
to provide the

flexibility of
modular design while retaining a thorough structure, enabling an easy

design
process. All the different processes (sensor measurements, measurement

processing, sensor fusion, map
building, path planning, task execution …) must be

coordinated in an efficien
t way in order to allow
accomplish a higher goal [9]. A number

of control strategies can be set up, varying from simple serial
sense
-
model
-
plan
-
act

strategies to complex hybrid methods. A discussion of some of these control
strategies

can be found in [13].

An interesting approach here, is to use fuzzy behaviours, partially

overriding each other, to build up complex navigation plans, as discussed in [10][12].

This research work
aims at implementing such a hybrid control strategy.

During the design of all the
se sub
-
aspects, the outdoor
nature of the robot has to be taken

into account. Outdoor robots face special difficulties compared to their
indoor

counterparts. These include totally uncontrolled environments, changing illumination,

thermal, wind
and solar co
nditions, uneven and tough terrain, rain, …


3.

Visual Sensing


3.1.

Introduction


Visual sensing is important to many robotic applications, including visually guided

mobile robots. When
observing biological examples such as humans, it can be noted that

almost all

navigational tasks are based
upon visual sensory input (the eyes). The aim to

match or even outperform this biological vision system has
driven an abundance of

research in the field of image processing, ranging from low
-
level colour tracking
and

segmentat
ion, motion analysis, optical flow and stereo estimation and activity detection

over mid
-
level
multi
-
view scene reconstruction techniques and human behaviour

detection to high
-
level context
-
aware
visual processing techniques.

The Robudem robot is outfitted

with a stereo camera system, consisting of
two digital

cameras. In order to maximize the information stream towards the navigation unit, two

different visual processing techniques

are used
: stereo vision and image motion analysis.



3.2.

Stereo Vision


Stereo
vision employs the difference in location between two cameras. This difference in

space leads to two
images where the same points can be found at different positions. The

goal of stereo disparity estimation is
finding the correct correspondences between im
age

points from the left and right camera. For this, we
employ the algorithm presented by

Birchfield et al. in [2]. The algorithm matches individual pixels in
corresponding scanline

pairs while allowing occluded pixels to remain unmatched, then propagates
the

information between scanlines by means of a fast postprocessor. The algorithm handles

large untextured
regions, uses a measure of pixel dissimilarity that is insensitive to image

sampling, and prunes bad search
nodes to increase the speed of dynamic pr
ogramming.

The computation is relatively fast, taking about 1.5
microseconds per pixel per disparity

on a workstation. This allows for near real
-
time stereo processing the
camera images at

about 6 frames per second, which is an acceptable speed, given the
dynamics of the robot.

The output of this algorithm is a dense depth map of the area in front of the cameras, as

shown on Figure 2.
On the depth map in Figure 2, nearby objects appear dark. The cross

on top marks the location of the
closest obstacle, which

is the darkest point on the depth

map and which corresponds here to the corner of
the couch on the lower left side.




Figure 2 : Stereo Processing


a) Left camera image; b) Right camera image; c) Dense depth map (white=far, dark = near)


The data
-
conten
t of this dense depth map must now be reduced to be useful for the

navigation controller. It
is the aim of this project to conceive an integrated simultaneous

localization and 3D mapping process
(SLAM3D) into the framework, but this is not

instantiated ye
t. As such, the dense depth map is
downprojected onto the ground surface,

such that it can be represented as a 2D line. This data is further
reduced in dimensionality

by calculating from the depth line the distance to the nearest obstacle on the left
dl
, i
n the

middle
dc
, and on the right
dr
, respectively. This data reduction is illustrated by Figure 3.





Figure 3 : Depth map data reduction: Top: Depth map (white=near, dark = far);

Bottom: Depth line with nearest distances to obstacles on the left, in t
he middle and on the right



3.3.

Image Motion Analysis


Motion analysis can provide extra information about the environment. The rationale

behind the usage of the
image motion for navigation purposes is that when large image

motion is detected, this is likely
due to
objects close to the camera (and thus close to the

robot), which should trigger the obstacle avoidance
module. On the other hand, when few

image motion is detected, this means that the way in front of the
camera is probably quite

clear of obstacles.

Multiple techniques stand at our disposal to estimate the image
motion. These methods

differ in their approach towards the main problem to be solved in image motion:
the

background estimation and subtraction process. As the camera system is installed on a

moving robot
system, background estimation is particularly difficult in this case, as it is

very hard to build up a model of
the background over a large amount of time. This

constraint limits the use of advanced background
estimation techniques like kerne
l

density estimators, mean shift or mixtures of Gaussians. As a result, the
frame difference

between successive frames was employed to find back the moving objects.



The threshold is chosen according to the robot velocity.



With
c
a constant describing

the relation between robot speed and image motion. Figure

4 shows this image motion
field as calculated by the right robot camera.




Figure 4: Image motion field of right camera: a) Image at ti
-
1; b) Image at ti; c) Image motion field


The resulting mot
ion field is then summed over the whole image to obtain one single

motion measure for
the camera image.


This calculation is performed once for the left camera and once for the right camera

image, leading to two
distinct image motion estimates.


4.

Behaviour
-
based Control Architecture


4.1.

General architecture


The control architecture describes the strategy to combine the three main capabilities of

an intelligent
mobile agent: sensing, reasoning (intelligence) and actuation. These three

capabilities have to be
integrated
in a coherent framework in order for the mobile agent to

perform a certain task adequately.

The working
principle of the proposed control architecture is sketched on Figure 5. There

are three distinctive modules
to be discriminated: Navigation (
on the right side on Figure

5), Mine Detection
-

Scanning (in the middle
on Figure 5) and Metal Detection (on the

left side on Figure 5). These three processes are controlled by a
watchdog, the robot

motion scheduler, which manages the execution of each mo
dule and decides on the

commands to be sent to the robot actuators. This robot motion scheduler is explained

more in detail in
Figure 6.




Fig 5.


This general scheme is now discussed more in detail for each of the three modules.


1. Navigation

Different

Sensors provide input for a Simultaneous Localization and Mapping module

Sensors:

-

GPS (Global Positionment System) gives absolute coordinates

-

IMS (Inertial Measurement System) gives acceleration (and speed and position by

integration)

-

US (Ultrasonic

sensors) give distance measurements to obstacles

-

IR (Infrared sensors) give distance measurements to obstacles

-

LASER gives line 3D data

-

Mine Sensor: The mine imaging module will return locations of mines, which

have to be represented on
the map and
which are of course obstacles themselves

As the SLAM module works with a global map, it
doesn’t have to re
-
calculate the whole

map from scratch every time, but the map can just be iterated to
improve the different

estimates, hence the loopback arrow. The S
LAM module outputs a global map with

obstacles and also with mines, thanks to the input from the mine imaging module. This

map is used by the
navigation module to calculate a safe path. The safe path is given as

an input to the robot motion scheduler
which

will transform it into a motor command and

execute it, unless another module has a higher priority
task (and trajectory) to perform.


2. Mine Detection

The Cartesian scanning mechanism makes a 2D scan with the metal detector. Mine

imaging tools
determine
the likelihood of mine occurrence and the exact position of

eventual mines. If a mine is found,
this will be reported to the robot motion scheduler,

which will take the appropriative actions (Stop
Scanning Metal Detection, Invoke motion

control based on th
e trajectory calculated by the navigation
module, Start Metal Detector,

see Figure 6). In addition to this the Mine detector acts as a sensor for the
SLAMalgorithm,

as it will return the locations of mines, which have to be represented on the

map and which

are of course obstacles themselves.


3. Metal Detection

The metal detector scans for metal in the soil. If no metal is found, it keeps on doing this

and the robot
keeps on moving. If a metal is found, this will be reported to the robot

motion scheduler, w
hich will take
the appropriative actions (Stop Metal Detection, Move

robot backwards a little, Stop robot, Start Scanning
Metal Detection, see Figure 6).


4.2. Robot motion scheduler


The robot motion scheduler (Figure 6) needs to arbitrate which of the mo
dules is

executed and which of
them can influence the robot actuators through robot commands.

Therefore, there are two main paths
through the robot scheduler, one for the (normal)

situation of exploring while avoiding obstacles and while
detecting metals a
nd one for the

situation where a metal is found and more thorough investigation is needed
(mine

detection) while the robot is standing still.


Fig 6.



Normal situation:


-

In an initial situation (default inputs), or when the “no mine found” or “mine

fou
nd” trigger is given, scanning metal detection is turned off;

-

The Navigation module gives at all time instances a safe path and trajectory, as

this module loops infinitely without interaction with the other modules;

-

This Trajectory is set as the trajec
tory to be executed, but with a low priority;

-

The Metal detector module is activated.


Mine detection :


-

If the “metal found” trigger is given, the metal detector is switched off;

-

The trajectory for the robot is set to a predefined movement, more spe
cifically, to

back off a little. This is done to be able to centre the scanning metal detection

better around the suspicious object. This trajectory has a high priority ;

-

When this movement is completed, the robot is halted, by giving a “no

movement” tra
jectory with a high priority;

-

Finally, the scanning metal detection module is activated.


4.3

Behaviour
-
based navigation framework


The general control scheme was translated into a behaviour based robot controlarchitecture. This option
was chosen because of
the flexible and modular nature of

behaviour
-
based controllers, facilitating the
design process. This document concentrates

fully on the development of a behaviour
-
based architecture for
the robot navigation and

path planning, as last years report concentr
ated already on the integration of the
metal

detector in the architecture and the implementation of the scanning metal detection. This

means that
from the global architecture, as shown in Figure 5, only the Navigation

module will be further discussed
from
here on.

In behavior
-
based robotics the control of a robot is shared between a set of purposive

perception
-
action units, called
behaviors
. Based on selective sensory information, each

behavior produces
immediate
reactions
to control the robot with respect
to a particular

objective, a narrow aspect of the
robot’s overall task such as obstacle avoidance or wall

following. Behaviors with different and possibly
incommensurable objectives may

produce conflicting actions that are seemingly irreconcilable. Thus a
major issue in the

design of behavior
-
based control systems is the formulation of effective mechanisms for

coordination of the behaviors’ activities into strategies for rational and coherent behavior.

This is known as
the
action selection problem
(also ref
ereed to as the behavior

coordination or behavior fusion problem).
Numerous action selection mechanisms have

been proposed over the last decade. A qualitative overview of
these approaches was

compiled in the course of this research project [3].

The main ad
vantage of using
behaviour
-
based approaches is that each of the different

behaviours can be designed and implemented
separately. For the Robudem robot, three

main sensing capabilities are installed for now: odometry, stereo
vision, image motion

analysis. T
hese three sensing capabilities are processed by separate behaviors. In this

context, the odometry information is used for guiding the robot to a given goal position.

The general
scheme of the behaviour
-
based navigation module is shown on Figure 7.



Fig
7.


The output of this behaviour
-
based navigation planner is a velocity
V
and turn angle
T

command to be sent
to the robot. How these properties are estimated is explained in detail

in the following section.


4.4


Behaviour design and fusion


The velocity
V
an
d turn angle
T
to be sent to the robot are dependent on the individual

input of each of the
different behaviours: goal seeking, stereo vision and image motion.

Therefore, each of these behaviours
calculates its own velocity and turn angle for the

robot to
be performed. These prescripts are then weighed
according to the activity level
A

of the specific behaviour according to the formulation of equation 1.4. The
activity level

of a behaviour describes to which degree this behaviour is relevant for the calcula
tion of

the final robot command
.





When the Emergency Stop behaviour is activated, no robot movement is possible, which

is a security
measure. The activity level for the front and rear steering behaviour decides

on the drive mode which will
be adopted
by the robot. The velocity and turn prescripts

are calculated at behaviour level as follows:


Stereo vision

The stereo vision behaviour receives as data
d
l

and
d
r
, the distances to obstacles on the left and right
side of the robot. The smaller the distance

to obstacles, the more careful and slow the robot must move.
The velocity is therefore directly proportional to the mean of the measured distances left and right.


2
l r
S S
d d
V c



(
0
.
1
)

With
c
S

a normalisation constant.

When the distance to obstacles on the left side is larger than the distance to obstacles on the right side
,
the robot shoul
d avoid these obstacles on the right by turning to the left
.
This relation between robot
turn angle and distances to obstacles is expressed by equation 1.6.




S S l r
T c d d
 

(
0
.
2
)



Image motion


The image motion behaviour receives as data
m
l

and
m
r
, the movement measured by the left and the
right camera. The more movement in the image, the
more probable there are objects close to the robot,
so the velocity should be lowered. The robot speed is as such inversely proportional to the image
motion, as expressed by equation 1.7.


1
2
l r
M M
m m
V c

 

(
0
.
3
)

With
c
M

a normalisation constant.

When the
movement

on the left side is larger than the
movement

on the right side
, the robot should
avoid these probable obstacles on the left by turning to the left
.
This relation between robot turn angle
and the movement in the images is expressed by equation 1.8.




1
M M l r
T c m m
  

(
0
.
4
)


Goal seeking


The goal seeking behaviour receives as data the estimate of the robot position


,
R R
x y

and
orientation
R


as calculated by the odometry and robot kinematics. This position and orientation is
compared to the desired goal position


,
G G
x y

and orientation
G

.

When the distance to the goal is
small, the rob
ot is approaching its goal and should drive slowly, so the speed is proportional to the
distance the goal.







2 2
,
P P d R G R G
V c x x y y
   

(
0
.
5
)


With
c
P,d

a normalisation constant.

According to the same reasoning, the robot orientation should be aligned with the
goal orientation, so the turning angle is in direct relation to the difference between
these t
wo angles, as expressed by equation 1.10.




,
P P R G
T c

 
 

(
0
.
6
)

With
c
P,


a normalisat
ion constant.


Now the velocity and turning prescripts
of the
individual

behaviours are known, they need to be fused.
Therefore, the activity levels need to be calculated. As noted before, these activity levels should reflect the
relevance of the specific
behaviour. The principle behind the calculation of the activity levels is that the
output of a behaviour should be stable over time in order to trust it. Therefore, the degree of relevance or
activity is calculated by observing the history of the output
-

a velocity and turn angle
-

of each behaviour.
This history
-
analysis is performed by comparing the current output to a running average of previous
outputs, which leads to a standard deviation, which is then normalized. For the stereo vision behaviour,
thes
e standard deviations are:


2
,
1
,,
2
,
1
,,
N
S j
i
j
S V V S k
k i h
N
S j
i
j
S T T S k
k i h
V
c V
N
T
c T
N



 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 





(
0
.
7
)

With
c
V

and
c
T


two
normalisation con
stant
s
.


The bigger this standard deviation, the more unstable the output values of the behaviour are, so the less
they can be trusted. This leads to an estimate for the activity level:






,,
1 1
S S V S T
a
 
   

(
0
.
8
)


For stability reasons, the activity level is initialized at a certain value (in general 0.5) and then iteratively
estimated using suc
cessive overrelaxation. This leads to the following equations describing the estimation
of the activity levels for the stereo vision, image motion and the goal seeking behaviour:







2 2
,,
1 1
,1,,,
2 2
,,
1 1
,1,,,
1 2
1 2
N N
S j S j
i i
j j
S i S i V S k T S k
k i h k i h
N N
M j M j
i i
j j
M i M i V M k T M k
k i h k i h
V T
A A c V c T
N N
V T
A A c V c T
N N
 
 
 

   
 

   
 
   
 
   
 
   
      
 
   
 
   
   
 
 
   
   
   
      
   
   
   
 
 
 
 


2 2
,,
1 1
,1,,,
1 2
N N
P j P j
i i
j j
P i P i V P k T P k
k i h k i h
V T
A A c V c T
N N
 
 

   
 
 
 
 
 
 
 
 
   
 
   
 
   
      
 
   
 
   
   
 
 
 
 

(
0
.
9
)

Here,


is the extrapolation factor for successive overrelaxation.


5.

Software Architecture


5.1.
MCA


As control architectures which aim to mimic human thinking risk of becoming highly complex, the choice
of a flexible, extendable and real
-
time capable software a
rchitecture is very important. This software
architecture has to ease the use of reusable and transferable software components. The chosen software
architecture, MCA (Modular Controller Architecture) [14] achieves this by employing simple modules with
stan
dardized interfaces. They are connected via data transporting edges which is how the communication
between the single parts of the entire controller architecture is managed. The main programs only consist of
constructing modules that are connected via edge
s and pooled into a group. This results in an equal
programming on all system levels. As modules can be integrated both on Windows, Linux and on RT
-
Linux without changes, they can be developed on Linux
-
side and then transferred later to RT
-
Linux. As
errors

in RT
-
Linux lead to system
-
hangs this development strategy prevents from many reboot cycles and
results in faster software development.

Each MCA module is determined by four connectors with the outside world: Sensor input (left below),
Sensor output (left

top), Control Input (right top), Control Output (right below). As a result sensor data
streams up, control commands stream down. The Sensor input and output are connected through a Sense
procedure which enables to process the sensor data and the Control i
nput and output are connected through
a Control procedure which enables to process the control commands. Sensor data flow is shown in yellow,
control command flow in red.


This modular structure is particularly convenient for behaviour
-
based architectures
as the individual
behaviours translate easily to corresponding MCA
-
modules.


5.2.
Proposed architecture: the global scheme

The

proposed global MCA software architecture, as it is depicted on
Figure
8

consists of three

main
groups:

o
ne for sensor
-
guided robot control
, o
ne for
s
canning metal detection

and o
ne for metal detection
.
The robot motion scheduler controls which of the
three

groups is executed and with which parameters
.
Each group consists of several modules and/or subgroups
.

As last years’ report concentrated on the
implementation of the rightmost metal detection and scanning metal detection groups, the focus will now
be entirely on the implementation of the navigation group on the left.





Sensor Output
Controller Input
Robot Motion Scheduler
Scanner
Control
Metal Detector
Acquisition
Metal Detector
Processing
SLAM
Behavior-based Navigation
US
Processing
GPS
Processing
Visual
Processing
LASER
Processing
Wheel Encoder
Processing
Robot Motion
Control
US
Acquisition
GPS
Acquisition
Visual
Acquisition
LASER
Acquisition
Wheel Encoder
Acquisition
Wheel Encoder
Setting
Metal Detector
Processing
Metal Detector
Control
Metal Detector
Acquisition
Figure
8
: MCA scheme

of the global software architecture

Figure
1
: MCA s
cheme of the glo
bal software architecture

5.3.
Proposed architecture: th
e navigation group


As the navigation group is implemented now, it only receives sensory input via the abstract visual sensors
(stereo vision and image motion analysis) and by the robot odometry. On
Figure
2
, the robot interface
s
ubgroup was not loaded, so the only sensor input (yellow lines) is coming from the StereoVisionSystem
group which will be discussed later and which hold all the visual acquisition and processing algorithms.
This sensor data is streamed to a EvaluationUnit
group, which will perform some postprocessing on the
stereo data and to the RobudemBehaviours group. This subgroup holds the modules for the behaviour
-
based navigational controller and will be discussed in a following section. The navigation controller
dis
tillates control commands from its sensor inputs and sends these to the robot. The MCAGUIBridge
group is a generic group collecting sensory data from all subgroups to be displayed on a graphical user
interface (GUI). In the same manner, it organizes for co
ntrol commands from the user on the GUI to be sent
to the different subgroups.



Figure
2
: The general scheme of the navigation controller

.


5.4. Proposed architecture: the visual sensing subgroup


The starting point of the visua
l sensing subgroup on
Figure
3

is the StereoFrameGrabberGroup which is a
subgroup containing the drivers and acquisition infrastructure for the stereo camera system. From here,
sensor data (i.e. camera images) stream upwards, fol
lowing in essence three main paths. One path is
towards the Movement analysis module which will calculate the image motion. Another path is towards the
disparity map generator, which will output a dense depth map of the environment. This disparity estimati
on
however requires the images to be rectified before they can be processed, which is performed by the
UndistortImages module. A third option is to record the images for analysis and debugging purposes.



Figure
3
: MCA scheme for

the visual sensing subgroup

5.5.
Proposed architecture: the behaviour
-
based controller subgroup


Figure
4

shows the MCA scheme for the behaviour
-
based controller subgroup. It consists of seven
behaviours, controlled (fused) by
a BehaviourSelector module.



Figure
4
: MCA scheme for the behaviour
-
based controller subgroup


The sensory input received by the behaviour
-
based controller subgroup consists of:



From stereo vision:
d
l
,
dc, d
r



From image motion
analysis:
m
l
,

m
r



From odometry:


,
R R
x y

and
R


This data will be processed by the stereo vision behaviour according to equations 1.5 and 1.6, for the image
motion behaviour by equations 1.7 and 1.8 and for the g
oal seeking behaviour by equations 1.9 and 1.10.

The BehaviourSelector module receives as input the output of the different behaviours and will calculate
the activity levels for each of these behaviours according to the equations 1.13.

All of the three mai
n behaviours also send the velocity prescript which was calculated to the Velocity
behaviour, where a fusion of the data will occur, using the activity levels, calculated by the
BehaviourSelector module, as described by equation 1.4. The EmergencyStop beha
viour, which can be
triggered externally by the user or internally when an obstacle is detected within a designated security
distance, ensures that the fused velocity command is only transferred to the robot in safe conditions,
otherwise the robot will rec
eive a zero velocity prescript.

For the turning behaviour, a similar approach is followed; only in this case there are two separate fusion
behaviours the main path planning behaviours can send their results to. This is related to the mechanical
structure o
f the Robudem robot, which has a two
-
by
-
two differential drive system, meaning front and back
wheels can be given a different turning angle, allowing for highly flexible maneuvering on difficult terrain.
These different drive modes are:



Car
-
like: no turnin
g on back wheels, only turning on front wheels: default mode



Shopping
-
cart
-
like: no turning on front wheels, only turning on back wheels



Crab
-
like: same angle turning on front and back wheels



Assisted car
-
like: turning on front wheels, inverted turning on
back wheels

It is our aim to let the behavioural controller (BehaviourSelector) decide which is the best drive mode given
the terrain circumstances, by setting the activity levels for the FrontSteering and RearSteering behaviours.
To achieve this, more res
earch is necessary to analyse the relation between outdoor terrain features and the
optimal drive mode. For now, the user can select on the graphical user interface the desired drive mode and
the activity levels
A
F

and
A
R

will be set accordingly. The Front
Steering and RearSteering behaviour will
then fuse the velocity prescripts from the path planning behaviours according to equations 1.4 and pass this
as a control output to the robot.


6. Results


The state and achievements of this research project can be
discussed by taking a look at the Graphical User
Interface (GUI) in
Figure
5

which was developed for the robot control in its current state. As can be
noticed, the interface is still very cluttered and research oriented, but it s
hows the important features of the
Robudem controller. On the upper left, the stereo camera images are visible, which serve for now as the
only sensor inputs for the robot. These stereo camera images are rectified to facilitate the calculation of the
depth

map. The rectified images are displayed underneath the camera images and below them, the dense
depth map can be found. The dense depth map is then postprocessed, as is shown on the images to the right
of the original depth map. In the middle of the interf
ace, the measurements of the abstract visual sensors
stereo vision and image motion analysis are shown using colored bars. These indicate for the stereo vision
part on the left:



The distance to obstacles on the left



The distance to obstacles in the middle



The distance to obstacles on the right

For the image motion sensor, these bars indicate:



The motion in the left camera image



The mean of the motion in the left and right camera



The motion in the right camera image


At the right of the interface, the activi
ty levels of the different behaviours are shown using colored bars. As
can be notices, the BehaviourSelector has decided in this case that the ImageMotion behaviour delivers
more reliable results than the stereo vision behaviour. This is in this case due t
o the lack of texture in the
camera images, which renders the dense depth map estimation less robust. As discussed before, the activity
levels for the VelocitySteering, FrontSteering and RearSteering are user
-
decided by selecting the drive
mode. In this ca
se, crab
-
like drive mode was selected on the GUI, resulting in activity levels of 1 for the
FrontSteering and RearSteering behaviour.




Figure
5
: Graphical User Interface of the Robudem navigation controller



7. Conclusions & F
uture research

In this report, the development of a control scheme for a semi
-
autonomous mobile robot for humanitarian
demining was discussed. Some aspects of this control scheme, like the scanning metal detection, were
already implemented before. This rep
ort focused on the implementation of the visual sensing capabilities,
the design of a behaviour
-
based navigation controller and its implementation into a modular software
architecture. Other aspects of the global control scheme, like mapping and navigation
, are still under
research.

For the future, we will concentrate on the improvement extension of the behavior
-
based path planning
controller as this navigation controller is an essential subsystem of the autonomous robot. Several
improvements to the curren
t navigation controller are still necessary to improve the robustness of the whole
system. These include:



Tuning of the different parameters of the behaviours for optimal speed and reliability by doing
extensive field tests



Optimizing the behaviour fusion
process using more advanced fusion techniques.



Decentralized processing of the different modules using multiple networked PC’s. For now, the
whole framework is being run on one single laptop computer. As the visual processing modules
require a lot of proce
ssing time, this system is constantly stressed to the maximum, which has
negative effects on system stability.



Integration of more types of sensors. In particular for the goal seeking behaviour, a more reliable
sensing method than odometry is absolutely ne
cessary. A GPS system is under investigation for
this purpose.



Another problem arising when developing behavior
-
based navigation controllers is how to
introduce a form of reflexive model
-
based reasoning into the reactive behavior
-
based architecture.
This p
roblem will be tackled by defining different levels of control, as sketched on
Figure
6
. We
do this in order to preserve the speed of the behavior
-
based reactive controller, while joining it
with a reflexive model
-
based controlle
r (which will inherently have a longer control loop).


Model
-
Based
Navigation
Controller
Simultaneous
Localization
And
Mapping
Sensors
(IR,
US
, Laser,
Vision
,

)
Behavior
-
Based
Navigation
Controller
Actuators
Robot
Task

Figure
6
: Reflexive / Reactive behaviour
-
based navigation controller



A reflexive map
-
based navigation controller (MBNC), which is able to interact with a reactive
behavio
r
-
based navigation controller (BBNC), will be investigated. The model
-
based controller
requires input from a simultaneous localization and mapping (SLAM) module and is therefore
much slower to react to changing environmental conditions. Through the BBNC, t
he sensors and
actuators could be linked in a fast control loop.

References


[1] R. R. Brooks and S.S. Iyengar. Multi
-
Sensor Fusion. Prentice Hall, New Jersey, USA, 1998

[2] S. Birchfield, C. Tomasi, “Depth Discontinuities by Pixel
-
to
-
Pixel Stereo”,
Inter
national Conference on Computer
Vision
, pp. 1073
-
1080, Bombay, India, 1998.

[3] D. Doroftei, “Behavior based Robot Navigation Techniques


State of the Art” Technical report & Lecture Notes
of The Royal Military Academy, Brussels, Belgium, October 2006

[4
] T. D. Larsen. Optimal Fusion of Sensors, Technical University of Denmark, Department of Automation, Lyngby,
September 1998

[5] J.C. Latombe, J.
-
C., Robot Motion Planning, Kluwer Academic Publishers, 1991.

[6] M. Lopez, F.J. Rodriguez, J.C. Corredra, Fu
zzy Reasoning for Multisensor Management, 1995 IEEE International
Conference on Systems, Man, and Cybernetics, Vol. 2, pp. 1398
-
1403, October 1995, New York, USA

[7] K. Nagatani, H. Choset, S. Thrun, Towards exact localization without explicit localizatio
n with the generalized
Voronoi graph. Carnegie Mellon University, USA

[8] R. R. Murphy. Sensor Fusion. In Handbook of Brain Theory and Neural Networks, Colorado School of Mines,
Dept. of Mathematical and Computer Sciences

[9] J. Budenske and M. Gini. Ach
ieving Goals Through Interaction with Sensors and Actuators. Department of
Computer Science, University of Minnesota, Minneapolis, USA, 1992

[10] A. Saffiotti, K. Konolige, E. H. Ruspini, A Multivalued Logic Approach to Integrating Planning and Control,
A
rtificial Intelligence Center, Menlo Park, Califonia, USA, 1995

[11] G. De Cubber, H. Sahli, F. Decroos, Sensor integration on a mobile robot, ICMCR02, International Conference on
Measurement and Control in Robotics, pp 89
-
92, Bourges, France, 2002

[12] H
. Schäfer, K. Berns,
RAVON
-

An autonomous Vehicle for Risky Intervention and Surveillance,
International
Workshop on Robotics for risky intervention and environmental surveillance


RISE, 2006

[13] G.G. Schenkel, Memory Management, Navigation and Map Buil
ding. Universität Stuttgart, Stuttgart, Germany,
1994

[14] K.U. School, J. Albiez, B. Gassmann, MCA


An Expandable Modular Controller Architecture,
Forschungszentrum Informatik an der Universitaet Karlsruhe (FZI), Interactive Diagnosis and Service System
s

[15] P. Sykacek, I. Rezek. Markov chain Monte Carlo methods for Bayesian sensor fusion. Kluwer Academic
Publishers, 2000

[16] S. Thrun and A. Bücken. Learning Maps for Indoor Mobile Robot Navigation. School of Computer Science,
Carnegie Mellon Universit
y, Pittsburgh, USA, April 1996

[17] G. Welch, G. Bishop. An Introduction to the Kalman Filter. University of North Carolina, Department of
Computer Science, Chapel Hill, USA, 1996

Annexe 3. WP T23
-
232


GPS

: préliminaires


JC Habumuremyi (Jan
-
Jun 2006)


1.

Le
s tâches préliminaires


Le programme RxControl se trouvant sur les CD fournis par la firme Septentrio, a été

installé sur l’ordinateur portable ASUS en Windows et en Linux. Ce programme permet

de communiquer avec le PolaRx au moyen des interfaces graphique
s.


2.

Travaux accomplis


Les programmes suivants ont été écrits :


-

Traitement des données brutes fournies sous le format NMEA GPGGA. Ces données

ont été obtenues en
utilisant le programme RxControl. Le traitement « offline » avait

pour but d’extraire le te
mps auquel les
mesures ont été prises, la longitude, la latitude,

l’altitude de l’antenne.De ces données extraites, une
conversion a été faite afin

d’avoir les coordonnées cartésiennes de ces différentes mesures. Les
coordonnées de

ces points pouvaient alo
rs être représentées en utilisant « matlab » (exemple à la

Figure 1 où on a effectué plusieurs déplacements le long d’une ligne avec l’antenne)

ou « gnuplot ». Ce
programme (qui devra être testé davantage afin de trouver des

« bugs ») se trouve sur l’ordin
ateur portable
ASUS dans le directoire (sous Linux)

suivant : /home/sharedWork/gpsProg/prog1/progGga.c.



Figure 1 : conversion « offline » des données NMEA GPGGA collectées sur le PolaRx en

coordonnées cartésiennes


Ce programme pourra être utilisé pour
l’étude de la cinématique (directe et inverse) du

robot Robudem et
établir des comparaisons avec des modèles obtenus avec

les encodeurs.

-

Traitement des données fournies
sous le format NMEA GPGGA au fur et à mesure de

leur arrivée. J’ai esquissé l’écritur
e de ce programme
que vous pouvez trouver dans le

directoire de l’ordinateur portable ASUS (sous Linux) suivant :

/home/sharedWork/gpsProg/gpsRt. Il faudra pour cela savoir envoyer au moyen

d’une commande les
données sous le format NMEA GPGGA et l’ordinate
ur ASUS

ou le PC104 les lira sur le port spécifié
(Série ou TCP). J’ai constaté lors des tests que

l’utilisation des commandes PolaRx en Linux n’envoi
e

pas
correctement le format

voulu (il y a des bruits). En windows, je

n’ai pas eu ce problème. Donc je
co
nfigurais au moyen des commandes le PolaRx en

windows puis j’utilisais Linux pour lire les données et
les traiter. Le programme est

encore à ses débuts (ce n’est qu’une esquisse). J’ai également commencé
l’écriture

des modules MCA que vous pouvez retrouver

dans le directoire

/home/sharedWork/mca2File/gpsRtGrp. Il faudra beaucoup d’études afin de bien

maîtriser la contrainte de
temps. L’utilisation du TCP améliorera encore le

programme au point de vue temps de réponse (il ne faudra
pas perdre de vue que dans

le cas du DGPS et du RTK la transmission des corrections se feront souvent par

ondes radios car certaines antennes restent fixes. Dans ce cas, je pense que la

connexion wireless TCP/IP
pourrait être appropriée). Donc il serait mieux de réécrire

ce program
me en utilisant les « sockets ».

I
l reste
encore

beaucoup à faire pour le traitement des données des PolaRx en temps. Il faudra aussi penser à
comment

réagir quand on perd la réception du signal et comment fusionner le GPS avec

d’autres capteurs.















































Annexe 4.
T23
-
232


External
preparatory

Contribution (VUB
-
RMA)

Multi
-
View Reconstruction for 3D Structure and

Motion Estimation


Geert De Cubber

3rd February 2007



1 An Introduction to Multi
-
View Reconstruction


Many m
ethods have been considered for reconstruction from several views. One

way to proceed is to
progressively reconstruct the scene, using three
-
view or

two
-
view techniques. Such a method may be
applied to any image sequence.

There are me
thods that can be used

in specifi
c circumstances.

The task of reconstruction becomes easier if we are able to apply a simpler

camera model, known as the
a±ne camera. Thi
s camera model is a fair approx
imation to perspective projection whenever the distance to
the scene is large

compared with the diff
erence in depth between the back and front of the scene.

If a set of points are visible in all of a set of
n
views involving an affine cam
era, then a well
-
known
algorithm, the factorization algorithm, as introduced

by Tomasi and Kanad
e in [8], can be used to compute
both the structure of

the scene and the specifi
c camera models in one step using the Singular Value

Decomposition. This algorithm is very reliable and simple to implement. Its

main dffi
culties are the use of
the a±ne camer
a

model, rather than a full pro
jective model, and the requirement that all the points be visible
in all views.

This method has been extended to projective cameras in a method known as

projective
factorization [7]. Although this method is generally satisfact
ory, it

can not be proven to converge to the
correct solution in all cases. Besides, it

also requires all points to be visible in all images.

Other methods for
n
-
view reconstruction involve various assumptions, such as

knowledge of four coplanar points in
the world
visible in all views [6], or six or

seven points that are visible in all images in the sequence. Methods that
apply

to specifi
c motion sequences, such as linear motion, planar motion or single axis

(turntable) motion
have also been developed.

The

dominant methodology for the general reconstruction problem is bundle

adjustment [9]. This is an iterative me
thod, in which one attempts to fi
t anon
-
linear model to the measured
data (the

point correspondences). The ad
vantage of bundle
-
adjustment is that
it is a very general method
that may be

applied to a wide range of reconstruction and optimization problems. It may be

implemented in such a way that the discovered solution is the Maximum Like
-
lihood solution to the
problem, that is a solution that is in
some sense optimal

in terms of a model for the inaccuracies of image
measurements. Unfortunately,

bundle adjustment is an iterative process, whi
ch can not be guaranteed to
con
verge to the optimal solution from an arbitrary starting point. Much research

in
reconstruction methods
seeks easily computable non
-
optimal solutions that

can be used as a starting point for bundle adjustment.
An excellent survey of

these methods is given in [9]. A common impression is that bundle
-
adjustment

is necessarily a slow techn
ique, but when implemented carefully, it can be quite

e
ffic
ient.

All these
techniques generally start from point o
r line correspondences over mul
tiple frames. It is necessary to have
correspondences over more than 2 frames if

one wants to stitch multiple v
iew
s together. Therefore, we will
fi
rst discuss the

case of three
-
view reconstruction and then show how to extend the structure

estimation
process to multiple frames.



2 Three
-
view reconstruction
:
Three View Geometry


In this section, we consider the case

of 3D reconstruction form three views.

Whereas for two views, the
basic algebraic entity is the fundamental matrix, for

three views this role is played by the trifocal tensor.
The trifocal tensor is a

3
X

3
X

3 array of numbers that relate the coordinates
of corresponding points

or lines in three views. Just as the fundamental matrix is determined by the

two camera matrices, and
determines them up to projective transformation, so

in three views, the trifocal tensor is determined by the
three camera matrices
,

and in turn determines them, again up to projective transformation. Thus,

the trifocal tensor encapsulates the relative projective geometry of the three

cameras.

There are several
ways that the trifocal tensor may be approached. Here, the

method followed

by Hartley in [3] is taken over.
This approach goes out form the

incidence relationship of three corresponding lines, a
s shown in fi
gure 1.



Figure 1: A line in 3
-
space is imaged as the corresponding triplet l,l
0
,l
00
in

three views indicated by their c
entres, C,C
0
,C
00
, and image planes. Conversely,

corresponding lines back
-
projected from the
fi
rst, second and third images all

intersect in a single 3D line in space. [3]




Next Model development: on request (available by
yvan.baudoin@rma.ac.be
)

or http://mecatron.rma.ac.be

Other contributions:

Feature Detection and Matching for 3D Structure and Motion Estimation (Geert De Cubber


3 feb 2007)

Epipolar Reconstruction for 3D Structure and

Motion Estimation

(Gee
rt De Cubber


3 feb 2007)




References

[1] Sebastien Cornou, Michel Dhome, and Patrick Sayd. Bundle adjustment: a

fast method with weak
initialisation. In
British Machine Vision Conference
,

2002.

[2] Andrew W. Fitzgibbon and Andrew Zi
sserman. Automatic c
amera recov
ery for closed or open image
sequences. In
5th European Conference on

Computer Vision
, volume 1, pages 311{326, Freiburg,
Germany, June 1998.

[3] Richard Hartley and Andrew Zisserman.
Multip
le View Geometry in Com
puter Vision
. Cambridge
University Press, New York, NY, USA, second

edition, 2004.