Structure and Motion

Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Online

Structure and Motion

for

General Camera Models



Gerald Schweighofer


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Motivation: Robots


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Motivation: Robots


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Motivation: User Guidance


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Agenda


Robust Pose



Online SaM


SaM for GCM


Online


Robust


Experiments


Feature Generation


Simulations


Real Sequenzes


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Robust Pose


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Robust Pose


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Robust Pose


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Robust Pose


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Robust Pose


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Robust Pose

4th order polynomial


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Results


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Results


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Results


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Online SaM for GCM



General Camera Model


Object space cost



Globally Convergent SaM


SaM as an optimization problem


closed form solutions for structure and camera position


Proven convergence


Online/Realtime SaM


constant amount of CPU time / frame


Robustness



Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

General Camera Model


Measurements are rays of light (c,v)


c ... a point


v ... a vector

Camera

v

c


An example


Stereo Setup






Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Object Space Cost for GCM


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Structure & Motion


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Closed form Solution for Structure


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Closed form Solution for Camera
translation


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Iterative Rotation estimation


Solve one iteration using SVD or quaternions


Results in a globally convergent algorithm.

closed form structure closed form translation


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Convergence


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Online/Realtime SaM


Assumption: old Frames stay constant.

pre
-
calculate


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Online/Realtime SaM


Assumption: old Frames stay constant.

pre
-
calculate


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Robustness


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Robustness


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Experiments


Simulation



Natural Landmarks


Artificial Landmarks



House Sequence


Bridging Marker less Environment


Laboratory Sequence



Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Simulation

Object: Cylinder

70 random points

diameter: 1 meter

height: 1 meter


Motion: Circle

diameter: 6 meter

100 frames / every 3.6
°


0.5 Pixel Gaussian noise



Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Simulation

10 ms := 100 Frames / sec



for SaM


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Simulation


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Artficial Landmarks



ARToolKit Marker


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Natural Landmark Tracking


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

House Sequence


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

House Sequence


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

House Sequence


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

House Sequence


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Bridging Marker less Environments


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Automatic Generation of Scene
Description


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Automatic Generation of Scene
Description



Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Automatic Generation of Scene
Description

standard deviation: 0.48 %


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Conclusion


Robust Pose



Structure and Motion for GCM


General Camera Model


Online Algorithm


Robust to Outliers



Experiments



Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Publications

Institute of Electrical Measurement and Measurement Signal Processing


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing



Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing



Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Natural Landmark Tracking


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Simulation


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Time Complexity


Gerald Schweighofer

RIGOROSUM 03.07.2008


Online SaM for GCM

Institute of Electrical Measurement and Measurement Signal Processing

Time Complexity

Frames

Points

Frames to optimize

affected Points

classical algorithms

proposed algorithms