Algebraic methods in Computer Vision

coatiarfAI and Robotics

Oct 17, 2013 (3 years and 9 months ago)

79 views

Center for Machine Perception
Department of Cybernetics, Faculty of Electrical Engineering
Czech Technical University in Prague
Algebraic methods in Computer Vision
Zuzana Kukelova, Tomas Pajdla, Martin Bujnak
2/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Motivation

Many problems in computer vision, robotics and other fields lead to
systems of polynomial equations

Camera calibration

Relative/Absolute pose estimation



















3/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague



SfM






Tracking
Motivation

3D reconstruction






Recognition
Require to solve camera calibration, relative/absolute pose estimation
4/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Motivation

Systems of equations are not trivial - many equations, many
unknowns, high degree







Contaminated input measurements => problems have to be solved
for many different inputs (RANSAC)

High or even real-time performance


















x
1
x
2

5/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Motivation

General algorithms → slow → not feasible




To solve these systems of polynomial equations special algorithms
have to be designed to achieve numerical robustness and
computational efficiency

Not general

Fast


















6/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague




One polynomial equation in one unknown
- Companion matrix
1 equation in 1 variable
→ companion matrix
→ eigenvalues
... a simple rule
Eigenvalues = solutions
7/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


System of linear polynomial equations
- Gauss elimination

System of n linear equations in n unknowns









We can represent this system in a matrix form
Perform Gauss-elimination → reduces the matrix to a triangular form
backsubstitution
8/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


System of m polynomial equations in n
unknowns

A system of m polynomial equations in n variables
with coefficients from




Our goal is to solve this system

Many different methods

Numerical methods

Symbolic methods

Grőbner basis methods

Resultant methods
9/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


System of m nonlinear polynomial equations in
n unknowns - Grőbner basis method

System of m equations in n unknowns




Idea

construct a new system of polynomial equations with the same
solutions as the initial one, but with a simpler structure

Grőbner bases

have the same solutions but are easier to solve

10/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


System of m nonlinear polynomial equations in
n unknowns - Grőbner basis method

The reality is much harder






Computation of Grőbner bases

is a NP-complete in m, n (takes very long time to compute, needs
huge space to remember intermediate results)

Fortunately in some cases, Grőbner bases can be obtained easier


Equations → Grőbner basis
no simple way
11/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


System of m nonlinear polynomial equations in
n unknowns - Grőbner basis method
How to find coefficients?
Bucheberger’s
algorithm ~
generalized
Gauss elimination
12/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Grőbner basis method w.r.t. lexicographic
ordering ~ generalized G-J elimination

General approach for constructing the Grőbner basis







Add new equations – monomial multiples of initial equations Perform Gauss-Jordan elimination
13/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


System of m polynomial equations in n
unknowns – multiplication matrix



m equations
in n unknowns
Buchberger’s algorithm
Generalized
companion matrix
Eigenvectors
Solutions
Grőbner basis
(grevlex ordering)
Polynomial division
Eigenvectors ~ solutions
14/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Grőbner basis method

Computing Grőbner basis (even in grevlex ordering) may be very
hard









Example: 4 polynomials, 3 variables, degree ≤ 6, small integer coeffs








Have extremly simple Grőbner basis







15/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Grőbner basis method

However when computed by the Buchberger’s algorithm over the
rational numbers w.r.t. the grevlex ordering x > y > z, the following
polynomial appears during the computation:









~ 80,000 digits
16/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Grőbner basis method
C o m p l e x i t y

Input
Grőbner basis
General algorithms can construct all GBs but
often generate many complicated polynomials
G e n e r a l
a l g o r i t h m

S p e c i f i c
a l g o r i t h m

17/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Specific Grőbner basis algorithm

All ``non-degenerate'' instances of the problem result in systems of
polynomial equations of the same form, i.e. systems consisting of
the same polynomials and differing only in coefficients



Always the same number of solutions

Grőbner basis w.r.t. some fixed monomial ordering is always the same
and differs for each instance only in coefficients arising from non-
degenerate measurements.

The way of obtaining the Grőbner basis is always the same

The way of obtaining all necessary polynomials for creating the
multiplication matrix is always the same







18/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Specific Grőbner basis algorithm

Find a short path towards the Grőbner basis/multiplication matrix
which is independent from the actual ``non-degenerate’’
coefficients, implement it efficiently







run long,
coefficient growth
numerical
problems
fast, no coefficient
growth, no numerical
problems
19/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Specific Grőbner basis algorithm

returns a very similar basis in the fraction of the second
20/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Specific Grőbner basis algorithm
Pose problem in .
Compute the number of solutions
and find the form of all necessary
polynomials for multiplication matrix
Offline
Find the way to these polynomials
(systematically generate
polynomials from the ideal)
Remove unnecessary
polynomials
Pose problem in .
Elimination template
Generate all necessary polynomials
for constructing the multiplication
matrix
Construct the multiplication matrix
and find its eigenvalues and
eigenvectors
Extract solutions from
eigenvectors
Online
21/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Specific Grőbner basis algorithm
25min
15min

Elimination template – hard to understand and implement






Requires knowledge from algebraic geometry




22/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Automatic generator of Grőbner basis solvers
Pose problem in .
Compute the number of solutions
and find the form of all necessary
polynomials for multiplication matrix
Find the way to these polynomials
(systematically generate
polynomials from the ideal)
Remove unnecessary
polynomials
Pose problem in .
Elimination template
Generate all necessary polynomials
for constructing the multiplication
matrix
Construct the multiplication matrix
and find its eigenvalues and
eigenvectors
Extract solutions from
eigenvectors
Input
Output
23/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Automatic generator of Grőbner basis solvers
24/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague






consider as a parameter






Hidden variable method: If B is square matrix then the system has
non-trivial solutions iff
- polynomial in
Hidden variable & polynomial eigenvalues
matrix form
25/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Polynomial eigenvalue method – an example







Rewrite in a matrix form
Rewrite in a form
~Polynomial eigenvalue formulation
26/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Relative pose problems

Estimation of relative position and orientation (rotation, translation) of
two cameras from point correspondences
1.
Calibrated cameras – 5-pt relative pose problem (BMVC 08)
2.
Cameras with unknown focal length – 6-pt focal length problem (BMVC 08)
3.
One calibrated and one up to focal length calibrated camera – 6-pt on-off
problem (ICCV 09)
4.
Cameras with unknown focal length observing a dominant plane – plane
+parallax + unknown focal length from 5 point correspondences (CVVT 10)
27/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague



Given 6 point correspondences in two images, find relative position,
orientation and focal length of partially calibrated camera












15 solutions










6 point focal length problem
28/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague



Epipolar constraint


Parametrize
the fundamental as a linear combination of a basis
of the space of all compatible fundamental matrices



Singularity of F


Transfer F to essential matrix E


Trace constraint of E


10 3
rd
and 5
th
order equations in 3 variables and with
30 monomials


6 point focal length problem
29/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague



Grőbner basis solution

Stewenius et al. - CVPR 2005 - 3x G-J eliminations of 12x33, 16x33
and 18x33 matrices + eigenvalue computation of 15x15 matrix

Kukelova et al. – ECCV 2008 – automatic generator – single G-J
elimination of 31x46 matrix + eigenvalue computation of 15x15 matrix


Polynomial eigenvalue solution

Kukelova et al – BMVC 2008

Quadratic polynomial eigenvalue problem (degree 2) -

=> eigenvalue computation of 20x20 matrix

Running time - 182μs


6 point focal length problem
30/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Relative pose problems for cameras with radial
distortion

Three new minimal problems for estimating one-parameter radial
distortion model and epipolar geometry from point correspondences
1.
Minimal algorithm for uncalibrated cameras with same radial distortion
and 8 point correspondences (CVPR 2007)
2.
Minimal algorithm for partially calibrated cameras with same radial
distortion and 6 point correspondences (CVPR 2008)
3.
Minimal algorithm for uncalibrated cameras with different radial
distortions in each image and 9 point correspondences (CVPR 2008)
31/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague



Results




8-point algorithm for uncalibrated cameras with
same radial distortions
Before correction After correction
Left image
Right image
32/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague


Absolute pose problems

Estimation of absolute position and orientation of one camera from 2D-
3D point correspondences – PnP problems
1.
Camera with unknown focal length – P4P+f (CVPR 2008)
2.
Camera with unknown focal length and radial distortion – P4P+f+k (ACCV
2010)
3.
Calibrated camera and camera with unknown focal length and radial
distortion with known vertical direction (rotation around one axis) – uP2P,
uP3P+f+k (ACCV 2010)
f
o
X
1
X
2
X
3
X
4
x
1
33/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague



Many problems in computer vision and other fields lead to systems
of polynomial equations

No general robust and efficient method for solving systems of
polynomial equations

Special algorithms have to be designed to achieve numerical
robustness and computational efficiency

Grőbner basis methods / polynomial eigenvalue methods

Automatic generator of Grőbner basis solvers => efficient
algorithms to many minimal computer vision problems














Conclusion
34/34
Zuzana K
ú
kelov
á

kukelova
@cmp.felk.cvut.cz,
Tomáš Pajdla

pajdla
@cmp.felk.cvut.cz, Center for Machine Perception, Czech Technical University

in Prague










http://cmp.felk.cvut.cz/minimal
THANK YOU FOR YOUR ATTENTION