From 2D to 3D and Stereo Machine Vision

jabgoldfishAI and Robotics

Oct 19, 2013 (3 years and 11 months ago)

60 views

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

1

From 2D to 3D and

Stereo Machine Vision

Helge Jordfald

Tordivel AS

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

2

3D Machine Vision Status


3D Machine Vision is currently only for
special tasks and expensive


3D machine vision can be marketed as 2.5
D when a 3D image is not created


It is difficult to select the appropriate 3D
method for a specific application due to
diversity in solution space



SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

3

Scorpion Vision 3D Strategy


Work hard to gain experience and exploit the
new 3D solution space


Teach our customers 3D


Create 3D training material and examples for
self study


Establish a Scorpion 3D vocabulary

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

4

Scorpion Vision 3D Vision


Take advantage of the smart Scorpion Framework
to implement the best for 3D Machine Vision
software solution


Scorpion Vision 3D shall consist of:


3D camera calibration, 3D Images, 3D Visualisation,
numerous ways to create and analyse 3D images


3D image creation can be stripelight, laser line,
stereo vision, laser grid and more


3D shall be an natural enhancement of Scorpion
Vision 2D

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

5

Content


Some history and the basic theory


2D Vision system


2D techniques for handling heights variation


3D Vision System


Basic 3D Vision techniques


Advanced 3D vision techniques


to be continued


Point Cloud Stereo Vision


Holography


structured light

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

6

Camera obscura


Latin for dark room


Light is admitted through a
narrow hole into a dark
chamber.


An inverted image is formed on
the opposing wall.


A device known for at least
2000 years


Chinese philosopher Mo
-
Ti
(5th century BC)


Aristotle (384
-
322 BC)


Leonardo Da Vinci (1490)


German astronomer Johannes
Kepler (early 17th century)

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

7

Descriptive Geometry


Gaspard Monge(1746
-
1818),

the father of descriptive geometry,

developed a graphical protocol

that

creates three
-
dimensional

virtual space

on a two
-
dimensional plane.


Monge became a scientific and mathematical aide
to Napoleon during his reign as
G
eneral and
E
mperor of France.



SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

8

Descriptive Geometry


D
efined as the projection of three
-
dimensional figures onto a two
-
dimensional
plane


The purpose is

to allow geometric
manipulations to determine


lengths
,
angles
,

shapes



and other descriptive information

concerning
the figures

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

9

Projection


When representing a 3
-
D object on the 2
-
D sheet of paper,
the number of dimensions is reduced from 3 to 2.


The general process of reducing the number of dimensions
of a given object is called
projection
.


T
wo different ways of doing this

a
ccording to the position
of observer;


Parallel Projections



infinitely far away from object


Per
s
pective
/Central

Projections


close to the object

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

10

Parallel projection


Descriptive Geometry
is based on Parallel
Projection, in most
cases parallel,
orthogonal projection.

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

11

Orthographic projection


Orthographic projection

is a means of representing a
three
-
dimensional (3D) object in two dimensions (2D). It
uses multiple views of the object, from points of view
rotated about the object's center through increments of 90
°
.

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

12

Perspective (Central) projection


In this case, where the
observer is relatively
close to the object, the
projectors form a
“cone” of projectors
.

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

13

Central Projection


Pin Hole Camera

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

14

How we see the world


Image Plane

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

15

The model we use

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

16

2D Vision System


Optical System calibration


Camera (CCD and electronics)


Lens


Camera/Lens position


2D Camera Model


Calibrate the camera in one plane


SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

17

2D Camera Calibration

step 1


Original image


After eliminating lens
distortion

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

18

Camera calibration


step 2


Camera positioned in
perspective


Camera corrected for
perspective

Calibration plane

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

19

Intermediate result


Creates a “Virtual”
pinhole camera with a
mathematical model
between calibration
plane and true image
plane


The 2D model
represents only the
object in one flat plane

True image plane

Calibration plane

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

20

Camera calibration


optional step 3


Image plane moved/scaled to object plane

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

21

2D Camera model


Object points in
calibration plane is
only correct in the
image plane

Calibration plane

True Image plane

Image plane

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

22

Challenges with 2D Camera Model


Product with different
heights or in different
layers

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

23

Multiple 2D camera models / references

Reference plane 1

Reference plane 2

Reference plane 3

3 different 2D camera calibrations, one model must

be selected based on the height of the object

External Reference tool

Possible to use linear approximation

in between the 2D camera models

Image plane 1

Image plane 2

Image plane 3

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

24

Challenge # 2


Inclined Geometry


Cannot be described with a 2D Model

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

25

Challenge # 3


Disorganised Products


Products with varying
angle relative to the
calibration plane


Cannot be described
by a 2D Model

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

26

Laser Triangulation


A laser system projects a
spot or line of light to the
target, and a camera
system takes an image of
its reflection.


The position of the spot /
line described the
elevation of the object
reflecting the spot/ line.

Triangular geometry between

laser, camera and reflection point


SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

27

Laser triangulation calibration

Measurement setup

Calibration object

+

Laser line in image

Object in laser plane

Measure point coordinates in image

and enter them + real coordinates

in External Reference Tool

Virtual camera in Laser plane

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

28

3D Scanning

Combine multiple laserline to create a 3D Image


Require scanning to move
one Virtual camera (to get
parallel laser lines)


X and Y are still only with
a 2D camera model


The 3D image is
represented as a 2D height
map

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

29

Summary 2D Camera Calibration


Describe Lens Distortion


Create a virtual camera with image plane equal to calibration plane


Use a calibration grid and the Calibrator Tool


Use multiple reference system to describe height variations


Create multiple 2d references


Use External Reference tool


Combine 2d references


Use Python to select the best 2D plane,


Use Python and Change Reference tool to interpolate between the different 2D planes


A simple concept to handle inclined objects


If possible locate 4 points in a plane


Use the four points to describe the inclined plane with External Reference tool


Note: Inclined object shall be handled in 3D

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

30

3D vision system


3D camera model


3D camera calibration


3D reference systems


Create a new 3D reference system


Moving from one 3D reference system to another
-

Robotics


3D (Monocular) reconstruction


Stereo vision

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

31

3D camera model


True central/perspective
projection to 2D image
plane


Each point in the 3D will
be projected correctly to
the Image Plane

Image plane

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

32

3D camera calibration


Measure minimum 7
points points in the 2D
image with different x,
y and z

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

33

3D Objects


3D point


x, y, z coordinates


3D line


Two 3d points


3D angle


Angle between two 3D
lines


3D plane (frame)


Origin x, y, z


Rotation around axis


Rx, Ry, Rz

X

Y

Z

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

34

Euler


Leonhard Euler

(1707


1783)
Swiss mathematician and
physicist. He published more
papers than any other
mathematician in history.


Introduced much of the modern
mathematical terminology and
notation and also renowned for
his work in mechanics, optics,
and astronomy.

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

35

Euler Angles

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

36

Fixed Angles (PRY)

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

37


Three rotations taken about fixed axes
(Fixed Angles) yield the same final
orientation as the same three rotations taken
in an opposite order about the axes of the
moving frame (Euler Angles)

Fixed angles versus Euler angles

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

38

Move the 3D Reference System


The Image Plane can be
moved to any position in
the 3D model using
Change Reference 3D tool


Specify new origin and
rotation

around

x, y, z

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

40

Definition of a 3D plane


A 3D point defines the
Origin


The normal vector of the
new 3D plane (Z axis) is
moved to the origin of the
original reference system


The projection of the
normal vector to x, y, z
axis defines the rotation of
the new 3D plane (A
normal vector has a
nominal magnitude of 1)

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

41

Stereo vision


What is stereo vision?


How to create a stereo vision system?


Stereo vision processing techniques


Key issues related to accuracy

SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

42

Stereo Vision Concept


The slightly different
perspectives from which
2 or
more cameras

perceive the
world lead to different images
with relative displacements of
objects


disparities
-

in the
different monocular views of
the scene


The size and direction of the
disparities of an object is a
measure of its relative depth;
absolute depth
-
information can
be obtained if the geometry of
the imaging system is known


SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

43

Scorpion 3D Camera Calibration


Remove lens distortion with Calibrator


Describe 3D Camera Models using
ExternalReference3D


SL
-
2007
-
0019
-
c From2D to 3D Machine Vision

44

Stereo Vision

Multiple images working in the same 3D space


Multiple cameras or images
calibrated in the same 3D space


Creation of a common 3D
Space


1. Calibrated multiple cameras
with the same 3D object


2. Know the displacment of the
camera or object in time


dynamic generation of common
3D space