Applications of Panoramic Annual

builderanthologyAI and Robotics

Oct 19, 2013 (3 years and 8 months ago)

68 views

Omnidirectional Vision Problems


Application
s

of Panoramic Annual
Lens (PAL)

Zoltán Vámossy

Vamossy.Zoltan@nik.bmf.hu

Budapest Tech

John von Neumann Faculty of Informatics

Zoltán Vámossy, Budapest Tech

2

IAR 2008

Prof. Ábrahám
’s picture

Zoltán Vámossy, Budapest Tech

3

IAR 2008

Zoltán Vámossy, Budapest Tech

4

IAR 2008

In memoriam Prof. Pál Greguss

Zoltán Vámossy, Budapest Tech

5

IAR 2008

Introduction
-

Imaging

Def.: Effort to map the 3D space conveyed by
signal waves on an Euclidean flat surface

Problem: How to obtain the
place

and
time

information

about a given object?


Traditional solutions: restricted part of
space: S
ee
-
Through
-
Window
(STW)
principle


Other approach:
Centric Minded Imaging

(CMI)

Zoltán Vámossy, Budapest Tech

6

IAR 2008

STW Imaging 1

Looking through the picture
plane


Natural horizon, vanishing
point


Space chunk


visual field,
<
-

spherical map


No 360
º panoramic view
at
once


Without depth information
-

feeling of perspective

Zoltán Vámossy, Budapest Tech

7

IAR 2008

STW Imaging 2

Zoltán Vámossy, Budapest Tech

8

IAR 2008

STW Imaging 3

‘The surrounding visual world
is a sphere’


The viewer is always on the
periphery of the
surrounding 3D space,


<
-
> never in the center of
the scene

Zoltán Vámossy, Budapest Tech

9

IAR 2008

CMI
-

Centric Minded Imaging

Zoltán Vámossy, Budapest Tech

10

IAR 2008

CMI
-

Centric Minded Imaging


Def.: CMI the term used to describe an optical
system capable to projecting a full 360º
panoramic field of view onto a 2D annular
format


Cylindrical visual field


No distortion


FOV is larger than that of the fish
-
eye
lenses

Zoltán Vámossy, Budapest Tech

11

IAR 2008

CMI Features


Unique projection of 3D space


2D shell of 3D environment


Ring shaped picture


cylindrical FOV


Proportional radial distance
information


1 vanishing point

Zoltán Vámossy, Budapest Tech

12

IAR 2008

The philosophy of CMI



CMI assumes that the geometric structure of
space encircling the spectator is
cylindrical
. The
radius of this cylinder is equal to the
vision
distance
, and a virtual
omnidirectional

panoramic view of the image volume shows up
on the wall of this imaginary cylinder.

Zoltán Vámossy, Budapest Tech

13

IAR 2008

Formation of an omnidirectional
view



This projection when
transformed onto a plane
surface creates an
annular
panoramic image

of the
environment, in which the
target points retain the
same
1:1

relation to each
other as it is in reality.

Zoltán Vámossy, Budapest Tech

14

IAR 2008

CMI Tool Types


The first who created such an
imaging block was the French
astronomer, A. Mangin, in
1878, when he designed a
telescope for the panoramic
observation of the sky

Zoltán Vámossy, Budapest Tech

15

IAR 2008

CMI Tool Types


Multiplex element design:


several elements with
coinciding optical axes


correction of various aberrations


not possible to miniaturize


Single glass block (PAL):


Reflecting and refracting
surfaces

Zoltán Vámossy, Budapest Tech

16

IAR 2008

PAL Optic


Biological
Background

Eye of a scallop


Pecten
maximus


‘SHELL’


Spherical, concave
multilayered structure



Reflecting argentea


2 interacting retinas

Zoltán Vámossy, Budapest Tech

17

IAR 2008

Technical realization of CMI
Systems


PAL
-

as described in Hung.
Pat. 192125, US Pat. 4566763,
Japan Pat. 192784, etc.
-

consists of a single glass block
with four reflective and/or
refracting surfaces that can be
plane, concave and/or convex,
resulting in

81 possible shapes!

but only a few of them have
real practical value
.

Zoltán Vámossy, Budapest Tech

18

IAR 2008

Zoltán Vámossy, Budapest Tech

19

IAR 2008

PAL

Panoramic Annular Lens (PAL)



Optical block


2 reflecting and 2 refracting
spherical surfaces

(convex, concave, flat)


Zoltán Vámossy, Budapest Tech

20

IAR 2008

PAL


Real and virtual image block
inside the optic
-
> additional
auxiliary lens


Miniaturized image volume of the
3D space encircling the imaging
block


It contains all the imaging point
data from the real 3D space


Center region does not take part in
the imaging

Zoltán Vámossy, Budapest Tech

21

IAR 2008

PAL


Image features 1


Sharp image from right up against the
lens surface out to infinity


Objects to the front of the optic are
imaged to the interior of the annular
image


Objects to the rear of the optic are
appear on the outer rim of the annular
image

Zoltán Vámossy, Budapest Tech

22

IAR 2008

PAL


Image features 2


The
horizon

of the optic where the first
reflecting and refracting surface
intersect, this plane is perpendicular to
the optical axis


The width of the image ring is
proportional of the acceptance angle of
the optic


The image points retain the same 1:1
relation of the original objects points


Interpretation with confusion

Zoltán Vámossy, Budapest Tech

23

IAR 2008

Panoramic Imaging Sensor

panoramic annular lens (PAL)

* 40 mm in diameter, C
-
mount

* view: H: 360, V:
-
15 ~ +20

* single view point (O)

p

p
1

pinhole

P
1

P

B

O

C

Ellipsoidal mirror

Hyperboloidal mirror

Zoltán Vámossy, Budapest Tech

24

IAR 2008

PAL Image


office

Zoltán Vámossy, Budapest Tech

25

IAR 2008

Unwraped PAL Image


office

Zoltán Vámossy, Budapest Tech

26

IAR 2008

PAL Image


St. Paul Cathedral

Zoltán Vámossy, Budapest Tech

27

IAR 2008

PAL Image


St. Paul Cathedral

Zoltán Vámossy, Budapest Tech

28

IAR 2008

How to unwrap the image?

Zoltán Vámossy, Budapest Tech

29

IAR 2008

How to unwrap the image?

Zoltán Vámossy, Budapest Tech

30

IAR 2008

Cylindrical
panoramic un
-
warping

Two Steps:

(1).
Center determination

(2) Distortion rectification

2
-
order polynomial approximation

Zoltán Vámossy, Budapest Tech

31

IAR 2008

Interpolation methods


Nearest neighborhood


Bilinear interpolation


Bicubic interpolation


Spline interpolation

Zoltán Vámossy, Budapest Tech

32

IAR 2008

Straightforward
-
looking

PAL


If a well defined portion of the
concave obstruction around the
optical axis of PAL is removed,
one can look
through

the imaging
block. Let us put a conventional
straightforward
-

looking optic

(SFLO) in the optical axis of
PAL; the vertical viewing field
will be increased resulting, e.g.,
in a total of 80º+20º.

Zoltán Vámossy, Budapest Tech

33

IAR 2008

Foveated PAL


If a rotatable and inclinable
mirror is placed in the optical
axis of SFLO, a vision system is
created, which is capable of
displaying with higher accuracy
any portion of interest of the 360
°

peripheral view, similarly to the
human foveal vision. Therefore,
this “
foveated


PAL has been
named
Humanoid Machine
Vision System

(HMVS).

Zoltán Vámossy, Budapest Tech

34

IAR 2008

Workshop drawing and realization
of the foveated PAL

Zoltán Vámossy, Budapest Tech

35

IAR 2008

HMVS


Humanoid Machine
Vision System


Foveal and peripheral FOV

Zoltán Vámossy, Budapest Tech

36

IAR 2008

PALIMADAR
-

Stereo

Zoltán Vámossy, Budapest Tech

37

IAR 2008

ENDOPAL application

Zoltán Vámossy, Budapest Tech

38

IAR 2008

First space
-
related application


PAL Attitude
Determination System

PALADS

was

launched

to

orbit,

as

a

part

of

the

Deep

Space
-
1

program,

from

Cape

Canaveral,

on

October

24
,

1998

Zoltán Vámossy, Budapest Tech

39

IAR 2008

Planned Planetary Application
-

Robocar

Zoltán Vámossy, Budapest Tech

40

IAR 2008

Future trends


The SATELLITE
-
M software
allows only processing of 25
HPAL image frames/second.
Preliminary tests proved that
with the Cellular Neural
Network
-
Universal Machine
(CNN
-
UM) of
AnaLogic
Computers, Ltd. (Budapest)

roughly 1000 image
frames/second would be
feasible


Aladdin Visual
Computer,
projects
images directly to the
CNN
-
UM.

Zoltán Vámossy, Budapest Tech

41

IAR 2008

Walking Robot Application Based on
PAL


Zoltán Vámossy, Budapest Tech

42

IAR 2008

Walking Robot Application Based on
PAL


Interaction with the terrain

Zoltán Vámossy, Budapest Tech

43

IAR 2008

Walking Robot Application Based On
PAL

Zoltán Vámossy, Budapest Tech

44

IAR 2008

Experiments

Zoltán Vámossy, Budapest Tech

45

IAR 2008

Navigation using the PAL
-
image

Real
-
time mapping and effect of filters:

Zoltán Vámossy, Budapest Tech

46

IAR 2008

Navigation using PAL
-
optic image

Zoltán Vámossy, Budapest Tech

47

IAR 2008

Position determination

Zoltán Vámossy, Budapest Tech

48

IAR 2008

Position determination step by step

Determination of

the vertical position of

the FOBOT

Layout

of the

PAL
-
image


Determination of

the spatial vectors

from the PAL
-
image

Edge

detection

Selection of the

followed points manually



Add
filters


3D transformation,


mapping the points


Location of the
ROBOT

Schoolyard

PAL
-
image

Zoltán Vámossy, Budapest Tech

49

IAR 2008

Suggested Reading
s and Software


Mubarak Shah, "Fundamentals of Computer Vision".


Emanuele Trucco, Alessandro Verri, "Introductory
Techniques for 3
-
D Computer Vision", Prentice Hall,
1998.




David A. Forsyth and Jean Ponce, "Computer Vision:
A Modern Approach", Prentice Hall, 2003.


Rafael Gonzales, Richard Woods: Digitale Image
Processing, Prentice Hall, 2002




OpenCV: Open Source Computer Vision Library

Zoltán Vámossy, Budapest Tech

50

IAR 2008

Thank you for your attention!