Vision Guided Robotics

flippinjapaneseΤεχνίτη Νοημοσύνη και Ρομποτική

2 Νοε 2013 (πριν από 3 χρόνια και 7 μήνες)

117 εμφανίσεις

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Vision Guided Robotics

and Applications in Industry and Medicine

Matthias Rüther

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Contents


Robotics in General


Industrial Robotics


Medical Robotics


What can Computer Vision do for Robotics?


Vision Sensors


Issues / Problems


Visual Servoing


Application Examples


Summary


SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Robotics


What is a robot?

"A reprogrammable, multifunctional manipulator designed to move
material, parts, tools, or specialized devices through various
programmed motions for the performance of a variety of tasks"

Robot Institute of America, 1979



Industrial


Mostly automatic manipulation of rigid parts with well
-
known shape in a
specially prepared environment.


Medical


Mostly semi
-
automatic manipulation of deformable objects in a
naturally created, space limited environment.


Field Robotics


Autonomous control and navigation of a mobile vehicle in an arbitrary
environment.

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Robot vs Human


Robot Advantages
:



Strength


Accuracy


Speed


Does not tire


Does repetitive tasks


Can Measure


Human advantages:



Intelligence


Flexibility


Adaptability


Skill


Can Learn


Can Estimate


SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004


Requirements:



Accuracy


Tool Quality



Robustness


Strength


Speed


Price



Production Cost


Maintenance


Industrial Robot

Production Quality

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Medical (Surgical) Robot


Requirements



Safety


Accuracy


Reliability


Tool Quality


Price


Maintenance


Man
-
Machine Interface

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

What can Computer Vision do for Robotics?


Accurate Robot
-
Object Positioning


Keeping Relative Position under Movement


Visualization / Teaching / Telerobotics


Performing measurements


Object Recognition


Registration

Visual Servoing

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Vision Sensors


Single Perspective Camera


Multiple Perspective Cameras (e.g. Stereo Camera
Pair)


Laser Scanner


Omnidirectional Camera


Structured Light Sensor

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Vision Sensors


Single Perspective Camera

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Vision Sensors


Multiple Perspective Cameras (e.g. Stereo Camera Pair)

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Vision Sensors


Multiple Perspective Cameras (e.g. Stereo Camera Pair)

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Vision Sensors


Laser Scanner

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Vision Sensors


Laser Scanner

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Vision Sensors


Omnidirectional Camera

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Vision Sensors


Omnidirectional Camera

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Vision Sensors


Structured Light Sensor













Figures from PRIP, TU Vienna

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Issues/Problems of Vision Guided Robotics


Measurement Frequency



Measurement Uncertainty



Occlusion, Camera Positioning



Sensor dimensions


SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Visual Servoing


Vision System operates in a closed control loop.


Better Accuracy than „Look and Move“ systems

Figures from S.Hutchinson: A Tutorial on Visual Servo Control

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Visual Servoing


Example: Maintaining relative Object Position

Figures from
P. Wunsch and G. Hirzinger.

Real
-
Time Visual Tracking of 3
-
D Objects with Dynamic Handling of Occlusion


SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Visual Servoing


Camera Configurations:


End
-
Effector Mounted

Fixed

Figures from S.Hutchinson: A Tutorial on Visual Servo Control

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Visual Servoing


Servoing Architectures

Figures from S.Hutchinson: A Tutorial on Visual Servo Control

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Visual Servoing


Position
-
based and Image Based control



Position based:


Alignment in target coordinate system


The 3D structure of the target is rconstructed


The end
-
effector is tracked


Sensitive to calibration errors


Sensitive to reconstruction errors




Image based:



Alignment in image coordinates


No explicit reconstruction necessary


Insensitive to calibration errors


Only special problems solvable


Depends on initial pose


Depends on selected features

target

End
-
effector

Image of target

Image of end
effector

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Visual Servoing


EOL and ECL control




EOL: endpoint open
-
loop; only the target is observed by the camera




ECL: endpoint closed
-
loop; target as well as end
-
effector are observed by
the camera

EOL

ECL

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Visual Servoing


Position Based Algorithm:

1.
Estimation of relative pose

2.
Computation of error between current pose and target pose

3.
Movement of robot



Example: point alignment

p
1

p
2

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Visual Servoing


Position based point alignment






Goal: bring e to 0 by moving p
1

e = |p
2m



p
1m
|

u = k*(p
2m



p
1m
)




p
xm

is subject to the following measurement errors:
sensor position, sensor
calibration, sensor measurement error



p
xm

is independent of the following errors:
end effector position, target
position


p
1m

p
2m

d

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Visual Servoing


Image based point alignment






Goal: bring e to 0 by moving p
1

e = |u
1m



v
1m
| + |u
2m



v
2m
|





u
xm
, v
xm

is subject only to
sensor measurement error



u
xm
, v
xm

is independent of the following measurement errors:
sensor
position, end effector position, sensor calibration, target position


p
1

p
2

c
1

c
2

u
1

u
2

v
1

v
2

d
1

d
2

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Visual Servoing


Example Laparoscopy

Figures from A.Krupa:
Autonomous 3
-
D Positioning of Surgical

Instruments in Robotized Laparoscopic

Surgery Using Visual

Servoing



SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Visual Servoing


Example Laparoscopy

Figures from A.Krupa:
Autonomous 3
-
D Positioning of Surgical

Instruments in Robotized Laparoscopic

Surgery Using Visual

Servoing



SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Registration


Registration of CAD models to scene features:

Figures from P.Wunsch:
Registration of CAD
-
Models to Images by Iterative Inverse Perspective Matching



SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Registration


Registration of CAD models to scene features:

Figures from P.Wunsch:
Registration of CAD
-
Models to Images by Iterative Inverse Perspective Matching



SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Tracking


Instrument tracking in laparoscopy

Figures from Wei:
A Real
-
time Visual Servoing System for Laparoscopic Surgery

SSIP 2004 Graz

© Inst. For Computer Graphics and Vision, 2004

Summary


Computer Vision provides accurate and versatile
measurements for robotic manipulators



With current general purpose hardware, depth and pose
measurements can be performed in real time



In industrial robotics, vision systems are deployed in a fully
automated way.



In medicine, computer vision can make more intelligent
„surgical assistants“ possible.