Machine Vision's In-Depth Picture - Siemens

geckokittenΤεχνίτη Νοημοσύνη και Ρομποτική

17 Οκτ 2013 (πριν από 4 χρόνια και 8 μήνες)

117 εμφανίσεις

90 Pictures of the Future | Fall 2006 Pictures of the Future | Fall 2006 91
Machine Vision | 3D Object Recognition
Cameras and laser scanners generate a 3D image
of their surroundings — here in Dr. Frank Forster’s
laboratory (large photo) and during the placement
of heavy containers (bottom).
Machine Vision’s
In-Depth Picture
Machines are getting better at recognizing 3D objects. Progress in this
area is being driven by stereo cameras, light stripe projection systems,
laser scanners — and the intelligent software that makes them possible .
ubai, free port. A fully loaded container ship
has barely docked, yet it’s time to hurry.
Wasted time is expensive. Towering gantry
cranes purr and clatter as they lower their hooks
and then hoist shipping containers, some of
which are over 12 meters long, ashore. Other
giants approach on rail tracks and proceed to
stack the cargo in the container storage area.
The cranes go through their motions as
always. But this time something is different.
Experts from Siemens Automation and Drives
(A&D) are on hand and their attention is focused
on a crane that’s about to maneuver a container
onto the bed of a huge truck. Superficially, the
crane looks no different from the others. But the
A&D pros know what’s concealed in the crane’s
trolley, which is slowly lowering the cargo
toward the truck. Laser beams from the crane
have detected the truck, scanned it, and created
its three-dimensional image. As a result, the
crane can now compute the vehicle’s position
with great precision.
Visible light signals guide the truck driver
precisely to his parking position under the
gantry crane. The laser system verifies this posi-
tion and communicates it to the crane’s control
system, which then manages the unloading of
the cargo onto the truck with precision in the
centimeter range. The truck positioning system,
they’ll work fully automatically. Here too, laser
technology will be useful, and preliminary tests
are already under way. To enable the crane to
deposit its load correctly in a huge stack of con-
tainers, the control system scans the container
landscape and stores it as a 3D image. Using this
image, the system automatically computes the
quickest path to each storage place and the
height to which the container must be hoisted in
order to avoid obstacles.
Alois Recktenwald, product manager at A&D
MC Cranes, is convinced that such driverless
gantry cranes with 3D object recognition will
be safer and more productive than their prede-
cessors. “Even now, the safety and efficiency of
the test cranes is several percent better than
today’s standard,” says Recktenwald .
Automated Forklifts. In addition to their work
on rail-mounted harbor cranes, experts from
Siemens A&D are also planning to equip forklifts
with laser scanners. The driverless vehicles will
then be able to cruise automatically through ware-
houses or factory shops without conventional
floor-mounted guide wires. “By the end of the
year, such autonomous vehicles will be ready for
large-scale production,” predicts A&D project
manager Walter Beichl as he starts up a proto-
type in a warehouse near Stuttgart. “First it has
pallet. But they’ll manage this feat within a few
months.” When that happens, information
regarding the position and destination of each
pallet will be transmitted by a master computer
via WLAN to the forklift.
Closely Watched Packages. Vision systems
are also set to transform postal sorting. At the
Cologne-Bonn airport, for instance, Siemens has
integrated an entirely new type of 3D object
recognition system into its Singulator package
sorting system. There, Siemens Industrial Solu-
tions and Services has installed four Singulators
for UPS to ensure efficient operation of package
conveyor belts. Each Singulator takes an initially
unsorted flow of packages of various sizes, all
aligned helter-skelter, and sorts it into a single-
But to line up the packages, the system must
recognize and record them. To this end, six video
cameras are installed above the conveyor. The
first pair are stereo cameras, which acquire a
three-dimensional view of the packages. This
initial image information is processed — in
conjunction with signals from photodiodes inte-
grated in the conveyor belt — to determine the
size, location and orientation of the packages.
As this processing takes place, mechanical sort-
ing takes place on the conveyor belt, which is
about 1.5 meters wide.
Narrower side-by-side conveyor belts and
rollers accelerate or slow down the packages
individually to cause them to rotate and line up
in a single-file. Four additional cameras monitor
the scene to ensure that everything runs
which was developed by Siemens and tested for
the first time in Dubai, functions flawlessly. In
other words, these containers, which weigh
many tons, can now be loaded and hauled away
faster and more safely than ever.
The rail-mounted gantry cranes that stack
containers in the storage area up to five high, in
rows of six, will also work faster in the future.
Siemens researchers plan to upgrade them so
to learn its route,” explains Beichl as he maneu-
vers the forklift among the storage shelves in the
building. As he does so, the laser beam emitted
by the navigation system on top of the forklift con-
tinues to scan the surrounding space from floor
to ceiling, scan line after scan line. From these
scanned lines, which represent local geography,
the forklift’s image processing software generates
and stores a 3D model of the route’s environment.
In a subsequent automatic trip, an internal
computer compares this map with new, live
images detected by the laser scanner. The fork-
lift precisely follows the route for which it’s been
trained, including proper speed and steering
angles. To achieve such accurate localization,
the navigation system uses fixed landmarks on
the ceiling, such as supporting beams, whose
images it has stored during the test drive.
The forklift’s efficiency will be further en-
hanced when it also learns to reliably detect pal-
lets in the laser image — and how to pick them
up automatically. “Unfortunately, the required
algorithms aren’t quite there yet,” says Beichl.
“They’ve learned to recognize pallets, but
haven’t quite learned how to precisely align their
forks with the pallet axes. As a result, they can’t
accurately calculate the angle of approach to the
| Facts and Forecasts
Pictures of the Future | Fall 2006 9392 Pictures of the Future | Fall 2006
already 9,000 cameras in public places — about four per
city block.
In the past, such systems utilized cameras that merely
transmitted their images to tape machines and monitors
(CCTV, Closed Circuit Television). But now there are more
and more digital cameras that transmit data to computers.
Presently, four to eight such cameras share one CPU. But in
just two to three years many cameras will have their own
CPUs. Conventional videotape will be superfluous. Using
intelligent software, the latest smart cameras can even use
data comparison to detect unusual behavior and trigger an
alarm (see p. 87).
By 2008, video cameras will be increasingly combined
with access control solutions. That, in turn, will increase
demand for biometric systems, especially those based on
face recognition. Market researchers also see a particularly
strong future trend toward totally digital solutions based
on the Internet Protocol. Every surveillance camera will
then essentially be a Webcam. What’s more, security per-
sonnel will increasingly be able to use mobile telephones
to record and transmit the actions of suspicious persons for
computer analysis, for instance in airports, railroad stations
and sports arenas.

Sylvia Trage
mage processing applications already range from indus-
trial uses and security systems to transportation and
medical technology. Even so, industry experts agree that
only about 20 percent of all possible applications have
been addressed so far. According to estimates provided by
a number of manufacturers, the worldwide market volume
for machine vision systems presently amounts to about 6.5
billion euros, with annual growth rates extending into the
double-digit range.
In the industrial area, image processing systems are
employed for quality control in virtually every sector. They
are used to inspect everything from computer displays and
the surfaces of gearbox components to printed circuit
boards for cell phones. Image processing is also useful in
metrology, where it is used in visually guided machines
and to recognize components, symbolic characters and
codes. Cameras can help robots recognize objects, such as
the shape and position of workpieces.
In Germany, industrial image processing has been
growing faster than other sectors of automation tech-
nology for several years. According to a study published in
July 2006 by the German Engineering Federation (VDMA),
sales volume in 2005 grew seven percent and now tops
the one billion euro mark. “For 2006 we project a growth
rate of nine percent, with the strongest advances coming
from export sales,” predicts VDMA expert Patrick
Schwarzkopf. In 2005 about 70 percent of component
sales involved cameras and smart cameras.
Among the latter, system functionalities — in other
words, the image sensor, processor and light source — are
integrated in a compact housing. Sales of these products
soared 23 percent between 2004 and 2005. The share of
frame grabbers, on the other hand, declined from 15 to 13
percent. This category includes PC cards for digitizing, stor-
ing and playback of image signals. The decline resulted
from the increased use of digital cameras that have a built-
in image processing system without a frame grabber, but
that use USB.
According to a 2005 study by Frost & Sullivan (F&S) the
market will see increasing growth in sales of gigabit Ether-
net cameras that can transmit high-resolution images from
a camera to a computer across a distance of several hun-
dred meters. And by 2007, 3D vision systems for robots
should be available, as should systems for the inspection of
semiconductor components with an accuracy of 4.5 micro-
Starting in 2010, smart cameras with neural networks
are expected to have the capability of categorizing objects
into many different classes — an important feature when it
comes to automatic sorting.
Image processing is vitally important in hospitals too.
According to F&S market researchers, the key development
in that sector is the growing importance of Picture Archiv-
ing and Communication System. PACS make it possible to
process, store and manage medical images, and have be-
come accepted as the standard in radiology. By 2010 ana-
lysts predict sales in Europe will reach $1.47 billion —
compared to $0.47 billion in 2003. An important growth
engine here is a reduction in costs, which are declining by
about ten percent annually. Another trend is the combina-
tion of two imaging modalities in a single system, such as
high resolution computed-tomography images paired with
nuclear medicine methods that visualize biochemical
In the auto industry too, image processing for driver
assistance systems is gaining in importance (Pictures of the
Future, Fall 2005, p. 46). Automakers use not only laser,
radar and ultrasonic sensors, but also cameras that can
perceive vehicles, lane boundaries, traffic signs and pedes-
trians faster than the human eye. According to a 2006 F&S
analysis, cameras will experience the strongest sales
growth among all onboard automotive sensing systems,
for instance in video-supported systems that sense lane
markers and issue a warning when a car strays from its
lane, and in parking assistance systems.
Eyes on London. Security systems are a particularly
strong market for video surveillance. According to an F&S
study, worldwide sales are expected to reach about $11
billion by 2008. At 44 percent, North America represents
the largest share of this market. In Europe, the United
Kingdomis driving this trend.
As a case in point, the authors of the 2004 EU study,
UrbanEye (, estimate that there are
more than four million private and public surveillance cam-
eras in the UK. That makes the United Kingdom the coun-
try with the largest concentration of video surveillance in
Europe. Around 6,000 cameras of the estimated half a mil-
lion cameras installed throughout London are located in
the city’s Underground system. In some streets, cameras
are mounted only 15 meters apart. Privacy advocates have
calculated that people in London are recorded by a surveil-
lance camera up to 300 times per day. But most Londoners
consider the undeniable successes in fighting crime more
important than the potential negative aspects of such
In the UrbanEye survey, 90 percent of London’s inhabi-
tants were in favor of cameras in public places (compared
with 25 percent in Vienna). In New York too, cameras are
multiplying rapidly. In Manhattan, for instance, there are
Image Processing:
A Booming Market
Machine Vision | 3D Object Recognition
Seeing with Sound
As part of the Cognitive Aid System for Blind People (CASBliP) project, a research initiative supported
by the European Union, Siemens is collaborating with universities and organizations for the blind in
developing a sensor system that uses audio signals to endow sightless people with spatial perception of
their surroundings. The concept for the sensor system, which is built into a pair of glasses, was developed
by Siemens Corporate Technology (CT). “We originally developed the sensor for pedestrian recognition by
cars,” says project researcher Dr. Peter Mengel. “But this solution is also very well suited as an orientation
aid for blind people.” A laser diode in a special pair of glasses scans its surroundings with infrared light
pulses up to five meters ahead, and with an angle of 60 degrees. Infrared light reflected by objects is
detected by a tiny scanning camera with 64 pixels. Differences in elapsed time are converted into a
distance profile of the immediate environment, which in turn is converted into audible signals.
The shorter the distance to the object, the higher the pitch of the sound — and conversely, the farther
the object, the lower the pitch of the signal. Thanks to the different angles from which the infrared light
is reflected, there is an audible right / left difference. By turning his or her head, an otherwise unaided
blind person can gain a nearly complete impression of object distances in the immediate environment.
As part of his dissertation on color-coded tri-
angulation, Dr. Frank Forster, a researcher at
Siemens Corporate Technology (CT), developed
a very promising method of 3D acquisition that’s
already used in several Siemens products. The
basic principle is simple. A projector illuminates
the object whose shape is to be detected with a
pattern of parallel, colored light stripes that are
subsequently deformed according to the geom-
etry of the object’s surface.
A camera records the resulting pattern, and
in a fraction of a second a computer program
composes a 3D image based on the pattern’s
deformation. The method has a number of
advantages: Since all that’s required are stan-
dard video system components, it is inexpensive
to implement. What’s more, it generates a 3D
record from a single video image, which means
that it can be used for anything from facial
recognition to detection of imperfections in
manufactured objects.
Faces and Hearing Aids. Color-coded triangu-
lation was initially used for facial recognition in
access control systems (see Pictures of the Future,
Spring 2003, p. 38). The advantage of this
method over other recognition methods, such
as color images, is obvious: 3D recognition is
more reliable, because the exact shape of the
face is difficult to imitate. In order to quickly take
advantage of Siemens’ technological lead in this
area, a cooperative agreement was concluded
between CT, Siemens Building Technologies and
Viisage Technology Inc., the global leader in per-
sonal identification systems.
Color-coded triangulation is also being put to
good use by other Siemens Groups. Siemens
Automation and Drives (A&D), for instance, uses
it in the latest generation of chassis tuning sys-
tems. In this system, the surfaces of rotating
vehicle wheels are scanned. Within seconds, the
system precisely measures the chassis with a
view to improving the quality of the vehicle.
After all, the more precisely the chassis is adjust-
ed, the greater its safety and driving comfort,
and the less tire wear. The system is used by
BMW and Porsche, among others, and will soon
be available to repair shops as well.
Color coding is also used in the iScan system,
which was recently introduced by Siemens Audi-
ologische Technik GmbH (S.A.T.). iScan allows
hearing-aid acousticians to produce digital
casts of the auditory canal that can be translat-
ed into in-the-canal devices. iScan scans the
canal and converts the image into 3D data.
Then, instead of mailing a physical cast for a
hearing aid to a device manufacturer for digi-
tization, they can send an electronic version by
e-mail. That’s a lot faster, simpler and more

Rolf Sterbak
3D object recognition based on the use colored stripes — so called structured light — is useful in applications
ranging from 3D face recognition to measuring suspension systems and ensuring a perfect fit for hearing aids.
Euros: millions
2005 2006 2007 2008 2009 2010 2011 2012
Millions of units
discrete parts
(46% of
total sales)
Component and
character recognition
Code reading
Automotive Onboard Sensing
Industrial Image Processing
Major applications of machine
image processing
High note
from right front
Audible impression: Mid-range
tone from left front
Infrared light pulse
Reflected signal
Audible impression:
Base note from
directly in front
Source: VDMA, July 2006, Values are for Germany alone
Source: Frost &Sullivan (2006), Figures are for world market
Source: VDMA 2006