Machine Vision Software The Key to Competitiveness

geckokittenΤεχνίτη Νοημοσύνη και Ρομποτική

17 Οκτ 2013 (πριν από 4 χρόνια και 27 μέρες)

98 εμφανίσεις

Vision Academy GmbH ?Konrad-Zuse-Str. 15 ?D-99099 Erfurt / Germany ?Tel. +49(0)361 / 4262-187 ?Fax -189 ?infopoint@vision-academy.org?www.vision-
academy.org
Machine Vision Software
The Key to Competitiveness
Dr. Wolfgang Eckstein
MVTec Software GmbH
www.mvtec.com
2© Vision Academy, Erfurt
Overview of the presentation
Application areas of machine vision
Typical applications from different industries
How to develop a machine vision application
Trends in machine vision software
3© Vision Academy, Erfurt
MVTec Software GmbH
A company snapshot
4© Vision Academy, Erfurt
MVTec Software GmbH
A company snapshot
MVTec is a leading international manufacturer of
standard software products for machine vision.
Our industrially proven software products HALCON and
ActivVisionTools are used in a large variety of
application areas, such as semiconductor industry,
inspection applications, medicine, and surveillance.
Besides, MVTec builds customized solutions - from
consultancy, studies, and prototypes up to integrated
products. Moreover, MVTec offers services, such as
technical support and customized trainings.
5© Vision Academy, Erfurt
Machine vision offers a broad range of application areas
Identification
 Optical character recognition
 Bar codes
 Data codes
Completeness check and print inspection
Measuring and comparison
Object and position recognition
Surface inspection
Texture and color inspection
Robot vision
6© Vision Academy, Erfurt
Machine vision is applicable to every industry
Automobile
Chemicals
Electrical components
Food
Glass production
Health care
Iron, steel, and metal
Machinery
Packaging
Paper Production
Pharmaceutical
Precision engineering
Printing
Rubber, synthetic material, foil
Semiconductors
Surveillance and Security
Transport
Wood and Timber
7© Vision Academy, Erfurt
Inspection of circuit breakers
Sensata Technologies, Germany
8© Vision Academy, Erfurt
Inspection of circuit breakers
Sensata Technologies, Germany
Description of the inspection task
Hair dryers can consume up to 1875 W of power to produce hot-air flow up to
40°F, or the maximum temperature the typical human scalp can handle without
burning. That's a lot of electrical energy, producing a lot of heat.
A bimetallic disk, measuring about 5 × 10 mm, forms the heart of a thermal
protector switch that breaks the electrical circuit if the internal temperature
exceeds a safe level.
Sensata Technologies manufactures bimetal disks and thermal protector
switches for many products, including coffee machines and electric motors. As a
bimetal disk is heated, the two metal layers expand at different rates. Eventually,
the disk will bend enough to break a physical connection with a nearby electrode,
cutting power to the circuit. After the device cools sufficiently, the disk returns to
its original shape, and the circuit can be energized.
The realization is based on traditional visible-light sensors, special ovens, and a
generic PC running a Visual Basic front end and HALCON image-processing
algorithms to test up to 1024 bimetal disks per hour with greater accuracy and
speed than previous test methods.
9© Vision Academy, Erfurt
Checking for defects on plastic rings
NP Plastics, Netherlands
10© Vision Academy, Erfurt
Checking for defects on plastic rings
NP Plastics, Netherlands
11© Vision Academy, Erfurt
Checking for defects on plastic rings
NP Plastics, Netherlands
Description of the inspection task
NP Plastics, a Dutch injection-molded plastics manufacturer, produces small
plastic rings–among its many other products. These plastic rings are used in the
closing mechanism of ring binders found in every office from Amsterdam to Hong
Kong.
To meet volume demand while improving quality, NP Plastics recently decided to
install a machine-vision system to inspect the small plastic rings. A custom
machine-vision solution was developed using two FireWire cameras, strobe and
LED illumination, and the HALCON image-processing software running on a
standard PC to check the rings for burrs, undershots, and other spatial defects.
12© Vision Academy, Erfurt
Simplifying of fruit picking with robots
ACRO Institute, Belgium
13© Vision Academy, Erfurt
Simplifying of fruit picking with robots
ACRO Institute, Belgium
Description of the inspection task
In response to Belgian farmers' problem to find reliable seasonal workers, the Automation Centre for
Research and Education (ACRO) Institute developed a robotic apple harvester.
The ACRO automated fruit-picking machine (AFPM) harvester uses a unique vacuum-gripper design to pick
the fruit and ease coordination between the vision system and robot controller. Mounted behind a common
agriculture tractor, the AFPM platform supports a Panasonic industrial robot to "pick" the fruit.
The image processing software provides the robot guidance coordinates, while a canopy and curtain, which
can be folded up during transportation, reduce the affects of ambient light. The AFPM needs one driver on
the tractor while it effectively handles the workload of six workers.
One of the most challenging problems was the design of the fruit gripper. The final gripper design is a
combination of a white flexible silicone cone surrounding a USB 2.0 color camera. The cone is fed by a
reversible vacuum/blower.
By placing the camera inside the gripper, the position of the camera is fully controllable. The camera can
point its optical axis at the apple, reducing image distortion and eliminating repetitive calibration steps during
apple picking. A final advantage is that the camera is protected against collisions or bad weather conditions,
as well as against direct sunlight.
Image processing is conducted on an industrial PC with 2 GHz Pentium IV microprocessor and 1 GByte RAM
running Windows XP. For image-processing software, the AFPM designers elected the standard machine
vision library HALCON. The system can pick 85% of all apples of a tree, similar to the achievements of
manual picking operations.
14© Vision Academy, Erfurt
High speed bottle inspection
BBull Technology , Germany
15© Vision Academy, Erfurt
High speed bottle inspection
BBull Technology , Germany
Description of the inspection task
To operate the enormous amount of containers, the machine vision application must
acquire and process four camera images in parallel running on a single PC.
Furthermore, the system supports other complex tasks. BBull has developed a procedure
to control swing cap bottles. This someway nostalgic manner to cap beer bottles currently
undergoes an irresistible renaissance. However, to control the swing caps and labels of
these bottles is a complex challenge. The metal bails must be in the right position, cap and
packing ring adjusted, the seal label undamaged, and defilements or damages in the
aperture area must be discovered. Moreover, before the packing process the barrels must
be checked for completeness.
16© Vision Academy, Erfurt
Automated eyeglass fitting
Rodenstock GmbH, Germany
17© Vision Academy, Erfurt
Automated eyeglass fitting
Rodenstock GmbH, Germany
Description of the inspection task
Imagine that you wear varifocal glasses and need new ones. You go to an optometrist and,
without time-consuming adaptations, measuring or trying on frame after frame,you look at
yourself in a mirror and - voila! - your chosen frame is on your nose.
Rodenstock GmbH of Munich has developed an optometrist service terminal that can do
just that. It enables automatic calculation of eyeglass frames and lenses customized to fit
an individual's face - all in about a second.
One camera captures the face from the front and the other, from below and the side. These
create aesthetically pleasing images, while being suitable for analysis.
The images are processed based on HALCON. The software delivers the real-world 3D
coordinates necessary for manufacturing the frames and lenses. By acquiring a calibration
plate before the first start-up of the ImpressionIST, inner and outer camera parameters can
be appointed to deliver the coordinates.
18© Vision Academy, Erfurt
High precision measurement of drilling tools
E. Zoller GmbH & Co. KG, Germany
19© Vision Academy, Erfurt
High precision measurement of drilling tools
E. Zoller GmbH & Co. KG, Germany
Description of the inspection task
The company E. Zoller GmbH & Co. KG is the leading manufacturer of presetter and
measuring machines in the world.
Besides the fine mechanics of the machine the central point is the exact measurement of
the tools in the video image. The complete analysis of the images is done with HALCON.
With the use of specific subpixel algorithms the geometric properties of the tools are
measured with an absolute precision better than two microns.
The camera calibration guarantees the same precision in the whole field of view. The
measure task is automatically extracted from the shape of the tool. And with the fast
routines the image processing is done in real-time.
20© Vision Academy, Erfurt
How to realize a machine vision application:
The necessary hardware components
Object
Camera
Lighting
Machine control
21© Vision Academy, Erfurt
How to realize a machine vision application:
The necessary hardware components
Nearly all application setups consist of the following parts:
 the object to be inspected,
 a camera (and lens) to acquire images of the object,
 illumination,
 additional machine control devices,
and, of course, a computer system of some kind (PC, smart camera, embedded system,
etc.) that interprets the images and controls the machine.
Although the computer system with the software is not directly visible on this slide, it is an
important part performing the complete interpretation of the incoming images.
22© Vision Academy, Erfurt
How to realize a machine vision application:
The control loop
23© Vision Academy, Erfurt
How to realize a machine vision application:
The control loop
This illustration shows a typical (simplified) setup of a machine vision application. The
object is illuminated with a specific light source and imaged using a lens and camera. The
image is transferred to the memory of a computer system (via frame grabber or directly).
There it can be processed by the software, and the results can be used to control the
process, e.g., by passing the object coordinates to a robot that is then able to handle the
object correctly.
24© Vision Academy, Erfurt
To develop the software solution for a given Task
you need the right tool
Task description
?
25© Vision Academy, Erfurt
Three kinds software approaches:
1. Library-based systems
Advantages
 Very flexible and powerful
 Complete control on a programming level
 Both usable on standard hardware and smart cameras
26© Vision Academy, Erfurt
Three kinds software approaches:
1. Library-based systems
Background information
Software development without using libraries is more or less impossible for many
tasks. Libraries enable the user to develop own applications by using reliable
functions. Instead of being occupied by writing low-level image processing
functions the user can focus on the development of the application itself. The
user can expect that a library for machine vision supports the concepts of the
host languages as much as possible. E.g. a library used in C++ must have a
class hierarchy as well as its own memory management.
The most important benefit is as well the most important drawback: the highly
flexible library still is one part of an application; the user-interface and the
interaction with the production process needs to be programmed in addition.
27© Vision Academy, Erfurt
Three kinds software approaches:
2. Component-based systems
Advantages
 Make use of Microsoft´s ActivX and .NET architecture
 Easy to learn
 Ready to go user interface components
 Configuration instead of programming
 Programming is possible
28© Vision Academy, Erfurt
Three kinds software approaches:
2. Component-based systems
Background information
Since the introduction of software components in the Microsoft Windows world
many people got aware of the concepts behind components. The first generation
of software components in Microsoft terminology was COM (Component Object
Model) and refers to a mechanism to establish interfaces between components
enabling a component to share data with other components via a native
Windows operating system support. Based on COM ActiveX was introduced.
Currently .NET has been introduced which offers various new features and is
therefore already widely used.
29© Vision Academy, Erfurt
Three kinds software approaches:
3. Closed system
Advantages
 User interface included
 Configuration instead of programming
 Easy to learn
 Both usable on standard hardware and smart cameras
Types of systems
 PC-based
 Smart cameras
30© Vision Academy, Erfurt
Three kinds software approaches:
3. Closed system
Background information
Users which won‘t like to program their machine vision application might look into
closed systems, i.e., executable programs. Closed systems offer a task-oriented
approach for solving a machine vision application: the control of the image
acquisition, image processing, and result visualization. Each step usually is easy
to parameterize.
The drawback here is the fixed control flow of the application which might not fit
into the particular application of the user. Also extensions of closed systems are
often not impossible, e.g., the (complete) adaptation of the user interface: an
OEM customer might not want to see the logo of a machine vision component
supplier on his machine.
31© Vision Academy, Erfurt
Trends in machine vision software
3D reconstruction and 3D object recognition
Parallel processing
Usability
32© Vision Academy, Erfurt
3D reconstruction and 3D object recognition
Stereo
Photometric stereo
3D matching
Depth fromfocus Sheet of light
Pose fromcontours Pose frompoints
Perspective matching
33© Vision Academy, Erfurt
3D reconstruction and 3D object recognition
34© Vision Academy, Erfurt
3D reconstruction and 3D object recognition
What Is 3D Vision?
3D vision means the utilization of 3D information with the aid of machine vision, allowing to approach
applications which so far could not be solved with classical 2D technologies. It includes two main objectives,
which both contain many different technologies:
 3D alignment – finding the 3D pose (position & orientation) of an object.
 3D reconstruction – determining the 3D shape of arbitrary objects.
From the Task to the Solution
For any project, in order to find a solution, you must identify the specific vision task. First, you should ask the
following questions before deciding which 3D vision method is the right one for you:
 What do I want to find out?
 How accurately do I want to measure?
 Which are the characteristics of my object?
 What are the general conditions?
35© Vision Academy, Erfurt
Speedup by parallel processing:
Multi core technology
0
1
2
3
4
5
6
7
8
1 2 3 4 5 6 7 8
threshold
derivate_gauss
median(7x7)
median(15x15)
Number of CPUs
Speedup
36© Vision Academy, Erfurt
Speedup by parallel processing:
Multi core technology
Various software packages allow parallel programming. Some of them support automatic
operator parallelization – the easy way of making use of multi core computers. An important
point is how well the speedup scales with the number of CPUs. A good library should offer
a significant speedup for 4 up to 8 CPUs. Another criteria when selecting a library is the
amount of operators which parallelize automatically.
Notes on the selected operators and the benchmark:
 The image size used for the test is 1280 x 1024
 derivate_gauss is used with sigma 5.0 and filter mode ‘gradient’
37© Vision Academy, Erfurt
Increased usability:
Generate code by configuring with assistants
38© Vision Academy, Erfurt
Increased usability:
Generate code by configuring with assistants
Modern integrated development environments for machine vision offer
 tools for visualization,
 prototyping,
 parameter tuning,
 Programming,
 and debugging
and all of these optimized for image processing.
Some environments additionally offer assistants to manage even complex tasks like image
acquisition, calibration and matching with a few mouse clicks over an intuitive GUI.
The assistants allow to make use of important functionality in a self-explanatory and purely
interactive way. So the desired results are available within only a few minutes. Although
everything is done interactively, the corresponding code can still be generated
automatically.
39© Vision Academy, Erfurt
Increased usability:
Automatic learning of image processing parameters
40© Vision Academy, Erfurt
Increased usability:
Automatic learning of image processing parameters
Tools like data coder readers are continuously being improved, so that they read data
codes even under extremely difficult conditions.
Besides this, some libraries offer the automatic optimization of parameters, by just giving a
few sample images to the code reader. Based on these, the systemadapts the parameters
so that speed and robustness are increased – without extra programming.
Both kinds of improvements highly ease the development and the setup of a system.
41© Vision Academy, Erfurt
Summary
Machine vision offers a broad range of
technologies for all kinds of applications
Virtually every industry benefits from cost
reduction, increased throughput and quality
According to the available technical
knowledge adapted programming and
configuration tools are available
New developments offer new inspections
tasks and increased performance