THERMOBOT: TOWARDS SEMI-AUTONOMOUS, THERMOGRAPHIC DETECTION OF CRACKS

lynxherringAI and Robotics

Oct 18, 2013 (3 years and 5 months ago)

63 views




T
HERMOBOT
:
TOWARDS SEMI
-
AUTONOMOUS,
THERMOGRAPHIC DETECTION OF CRACKS


C. Eitzinger
(
1
)
, S. Ghidoni
(2)
,
E.Menegatti

(
2
)




(
1
)

Profactor GmBH, Im Stadtgut A2, 4407 Steyr
-
Gleink, Austria

(
2
)
Department of Information Engineering
, University of Padova

Via
Gradenigo 6/A

I
-
35131 Padova, Italy









ABSTRACT.
The
detection of
cracks in parts of complex geometry
requires a cumbersome
pro
c
ess based on

magnetic particle inspection

. This includes the application and removal of
liquids and is difficult to autom
ate. In this paper a semi
-
autonomous system for crack
detection is proposed that uses a robot to move a part in front of a thermographic image
acquisition system
. At th
e heart of the inspection system is a l
aser

combined with a thermo
-
camera
that
will prov
ide image information t
o

enable the robust detection of crack
s

on or near
the surface.

The analysis of the heat flow will reveal any inhomogen
e
ities such as cracks in
the part.
This is combined with automatic path planning for the robot to

enable the

inspe
ct
ion
of complex parts. The system concept is presented and details about the various system
components are explained.


INTRODUCTION

Despite considerable progress in the automation of production processes visual inspection of

products and parts is still d
one manually in a wide variety of inspection tasks.
The
current
procedure for crack detection is a process that dates back to the 1920s and is called


magnetic
particle inspection
”. This method is infamous in industry, because it is a

cumbersome, dirty
pro
cess that is often done manually even in otherwise fully automatic

production lines. The
component to be tested is magnetized before applying a suspension of

finely divided coloured
or fluorescent magnetic particles. Cracks or inclusions cause the

magnetic

flux to break the
surface forming free magnetic poles and the magnetic particles will

collect at these locations
indicating a crack. Using UV
-
light the fluorescent particles in the

suspension are activated to
increase the visibility of the cracks. Inspect
ion

of complex parts is usually done manually,
whereas for simple geometries machine vision

systems can be used for automatic detection.

The project ThermoBot
aims at replacing this old method for crack detection with a new
technology that

is based on auto
nomous inspection robots using thermography to recognize
cracks on parts of

complex geometry. The robot will scan the whole part with a thermo
-
camera and analyse the

heat
-
flow to find cracks and other defects hidden under the surface.
To achieve this, the

project
aims at making progress in the following areas
:



thermographic process
es

and process model
s

for the automatic

detection of cracks in
parts of complex geometry, varying surface structures and for different

materials.



A
utomatic path and motion plannin
g
that uses
a
thermographic process model to
automatically generate a path for the

inspection robot from 3D CAD data.



T
hermo
-
image analysis methods

that not only aim at detecting cracks

and making an
accept/reject decision for the whole part, but also hav
e the capability of self
-
evaluating
the
ir

performance
in

the crack detection.

The following sections will describe the state of the art with respect to these areas and
explain the progress that has been made so far in solving the open technological and sc
ientific
questions.


STATE OF THE ART


Models of thermographic imaging

Most frequently the models used in applications

of thermography

have their focus on
conduction of heat in solids in the context of the energy source, which is used to heat up the
spec
imen under inspection

[1,

2,

3,

4]. Heat transport is the main mechanism that enables
the
rmography to
detect defects such as cracks. In addition to heat flow in
side

the part two
additional models are required to fully cover the thermographic image acquisit
ion process.
The first one deals with the radiation from the surface of a specimen, which includes the
actual (heat) signal as well as components [5] such as noise, reflections from environment or
the influence of the surface properties. The second one, wh
ich is much simpler, deals with the
properties of sensor and imaging considerations [6,

7].

Adding radiation from environment,
on
e

creates a model that describes the “radiometric measurement problem”

[8
]. The model for
description of the surface temperatur
e distribution and evolution is based on heat production
and transfer in a solid. For crack detection an in
-
plane flux must be induced, to obtain
measurable temperature differences along the crack. This can be achieved by heating up only
a small area of th
e surface. From this “hot spot”, heat will flow mainly in parallel to the
surface, and this flow is interrupted by a crack. To improve robustness

of the detection
, an
evaluation method [
9
] was developed, that identifies heat flux phenomena by watching the
evolution of temperature profiles. Description of temperature as consequence of heat
conduction is based on the differential equations of heat conduction, for which explicit
solutions exist only for simple models [1
0
,

1
1
]. In more complex situations simula
tions [1
2
]
such as finite elements methods (FEM) are used to predict signals [1
3
]. In case of real time
applications, current simulation methods would take too much time, so it is mainly used for
design purposes.


Robot path planning for inspection tasks

To calculate a path on a 3D object that can be used for inspection, several constraints need to
be adhered. For example, the camera has to be in a certain (range of) distances from the part
and it has to view the relevant surface under a certain angle. Thi
s pro
blem is inherent in many
camera
-
based inspection problem
s
, such as completeness inspection [1
4
] or for automated 3D
object reconstruction [
15
]. It aims at the automatic generation of an (optimal) sequence of
viewpoints that satisfy certain constraints
, such as full coverage of the part

[
16
]
. This is
known as


view planni
ng problem”. In order to solve the associated optimization problem
algorithms that can deal with discrete search spaces, such as genetic algorithms [
17
] or
particle swarm optimization a
re regularly used.


Thermo
-
image analysis

Most of the systems proposed in literature for thermographic image processing are quite
basic: the processing work flows are composed of thresholding, edge detection, region
growing, and template matching [
1
8]. H
owever, more advanced image processing techniques
have been proposed. Besides traditional techniques coming from the field of computer vision,
several specific methods have been developed for thermal image processing [1
9
]. These
unique techniques are some
times based on the underlying heat
-
conduction physics. The
methods are used either at image preprocessing and/or processing stages [
20
]. For instance,
thermal contrasts have the advantages of being little sensitive to noise and to th
e surface
optical prope
rties
. Another image processing technique which proved to be useful for image
segmentation is pulsed phase thermography (PPT). This is a processing method in which the
thermal images are transformed from the time domain to the frequency domain to calculate

the phase and the amplitude of the signal in the images [
21
]. The phase is particularly
advantageous since it is less affected by environmental reflections, emissivity variations, non
-
uniform heating, surface geometry and orientation.
It should be noted f
or the sake of
completeness that all the image segmentation algorithms developed in computer vision are also
applicable in thermo
-
image analysis. We just cite [
22
] and [
23
] as recent results that explain
adaptive image segmentation methods as well as optim
ization methods that can automatically tune
segmentation algorithms to a particular task.


OUR APPROACH


System overview

The hardware components of the automatic inspection system include a robot, a heating system (in
our case a laser unit), a thermocame
ra and a processing unit. While the robot itself can be any
general purpose robot, the robot’s control unit has to fulfill the requirement that accurate, time
-
stamped position information (joint angles) can be acquired at high frequency. This is required t
o
synchronize image acquisition and robot motion. The laser unit’s main function is to locally heat
the part. The specific parameters of the laser substantially depend on the thermodynamic properties
of the parts to be inspected. For the thermocamera the k
ey feature


aside from high resolution


is
a sufficiently high frame rate to allow the acquisition of image sequences at high frame rates. This
is particularly important for metallic parts, where the heat dissipates in less than a second. A central
proce
ssing unit links all the component and collects the robot’s position as well as the camera
image for each time instance. These data are then processed by image segmen
t
ation and
classification algorithms to distinguish betwe
e
n different types of defects and

to find a final
good/bad classification for the whole part.

It should be noted that for practical reasons in many applications the robot will be holding the part
and not the camera/laser unit.


Model of thermographic imaging

The thermographic imaging mod
el is at the heart of the path planning
algorithm
. The model
basically answers the question which are
a

on the part can be inspected
, if the part is placed in a
certain position and orientation under the camera.




Figure
1
. Area that can be inspected (basic concept).


The area that can be inspected is determined by multiple factors. In the most simple case of a
flat surface placed at an ideal angle under the camera and laser the area will have a ring
-
shaped form as shown in figure
1
. At the cent
re

of the ring there is the laser spot that results in
a bright, white region in the image, where the pixels of the thermocamera are fully saturated
and no analysis is possible. As the heat dissipates isotropic
ally the signal becomes weaker
until no more contrast is achieved. This area defines the outer edge of the ring
-
shaped region.
During inspection the ring is wandering across the surface leaving a trail wh
ere the part has
been inspected
.
Cracks will become
visible as a temperature gradient that is caused by the
interruption of the heat flow.
It should be noted that the sensitivity of the detection depends
on the orientation of the crack relative to the
laser spot. If the crack is propagating radially
from th
e laser spot, it will not be visible as there is no heat flow across the crack.

In the more realistic case of a non
-
flat part, the situation becomes significantly more complex.
The model has to consider that laser and camera are not placed in an ideal po
sition relative to
the part’s surfaces and that the heat propagates in a non
-
flat area. An approximation of the
area that can be checked, may be obtained by projecting the ring
-
shaped region onto the part’s
3D surface. Additionally, self
-
occlusions of the
part have to be considered as well as areas of
high curvature, where the above mentioned approximation is invalid. Those areas have to be
excluded from the region that can be checked. The calculations that are involved in such
estimations can only be done
numerically.


Robot path planning

The task of path planning is to generate a sequen
c
e of viewpoint
s

through which the robot has to
pass. This
sequenc
e
ha
s

to fulfill a few strict boundary conditions:



all relevant areas on the part’s surface have to be cov
ered by acquiring images from
these viewpoints
;



when passing through the vi
ewpoints the part must not collide with the robot, the
camera or other devices in the workspace of the robot;



all the viewpoints must be within the reach of the robot.


These condit
ions apply to all robot
-
based inspection systems that include camera as the main
sensor system. Thermography, however, has one additional constraint that is quite difficult to
fulfil
. In order to do a proper analysis of the heat flow, each position on the
part has to be
recorded in several images at specific time intervals. Therefore an additional requirement is
that



the path of the robot has to follow a specific trajectory, that ensures that the
viewpoints are reached at a particular time.


It should be no
ted that the motion of the robot has to be continuous and not a start/stop motion
as is possible e.g. in standard machine vision applications.

This set of boundary conditions still allow a large (actually infinite) number of different
solutions for the se
quence of viewpoints that is chosen for the particular inspection task.

Even
if the restrictions of the robot’s kinematics are fully considered, there is still a wide range of
different paths.

Therefore, an additional optimization cr
iterion is required tha
t reduces

the set
of possible solution to (ideally) a single on
e
. In many real
-
world applications this
optimization criterion corresponds to



the time required for inspecting all areas on the part should be minimal.


While this optimization criterion may s
ound trivial it poses a number of challenges.
Estimating the time required for following a complex path in 6 degrees of freedom requires
detailed knowledge about the dynamical behaviour of the robot as well as detailed
consideration of the thermodynamic pr
ocesses that are involved.


The starting point for path planning is a 3D CAD model of the part. This model is enhanced
by splitting the region into small elements that correspond t
o one
main position (“
process
points
”) of the inspection process. Each surfa
ce element is augmented with an outward
pointing normal vector that points towards the ideal position of the
laser/cam
era unit as shown
in figure 2.



Figure
2
. Process points on small section of a crankshaft

with a small outward pointing
normal vector.


Given the fact that there is an ideal operating distance for the image acquisition unit, a space
of ideal positions can be generated from which the surface can be inspected.

Tests on objects of different complexity included parts with an essentially flat
surface that
have only low curvature as well as more complex parts such as the crankshaft shown in the
figure above. Path planning was able to generated useful paths for both of these test parts. The
current implementation of the path planning just tries t
o find a feasible path for the robot that
complies with the above mentioned boundary conditions. It does not yet use an optimization
criterion to e.g. reduce the time needed for the inspection task. For inspecting the relevant
regions on a whole crank shaf
t a total time of a few minutes is required, but there is still
substantial potential for optimization so that cycle times below 1 minute can be expected.


Thermo
-
image analysis

Automatic processing of thermo
-
image was implemented for two kinds of sample p
arts
: a
metal crankshaft and a carbon fiber sideb
lade, shown in figure 3.




Figure 3. Sample parts tested using the thermo
-
image analysis: a carbon fiber sideblade (left)
and a metal crankshaft (right).


Such sample parts were cut into smaller
porti
ons

that were analyzed, in order to ease the
acquisition process. The thermal excitation method is different for the two types of material,
because a strong power is needed for changing the thermal state of metal, while carbon fiber
would be damaged by
suc
h
a strong heating. A high
-
power laser was used in the former case,
while small portions of carbon fiber parts were heated using
a thermal chuck
, even though in
the final system configuration this will not be feasible because of the complex geometry of
suc
h
part
s
.

Image analysis techniques employed for crack detection on metal parts
focus on the laser spot.
The high amount of energy concentrated in a small spot causes a strong heat flow to the
surrounding areas. Cracks modify the heat transfer profile
, and
can therefore be detected by
measuring the regularity of the heat status around the laser spot.

An example of detection
result can be seen in figure 4

(left): a large crack cause
s

the heat transfer to be less effective
towards the left part of the image. T
he analysis of heat gradients depends on multiple
parameters, including the laser power and the speed at which it scans the part under
inspection. Patterns are in fact circular at lower speed, around 40 mm/s, while they become
oval
-
shaped

at higher speed.
Laser power also has an impact, since it should be strong enough
to cause heat transfer
s
visible in the thermo
-
images. Common values are around 7
-
10 W.

Images of carbon fiber parts
show a more uniform thermal status, given by the different
heating system
:
the thermal chuck has a surface
size
similar to
that of
the sample
. The parts
are framed from one side, and heated on the opposite
face
. The
alteration of the thermal status
of the part
is capable of highlighting intrusion of unwanted material inside the f
iber element.
Image processing is exploited to segment the single fiber elements and to analyze the content
of each one, that must be uniform without strong gradients. In figure 4 (right) the output of
the image analysis is shown: fiber elements are marked

using elliptical contours, whose color
indicates whether the element has uniform color (green) or not (red). Some false positives are
visible towards the borders, because the contrast is too low in such locations.
This will be
fixed in the final system co
nfiguration, when images acquired with the final heating system
will be available.



Figure

4. Automatic detection of a crack in a metal part (left) and carbon fiber element
segmentation and analysis (right).


CONCLUSIONS

In this paper we presente
d the progress that was made towards the implementation of a system
for the semi
-
autonomous detection of cracks in complex parts
.
The system is based on
thermography and uses a robotic system to position the part in front of the camera. Automatic
path plan
ning allows a quick setup of the system for new geometries. First results indicated
that a robust detection of the crack is possible using image analysis methods and that the
automated path planning allows the inspection of a complex part in a few minutes.

Future
work includes the optimization of all components in order to speed up the inspection process
and the setup of the whole inspection system. Additional work on defect classification and
extending the range of possible materials is also needed.


ACKNO
WLEDGMENTS

The research leading to these results has received funding from the European Union Seventh
Framework Programme (FP7/2007
-
2013)under grant agreement No. 284607.

REFERENCES

[1] K.
-
J. Kremer, W. Kaiser, P. Möller

(1985)
: Das Therm
-
O
-
Matic
-
Verfahr
en


ein
neuartiges Verfahren für die Inl
i
ne
-
Prüfung von Stahlerzeugnissen auf Oberflächenfehler.
Stahl u. Eisen 105, 71
-
76

[2] J.Baumann, U.Netzelmann, R.Bilgram, T.Hierl, V
.Carl, H.R.

Subach, G. Zenzinger.
(2005)

“Untersuchungen zu aktiver thermografisch
en Prüfmethoden zur Absicherung von
Hochleistungsfertigungsverfahren


Ergebnisse eines BMBF
-
Projekts”, DGZfP
-
Berichtsband
94, DGZfP
-
Jahrestagung, Rostock

[3] B. Oswald
-
Tranta
. (2004)
: Thermoinductive Investigations of Magnetic Materials for
Surface Cracks
.
Verformungskundliches Kolloquium 23, 229
-
237, Herausgeber: B.
Buchmayr, Lehrstuhl für Umformte
chnik, Montan Universität

[4] Xavier P. V. Maldague
. (2001).

Theory and practice of infrared technology for non
-
destructive testing, ISBN 0
-
471
-
18190
-
0

[5]
E.G
rinzato, V.Vavilov, P.G.Bison, S.Marinetti, C.Bressan
. (1995)
, Methodology of
processing experimental data in Transient Thermal NDT, SP
IE Vol. 2473 / 167,

[6]
A. Rogalski, K. Chrzanowski. (2002).

Infrared devices and techniques, Opto
-
Electron.
Re
v., 10, no
. 2

[7]
G.Eppeldauer, M.Racz. (2000).

Spectral power and irradiance responsivity

calibration of
InSb working
-
standard radiometers, APPLIED OPTICS, November 2000, Vol. 39, No. 31

[
8
]
H.Kaplan.(2007).

Practical Applications of Infrared Thermal Sensing and
Imaging
Equipment, SPIE 2007, ISBN 978
-
0819467232

[
9
]
G.Traxler, P.Th
anner, W.Palfinger, A.Poetscher. (2010)

Advanced evaluation for
thermographic crack detection with inductive exitati
on for steel billets,

Budapest, 20th
manufacturing conference

[
10
]
H.S.
Carslaw, J.C.Jaeger. (1959)

Conduction of Heat in Solids, 2nd Edition, Oxford
University Press, ISBN 0
-
19
-
853368
-
3

[11
] Özisik, M.

N. (1989). Boundary Value Problems of Heat Conduction. Dover Publications
Inc., New York, first edition

[12
]
U.Siemer. (2006
)
Simulation and Evaluation of New Thermographic Techniques for the
Deployment in the Automotive Industry
, ECNDT
-

European conference on non
-
destructive
testing.

[13
] H.D.Baehr, K. Stephan: Wärme und Stoffübertragung, Springer 5.Auflage, ISBN
-
10 3
-
540
-
3233
4
-
1

[14] T.H.

Park, H.J. Kim, and N. Kim.( 2006).

Path Planning of Automated Optical
Inspection Machines for PCB Assembly Systems,
International Journal of Control,
Automation, and Systems, vol. 4, no. 1, pp. 96
-
104, February 2006

[15] W. R. Scoot and G. R
oth. (2003).

View Planning for Automated Three
-
Dimensional
Object Reconstruction and Inspection,
ACM Computing Surveys, Vol. 35, No. 1, March
2003, pp. 64

96.


[16
] Tarbox, G., Gottschlich , S.
(

1995
)
. Planning for complete sensor coverage in
inspection.

Comput. Vis. Image Understand. 61, 1 (Jan.), 84

111.

[17
] C. G. Lu, D.

Morton, M. H. Wu and P. Myler. (1999)
Genetic Algorithm Modelling and
Solution of Inspection Path Planning on a Coordinate Measuring Machine (CMM),
International Journal of Advanced Ma
nufacturing Technologies (1999) 15:409

416

[
1
8] M. Pilla, M.

Klein, X. Maldague, A. Salerno. (2002)

New Absolute Contrast for Pulsed
Thermography, in: D. Balageas, G.M. Busse, G. Carlomagno (Eds.), QIRT 2002, pp. 53

58.

[1
9
] C. Ibarra
-
Castanedo, D. Gonzale
z, M. Klein, M. Pilla, S. Vallerand, X. Maldague
.
(2004).

Infrared image processing and data analysis, Infrared Physics & Technology, Volume
46, Issues 1
-
2, Workshop on Advanced Infrared Technology and Application, December
2004, Pages 75
-
83, ISSN 1350
-
449

[20] X. Maldague. (2001)

Theory and Practice of Infrared Technology for NonDestructive
Testing, John Wiley
-
Interscience, 2001, p. 684

[21
] Maldague X. and Marinetti S., “Pulse Phase Infrared Thermography”, J.Appl. Phys., Vol.
79, pp. 2694
-
2698, 1996.

[22
]

Peters S.

(2008)
.

Automat
ed Design of a Hybrid Texture A
n
a
ly
sis System for Defect
Detection.,

OAGM Workshop, Hagenberg, 2008

[23
]
Martin V., M. Thonnat. (2007).
A Learning Approach for Adaptive Image Segmentation,
in Scene Reconstruction, Pose Estimation
and Tracking, Book edited by: Rustam Stolkin,
ISBN 978
-
3
-
902613
-
06
-
6, pp.530, I
-
Tech, Vienna, Austria, June 2007