Latest Developments in Image Processing on
JET
by
Andrea Murari
1
, J.Vega
2
, T.Craciunescu
3
, P.Arena
4
,
D.Mazon
5
, L.Gabellieri
6
, M.Gelfusa
7
, D.Pacella
6
, S.Palazzo
4
,
A.Romano
6
, J.F.Delmond
8
, A. De Maack
9
, T.Lesage
8
1
2
4
5
3
9
6
8
7 University of Rome
“
T潲oV敲条瑡
”
6
CODAS: Raw Data
Total Raw data: a record
of almost 35
Gbytes
per
shot has been reached
which keeps JET increase
in stored information in
line with the Moore law.
JET Database exceeds
100 Terabytes
About 50% are images
Cameras: Visualization
In total more than 30
cameras operational (
PIW
protection
).
New visualization tools
are indispensable for
the analysis (PinUp)
A new specialist is
rostered in the control
room: the VSO (Viewing
Systems Officer)
Goals of Imaging in JET
Goals of imaging:
o
Imaging of the IR emission from the wall for
portection and physics studies
o
Imaging of edge instabilities (ELMs, MARFEs
etc) for phyics and to assess their effects on
the wall.
o
Overview of the general discharge behaviour
Issues of Imaging in JET
Issues posed by the exploitation of images:
o
Information retrieval (discussed in detail last
meeting)
o
Image registration
(vibrations and interference)
o
Integration of models (see V.Martin Talk)
o
Real time identification of events
o
Extraction of quantitative information for
physics studies (see T.Craciunescu Talk)
Mathematical indicators
•
8 different mathematical indicators for
vibration detection have been investigated:
•
Normalized cross

correlation
•
Shannon entropy
•
Tsallis entropy
•
Renyi entropy
•
Alpha entropy
•
Shannon mutual information
•
Tsallis mutual information
•
Renyi mutual information
Entropy
Mutual
information
Normalized cross

correlation
Normalized cross

correlation
Shannon Entropy
Tsallis
entropy
/
Sq
entropy
q : degree of non

additivity
Equal
when
q
1
p
i
: probability of finding the system in
each possible state i (or residual i)
k : Total number of possible states
(or number of possible residuals)
Additive and Non additive entropy
Applications of non additive
entropy
Shannon entropy is additive because it assumes that there are
no correlations between the systems being added
Tsallis entropy is not additive because it can take into account
these correlations.
Tsallis entropy is not additive. For a sum of two systems
A1
and
A2
Tsallis entropy is finding many applications from
statistical mechanics to signal processing, image
processing etc
S
q
(A1 + A2 ) = S
q
(A1) + S
q
(A2) + (1

q) S
q
(A1) S
q
(A2)
In the case of camera movements, the difference between
two frames presents long range correlations
These long range correlations, which are less pronounced, in
case of objects moving in the still field of view of a camera,
can be emphasised by the proper selection of q in the
Tsallis
entropy.
Application of Tsallis entropy
to image registration
Tsallis
Entropy
:
higher
sensitivity
Shannon Entropy : 0
Sq entropy : 0
Shannon Entropy :
0.61
Sq entropy :
3.16
Shannon
Entropy
:
0.81
Sq
entropy
:
3.99
Background
Matrix
Object
Matrix
q=0.1
Shannon
Entropy
:
+0.23
Sq
entropy
:
+0.83
Red: Tsallis entropy versus row shift
Blue: Shannon entropy vs row shift
Mutual information
Renyi definition
The Wide Angle Camera KL7
provides a view of the main vessel
in the IR
•
The Camera seats at the end of and
endoscope with many optical components
whose position is not monitored
•
No reliable reference points in the field
of view
Image registration: diagnostic
All the major typical events are included
Plasma current between 2 and 3.5 MA
Toroidal
field between 1.9 and 3.4T
Statistics of frames observed in JET
•
A database of 69 videos and almost 40000 frames has been
analysed manually to determine the cases with movements.
Comparison Entropies
The vertical lines indicate the period with vibrations
Comparison Mutual Informations
Statistics: Threshold
•
Method: determination of a threshold discriminating
between the frames with and without movements
No
mouvement
Mouvement
Succes Rate: Overview
Conclusions
Threshold
% of good
results
Frame where no
movement is
wrongly
detected
Frame where
movement is
wrongly
detected
Normalized
cross

correlation
0.94
71.66
14.84
3.78
Shannon
entropy
1.6
84.17
15.35
0.48
Shannon
mutual
information
0.62
78.09
0.47
21.44
Tsallis
entropy
25
86.19
6.66
7.15
Tsallis
mutual
information
0.58
79.98
0.48
19.54
Renyi
entropy
8
84.70
15.14
0.16
Renyi
mutual
information
1.28
79.80
2.58
17.62
•
The result is that entropy of Tsallis
is the best among the other
entropies.
•
The mutual
information
with
Tsallis
definition is
the best definitions
among from the
definition of mutual information and NCC.
Success Rate: missed and false alarms
86,19%
6,66%
7,15%
Tsallis entropy analysis
Correct analysis
Frame where no mouvement is wrongly detected
Frame where mouvement is wrongly detected
False alarms
Missed alarms
Succesfull
identifications
Registration: Method Comparison
•
A synthetic videos has been shifted by 10 rows and then two
of the best indicators have been tried to register it.
Shift
Application to video 73851, frame 786
•
Frame 786 is chosen among
frames with vibrations.
The result of the Tsallis mutual information, which is
shown below, is the matrix must be shift by two rows
leftwards.
Verification
Mean(value of
pixel)=1.3864
Mean(value of
pixel)=1.2788
•
Subtraction of the frame affected by the movement and
the reference frame before and after the registration
shows a clear improvement. More effective in the main
chamber because the divertor is affected by ELMs
Image Analysis: Hot spot detection
The white areas represent the
potential hot regions, parts of the
wall which reach a to high
temperature.
11,300 frames have been analysed
manually
A C++ algorithm to be run on a
serial machine has been developed
to automatically identify the hot
spots (100% success rate in terms
of image processing not physics)
•
Infrared Wide Angle
View: Size of IR
images: 496x560
pixels
Assumption: the temperature
map provided is correct
Reference serial algorithm:
computational time
•
For
traditional
serial
algorithms,
the
computational
time
depends
on
the
content
of
the
image
.
A
potential
problem
for
real
time
applications
Computational
time versus
number of
white pixels
Computational
time
evolution
during a
discharge
•
Array
of
cells
–
Information
for
each
cell
:
•
State
(mapped
to
greyscale
value)
•
Input
•
Output
(dependent
on
state)
–
Each
cell
is
connected
to
a
set
of
neighbours
(usually
belonging
to
a
3
x
3
square)
–
A
state
equation
defines
the
time
evolution
of
the
cell
:
Cellular Nonlinear Networks
ij
j
i
S
l
k
C
kl
j
i
S
l
k
C
kl
ij
ij
z
u
l
k
j
i
B
y
l
k
j
i
A
x
x
r
r
)
,
(
)
,
(
)
,
(
)
,
(
,
;
,
,
;
,
where
x
ij
is
the
state
of
the
cell,
y
kl
the
output
and
u
kl
the
input
.
•
CNNs
are
a
new
computational
paradigm
.
If
supported
by
an
adequate
memory
they
have
the
same
computational
power
of
Universal
Turing
machines
but
with
the
benefit
of
parallelism
.
•
A,
B
:
feedback
and
input
synaptic
operators
–
They
define
how
the
state
evolves
and
how
neighbour
cells
influence
it
.
–
For
image
processing,
they
define
the
kind
of
filter
implemented
by
the
CNN,
and
are
usually
3
x
3
matrices
a

1,

1
a

1,0
a

1,1
a
0,

1
a
0,0
a
0, 1
a
1,

1
a
1,0
a
1,1
ij
j
i
S
l
k
C
kl
j
i
S
l
k
C
kl
ij
ij
z
u
l
k
j
i
B
y
l
k
j
i
A
x
x
r
r
)
,
(
)
,
(
)
,
(
)
,
(
,
;
,
,
;
,
•
z
ij
is a bias constant.
•
The set
(A, B, z)
is called a
template
.
Nonlinear
(morphological) operators can be implemented
y
i

1,j

1
y
i

1,j
y
i

1,j+1
y
i,j

1
y
i,j
y
i,j+1
y
i+1,j

1
y
i+1,j
y
i+1,j+1
Summation of dot
products
Cellular Nonlinear Networks
1.
Directed Growing Shadow
•
This
template
create
“shadows”
from
white
pixels
by
increasing
the
objects
.
The
template
was
customized
so
that
the
main
direction
of
growth
is
horizontal
.
This template allows
merging small close
regions
–
this corresponds
to the clustering operation
of the serial algorithm.
To be classified
as hot spot
To be eliminated
2.
ConcaveFiller
•
The
ConcaveFiller
template
is
applied
in
order
to
avoid
that
the
following
shrinking
phase
might
separate
the
regions
unified
by
DirectedGrowingShadow
.
S.Palazzo, A.Murari et al
REVIEW OF SCIENTIFIC
INSTRUMENTS
81
,
083505 2010
3.
Object Decreasing
•
Object
Decreasing
is
applied
in
order
to
rescale
the
objects
back
to
their
original
size,
while
keeping
the
merge
regions
united
.
•
Object Removal
allows
to remove “small
objects”
How to implement different processing algorithms to
different parts of the images?
Space

varying CNNs
•
The implementation approach is based
on the definitions of
regions
in the
input image.
•
The image is divided into a grid of
rectangular cells (regions), by
specifying the coordinates of the
grid’s rows and columns.
•
Each region is then assigned its own
sequence of templates, which can
differ from other regions in terms of
number of templates to be applied,
number of iterations or templates’
coefficients. Mathematics already
developed.
•
The total computation time will depend
on the longest template sequence
among all regions.
CNN implementation on FPGA
Core
array
architecture
A
core
takes
as
input
a
stripe
of
the
image
(or
the
output
of
the
upper

row
core)
and
computes
the
next
iteration
.
All
cores
in
a
column
process
the
same
part
of
the
image
.
All
cores
in
a
row
execute
the
same
iteration
(on
different
input
stripes)
.
Parallelism
is
provided
by
adding
columns
to
the
array
–
that
is,
by
dividing
the
image
into
more
parts,
to
be
independently
processed
.
Hot spot detection
•
The new algorithm divides the image
into different number of regions on
which it is possible to:
–
Apply customized temperature thresholds,
for example a higher one in the bottom

left
divertor’s region.
–
Apply region

specific template sequences,
in order to improve the global detection
accuracy.
Deterministic
computational
time
Implementation
with
FPGA
using
cores
Total
computation
time
with
a
100
MHz
clock
and
1
column
of
cores
:
10
6
∙
10
ns
=
10
ms
→
Maximum
frame
rate
:
100
fps
It
is
possible
to
increase
the
frame
rate
by
adding
parallelism,
i
.
e
.
more
columns
in
the
core
array
architecture
.
With
a
10

column
core
array,
the
computation
time
is
reduced
to
1
ms
,
and
the
maximum
input
frame
rate
becomes
1000
fps
.
Conclusions
o
Bidimensional measurements are the new frontier in
plasma physics (they are a step forward comparable
to profiles)
o
Videos contain a wealth of information which can
give a very significant contribution to both the
understanding of the physics and the real time
control of fusion plasmas (including protection)
o
Image manipulation: many tools are on the market
but they are not always exactly what is needed and
therefore significant level of development is required
Apha entropy
Tsallis definition
Results
•
The figure below shows the I
α
entropy. This
entropy does not provide coherent and
understable results so it will not be used in the
following.
Comments 0
Log in to post a comment