Murari Image Processing JET - ENEA - Fusione

molassesitalianΤεχνίτη Νοημοσύνη και Ρομποτική

6 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

47 εμφανίσεις

Latest Developments in Image Processing on
JET



by
Andrea Murari
1
, J.Vega
2
, T.Craciunescu
3
, P.Arena
4
,
D.Mazon
5
, L.Gabellieri
6
, M.Gelfusa
7
, D.Pacella
6
, S.Palazzo
4
,
A.Romano
6
, J.F.Delmond
8
, A. De Maack
9

, T.Lesage
8

1

2

4

5

3

9

6

8

7 University of Rome


T潲oV敲条瑡


6

CODAS: Raw Data

Total Raw data: a record
of almost 35
Gbytes

per
shot has been reached
which keeps JET increase
in stored information in
line with the Moore law.

JET Database exceeds
100 Terabytes


About 50% are images


Cameras: Visualization

In total more than 30
cameras operational (
PIW
protection
).

New visualization tools
are indispensable for
the analysis (PinUp)

A new specialist is
rostered in the control
room: the VSO (Viewing
Systems Officer)

Goals of Imaging in JET

Goals of imaging:

o

Imaging of the IR emission from the wall for
portection and physics studies

o

Imaging of edge instabilities (ELMs, MARFEs
etc) for phyics and to assess their effects on
the wall.

o

Overview of the general discharge behaviour

Issues of Imaging in JET

Issues posed by the exploitation of images:

o

Information retrieval (discussed in detail last
meeting)

o

Image registration
(vibrations and interference)

o

Integration of models (see V.Martin Talk)

o

Real time identification of events

o

Extraction of quantitative information for
physics studies (see T.Craciunescu Talk)

Mathematical indicators


8 different mathematical indicators for
vibration detection have been investigated:


Normalized cross
-
correlation


Shannon entropy


Tsallis entropy


Renyi entropy


Alpha entropy


Shannon mutual information


Tsallis mutual information


Renyi mutual information

Entropy

Mutual
information

Normalized cross
-
correlation

Normalized cross
-
correlation


Shannon Entropy


Tsallis

entropy

/
Sq

entropy


q : degree of non
-
additivity

Equal

when

q


1

p
i

: probability of finding the system in
each possible state i (or residual i)

k : Total number of possible states

(or number of possible residuals)


Additive and Non additive entropy

Applications of non additive
entropy

Shannon entropy is additive because it assumes that there are
no correlations between the systems being added



Tsallis entropy is not additive because it can take into account
these correlations.

Tsallis entropy is not additive. For a sum of two systems
A1

and
A2



Tsallis entropy is finding many applications from
statistical mechanics to signal processing, image
processing etc

S
q

(A1 + A2 ) = S
q

(A1) + S
q

(A2) + (1
-
q) S
q

(A1) S
q

(A2)



In the case of camera movements, the difference between
two frames presents long range correlations


These long range correlations, which are less pronounced, in
case of objects moving in the still field of view of a camera,
can be emphasised by the proper selection of q in the
Tsallis

entropy.

Application of Tsallis entropy
to image registration

Tsallis

Entropy
:
higher

sensitivity


Shannon Entropy : 0


Sq entropy : 0


Shannon Entropy :
0.61


Sq entropy :
3.16


Shannon
Entropy

:
0.81


Sq

entropy

:
3.99


Background
Matrix


Object
Matrix

q=0.1


Shannon
Entropy

:
+0.23


Sq

entropy

:
+0.83

Red: Tsallis entropy versus row shift

Blue: Shannon entropy vs row shift

Mutual information

Renyi definition

The Wide Angle Camera KL7
provides a view of the main vessel
in the IR


The Camera seats at the end of and
endoscope with many optical components
whose position is not monitored



No reliable reference points in the field
of view

Image registration: diagnostic


All the major typical events are included


Plasma current between 2 and 3.5 MA


Toroidal

field between 1.9 and 3.4T

Statistics of frames observed in JET


A database of 69 videos and almost 40000 frames has been
analysed manually to determine the cases with movements.


Comparison Entropies

The vertical lines indicate the period with vibrations

Comparison Mutual Informations

Statistics: Threshold


Method: determination of a threshold discriminating
between the frames with and without movements

No
mouvement

Mouvement

Succes Rate: Overview

Conclusions
Threshold
% of good
results
Frame where no
movement is
wrongly
detected
Frame where
movement is
wrongly
detected
Normalized
cross
-
correlation
0.94
71.66
14.84
3.78
Shannon
entropy
1.6
84.17
15.35
0.48
Shannon
mutual
information
0.62
78.09
0.47
21.44
Tsallis
entropy
25
86.19
6.66
7.15
Tsallis
mutual
information
0.58
79.98
0.48
19.54
Renyi
entropy
8
84.70
15.14
0.16
Renyi
mutual
information
1.28
79.80
2.58
17.62

The result is that entropy of Tsallis
is the best among the other
entropies.


The mutual
information
with
Tsallis
definition is
the best definitions
among from the
definition of mutual information and NCC.


Success Rate: missed and false alarms

86,19%
6,66%
7,15%
Tsallis entropy analysis
Correct analysis
Frame where no mouvement is wrongly detected
Frame where mouvement is wrongly detected
False alarms

Missed alarms

Succesfull
identifications

Registration: Method Comparison


A synthetic videos has been shifted by 10 rows and then two
of the best indicators have been tried to register it.


Shift

Application to video 73851, frame 786


Frame 786 is chosen among
frames with vibrations.
The result of the Tsallis mutual information, which is
shown below, is the matrix must be shift by two rows
leftwards.

Verification

Mean(value of
pixel)=1.3864

Mean(value of
pixel)=1.2788


Subtraction of the frame affected by the movement and
the reference frame before and after the registration
shows a clear improvement. More effective in the main
chamber because the divertor is affected by ELMs

Image Analysis: Hot spot detection

The white areas represent the
potential hot regions, parts of the
wall which reach a to high
temperature.

11,300 frames have been analysed
manually

A C++ algorithm to be run on a
serial machine has been developed
to automatically identify the hot
spots (100% success rate in terms
of image processing not physics)


Infrared Wide Angle
View: Size of IR
images: 496x560
pixels

Assumption: the temperature
map provided is correct

Reference serial algorithm:
computational time


For

traditional

serial

algorithms,

the

computational

time

depends

on

the

content

of

the

image
.

A

potential

problem

for

real

time

applications

Computational
time versus
number of
white pixels

Computational
time
evolution
during a
discharge


Array

of

cells


Information

for

each

cell
:


State

(mapped

to

greyscale

value)


Input


Output

(dependent

on

state)


Each

cell

is

connected

to

a

set

of

neighbours

(usually

belonging

to

a

3
x
3

square)


A

state

equation

defines

the

time

evolution

of

the

cell
:

Cellular Nonlinear Networks





ij
j
i
S
l
k
C
kl
j
i
S
l
k
C
kl
ij
ij
z
u
l
k
j
i
B
y
l
k
j
i
A
x
x
r
r









)
,
(
)
,
(
)
,
(
)
,
(
,
;
,
,
;
,

where

x
ij

is

the

state

of

the

cell,

y
kl

the

output

and

u
kl

the

input
.


CNNs

are

a

new

computational

paradigm
.

If

supported

by

an

adequate

memory

they

have

the

same

computational

power

of

Universal

Turing

machines

but

with

the

benefit

of

parallelism
.



A,

B
:

feedback

and

input

synaptic

operators


They

define

how

the

state

evolves

and

how

neighbour

cells

influence

it
.


For

image

processing,

they

define

the

kind

of

filter

implemented

by

the

CNN,

and

are

usually

3
x
3

matrices

a
-
1,
-
1

a
-
1,0

a
-
1,1

a
0,
-
1

a
0,0

a
0, 1

a
1,
-
1

a
1,0

a
1,1





ij
j
i
S
l
k
C
kl
j
i
S
l
k
C
kl
ij
ij
z
u
l
k
j
i
B
y
l
k
j
i
A
x
x
r
r









)
,
(
)
,
(
)
,
(
)
,
(
,
;
,
,
;
,


z
ij

is a bias constant.


The set
(A, B, z)

is called a
template
.

Nonlinear
(morphological) operators can be implemented





y
i
-
1,j
-
1

y
i
-
1,j

y
i
-
1,j+1

y
i,j
-
1

y
i,j

y
i,j+1

y
i+1,j
-
1

y
i+1,j

y
i+1,j+1

Summation of dot
products

Cellular Nonlinear Networks

1.
Directed Growing Shadow


This

template

create

“shadows”

from

white

pixels

by

increasing

the

objects
.

The

template

was

customized

so

that

the

main

direction

of

growth

is

horizontal
.


This template allows
merging small close
regions


this corresponds
to the clustering operation
of the serial algorithm.


To be classified
as hot spot


To be eliminated

2.
ConcaveFiller


The

ConcaveFiller

template

is

applied

in

order

to

avoid

that

the

following

shrinking

phase

might

separate

the

regions

unified

by

DirectedGrowingShadow
.

S.Palazzo, A.Murari et al
REVIEW OF SCIENTIFIC
INSTRUMENTS
81
,
083505 2010

3.
Object Decreasing


Object

Decreasing

is

applied

in

order

to

rescale

the

objects

back

to

their

original

size,

while

keeping

the

merge

regions

united
.


Object Removal
allows
to remove “small
objects”


How to implement different processing algorithms to
different parts of the images?



Space
-
varying CNNs


The implementation approach is based
on the definitions of
regions

in the
input image.


The image is divided into a grid of
rectangular cells (regions), by
specifying the coordinates of the
grid’s rows and columns.


Each region is then assigned its own
sequence of templates, which can
differ from other regions in terms of
number of templates to be applied,
number of iterations or templates’
coefficients. Mathematics already
developed.


The total computation time will depend
on the longest template sequence
among all regions.

CNN implementation on FPGA


Core

array

architecture


A

core

takes

as

input

a

stripe

of

the

image

(or

the

output

of

the

upper
-
row

core)

and

computes

the

next

iteration
.


All

cores

in

a

column

process

the

same

part

of

the

image
.


All

cores

in

a

row

execute

the

same

iteration

(on

different

input

stripes)
.


Parallelism

is

provided

by

adding

columns

to

the

array



that

is,

by

dividing

the

image

into

more

parts,

to

be

independently

processed
.

Hot spot detection


The new algorithm divides the image
into different number of regions on
which it is possible to:


Apply customized temperature thresholds,
for example a higher one in the bottom
-
left
divertor’s region.


Apply region
-
specific template sequences,
in order to improve the global detection
accuracy.


Deterministic

computational

time


Implementation

with

FPGA

using

cores


Total

computation

time

with

a

100

MHz

clock

and

1

column

of

cores
:

10
6


10

ns

=

10

ms



Maximum

frame

rate
:

100

fps


It

is

possible

to

increase

the

frame

rate

by

adding

parallelism,

i
.
e
.

more

columns

in

the

core

array

architecture
.

With

a

10
-
column

core

array,

the

computation

time

is

reduced

to

1

ms
,

and

the

maximum

input

frame

rate

becomes

1000

fps
.


Conclusions

o

Bidimensional measurements are the new frontier in
plasma physics (they are a step forward comparable
to profiles)

o
Videos contain a wealth of information which can
give a very significant contribution to both the
understanding of the physics and the real time
control of fusion plasmas (including protection)

o

Image manipulation: many tools are on the market
but they are not always exactly what is needed and
therefore significant level of development is required


Apha entropy

Tsallis definition

Results


The figure below shows the I
α

entropy. This
entropy does not provide coherent and
understable results so it will not be used in the
following.