TC20100506_designx

streakconvertingΛογισμικό & κατασκευή λογ/κού

13 Δεκ 2013 (πριν από 3 χρόνια και 10 μήνες)

73 εμφανίσεις

Diamon/Laser

Design proposal



Marek Misiowiec

BE/CO/AP

May

20
1
0

Core Overview

Multi
-
tier, distributed, fail
-
safe system processing
diagnostic data


collected repeatedly from a
variety of sources
. Raw data of

different shapes

is transformed into
unified formats

and further

analyzed on a middle layer according to
well
-
defined rules
.








Actions may be taken, in form of
alarms

or
monitoring information

delivered to clients. Processed data is made available as
JAPC

parameters
or for
dedicated GUIs
. It is stored in long term archive

too.
Downward
communication with the source devices is possible.



2

devs

app
s

DAQ

raw
data

unified
data

SRV

clients

client
view

Core Overview


Diamon/Laser core responsibilities:


data acquisition for various sources


push & pull modes


communication through layers, publishing


data transformation, processing, analysis, conditioning


failover, database separation, scaling


complementary functionalities:


rules engine


logging, archiving, configuration updates


non
-
core:


definitions & rules, data provider’s workflow


GUIs, dedicated client applications

3

Importance of a change


Nowadays:


upgrades mess


stale sources, legacy constraints, dusty components


meeting only part of
the
req
uirement
s


partial data, simplistic thresholds, no conditioning


monitoring based on alarm
s
-

D
iamon

on L
aser


misuse of the system, overhead


mediocre scaling


bad
ly
-
behaving sources/clients can
not

be detached/isolated


legacy solutions hiding in Laser world


SonicMQ, OC4J, EJBs, Hibernate, Sleepycat


...and no standard CO components in place


4

Importance of a change



We would like to:


widely use
CO solutions

and help to establish others worth it


better
drive interactions

with the bottom and upper tiers


consolidate

logics in system core


be able to
draw

more
conclusions

on collected data


sustain

upgrades, updates, dependency changes



5

Transition

6

JMS
Notifier
oc
4
j
-
diam
on

Alarm
Archive
Agent Messages
Checks
surveillance
Alarms
,
filters
non registered
Alarms
Result Messages
Sends update
/
Requests config
CLIC Agents
Central Agents
CMW
JMX
WIE
TWIDO
SNMP
PING
jdaemon
Configuration
Agents
to be
configured
configures
commands
commands
Router
CCC Consoles
LASER
ALARMS
YAMI protocol
JMS
Transition

7

JMS
Notifier
oc
4
j
-
diam
on

Alarm
Archive
Agent Messages
Checks
surveillance
Alarms
,
filters
non registered
Alarms
Result Messages
Sends update
/
Requests config
CLIC Agents
Central Agents
CMW
JMX
WIE
TWIDO
SNMP
PING
jdaemon
Configuration
Agents
to be
configured
configures
commands
commands
Router
CCC Consoles
LASER
ALARMS
YAMI protocol
JMS
OC
4
J
,
EJBs
,
Hibernate
,
BerkeleyDB
OC
4
J
,
EJBs
,
Hibernate
,
BerkeleyDB
second
SonicMQ
first
SonicMQ
In
-
house DB
In
-
house DB
SNMP
,
CMW
,
variety of data
formats
directly to
devices
not enough
data
YAMI specific
protocol
Transition

8

JMS
spring

Alarm
Archive
Agent Messages
Result Messages
Sends update
/
Requests config
CLIC Agents
Central Agents
CMW
JMX
WIE
TWIDO
SNMP
PING
jdaemon
Configuration
Agents
to be
configured
configures
commands
commands
CCC Consoles
YAMI protocol
JMS
LASER
PROXY
LASER
CMW
Transition

9

spring

Alarm
Archive
Result Messages
Sends update
/
Requests config
CLIC Agents
Central Agents
CMW
JMX
WIE
TWIDO
SNMP
PING
jdaemon
Configuration
configures
commands
CCC Consoles
YAMI protocol
JMS
LASER
PROXY
LASER
CMW
Transition

10

spring

Alarm
Archive
Result Messages
Data Acquisition Layer
CLIC
AGENTS
CMW
JMX
WIE
TWIDO
SNMP
PING
Configuration
commands
CCC Consoles
JMS
LASER
PROXY
LASER
CMW
Rules
,
thresholds
,
distribution
,
analysis
,
publishing
...
JAPC
other applications
Technical outcome

Then:


unified view of complete data


sound construction on CO materials


OC4J

-
> Spring,
Hibernate

-
> JDBC Templates,
EJBs

-
> POJOs

SonicMQ

-
> ActiveMQ,
multitude of data formats

-
> JAPC

in
-
house database

-
> CCDB + Logging


wider choice of middleware: CMW, YAMI, JMS


focusing logics in one place


rules, thresholds, alarm generation


conditioning: machine mode, working hours,
state of
subsystems


driving the whole process


detachable
sources
,
clients


less restart/reboot downtime


convergence with similar efforts


11

information exchange

12

japc
-
ext
-
yami

japc
-
ext
-
cmw

japc
-
ext
-
snmp

jms
-
to
-
japc proxy

japc
-
ext
-
?

Core Services

rules
thresholds

alarm
generator

externals

conditions

publisher

w
-
flow

consoles

client middleware: JMS

client

inter

actions

other apps

JAPC

provider

Config

DB

archiver

Archive

DB

Core

DAQ [1]

DAQ [2]

DAQ [n]

tackle

problematic
sources

Clic

agents

CMW

srcs

Laser

Wie, Twi

?

other

sources

source
middleware
protocols

Available technologies



Building blocks:


CO standard components eg.
JAPC


CO less standard, but demanded eg.
japc
-
ext
-
yami


Laser

dev

code base
,
ready
to reuse

eg.
caching
,
distribution


TIM being refurbished and struggling
with
s
imilar

future


other CO projects:
INCA


13

Lessons learned from Laser



eradication of legacy concepts

&
technologies

pays off


to scale well, we need to distribute processing power


a supervisor over multiple workers


dividing is difficult


separation of concerns


single processing unit as a set of loosely coupled actions


patterns: chain of responsibilities, observable


upward flow: source input
-
> processing
-
> publishing


cater for
database
outages through local
caching


JMS+RMI as a middleware for all parties involved


more advanced features of JMS channel help


...but no JAPC concepts were introduced at all


14

Device/property model



to clarify the current incoherent view


strong incentives:


speak one language


let everyone fit (Logging)


common tools, well
-
established in CO world


migration of legacy sources


expose properties


eradicate laser
-
source as it is


inte
r
action with devices through
JAPC calls


domain entities:


Alarm, Alarms, Diagnostic,...


15

JAPC extensions



to
unif
y

view on a variety of devices/applications



available now:


CMW

(japc
-
ext
-
cmwrda) and

JMS (remote)

as
most popular



we need some more:


Y
AMI through japc
-
ext
-
yami is already available
,

used in new
Clic

agent


SNMP
through japc
-
ext
-
snmp under development

16

Bottom tier


Experience gained in:


current Diamon and Laser


eg. CMW Alarm Monitor as a huge data acquisition process


LHC C
oncentrators

,
similar
problems


INCA data acqusition layer


Progress:


17

source type

current

planned/
done

Clic Agents

JMS, YAMI

japc
-
ext
-
yami

CMW Alarm Monitor

Laser gateway

JAPC

device

Laser sources

C/Java laser
-
source

Laser
-
to
-
JAPC proxy

Wiener,

Twido

SNMP central agents

japc
-
ext
-
snmp

Ping

central

agent

JAPC device

Already developed



JAPC extensions


current version of YAMI meets our requirements


Diamon Clic agent


fully YAMI
-
based, JAPC values


CMW Alarm Monitor

refurbished


6000 subscriptions

to GM and FESA devices


will be incorporated into new core


proxy
between
Laser
production alarms and JAPC values


for the legacy sources

18

Open questions


Still to think over:


best components and collaborations to choose


level of distribution and scaling


shared memory, distributed cache


storage: short
-
term, long
-
term, Logging DB


rules handling: homemade, proprietary


...and many others

19