ppt - ALICE DCDB Website - Cern

righteousgaggleΔιαχείριση Δεδομένων

31 Ιαν 2013 (πριν από 4 χρόνια και 8 μήνες)

105 εμφανίσεις

Warsaw University of Technology

Alice detector databases

architecture, data structures and general overview

Wiktor Peryt

&
Tomasz

Traczyk


in collaboration with student (also Ph.D.student) team from WUT

Piotr Mazan, Dominik Tukendorf, Piotr Szarwas, Michal
Janik, Dawid Jarosz, Bartek Pawlowski, Pawel Szelenbam,
Jacek Wojcieszuk




ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

2

Alice databases


architecture

Central database


Placed at CERN

(temporarily placed at WUT)


Plays role of central repository


Contains


central inventory of components


copies of data from laboratories


metadata, e.g. dictionaries


Satellite databases


Placed in laboratories
-
participants


Contain


source data


produced at laboratories


delivered by manufacturers


working copies of data from central
repository


partial copies of metadata (read only)

Communication


Passing messages in XML


Mainly off
-
line (batch processing)


No satellite
-
satellite communication!

Central
database

Satellite databases

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

3

Central + satellite architecture






pros and contras

Advantages


Labs can input data almost independently


even if communication problems occur


Data from different satellite databases can be
integrated


All the data is available from one site


it’s easier to create applications


access rights can be managed in easier
way


Data can be restored from central database
even if local database is broken


It’s possible to avoid satellite
-
to
-
satellite data
transfers


At the end of the production phase all the data
will be stored in one database


no further
integration is necessary

Disadvantages


Data must be transferred between central and
satellite databases


Data from satellite databases must be
integrated in central database each time it is
uploaded from satellite databases


Data integrity is difficult to preserve


copies must be properly updated


if the same data is changed concurrently in
more than one database, a conflict can
occur, which must be detected and solved


ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

4

Central database versus satellite databases

Central database


Usage characteristics


very large data volume


many concurrent users


twenty
-
four
-
hour utilisation


critical data (breakdown may be very
costly)



Necessary features


transaction processing


procedural constraints (triggers)


7
×

24 availability


high reliability


continuous administration

by professional DBA team


on
-
line backup


data partitioning




Proper solution


high
-
end commercial DBMS software

is necessary

Satellite databases


Usage characteristics


average or small data volume


only few concurrent users


used only few hours per day


not critical data (can be restored from
backup or from central database)



Necessary features


transaction processing


procedural constraints (triggers)


should not need complex administration
(no trained personnel at labs
-
participants)


should be monitored remotely

by developers


off
-
line backup procedures are sufficient


data may not need to be partitioned


should not be costly!



Proper solution


open
-
source database can be used

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

5

Proposed technology

Central database


Oracle RDBMS



Advantages


very stable and reliable


support for transaction processing


built
-
in procedural language


triggers


support for complex data types

and BLOBS


support for VLDB (very large databases),
e.g. data partitioning


7
×

24 availability (on
-
line backup, etc.)



Disadvantages


quite expensive


complex and difficult to administer

Satellite databases


PostgreSQL



Advantages


free of charge


quite easy to administer


stable enough


support for transaction processing


built
-
in procedural language


triggers


support for complex data types

and BLOB objects



Disadvantages


not
very

fast (but fast enough for this
particular application)


no support for distributed processing

(data replication, etc.)


no support for heterogeneous systems


no support for VLDB


no 7
×

24 availability

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

6

Data interchange


technology

Problems


Centrally maintained data (e.g. dictionaries)
must be replicated to satellite databases


Data entered/edited in satellite databases must
be


copied to central repository


interchanged between satellite databases


System is heterogeneous


no built
-
in support for distribution, data
replication, etc.


no support for direct data interchange
(gateways)


Proposed solution


XML
-
based data interchange


data is exported to XML documents


XML documents are sent over the network


data is imported from XML into database
structures

Advantages


XML is very flexible


any data structures can be represented


XML documents have readable format


contain metadata


easy to debug


can even be written/changed by human


XML can be written and read (parsed) by
standard (and free) software tools

(XML parsers)


XML documents can be transformed by
standard software (XSL processors) and
presented using standard tools (e.g. HTML
browsers)


Disadvantages


XML documents are quite big


should be sent compressed

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

7

Central
database


Satellite databases

2. Check
-
in

3. Check
-
out

1. Creation

4. Destruction

Monitoring of components

Central database


Central inventory of components contains


actual location and status of each
components


history of each component


Physical flow of components


Creation of new component must be signalled
to central inventory


a globally unique identifier for each
component is generated by lab and
registered in central inventory


Arrival and departure of each component
to/from any laboratory must be recorded in the
inventory


Destruction of the component must also be
signalled to the inventory


Solution


Components must be “checked
-
out” and
“checked
-
in” from/to central inventory
before/after they are processed by satellite
databases


Messages are passed as XML documents



ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

8

Components’ flow and data flow

Problems


Lab may need to modify components’ data even
if it does not have the component itself


Data transfer from satellite database to some
auxiliary database (e.g. on notebook) may be
required


To avoid possible conflicts we should not allow
concurrent modifications of the same data


some kind of locks is required


There is a danger of deadlocks

(e.g. when one lab locks data set A and the
second lab locks B, then the first lab requests
B, and the second lab requests A)


Temporary communication problems should not
stop the work

Proposed solution


Check
-
in / check
-
out of the data must be
separated from check
-
in / check
-
out of the
physical component


data is checked in/out when the transfer of
read
-
write access rights is necessary


component is checked in/out when it is
physically transferred from one lab to
another


There must be some way to override check
in/out and to force modifications of not
-
checked
-
out data


conflicts resolution must occur after
changes in forced mode are finished


There must be some kind of deadlock detection
and resolution

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

9

Structure of Satellite Database

Main assumptions


Satellite databases will work in co
-
operation
with central database


The same data structure should be used for


all labs


all detectors and component types


History of data changes is recorded only if
absolutely necessary

(in central database changes of data will be
audited in more detailed way)


Series of test results should be stored in
efficient and easy
-
to
-
query way

Proposed solution


Generic data structure should be used


Generic “core” application will be created


with possibility to create some more
specialised application modules for
particular labs


Natural primary keys are preferred


alphanumeric codes used in most cases


lead to more natural queries


but compound keys must be used


artificial (numeric) identifiers used only
when no natural key can be supplied


Design of primary keys enables the data to be
easily integrated into central database (no key
conflicts should occur)


Complex object
-
relational data types (“nested”
tables) are used to store series of results

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

10

Generic structures

“Typical” structure


Separate column for each property


Separate set of tables for each type of objects
(having different set of properties)

Generic structure


Dictionary
-
based


dictionaries of object types, properties, etc.


One generic set of tables for all objects

Object type C

# Id

* Property C1

* Property C2

* Property C3

...

Object type B

# Id

* Property B1

* Property B2

* Property B3

...

Object type A

# Id

* Property A1

* Property A2

* Property A3

...

Object

# Id

* Name

...

Type

# Id

* Name

...

Property

definition

# Id

* Name

*
Optional?

* Data
type

...

Property

value

* Value

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

11

Generic structures

“Typical” structure


Separate column for each property


Separate set of tables for each type of objects
(having different set of properties)

Generic structure


Dictionary
-
based


dictionaries of object types, properties, etc.


One generic set of tables for all objects

Object type C

# Id

* Property C1

* Property C2

* Property C3

...

Object type B

# Id

* Property B1

* Property B2

* Property B3

...

Object type A

# Id

* Property A1

* Property A2

* Property A3

...

Object

# Id

* Name

...

Type

# Id

* Name

...

Property

definition

# Id

* Name

*
Optional?

* Data
type

...

Property

value

* Value

Entity (~ table)

# Unique identifier (~ key)

Attribute (~ column)

“One to many” relationship

-

-

-

Optional relationship

–––

Obligatory relationship

“Many to many” relationship

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

12

Generic structures


pros and contras

Advantages


The
same

structure can be used in all labs


it is therefore
much
easier to maintain


The
same

generic application can be used on
top of all the databases


Structure of the central database will be very
similar to the structure of satellite ones


it’s easy to understand the relationships
between central and satellite data


no complicated translations between
satellite and central structures are required


it’s easier to introduce changes to both
structures


The structure is flexible


new types of components can be
introduced with no need to change the
database structure


The structure is quite simple


less than 20 tables


quite easy to understand and to memorise

Disadvantages


The structure is not so straightforward


There may be more performance problems
comparing to not
-
generic structures

(in our case the performance of the generic
structure has been tested and proved to be
sufficient)


The data dictionaries must be created and
distributed to satellite sites


Procedural constraints (triggers) are necessary
to enforce data integrity (e.g. proper typing of
values)


The generic application is a bit more difficult to
create than “normal” one (but we have to create
and to maintain only
one

universal application
instead of many different specialised
applications)

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

13

Satellite Database


data structures

DETECTOR
# DETECTOR CODE
* NAME
o DESCRIPTION
ALLOWABLE VALUE
# SEQUENTIAL NO
* VALUE
o HIGH VALUE
o MEANING
o NUMERICAL VALUE
DATA TYPE
# DATA TYPE CODE
* NAME
* ELEMENTARY TYPE
o DESCRIPTION
BLOB TYPE
# MIME TYPE
o NAME
MANUFACTURER
# MANUFACTURER CODE
* NAME
* IS LAB
o DESCRIPTION
PROCESS PARAMETER
o VALUE
BLOB
# BLOB ID
* BLOB VALUE
o BLOB DATE
o FILE NAME
o FILE PATH
o FILE OWNER
o DESCRIPTION
PROCESS
# PROCESS DATE
o DESCRIPTION
o X SIZE
o Y SIZE
o Z SIZE
o VALUE TABLE
o PROCESSED BY
PARAMETER
# VALID FROM
o VALID TO
o VALUE
DEFINI TI ON OF PROCESS
PARAMETER
# PARAMETER CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o DESCRIPTION
* IS RESULT
* IS ACTIVE
DEFINI TI ON OF PROCESS
# PROCESS CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o X LABEL
o Y LABEL
o Z LABEL
o DESCRIPTION
* IS TEST
* IS ACTIVE
DEFINI TI ON OF PARAMETER
# PARAMETER CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o DESCRIPTION
* IS ACTIVE
COMPONENT STATE
# VALID FROM
o VALID TO
* EXISTENCE
o QUALITY
o ACCEPTED
* IS COMPONENT CHECKED OUT
* IS DATA CHECKED OUT
COMPONENT DERIVATION
# VALID FROM
o VALID TO
* DERIVATION TYPE
o LOCATION
o POSITION_NO
o DESCRIPTION
COMPONENT GROUP
# GROUP CODE
* NAME
o DESCRIPTION
COMPONENT TYPE
# TYPE CODE
* NAME
* IS ACTIVE
o DESCRIPTION
COMPONENT
# COMPONENT ID
o USER CODE
o SERIAL NUMBER
o DESCRIPTION
* IS SKELETON
belongs to
includes
belongs to
consists of
of type
.
of type
.
of type
.
belongs to
is contrained by
of type
defines
belongs to
consists of
belongs to
consists of
delivered by
delivers
manufactured by
creates
of
described by
defined by
defines
of component
has
of test
has
for
tested by
defined by
defines
of
described by
defined by
defines
.
has parameters
of type
processed by
of type
described by
describes
in
source of
source of
derivative
derivative
in group
consists of
of type
includes
ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

14

DETECTOR
# DETECTOR CODE
* NAME
o DESCRIPTION
ALLOWABLE VALUE
# SEQUENTIAL NO
* VALUE
o HIGH VALUE
o MEANING
o NUMERICAL VALUE
DATA TYPE
# DATA TYPE CODE
* NAME
* ELEMENTARY TYPE
o DESCRIPTION
BLOB TYPE
# MIME TYPE
o NAME
MANUFACTURER
# MANUFACTURER CODE
* NAME
* IS LAB
o DESCRIPTION
PROCESS PARAMETER
o VALUE
BLOB
# BLOB ID
* BLOB VALUE
o BLOB DATE
o FILE NAME
o FILE PATH
o FILE OWNER
o DESCRIPTION
PROCESS
# PROCESS DATE
o DESCRIPTION
o X SIZE
o Y SIZE
o Z SIZE
o VALUE TABLE
o PROCESSED BY
PARAMETER
# VALID FROM
o VALID TO
o VALUE
DEFINI TI ON OF PROCESS
PARAMETER
# PARAMETER CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o DESCRIPTION
* IS RESULT
* IS ACTIVE
DEFINI TI ON OF PROCESS
# PROCESS CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o X LABEL
o Y LABEL
o Z LABEL
o DESCRIPTION
* IS TEST
* IS ACTIVE
DEFINI TI ON OF PARAMETER
# PARAMETER CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o DESCRIPTION
* IS ACTIVE
COMPONENT STATE
# VALID FROM
o VALID TO
* EXISTENCE
o QUALITY
o ACCEPTED
* IS COMPONENT CHECKED OUT
* IS DATA CHECKED OUT
COMPONENT DERIVATION
# VALID FROM
o VALID TO
* DERIVATION TYPE
o LOCATION
o POSITION_NO
o DESCRIPTION
COMPONENT GROUP
# GROUP CODE
* NAME
o DESCRIPTION
COMPONENT TYPE
# TYPE CODE
* NAME
* IS ACTIVE
o DESCRIPTION
COMPONENT
# COMPONENT ID
o USER CODE
o SERIAL NUMBER
o DESCRIPTION
* IS SKELETON
belongs to
includes
belongs to
consists of
of type
.
of type
.
of type
.
belongs to
is contrained by
of type
defines
belongs to
consists of
belongs to
consists of
delivered by
delivers
manufactured by
creates
of
described by
defined by
defines
of component
has
of test
has
for
tested by
defined by
defines
of
described by
defined by
defines
.
has parameters
of type
processed by
of type
described by
describes
in
source of
source of
derivative
derivative
in group
consists of
of type
includes
Satellite Database


data structures

Dictionaries


Maintained in central repository


Supplied from central database to labs


Read
-
only in satellite databases

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

15

Satellite Database


data structures

DETECTOR
# DETECTOR CODE
* NAME
o DESCRIPTION
ALLOWABLE VALUE
# SEQUENTIAL NO
* VALUE
o HIGH VALUE
o MEANING
o NUMERICAL VALUE
DATA TYPE
# DATA TYPE CODE
* NAME
* ELEMENTARY TYPE
o DESCRIPTION
BLOB TYPE
# MIME TYPE
o NAME
MANUFACTURER
# MANUFACTURER CODE
* NAME
* IS LAB
o DESCRIPTION
PROCESS PARAMETER
o VALUE
BLOB
# BLOB ID
* BLOB VALUE
o BLOB DATE
o FILE NAME
o FILE PATH
o FILE OWNER
o DESCRIPTION
PROCESS
# PROCESS DATE
o DESCRIPTION
o X SIZE
o Y SIZE
o Z SIZE
o VALUE TABLE
o PROCESSED BY
PARAMETER
# VALID FROM
o VALID TO
o VALUE
DEFINI TI ON OF PROCESS
PARAMETER
# PARAMETER CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o DESCRIPTION
* IS RESULT
* IS ACTIVE
DEFINI TI ON OF PROCESS
# PROCESS CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o X LABEL
o Y LABEL
o Z LABEL
o DESCRIPTION
* IS TEST
* IS ACTIVE
DEFINI TI ON OF PARAMETER
# PARAMETER CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o DESCRIPTION
* IS ACTIVE
COMPONENT STATE
# VALID FROM
o VALID TO
* EXISTENCE
o QUALITY
o ACCEPTED
* IS COMPONENT CHECKED OUT
* IS DATA CHECKED OUT
COMPONENT DERIVATION
# VALID FROM
o VALID TO
* DERIVATION TYPE
o LOCATION
o POSITION_NO
o DESCRIPTION
COMPONENT GROUP
# GROUP CODE
* NAME
o DESCRIPTION
COMPONENT TYPE
# TYPE CODE
* NAME
* IS ACTIVE
o DESCRIPTION
COMPONENT
# COMPONENT ID
o USER CODE
o SERIAL NUMBER
o DESCRIPTION
* IS SKELETON
belongs to
includes
belongs to
consists of
of type
.
of type
.
of type
.
belongs to
is contrained by
of type
defines
belongs to
consists of
belongs to
consists of
delivered by
delivers
manufactured by
creates
of
described by
defined by
defines
of component
has
of test
has
for
tested by
defined by
defines
of
described by
defined by
defines
.
has parameters
of type
processed by
of type
described by
describes
in
source of
source of
derivative
derivative
in group
consists of
of type
includes
Components’ data


Created and updated in satellite
databases at labs


Copied to central repository

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

16

Data structures


explanation

Component


Identified by


globally unique internal numerical Id
(contains lab prefix and locally unique
sequence number)


user code (locally unique)


serial number

(unique for specific component type)

COMPONENT TYPE
# TYPE CODE
* NAME
* IS ACTIVE
o DESCRIPTION
COMPONENT
# COMPONENT ID
o USER CODE
o SERIAL NUMBER
o DESCRIPTION
* IS SKELETON
MANUFACTURER
# MANUFACTURER CODE
* NAME
* IS LAB
o DESCRIPTION
COMPONENT GROUP
# GROUP CODE
* NAME
o DESCRIPTION
of type
includes
manufactured by
creates
in group
consists of
Component type


Identified by globally unique type code

and name


Each component belongs to exactly one
component type


Component cannot change its type

(the relationship is not transferable)


Manufacturer


Each component has one manufacturer

(lab
-
participant or external manufacturer)


Group of components


Auxiliary grouping of components

Non transferable relationship

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

17

Data structures


explanation

State of component


Existence


exists


assembled


included in compound component


broken


destroyed


Final quality assessment (in percents)


Final acceptation mark


Check
-
out/check
-
in states



Full history of state changes is recorded


each change of the state causes insertion of

a new record in
COMPONENT_STATES

table

COMPONENT
# COMPONENT ID
o USER CODE
o SERIAL NUMBER
o DESCRIPTION
* IS SKELETON
COMPONENT STATE
# VALID FROM
o VALID TO
* EXISTENCE
o QUALITY
o ACCEPTED
* IS COMPONENT CHECKED OUT
* IS DATA CHECKED OUT
describes
in
Relationship is part of
unique id (id consists
of the relationship and

# attribute)

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

18

Data structures


explanation

Component assemblage/partition


The data structure represents a graph (digraph)


nodes represent components


arcs represent composition/derivation:


a component consists (is assembled)
of other components


a component is derived (e.g. by
partition or disassemble) from another
one


The location of the component in compound
component (e.g. location slot no) can also be
recorded



Full history of the assemblage/partition
processes is recorded

COMPONENT
# COMPONENT ID
o USER CODE
o SERIAL NUMBER
o DESCRIPTION
* IS SKELETON
COMPONENT DERIVATION
# VALID FROM
o VALID TO
* DERIVATION TYPE
o LOCATION
o POSITION_NO
o DESCRIPTION
source of
source of
derivative
derivative
Arcs of the graph

Nodes of the
graph

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

19

Data structures


explanation

Parameter values


Value is stored in text format


values can be converted to proper data
types as needed, e.g. to calculate an
aggregation (sum, average, etc)


Each value change is validated against value
data type by database trigger


Full history of changes of component
parameters’ values is recorded


each change of the value causes insertion
of a new record in
PARAMETER

table

COMPONENT TYPE
# TYPE CODE
* NAME
* IS ACTIVE
o DESCRIPTION
DEFINI TI ON OF PARAMETER
# PARAMETER CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o DESCRIPTION
* IS ACTIVE
PARAMETER
# VALID FROM
o VALID TO
o VALUE
COMPONENT
# COMPONENT ID
o USER CODE
o SERIAL NUMBER
o DESCRIPTION
* IS SKELETON
of type
includes
of type
described by
defined by
defines
of
described by
Definition of parameter


Each component can have several parameters


Components of the same type have identical
set of parameters (certainly, values of these
parameters may vary)


Parameter is identified by a parameter code
and a component type code


ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

20

Data structures


explanation

Data types of parameters


Data type of each parameter must be defined


Elementary data types are


string


number (float)


integer number


Derived data types can be defined


by enumeration


each allowable value can have
descriptive label (
Meaning
)


each allowable string value can have
its numerical counterpart defined
(
Numerical Value
)


by restriction of range of allowable values


Value

and
High Value

attributes
are used


DEFINI TI ON OF PARAMETER
# PARAMETER CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o DESCRIPTION
* IS ACTIVE
DATA TYPE
# DATA TYPE CODE
* NAME
* ELEMENTARY TYPE
o DESCRIPTION
ALLOWABLE VALUE
# SEQUENTIAL NO
* VALUE
o HIGH VALUE
o MEANING
o NUMERICAL VALUE
of type
.
belongs to
is contrained by
ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

21

Data structures


explanation

Processes


Processes are used to store information related to


tests


manufacturing processes (e.g. assemblage)


Each component can be described by


several types of processes


many instances of each process type

(e.g. many repetitions of particular test type)


Each process can be related to a “manufacturer”



an institution which performs the process



MANUFACTURER
# MANUFACTURER CODE
* NAME
* IS LAB
o DESCRIPTION
COMPONENT
# COMPONENT ID
o USER CODE
o SERIAL NUMBER
o DESCRIPTION
* IS SKELETON
PROCESS
# PROCESS DATE
o DESCRIPTION
o X SIZE
o Y SIZE
o Z SIZE
o VALUE TABLE
o PROCESSED BY
DEFINI TI ON OF PROCESS
# PROCESS CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o X LABEL
o Y LABEL
o Z LABEL
o DESCRIPTION
* IS TEST
* IS ACTIVE
COMPONENT TYPE
# TYPE CODE
* NAME
* IS ACTIVE
o DESCRIPTION
for
tested by
belongs to
consists of
defined by
defines
belongs to
consists of
of type
processed by
of type
includes
delivered by
delivers
ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

22

Data structures


explanation

Compound processes


Compound process can include other
processes


recursive relationships denote
hierarchies of processes

PROCESS
# PROCESS DATE
o DESCRIPTION
o X SIZE
o Y SIZE
o Z SIZE
o VALUE TABLE
o PROCESSED BY
DEFINI TI ON OF PROCESS
# PROCESS CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o X LABEL
o Y LABEL
o Z LABEL
o DESCRIPTION
* IS TEST
* IS ACTIVE
defined by
defines
belongs to
consists of
belongs to
consists of
Complex test data


Complex process results can be stored


max. 3
-
dimensional data (
X
,
Y
,
Z
)


series of results (
Value Table
)


whole series is stored in one table row


reasonable query performance is ensured


complex object
-
relational data type is
used


each data cell can be retrieved
using SQL query


Numerical results are stored in text format


scale and precision of numbers are preserved;
scientific notation can be used


text values can be converted to proper data
types as needed


Recursive relationship

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

23

DEFINI TI ON OF PROCESS
# PROCESS CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o X LABEL
o Y LABEL
o Z LABEL
o DESCRIPTION
* IS TEST
* IS ACTIVE
PROCESS
# PROCESS DATE
o DESCRIPTION
o X SIZE
o Y SIZE
o Z SIZE
o VALUE TABLE
o PROCESSED BY
PROCESS PARAMETER
o VALUE
DEFINI TI ON OF PROCESS
PARAMETER
# PARAMETER CODE
* NAME
o SEQUENTIAL NO
o UNITS OF MEASURE
o DESCRIPTION
* IS RESULT
* IS ACTIVE
DATA TYPE
# DATA TYPE CODE
* NAME
* ELEMENTARY TYPE
o DESCRIPTION
ALLOWABLE VALUE
# SEQUENTIAL NO
* VALUE
o HIGH VALUE
o MEANING
o NUMERICAL VALUE
defined by
defines
of
described by
defined by
defines
.
has parameters
of type
.
belongs to
is contrained by
Data structures


explanation

Parameters of processes


Each process can have several parameters


Processes of the same type have identical set
of parameters


Process can have


input parameters


output parameters (results)






Data types of these parameters are defined
similarly to data types of components’
parameters


Only current values of process’ parameters are
stored (no history of value changes)


ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

24

Data structures


explanation

BLOBs (Binary Large Objects)


Can be used to store binaries


pictures (photographs, etc)


other binary files


Should
not

be used to store structured data
(e.g. results of a test in a spreadsheet format)


SQL cannot query binary data


BLOBs can be connected to


components


processes (e.g. tests)

COMPONENT
# COMPONENT ID
o USER CODE
o SERIAL NUMBER
o DESCRIPTION
* IS SKELETON
PROCESS
# PROCESS DATE
o DESCRIPTION
o X SIZE
o Y SIZE
o Z SIZE
o VALUE TABLE
o PROCESSED BY
BLOB TYPE
# MIME TYPE
o NAME
BLOB
# BLOB ID
* BLOB VALUE
o BLOB DATE
o FILE NAME
o FILE PATH
o FILE OWNER
o DESCRIPTION
of component
has
of test
has
of type
defines
Exclusive arc:

each BLOB
must be related
to

a process
or

to
a component

ALICE ITS meeting, Dec. 3
rd
, 2001 CERN

25

Monitoring of remote databases

Problem


Many local databases


Trained personnel not available at all labs
-
participants


We need to be able to monitor the the state of
remote servers from one central place


Some monitoring system is needed


Main functions


Performing tests on remote computers, e.g.


connection checking


parsing remote databases’ log files


Performing actions


inform administrators about errors found on
their servers during tests, e.g. by e
-
mail.


Gathering and reporting information on tests’
results

Proposed solution


Remote Computer State Analysis System


Design of RCSAS


Three
-
layer architecture


database (PostgreSQL)


stores all
necessary information


main program (business logic)


split into two parts (client and server)


Web application
--

for data presentation


Security


connection between server and client over
SSL


username / password authentication
between server and client


Extensibility


new tests and actions can be easily added