Data provider usage

musicincurableData Management

Jan 31, 2013 (4 years and 7 months ago)

208 views

Training on the management of the

End to End data Management (E2EDM)

prototype system at IODE NODCs

E2E Data Provider

Requirements, installation, features, setup, usage and maintenance

E2E Data Provider. Features




Providing connection between local data sets of Data Centers
and distributed system


Access to the local data source (databases, files)


Data source/resources registry and metadata update


Schedule mechanisms


Generation and updating of resource descriptions in compliance
with the current status of the local data set


Codes and dictionaries mapping


Request handling and data delivery


Security services


Administration tools (CMS)

Ostende,
2007

Requirements to local data system




Rational database management systems: Oracle, MS SQL
Server, MySQL, PostgreSQL, Interbase, ODBC, …


Structured data files in “flat” (non
-
hierarchical) data formats:
ASCII files with separator or fixed values positions


Object files: electronic documents, images, data files in
formats, which are not handled by the E
2
EDM technology
software


Web
-
applications performing an access to local data sets

Ostende, 2007

Software and hardware requirements




Windows OS (XP, 2000, 2003) or Unix
-
like OS (RedHat, Debian,
Sun Solaris)


Apache web server 1.3.27 or higher


PHP 4.0 or higher


Java2 SDK 1.5.0 or higher


JBoss application server 4.0.0

Ostende, 2007

Software requirements

Recommended technical requirements


Intel Pentium Core 2 Duo 2Ghz, 2Gb RAM, 10Gb HDD


Network connection 100Mbit/s speed

Apache web server installation



Ostende, 2007

1.
Start distribution installation,
defining server name.


Server name is a DNS or
computer name

2.

Select custom setup type to
specify installation path and
components

Apache web server installation



Ostende, 2007

3.
Specify installation path to
c:
\
usr
\
local


It’s better to use path that
doesn’t contain any space
characters



4. Final step


After successful installation
apache system service is
managed from the “Services”
section of Windows Control
Panel

PHP installation (continue)



Ostende, 2007

PHP is an language interpreter that running under Apache web server.

Run the executable installer and follow the instructions:

1.
Specify installation path to directory c:
\
usr
\
local
\
php
-
4.4.4 (It’s a default path in
installer)

2.
To make sure that the php libraries can be found by Apache web server copy
php4ts.dll and sapi
\
php4apache.dll from installation directory to Windows system
directory (e.g. c:
\
windows
\
system32)

3.
Additionally, in order to work with DBMS except MySQL you need to uncomment
proper extension in php.ini configuration file and put dll library to c:
\
windows
\
system32
path. To work with MS SQL Server put extensions
\
php_mssql.dll to this path.

4.
Copy the file, php.ini from additional files in Data Provider distribution to c:
\
windows.

Make sure to rename php
-
4.1.1.ini file from distribution to php.ini. All necessary
settings in this file are done for successful work of Data Provider with MS SQL Server.

Note, that MS SQL Client must be installed on the server also.

PHP installation (continue)



Ostende, 2007

5.
Overwrite c:
\
usr
\
local
\
apache
\
conf
\
httpd.conf Apache configuration file from
Data Provider distribution. Before copying httpd
-
1.3.28.conf to Apache conf
directory rename it to httpd.conf. This file loads php module into Apache web
server.


Note: By overwriting your configuration file with file from distribution
you overwrite all previous settings. In case of Data Provider you have
to edit httpd.conf file after overwriting and set your server name again.


It can be done by searching the string starts with “ServerName” in this
file and typing correct value

Database Access Service (DiGIR) installation



Ostende, 2007

1.
Run digir
2
.exe installer from Data Provider distribution and follow
instructions. Installation path is c:
\
www
\
digir
2
by default.

2.
Make sure that DiGIR

is installed correctly by typing in your web
browser address:

http://localhost/digir2/admin

Java SDK installation



Ostende, 2007

1.
Run jdk
-
1_5_0_04
-
windows
-
i586
-
p.exe installer from Data Provider
distribution and follow instructions. Specify installation path to
c:
\
jdk1.5.0_04.

2.
After installation create system variable JAVA_HOME and set it to
c:
\
jdk1.5.0_04 path

To set system variable click right mouse button on My Computer and
choose Properties. On Advance tab press Environment variables
button below. Then in system variables section create JAVA_HOME
variable

3.
Installation complete and all java application will use this JAVA_HOME
variable pointing to installed Java SDK

JBoss application server installation



Ostende, 2007

1.
Copy JBoss from Data Provider distribution to c:
\
jboss
-
4.0.0 path

2.
Go to c:
\
jboss
-
4.0.0
\
bin directory and run !installJbossAsService.bat file to install
JBoss as system service.

3.
After installation JBoss system service is managed from the “Services” section of
Windows Control Panel

4.
Server log is placed to c:
\
jboss
-
4.0.0
\
server
\
default
\
log
\
server.log file. All errors that
occurs can be read from this log file.

5.
Make sure that JBoss server application works properly by typing address in web
browser:

http://localhost:8080/

Data provider management system


installation



Ostende, 2007

1.
Run dpms.exe installer from Data Provider distribution to

c:
\
jboss
-
4.0.0
\
server
\
default
\
deploy path

2.
Restart JBoss application server for application deployment

3.
Make sure that application works properly by typing address in web browser:

http://localhost:8080/dpms/

Data provider setup



Ostende, 2007

Administration section: Configure settings


Setup parameters and define paths of locating system files:

Data Provider identifier, server name and port, path to application files, path to DiGIR

Data provider setup



Ostende, 2007

Administration section: Configure schedulers


Data Provider performs automated updating of each resource description which data
set is in DBMS or structured files.

Data provider setup



Ostende,
2007

Administration section: Localization of system elements


Localization of system elements consists in specifying those system elements, which
are relevant for the data sets of a specific Data Center and will be used for resource
descriptions.


Data provider usage



Ostende, 2007

Operator section: Manage information resources


Entry point for resource descriptions management.


Data provider usage



Ostende, 2007

Operator section: Create new resource description


2 ways of registrations: create new description or import existing


Registration steps:


Identification and connection to data (if data source is DPMS or structured files)

1.
Data presentation and dissemination

2.
Time and geographic characteristics

3.
Contacts

4.
Data hierarchy setup setting structural units of data set (subsets) and data records
structure


Data provider usage



Ostende, 2007

Operator section: Create new resource description

Identification and connection to data


Data provider usage



Ostende, 2007

Operator section: Create new resource description

Edit resource mapping


Metadata section contains identification and contact information

Data provider usage



Ostende, 2007

Operator section: Create new resource description

Edit resource mapping


Datasource section: data base and login account settings

Data provider usage



Ostende, 2007

Operator section: Create new resource description

Edit resource mapping


Tables section: specify table or set of tables linked by foreign keys

First step: select root table and key field (primary key is preferable)

Second step: select foreign key in root table, select foreign key from another table and
press “add” button

Data provider usage



Ostende, 2007

Operator section: Create new resource description

Edit resource mapping

: filter


Specify criteria of data sampling out of the database. It is used when resource is a
part of the database, determined by the period of observation, geographic location
and other characteristics.

Data provider usage



Ostende,
2007

Operator section: Create new resource description

Edit resource mapping

: Mapping


Define a one
-
to
-
one mapping between system data elements (codes/names) and
local data elements *tables and table fields). To save time it is recommended to
prepare and use a working table of mapping.

Data provider usage



Ostende, 2007

Operator section: Create new resource description

Data presentation and dissemination


Define a space
-
time structure of local data provided by resource as well as form and
conditions of presenting information to users.

Data provider usage



Ostende, 2007

Operator section: Create new resource description

Time and geographic characteristics


Set time and geographical characteristics of data provided by resource

Data provider usage



Ostende, 2007

Operator section: Create new resource description

Contacts


Enter contact information of three categories:


Role “generator”


organization that has a rights of data owner (or entitled to a right of
ownership) of the data set provided by resource;


Role “resource provider”


Data Center that provides resource


Role “point of contact”

resource description author

Data provider usage



Ostende, 2007

Operator section: Create new resource description

Data hierarchy


1. Defines resource structure


data segmentation (granulation) into resource
instances,


2. Defines data structure


data segmentation (granulation) into transport data file
records


3. Resource structure setup is based on logical and physical basis. Logical is made by
instance key. Physical is one data file


one resource instance

Data provider usage



Ostende, 2007

Operator section: Resource maintenance


Registered resources

Managing instances, codes mapping
settings, tasks

Resources under registration

Eliminating errors and warnings in
resource descriptions

Data provider usage



Ostende,
2007

Operator section: Resource maintenance


Task creation

Information resources should by ready to meet requests from Integration Server. To assure
actuality of resources it is necessary to set frequency of automatic resource description
updating and instance generation

Data provider usage



Ostende, 2007

Operator section: Codes


This section is intended to provide an access to system codes used for
resource descriptions

Data provider usage



Ostende, 2007

Operator section: Dictionaries


This section is intended to provide an access to system dictionaries
(guides) used for code unification of local data


Data provider usage



Ostende, 2007

Operator section: Data elements


List of elements, which used in resource descriptions

Operations: sort the list of system elements by the chosen field,



obtain the full list of codes without paging,




copy list to one of the formats: excel, word, pdf, csv

Data provider usage



Ostende,
2007

Operator section: Help


All documentation and contacts