Local (ESIMO) System - FTP

obtainablerabbiData Management

Jan 31, 2013 (4 years and 6 months ago)

133 views

IODE Ocean Data Portal
-

ODP


The objective of the IODE Ocean Data Portal (ODP) is to
facilitate and promote the exchange and dissemination of marine
data and services.


This is achieved through a network of interconnected National
Oceanographic Data Centres (NODCs) which can provide
seamless access to collections and inventories of marine data
from these data centres.


The Ocean Data Portal provides the full range of processes
including data discovery, access, and visualization.


The ODP supports the data access requirements of all IOC
programmes

areas.

ODP
-

Access to data

The IODE ODP provides on
-
line access to distributed ocean data:



at operational and delayed
-
mode time scales


at various processing level (observation, climate, analysis,
forecast)


across both oceanographic and marine meteorological domains


from multiple data source formats


ODP
-

Standards
are the key to

interoperability


Data Discovery requires a common metadata standard and
standard vocabularies to ensure that the distributed datasets are
discoverable and conformable.


Data Visualization
allows interactive presentation of data and
products using web map service such as WMS and WFS.


Data Access provides ability to download data using common file
formats such as NetCDF, XML, ASCII.



Interoperability with existing systems: IODE is working
closely with the Joint WMO
-
IOC Technical Commission
for Oceanography and Marine Meteorology (JCOMM)
to ensure the ODP is interoperable with the WMO
Information System (WIS)

ODP
-

End to end (E2E) technology

E2E

Integration Server

E2E

Data Provider

HTTP communication

Web
-
services

Local Database
system

Local Data file
system

Data Centers Network

JCOMM/IODE

Data Portal

NetCDF transport

data
-
file

Object transport

data
-
file

Response message

Request message

User access


Security

Registry

Discovery

Delivery

Monitoring

Connection

Mapping

Data Access


Data

Data provider nodes

The goal of this E2E technology
is to integrate the non
-
homogeneous local data
systems into unified distributed
marine data system, that will
provide the transparent
exchange between these local
systems and a end
-
user.

HTTP communication

Web
-
services

CENPAT

INIDEP

CEADO

DNA

CRIBAB

etc. etc.

MINCyT

Based

on the “client
-
server with mediator and wrappers”
:


wrapper (Data Provider)

provides access to data or metadata of
the local data system that exist
s

in DBMS, structured
,
formatted
and object data files (such as images, video files, documents, etc.).
As soon as the wrapper is installed
on

the local data system, the
latter becomes a data source for the distributed data system;


mediator (Integration Server)

integrates data from various local
data systems interacting with wrappers (Data Providers) and with
other mediators (Integration Servers
,
other portals accumulating
descriptive metadata). This makes possible to construct a complex
network as a federation of data sources to meet the needs of
various projects and applications.

ODP
-

E2E technology architecture

DATA PROVIDER

DP


DP provides access to data and metadata of the local data
systems. When the wrapper is installed in the local data system,
the latter becomes a data source for the distributed data system.


DP processes the local data sets and in a semi
-
automated way
generates the discovery metadata
.


These services are based on the OpenDAP data (point, profile
and grid) structures and specific metadata model based on ISO
19115.





Scope

How to become a DP

The data center which agrees to be the DP should provide:



the middleware for communications: application server available
for HTTP protocol,


installation of the DP software OR use light DP extension,


registration of the data source and its discovery metadata,


support of the local data system

How to become a DP

Light DP
extension:



Allows integration of data from data
centres

unable to install
the DP software.

In this case the owner of the DP must create new user with
login and password and provide this information to remote user




Data centres can use remote DP for providing
catalogs

of data
to ODP distributed system


1
0

1
0

DBMS

Dedicated
Server

JBoss Application
server

Apache web
-
server

PHP

Structured and objective data
files

Database access
service

DiGIR

DP web application

It is recommended to use a
dedicated
computer with these
minimum characteristics: CPU 1GHz or more, 2 GB RAM, 300 Mb
hard disk space, running under Windows or Linux.

Installing DP

Network requirements


HTTP and SOAP protocols must be available



JBoss AS port must be opened in firewall settings



IP
-
address verification: DP receives requests only from the
Integration Server


Data in relational Database Management System (DBMS): Oracle,
MS SQL Server, MySQL, PostgreSQL, etc.;


Structured data files with non
-
hierarchical data formats (CSV,
TSV, …)


Object data files: documents, images, data which has format not
supported by ODP technology


Links (web site pages, web
-
applications, URLs, web
-
services)

Supported data storage types


Data in database inside local network with the Data Provider
software


Structured data files: upload to the Data Provider server or
specify URL to data files location (FTP, HTTP)

Data Provider

DBMS

files

Local network of Data Centre

FTP, HTTP

Internet

How to provide the data

The
local data administrator
should provide:




design of resources;


data source registration;


discovery metadata registration;


provision of data


Functional requirements

Use web
-
interface of the Data Provider to generate, update and
maintenance discovery metadata


How to generate metadata




Provides the unification of specifics of storage structures and data
coding systems of the local data systems through the mechanism
and tools of translating local formats and codes to the common
transport data file and system codes.


Code

list



ships

Local

(ESIMO)

System

(IMO)

23640

8806802

23641

9347621


..

……
.


Local

(ESIMO)

System

(WMO)

684

21821


Coding and decoding issues


define the schedule for updating of discovery metadata,


check the local data

availability using the report submitted by
Integration Server,


take the needed actions to provide the data source actuality

(connection, data files storage availability),


Resource life
-
cycle