Remote Access Update - Unidata

computerharpySoftware and s/w Development

Dec 2, 2013 (3 years and 11 months ago)

78 views


Remote Data Access
Update

Dr. Dennis Heimbigner

Unidata netCDF Workshop

July 26, 2011

Goals


Exploit new dispatch architecture to support a
number of new remote access protocols and
associated data models


DAP 2


fully supported now


CDM Remote


in testing


netCDF
-
RPC


under development


DAP4


planning stage


Others as they gain traction

General issues


Serialization/De
-
serialization


Convert data (and meta
-
data) on the server to
some linear form suitable for transmission across
the internet


E.g DAP 2 has DDS and DAS for meta
-
data and
uses XDR for raw data


Underlying client
-
server protocol


All current and planned will use HTTP protocol


=> something like libcurl on client side and a
servlet engine (e.g. tomcat) as the server.

CDM Remote


Protocol developed at Unidata by John Caron


Goal is to transmit complete CDM data and model


CDM is netCDF
-
4
-
like model used in Thredds Data
Server


C
-
client converts CDM to netCDF
-
4 only


Rationale: DAP2 is not powerful enough to
transport complete CDM model


(De
-
)Serialization provided by Google protobuf
system.


Underlying transport is HTTP


Status: working, but needs much more testing for
edge cases

netCDF
-
RPC


Protocol developed at Unidata by Dennis
Heimbigner


Rationales:

1.
Provide a very light
-
weight client for use on limited
devices (e.g. field sensors, cell
-
phones, etc.)

2.
Allow netCDF
-
Java read/write access to HDF
-
5 via
netCDF
-
4 (also keep up with changes in HDF
-
5)


(De
-
)Serialization provided by Google protobuf
system.


Underlying transport is HTTP (for initial tests)


Status: under active development

DAP4


Proposed new version of OPeNDAP DAP2
protocol


Essentially equivalent to netCDF
-
4 model


Add new atomic types


Add groups and shared dimensions


Use sequences in place of vlen


Use modified version of DAP2 serialization


Use HTTP as underlying transport protocol


Status: planning state => on hold

Questions?