THREDDS Data Server

indexadjustmentInternet και Εφαρμογές Web

13 Νοε 2013 (πριν από 3 χρόνια και 9 μήνες)

154 εμφανίσεις

THREDDS Data Server

John Caron

Unidata/UCAR


Mar 14, 2005

Outline

1.
THREDDS Catalog Services

2.
THREDDS Data Server

3.
NetCDF
-
4

4.
NetCDF
-
Java 2.2

THREDDS


Thematic Realtime Environmental
Distributed Data Services


Middleware between data providers and
data users


Enable discovery and use of scientific data


Not a data portal


XML over HTTP


Java Servlets


HTTP Server




THREDDS Catalogs

Client

Application

Datasets

Catalog.xml

hostname.edu

Catalog
Generator

Data Server(s)

DODS, ADDE, FTP, HTTP






Catalog.xml

Catalog.xml


HTTP Tomcat Server









Dynamic Catalogs, data services

Client

Application

Datasets

Catalog.xml

hostname.edu

Data Server(s)

DODS, ADDE, FTP, HTTP

Query Resolver Service



DQC.xml

Catalog Service



Catalog
Generator

Catalog.xml

Resolver Service


URI

URL

Dataset Query Capability (DQC)



Dynamic and very large datasets



NEXRAD (level 3) from IDD


150 stations


20 products


Irregular times


The DQC describes the queries that the data
server is capable of responding to.

Resolver Services


Logical Dataset, eg “latest ETA model run”


Dataset with Service type “Resolver”


On the client, the URI of the logical
dataset is sent to the data server


The server finds what is available and
returns an actual dataset URL.


Limitations


Catalogs are very general


“data protocol
agnostic”.


Automatic metadata extraction needs to
“crack” the datasets.


Data Servers must be separately installed,
maintained, etc


THREDDS Data Server


Add the data access!


100% Java


single thredds.war


requirement, not goal


Data input is NetCDF Java 2.2 library


Data output:


OpenDAP


WCS (in progress)


FTP, others (future)


HTTP Tomcat Server









THREDDS Data Server

Datasets

Catalog.xml

hostname.edu

THREDDS Server


OPeNDAP


WCS

Application

NJ22

library

IDD Data

THREDDS OpenDAP Server


Catalogs instead of dods_dir


Next version of C++ library will also have
THREDDS catalogs


Catalogs automatically generated


Based on DODS
-
Java library and the
Aggregation Server.


Aggregation probably done at nj22 level


THREDDS WCS Server


WCS 1.0 specification


Proposing to add NetCDF as “blessed”
binary file return type


OGC interoperability experiment


Difficult to find clients to work with


Working with CADCorp client


Liping Di (George Mason)


NASA GSFC ‘Earth Science Gateway’


Looking for others

THREDDS WCS Server


Return formats


Geotiff floating point


NetCDF binary


No reprojections for now


Must have Grid coordinate system


Issues


Vertical coordinates


Time coordinates


Large number of coverages (granularity)

NetCDF
-
4

NetCDF
-
Java 2.2

NetCDF
-
4


Project funded by NASA to create new version of
netCDF using the HDF5 file format.


“Extend and merge” netCDF and HDF5:


Widespread use and simplicity of netCDF
-
3


Generality and performance of HDF5


Specifically, we are funded to create netCDF
-
4 C library
API, using HDF5 library underneath.


Russ Rew (PI), Ed Hartnett (Unidata), Mike Folk (co
-
PI),
Quincey Koziol (NCSA)


16

NetCDF
-
4 C Library

HDF5 Library

netCDF
-
4 Library

netCDF
-
3

Interface

NetCDF
-
4 Architecture

NetCDF
-
4 and Java


100% Java library for netCDF
-
4 files possible?


NetCDF
-
Java 2.1 already a 100% Java library for
netCDF
-
3 files (and OPeNDAP)


NetCDF
-
Java 2.2: read HDF5 to determine what
netCDF
-
4 data model should be


“Common Data Model” emerged for netCDF, HDF5 and
OpenDAP.


OpenDAP 4.0 protocol being worked on


The 3 groups are actively discussing


NetCDF
-
Java 2.2 (nj22)


100% Java library for netCDF,
HDF5, OpenDAP, and more


THREDDS integrated


Under development


Alpha release: Nov 2004


Beta release: May 2005


Release: summer 2005

NetcdfDataset

Application

Scientific Datatypes




Grid

Station

Radial

NetCDF
-
Java
version 2.2
architecture

ADDE

OpenDAP

THREDDS

Catalog.xml

NetCDF
-
3

HDF5

I/O service
provider

GRIB

GINI

NIDS

NetcdfFile

NetCDF
-
4



Nexrad

DSMP

Trajectory

Image

I/O Service Provider

Implement this interface:



public interface IOServiceProvider {



boolean isValidFile( RandomAccessFile raf);




void open( RandomAccessFile raf, NetcdfFile ncfile);



Array readData( Variable v2, List section);




// only if you use Structures


Array readNestedData( Variable v2, List section);


}


File Format

#N

File Format

#2

File Format

#1

CDM

Visualization

&Analysis

Goal: N + M instead of N * M things on your TODO List

NetCDF file

OpenDAP Server

WCS Service

Common
Data Model
(CDM)

Coordinate Systems



Same underlying mathematics as VisAD, ASCII

Scientific Data Types


Point / Station (metars)


Unconnected


DQCs used with IDD station/radar data


Trajectory (floating buoys)


1D connected


Radial (NEXRAD)


elevation ,azimuth, distance


Grid (Model output)


cartesian x, y, z


WCS


HTTP Tomcat Server









THREDDS Data Server

Datasets

Catalog.xml

hostname.edu

THREDDS Server


OPeNDAP


WCS

Application

NJ22

library

IDD Data

NJ22 library

NJ22 UI library

Summary


THREDDS data server combines Catalog
Services and data servers


As nj22 matures, data server will have
more functionality


OpenDAP, WCS


NetCDF
-
4: simplicity of netCDF and
flexibility of HDF5