4-Dimensional Weather Data Cube

tendencyrheumaticInternet and Web Development

Nov 12, 2013 (3 years and 9 months ago)

173 views


4
-
Dimensional Weather Data Cube

Web Coverage Service

Reference Implementation
(
WCS
RI)

Architecture and Design


Version
2.0

01
/
15/2010







National Center for Atmospheric Research

MIT Lincoln Laboratory

National Oceanic and Atmospheric Administration/Glo
bal Systems Division


DOCUMENT REVISION REGISTER

Version

Date

Content Changes

Editors

Contributors

0.1

03/16/2009

Preliminary draft

Rob Weingruber

Aaron Braeckel


0.2

04/10/2009

Revised to s
upport a system design
template more closely tied

to the
WFSRI D
esign and Architecture
Document. Multiple other content
refinements

Aaron Braeckel


1.0

04/30/2009

Updated to be more similar
in format
and to include

additional

relevant
content from

WFSRI
Design and
Architecture
Document
v0.
1
.
Incorporated comments fr
om NNEW
PO
and
labs.

Aaron Braeckel

Rob Weingruber

1.1

12/8/2009

Initial draft of 2.0 document changes
.
Updated to reflect design changes and
experience gathered from
implementing WCSRI 1.0, updated to
reflect the current status of SWIM, and
generally im
proved wording and
content

Aaron Braeckel


2.0

1/15/2010

Finalized revisions from 1.1

Aaron Braeckel



Please direct comments or questions to:

Rob Weingruber

National Center for Atmospheric Research

Research Applications Laboratory



3450 Mitchell Lane



Boulder, CO 80301



weingrub@ucar.edu



(303)497
-
2850



Aaron Braeckel

National Center for Atmospheric Research

Research Applications Laboratory

3450 Mitchell Lane

Boulder, CO 80301


braeckel@ucar.edu


(303)497
-
2806



Terms of Use


NNEW Documentation


The following Terms of Use applies to the NNEW Documentation.

1.

Use. The User may use NNEW Documentation for any lawful purpose without any fee or cost.
Any modification, re
production and redistribution may take place without further permission so
long as proper copyright notices and acknowledgements are retained.

2.

Acknowledgement. The NNEW Documentation was developed through the sponsorship of the
National Oceanic and Atm
ospheric Administration and the Federal Aviation Administration.

3.

Copyright. Any copyright notice contained in this Terms of Use, the NNEW Documentation, any
software code, or any part of the website shall remain intact and unaltered and shall be affixed
to
any use, distribution or copy. Except as specifically permitted herein, the user is not granted any
express or implied right under any patents, copyrights, trademarks, or other intellectual property
rights with respect to the NNEW Documentation.

4.

No End
orsements. The names, MIT, Lincoln Labs, UCAR and NCAR
, may

not be used in any
advertising or publicity to endorse or promote any program, project, product or commercial
entity.

5.

Limitation of Liability. The NNEW Documentation, including all content and
materials, is
provided "as is." There are no warranties of use, fitness for any purpose, service or goods,
implied or direct, associated with the NNEW Documentation and MIT and UCAR expressly
disclaim any warranties. In no event shall MIT or UCAR be liab
le for any damages of any nature
suffered by any user, or any third party resulting in whole or in part from use of the NNEW
Documentation.




Table of Contents


Part 1: Introduction

................................
................................
................................
................................
.......

8

1

PURPOSE

................................
................................
................................
................................
...............

9

2

DESIGN GOALS

................................
................................
................................
................................
....

11

Part 2: Proposed Software Architecture

................................
................................
................................
.....

13

3

ARCHITECTURE AND DESIGN OVERVIEW

................................
................................
............................

14

3.1

SYSTEM ARCHITECTURE OVERVIEW

................................
................................
..........................

14

4

SUBSYSTEM DECOMPOSITION (MODULES)

................................
................................
........................

16

4.1

WCSRI Tiers

................................
................................
................................
................................

16

4.1.1

Protocol Tier

................................
................................
................................
..........................

16

4.1.2

Domain Tier

................................
................................
................................
...........................

17

4.1.3

Data Source Tier

................................
................................
................................
....................

18

4.2

SUBSCRIBER MANAGEMENT

................................
................................
................................
.....

19

4.3

LOGGING

................................
................................
................................
................................
....

20

4.4

MONITORING

................................
................................
................................
.............................

20

4.5

AUDITING

................................
................................
................................
................................
...

21

4.6

WCS PROTOCOL

................................
................................
................................
.........................

22

4.6.1

GetCoverage “store” attribute

................................
................................
..............................

22

4.7

WCS PROTOCOL EXTENSIONS
................................
................................
................................
....

22

4.7.1

Subscription
-
Specific Extensions

................................
................................
...........................

22

4.7.2

Extended Trajectory Operations

................................
................................
...........................

22

4.7.3

Coordinate Reference System Extensions

................................
................................
............

24

4.7.4

Units and Measures Extensions

................................
................................
............................

25

4.7.5

Geographic Subsetting Extensions

................................
................................
........................

26

HARDWARE/SOFTWARE MAPPING

................................
................................
................................
............

27

4.8

SOFTWARE ENVIRONMENT

................................
................................
................................
.......

27

4.8.1

Programming Languages

................................
................................
................................
.......

27

4.8.2

Web Application Framework

................................
................................
................................

27


4.8.3

Data Management

................................
................................
................................
................

28

4.8.4

Build and Deployment

................................
................................
................................
...........

28

5

PERSISTENT DATA MANAGEMENT

................................
................................
................................
.....

29

5.1.1

Users/Roles

................................
................................
................................
...........................

29

5.1.2

Coverages

................................
................................
................................
..............................

29

5.1.3

Application Data

................................
................................
................................
....................

31

5.1.4

Lifecycle Data

................................
................................
................................
........................

31

6

ACCESS CONTROL AND SECURITY

................................
................................
................................
.......

33

6.1

ACCESS RESTRICTION

................................
................................
................................
.................

33

7

GLOBAL SOFTWARE CONTRO
L

................................
................................
................................
............

34

8

BOUNDARY CONDITIONS

................................
................................
................................
....................

35

APPENDIX A
-

ADDRESSED REQUIREMENTS

................................
................................
...............................

36

APPENDIX B
-

ACRONYMS

................................
................................
................................
...........................

40

APPENDIX C
-

DEFINITIONS AND TERMS

................................
................................
................................
.....

42

APPENDIX D
-

REFERENCES

................................
................................
................................
.........................

43




Table of Figures


Figure 1. 4
-
D Wx Data Cube Components

................................
................................
................................
..

10

Figure 2. Overview of the WCSRI 3
-
tiered System

................................
................................
......................

15

Figure 3. Temporal Trajectory

................................
................................
................................
....................

23

Figure 4. Temporal Trajectory CRS

................................
................................
................................
.............

24

Figure 5. WCS

Trajectory UML

................................
................................
................................
...................

24



Part 1:

Introduction


1

PURPOSE

To satisfy the requirements of Next Generation Air Transportation System’s (NextGen) 4
-
Dimensional
Weather Data Cube (4
-
D Wx Data Cube), it is envisioned that a number of comp
onents will comprise the
physical solution. The Next Generation Air Transportation System (NextGen) Network Enabled Weather
(NNEW) Program has recommended the adoption of several standards to be used as part of that solution.
To provide a definitive interp
retation of these standards, a production
-
quality implementation for
operational parties, and a resource for operational data providers that choose to implement these standards
differently, the NNEW Program will create reference implementations of key data

distribution standards:
the Web Feature Service and Web Coverage Service. These reference implementations are developed and
licensed in an open, standards
-
based fashion to allow broad usage.

High
-
level requirements for the 4
-
D

Wx

Data

Cube have been estab
lished over the past

several years by
the Joint Program Development Office (JPDO). Documents describing these requirements include the
“Concept of Operations for the Next Generation Air Transportation System [
i
@??GRFXPHQW??WKH?³)RXU
-
Dimensional Weather Fun
ctional Requirements for NextGen Air Traffic Management [
ii
]”, and others.
With these high
-
level requirements in mind, the lower level “NNEW IT CONOPS [
iii
@´?FDSWXUHG?D?VHW?RI?
4
-
D

Wx

Data

Cube requirements relevant to network
-
enabled weather
.

The NNEW IT CON
OPS
document provides a representative set of use cases that are intended to aid system architects in
understanding typical usages of the 4
-
D

Wx

Data

Cube. Though not exhaustive, the selection of use cases
represents an attempt to ‘span the problem domain’
, covering both functional and non
-
functional
scenarios.

The NNEW P
rogram also developed a
WCS
RI
R
equirements
D
ocument [
iv
] that focuses more closely on
implementation issues than the NNEW IT CONOPS document an
d includes further context and
background. This document describes the high
-
level interaction and design of the
WCS
RI based on
requirements identified.

Requirements mentioned in this document that come from the
WCS
RI
R
equirements
D
ocument are
shown in bol
d, similar to
Requirement 1.2.3
. These note
s refer to the version of this R
equirements
D
ocument indicated by [
iv
].

This document describes the architectural aspects and design of the
Web Coverage Service

Reference
Implementation (
WCS
RI), which is well suit
ed for serving geographically filtered
coverage

data. This
document has two purposes:

1.

Describe the design of this phase of development in adequate detail to enable the development of
the
WCS
RI. This description will facilitate subsequent implementation an
d deployment phases
of
the
4
-
D

Wx

Data

Cube
.

2.

Describe how the
WCS
RI technology will integrate with other technology components, such as
the FAA System Wide Information Management Program (SWIM) and 4
-
D Wx Data Cube Web
Feature Service Reference Implementat
ion (
WF
S
RI).

Note:

This document is intended to describe only high
-
level architectural and design concepts.
Implementation details are left to other forms of documentation.


Because the design and implementation of the
WCS
RI is iterative, this document add
resses a subset of the
total requirements of the
WCS
RI. The initial design phase must precede implementation, but the design
and implementation work may be considered to be on somewhat separate iterations after that point.
Therefore, the initial design w
ork may enable several phases of implementation before further iteration is
made on the design. The requirements addressed by this document are listed in

APPENDIX A
-

ADDRESSED REQUIREMENTS
.

Important Note:

This document is not in
tended to describe the functionality included in any particular
WCS
RI release, merely the high
-
level design concepts that may be used as guidance in implementing the
functionality once the appropriate
WCS
RI implementation phase is reached. Additionally, t
his document
describes architecture and design concepts for the
WCS
RI, but these concepts will almost certainly be
adjusted as implementation progresses.

Apart from th
ose included in the OGC
WCS

specification, some requirements are imposed on the
WCS
RI
sim
ply because it is intended to be a component within the 4
-
D

Wx

Data

Cube. This includes capabilities
such as support for real
-
time data dissemination, flexible architectural configuration, infrastructure
support for logging and monitoring, lifecycle manage
ment of data, and support for accident
reconstruction.

In addition, the
WCS
RI must integrate with other
4
-
D

Wx

Data

Cube components (see
Figure

1
). The
4
-
D

Wx

Data

Cube’s
W
F
S
RI is intended to provide access to
non
-
gridded

data, and is relatively isolated
from
other
4
-
D Wx Data Cube components
. Therefore, it is not envisioned that integration between the
WCSRI and
WF
S
RI need occur. However, the 4
-
D

Wx

Data

Cube Registry/Repository will contain
registry information ab
out each
WCS
RI installation, such as
coverage
metadata and
WCS

service
endpoints. Therefore some level of integration must occur between the two, though no implication is
made as to the specific direction of communication or dependency between the
WCS
RI in
stallations and
the Registry/Repository.


Figure
1
.
4
-
D Wx Data Cube

Components


2

DESIGN GOALS

There are several goals for the Reference Implementation of a
Web Coverage Service
:



Dependability
:

o

Robust implementation of the
WCS

1.1
specification itself. An implementation of the
standard promotes interoperability across the broad spectrum of weather data producer
and consumer communities.

The
WCS
RI should be robust enough to withstand invalid
user inputs.

o

Fault Tolerance. The
WCS
RI s
hould be fault tolerant to network and machine failures
and be able to recover gracefully from such failures. Fault tolerance here implies that
a

WCS
RI crash does not result in data corruption or any security malfunction.


o

Operational quality. The
WCS

Refe
rence Implementation is intended to guide future
development of vendor implementations, as well as providing an operational server.
Because the
WCS
RI will be deployed as part of the 4
-
D

Wx

Data

Cube initial operating
capability (IOC), the implementation m
ust be of production/operational quality and must
support enterprise
-
level capabilities
such as monitoring and security
-

either within the
WCS
RI itself or through its surrounding infrastructure.

o

Scalability.
It is almost certain that t
he 4
-
D Wx Data Cube
architecture
will be

built to
scale within each node by adding new pieces of hardware. The
WCS
RI must
accommodate this architecture, and therefore the addition of a piece of hardware to a
node with the
WCS
RI on it must scale well. The primary characteris
tic is that the
WCS
RI scale horizontally (i.e. it scales well as new computers are added to a group)
rather than vertically (where additional CPUs, memory, and other capability
are

added to
a single piece of hardware).



Performance
:

o

Response Time. The
WCS
RI

should be able to provide fast response time (on the order of
milliseconds/seconds) once a user request has been issued and received.

o

Throughput
. The
WCS
RI must be able to handle several concurrent requests at the same
time.



Maintenance
:


o

Modifiability.
To accommodate future modifications to the
WCS

specification and the
corresponding GML schema specification, the
WCS
RI should be modifiable so that it can
be easily updated.

o

Extensibility. The
WCS
RI will be implemented in stages. For current development as

well as for future development, it is essential that the
WCS
RI be extensible, that is
,

it is
easy to add new functionality.





Usability
:

o

Out
-
of
-
the
-
box solution. The intention of the
WCS
RI is to provide a baseline
implementation that may be used, out of th
e box, by any gridded data Service Provider
participating in the 4
-
D Wx Data Cube. This implementation should be easy to install and
configure by the Service Provider. This will alleviate the need for Service Providers to
develop their own custom
WCS

solut
ions.




Other
:

o

4
-
D

Wx

Data

Cube integration. There are several NNEW
-
foundational components that
will comprise the 4
-
D

Wx

Data

Cube
-
, including the
WCS
RI, WCSRI and the
Registry/Repository. Though the
WCS
RI may be functional within an isolated
environment,
it will also need to integrate with the other components in order for it to be
operational within the 4
-
D

Wx

Data

Cube.

o

SWIM and FTI integration


The FAA SWIM Program provides an infrastructure for
service
-
oriented capabilities within the FAA. The Federa
l Telecommunications
Infrastructure (FTI) provides a network infrastructure. Since at least a portion of the
4
-
D

Wx

Data

Cube
-

will be run within the FAA, within SWIM and over FTI, the WCSRI
must integrate with that infrastructure.




Part 2:

Proposed Soft
ware Architecture


3

ARCHITECTURE AND DESIGN OVERVIEW

This section provides an overview of the architecture and design of the
WCS
RI.

3.1

SYSTEM ARCHITECTURE OVERVIEW

This section gives a high level overview of the
WCS
RI architecture including a brief description
of the
sub
-
systems that comprise the overall
WCS
RI system and the interactions between the sub
-
systems.

The
WCS
RI architecture follows the general model of a
closed

layered

system. A
layered

system is an
ordered set of
layers

or
tiers
, where
a layer or
tie
r is built using the functionality of the layers below it and
the layer itself provides the implementation basis for the layers above it. While the objects in each layer
can be independent, there is typically some coupling between the objects in the differ
ent layers.
Knowledge is one
-
way only


a sub
-
system knows of the layers below it, but has no awareness of the
layers above it. A client
-
server relationship exists between the upper layers (users of services) and the
lower layers (providers of service) [Ru
mbaugh]. In a
closed

architecture, each layer is built only in terms
of the immediate lower layer. This reduces the dependencies between the layers and allows changes to be
made easily as the interface of any layer impacts only the layer above it. This is
in contrast with an open
architecture that allows each layer to access any of the layers below it. While an open architecture reduces
the need to wrap or duplicate operations at each level, it violates the principle of information hiding and
changes can ri
pple through the entire system. To allow for future extensibility and modifications, the
WCS
RI is being designed as a closed, layered system.

Note:
More specifically, this approach corresponds to a 3
-
tier layered approach with the three principal
layers f
or enterprise applications: Presentation, Domain, and Data Source [
v
].

Figure
2

shows the three basic tiers, as well as major components of each
tier.
Each tier is described in
more detail in section
4
.

M
ultiple protocols may be eventually supported by the WCSRI at the same time. To allow for future
protocol modifications and additions, the WCSRI is designed with a pluggable protocol layer that ties into
the common functionality and data store. There
fore, if newer versions of the WCS, JMBL
,

or OpenDAP
protocols become
necessary

at some point in the future
,

they
may
be added with minimal changes to the
remainder of the code base.

Similarly, the WCSRI is designed to support a pluggable data source tier.

This will allow for future data
store file formats (e.g.
,

relational database
), delegating access to other WCSRIs, or alternative approaches
to data storage such as a mix of relational database and files.

I
nteractions with other WCSRI instances are plann
ed (and are included at a high level in the design) but
are left to subsequent design phases to be fully fleshed out. The pluggable data source tier will assist in
allowing for different communication patterns, such as delegation to an upstream WCSRI.

Ther
efore, all user communications are handled by the protocol tier and are passed to the domain tier for
handling.

In the case of data notifications the

domain tier
Notification
Engine

will

monitor the gridded

data file store and will notify the protocol tie
r
of the need
to send messages to subscribers u
sing a specific
technology (such as the FUSE Message Broker:
A
ctiveMQ).


Figure
2
. Overview of the WCSRI
3
-
tiered System


4

SUBSYSTEM DECOMPOSITION (MODULES
)

This section gives a more de
tailed overview of the
WCS
RI modules. In contrast to
Section
3.1

and
Figure
2

which provide a layered view of the overall architecture, this section takes a more functional point of
view providing a mo
re in
-
depth look at how sub
-
systems across the layers can be used to accomplish
desired functionality.

Note:

these modules are logical groupings, and may or may not be segregated exactly as shown in the
final implementation.

4.1

WCSRI Tiers

This section

descri
bes in detail the three
-
tiered architecture briefly described in Section
3.1
.

4.1.1

Protocol Tier

Per the WCSRI
Requirements Document

[
iv
], the initial need for the WCSRI will

be to suppor
t WCS
version 1.1
. However, future versions of the
OGC
WCS specification as well as other additional protocols
will need to be supported in the long term. Protocols and data formats tend to change much faster than the
functional aspects of retrieval servic
es. To allow for future protocol support, the WCSRI will be
implemented with a pluggable protocol tier that will allow multiple protocols to be implemented and
plugged in to the WCSRI without directly affecting the core functionality.

The protocol tier ha
s two basic responsibilities: request parsing and response generation. Request parsing
involves taking a request (most notably a WCS 1.1.2
GetCoverage

request) from a consumer and
translating it into
domain

objects that represent the request semantics. R
esponse generation takes
domain

object results and formats them as a WCS XML response, a NetCDF file, or whatever representation is
appropriate for that particular protocol.

Request parsing produces a generalized set of
domain

objects that represent the lo
gical intent of that
request that are independent of any particular protocol. Therefore, the domain layer will make no
reference to HTTP, SOAP, WCS, or any other transmission technology and will instead represent
Requests, Constraints, and other abstract
information related to serving

gridded data
. The domain tier is
described in further detail in Section
4.1.2
.

Each of the elements of a domain response will be appropriately transformed from their domain
repres
entations into a protocol
-
appropriate response equivalent. For example, a critical error in the
domain tier (e.g.
,

a simple Java string or object) will be returned through the WCS 1.1.2 protocol tier as a
WCS 1.1.2
-
compliant error. The same error from an
other request using OpenDAP would be translated
and returned as an OpenDAP error.

As protocols are added they will be encapsulated as new modules. Therefore, protocol modules may be
plugged in and configured relatively independently of other protocols, th
e core domain logic, and the
backing data store.

For example, protocol modules
might

eventually include:




WCS 1.1.2



WCS 1.1.2, with 4
-
D Wx Data Cube extensions



WCS 2.0



OpenDAP 2.0



Web Coverage Processing Server (WCPS) 1.0



Etc.

All of these protocols can mak
e use of the same underlying domain logic, which enables for a unified and
well
-
tested code base for this behavior. Additionally, this separation of concerns is a good illustration of
modularity, in that allows protocol components change relatively indepe
ndently of the rest of the system.

Within the SWIM Service Container (FUSE), the WCS 1.1.2 support may be implemented using CXF
and JAXB bindings. This module will include all the generated Java classes and all code related to
parsing these generated clas
ses into the domain model, as well as the logic required to format a NetCDF
-
CF and/or XML response.

4.1.2

Domain Tier

The domain tier is a set of objects that represent gridded data serving requests and responses
independently of a particular protocol or data st
orage mechanism. This tier is an abstraction of the
superset of functionality that is implemented by all protocols and data storage mechanisms. For example,
if one implemented protocol includes support for re
-
gridding with a different resolution and anot
her
protocol has support for unit conversion, the domain tier would include objects for representing both
functional elements.

The two core concepts in the domain tier are the Request and Response. Because these are part of the
domain tier, neither specif
ically includes information on transport mechanisms, data formats, or protocols.
Requests may eventually include:



A basic gridded data Request



A point time series Request



A gridded dataset description Request (e.g.
,

a list of times or its bounding box)



Et
c.

Requests include information such as constraints/filters (such as temporal and spatial bounds), the desired
coverage identifier, the desired output format identifier, and so on.

Corresponding to each Request type, there will be a Response:



A basic gridd
ed data Response




A point time series Response



A gridded dataset description Response



Etc.

Responses encapsulate request status, including warnings and errors (such as when a Request does not
include sufficient information or is improperly formed) and the r
esponse data (gridded data values).

4.1.2.1

Coordinate Reference System Module

The Coordinate Reference System

module
deals with

EPSG codes, projected coordinate reference
systems (
e.g.
,

Lambert Conformal, Stereographic) and transformations between different coord
inate
reference systems.
The module

also

includes support for defining gridded coordinate reference systems
and their relationship to a base reference system. This is likely to be implemented as part of

the domain
tier because it is a common need across

different backing stores.

4.1.3

Data Source Tier

This tier will be given Request objects from the domain tier, will access data storage (such as files or a
relational database) and will build appropriate domain Response objects from that information. This tier

handles all the data storage
-
specific filtering, retrieval
,

and domain Response generation. The domain tier
will make calls against the data source tier, but will not know anything about the file formats of the
backing store, how data
are

filtered, or an
y of the other storage
-
specific details.

4.1.3.1

Physical data store access

This is a module that can access a set of files on disk and in most cases perform filtering and
transformation functions against a backing data store (such as a directory of data files).

The most
prominent example of this type of module is one that accesses a directory structure that contains NetCDF
-
CF files.

4.1.3.2

WCSRI to WCSRI Interactions


The 4
-
D

Wx

Data

Cube will likely be composed of a hub and spoke architecture that requires
communica
tion across WCSRI instances. There are many topologies that may be used for the
4
-
D

Wx

Data

Cube in order to optimize request performance, data flow latency, bandwidth, physical node
capabilities, network topology, and other considerations. A notional depi
ction of some of the roles a
WCSRI may take on is described below. In general, the WCSRI will be designed flexibly to
accommodate a broad range of different topologies and specific architectures. This will probably require
communication, intelligent data m
anipulation, and caching mechanisms across WCSRI nodes.

In order to achieve this design goal, the WCSRI will leverage the data access façade as mentioned
previously. Access to data available in another WCSRI may be considered another type of data store
acc
ess module, in that it can be exchanged with another module that access
es

files from disk on the local
machine. Specific implementations of that façade may differ not only by data access mechanism, but also
by the type of WCSRI interactions supported. Dif
ferent façade implementations may be implemented to
support the following communication patterns:




Aggregator

A WCSRI instance may act as an
Aggregator

for a specific dataset. This policy involves taking
pieces of the same dataset from more than one upstrea
m node. This aggregates remote files to a

single (local) data source.(i.e.,

pull the remote files from several data sources/WCSRIs to a single
data source/WCSRI). Or aggregate dynamically, as a Delegator. An example of this pattern
would be a WCSRI that
aggregates and stores local radar data that
are

available
from
three

different sources upstream, each with a different geographic area. This would be available from
the aggregator as a single radar coverage with all the data available from upstream nodes.



Delegator (without caching)

The non
-
caching delegator pattern is one where a WCSRI will forward data requests to one or
more upstream WCSRIs, and does not store data locally.



Delegator with dynamic cache

A delegator with a dynamic cache is similar to a
non
-
caching delegator, except that frequent
requests for the same remote data will sometimes be stored locally to optimize performance.
There are many types of caching policy that may be used. The cache may manifest itself as
pub/sub on a temporary or per
manent basis (the latter being Repeater with Permanent Cache).
This cache can reside on the data store or in memory, depending on caching policy and
performance decisions.



Repeater

A WCSRI may s
ubscribe to an upstream WCSRI for updates to a particular cov
erage (such as
new data arrival). This subscription may optionally include filtering information to only be
notified of updates to that coverage that match the filtering criteria (such as geographic area).

4.2

SUBSCRIBER MANAGEMENT

T
he WCSRI will support the
capability to notify data consumers when data becomes available
that
match
es

desired filtering criteria. The default implementation will be based off of the FUSE Message
Broker (ActiveMQ)

and SOAP
-
related notification standards
. W
hile th
ese

are

the only
currently
identified notification technology requirement, the WCSRI will be built to support a general capability
that can be used for a broader range of notification technologies in non
-
FUSE environments, such as
ATOM feeds, other JMS solutions, and HTTP
callbacks.
A
core WCSRI notification engine

will send
notifications to pluggable modules

which are triggered when the WCSRI discovers new data that matches
subscription criteria. The core WCSRI notification engine will be a part of the domain and/or data

source
tiers.

This mechanism will be
closely related to

the WCS protocol filtering criteria, so the means to receive
notifications and the means to grab data in an ad
-
hoc manner are relatively uniform. This allows for a
similar code base and protocol to
identify filtered subsets of interest to consumers. Therefore,
subscription requests by data consumers will include WCS
-
like filtering syntax.


This subscription is held until it expires, and with the default notification technology a FUSE Messaging
Brok
er channel is opened for that particular subscription. When new data
are

discovered by the WCSRI
that matches the filtering criteria for that subscription, a notification is sent to the data consumer through
the channel. The data consumer may then reques
t data through typical WCSRI means.

The Subscriber

Bridge and the Subscriber together manage the real
-
time
, asynchronous

dissemination of
coverage

data
.

The Subscriber

Bridge provides an interface to the
data consumer
, while the
Notification
Engine

provide
s
the basic notification capabilities independently of a particular delivery mechanism
.

This relationship is shown in
Figure
2
.



Notification Engine
: The

Notification Engine is the basic, protocol
-
independent piece of logic
that i
dentifies when coverage data relevant to a data consumer has arrived, and
passes
information to

a Subscriber Bridge to notify interested data consumers.




Subscriber Bridge
:

The Subscriber
Bridge
handles

the data encoding and format conversions
required by
the subscriber, as well as the delivery of the message to the subscriber.

4.3

LOGGING

Logging needs will potentially cross all portions of the WCSRI, and are the most prominent example of a
cross
-
cutting concern. Logging will be built on top of the Simple Logg
ing Façade for Java (SLF4J).
SLF4J is a façade that provides a flexible mechanism to support a number of underlying Java logging
packages (
e.g.
,

LOG4J, central Java Logging API, etc). SLF4J with Log4J is the default logging
approach in FUSE, and is broad
ly applicable in any environment.

There are several types of logging:



Development/debugging
. Used to trace problems or behavior for specific cases or short periods
of time. This will be implemented with AOP or with hard
-
coded calls in Java source



Access/

traceability/runtime
. Used to log requests, responses, and is generally used as a way to
have a background log of accesses. This can be compared to the access log of the Apache web
server. This will also include basic information on notifications and ot
her non
-
request/response
communications. Implemented as AOP.



Error
. Used to log all errors and warnings. Similar to the error log of the Apache web server.
Implemented in AOP, and possibly hard
-
coded Java source.



Request reconstruction
. Extremely detail
ed logging sufficient to reconstruct a request and
response from a data consumer at a particular time. Unclear whether this will be best
accomplished through AOP or other techniques

4.4

MONITORING

Monitoring information collected and exposed by the
WCS
RI incl
ude
s
:



Available
coverage
s with basic metadata and configuration information, such as what protocol(s)
are currently offered




WCS
RI service configuration status, including the exposed endpoints



Performance information (gathered on a per
-
coverage

basis)

o

Mean
response time

o

Mean backing store seek time (response time for the database layer alone)

o

Mean request (user communication layer) parsing time, for incoming request translation
into domain objects

o

Mean response (user communication layer) generation time, for

outgoing response
generation after data is retrieved



Per
-
coverage

request counts (a count of all requests by type, broken down per coverage)



Critical error status

Additional monitoring and management capabilities will be added as needed, particularly as
SWIM
information becomes available.

Monitoring support is provided by the Logging sub
-
system in the Information Retrieval
layer

that logs all
activities in the Logging Database. The Logging sub
-
system provides the interface for logging at different
levels

with the ability to turn on or off different types of logging. Additionally, logging information
collected will be collated and common statistics computed periodically by the Logging sub
-
system to
provide better monitoring performance.

4.5

AUDITING

Auditing f
unctionality will be supported primarily through logging mechanisms and specialized request
support. This will involve additions to the protocol tier to support qualified users making auditing
requests. It is desirable to segregate auditing functionality

from the rest of the WCSRI to provide a
separation of concerns, but it is not currently known whether this will be possible because of the probable
close relationship between auditing requests and standard data requests.

Auditing requires two components:
a logging mechanism to collect information sufficient to later
reconstruct a consumer request, and request support that allows for the retrieval of this historical
information.

Note:
A
uditing support will not save every request from and response to data co
nsumers.
While the
raw
data will be

preserved

so the responses can be replicated
,
the
filtered response data
for every request
will

not

be kept
.
Therefore
,

it is truly request reconstruction rather than a verbose logging mechanism.
Saving the actual res
ponses sent to all clients over 15 days would be very prohibitive for 4
-
D Wx Data
Cube
nodes
, and the responses sent to consumers can be replicated
by keeping the necessary amount

of
raw
data results

rather than
all of the
responses
.


4.6

WCS

PROTOCOL

4.6.1

GetCovera
ge “store” attribute

The WCS specification provides two mechanisms for returning the coverage binary files from a
GetCoverage

request. Though the
GetCoverage

response consists of XML, the actual coverage data file(s)
can be returned in a SOAP message with
attachments, or it can be stored temporarily on a server for
HTTP download access.

The preferred approach to returning binary data is to return the data file(s) as a SOAP message with
attachments. The alternative approach of storing the data temporarily o
n the server has implications with
respect to scalability and statelessness, complicates the security requirements for a single GetCoverage
request (i.e., to get a single coverage requires the initial, secure request that stores the temporary data file
on
the server, and then requires an additional secure request to retrieve that temporary data file), and
inherently complicates the policy used for storing the temporary data files such as configuring how long
each file should be saved. Therefore, the SOAP w
ith attachments approach will be used by the WCSRI
and storing resu
lt files will not be supported

initially
.

4.7

WCS PROTOCOL EXTENSIONS

To handle Get
Coverage

requests in a uniform manner, while allowing for the results to be returned as
a
standard request/re
sponse or alternatively allowing the client to set up a persistent query/subscription, it is
proposed that the OGC
WCS

specification be extended to allow clients to specify the
method of result
delivery.

This would allow seamless extension of Get
Coverage

w
ithout requiring any external
modifications to the Get
Coverage

interface itself.

This approach
can
also
be common across other OGC
service specifications
, such as the WFS
.

4.7.1

Subscription
-
Specific Extensions

To adequately support persistent queries, the
WCS

G
et
Coverage

operation

will
be extended to allow
clients to
specify subscription parameters, including:



Latency
: Maximum tolerable delay for the transmission of a message to the subscriber



Window
: The length of subscription time. The client is subscribed fro
m now to forever by default,
but the subscriber has the option to specify the duration.



Update Frequency
: To control message frequency at the subscriber, the subscriber can specify
how often they would like to receive updates, for example once every minute
, or once every hour.



Renewal:
Allows the renewal of an already configured subscription

4.7.2

Extended Trajectory Operations

The WCSRI will support the coverage geometries, including the temporal trajectories, as
specified
previously in the WCSR
I Requirements Do
cument. This section describes how trajectories, cross
sections, and other complex geometries will be specified by an extension to the WCS specification.

A picture of a corridor along a temporal trajectory is shown in

Figure
3
. Th
e temporal trajectory is defined
by a CRS anchored in space and time, as shown in red in

Figure
4
. The CRS has an origin located at the
latitude, longitude, altitude, and time of the first waypoint of the trajectory. The X axis of

the CRS travels

along the temporal trajectory, from waypoint to waypoint, with a resolution equal to the number of
sample points taken along the path. The Y axis of the CRS is in the vertical, altitude direction, with the
positive axis increasing in altit
ude in units of kilometers. The Z axis of the CRS is the distance orthogonal
to the trajectory in the latitude/longitude plane, and is in units of kilometers.

Figure
5

depicts UML illustrating how the WCS specification supports t
his type of subsetting operation.
The light green objects represent the requested coverage subset. The “trajectoryCRS” is the GridBaseCRS
of the requested coverage grid. The example shows that the CRS will consist of 500 sample points in the
X direction, s
ampled along the trajectory path defined by the appended waypoints. Each waypoint is
defined by a latitude, longitude, altitude, and time, and is appended to the URI (or URL) used to define
the trajectory’s CRS. By dynamically defining the “trajectoryCRS”
object, the coordinate reference
system for the trajectory becomes anchored in space and time.

The “corridorCRS” object then describes the GridCRS that defines the resulting 3D coverage grid relative
to the “trajectoryCRS.” The corridor grid’s origin is lo
cated at the origin of the “trajectoryCRS.” The
“gridOffsets” represents the resolution of the resulting corridor grid in each of the three dimensions: the X
resolution is one sample point (sample points are taken evenly with respect to distance and time a
long the
trajectory’s path), the Y resolution is 0.1 km, and the Z resolution is 0.5 km. The BoundingBox specifies
the extents of the 3D corridor grid relative to the “trajectoryCRS.” The example shows that the 3D
corridor grid will extend from 0 to 500 sa
mple points in the X direction,
-
2 to +2 km in the Y direction,
and
-
10 to +10 km in the Z direction.

Simpler geometry subsetting can be performed in the same fashion. For example, a vertical cross
-
section
through a single forecast result (i.e., a single v
alid time) can be obtained by specifying each of the
waypoints using the same valid time, and by specifying the “BoundingBox” of the resulting grid to have 0
km extent in the Z direction (
e.g.,
, lowerCorner = “0
-
2 0” and upperCorner = “500 2 0”). A simila
r
technique can be used to extract horizontal cross
-
sections along a trajectory as well.


Figure
3
. Temporal Trajectory




Figure
4
. Temporal Trajectory CRS


Figure
5
. WCS Traj
ectory
UML

4.7.3

Coordinate Reference System

Extensions

The
WCS 1.1
spec allow
s

a server to advertise a set of fixed CRSs that may be requested by a data
consumer

for returned coverages
.


This limited set of CRSs is specified as either electronic definitions or

as authority codes (such as EPSG codes). A full gridded coverage
CRS
consists of several components: a
geodetic CRS

(including several pieces of information such as the assumed shape of the earth
),

a
projection CRS

(Lambert Conformal Conic, Mercator, etc.

with appropriate origins and other projection
parameters)
, and a
grid CRS that specifies the location of the grid relative to the projected space.

The meteorological community often makes assumptions with regards to the geodetic CRS for
computational reas
ons, and deals with a broad variety of projections, units, and vertical coordinate
systems. Lacking the ability to specify the full variety and configurations of each portion of a coverage
CRS is a long term maintenance burden. As new CRSs come into use
or are needed by downstream
systems, their definitions must be added to a long list of those advertised and supported for conversion.

Therefore
, it is much more practical, simpler to implement, and much more flexible to allow data
consumers to fully define

a CRS at request time. This allows the full capabilities of the re
-
projection
capabilities to be advertised and used in a
compact and
straightforward fashion, and does not limit data
consumers to a fixed set of CRSs.

Since the set of desired configurati
ons is infinite, it is necessary to
support these

richer CRS
mechanism
s
.

To support this, t
he WCS 1.1 specification will be extended
in two ways to
allow a
WCS server to
advertise and a
data consumer to
request the full range of CRSs.

1.

The WCS server shal
l advertise the projection types, projection parameters, datums, and
dimensional units that may be requested by a data consumer

2.

The WCS server shall extend the WCS protocol (the Domain portion of the GetCoverage and
related
data access
operations) to allow

a data consumer to fully specify a desired CRS
for a
returned coverage
in a standards
-
based manner

(most likely either Well
-
Known Text or GML)

4.7.4

Units and Measures Extensions

CRS definitions in GML and Well
-
Known Text have mechanisms for indicating the unit
s of particular axes,
such as the vertical axis being in units of kilometers. However, these mechanisms are not always
sufficient to describe the breadth of units used in the meteorological community (such as pressure,
differentiating
feet above ground le
vel
and

feet above mean sea level) nor are they consistent with the
NetCDF Climate and Forecast convention’s unit names.

The fundamental
drivers for the design

may be summarized as follows:

1.

A full set of meteorological units must be supported, including
a distinction between “above
ground level”, “above mean sea level”, and reasonable support for flight levels (which may not
always be

used
in terms of hundreds of feet MSL!) .
Additionally, pressure
-
based unit types
must be
supported
. The Unified Code fo
r Units of Measure specification and libraries are a good
starting point for these definitions, and are recommended in GML


2.

Consistency
among

the WFS (and therefore GML) specification and other existing WCS
-
related
specifications (WKT, NetCDF/CF)
. Ideally
a data consumer would be able to deal in one set of
unit definitions across the entire 4
-
D Wx Data Cube enterprise

To address the
se needs
, there are a couple of options:

1.

Make use of a meteorological extension to the Unified Code for Units of Measure that i
ncludes a
unit name for each CF unit and each
GML/WKT unit
.
Unit n
ames used in returned data files

would be in CF unit names
,
and
CRS definitions

can be specified in any CF or GML/WKT name.
This would result in mo
re than one name for many units (aliases)

and name clashes between CF
and GML/WKT would need to be resolved. A namespace (“gml:”, “cf:1.1:”, “wkt:”) preceding
the unit name could be used for disambiguation

2.

Return data with CF unit names, and deal in WKT/GML unit names for re
-
projection. This
pu
shes the burden onto the client for
understanding and associating

the two unit systems
. This
also requires that the WCS server internally be able to work with both systems

3.

Define a meteorological extension to the Unified Code for Units of Measure that inc
ludes a
single unique unit name for every unit of measure from CF and WKT/GML. These names would
be consistently used across all operations in the WCS. The big issue with this approach is that it
most likely breaks both CF compliance (and
therefore
NetCD
F library support) and
probably
WKT support. It is unclear if this would be compatible with GML CRS unit definitions

4.7.5

Geographic Subsetting Extensions

The WCS 1.1 spec allows for geo
-
rectilinear (lon/lat bounding box) filtering. However, it is necessary t
o
support the retrieval of sectors, routes, airways, and other geographically complex regions.
The WFS
specification includes support for this functionality
through

the OGC Filter Specification
, which is a SQL
-
like language for data filtering
.
The WFS
sp
ecification includes support for advertising

the components
of the Filter Spec that are implemented by the server, thereby allowing
implementers

of the
specification to only implement
relevant

portions

and not necessarily the entire query language
.

To achi
eve similar levels of support, t
he WCS will
extend coverage retrieval operations

to mirror the
WFS support for the OGC Filter Spec
.


This implementation will include

enough depth to allow coverage
data to be retrieved by a 2
-
D or 3
-
D series of points descr
ibing the outer boundary of the desired region.

This functionality will be advertised and implemented in a manner
maximally
consistent with the WFS
specification to allow similar queries

to be made against both server types
.

TODO:

describe the exact mecha
nism by which the OGC Filter Spec support is implemented within the
framework of the WCS 1.1 specification.


H
ARDWARE/SOFTWARE MAPPING

Mapping of the components (sub
-
systems) to processors enables identification of potential concurrency
among subsystems and

to address performance and reliability design goals. From a performance
perspective, where minimizing delay is a primary criteria, all
WCS
RI modules should be placed on a
single server machine. However, thr
ough
put and processing speed dictate that the dat
abase server be
placed on a separate machine.
The proposed
initial placement of the
WCS
RI components
is to have all

but
the data management server being placed on one server and the data management server placed on a
separate server.

Future versions of the

WCS
RI will consider a more distributed architecture with a caching infrastructure
to provide better performance for edge clients.

4.8

SOFTWARE ENVIRONMENT

This section describes some of the central technology decisions for the
WCS
RI. These decisions are
unli
kely to be affected or significantly changed as a result of experience gained through the
implementation process.

4.8.1

Programming Languages

The
WCS
RI will be implemented in Java. Java is one of the primary programming languages currently in
use for building w
eb services, and provides a broad and mature set of functionality. Almost all SWIM
Implementing Programs (SIPs) are using Java as an implementation basis, and the NNEW
implementation team has extensive experience with development in Java. Java has a numb
er of
advantages (
e.g.,
, platform independence and mature
common
libraries) that are well described elsewhere.

4.8.2

Web Application Framework

The SWIM program purchased and makes available two basic technology stacks: the FUSE ESB
framework and the Artix ESB fr
amework, both offered by Progress Software. A minority of the SIPs
have considered and/or made use of the more heavy
-
weight Oracle Application Server stack based on
WebLogic. The Oracle Application Server comes as part of an FAA
-
wide Oracle license.

Whil
e the two Progress Software ESBs may eventually be merged, the Artix ESB is currently oriented
towards CORBA and legacy environments. The FUSE ESB is oriented towards Java Web Service
environments. The FUSE ESB is also available as open source software,
in addition to its licensed and
supported form. Because of the lightweight, Java
-
centric, open source yet supported nature of the FUSE
ESB, it was chosen as the primary ESB for the
WCS
RI. While the Oracle Applic
ation Server is mature
and well
supported,
it would present a larger learning curve, relatively similar capabilities, and licensing
fees for 4
-
D Wx Data Cube participants outside of the FAA.

As stated in the
WCS
RI Requirements
D
ocument, the
WCS
RI is designed to maximize its functionality
across a b
road range of environments. While the FUSE ESB will be used heavily inside the FAA and
SIPs, it may not be available or desirable outside of these groups. Therefore, FUSE functionality will be
leveraged for what it provides but it will be separable from
core
coverage
s desirable in other

environments. To maximize adoption and utility, the
WCS
RI will be able to run in both the FUSE ESB
and
a servlet container such as
Apache Tomcat. Tomcat is an open source Java servlet container that may
be considered a b
aseline Java web application server.

4.8.3

Data Management

The WCSRI will use a flat file database as its primary persistence layer. Given that the FAA has a site
-
wide license, the Oracle Database Server may eventually be used within the FAA for storing other
i
nformation unrelated to raw coverage data, such as user/role configuration.
The Oracle Database Server
is also the primary persistence mechanism in use by the WFSRI, and as such the shared capabilities (such
as authorization) between the WCSRI and WFSRI c
ould be implemented
on a similar platform
.

4.8.4

Build and Deployment

The FUSE ESB is heavily based on the Spring Framework. The FUSE ESB also provides Maven tooling
to ease the building, packaging and deployment of components (e.g., web services, Spring configu
ration,
Camel routing) into the ESB. The
WCS
RI will be developed and will fit into FUSE through the Spring
Framework, and will use the Maven environment as a primary means for build and deployment. Spring
and Maven are both appropriate for use outside th
e FUSE ESB, and are suitable for a broad range of
operating environments.



5

PERSISTENT DATA MANAGEMENT


The
WCS
RI needs to deal
with
several different types of information that must be persisted over time.
These o
bjects include Users,
Coverage
s, Application

Data (
Coverage

Data Set
),
Lifecycle Policies,
S
ervice
Provider Metadata,

and

S
u
bscriber Information
.

5.1.1

Users/Roles

Each service provider can grant/revoke access privileges to and from users for their published
coverage
s.
This information is persisted as par
t of the User/Role database and includes information such as:



User Identifier



User Name



User Password



Capability Matrix [
Coverage
,

Operation]: The capability matrix associates a (
coverage
,

operation)
pair with a user, where operation defines the OGC
WCS

op
erations that a user can invoke on the
specified
coverage
.



Role: This identifies the role of the user as per a globally defined and managed set of roles. Roles
can be used to control QoS for user classes, if needed.

Note:

A design decision may be made to
implement the access control as an access control list associated
with
coverage
s, which would
eliminate

the need for a capability matrix.

5.1.2

Coverage
s

At the core of the
WCS
RI are the
coverage
s that are served up by the
WCS
. Each
coverage

carries with it
met
adata that describes the
coverage

and the operations that can be performed on it, as well as
information on its provider.

Service Providers will be expected to directly populate the backing store. On a per
-
coverage basis, the
WCSRI will be configured with

data scrubbing policies to ensure that an appropriate amount of data
is

retained.

The creation of a coverage entails the following steps:

1.

Service Provider
registers their coverage in the
Registry/Repository

2.

Service Provider configures the WCSRI with the

appropriate
coverage
information

3.

Service Provider customizes the WCSRI to include information on scrubbing policies, ontological
identifier(s), access control policies, monitoring detail, auditing policies, and similar information
that is specific for eac
h Service Provider

4.

The WCSRI monitors the backing store. When scrubbing is appropriate, the WCSRI removes
data from the backing store


The WCSRI will allow the Service Provider to configure the temporary file location, but the details of
their management
i
s likely to be

hidden from Service Providers.

5.1.2.1

Populating a Coverage’s Data Files

The default backing store will consist of a single directory for each coverage that contains data for that
coverage (the RUC model, for example). This directory will contain
configuration information for the
WCSRI behavior relating to that coverage (e.g., the WCSRI to WCSRI communication role or pattern),
metadata relating to that coverage, and several nested directories with data file names that include
information such as fo
recast generation time and valid time. The information encoded in directory and
file names will allow for the identification of each atomic coverage component. The Service Provider
will populate these directories with data files with the correct names, a
nd these will be discovered by the
WCSRI and made available to data consumers.

At some point an API may be provided to Service
Providers for adding data to the WCSRI.

This approach to a backing store is efficient for ad
-
hoc subsetting requests that do not

cross many data
files. It is anticipated that the vast majority of requests will require only one file to be opened,
which

will
allow fast lookups to satisfy requests. Time series requests are example
s

of request type
s

that
are

not
quickly satisfied wit
h this approach, but are almost certain to be less frequent than standard
,

filtered
-
coverage requests.

The implications of this approach are:



The Service Provider should not place files in the backing store with names recognized by the
WCSRI until the file
s are completely written. An example process to accommodate this would
be:

1.

Service Provider receives
a

legacy

data file,
which

is written to disk

outside of the WCSRI’s
root data directory

2.

Service Provider converts this file to
a form that is readable by
the WCSRI
. The
converted

file is placed in the appropriate coverage data directory, but with a name that is not
recognized by the WCSRI as being part of the backing store (datafile.nc.tmp)

3.

Service Provider renames datafile.nc.tmp to datafile.nc, which is
a valid file name that is
recognized by the WCSRI
. This ensures that data availability is an atomic operation, and the
WCSRI does not ever attempt to access a partially
-
written file

4.

WCSRI finds the new file and exposes it as part of the coverage



A single
data “atom” is contained in a single file. Data atoms are the smallest piece of data that is
intended to be exposed by the WCSRI as it is received from upstream sources
, which is highly
dependent on how the data is naturally
generated and/or
distributed b
y
these

sources
. An atom in
the case of forecast data might be a single forecast grid of a forecast run (a single forecast/valid
time combination).
The RUC forecast dataset includes many fields (air temperature, dewpoint,
and others)
which

are all releas
ed as a single forecast. An atom in this case would be a single file
that contains all of these fields, since they are being
distributed as a single
forecast.
An atom in
the case of radar data might be a single 360 degree scan volume. These atoms are co
verage
-

specific, and one source of radar data may require a different data atom than another.

If data
quality information is generated or distributed with a grid, it may also be exposed as part of an
atom.

Therefore, each Service Provider is responsible f
or populating the backing store with appropriately named
and located data files. Each coverage directory includes metadata, coverage
-
specific WCSRI
configuration, and data files, and the WCSRI will monitor and change behavior appropriately when
configurat
ion files are modified.

5.1.2.2

Temporary Files

The WCSRI will temporarily write to a directory on disk to enable several subsetting cases and other
operations. While every operating system defines a temporary directory, this directory may or may not be
suffici
ent or desirable to use as the WCSRI temporary directory because of data volumes, access
restrictions, or Service Provider policies. For example, the default OS temporary directory may be on a
lower speed disk, or may be on a disk partition that is of ins
ufficient size to support large gridded files.
Therefore, the central WCSRI configuration will include an option to specify a temporary directory.

5.1.2.3

Data Formats

The Java NetCDF API [
vi
] will be utilized to access files in the backing store. The NetCDF API
provides
an abstraction layer above that of a particular file format, as well as implemented support for GRIB,
NetCDF 3, NetCDF 4, and a number of other file formats.
Any data file that is readable through the Java
NetCDF API as a grid feature type may be

used as a data file in the backing store.

When responding to data requests, the WCSRI will return Climate and Forecast convention
-
compliant
NetCDF 4 files as the primary protocol/exchange format. The WCSRI will also have support for GRIB 2,
but
the
exact

extent of

this support
has

not yet been determined.

5.1.3

Application Data

For the
WCS
RI, the Application data represents the collection of
coverage

instances. Each application or
service provider can expose the desired
coverage
s by
adding a new directory to be

scanned

in the WCSRI
configuration
.

5.1.4

Lifecycle Data

The deletion of

old data
no longer useful to data consumers
is
an example of a lifecycle policy, and is
commonly referred to as scrubbing.
Scrubbing policies are configurable on a per
-
coverage basis, and

includes information such as:



Policy (Data Range, Action): This defines the range of the
coverage

instances as a function of
time and the action to be performed on the data. Every policy must have at least two stages, with
a current stage defining the dat
a that is to be considered
current

and a final stage which defines
the data that is to be considered stale or old on which some end of life action can be performed.
Policies may define any number of interim stages. Currently proposed actions include: NOP (
do
nothing), Archive (archive data), and Delete (remove data).




Execution

Frequency: The frequency with which the lifecycle checking and conformance should
be performed.


6

ACCESS CONTROL AND SECURITY

There are two major categories of security concern
:

those t
hat change with the security environment and
those that are uniform for every environment. An example of a concern that is uniform across all security
environments is data validation. This is logic that is very specific to a particular protocol and/or req
uest
type and must take place at least partially in protocol modules and the domain tier. Data validation is a
universal need, and this will not be configurable.

An example of a security concern that
does

need to adapt to a variety of security environment
s is
authorization and authentication
. Different Service Providers have greatly varying needs for securing
particular functions and
coverage
s, and it is therefore desirable that access restriction be highly
configurable. This includes restricted access to

particular
coverage
s
.

The
WCS
RI uses a layered authorization strategy, with coarse
-
grain authorization being handled by
implementing r
ole
-
based security mechanisms on top of standard FUSE role management technologies.
As central SWIM security mechanisms a
re put in place, they will be adopted. Until this time, standard
FUSE access mechanisms will be sufficient.

Fine
-
grain
ed

access to
coverage
s will be handled at the database level wherein user
privileges

will be
checked against the capability matrix stored

as part of the User Database and set up by the Service
Provider. Given that t
he 4
-
D Wx Data Cube functionality requires both secured and unsecured
communicatio
ns for different classes of use, default “anonymous” users will be set up.

6.1

ACCESS RESTRICTION

Ro
le
-
based access restriction will take place on several fronts:

Coverage



determines which roles have access to any data from a
coverage

Maintenance



determines which roles have remote access to maintenance information, which includes
monitoring informati
on and web application control mechanisms (such as those provided by FUSE to
start or stop services). This only includes remotely

accessible maintenance information and updates.
Those that are accessible or handled through configuration files are assumed

to be secured by standard
operating system user management mechanisms
.

Provider


determines which users have administrator
privileges

to allow them to
manage the
configuration of the WCSRI
.


7

GLOBAL SOFTWARE CONTROL

It is envisioned that all sub
-
systems of

the
WCS
RI
will
run within the same process, using a threaded
control flow paradigm to increase response time and
throughput

of the overall system.


8

BOUNDARY CONDITIONS

This section outlines the administrator use cases that need to be considered and planne
d for. These
include:



Startup of the
WCS
RI



Shutdown of the
WCS
RI



Exception Cases: for handling failures


APPENDIX A
-

ADDRESSED REQUIREMENTS


This section lists the requirements from [
iv
] that are

at least partia
lly

addressed by this version of the
document.

Requirement
Number

Requirement

Description

4.1.1

The WCSRI shall provide the capability for the Service Provider to specify the
available coverages or datasets that will be served by the WCSRI.

4.1.2

The WCS
RI shall provide the capability for the Service Provider to configure the
following for each dataset offered by the WCSRI:

1.

A unique identifier for the coverage.

2.

Name of the coverage.

3.

The directory for the coverage data files.

4.

Descriptive metadata for the
dataset.

4.1.3

The WCSRI shall provide the capability to serve virtually aggregated datasets.

4.1.4

The WCSRI shall provide configuration capability for required information
regarding the files that comprise an aggregated dataset, such as file storage
di
rectories, file names and geographic extents (in the case of tiling).

4.1.5


The WCSRI shall provide a lifecycle management system that will delete data
beyond a certain time period or archive data for a specified period of time.

4.2.1

The WCSRI shall al
low subsetting by zero or more fields (e.g., temperature,
reflectivity). If no fields are indicated, all available fields shall be returned.

4.2.1.1

The WCSRI shall support geometric subsetting of coverage volumes, including the
following:



3
-
D Volume. Fo
r this case, a 3
-
Dimensional bounding box may be specified
with dimensions defined by X, Y, Z value (such as
longitude/latitude/altitude) ranges. The result is a 3
-
Dimensional rectilinear
volume.



Horizontal 2
-
D Cross
-
section. If a constant altitude level i
s specified, a
horizontal (latitude/longitude) slice will be taken through the coverage
volume.



Vertical 2
-
D Cross
-
section. For this case, a vertical slice is taken through a

coverage volume. The path for the cross
-
section is defined by a set of XY
waypoin
ts and a sample density.



Sounding. A coverage volume may be subsetted by requesting a vertical
column of data whose location is specified by a latitude/longitude point.



1
-
D Point. A coverage volume may be subsetted by requesting a single point
specified b
y X, Y, and Z values.



Trajectory. A coverage volume may be subsetted along a trajectory, or a 3
-
D
path, by specifying the latitudes, longitudes, and altitudes of two or more
ordered waypoints. The returned coverage will be a “line” of data along a 3D
path,

where the geo
-
locations of the interpolated data points are determined
with either Euclidean or Great Circle geometry.



Corridor. A coverage volume may be subsetted by extruding a rectangular
area along a trajectory. A rectangular cross
-
section, orthogonal

to and along
a trajectory can be extracted from the coverage volume by specifying the two
or more ordered waypoints, the vertical range and the horizontal range. This
rectangular cross
-
section is a corridor. In addition, this is a superset of the
horizont
al and vertical cross
-
sections, and may be used for cross
-
section
subsetting.

4.2.3.1

The WCSRI shall be capable of providing a list of valid times from datasets that are
temporally aggregated.

4.2.3.2

The WCSRI shall provide the capability to constrain

coverages by requesting a
single valid time.

4.2.3.3

The WCSRI shall provide the capability to constrain coverages by requesting a range
of valid times.

4.3.2

The WCSRI shall expose CF
-
NetCDF4 as a file format at the interface level
.

5.1

The
WCSRI shal
l support the WCS

version 1.1

specification.

5.2

The WCSRI shall be designed in a manner such that future WCS specification
versions are supported through a pluggable protocol layer.

5.3.1

The WCSRI shall support Key
-
Value Pair (KVP) encoding over http G
ET for the
GetCapabilities operation.

5.3.2

The WCSRI shall support Plain Old XML (POX) encoding over http POST for the
GetCapabilities operation.

5.3.
3

The WCSRI shall support Simple Object Access Protocol (SOAP) encoding over
http POST for the GetCapab
ilities operation.

5.3.4

The WCSRI shall support the
Sections

and
AcceptFormats

parameters.


5.4.1.1

The WCSRI shall provide a means for the Service Provider to statically configure
metadata describing the server for the following
GetCapabilities

elements
:
ServiceIdentification
,
ServiceProvider
,
OperationsMetadata

and
Contents.

5.4.3.1

The WCSRI shall provide a means for the Service Provider to configure the path for
storing temporary files and scrubbing instructions (e.g., maximum file age) for stored
re
quest results and other temporary file purposes.

5.4.4.1

The WCSRI shall provide the capability to determine the following, on a per
-
coverage basis, either through programmatic means or Service Provider
configuration:



Unique Coverage Identifier



World Geod
etic System 1984 (WGS84) Bounding Box for the coverage



Supported coordinate reference systems (CRS), including geographic
projections, vertical and compound.



Native CRS


5.4.4.2

The WCSRI shall support “application/x
-
NetCDF” as the value for the
supported
Format

attribute. This is the
Multipurpose Internet Mail Extensions

(MIME)
type for CF
-
NetCDF4 binary coverage data.

5.5.2

For the
DescribeCoverage

operation, the WCSRI shall support SOAP encoding over
HTTP POST.

5.7.2

For the
GetCoverage

operation, the
WCSRI shall support SOAP encoding over HTTP
POST.

6.1

The WCSRI installation shall include, at a minimum, a description of the procedure
for adding and removing offered coverage types, and updating the metadata for those
coverage types.

6.1.1

The WCSRI s
hall provide a low
-
latency means to distribute filtered data (such as
geometric and temporal subsets) to interested data consumers. The mechanism used
here shall be consistent with what is used by other 4
-
D Wx Data Cube fundam
ental
components, such as the
WF
SRI, to distribute filtered data.


6.5.1

The WCSRI shall provide remotely accessible monitoring information relating to the
configuration status, WCSRI service status, health of upstream and downstream
WCSRI node interactions, dataset availability, data
storage size, data availability,
data scrubbing information, web service request counts, web service performance,
and error tracking/reporting. This capability shall be configurable to permit different
levels of monitoring detail and will leverage the SWIM

monitoring infrastructure.

6.6.1

The WCSRI shall provide auditing capabilities that allows an authorized client to
gather historical (i.e., 15 days minimum) information regarding system usage and
health. This information will include request/response det
ails. Note: These
capabilities may leverage those provided by the SWIM infrastructure.

6.7.1

The WCSRI shall provide the capability to configure the level of logging and
specifics as to what is logged. Note: This will be consistent with SWIM
infrastructur
e and other 4
-
D

Wx

Data

Cube components.

6.8.1

The WCSRI shall perform stringent data validation for communications, including
but not limited to requests for data and messages/notifications sent from upstream
nodes.

6.8.3

The WCSRI shall provide configu
rable role
-
based access to WCSRI web service
endpoints. Implementation of this requirement will leverage SWIM infrastructure
security solutions.

6.8.4

The WCSRI shall provide configurable role
-
based access to WCSRI
-
offered
coverages and their data fields.

Implementation of this requirement will leverage
SWIM infrastructure security solutions.

7.1

The WCSRI shall implement infrastructure capabilities in a manner compliant with
the SWIM service container, and shall leverage SWIM mechanisms wherever
appropri
ate.

7.2

The WCSRI installation shall include WSDL definitions of the web services that it
provides.

7.3

The WCSRI shall support the deployment and installation of the WCS operations
(e.g., GetCapabilities, DescribeFeatureType, GetFeature into the SWIM s
ervice
container, either through installation scripts or documentation for Service Providers.

8.
2

The WCSRI shall run on Linux and Windows platforms.



APPENDIX B
-

ACRONYMS


API


application programming interface

CITE


Compliance & Interoperability Testi
ng & Evaluation

CONOPS

Concept of Operations

CF


Climate and Forecast

CRS


coordinate reference system

DOD


Department of Defense

ESB


Enterprise Service Bus

FAA


Federal Aviation Administration

FTI


Federal Telecommunications Infrastructure

FUSE


An open
sour
ce Enterprise Service Bus (ESB).

The SWIM Container.

GSD


Global Systems Division

HTTP


Hypertext Transfer Protocol

IOC


Initial Operating Capability

ISO


International Organization for Standardization

IT


information technology

JMBL


Joint METOC Brok
er Language

JPDO


Joint Program Development Office

METOC

Meteorological and Oceanographic Data

MIME


Multipurpose Internet Mail Extensions

MIT


Massachusetts Institute of Technology

NAS


National Airspace System

NextGen

Next Generation Air Transportation S
ystem

NCAR


National Center for Atmospheric Research

NNEW


NextGen Network Enabled Weather

NOAA


National Oceanic and Atmospheric Administration

OGC


Open GeoSpatial Consortium

OPeNDAP

Network Data Access Protocol

POX


plain
-
old
-
XML

RAL


Research Applicati
ons Laboratory

RUC


Rapid Update Cycle

SAS


Single Authoritative Source

SOAP


The technology formerly known as Simple Object Access Protocol

SWIM


System Wide Information Management

TAF


Terminal Aerodrome Forecasts

UOM


Unit(s) of measure

URI


Uniform Res
ource Identifiers

URL


Uniform Resource Locators

URN


Uniform Resource Names

WCS


Web Coverage Service

WCPS


Web Coverage Processing Service


WCSRI


Web Coverage Service Reference Implementation

WGS84


World Geodetic System 1984

XML


Extensible Markup Langu
age


APPENDIX C
-

DEFINITIONS AND TERMS


4
-
Dimentional Weather Data Cube



A
virtual
entity comprised of a federation of distributed weather
data sources, which provides a common source of weather information to all users of the NAS.

Coverage



A conceptual

dataset representing a single gridded data product, which may be composed of
several different data files and fields, and may pertain to a general time range. A dataset.

Dataset



May be composed of several files or a time series of data, that comprise a
single logical
product. Examples include RUC, Icing/CIP, GOES satellite imagery, etc. This is equivalent to the ISO
19115 “Dataset series” terminology.

Initial Operating Capability



The 4
-
D

Wx

Data

Cube Initial Operating Capability. This includes an
init
ial (not CONUS
-
wide) operational demonstration of capability. Subsequently a full CONUS
-
wide
deployment will take place.

Offered Coverage



A high level dataset/coverage offered by a particular WCSRI installation.

Open Geospatial Consortium



A standards o
rganization that is leading the development of standards
for geospatial and location based services.

Registry/Repository



The registry/repository reference implementation for the 4
-
D

Wx

Data

Cube.

Service Provider



An entity (a person or organization) th
at hosts an installation of the WCSRI, and is
the original provider of zero or more datasets.

United States National Airspace System



The national aviation system of the United States of
America. It is a large operational system, and as such includes st
rict operational requirements relating to
security, quality of service, and related critical infrastructure.

Web Feature Service



A service standard developed by the Open Geospatial Consortium that
encompasses a standard protocol for accessing feature
-
bas
ed (typically non
-
gridded) data.


APPENDIX D
-

REFERENCES





[
i
] JPDO, “
Concept of Operations for the Next Generation Air Transportation System

, Version 2.0, June
2007.

[
ii
] JPDO, “
Four
-
Dimensional Weather Functional Requirements for NextGen Air Traffic Management
”,
Version 0.1, January 2008.

[
iii
] NNEW, “
NextGen Network
-
Enabled Weather


IT CONOPS
”, Version 3.2, August 20, 2008.

[
iv
] NNEW,

4
-
Dimens
ional Weather Data Cube Web
Coverage

Service Reference Implementation
Requirements”
, Version 1.2
, February 27, 2009.

[
v
] Fowler, Martin et al.
Patterns of Enterprise Application Architecture
, Addison
-
Wesley, 2002

[
vi
] The NetCDF Java Library Home.
http://www.unidata.ucar.edu/software/netcdf
-
java/