or some relevant web services

insidiousbehaviorSecurity

Nov 3, 2013 (4 years and 1 month ago)

75 views

What is a Web Service


(for) Developing Applications for

Real
-
time Environmental Data



Bryan Lawrence

Head, NCAS British Atmospheric Data Centre


P.I. The NERC DataGrid.


or some relevant web services


British Atmospheric Data Centre

http://badc.nerc.ac.uk

Outline


Question: What is a Web Service?

Answer: Acronym Soup!


Fundamentals (XML+)


SOAP and REST


WSDL


OGC Web Services


WFS, WCS, WFS


UNIDATA


LDM


NDG Web Services




British Atmospheric Data Centre

http://badc.nerc.ac.uk

What is a web service?

Googling gives:



Answers.Com: Web
-
based applications that dynamically interact
with other Web applications using open standards that include XML,
UDDI and SOAP.


(Yuck: UDDI?)


emergingtech.ittoolbox.com:


A Service
-
Oriented Architecture (SOA) is a collection of services
or software agents that communicate freely with each other.


Sub
-
topic definition: Web Services protocols and standards are
the technology that promote the sharing and distribution of
information and business data. A protocol is a standard method
for transmitting data through a network. There are many
different specialized protocols to accommodate the many kinds
of data that might be transmitted.


British Atmospheric Data Centre

http://badc.nerc.ac.uk

SOAP

Simple Object Access Protocol

Horrible looking XML specification, but
lots of SOAP pacakges available which
are (generally easy to use), eg:

from SOAPpy import SOAPproxy

url=http://services.xmethods.net:80/soap/servlet/rpcroute
r’

n=‘urn:xmethods
-
Temperature’

server=SOAPProxy(url,namespace=n)

temperature=server.getTemp(‘27502’)

Returns the temperature at U.S. zipcode 27502 (from
Dive Into Python, Mark Pilgrim)

(
One knew a priori that a method named getTEmp would return a float given an argument of a zip
code, provided one used the right namespace


How?)


British Atmospheric Data Centre

http://badc.nerc.ac.uk

WSDL
:Web Services Description Language

>>>from SOAPpy import WSDL

>>>Server=WDSL.Proxy(‘path/to/WSDL’)

>>>server.method.keys()

[u’doGoogleSearch’,u’doGetCachedPage’ …]

>>>callInfo=server.methods[‘doGoogleSearch’]

>>>for arg in callInfo.inparams:

. . . print arg.name, arg.type


key (u’http://www.w3.org/2001/XMLSchema’,u’string’)

. . .

Define expected messages for a service, and
their (input or output parameters).

An interface will group together a number of
messages (operations)

Bind an Interface via a definition to
a specific transport (e.g. HTTP) and
messaging (e.g. SOAP) protocol

The network location where the service is
implemented , e.g. http://localhost:8080


British Atmospheric Data Centre

http://badc.nerc.ac.uk

REST’ful Web Services


REST (representational state transfer) is an
approach for getting information content
from a Web site by reading a designated
(ok, you need a URI) Web page that
contains an XML (Extensible Markup
Language) file that describes and includes
the desired content.


That’s it! You use HTTP (get,put,…) to a
URI, with XML as the payload …


British Atmospheric Data Centre

http://badc.nerc.ac.uk

Web Services the confusion:


(Tim Ewald): “Today, I see three camps:


1) It's all about SOAP messages …,

2) It's all about WSDL,

3) It's all about XML/HTTP.


Microsoft is definitely in the first camp, focusing on doing things
with SOAP whether or not they are ultimately represented
as XML or a binary data rep.

IBM and Iona are in the second camp, focusing on describing in
WSDL, whether or not they are ultimately represented as
SOAP or a binary protocol.

Lots of developers and some big companies like Amazon are in
the XML/HTTP camp, focusing on getting data from one
place to another in a way they can consume it.


[Tim Ewald: http://pluralsight.com/blogs/tewald/archive/2004/10/18/2875.aspx]


British Atmospheric Data Centre

http://badc.nerc.ac.uk


Dimensions of interoperability for
earth science data

Standards

access

ISO / OGC


British Atmospheric Data Centre

http://badc.nerc.ac.uk

OGC web services


Data access services, e.g.:


Live Access Server, NDG DataExtractor


GADS


DODS/OPeNDAP


OGC

web services


OGC web services


Web Map Service


Web Feature Service


Web Coverage Service

[
Open Geospatial Consortium (OGC): International consortium of nearly 300
companies, government agencies and universities participating in a
consensus process to develop publicly available geoprocessing
specifications]


British Atmospheric Data Centre

http://badc.nerc.ac.uk

OGC web services


Web Map Service (WMS)


Three operations:


GetCapabilities

(required)


GetMap

(required)


GetFeatureInfo

(optional)


Typically support HTTP GET binding, SOAP
being considered


GetCapabilities

request


<Service>

metadata for service as a whole (ISO 19115 compliant)


<Capability>

metadata describes request bindings, exceptions, and:


<Layer>
s: title, name, SRS, bounding box, keywords, style; nested
layers inherit from parent

eg, GLOBE program WMS server


GetCapabilities
:

http://viz.globe.gov/viz
-
bin/wmt.cgi?REQUEST=GetCapabilities&SERV
ICE=WMS


British Atmospheric Data Centre

http://badc.nerc.ac.uk


e.g.: ERA40 re
-
analysis surface air temperature, 2001
-
04
-
27


deegree open
-
source WMS modified with netCDF connector

Overlaid with rainfall from

globe.digitalearth.gov WMS server

OGC web services


British Atmospheric Data Centre

http://badc.nerc.ac.uk

Web Coverage Service (WCS)


cf DODS/OPeNDAP/GADS


A geographic
coverage

is raw (typically gridded) data


WCS similar to WMS, but provides
data
, not images

(cf DODS/OPeNDAP, GADS)

Operations:


GetCapabilities


DescribeCoverage


GetCoverage

Coverages supplied in
well
-
known
-
binary
format:


GeoTIFF, HDF
-
EOS, DTED, NITF, GML,
netCDF???

OGC web services


British Atmospheric Data Centre

http://badc.nerc.ac.uk

Web Feature Service (WFS)


Provides access to geographic
feature instances


Features are defined by
application schema

compliant with the Geography
Markup Language (GML)


Operations:


DescribeFeatureType

(required): returns XML schema for the feature


GetFeature

(required): allows retrieval of features in XML compliant to the feature
schema. Selected features may be constrained through Xpath expressions


Transaction

and
LockFeature

(optional): for inserting, updating and deleting
feature instances


GetCapabilities
: describes capabilities of WFS (as for WMS)


Typically used for vector data (eg cruise tracks), but can be any application
schema

OGC web services


British Atmospheric Data Centre

http://badc.nerc.ac.uk


FOSS implementations


server:


deegree


geoserver


mapserver


client:


quickWMS


www.wmsviewer.com



List maintained on NDG wiki at
http://ndg.nerc.ac.uk/kwiki/ndg/index.cgi?
OtherProjects

OGC web services


British Atmospheric Data Centre

http://badc.nerc.ac.uk

OGC web services

Summary


Web Map Service (WMS, ISO 19128)


GetCapabilities


GetMap


Web Feature Service (WFS)


GetCapabilities


DescribeFeatureType


GetFeature


Transaction, LockFeature


Web Coverage Service (WCS)


GetCapabilities


DescribeCoverage


GetCoverage


Catalog Service for the Web (CSW)


very new!


REST, not SOAP (currently)



British Atmospheric Data Centre

http://badc.nerc.ac.uk

Climate Science Modelling Language
(CSML, a GML application schema)

http://ndg.nerc.ac.uk/csml


Defines seven features of interest!

NDG: CSML

CSML feature type

Description

Examples

TrajectoryFeature

Discrete path in time and space of a
platform or instrument.

ship’s cruise track, aircraft’s flight path

PointFeature

Single point measurement

raingauge measurement

ProfileFeature

Single ‘profile’ of some parameter along a
directed line in space.

wind sounding, XBT, CTD, radiosonde

GridFeature

Single time
-
snapshot of a gridded field

gridded analysis field

PointSeriesFeature

Series of single datum measurements

tidegauge, rainfall timeseries

ProfileSeriesFeature

Series of profile
-
type measurements

vertical or scanning radar, shipborne ADCP,
thermistor chain timeseries

GridSeriesFeature

Timeseries of gridded parameter fields.

numerical weather prediction model, ocean
general circulation model


British Atmospheric Data Centre

http://badc.nerc.ac.uk

ProfileFeature

GridFeature

ProfileSeriesFeature

CSML


British Atmospheric Data Centre

http://badc.nerc.ac.uk


Discovery Service (here now)

Data Extractor (next week)

Vocab Service (next month)

Attribute Authority (next couple of months)

Data Provider (next couple of months)

DeliveryBroker (later this year)

Browse Service (a long way off)

All Open Source, documented, clear APIs

NERC DataGrid; web services coming soon


British Atmospheric Data Centre

http://badc.nerc.ac.uk

NERC DataGrid: Discovery Service


British Atmospheric Data Centre

http://badc.nerc.ac.uk

NDG: dataExtractor and GeoSplat


British Atmospheric Data Centre

http://badc.nerc.ac.uk

UNIDATA

Unidata (
www.unidata.ucar.edu
) provide data feeds
using their Local Data Manager (LDM) technology
to push data around the U.S. (and further afield) in
the Internet Data Distribution system (IDD).


Users to specify in advance which data should be
delivered to their local systems. The IDD then
delivers the data as soon as they are available.


The IDD is a data subscription service,
implemented in such a way that delivery (and
often processing) are triggered by external events.


WMO considering the use of LDM for some data
movement.


British Atmospheric Data Centre

http://badc.nerc.ac.uk

Unidata: LDM

LDM Delivery


Scalable


Hierarchical fan out


Relying on relay sites with
appropriate resources


Flexible


New data products can be
introduced from any node in
the system


British Atmospheric Data Centre

http://badc.nerc.ac.uk

UNIDATA IDS Topology


British Atmospheric Data Centre

http://badc.nerc.ac.uk

Summary


1)
Web services not just about SOAP and WSDL …

2)
We
should
use

1)
web service standards,

2)
WS mechanisms optimised for environmental
applications

3)
Currently need to extend GML (CSML)

4)
Options for data push if UK community want it
based on LDM (or similar) technologies …

5)
NDG can (and is) building a range of
environmental web services