Web Services at IRIS DMC

insidiousbehaviorSecurity

Nov 3, 2013 (3 years and 7 months ago)

74 views

Web Services at IRIS DMC

Joanna Muench

Linus Kamb

Outline


Web service development process at
DMC


Demos


SOSA seismogram processing


UPDS product handling


DHI database access

Developing a web service


Interface creation with a wsdl


Code generation


Building the backend


Client side development


Deployment


Testing and logging


Reports from the field

Creating an interface


WSDL (web service definition
language) is public interface to
service


Best practice
-

define wsdl first


Avoids language specific
constructs


Guarantees* interface intent


But
-

complicated, easy to
make mistakes


<?xml version="1.0" encoding="UTF
-
8"?>

<wsdl:definitions name="dhi_webservice"


targetNamespace="http://dhiWS.dmc.iris.edu/wsdl"


xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/"


xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"


xmlns:tns="http://dhiWS.dmc.iris.edu/wsdl"


xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"


xmlns:xsd="http://www.w3.org/2001/XMLSchema"


xmlns:xsd1="http://dhiWS.dmc.iris.edu/types">


<wsdl:documentation xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/">Created
using Cape Clear Studio SOA Editor>


<wsdl:types>


<xsd:schema


targetNamespace="http://dhiWS.dmc.iris.edu/types"


xmlns:SOAP
-
ENC="http://schemas.xmlsoap.org/soap/encoding/"


xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"


xmlns:xsd="http://www.w3.org/2001/XMLSchema"


xmlns:xsd1="http://dhiWS.dmc.iris.edu/types">


<xsd:import namespace="http://schemas.xmlsoap.org/wsdl/"/>


<xsd:import namespace="http://schemas.xmlsoap.org/soap/encoding/"/>


<xsd:element name="TimeSpan">


<xsd:annotation>


<xsd:documentation>Defines a time span with start and end
times.</xsd:documentation>


</xsd:annotation>


<xsd:complexType>


<xsd:sequence>


<xsd:element maxOccurs="1" minOccurs="1" name="startTime"
type="xsd:dateTime"/>


<xsd:element maxOccurs="1" minOccurs="1" name="endTime"
type="xsd:dateTime"/>


</xsd:sequence>


</xsd:complexType>


</xsd:element>


Tools for WSDL creation


Cape Clear SOA Editor


Graphic interface


WSDL validator


Great manual (and free!)


Other editors:


Altova XMLSpy


Sonic Software


or, XML Editor + WSDL
validator

Code generation


Tools that translate wsdl into a web service
engine in your favorite language


Provide stubs and bindings, making server
and/or client generation easier


Available in many languages


Java: Axis, GLUE, JibxSOAP


Perl: SOAPLite


C#: .NET


C++: gSOAP, Axis

Building the backend


Implement desired functionality


Various architectures possible


Maintain flexibility through interfaces


Test


Fill in stub methods in generated code,
calling your implementation


You’re ready to deploy!


Building the client


Deployment is more
interesting with a client


Client skeleton can be
generated from wsdl


Could be command line
driven, web application
or full application


Common practice to
have a client internal to
server project, for testing
new ideas

Server Deployment


Most web service development environments
have deployment tools


Automation is essential
-

useful tools like ant


Three levels of deployment

Environment

Function

Updated

Local

Development

Every 5 min

Test

Load test, etc.

Daily (or less)

Production

Public service

New release only

Testing and logging


Testing at all levels


Unit tests in dev


Functional tests


Performance tests


Automate and run
frequently



Logging


Debugging


Performance
monitoring


Usage monitoring

Reports from the Field


Can be easy but…


Setting up deployment, classpaths, etc
can be tricky


Tools aren’t foolproof (at least not the free
ones)


Attachment specification still vague

IRIS DMC web services


SeismoProcess
-

component of SOSA
(Synthetic and Observed Seismogram
Analysis)


UPDS (Uniform Product Distribution
Service) revolves around web service


DHI (Data Handling Interface) web
service

SeismoProcess Service


Provides additional processing capabilities for
SOSA, a Java GUI application


Part of SCEC CME


Motivation:


Update and expand processing capabilities without
re
-
installing application


Prototype for possible more general seismogram
processing service


Released as part of SOSA 1.0, but not yet
running on production server

SeismoProcess Service

SeismoProcess Service


Implementation details


Attachment style SOAP with Attachments
(SwA)


Seismograms sent as SAC files


Issues


Attachment configuration


Performance

UPDS
-

Motivation


Uniform Product Distribution System


Submit, archive, query, and access


Handles XML “Data Products”


Many different types of products


Unknown future products


Extensible products, product list

UPDS
-

Design


Submission service


Query/Access service


Unstructured XML


Application and Web interfaces


Flexible interfaces

UPDS
-

Overview

DATA

PRODUCT

PRODUCER

PRODUCT

ARCHIVE

SUBMISSION

WEB SERVICE

Product

Handler

Module

Product

Handler

Module

Product

Handler

Module

Product

Handler

Modules

<XML>

<PRODUCT>

</XML>

SUBMIT CLIENT

Web

Access

QUERY / ACCESS

WEB SERVICES

CLIENT

PROGRAM

UPDS
-

Overview

DATA

PRODUCT

PRODUCER

PRODUCT

ARCHIVE

SUBMISSION

WEB SERVICE

Product

Handler

Module

Product

Handler

Module

Product

Handler

Module

Product

Handler

Modules

<XML>

<PRODUCT>

</XML>

Web

Access

QUERY / ACCESS

WEB SERVICES

CLIENT

PROGRAM

SERVLET

Q/A WEB

SERVICES

Q/A LIB

Q/A LIB

UPDS
-

Issues


“Pure” XML vs binary products


Large products


“Pure” WS vs multi
-
protocol approach


Large result sets


Attachment handling


Performance

DHI web service


Web service to CORBA
-
based DHI


SCEC CME in conjunction with USArray


Provides subset of full DHI capabilities


Motivation:


Language agnostic service


Available over port 80


Provide most commonly accessed DHI functions,
simplified data types


Not yet beta release

DHI web service

DHI web service


Implementation details


Attachment style


Seismograms sent as mSeed, SAC or link to
URL for download


Issues


Attachment configuration problems in Axis


Performance of underlying CORBA service

Recap


Can be easy but… hard to do right


Setting up deployment, etc can be tricky


Tools aren’t foolproof (at least not the free
ones)


Attachment specifications still vague