Cooperative Agreement # NNS04AA78A
1
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
SENSOR TO USER
Progress Report
-
2
nd
Year (Q2
-
Q4)
Prepared by
Dr. Erick Malaret, ACT Corp. (703
-
742
-
094)
Email: malaret@actgate.com
Cooperative Agreement # NNS04AA78A
2
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Description and Objectives
Applied Coherent Technology Corporation (
ACT
), the Naval Research
Laboratory (
NRLSSC
) Oceanographic Division, and our
NOAA
partners,
are under a cooperative agreement with NASA to maximize the value of
NASA/EOS Earth science observations to decision makers related to
coastal, marine, and ocean environments.
ACT and NRLSSC are integrating real
-
time ocean systematic
measurements from NASA and NOAA satellites and available coastal
observations along with coastal ocean model output into an automated real
-
time database of ocean weather in the Gulf of Mexico.
This project focuses on monitoring coastal ocean properties and it is aimed
at providing new real
-
time support to NOAA efforts initially in the following
focus areas: Harmful Algal Blooms (HABs), Coastal Flooding /freshwater
discharge, and Hypoxia.
ACT in collaboration with NOAA will expand the project to other U.S. coastal
areas and add additional NASA,NOAA,DOD and foreign data sources.
Approach
The on
-
going effort is divided into an application component and a
and product research/development component.
•
The
Application Component
[
by ACT
]
–
provides an
architecture that addresses issues related to data ingestion,
processing, distribution, access, and decision support.
The key
Decision Support Server and Fusion Server will be installed at
NRL/SSC, followed by other servers at different NOAA sites.
•
The
Product Research/Development component
[by
NRLSSC]
–
provides state of the art oceanographic expertise and
research addressing issues related to generation of new
geophysical data products as part of a production pipe
-
line.
Schedule and Key Deliverables
Year 1:
Application Component
–
a) Req. Gathering, b) Install high end
HW/SW to ingest/process/serve data, c) enhance current Fusion
server & Decision Support Tool Technology.
Product R&D Component
-
….
Year 2:
Application Component
–
a)
Demonstrate widely available internet
access to all data levels, b) develop initial Hazardous Algal Bloom
product based on satellite/model
data in GIS and other compatible
formats, c) add new data streams (e.g. NPP VIIRS)
Product R&D Component
-
….
Year 3
-
5:
Application Component
–
a) Expand to other application areas,
geographic areas, and data sources, b) identify and customize
architecture to provide additional decision support tools, assist in
transition of application to operation.
Product R&D Component
-
….
Initial Implementation
2
nd
year Started May 2005
Note: Big Scope of Cooperative
Agreement Scope and participants
remains unchanged
Cooperative Agreement # NNS04AA78A
3
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
De
The next slides will concentrate on
Application Component progress in the
last 7 months
Cooperative Agreement # NNS04AA78A
4
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
WIPE
(server)
REACT
(CSC User)
Fusion Server at
NAVY/NRL
-
SSC
REACT
(NOS User)
REACT
(Field Analyst)
WIPE
(
servers
)
REACT
NOOA Fusion Servers at ACT
Web Browsers
(HTML and Applet Access)
SOAP
WSDL
HTTP/
CGI
Local
+Satellite Imagery
+MODELS
+ …
…
HAB Test
-
bed for NOAA: Present Network Configuration
NOAA/CW
•
MODIS data
NOAA/NDBC
•
Buoy data
FNMOC
•
Many models
OTHER
+in
-
situ obs.
+satellites Obs.
NOAA/CW
NOAA/NDBC
NOAA/NCEP
optional access
directly by REACT
NRL
-
SSC
•
MODIS OBS
•
MODELS
•
MODIS VAPs
FTP
&
HTML xfers
Final HAB Bulletin For Decision
Maker (user Type 3)
HAB bulleting creation Wizard
NOAA/NOS/CSC Analyst Team (user Type 2)
Data providers
(user Type 1)
Cooperative Agreement # NNS04AA78A
5
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Apply WIPE/REACT
To HAB’s needs by
NOAA
Cooperative Agreement # NNS04AA78A
6
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
DATA INGESTION PROGRESS
Cooperative Agreement # NNS04AA78A
7
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Cooperative Agreement # NNS04AA78A
8
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
•
Ingestion Data Aspects and Engines (continue)
–
NRL Model Data
•
Model data ingestion engines were enhanced in REACT to rapidly open any of the NRL
volumetric NetCDF files. The files can be open via the REACT interface.
•
Visualization tools were added to REACT to simplify the inspection of volumetric model
data.
–
Layer animation
–
File animation
–
Data inspection with row/col/depth proves
–
Interactive Intensity Mapping
–
NRL Level 2
-
4 HDF Products
•
Integrated the ability to read any of the sub
-
datasets included in the NRL HDF files.
•
The REACT interface provides a hierarchical file layout over the file system and the
HDF file
•
User can load desired set of sub
-
dataset layers simultaneously, and remap into desired
cartographic projection
•
Visualization capabilities similar to the model data were added.
Done: multiplicative and additive scaling constants are applied.
–
NOAA Coast Watch
•
After Meeting with Coast Watch it was clear that ALL NOAA/CoastWatch products are
being saved in CW HDF format.
•
Added capabilities to read/load all the sample data provided by CW to ACT
–
NOS Needed Data
•
Incorporated the Buoys data and filtering algorithms
•
In
-
situ observations can not be accessed automatically at this time
DATA INGESTION PROGRESS (continue)
Cooperative Agreement # NNS04AA78A
9
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Data Ingestion and Status of WIPE Server(s)
NRL/SSC
•
The WIPE server at NRLSSC has been working since October 05.
•
NRL/SSC has given security approval to the machine, hence it can be access over
the internet as a web server.
–
The web page is:
http://cesium.nrlssc.navy.mil
•
Met with NRL/SSC at ACT, and identified automatic process to populate the WIPE
server at NRL/SSC with both MODIS L2
-
L4 products and with MODEL data.
–
The advantage of the approach taken is that it does not replicate in any way Hark Disk
storage at NRL/SSC. It uses the same hard disk area that their DODS server is using.
•
Training on WIPE was done in November
–
ACT is presently incorporating the suggestion received from NRL
Basemap MODIS COMPOSITE NCOM TEMP
Cooperative Agreement # NNS04AA78A
10
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Cooperative Agreement # NNS04AA78A
11
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Cooperative Agreement # NNS04AA78A
12
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Data Ingestion and Status of WIPE Server(s) (continue)
NOAA CW Servers at ACT:
•
Working. Tested with Sample CW
-
HDF format data. All modes
working, except the Swath data sets.
•
Bow
-
tie projection problems in the MODIS L1B data.
–
To save computational time we were not using all the ground control
points in the MODIS data. This resulted in visible bow
-
tie projection
problems that were not being handled correctly.
–
Investigated used of third party SW
decided against it after
discussing with NRL/SSC that all that is needed is to use ALL the GCP
information.
–
Made significant modifications to the processing of Ground Control
Point to generate a properly corrected geo
-
referenced image.
–
The new approach is more computationally intensive, hence it takes
longer to generate a full area mosaic at full resolution.
5
-
10 minutes.
Cooperative Agreement # NNS04AA78A
13
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Status of HAB WIPE Server at ACT
•
Presently working on getting all the data automatically pushed into the WIPE server
at ACT.
•
Some of the cell count data is at FTP sites. Presently embedding at FTP library into
WIPE to simplify process of getting data from remote servers via FTP.
Data Ingestion and Status of WIPE Server(s) (continue)
Cooperative Agreement # NNS04AA78A
14
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
SOFTWARE DEVELOPMENT PROGRESS
Cooperative Agreement # NNS04AA78A
15
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
•
MSHELL
-
Image
Processing Interpreter
Language developed by
ACT. The language has a
high level syntax similar to
Matlab and IDL.
•
ProVIEW
-
Professional
Windows Environment for
Windows (ProVIEW).
ProVIEW has extensive
capabilities in image
processing, analysis, and
map generation
.
•
WIPE
–
WWW Information
Processing Environment.
Data Assimilation,
Processing, and distribution
system with batch and
interactive capabilities.
MSHELL Provides the Computational Engine Used by ProVIEW & WIPE
ProVIEW Provides Many of the Graphical Capabilities Used Under WIPE
WIPE’s building blocks have been used in multiple mission critical operations
Rapid Environmental Assessment Composition Tools (REACT)
has been implemented and continues under rapid development.
It captures elements of MSHELL+ProVIEW+WIPE
all under an easy to deploy stand alone application, and
portable GUI environment.
During 2
nd
-
3
rd
quarter continue expanding the tool space in REACT in
response to needs of the HAB needs
CORE SW Development Progress
Cooperative Agreement # NNS04AA78A
16
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
•
As part of our NASA Cooperative Agreement, ACT has been participating in the
Technology Infusion working group. The Technology Infusion has identified many
capability blocks required in systems to support the EOS needs.
•
REACT inherits key components of the ACT’s SW remote sensing SW tools
–
MSHELL
-
interpreter language with extensive capabilities for multidimensional data
manipulation
–
ProVIEW
-
graphical user interface around MSHELL
–
WIPE
-
provides fusion center architecture, with over 200 streams
•
ACT has been using REACT and WIPE as a platform/architecture to demonstrate
technology infusion from the OpenSource community, i.e.
–
Qt
–
provides a portable library in C++ that allows GUI and system level support (
A prototype
version of REACT has been implemented for LINUX and Windows.)
–
GDAL
–
the Geographic Data Abstraction Library is used by REACT to perform raster IO over
many geo
-
graphic data formats
new formats can be easily added following an OpenSource approach empowering developers
to rapidly contribute in the adaptation of new formats. GDAL has been extended to
support : CW
-
HDF, NRL
-
HDF, PCI
-
AUX (extended SRS support)
–
gSOAP
–
used to SOAP enable REACT
the
developer needs to primarily concentrate on the
core subroutine functionality.
CONTINUED EXPANDING THE SOAP INTERFACE
–
PROJ4
–
a cartographic library with support to most of the projections used by USGS
–
SQLITE
–
a built
-
in SQL engine
–
OpenGIS/OGC
–
describes protocols/methods used for the request of products (This is work in
progress)
CORE SW Development Progress (continue)
GDAL + PROJ4 +
GSOAP+…
MSHELL
(Interpreter+ scripts!)
REACT
Cooperative Agreement # NNS04AA78A
17
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
•
REACT is been configured to
provide Web Services
for geo
-
spatial data
–
Web Services Requirements are:
•
Available over the Internet
•
Not tied to any operating system or programming language
•
Self
-
describing via a common XML grammar
•
Discoverable via a simple find mechanism
•
REACT captures a WIPE like type of interface for accessing inventory queries and
generating mosaics, a set of five core functions are being added to the
REACT/MSHELL scripting language resulting in a
LARGE space of remote
computing capabilities/options
! (implemented using SOAP)
–
$result
= rmshell_execute($server, $scriptCommandList )
–
Data
= rmshell_getVar( $server, $variableName)
–
$result
= rmshell_putVar( $server, $variableName)
–
$result
= rmshell_getFile( $server, $inFile, $outFile )
–
$result
= rmshell_putFile( $server, $inFile, $outFile)
ADDITIONAL FUNCTIONS WERE ADDED TO HANDLE ASYNCHRONOUS REQUESTS
•
The above functionality can be exported to other applications via WSDL (an XML
grammar for specifying public interface to the web service)
GAVE A PRESENTATION TO THE ESIP FEDERATION MEETING IN SAN DIEGO, ON
WEB SERVICES USING WIPE&REACT.
CORE SW Development Progress (continue)
Cooperative Agreement # NNS04AA78A
18
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
REACT
REACT
REACT
WIPE
Script/algorithm or data
result or data
Script/algorithm or data
result or data
WIPE and REACT share the same processing engine
REACT is configured to handle local user and a small number of remote applications.
REACT can support remote applications as a WIPE
-
LITE system
Client side
(Field System)
Server side
(Field or Fusion Center System)
Web Enabled
Application
WIPE
Or
REACT
Script/algorithm or data
result or data
CORE SW Development Progress (continue)
Cooperative Agreement # NNS04AA78A
19
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
•
REACT can be seen as a WIPE
-
LITE system with more flexible visualization
capabilities, and network centric capabilities.
•
A
prototype windows version
of REACT can be downloaded from ACT, i.e.
http://www.actgate.com/reason
CORE SW Development Progress (continue)
Cooperative Agreement # NNS04AA78A
20
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
DECISION SUPPORT TOOLS PROGRESS
Cooperative Agreement # NNS04AA78A
21
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Decision Support Tools means different things to different users, e.g.
–
Data Provider
need coverage and cartographic
maps/mosaics with simple to use query engines
–
Analyst at Fusion Center
•
Visualization Tools, e.g animations of layers, volumetric vis.
•
Implementation of a Process in the form of a Wizard
•
Analysis: Multivariate Detection/Estimation/Prediction/…
–
High Level Decision Maker, e.g. Coastal Manager
GO/NoGo results with access to supporting data
•
ACT has been working during the last year in the above end
-
user
categories with requirements driven by the needs of:
–
NRL,
–
NOAA/CW
–
NOAA/NOS
–
NOAA/CSC
–
NOAA’s end
-
users
DECISION SUPPORT TOOLS …. The Challenge
Cooperative Agreement # NNS04AA78A
22
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
•
NOAA/NOS, NOAA/CSC, NASA/SSC, NOAA/NCDDC, and ACT met in February
-
05
to discuss the HAB Bulleting creation process and future architecture developments.
•
As the result of the meeting a number of concrete suggestions were provided by NOS
and CSC. (see following slide).
DECISION SUPPORT TOOLS PROGRESS (continue)
Cooperative Agreement # NNS04AA78A
23
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Development Operations Delivery
•
[done]Use WIPE for
new data stream
ingestion
•
[done]Easy Incorporation of
other geographies
•
[done]Mosaic Images
•
[done]Alternate look from
same data => flicker
•
[done]View data values
from multi
-
sensors at same
time
•
Skill assessment
match in
-
situ data and
imagery
•
[done] Output data
•
[done] Oriented geo
-
tiff
Requirements
•
[done] Save view settings
•
[done] Archive data used in view
•
[done] Access from multiple sites
•
[done] Use of consistent color tables
(use same tables as CW)
Outcome: NOAA/NOS/CSC requested ACT to demonstrate how WIPE/REACT could be
used reproduce current HAB Bulleting
creation process in REACT/WIPE
Possible Advantages:
•
Proposed approach could provide clear view of process and problems
•
Need web centric creation process (can be a browser or application)
•
Explore REACT/WIPE as next generation environment
DECISION SUPPORT TOOLS PROGRESS (continue)
Cooperative Agreement # NNS04AA78A
24
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Cooperative Agreement # NNS04AA78A
25
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Cooperative Agreement # NNS04AA78A
26
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Cooperative Agreement # NNS04AA78A
27
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Cooperative Agreement # NNS04AA78A
28
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Cooperative Agreement # NNS04AA78A
29
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
HTML output
+ PDF output
Cooperative Agreement # NNS04AA78A
30
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
SPEED Data Cube Generated
on the Fly from U&V
components
Cooperative Agreement # NNS04AA78A
31
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
EXAMPLE OF READING AND DISPLAYING: generic formats, NRL’s
MODEL DATA, MODIS L1B, CW Products, …
Cooperative Agreement # NNS04AA78A
32
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
EXAMPLE OF READING AND DISPLAYING NRL’s MODEL DATA
DATA
Probes
On
Active
Image
Cooperative Agreement # NNS04AA78A
33
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
EXAMPLE OF READING AND DISPLAYING NRL’s MODEL DATA
Metadata information
Easy
Access to
Any Layer,
Including
Volume Animation
Cooperative Agreement # NNS04AA78A
34
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
EXAMPLE OF NRL L2
-
L4
File System Navigation and
HDF Data Structure Navigation
Cooperative Agreement # NNS04AA78A
35
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Cooperative Agreement # NNS04AA78A
36
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Meetings/Training/Demos
–
Supported Different ESIP Federations Meetings
–
Gave Training on WIPE at NRL (2 days)
–
Explore Training on REACT (over the WEB)
•
For Example, Gave Presentation to CSC on REACT using
VNC
Cooperative Agreement # NNS04AA78A
37
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
ACT’s REASON
Web page Is up an running
http://www.actgate.com/reason
Provides:
•
Report Summaries
•
Actual Data Access
•
DST Demos Products
•
Tutorials
•
…
…LIVE DEMO …
Cooperative Agreement # NNS04AA78A
38
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
•
NRL Related
–
Work over WIPE enhancements/customization requested by NRL
-
SSC
•
Custom product generation
•
Virtual products on the fly…
•
Legend
•
…
–
Further Training to NRL on Use of WIPE/REACT
•
NOAA Related
–
Fine
-
tune HAB Wizard based on inputs from NOAA/NOS/CSC/…
–
Provide Training as needed to NOS and CSC
•
ESIP Federation Related
–
Demonstrate Web Services Capabilities of WIPE/REACT (SOAP/WSDL)
•
Improve interfaces with OGC and OpenDap
•
Demonstrate with ESIP Federation Web Services Chaining (using WIPE as a prototype)
•
Use REACT as a test case for Technology Infusion Process being Developed by the SEEDS Working
Group on Technology Infusion
•
Investigate interface WIPE within NOAA/CLASS Architecture
•
Start Investigation Next Year Target Application Areas Based on Team
Recommendations
Next Quarter Main Effort …
Cooperative Agreement # NNS04AA78A
39
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Summary
•
All ACT WIPE/REACT resources are accessible via ACT’s reason web page, i.e.
–
http://www.actgate.com/reason
–
On
-
line demos are being constructed to simplify end
-
user training
•
NRLSSC WIPE Server is actively receiving NRL data streams
–
NOAA can access all NRL data directly from WIPE/Applet interface or from WIPE/REACT
interface! (over 1,200 different fields populated daily)
•
REACT has matured to a state that NOAA/NOS/CSC can:
–
Test its use for HAB analysis tools
–
Test its use for HAB bulleting creation
–
ACT WILL WORK WITH THEM IN SIMPLIFYING USAGE OF TOOL
•
WIPE Axon Architecture for Decision Support Tools is working
–
Need testing and use …
Cooperative Agreement # NNS04AA78A
40
Applying NASA/EOS Data to Coastal Zone Management Application Developed from Integrated Analyses
SENSOR TO USER
Possible Items for Discussion
•
Are we heading (REACTing) in the right direction?
•
How can we help make the NRL products easier to evaluate by NOAA?
•
How can we make the emerging architecture more useful to NOAA?
•
How can we make the large volume of unique data more useful to a larger audience?
•
Can the data in the WIPE servers be accessed by non
-
team members?
–
The goal is to raise awareness of the many products available and the web
-
services infrastructure
•
Can we interface WIPE better with NOAA’s CLASS architecture?
…NRL and NOAA suggestions go here …
Enter the password to open this PDF file:
File name:
-
File size:
-
Title:
-
Author:
-
Subject:
-
Keywords:
-
Creation Date:
-
Modification Date:
-
Creator:
-
PDF Producer:
-
PDF Version:
-
Page Count:
-
Preparing document for printing…
0%
Comments 0
Log in to post a comment