Visualization Cookbook - FTP Directory Listing - University of South ...

assistantashamedΔιαχείριση Δεδομένων

29 Νοε 2012 (πριν από 4 χρόνια και 6 μήνες)

343 εμφανίσεις

SEACOOS Data Visualization Cookbook


**Draft**



Table of Contents


I.

Introduction

................................
................................
................................
...........................

4

II.

Database Specs

................................
................................
................................
.....................

5

A.

PostgreSQL database

................................
................................
................................
..........

5

1.

Remote Sensing
I
diosyncrasies (
content under development
)

................................
.......

5

2.

Impl
ementation
D
etails

(
content under development
)

................................
..................

10

B.

PostGIS Geospatial Enablement

................................
................................
.......................

10

1.

Implementation Details

(
content under devel
opment
)

................................
..................

10

C.

Software Installation and
C
onfiguration

(
content under development
)

............................

11

III.

Web Mapping with MapServer

................................
................................
.....................

11

A.

Mapserv CGI

................................
................................
................................
....................

11

1.

Implementation Details

(
content under development
)

................................
..................

11

B.

PHP
-
MapScript

................................
................................
................................
.................

12

C.

Product Links

................................
................................
................................
....................

12

D.

Support Applications

................................
................................
................................
........

12

1.

Implem
entation Details

(
content under development
)

................................
..................

13

E.

Software Installation and Configuration (
content under development
)

............................

13

IV.

Data E
xploration Applications

................................
................................
......................

13

V.

OGC web services

................................
................................
................................
...............

13

A.

External Web Service Audience

................................
................................
.......................

14

1.

Implementation
D
etails

(
content under development
)

................................
..................

14




[
Some contents from Jeremy’s SURA TWIKI post:
http://twiki.sura.org/bin
/view/Main/DataStandards?skin=print
]


SEA
-
COOS presented the chance to define data standards and integrate in
-
situ observational,
modeling and remote sensing data products from several institutions and programs. The GIS
visualization provides immediate fe
edback and validation as to the effectiveness of integration
efforts while also raising further questions about the quality and display of data.

The in
-
situ and remote sensing layers are presented in the
MapServer

GIS and allow for raster
query
-

drilling through data layers to obtain the point values. This allows calibration/validation
between these products. GIS animation and time series graphs are also provided for all products.

For the in
-
s
itu and modeling products, each involved institution was able to setup a
DODS/OPeNDAP netCDF server to share a netCDF file representing the past 2 weeks worth of
data. While this data is available via DODS/OPeNDAP, the amount of data being transmitted
was
only a few kilobytes, so for performance reasons when aggregating the data at the central
relational database the file was uploaded directly(not utilizing the DODS/OPeNDAP API) and
parsed with perl netCDF libraries
http://www.carocoops.org/bb/viewtopic.php?t=104

.

In
-
situ observational

MapServer

GIS
http://
nautilus.baruch.sc.edu/portal_obs


DODS/OPeNDAP data access
http://trident.baruch.sc.edu/dods/seacoos


CSV (Comma Separated Value/Excel) files and perl generation programs
http://nautilus.baruch.sc.edu/seacoos_data/CSV/


After many telephone and email discussions between institutions, a few data standard
proposals for a
SeaCOOS

in
-
situ netCDF format were developed. While the format
currently focuses on a few different cases(fixed
-
point, fixed
-
profiler, fixed
-
map, moving
-
point
-
2D, moving
-
point
-
3D and moving
-
profiler), an example of this format can be seen
at t
he following data provider directories

http://nccoos.unc.edu/data/nos/proc_data/latest_v2.0


htt
p://seacoos.skio.peachnet.edu/proc_data/latest_v2.0


http://nccoos.unc.edu/data/nws_metar/proc_data/latest_v2.0


http://trident.baruch.sc.edu/usgs_data


http://trident.baruch.sc.edu/storm_surge_data/latest


h
ttp://seacoos.marine.usf.edu/data/seacoos_rt_v2


http://oceanlab.rsmas.miami.edu/ELWX5/v2


http://aigeann.ucsc.edu/cimtLatest


http://trident.baruch.sc.edu/po.daac_data/OVW


http://nccoos.unc.edu/data/nc
-
coos/latest_v2.0


http://oceanlab.rsmas.miami.edu/wera


This site lists all the canonical forms which were under consideration

http://nccoos.unc.edu/data/test_cdl/


Here is documentation on the current SEA
-
COOS CDL v2.0 format

SEACOOSv2.0


Charlton's perl programs which gather the latest data(..._latest.
nc) from providers on a
periodic basis (which has come to be termed the data 'scout') and converts these netCDF
files to SQL INSERT statements to populate the relational databse is listed here
http://nautilus.baruch.sc.edu/seacoos_data/CSV/data_scout/


One of the main points developed was to allow for the most complex data model and let
everything else fall out as a subset of that case. With this thinking in mind, the best
way
to model the data is to make all variables(including latitude, longitude and depth) a
function of time. The other data forms allowed for programmatic 'shortcuts' based on the
types of dimensions presented in the file
-

for instance if latitude and long
itude were each
a dimension of 1 point, then the file was processed as a fixed station. Most of the debates
centered around whether description should be carried in the variable attributes or in the
dimension or variable naming convention. COARDS, CF conve
ntions were adopted
where possible.


Modeling

SeaCOOS

model output integration website(3 merged area domains covering the
Southeast)
http://nautilus.baruch.sc.edu/model_rs


The following is a link to the model output for the top
-
right section

http://nccoos.unc.edu/data/nc
-
coos/model_data/quoddy/
forecast/


Modelers needed to resolve within their group resolution and interpolation issues
regarding time and other variables as they related to the model output, model region
overlaps and their display. Display of the results via the GIS helped articul
ate various
projection, alignment and resolution issues. Since all the model data was homogenous
area/field oriented data, deciding on a common netCDF representation was fairly
straightforward.


RemoteSensing
?


MapServer

GIS
http://nautilus.baruch.sc.edu/rs


Remote sensing listserv
http://caro
-
coops.org/cgi
-
bin/wilma/remotesensing



Aggregation Process

In
-
situ and Modeling

The process for in situ and model data which is being formatted to a netCDF file
is:

1)Pick a
physical variable of interest (like wind speed&direction, sea surface
temperature)

Each variable is defined within it's own table(one record for each measurement).
So there is a table which contains station id, time, latitude, longitude, depth,
wind_speed
, wind_direction, associated metadata fields. Another table which
contains row_id,station id, time, latitude, longitude, depth,
sea_surface_temperature, associated metadata fields. And so on...

SQL allows us to join separate table data into views of combi
ned data according
to user query needs(so far we haven't needed to do this). Currently the separate
variable tables are used to generate separate variable data maps and these maps
can be superimposed via the GIS.

2)Define how the variable will be defined
in time and location for
SeaCOOS
,
everyone has decided to use the same time frame of seconds(this can be a floating
point number to indicate subsecond intervals) since 1970
-
01
-
01 for location
cons
iderations
SeaCOOS

has developed a framework for datatypes(relating to the
degrees of locational freedom of the measurement point(s)) which gives
guidelines on how this should be defined in a netC
DF file via dimensions and
attributes

3)Considerations for display purposes
-

metadata fields are added which take into
consideration the measurement and
-

whether it should be shown in the display
-

can be normalized given an agreed upon normalization fo
rmula
-

orientation as it
relates to the agreed group locational framework
-

how data is interpolated and
chosen for display

The above steps result in the
SeaCOOS

netCDF convention as a specific
syntax of
dimensions and attributes which can be programmatically parsed by Charlton's
perl script (scout) which downloads the netCDF files via HTTP from data
providers and populates the variable tables or alerts the provider when there is a
problem with t
he data being input.

Charlton then uses
MapServer

functions to convert from the table data to a raster
image which is displayed as a data layer in the GIS.

For each new physical in situ variab
le to be added, aside from what the naming
convention that will be for the new variable, step 3 is the only step which should
be repeated. Steps 1&2 are a one time initial group discussion/decision processes
which are subject to periodic consideration and
revision if need be.

Step 3 also tends to take the product(GIS in this case) considerations into mind,
whereas the work accomplished in steps 1&2 should be universally applicable for
aggregation needs across a variety of products.


RemoteSensing
?


The process for remote sensing data using PNG files is:

The PNG files themselves contain locational metadata used for
placement/interpretation of the raster image w
ithin the GIS and the filenames
contain timestamp information and this timestamp is used by Charlton's code to
determine which image should be displayed for given temporal conditions. No
underlying relational database support is needed for the initial disp
lay of the
images within the GIS.

Relational database support is currently used to support raster query functions(the
raster image is converted to a table lookup of RGB values which correspond to
variable measurement). This functionality should be tempora
ry as
MapServer

should better support raster query in the future.

I.


Biological

The process for biological data is still up for discussion. One way of thinking of
datatypes is as point, raster a
nd vector/coverage. I think we have working models
for working with point(in situ) and raster(PNG, etc), but with biological datasets,
I'd say my thinking would be to possibly add a time dependent vector/coverage
which defines a collection or population lo
cational boundary. As a table
representation, this would mean that each population/collection would have its
own time indexed table with a link to a vector/coverage subtable. This would keep
time as a rigid searchable index and allow a population to be def
ined as a
collection of points or a boundary of an area. Populations could still be described
at a point or as part of a raster(image), the vector/coverage datatype would be
more to accommodate population datatypes which don't fit well into our existing
fr
amework of point and raster datatypes.


I.

Introduction


After SEACOOS data are collected and aggregated, visualization procedures are implemented to
represent this data for constituent user groups. These procedures provide immediate feedback
and validation

of SEACOOS data aggregation efforts, quickly addressing integration issues
about data projection and resolution. These procedures are built using open source solutions
where possible. The methods presented below encompass a significant amount of develop
ment
work that has coalesced into
a robust
data

visualization effort. This effort is an initial step
toward leveraging SEACOOS project data into national and international ocean observing
efforts.



This documentation is intended to serve as both an overv
iew of the SEACOOS data
visualization efforts and a technical reference (under development). Each section contains a
topical overview followed by sections on technical details and implementation requirements.

II.

Database Specs


SEACOOS uses the open source P
ostgreSQL relational database to store
in
-
situ

and remotely
sensed data collected from project partners. PostgreSQL can be accessed by a number of front
-
end applications and can also be enabled for spatial querying and mapping. This enablement is
achieve
d using the PostGIS extension for PostgreSQL. PostGIS adds several geometric objects
to the supported datatypes in PostgreSQL. This combination can then function as the spatial
database engine for all subsequent SEACOOS data visualization.


A.

PostgreSQL

database


SEACOOS data is stored in two PostgreSQL database instances at the University of South
Carolina (USC). One instance contains the
in
-
situ

observations and remotely sensed data. The
other contains model output data and duplicate in
-
situ observat
ions, used for “round
-
robin”
updating. The databases are partitioned into separate tables for each in
-
situ observation variable,
remotely sensed raster layer, and model variable layer per hour. The remotely sensed tables do
not house the actual images bu
t pointers to the image files and their ancillary boundary files.
The database is used to execute raster queries, which require the image RGB values to be
referenced against a look
-
up table of actual variable measurements.


1.

Remote Sensing idiosyncrasies

(
content under development
)

Processing by Charlton


e.g.,
composite procedure
, etc


general image fetching



fetch file based on embedded timestamp



remove background and watermarks using ImageMagick



cleaned image has embedded timestamp in the filename, ass
ociated
.wld file, and has a matching reference in a DB lookup table



compositing



MODIS RGB images are also part of a compositing process



After a new RGB has been fetched, a script called
build_composite.php [show script] is called which creates a resulta
nt
image of the past n passes (in our case 4) that are no older than y days
(in our case 4).



This new image is similarly included as part of the DB lookup process
as non composites are.

Storage


file system wit
h pointer stored in PostgreSQL?


file system



remotely sensed images are stored on the file system based on their
layer name. e.g.: avhrr_sst, modis_chl, modis_rgb_composite, etc.



And the files under these directories have embedded timestamps as
part of their filenames as well as a corresponding geo
referncing file
that MapServer requires. E.g. avhrr_sst_2004_04_01_03_06.png,
avhrr_sst_2004_04_01_03_06.wld,
avhrr_sst_2004_04_01_04_49.png,
avhrr_sst_2004_04_01_04_49.wld.

PostgreSQL pointer



Distinct tables contain pointers to all remotely sensed images

on the
filesystem and is accessed by timestamp. E.g. [show raster_oi_sst
table DDL] whose contents may be:


pass_timestamp | local_filename

---------------------
+
------------------------------------


2004
-
08
-
01 17:30:00 | oi_ss
t/oi_sst_2004_08_01_17_30.png


2004
-
08
-
02 17:30:00 | oi_sst/oi_sst_2004_08_02_17_30.png


2004
-
08
-
03 17:30:00 | oi_sst/oi_sst_2004_08_03_17_30.png



Data Structures / Canonical forms

The structure of temporal, geospatial data as it is stored in various for
mats should
hopefully be capable of having its structural elements described in a handful of forms.
Describing and labeling these forms(deciding what should be abstracted away as well)
are the beginning steps before automated programmatic conventions, labe
ls and
processing can be utilized in data transformation.

As an example, two predictable forms for storing buoy data are



'by station' where the tablename is that of the station and each row corresponds to
all the variable reading for a given time measure
ment



-

or
-




'by variable' where the tablename is that of the variable measured and each row
corresponds to a measurement time, station id and possibly lat, long and depth
describing the measurement point and the measurand value.

Currently the GIS favors

a 'by variable' approach which corresponds to variable data
layers. I appreciate this format for its conciseness and amenability to query and resultset
packaging(the ability to mix and match variables which have a similar reference scheme
on each variable

table). Issues of varying temporal sampling resolutions across multiple
stations are better handled in this form. We are developing programs to convert other
table formats to this format.

see also
Going from Datalogger files to 'Point' form on a relational database



pointForm

(by Variable form, used with point and moving point data)


The following represents a basic table layout which might be implemented o
n a
postgresql database.

CREATE TABLE <my_table>

(row_id SERIAL PRIMARY KEY,


row_entry_date TIMESTAMP with time zone,


row_update_date TIMESTAMP with time zone,


platform_id INT NOT NULL,


sensor_id INT,


measurement_date TIMESTAMP with time zone,


measu
rement_value_<my_var> FLOAT,


--

other associated measurement_value_<my_var> added here as well


latitude FLOAT,


longitude FLOAT,


z FLOAT,


z_desc VARCHAR(20),


qc_level INT,


qc_flag VARCHAR(32));


<my_table> and <my_var> could be the same for simple va
riables like 'sst'( <my_table>
= 'sst', <my_var> = 'sst'> )

or different for tandemly collected variables like where ( <my_table> = 'winds',
<my_var_1> = 'wind_speed', <my_var_2> = 'wind_from_direction', <my_var_3> =
'wind_gust' ).


row_id, row_entry_date,

row_update_date are my preferences for internal table row
references


platform_id (or station_id, container_id)


an integer code which references a lookup
table for platform information. Could also be a string label for simpler configurations.


sensor_
id
-

an integer code which references a lookup table for platform information.
Not neccessarily required for simpler configurations(or could be a string label) but adding
an additional degree of detail and flexibility at the sensor level.


measurement_d
ate
-

the time the variable was measured with time zone reference


measurement_value_<my_var>
-

the value(s) of the measured variable(s). There could
be one or several measurement_value_<my_var> listed here.


latitude, longitude
-

unit decimal degree


z
-

depth or height reference, unit meters


z_desc
-

a string label which describes z for averaged values or specific boundaries such
as 'surface', 'bottom', 'average', 'bin1','bin2'... Optional.


qc_level
-

integer value of [0,1,2,3,
-
9] corresponding to hig
h
-
level qc process

qc_flag
-

space separated listing of qc flags which may apply to this measurement

Adding a general index(assuming a station level reference to individual variables) to
prevent duplicate measurements.

CREATE UNIQUE INDEX i_<my_table> ON
<my_table>
(platform_id,sensor_id,z,z_desc,measurement_date);

Adding a
PostGIS

2
-
dimensional geospatial reference column type(see this link on
PostGIS

enabling a
PostgreSQL

database
http://www.carocoops.org/misc/phpBB2/viewtopic.php?t=229
)


--

a
dd the geometries

select AddGeometryColumn('<my_dbinstance>','<my_table>','the_geom',
-
1,'POINT',2);


--

add geometry index

create index <my_table>_gist on <my_table> using gist(the_geom);

Here are database descriptions of the wind and sst tables which Char
lton currently
utilizes. There are matches to the structure listed above with additional fields added for



GIS display purposes (for example, should this be displayed in the GIS or not)



unit converted fields (for example, celsius to fahrenheit conversions
)



frame of reference (for example, does a positive z value correspond to up or
down)



contact or standards metadata (for example, provider URL, standard XML fields
for WMS, WFS or other standards metadata)

We make efforts to keep from 'normalizing' the t
able into subtables preferring a single
table approach with redundancy in certain fields. Since the storage needs are low to begin
with this means we can keep things conceptually and operationally simple. Table
preformance can be further optimized by parti
tioning, and familiarity /use of VACUUM,
COPY, CLUSTER commands and other indexing schemes which can be applied
similarly across these repeated table stuctures.


sea_coos_obs=#
\
d wind_prod


Table "public.wind_prod"



Column | Type | Modifiers

-----------------------------
+
-----------------------------
+
-------------------------------------
---


station_id | character varying

| not null


time_stamp | timestamp without time zone | not null


z | double precision | not null


wind_speed | double precision |


wind_from_direction | double pr
ecision |


label_theta | double precision |


label_char | character(1) |


lon | double precision |


lat | double precision

|


title | character varying |


institution | character varying |


institution_url | character varying |


institution_dods_url | character varying

|


source | character varying |


refs | character varying |


contact | character varying |


report_time_stamp | timestamp without time zone |


s
how | integer |


secs_from_report_time_stamp | integer |


secs_from_time_stamp | integer |


the_geom | geometry | not null


w
ind_speed_knots | double precision |


can_be_normalized | character varying |


normalized_wind_speed | double precision |


normalized_wind_speed_knots | double precision |


normalize
d_label_char | character(1) |


positive | character varying |


label_z | double precision |


wind_from_direction_compass | character varying |


wind_speed_mph

| double precision |


normalized_wind_speed_mph | double precision |


label_char_knots | character(1) |


label_char_mph | character(1) |


normalized_label_char_kno
ts | character(1) |


normalized_label_char_mph | character(1) |


value | character varying |


value_knots | character varying |


value_mph | cha
racter varying |


normalized_value | character varying |


normalized_value_knots | character varying |


normalized_value_mph | character varying |


seq | integer

| default
nextval('wind_prod_seq'::text)

Indexes:


"pk_id_time_stamp_z" primary key, btree (station_id, time_stamp, z)


"wind_prod__gist" gist (the_geom)


"wind_prod__id_z_rep_time" btree (station_id, z, report_time_stamp)


"wi
nd_prod__oid" btree (oid)


"wind_prod__report_time_stamp" btree (report_time_stamp)


"wind_prod__seq" btree (seq)


"wind_prod__show" btree ("show")


"wind_prod__station_id" btree (station_id)


"wind_prod__time_stamp" btree (time_stamp)

Check

constraints:


"$1" CHECK (srid(the_geom) =
-
1)


"$2" CHECK (geometrytype(the_geom) = 'POINT'::text OR the_geom IS NULL)


"can_be_normalized_yes_no" CHECK (can_be_normalized::text = 'yes'::character varying::text OR
can_be_normalized::text = 'no':
:character va

rying::text OR can_be_normalized::text = ''::character varying::text)


"positive_up_down" CHECK (positive::text = 'up'::character varying::text OR positive::text =
'down'::character varying::text OR positive::

text = ''::character varying:
:text)

Triggers:


wind_labels_and_times_trig BEFORE INSERT OR UPDATE ON wind_prod FOR EACH ROW EXECUTE
PROCEDURE wind_labels_and_times()


sea_coos_obs=#
\
d sst_prod


Table "public.sst_prod"


Column

| Type | Modifiers

------------------------------
+
-----------------------------
+
------------------------------------
---


station_id | character varying | not null


time_sta
mp | timestamp without time zone | not null


z | double precision |


temperature_celcius | double precision |


lon | double precision |


lat

| double precision |


title | character varying |


institution | character varying |


institution_url | character varying |


institut
ion_dods_url | character varying |


source | character varying |


refs | character varying |


contact | character varying |


report_time_
stamp | timestamp without time zone |


show | integer |


secs_from_report_time_stamp | integer |


secs_from_time_stamp | integer |


the_geom

| geometry | not null


positive | character varying |


label_z | double precision |


temperature_fahrenheit | double precision |


value_temper
ature_celcius | character varying |


value_temperature_fahrenheit | character varying |


seq | integer | default
nextval('sst_prod_seq'::text)

Indexes:


"pk_id_time_stamp" primary key
, btree (station_id, time_stamp)


"sst_prod__gist" gist (the_geom)


"sst_prod__id_z_rep_time" btree (station_id, z, report_time_stamp)


"sst_prod__oid" btree (oid)


"sst_prod__report_time_stamp" btree (report_time_stamp)


"sst_prod__seq" btr
ee (seq)


"sst_prod__show" btree ("show")


"sst_prod__station_id" btree (station_id)


"sst_prod__time_stamp" btree (time_stamp)

Check constraints:


"$1" CHECK (srid(the_geom) =
-
1)


"$2" CHECK (geometrytype(the_geom) = 'POINT'::text OR the_g
eom IS NULL)


"positive_up_down" CHECK (positive::text = 'up'::character varying::text OR positive::text =
'down'::character varying::text OR positive::

text = ''::character varying::text)

Triggers:


sst_labels_and_times_trig BEFORE INSERT OR UPDATE
ON sst_prod FOR EACH ROW EXECUTE PROCEDURE
sst_labels_and_times()



Access by mapfile


lookup tables and projection files?


lookup tables and projection files



For a given request of a remotely sensed layer at a given time,
MapScript code references the co
ntrol .map file and sets the DATA
clause to define the base raster image to fulfill the users space and time
query.

2.

Implementation details

(
content under development
)

PostgreSQL database schema



[show schema]

Database massaging for visualization purposes
-

fields added?



Certain in
-
situ data requires MapServer
-
specific fields to be populated in
order to create maps. In
-
situ data must be collected into an hourly snapshot
table. See function set_wind_prod_show() as an example of how in
-
situ
winds are selected

to appear in an hourly snapshot table. The flow of
information is that raw in
-
situ data is inserted into a production table, e.g.
wind_prod. set_wind_prod_show() is run after a complete raw ingestion is
finished. The resulting records that are flagged
as being eligible to be
included in the hourly snapshot table,are collected and inserted into
wind_map.

How to access the PostgreSQL instances for mapping


“Data” parameter in mapfile



Most of the time
-

and space
-
specific programming occurs in
map_session.p
hp. MapScript allows for the dynamic setting of the
DATA and FILTER parameters in the base .map file which MapServer
will use to determine what time and space to display. [show wind_obs
clip from map_session.php].



In addition to time and space sensitivit
ies, map_session.php also
handles unit specifications, e.g. wind speeds in knots, miles per hour,
and meters per seconds. MapScript adjusts the DATA parameter
accordingly.



B.

PostGIS Geospatial Enablement

The PostgreSQL database is “spatially enabled” usi
ng the PostGIS extension, creating a back
-
end spatial database. This extension converts the raw locations of SEACOOS observation data
and stores them in a GIS
-
accessible geometry column, enabling mapping and spatial query
functionality. GIS mapping appli
cations then point to these new columns to render the
geometric topologies they contain. PostGIS fields can also be populated from other common
GIS data formats such as ESRI shapefiles.


1.

Implementation Details

(
content under development
)

PostGIS comma
nds

[Charlton doesn’t know what this means ???]

Shapefile to
PostGIS co
n
version
s



Many of the static base data layers are converted from shapefiles into PostGIS
tables by using the shp2pgsql command.

C.

Software Installation and configuration (CBC, JC, and CP
help)

Installation Process


precursor configurations/libraries ready?



See seacoos_install.txt in
http://caro
-
coops.org/resources/jpl/
.

Servers and connections

PostgreSQL database:
http://www.postgresql.org/

PostGIS extension:
http://postgis.refractions.net/

III.


Web Mapping with MapServer


Visualization of SEACOOS data over the web utilizes an open source
mapping

pl
atform known
as MapServer. MapServer is well adapted for use with PostgreSQL/PostGIS and can power
open web mapping services within a flexible, scriptable environment. Although MapServer can
parse ESRI data formats, these are not required and open source

customization is encouraged.
MapServer utilizes a “mapfile” (*.map) to setup parameters and control map images for each
map instance. Elements of this mapfile may then be manipulated via URL query statements,
giving MapServer its web mapping flexibility
. The instance powering SEACOOS maps is
housed at USC, local to the SEACOOS aggregate database.




A.

Mapserv CGI

MapServer is used in two modes to map SEACOOS data. For more basic applications, the
mapserv CGI is used. The gives control over basic elemen
ts of the mapfile via a URL query:
toggling map layers on/off, adjusting the map size, and panning the map extents. More detailed
changes require editing the MapServer mapfile served from USC. For custom GIS applications,
GUIs are created that generate U
RL query strings, which control the map images from the USC
MapServer. The mapserv CGI powers the OGC web services (WMS and WFS) available from
USC.

1.

Implementation Details

(
content under development
)

Mapfile details


“data” parameter: SQL to hit Postgre
SQL DB

[explained above]

WMS and WFS instances and links



SEACOOS maps and underlying data are accessible via WMS and WFS OGC
requests. Customized Perl wrappers were written to accommodate time
parameters. [show the wrappers]. Descriptions of how to acce
ss these OGC
-
ready layers can be found here:
http://caro
-
coops.org/bb/viewtopic.php?t=331

(for WMS and WFS in
-
situ layers) and
http:
//caro
-
coops.org/bb/viewtopic.php?p=564

(for WMS remotely sensed layers).

Getting images into MapServer

[explained above]

Remote Sensing mapfile and visualization details


lookup tables

[explained above]





B.

PHP
-
MapScript

A second mode of function for
MapServer utilizes an additional module called MapScript to
control the same base mapfile. SEACOOS uses the PHP variant of MapScript because of PHP’s
ability to work with HTML. This functionality enables much more control over the mapfile from
a web brow
ser. Most parameters in the mapfile can be controlled and specified via the URL
query string. In addition to the abovementioned CGI functionality, URL queries can be
constructed to adjust the size and location of the scalebar, move the legend placement,
change
layer unit settings, and place watermark images. This functionality works behind the GUI
products driving the Interactive Observations Map, the Interactive Model Display, and the
Remote Sensing Development Map now served at USC (links below). Comp
onents of this
functionality have been leveraged to drive some of the internal data exploration efforts
mentioned below.


C.

Product Links:

Interactive Observations Map:
http://seacoos.org/Data
%20Access%20and%20Mapping/

Interactive Model Display:
http://seacoos.org/Model%20Output%20and%20Mapping/

Remote Sensing Development Map:
http://nautilus.baruch.sc.edu/portal_rs/


1.

Implementation Details

(
content under development
)

PHP MapScript Code?

[map_session.php explained above]



[give link to portal_rs source code tree] In addition to map_session.php,
seacoos_bathy_contents.ph
p and seacoos_bathy.phtml are top
-
level control
files that contain references to underlying code that defines the functionality
of the interactive maps.

SEACOOS region extents



Predefined regions for quick access are defined within the .map file and are
ref
erenced upon interactive map startup.



D.

Support Applications


Several other open source solutions are used to graph, animate, and query SEACOOS data.
GIFsicle and AnimationS (AniS ) are used to create and control data animations over the web.
ImageMagick

(and perl::ImageMagick) is used for image manipulation and to execute raster data
queries. Mouseover query functionality is enabled with the searchmap component of
MapServer, creating an imagemap of the existing map image for queries. Graph Plotting Mod
ule
for Perl (GD::Graph) is used to generate time series graphs. All of these tools are scriptable and
run behind the SEACOOS interactive maps.



1.

Implementation Details

(
content under development
)

ImageMagick
details
for image manipulation and raster q
ueries

[show xy2sst.pl as
example]

A
nimation Routine


user details [show mk_anim_gif.php as example]


E.

Software Installation and Configuration (
content under development
)

Installation Process


precursor configurations/libraries ready
?

[explained above]

Ma
pServer:
http://mapserver.gis.umn.edu/




IV.

Data Exploration Applications


Further data exploration and visualization has been enabled to allow researchers quick access to
the SEACOOS database. These tools are
web pages that rely on PHP to interact with the
PostgreSQL database and MapServer, presenting database content ranges and simple maps. The
following pages are in use by SEACOOS researchers and are updated in near real time:


A
data

overview page

displays

a list of min and max timestamps for all SEACOOS interactive
map data (model input data and observations data). It also provides links to individual pages for
each data layer displaying the specific time slices available for each individual layer. Acces
s is
also available to map images of each layer and each time and date stamp, across 3 selectable
regions. While these pages are not intended for the general public, they provide on
-
demand
access and visualization to the entire SEACOOS database for a dist
ributed research community.

http://nautilus.baruch.sc.edu/seacoos_misc/show_sea_coos_obs_time_ranges.php


The
data animation page

takes URL query string parameter
s and creates animations of data
ingested by SEACOOS. The animation routine combines maps and graphs for most SEACOOS
data. Users have control over the GIS layers, scale, platforms to graph, and time step. These
animations are then served via another PHP

generated page with full animation movement
controls. These animations are created, stored, and served at USC until the user asks for them to
be removed.

Output example:



The flexibility of PHP


MapScript allows other SEACOOS institutions to utili
ze the USC
database for further visualization development. A
cached observation page

serves static images
each hour for a variety of SEACOOS data layers and sub regions. This page is supplied by a
script that sends modified URL query strings to the USC M
apServer and caches the map images
that are returned.

http://seacoos.org/Data Access and Mapping/cached
-
images/


V.

OGC web services


External presentation of SEACOOS data is enabled
compliant with standards set by the Open
Geospatial Consortium (OGC). OGC web mapping services are XML based and are intended to
be platform independent, web
-
enabled, representations of GIS data. The services can be
accessed, controlled, and presented by

web browsers as well as other GIS software platforms (i.e.
ESRI Interoperability toolbar, GAIA application). SEACOOS OGC compliant services rely on
the mapserv CGI engine. SEACOOS provides a Web Mapping Service (WMS) and Web
Feature Service (WFS). The
WMS feed returns a static, georeferenced image to a user’s browser
or GIS platform, while the WFS feed returns actual feature data, allowing visualization control
over these data externally.


A.

External Web Service Audience

Other ocean observing research pro
jects currently ingest SEACOOS OGC feeds, the OpenIOOS
project (
www.openioos.org
) and the GoMOOS project (
www.gomoos.org
). Several other local
GIS projects also display SEACOO
S data via WMS feeds. This method of integration is a
critical step in the desired aggregation of ocean observing data across regional, national, and
global scales.


1.

Implementation details

(
content under development
)

Wrapper script around a mapfile with s
et of metadata requirements

[explained above]

Open Geospatial Consortium:
http://www.opengeospatial.org




**Draft**