User Assessment Questionnaire Results

townripeData Management

Jan 31, 2013 (4 years and 8 months ago)

210 views

Hydrologic Information Use and
Needs Assessment

Utah State University


Examples of Existing Systems


User Needs Assessment Survey Results


David G Tarboton

with input from

Jeff Horsburgh, Rob Gillies, Christina
Bandaragoda, John Weeks, Doug Ramsey

Some existing hydrology related
information systems at USU


Bear River Watershed Information
System


Intermountain Region Digital Image
Archive (IRDIAC)


Climate and Weather Data Server


UNIDATA port

Internet Based WIS

Bear River
http://water.usu.edu/wis/bear_river/

Great Salt Lake
http://greatsaltlake.utah.edu/



Web browser as client software


All databases and data analysis
managed on master server


ESRI’s ArcIMS powers the map
server and provides
visualization of GIS datasets


SQL Server and Time Series
Analyst manage time series
data and provide access to
analysis and visualization

Servers


Web Server Dell PowerEdge 700, $3000


Database Server Dell PowerEdge 2800, $3000


Base Station for real time data collection, Pentium III 600
MHz, < $1000 now.


Operating Systems


Windows Server 2003 and Windows XP


Software


IIS v6.0


.Net Framework v1.1


ProEssentials v5 (for time series plotting)


ArcIMS v4.0


ArcGIS


Microsoft SQL Server 2000 Standard Edition

Disk Space


223 GB hard drive/~20 GB used


Communication


100Mb connection to USU internet


Operators


Jeff Horsburgh


Research Engineer with expertise in
data analysis, data manipulation, GIS, modeling, custom
GIS application development, etc.


Rick Satterfield


Graduate Student system programmer
with expertise in developing and deploying Internet
applications


Crystal Yap


data technician (downloading, formatting,
etc.)


Amber Spackman


data technician (downloading,
formatting, etc.)

http://earth.gis.usu.edu


Servers


Sunfire 280R server with 2 1.2 GHz processors and 8GB
RAM, ~$20,000


Partnership in Beowulf Linux Cluster


Windows Server to support ArcGIS geodatabase
processing


Operating Systems


Solaris, Linux, Windows Server XP


Software


MSQL


Apache and Tomcat web services


MapServer, PHP, PHP Mapscript


GDAL and IDL/ENVI for image processing


ESRI Geodatabase Server

Disk Space


2TB at 95% capacity


Communication


1GB connection to USU internet


Operators


1 FTE programmer/analyst with knowledge of C, PHP, IDL, GIS
programming languages and scripts.


½ FTE web developer


¼
-

½ FTE System Administrator with knowledge in Solaris, Linux,
and PC networking architecture, security, installation and
integration if 3rd
-

party software, etc.


½ FTE Project Manager to oversee and manage daily activities.


¼


½ FTE Project Director.


3 student technicians (1.5 FTE) that organize, process imagery and
data, populate the image database, and provide programming and
web support.


2 postdoctoral fellows to provide research support

Climate and Weather Data Server

Integrated Weather and Climate
Database

Goals


Data Ingestion from Many Sources


WEB Accessible


User Friendly Navigation and Interfaces


User Friendly Data Output


High Quality Reports and Graphs


High Quality Maps


All available climate data in one location

Currently offline
-

undergoing restructuring!

Servers


One Windows PC, $3000, System Management
Front end


Two Redhat Linux PC, $3000, Apache Web
Servers


1TB RAID Disk Array $6000

Operating Systems


Windows Server 2003


RedHat Enterprise Linux


Software


PostgreSQL


Apache Web Services


In house developed applications

Disk Space


120GB, 30% full


250GB, 40% full


1000GB (1TB), 60% full


Communication


100MB connection to USU internet


Operators


Systems Administrator
-

MS Windows, RedHat Linux


Programmer to build custom utilities: Java, C, C++, Perl


Software Management and Maintenance
-

Third Party
or In house


Subject Matter Expert
-

Climate

UNIDATA
-

access and visualization of
meteorological information

IDD Data Feed

Data is ingested using a
client / server program
-

Local Data Manager.

Scripts are configured to
select data through
pattern matching
techniques which define
data requested and
amount.

Currently USU ingests
1GB / hour.

Visualization and Analysis


Several custom
software packages
are made available for
analysis

and
visualization

of data


Interactive Data
Viewer (IDV)
-

shown
on left


McIDAS


GEMPACK

Servers


Solaris Ultra 10, < $500 now


Operating Systems


Solaris


Software


LDM


Unidata


McIdas

Disk Space


250GB, 90% full


Communication


10MB connection to USU internet


Operators


Systems Administrator
-

Solaris


Programmer to build custom utilities: Java, C,
C++, Perl


Software Management and Maintenance
-

Third
Party or In house


Subject Matter Expert
-

Climate

User Needs Assessment Survey
Results

18 USU Respondents

Respondents
0
2
4
6
8
10
12
Faculty
Grad Student
Undergrad
Student
Other
Univ. Pro.
Other
Operating System
0
2
4
6
8
10
12
Windows
Mac/OS X
Linux
Solaris
Unix
Other
Minutes to complete survey
0
2
4
6
8
10
12
5
10
15
20
25
30
35
Mean: 15.5
User Needs Assessment Survey
Results

36 CUAHSI Respondents

Respondents
0
5
10
15
20
25
30
Faculty
Grad Student
Undergrad
Student
University
Professional
Working
Professional
Other
Windows
MAC/OS X
Linux
Solaris
Unix
Other
Operating System %
0
20
40
60
80
100
Minutes to complete survey
0
2
4
6
8
10
12
0
5
10
15
20
25
30
Mean: 15.5
CUASHI Survey Respondent Specialities
0
2
4
6
8
10
12
14
16
Engineering:Hydrology/Water
Resources
Engineering:Groundwater
Geology
Computer Science
Climate/Meteorology
Biology
GIS
Engineering: Electrical
Water Quality/Chemistry
Geomorphology
Software
-

USU

Other Software
mentioned:

SAS
-

3

Surfer
-

2

R, Groundwater
Vistas, STAMMT
-
L, HEC
-
RAS,
Adobe Illustrator,
Kaleidagraph, MS
SQL Server,
ArcIMS,
MapWindow,
Macromedia,
Minitab,
SigmaPlot


Please rate each of the following software packages and programming languages
with respect to how important they are for hydrologic analysis in your research.

4.5
3.2
2.4
2.3
2.3
2.2
2.0
1.8
1.8
1.6
1.3
1.1
Excel
ArcGIS/ArcView
C++
Visual Basic
Matlab
MS Access
FORTRAN
SPSS
S-Plus
Java
Mathematica
GRASS
Raw mean scores; n = 18
1=Never use or do not find useful; 2=Have used but do not rely on this;
3=Use occasionally and am comfortable with its use; 4=Use often;
5=Use frequently and find indispensable.
Software
-

USU

Please rate each of the following software packages and programming languages
with respect to how important they are for hydrologic analysis in your research.

Excel
0
5
10
15
1
2
3
4
5
Mean: 4.5
MS Access
0
5
10
15
1
2
3
4
5
Mean: 2.24
Matlab
0
5
10
15
1
2
3
4
5
Mean: 2.28
Mathematica
0
5
10
15
1
2
3
4
5
Mean: 1.29
S-Plus
0
5
10
15
1
2
3
4
5
Mean: 1.76
SPSS
0
5
10
15
1
2
3
4
5
Mean: 1.82
ArcGIS/ArcView
0
5
10
15
1
2
3
4
5
Mean: 3.17
GRASS
0
5
10
15
1
2
3
4
5
Mean: 1.11
FORTRAN
0
5
10
15
1
2
3
4
5
Mean: 2
Visual Basic
0
5
10
15
1
2
3
4
5
Mean: 2.33
C++
0
5
10
15
1
2
3
4
5
Mean: 2.39
Java
0
5
10
15
1
2
3
4
5
Mean: 1.65
Software
-
CUAHSI

Rate the following software packages and programming languages with respect to
how important they are for hydrologic analysis in your research

4.0
3.9
3.3
3.2
2.7
2.6
2.5
2.4
2.4
2.3
2.2
2.1
2.1
1.9
1.9
1.8
Excel
ArcGIS/ArcView
FORTRAN
C/C++
Java
MS Access
Visual Basic
Matlab
SQL/Server
Modflow
Adobe Illustrator
HEC models
GMS, WMS, SMS
R
SWAT
Sigma Plot
1 =Never
use or
do not
find
2=Have
used, but
do not rely
5=Use
frequently
and find
indispensable
4=Use
often
3=Use
occasionally
and am
comfortable
with its use
Raw mean scores; n = 36
Other Software Mentioned


Rockware (2)


Imagine/IDL (2)


MySQL (2)


Perl (2)


RiverTools


SWS


Tcl/Tk(GUI)


Web


Python “2ope”


Visual/Modeling


IDL/PV
-
Ware


Global Mapper


TELEMAC


Virtualization


UML/VMware/etc


PHREEQC


Aqua Chem


IPW


GEMPAK


GRADS


Hydrus2D


DSSAT


Topmodel


ArcIMS


Pro Essentials

4.0
3.9
3.3
3.2
2.7
2.6
2.5
2.4
2.4
2.3
2.2
2.1
2.1
1.9
1.9
1.8
Excel
ArcGIS/ArcView
FORTRAN
C/C++
Java
MS Access
Visual Basic
Matlab
SQL/Server
Modflow
Adobe Illustrator
HEC models
GMS, WMS, SMS
R
SWAT
Sigma Plot
1 =Never
use or
do not
find
2=Have
used, but
do not rely
5=Use
frequently
and find
indispensable
4=Use
often
3=Use
occasionally
and am
comfortable
with its use
Raw mean scores; n = 36
Software
-
CUAHSI

Rate the following software packages and programming languages with respect to
how important they are for hydrologic analysis in your research

Rate the following software packages and programming languages
with respect to how important they are for hydrologic analysis in
your research - CUAHSI
1.7
1.7
1.7
1.7
1.6
1.6
1.6
1.5
1.5
1.3
1.3
1.2
Surfer
SPSS
SAS
HSPF
S-Plus
GRASS
Visual Modflow
Mathematica
PostgreSQL
Tecplot
Groundwater Vistas
Kaleidagraph
1
=Never use or
do not find useful
2
=Have used, but
do not rely on this
5
=Use frequently
and find
indispensable
4
=Use often
3
=Use occasionally
and am comfortable
with its use
Raw mean scores; n = 36
Please rate each software functionality with
respect to the priority for including this functionality
in a hydrologic information system
-

USU

Rate the software functionality with respect to the priority for
including in a hydrologic information system
4.9
4.2
4.2
4.1
3.8
3.6
3.5
3.3
2.9
2.8
Data storage and retrieval
Visualization of spatial data
Visualization of time series data
Building relational links
Efficient coupling with 3rd party analysis software
Presentation to non-technical audiences
Development of publication quality figures
Numerical analysis
Multivariate statistical analysis
Univariate statistical analysis
1
=Low priority
5
=High priority
Raw mean scores; n = 18
Rate the software functionality with respect to the priority for including
in a hydrologic information system - CUAHSI
4.5
4.4
4.2
4.1
4.1
4.1
4.1
4.0
3.9
3.9
3.9
3.9
3.7
3.4
Store and retrieve digital products from a hydrologic
digital library
Include GIS data on terrain, soils, land cover, geology,
stream networks
Allow connection to hydrologic models
Include information from weather and climate models,
and Nexrad
Design metadata and develop tools for preparing it
Support intelligent searching for hydrologic data,
models, reports and papers
Include remote sensing information
Automatically harvest hydrologic observation data from
agency websites
Couple hydrologic fluxes among atmosphere, surface
and groundwater
Provide access to real time hydrologic and weather
data
Allow downloads from this database in delimited ascii
format
Synthesize national, state and local data into a single
space-time database
Allow connection to statistical analysis systems
Include modeling capabilities for flow and transport
1
=Low priority
5
=High priority
Raw mean scores; n = 36
Independence
-

USU:

Please indicate the following on a scale of 1
-
5, where
1=not important, 5=essential

all computer systems
0
2
4
6
8
10
12
1
2
3
4
5
Mean: 4.53
independently of 3rd party software
0
2
4
6
8
10
12
1
2
3
4
5
Mean: 3.82
HIS software should work
on all computer systems

HIS software should work
independently from any 3rd
party software

Independence
-

CUAHSI:

Please indicate the following on a scale of 1
-
5, where
1=not important, 5=essential

HIS software should
work on all computer
systems (Windows,
Linux, Mac, Unix)

HIS software should
leverage commercial
software systems (e.g.
ArcGIS, SAS)

All
0
5
10
15
20
25
30
1
2
3
4
5
Mean: 4.03
leverage
0
5
10
15
20
25
30
1
2
3
4
5
Mean: 3.46
Open Source
0
5
10
15
20
25
30
1
2
3
4
5
Mean: 4.09
HIS should be
implemented using open
source software

List data standards and file formats that you
think CUAHSI should adopt
-

USU


NetCDF


check the ALMA convention for Land
Surface Modeling


Grib


USGS


NASA
-

hdf










Are there data for which CUAHSI should
define standards
-

USU


Standards are important, but I do not know what they all are.



Streamflow, Precip, Landuse, Topography, Landcover



Unpublished data in technical reports must be protected, esp when
graduate theses and dissertations are involved


Of course, mostly in terms of denomination of the variables. Check
ALMA standards as an example.


Time series of any sort should have a standard that controls how
they are stored and distributed (this is primarily so that qualifying
comments or metadata are not lost).


Data should probably be available in a variety of export formats.


EPA storet
-

have good chemical data AND flow from the same
sample site. Otherwise it is pretty useless.

List data standards and file formats that you
think CUAHSI should adopt
-

CUAHSI


Time series


Space time grids


Some image formats


Hydrologic Observations


ISO


HDF


NetCDF


NASA DIF


GeoTiff


XML based formats


At least conversion from difficult data formats


Flat files


Data model for time series needs more development


How should I know? I’m just a user.


Any raster grid supported by ESRI


Ascii, shapefile, MS Access, XML


XML, XML, and XML

Are there data for which CUAHSI should
define standards
-

CUAHSI


Probably


Transboundary datasets


Sediment transport


Domain specific data


whatever is going to be
offered to users


Adapt existing standards


In situ observational data


such as SCAN
network data


Metadata format and ontology structure


Vector data, raster data, image data

Rate the ease of access to hydrologic data and information
3.7
3.6
3.3
3.3
3.1
2.9
2.9
2.8
2.8
2.6
2.5
2.5
2.1
2.0
1.8
1.5
USGS Historical Streamflow
USGS Real Time Streamflow
National Elevation Dataset
SNOTEL data
National Land Cover Data
National Hydrography Dataset
NCDC Precipitation
PRISM Precipitation Data
LANDSAT Satellite Imagery
STATSGO soils data
SSURGO soils data
USGS National Geology data
EPA STORET Water Quality Data
NEXRAD Radar precipitation
University of Washington Gridded Meteorological
Data (Maurer, et al., 2002, J. Climate 15, 3237-
NCEP North American Regional Reanalysis
(NARR) climate data
1
=Low
5
=High
Raw mean scores; n = 18
USU

Rate the ease of access to hydrologic data
and information
-

CUAHSI

3.8
3.8
3.8
3.8
3.5
3.4
3.1
3.0
3.0
2.9
2.9
2.9
2.8
2.6
2.6
2.6
2.6
2.5
2.5
2.5
2.4
2.3
USGS Real Time Streamflow
National Land Cover Data
National Elevation Dataset
USGS Historical Streamflow
National Hydrography Dataset
USGS Hydrologic Landscape Regions
SSURGO soils data
STATSGO soils data
PRISM Precipitation Data
USGS National Geology data
NCDC Precipitation
Groundwater level
LANDSAT Satellite Imagery
Water quality/Chemistry
University of Washington Gridded
Meteorological Data (Maurer, et al., 2002, J.
SNOTEL data
EPA STORET Water Quality Data
Real-time weather and Nexrad data from
Unidata
NCEP North American Regional Reanalysis
(NARR) climate data
Water use
NEXRAD Radar precipitation
Evapotranspiration
1
=Low
5
=High
Raw mean scores; n = 36
Rate the priority for data inclusion in HIS
4.8
4.8
4.7
4.6
4.4
4.4
4.4
4.2
4.2
4.1
3.9
3.9
3.9
3.9
3.7
3.7
National Elevation Dataset
USGS Historical Streamflow
NCDC Precipitation
National Hydrography Dataset
National Land Cover Data
SNOTEL data
USGS Real Time Streamflow
EPA STORET Water Quality Data
NEXRAD Radar precipitation
PRISM Precipitation Data
STATSGO soils data
SSURGO soils data
LANDSAT Satellite Imagery
University of Washington Gridded
Meteorological Data (Maurer, et al.,
USGS National Geology data
NCEP North American Regional
Reanalysis (NARR) climate data
1
=Low
5
=High
Raw mean scores; n = 18
Other data mentioned

Water Use/Diversion
data
-

2

ET, Groundwater level,
Water
Quality/Chemistry,
Energy balance
components (latent and
sensible heat)



USU

Rate the priority for data inclusion in HIS
-

CUAHSI

4.7
4.7
4.6
4.6
4.6
4.6
4.6
4.5
4.5
4.5
4.5
4.4
4.4
4.4
4.4
4.4
4.3
4.3
4.2
4.0
4.0
3.8
National Land Cover Data
Groundwater level
NCDC Precipitation
USGS Historical Streamflow
Water quality/Chemistry
National Hydrography Dataset
NEXRAD Radar precipitation
EPA STORET Water Quality Data
USGS Real Time Streamflow
SNOTEL data
USGS National Geology data
National Elevation Dataset
Water use
Evapotranspiration
PRISM Precipitation Data
USGS Hydrologic Landscape Regions
SSURGO soils data
STATSGO soils data
LANDSAT Satellite Imagery
NCEP North American Regional Reanalysis (NARR)
climate data
Real-time weather and Nexrad data from Unidata
University of Washington Gridded Meteorological
Data (Maurer, et al., 2002, J. Climate 15, 3237-
1
=Low
5
=High
Raw mean scores; n = 36
Other National Datasets you think
should be considered
-

CUAHSI


Corps of Engineers Stream stage/discharge


WRRI


Institutes


Reservoir storage/elevation/chemistry (Corps of Engineers, Bureau of Reclamation)


National orthophotography


Higher resolution HUCs


Urban/anthropic/water
-
related infrastructure


General geology


Population density


Various NRCS/WCC & NOHRSC data


NADP precip chemistry


Stream sediment data


Hydrogeologic data (K, Chem, etc.)


Snow courses


Vegetation maps


US Census/population data


Ecological data/ecoregions


Agricultural census data

Local Datasets that you think
should be included
-

USU


Geomorphic/Channel Characteristics, (.shp)


Fish Distribution and abundance (Excel)


Other biotic data on streams( Excel)


Abiotic data on streams (Excel)


LIS (Gribs)


U Washington (NetCDF)


Unidata


Real time river diversions and reservoir elevations (txt)


Real time water quality monitoring data


Real Time climate/weather observations (like Mesowest)


Local government records indicating land use changes


US govt Population Census data


Meteorological and edaphic state variable data from CSI loggers


Precipitation (ASCII, Excel))


Soil Moisture (ASCII, Excel)


Soil temperature (excel)


Soil characteristics (ascii)


Meteorological data (wind speed, humidity, air temperature, solar radiation) (ASCII)



Local Datasets that you think
should be included
-

CUAHSI


Borehole data


geologic electric and
geologic logs


locations


Seismic reflection lines


NPDES sites


Geologic surface profiles with original
data points


Groundwater raw water chemistry


Irrigation diversions


Pressure, moisture, concentration
(SQL)


USGS FRI


Forest growth data


National rangeland condition data


US breeding bird survey


Airborne Lidar


filtered bare earth
and first return


High resolution orthorectified aerial
photographs


Various mountain datasets


RAWS & other mesonets



Precipitation isotope data


Calsil (data system), Cdoc (data
system)


Weather Mesonets


High level land use


High resolution topography


Land management practices


Water management practices


Model input for specific watersheds


Met and flux tower


Realtime water quality data


Diversions, water management, water
rights


Water use


GAP analysis

Rate the importance of the following roles of an HIS
4.8
4.4
3.8
3.7
3.7
3.6
3.6
3.4
3.1
Retrieval of relevant National, Community, and
Hydrologic Observatory datasets
Uploading, archival and sharing of hydrologic data
with collaborators and the CUAHSI community
Source or repository of educational information
(e.g. online lectures, seminars or educational
modules)
Source or repository of information necessary for
the interpretation of HIS data (e.g. what do soil
moisture measurements mean for a user not an
Development of community data models and
standards for data representation
Interfacing of hydrologic datasets in standard
format with 3rd party analysis software
Sharing of models, software and hydrologic
simulation systems
Development and management of community
models and software
Software system for hydrologic analysis
Raw mean scores; n = 18
1=Do not
include
this
2=Low
priority for
this
capability
3=Good
idea,
include if
possible
4=Important
, HIS should
place high
priority on
this
5=This is
an
essential
role for HIS
USU

Rate the importance of the following roles of an HIS - CUAHSI
4.7
4.6
3.8
3.8
3.5
3.5
3.5
3.4
2.8
Retrieval of relevant National, Community, and
Hydrologic Observatory datasets
Uploading, archival and sharing of hydrologic data
with collaborators and the CUAHSI community
Interfacing of hydrologic datasets in standard
format with 3rd party analysis software
Development of community data models and
standards for data representation
Source or repository of educational information
(e.g. online lectures, seminars or educational
modules)
Software system for hydrologic analysis
Sharing of models, software and hydrologic
simulation systems
Source or repository of information necessary for
the interpretation of HIS data (e.g. what do soil
moisture measurements mean for a user not an
Development and management of community
models and software
Raw mean scores; n = 36
1
Never use
or do not find
useful
2
Have used,
but do not
rely on
5
Use frequently
and find
indispensable
4
Use often
3
Use occasionally
and am comfortable
with its use
Additional Comments/Suggestions


Transboundary approach should be included into the HIS project (e.g., all
watersheds along the US/MX border are binational on both surface and GW
hydrology


Need tools to easily register raster data at different scales (e.g., MODIS,
Landsat, orthophotos, NED, gridded climate)


Consider uncertainty


Consider anthropic influences


Prioritize ingestion and dissemination of data, methods for data
visualization, manipulation and, analysis


Need attention to ease of use, intuitive interfaces, responsiveness, etc.


Don’t forget the hydrogeologists, hydrochemists, and stream
geomorphologists


Is there more interest in accessing unfiltered data than model results?


Each observation needs quality information (adequacy of time series data
model)


Priority on simple data model for hydrologic observations with derivatives for
groundwater, streamflow, etc. Priority 2 = data fusion and assimilation
technology


Needs


integrated models, data to models, standard models, software for
analysis of model output and uncertainty