KUG 2009 Delphi and .Net

musicincurableΔιαχείριση Δεδομένων

31 Ιαν 2013 (πριν από 4 χρόνια και 9 μήνες)

167 εμφανίσεις

What's New in Hydstra

Peter Heweston
-

Aug 2009

KUG 2009

Strategic Drivers


.Net


64 bit Windows


Kisters TSM


Water Act 2007


Modelling and Data Use


System Enhancements

KUG 2009

.Net

KUG 2009

.Net


Microsoft is pushing to .Net


Language independence


VB, C++, C#, J#, Delphi


Rich base class library (I/O, graphics, database, XML etc)


Security


Web orientation


Interactive web sites using ASP.NET, Silverlight, etc


Web services


Simplified deployment


No 'DLL hell'


Different versions can run side by side


Optimisation for different CPUs is possible

KUG 2009

Delphi and .Net


Delphi 2007 provided dual mode compilation
-

Win32 and
.Net


Hydstra 9.6 and Hydstra 10 are compiled with Delphi 2007


Microsoft hired the lead Delphi designer


Anders Hejlsberg (Turbo Pascal, Delphi etc)


C# conceptually very similar to Delphi


We had virtually completed a .Net port of Hydstra


Using conditional compilation to preserve single source code



KUG 2009

Delphi and .Net


May 2008 Delphi sold to Embarcadero Technologies


Embarcadero announced two products


Delphi 2009
-

continues the Delphi path for Win32


Delphi Prism
-

new .Net compiler


Problem
-

no VCL for .NET in Prism at this stage


We would need to redevelop all GUI objects
-

a major task


Hydstra will remain Win32 for a while longer, until the fog
clears


Win32 development continues apace


KUG 2009

TSM Integration


For a while we were stuck in our TSM integration because
of compiler limitations


Recently Embarcadero announced unmanaged exports
facility


This allows a .Net DLL to expose itself to the Win32 world
as a native Win32 DLL


Allows for object links between Hydstra and TSM


Integration effort can re
-
commence


KUG 2009

64 Bit Windows

KUG 2009

64 Bit Windows


Many new servers come loaded with 64 bit Windows


All device drivers need to be rewritten for 64 bit


32 bit apps run under 64 bit just fine


16 bit apps do NOT


Hydstra 9 does NOT run under 64 bit windows


Hydstra 10 runs fine, as a 32 bit app


64 bit Delphi coming in 2010 (maybe)

KUG 2009

V10 32
-
bit Tools and Utilities


All 16 bit command line tools and utilities have been
upgraded / replaced and are now 32
-
bit


All tools deal correctly with long file names


Hydstra V10 runs under 64
-
bit Windows


Users should not notice any difference unless they are
writing advanced batch jobs


Scripts in INIPATH may need attention


KUG 2009

Long File Names and Spaces


Hydstra V10 should uniformly deal with long file names
everywhere


Hydstra runs under a long directory root






but



No Spaces


Like C:
\
Program Files or C:
\
Documents and Settings


Every job would need to quote every parameter and they don’t

KUG 2009

Finding 16 Bit References


HYFIND.HSC is delivered in Hydstra V10


Available for 9.6


ask us


Download from http://hydstra.kisters.com.au/index.htm


Searches

INIPATH for obsolete tools


Help file
documents

tool changes


Help file
suggests

workarounds


* Patchable to V9.6

KUG 2009

TSM

KUG 2009

TSM Integration


Time Series Manager is the Kisters server that will
eventually underlie all Kisters products


TimeStudio, Wiski, Hydstra, AquisNet, Belvis, …


Much work in Hydstra is being done to align concepts with
TSM


Basic data tree


Datasources


Variables


etc

KUG 2009

All Database Tables in DBF


Having tables scattered across multiple folders makes it
difficult to manage progress towards modern server
-
based
databases and .Net


All tables now live in DBFPATH


UPGRADE copies them there for you


All users get all tables
-

simplifies UPGRADE


Scripts and other commands that access obsolete paths will
display warnings, and will very likely require attention


Obsolete paths are:


SECTPATH, RATEPATH, GAUGPATH, QDBPATH, RESPATH,
GDBPATH, LDBFPATH, OPSPATH, OSYSPATH


HYFIND option [V10_BADPATHS] will
find

them

KUG 2009

Alphanumeric Variables


Phase One


V10 sees the introduction of phase one of alphanumeric
variables.


Don't need to do anything immediately


Alphanumeric variables fit better with TSM concepts


Provide larger variable space for biologicals


During the life of V10 users will need to decide about
alphanumeric names, subvariables


Separate presentation

KUG 2009

Support for Corporate Databases


Hydstra V10 should run over mainstream databases


Oracle


PostgreSQL


SQL Server


Not sure about all script tools


Time series remain in TS files


Perl data sources only work on TS files.


Will cost more $$$ to migrate


Need friendly user to help us finalise testing

KUG 2009

V10 System Enhancements

KUG 2009

Look and Feel


All forms and dialogs respect Windows Vista themes:



KUG 2009

HYXPLORE Enhancements


HYXPLORE

has several enhancements


Run HSC jobs by name from Run box (.HSC optional)


Run HYMANAGE jobs by name from Run box


History of Run box preserved over runs


List locked TS files


Test path permissions


Open HYDLOG.TXT


Prevent accidental saving (and loss) of menus


Doesn't try to register .HSC


XDOCs
-

external supplied documentation


user extensible to access other documentation


not site related

KUG 2009

HYMANAGE Enhancements


HYMANAGE

has many usability enhancements


Drag column borders in browse mode


Consistent colours in form and browse mode


Preserves filters on multiple tables within a manage


Quickfind on primary key or sorted column


Replace field value for filtered records (constants only)


Enhanced Statistics summarises text fields


Classify by unique values

shows count of all values


Load saved queries from command line


HYMANAGE RATINGS /QUERY=RATEPER.FLOWRATINGS

KUG 2009

HYMANAGE menus now more standard


The menu bar in HYMANAGE now conforms more closely
to Hydstra and Windows conventions


Reports are now created via File | Print


Hyconfig and Hydebug under Tools


Fonts under View

KUG 2009

Live Code Update


This feature has been available since 9.6


A TS=3 user can add a new code from any coded field


Add a 'wrong value' and tab out of field


Dialog appears to help you add a new code


KUG 2009

New CSH Format Supported


CSH=CSV with Headers


Exported by HYMANAGE and HYDBUTIL


Imported by HYMANAGE and HYDBUTIL


Easier than XML and still allows data to possibly cross
Hydstra versions


Example

KUG 2009

SVRIMP Enhancements


SVRIMP has been reworked to significantly improve
reliability and performance


Marshals incoming files by site to reduce Hydstra file access


Processes 10
-
20 files/sec sustained


Preserve TS file edits (usually)


Much improved backlog processing


Restructured

INI tab


Suggest new settings


Folder Scan Limit=5000


Batch Processing Limit=100


Shut down with SVRIMP /K for backups and replications

KUG 2009

SVRIMP Enhancements


Version 2 CSV format supported


#V2, site,
datasource
, variable, time, value, quality, datatrans,
maxgap, DECUM, comment


Send files to different datasources


V1 format still supported


KUG 2009

Ordered Site Lists


Some site list expressions preserve order and dupes


GROUP, FILE, PERLLIST,CONCAT


Allows for run of river lists etc


Use SORT if you don’t care


SORT(FILE(c:
\
temp
\
sites.csv))


Warning
some existing lists may now allow duplicates


Wrap in SORT() to fix

KUG 2009

HYSTNS from Clipboard


New Perl script HYSTNS.COPY.PL copies HYSTNS list
from clipboard


Almost any delimiter is recognised


for example copy the following text


hydsys01,

410776 / / 201235a##310996b

456722

301876,hydsys01


Then run
HYSTNS

SORT(PERLLIST(HYSTNS.COPY.PL))

KUG 2009

Text Editing of GROUPS


You can now edit GROUPS lists

using a
text editor


Support copy and paste


More convenient than adding them

row
-
at
-
a
-
time

KUG 2009

VARIABLE


VARIABLE

table updated to prepare for Alphanumeric
variables in V11


Proposed Alpha ID field preparing for V11


DOMAIN field to classify variables


Parameter Screens
have new variable

tree object to look up variables


By Domain


by Report Type


by Units


by word (with quicksearch)


Report Decimals


KUG 2009

Web style plots from HYPLOT


HYPLOT is now able to produce “bare” plots for use in web
pages.


Allows web designers to create titles and labels using HTML


May do other ‘widget’ style plots as required

KUG 2009

HYDATSUM


Show Data Holdings


HYDATSUM has been enhanced to
show

WQ and GW
holdings

KUG 2009

HYCLIPIN


Import from Clipboard


Import time series or database data from Excel


Export database data to Excel


Allows Perl processing step


Easy to build
standard

data entry sheets for time
-
series or
tables


Useful for
Excel

data

manipulation of tables


KUG 2009

HYAUDIT Enhancements


Post
-
processing

script HYAUDIT.XMLSPLIT.PL XML splits
XML into single
files


HYAUDIT.SUMREP.HSC

gathers selected files together
into a colour
-
coded intelligent HTML report


Click on errors in
red

to see full text


Open offending site in WB, HYMANAGE


Rerun HYAUDIT for that site when

error has been fixed

KUG 2009

HYAUDIT Helpers


HYAUDIT.STNINILIM.HSC

computes limits from data and
loads them into STNINI


Should only be run on clean data


HYAUDIT.STNINISET.HSC

loads constants into STNINI


Ready for use in HYAUDIT


KUG 2009

HYREP CHECK


Checks

TS files for problems


Zero length files


Corrupt TS file (invalid flag byte)


Unregistered variable




Unregistered quality code


Invalid datatrans code


Index points to a block which does not exist


Overlaps in index


Also leaves a
CSV

file in TEMPPATH HYREP.CSV


Useful in highly automated systems

KUG 2009

HYWOTSUP


Scans a variety of log files and reports errors


Configure where to look for log files


svrimp=&hyd
-
logpath.svrimp
\


hydlog=&hyd
-
dbfpath.hydlog
\


path01=&hyd
-
temppath.hydlog.txt


Run it daily to find SVRRUN
problems

KUG 2009

HYDBSQL


Perform SQL queries on (Foxpro) tables


Supports quite complex one
-
line SQL statements


SELECT STN,MEAS_DATE,M_GH,FLOW FROM GAUGINGS
WHERE FLOW=(SELECT MAX(FLOW) FROM GAUGINGS
GROUP BY STN) GROUP BY STN


Example

KUG 2009

Changes to the Format of Binary TS Files


We have made some changes to the format of binary time
series files


V10 TS files
not

readable by V9


V9 files
are

readable by V10


Related agencies better upgrade together


Values are now stored and read back rounded
exactly
to
your nominated precision


0.125m, stored to 0.001m precision, will come back as 0.125m,

not .012496431


Alphanumeric variables will be fully released in V11


We are defining the binary tags in advance, as part of V10



KUG 2009

New Lossless Compression

V9

V10

KUG 2009

Storage space consequences


Reduced the space required by Hydstra to store typical time
series data. Expected savings:


Typical water level data (with point
-
removal compression)


5
-
15% reduction


Typical water level data (no compression)


25
-
35% reduction


Typical rainfall data


20
-
25% reduction


DIPNR archive:


15% reduction


Lots of small blocks:


size may increase

KUG 2009

0
50
100
150
Hydstra Pty Ltd.
HYSECPIC V20 Output 05/05/2009
Cross section status report
Site
HYDSYS01 Hydstra Test Station
Height
4.272
metres
Velocity
[ ]
Time
04:00_29/08/1974
Flow
629.3
m3/s
Cease to flow
0.017
metres
Flood Reference Levels
Below flood classification
Minor
Moderate
Major
Distance in metres (left bank to right bank)
Gauge height in metres
1982 Flood (9.2 m)
Victoria River Bridge
Walking Trail
Boat Jetty
0
2
4
6
8
10
HYSECPIC


Enhanced to allow coloured ground:


KUG 2009

Daily River Report


As a project for DWE we implemented DWRRR, the Daily
River Report


Reports by catchment in stream order for the last 7 days


Configurable via INI file


Rebuilt ever 30 mins in the morning


FTP output to web, email process report to manager


Example

KUG 2009

Hydstra/WEB


We have released Hydstra/WEB and have a couple of
clients up and running:


One in Australia


One in Italy


KUG 2009

TSADMIN manage


TSADMIN

manage centralises all TS related tables


Used by a TS=2 or TS=3 data manager


Variables


Qualities


Groups


Generalised Logger configuration


Digitiser related tables


ARCHSIZE


etc


KUG 2009

GWREP


GWREP produces a nice HTML report from GW tables


Example

KUG 2009

HYPERUPD


Writes actual datasource for TS files


Keep separate PERIOD entries for A and T files


Update for composite datasources much faster


Example

KUG 2009

TS File Locking


Time
-
series files are now logically locked while they are
being written


Locks last over multiple operations, preventing interruption


Should make SRVRUN triggers more robust


HYXPLORE option shows
locked

TS files


Lock files exist in TS directory alongside TS files


C:
\
hydstra
\
systems
\
g
\
HYD
\
DAT
\
HYD
\
HYDSYS01.A


C:
\
hydstra
\
systems
\
g
\
HYD
\
DAT
\
HYD
\
HYDSYS01.AX


C:
\
hydstra
\
systems
\
g
\
HYD
\
DAT
\
HYD
\
HYDSYS01.A.LOCK(READ)


C:
\
hydstra
\
systems
\
g
\
HYD
\
DAT
\
HYD
\
HYDSYS02.A


C:
\
hydstra
\
systems
\
g
\
HYD
\
DAT
\
HYD
\
HYDSYS02.AX


C:
\
hydstra
\
systems
\
g
\
HYD
\
DAT
\
HYD
\
HYDSYS02.A.LOCK(READ)

KUG 2009

HYCSV.DETAILS.PL


Postprocessor on HYCSV


Documents variables and qualities output


Publishing HYCSV data on the web


Sending HYCSV data to other users


Example

KUG 2009

HYSETTIM


HYSETTIM.BAT sets current date and time parameters as environment
variables:












e.g. MKDIR %PTMPPATH%%DATEDIR%


DATEDIR=2009
\
07
\
29

DAY=29

DAYNAME=WED

DOW=3

HOUR=22

MINUTE=40

MONTH=07

MTHNAME=JUL

QUARTER=3

SECOND=38

YEAR=2009

KUG 2009

Multiline Parameter Screens


On multiline programs like HYPLOT, HYCSV etc, '=" means
same site as above


KUG 2009

Instruments


New STOCKTAKE date field in INSTREG


record the last date the instrument was seen


INSTCAL has TIME as well as DATA so multiple
calibrations allowed on one day


HYINSCAN

provides scanner support for portable battery
powered scanners


Opticon OPN 2001


Visit site


scan labels


return to office


paste into HYINSCAN

KUG 2009

New Perl Version


Hydstra 10 distributed with Strawberry Perl 5.10


Many new modules


Support for Perl Critic


HYCRITIC.HSC for
testing

Perl code

KUG 2009

Perl Training Notes


In May 2009 we provided advanced Perl training


Course material included in
Help

file


Includes notes on new Windows scripting capabilities

What's New in Hydstra

Peter Heweston
-

Aug 2009