JAS3 Current Status and

coordinatedcapableSoftware and s/w Development

Nov 4, 2013 (3 years and 9 months ago)

211 views

JAS3


Current Status and
Prospects

by Victor Serbo, SLAC

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Contents


Introduction to JAS3 (Java Analysis Studio)


Basic: Opening Files, Plots, Printing, etc.


Web Browser, Editor, Java Compiler, Scripting


Miscellaneous: Spreadsheet, Configuration Save/Restore,
Preferences, etc.


Analysis


AIDA


Abstract Interfaces for Data Analysis


JAIDA


Java Implementation of AIDA


AIDA JNI


using JAIDA from C++


Analysis with JAIDA and JAS3


JAS3 Plugins


What is Plugin, examples


Demo: AmbientPlugin and Geant4 Plugin


Demo: JAIDA Web Interface

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAS3 Overview


A completely new version of JAS


Design based on
Application Shell
, into which many (optional)
modules can be plugged


Highly customizable

for different application domains


HEP/Astrophysics/Other


Data analysis/Online Monitoring/GRID analysis


Experiment/User specific modules


Modules can be updated independently of shell


Possible to release bug fixes fast


Includes support for programming in many languages


Scripting
: Pnuts, Python, ….


Command prompt


Java (compiled)


Analysis (histograms, tuples, fitting) based on
AIDA standard


Not technically backwards compatible with JAS2


But migration is straightforward.

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAS3 Lightning Tour


Tour designed to
give you an
overview of the
capabilities of JAS3,
you can try them
out for yourself:


http://jas.freehep.org/jas3

Welcome Page, gives initial
info and links to example
scripts and programs

Memory
monitor

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Opening Files

Use file
menu

Drag from
explorer

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Plot Manipulation and Printing

Can send
individual
plots or full
page
direct to
printer

Or save as PS,
EPS, PDF, SWF,
SVG, PNG, GIF…

Or
copy/paste
into Word,
PowerPoint
etc
.

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Java Editor, Compiler and Loader

Built
-
in Java
compiler

Tree
shows
loaded
programs

Built
-
in
editor for
writing
analysis
code

JAS3 allows any Java program to be
loaded. This example “main routine” is
taken directly from the AIDA manual

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Miscellaneous Features

Save/Restore
configuration

User
Preferences

Plugin
Manager

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAS3 Spreadsheet


Simple spreadsheet plugin


for


Displaying results


Calculations


Simple Plots


Supports reading/writing


.csv files


Excel

files


Cut/Paste with Excel
etc


Coming Soon…


Scripting interface


GUI for building plots


User defined functions


Java, scripting


ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAS3 Scripting


JAS3 has multi
-
language OO scripting support


Command line, Console, Editor


Major components (e.g. AIDA) have scripting interfaces


Currently have plugins to support


Pnuts



syntax almost identical to Java, fast, well documented
and feature complete


Python



Use Jython (the Java implementation of Python)


Limitation: can not call Python libraries written in C


More scripting languages can be added


not restricted to Java implementations (e.g. could use C
-
Python, JPE)


ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Pnuts Language


Currently support Pnuts scripting language


Complete and well documented


http://javacenter.sun.co.jp/pnuts/doc/guide.html


Fast (although not as fast as compiled Java)


Syntax very similar to Java; can compile script into Java
bytecode or convert it to Java code


Can easily call compiled Java classes from scripts


best of both
worlds

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAS3 Scripting II

Can also
write and run
scripts

Console allows
direct interaction
with scripting
language

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

AIDA
-

Abstract Interfaces for Data Analysis


AIDA is a standard set of interfaces for creating and manipulating
histograms, n
-
tuples and related data analysis objects, like Plotter,
Fitter, etc.


AIDA defines storage XML format.


It has been created cooperatively by a group of developers working
on high
-
energy physics data analysis tools. The goal of the AIDA
project is to provide the user with a powerful set of interfaces which
can be used regardless of which analysis tool they are using.



Only define “protocol” for analysis objects, no internal details


AIDA defines behavior, analysis tool provides implementation


C++ and Java versions of interfaces are generated from the AID
files.

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

AIDA Examples


IHistogram1D

-
IAxis axis()

-
void fill(double x)

-
void fill(double x, double w)

-
double mean()

-
double rms()

-
double binEntries(int index)

-
double binError(int index)

-
….



For more details on AIDA,
see:
http://aida.freehep.org



IAxis

-
int bins()

-
double lowerEdge()

-
double upperEdge()

-
double binLowerEdge(int index)

-
double binUpperEdge(int index)

-
double binWidth(int index)

-
int coordinateToIndex(double x)

-
….


ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAIDA


JAIDA = Java implementation of AIDA


Complete implementation of AIDA interfaces


With a few exceptions


Support for plot styles is somewhat limited


Some missing features in ITree, like cp, mv


Uses Java “service provider mechanism” for dynamic loading of
services at Runtime


Can be used to add support for new fitters, stores etc.


Have defined “developer level” interfaces for IOptimizer, IStore etc


Uses Java Expression Library (JEL) for runtime compilation of
expressions (for tuple evaluators and functions)

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAIDA


Features continued…


Fitting


support for multiple fitting engines (optimizers)


UNCMIN (pure Java)


Minuit (needs minuitAdapter native library)


Support for multiple fitting methods


LeastSquares
,
Chi2
,
BinnedMaximumLikelihood

and
UnbinnedMaximumLikelihood



Stores


Read/Write support for AIDA XML format (with built
-
in gzip)


Read support for HBOOK/PAW files (needs hbookAdapter native library)


Read support for Root files (write support hopefully coming soon)


Future plans to add:


SQL interface, binary store format, text file support …


Plotting


Two implementations of IAnalysisFactory


BatchAnalysisFactory


dummy plotter, ignore calls to IPlotter


AnalysisFactory


Full plotter functionality


Multi
-
Threaded


plots update in real
-
time as they are filled


No need to call interact() or refresh()


Support for embedding plotter


In other Java GUI’s


In Java servlets for web based applications.


Graphics export in many formats including:


PDF, EPS, SVG, SWF, PNG, GIF, JPG


ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

AIDA
-
JNI


C++ interface uses AIDA
-
JNI

C++ program

AIDA

AIDA
-

JNI

AIDA

JAIDA (Java)


AIDA
-
JNI partially generated by AID, partly by hand


Does support full AIDA functionality

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Using JAIDA with Java

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Using JAIDA with Python

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Using JAIDA with C++

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Future Plans for JAIDA


Complete implementation of missing features


Continue to track improvements to AIDA standard


Add support for function arithmetic


Web Interface


Support for “Remote AIDA”


Support for more storage formats


Need good binary IO format .. either AIDA standard or roll our own



Future AIDA developers’ meetings

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAS3 and AIDA


JAS3 uses JAIDA
-

supports all AIDA functionality, including


Histograms (includes arithmetic, projections,
etc
.)


Clouds (unbinned histograms, scatterplots)


Plotter


Tuples


Fitting


AIDA interfaces allow for multiple fitters


Uncmin
--

pure java minimizer


Minuit
--

Fortran called by Java Native Interface (JNI)


IO


AIDA XML, PAW, Root


JAS3 supports user interaction with AIDA in three ways


Scripting (Pnuts, Python etc)


Compiled (Java) code


GUI


Plotting, Fitting, Cuts etc.

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Graphical Interface to AIDA

Histograms,
Clouds, Tuples
all presented in
AIDA tree

.aida files,
.hbook files,
.root files all
presented as
AIDA objects

Drag items onto
page, or use
(popup) menus

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Histogram

Profile

ScatterPlot

XY Data

(More appropriate for

smaller data sets)

Tuple Explorer
-

Plots

Works with any tuple,
read from file or
dynamically created

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Tuple Explorer


Define Columns

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Tuple Explorer
-

Cuts

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Tuple Explorer
-

Tabulate

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

To be used with record loop

Tuple Explorer


Record Source

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Record Sources

Opening record
(or event)
based files
causes the run
control toolbar
to appear

Supports random access
and “tagged” data sets
(mainly for event
displays)

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAS3 Plugins


JAS3 is based on the Freehep Application Framework, which
provides a basic application framework into which extension modules,
or "plugins" can be loaded.



The application framework provides general purpose functionality,
such as top level GUI (windows, menus
etc
.), help system, storing
and retrieving user preferences
etc
.



All of the data analysis specific functionality of JAS3 is provided by a
set of extension modules.



Modules communicate with each other via service interfaces and
notification events. JAS3 comes with a standard set of service
interfaces and notification events, but new plugins can also define
their own interfaces and events.


ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAS3 Plugin II


Plugin functionality:


Do some plugin
-
specific task


Use services provided by other plugins


Provide services to other plugins


Modify JAS3 menus and pop
-
up menus



“Core” Plugins


provide basic services


Simple: File Opener, Memory Status, Status Bar, …


More Complex: Web Browser, Plotter, Console, XMLIO, …



“Extensions”


separate packages


Included with JAS3 distribution


AIDA, TupleExplorer, JAVA Compiler and Editor, Spreadsheet, …


Specific to particular Experiment


BaBar Ambient DB Browser: AmbientPlugin, Geant4 Plugin, etc.


Partial Tutorial on writing a plugin:
http://jas.freehep.org/jas3/developers/tutorial.jsp

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAS3

(Java)



JAS3 Plugin Demo I


AmbientPlugin:


Access BaBar Ambient DB via CORBA


Present Time History as an AIDA object (DataPointSet)


BaBar

Ambient DB

CORBA
Server
(C++)

AmbientPlugin

(Java CORBA
Client)

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAS3

(Java)



JAS3 Plugin Demo II


Gean4 Plugin

Geant4
Application
(C++)

JNI Layer

Java RMI
Server

Geant4 Plugin

(Java RMI
Client)

AIDA Web Interface

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Goals


Provide access to Histogram plots without requiring any
software installation


Putting results on to the web for collaborators etc.


For example for viewing results of “quality control” plots


Viewing “live” histograms on the web for


Online monitoring


Monitoring progress of batch jobs


Add as much functionality as possible in a “generic” way
so that it can be used by others (with minimal work)


Complete “out
-
of
-
the
-
box” web application


Packaged as a .war file for use with any J2EE app server


(e.g. Tomcat)


Configurable via a simple .ini file (or better via a web interface)



ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Architecture

.root

.paw

.aida

Application Server

(Tomcat)

Web Application

JAIDA

AIDA

Servlet

SQL



Remote

AIDA

C++ App

Java App

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Features


Current Feature set


Display of “AIDA Tree” for selection of histograms


Displays plots as .png or .gif images in browser


Allows (limited) control of image properties


Allows (limited) control over plot properties


Allows comparison to reference plots (including Kolmagorov test)


Allows export of plots to PDF, EPS, SVG, SWF, GIF, PNG,
etc.


Planned features


Extend to support n
-
tuples as well as histograms


Dynamic definition of new columns and cuts (evaluators, filters)


Web based fitting (including function definition)


Support multiple plots per page


Thumbnail browser


Extend which plot options can be configured through web


Allow users to retain settings via cookies

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Current Interface

Can try from the web:

http://glast02.slac.stanford.edu:8080/
aida/welcome.html

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

Status


Currently released JAS3 version 0.7.3 (0.7.4 next week)


AIDA functionality is quite solid


Compiler, Loader, Record Loop all quite recently added,


Certainly still some rough edges


Documentation limited but available


Built
-
in example scripts and programs


Tutorial on web


If you used JAS2 you will find some functionality not yet
ported to JAS3


Remote (client/server) access to data.


3D Lego/Surface plots

ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAS3 Conclusion


JAS3 is a powerful and flexible Analysis Tool


JAS3 uses Plugin Architecture


Easy to add new functionality


Easy to fix problems with existing modules


do not have to wait
for new release of all libraries


JAS3 release 0.7.3 is out


try it!


It is still a beta version, so bugs happen


We need your input to make JAS3 better!


Report bugs, problems, suggestions at
http://bugs.freehep.org


Do not have to be on JAS team to be JAS developer


ideas/code for
general
-
purpose plugins are always welcome


Try writing JAS3 plugins for your own needs: for Experiment and
Home


ACAT03 1
-
5 December 2003


JAS3


Current Status and Prospects, V. Serbo

JAS3 Links, More Info


Report Issues


http://bugs.freehep.org/


JAS


Java Analysis Studio
-

http://jas.freehep.org


JAS3


http://jas.freehep.org/jas3


JAIDA


http://java.freehep.org/jaida/


AIDA


http://aida.freehep.org


FreeHEP
-

http://www.freehep.org



FreeHEP Java Libraries
-

http://java.freehep.org


WIRED


http://wired.freehep.org