OGCE-Overview-SciDAC-TG09

judgedrunkshipΔιακομιστές

17 Νοε 2013 (πριν από 3 χρόνια και 6 μήνες)

93 εμφανίσεις

OGCE Overview: Portals,
Services, Workflows,
Gadgets, and Tags

Marlon Pierce (IU), Suresh Marru (IU), Gregor von Laszewski
(RIT), Mary Thomas (SDSU), Nancy Wilkins
-
Diehr (SDSC), Jeff
Sale (SDSC), Maytal Dahan (TACC), Rion Dooley (TACC), and
Wenjun Wu (UC)


Funding: NSF SDCI, TeraGrid RP and GIG (synergistic)


Links for More Information


Website:
http://www.collab
-
ogce.org


News/RSS/Blog:
http://collab
-
ogce.blogspot.com


Email:
discuss@ogce.org


SVN Code Repository:
http://ogce.svn.sourceforge.net/viewvc/ogce/



And get tar.gz’s


Source Forge:
http://sourceforge.net/projects/ogce

Project

Description

Status

Grid Portal

Packaged, one step installation of
Grid portlets into
Tomcat+Gridsphere

Stable, V2.5
for TG09

Axis Services

One step install of Resource
Discovery and Prediction Services.
More services to come.

Stable, V1.0
for TG09.

Workflow Suite

Support for scientific workflows,
based on LEAD tools. Suresh will
describe next.

Dev. Release
for TG09

GTLAB

JSF Tag Libraries for building Grid
portlets and gadgets

Dev. Release
for TG09

JavaScript COG

(Cyberaide)

JavaScript libraries for building Grid
gateways.

Dev. Preview
for TG09

Gadgets, Open
Social Portals

Open Social containers, gadgets,
OAuth, OpenID

SVN, will build

Incubator


projects

Pylons portals, metascheduling
services, bio
-
gadgets, etc.

SVN if you dare

Tutorial will focus on the bold
-
faced items.

OGCE and Gateways


We develop and package software for use by TeraGrid
Science Gateways and other resources


BioVLAB use OGCE tools to run on Amazon


A lot of this comes from active Gateways.


Information Services (GPIR, QBETS): TeraGrid User Portal


Workflow tools: LEAD


Resource Discovery Service, File Browser Applet: TGUP, GridChem


SIDGrid, OLSG


We contribute codes back to these projects.


Gateways


OGCE

Software


Preview: Workflow, Open
Social Gadgets, and Tab
Libraries

WRF
-
Static
running on
Tungsten

OGCE Workflow Tools (Suresh Marru)

Open Life Science Gateway Bio
-
gadgets

(Wenjun Wu)

Job Submission History gadget

Blast gadget

ClustalW gadget

GTLAB Gadgets

Code

Base

GTLAB Portlets

Code

Base

Tomcat

Web

Server

Tomcat Web Server

+

Portlet Container

iGoogle

Client

Aggregator

Compile and deploy into server

Decorate with bridge
and container jars, xml
config files.

GTLAB gadgets
can also run as
portlets with no
code changes by
using the JSF
portlet bridge.

GTLAB MyProxy Example

<h:form id="form">


<h:outputText value="Username:"/>


<h:inputText value="#{resource.username}" required="true"/>



<h:outputText value="Password: "/>


<h:inputSecret value="#{resource.password}" required="true"/>



<h:outputText value="MyProxy Server: "/>


<h:inputText value="#{resource.myproxyHostname}" required="true"/>



<o:submit id="submit" value="Submit" binding="#{builder.body}"





action="#{builder.junkAction}" >


<o:multitask id="multi" persistent="true" taskname="#{resource.taskname}">


<o:myproxy id="mypr" hostname="#{resource.myproxyHostname}"




lifetime="2" password="#{resource.password}" port="7512"




username="#{resource.username}"/>


</o:multitask>


</o:submit>

</h:form>

Packaging, Building and Testing


All builds are designed to be
self contained


Use Apache Maven 2.0


Download includes everything you need.


Portal, Axis Services, and Xregistry all
build nightly
on the
NMI Build and Test facility at UW.


17 Linux platforms, Mac PPC, and Mac X86


Java 1.5


Apache JMeter
test suite
for the portal.


Run against your installation


Automated tests nightly

JMeter test suite

OGCE Portal

Components for Science Portals


OGCE is founded on the principal that portals
should be built out of
reusable parts
.


Key standard in our first phase: the JSR 168 portlet
specification.


Portlets can run in multiple containers


uPortal, Sakai, GridSphere, LifeRay, etc.


Allows us to build Grid specific components and
deploy along side
third party components


Sakai collaboration tools, RSS feed readers,
contributed portlets, etc.


Future (?):
Open Social
compliant
Google Gadgets


Facebook apps

Downloading and Installing OGCE
Portal Software


You can have a basic portal up and running in a
few minutes.


Installation process:


Update properties in pom.xml to match your
Grid.


Run “
mvn clean install



That’s it

OGCE GPIR portlet can interoperate
with TeraGrid and your own GPIR
services.

Manage TeraGrid MyProxy
credentials with the OGCE
ProxyManager portlets.

OGCE file management client
portlets interact with TeraGrid
GridFTP servers.

General purpose batch and interactive job
submission to GRAM, WS
-
GRAM is supported.

Dashboard Portlet

19

Check on your jobs with a simple dashboard
portlet.

Queue forecasting portlets work with the
NWS QBETS to predict wait times and
deadlines.

Condor and Condor
-
G

OGCE IFrame Portlet can be
used to integrate external
sites.

Under the Hood: Building Components


Portlets can be built with our
Apache Velocity bridge
.


Still some interest in this.


“Template” or “steal this code” approach.


OGCE portlets use the
Java COG Abstraction Layer
to
access Grid services.


Provides a buffer over different Grid toolkits and
versions.


Provides ability to compose graph
-
based workflows.


Tutorial in itself


We have also developed
Grid Tag Libraries and Beans
(GTLAB) to simplify development process.

Some OGCE Services

Information Web Services and build framework

Resource Discovery Service


Extracted from the TeraGrid User Portal


Contributed by Rion Dooley, TACC


Axis2 Web service


Integrates information from INCA, GPIR services.


What machines are in my Grid


Which ones are actually running?


Works with TeraGrid deployed services, but you can use
it with your own information service installations.

Resource Prediction Service (RPS)


Predicts an optimal set of resources for running
scientific applications


Predicts a set of resources on which the sum of
data
transfer time, queue wait time and compute time
for the
application will be minimum


Data transfer time:
Uses input data size and NWS info


Queue wait time
: Uses QBETS info


Compute time:
Uses application performance info


Axis2 web service with a MySQL backend


RPS is a subset of the Fault Tolerance and Recovery
service used in LEAD and VGrADS

Packaging Services


Everything builds with one command


mvn clean install


We use Apache Maven 2 for builds.


Structured, modular builds and consistent layouts


Build a single module:


mvn clean install

f somemod/pom.xml


We include Apache Tomcat in the download, so all you need
is Java and MySQL.


Suitable also for the NMI build and test system.


Want the Javadoc? mvn javadoc:javadoc

Upcoming Services


XMC
-
CAT
metadata cataloging service with JSDL sample
implementation.


See also our demo on Wednesday.


Swarm Web Service
for mass job submission and
management.


http://escience2008.iu.edu/sessions/SWARM.shtml


GPIR Web Service
for information on your Grid. This is
being repackaged; older packaged version (but same
code) is already available.


Workflow suite services will be integrated in the future.