Portal_Software

righteousgaggleData Management

Jan 31, 2013 (4 years and 9 months ago)

285 views

SPARRO Group, University of Regina

1

Portal Software:

Browser
-
based Monte Carlo

Zisis Papandreou

University of Regina

GlueX Collaboration Meeting

JLab, December 11
-
13, 2003

SPARRO Group, University of Regina

2

Outline


Grid2003 Conference


Grid Services


Regina Portal


Application framework, Globus


Meta queuing


Collaboration tools:


GlueX Forum, Database Administrator

SPARRO Group, University of Regina

3

Grid2003 Conference
(Ed, Zisis)


Phoenix, AZ, November 16
-
17, 2003


Concurrent with SC2003 (8,000 hackers)


Interactive and coached tutorial:


How to Build a Grid Service using the
Globus Toolkit
®

3 (Argonne people)


Written in Java


Built (JDK) and run on Ed’s RH9.0 linux laptop

SPARRO Group, University of Regina

4

GT3 Architecture


Core:

Grid Services


Security:

SSL, X.509


Base:

Job Services,
Index Services, RFT


Data:

Replica
Management
(data subsets)


Other:

foreign grid
services

OGSA

SPARRO Group, University of Regina

5

javac Math.java

java org.apache.axis.wsdl.Java2WSDL

-
P MathPortType

-
S MathService

-
l http://localhost/ogsa/services/core/first/MathService

-
n http://first.core.gt3tutorial/Math gt3tutorial.core.first.impl.Math


package gt3tutorial.core.first.impl;

public interface Math

{

public int add(int a, int b);

public int subtract(int a, int b);

public int multiply(int a, int b);

public float divide(int a, int b);

}


Creating a Grid Service


Write WSDL directly


Generate WSDL from
interface language (Java)


Decorate it


Generate stubs


Implement it

Web Services Description Language

SPARRO Group, University of Regina

6

Exercises accomplished


Grid Services / GT3


FileShare Service
deployment


File sharing with
identifiers


Adding service data


Register with
community index


Lifetime management


File searching


Security


Transience: create
and destroy
FileShares


Grid Service
Publishing

SPARRO Group, University of Regina

7

Demonstration:

FileShare Grid Service


(Ed Brash)

SPARRO Group, University of Regina

8

Regina Portal: Beefed up


Glasgow:
job submission, review (PHP)


New features:


ColdFusion Application Framework (CFML)


User registration & authentication


Integration with GridPort TK, Globus


Upload XML Geometry and compile


Experiment or commit executables to db

https://www.phys.uregina.ca/GlueX/

SPARRO Group, University of Regina

9

Demonstration:

Regina Portal

(Zisis Papandreou)

SPARRO Group, University of Regina

10

Grid Computing


“The Grid”:

distributed computing infrastructure (both
software and hardware) that provides access to high
-
end
computational capabilities


G
rid computing enable
s

the shared use of computational
resources such as data archives and simulations.


R
esource owner
s

offer

resources subject to constraints


R
esource consumers

also place constraints on properties

SPARRO Group, University of Regina

11

Queuing, Scheduling &

Meta Scheduling

(Aadila Soobrattee)


Regina
openSpace

cluster (50 CPUs) is governed by
OpenPBS

local queuing software


Need
capabilities of scheduling workload among a
computer grid where local scheduling software and local
policies may differ
:
meta
-
scheduler


Schemes:


Centralized: surface info, locals dispatch jobs


Hierarchical: meta and local share info; no flexible re
-
routing


Distributed:
scheduler/dispatcher model; flexible, scalable

SPARRO Group, University of Regina

12

Distributed Scheduling

Meta-
scheduler
Local
Scheduler/
Dispatcher
Local
Scheduler/
Dispatcher
Local
Scheduler/
Dispatcher
Jobs
Job flow
Information flow

M
eta

scheduler
s

at every site


J
obs submitted to meta

scheduler where job originates


M
eta

schedulers query each other

to collect instantaneous load
info
;
job is transferred to the site with the lowest load


Dagman, Condor/G, Nimrod/G, GT3/Platform,

Silver/Maui

SPARRO Group, University of Regina

13

Silver/Maui


Silver provides grid scheduling services for clusters using
OpenPBS
, can operate on all major
UNIX

operating systems
and is capable of operating with or without
Globus

Jobs
Jobs
Jobs
Meta-Scheduler
Local Scheduler
Meta-Scheduler
Local Scheduler
Meta-Scheduler
Local Scheduler
Silver:


reduce
s

average job queue
time

b
y enabling access to a
larger set of computer
resources


intelligently decompose
s

jobs
into multi
-
system sub
-
jobs

to
improve job turnaround time


manage
s

site
-
dependent job
environment and resource
requirements


support
s

jobs which require
co
-
allocated resources which
expand multiple systems


support
s

jobs too large to run
at any single site by spanning
jobs across multiple sites


http://www.supercluster.org/

SPARRO Group, University of Regina

14

1. User logs into Globus and
gets

Globus
credential

2. User submits job from site A via sqsub
client

3.
sqsub

translates job to global job and
stores job and credential in SILVER's
global queue

4. SILVER determines list of feasible
resources based on resources requested

5. SILVER queries Globus and determines
machines
that
support the specific
credential

6. SILVER queries each feasible resource
and obtains matrix of resource availability
times

7. SILVER determines resource
associated with earliest job completion
time (site B)

8. SILVER reserves resources on site B

9. SILVER stages input data and
executable to site B using Globus file
transfer (gssftp)

10. SILVER translates global job to RSL
and submits job to Globus with 'run on site
B' request

11. Globus translates RSL job to resource
manager command language and submits
the job to resource manager on site B.

12. MAUI detects job
,

executes
it

on
reserved resources

13. job completes

14. MAUI creates accounting record,
removes reservation, and updates SILVER

15. Output data is staged to requested
destination using Globus file transfer

SPARRO Group, University of Regina

15

Collaboration Tools


GlueX Forum

(at UConn)


http://zeus.phys.uconn.edu/forums/gluex.org/index.php


Web
-
based email forum (written in PHP)


Category
-
based email, threads, moderators


PostgreSQL

Database Administration


pgAdminIII (win & unix)


Remote connection to db servers


Create/delete tables, enter data, view data

SPARRO Group, University of Regina

16

What’s next?


Expand portal further:


Install Meta Queuing System at Regina


Web/Grid Services (document uploading, indexing and
searching)


XML Data Transfer (Web Distributed Data Exchange


WDDX)


Communications:


Teleconferencing:
AT&T


Videoconferencing:
VRVS

or
Polycom


Email threaded discussions:
GlueX Forum