GregorVL - Ogce

quaggahooliganInternet και Εφαρμογές Web

5 Φεβ 2013 (πριν από 4 χρόνια και 4 μήνες)

165 εμφανίσεις

Rochester Institute of Technology

Experiment Management with
Microsoft Project

Gregor von Laszewski

Leor E. Dilmanian


http://www.cyberaide.org

08:30:25

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

1

Outline


Introduction


Cyberaide.org


Management Requirements of e
-
Science


Microsoft Project for Managing Tasks


Extending Microsoft Project for e
-
Science


Conclusion


References


Acknowledgements


08:30:25

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

2

Introduction
-

Background


E
-
Science

is
computationally or data
intensive science, carried
out in a distributed
computing environment.



Cyberinfrastructure

is a
research environment
supporting advanced data
acquisition and information
processing services over
the internet.



Experiment Management

is the management of a
large number of
experiments over
Cyberinfrastructure

08:30:25

3

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

Introduction


Motivation


There is a high entry barrier into Grid computing.



Experiment Management on the Grid or Cyberinfrastructure
is a complicated affair.



Issues include:


Application Design


Scheduling of Large Scale Resources


Orchestration of Activity


Monitoring Execution and Quality of Service


Data Acquisition


08:30:25

4

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

eScience Experiment Management

von
Laszewski’s


Meta Computer



Workflow & resource utilization,
fault tolerance

CoG Kit


Workflow
Abstractions

CoG Kit


Grid Ant


CoG
Experiment


CoG Karajan

Cyberaide


Shell


Project


Experiment
Management


Super
-
Gram

3/18/2013

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

5

Queuing

Systems

Globus

(Condor,Unicore, …)

Social Networks

Clouds

Cyberaide

08:30:25

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

6


is a framework
used to simplify
Cyberinfrastructure
development
.


Microsoft Project


Is used for planning a project.


Workflow



a set of operations which contain the following:


Task



a unit of work.


Dependency



among two tasks, used to specify ordering
.


Resource



people or machines used to carry out work.


Assignment /Mapping


of resources to tasks, to execute the
workflow.


This makes it a job

08:30:25

7

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

Microsoft Project

Microsoft Project is used to by project managers to manage
resources and tasks. Shown above: a
Finance and Accounting
System Implementation

project.

08:30:25

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

8

Source: Microsoft Project

E
-
Science Requirements


Focus on Science


Well known, familiar user
interface


Easy to Navigate


Effectively manage large
volumes of information


Advanced features:


Time based workflows
with soft deadlines.


Calendar & resource
views, toolbars, forms &
graphical indicators…


Integrated User Experience


Design


Easily specify a distributed
application.


Execution


Seamless scheduling and
execution on the Grid.


Monitoring


Progress & Quality of
Service.

08:30:25

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

9

Cyberaide Project


Cyberaide contains a prototype called
Cyberaide Project
.


It uses the Microsoft Project software package as an
elementary component.


Based on the methodology of project management
software.


It is used to…


Design a high level application.


Execute the workflow on the Grid.


Track progress and Quality of Service during execution.

08:30:25

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

10

Cyberaide Project

08:30:25

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

11

Microsoft Project

Console Window

Proxy Initialization

Manage Workflow

Task Table

Gantt Chart /

Track Workflow

Implementation


Technologies


Visual Studio Professional 2008


C# and Visual Basic


Microsoft Project 2007 Professional (client)


Microsoft Project Primary Interoperability
Assembly


Java CoG Kit


Apache CLI

08:30:25

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

12

Internal Design

08:30:25

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

13

The Cyberaide Project Class



Console application written in C#



Controls execution



Command registrations


Command Interfaces



Uniform way to specify a
command


Command Classes

addTask, delete, addResource, etc.


Integration with other
Cyberaide components

An easy client interface
provides access through
a mediator service to a
variety of different
services.

3/18/2013

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

14

MS
Project

Ruby

Python

Java

Mediator & Abstractions

TeraGrid

Amazon

Cloud

Faceboo
k

iGoogle

Social

Networks

Globus

GPIR,
NWS, …

Cloud

Grid

Client

Services

Javascript

C#

Commands


Workflow Manipulation:


Add/Delete/Edit task


Add/Delete Dependency


Add/Remove Resource


Workflow Execution


Run
-
all


Import/Export in JSON format


Load (will also merge)


Save

08:30:25

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

15

Example
-

Starting

3/18/2013

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

16

Example


Design a Project

3/18/2013

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

17

Example


Execute & Observe

3/18/2013

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

18

Example
-

Result

3/18/2013

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

19

Example
-

Result

3/18/2013

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

20

Summary


Task Table


Resource Mapping
-
> Jobs


Integration with TeraGrid


through CoG Kit to TeraGrid


We have are developing a Grid mediator

(e.g. a
Super Gram Service)

3/18/2013

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

21

08:30:25

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

22

Conclusion


Cyberaide Project can be used to design applications in e
-
Science


Supports the process of developing, executing and monitoring
workflows for the Grid



Familiarity, simplicity and user friendliness are the major
considerations


Useful for the casual Grid user familiar with or using Microsoft Project



Spend less time learning software, and more time making
discovery


Increases productivity



laszewski@gmail.com

08:30:25

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

23

References

[1] G. von Laszewski and L. E. Dilmanian, “e
-
Science Project and Experiment Management with Microsoft Project," Rochester
Institute of Technology, Rochester, NY 14623,
laszewski@gmail.com. [Online]. Available:
http://cyberaide.googlecode.com/svn/trunk/papers/
08
-
project/vonLaszewski
-
08
-
project.pdf

[2] G. von Laszewski and M. Hategan, “Grid Workflow
-

An Integrated Approach,” in Technical Report., Argonne National
Laboratory, Argonne National Laboratory, 9700 S. Cass Ave., Argonne, IL
60440, 2005. [Online]. Available:
http://www.mcs.anl.gov/gregor/
papers/vonLaszewski
-
workflow
-
draft.pdf

[3] I. Taylor, E. Deelman, D. Gannon, and M. Shields, Eds., Workflows for e
-
Science: Scientific Workflows for Grids. Springer,
2007, iSBN: 978
-
1
-
84628
-
519
-
6.

[4]
J. Frey, T. Tannenbaum, I. Foster et al., “Condor
-
G: A Computation
Management Agent for Multi
-
Institutional Grids,”
Proceedings of the Tenth IEEE Symposium on High Performance Distributed Computing (HPDC10), 2001.

[5] G. von Laszewski, J. Gawor, P. Lane, N. Rehn, M. Russell, and K. Jackson, “Features of the Java Commodity Grid Kit,”
Concurrency and Computation: Practice and Experience, vol. 14, pp. 1045

1055, 2002. [Online]. Available:
http://www.mcs.anl.gov/gregor/papers/vonLaszewski
--
cog
-
features.pdf

[6] G. von Laszewski, B. Ruscic, K. Amin, P. Wagstrom, S. Krishnan, and S. Nijsure, “A Framework for Building Scientific
Knowledge Grids Applied to Thermochemical Tables,” The International Journal of High Performance Computing
Applications, vol. 17, no. 4, pp. 431

447, Winter 2003. [Online]. Available:
http://www.mcs.anl.gov/gregor/papers/vonLaszewski
--
knowledge
-
grid.pdf

[7] “Grid Workflow :: Workflow Composition Tools,” Web Page. [Online]. Available:
www.gridworkflow.org/snips/gridworkflow/space/Workflow+Composition+Tools

[8] G. von Laszewski, “Java CoG Kit Workflow Concepts,” Journal of Grid Computing, Jan. 2006,
http://dx.doi.org/10.1007/s10723
-
005
-
9013
-
5. [Online]. Available:
http://www.mcs.anl.gov/gregor/papers/
vonLaszewski
-
workflow
-
taylor
-
anl.pdf

[9] G. von Laszewski, A. J. Younge, X. He, and F. Wang, “GridShell: Interactive Task Management for Grid and Cluster
Computing,” (submitted for review), Sep. 2008.


Acknowledgements


NSF NMI


NSF DDDAS

3/18/2013

Service Oriented Cyberinfrastructure Lab, http://grid.rit.edu

24