MOCCA - Distributed and High Performance Systems Group

bricklayerbelchedInternet και Εφαρμογές Web

5 Φεβ 2013 (πριν από 4 χρόνια και 7 μήνες)

200 εμφανίσεις

1

An ADL
-
based Support for Component Applications
on the Grid

Tomasz Bartyński, Eryk Ciepiela, Joanna Kocot,
Przemysław Pelczar,
Maciej Malawski
, Marian Bubak

Institute of Computer Science &

ACC CYFRONET AGH

Kraków, Poland



2

Background: MOCCA Component
Environment based on CCA and H2O



Scalable to different environments


lightweight platform


dynamic, pluggable, reconfigurable


Facilitated deployment on shared resources


Static virtualization by using H2O kernel as a daemon


Dynamic virtualization using a pool of transient containers


Communication adjusted to various levels of coupling


Offered by RMIX library of H2O


Parallel extensions for CCA: multiple ports


Facilitating programming



Composition in time: Python, Ruby Scripting


High
-
level: Virolab/GridSpace

programming environment


Supporting multiple languages


Integration with Babel

(in progress)


Interoperability with grid standards


Web Services


Grid Component Model (ProActive/Fractal)


Adapted to unreliable Grid environment


reconfiguration of connections,

locations, bindings


fault
-
tolerance support:

migration and checkpointing

(future work)


3

Motivation



Component
model


Suitable for the distributed environment


Allow
s

to manage complex applications


Support
ed

by standards


Support
s

scientific applications


Facilitat
es

programming


Running component application involves:


Resource discovery


Deploying components


Connecting components


Destroying components


Additional requirements:


Multiple component connections


Dynamic changes of resource pool


Now these have to be done manually

4

Requirements


Facilitate running component
-
based applications
in Grid environment: automate process using a
Manager


A
model

of distributed component application to
enable management


Provide a
description

of application model


Enable developer to compose complex, scalable
applications, that would adapt their architecture
to the environment


5

MOCCAccino Concept


ADLM (ADL for MOCCAccino)


XML based language
for:


Describing types and number of components and their connections


Concept of hierarchical component groups


Facilitating modification of the quantitative properties of architecture
-

parametrization


Optional information to specify resources


Metadata:
Hints for deployment of components (whether they are
computation intensive or communication intensive).


Information related to aspects of deployment (e.g scheduling, policies of
adaptation)


Application Manager


responsible for:


Discovering available kernel pool


Planning optimal location of components


Deploying components in specified kernels


Connecting components


Management of the application

6

ADLM: Main Elements


component
-
class



defines a type of component


component
-
group



instances of component class


independent, but their connections follow the same pattern


weight

specifies their computational needs


connection



connects groups of components:


includes communicational needs


specifies relations


qualifier

specifies multiplicity and enables referring to each instance from group


<
component
-
class
name
=
"Ping"
classname
=
"foo.bar.Ping"

codebase
=
"http://foo.org/my
-
app.jar"
>


<
provides
>


<port
name
=
"MyGoPort"
classname
=
"foo.bar"
/>


</
provides
>


<
uses
qualifier
=
"list"
>


<
port
name
=
"pongs"
classname
=
"foo.bar.PongPort"
/>


</
uses
>

</
component
-
class
>

<component
-
group

name
=
"ping"
component
-
class
=
"Ping"
weight
=
"5"
>


<
connection
usesPort
=
"pongs"
qualifier
-
attribs
=
"length=2"

providesPort
=
"PongPort"
weight
=
"5"
shared
=
"false"
>


<
component
-
group
name
=
"pongs"
component
-
class
=
"Pong"
weight
=
"5"
>


<connection
usesPort
=
"zonks"
qualifier
-



attribs
=
"keys=
left
;
right
"
providesPort
=
"ZonkPort"
weight
=
"5
” ”
shared
=
"false"
>


<
component
-
group
name
=
"zonks"
component
-
class
=
"Zonk"
weight
=
"5"
/>


</
connection
>


</
component
-
group
>


</
connection
>

</
component
-
group
>

Pong
Ping
list
map
index
: 0
key
:

left

index
: 1
key
:

right

Zonk
Pong
Zonk
Zonk
Zonk
map
key
:

left

key
:

right

Pong
Pong
Ping
Ping
list
map
index
: 0
key
:

left

index
: 1
key
:

right

Zonk
Zonk
Pong
Pong
Zonk
Zonk
Zonk
Zonk
Zonk
Zonk
map
key
:

left

key
:

right

7

Why ADLM?

Similar approaches:


Fractal ADL


Composite components vs. component groups


IDL for Corba Component Model


No support for hierarchies


A broad overwiev of other ADLs can be found at
http://www.sei.cmu.edu/architecture/adl.html


None of existing ADL approaches satisfies
all
MOCCAccino requirements, such as:


building
hierarchical

component
application
s


describing components' characteristics


e.g.
computational or communicational needs


specifying dynamic behaviour of the application

8

MOCCAccino Architecture

The Manager is built from these components

9

MOCCAccino Architecture: Conversion
between ADLM and AOM


ADLM Unmarshaller


Parses the ADLM file


Converts it to Application Object Model (AOM)


Creates application components graph


Component Graph Builder (CGB)


Validates and resolves the application model


Returns component graph, assembled from plain component instances


ADLM Marshaller


Serializes the application to an XML (ADLM) file


Deployment Planner


Creates Deployment Plan


assigns H2O kernels to each application's MOCCA component


D
ecisions basing on the fallowing heuristics:

-

computation repels components

-

strongly repel
l
ed components in different H2O kernels

-

com
m
unication attracts components



strongly attracted components in the same kernel


Kernel Information Provider (KIP)


Gives information about available H2O kernels (used by Deployment Planner)


Two modes:


Static mode


list of kernels specified in a file


Dynamic mode


kernels information obtained from HDNS


Application Deployer


Deploys the application (following the Deployment Plan)


Returns a handle to the deployed application

10

Test Application


Simple test
application


6 component
classes


Service


Input


Output


Master (root
component)


Worker


Subworker

AOM Snoop visualization

11

Prototype Status


Full support for static use case


ADLM


Can be used to describe even complex static applications


Implementation


Deployment Planner


supports simple algorithms (round
-
robin,
random)


no optimization


Kernel Information Provider


retrieves information from HDNS


ADL Unmarshaller / ADL Marshaller


support this version of
ADLM, must be adjusted when ADLM changes


Other architecture components support static use cases and can
be easily adjusted to dynamic applications

12

Current and Future Work


Support for altering application structure at run
-
time


adapt to changes in environment


Add information defining policy for using new resources to ADLM


Add support for notifications to Kernel Information Provider


Add support for notifications to Application Deployer


Test on applications that will have diversed
requirements


Domain decomposition


Branch
-
and
-
bound application


Genetic algorithms/strategies


Scientific application...

13

Integration with CoreGRID


CCA
-
GCM interoperability


collaboration with INRIA
-
Sophia


Creating glue between CCA nd Fractal


ProActive


MOCCA interoperability


Research group on deployment


Colaboration with INRIA
-
Rennes


Usage of ADAGE tool for deployment

14

References


Malawski

M.
, Kurzyniec

D.
, Sunderam

V
.
:

MOCCA


towards a
distributed CCA framework for metacomputing
. In Proceedings of
the 10th International Workshop on High
-
Level Parallel
Programming Models and Supportive Environments (HIPS2005),
2005. IEEE Computer Society


M. Malawski, M. Bubak, M. Placek, D. Kurzyniec, and V. Sunderam.
Experiments with distributed component computing across grid
boundaries. In Proceedings of the HPCGECO/CompFrame
workshop in conjunction with HPDC 2006, Paris, France, 2006..


Kurzyniec D. and others:
Towards Self
-
Organizing Distributed
Computing Frameworks: The H2O Approach.
Parallel Processing
Lett, 2003.


Architecture Description Languages
:
http://www.sei.cmu.edu/architecture/adl.html


MOCCA:
http://www.icsr.agh.edu.pl/mambo/mocca


15

Thank You

16

MOCCAccino basis
: CCA and H2O


Common Component Architecture
(CCA)


Component standard for HPC


Uses and provides ports described in SIDL


Support for scientific data types


Existing tightly coupled (CCAFFEINE) and
loosely coupled, distributed (XCAT)
frameworks


H2O


Java
-
based d
istributed resource sharing
platform


Providers setup H2O kernel (container)


Allowed parties can deploy pluglets
(components)


Separation of roles: decoupling


Providers from deployers


Providers from each other


RMIX: efficient multiprotocol RMI extension

Container
Provider host
Deploy
Lookup
& use
Provider
Client
<<create>>
B
A
Provider
<<create>>
A
B
Container
Lookup
& use
Client
Deploy
Provider
,
Client
,
or
Reseller
Provider host
Traditional model
Proposed model
Container
Provider host
Deploy
Lookup
& use
Provider
Client
<<create>>
B
B
A
A
Provider
<<create>>
A
A
B
B
Container
Lookup
& use
Client
Deploy
Provider
,
Client
,
or
Reseller
Provider host
Traditional model
Proposed model
http://home.agh.edu.pl/malawski/MOCCA
-
Seminarium
-
Nov06.ppt


17

MOCCAccino basis:
MOCCA


A basic component framework, CCA implementation


Each component is a separate pluglet


Dynamic remote deployment of components


Components packaged as JAR files


Security: Java sandboxing, detailed access policy


Using RMIX for communication


efficiency, multiprotocol interoperability


Flexibility and multiple scenarios


as in H2O


MOCCA_Light: pure Java implementation


Java API or Jython and Ruby scripting for application asssembly


http://www.icsr.agh.edu.pl/mambo/mocca

http://home.agh.edu.pl/malawski/MOCCA
-
Seminarium
-
Nov06.ppt


18

MOCCAccino Architecture: Manager


Uses other
MOCCAccino
components to
control the
application
deployment


Launches the
deployed
application


Monitors and
manages the
application

19

MOCCAccino Architecture: KIP Details


Kernel Information
Provider


A

M
OCCA

component
running in H2O kernel


P
rovides information
about res
o
urces to the
Manager and
D
eployment

Planner


C
ommunicates with
HDNS (
JNDI
) to retrieve
needed information


Can also be configured
to read a static list of
kernels from file