A Dynamic Instrumentation Framework for Distributed Applications

compliantprotectiveΛογισμικό & κατασκευή λογ/κού

1 Δεκ 2013 (πριν από 3 χρόνια και 8 μήνες)

108 εμφανίσεις


A Dynamic Instrumentation Framework for Distributed
Applications

(M.Phil Transfer Report)



Denis Reilly

May 2002




Supervisors:

Prof. A. Taleb
-
Bendiab and Dr. Carl Bamford







Distributed Multimedia Systems Engineering Group

School of Computing and Ma
thematical Sciences

Liverpool John Moores University


2



CONTENTS


1.1

D
ISTRIBUTED
A
PPLICATIONS

................................
................................
................................
......

4

1.2

S
OFTWARE
I
NSTRUMENTATION

................................
................................
................................
..

5

1.3

I
NSTRUMENTATION
F
RAMEWORKS

................................
................................
.............................

5

1.4

D
EVELOPING AN
I
NSTRUMENTATION
F
RAMEWORK


T
HE
R
ESEARCH
P
ROBLEM

........................

5

2.1

F
OUNDATIONS

................................
................................
................................
............................

6

2.2

S
TATE OF THE
A
RT
D
EVELOPMENTS

................................
................................
...........................

7

3.1

A

P
RACTICAL
S
OLUTION

................................
................................
................................
.............

8

3.2

E
XTENDING
B
EYOND
O
THERS


T
OWARDS
I
NTELLIGENT
I
NSTRUMENTATION

...........................

9

4.1

I
NSTRUMENTATION
P
ARAMETERS

................................
................................
.............................

10

4.2

I
NSTRUMENTATION
M
ONITOR
S
ERVICES

................................
................................
..................

11

4.3

M
ONITOR
M
ANAGEMENT
S
ERVICES

................................
................................
.........................

12

5.1

O
VERVIEW OF
J
INI
T
ECHNOLOGY


C
ODE
M
OBILITY

................................
................................

14

5.2

D
YNAMIC
P
ROXIES

................................
................................
................................
...................

16

5.3

I
MPLEMENTING
M
ONITOR
S
ERVICES

................................
................................
........................

17

5.4

I
MPLEMENTING
M
ANAGEMENT
S
ERVICES

................................
................................
................

19

5.5

E
VALUATION

................................
................................
................................
............................

22

6.1

C
ONCLUSIONS AND
W
ORK
C
OMPLETED

................................
................................
...................

24

6.2

P
ROPOSED
W
ORK FOR THE
P
H
.D.

................................
................................
.............................

25


ABSTRACT

................................
................................
................................
................................
..........................

3

1.

INTRODUCTION

................................
................................
................................
................................
....

3

2.

INSTRUMENTATION


PAST AND PRESENT

................................
................................
.................

6

3.

INTELLECTUAL CHALLEN
GE

................................
................................
................................
..........

8

4.

DEVELOPM
ENT OF THE DYNAMIC I
NSTRUMENTATION FRAME
WORK

.........................

10

5.

IMPLEMENTING THE DYN
AMIC INSTRUMENTATION

FRAMEWORK

..............................

13

6.

PROGRESS REVIEW

................................
................................
................................
...........................

24

REFERENCES

................................
................................
................................
................................
...................

26


3

Abstract

Distributed applications have the capabilities to meet today’s enterprise computing
systems. They allow remotely located services and resources to be accessed and provide
redundancy and fault tolerance capabilities along wi
th an overall impression of a single
integrated computing facility. However, distributed applications are notoriously difficult
to develop and manage due to their inherent dynamics, which manifest as application
component and service configurations that ma
y change “on
-
the
-
fly”. Middleware
technologies dramatically simplify the development of distributed applications, but, until
recently, little attention has focused on combining software instrumentation techniques
with middleware technologies to understand
dynamic behaviour and assist with the
runtime management of distributed applications.

This research is concerned with the development of a
dynamic instrumentation
framework
, which provides support to measure and monitor distributed applications. The
frame
work adopts a
service
-
oriented

approach, which provides capabilities for runtime
insertion/removal of “intelligent” instrumentation services to measure and monitor the
complex application service groupings that exist within a distributed application. The
d
esign and development of the framework is underway and a prototype demonstrator
system has already been developed, using Jini middleware technology, and evaluated
using an existing Jini application. The proposed work for the Ph.D. will extend on this
proto
type to incorporate mechanisms, which describe how complex groups of application
services may be instrumented through an understanding of the dependencies that exist
with the groupings.

The report is structured as follows: section 1introduces the elements
and concepts that
feature in the research before stating the main research problem to be addressed. Section
2 provides a literature review of software instrumentation and current “state
-
of
-
the
-
art”
practice relating to distributed applications. Section 3 c
onsiders the challenges facing the
development of the dynamic instrumentation framework and identifies the novel
contribution of the current and future research work. Section 4 describes the design and
development of the instrumentation framework. Section
5 considers a prototype
implementation of the framework and describes an early evaluation of the
implementation through a case study. Finally, section 6 summarizes what has been
achieved so far, and what needs to be addressed following transfer to the Ph.D
.

1.

Introduction

Like any disciplined approach to tackling a problem, one must first be clear about the
elements and concepts that are central to the problem. This introductory section, defines
the terms: distributed application, middleware, software inst
rumentation and
instrumentation framework and ends by identifying the main research problem to be
addressed.


4

1.1

Distributed Applications

Emmerich, [1], defines a distributed application as: “a collection of autonomous hosts
that are connected through a co
mputer network with each host executing components and
operating a
distributed

middleware

to enable components to coordinate their activities
giving the impression of a single, integrated computing facility”. The distributed
middleware, or simply middlewar
e, plays a crucial role by providing APIs and support
functions that effectively bridge the gap between network operating system and
distributed application components and services. The development of distributed
applications is greatly simplified by middl
eware, but it is still a daunting task due to the
possibility of different middleware technologies and/or different protocols and the
dynamic behaviour inherent in distributed applications, which can give rise to
component/service reconfigurations that occ
ur “on
-
the
-
fly”.

Current middleware technologies, which include DCOM, CORBA, Enterprise Java Beans
(EJB), Jini and Web Services, all share common ground in that they allow clients to
access and use remote services via “stubs” or proxies, which are downloa
ded over a
network from a service provider to a service user or client. A proxy gives the impression
that a client has a local copy of the service, which it intends to use. The client proceeds to
use the service by invoking the methods specified by the pro
xy and the middleware
technology takes care of the routing of these invocations back to the actual service. By
doing so, middleware is seen to provide
location transparency

in that clients are able to
use services irrespective of their physical location. A
lso, if for some reason a service
should fail or cause problems then, through middleware, a suitable alternative can be
sought from elsewhere thereby providing redundancy and fault tolerance capabilities.

The components in a distributed application are sel
f
-
contained binary implementations,
which consist of one or more objects. Components communicate with each other through
connectors, which are implemented via software interfaces and distributed applications
are often described by a
component
-
connector

abs
traction. However, components also
provide application services, which form the basis of the alternative
service
-
oriented

abstraction of a distributed application, which regards the application as a
federation

of
services distributed over a network. This s
ervice
-
oriented abstraction to middleware
-
based distributed application development considers a service as a
logical

concept such
as a printer, or chat service that can be discovered dynamically by a client and used
according to a contract of use. In a non
-
trivial distributed application services are
arranged into logical groupings, or
complex entities
, were a grouping reflects the type or
nature of service provided.



5

1.2

Software Instrumentation

Software instrumentation
1

may be defined as additional softwa
re features, which are
developed specifically to measure the performance and/or monitor the behaviour of a
distributed application and, as such, play no role in the main processing undertaken by
the application. Until recently, little attention has focused

on the

instrumentation of
distributed applications, or more particularly on
instrumentation

as a middleware service
to monitor dynamic behaviour. This is not to say that instrumentation has gone unnoticed,
since it has been applied for sometime in softwar
e engineering to debug and test software
applications and also for performance monitoring. Traditional, instrumentation
approaches involved the insertion of additional software constructs at design
-
time, or
when the system was off
-
line, during maintenance,

to observe specific events and/or
monitor specific parameters. This
static

instrumentation can be used with distributed
applications, but only with limited success due to their dynamic runtime characteristics.
This suggests a need for
dynamic

instrumentat
ion that can be applied at
runtime

to
accommodate any architectural reconfigurations and hence provide a faithful
representation of the systems behaviour.

1.3

Instrumentation Frameworks

A refinement of the common definition of an application framework, giv
en in [2], to suit
instrumentation may be stated as: “an instrumentation framework is a collection of
instrumentation services and interfaces that embodies an abstract design for solutions to a
family of related instrumentation problems or tasks. An instru
mentation framework also
specifies a collection of contracts, or rules that specify how instrumentation services can
work together”. Some of these rules may be mere conventions and other rules may be
easy to follow because the framework provides suitable c
ode along with the interfaces.
For example, GUI frameworks provide default behaviour for GUI application
components (windows, menus, and dialog
-
boxes). If the default behaviour is not
replaced, a component is expected to follow the rules, such as implement
ing the correct
user interface guidelines of the underlying platform. Other rules may actually be enforced
by the framework, such as drawing routines that prevent an application from drawing
outside of its windows. Enforcement means that the framework prov
ides some services
only through safe code that guarantees the necessary invariants (e.g. The invariant
"drawing always occurs inside of the application's windows").

1.4

Developing an Instrumentation Framework


The Research Problem

The service
-
oriented abs
traction proves extremely useful for developing instrumentation
as services, in much the same way as any other application service, and serves as the
basis for the development of the dynamic instrumentation framework. The development
of such a framework ra
ises a number of questions, which include:




1

The term instrumentation is used henceforth to refer to “software instrumentation”


6

(1)

What parameters of an application need to be monitored?

(2)

What type of instrumentation is required to monitor these parameters?

(3)

How can this instrumentation be managed in relation to the application?

(4)

What are the ru
les and invariants necessary to provide easy to use generic
instrumentation that exhibits a degree of intelligence?

This report considers the first three of these questions and postpones the fourth question
until subsequent research has been conducted, sin
ce it raises issues, which have not yet
been considered in detail. However, the issues raised by the fourth question are discussed
in section 3 to highlight the challenges that lay ahead.

2.

Instrumentation


Past and Present

This section reviews the use o
f software instrumentation from its initial foundations to the
“state of the art” developments relating to distributed applications.

2.1

Foundations

Originally, instrumentation was used to debug and test applications that run on single
processor machines a
nd for analyzing the performance of real
-
time systems. The parallel
computing community later adopted instrumentation to debug, evaluate and visualize
parallel applications. More recently distributed application developers have recognized
the potentials of

instrumentation, used in a
dynamic

regime, to monitor and manage
today’s distributed applications.

Probably the earliest documented use of software instrumentation was that of
application
analyzers
, first considered by Satterthwaite, [3], for monitoring t
he performance of
Fortran programs. These application analyzers consisted of:

(1)

An instrumentation system

(2)

A monitoring and display system.

Application analyzers were used as part of a more comprehensive test environment, [4],
and were closely associated (eve
n integrated) with the compiler, through which they
could be switched on or off by a compiler directive. Two of the main problems of
application analyzers, as noted in [4], were: first, they relied on source code
instrumentation, which was not always possi
ble when the program relied on additional
pre
-
compiled libraries. Second, the instrumentation code often affected program
performance, which presented problems in real
-
time applications.

Significant advances in the use of instrumentation came from the para
llel computing
community through the development of support tools to debug, evaluate and visualize
parallel programs, [5] and [6]. These advances saw instrumentation applied in a much
more structured manner, which followed a “tools” or “library” based appr
oach, [7] and

7

[8]. Emphasis was placed on analyzing processes and the transportation and storage of
data, within a parallel application. Visualization tools, [9] and [10], were usually based on
GUIs developed using graphics libraries such as OpenGL, Tcl/Tk

and X/Motif that
provided the user with a consistent view of the application and its environment.
Performance, which is significant in parallel programs, was evaluated using
unobtrusive

instrumentation that did not carry an additional computational overhe
ad. The parallel
computing instrumentation tools were capable of synchronizing with applications and
providing limited interaction facilities, but they were still generally static in nature.

2.2

State of the Art Developments

The more recent “state of the a
rt” developments have adopted either the component
-
connector or service
-
oriented abstraction of a distributed application to provide dynamic
instrumentation capabilities. Sun Microsystems has made a significant contribution
through the Java Management Exte
nsion API (JMX), [11], which is an optional extra to
Java v1.3 that facilitates the instrumentation of Java
-
based applications. JMX uses
Management Beans

(
MBeans
), which are arranged into instrumentation and agent levels
to monitor distributed services. Th
e MBean approach is also used by the Openwings
community, [12], who adopt the component
-
connector view of a distributed application,
through which components are connected together using protocol specific connectors that
plug into ports, within the compone
nts, to facilitate synchronous and asynchronous
communications between components.

The DARPA funded initiative for
Dynamic Assembly for System Adaptability,
Dependability and Assurance (DASADA)

is actively investigating the use of software
gauges and probe
s to dynamically deduce component configurations and examine
distributed applications as an assemblage of components, [13], [14] and [15]. Reflective
techniques are used by Diakov
et al.

in [16] to monitor distributed component
interactions by combining CO
RBA’s
interceptor

mechanism together with Java’s thread
API to “peek” into the implementation of CORBA components at runtime.

More specific to Jini
-
based systems, the Rio
2

project, [17], has made a significant
contribution through an architecture that simp
lifies the development of Jini service
federations by providing concepts and capabilities that extend Jini into the areas of
quality of service (QoS), dynamic deployment and fault detection and recovery. Rio
makes use of
Jini Service Beans

(JSBs)
,
Monitor
Services

and
Operational Strings
, where
the latter are used to represent the collection of services and infrastructure components as
an XML document. Also of interest in Rio is the
Watchable

framework, which provides a
mechanism to collect and analyze prog
rammer
-
defined metrics in distributed
applications.

Further smaller
-
scale Jini
-
based contributions come from [18] and [19]. In [18] Fahrimair
et al.

describe the
Carp@

system, which is a reflective based tool for observing the



2

Rio is a Jini Communit
y [21] project


8

behaviour of Jini services.
C
arp@

combines the ideas of reflection together with Jini
services to provide a
meta

architecture that reflects on a Jini application.
Carp@

instruments a Jini application using a model based on the use of
Carp@ Beans

for
instrumentation that communicate th
rough channels (or connectors) and ports. In [19]
Hasselmeyer and Vo


describe a generic approach to instrumentation that effectively
instruments a Jini lookup service using Java’s dynamic proxies to trace service
interactions in a Jini federation. In [20]

Hasselmeyer extends this earlier work by
considering the management of
dynamic service dependencies

in
service
-
centric

applications.

3.

Intellectual Challenge

Distributed applications have many benefits, which meet the needs of today’s enterprise
computin
g systems, which include: facilities to access remotely located services and
resources, redundancy and fault tolerance, as well as flexibility and adaptability.
However, as mentioned previously, these benefits carry a cost of increased complexity
making di
stributed applications difficult to develop and manage. These difficulties arise
because distributed applications can change at runtime such that the set of components
and services and their interconnections and communications with clients can change from
one instant to the next. Distributed applications also allow different middleware
technologies (DCOM, CORBA, EJB, Jini and Web Services) and different network
protocols (TCP/IP and UDP) to co
-
exist, which can cause inconsistencies and
compatibility problem
s within a distributed application.

3.1

A Practical Solution

Dynamic instrumentation provides a practical solution to overcome the dynamic and
diverse technology difficulties, which is reinforced by the recent state of the art
developments mentioned above.

However, there is still scope for further development
because much of the state of the art dynamic instrumentation developments concentrate
on providing instrumentation services for measuring or monitoring specific parameters,
paying little or no attentio
n towards the instrumentation infrastructure and overall
integration of instrumentation into a distributed application. In short, there is a need for
an approach, which goes some way to addressing question (4), as stated in section 1.4, in
order to unify t
he individual advances and developments to provide a common
framework for describing, developing and integrating instrumentation.

The best effort along these lines comes from [15], which provides a draft proposal for the
DASADA Gauge Infrastructure
. [15] c
onsiders instrumentation as low
-
level probes and
higher
-
level gauges (constructed from probes) interconnected through probe and gauge
buses. [15] also provides definitions and specifications, so that users and application
developers can understand the rule
s and regulations relating to the use of instrumentation.
This type of complete approach to instrumentation, which encompasses the description,
development and integration of instrumentation, is encouraging, but there is still scope
for going even further
by considering some of the more fundamental issues relating to the

9

integration of instrumentation. One such fundamental issue is that of dynamic service
dependencies, which are a crucial pre
-
requisite to understanding: first, both the structure
and dynamic
s inherent in a distributed application and second, for developing rules,
which specify how instrumentation services can work together.

3.2

Extending Beyond Others


Towards Intelligent Instrumentation

Service dependencies, considered further in [20], gove
rn how the services within a
distributed application depend on each other as well as external (third party) services.
These dependencies must be resolved, understood and managed before a complex
grouping of services can be instrumented. Service dependencie
s are an important issue in
distributed applications and they impact on the redundancy, fault
-
tolerance and flexibility
of a distributed application. Dependencies may occur as one application service
depending on the availability of another. Alternatively,

they may occur as dependencies
on third
-
party services such as the availability of a web
-
server, or even network or
hardware services. All of the current state of the art dynamic instrumentation
developments place the onus of dealing with service dependen
cies on the developer. In
other words, the current developments provide instrumentation services, but leave it up to
the developer to decide what needs to be instrumented and how to organize and group the
instrumentation. The main contribution of this work

will extend that bit further beyond
other existing approaches by considering service dependency issues in conjunction with
instrumentation services to provide a framework capable of “intelligently” instrumenting
the complex entities within a distributed a
pplication.

From an architectural perspective, the framework intends to supplement the architectural
layers that constitute the concept of a distributed application with an additional
instrumentation service layer (shown as shaded in Figure 1 overleaf) tha
t can be used to
assist in the development and runtime measurement and monitoring of a distributed
application. The relative position of the instrumentation service layer means that
instrumentation services can observe application service/middleware servic
e interactions
as well as the interactions between each of the former and local operating systems (OS)
and virtual machines.


10


Figure 1: architectural layers of a distributed application

4.

Development of the Dynamic Instrumentation Framework

This section
considers the first three questions raised in the introductory section (Section
1.4), which are repeated below:

(1)

What parameters of an application need to be monitored?

(2)

What instrumentation is needed to monitor these parameters?

(3)

How can this instrumentation

be managed in relation to the application?

As mentioned previously, consideration of the fourth question exceeds the scope of this
report and is to be addressed after transfer to the Ph.D. Current work has focused on
distributed applications implemented u
sing Jini technology, [22], but many of the
concepts apply equally to other middleware technologies, particularly CORBA.

4.1

Instrumentation Parameters

The selection of a series of parameters depends largely on the abstraction of a distributed
application.

The abstraction used by Openwings and the
Carp@

system is based on
components, channels (or connectors) and ports. Through this abstraction components,
which consist of one or more objects, communicate through channels (or connectors) that
are plugged int
o ports on the components. The alternative abstraction, used in Rio and
[19], is the service
-
oriented abstraction that regards a distributed application as a
federation of services and clients were clients may be either “pure” clients or service
providers
themselves in a peer
-
to
-
peer regime.

The approach adopted for this work uses the same service
-
oriented abstraction as Rio,
and [19], which based the selection of parameters primarily on services (application

11

services and lookup services), but also took int
o account the distributed Jini application
as a single entity (i.e. the federation of services, pure clients and Java Virtual Machines
(JVMs)), as shown in Table 1. Although Table 1 does not provide an exhaustive list,
these parameters can provide useful i
nformation relating to performance and behaviour of
Jini applications. The parameters of Table 1 (with the exception of client/service access
patterns) are all regarded as “low
-
level” parameters. However, the re
-
factoring of these
parameters, to be conside
red in future work, would produce additional “higher
-
level”
parameters or “metrics”, which may be used by application developers to assess the
performance and behaviour of a distributed application.

Category

Parameters




Application Service

number of c
lients

dependent services

period of use before discarded

events generated

events received

member function access

values of attributes



Lookup Service

frequency of client access

frequency of service registration

number of services registe
red

size of serialized code

Physical location of service



Jini Application

number of services per group

number of groups

number of pure clients

number of JVMs

client / service access patterns

Table
1
: instrumentation
parameters according to category

Next is the consideration of question (2) concerning the instrumentation that is needed to
monitor these parameters.

4.2

Instrumentation Monitor Services

Within Table 1 there are four different types of parameter:

(1)

Informati
on parameters


such as the “id.” of a service or its physical location,
which often occur as strings and IP addresses.

(2)

Numeric data parameters


such as the number of clients or number of application
services registered with a lookup service.

(3)

Dynamic para
meters


such as the invocation of a service method, or specific
events generated and/or received by a service, which are event driven.


12

(4)

Complex parameters


such as client/service access patterns, which are obtained
by post
-
processing the previous paramete
rs.

In order to monitor the different types of parameter an
instrumentation

monitor

service

(or simply monitor service) is used for each application service to be instrumented. The
monitor service is implemented as a Jini service that is rendered “invisibl
e” to clients and
can make use of two instrumentation utility classes, namely
logger

and
analyzer
, which
are standard Java classes (not Jini services). The responsibilities of the monitor service
and logger and analyzer utilities, with respect to the diffe
rent parameter types, are:

(1)

Monitor service


“connects” with an application service or a middleware lookup
service and monitors dynamic parameters and invokes logger and/or analyzer
utilities.

(2)

Logger utility


invoked by a monitor to record information and

numeric data
parameters.

(3)

Analyzer utility


invoked by a monitor to post
-
process parameter types (1), (2)
and (3) and hence compute complex parameters.

Finally question (3) concerning the management of the instrumentation is considered.

4.3

Monitor Manage
ment Services

Monitor services alone can only monitor behaviour and request the use of loggers and
analyzers. They need to be created and controlled during their lifecycle, which suggests a
need for
monitor management services

(or simply management service
s). The
management services create and interact with the monitor services, during their lifecycle,
triggered by specific remote instrumentation events generated within a Jini federation.
Because management services need to be aware of such remote events th
ey too must be
Jini services and must first register with a Jini middleware lookup service. Essentially the
management services provide the dynamic capabilities to the instrumentation framework
by responding to event requests for instrumentation services.
They are also responsible
for maintaining information relating to the current monitor service configuration, which
provides a
snapshot

of the current instrumentation services. An overview of the
instrumentation framework architecture is shown in Figure 2,
which is explained further
overleaf.


13


Lookup
Services

Application
Services

JVMs

Monitor
Services

Management
Services

Analyzer
Utilities

Observations
and
Measurements

Monitor
Configuration

Jini Application

Instrumentation

System

Logger

Utilities

Clients


Figure 2: instrumentation framework architectural overview

The right side of Figure 2 shows a Jini application, which consists of: one or more
application services; one or more lookup service
s; one or more pure clients and one or
more JVMs. The left side of Figure 2 shows the instrumentation system that monitors the
Jini application, which consists of: one or more monitor services, each of which is
associated with a single management service a
nd a single optional logger and/or a single
optional analyzer; a single monitor observations and measurements record and a single
monitor configuration record. Each management service first registers with a middleware
lookup service and then creates and in
teracts with a monitor service. By registering with a
lookup service a management service is made aware of remote instrumentation events
that may occur within a Jini application. The logger and/or analyzer instrumentation
utilities, which are not Jini serv
ices, can be called by a monitor service to record or
compute parameters. Also the logger utilities can directly log parameters of the
applications JVMs and the analyzer utilities compute information, which, when gathered
together, reflects the performance

and behaviour of the Jini application as a single entity.
The monitor services, logger and analyzer utilities can all update an observations and
measurements record, which could range from a series of files to a detailed GUI. Finally,
the management servi
ces maintain a monitor configuration record that reflects the current
monitor service configuration, which again could be a series of files or an XML
document.

5.

Implementing the Dynamic Instrumentation Framework

This section considers the on
-
going develo
pment and implementation of the
instrumentation framework and in particular, how instrumentation services can be
implemented by combining Jini’s code mobility capabilities, Java’s dynamic proxies and
Jini’s event model. The section finishes with the descri
ption of an early case
-
study used
to evaluate the instrumentation framework.


14

5.1

Overview of Jini Technology


Code Mobility

As mentioned previously, a Jini application consists of a collection of application
services, a collection of clients that use the
application services, a collection of lookup
services, with which the application services register, and a collection of JVMs that may
be distributed across several computing platforms. In order to provide an overview of Jini
material from Newmarch, [23],
is used to help consider a simple Jini application that
consists of three components: an application service (or simply a service), a client and a
lookup service. There is also an additional “hidden” fourth component, which is a
network

connecting the prev
ious three together, and this network will generally be
running TCP/IP (although the Jini
specification

is independent of network protocol).

Jini’s API provides
code mobility

in that code can be moved around between these three
components, over the network
, by
marshalling

the objects. This marshalling involves
serializing the objects (using Java’s Serialization API) in such a way that they can be
moved around the network, stored in a “freeze
-
dried” form, and later reconstituted by
using included information

about the class files as well as instance data. This marshalling
is represented in Figure 3 using the block arrows with broken lines.

Two events must take place in order for the client to use the application service:

(1)

First the application service, which c
onsists of an implementation and a proxy,
must register with a Jini lookup service. Sun Microsystems Jini implementation
provides a lookup service (called
reggie
), which “listens” on a port for registration
requests. When a request is received a dialog bet
ween the application service and
lookup service takes place after which a copy of the service proxy is moved to and
stored in the lookup service.

(2)

Second the client must find the service. This again, unsurprisingly, involves the
lookup service, which also l
istens for incoming requests from clients that want to
use services. The client makes its request using a template, which is checked
against the service proxies that are currently stored on the lookup service. If a
match is found a copy of the matching ser
vice proxy is moved from the lookup
service to the client.

At this stage there are three copies of the service proxy in existence (Figure 3 overleaf),
one in the application service, one in the lookup service and now one in the client. Jini
application ser
vice proxies are implemented as Java interfaces that specify the signatures
of methods, which are implemented by the service implementation. The client can
interact with its copy of the service proxy by invoking any of the specified methods.
These method i
nvocations are then routed back to the service implementation, typically
using Java’s remote method invocation (RMI), resulting in the invocation of a method in
the service implementation. The overall effect of this routing is that the client “thinks”
that

it has its own local copy of the service implementation and proceeds to use the
service by invoking its methods being unaware of the physical location of the service
within the network.


15


Network

Lookup Service

Service

Implementation

Service

Proxy

Client

Service

Proxy

Service

Proxy


Figure 3: Jini client/service interaction

The three code listings (below and overleaf) show how a simple Calculator application
service is coded in Java. The first listing represents the proxy, which specifies the
methods
add
,
sum
,
mult

and
div
. This proxy, represented as the ellipses in Figure 3
, is
the code that will be marshalled over the network and used by the client to interact with
the service. The second listing represents the service implementation, which actually
implements the Calculator service, with the details of service registration

omitted to avoid
unnecessary detail. The third listing represents a typical client that uses the Calculator
service. The top half of the client code shows the Jini API methods, which are used to
acquire a
CalculatorProxy

interface and the bottom half show
s how the client uses
the service by invoking its methods, via the proxy, to perform simple arithmetic.


Listing 1: Calculator proxy


16


Listing 2: Calculator implementation


Listing 3: Calculator client

5.2

Dynamic Proxies

The
proxy

is a design pattern, s
pecified by Gamma
et al.
, [24], which is frequently used
in object
-
oriented programming to force method invocations on an object to occur
indirectly through a second object


the proxy object, which essentially acts as a
surrogate or delegate for the under
lying object being proxied. Several variations on the
basic proxy, also described in [24], are: access, remote, virtual and dynamic proxies. Sun

17

Microsystems realized the potential of the dynamic proxy for implementing strongly
typed, or
type
-
safe

Java app
lications and introduced the Dynamic Proxy API in Java v1.3
for use in conjunction with Java interfaces. The paragraph below draws on material
presented in the Java language API documentation, [25], to explain the mechanism of the
dynamic proxy.

Java uses
interfaces to define
behaviour

that can be implemented by any class anywhere
in a class hierarchy. A Java interface defines a set of methods but does not implement
them, instead relying on a class to implement the methods, which thereby agrees to
provide t
he behaviour specified by the interface. A dynamic proxy class extends Java’s
use of interfaces by providing capabilities to actually implement a list of interfaces that
are specified at runtime. A method invocation through one of the interfaces on an
inst
ance of a dynamic proxy class is encoded and dispatched to another object through a
uniform interface. In more detail, the method invocation is dispatched to a special
invocation handler

method in the dynamic proxy instance, where it is encoded, using
Java
’s reflection API, to deliver a Java object (
java.lang.Object
) identifying the
method that was invoked and an array of Java objects containing the arguments used in
the invocation. This provides an extremely powerful tool for implementing
instrumentation,
because dynamic interactions between application services and clients,
occurring as method invocations, can be made available to an instrumentation monitor
service (i.e. a dynamic proxy) in Java object form!

5.3

Implementing Monitor Services

The dynamic pr
oxy provides an ideal mechanism for the implementation of the monitor
service. Section 5.1 stated that Jini application service proxies are implemented as Java
interfaces, so an application service proxy can be supplied as the list of interfaces that a
mon
itor service, as a dynamic proxy, should implement. This is illustrated overleaf in
Figure 4, in which the top half of the figure shows the same basic Jini application that
was considered previously in Section 5.1 and the bottom half now shows the applicat
ion
service proxy (small ellipse) “wrapped up” in a monitor service (large ellipse) to create a
“compound” proxy, such that the monitor service (i.e. dynamic proxy) implements the
application service proxy (i.e. Java interface).

The power of the dynamic pr
oxy, when used as a wrapper for the application service
proxy, is that it allows normal access to the application service proxy from the client to
proceed as before. However, after a client has invoked a service implementation method,
via the application s
ervice proxy, then control returns back to the monitor service, which
now has the Java objects that identify the method invoked and the parameters used in the
invocation along with any return values. When used in this way the monitor service can
be regarde
d as providing an
invocation handling

function that provides information,
which is extremely useful for observing Jini client/service interactions.


18


Network

Network

Lookup S
ervice

Lookup Service

Service

Implementation

Service

Proxy

Client

Service

Proxy

Service

Proxy

Service

Implementation

Client


Service

Proxy

Monitor Service

(Dynamic Proxy)

Service

Proxy


Service

Proxy

(a)

(b)


Figure 4: monitor service overview

Figure 5 shows an outline UML representation
of a monitor service that instruments an
application service, which implements the methods:
method1
,
method2

and
method3
.


Figure 5: monitor service UML class diagram


19

5.4

Implementing Management Services

Monitor management services are implemented just li
ke any other Jini service, which
requires that they register with a lookup service, but what is significant to their
implementation is their event handling capabilities, based on Jini’s remote event API.

In a typical instrumentation scenario management ser
vices are created on demand by an
instrumentation factory (considered in section 5.5). After creation management services
register with a lookup service through a management proxy (again, a Java interface) and
then wait for instrumentation requests from wi
thin the Jini application. Requests are made
using specific remote instrumentation events and a management service is notified of
these events by the lookup service with which it has registered. If an event is received for
an application service to be inst
rumented, before use by a client, then the management
service creates a monitor service (as a dynamic proxy) and adds the application service
proxy to the monitor service. The client then receives the monitor service containing the
application service prox
y. The client can then invoke methods on the application proxy
and after each invocation control returns to the monitor service as considered previously
in section 5.3.

If an application service is currently being used by a client in an “un
-
instrumented” s
tate
then it can be dynamically instrumented by arranging for the management service to
notify the client that its copy of the application service proxy is “out of date” through a
specific remote instrumentation event. After this notification the managemen
t service sets
about the creation of a new compound proxy by creating the monitor service to which it
then adds the application service proxy as before. The client is then sent the new revised
proxy, which is the monitor service containing the original app
lication service proxy and
as before the monitor service can monitor method invocations made on the application
service proxy.

Essentially, management services orchestrate the dynamic instrumentation by acting on
Jini remote instrumentation events. However
, management services also maintain
configuration information about the current instrumentation monitor services, within the
federation, through the use of an XML document, similar to the
Operational Strings

used
in Rio. The XML document may be accessed by

any client, application service, monitor
service or management service, within the application, or even by the user, through a
GUI, to provide a snapshot of the current instrumentation configuration.


20


Lookup Service

Lookup Service


Monitor Service

(Dynamic Proxy)

Observations

and
Measurements

Monitor

Configuration

Service

Proxy

Service

Im
plementation

Service

Proxy

Client

Service

Proxy

Service

Implementation

Client

Service

Proxy

Service

Proxy

Service

Proxy

Service

Proxy

Management
Service

Logger Utility

Service

Proxy

(a)

(b)

Management

Proxy

Monitor

Service

Management

Proxy

(Network not shown)


Figure 6: management servic
e overview

Figure 6 essentially supplements the previous Figure 4 with a management service
(assumed to have been created by an instrumentation factory), which has registered with
a lookup service, through its management proxy, so that it may be notified o
f remote
instrumentation events within the application. On receiving an instrumentation event, the
management service has then created a monitor service and added the application service
proxy to the monitor service and registered the monitor service with
the lookup service.
The monitor service has requested the use of a logger utility (but no analyzer


simply to
avoid diagram clutter), which is used to assist in maintaining the observations and
measurements record that reflects the service/client interact
ion. Finally, the management

21

service updates the monitor configuration record to reflect the addition of a new monitor
service. An outline UML representation of the management service is shown in Figure 7.


Figure 7: management service UML class diagram

F
igure 8 shows the front
-
end GUI to the current instrumentation framework prototype.
The GUI allows different client/service configurations to be established instantly by
selecting from the client and service lists and instrumentation service and utility co
ntrols
allow these configurations to be instrumented dynamically.


Figure 8: instrumentation framework GUI


22

5.5

Evaluation

As mentioned previously, the development and implementation of the framework is
ongoing. However, to date, the basic architecture und
erlying the framework has already
been implemented and evaluated using an existing Jini application, namely EmergeITS
3
,
[26], which was already conveniently available from previous work. EmergeITS is
intended to realize the concept of
intelligent

networked

vehicles
, primarily for use by the
emergency fire service. Essentially EmergeITS allows emergency fire service personnel
to access a variety of application services, from centralized corporate systems through
remote in
-
vehicle computers and Palm and WAP p
hone devices. Figure 9 shows a
simplified version of the EmergeITS architecture, which consists of a collection of
application services and a service manager responsible for registering application services
and managing their leases. Application services a
re discovered and used accordingly by
one or more in
-
vehicle client computers.


Figure 9: EmergeITS architecture

An important design decision facing EmergeITS was the choice of service
implementation. The alternatives were “conventional” lease
-
based servi
ces, which
subclass the Java RMI
UnicastRemoteObject

class, or “lazy” activation
-
based
services, which subclass the Java RMI
Activatable

class. Conventional lease
-
based
services exist within a server, where they are kept alive, consuming memory, even when
idle, until they are eventually discarded. A lazy activatable service, considered further in
[23], registers with a lookup service, via a proxy, just like a conventional service, but
after registration, when the activatable service becomes idle, it is allo
wed to “die” or
“sleep”, thereby consuming no memory. The Java RMI daemon,
rmid
, maintains
references to the dormant service so that it can be resurrected when needed by a client.
On first impression, activatable services seem like an attractive option. Ho
wever, the



3

EmergeITS is a collaborative project between the School of Computing and Mathematical Sciences at
Liverpool John Moores University and Merseyside Fire Service <http://www.cms.livjm.ac.uk/emergeits>


23

trade off with activatable services is that although memory usage is reduced a new JVM
needs to be started, when a service is called into use (i.e. its methods are invoked). The
decision to “mix and match” both types of service implementation wa
s based on intuition
in that it was self evident as to which services would be accessed frequently
(implemented as conventional lease
-
based services


shown as plain rectangles in Figure
9) and which would be accessed infrequently (implemented as activatab
le services


shown as shaded rectangles in Figure 9).

Recently, the EmergeITS application was instrumented to essentially “field
-
trial” and
evaluate the mechanics of the instrumentation framework and also to confirm the
intuition regarding the previously
mentioned EmergeITS design decision. An Instrument
Factory, controlled by an Instrumentation Manager (a main driver program), was used to
create instrumentation services (management and monitor services) as and when they
were needed. The instrumentation se
rvices were applied such that conventional lease
-
based services were instrumented when first registered with a lookup service (i.e. before
use by any client) and activatable services were instrumented on activation (i.e. whenever
used by a client).

The pa
rameters of interest, during the trial, were the number of JVMs started, as a
consequence of invocations made on activatable services, and the memory conserved
through the use of activatable services. Logger and analyzer utilities were used to record
and p
ost
-
process the information acquired by the instrumentation monitor services
respectively. This information was stored in the Application Recorder, which was used to
gather the performance and behavioural parameters obtained by the individual
instrumentati
on services to record the overall performance and behaviour of the
EmergeITS application as a single entity. Figure, overleaf, 10 shows a snapshot of the
instrumentation system in operation after the creation of five instrumentation services:
IS1
,
IS2
,
IS3
,
IS4

and
IS5
. Only one client is included, to avoid clutter, which is shown
accessing the Database/XML Servlet activatable service and the figure is annotated with
the sequence of events necessary to dynamically instrument this activatable service.


24


Sequence of events (represented in the figure with hatched lines):

1.

Client3 invokes a method on activatable Database/XML Servlet
service

2.

Instrumentation Manager requests a new Instrumentation service
IS3

3.

Instrumentation Factory produces
IS3

4.

IS3
is dynamical
ly “connected” to Database/XML Servlet service

5.

IS3
“connects” with Instrumentation Manager so that the latter can
receive information from
IS3
and in turn update the Application
Recorder


Figure 10: instrumentation of EmergeITS

6.

Progress Review

This final section reports on what has been achieved so far and what needs to be
addressed following transfer to the Ph.D.

6.1

Conclusions and Work Completed

(1)

Static instrum
entation is insufficient for monitoring and measuring the behaviour
and performance of distributed applications. Dynamic instrumentation is needed
to accommodate the dynamics inherent in distributed applications and provide a
faithful representation of the
ir performance and behaviour.


25

(2)

Dynamic instrumentation has grasped the attention of several major contenders
interested in the development of distributed applications, namely Sun
Microsystems, the Jini Community, the Openwings Community and DARPA,
which has

led to cooperative efforts resulting in the development of JMX, Rio,
Openwings and DASADA architectures respectively. However, these efforts do
not completely address question (4) (section 1.4) or provide a generic framework
for describing, developing and

integrating instrumentation.

(3)

Four important questions that need to be addressed, relating to the development of
a dynamic instrumentation framework are: what should be monitored, what
instrumentation is needed to perform the monitoring, how can the instru
mentation
be managed and what rules and invariants are needed to enforce the integration of
the instrumentation? The first three questions have been considered through this
report and the fourth will be considered following transfer to the Ph.D.

(4)

A prototyp
e instrumentation framework based on the use of a monitor service per
application service, which can call on the use of logger and/or analyzer utilities to
monitor application services and lookup services has already been developed and
implemented using Ji
ni technology. Management services have also been
developed and implemented to control, coordinate and maintain configuration
information relating to the instrumentation monitor services.

(5)

The prototype instrumentation framework has been evaluated, using an

existing
Jini application, to assess the performance impact of design decisions relating to
the choice between conventional lease
-
based and lazy activatable service
implementations.

6.2

Proposed Work for the Ph.D.

(1)

To re
-
factor the low
-
level instrumentatio
n parameters of Table 1 to produce
meaningful metrics against which performance and behaviour of a distributed
application can be assessed. It is anticipated that metrics, such as access patterns
and application granularity, will provide useful information

to assist the design,
development and runtime management of distributed applications (2 months).

(2)

To consider the issues of dynamic service dependencies, so that dependencies
can be resolved, understood and managed to facilitate the intelligent
instrumenta
tion of complex entities within a distributed application. This is a
major piece of work, central to understanding the structure and dynamics of a
distributed application, that will require:

a.

Conceptualisation of dependency relationships based on a digraph
consisting of nodes and directed edges


26

b.

Development of dependency digraph APIs that maintain “up to date”
dependency information and make structural and behavioural information
available to instrumentation.

c.

Use of dependency relationships to assist the plac
ement of instruments and
facilitate network
-
centric instrumentation.

(7 months)

(3)

To extend the ideas of the framework to consider further distributed middleware
technologies, particularly CORBA and EJB. It is anticipated that this will lead to
generic frame
work rules, guidelines and usage models that are common to
several middleware technologies (3 months).

(4)

To continue the evaluation of the instrumentation framework, through case
studies, and to compare results against those of others concerned with distribu
ted
application management, measurement and monitoring. Existing applications
have been sufficient for the case studies completed to date. However, new
applications will be required for further case studies that will consider and
evaluate:

a.

The dependencies

relationships, mentioned in point 2.

b.

Issues of scale (i.e. instrumentation of large
-
scale distributed applications)

(3 months).

(5)

To disseminate the research results and prepare and write the PhD. Thesis (9
months).

References

[1]

Emmerich, W, “Engineering
Distributed Objects”,
John Wiley & Sons Ltd.,
ISBN: 0
-
471
-
98657
-
7, 2000.

[2] Szyperski, C., “Component Software
-

Beyond Object
-
Oriented Programming”,
Addison
-
Wesley and ACM Press
, ISBN: 0
-
201
-
17888
-
5, 1998.

[3]

Satterthwaite, E., “Debugging Tools for High

Level Languages”,
Software Practice
and Experience, 2
, 1972, pp. 197
-
217.

[4]

Sommerville, I., “Software Engineering, Fourth Edition”,
Addison Wesley
Publishers,
ISBN 0
-
201
-
56529
-
3, 1992.

[5]

Rover D.T., “Performance Evaluation: Integrating Techniques and

Tools into
Environments and Frameworks”,
Roundtable, Supercomputing 94,
Washington DC,
November 1994.


27

[6]

Simmons, M. and Koskela R. (editors), “Performance Instrumentation and
Visualization”,
ACM & Addison Wesley Publishers,
1990.

[7]

Waheed, A. and Rove
r D.T., “A Structured Approach to Instrumentation System
Development and Evaluation”,
Proceedings of ACM/IEEE Supercomputing
Conference (SC’95)
, San Diego California, 1995.

[8]

Geist, G.; Heath, M.; Peyton, B.; Worley, P., “A User’s Guide to PICL”, Technic
al
Report, ORNL/TM
-
11616, Oak Ridge National Laboratory, 1991.

[9]

Heath, M. and Etheridge, J.A., “Visualizing the Performance of Parallel Programs”,
IEEE Software 8(5)
, September 1991, pp. 29
-
39.

[10] Hao, M.C.; Karp, A.H.; Waheed, A.; Jazayeri, M., “VIZI
R: An Integrated
Environment for Distributed Program Visualization”,
Proceedings of International
Workshop on Modeling, Analysis and Simulation of Computer and
Telecommunications Systems (MASCOTS ’95) Tools Fair
, Durham, North Carolina
1995.

[11]

Sun Micro
systems Inc., “Java Management Extensions: Instrumentation and
Agent Specification, v1.0”, May 2000.
<http://java.sun.com/products/JavaManagement/download.html> (accessed January
2002).

[12]

Openwings Community, “Openwings Overview, Alpha v0.7”,
<http://ww
w.openwings.org> (accessed January 2002).

[13]

Wells, D.L. (Object Services and Consulting Inc.) and Nagy, J. (Air Force
Research Laboratory), “Gauges to Dynamically Deduce Componentware
Configurations”, DASADA Project List, DARPA (Program Sponsor),
<http:
//schafercorp
-
ballston.com/dasada/projectlist.html> (accessed January 2002).

[14]

Wolf A.L. (Univ. Colorado) and Kean, E. (Air Force Research Laboratory),
“Definition, Deployment and Use of Gauges to Manage Reconfigurable Component
-
Based Systems”, DASADA P
roject List, DARPA (Program Sponsor),
<http://schafercorp
-
ballston.com/dasada/projectlist.html> (accessed January 2002).

[15]

Garlan, D.; Schmerl, B.; Chang, J., “Using Gauges for Architecture
-
Based
Monitoring and Adaptation”,
Working Conference on Complex

and Dynamic Systems
Architecture
, Brisbane, Australia, 2001.

[16]

Diakov, N.K.; Batteram, H.J.; Zandbelt, H.; Sinderen, M.J., “Monitoring of
Distributed Component Interactions”,
RM’2000: Workshop on Reflective
Middleware
, New York, 2000.

[17]

Jini Communi
ty, “Rio Architecture Overview”,
Rio Project,

<http://www.jini.org/projects/rio>
(accessed January 2002).

[18]

Fahrimair, M.; Salzmann, C.; Schoenmakers, M.; “A Reflection Based Tool for
Observing Jini Services”,
In Cazzola et al. Reflection and Software E
ngineering
LNCS 1826, Springer Verlag
, June 2000.


28

[19]

Hasselmeyer, P. and Vo

, M., “Monitoring Component Interactions in Jini
Federations”,
SPIE Proceedings vol. 4521,

ISBN 0
-
8194
-
4245
-
3, August 2001, pp.
34
-
41.

[20]

Hasselmeyer, P., “Managing Dynamic Ser
vice Dependencies”,
12
th

International
Workshop on Distributed Systems: Operations and Management (DSCOM 2001),

ISBN 0
-
8194
-
4245
-
3, Nancy, France, October 2001.

[21]

Jini Community, <http://www.jini.org> (accessed January 2002).

[22]

Sun Microsystems Inc.,

“Jini Architecture Specification


v1.1”, October
2000.<http://www.sun.com/jini/specs> (accessed January 2002).

[23]

Newmarch, J., “Jan Newmarch’s Guide to Jini Technology”, version 2.08, 12
June 2001, <http://pandonia.canberra.edu.au/java/jini/tutorial/J
ini.xml> (accessed
January 2002).

[24]

Gamma, E.; Helm, R.; Johnson, R.; Vlissides, J.; Booch, G.;


Design Patterns”,
Addison
-
Wesley Publishing Company,
ISBN: 0
-
201
-
63361
-
2, 1995.

[25]

Sun Microsystems Inc., “Java 2 SDK, Standard Edition Documentation v1.3
.1”,
November 2001, <http://java.sun.com/j2se/1.3/docs.html>, (accessed January 2002).

[26]

Reilly, D. and Taleb
-
Bendiab, A. “A Service Based Architecture for In
-
Vehicle
Telematics Systems”,
IEEE Proceedings of 22
nd

International Conference on
Distributed
Computing Systems (ICDCS 2002) WORKSHOPS
-

International
Workshop of Smart Appliances and Wearable Computing (IWSAWC 2002)
, Vienna,
Austria, 2002.