Delivery of Services on Any Device: From Java Code to User Interface

mammetlizardlickΛογισμικό & κατασκευή λογ/κού

3 Νοε 2013 (πριν από 3 χρόνια και 7 μήνες)

52 εμφανίσεις

Delivery of Services on Any Device: From Java Code to User

Davide Carboni, Andrea Piras,

Stefano Sanna, Gavino Paddeu


Ce湴敲⁦潲⁁摶 湣e搠d瑵摩e猬s

sf⁳瑲 摡 佶l獴Ⱐw.f⸠䵡cc桩慲e摤甬d


Sylvain Giroux

Dep. of Mathematics and Computer
Science, University of Sherbrooke

Sherbrooke, Canada


The design and the implementation of software for mobile comp
uters and for pervasive computing
environments entail several issues and lead to new requirements. Applications must handle at run
time the heterogeneity of delivery contexts in terms of devices features, network bandwidth,
operating systems and so forth.
The issues related to the access belong both to delivery (can the
device run the service?) and to the user interface (how the user interact with the service through
device?). This paper describes, on one hand, an approach where user interfaces are generate
d at
time in a mobile or pervasive context, and on the other hand, a deployment strategy that,
taking charge of distributing the code according to the device capacities, leads to the “application



The design and the implemen
tation of software for mobile computers and for pervasive computing
environments [Weiser, 1993] entail several issues. Enhancing physical surroundings with
computing capabilities leads to new requirements: quoting from [Banavar et al., 2000]
"A device is

portal into an application/data space, not a repository of custom software managed by the user.
An application is a means by which a user performs a task, not a piece of software that is written
to exploit a device's capabilities. The computing environmen
t is the user's information
physical surroundings, not a virtual space that exists to store and run software."

Thus, applications must handle at run
time the heterogeneity of delivery contexts in terms of
devices features, network bandwidth, opera
ting systems and so forth. In fact, a user may want to
access his personal data/applications from different devices in different context of use: from
desktop PC to cellular phones. The issues related to the access belong both to delivery (can the
device ru
n the service?) and to the user interface (how the user interact with the service through
device?). Delivery must cope with network bandwidth, processing power, operating systems, etc.
computer interaction must deal with screen size (if any), colors,

and input/output
modalities. designing and implementing a brand
new interface for each category of devices is a
tedious and error
prone task. Moreover, when applications get mobile or pervasive, one even does
not know beforehand what devices will be invol
ved. This paper describes an approach where user
interfaces are generated at run
time in a mobile or pervasive context. On the one hand, there are
rendering engines that, given a category of device, generate the appropriate user interfaces (UI)
reflection on Java code in order to understand which data can be accesses by the user
and which actions are permitted on that data. On the other hand, there is a multi
deployment strategy that, taking charge of distributing the code according to the

device capacities,
leads to the “application apportioning”[Banavar et al., 2000].


Target Devices

Assuming that accessing the same application from unpredictable delivery contexts involves the
automatic UI generation at run
time, we need at least a kind o
f categorization which groups
together devices with similar characteristics. Due to the fact that in our research centre, the Java
language is widely used to develop network distributed applications, our categorization is also
affected by the type of Java
Virtual Machine provided with any target device. We have identified
three main categories: fat clients, thin clients, and Web
like clients. The category of fat clients
groups together Personal Computers, Unix workstations, and laptops. All of them are capa
ble to
execute a standard Java Virtual Machine, to download mobile Java code, and are provided with
WIMP (Windows Icons Mouse Pointing) input/output modalities. In fat clients a portion of the
application code (the entire application in some cases) is move
d across the network to the target
device. On the other hand, thin clients cannot download nor execute mobile code, and Java classes
must be loaded from the local disc. So, as the application code cannot reach the device, we pre
install a service viewer w
hich can connect to the application and exchange data using an XML
protocol similar to the Web
Service protocol. Such a service viewer is able to generate an
appropriate user interface depending on the data received in the XML, and to interact with the
ote object invoking its methods.
Thin clients category groups
personal digital
assistants, smart phones, and set
top boxes. The last category is that of Web
like clients. These
devices have no Java Virtual Machine at all but they are equipped with

a third

party browser
enabled phones, VoiceXML [Voice, 2000], and HTML

browsers). Such devices can
connect via HTTP to a Web server and download documents compliant to their specific mark
languages. For instance, a WAP phone will work only with WM
L [Wireless, 2001] documents, a
voice browser will work only with VoiceXML documents and so forth.



The rationale behind our approach is that the code of an interactive system can be decomposed in
two main parts: models and views. Such a decompos
ition is borrowed from the object oriented
programming practice in which the well known Model
View Control, (see Observer pattern in
[Gamma et al., 1995]),

design pattern has been formalized and widely used in the design of
graphical user interfaces. Such
a pattern
organizes code for a clear separation of the logic (the
Model) and the UI (the View
Controller). In object oriented systems models and views are
objects, and objects are reactive systems that can be activated by the invocation the methods in
r public interface (the set of operations that can be activated by a client). A model is
independent from the delivery context while views are strictly affected by it. As an example we
can consider a pocket calculator which is a metaphor for some applicati
ons (e.g. the calc.exe in
Microsoft Windows). Such a metaphor can be implemented as a model object which is activated
whenever the user invokes the execution of a method that appends a new digit to the display or a
method that selects the mathematical oper
ation to be performed. Such a model is independent
from the graphical user interface, the only thing to adjust at run
time is "where" such an object is
loaded and executed. In the case of fat clients, this model moves across the network, is loaded by
the u
ser Java Virtual Machine, and there executed. In the other cases (thin and Web
like clients),
an intermediate remote tier (remote with respect to the user), performs code loading, code
execution, and communicates with the user terminal using an ad
hoc prot
ocol. On the other hand,
a calculator view is strictly dependant on the user terminal. In the case of fat and thin clients, the
view object is generated at run
time by means of graphical components such as buttons, text
fields, dialogs and so forth.

d, views are concrete “interactors” and must be defined by
composing concrete widgets belonging to a specific toolkit.
In the case of Web
like clients, view
objects are only a mark
up presentation of the model: HTML for Web browsers, WML for WAP
browser an
d so forth. So, in the latter case, the final presentation is delegated to a pre
party browser.


From Java Code to User Interfaces

A composition of model objects can be transformed at run
time into a structure of concrete and
interactive ob
jects by means of a “rendering process”. In this process, widgets such as text fields,
radio buttons, and more sophisticated controls, are deployed in place of data they refer to: strings,
numbers, booleans and more complex data types like object arrays an
d collections. The advantage
of this approach is that the same model can be accessed from any device for which has been
implemented a rendering engine
. We have developed on one hand a framework for defining and
composing models, and on the other hand, a
set of rendering engines that we call MORE (Multi
platform Rendering Engine). At run
time, the rendering engine builds on
fly the right view by
transforming models into structures of concrete and interactive objects. The rendering process uses
on to extract directly from the code the “displayable” information. Complex models are
viewed as composite graphical objects and the rendering process is applied recursively till atomic
interactors are obtained. The process of inspecting composite models t
o generate a graphical user
interface can be summarized by the following algorithm:

input = a model object

output = a view object

Create a container for the actual platform //e.g. a JFrame in Java Swing toolkit

For each field of the object:

up a ru
time editor for the field.

If success: add the editor to the container.

Else: add a link that when activated will inspect the field with
this algorithm.

For each method of the object:

Add a trigger //(e.g. a button)

Once the container is filled with all

its components the following events are handled:

When a trigger is activated execute the corresponding method.

When a link is activated process the corresponding field as input for the

A run
time editor is a component aimed to write/read the v
alue of an object field. For instance, if
our model contains a field of type
, the above algorithm looks up a run
editor for such a type, that will likely be a text field graphic component. To foster new
applications, our system provid
es run
time editors for some basic Java types:
, and
. Besides, it provides run
editors for composite types like
, and
. For those
types, the
editor allow users to select, display, modify, remove, and add elements to the collection
or to the object array. Finally, during the development of multi
media and geo
applications, emerged the need for some new Java types:


is Java interface that provides information about the physical position of an object
either in a Cartesian format (latitude and longitude) or in a topological format (city, street, number,
etc.). An

component is displayed a
s a map centered on the barycentre of the array, in
which the user can zoom in, zoom out, select an object and display it in a separate view. The last


We refer to “rendering engine” as a system able to perform a “rendering process”.

type we added is the

type which is basically a wrapper to a multimedia file
of type am
ong JPEG, GIF, MP3, wav, and mpeg.


An application example

This section illustrates a trivial object model for a Museum and its Pictures (Fig. 1a). A Museum
instance contains a collection of Picture and a

field containing the name of the
ty. Accessors methods like


are used to read and write the value
of fields, while

is used to give a text description of a whole object. The other methods
are displayed as buttons. The main interaction with this syste
m is to browse the list of painting,
select one, and eventually invoke the operation

on the Picture.

Figure 1a.
UML Diagram

Swing User Interface

ure 1c.
HTML Interface for iPaq

Figure 1d.
WML Interface

The Java Swing (Fig. 1b) implementation of the rendering engine yields an interface where
browsing the collection and viewing items are performed at the same time in a s
ingle composite
frame. On the other hand, in a palm computer (Fig. 1c) and in the WAP user interface (Fig. 1d),
due to the severe screen size constraints, browsing the collection and viewing items are performed
in two separate steps.




tation Status

We have implemented rendering engines with all the run
time editors we introduced in §

for the
following platforms: fat clients such as laptops and workstations; thin clients such as palm
computers, interactive TV
, and smart phones; Web
like clients such as Web browsers, WAP
browsers, and VoiceXML browsers. In the case of Web
like clients, the result of the rendering
process is an intermediate XML representation which is finally translated in a mark
up language
table for the actual browser. MORE uses a XSLT processor and an XSL style sheet to perform
such a transformation.



Our framework provides some immediate and considerable advantages. MORE makes “design
once, display anywhere” closer to reality. A

programmer solely need to write the model in the
form of Java code. A model then can be deployed on any device for which there is a rendering
engine. It is easily extensible as new devices categories emerged. As technology evolves, it is not
necessary to
rewrite the UI but simply to adapt the rendering engine for this device category.

This approach is more germane to Silica [Rao, 1991] and plug
ins than to model
based ones
[Paterno, 2000]



Reflection of Java code allows to extract the structur
e of Java types and our system uses this
structure to generate a user interface. Therefore, the information obtainable by the reflection is not
always sufficient. On one hand, the fields of an object have names compliant with Java syntactical
rules but tha
t can result scarcely readable. For instance, a field

should be
visualized as “Departure date”. Such a transformation could be done automatically decomposing
strings according to the position of their capital letter, assuming that programmers

follow a precise
naming convention. On the other hand, fields and methods of an object are not ordered in the code
with a precise criteria, thus the view is generated laying out the components with a unpredictable
position. Due to the fact that views are
automatically generated at run
time, programmers do not
have a complete control of the presentation. To address these issues, our rendering engine takes as
input a supplementary information, in form of a text file, which explains how names must be

by aliases (allowing also the internationalization of the user interface), and how fields
and methods must be laid out in the container.



G. Banavar et al.
Challenges: An Application Model for Pervasive Computing.
, pp. 266

Gamma, E., Helm, R., Johnson, R., & Vlissides, J. (1995) Design Patterns, Addison

Paterno, F. (2000) Model
based desing and evaluation of interactive applications, Springer.

Rao, R. (1991) Implementational Reflection in Silica,
ECOOP 1991

512, Springer
pp. 251

Voice eXtensible Markup Language (VoiceXML) v1.0 (2000),

Wireless Markup Language v2.0, Wireless Application Protocol WAP

Weiser, M. (1993) "Some Computer Science Problems in Ubiquitous Computing,"
Communications of the ACM