Project Proposal and Plan ENGG4801 Subscription Protocols and gathering operational statistics of remote experiments

secrettownpanamanianMobile - Wireless

Dec 10, 2013 (3 years and 8 months ago)

78 views

1

Project Proposal and Plan
ENGG4801
Subscription Protocols and gathering
operational statistics of remote
experiments













Supervisor: Mark Schulz
Student: Mikeal Hooley
41420517

2

Contents
Executive Summary ................................................................................................................................. 3
Topic Definition ....................................................................................................................................... 4
The iLabs Project ................................................................................................................................. 4
LabClient ......................................................................................................................................... 5
ServiceBroker .................................................................................................................................. 5
LabServer ........................................................................................................................................ 5
LabEquipment ................................................................................................................................. 5
This Project (SBS) ................................................................................................................................ 6
Concept ........................................................................................................................................... 6
Motivation ....................................................................................................................................... 7
Scope ............................................................................................................................................... 7
Future Extension ............................................................................................................................. 8
Project Plan ............................................................................................................................................. 9
Technologies ....................................................................................................................................... 9
LabServer SUS ................................................................................................................................. 9
Subscription Server ......................................................................................................................... 9
SBS Client ...................................................................................................................................... 10
Proposed Features ............................................................................................................................ 12
LabServer SUS ............................................................................................................................... 12
Subscription Server ....................................................................................................................... 12
SBS Client ...................................................................................................................................... 12
Timed Targets ................................................................................................................................... 13
Bibliography .......................................................................................................................................... 14



3

Executive Summary
The project detailed and outlined in this report is tentatively named the Subscription Based Statistics
(SBS) system. It is intended to build on top of the current iLab project by creating a client that will
register to the LabServer via a subscription based communications approach, and will provide
statistics and general data relating to the usage of the LabServer. It will do this via three main
components, the LabServer Subscription Updater Service (SUS), the Subscription Server, and the SBS
client.
The SUS will be a service built on top of the current LabServer that will update the Subscription
Server periodically via an asynchronous call. The data sent will be in xml and will provide a base for
the Subscription Server to update to the SBS clients.
The Subscription Server will receive updates from the SUS and be capable of storing these xml files
either in a local directory store, or in a local database. It will also be capable of receiving
subscriptions from SBS clients to the data, and periodically update these clients when new data is
available (or upon subscription). The subscribing and updating components of this part of the system
will likely exist however, on the server side of the SBS client, and use a duplex services call to allow
clients to register.
The SBS client, being the main part of the system, and the interface for the entire user base, is where
most of the developmental work will occur. It is hoped that by the end of development, this client
will be capable of effectively and efficiently providing statistics of experiments held on the
LabServer, with extremely little impact on the LabServers usage. In addition to the obvious 'current
status' pages that will be available on the SBS client, three main significant components are planned
to exist by the end of development. These are:
· A calendar for selected experiments showing times of activity, and capable of displaying
individual experiments that occurred along with their parameters and results
· A charting wizard allowing a user to create a chart comparing certain attributes of selected
experiments over a chosen time period (eg. select line chart, select queue lengths attribute,
select radiation experiment, and choose 12pm to 6pm yesterday)
· Some graphical display capable of showing for selected experiments, where in the world they
are being used, and in what quantities (using ip addresses or google tools)
The timed targets outline for completion of various components of this project is available in the
Project Plan section of this report. If the reader is unfamiliar with the current iLabs project, this is
briefly outlined in the Topic Definition section of this project.
4

Topic Definition
The iLabs Project
The purpose of this thesis is to extend on and provide support to another larger and existing project
called the iLabs project. This project is already active, and has a user-base that spans around the
world.
In education at the moment, there exists a requirement for high-cost, high-maintenance, and high
quality laboratory equipment for the purpose of practical scientific learning experiences. This
requirement exists, not just in advanced wealthy area schools, nor is it confined to tertiary or higher
level education. Average students, all over the globe have this need, and it is because of this need,
that the iLabs project began.
The core idea of the iLabs project is to allow students from around the world to experience the use
of real life laboratory equipment, which is either too complex or too expensive for their own
educational institution to manage. It has been shown that the learning experience gained from
working online with a real piece of hardware (or at least one that the subject believes is real) is far
greater than one gained from working with a simulation of the experiment. Because of this,
necessary hardware can be placed (as it is at uq) onto an online service, that provides students
around the world with an online interface to run experiments, with their own parameters and setup,
and on advanced pieces of hardware that would otherwise be unavailable to them.
The experiments themselves are setup in various locations around the uq campus and are currently
used by students around the world (and soon even in Queensland). Several pieces of software (and
the hardware of the experiment), enable this worldwide use of equipment. These include the
LabClient, the ServiceBroker, the LabServer, and the LabEquipment. These components are
discussed on the next page in more detail.

Figure 1 – Ilabs System with focus around ServiceBroker
5

LabClient
The LabClient is essentially any web-based or form-based client that the student (or any other user)
accesses to use the iLabs project. It communes only with the ServiceBroker and is outside the scope
of this project.
ServiceBroker
The ServiceBroker is a piece of middleware that has a valid authentication to talk directly to the
LabServers and is used to manage the experiments of all the LabClients. Its features can vary and it is
also outside the scope of this project. It is however hoped that an extension on the subscription
features of this project could decrease the need for some features of this component. This will be
discussed in more detail in the Future Extension section of this report.
LabServer
The LabServer is one of the main responsibilities of the CEIT office in regards to the iLabs project. Its
responsibility is to receive and authenticate requests from the ServiceBroker to run experiments. It
then parses the setup of the experiment, and according to that, inserts the authenticated
experiment into a queue to be run through an experiment engine that has an appropriate driver for
the LabEquipment. It can also handle requests for estimated execution time and cancellation of
experiments, and makes the results available (via xml) when the experiment has reached
completion.
LabEquipment
The LabEquipment is the actual hardware of the experiment. It varies from complex radiation
experiments to motor controlled devices useful for learning about control systems. It is connected to
the LabServer via the LabServer’s experiment engine, which holds drivers for all the pieces of
LabEquipment it is responsible for.

6

This Project (SBS)
For the purpose of this report, the project that is involved with this thesis proposal, and will be
discussed in detail in this section will be referred to as SBS (Subscription based Statistics). Note that
this is a tentative naming for the sole purpose of this written report and will likely be altered or
changed completely throughout the lifetime of the project.
Concept
It has become a point of interest amongst the iLab Developer team, and also amongst its users, as to
where in the world users are coming from, and in what levels of demand the LabServer (specifically
the CEIT LabServer) experiments are in. To satisfy this interest – the idea arose of an additional
client, the SBS client, that could be attached directly to the LabServer, and that provides detailed
statistical information concerning the iLabs usage. Exactly how and what types of statistical
information to be displayed will be discussed further in the Project Plan section of this report.
It has also been realised, that an issue could arise from activation of this project in terms of server
usage. Since the SBS system would be attached directly to the LabServer, and many SBS clients could
be synchronously attached, this poses a severe problem. Normally under such circumstances, a
simple asynchronous polling of the LabServer over a significant time delay would be sufficient,
however future considerations for extensions on this project leave this method lacking, as traffic will
increase dramatically when the number of SBS clients increases.
A proposed solution to this method comes in the subscription feature of the SBS client. Using a
separate server, clients can subscribe to different types of information from the LabServer. Then
when a change is made to the LabServer information or a time period has passed, the LabServer
sends a single, small volume xml file to the separate Subscription Server containing the
subscriptions, which then proceeds to notify all the subscribed clients of the change. This method
will significantly decrease the LabServer usage by moving the responsibility of notification to the
separate Subscription Server. Technologies for this separate server are available and will be
discussed briefly in the Project Plan section of this report.

Figure 2 - The proposed Subscription Server based SBS client

7

Motivation
The motivation for development in a system such as the SBS spans multiple areas. First and
foremost, a system such as this will provide developers and users with a reliable and up to date
indication of where and how the CEIT LabServer is being used. This information would not only be
useful for publication of results or maintenance (e.g. A piece of equipment returning inconsistent
results), but would also spur on additional funding and allow developers to better realise areas of
demand. It might also enable potential user bases to better realise the capabilities of the system and
how widely it is being used, possibly spurring them to adopt it themselves.
Beyond this, there are future considerations for the project (especially concerning the subscription
model) that will be discussed in the Future Extension section of this report.
Scope
Properly describing the scope of the SBS project is extremely important. The reason for this, is not
only because of the massive size and scope of the iLabs project, and even larger when you include
the ServiceBroker and its LabClients, but because any alteration of code in these areas could affect
the performance of the LabServer itself. There are three main components to the SBS project, and
these are outlined in detail below.
LabServer SUS (Subscription Updater Service)
This service is an extremely important part of the SBS project, whilst also being the least visible. Its
responsibility will be to update the Subscription Server of changes that occur in the LabServer
concerning the experiments. The reason utmost care and consideration must be made in the
planning and development of this portion of the project is because it will be the sole component of
the SBS project that will have an effect upon the performance of the LabServer.
Another attribute of the SUS that is worth pointing out is that its connection to the Subscription
Server is monodirectional (SUS to Server), and asynchronous. That is to say that no connection is
constantly maintained, all that happens is the SUS will submit an updated xml file with new
information to the Subscription Server at intervals defined by either a time period, or availability of
some important new information, and will never deal with requests.
Subscription Server
The Subscription Server is the component of the SBS system that provides its robustness and ability
to extend upon the systems scope for future development. This server is responsible for two things.
First and foremost it must allow SBS clients to subscribe to the xml data from the LabServer, and
secondly, it must be capable of receiving updated data from the LabServer and notify all its
subscribers of the change. Current prototype plans for this functionality will only allow a single
subscription to the entire LabServer xml data, however it is possible that during development of the
SBS client, separate subscriptions to different parts of the LabServer xml data become available in
order to reduce the load on the system and increase robustness.
SBS client
The SBS client will be the main workload of the project, and also the most visible component. Its
features and functionality are as yet completely undefined, yet to fulfil the motivations and to create
a successful outcome for the project – it will need to satisfactorily provide statistically relevant
information concerning the current and past operation of the iLab projects experiments. Some ideas
8

for features and functionality that would provide this are outlined in the Project Plan section of this
report, however these features are expected to be revisited and reanalysed during development,
with the possibility of further extensions.
In order to receive the data for its function, the SBS client must subscribe to the Subscription Server
and receive updates from it when it in turn is updated of the LabServer status. This passage of
information will all be done via xml.
Future Extension
It has long been a desire of the LabServer development team to reduce the iLab projects reliance on
the ServiceBroker. The reason this is the case is because the LabServer package offers no user
friendly interface for experiments, and is only capable of authenticating requests from a
ServiceBroker. This means that opening the LabServer to public directly would potentially bury the
system in dealing with user requests and validation.
However, with the introduction of this project and its subscription based communication, a new
method of application execution could be permitted - which would remove all reliance on the
ServiceBroker as anything more than a security service. By implementing the Subscription Server,
and allowing it to communicate with the LabServer in a bidirectional fashion (a concept that must be
explored further to implement this extension), you could place a LabClient in place of the SBS client,
obtain an authentication code from you're ServiceBroker, and then simply communicate directly
with the LabServer through the Subscription Server without any further reliance on the
ServiceBroker for notifications or experiment handling.
The plausibility's and possible benefits of this concept will be explored further by the development
team at the end of the SBS systems development.
9

Project Plan
Technologies
This section of the report is concerned with the available technologies that could be implemented in
each component of the SBS system. Some of these technologies will be prototyped to better
compare and contrast benefits of alternatives, and others will be discarded in the face of significant
flaws in their capabilities.
LabServer SUS
The LabServer SUS system will not largely benefit from any new technology not already present
within the existing LabServer. It will extend upon the existing C# codebase of the LabServer system
by providing an automated client to the Subscription Server that uploads xml updates of the
LabServer system status.
Subscription Server
An available technology that could assist the Subscription Servers operation is the concept of xmpp.
XMPP is a widely available technology that is used commonly worldwide in instant messaging. There
are freely available downloads online that offer server packages to make use of this technology, and
it is possible that it could be easily adapted to handle the subscription and xml messaging features
required by the SBS system. The benefit of using this technology (apart from the possible rapid
deployment), is that it has been widely tested in Instant Messaging applications all over the world,
and is known to be extremely robust, capable of maintaining several thousands of simultaneous
modules.
An alternative to this technology would be to create a lightweight C# service. Being a service, the
LabServer code could simply contain a client to this service, and make a cross domain call via use of a
clientaccesspolicy.xml file (in the case of WCF) to make an xml update whenever required. The
service could respond to these updates by using the duplex services package (or other similar
package depending upon the SBS client) to make a server side call to all registered SBS clients of that
duplex method.
An issue that is recognised in both of these methods is the circumstance where the SBS client
requires a large amount of historic data from the LabServer, in which case the best option would be
to store historic data on the Subscription Server somehow. This means that rather than being
consistently updated by the LabServer with these large xml files - the LabServer can make a lazy
update (eg. once per day) of either the entire history (with some reasonable cut-off date) or just an
update of the last days activities to append to the existing xml history on the Subscription Server.
The server, in turn, would only need to respond to subscription calls from clients to send this data,
and the also be on a lazy update (eg. daily or whenever it receives new data from the LabServer).
The capabilities of the xmpp technology to be able to satisfy this requirement are uncertain.



10

SBS Client
There are several technologies that could assist in creating a SBS client. Whatever is chosen, it must
satisfy the following:
· Must allow server side calls to be able to be updated from its subscriptions and not have to
continuously poll the Subscription Server
· Must have a comprehensive charting library and UI control set for better display of statistical
information, and ease of development and use
· Must be sufficiently cross platform to be able to run on Windows, Mac, and Linux, would also be
beneficial to be functional on portable wifi devices such as IPhones
· Must be capable of being hosted and maintained effectively and easily on a CEIT server
· Must be a relatively recent technology so as not to be outdated and become no longer
supported in the near future
Finding a technology that comprehensively satisfies all these requirements is not easy. Flash/Flex,
Java Applet, Adobe AIR are just a few and all are lacking in one area or another, whether it be a lack
of complex server side implementation, a lack of libraries, or a lack of compatibilities.
A technology that does stand out however, is Microsoft Silverlight. It is capable of working with a
.NET server side, which would maintain the current LabServer codebase in a single code format, and
is extremely powerful, containing the duplex services which allows server side calls based on client
registry (similar to subscription). It contains an extremely comprehensive set of controls and charting
libraries (not to mention third party libraries), and also contains a comprehensive list of effects and
UI stylings (especially with the assistance of the Microsoft Expression Studio). It can be hosted in IIS7
so can definitely be managed by the CEIT servers as the LabServer is hosted in the same fashion.

Figure 3 - A simple example of the Microsoft Silverlight Charting Libraries
11

The downside to the use of the Microsoft Silverlight technology is that it isn't normally supported on
Linux or portable wifi devices such as the IPhone at the moment. Novell Mono's Moonlight however
has gone some way into filling this gap, already providing a freely available open source
implementation of Silverlight for Linux and other Unix based operating systems, and work on
extending this implementation to devices such as the IPhone is underway.
12

Proposed Features
This section of the report is concerned with the significant features that could exist in each
component of the SBS system. Some of these features are absolutely necessary to satisfy the
motivations of the project, while others are simply ideas that may flow through to develop or might
be discarded in the face of implausibility, security, or any other factor. This is also by no means a
complete list of features that will be pursued, just an outline of the major ones.
LabServer SUS
The SUS will have a single feature only - that is to periodically, whether by a timer or by some
significant enough change in the LabServer usage, send an xml file of data to the Subscription Server.
In the early prototypes, it is likely that this information will be sent via a single xml file that will be of
small volume, however it is the intention of the developer to make several different periodic xml
updates (perhaps one for each page, section or tab of the SBS client), that updates separately from
each other and at different times.
Subscription Server
The Subscription Server must have 3 main features. Firstly, it must contain a service for the SBS
client to be able to subscribe to it (this could exist either within the subscription server, or on the
serverside of the SBS client).
Secondly, it must be able to store the xml data of currently available LabServer information. This
feature is especially important for parts of the SBS client that require large amounts of historic
information, otherwise the Subscription Server would be heavily reliant on rapid consistent updates
from the LabServer SUS to service these clients.
Thirdly, it must be able to redirect the xml data to its subscribed clients, both at subscription time
(with use of its saved xml), and at every periodic update of that data that the Subscription Server
receives from the LabServer SUS.
SBS Client
The SBS client is the actual user interface of the entire SBS system. Its required features can't yet be
clearly described as it is unknown what levels of data complexity the LabServer SUS and Subscription
Server are capable of supplying to this interface. Once a simple prototype, for example a very basic
client that displays a current queue length for various LabExperiments is created, the possibilities
and plausibility's of various additional features will become more clear.
Nevertheless, some ideas and concepts that are hoped to exist in the final client include:
· A calendar for selected experiments showing times of activity, and capable of displaying
individual experiments that occurred along with their parameters and results
· A charting wizard allowing a user to create a chart comparing certain attributes of selected
experiments over a chosen time period (eg. select line chart, select queue lengths attribute,
select radiation experiment, and choose 12pm to 6pm yesterday)
· Some graphical display capable of showing for selected experiments, where in the world they
are being used, and in what quantities (using ip addresses or google tools)
13

Timed Targets
The following is a rough estimate and guide for timed targets in the development of the SBS system.
Most of the items in this list are only large and general components such as some of those
mentioned in the proposed features.
Item

Semester/Week

Date

Create a dummy LabServer for testing

with very a
basic SUS
1/6

16/4/10

Create a prototype of the Subscription Server

that only transfers data, doesn't keep a copy
1/7

23/4/10

Create a prototype of very simple SBS client
showing Subscription Server connection
1/8

30/4/10

Develop the charting wizard

along with its
required Subscription Server and SUS
implementations
1/10

14/5/10

Improve the Subscription Server to store xml and
reduce reliance on consistent SUS updates
midyear

23/7/10

Expand the client to include historical data of
experiments (e.g. calendar)
2/3

13/8/10

Develop a method of viewing where in the world
the users of various experiments are
2/6

3/9/10


The few weeks left available at the end of the project have been deliberately left empty to provide
for any additional time required to publish a presentable SBS client, and to prepare the thesis report,
presentation, and demo, which are all due towards the end of the second semester.

14

Bibliography

P.Tweed, "Server calling the client - duplex WCF Services in Silverlight", Exploring and explaining the
mysteries of .NET, July 2009. [Online]. Available: http://geekswithblogs.net/PeterTweed
/archive/2009/07/28/server-calling-the-client---duplex-wcf-services-in-silverlight.aspx
[Accessed: 7/4/2010]
ignite realtime, "Openfire 3.6.4", May 2009. [Online]. Available: http://www.igniterealtime.org/
projects/openfire/index.jsp [Accessed: 8/4/2010]
mono, "Moonlight", December 2009. [Online]. Available: http://www.mono-project.com/Moonlight
[Accessed: 6/4/2010]
L.Payne, "ILab LabServer Project", December 2009. [Online]. Available: http://ceit.uq.edu.au/wiki/
index.php/ILab_LabServer_Project [Accessed: 10/3/2010]
B.Czernicki, "Silverlight advantages over Flash", Silverlight Hack, August 2008. [Online]. Available:
http://www.silverlighthack.com/post/2008/08/12/Silverlight-advantages-over-Flash-%28not-
another-Silverlight-vs-Flash%29.aspx [Accessed: 10/4/2010]
Microsoft, "Silverlight for Windows Phone", Microsoft Silverlight, 2009. [Online]. Available:
http://www.silverlight.net/getstarted/devices/windows-phone/ [Accessed: 12/3/2010]