RTI Interoperability Study Group Final Report

mustardpruneΔίκτυα και Επικοινωνίες

23 Οκτ 2013 (πριν από 3 χρόνια και 9 μήνες)

202 εμφανίσεις

RTI Interoperability Study Group Final Report


Michael D. Myjak

The Virtual Workshop

P. O. Box 98 Titusville, FL 32781

<mmyjak@virtualworkshop.com>


Duncan Clark

IBM Hursley Services & Technology

Winchester, Hampshire, U.K. S021 2JN

<Duncan_Clark@uk.ibm.
com>


Tom Lake

InterGlossa Ltd.

31A, Chain Street, Reading, UK RG12HX

<Tom.Lake@glossa.co.uk>


K
EYWORDS

HLA,

RTI,

I
NTEROPERABILITY
,

M&S,

S
IMULATION



A
BSTRACT
:

Under the High Level Architecture (HLA), as defined today, federates are limited to interopera
ting within
the context of a single federation execution, the data for which is described in a common Federation Object Model
(FOM). However, federates and a common FOM do not create an interoperable environment. Instead, federate
interoperability in the
context of HLA is accomplished using the HLA Federate Interface Specification, an Application
Program Interface (API) definition which refers to services provided for by a distributed software component referred
to as the Run
-
Time Infrastructure (RTI). In
other words, inter
-
federate interoperability is totally dependent upon an
RTI, the definition for which is not part of the HLA specification.


In this paper, the RTI Interoperability Study Group addresses the issue of functional HLA interoperability and
re
usability. We recognize that without a standardized RTI or RTI
-
to
-
RTI
interfaces, RTI from different vendors are
functionality neither compatible nor interoperable with one another. The result is the adoption of
a specific RTI,
produced by a specific vendo
r, often for only a limited set of platforms. This results in the loss of reusability, leading to

distributed stove piped solutions that are not very different from their noninteroperable brethren: the stand
-
alone
systems. Therefore, we've

identified the

need to expand upon the existing definition of the High Level Architecture. W
e
identify a new taxonomy, based upon the concept of HLA interoperability in a heterogeneous environment, and
introduce the concept of a Federation Community.

Because differing

definitions and views of interoperability result in

standards that are neither compatible nor interoperable, we define the term Interoperability, with respect to
distributed simulation systems.
Further, we describe the support constructs necessary to ach
ieve functional
interoperability under our expanded definition of HLA, and evaluate these in light of several common use case
scenarios.
We describe the four architectural variants of HLA systems, specifically dealing with the interoperability of
heteroge
neous or homogeneous RTI or FOMs. And we
present four different approaches to achieving interoperable
solutions with HLA: the
Gateway, Proxy, Broker, and protocol
-
based solutions
, each requiring varying levels of
sophistication and complexity to interface

between the four architectural variations of the HLA
.


1

I
NTRODUCTIO
N AND
O
VERVIEW

Interoperability can be a very fragile and yet intangible
thing, with different meanings and different requirements
to different people. It can be measured, declared, and
even experienced, right up until the time it's lost. And
interoperabi
lity within HLA is just as ethereal. For
example, we "know" when two software modules
interoperate only after long sessions of testing, analysis
and evaluation, as software typically evolves from brittle
prototype through robust system in a process of cont
inual

evolution. Yet are we
sure

that we have an interoperable
solution? Experience has taught us that we can often tell
rather quickly when two software modules will not
interoperate, for it is easy to disprove a hypothesis. One
need only find an insta
nce where the hypothesis is false.
To prove that a thing is true, however, takes experience,
practice, and often patience. And achieving and then
maintaining interoperability between distributed
computing applications can be very difficult to prove
indee
d.



Figure
1
. The OSI Reference Model.

1.1

Background

To strive for interoperable solutions is a noble goal. To
further this quest and embrace software reusability, as the
HLA is trying to do, is to follow

in the footsteps of those
that came before us: the International Standards
Organization and the
Open System Interconnect (OSI)
initiative. The OSI standards, taken as a whole, were
designed to provide reliable, end
-
to
-
end communications
between computers

of different origin, thus removing an
applications constraint of commercial dependency from a
particular vendor or platform [Chapin, 89]. The OSI
standards gave the computing industry the capability to
exchange files, data and electronic messages through

well
-
defined and reliable communic
a
tions services. In short,
applications were made to interoperate over
heterogeneous environments. Furthermore, the design
goal of the OSI initiative was accomplished entirely
without the knowledge of the architectural d
etails of any
vendor’s platform.

1.2

Then…

Recall that as this decade began, there was no commodity
"Internet" like we have today. Personal Computers (PCs)
based on a clone of the original IBM PC didn't, for
example, speak AppleTalk. The Macintosh didn't spe
ak
DECNet, and the Novell Network Operating System, based
on IPX, didn't communicate with either (or any, as some
claim). This was the great age of protocols, where every
vendor had one to call their own. Intergraph purported
XNS, while Symbolic's touted

ChaosNet, and so on.
Network vendors had a venerable field day, as they
developed new and proprietary solutions called
"Gateways" that would transfer information from one
network to another. It was from this climate that the OSI
protocol standards evolv
ed, born out of a need to allow
isolated systems or “closed” groups of hosts to be
internetworked

together [Comer, 91]. In other words,
these systems were now “
open
” to one another,
r
e
gardless of the specific vendor platform, opera
t
ing
system or applicati
ons involved.
1



As a basis for development of the OSI standards, ISO
developed the 7
-
layer reference model to partition the
problem into discrete
layers
. These conceptual layers
provided a framework for u
n
derstanding the complex
processes involved in co
mputer communications.
Without such a model, the problem of making
applications, distributed across heterogeneous computer
systems, communicate and thus
interoperate

with one
another, was considered intractable.


In order to get a handle on this intracta
ble problem, the
ISO reference model defined a protocol as “… the private
language and procedures that are defined for each layer”
[Rose, 90]. It is these protocols that allow one to specify
(and unde
r
stand) the communications interface at the top
-
most la
yer without knowing the details of an u
n
derlying
layer (including the vendor's hardware or software
architecture). Ultimately, it is this prior knowledge that
permits user applications to interoperate.


The ISO reference model, shown in
Figure
1
, denotes the
application layer protocol at the top of the stack. This
protocol is principally concerned with providing a well
-
defined and reliable API to support the end
-
user’s
applic
a
tion
and

from shielding it from the underlying
protocol
s (e.g., transport, network, hardware, etc.).
Conversely, the transport, network, and data link layers
are primarily concerned with the connection
-
oriented
transmi
s
sion of data only between the layer immediately
above or below.




1

For additional information on the OSI protocols, refer to
[Rose, 90], [Henshall, 88], [Stallings, 87] and [Knightson,
88]. A detailed listing of all r
elevant OSI standards can be
found in [Chapin,
.
89].

1.3

And now…

However, we do not

use the OSI protocols today over the
commodity Internet. The requirement of OSI, that
reliability in communications be maintained at each level of

the model hierarchy, was shown as being fundamentally
flawed. Certainly OSI networks themselves were relia
ble,
but this in no way diminished the application's
requirement of ensuring functional reliability. For example,
if a Network File System (NFS) write command is executed,
the sending application, the one requiring the
acknowledgement, is not interested i
n receiving that
acknowledgement from then layer immediately below, or
even from the transport layer of the target host. Rather
the acknowledgement desired became one indicating that
data has been successfully written through the cache and
onto disk, a fu
nctional task only the application can
answer.


Thus a great debate arose over
where

certain functionality
should be placed within a communication architecture.
Further, choosing the proper boundary between functions
is perhaps the primary activity of t
he designer of computer
architectures. The design principles that provide
guidance in this choice of function placement are arguably
among the most important tools a systems designer carries

with him or her. [Saltzer, 84]


Consider for a moment, the usefu
lness of this argument as
it pertains to reliable communication in a shared
distributed simulation application like the HLA. In
particular, should the function of reliable communications
be managed internally, at each hop along the way from
source to dest
ination(s)? After all, reliability might be
accomplished using a number of different techniques;
such as selective redundancy in the form of packet
checksums, sequence number checking, or internal retry
mechanisms, for example. By exercising enough care, t
he
probability of an undetected bit error occurring along the
communications path is reduced to an extremely low level.
The question here, however, is not whether or not these
low
-
level data validation techniques are
valid
, but whether

or not they are
use
ful

to a higher level, distributed
application.


In a system like that represented by the High Level
Architecture, one which embodies end
-
to
-
end
communications as a fundamental premise, the architect
usually draws a modular boundary around the
communicatio
n subsystem and defines a firm interface
between it and the rest of the system.
2

In the HLA



2

Traditionally, this Application Program Interface (API) is
the roll of an application layer protocol. [Myjak, 97]

however, this boundary is defined between the RTI and
the federate. When defining this API, it becomes apparent
that there are a plethora of functions, each of whic
h might
be implemented in one of several ways:



By the communication subsystem



By the sending or receiving host



By the sending or receiving RTI component



By a server or client application, or



As a joint venture among two (or more) peers.


In reasoning abo
ut the choice of where to place certain
functionality, the requirements of the application provide
the basis for the following argument:


Can the function in question be completely and
correctly implemented only with the knowledge and

help of the applicati
on standing at the endpoints of
the system?


This line of reasoning, often directed against low
-
level
function implementation, is called the end
-
to
-
end
argument. [Saltzer, 84]


It was precisely this line of reasoning that the developers
of the Internet Pro
tocol (IP) chose to follow when they
evolved their own 5
-
layer model, shown in
Figure
2
.
Under IP, the cost of reliable communications below that
managed by the hosts at the endpoints was considered
redundant and inefficient, par
ticularly if the transport layer
had to guarantee end
-
to
-
end delivery anyway.



Figure
2
. The Internet Protocol Model.

There is an important lesson here for the simulation
community that can be drawn fr
om these humble
beginnings. The ISO OSI initiative felt that
interoperability in a heterogeneous environment was
unmanageable without a model that defined an a' priori
solution down to the "bits on the wire" level. The
evolution of the Internet Protocol d
emonstrated that
refinement and improvements could still be made, further
streamlining the communication process. Both of these
architectures support application
-
level interoperability
without dependencies on a particular vendor or
implementation. Both o
f these architectures were
designed from the bottom
-
up, with the intention
guarantees of interoperability could only be constructed
upon a solid foundation. Therefore, how can we, as a
community, declare the construction of an interoperable
federation, wh
en said federation is based entirely upon a
nonstandard RTI? Indeed, this is one of the questions
that the RTI Interoperability Study Group has been
wrestling with.

1.4

The High Level Architecture (HLA)

End
-
to
-
end interoperability in a heterogeneous
environ
ment should, by now, be a lesson learned, rather
than a lesson lost.

We should know that first obtaining
and then second maintaining software that is functionally
interoperable is a very difficult task, one that should not
be undertaken lightheartedly. I
ndeed, it takes a lot of work

to develop a specification that describes an interoperable
implementation in an unambiguous fashion. True
interoperability among dissimilar platforms from various
vendors must be built from a solid foundation (i.e., the
bottom
-
up approach).

1.4.1

The Nuts and Bolts of HLA

Within the HLA, several low
-
level factors need to be taken

into consideration in the design, implementation
and
specification

of the RTI that affect both usability and
interoperability. For example, maintaining t
he shared
internal state among RTI local components must be
consistent across the entire federation. If this consistency

varies from vendor to vendor, then RTIs will not be able to
interoperate, and one level, a very important level, of
interoperability w
ill have been lost: interoperability in a
heterogeneous environment. Further, some mechanism
must be made available to support late joining federates or
federates that resign early. In the case of late joiners, both
federate
-
level state information as we
ll as RTI internal
state information must be passed to both the late joining
federate and its local RTI component, respectively. In the
case of the early retirement of a federate, some mechanism
within the simulation must be made available to transfer
the

ownership and state (both public and private) for the
objects managed by the retiring federate. Similarly, the
resigning local RTI component must be dealt with, as this
change in RTI internal state must be incorporated into the
operational federation.


T
here are other factors involving HLA interoperability as
well, factors that may appear to be more obscure, both to
the simulation application developer as well as to the RTI
vendor. For example, we've all heard the cry for an HLA
network protocol, one tha
t in part, draws similarity to the
DIS application layer protocol. However, such an
undertaking is not trivial under HLA. Such a protocol
must be able to scale well, perhaps on the scale of the
Internet, in order to support large numbers of users
and

obj
ects. Such a protocol must also take into consideration
the reliability or resilience issue of each underlying data
stream used to support the RTI. Indeed, the HLA RTI
itself must have a foundation upon which it is built; a
communication architecture. Of
ten we find that there is a
tradeoff between real
-
time interactivity, scalability and
congestion control. And of course, if HLA is truly going
to be useful, the HLA RTI must be capable of
interoperating over a heterogeneous environment.

1.4.2

Functional Interoperability

Beyond the low
-
level interoperability issues are those of
the higher level simulation: func
tional interoperability.
This is the level of interoperability that most interests
simulation developers and users. If the end result is a loss
of functionality, or the generation of ambiguous or
inconsistent results, or that a given implementation cannot

be validated, then interoperability at the functional level
has been lost. For instance,
the Logistics community
desires HLA federates that not only publish their own
objects, but also divest ownership of them as well. To this

community, this is a funda
mental premise of
interoperability at a functional level.
[SISO
-
OSP, 99]
However, HLA as it is defined today, only supports the
acquisition and divestiture of an object's attributes. From
the logistics standpoint, once an item is shipped, they are
done
with it. Certainly, one can understand why a
logistics simulation would not want to maintain ownership
of all the objects shipped into a
Theater Level

simulation
during run time.


Not only is interoperability difficult to quantify, but it is
just as diffi
cult to ensure that it has been achieved,
particularly at the functional level. For that matter, how
does one know whether a given implementation will
interoperate with another? Can anyone truly
guarantee

interoperability? Perhaps not to 100%, but result
s
approaching that have been achieved. However, it is
possible to test for interoperability, at a functional level,
by using a process known as functional equivalence or
replacement. By replacing one implementation with
another, written from the same spe
cification but developed

from a genetically different code bases, a lot can be
learned about the interoperability derived from said
specification. Should the replacement operate in
precisely

the same manner as the original, having passed all
possible oper
ational tests, then one can at least be
assured that functional interoperability has been
identified, if not achieved, in this case.

1.4.3

The Summation on HLA

All this makes HLA interoperability (and thus reusability)
rather problematic. HLA is defined by a se
t of Rules [draft
IEEE 1516], a Federation Object Model (FOM) that is
described in Object Model Template (OMT) format [draft
IEEE 1516.1], and the Federate Interface Specification
[Draft IEEE 1516.2]. However, HLA federates, defined in
accordance with the

HLA Rules, and using a FOM defined
in OMT format, cannot interoperate with one another
using the Federate Interface Specification alone. Instead,
a fourth piece to this puzzle is required: the ubiquitous
Run Time Infrastructure (RTI). However, the RTI i
s not
part of the HLA specification, nor is the Federate Interface
Specification complete enough to provide a concise
definition for how to construct an HLA RTI. Further, the
specification as currently written, is ambiguous and lacks
both operational expe
rience and interoperability testing.


For HLA RTIs to be developed that interoperate, a
complete, concise, and unambiguous specification for
both the RTI and all of its interfaces needs to be
developed. Each RTI interface needs to be capable of
communicat
ing with another RTI local component in an a'
priori fashion, with an understanding of all of the various
low
-
level parameters (e.g., to support internal time
management and DDM state) and nuances required for
interoperable execution.


Limiting the HLA to

operate within the predefined context
of a common, homogeneous FOM, over a vendor
-
specific
RTI implementation, will lead the M&S community directly
back to the stovepiped days of SIMNET. By requiring all
federates to join, then publish, then subscribe, t
hen
register their objects before a given execution can begin,
eliminates the ability for federates to join late. Not
permitting object transference can similarly limit the ability
of a federate to depart early.

1.5

Terms of Reference

The Study Group mechani
sm is intended to provide a wide
range of flexibility in filling in the gaps between the
requirements identified by the Simulation Interoperability
Workshop Conference Committee (CC) forums, and
requests for product developments, administered by the
Standa
rds Activity Committee (SAC). Study Groups are
established using the Terms of Reference (ToR) and may
include defining key terminology, recommending
modifications to SISO processes, or generating a plan for
(or an initial prototype of) a proposed SISO pro
duct.
Upon their conclusion and final report, the SISO EXCOM
may then decide to form follow
-
on Study Groups to further

develop certain recommendations or to generate a
Standards Nomination that may (if approved) proceed
through the Standards Development P
rocess defined in
the SISO Policy and Procedures. The RTI Interoperability
Study Group ToR were approved in the fall of 1998, and
are as follows:



Define the term "interoperability" as applied to
HLA interoperability and the derived issue of RTI
to RTI in
teroperability.



Provide an assessment of current RTI to RTI
interoperability limitations indicating the
architectural and configuration issues.



Provide an assessment of the community impacts
that may result if RTI to RTI interoperability is not
achieved.




Establish through community discussion and input
the requirements for RTI to RTI interoperability.



Examine and evaluate the benefits and impacts of
possible approaches to achieving RTI to RTI
interoperability.



Recommend an approach that will support an

appropriate and necessary level of interoperability
between RTI implementations.



Figure
3
.

Distributed Simulation
Interoperability Layer
Model
.

1.6

Report Structure

The remainder of this paper reports back o
n the study
groups findings and hence fulfills the Study Group's
Terms of Reference. Section 2 summarizes the problem as
identified in this section and puts forward a structure for
discussing the issues associated with HLA
Interoperability. Section 3 prov
ides a high level view of
the different types of User Requirement expressed to the
RTI Interoperability Study Group together with a
definition of the terminology used in this report. Section 4
provides a number of "use case" examples that explain the
requ
irements for HLA interoperability and provide some
perception on the impacts of not achieving interoperability

for some potentially realistic situations. Section 5
identifies some solutions for achieving interoperability.
Some of these could be realized w
ithin the scope of
current non
-
interoperable RTI implementations, while
others require new specifications or concepts yet to be
evolved. Section 6 summarizes the conclusions of the
working group.

2

D
EFINITIONS
&

L
EXICON

Over the course of the past year, the

RTI Interoperability
Study Group drew to closure on the problems with HLA
Interoperability. To resolve this issue, in light of the
group's definition for Interoperability, a need was
expressed to expand upon the existing HLA lexicon. This
was necessary
because the existing documentation didn't
cover some of the architectural variations that could easily
be extrapolation given a non
-
standard RTI, the foundation
of the existing HLA. For this reason, we needed to evolve
new terminology that complimented ex
isting HLA
nomenclature. The next section covers the RTI
Interoperability Study Group Lexicon. The section
following that addresses the implications and variations of
the High Level Architecture.

2.1

HLA RTI Lexicon

Adjacent Federation



Where two or more f
ederations
have similar objects and/or interactions, but are
defined by two or more different FOMs.

Communication Architectures



The principal
architectures used to provide distribution of RTI state
information between RTI components in an execution.
The
re are three types (plus hybrids): Networked,
Shared Memory, and Parallel architectures.

Community



is a combination of federations and RTIs
working together to achieve a common goal. There are

4 combinations of homogeneous and heterogeneous
federations a
nd RTI.

Federate Gateway



A translation device that
interconnects two (or more) federates (unlike the
Federate Proxy), each presumably joined to different
federations or non
-
HLA simulation applications,
within the same federation community.

Federate Proxy



A translation device that interconnects
two (or more) RTI Implementations using the Federate
Interface specification (unlike the Federate Gateway),
within the same federation community.

Homogeneous Federation



A named set of federates
within a communit
y using a common Federation
Object Model (FOM).

Homogeneous FOM



A common Federation Object
Model (FOM) used by a named set of federates within

a community.

Heterogeneous Federation



Named sets of homogeneous
federations, within a community, that use mu
ltiple
FOMs. There are three classes: Adjacent Federation,
Intersecting Federations, and Hierarchical
Federations.

Heterogeneous FOM



Sets of Federation Object Models
(FOMs) used by named
sets

of federates, where each
set is contained within a homogeneous

federation
within a community.

Hierarchical Federations



One federation appears as a
federate or federate component of another federation.

Heterogeneous RTI



An RTI that includes several RTI
Implementations that are not capable of directly
sharing state

so as to support the rules and
specifications of the HLA.

Intersecting Federations



A federation that shares some
common objects and/or interactions with another
federation.

Local (or Federate) RTI component



A component of an
RTI that is installed an
d implements the RTI Federate
Interface Specification (e.g., RTI Ambassador) to a
single federate.

RTI



A global collection of one or more RTI components
that communicate data and control information
necessary to support multiple federates in accordance
with the rules and specifications of the HLA.

RTI Broker



Is a translation device that uses the RTI
-
to
-
RTI API to pass not only federate level data (as is
usual and customary for the RTI) but to communicate
RTI internal state information between two (or
more)
RTIs, potentially from different vendors or
conforming to different communication architectures.

RTI Component



A component of an HLA RTI
implementation.

RTI Implementation



An RTI from a single vendor that
shares common private code and state info
rmation;
RTIs that do not naturally share identical state are
different implementations, even if they are from the
same vendor.

RTI Interoperability Protocol



A protocol capable of
transporting federate information and RTI internal
state between RTI impl
ementations or between local
RTI components in a manner independent of RTI
vendor.

2.2

Architectural Implications

The High Level Architecture evolved around the idea that
a single federation, composed of named collection of
federates, and joined through a vend
or
-
specific RTI would
interoperate using only data defined in the shared
common FOM. In the Study Group's Interim Report,
presented at the Spring SIW in March 1999, we called this
variant of the HLA the "Homogeneous Federation
coupled to a Homogeneous RTI
." Since a federation
could potentially involve some connectivity to another
federation using communications outside of the RTI, it is
conceivable that a
heterogeneous

federation could also be
constructed. In a heterogeneous federation, different
Federati
on Object Models are involved. For example, a
federation could support multiple levels of security.
[Filsinger, 97] But interoperability problems quickly arise,
such as the architectural limitations identified with
ownership transfer. [Bouwens, 98] Further
, if two RTI
vendors decided to create an agreement (e.g.,
Interoperability Protocol) where by their two dissimilar
RTIs would interoperate with one another, then a
heterogeneous

RTI could potentially evolve.


The following sections present these four arch
itectural
variations of heterogeneous and homogeneous RTIs and
federations. Each variation, whether homogeneous or
heterogeneous FOM or RTI, represents a federation
community. In each instance, data is passed between the
federates. In the cases involvin
g heterogeneous FOMs, a
translation mechanism is necessary, and will be discussed
in section
5
,
Range of Solutions and Issues
. Similarly,
communication mechanisms for dissimilar RTI
implementations wi
ll also be discussed.



Figure
4
. Homogeneous FOM coupled with a
Homogeneous RTI.

2.3

Homogeneous FOM & Homogeneous
RTI

This is the current mechanism by which interoperability is
achieved between federates usin
g HLA today. This
method is already producing benefits and accounts for
most of the HLA applications that have been developed to

date.

2.4

Homogeneous FOM & Heterogeneous
RTI

In this situation communities have been concerned that a
single RTI Implementation
will not be able to meet their
needs for a variety of possible reasons:



Platform Support
: including hardware, operating
system and language.



Communications Architecture
: The means of
distribution may be different in different parts of the

community.



Perfor
mance
: Performance needs for particular
services or particular aspects may need to be
optimized.



Services
: There may be situations where an RTI
may deliver performance in a specialist area but not
deliver the necessary services.

There are many reasons why
a single, monolithic
approach to constructing HLA RTI implementations will
not work. HLA interoperability in a heterogeneous
environment is but one answer. Further, compelling
evidence has already shown that this path will
eventually lead the community ba
ck to stove piped
implementations. [Briggs, 98]



Figure
5
. Homogeneous FOM coupled with a
Heterogeneous

RTI.

The simple solution here, and often reiterated within the
RTI and Communications Forum, was to
develop an "on
the wire protocol" to permit RTIs from dissimilar
vendors to communicate. However, I think we will show
that while this might indeed be the optimal approach, it
is not the easiest of undertakings.

2.5

Heterogeneous FOM & Homogeneous
RTI

Situati
ons have arisen where projects wish to use multiple
federations working together to achieve their objectives.
These objectives may be to support multiple levels of
resolution within the object description of a FOM, or to
devise different levels of fidelit
y or object decomposition.
This arises from situations such as:



Figure
6
.
Heterogeneous

FOM coupled with
a Homogeneous RTI.



Use of legacy systems

where it may be difficult or
cost
-
prohibitive to develop
new FOMs.



Information Security

where some information
needs to be filtered or declassified before passing
to federates in another federation.



Differing Levels of Fidelity
where objects exist in
two federations but with different representations

Whatever th
e reason, the HLA architecture should be able
to support these requests in an open and consistent
manner.

2.6

Heterogeneous FOM & Heterogeneous
RTI

In the generalized community, cost
-
effective vendor and
platform
-
independent solutions may drive situations
whe
re both heterogeneous FOM and heterogeneous RTI
must interoperate. [Briggs, 98] The concept of a Joint
NATO federation, or a Joint U.S. Services federation,
where each service purports a different set of
requirements, is anticipated. The interoperability
issues
underlying the RTI services (e.g., time management,
ownership management, etc.) may yield architectures such
as this.



Figure
7
. Heterogeneous FOM coupled with
a Heterogeneous RTI.

3

P
ROBLEM
S
TATEMENT

What is Interoperability? What does it mean to have
achieved interoperability and how will we know when
we’ve achieved it? Will it enhance the art of distributed
simulations or will simple equivalence suffice? For that
matter, what does it mean to have
equivalent or different
RTIs? These are the questions the RTI Interoperability
Study Group has wrestled with.


It is usual in simulation practice to consider the modeled
events (the
enactment
) to be the logical content of the
simulation: the result of our

efforts, if you like. These
include the various and contingent details of the scenario,
how we carried out the simulation and what we call the
physical content of the simulation, containing all the detail
of the actual apparatus and runs that produced the

results.


As an analogy, it may be helpful to think of a program in a
programming language. The program should have a
meaning that is the same for any processor or platform,
with the exception of properties (e.g., timing, precision,
safety, etc.) which ma
y not be specified in the language
semantics directly. Of course, with the appropriate
techniques we can construct a program that can measure
the precision or performance characteristics of the
processing environment. Such programs are dependent
on the u
nspecified aspects of the programming language
semantics; they are not illegal but they neither are they
portable between environments
--

indeed they are created
to distinguish between them. So by analogy, we would
like to characterize all portable progra
ms written in our
hypothetical language. By testing the correctness of the
language implementation by testing and analysis,
ensuring that the portable programs produce the
(expected) standard results in the particular
implementation.


So if we construct a
simulation model by building
federates, then we should be able to specify the meaning
of the simulation independently of the precise RTI that is
used to execute the simulation, so long as it be conformant

to the HLA standards. So is there an equivalent in
HLA to
the notion of a portable program, that is, an
implementation of a model, which doesn't depend on
implementation details of an RTI to produce its results? To

answer this means understanding fully the definition of
the HLA.


The HLA framework (first
IEEE draft standard) does
clearly distinguish logical events in the simulation from
contingent events (as far as timed events are concerned).
Updates, interaction sends, and object deletes are logical
events, whilst registration, subscription and so on are

not
3
. For example, a simulation might be portable between
RTI implementations if it relies on the exact time and
sequence of subscription or registration events, so long
as it is sufficiently early in the simulation execution to
produce the required resul
ts. Unfortunately, there appears
to be no real way to ascertain how early is early enough,
and hence there is no way to write simulations that are
genuinely portable between RTIs in this sense. Given the
complexity of the Federate Interface, it is not yet
clear how
localized these difficulties are or how many there will be. It
may be that, given sufficient effort, they can be cleared up
in the process of producing an IEEE standard.


The desirable position is surely that we should be able to
give a clear dis
tinction between what is a logical part of
the simulation and what is physical and contingent, so as
to be able to judge whether RTIs are really equivalent in
their treatment of portable simulations.


Having a full understanding of RTI portability as
descr
ibed above then allows one to ask for the same
equivalence if only
part

of a system uses a different RTI
or different FOM. As we shall show, this is the problem
that arises in realistic situations that some simulation
owners will be facing.

3.1

Overview

The
Introductio
n and Overview

pointed out that the
simulation application level should not be the first area to
try to achieve end
-
to
-
end interoperability. Further, we
need to learn from previous, proven experiences with
interoperabili
ty in heterogeneous environments, such as



3

This leaves room for different RTI implementations to
support different conventions, l
eading to non
-
portable
results.

the Internet and existing international Standards (e.g., ISO,
IETF, etc.). We should note that these organizations
adopted layered models. While arguably they may have
some minor faults, they are nevertheless esse
ntial for
understanding the issue of end
-
to
-
end interoperability in a
heterogeneous environment.



Figure
8
. Interoperability Model based on
existing HLA implementations.

Similarly, in simulation applicatio
ns, there are equivalent
layered structures (as shown in
Figure
3

above) to be
found. For the existing definition of HLA, these are
shown in
Figure
8
. How we propose to achieve
heterogeneous interoperabi
lity at each of the layers
identified in the HLA, will be described in the following
subsections.

3.2

A Solid Foundation

Historically, interoperability within a heterogeneous
environment has not evolved from a top
-
down approach.
Instead, heterogeneous interop
erability evolves from
having developed a solid, underlying foundation, i.e., a
bottom
-
up approach. This means beginning with a solid
Communication Architecture, which in turn supports the
necessary and needed RTI Services. RTI Service Level
interoperabi
lity then supports the constructs necessary to
support Model Level Interoperability and so on. All of
which are required to support the end
-
to
-
end Application
Level (i.e., functional) Interoperability.


It is important to keep in mind that existing HLA RT
I
implementations already exhibit all of these layers; they
are merely bundled into the various vendor
-
proprietary
implementations available today (
Figure
8
). The Study
Group analyzed these layers and identified where
protocols o
r interfaces should be published or specified,
we began to identify the problem space. Further, this
analysis showed just how much of the RTI was open and
available for achieving heterogeneous interoperability
with HLA in its current form.

3.3

Interoperabilit
y Defined

HLA Interoperability in a heterogeneous environment is
going to be difficult to achieve. Make no mistake; there
are many hurdles that have to be over come even in the
lower layers. [RFC 2502] But that doesn't mean that there
isn't work to do wit
hin the RTI, so that the upper
-
layers
can support interoperability. For example:



The HLA must be capable of scaling to support large
numbers of federates, objects and users.



Support for real
-
time interactivity
and

reliability is
necessary and fundamental
to a large portion of the
community. (This is not necessarily a limitation of
HLA per se, but of the underlying communication
architectures.)



Interoperability in heterogeneous environments is
necessary to realize cost effective reuse and
widespread adopti
on.



Support for late joiners and management of early
resigns is necessary and fundamental.



Object
-
oriented features such as abstraction,
encapsulation, inheritance, specialization, and
polymorphism can all provide additional power and
enhance code reuse, a
nd thus speed adoption within
the community.



Ownership transfer mechanisms for objects that are
interoperable at all levels (public
and
private data,
and

behaviors).



Maintaining a shared virtual environment that is
state
-
consistent throughout the entire f
ederation
community.



Recognition that consideration for interoperability
between the federate (e.g., simulation application) and
the RTI is only a small part of the interoperability
puzzle.


With these criteria in mind, the RTI Interoperability Study
Group

proposes that the following definition of
Interoperability be adopted by SISO:


"
Interoperability

means there is functional
equivalence provided by interchangeable
components within a system or process in order
to allow its components to be able to work
t
ogether with no prior agreement over an agreed
upon data communications path."


Each level of interoperability identified in
Figure
3

is
described in the next four sections, and is based up this
definition of interoperability.

3.4

App
lication Level Interoperability

Application (or Federate Application) Level
Interoperability is about being able to effectively and
unambiguously interact
functionally
between entities
managed by the same or different federates. By analogy,
two objects re
presenting entities interacting with one
another may be co
-
located on the same federate or
distributed across two distinct federates. The
transactions (e.g., attribute updates, interactions, etc.)
between said objects may or may not occur through the
HLA
RTI, depending upon where in the federation they
reside.
4

Objects that are located within distinctly different
federates will, according to the HLA rules, interact with
one another using the HLA RTI, as per the HLA rules and
Federate Interface Specificatio
n. Applying the definition
of interoperability to the Application Level, and by
observing minimal appropriate rules for the programming
of simulations, one should be able to achieve
identical
results

whether or not the objects involved are located on
the
same federate or placed on different federates
anywhere in federation community. In other words, no
matter where a particular object resides within a federation
community, precisely the same results are obtained. (This
is what the top
-
most arrow implies i
n
Figure
9
.) This view
of Application Interoperability is not really addressed yet
by HLA. However, when new simulations are designed
and built there is need for an architectural approach which
goes beyond mere interoperation. Ou
r view of Application
Level Interoperability should be the starting point for such

architectural guidance.

3.5

Model Level Interoperability

In the context of this report, Model interoperability is
about being able to effectively and unambiguously
communicate s
imulated entity state information and
behavior between two federates. Consequently, model
-



4

Resultant attribute updates for co
-
located objects might
still be transmitted through the RTI because of prior
subscription requests.

level interoperability provides a lower level of interactivity
than does the functional
-
level described above.



Figure
9
.
HLA Interoperability Layer Model.


The current means of achieving model level
interoperability within HLA is to use a FOM to define the
complete set of information that is to be exchanged
between federates in a federation community. Each object
mu
st be assigned a class that then uniquely defines the
attributes that are available to describe that object. This is
termed a Homogeneous FOM. Situations have arisen
where projects wish to use federates together that share
information about common object
s but represent those
objects in different ways. In HLA at present a federation
has only one FOM so the concept of multiple FOMs or
Heterogeneous federations is difficult to understand. The
basic situation that we are describing here is when an
object is

visible or partly visible to different federates
using multiple representations of the same underlying
state. A FOM or collection of FOMs that would allow
these multiple views and representations we are terming
heterogeneous.

3.6

Service Level Interoperabil
ity

Early in our research, we were challenged how find a way
to allow multiple RTIs to be interconnected and yet still
provide the logical equivalent of one runtime
infrastructure. At first, the problem seemed intractable
(and in some cases, we were as mu
ch told this was so.)
The problem, as we will describe, involves the sharing of
appropriate RTI internal state information to allow RTI
services from different vendors to interoperate.


Each RTI, according to the Federate Interface
Specification, is requi
red to support six services to each
federate in a federation: federation management,
declaration management, object management, (attribute)
ownership management, time management and data
distribution management. For two dissimilar RTIs to
communicate, a c
ommunications path must be established
for each service, such that one RTI service must map onto
the same service in another RTI. In other words, a sharing
of information must take place between the two dissimilar
yet cooperating RTIs.



Figure
10
.
HLA Interoperability Service
Layer Model.

It may not be necessary or even desirable for the federates
served by one RTI to see the federates served by another.
For example, if a synchronization point is to be e
stablished
using the optional list of Federate IDs, then the RTIs must
negotiate how an RTI in one federation will impart its
federate ID information to the RTI (and thus the federate)
located in another federation. They may merely agree to
use a single r
epresentation (as has been described in
bridged federations and hierarchical federations in
[Bouwens, 98] & [Myjak, 98]). In each of these two latter
instances, the object in the concealed federates appear to
reside in the interface of a proxy federate wit
hin a
federation community. Alternatively, a gateway may also
be used when connecting dissimilar simulations, such as
SIMNET, DIS and HLA together. [Wood, 97]


In each case, Service Level interoperability between
different RTIs requires that some common u
nderlying
mechanism be developed that permits the RTIs to
cooperate and exchange state information. Application
Program Interfaces (APIs) or Protocols would be imposed
on all HLA RTIs. (It can easily be shown that a gateway
knowledgeable of only the Fede
rate Interface is
inadequate to support RTI Service Level Interoperability.)
However, even though the present federate interface is
not quite wide enough to allow bridging of all RTI
services, much useful work could be done with brokers
knowledgeable of a

proposed RTI
-
to
-
RTI Interface.



Figure
11
. HLA Interoperability
Communication Layer Model.

3.7

Communications Level Interoperability

Communications Level Interoperability is about being able
to functionality
and unambiguously communicate service
-
level interactions between components within in a Run
Time Infrastructure, or between similar components
produced by different RTI manufacturers. In a perfect
world, this would imply that there are even
interchangeabl
e components
within

a given RTI
implementation (e.g., vendor A's time manager, vendor B's
DDM service) that could be exchanged for use by
different federates. For example, if a given federation
requires a specialized form of time management, a "plug
and p
lay" module could be downloaded and integrated
into the remaining RTI environment and it would still work.

Functional equivalence doesn't necessarily have to mean
that components of a given RTI are "hot swappable" like
network interface cards, only that w
hen one service
module is replaced with another, functional equivalence is
maintained at the service layer to communication layer
boundary.


Runtime Infrastructure Service Level Interoperability is
supported through the lowest identified interoperability
l
ayer: the Common Communication Architecture. This is
the mechanism that the RTI Service Layer uses to
communicate, for example, between local RTI components.
On aspect of interoperability that falls to the
communications level is interoperability in a
he
terogeneous environment. This is where the rubber
meets the road. No matter how well designed the service
level API might be, if the lower layer cannot interoperate
across a vendor
-
independent specified communication
architecture, then higher order inter
operability will not be
possible.


For the purposes of this study, three different architectural

variations have been identified at this level. These are the
networked communications architecture, the shared
memory communications architecture, and the par
allel
communications architecture. Of course, hydrides are also
possible, potentially comprising a fourth variant.

4

U
SE
C
ASES

Over the duration of the study, the requirements for
interoperability have been expressed in a huge number of
different ways, begin
ning with gateways and bridge
federates to "On
-
The
-
Wire" protocols. The study group's
first job however, was to try and classify these different
requirements and come up with a consistent set of
terminology that everyone was happy with. To that end,
four

(4) different solution classifications have been
derived. These are the
Gateway
,
Proxy
,
Broker
5
, and



5

The term "Bridge" has been deprecated from this lexicon
because of its overloaded mean
ing. Some have
inappropriately used it to mean gateway, others argued
Protocol

solutions. Each of these solutions will be further
explained in Section
5

Range of Solutions and Issues
,
below.


The study group found that jumping into a discussion
about the benefits of one particular solution over another
is hardly worth while without first having an
understanding of the use cases that drive one to such a
solution in the
first place. So in order to clearly illustrate
the requirements for interoperability, a use case example
will be put forward. We present a simple hypothetical
6

situation, involving the development and use of a number
of realistic simulations to solve a r
ange of potentially
realistic problems. Our objective is to limit the scenarios
and scope to that necessary to illustrate the
interoperability requirements and to explain our solutions
and rational for developing these variations to the High
Level Archite
cture (beyond that which is common practice
today), namely heterogeneous RTI or heterogeneous
FOM situations.

4.1

The federates: An Overview

Our example problem scenario is based on a common
defense simulation scenario: naval air defense assessment
and trainin
g. The scope of this problem tends to combine
different services (especially air and sea) but also has
relevance to land forces with littoral operations. The
problem also straddles the nature of simulation from
operations assessment in the development of

tactics,
through training for both command team and weapon
operators, down to engineering federates involved in R&D

for weapon systems performance and development.


A number of federates are put forward that have a
functional role in the problem context a
nd could be
usefully put together in different ways for solving
different aspects of the problem. The important aspect of
this exercise is to explain the differences in the supporting
architectures, both of the federates
and

their implications
on the simu
lation infrastructure (i.e. the RTI) that has to
support them. For this example, the federates have been
chosen to have similar but different information
requirements, as described in the next sections.





that bridging relates to network communication
exclusively. Thus the term "Broker" was chosen instead.

6

Hypothetical use cases were used, based on existing
work, only because experienc
e and use of HLA is still in
its infancy.

4.1.1

The Flight Simulator

The flight simulator federate i
s meant to be typical of the
visual "man
-
in
-
the
-
loop" flight simulators used in legacy
DIS systems. In our example, this flight simulator has been
converted to use the newly standardized Real
-
time
Platform Reference FOM (RPR FOM). Objects in this
system a
re capable of providing both position and
appearance updates with reasonably detailed dynamics
models to allow their behavior to appear realistic to a
human operator or
interactor
. The flight simulator
federate will also accept position updates from other

platforms for visualization of
out of the window

views and
for simplified sensor input. As a legacy simulation, the
cost of altering the original application to use the HLA in
native mode is prohibitive. Just making the simulation
compliant though the R
PR FOM using for example a
DIS/HLA Gateway, has already produced significant cost
overruns. Further consideration for additional
modifications would be neither tolerated nor effective.

4.1.2

The Weapon Simulator

The weapon simulator is representative of engineer
ing
simulations used for Research and Development. These
systems are typically very detailed. Although the
information in terms of platforms and their position is in
common with the other federates, the update rates and
overall complexity of the algorithm
s supporting the
generation of this information is of much greater fidelity
than other federates. In this respect, the weapon simulator

federate may not run in real time and when not using
parallel processors to improve the execution times of
routines con
tained within the simulation. However, when
coupled with the use of parallel processors and the use of
optimistic time management techniques, this federate
could achieve real
-
time performance though parallelization
and other performance improvements in an

event
-
based
simulation.
7


The weapon simulator federate as well as its physical
representations of certain platforms uses electromagnetic
interactions for either passive sensing or active radar
sensing devices. It includes aspects of data fusion in
terms

of weapon system performance and the benefits of
improved target information and identity.

4.1.3

The Threat Missile Simulation

This system relies on a mixture of simulators and
stimulated systems together with "hardware
-
in
-
the
-
loop"
precision devices. By its n
ature, the threat missile



7

This simulation would be well represented by the sort of
work that has been undertaken on EADTB.

simulation has a high degree of security associated with it
and although it provides the best representation of the
behaviors of various threat systems, the degree to which
the information may be released will need to be controlle
d.
The simulation operates in real time in order to stay
synchronized with the hardware in the loop but the
granularity of updates and the level of synchronization
needed is much more demanding (perhaps even an order
of magnitude) above that of the Flight

simulator.


The detail in the threat is paramount in this simulation, but
targets are represented relatively simply. Although the
positions of targets are represented similarly to that of the
other models, the details of electromagnetic returns are
impor
tant here, especially environmental effects such as
multipath and clutter. The format(s) for representing this
type of information has to be very specialized to allow the
effects to be converted into real signals that can be
interpreted accurately and cor
rectly by the hardware
-
in
-
the
-
loop.

4.1.4

The Command Team Trainer

The emphasis on the command team trainer is to bring the
command post and operator consoles together into a
system in order to train a task force command team in an
operational context on the use

and management of various
weapon systems and sensors. Of particular importance is
the ability to provide representative information to each
member of the command team and to stress his or her
decision making and teamworking ability. To this effect
the t
rainer is supported with a scenario generator based
on Semi Automated Forces (SAFs) technology. This is
representative of many of the simulations run using the
Aggregate Level Simulation Protocol (ALSP) or even
possibly the types of exercises run during S
TOW or
Unified Endeavor. A final architectural challenge is that
the command team trainer may need to operate at sea
during transits, in which case the distribution mechanism
will need to rely on radio communications rather than the
readily available Ethe
rnet LANs or ATM WANs.


As with the previous simulations, the same information is
needed in terms of platforms, threats and the capabilities
of systems. However the detail now is in the information
provided by the systems to the command team rather than
t
he behavior of the systems per
-
se.

4.2

Common Scenario Objects

This section looks at a summary of the objects that are
common to the simulations. This should not be construed
as trying to identify a FOM but rather to look at the real
world objects that would

be represented in each of the
simulators. Each simulator may choose to represent the
objects in a manner that is appropriate in fidelity,
resolution, and security with attributes that are relevant to
the overall simulation.

4.2.1

Platforms

Simulated platforms a
re important to all military
simulations, either as threats, friendly ships or hostile
aircraft within the scenario. Position and trajectory will
always be important but with different fidelity
requirements. Identity and trajectory may also be used as
“p
erceived data” by some simulators that. This may be a
question of "need to know" or perhaps the specific
simulator does not have the accuracy or behavior
constraints of the required sensor systems modeled.

4.2.2

Electromagnetic Signals /Beams

Many of the simula
tors involve the representation of the
electromagnetic environment. If they are to use their
sensor models in a cooperative way, then they must be
able to accurately interpret the ways in which these
electromagnetic signals are represented. This can become

a rather sticky issue, particularly where legacy systems are
concerned. The cost of retrofitting such a system in order
to agree upon a given object's representation could be
cost prohibitive.

4.2.3

Command messages

The command team trainer simulator exhibits
simplified
behavior, particularly with regard toward the source and
decision accuracy of tasking and management messages
relative to the engineering models. In all of our simulators
however, the concepts of command level messages, either
as sensor tracks
or as tasking orders, are set in an
operational context and will be important in our scenarios.

4.3

A Possible Life History

4.3.1

Disparate Evolution and HLA Compliance

Initially our simulators were developed in isolation to meet
their original design requirements.

As use of HLA is now
mandated, each system will have had to adopt some
strategy for becoming HLA compliant. This means they
will now be capable of interfacing with an HLA RTI
through the services defined by the Federate Interface
Specification, in order t
o achieve their goal. It has been
assumed that all of the simulators described herein will be
running on a heterogeneous variety of processing
platforms. To achieve such a federation, the RTI services
will likely be configured differently with respect to

each of
our different simulations. Further, platform compatibility
as well as the performance of the RTI(s) will be strong
factors in the choice of a given supplier. In addition,
development of the FOM will be a very subjective topic
for the legacy syst
em managers. The need to have some
compatibility with other simulations in the future may also
be traded off against the information exchanged currently
in the soon
-
to
-
be proposed simulation scenarios.

4.3.2

Integration in the RPR FOM

At some point the simulat
ors are all brought together to
fulfill a common air defense assessment. As an example,
an experiment needs to be undertaken to assess the
operational value of a newly proposed weapon system
when used against a range of threats. The flight simulator
woul
d be used in a hostile role for targeting and launching
the threat, which is then simulated by the threat simulator.
The weapon simulation is used to determine the
effectiveness of the federation community when tasked by
the command team. Delivering info
rmation to the
command team (in conjunction with other sensor systems)
and showing how the whole weapon system concept
would help the command team counter the threat under a
variety of targeting conditions.


It is assumed that at this stage, all FOM requir
ements
could be met using the RPR FOM and that information
interpretation of the objects and their attributes would not
be a problem. There would however be problems in using
a single RTI since the services and performance
requirements of our collection o
f federates cannot be met
by a single RTI supplier. This sets the context for the
issues discussed in Section
4.5

Service Interoperability
,
and Section
Communications Level Interoperability
, &
their possible solutions in Sections
5.3

and
5.4
.

4.3.3

Mixed Fidelity operation

In this situation we assume that all the implementation
difficulties described in Section
4.5
,
Service
Interoperability
, have been overcome or can be resolved
by using a single homogeneous RTI that offers the
requisite performance. However, to achieve this tradeoff,
the simulators have to use representa
tions of the objects
that are appropriate to
their

mode of operation. They
cannot afford to translate to a common format or FOM due
to lost fidelity. While the objects in the simulated scenario

are the same, each simulated object may have multiple
repres
entations, each being documented effectively in
different FOMs (i.e., heterogeneity). This provides a
situation in which to discuss the issues associated with
heterogeneous federations, as discussed in Section
4.4

&
solutions
in Section
5.2
. Issues of how secure federates
dealing with classified information may translate or
declassify certain attributes are also discussed in this
section.

4.3.4

Compound situations

Given the nature of the individual situa
tions described
previously, it is quite likely that when these simulators are
brought together, there will be elements of both model,
service and communications interoperability that will need
to be dealt with and resolved. While there may be issues
that
relate to the compound situations that will certainly
arise, their respective details are not the focus of this
report.

4.4

Model interoperability

The model interoperability issue has been defined in
Section
3.5

as being the need t
o exchange entity state
information. The types of federation requirements for this
level of interoperability has been further expanded in
Section
2.5

Heterogeneous FOM & Homogeneous RTI
.
This section

illustrates how the use case situation
described in Section
4.3.3

can lead to these requirements.
Sections
4.4.1

through
4.4.3

describe three general ways in
which the
se heterogeneous FOMs can be viewed:
Intersecting, Adjacent and Hierarchical Federations.
Sections
4.4.4

through
4.4.6

explain why we're prevented
from using a single FOM (again from the Use Case
p
erspective).

4.4.1

Intersecting Federation

In an Intersection Federation, two named federations
share some common objects and/or interactions with
another federation. In effect, they operate over two FOMs
that are nearly identical in many respects, but not
equi
valent. As an example of an intersecting federation,
consider the command team trainer and the flight
simulator. The flight simulator uses (at least part of) the
RPR FOM. It requires positional information with enough
detail to be able to visualize the
platforms in terms or their
locality, their articulated parts and appearance. Similarly,
the command team trainer needs positional information in
the same way, but has a much larger emphasis on the
actual messages passed between platforms (and hence
betwe
en team trainees). This latter information does not
exist in the RPR FOM, so although this positional
information could be shared, using an identical
representation is not possible, as some other information
must be declared in the FOM to in order to supp
ort the
command team simulator. Therefore, while the majority of
information exchanged within the intersection federation
can continue normally with the common elements of data
being exchanged, all of the data necessary is not being
delivered between all
of the joined federates. In effect, the
resulting FOM is the union of the two distinctly different
FOMs.

4.4.2

Adjacent Federation


In an adjacent Federation, two or more federations have
similar

objects and/or interactions, but are defined by
two
or more diffe
rent FOMs
. This is an example of different
fidelity and resolution. In our adjacent federation, we
consider the interaction between the weapon simulator
and the flight simulator (as a threat target for the weapon).
In this case the flight simulator uses

the RPR FOM for
position while the weapon simulator uses a local Cartesian
representation of threat and target (i.e., missile) positions
in order to achieve its weapon guidance integration
algorithms. So while the position of both flight simulator
(aircr
aft) and the weapon system (missile) are present in
both FOMs, their internal representation is entirely
different. Neither simulator will change their
representation since to do so would imply rewriting the
basic model algorithms (or embedding translator
s) in each
of the federates. Further, rewriting either federate to
translate data to some common FOM representation
should not be negotiable. Instead, we have a clear case
where we need to have multiple representations for the
same given attribute.

4.4.3

Hierar
chical Federation

Recall that in a hierarchical federation, one federation
appears as a
federate

(or federate component) in another
federation. Put another way, in a hierarchical federation,
some of the objects owned by one named set of federates
appear i
n an abstracted form to other “higher level”
federates in the federation tree. As an example, consider
the weapon system simulator. As an engineering
simulation it has been developed in a very modular
fashion with different federates representing differe
nt
parts of the weapon system. As an air defense weapon,
we could consider different systems being a target:
locating radar, a tracker radar and an illuminator or
Command Line of Sight (CLOS) link, the weapon launcher
as well as the missile or missiles in

flight could all be
different federated systems. The simulated missiles
themselves may be split up into other federates, such as
the seeker, missile propulsion and aerodynamic models
and guidance algorithms. It's not to difficult to imagine
one such fede
ration for each missile. Note that these
lower
-
level federates will be passing information between
each other that will be of very little relevance to other
simulations, with the exception of perhaps appearance,
position, attitude and targeting informatio
n being
exchanged with the command team trainer. So there will be

some information that we consider relevant to the other
simulations.


In this hierarchical situation it is also likely that the level of

representation of shared objects will have differen
t
fidelities and granularities, not to mention security. For
example the missile position could be timestepped and
updated at 1
-
millisecond intervals, while most of the other
simulations were expecting Dead Reckoned positions with
updates in excess of 1
-
s
econd per iteration. In order to
realize a hierarchical federation, we will need to provide a
means of effective data filtering and aggregation and
possibly the generation of alternative representations.
These are to ensure that traffic that is only asso
ciated with

the weapon system simulator does not unduly load down
the other federates or burden the RTI and network. In
practice the simulation providing the low
-
level federation
will have its own LAN designed to take that traffic. We
would need to ensur
e that this traffic did not get relayed
across to LANs or WANs serving the higher level
federations in order to maintain reasonable performance.

4.4.4

Cost Prohibitive Modification of Legacy
Systems.

In our use case scenarios, all the simulators have already
bee
n converted to HLA and have their own Simulation
Object Models (SOMs) that they use internally. Stand
-
alone federates such as the Flight Simulator will have
chosen a FOM that allows them to be “Plug and Played”
with other simulations using the same FOM.
In this case,
it would be uneconomical to convert to anything other
than that FOM. The command team trainer is built up of a
large number of federates already. To change its FOM
could imply altering every single federate in that
simulation. Similar argu
ments apply to the other federates
with the emphasis being that the amount of work needed
to get a simulation to work would be a real limitation on
reusability, and the viability of achieving the project
objectives.

4.4.5

Information Security

The threat simulato
r needs to distribute classified
information about the behavior of the threat between its
own federates. However, this information must not be
allowed outside of the simulator boundary in its “correct”
secure form. It is often very difficult to determine

what
exactly makes information releasable (or not) from a
simulation. If we consider that the trajectory of the missile
threat prior to seeker activation is classified, this could
imply that the position of the threat declared to other
simulations couldn
't be the same value as that used within
the threat simulator itself. It must be representative (for
example, allowing weapon sensors to calculate whether
they would see it during this initial phase) but cannot give
away the classified behavior.


In effec
t, we must maintain two different representations
of the threat missile position. One for use internal for the
threat simulator (remembering that this involves multiple
federates) and one for use external from the threat
simulator.

4.4.6

Differing Levels of Fid
elity

The hierarchical federation has already identified that the
different representations may have different fidelities or
granularities associated with them. This could be simply a
variation in degrees of freedom or derivatives to which an
attribute s
uch as position was expressed. In many cases
the fidelity differences are much subtler and relate to the
way the attribute value has been calculated. As an
example of this we can consider the Anti Radiation Missile
(ARM) missile threat that is looking fo
r radar beams on
which to home. (The weapon system radar model will
probably use detailed radar equations to calculate
stochastic probabilities of detection and work out when it
could first detect the incoming missile.) It may then be
presenting the rada
r beam as a complete scan (in its
aggregated form) in order to visualize the weapon scan
sector or to allow simple Electronic Support Measures
(ESM) detections in the flight simulator. The threat model
on the other hand needs real time evaluation of the a
ctual
radar beam as a result of its scan (at 1
-
millisecond
intervals). It also needs to know the waveform being
transmitted and its exact amplitude profile from the
intersection of the antenna pattern and scan. So although
the “emitter beam” attribute is

still the same, each
simulation has radically different expectations of the
information that can be obtained from it.

4.5

Service Interoperability

The service interoperability issues have been defined in
Section
3.6

as being the n
eed to have different RTI
vendor's services be able to communicate with each other,
thus implementing a consistent RTI
-
to
-
RTI service
interface. Federation requirements for this type of
interoperability have been further expanded in Section
2.4
,
Homogeneous FOM & Heterogeneous RTI
. This section
illustrates how the Use Case situation described in section
4.4.2

can lead to these requirements.

4.5.1

Platform Support:

The availabili
ty of RTIs for particular platforms and
operating systems has already become an issue for some
of the less popular platforms. As an example we could
consider the federates being hosted on the following
platforms:

Command Team Trainer

PCs

Windows NT

Weapo
n Simulator

SP2

AIX

Threat Simulator

Various

Real
-
Time OS

Flight Simulator

SGI

IRIX

While many of these platforms are supported by current
RTIs, some are not (for example, VxWorks). Still other
platforms and operating systems are being used in various
sim
ulators and simulation systems, including legacy
systems and the growing world of embedded and C4I
systems. In addition, language bindings can make a great
deal of difference as to what is achievable
and

economically. There are still many simulators (and

thus
potential federates) that are written in languages such as
Fortran or LISP that have great difficulty interfacing to the
current APIs written in C or Ada. This kind of availability
severely limits the delivery of federated simulations,
reduces reuse
, and stunts interoperability. Unless we
forget our lesson in heterogeneous interoperability from
the developers of the Internet, to obtain the necessary
flexibility and economical growth in HLA systems,
individual RTI implementations will need to be
inte
roperable.

4.5.2

Performance:

In their standalone form, our example federated simulation
systems have very different RTI performance
requirements. The weapon System simulator makes great
demands on the time management system. It has to allow
for different pro
cessors to stay synchronized and yet gain
the benefit of parallel processing in order to keep the
simulation (run) time equivalent to that of "wall
-
clock"
time. This is necessary for the command team trainer to
keep up with the weapon simulator.


The co
mmand team trainer is often used in large
-
scale
exercises and needs to have good DDM performance to
provide the scaleability needed. The flight simulator is
more concerned with end to end latency and ensuring that
operator reaction times are not exceeded.

The threat
simulator with its links to real systems needs to maintain
high throughput for video signals and pulse descriptors.


Each of these service performance requirements, although
only needed locally to maintain the simulation, will affect
or will be

affected by the RTI chosen for that particular
simulation. In the end, when all simulations are working
together, this will require different RTIs to work together,
i.e., to be interoperable.

4.5.3

Services:

In a similar argument to dealing with performance,
different

simulators will make use of RTI services differently. A
good example of this might be time management. To
continue our example, the weapon system simulator would
use optimistic time management. Other simulators may be
using conservative mechan
isms, while a few human
-
in
-
the
-
loop systems will use real time. An ancillary
consideration here is that not all services are implemented
by every RTI vendor (although the current requirement is
that they should). A further question might be to ask if
the

services have all been implemented in a conformal
way? One would expect that they would, but since there
is currently no requirement for RTI
-
to
-
RTI interoperability,
we may not know if there is functional equivalence at the
service level today.


Another
example of where some simulators would use a
particular service while others would not, is the DDM
service (e.g., the Command team trainer) and Ownership
management (e.g., the threat simulator for swapping from
simulated behavior to real for certain aspect
s of the
scenario). These all point to the fact that it will often be
more effective for a simulator to choose an RTI that
implements the particular services it wants. If you then
have to make a given simulator work with another
simulator that has select
ed a different RTI, then the RTIs
must interoperate, or one or both of the simulators will
have to be altered. Rebuilding the simulations to use a
common RTI will not be adequate. The net result is that
stovepiped solution sets will evolve once again.


F
inally, there is the marked issue of RTI service
interoperability. To date, there is very little interaction
between the various RTI services. Yet in some cases,
service interaction (i.e., coupling) is necessary and
warranted. Such is the case with obje
ct ownership
transfer and time management. Given that some federates
may choose to employ optimistic time management
schemes while others do not can yield nondeterministic
and potentially undesirable service interactions. These
interoperable problems can

occur within a single vendor's
RTI implementation, but they are also a concern when two
dissimilar RTI attempt to interoperate.

4.6

Communications Level Interoperability

RTI Communications level interoperability issues, as
identified in section
3.7
, involve three different yet
potentially interoperable communications architectures:
networked, shared memory, and parallel systems. Even
though these fundamental architectures appear to be
completely different, it is important to note

what each has
in common. And that is the need to be able to directly
interface with and support the RTI service layer directly
above it.

4.6.1

Networked Communication Architectures

Networking implementations offer the most cost effective
and flexible alternati
ve in end
-
to
-
end communications to
simulation managers today. With the growth of the
Internet
8

exploding at a 341,634% annual growth rate, it is
hard to imagine doing anything else. Perhaps this explains

why the vast majority of all HLA federations built

today
are based on networked communication architectures, and
the Internet Protocol in particular. But not all networked
communication architectures are built the same. In fact,
there are literally hundreds of protocols available.
Unfortunately, most o
f them do not interoperate, and some

may interoperate with the assistance of a gateway or proxy

agent. Nevertheless, networked solutions far out number
the other two communication architectures combined.


At the present, both the Telephone Companies and
I
nternet Service Providers see simulations and shared
virtual world applications as significant advancements
(and potentially new services). They may not be far from
the truth. Unfortunately, the net result to date has been
that protocols are often develo
ped with one specific
purpose in mind, a purpose that is not always conducive
to the needs of distributed simulation. Thus today, no
protocol has been developed that is specifically designed
to meeting the requirements of distributed simulations.
This is

particularly disappointing. Take for example the
underlying data link layer and hardware of ATM, which
indeed provides features which could be useful to RTIs
(e.g., connection
-
oriented flow control, Quality of Service,
etc.) but whose control is hidden b
y the intervening layers
of the protocol stack (i.e., network, transport, and
application layers). Conversely, there are features that
distributed simulations need (e.g., multicast
communications, both reliable and best effort; embedded
network management,

etc.) that are not supported by
ATM link layer protocols. ATM after all, was designed for
telephone (voice) traffic, which typically only involves
two parties. Further, not all distributed simulations operate
over networked communications architectures.
Instead
they may involve other means of communications which
are not (at least directly associated with) networking or
networked communications: Parallel and massively parallel
architectures, and shared memory architectures. Each of
these communication ar
chitectures presents an interesting
set of interfaces that may appear to be quite different from



8

Source: Hobbes' Internet Timeline v4.1
http://www.isoc.org/guest/zakon/Internet/History/HIT.html

one another. Nevertheless, heterogeneous
interoperability at this layer may still be accomplished.


The achievement of a common underlying protocol to
support

RTI
-
to
-
RTI communications might seem hard
without an Open Source RTI vehicle. However, there is a
possible negotiated approach in which different RTIs can
negotiate their best common set of protocols for a given
configuration. In other words, the answer
here lies not
necessarily in the fact that there are several Networked,
Shared Memory, and Parallel architectures involved,
perhaps even within the same federation community, but
rather that each architecture supports a well
-
defined
interface. And the key

to what this interface needs to
manage can be found in the layer above: the Service Layer
Interoperability specification.


The API defined at the service layer describes precisely
what two cooperating RTIs need in order to interoperate.
Declaring a commu
nications level interoperability protocol
may be as simple as defining a one
-
to
-
one (1:1) mapping
between the service level API and underlying
communications architecture. To accomplish this
however, requires access to an open and flexible run time
infras
tructure, and agreement and standardization among
RTI developers. This
is

something which SISO SAC
should begin working on.

4.6.2

Shared and Parallel Communications

When the RTI Interoperability Study Group was formed,
the RTI and Communications Forum requested

that an
"on
-
the
-
wire" protocol be investigated as a potential
solution to RTI
-
to
-
RTI interoperability. In general, this
made a lot of sense. This was later reinforced by the
results of the study group, when they realized that the
service layer interoper
ability requirements could be
defined with a single API. And since an "on
-
the
-
wire"
protocol typically results in a 1:1 mapping of this API to
the "bits on the wire," this all made perfect sense.
However, not all RTI communications architectures are
repr
esented by networked solutions.


Shared memory architectures, while fundamentally similar
to networked communications today, offer an interface
into the RTI services that is unique: direct memory access.
This means that rather than interfacing with a prot
ocol to a

communications socket, access to data can be obtained
directly, through internal memory reads and writes. This
type of communications is more closely related to that
found internally on parallel processing systems. These
latter architectures ty
pically use some form of message
passing system to communicate their information from
within. While different from the protocol interface of a
communications network, they are at the same time,
similar.


The types of federation requirements for this leve
l of
interoperability has been further expanded in Section
2.3
.
This section illustrates how the Use Case situation
described in Section
4.4.2

can lead to these requirements.

4.6.3

Communications Services

As an example of an adjacent federation we will again
consider, the interaction between the weapon simulator
and the flight simulator (as a threat target for the weapon).
In this instance, the flight simulator contains a human in
the loop, and thus operat
es in real
-
time (i.e., federation
time is synchronized with wall
-
clock time). In this case the
flight simulator, using the RPR FOM, supports a common
networked interface. The weapon missile simulator used
to generate some representation of threat and tar
get
positions is supported by two parallel systems, using a
shared memory interface. This hybrid architecture now has

the need to be able to communicate with the networked
architecture of the flight simulator.

5

R
ANGE OF
S
OLUTIONS AND
I
SSUES

5.1

Overview

The req
uirements identified and described in the previous
sections imply that the current definitions of the HLA and
implementations of the RTI do not deal with all the current
issues of interoperability and heterogeneity. This section
looks at some of the implic
ations for the HLA if these
requirements are to be realized. By examining and
discussing the implications, it is hoped that a number of
solutions or ways forward may become apparent. In this
section we introduce the concepts of a Federate Gateway,
a Fede
rate Proxy, an RTI Broker and an RTI
Interoperability Protocol as defined in section
2.1
.

5.1.1

Federate Gateway

A Federate Gateway is a translation device that
interconnects two or more Federates or interconnects one
or more Federat
es to one or more legacy applications or
systems for the purpose of joining a federation
community. Unlike the Federate Proxy, Federate Gateways
support no direct HLA interface. The Federate joined to
the Federate Gateway provides the interface to the
Fed
erate's local RTI component, according to the Federate
Interface Specification. The interface between the
Federate and the Gateway is unspecified. The Federate
Gateway provides the connection between federates in
separate named federations (or to legacy
simulations) to
create a federation community. While the Federate
Gateway does not appear as a Federate to any of the
named federations to which it connects, the Federate
Gateway is visible however, and a member of the
Federation Community.

5.1.2

Federate Proxy

Unlike the Federate Gateway, a Federate Proxy is a joined
federate in the named federations to which it connects.
The Federate Proxy provides the connection between
RTIs in separate named federations as defined by their
FOMs or between an RTI and legacy
simulations to create
a federation community. (This is the approach that has
been previously called 'federate bridge' or `bridging' of
federations.) The Federate Proxy appears as a Federate
within a named federation, and when so connected, makes
objects o
f the remote federation(s) or legacy simulation
appear to be present within itself to the local joined
federation.

5.1.3

RTI Broker

The RTI Broker functions in a way that is similar to the
Federate Proxy in that it provides the connection between
RTIs in separat
e named federations (as defined by their
FOMs). However, unlike the Federate Proxy, the RTI
Broker uses the Service Level API to use and
communicate information beyond that defined in the
Federate to RTI Interface. The RTI Broker passes not only

federate

level data but also communicates and arbitrates
RTI internal state information between two (or more) RTIs,
potentially from different vendors, or in support of the
common use of different communications architectures.
An RTI Broker may have multiple conn
ections to
federations and may itself be distributed. An RTI requires
that a standardized interface to RTI services in order to
connect dissimilar RTIs or local RTI components within a
vendors RTI implementation. This interface is identified
as the Servic
e Level Interoperability API in the layered
model of Section
1
,
Figure
3
.

Distributed Simulation
Interoperability Layer
Model
.

5.1.4

RTI Interoperability Protocol

The RTI Broker approach may well not be the

most
efficient way to communicate between RTI
implementations in heterogeneous environments, where
differences in language, processor or vendor can inhibit
interoperability. This leads us to the final, low
-
level
solution: the RTI
-
to
-
RTI Interoperability P
rotocol. This
protocol may be used within a vendor's RTI to
interconnect local RTI components, as well as between
RTI implementations to communicate RTI internal state
and federate level data. The protocol should still be
independent of the underlying com
munications
architecture except for some parametric knowledge of the
characteristics of that architecture, related to its efficient
use. Specific implementations standardizing this protocol
will be necessary to ensure compatibility and
interoperability ac
ross all vendor's RTIs.

5.1.5

Heterogeneous FOMs

The translation problems that arise with Heterogeneous
FOMs within a federation community can be best
addressed with either the Federate Gateway or Federate
Proxy approach. These two mechanisms are designed to
in
teract with HLA Federates and to translate Federate
Object Model level data. The Federate Gateway is the
translation device of choice to interconnect HLA federates

with legacy systems, as they are not required to maintain
the Federate to RTI interface wit
hin the federation
community. The communication mechanisms used by a
Federate Gateway vary and may be entirely outside those
services provided by the Federate Interface Specification
(i.e., outside of the HLA). Put another way, the Federate
Proxy operate
s below the level of the Federate Gateway.
The Federate Proxy, unlike the Federate Gateway, indeed
uses the Federate Interface Specification to translate
information from one federation to another. In this
fashion, a proxy may be capable of translating i
nformation
between two dissimilar federations (e.g., solving the
problem of the heterogeneous FOM).

5.1.6

Heterogeneous RTIs

With heterogeneous RTIs it is necessary to connect
separate federations employing different RTI
implementations. Only the RTI broker and
RTI
Interoperability Protocol solutions have the ability to
transmit RTI internal state information and so these are the

only solutions that can provide the full range of HLA
services across different RTI implementations.


The RTI Broker functions at a le
vel below that of the
Federate Proxy. It does not use the Federate to RTI
Interface but rather is involved in the translation of
information between two or more RTI implementations or
between local RTI components within a given RTI
implementation. The RT
I Broker uses the Service Level
Interoperability API identified above. The RTI Broker
uses the API to pass not only federate level data (as is
usual and customary for the RTI) but it also communicates
and arbitrates RTI internal state information between
two
or more RTIs, potentially from different vendors, or in
support for bridging across different communication
architectures.


The RTI Broker will support the interchange of internal
RTI state information between RTIs using the RTI
-
to
-
RTI
API. Such a mes
sage passing scheme is however, not
necessarily the most efficient way to communicate
between RTIs in heterogeneous environments, where
differences in language, processor or vendor can inhibit
interoperability. This then leads us to the final, low
-
level
so
lution: the RTI
-
to
-
RTI Interoperability protocol. The
RTI
-
to
-
RTI Interoperability protocol is very closely related
to the RTI
-
to
-
RTI API used by the RTI Broker. This
protocol will permit any two RTI Implementations (of any
vendor) to communicate with one
another directly through
some common medium, whether that medium is a network,
shared memory architecture, or a parallel machine's internal

message passing system. In other words, the RTI
-
to
-
RTI
Interoperability Protocol can be independent of the
underlyi
ng communications architecture. Specific
implementations standardizing this protocol will be
necessary to ensure compatibility and interoperability
across all vendors' RTIs, and to ensure a one
-
to
-
one (1:1)
mapping with the RTI
-
to
-
RTI API.

5.2

Model interoper
ability

In a heterogeneous FOM we have a situation where an
object that has the same interpretation in the real world
has multiple alternative representations (for the same
aspect of object state) needed by the different federates.
The reasons for requiri
ng multiple representations or
views of state are varied and were described previously
ranging from security, through partial overlap of data to
simple alternative representations. These solutions and
issues focus on possible means of achieving
correspond
ence between alternative representations and
do not address the specific requirements identified earlier.
This is shown diagrammatically below in
Figure
12
.
Representation Equivalence Example.



Figure
12
. Representation Equivalence
Example.

In this example, the key issue is that there is one object
(the ship) but two different adjacent federations that want
to describe that object (i.e., receive updates about it) in
two different

ways. In this case we have equivalent
classes (Ship and Vessel) with identically named attributes
(Position). The only real difference is the way the Position
attribute is represented with federate (or named set of
federates) using some flat earth X, Y,

Z system and the
other federation using Latitude longitude and altitude.
However, as with many heterogeneous FOMs, the
representations can be far more complex. One example
might be an aggregation where one set of federates
considers a whole fleet of shi
ps as a single entity, while
another federation might consider them as discrete ships.
Further, the example of a naval fleet is not unique to this
scenario, as ground vehicle simulations have also
encountered similar problems when dealing with individual

weapons platforms verses brigades of such platforms.


There are therefore two issues to consider in a
heterogeneous FOM. The first is how do we define these
alternative representations in a FOM. The second is how
do we keep these alternative representati
ons in step.

5.2.1

Heterogeneous FOM Representations

Within a FOM we have two basic options to handle these
alternative representations




Class Equivalence
-

We define different classes to
hold the different attributes for each representation.
This means that eac
h object can only be of one
variant and federates subscribing to the other
representation will not discover the objects (except
as a common base class) and have no access to the
common state. It implies that the single object
actually has to be represent
ed as two objects.



Attribute Equivalence
-

We define a class that
contains both sets of attributes.


In this case the
different federates only know about the
representation attributes that they have subscribed
to, thus have no access to the common state.


Both these situations have disadvantages. First, there is
no way of describing within the FOM the equivalence
between the two representations (nor can they be the
same object). A second issue involves the idea of having
a common attribute name (such as

position) that maps
onto the meaning associated with the receiving
federation's interpretation of that objects position state.
This is a valuable interoperability concept that is now lost
because we have to change the name so that we may
differentiate be
tween the subtle variances in its
representation. These issues point to a need for a
representation clause within the FOM perhaps as a means
of defining equivalence and allowing the translation
mechanism(s) to be described next to the object be
implemente
d.

5.2.2

Representation Translation

In either of the previous situations there needs to be a
way of keeping the two (or more) representations
consistent for any given object.


The onus could be left to individual “translation”
federates responsible for mapping o
ne representation onto

another. This removes the need to do the translation from
any given federate (or set of federates). However, this
leads to an interesting question about ownership in that
although the “translation federate" or proxy would “own”
an
object or equivalent attributes, the real state is actually
owned by the originating federate.


The RTI might know about the existence of the attributes
(through the FOM) but has no responsibility for
translating the information (nor should it). However,
it
might be in order for the RTI (through the FOM or through

multiple FOMs) to denote class and attribute equivalence
for objects that have multiple alternate representations. If
we did define an equivalence relationship in the FOM, the
RTI could then mai
ntain the proxy roles in keeping all
representations up to date. This would probably have to
be be done in a similar manner to attribute ownership.


With the number of FOM based code generators being
developed, another approach might be analogous to
“plug
-
ins” for browsers. In this instance, the RTI could
call up a particular "plug
-
in" proxy translator if the format
that it wanted information to be delivered in was not its
own local federate's preferred representation. Effectively,
a single representation

would exist within the federation,
but the local RTI component can translate between those
representations a' priori, and that operation could be
requested for or provided by its local federate (like a plug
-
in to Netscape).

5.2.3

Class Based Filtering

A final
issue with heterogeneous federations is associated
with performance. The FOM and its associated execution
data (a.k.a. FOM Document Data) file are used to achieve
class based filtering. This class based filtering has a much
lower filtering overhead than
say DDM techniques, and
tends to be used more frequently in current federations.
In heterogeneous federations, different representations
(especially classes) might be used to identify different sets

of information exchange between sets of federates. It
w
ould be typical for each representation set (i.e., each
federation's FOM) to support a set of closely coupled
federates, with the overlap between FOMs representing
the loose coupling between the sets.

5.2.4

Summary

In order to achieve the sort of Model Level
Int
eroperability expected by simulations in the future, it
seems likely that the FOM and OMT concepts will have to
be developed further. A good start in this direction
appears to be the Base Object Model (BOM)
representation originally proposed by SISO's Ref
erence
FOM Study Group. Certainly, this monolithic approach to
FOM development has its limitations. By building up
models hierarchically from BOMs, and then providing
better mechanisms to manage them, we may be able to
over come all of the model level in
teroperability issues.
Further, this work will need to be undertaken in concert
with the Service Level API development and future
Interface Specifications work, such that the RTI
developers can then support both federates and or plug
-
in
proxies to realize

these interoperability objectives.

5.3

Service Interoperability

5.3.1

Interconnection Mechanisms

Large parts of RTI service functionality can be considered
reasonably simply in terms of getting messages from one
place to another. For these instances, the network la
yer
takes the burden of interoperability. Other parts of the RTI
are more complex and demand synchronization, agreement,

and cooperation among its distributed components. In
addition the concepts defined in the Federate Interface
Specification are not alw
ays sufficient to support
interoperability (or reliability or deterministic results).
Using only the federate to RTI interface, LBTS is
insufficient information to support Time Management
interoperability over across a proxy or through a gateway,
intercon
necting two federates and using some out
-
of
-
band communications method. In short, the information
made available to the federates from the RTI does not
contain enough information for the federates to be able to
discern the RTI's internal state information
.


There are two leading ideas in this general view of RTI
service interoperability. The first is the idea of point
-
to
-
point connections, either by an RTI
-
to
-
RTI API (via an
RTI broker) or by the use of proxy federates, which are
members of more than one
federation in a federation
community. In this instance, each federation supporting
proxy federates would communicate through an extended
Federate Interface (FI), and be used to establish the
common services. Proxy Federates are attractive because
they trea
t the RTIs as black boxes behind their FIs. It also
seems that the extensions required to permit service
interoperability might not be impossibly great.


The second idea is that of group protocols in a network.
This suggests that RTIs could be implemented
above a
common set of group protocols (above the
communications layer) which would allow interoperability.
These group protocols might themselves require
gateway/point connections
-

but this would be
implemented on a protocol by protocol basis
-

common
arc
hitecture would be the dominant feature.


Another significant choice, particularly for the point
connection approach is between tree
-
structured
interconnect of regions each having a particular RTI and
more general interconnect. Tree
-
structured interconnect

has many attractive properties
-

e.g. a single path between
any two points, but for this reason suffers from the lack of
resilience associated with more general networks.


At this stage it is helpful to consider what sort of
agreement would be needed in o
rder to facilitate
interoperability between two different RTIs.

A typical configuration involving two RTIs is shown
below.



Figure 12.
Example RTI to RTI interface.


We can now consider the implications on RTI to RTI
interoperabil
ity by looking at the RTI state information
that flows between Local RTI components in each of the
Federate Interface Specification service groups.


5.3.2

Federation Management

The first aspect of federation management needed is a
consistent interpretation of a
FOM or FDD file. This
ranges from ensuring that the same file is actually being
used by each Federate to ensuring that the same handles
are allocated to each class, interaction, attribute and
parameter.

The second aspect is related to the identification

of
federates that are actually in the federation. This includes
allocating consistent (and unique) handles to the
federates while still understanding which federates are
“serviced” by which RTI.

The third aspect involves the other federation
management

services such as synchronization points and
save and restore. While the protocols themselves may be
well defined at a federate level, the internal protocols used
between Local RTI Components are open to interpretation.

A definitive statement on how the
information is moved
from one RTI to another would be essential. Similar
protocol definitions are likely to be needed for all RTI
services.


5.3.3

Declaration Management

Declaration management is relatively simple at a federate
level but internal to an RTI the
re are a number of issues
that are open to interpretation. The consistency of
handles has already been covered under Federation
management. The implication of the Interface
Specification descriptions for subscription services is that
subscriptions are di
stributed to all Local RTIs. It may be
possible that an RTI uses a single Central Federation
Execution to find publish and subscribe intersections and
hence pass to publishers the need to provide certain
attributes. A protocol will be needed between RTIs

that at
least transfers all the publishes and subscribes made by a
particular federate.


Declaration management for DDM is further complicated
since there is no implementation definition for how regions

(or matching regions) are calculated. There appears

to be
a notion of channels that deliver to common sets of
federates (and often implemented as a multicast group).
This channel would probably need to be made into an
explicit concept in an RTI to RTI interface.


5.3.4

Object management

In a bridged or a hierar
chical federation, object attributes
appear to reside in a bridge or representative federate
although they are actually in some concealed federate. For
tree
-
structured federations it is clear that federate is the
representative
-

in other cases the decisio
n is open and
has to be made in a consistent manner.


Unique and consistent naming is required for objects


this requires some consistency or agreement.


Data formats are specified in the FOM but by reference to
external standards. It will be necessary fo
r these to be in
common or translatable formats for RTIs to interoperate.
Even then the message formats, structures and AHVPS
representation will need to be translated or tolerated as
messages pass between federations.


Consistency in the service descripti
on terms (reliability
etc.) is also required between interoperating federations.

5.3.5

Time management

Time Management is a particularly successful area in
HLA. HLA allows federates to be written so that federates

constrained by their own real
-
time clock and fe
derates
constrained by federation time can interoperate. It is true
that some knowledge and understanding is required to
achieve this. Perhaps federate construction tools will
evolve to make this easier.


Where different time management regimes interoperat
e,
especially with radically different time scales, it is
necessary to take an engineering approach to the use of
resources, that is, to be aware of the resource load implied
by a particular update rate. It is sometimes necessary to
represent a single comp
onent simulation by two federates
so that a slowly moving and a fast changing face can be
presented to appropriate parts of the network. None of
this is automatic at the moment.


Issues of numerical stability and accuracy, in connection
with timeliness of
data updates have so far only been
touched on in SISO discussions. These will be important
for validation of federations as a whole and hence
application interoperability.


With time management we are entering the arena where
group protocols are significan
t. Interoperability of
different RTIs requires that they support common
underlying concepts. In general, the gateway idea is not
enough.


A problem for time management interoperability is that the
LBTS information available via the federate interface of th
e

RTI is not sufficient to allow bridging of federations.
Indeed the rationale for hierarchical federations is partly
based on the availability of additional time
-
related
information from the lower federations that can use
specific RTIs.


5.3.6

Ownership Managem
ent

The ownership management protocol appears complex but
can be more easily understood when it is appreciated that
commitment to an exchange always takes place at the
currently owning federate.


A preliminary analysis suggests that the ownership
managemen
t protocol is bridgeable.


5.3.7

Data Distribution Management

Data Distribution management services have been
developed to provide a content based filtering service (as
opposed to the class based filtering available through
Declaration Management) to make HLA mo
re scaleable
and to deliver better performance. The regions and
dimensions used in DDM do not have to be related to data

in the FOM, providing flexible use to allow even different
federates to communicate together in a tightly coupled
region. However alt
hough the interface between the RTI
and the federate is defined, the implementation of DDM is
left to the RTI developer. To achieve interoperability
between heterogeneous RTIs requires a common
understanding of a number of issues:

The first key issue is t
he concept of a channel. Most RTIs
that have provided DDM have used multicast groups as a
means of sending information to a finite determined set of
Local RTI Components (and hence to their associated
Federates). The important part of the channel is a me
ans
of identifying the federates associated with it and the
types of information that goes down it. While this must
be provided internally within all RTIs there is no standard
to identify how this information might be passed between
heterogeneous RTIs.


T
he second related concept regards the quality of service
provided over a channel. This refers back to the issues
mentioned in object management in that if channels are to
be shared between heterogeneous RTIs there must be an
agreed way of defining that qu
ality of service. This relates

to delivery ordering as well as reliability.


5.3.8

Heterogeneous RTI Summary

This section has identified the implications and challenges
if we are to achieve interoperable RTIs. The difficulties are

not trivial and many issues a
re manifesting themselves as
problems in federate to federate interoperability with a
homogeneous RTI. The current RTI Interface
specification should be viewed as a starting point. It
defines a great deal of the essential data that would need
to be moved

between heterogeneous Local RTI
components. We need to continue developing the ideas
and interfaces with the goal of RTI to RTI interoperability
in mind, minimizing (but not denying) changes to the
Interface Specification. Where issues relate purely to
RTI
to RTI interoperability, it is suggested that a
supplementary Inter
-
RTI Interface specification be
developed.

5.4

Communications Interoperability

Communications level independence and the most
efficient use of available resources are the contradictory
requ
irements placed on the interface between the RTI and
its communications support layer. The difficulties are
heightened by the fact that the RTI needs to deal with
group protocols in handling multicast groups and time
management.


To satisfy these conflicti
ng requirements is the challenge
that RTI developers face.


Such problems are dealt with by abstracting appropriate
functionality with the supporting layer, by parameterizing
it and splitting knowledge of the communications layer
into that which must be kn
own by the RTI, and that which
can be supplied by the application layer and simply
passed through the RTI. In practice, the construction of
software to meet these requirements is still challenging,
often requiring considerable analytic and algorithmic
inpu
ts.


A focus on these requirements is more helpful than
analogies to a distributed operating system. It is the
provision of distributed services that is the common
feature, not the functionality of operating systems.

6

C
ONCLUSIONS

HLA provides for the constr
uction of a shared virtual
world dealing in object attributes and interactions, a world
in which a collection of simulations can interact. Another
view of HLA, perhaps more appropriate to civil
applications is as the controlling framework for a
distribute
d whiteboard, or corporate video conference on
which distributed agents work, or even a multi
-
player
Internet
-
based game. From this point of view it is much
clearer as to why we would want to bring together
federations, to be able to interoperate, over eve
ry device
in a kind of I/O computing fabric.


In the current fledgling state of HLA, the interoperation of
simulations requires prior preparation, intense FOM
design, and perhaps even preliminary implementations
leading to simulation wrapper software, and
so on. But
rather than dwell on the challenges we face today, we
should look forward to a state in which HLA is much
easier to use, easier to compose, and definitely easier to
interoperate between our various simulations and
distributed agents.


Admitte
dly, there are two fundamental approaches to
obtaining RTI Interoperability, representing solutions
both above and below the level of the Federate to RTI
Interface. Above this division line are methods involving
the HLA Gateway and Federate Proxy, two meth
ods that
utilize or represent data contained strictly within the
Federate, or information available only through the
federate to RTI interface, respectively. Below this line we
identified the RTI Broker and associated API and a
Network Interoperability Pr
otocol.


HLA Gateway solutions provide an opportunity for non
-
HLA compliant simulations to evolve an interface that
allows them to interact with an HLA Federation. By simple

extrapolation, HLA Gateways can also create interfaces
between HLA Federations, y
ielding new and interesting
architectural variations of HLA Federations we called:
Intersecting, Adjacent and Hierarchical Federations.
When taken together, they form a Federation Community.
Similarly, Federate Proxies create interconnections
between HLA

Federations by creating a
bridge

between
RTI implementations, potentially RTI implementations
manufactured by potentially different vendors.


Below the level of the Federate to RTI Interface, we
discussed two possible and workable solutions that would

all
ow local RTI components to communicate and even
support interoperability between different vendors'
implementations. Both were seen as ways that SISO might
develop new infrastructures in totally different ways.
Initially, the RTI and Communications Forum

asked for a
solution set that yielded an "on the wire protocol." The
result was a proposal for a Networked Interoperability
Protocol. However, the RTI Interoperability Study Group
felt that while that might be the best long
-
term solution, it
might inhib
it experimentation and possible development.
However, a compromise was reached with the proposal of
an RTI
-
to
-
RTI API.


This low
-
level API would still provide a similar construct
to that of the Networked Interoperability Protocol (i.e., a
solution to the
"other" RTI interface). The difference
being that this approach is more of an algorithmic
solution. It would permit RTI developers to not
necessarily agree on
how

an RTI should be built, but
rather on
what

information RTIs needed to know (with
respect to

their internal state) in order to be able to
interoperate with one another. Of course, once an RTI
service level API is identified, a networked based
approach would be relatively straightforward. Indeed,
such an approach would involve a simple one
-
to
-
on
e (1:1)
mapping of the service level API used by the RTI broker
to a protocol header and body construct. The differences
required for the different communications architectures
identified (i.e., networked, shared memory, parallel and
hybrid) could easily
be postulated in light of an agreed
upon RTI
-
to
-
RTI API.


Experiments and analyses of RTI Interoperability are now
needed to prepare the HLA for a wider role. Experience
with the DIS to HLA Gateway has shown that that at one
level, interoperability is usef
ul. What is needed now is to
prototype the RTI
-
to
-
RTI broker and associated API.


HLA has set simulation developers thinking about the
importance of interoperability and has provided a way of
doing it. The ability of a simulation to interoperate with
an
other is now much better because of HLA. But


the
story does not stop there. The issues have
not

all been
resolved, many will not appear until it is too late to do
something about it UNLESS we start preempting the
issues by some forward thinking.


HLA a
nd its implementations need to be cognizant of this
and amenable to controlled change and evolution.
HLA
has a very important immediate purpose
--

to support fixed
combinations of existing simulations. However, greater
ease of use and support for construct
ion of federations
involving new simulations are also desirable. As much as
anything HLA needs to find a process by which it can
evolve. The long road from ARPANET to Internet was
only possible through a period of experimentation and the
gradual constructi
on of a fruitful and truthful process for
evolution of standards. In this case, interoperability was a
paramount goal. SISO should align itself with this goal,
and with driving the future of HLA. It is this that HLA
now needs if it is to repay the effor
ts of its originators with

continued growth and uptake.


Some issues stand out that we must concentrate on if we
are to achieve our simulation interoperability goals :




We need identify what simulation interoperability
means to SISO; we urge you to accept
the study
group's definition as your own.



We must develop a means by which different RTIs
can talk to each other


we need to be able to have
heterogeneous RTIs. Heterogeneous
interoperability is good; its good from a cost
-
effective standpoint, it's good f
rom a competition
standpoint, it's good from the standpoint of
community growth.



We must have a better means of agreeing and
translating alternative model data representations
(Model Interoperability). The FOM and OMT (and
their relationship with the RTI)

will need to be
extended.



We must be able to take advantage in new
networking technology and distribution techniques
and learn from the lessons in the evolution of the
Internet.



We need to review and document the protocols
implied by the interface specifi
cation as a layered
protocol.

7

E
PILOGUE

The dedicated volunteers and contributors of the RTI
Interoperability Study Group have worked diligently to
meet the requirements of the RTI Interoperability Terms of
Reference and to creatively capture the diversity
and
interest of the SISO community. They deserve the thanks
of the simulation community.


The SISO Support Staff, led by Allison Griffin, have
supported this Study Group. They added the energy
required to help us reach the SISO community. The RTII
SG tha
nks you.

I would also like to take this opportunity to thank all of the
folks involved with RTI Interoperability Study Group.
Your comments and insightful discussions have brought
us to where we are.








Ilie Stiharu
-
Alexe

Andre Granger

Uwe K.J. D
ompke

Andreas Tolk

Marco Schumann

Roland Jesse

Jan Tegnir

Ulf Johansson

Mikael Karlsson

Martin Hjelmstedt

Odd Romell

Uwe Krosta

Kay Pixius

Terry Gill

Emmet Beeker

Lawrence Liang

Stephen Turner

Thomas Lake

Kelvin Martin

Steve Jones

Koby Bar
-
Tal

Mitch Peckham

Robert Guy

Ernie Payne

Jesse Zydallis

Mike Bachmann

Ernie Payne

Mark Adducchio

Jim Evans

Jerry Arnett

Francis E. Porras

Patrick Getchell

Mark E Crooks

John Gray

David Mckeeby

Lawrence Goldberg

Walter Peterson

Marilyn Rari
ck

Gail Palmisano

Stephen Swenson

Jeff Fischer

Ted Mulder

Jeffrey Wallace

Daniel Paterson

Tomas Bozack

Ray Drake

Roger Wuerfel

Katherine Morse

Glenn Valentine

James Shiflett

Darren Law

Steve Bachinsky

Jaime E. Cisneros

Robert Howard

Keith
Briggs

Ronnie Turrentine

Jay Graham

Ken Hunt

Anthony Lashley

Thomas Stanzione

Sven Galloway

Don Theune

Michael O'Connor

Clint Parkerson

Alexander Baidin

Karl Shepherd

Jean
-
Pierre Floch

Jean Dostert

Steven R. Drake

Reuben Jones

Nicholas Stra
uss

Rune Lillesveen

Brent Goodwin

Duncan Suttles

John Shockly

David Decamp

Jennifer Cormier

Dan Denny

Alexander V. Baidin

Gordon Miller

Burt Grippin

Duncan Clark

Gerry Magee

Damian Popple

Carl Byers

Jerome Sanders

Brian Fuehrer

Gary Buker

Kent Farwell

Farid Mamaghani

Mary Kruck

Jon Thornburg

Robert

Glenn Gross

Peter Storch

Kimberly Neff

Michael Myjak

Terrell Burks

W. Garth Smith

Michael Hieb

Jim Hollenbach

Sharon Hardy

Michelle Bevan

Mark Raker

Tomoya Tsusue

Masakazu Furui
chi

David Pivetta

Daniel Michel

Ho Kyoung Kim

Javier Velasco

Mr. David Brown

Fang_Tzong Cherng

Allison Griffin

Richard Fujimoto

Thom Mclean

Margaret Loper

Hasan Timucin Ozdemir

Dan Coffin

Reed Little

Insub Shin

Geoffrey Barbier,

Dr. Leigh Lun
sford

8

B
IBLIOGRAPHY

[Arpaci
-
Dusseau, 96]

Arpaci
-
Dusseau, Remzi H.,

Communication Behavior of a Distributed
Operating System,”

Masters Thesis, University of
California at Berkeley, May 1996,

www.cs.berkeley.edu/~remzi/Text/masters.html


[Beebe, 97]

Beebe, B., Bouwens, C., Braudaway, W.,
Harkrider, S., Ogren, J., Paterson, D., Richardson, R.,
and Zimmerman, P., “
Building HLA Interfaces for
FOM Flexibility: Five Case Studies
,” Proceedings of

the 2
nd

Simulation Interoperability Standards
Organization (SISO) Simulation Interoperability
Workshop, Orlando FL, September, 1997.

[Bjornson, 92]

Robert Bjornson., “
Linda on
Distributed Memory Multiprocessors
,: Ph.D.
Dissertation, Technical Report 931,

Yale University
Department of Computer Science, Nov. 1992.

[Bouwens, 98]

Bouwens, C., Hurrell, D., and Shen, D.,

Implementing Ownershop Management with a
Bridge Federate
,” Proceedings of the 3
rd

Simulation
Interoperability Standards Organization (SISO)
S
imulation Interoperability Workshop, Orlando FL,
March 9
-
13 1998, pp. 1126
-
1131.

[Briggs, 98]

Briggs, Keith, “
A Required RTI Gateway
As A Solution To RTI Interoperability
,” Proceedings
of the 3
rd

Simulation Interoperability Standards
Organization (SISO) Si
mulation Interoperability
Workshop, 97S
-
SIW
-
188, Orlando Florida, March
1998.

[Chapin, 89]

Chapin, A. L., “Status of OSI
Standards,” Computer Communication R
e
view, Vol.
19, no. 3, pp. 99
-
118, July 1989.

[Comer, 91]

Comer, D. E.,
“Internetwor
k
ing with

TCP/IP,”

Prentice Hall, Englewood Cliffs, New Jersey,
1991.

[Dasgupta, 99]

Dasgupta, Partha, “
Network Operating
Systems
,” Department of Computer Science and
Engineering, Arizona State University, sponsored by
New York University,
http://milan.milan.cs.nyu
.edu/partha/DOS/nos.html
.

[Filsinger, 97]

Filsinger, Jarrellann and Landrum, E.
Taylor Jr., “HLA Security Guard Federate
,

Proceedings of the 1
st

Simulation Interoperability
Standards Organization (SISO) Simulation
Interoperability Workshop, 97s
-
SIW
-
163, O
rlando
Florida, March, 1997

[Geist, 94]

Geist, A., with Adam Beguelin, Jack
Dongarra, Weicheng Jiang, Robert Manchek, and
Vaidy Sunderam, “
PVM: Parallel Virtual Machine A
Users' Guide and Tutorial for Networked Parallel
Computing
,” MIT Press, 1994.

[Hensha
ll, 88]

Henshall, J., and Shaw, W.,

ISO

Explained: End
-
to
-
End Computer Communications
Standards,”

John Wiley & Sons, New York 1988.

[IEEE 1516]

SISO High Level Architecture Standards
Development Group, “
Draft Standard for Modeling
and Simulation (M&S) Hi
gh Level Architecture
(HLA)
-

Framework and Rules
,” Draft specification,
http://www.sisostds.org/stdsdev/ Nov. 1998.

[IEEE 1516.1]

SISO High Level Architecture Standards
Development Group, “
Draft Standard for Modeling
and Simulation (M&S) High Level Archit
ecture
(HLA)
-

Federate Interface Specification
,” Draft
specification, http://www.sisostds.org/stdsdev/ Nov.
1998.

[IEEE 1516.2]

SISO High Level Architecture Standards
Development Group, “
Draft Standard for Modeling
and Simulation (M&S) High Level Architec
ture
(HLA)
-

Object Model Template (OMT),”
Draft
specification, http://www.sisostds.org/stdsdev/ Nov.
1998.

[Knightson, 88]

Knightson, K. G., and Knowles, T., and
Larmouth J.,
“Standards for Open Systems
Interconnection,”

McGraw
-
Hill, New York, 1988.

[My
jak, 98]

Myjak, Michael D., Carter, Russell L.,
Wood, Douglas D. and Mikel Petty, "
A Taxonomy Of
Multiple Federation Executions
", Proceedings of the
20
th

Interservice/Industry Training, Simulation, and
Education Conference, Orlando FL, December, 1998.

[Ros
e, 90]

Rose, Marshal T.,
“The Open Book: A
practical Perspective on OSI,”

Prentice Hall,
Englewood Cliffs, New Je
r
sey, 1990.

[Pullen, 97]

Pullen, J. Mark, Myjak, Michael and
Chris Bouwens, "
Limitations of THE Internet
Protocol Suite for Distributed Simulat
ion in the
Large Multicast Environment
," Proceedings of the 1
st

Simulation Interoperability Standards Organization
(SISO) Simulation Interoperability Workshop, 97S
-
SIW
-
136, Orlando Florida, March 1997.

[RFC 2502]

Pullen, J. Mark and Myjak, Michael, and
Ch
ristina Bouwens, ‘
Limitations of Internet Protocol
Suite for Distributed Simulation
,” Internet
Engineering Task Force, Request for Comments No.
2502. http://www.ietf.org/rfc/rfc2502.txt

[Saltzer, 84]

Saltzer, J. H., Reed, D. P., and D. D.
Clark, “
End
-
toEn
d Arguments in System Design
,”
IEEE/ACM Transactions on Computer Systems, Vol.
2, No. 4, Nov. 1984, pp. 276
-
288.

[SISO
-
OSP, 99]

Simulation Interoperability Standards
Organization Outbrief Summary Points, Biannual
Simulation Interoperability Workshop, Logis
tics
Forum points LOG KP2 and KP3, Spring, 1999.
http://www.sisostds.org/siw/99Spring/outbrief.htm

[Stallings, 87]

Stallings, A.,
“Handbook of Computer
-
Communications Standards, Vo
l
ume 1: The open
Systems Interconnection (OSI) Model and OSI
-
Related Standar
ds,”

Macmillian, New York, 1987.

[Stevens, 90]

Stevens, W. R., “UNIX Ne
t
work
Programming,” Prentice Hall, Englewood Cliffs, New
Jersey, 1990.

[Wood, 97]

Wood, D., Cox, A., and Petty, M. (1997).

An HLA Gateway for DIS Applications
”,
Proceedings of the 19
th Interservice/Industry
Training, Simulation, and Education Conference,
Orlando FL, December 1997.