Implementations of Hierarchical Federations

loyalsockvillemobΔίκτυα και Επικοινωνίες

27 Οκτ 2013 (πριν από 4 χρόνια και 13 μέρες)

123 εμφανίσεις

Implementations of Hierarchical Federations

Michael D. Myjak

Sean T. Sharp

The Virtual Workshop & HLA Products

P. O. Box 98

Titusville, FL 32781

<
mmyjak@virtual workshop.com
>

<
ssharp@virtual workshop.com
>



K
EYWORDS

HLA,

RTI,

R
EAL
T
IME
P
ROTOCOL
,

E
MBEDDED
S
IMULATION
,

RTOS,

J
AVA
,

J
AVELIN
S
IMULATION


A
BSTRACT
:

The High Level Architecture (HLA) supports the interoperation of sets of simulations

within the context
of a single Federation Object Model (FOM). This is accomplished using the HLA Federate Interface Specification,
which refers to services provided for by software (a Run
-
Time Infrastructure) that is not standardized under the
current HLA

specification. Under HLA, federates are limited to operating within the context of a single federation
execution, defined by the FOM and communicating through a single RTI. But a non
-
standardized RTI presents an
interesting loophole though which many us
eful simulations involving multiple federations and multiple RTIs may
exist.


In this paper, these issues of HLA interoperability and reusability, as well as the ramifications of not having an RTI
standard supporting the whole HLA concept are discussed.
We present some of the relevant background
information on how interoperable computer systems came to be and where HLA stands in light of this. We present a
few different approaches to providing interoperability between multiple federations, including the
Gateway, Proxy,
Broker, or protocol
-
based implementation. We describe each of these solutions, as well as some of the issues
involved in supporting them.




Given a choice between two explanations, choose the simplest
--

the explanation which
requires th
e fewest assumptions.


William of Occam




1.

I
NTRODUCTION

The
High Level Architecture (HLA)

is an attempt to
address two key issues in the world of Modeling and
Simulation (M&S): "…promoting interoperability
between simulations and aiding the reuse of mode
ls in
different contexts." [IEEE 1516] These goals are by all
accounts, admirable. However, the particular draft of the
HLA standard being balloted through the IEEE during the
summer of 1999, falls short of obtaining both of these
goals.


1.1.

B
ACKGROUND
H
IS
TORICAL
C
OMMENTS

In recent years we have witnessed the evolution of
federated

simulation systems. (
Figure
1
) Federated
simulations claim to enhance the development,
deployment and interoperability of
federates

(e.g.,
independentl
y developed simulation applications,
components or related tools) that build upon a rich
heritage of simulator interoperability. We can trace the
roots of modern federated simulations to the development
of SIMNET in the early 1980’s, but tracing the
devel
opment of
interoperability

among distributed
systems is a little less direct.


As SIMNET experience evolved, so did standardization,
with interoperability as a principal goal. This came in the
guise of Distributed Interactive Simulation (DIS) in the
late
1980’s and later, the Aggregate Level Simulation
Protocol (ALSP) in the early 1990’s.
1

Integrating these
two concepts together (DIS and ALSP) has been dubbed
Advanced Distributed Simulation (ADS) and is seen as
another milestone in the development of hybri
d
simulation systems. ADS concepts, upon which the High
Level Architecture is based, are the legacy that grew out



1

Note that most DIS and ALSP
-
based systems are still in
use today.

of SIMNET and distributed networked computing age,
and is the basis for this paper.


1.2.

E
VOLUTION TO
R
EVOLUTION

Consider the evolutionary change
s in software and
networking during the evolution of SIMNET (through
ADS) that permitted dissimilar simulation applications to
evolve interoperable solutions. Recall that this was a time
before the
Commodity Internet
2

existed. In 1984, the
Defense Advanc
ed Research Projects Agency (DARPA)
released the management of the ARPANet over to the
National Science Foundation, finally declaring that Inter
-
Networking (i.e., the Internet) was no longer an
advanced

research application. In other words, DARPA considere
d
the Internet to be on par with other engineering projects.
That's not to say that there weren't other research issues to
be resolved (e.g., multicasting, service quality
management, the World Wide Web, etc.).


Nevertheless, when the ARPANet transitioned

over to the
NSFnet, there were barely 100K nodes attached to the
ARPANet, Intel 8086 computers were
hot
, and fledgling
Microsoft witnessed the release of the Apple Macintosh
computer, complete with a window system, mouse, and
built
-
in networking. Bridges

and Gateways provided the
interconnecting function joining often
-
dissimilar
networks. And where protocols were concerned, the
norm was to support literally dozens of different,
generally non
-
interoperable protocols over the same
network. In short, the g
eneral concept of interoperability
still eluded us. Vendors who saw profit in being able to
dominate the network enterprise avoided protocol
standardization. So it's quite understandable now that
SIMNET, another DARPA project, was not based upon
DARPA's o
wn Internet Protocol (IP). The fact is that
understanding, practice, and experience with the use of IP
was rather limited at the time. People did not know the
level of interoperability that would one day be achieved
with IP and its suite of protocols, ev
en through it would
one day change the world.


In the 1970’s and 1980’s, many applications and products
(including simulation applications) were not
interoperable. In fact, the norm was quite the contrary.
These systems were typically "one
-
of
-
a
-
kind" solu
tions,
often
-
employing vendor
-
specific hardware and operating
system platforms, as well as vendor
-
dependent software
implementations. (Computing was expensive in this age
of homogeneous system solutions.) Software was rarely
portable, as porting costs we
re prohibitively expensive.
Still, then as now, many users and developers alike were
concerned about software reuse and interoperability.
Heterogeneous environments were slow to evolve, but



2

The Internet, as we now know it today.

interoperability in the general sense was not yet
achievable. F
or these reasons and more, the International
Standards Organization (ISO) developed the Open
Systems Interconnection (OSI) initiative. [Chapin, 89]
[Comer, 91] [Stallings, 87]


The acceptance and popularity of IP continued to grow
through the 1980's, as di
d interoperable systems based on
the OSI model.
3

With the introduction of the NSFNet,
routing became the norm and the dominance of the IP
protocol stack became clear. The number of “other”
routable protocols began to diminish rapidly. XNS,
AppleTalk, IPX,
even DECNet quickly fell by the
wayside. However, the simulation community was slow
to adapt to this new infrastructure model. Neither ALSP
nor DIS (version 1.0)
4

were based on the IP protocol
stack.


In 1991, the NSF
lifted its restriction on commercia
l use
of the NSFNet and the modern Commodity Internet was
born. Along with the birth of the Internet came the first
“point
-
n
-
click” Browser, called
Gopher
. (Remember
that?) Two years later we saw the evolution of
Mosaic.

The birth of the World Wide Web
followed in 1993.
Thus began one of the greatest success stories for
interoperability. By 1996, the Internet was growing at the
astounding rate of 341,634% (according to the NSF),
supported over 150 countries and connected over 10
million hosts. By July
1998, there were over 168 million
hosts on the Internet and IP address space is now
becoming scarce. Consequently,
version 2 of DIS aligned
itself with the Internet Protocol Suite. But not ALSP, and
not HLA. (Are we just slow to learn?)


This is the succe
ss story of interoperability in
heterogeneous environments.

Today the value of a
standards driven, system
-
independent communications
mechanism is well understood. Software development
costs are manageable because of vendor competition and
platform indepe
ndence. Therefore, the long
-
term success
of the HLA and interoperable federations will hinge on
the ability to standardize HLA in terms of IP.


Relatively speaking, distributed simulation is a small
community (when compared to the World Wide Web or
the sc
ale of the Internet). But there are some significant
changes on the horizon. Applications such as distributed
simulation and shared immersive environments had been
presented as the driving force behind the development of
the Next Generation Internet. As

an application group,
this type of distributed application is quite capable of



3

In the mid 1980's, the OSI model fell into disfavor with
advent of the "End
-
to
-
End" argument. [Saltzer, 84]

4

The first version
of DIS was essentially SIMNET with
an IEEE standardization stamp.

stressing the low
-
latency, high
-
bandwidth capabilities of
even the fastest network infrastructure. Further, the
application and middleware layer technology being
developed tod
ay in the simulation community may one
day find its way into a wide variety of non
-
military
applications, such as immersive telemedicine,
teleinstrumentation, and entertainment. So the
technology, properly applied, holds great promise.


2.

HLA

IN THE
S
POTLIG
HT OF
H
ISTORY

The High Level Architecture (HLA) supports the
interoperation of federates
5

within the context of a
single

Federation Object Model (FOM). Federates use the HLA
Federate Interface Specification to communicate FOM
data with one another through
a software component
called the Run
-
Time Infrastructure (RTI).
The RTI
software therefore, is to provide the services referenced in
the Federate Interface Specification. This interface is then
used by federates to coordinate operations and data
exchange d
uring a runtime execution. However, while
t
he HLA Federate Interface Specification identifies the
services, standardization of the RTI
6

is not. While the
HLA
depends

upon the RTI to communicate, it is not
considered part of the HLA specification. Yet an
y HLA
implementation requires a FOM, a set of cooperating
federates,
and

an RTI. Go figure.


2.1.

T
HE
G
ROWTH OF
HLA

Through the HLA, federates (e.g., C4I simulations,
embedded applications, tools, video games, interfaces to
other applications) are combined with

the RTI and a FOM
into a set called the
federation
. A run of a given
federation is a
federation execution
. Under HLA, and
according to the Draft [IEEE 1516] suite of specifications,
the implied mode of operation is for federates to
cooperate within the c
ontext of a single federation
execution. (See
Figure
1
.) However, the definition of an
HLA Federation sans RTI leaves open several interesting
architectural possibilities, namely the possibility that a
federate may itself, be re
presented by another federation,
or perhaps be
simultaneously

joined to two (or more)
concurrent federation executions (supported by the same,
similar or perhaps even different FOMs).


If two or more concurrent federation executions, of the
same or differe
nt federations
7
, could have one or more
federates in common, then presumably the common



5

Federates are
applications, tools, loggers, viewers, etc.,
[read
anything
].

6

A complete, concise, functional and unambiguous
description of the RTI is not provided in [IEEE 1516.1].

7

Dif
ferent federations are defined by having different
FOMs.

federate(s) can be said to "broker" the exchange
information between federation executions. In this way,
one federation execution can exert influence (or yield a
profou
nd impact) upon the events of another. This
process of brokering information between federations
gives rise to the notion that an entire federation might be
perceived as a "pseudo" federate by another, perhaps
"higher level" federation. Such "hierarchica
l federations,"
loosely referred to as a
federation community
, are
similarly undefined within the current context of HLA. (A
Community
is a combination of federations and RTIs
working together to achieve a common goal.)

The result
of this is that several d
istinct types of multi
-
federation
execution possibilities exist. The remainder of this paper
will describe some of these architectural variants of HLA
federations, some of the performance efficiencies (e.g.,
scalability, usability, etc.) that can arise fr
om them, and
how we propose to support these proposed simulation
implementations using
J
AVELIN
.



Figure
1

Prototypical HLA Federation

3.

HLA

AS
M
IDDLEWARE

It is important to both understand our past, so we

don't
repeat the same mistakes, while at the same time, its also
important to understand our future and where HLA fits in
the grand scheme of things. Lately, the realization of the
limitations of the existing IP protocol 5
-
layer stack has
become apparent

[Myjak, 97]. In particular, the top layer,
describing the Application Layer Protocol has given way
to a new term called
Middleware
. This is because the
concept behind an Application Layer Protocol was that
there would be a 1:1 mapping between in
-
bound re
quests
from the application and out
-
bound Protocol Data Units
(PDUs) to the Transport layer below. A middleware layer
on the other hand, may not have such restrictions as
additional functionality has been added. Further, its now
recognized that in order
to obtain an appropriate balance
between performance of the network and service quality
of the application stream(s), some control information
must be obtained directly from the application (or perhaps
as a function of the Middleware). After all, the netw
ork
infrastructure doesn't understand an application's data any
better then the HLA RTI understands Federate state
information. Therefore, while middleware products are
now recognized as both feasible and useful, we recognize
that federated simulations lik
e HLA do not fit neatly into
the application layer in the IP 5
-
layer model. Thus
without a standard implementation for the HLA RTI,
interoperability concerns with continue to exist.


3.1.

HLA

I
NTEROPERABILITY

Today HLA is forging new ground and is the basis b
ehind
the development of federated simulation systems.
Sponsored and developed by the Defense Modeling and
Simulation Office (DMSO), the HLA provides an
architectural
framework

and an interaction specification
that addresses the interoperation of dissimil
ar simulations
at the Application Program Interface (API). HLA is also
being investigated for use in acquisition confirmation,
software integration and testing, as well as the reuse of
legacy simulation systems.


Common knowledge and experience with the m
odern
Internet, protocol development, the Open Systems
Initiative, and even common use and experience with
interactive applications demonstrates the fact that
interoperability is difficult to achieve at best; ubiquitous
interoperability for an architecture
, doubly so. To that
end, a well
-
known axiom in the end
-
to
-
end underworld of
the Internet, one often attributed to Dr. Steve Deering
drives this point home:



Interoperability is inversely proportional to
Flexibility.


Yet "
Flexibility is the aim in the de
finition of the HLA…"
(at least according to these same HLA Rules). So how is
this possible? Dr. Van Jacobson, a principle architect of
the current Transmission Control Protocol (TCP), has
often said that the more dials, knobs and switches one
makes avail
able to an application programmer, the less
likelihood one will have of being able to interoperate. So
if HLA is so highly flexible, then how can it also be
Interoperable? The short answer is, of course, that it
cannot.


So how then does the HLA claim In
teroperability? By
definition, the HLA claims to support the interoperation
of federates
8

within the context of a single Federation
Object Model

(FOM).
So in HLA terms, Dr. Jacobson's



8

HLA federates can be simulation applications, tools,
loggers, viewers, or any other software application.

"knobs, dials, and switches" are actually HLA FOM data.
But how are th
e actual attributes and interactions passed
about? The answer lies in HLA Rule #3:



"During a federation execution, all exchange of
FOM data among joined federates shall occur via
the RTI" (Run Time Infrastructure).


HLA Interoperability therefore is accomp
lished by
federates using the (draft) HLA Federate Interface
Specification. [IEEE 1516.1]
Such federates use this
specification to communicate a' priori FOM data (or some
proper subset thereof) with one another through
another

software component, the RTI.

Further, note that while the
Federate Interface Specification describes the federate's
interface to a collection of services, these services, and
how they interoperate or distribute their own state, is
ostensibly provided for by an RTI, and is
not part o
f the
HLA specification
! In other words, the success of HLA
interoperability resides with a requirement to use a
middleware implementation, the complete description of
which is
not yet standardized
!


3.2.

HLA

R
EUSABILITY

Certainly, federates are grouped into s
ets which employ a
common FOM. Such groupings are called a federation,
and when run, are called a federation execution. During a
federation execution, the FOM data (i.e., attributes and
interactions) specifies a contract between members of a
federation.

This contract is based on the type of class
attributes and interactions that are to be supported.
However, the FOM does not describe an object in its
entirety (i.e., behaviors, private data, etc.). Only publicly
defined class attributes and interaction
classes are
described by the FOM. (See draft [IEEE 1516.2].)


So where is the software reuse? One might claim, that by
defining a Reference FOM [RFOM SG], federations may
obtain federate software reuse by agreeing to use a
common set of data. However, ma
ny factors may inhibit
the most straightforward or direct reuse of software.
Suppose for example, that some federation exists with a
well
-
defined collection of federates, a known Reference
FOM, and a particular vendor's specific RTI. Further,
suppose tha
t this federation has since been "certified" by
going through some sort of Verification, Validation and
Authentication (VV&A) analysis. Certainly, such a
federation might be reused over and over again in some
particular training scenario. However, when o
ne makes a
change to this infrastructure, say update the RTI or
replace a federate, then the entire federation would have
to be recertified because a
new
federation has superceded
the previous federation. This is no longer the same
infrastructure because
some of the "knobs, dials and
switches" have changed.


While only a simple demonstration, consider what
happens when changes are required to the FOM. Turn a
dial, throw a switch, and once again, a whole new
federation exists. In fact, we estimate that the

FOM is
more far more likely to change than the federates will.
Once again the federation must be recertified against the
VV&A interoperability requirement. (This can get
expensive!)


Now consider for a moment that we really desire vendor
independence in

federate software and in RTIs. Suppose
that these suggested changes are commonplace, being
performed and thus generating new federations often,
even when used in even the most stable of training
environments. Isn't this the desired norm for HLA
operation
s? What happens when say, a new FOM is
required every time a federation execution is desired? A
process termed the FOM
-
a
-
roma quickly ensues. Can
such a software environment ever obtain certified
interoperability (through VV&A) in such a way that the
inve
stment in federate software development can be
reliably reused? Once again, the answer is no. Not
because federates and FOM objects cannot be reused
independently, but because they cannot be reused
independently when constructed over non
-
standard
infrastru
ctures.
Change any one of these, and any savings
in terms of reusability are lost in interoperability
revalidation.



4.

M
ULTIPLE
F
EDERATIONS


4.1.

F
EDERATES AND
F
EDERATIONS

As stated in the HLA Federate Interface Specifications
(DoD, 1998b), "A
federation

is the

combination of a
particular FOM, a particular set of federates, and the RTI
services.” This model of interoperation is generally
considered to be self
-
contained, and would be if all three
components were standardized. That is to say that the
basis for c
ommunication, the RTI, will pass data
described by the FOM, at the disposition of the federates
during a federation execution. However, it is
architecturally conceivable if not inevitable that
simulations may interoperate across FOMs, and in
federation co
mmunities, since the standardization of RTIs
is as yet undefined. This notion of multiple federation
interoperation

and how such federation configurations
might be structured requires careful interpretation of the
balloted HLA rules [IEEE 1516], as will b
e discussed
later. The current rules are given here for reference
(
Table
1
).


The original developers of the HLA had intended that a
simple, single federation model (e.g., federates, FOM, and
RTI) would suffice for the vast major
ity of federated
simulations (as in
Figure
1
). Indeed, some considered this
architecture to be complete. However, [Briggs, 98]
showed that this closed
-
notion of federation architecture
will lead to "stove
-
piped" solutions, as dif
ferent service
requirements will eventually lead to multiple, non
-
interoperable, RTI implementations. Further, the goal of
HLA to support all models and simulations will not be
met, as some models cannot be simulated using existing
federated simulation tec
hnology (i.e., security, varying
levels of fidelity, embedded system devices, etc.).


Federation Rules

Rule 1

Federations shall have an HLA FOM,
documented in accordance with the HLA OMT.

Rule 2

In a federation, all simulation
-
associated object
instance

representation shall be in the
federates, not in the run
-
time infrastructure
(RTI).

Rule 3

During a federation execution, all exchange of
FOM data among federates shall occur via the
RTI.

Rule 4

During a federation execution, joined federates
shall inte
ract with the RTI in accordance with
the HLA interface specification.

Rule 5

During a federation execution, an instance
attribute shall be owned by at most one
federate at any given time.

Federate Rules

Rule 6

Federates shall have an HLA SOM,
documented

in accordance with the HLA OMT.

Rule 7

Federates shall be able to update and/or reflect
any instance attributes and send and/or receive
interactions, as specified in their SOMs.

Rule 8

Federates shall be able to transfer and/or
accept ownership of insta
nce attributes
dynamically during a federation execution, as
specified in their SOMs.

Rule 9

Federates shall be able to vary the conditions
(e.g., thresholds) under which they provide
updates of instance attributes, as specified in
their SOMs.

Rule 10

Fe
derates shall be able to manage local time in
a way that will allow them to coordinate data
exchange with other members of a federation.

Table
1

The HLA Rules.

4.2.

D
EFINITION

We define a
multiple federation

(or
multi
-
federation
) as a

set of more than one currently executing federations to
which one or more federates are simultaneously joined.
Multi
-
federations may include one or more FOMs. This
logically gives rise to a new term, a Federation
Community
. [RTII SG] In a federation co
mmunity, intra
-
federation federate communication would utilize a
common FOM, while inter
-
federation communications
could embody several different FOMs and mechanisms
that otherwise in some way use the events of one
execution to influence events in another.


4.2.1.

The Precedent For Multiple Federations

The HLA proto
-
federation experiments were quickly
followed by a proposed HLA security architecture design
[Filsinger, 1997]. That architecture indicated that a single
(i.e., common) FOM would not suffice for federa
tion
executions requiring multiple security levels. Such
federations would require some special action to be taken
in handling classified data (e.g., secret
-
high to
unclassified). To resolve these issues, the HLA security
architecture proposed the use of

a security guard process.
The guard process would “scrub” the classified
information from data flowing to unclassified federates
and/or augment the unclassified data with relevant
classified information when flowing in the reverse
direction.


The secur
ity details, while interesting, are not of concern
in this paper. What is interesting however, is relationship
between the guard process and the adjoining federations,
as it has a direct parallel to the old ARPANet Gateways.
Gateways (before Routers) wer
e responsible for protocol
translation, a process where by the integrity of the
application level data was preserved while the
transporting network protocol (e.g., IPX, ChaosNet, etc.)
was exchanged between dissimilar network domains. The
security archite
cture design is clearly based on a single
federate (the “HLA Security Guard Federate”) connected
to two concurrently executing federations. [Filsinger,
1997] This "gateway" architecture certainly provides the
precedent and initial example of multi
-
federat
ions.


Therefore, the question is no longer one of whether or not
such multi
-
federations
can

interact, but rather
how

will
two (or more) federations cooperate and interoperate in a
community. Since HLA provides no defined support
services for federation c
ommunity interoperation, a
secondary question must be raised as to what kind of
additional services might be required within or by the RTI
to facilitate such interoperability.


Similar issues have recently been raised relative to other
scenarios (e.g., mul
tiple levels of detail, differing fidelity
requirements, etc.) all of which appear to require or at
least involve multi
-
federation configurations. [Fidelity
SG]


4.2.2.

Multiple Federations And The HLA Rules

On the surface, it would appear by definition that mult
iple
concurrently executing federation executions could not
interact with one another. After all, a federation is
defined by its federates, a singular FOM, and an (albeit
non
-
standardized) RTI. However, non
-
interoperable
federations can diminish the over
all potential (not to
mention the scalability) of federated simulation
technology. [Briggs, 98] Multi
-
federations are only
interesting when inter
-
federation communications can
exist. But how is this to be accomplished? That involves
a thorough understan
ding of the HLA rules.


The HLA Rules describe precisely how federates and
federations are permitted to interact (
Table
1
). Note that
these rules state specifically how and under what
conditions the Federate Appli
cation Program Interface
and Federation Object Model are to be used by a
federation. We note that according to the draft rules,
federation community interaction is tenuous at best. One
interpretation of the rules indicates that all federations
must take
into account all of the data being represented by
all relevant FOMs; this as some have suggested, is called
the “SuperFOM” model. An alternative to the SuperFOM
approach would be to utilize mechanisms and methods of
communication
outside

of the federation
, to communicate
with other entities (e.g., federates, sub
-
components)
“outside” of the RTI. This “out of band” communication
mechanism can be used to communicate FOM data
outside of a given federation if it is not directed at another
federate
within

the
source federation. But perhaps even
more importantly, as we noted above, the implementation
of the RTI is
outside

of HLA! This leaves open an
interesting opportunity, as one would expect that an HLA
RTI would implement the Federate Interface
Specification
, one might not expect that the same RTI
might provide other features and additional services
outside
of the HLA definition in support of a federation
community.


5.

T
YPES
O
F
M
ULTIPLE
F
EDERATIONS

Federation connectivity, which gives rise to a federation
commu
nity, falls into one of 4 fundamental combinations
of homogeneous and heterogeneous federations and RTI.
Each of which can be spanned by either a Gateway, a
Proxy (previously known as a Federate Bridge), a Broker,
or an "on
-
the
-
wire" protocol. Each of the
se four will be
described in detail below.


5.1.

F
EDERATION
G
ATEWAY

A federation
gateway

is a device that performs a
interconnecting translatory function between two discrete
federation executions. (See
Figure
2

A typical Federation
Gateway.
)


In the Federate Gateway solution, a connection point is
provided within the scope of the joined federate
applications, which then transfers information to and
through the gateway. Connectivity
and

translation can
occur between two (or more) dis
tinct heterogeneous
Federations in this fashion,
or

between a Federation(s)
and/or some other external
non
-
HLA

application. In other
words, it is important to know that a federation gateway
provides an "out
-
of
-
band" communication path between
federation ex
ecutions.


The security Guard Federate for example, is an instance
of a Federate Gateway operation. With the security
model, the guard federate is performing a gateway
process. It is not only providing one federation (via the
joined federates) with stat
e information about another
federation, but it is also translating sensitive information
into unclassified data in one direction, and enhancing data
transfer information in the other. In this instance, there is
also the potentially that data augmentation
is also
occurring, with either fidelity or resolution (or both) being
applied to enrich the inbound data.



Figure
2

A typical Federation Gateway.


In the security model, both federations have dissimilar

FOM data (i.e., classified vs. unclassified data). But this
may not always the case. Take for instance, the DIS to
HLA Gateway developed by the Institute for Simulation
and Training. [Wood, 97] In this model, the gateway is
providing a translatory fun
ction between DIS
-
based
applications on one side, and an HLA federation on the
other. In this case, the HLA Gateway performs the
protocol translation between DIS PDUs and an internal
HLA Federate, which in turn communicates to the HLA
RTI through the Fede
rate API.


5.2.

F
EDERATION
P
ROXY

The HLA rules do not specifically prohibit or permit a
federate from being attached to multiple, concurrent
federations. Indeed, this leaves the whole subject of
multi
-
federation interoperation in a rather gray area.
HLA feder
ate rules indicate that the federate must have a
SOM that specifies [completely] the elements
and

interoperability requirements (e.g., classes, attributes,
interactions, transport type, etc.) of that federate. The
current draft of the HLA federation rules

appears to
indicate that only one FOM is permitted in a federation,
and that “… all exchange of FOM data among federates
shall occur via the RTI.” (Note the singular reference to
both RTI and FOM.) There is neither mention of, nor
prohibition of a singl
e federate being attached to multiple,
concurrently executing federations. This is the realm of
the Federate Proxy. (See
Figure
3

A typical proxy
federate architecture.
)



Figure
3

A typical proxy federate architecture.

When using a Federate Proxy, multi
-
federation
interoperability is accomplished using only the services
defined in the Federate Interface Specification API. In
operation, the federate proxy appears as any other
fe
derate to each federation execution. However, the
results of either federated simulation may now be altered,
as each federation now has some influence upon the other
(as defined by the simulation application developer).
Federate proxies can perform variou
s scenario
-
dependent
functions such as performing data replacement, data
composition, data decomposition, or other data
transformation operations. Recent experiments of this
type, described in [Beebe, 1997] and [Bouwens, 98],
have been characterized by t
he configuration of one
federate as a member of at least two federations. Although
federates proxies are not explicitly permitted by Draft
HLA Rules, [Bouwens, 98] indicated that in this
configuration, the bridge federate is actually more
complex than the
Gateway operation because a separate
federate ambassador and FOM is often used for each
adjoined federation.


Historically, the terms
federate bridge

and
federate
gateway

have been used almost indiscriminately in an
attempt to characterize a federate in a
proxy service,
managing asymmetric data flows, and applying bi
-
directional transformation functions as necessary. These
systems have been dependent upon the Federate Interface
Specification, as well as relevant FOM data (potentially
for two or more federa
tions being adjoined). Given that
this is the definition of a proxy service, it should be clear
that a universal proxy capable of joining federates and
federation communities indiscriminately, will not be
possible. Each proxy must build a unique set of
t
ransformation operations between any two representative
FOMs/federations. And since the transposition of such
data is not reciprocal, each data flow will require its own
transformation matrix (hence asymetrical).


5.3.

F
EDERATION
B
ROKERS

In principle, the brok
er is neither a gateway nor proxy
federate because as shown in
Figure
4
, brokers
interoperate directly with the RTI. That is to say, they are
not constrained to the Federate Interface Specification.
Instead, RTI
-
brokered interop
erability is accomplished
through an as yet undefined RTI
-
to
-
RTI API. This
a
p
proach allows for heterogeneous RTI implementations
to communicate directly with one another, or between
homogeneous RTI components in a true distributed
environment.


While the
concept (and contents) of an RTI to RTI API
are not the focus of this paper, it is worth while to broach
the subject here. According to the HLA rules, the HLA
RTI shall contain no federation application state
information. It maintains only whatever knowl
edge and
state information is necessary to service the supporting
joined federates. In short, it is not a repository and wealth
of knowledge from which federates can draw upon. This
distinction is crucial to understanding some of the basic
requirements o
f an RTI
-
to
-
RTI API. From each
federation's perspective, the directly connected local RTI
component must be capable of managing the local
federation's internal state.


For one federation to communicate with another, using
brokers, information must be pas
sed in some a' priori
fashion. The information passed is both federation
-
global
FOM data, as well as RTI internal state information.
Such transfers between RTIs and/or RTI components will
include information that may in part be documented in the
FOM, and

data which also describes the current state of
the overriding federation. Further, data must flow from
federation
F
1

into federation
F
2

(and visa versa), using a
broker just like a network bridge passes (or filters)
information, without modification or i
nvolving any
protocol translation, or modification of the federate state
data being passed between the two joined federations in a
community.



Figure
4

A typical brokered federation architecture.

5.4.

F
EDERA
TION
P
ROTOCOLS

While gateways, proxies and brokers may receive the
most attention in the near term, they are by no means the
only example or possible interpretation for handling
multi
-
federation interaction. As proposed, gateway or
proxy federate performs

a rather complex set of
functionality (i.e., asymmetrical, bi
-
directional data
transformation between two federations with often
varying Quality of Service requirements). Their
differential information transformation between the
destination federation all
ows them appear (at least to the
destination federates) as locally attached federates. But
what about true federation interoperability and the
ultimate answer to RTI
-
to
-
RTI interoperability? What is
the ultimate solution? Isn't the RTI
-
to
-
RTI API proposed

for use by RTI brokers sufficient to do the job? After all,
a well
-
defined interface between RTIs and/or RTI
components should be sufficient.



Figure
5

Federation RTI
-
to
-
RTI Protocol Solution.

The an
swer is no, and for the same reason that the
Federate API wasn't sufficient for to solve the federated
simulation interoperability problem: there still exists a
lower layer, upon which higher layer interoperability is
still dependent. API has to be transl
ated into machine
language sometime. And that final translation needs to
have a 1:1 mapping, just like application layer protocols
in the IP 5
-
layer model. The bits that actually make it to
the network protocol layer must have a known meaning; a
precise
determination. Often times this is referred to as
the
“On the wire”

approach to interoperability. In short,
this is accomplished by requiring RTI ve
n
dors to use a
standard protocol interface for communications between
whole RTI implementations and optiona
lly, between local
RTI components. (Note: this approach allows Local RTI
Components to share state information d
i
rectly.)


6.

I
SSUES

It should make little architectural difference whether or
not a federate application is managed by a single
processor, is supp
orted by multiple process threads, or is
distributed across several platforms. Further, it should
also make little difference how federates and federations
are composed in order to provide an adequate and
sufficient simulation scenario. Even from a single

host,
Intra
-
federate communications can occur outside of the
HLA RTI. Therefore, it doesn't take any great leap of
faith to see that Inter
-
federation communications can also
occur outside the HLA federate API. So whether a
federate (or RTI) is composed
of components that reside
on separate hosts, or run as multiple threaded processes
within a single host, should pose little or no difference to
a conformant HLA federation. The end result is that the
supporting architecture should be modular and robust
en
ough to support these variants and
still

interoperate.


Assuming a modular distributed architecture, a
component federate or RTI implementation may represent
a significant performance improvement for certain
applications (e.g., real
-
time simulations, C4I
and other
wireless systems). This improvement may be obtained by
bringing more processors to bear on the computationally
intensive aspects of the simulation, without unduly
increasing communications load through the RTI. There is
a lot of room for growth
here.


Many types of complex high fidelity simulations are
naturally distributed and require moderate bandwidth
exchange of interface data. An example is the
multidisciplinary coupling of elastic structural and fluid
dynamics solvers in the modeling of ai
rcraft flight
dynamics. Whereas the other federates in the federation
at large may be interested only in location, displacement,
and other external characteristics of the complete federate
simulation object(s), high fidelity often requires
significant exc
hange of internal data among the
distributed federate components. Thus, by additionally
partitioning the network load by federate or RTI
component functionality (rather than solely by simulation
instance attribute locality), the scalability of the overall

HLA simulation is thus enhanced. This functional
partitioning is explicitly defined in the hierarchical multi
-
federation architecture of HLA federation communities.

7.


B
IBLIOGRAPHY


[Arpaci
-
Dusseau, 96]

Arpaci
-
Dusseau, Remzi H.,

Communication Behavior of

a Distributed
Operating System,”

Masters Thesis, University of
California at Berkeley, May 1996,

www.cs.berkeley.edu/~remzi/Text/masters.html



[Beebe, 97]

Beebe, B., Bouwens, C., Braudawa
y,
W., Harkrider, S., Ogren, J., Paterson, D.,
Richardson, R., and Zimmerman, P., (1997).

Building HLA Interfaces for FOM Flexibility:
Five Case Studies
”, Proceedings of the 1997 Fall
Simulation Interoperability Workshop, Orlando
FL, September 8
-
12 1997,

pp. 1027
-
1034.

[Bjornson, 92]

Robert Bjornson., “
Linda on
Distributed Memory Multiprocessors
,: Ph.D.
Dissertation, Technical Report 931, Yale
University Department of Computer Science, Nov.
1992.


[Bouwens, 98]

Bouwens, C., Hurrell, D., and Shen,
D. “
Imp
lementing Ownership Management with a
Bridge Federate
”, Proceedings of the 1998 Spring
Simulation Interoperability Workshop, Orlando
FL, March 9
-
13 1998, pp. 1126
-
1131.

[Briggs, 98]

Briggs, Keith, “
A Required RTI
Gateway As A Solution To RTI Interoperabili
ty
,”
Proceedings of the 3
rd

Simulation Interoperability
Standards Organization (SISO) Simulation
Interoperability Workshop, 97S
-
SIW
-
188, Orlando
Florida, March 1998.


[Chapin, 89]

Chapin, A. L., “Status of OSI
Standards,” Computer Communication R
e
view,
Vo
l. 19, no. 3, pp. 99
-
118, July 1989.

[Cohen, 94]

Cohen, D., “
Back to Basics
”, Proceedings of
the 11
th

Workshop on Standards for Distributed
Interactive Simulation, Orlando Florida, 1994

[Comer, 91]

Comer, D. E.,
“Internetwor
k
ing with
TCP/IP,”

Prentice
Hall, Englewood Cliffs, New
Jersey, 1991.

[Dasgupta, 99]

Dasgupta, Partha, “
Network
Operating Systems
,” Department of Computer
Science and Engineering, Arizona State University,
sponsored by New York University,
http://milan.milan.cs.nyu.edu/partha/DOS/nos
.html
.

[DoD 98a]

U. S. Department of Defense “
Draft
Standard [For] Modeling and Simulation, High
Level Architecture (HLA)


Framework and
Rules
”, U. S. Department of Defense, Document
available on
-
line at http://hla.dmso.mil, February 5
1998.

DoD (1998b).

U. S. Department of Defense “
Draft
Standard [For] Modeling and Simulation, High
Level Architecture (HLA)


Federate Interface
Specification
”, U. S. Department of Defense,
Document available on
-
line at http://hla.dmso.mil,
February 5 1998.

[Filsinger, 97]
Filsinger, J., (1997). “
HLA Security
Guard Federate
”, Proceedings of the 1997 Spring
Simulation Interoperability Workshop, Orlando
FL, March 3
-
7 1997, pp. 1015
-
1023.

[Geist, 94]

Geist, A., with Adam Beguelin, Jack
Dongarra, Weicheng Jiang, Robert Manchek,

and
Vaidy Sunderam, “
PVM: Parallel Virtual Machine
A Users' Guide and Tutorial for Networked
Parallel Computing
,” MIT Press, 1994.


[IEEE 1516]

SISO High Level Architecture
Standards Development Group, “
Draft Standard
for Modeling and Simulation (M&S) Hig
h Level
Architecture (HLA)
-

Framework and Rules
,” Draft
specification, http://www.sisostds.org/stdsdev/
Nov. 1998.

[IEEE 1516.1]

SISO High Level Architecture
Standards Development Group, “
Draft Standard
for Modeling and Simulation (M&S) High Level
Archite
cture (HLA)
-

Federate Interface
Specification
,” Draft specification,
http://www.sisostds.org/stdsdev/ Nov. 1998.

[IEEE 1516.2]

SISO High Level Architecture
Standards Development Group, “
Draft Standard
for Modeling and Simulation (M&S) High Level
Architect
ure (HLA)
-

Object Model Template
(OMT),”
Draft specification,
http://www.sisostds.org/stdsdev/ Nov. 1998.

[Fidelity SG]

Gross, David C., Ed. "
Report from the
Fidelity Implementation Study Group
,"
Proceedings of the 5
th

Simulation Interoperability
Standard
s Organization (SISO) Simulation
Interoperability Workshop, 99s
-
SIW
-
167, Orlando
Florida, March, 1999

[Krupczak, 97]

Krupczak, B., K. L. Calvert, and M.
H. Ammar, “
Increasing the portability and
reusability of protocol code
,” IEEE/ACM Trans.
Networking, Au
g. 1997, pp. 445

59.

[Krupczak, 98]

Krupczak, Bobby, with Kenneth L.
Calvert, and Mostafa H. Ammar, “
Implementing
Communication Protocols in Java
,” IEEE
Communications Magazine,

0163
-
6804/98,
October 1998

[Myjak, 97]

Myjak, Michael D., and Sean Sharp,

HLA

RTI: An Application Layer Protocol,

Proceedings of the 1
st

Simulation Interoperability
Standards Organization (SISO) Simulation
Interoperability Workshop, 97s
-
SIW
-
112, Orlando
Florida, March, 1997


[Myjak, 98]

Myjak, M., Wood, D., Carter, R., and
Petty,
M. (1998). “
A Taxonomy of Multiple
Federation Executions
”, Proceedings of the 20
th

Interservice/Industry Training, Simulation, and
Education Conference, Orlando FL, December 1
-
4
1998.


[Myjak, 98]

Myjak, Michael D., and Sean Sharp,

Java Real
-
Time RTI,
” P
roceedings of the 4
th

Simulation Interoperability Standards Organization
(SISO) Simulation Interoperability Workshop, 98f
-
SIW
-
244, Orlando Florida, October, 1998

[Myjak, 99]

Myjak, Michael D., and Sean Sharp,
with Keith Briggs, “
Javelin,
” Proceedings of th
e 5
th

Simulation Interoperability Standards Organization
(SISO) Simulation Interoperability Workshop, 99s
-
SIW
-
158, Orlando Florida, March, 1999

[RTII SG]

RTI Interoperability Study Group
Final Report, Proceedings of the 6
th

Simulation
Interoperability Stan
dards Organization (SISO)
Simulation Interoperability Workshop, 99f
-
SIW
-
1,
Orlando Florida, September, 1999

[Saltzer, 84]

Saltzer, J. H., Reed, D. P., and D. D.
Clark, “
End
-
toEnd Arguments in System Design
,”
IEEE/ACM Transactions on Computer Systems,
Vol.
2, No. 4, Nov. 1984, pp. 276
-
288.

[Stallings, 87]

Stallings, A.,
“Handbook of
Computer
-
Communications Standards, Vo
l
ume 1:
The open Systems Interconnection (OSI) Model
and OSI
-
Related Standards,”

Macmillian, New
York, 1987.

[Wood, 97]

Wood, D., Cox, A., an
d Petty, M.
(1997). “
An HLA Gateway for DIS Applications
”,
Proceedings of the 19
th

Interservice/Industry
Training, Simulation, and Education Conference,
Orlando FL, December 1
-
4 1997, pp. 511
-
520.


8.

8


A
BOUT THE
A
UTHORS

M
ICHAEL
D.

M
YJAK

Is Vice President o
f Research and
Development, co
-
founder and Chief Technical Officer of
The Virtual Workshop, Inc., where his current role is as
chief architect of
J
AVELIN

and the Java RTI. In 1982, Mr.
Myjak received two Bachelor of Science Degrees from
Clemson University
, one in Computer Science and the
other in Engineering Technology. He obtained his Master
of Science Degree in Computer Science
-

Systems from
the University of North Texas in 1988 while employed
with the Computer Science Laboratory, Corporate
Research an
d Development labs at Texas Instruments.
Prior to founding The Virtual Workshop, Mr. Myjak was
a Senior Research Scientist with the Institute for
Simulation and Training, at the University of Central
Florida. Mr. Myjak has been an active participant in
Mod
eling and Simulation standards activities for a
number of years, and has recently completed a term of
office as Chair of the Run Time Infrastructure and
Communications Forum under the Simulation
Interoperability Standards Organization (SISO). He
currently
Chairs the Run Time Infrastructure
Interoperability Study Group, and was recently re
-
elected
as Vice Chair of SISO’s Standards Activity Committee
(SAC). Mr. Myjak also Chairs the Internet Engineering
Task Force’s (IETF) Large Scale Multicast Application
(L
SMA) working group, and is active in the Web 3D
consortium's Virtual Reality Transfer Protocol Working
Group, and the Internet Research Task Force’s Reliable
Multicast Research Group.


S
EAN
S
HARP

is a graduate of the University of Central
Florida, where he

graduated Magna Cum Laude in
Computer Science. Mr. Sharp is currently the Principle
Software Engineer at The Virtual Workshop, and comes
to TVW as former research assistant at the Institute for
Simulation and Training, where he has several years of
experi
ence in simulator networking and analysis. For his
contributions to the performance analysis of the Platform
Proto Federation, Mr. Sharp received the 1997 Student
Researcher Award from IST. Mr. Sharp is working on
obtaining his Masters Degree in Computer S
cience from
UCF.