QoS Requirements of Multimedia Applications

bunchlearnedNetworking and Communications

Oct 30, 2013 (3 years and 8 months ago)

82 views


1

QoS Requirements of Multimedia Applications

Brett Berliner, Brian Clark and

Albert Hartono

Department of Computer Science and Engineering

The Ohio State University

Columbus, OH 43210

{berliner, clarkbr,
hartonoa}@cse.ohio
-
state.edu



Abstract


Quality of

Service requirements
are very important to
multimedia applicati
ons. Ensuring that these requirements
are met is ke
y to many of today’s
applications and
creating
new technologies

to ensure that stricter requirements
can
be met will help create new device
s
in the future. This is a
study on the values of the QoS Requirements for Multimedia
Applications.


Introduction


When the internet was designed, it was intended to be used
for transfer of text or other simple data types, where th
e
level of service did not matter. The only thing that mattered
was reliability.
Concepts

such as delay, jitter and packet
loss percentages did not effect the service
, the only thing
that m
attered was that the service existed
. When the
internet started
being
used for applications

such as internet
telephony, streaming live video and even remote surgery,
things like jitter and packet

loss began to matter
.
Multimedia, unlike text, has a need for service guarantees or
else the services become useless. Tryi
ng to carry on a
conversation when words arrive out of order would be quite
frustrating. Thus, the introduction of multimedia into the
internet led to the concept of quality of service.


1. Voice


With the growth of the internet and easier access to high
speed internet connections, more and more people are
turning towards computer networks to handle their long
distance voice communication instead of the traditional
telephone system. Using the internet to replace standard
telephone lines has many advantage
s. One of the biggest
advantages being that using the internet for voice
communication eliminates the concept of “long distance”.
Most companies that provide internet based voice
communication charge a monthly rate and do not charge on
a per
-
minute basi
s like traditional telephone companies.
The most common way of transmitting voice over the
internet is by Voice over Internet Protocol, or VoIP
.


1.1 Raw Data for VoIP


The process of sending human voice over a computer
network starts with a person speaki
ng into a PC
microphone. The sounds waves produced by the voice must
be translated into an electrical signal in order to be sent over
a network. This process of converting the analog signal to a
digital one is call digitization. In order to digitize hum
an
voice effectively a sample is captured 8,000 times per
second, or given a sampling rate of 8 kHz. It is standard to
use 8 bits per sample which results in a minimum data
transfer rate of 64
Kbps
. At the application layer this digital
signal is encoded

and decoded by a codec. The sampling
rates, bit rates and extra information about popu
lar codecs
Figure 2: Delay Contribution by Components [2]

Delay Component

Maximum Delay
Contribution (ms)

Comments

Packetization

30

The process of con
verting the actual digital signal into packets.

Serialization

40

This delay is only incurred when using modems.

Dejittering

30

Done to compensate for jitter introduced by networks. This assumes
a 1x dejitter buffer is used.

Presentation

17

The delay in
troduced by actually presenting the information to the
human recipient.

Figure 1:

Audio Compression Standards (Codecs)

[1]

Codec Name

Sampling Rate

(kHz)

Bit Rate (
Kbps
)

Delay Contribution

(ms)

Miscellaneous

G.711

8

64

< 1

-

GIPS Enhanced
G.711

8

Vari
able

-

Voice activity detection

G.723.1

8

5.3 and 6.3

100

-

G.728

8

16

2

Optimized for low delay

G.729

8

8

10

Voice activity detection


2

can be found in Figure 1
. Like any other calculation,
encoding and decoding voice signals takes some finite
amount of time. The delay contributions of the
various
codecs

are also presented in
Figure 1
.

Each of these
compression codecs introduces a different amount of delay.
The delay introduced comes from various sources. The
upper bounds of some of the factors contributing to this
delay are presented in
F
igure 2
[3]
.

Once the voice is encoded with a particular codec it is
transmitted over the internet using internet protocol l(IP).


Since IP is a best
-
effort ser
vice the QoS is not perfect and
some delay, loss and jitter is encountered. When the
encoded

signal reaches its intended destination it is decoded
using the same codec used by the receiver. Finally the
decoded audio signal is presented to the receiver through a
speaker.


1.2 Need for Delay Reduction


From Figures 1 and 2
, one can see that the de
lay introduced
from a codec alone can approach almost 120 ms. This
number does not include the other various delays introduced
by the network such as propagation delay, queuing delay
and transmission delay. Based on the ITU recommendation
G.114, the dela
y in a telephone call should be less than 100
-
150 ms. The reasoning behind this is a psychological
factor. If the delay is much more than this the caller will be
dissatisfied with the service. Even though a delay of 100
-
150 ms is acceptable most QoS req
uirements for VoIP ask
for 50
-
80 ms of delay or less.


1.3 Solutions


Since delay must be minim
ized to ensure satisfactory
telephone service we must employ some techniques to
reduce this delay. The first major way to speed up voice
communication is to compress the audio signal. If the sheer
size of the data being transported is reduced it will arr
ive at
the destination quicker. Some notable low bit rate
compression algorithms used are
ITU G.723.1 and G.729A.
Another way to reduce the payload of transmitting voice
over IP is to use silence suppression. Due to the fact that
during normal telephone

conversation one person talks
while the other listens, only 50% of the full duplex
connection is used at a time. Also, voice packets are not
transmitted during the silence observed in between words.
By not sending packets containing “dead air” approximat
ely
10% of the bandwidth is reduced. These two techniques
total up to a 60% reduction in bandwidth from silence
suppression.


2. Video


Video traffic is being sent more and more often in today’s
internet and will only increase in the future. Applications

such as video conferencing are becoming business standards
and many websites, CNN.com for example, offer videos on
demand. Today many homes even have digital cable
television service which transmits video information over a
network. Since video imaging
requires lots of data,
compression and reservation protocols are going to become
necessary to support the future of video in networking.


2.1 Raw Data


In order to achieve studio quality picture a video stream is
broken up into 30 frames per second. Each
of these frames
contains 525 lines. In each of these frames the y value, or
luminance, is sampled at 13.5 MHz and the two
chrominance values, u and v, are sampled at 6.75 MHz.
This total data rate comes out to (13.5 + 6.75 + 6.75) * 8 =
216
Mbps
. Due to

this extremely high bit rate, obviously
compression techniques are required to transmit video over
the internet. For different transmission lines different
compression is required. If a channel supports higher
bandwidth then less compression is needed.

Conversely, if
a channel has lower bandwidth then a higher compression
ratio is necessary to view the data. This p
oint is illustrated
in Figure 3
. It can be seen that for slower channels

sending
video, even of lower quality, is just not feasible due to
the
enormous compression ratios needed.


Video uses techniques to compress individual frames, like
JPEG does, but also uses motion prediction to compress the
data further. In fact, most of the time the bit rate required
for video transmission is dependant

solely on motion within
the images. Factors such as screen size, resolution and
scanning rates are almost irrelevant. Motion is defined in


Figure 3:

Required Compression Ratios for Package Television


[4 ]



NTS C TV

HDTV

Fi l m Qual i ty

Channe l

Bi t Rate

1 6 8 Mb/s

9 3 3 Mb/s

2 3 0 0 Mb/s

PC l oc al LAN

30 kb/s

5,600:1

31,000:1

7
6,000:1

Mode ms

56 kb/s

3,000:1

17,000:1

41,000:1

IS DN

64


NQQ kbLs

NINSSWN

SIQMMWN

NSIMMMWN

T
-
1, DSL

1.5 Mb/s

112:1

622:1

1,500:1

Ethernet

10 Mb/s

17:1

93:1

230:1

T
-
3

42 Mb/s

4:1

22:1

54:1

Fiber Optic

200 Mb/s

1:1

5:1

11:1


3

increments of 1k (1024) pixels/second. In normal television
this translates to approximately one square inch of changed
image per second. This change does
not need to be in one
contiguous block, it can be scattered througho
ut the entire
image. Figure 4
illustrates this by showing the number of
simultaneous channels various types of links can support for
different rates of motion.


2.2 Delay Introduced By Co
mpression


Every computation takes some time and
compressing/decompressing video is no exception. More
often than not, the latency introduced by this process is
much greater than the latency introduced by digitization and

digital processing in uncompress
ed format. Since most
video is very data intensive a high compression ratio is
needed. The greater the compression ratio used, the greater
the latency introduced. Typically the delay introduced by
encoding and decoding in a distribution and/or broadcast


scenario is several seconds [5].


3
.
Interactive Gaming


Recently, int
eractive multimedia
, such as network gaming,
remote visualizations, remote

surgery and tele
-
immersion,
has

become a very large part of the


still developing internet. Com
pared to vide
o and voice,
these

type
s of applications often have

QoS requirements
that are even tougher to satisfy

than video and voice

program
. This is often due to the fact that these
applications can generally not afford to lose packets or
suffer from any noticeabl
e latency, or there is a good chan
ce
the experience will be affected, if not ruined
.



Among researchers,
there is a belief that the lower bound
on the acceptable delay from interactive multimedia is 15
ms, which is the amount of time it takes for a 66Hz m
onitor
to draw a single frame. With a lower delay requirement, the
monitor could not keep up, and therefore, most of these
methods could not be implemented

[6].
Figure 5
shows the
average delay requirements for interactive multimedia in
comparison to tho
se of video and voice.



3
.
1

Definition



One type of interactive mul
timedia is interactive gaming.
Interactive gaming, in this case, refers to players on their
own machine connecting remotely to other machines to
compete in the same event against each ot
her. The device
used to connect could be a PC, a console game system or a
handheld device. Each of the devices already has most of
the game data, such as the engine and the graphics, so only
certain data needs to be sent to the central server. This data

may include character positioning and orientation, as well
as their current action, and the central server sends the
pertinent data to the connected computers for processing.


3.2

QoS Requirements


These requirements help ensure that gameplay is a smooth,
realistic experience for all users with a minimum internet
co
nnection, depending on the game. Even the inability to
meet one of th
ese requirements often will completely ruin
gamers’ experiences while playing. The QoS requirements
that most directly affect interactive gaming are
[7
]:

1.

a minimum amount of throughput

2.

an acceptable end
-
to
-
end delay

3.

low jitter

4.

low packet loss rate

5.

high d
ependability


3.3

Throughput


Throughput is a QoS requirement that varies from game to
game. Most games only require 56K dial up connections
(40 kpbs) to run smoothly. For example, two
of
today’s
most popular online games, Guild Wars and Counter
-
Strike
,
can both be played online with a 56K connection. Counter
-
Strike, for example, only needs around 16
Kbps

per
connected user to avoid slowdown

[7]
. This number can
vary greatly depending on the
genre

of game. Games w
here
players have to take turns,
such

as Massive Multiplayer
Figure 4:

Number of Television Channels for Various Averaged Motions Within the Images [4]


Average
Motion

Ver
y Slow

Slow

Normal

Fast

Pixel Change
Rate

2 kp/s

4 kp/s

8 kp/s

16 kp/s

Channel

Bit Rate

12 kb/s

24 kb/s

48 kb/s

96 kb/s

PC local LAN

30 kb/s

2.5

1

0

0

Modems

56 kb/s

4

2

1

0

ISDN

64


NQQ kbLs



S

P

N

T
-
1, DSL

1.5 Mb/s

125

62

31

15

Ethernet

10 Mb/
s

833

416

208

104

T
-
3

42 Mb/s

3500

1750

875

437

Fiber Optic

200 Mb/s

16,666

8,333

4,166

2,083


Figure 5:

Delay Requirements for Data Types

[6]

Application

Video

Voice

Intera
ctive
Multimedia

Delay (ms )

150

150+80

15


4

Online Role Playing Games (MMORPGs) like Everquest or
World of Warcraft, can allow for slower links, as the data
can update while the player is waiting their turn. As a
result, these games
often only require

a
30
-
40
Kbps link
.
Thi
s also applies to real time strategy games such as
Command and Conquer, where the player tells
their
units
what to do, and while the unit is processing,
the server can
send

receive data. These games generally hover around 20


30
Kbps
, although the newer
the game, the higher the link
speed necessary. Very new first person shooters (FPS),
such as Battlefield 1942, can be
played with 16 players on a
40 K
pbs connection. However, to take full a
dvantage of all
of the vehicles and

weapons, as well as allow all

64 possible
players at a time, each user must have a broadband
connection around 250
Kbps
[8]
. Its sequel, Battlefield 2,
needs around that level and offers no guarantees for those
with less speed. In fact, for highest performance with 64
players,

a lin
k of 2 Mbps is necessary
. The following table
shows what type of games need around how much speed.



3.4

End
-
to
-
End Delay


The most important QoS requirement for online gaming is
definitely end
-
to
-
end delay. End
-
to
-
end delay is a major
factor in what gamers call lag
, which is

bas
ically a slang
term for la
tency. Lag refers to
the delay between when a
command is issued by the player, and when it happens on
the screen. A large amount of lag can completely ruin a
gamer’s experience.

For elite performance, 50 ms

or l
ess for
end
-
to
-
end delay
is optimal. Howe
ver, this is only
absolutely necessary
for certain games and game types.
While 50 ms is a good value for very intense first person
shooters like Battlefie
ld 2
, like throughput, older first
person shooters such as Counter
-
Strike, and real
-
time
strategy gam
es and MMORPGs, can survive with a higher
delay, as they do not need as fast of a link. Specifically,
most of these games can run with a delay of 150 ms or
below

[9]
.
Figure 6
shows the acceptable am
ount of delay
across game types, while Figure 7 shows
how Counter
-
Strike is affected by end to end delay.


3.5

Jitter
and Packet Loss Rate


Jitter
will cripple an online game. Packets are timely, and
any amount of jitter that allows packets to be received late,
or even worse, out of order,
affect gameplay the same
as
packet loss. As the paper

says, even the users who suffered
a delay of 40 ms, but with a jitter of 20 ms, were affected

greatly. All of the users reported horrible gameplay during
this experiment, and the delay never was over 100 ms,
which in the game they played (Unreal Tou
rname
nt 2003),
would not be devastating
, but some of the users could not
even continue the game

[11]
. This is due to the jitter. If the
packets don’t arrive on time, then game information can be
lost. Even worse, if the packets arrive out of order, then the
packet that arrives late is useless, and the possibly important
information is lost.


In network gaming, only data that is relevant to the game is
usually sent. As a result, there can be almost no packet loss,
since all data is important. Effectively, t
h
e packet loss has
to be at 0% f
or a game to run smoothly. The only way that
a packet loss is acceptable is if the packet contains
insignificant details. However, almost no packets (if any)
contain any insignificant details. Therefore, packet loss can

be

crippling to a game session.


3.6

Interactive Gaming Dependability


Like most multimedia, network gaming generally uses UDP
for transmission, due to the fact that it is a repeated
transmission to a single source, and the transmission is
usually small. In

addition, there is no time for TCP
connection establishment

and acknowledgement
, as speed is
everything. Therefore, packets cannot be retransmitted, so
the link needs to be dependable. It also needs to always be
open. Even if only two players are again
st each other in a
Battlefield 2 session, the ability to add in 62 additional
players must be there. They cannot simply open up the line
when it’s needed, because

if there isn’t room to be opened
up, the game session will fail
.


Also, the server must be a
ble to handle a constant stream of
packets. Although each packet is very small (around 100
bytes), the packets come at a steady stream, depending on
game, varying from 30 ms to 100 ms. For most games, the
maximum amount of traffic must be assumed (that is
, every
game room is full), so
if maximum traffic is actually
transmitted
, the network

is prepared [7
]
.


3.7

Mobile Network Gaming


Mobile network gaming is played on devices such as PDAs
and cell phones. When gaming on these devices, there is
generally m
uch less data to send than PC games, due to the
processing limitations of the devices. As a result of these
limitations, technology, bandwidth and time are rarely
devoted to mobile games, meaning these games don’t
particularly work well

(if at all)
. Thi
s, however, is
beginning to change as handheld gaming systems like the
Sony PSP become popular.



Figure


6:

Throughput and Delay Across Game Types

Game Type

Basic

RTS

MM

ORPG

Basic
FPS

Intense
FPS

Throughput
(
Kbps
)

40

20
-
30

30
-
40

40
-

250

250


㈰〰

End to End
Delay (ms)

50

150
-

200

150

150

50

Figure 7:
End
-
To
-
End Delay in Counter
-
Strike [10]

< 50 ms

50
-
100 ms

100
-
150 ms

150
-
200 ms

> 200 ms

Excellent gameplay

Good
gameplay

Noticeably decreased
gameplay

Sign
ificantly effected
gameplay

Intolerable gameplay


5

The two major transport services for PDAs and cellular
phones are GPRS (General Packet Radio Service) and
UMTS (Universal Mobile Telecommunications Systems).
GPRS is a non
-
voice service that works in unison with
mobile devices to send their data. GRPS is referred to as

“always connected”, since it can perform an almost instant
transmission. Theoretically, GRPS can send data at a rate
of 171.2
Kbps
, but in re
ality, to reach this, an operator must
grant them all of the bandwidth, referred to as timeslots.
Since operators will rarely, if ever, do this, in reality, the
bandwidth is

less than 1/8
th

of this at many times, and only
1/4
th

of it at best [12].

UMTS i
s a

type of
mobile
transmission that

relies on radio spectrum transmission.
Again, theoretically, UMTS can transmit anywhere from
384
Kbps

to 2
Mbps
, but again, this is only if the space is
reserved

[13]
. A study done in Germany
on

a mobile
volleyball ga
me shows the problem with
UMTS and GRPS
that doesn’t befall console and PC gaming


mobile
networks are just not set up for over
-
provisioning that is
necessary to meet gaming QoS requirements. For instance,
the simple volleyball game had very little data
to send
(approximately 20 states per minute, which each state
containing very little data), but the game was very tough to
play, due to the fact that the delay was always between 100
a
nd 200 ms [14
]
.


Although the Sony PSP contains much better graphics th
an
a PDA or a mobile phone, th
e reason is that it contains it
s
own wireless card that conforms to the IEEE 802.11b
standards

[15]
. The devices only communicate with a
wireless router, wh
ich already has the setup for ensuring
QoS requirements are met

(as t
hey would be for laptops or
PCs connected to the wireless router)
. As a result, the PSPs
are able to be played around a wireless router. True mobile
gaming is not effective yet.


4
. Remote Visualizations


Remote visualizations are
another
type of multime
dia
becoming a reality as the internet continues to grow.
Basically, a remote user connects to a data set that is either
generated or has previously been generated. The key is that
the data must be interactive


for example, the user
can
often do things
like rotate the data, zoom in, and even add
in slice planes to see inside the data. Right now, however,
there is just too much in the way of performing consistent
remote visualizations.


4
.1
QoS Requirements


To perform consistent, timely visualization tr
ansmission, a
few QoS requirements need to be met.

If these
requirements cannot be met, the user will almost certainly
be unable to have the patience to get their desired results.
Since the size of the raw data is often in the gigabyte range,
it is clear

that remote visualizations require great precision.
Thus, users need to be able to single out important parts of
the
data. This can often take many actions, and if a user
must wait for every
action to perform, they will not want to
use the system.

The
QoS requirements that most directly
affect remote visualizations are:


1. low delay

2
. high throughput/bandwidth

3.
low latency


4
.2 Delay


Generally, t
he way remote visualizations work is simple


a
computer collects the input, storing it as raw data. Th
at data
is then turned into triangles,
which are translated to an
image, and then only the image is sent to the user, who then
views the image or executes a command, modifying the
data and forcing the system to take a new snapshot.
Therefore, most of the
data is handled at the side of the
computer that is given
the raw data. [16
]


For the calculations, the data gathering computer first has to
receive the raw data. Often, this data can approach size of
gigabytes or terabytes, but due to visualization algor
ithms,
the size can often be dropped down to around 100 MB. The
size of the raw data is listed here on as ‘n’. Next, the
number of triangles per frame in the image can be written as
K, where K is often 500,000 (but can range from 50,000 to
1,000,000). W
ith n and K, the delay to generate the
triangles from raw data is
on the order of O(log(n) + K),
with the actual time depending on the CPU
[16]
.


With time determined, next is size. The siz
e of the triangles
in bytes is
3 (
dimensions
) * 3 (points) * 4 (flo
at number) * 2
(shading) + 12 (color) = 86 bytes. So with 500,000
triangles, there will be 43 MB of data. Then, to make them
an image, the data must be copied to the graphic card, so
that will take around 43 M * the copying speed, and then
an
average gra
phic card can take around 100M triangles per
second, so the time to process one frame is 500,000 / 100M,
or 5 ms

[
16
]
. Then, the only delay left is how long it takes
to get to the recipient
, which
must as

small as

possible,
because otherwise, the phenomen
on that nothing is
happening can occur. If a user goes to zoom in on a data
set, and the processing takes 30 seconds, it will be a highly
frustrating process. However, with the large amount of data
to be dealt with at the original computer, this is part
of the
reason remote visualizations will not work.

Figure 8 shows
a simplified version of how the data is collected and
transferred.


Figure 8
: Remote Visualization Data Process



6


4.
3

Throughput/Bandwidth


This is a big

factor that keeps remote visualiza
tion from
being possible.
The transmission speed is the last key.
Obviously, a high speed link is needed at the end, because
the transmission both ways must be as quick as possible, so
that the only delay is at the computer gathering data.


When performi
ng remote visualizations that do NOT work
like the above one, usually, a link speed about around 700
Mbps

is necessary. This is to make sure the data (which
would be the 100M) is transmitted in a timely fashion. The
more observers of the data the more ti
me is needed. If that
compression is put into motion, then a link of only 100


200
Mbps

is necessary
. Again, the more observers, the
more the overall link will need (so 2 users will be around
200


400
Mbps

from the central server, and so on). In
addit
ion,
if stereoscopic rendering (that is, creating the
illusion of depth in the image), the amount of bandwidth
needed is doubled [17].


4.4 Latency


Although QoS data on exact requirements for latency are
difficult to track down, a simple experiment using
a
visualization kit (VTK, in this case), can demonstrate the
effect of latency on distance visualizations [26]. On VTK’s
website, they offer many sample programs in C++ that
use

to perform visualizations. Although these occur solely on
the user’s compute
r, the “sleep” command from
<windows.h> can be used to simulate
remote visualization
delay
.
Since the command
sleep(50)
tells the processor to
sleep for 50 ms before
continuing on to the next program.
Inserting a
sleep(X)
command wherever a command appea
rs
in the VTK program will cause the program to pause for X
ms before it continues. Since every time a command is
issued, the computer waits a certain amount of time to begin
it, this is a very effective latency simulation.


A simple, unscientific test
re
veals that around

150 ms sleep
time starts to noticeably affect the program.
It is not a deal
breaker, but it is frustrating at times. At between 225 and
250 ms, the latency becomes impossible to work with.
Even a simple task such as rotating the data t
o see a
different side of the visualization becomes arduous. Both of
the users who attempted to experiment with the remote data
with a delay of 250 ms became too frustrated before
finishing any tests with slice planes. As a result, even
though these numb
ers are unscientific, it is clear what kind
of effect latency has on remote visualizations.


5
. Telesurgery


Telesurgery
-

surgery performed at a certain distance
-

is
one aspect of telemedicine. The introduction of robotic and
computer technology into sur
gical operations allows
dexterity to be increased and surgical procedures to be
carried out from a certain distance. Through communication
lines, digitized information can be transmitted to remote
locations, enabling surgeons to operate on patients located

distantly. Challenges to this concept are numerous, but the
most essential limitations have been the dependability or
quality of service of the communication lines and the issue
of latency, which is the delay time from when the hand
motion is initiated by

the surgeon until the remote
manipulator actually moves, and the image is s
hown on the
surgeon’s monitor [20
].


5
.1 QoS Requirements


Even though there is little practical experience of
telesurgery at present, it is clear that successful telesurgery
will

require a data transfer of robot commands, video and
voice signals, text, computer data, as well as stored and
real
-
time medical images. A list of provisional network
requirements for telesurgery can therefore be identified.
These QoS requirements include

[1
8
]:

1.

reliability

2.

an acceptable end
-
to
-
end delay

3.

multiplexing of various data rates

4.

low data error rate (BER
-
bit error rate)

Other requirements and desirable features are likely to
emerge as further telesurgery trials are conducted and more
experience is

gained.


5
.2
Reliability and Error Rate


In telesurgery, since human lives are at stake, the
consequences of an error in transmission could be very
serious, and therefore reliable techniques of networking
must be acquired.


Since the data rate associated

with robot commands is very
low (typically 19.2
Kbps
), there is therefore ample scope to
protect each message with error
-
protection coding. Each
time the operator issues a command, the transmitting
equipment can send it more than once to the receiving end
.
The receiving end can then echo the command back to the
sending end. Only when the command is received and
echoed correctly, say three times in succession, would the
command be executed at the receiving end [1
8
].


One apparent threat to safe telesurgery
would be a power
cut occurring in the network. Opportunely,
telecommunications network operators employ battery
arrays and ‘hot standby’ generators to take over the task of
powering the network in the event of a mains power failure.
A power cut on the netw
ork must be virtually unnoticed by
the user [1
8
].


5
.3 Time
-
Delay


There is a major constraint that could lead to disastrous
results during surgery, namely time delay. The surgeon
therefore views his or her movements on the computer

7

interface as they are
happening. If the surgical system were
removed to a more distant site, however, it would introduce
a time delay. Visualization of the operating field could be
milliseconds or even seconds behind the real
-
time
manipulations of the surgeon. Studies showed th
at the
acceptable limit of time delay in terms of a surgeon’s
perception

of safety was roughly 330 ms [19
]; satellite
transmission, for example, would introduce a delay of more
than 600 ms.


On September 7th, 2002, the world’s first human long
-
distance ope
ration was performed between New York, USA
and Strasbourg, France (14,000 km distance), demonstrating
the feasibility and safety of performing a complete surgical
operation from remote locations. The two sites were
connected through a high
-
speed terrestria
l optical
-
fiber
network that transports data through dedicated connections
using Asynchronous Transfer Mode (ATM) technology. A
bandwidth of 10
Mbps

has been reserved through a network
that interconnects applications at both sites using a network
terminati
on unit (NTU), which provides a multiservice path
to different applicati
ons [19
].


By monitoring both NTU units at the two ends, the number
of lost packets and the communication quality were
measured. It was revealed that no ATM packet was lost
during any

surgical procedure. The round
-
trip delay by
ATM transport was 78 − 80 ms. Adding 70 ms for video
coding and decoding, plus a few milliseconds for rate
adaptation and Ethernet
-
to
-
ATM packet conversion,
movements executed by the surgeon in New York were
app
arent with
in 155 ms on his video screen [19
].


Another technology which impacts the time
-
delay
requirement of telesurgery is video compression algorithms
(codecs). It is important that new codecs produce video that
is of higher quality, low latency (< 100

ms), and degrades
gracefully.


5
.4 Simultaneous Transfer of Various Types of
Data


The various transferred data in telesurgery consists of video,
voice, images, robot commands, text, and computer data.
These data are originated from different equipments a
nd
have different data rates. Therefore, it is crucial that the
network must have the ability to simultaneously transfer
data from sources with widely differing data rates. One
networking technology that is ideal for telesurgery is, for
example, ATM, due t
o its special ability to multiplex
sources with different data rates and its low cell loss rate
[1
8
].


6
. Tele
-
Immersion


Tele
-
immersion has since entered the NGIand Internet2
vocabulary. In the applications section of the Computing
Research Association’s
“Research Challenges for the NGI,”
tele
-
immersion
was one of five key technologies identified
as necessary
for the future use of the NGI [25
]:


Tele
-
immersion.
Tele
-
immersion will enable users in
different locations to collaborate in a shared, virtual, or
simulated environment as if they are in the same room. It is
the ultimate synthesis of networking and media technologies
to enhance collaborative environments. Tele
-
Immersive
applications must combine audio, video, virtual worlds,
simulations, and many oth
er complex technologies. They
will require huge bandwidth, very fast responses, and
guarantees of delivery.


6
.1 QoS Requirements


In general, the QoS requirements for tele
-
immersion include
the following four important factors:

1.

extremely high network band
width

2.

low latency

3.

constant jitter

4.

guarantees of delivery


6
.2
Challenges of Tele
-
Immersion


Tele
-
immersion has emerged as a high
-
end driver for the
Quality of Service (QoS), bandwidth, and reservation
efforts envisioned by the NGI and Internet2 leadership.

From a networking perspective, tele
-
immersion is a very
challenging t
echnology for several reasons [24
]:



The networks must be in place and tuned to
support high
-
bandwidth applications



Low latency, needed for 2
-
way collaboration, is
hard to specify and gua
rantee given current
middleware



The speed of light in fiber itself is a limiting factor
over transcontinental and transoceanic distances



Multicast, unicast, reliable and unreliable data
transmissions (called “flows”) need to be provided
for and managed by
the networks and the operating
systems of supercomputer
-
class workstations



Real
-
time considerations for video and audio
reconstruction (“streaming”) are critical to
achieving the feel of telepresence, whether
synchronous or recorded and played back



The com
puters, too, are bandwidth limited with
regard to handling very large data for collaboration



Simulation and data mining are open
-
ended in
computational and bandwidth needs

there will
never be quite enough computing and bits/second
to fully analyze, and sim
ulate reality for scientific
purposes


6
.3 Lag


Lag is the term used to describe the perceived sum of all the
sources of latency in a system. Typically, it is thought of as
the delay between action in the real world and the perceived
response of the system

to that action. Lag is the critical

8

issue for usability; reducing lag is a major technical
challenge. Communications latency is only one component
of tele
-
immersion lag. Effective solutions to reducing lag
must attack the component sources of latency at a
ll levels of
the system. Sources of latency in the communications
system are transmission latency, bandwidth or transfer
latency, switching or routing latency, contention, and
protocol latency.


Most users have difficulty manipulating objects
in VR once
la
g exceeds 200 ms [24
]. When the virtual display is
coupled with the real world, as in tele
-
robotics, this limit is
approximately 30 ms. Non
-
network components of the VR
system often together exceed 200


300 ms, so there is
actually very little room for wi
de
-
area communications
delay in the lag budget.


6
.4 Jitter


Jitter in the network will more greatly impact collaborat
ive
coordination than latency [23
]. Higher latencies with low
jitter will still allow collaborators to make reasonable
predictions of how
an environment will behave (albeit
overall task performance will decline.) However high jitter
reduces predictability and hence collaborators are forced to
employ a purely sequential interaction strategy


6
.5 Tele
-
Immersion Flow Types


Progress in all thes
e areas, however, is expected; tele
-
immersion serves as an integrating technology as pieces of
the solution are contributed by the community and
computer/networking industry. The following table,
developed in discussions with Rick Stevens, director of the
Math and Computer Science Division at Argonne National
Lab, gives our current best
estimations
and
opinions
of the
attributes of the nine types flows simultaneously needed for
an n
-
way compute and data
-
intensive audio, video, and
haptic (
touch) tele
-
immers
ive session [24
].


Each row indicates data flow types:



Control information
:

data u
sed to manage the

session, to authenticate users or processes, to
launch processes, to control the display or tracking
systems, and to communicate out of band between
the wo
rld servers and VR systems.



Text

provides simple communications capability
within sessions for simple note taking and passing.



Audio

gives ambient auditory cues, allows voice
communications among users, and is used to issue
commands via voice recognition
and speech
synthesis.



Video

can allow teleconferencing or remote
monitoring displayed within the virtual world.



Tracking

is achieved with location and orientation
sensors, and captures the position and orien
tation
of the user. Typically this data is streamed to the
computer responsible for computing the
perspective of the scene. Tele
-
immersion requires
tracking data to be shared among sites.



Database

is the heart of a tele
-
immersion
application world. The da
tabase contains the
graphical models of virtual scenes, objects, and
data, and since the database is used to provide the
models that are rendered, it must be maintained in
a coherent state across multiple sites. Databases
might be as simple as shared VRML
files or as
complex as multi
-
terabyte scientific datasets, VR
extensions of video serving.



Simulation

provides the basis for dynamic
behaviors, like responding to the users’ actions.
Small
-
scale simulations often run on the computer
also generating the VR
experience, but frequently
the simulation will need a dedicated
supercomputer. User input is captured and
transmitted to the simulation via the network and
the simulation will generate an update, which is
then propagated to each user site for local
renderi
ng. Typically the data transferred to the
simulation is considerably smaller than the data
returned by the simulation.



Haptics

include force and touch sensing/feedback
devices and use a variety of sensors and actuators
that are “attached” to the hands and
/or legs of
users. Some systems now generate haptic “images”
that augment or replace visual images Haptics are
particularly sensitive to latency and jitter
(instantaneous variations in latency).



Rendering

is the transformation of geometric
information int
o images for display. All VR
environments primarily render graphics locally. As
networks provide bandwidth adequate for
compressed HDTV, however, it will become
reasonable and efficient for scenes to be rendered
remotely and transmitted to each site in rea
l time.


The each flow
-
type attribute is explained in the following
list:



Latency

is the sum of all delays in the system, from
the speed of light in fiber, to operating system
Figure 9
: Tele
-
Immersion Data Flow Types

Type

Latency

Bandwidth

Reliable

Multicast

Security

Streaming

DynQoS

Control

< 30 ms

64Kb/s

Yes

No

H
igh

No

Low

Text

< 100 ms

64Kb/s

Yes

No

Medium

No

Low

Audio

< 30 ms

Nx128Kb/s

No

Yes

Medium

Yes

Medium

Video

< 100 ms

Nx5Mb/s

No

Yes

Low

Yes

Medium

Tracking

< 10 ms

Nx128Kb/s

No

Yes

Low

Yes

Medium

Database

< 100 ms

> 1GB/s

Yes

Maybe

Medium

No

High

Sim
ulation

< 30 ms

> 1GB/s

Mixed

Maybe

Medium

Maybe

High

Haptic

< 10 ms

> 1 Mb/s

Mixed

Maybe

Low

Maybe

High

Rendering

< 30 ms

>1GB/s

No

Maybe

Low

Maybe

Medium



9

overhead, to tracker settling time and screen
refresh



Bandwidth

is the bits/seco
nd the system can
transmit



Reliable

flows are verified and retransmitted if bad



Multicast

flows go to more than one site at once



Security

involves encryption overhead that may or
may not be warranted or legal



Streaming

data is a constant flow of informatio
n
over time, as with video, audio and tracking



Dynamic QoS

can provide ways to service bursty
high
-
bandwidth needs on request


Conclusion


As the internet has evolved it has certainly improved the
quality of service. What was once only able to support
sim
ple text transfer is now capable of allowing surgery to
be performed from 3,000 miles away. Simple multimedia
applications such as voice traffic and streaming video are
very easy to do with today’s technologies and applications
that require precise data t
ransfers with large payloads,
which were once inconceivable, are starting to become
commonplace. Technology will spawn more and more
complex applications, each requiring finer and more
guaranteed quality of service. In order to support these
future techn
ologies more effective means of ensuring
quality of service are going to need to be developed as well.


REFERENCES


[
1] Ballard, Buzz, Blake, Leslie, Macik, Keith,
Montemayor, Ramiro, “Sip Phone Codec Testing Project”,
May 2004; http://voip
-
itec.tamu.edu/
nit/Final%20Presentation.pdf


[2] Voiceage: The World’s Premier Supplier of Speech and
Audio Codecs;
http://www.voiceage.com/g729.php


[3] Birin, Gil, “Voice over Frame Relay, IP and ATM”;
http://www.protocols.com/papers/voe.htm


[4] “Image and Video Compr
ession Techniques”;
http://www.autosophy.com/videcomp.htm


[5] “DTV Latency”;
http://www.tvtechnology.com/features/Tech
-
Corner/f
-
rh
-
latency.shtml


[6] Xuan, Dong.

Sept 2005
. “Delays

in the Internet”


[7] “Aperto Networks Solutions: Internet Gaming”;
http:
//apertonet.com/en/solutions/solutions/gaming.shtml


[8] “Amazon.com: System Requirements: Battlefield 2”;

http://www.amazon.com/exec/obidos/tg/stores/detail/
-
/videogames/B00009V3NW/tech
-
data/002
-
0196654
-
5085624


[9] Borg,
Seth, Girard, Eric, Sheldon, Nath
an, “The Effect
of Latency on Performance of Warcraft III”;
http://web.cs.wpi.edu/~claypool/mqp/war3/mqp.pdf


[10] Almasbakk, Hans, Brekne, Tonnes, Overby, Harald,
“Online Gaming: An Overview”;
http://www.q2s.ntnu.no/q2sfc/uploads/online_gaming.pdf


[11]

D
egrande, Natalie, De Vleeschauwer, Danny, Lamotte,
Wim, Monsieurs, Patrick, Quax, Peter, “
Objective and
Subjective Evaluation of the Influence of Small Amounts of
Delay and Jitter on a Recent First Person Shooter Game”;
http://www.sigcomm.org/sigcomm2004/w
orkshop_papers/n
et608
-
quax.pdf


[12]
“What is General Packet Radio Service?”;
http://www.gsmworld.com/technology/gprs/intro.shtml


[13] “What is UMTS?”;
http://www.umts
-
forum.org/servlet/dycon/ztumts/umts/Live/en/umts/What+is
+UMTS_index


[14] Busse, Marcel
,
Effelsberg, Wolfgang, Lamparter,
Bernd, Mauve, Martin,

“Lightweight QoS Support for
Network Mobile Gaming”;
http://www.coe.montana.edu/ee/rwolff/Games%20Project/li
terature%20search/QoS%20suuport%20
-
%20mobile%20games.pdf


[15]
“The PSP FAQ”;
http://psp.ig
n.com/articles/513/513175p1.ht ml


[16] Xuan, Dong. Nov. 2005. ”Preliminary Data for
Remote Visualization”.


[17] “ESLEA: HPC
-

Further Information”;
http://www.eslea.uklight.ac.uk/sp_hpc_further_info.html


[
1
8
]
Smithwick, M. 1995. "Network Options for Wid
e
-
Area
Telesurgery."
Journal of Telemedicine and Telecare

1(3):131
-
38.


[19
]
J Marescaux, J Leroy, M Gagner, F Rubino, D Mutter,
M Vix, S E Butner and M K Smith, “Transatlantic Robot
-
Assisted Telesurgery”,
Nature
, 413 (6,854) (2001), pp.
379

80.

[20]

Mares
caux J, Rubino F. Telesurgery, “Trends
Including Robot Assisted Technology”,
Business Briefing:
Global Surgery
, October, 2003.

[21
]
Butner S. E., Ghodoussi M., “Transforming a Surgical
Robot for Human Telesurgery”,
IEEE Transactions on
Robotics and Automat
ion
, Oct. 2003.

[22
]
Cai Meng, Tianmiao Wang, Wusheng Chou, Sheng
Luan, Yuru Zhang, Zengmin Tian, “
Remote surgery case:

1
0

robot
-
assisted teleneurosurgery”,
Robotics and Automation,
2004. Proceedings. ICRA '04. 2004 IEEE International
Conference on
, Volume: 1
,


On page(s): 819
-

823 Vol.1.

[23
]
Jason Leigh, Oliver Yu, Dan Schonfeld, Rashid Ansari,
Eric He, Atul Nayak, Jinghua Ge, Naveen Krishnapasad,
Kyoung Park, Yong
-
joo Cho, Liujia Hu, Ray Fang, Alan
Verlo, Linda Winkler, Thomas DeFanti, “Adaptive
Networking
for Tele
-
Immersion,”
Proc. of the 5th
Immersive Projection Technology/ 7th Eurographics Virtual
Environments Conference (IPT/EGVE)
, May 16
-
18, 2001,
Stuttgart, Germany, pp.199
-
208.


[24
]
J. Leigh, Johnson, A., DeFanti, T., et al., "A Review of
Tele
-
Immersi
ve Applications in the CAVE Research
Network," presented at
IEEE VR99
, Houston, Texas, 1999.

Tom DeFanti, Dan Sandin, Maxine Brown, Dave Pape,
Josephine Anstey, Mike Bogucki, Greg Dawe, Andy
Johnson, Tom Huang, “
Technologies for Virtual
Reality/Tele
-
Immers
ion Applications: Issues of Research in
Image Display and Global Networking”
,

European
Commission/National Science Foundation Advanced
Research Workshop on "Human
-
Centered Computing,
Online Communities, and Virtual Environments"
, Judy
Brown, Andy van Dam,
Rae Earnshaw, Jose Encarnacao,
Richard Guedj, Jennifer Preece, Ben Shneiderman, John
Vance, eds., Chateau de Bonas, France, June 1
-
4, 1999.


[
25
]
J.E. Smith and F.W. Weingarten (eds.), “Research
Challenges for the Next Generation Internet”,
Computing
Research Association
, 1997, p. 20.


[26] “VTK: Visualization Toolkit”;

www.vtk.org