39912116 - Telecommunications Industry Association

existencetubNetworking and Communications

Oct 26, 2013 (4 years and 7 months ago)


Telecommunications Industry Association



Clearwater FL 01Dec 1999


Technical Committee TR
30 Meetings


Satchell Evaluations


Stephen Satchell


(775) 832


(775) 831




Adapting TSB 38 to TCP/IP Transport




Members of TR
30.3 and meeting attendees


Copyright Statement


The contributor

grants a free, irrevocable license to the Telecommunications Industry Association
(TIA) to incorporate text contained in this contribution and any modifications thereof in the creation
of a TIA Standards publication; to copyright in the TIA's name any TIA

Standards publication even
though it may include portions of this contribution; and at the TIA's sole discretion to permit others
to reproduce in whole or in part the resulting TIA Standards publication.

Intellectual Property Statement

The individual prep
aring this contribution does not know of patents, the use of which may be
essential to a Standard resulting in whole or in part from this contribution

Prior Publication

This paper has been published on the Web since October 24, 1999. Holders of this p
aper are
hereby given license to reproduce this paper for use within their company only. Anyone wishing to
republish or distribute copies of this paper outside their own company should contact the author
for licensing arrangements.


This contribut
ion provides, in contribution form, the rationale and proposal that the author, while
editor of PN 3509, included into the draft Standard for consideration by the Committee. Much of
the work included in this paper is new material, influenced by the author
’s work on the manuscript
for the book
Linux IP Commentary

to be published by CoriolisOpen Press as part of their series
on the Linux Operating System. Indeed, it is possible that some additional material will be
presented during the meeting as the autho
r finishes the chapter on the TCP protocol just before
the meeting.

This paper takes the standard modem test procedure standard TIA TSB 38 and identifies
corresponding procedures to implement testing in the TCP/IP domain. It proposes a checklist to
be incl
uded in any proposed standard. Finally, it attempts to predict the difference in
measurement between the standard TSB 38 method and a TCP/IP based method.

This paper also suggests some implications of using TSB 38 procedures with existing V.34
modems, whe
n those modems are used with Ethernet

connected router access servers.


The Telecommunications Industry Association (TIA) Telecommunications Systems Bulletin (TSB)
38 describes standard testing procedures for analog modems. These techniques, d
since 1950 and passed on as "stories around the campfire" until codified in 1991 by TIA Technical
Subcommittee TR
30.3, are used to measure the performance of a modem. The techniques
utilize raw binary datastreams in their procedures, with no prot
ocol whatsoever. This paper
describes a proposal for adapting these procedures to a protocol environment; for analog
modems, this means encapsulating TCP/IP within a PPP link.

The selection of TCP/IP as the target transport system was not made lightly. Fi
rst, the major use
of modems today is Internet access via the Public Switched Telephone Network (PSTN). Indeed,
the ITU Recommendation V.90 modems were designed specifically to support this application. Of
the many methods used to carry data, by far the mo
st popular method is to use Point to Point
Protocol (PPP) to encapsulate TCP/IP packet traffic. TCP/IP is also used for broadband access,
services such as cable data modem and Digital Subscriber Line (DSL) service, so from a testing
standpoint developing a

procedure to use TCP/IP means that it can be applied to system testing
in a wide variety of environments.

Many of the discussions directed toward PPP transport of TCP/IP packet streams also apply to
the SLIP and SluRP methods of encapsulating TCP/IP pack
ets onto asynchronous connections.

The goal of this paper is to make the translation of TIA TSB 38 procedures from the datastream
environment to the TCP/IP packet environment as faithful as possible, so that results from either
style of testing can be comp
ared side by side. The language of this paper will be in TCP/IP terms.

TSB 38 Test Procedure Overview

In TIA TSB 38, a number of test procedures are defined. These procedures are designed to be
used in conjunction with a connection network model that is
appropriate for the communications

system being tested. Standard two
wire modems utilizing non
PCM modulation methods (such as
V.32 bis and V.34) are tested using these procedures in conjunction with the TIA 3700 Standard
network model; V.90 modems use the

network model defined in TIA Project Number (PN) 3857.
Cellular testing uses the same procedures over a network model defined for that environment.
Finally, other media can define network models over which these procedures, suitably modified,
can be run.

The five standard tests and the one new test defined for modems are:

1. Throughput versus network model coverage (NMC),

2. Throughput versus file type (TFT),

3. Character echo delay (CED),

4. Block acknowledgement delay (BAK), and

5. Call Connect rel
iability (CCR).

6. Throughput over a sustained circuit (TSC, in PN 3509)

(N.B.: The other three tests defined are the Bit error ratio measurement, the Block error ratio
measurement, and the Message error ratio measurement. These tests are specific to mea
performance of the data link hardware, and require special access to perform.)

When you look at the actual procedures, you see that the tests break down into the following
types of tests:

Data throughput measurement

Data delay measurement

Data c
onnection reliability measurement

First, we'll look at the TSB 38 definition for each of these tests, and then discuss how to
implement parallel versions of the test in our TCP/IP
based test suites.

Data Throughput Measurements

TSB 38

In TSB 38, data t
hroughput is performed by establishing a connection, transmitting data over the
channel until a specific stop condition is obtained, receiving the last character transmitted, then
reporting the throughput in character seconds defining the time interval as
the difference in time
between receiving the last character and transmitting the first character. This method of
calculating the throughput therefore includes the effect of propagation delay in the channel,
processing delays in the modem units as well as i
ntermediate channel processing delays, and
proessing delays in the DTEs themselves. During the transfer, TSB 38 mandates that

start immediately upon connection establishment

continue without any time delays (no idle time, not even a single
extra stop bit)

be error free; that is, the test is rendered invalid if incorrect data is received.

The intent of these requirements is that the modem would hit the ground running, which is needs
to be able to do for a number of applications. "Connection

establishment" is defined as that instant
in time when both modems signal their ability to accept data from their respective data terminal
equipment (DTE); for modems, that's a "CONNECT" message sent from the modem to the DTE,
a carrier
detect signal asse
rted, and a clear
send signal asserted on both modems.

Via TCP and IP

Adapting the TSB 38 procedure to operate over PPP, IP, and TCP requires careful attention to the
details of establishing a modem connection, PPP connection, and TCP connection. Furt
considerable care is needed to ensure that the best use is made of the TCP connection when
transferring data. Network topology and collateral interference must be carefully crafted and
controled to avoid astonishing side effects. Finally, the accuracy

of the measurement requires that
the condition be stated exactly, and in such a form that it can be implemented.


Described in RFC 1661 .

Establishing a PPP connection using a modem involves yet one more handshake sequence in the
sequence of protoco
ls. The PPP dialer places the call, and upon detection of the establishment of
the underlying modem connection the PPP system then sends out "bid" packets in order to begin
the session. The initial handshake sets up the options to be used on the channel, a
nd initializes
the channel for use. Once the PPP protocol implementations are satisfied that everything is set
up, they indicate their willingness to carry traffic to the upper protocol (IP in this case).

PPP is a frame

oriented protocol, and uses a speci
al character as a frame character. This means
that if the frame character or the escape character appears as data in the PPP frame, the single
character is transmitted as a two

character sequence. By convention, the ASCII control
characters are also escape
d so that the underlying asynchronous channel will not see any control
characters. This behaviour may be modified during the initialization handshake to limit the
characters that are required to be escaped. These escape characters add to the fixed seven
aracter overhead per frame; each frame can hold up to 1,500 payload characters. This
maximum transmission unit

can be modified by prearrangement, although the frame check
sequence will not hold up with longer transmission units.

Most of the time, the PPP l
inks utilize the ITU V.42 error control protocol (and V.42 bis
compression protocol) to reduce the overhead introduced by PPP by allowing larger packets at
this level. The in

modem error control is more robust than that provided by PPP, and can
maintain a
higher throughput. The in

modem compression does a much better job than the Stac

based compression defined for PPP, for those connections in which compression should be
enabled. The total capacity of the PPP

enabled channel, assuming all these capabilities

are in
place, is within on average three percent of the carrying capacity of the raw modem channel.

In real

world situations, there is an authentication phase to PPP. Some PPP servers use a
standard LOGIN/PASSWORD sequence outside of protocol to authenti
cate a session. The PPP
protocol has several authentication methods defined for it:

PPP Challenge Handshake Authentication Protocol (CHAP) (RFC 1994)

Password Authentication Protocol (PAP) (RFC 1661)

Other authentication protocols may be defined in the

There is a startup time penalty when any of these forms of authentication is enabled. This is
because by definition the authentication must be completed before any user data may be


The Internet Protocol (IP) imposes a 20

byte hea
der overhead for each packet. In a testing
situation, there is no reason to include any IP

level options in the header packet, so the header
can be considered to be a fixed size. Van Jacobson compression (described in the section on
TCP) may be used with P
PP (or with CSLIP) to reduce this overhead load when performing
throughput measurements; the effective IP header size, when there is only one datastream over
the PPP link, is on average ten bytes.

The conversion between packets that are arbitrarily large,

up to 64 kilobytes, and packets that can
fit through the transmission path (usually limited to 1500 or 576 bytes) is performed at this layer.
Any TCP packet that's too large will be fragmented, or broken into multiple IP packets. The packet
fragments are
transmitted through the network, collected at the receiving system, and then
reassembled. The risk here is that fragments may not be transmitted in the original order (indeed,
there are implementations that favor transmitting the last fragment first, then
the fragments in
reverse sequence until the first fragment is transmitted) and so delivery of the data to the receiver
has to wait until all the fragments have been collected in the receiving system. Fragmentation can
take place not only in the transmittin
g system but in any intermediate router, when the maximum
transmission unit (MTU) is less than the size of the packet to be transmitted.

The advantage of having IP fragment the packets is that there would be only one higher
protocol header (such as
the TCP header). The disadvantage is that the logic of the testing
system has to take into account the behavior of fragmented packets, and know the fact that
virtually every implementation will not deliver data until packet defragmentation is complete.

other interesting variable is the routing algorithm used by IP's network layer implementation.
Given a pair of routes with equal "cost”, a given IP routing implementation may select one path
over another, or allocate data evenly across both paths. This can

exchange fragments

or even
IP packets if they are small enough

so that the data appears to the receiving application in


For throughput testing, nothing beats Transmission Control Protocol (TCP) for carriage of the
data. This didn't used to b
e true; at one point in TCP/IP's history, UDP packet traffic was faster
and therefore used in any "shoot

outs" among products.

TCP adds its own 20

byte header to the collection. Coupled with IP and PPP, that means that
there are 47 bytes of overhead for e
ach and every packet transmitted over a modem channel. Or
not. One way to reduce the overall overhead is to use Van Jacobson header compression (RFC
1144, adapted to PPP in RFC 1548 sections 6.6 and 6.7), a method that reduces or eliminates the
redundant o
r predictable contents of IP headers from the PPP datastream and reintroduces those
items at the far end of the data transfer. All modern implementations of modem PPP support this
option. Van Jacobson compression works only with TCP streams; any UDP stream
s are
transmitted in their original form.

If a testbed is implemented with custom programming that utilizes a bare TCP connection, there
are no surprises. If you try to conduct testing with any of the well

defined applications, you will find
some interest
ing quirks. TELNET has to escape binary data, or drop the binary data on the floor,
depending on how it's configured. FTP is better in this regard. Both of these popular applications,
though, make use of the URGENT flag in TCP to send control data, a proce
ss that causes a
synchronization event as the URGENT flag indicates a change in operating mode. In addition, at

the beginning of a TELNET or FTP session there is a control exchange in which a number of
parameters are exchanged. Determining when this sequen
ce ends and actual data transfer starts
is difficult at best.

TCP itself has some curves to throw, as well. Remember that TCP was designed to "play nice" in
a sea of other systems; all of these features can affect the repeatability of tests:

Path MTU Dis

The maximum size of a packet that can traverse a network link varies from
a low of 128 bytes to a maximum of 65,535 bytes. Ethernet, for example, has a MTU payload of
1500 bytes because its 32

bit CRC can't handle larger packets given Ethernet's in
herent soft

error rate. That 128

byte MTU is for a communications channel with a 12

bit CRC

those are few and far between now.

Some implementations of TCP implement a procedure called MTU Path Discovery to probe the
channel and determine the
maximum transmission unit that can travel through the channel without
fragmentation. This is done by using the Don't

fragment flag in the IP packet, and watching for
ICMP messages that say that someone wanted to fragment the packet but couldn't. The sendin
system then adjusts downward the MTU until a packet gets through.

For throughput testing, this would increase the overhead for TCP packets, not to mention blocking
data for the first few seconds as the transmitting system tries to determine the MTU of t
he path.

Slow Start Algorithm:

When a TCP connection is first opened, the network software has no idea
how fast each router/bridge/whatever in the path can accept data, let alone what the far

system can do.

To prevent a smash of data from causing dat
a to be spilled all over the floor, a TCP transmitter
must start by sending a single segment, then wait for the ACK from the other end. The
“congestion window” is then doubled in size, from one segment to two. This continues until the
congestion window is
the same size as the advertised window (the amount of buffer the receiver
claims is open for data) or until packets start to become lost.

On local

area networks where there is no intermediate router, this slow

start algorithm will slow
down the start of d
ata transmission slightly. The effect is negligible because when modem
compression is enabled the dictionaries are just starting to learn the data; when compression is
disabled the only time that the modem channel will be quiet will be after the first pack


ACK Procedure:

This attempts to reduce the number of acknowledgments transmitted
by trying to "piggy
back" acknowlegements with the transmission of data, or delaying the
transmission of an ACK so that multiple data packets can be acknowledged

at the same time with
a single ACK packet. This affects throughput testing by permitting the sending application to "use
up" the transmission window, and then needing to wait until the receiving application successfully
sends an acknowledgement to indicat
e there is now room to send more data.

When coupled with the slow

start algorithm, this causes the TCP protocol to behave as a half

duplex protocol for one packet. This in turn emphasizes the round

trip delay of the modem at the
beginning of a connection.

Worse, it can mask poor start


connection performance by letting
the delays caused by slow

start and delayed

ACK to mask error events.

Nagel algorithm:

The Nagle algorithm, named for the person who proposed it, tries to stuff as
many characters into a
packet as possible by delaying the transmission of data until "enough"
data is queued, or a sufficient time interval has passed from the system's receipt of the last
character from the application before sending the packet. The parameters of this algorithm
, the
transmission threshold and the time delay before transmission of "runt" packets varies from
implementation to implementation, but tend to be optimized toward interactive applications such

The only concern with the Nagle algorithm is that t
he testing software should present enough data
to the networking support software in each call such that there is no delay in transmitting a packet.
In practice, the software can query the network support to determine the maximum segment size
(MSS) and pre
sent data in buffers of that size. This eliminates any delay that would be caused by
presenting data in smaller groups (8 or 16 bytes at a time).

When testing a new TCP system, it's a good idea to run a packet sniffer in order to understand
the interactio
ns between the two sides of the conversation.


Watch out carefully for indistriminate use of domain names to identify systems when running in a
test environment. The response time for DSN queries can vary significantly, even when running
on a complete
ly private local area network. It's best if you use absolute IP addresses to address
systems, so as to avoid the variable of DNS lookup requests.

Other protocols

The interactions of other protocols, such as Address Resolution Protocol, play an insignific
ant part
when testing in a PPP environment, and may be safely ignored.

The implementation of PPP performs the task of identifying when the modem

based channel is
ready to accept data. The status of the PPP channel can be sampled by the test implementation

code. When the channel reports an UP status, the two programs can using the standard TCP
handshake to establish a connection, and begin transmitting data. The volume of data must be
large enough to avoid having the slow

algorithm and the delayed

ACK algor
ithm skew the test

Throughput testing using TCP/IP is very useful, and can be used to compare modem
performance as long as the network conditions are identical from test to test. In order to make
tests repeatable, use this checklist:

Condition P
PP to escape as few characters as possible, preferably none other than the two
flag characters

Use numeric IP addresses instead of IP names

Use a raw TCP connection (referred to as stream mode, clear TCP, raw TCP, or base
channel TCP.

Condition the soft
ware to use the MSS as the transmit element size

Eliminate as much non

test traffic off the link(s) as you can

Handle the case where data is delayed by fragmentation.

Transfer enough data, to avoid distortion by slow
start and delayed

When compari
ng throughput measured by the TCP/IP method with throughput measured by the
datastream method, remember that the modem's signal converter has to use more bits to transmit
the same data. The actual amount of extra data depends on setup; however, if the
mmendations in this paper are followed the difference is around three percent.

Data Delay Measurement

The measurement of delay using TCP/IP is considerably simpler, and in fact there are a couple of
ways to get the job done, each with its advantages and

TSB 38

TSB 38 measures delay using single characters, exchanged in a lockstep manner, and using a
block and a single character, again exchanged in lockstep. The first method duplicates what
happens when a terminal is used on a modem

ted terminal server with character echo
supplied by the server. The second method measures how modems react when transmitting files
using the classic Ward Christensen XMODEM protocol.


One way to move this test into the TCP/IP environment is to use

a suitably configured TCP
channel and use it exactly the same way as one would use the modem's datastream channel to
perform exactly the same test procedure as described in TSB 38. Virtually every TCP
implementation has a way to turn off the Nagle algorit
hm; for example, the BSD Unix method is to
use the function setsockopt and specify that the option TCP_NODELAY be set to TRUE. This
means that when the software presents a single character, that character is transmitted...along
with the 47 bytes or so of h
eader information necessary to carry it! With Von Jacobson
compression enabled, though, this is actually the lowest

overhead method of performing the test.

For the Block Acknowledgement Delay test, sending a single character followed by the rest of the
ock lets the receiver measure the time required to receive the first and last byte, as required in
datastream mode by TSB 38.

...or UDP...

In order to reduce the packet overhead, we can look to another protocol: User Datagram Protocol
(UDP). The eight

yte UDP header is considerably smaller, but it means that Von Jacobson
compression can't be used

Von Jacobson compression does not support UDP packets. The
standard utility traceroute uses UDP packets, for example, to probe the path of a connection and
determine the time required for each hop of an Internet connection path.

Because UDP is a datagram protocol, there are none of the system

imposed flow control issues
in UDP. There is also no error control, none at all

your application has to detect and de
al with
lost packets and out


order packets.

For the Character Echo Delay test, UDP packets carrying a single character can be exchanged in
the identical manner that the individual characters themselves are handled in TSB 38, random
delay and all.

the Block Acknowledgement Delay test, a 133

byte packet can be transmitted in a UDP
packet without worry about the packet being fragemented on the way. This wouldn't be true for
variations on the BAK test, which use blocks of 512, 1024, and 1500 bytes; ind
eed, that last would
most likely cause fragmentation because the most common MTU is 1500 bytes, set by Ethernet.
In order to know when the first and last character arrived at the receiver, though, the transmitting
program would have to send two packets, on
e with a single byte, and one with 132 bytes.

...or ICMP

Virtually every TCP/IP implementation running on a general

purpose computer has on it an
application that performs round
trip measurement: PING. The problem with the standard PING

program is that t
here is no provision for measuring the time for each direction, just the overall

trip delay time. There is another ICMP call, called the timestamp request and reply. No
standard utility exists that uses this facility, but it is avaialble in standard
implementations of
TCP/IP and could be utilized. There is no variation of this utility that will work to mimic the BAK
test, as the packets exchanged are the same size just as they are with the ping packets.

From the analysis above, the ideal method for m
easuring delay rests either in the TCP domain or
in the UDP domain. While the ICMP method (particularly using ping which already exists) is
attractive, it deviates a bit from the information about modems that TSB 38 was designed to

As for checkli
sts, the only item to add to the throughput checklist would be that when performing
the real

time tests, turn the Nagle algorithm off and be very, very careful about the packet sizes
you present to the network software.

Call Connect Reliability

Nothing s
pecial is required for call connect reliability in the TCP/IP domain. Indeed, the fact that
PPP has to negotiate a connection means that if the PPP interface ever reports UP status you
have a completed link. So the major difference in the CCR test is that
you let PPP determine if it
could establish a link, and use the time the interface reports UP status as the success time.
(Moreover, many PPP implementations will record the time it saw a successful establishment,
further eliminating work.)


e modification of the test procedures from the datastream methods of TSB 38 to the TCP/IP
methods we want to use has a few potholes, particularly in measuring throughput accurately and
repeatedly. This paper makes several suggestions for reducing the diffe
rences as much as
possible from run to run, from TCP/IP implementation to TCP/IP implementation, from lab to lab,
from tester to tester.

There is no way that we can achieve any goal of having TCP/IP based tests that duplicate exactly
in the TCP/IP domain
the results we get in the datastream mode. What we can do, and what this
paper tries to show, is that results can be compared once certain overhead information is known;
from that overhead information we can calculate the necessary correction factors.