Evaluation of IPv6 and Comparative Study with Different ...

painlosososSoftware and s/w Development

Jun 30, 2012 (5 years and 1 month ago)

458 views

Evaluation of IPv6 and Comparative Study with Different Operating Systems
Sulaiman Syed Mohamed
FIT, Multimedia University,
63100, Cyberjaya, Malaysia
sulaiman@mmu.edu.my
Asaad Y Malik Abusin
FIT, Multimedia University,
63100, Cyberjaya, Malaysia
asaad.malik@mmu.edu.my
David Chieng
FOE, Multimedia University,
63100, Cyberjaya, Malaysia
htchieng@mmu.edu.my
Abstract
With the increasing popularity in recent years,
many popular Operating System (OS) venders
worldwide have incorporated Internet Protocol
version6 (IPv6) into their OS. The performance of the
IP stack, together with the behavior of the OS greatly
affects the efficiency of network applications built on
top of it. Like any other protocols, the acceptance of
various IPv6 implementations heavily relies on the
end-user performance. Hence with the availability of a
variety of IPv6 implementations, it is essential to
evaluate their performance under different operating
systems. An investigation of IPv6 stack performance on
three different operating systems namely,
Windows2003, Redhat Linux 9.0 (Redhat9.0) and
FreeBSD4.9 is discussed in this paper. Performance
analysis and comparison of the three IPv6 stacks, some
interesting points raised by the experimental results
are addressed.
Keywords - Internet, IPv6, Performance,
Windows2003, Redhat9.0 and FreeBSD4.9
1.Introduction
IPv6 [1] becomes the choice of next generation
Internet. The Internet Engineering Task Force (IETF)
has introduced IPv6 with a mission to meet the
growing demands of the future Internet. IPv6 was
designed not only to increase the address space, but
also includes unique benefits such as scalability,
security, simple routing capability, easier configuration
“plug and play”, support for real-time data and
improved mobility support [2, 3]. In recent years, IPv6
have received significant attention by researchers,
educational institution, software vendors and end users.
Many vendors have pre-release IPv6 implementations
and there are several free, publicly available
implementations for BSD and Linux variants [4].
FreeBSD4.9 stable release has included the KAME
stack and the Linux variants are supported with
USAGI. Early in 1998, Microsoft developed and
released a public implementation of IPv6 with its
source code for testing, research, and educational
purposes [5], named Microsoft Research IPv6
(MSRIPv6 1.0). As of this writing the latest release is
MSRIPv6 1.4. The first fully supported release of the
Microsoft IPv6 stack is included with Windows2003
Server designed for full production use. Obviously
Performance is an important criterion for the wide
acceptance of any of these IPv6 stack implementations.
Hence with the availability of a variety of IPv6
implementations, it is essential to evaluate their
performance under different OS. However, only a few
works are found to evaluate the performance of IPv6 at
the operating system level. We choose to evaluate the
performance of IPv6 stack with three real-time popular
Operating Systems namely Windows2003,
FreeBSD4.9 and Redhat9.0. All the experiments were
conducted uniformly on native IPv6 network and a
comparative analysis is made in order to understand
the impact of OS in IPv6 stack performance in real-
time.
The rest of this paper is organized as follows:
Section 2 discusses some of the similar works done by
other researchers in estimating the performance of
IPv6. Section 3 and 4 describes the proposed
experimental setup and the metrics to evaluate the
performance of IPv6 respectively. Section 5 details the
results of evaluation. Section 6 finally concludes the
paper.
2.Related works
A first attempt to develop a prototype of IPv6 stack
by Microsoft for their Windows NT OS is discussed in
[6]. The work presented evaluates the IPv6 prototype
using TCP. Peter Ping Xie [7] evaluated the MSR IPv6
BETA protocol stack for Windows NT 4.0 operating
system. He measured latency, throughput and
processing overheads in order to evaluate IPv6 but
Proceedings of the Third International Conference on Information Technology and Applications (ICITA’05)
0-7695-2316-1/05 $20.00 © 2005 IEEE
lacked comparison with other implementations. In
recent years, MSR IPv6 BETA has been replaced by
newer releases of improved IPv6 stack. Seiji Ariga, et
al. [8] evaluated KAME IPv6 protocol stack using an
application named Digital Video Transmission
developed by Keo University. A software router was
used in their test bed and compared IPv4 vs. IPv6
under FreeBSD platform.
Ettikan [9, 10] evaluated and compared the
performance of KAME IPv6 implementation against
IPv4 using a ported ping and ftp application with two
metrics, throughput and latency. Sherali Zeadally, et al.
[11] evaluated and compared IPv6 protocol stack
offered by Solaris 8.0 against Microsoft IPv6
Technology Preview for Windows 2000. The work
presented used two hardware routers. According to the
author, impartial conduction of experiments was very
difficult due to an undocumented bug they faced with
UDP of Windows IPv6 stack.
The related works presented here clearly indicates
the need for comparative performance analysis of IPv6
stacks offered by popular OS. Furthermore evaluation
of IPv6 Linux variant could not be found to the best of
our knowledge. Few works were attributed to earlier
implementations of Windows OS. However, at this
point of writing however, production quality IPv6
implementation is available from Microsoft for its
Windows XP and Windows2003 and we are using
Windows2003 implementation in our experiments.
3.Experimental setup
We proposed to setup three different test-bed
configurations for our experiments. The test lab
consisted of three Intel Pentium III 733 MHz systems
with 128MB of RAM and 30GB of HDD storage. All
the systems were installed with Windows2003,
FreeBSD4.9 and Redhat9.0 along with respective IPv6
implementations provided by the vendor.
Figure 1. Loop back connection
Test-bed I uses a single system with the
configuration shown in Figure 1. The loop back
interface is utilized to perform the experiments. We
include this test-bed in our experiments in order to
avoid the external factors such as bandwidth, cable
noise and collision, intermediate nodes, etc that may
affect the performance of a network protocol stack in
real time. Hence, the results obtained will show the
actual efficiency of the stack.
Figure 2. Direct link connection
Test-bed II consists of two systems connected
directly to each other using an Ethernet hub. The
experimental setup is shown in Figure 2. This depicts
the scenario of communication between two systems
on the same network (LAN) in real time.
Figure 3. Direct link connection via router
In test-bed III, we introduced a router in between
two systems so that the systems can communicate with
each other as shown in Figure 3. The scenario is
similar to communication between two systems on
different network, between a system in the internal
LAN and a system in the Internet, for example.
4.Performance evaluation
We used the standard metrics for protocol
evaluation in our tests. We measured round trip time in
nanoseconds (ns) for TCP, throughput in megabits per
second (Mbps) for both TCP and UDP protocols,
socket creation time in ns for stream (TCP) and
datagram (UDP) sockets. We also measured
connection time for an application to establish a TCP
connection in ns. Finally, we measured the percentage
of CPU utilization for all the measurement tests and
number of connections that can be established per
second.
Proceedings of the Third International Conference on Information Technology and Applications (ICITA’05)
0-7695-2316-1/05 $20.00 © 2005 IEEE
4.1.Evaluation methodology
Generally, our approach to measure the
performance metrics is straightforward. We measured
the performance metrics by injecting data into the
network by using our measurement codes. We collect
the timestamps before and after the interested points
using our low-level timer function. All codes are
written in C language. The programs for FreeBSD4.9
and Redhat9.0 use BSD as the network interface.
Similarly the code for Windows2003 uses Winsock
2.0. We used the GCC (GNU Compiler Collection)
compiler in all three operating systems to compile our
codes. The tests were conducted using payload sizes
varying from 32 bytes to 65535 bytes. The tests were
repeated to rule out the inconsistency of the obtained
results.
4.2.Timer accuracy and measurement
overhead
We embedded few assembly instructions in our
measurement codes to obtain the current system time
from the Pentium CPU hardware tick counter. We used
the assembly instruction RDTSC (ReaD Time Stamp
Counter) that gives us a time resolution of around
1.3643 nanoseconds with a 733MHz speed Pentium III
CPU. However like every other timer function it also
has some time overhead, which must be calculated and
adjusted from the original measurements obtained. To
do this we wrote a special code to obtain the time
stamps consecutively with no other instruction running
in between them. The time difference between the two
consecutive time stamps is the time overhead of the
timer. We have repeated this test and got an average
timer overhead of 33 CPU ticks.
5.Result analysis
At this stage, the experiments for test-bed I & II are
performed and measurements have been done only for
round trip time (RTT) and throughput. The analysis of
results obtained is as follows:
5.1.Test bed – I: loop back
5.1.1.TCP round trip time. The RTT of TCP for
smaller packet size ranging from 32 – 1500 bytes is
shown in Figure 4. From the graph, it can be deduced
that Windows2003 has an additive increase in RTT
over the RTT of FreeBSD4.9 from packet size 150
bytes onwards. With FreeBSD4.9 and Windows2003
we can observe minor increase in RTT with increase in
packet size whereas Redhat9.0 maintains a constant
RTT of 30000 ns. But an increase in RTT with
Windows2003 after 1440 bytes packet size is observed.
0
20000
40000
60000
80000
100000
120000
140000
160000
0 256 512 768 1024 1280 1536
Payload (bytes)
RTT (nano seconds)
FreeBSD 4.9
RedHat 9.0
Windows2K3
Figure 4. Loop back TCP round trip time
Figure 5 shows the RTT of TCP for larger packet
size ranging from 32 bytes – 64 KB. The performance
of Windows2003 is very poor and RTT increases
linearly with increase in packet size. Redhat9.0 keeps
the RTT well controlled and increases very gradually
with larger packet sizes. FreeBSD4.9 seems to be
better than Windows2003 but worse compared to
Redhat9.0.
0
1000000
2000000
3000000
4000000
5000000
0
8192
16384
24576
32768
40960
49152
57344
65536
Payload (bytes)
RTT (nano seconds)
FreeBSD 4.9
RedHat 9.0
Windows2K3
Figure 5. Loop back TCP RTT global view
5.1.2.TCP throughput. Figure 6 shows the TCP
throughput performance analysis for smaller packet
size ranging from 32 to 1500 bytes for all the three
operating systems. It can be inferred from the graph
that all the three operating systems exhibit steady
gradual raise in the performance for this small packet
size range. Redhat9.0 beats the performance of other
two operating systems and touches a peak throughput
of 370 Mbps for the packet size around 1500 bytes.
Windows2003, on the other hand, lies in the opposite
extreme, reaching only 100 Mbps for the same range.
FreeBSD4.9 is comparatively better than
Windows2003 but still lags behind Redhat9.0 of the
order of 200 Mbps. Interestingly, Windows2003 starts
fragmenting the packets exceeding size of 1440 bytes
Proceedings of the Third International Conference on Information Technology and Applications (ICITA’05)
0-7695-2316-1/05 $20.00 © 2005 IEEE
and hence a fall in throughput for this packet size is
observed.
0
50
100
150
200
250
300
350
400
0 256 512 768 1024 1280 1536
Payload (bytes)
Throughput (Mbps
)
FreeBSD 4.9
RedHat 9.0
Windows2K3
Figure 6. Loop back TCP throughput
The performance of throughput shown in Figure 7
is for larger packet size range. Throughput of
Windows2003 rises to 350 Mbps for the range 0 to
16000 bytes and remains constant in the range 16000
to 50000 bytes. However the throughput of
Windows2003 experiences a slight decline for packet
size greater than 500000 bytes. Steady increase in
throughput is observed for FreeBSD4.9 till the packet
size of 15000 bytes and remains approximately
constant for higher packet sizes above 500000 bytes.
The throughput can be approximated to 800 Mbps in
this region. Steep increase in throughput up to a value
of 1900 Mbps is obtained for Redhat9.0 till packet size
becomes 150000 bytes. There after for packet size
greater than 15000 bytes, throughput increases
gradually up to 2300 Mbps and remains around that
value.
0
500
1000
1500
2000
2500
0
8192
16384
24576
32768
40960
49152
57344
65536
Payload (bytes)
Throughput (M b p s
)
FreeBSD 4.9
RedHat 9.0
Windows 2K3
Figure 7. Loop back TCP throughput global view
A drop in throughput at 14436 bytes & 16384 bytes
are detected for FreeBSD4.9 and Redhat9.0
respectively.
5.1.3.UDP throughput. The UDP throughput for
packet size range 32 - 1500 bytes is given in Figure 8.
The UDP throughput performance is comparable to the
performance of TCP throughput for this smaller packet
size range under the Loop Back test bed. Linear
increase in throughput is perceived for all the three
operating systems of our interest. Windows2003 being
the poor performer reaches 100 Mbps for 1500 bytes.
FreeBSD4.9 is comparatively better than
Windows2003 and touches 200 Mbps for 1500 bytes.
Redhat9.0 shows high performance of 500 Mbps for
1500 bytes.
0
100
200
300
400
500
600
0 256 512 768 1024 1280 1536
Payload (bytes)
Throughput (Mbps
)
FreeBSD 4.9
RedHat 9.0
Windows2K3
Figure 8. Loop back UDP throughput
Figure 9 shows the UDP throughput for packet size
from 32 bytes to 64 Kbytes. Here again the UDP
performance throughput can be compared to TCP
throughput under Loop Back test bed for the same
packet size range. Windows2003 rises to a maximum
of 400 Mbps in the range 20000 – 38000 bytes but falls
to 350 Mbps after this range. FreeBSD4.9 has a
maximum of 1000 Mbps and decreases for packet sizes
greater than 40000 bytes. Redhat9.0 surpasses
FreeBSD4.9 and Windows2003 in performance and
has a maximum throughput of 3100 Mbps.
0
500
1000
1500
2000
2500
3000
3500
0
8192
16384
24576
32768
40960
49152
57344
65536
Payload (bytes)
Throughput (Mbps
)
FreeBSD 4.9
RedHat 9.0
Windows2K3
Figure 9. Loop back UDP throughput global view
5.2.Test bed – II: direct link
5.2.1.TCP round trip time. Figure 10a. shows RTT
of TCP for smaller packet size range. As it is evident
from the graph, Windows2003 and FreeBSD4.9 have
similar performance. Redhat9.0 though performs little
Proceedings of the Third International Conference on Information Technology and Applications (ICITA’05)
0-7695-2316-1/05 $20.00 © 2005 IEEE
better than FreeBSD4.9 and Windows2003 as shown in
the enlarged view Figure 10b. Its rate of increase in
RTT coincides with the rate of increase of RTT with
FreeBSD4.9 or Windows2003.
0
500000
1000000
1500000
2000000
2500000
3000000
3500000
0 256 512 768 1024 1280 1536
Payload (bytes)
RTT (nano seconds)
FreeBSD 4.9
RedHat 9.0
Windows2K3
Figure 10a. Direct link TCP round trip time
1000000
1100000
1200000
384 448 512 576
Payload (bytes)
RTT (nano seconds)
FreeBSD 4.9
RedHat 9.0
Windows2K3
Figure 10b. Direct link TCP RTT – enlarged
0
20000000
40000000
60000000
80000000
100000000
120000000
140000000
160000000
180000000
0
8192
16384
24576
32768
40960
49152
57344
65536
Payload (bytes)
RTT (nano seconds)
FreeBSD 4.9
RedHat 9.0
Windows2K3
Figure 11. Direct link TCP RTT - global view
Figure 11 shows the performance of RTT for larger
packet size range. Here Windows2003 has
comparatively better performance than the other two
operating systems. On the other hand FreeBSD4.9 and
Redhat9.0 overlie each other and a closer look at the
graph reveal that Redhat9.0 has slightly better
performance than FreeBSD4.9.
5.2.2.TCP throughput. Figure 12 outlines the
performance of throughput for tiny packet size ranging
from 32 to 1500 bytes. As it is clearly seen, the
behavior of all the three operating systems under
consideration is very close to each other. The better
performance from least to higher is in the order
FreeBSD4.9, Windows2003 and Redhat9.0. The
average difference between the performance of
FreeBSD4.9 and Windows2003 is of the order of 200
Kbps. Similarly the average difference between the
performance of Windows2003 and Redhat9.0 is 400
Kbps. In all the three operating systems at hand, we
can note the increase in throughput with increase in
packet size.
0
1
2
3
4
5
6
7
8
9
0 256 512 768 1024 1280 1536
Payload (bytes)
Throughput (Mbps
)
FreeBSD 4.9
RedHat 9.0
Windows2K3
Figure 12. Direct link TCP throughput
0
1
2
3
4
5
6
7
8
9
0
8192
16384
24576
32768
40960
49152
57344
65536
Payload (bytes)
Throughput (Mbps
)
FreeBSD 4.9
RedHat 9.0
Windows2K3
Figure 13. Direct link TCP throughput global view
Results of throughput for huge packet size from 32
bytes to 64 KB are shown in Figure 13. It can be
viewed from the graph that the performance of
Windows2003 is better compared to FreeBSD4.9 and
Redhat9.0. It maintains almost a constant throughput of
8.5 Mbps. Redhat9.0 and FreeBSD4.9 on the other
hand, highly overlaps in performance. On keen
observation, we can conclude that Redhat9.0 performs
well compared to FreeBSD4.9.
5.2.3.UDP throughput. UDP throughput for packet
size from 32 – 1500 bytes is shown in Figure 14. As
Proceedings of the Third International Conference on Information Technology and Applications (ICITA’05)
0-7695-2316-1/05 $20.00 © 2005 IEEE
seen in figure, only subtle changes are noted in the
performance of the operating systems under testing.
However, Redhat9.0 has higher performance compared
to FreeBSD4.9. FreeBSD4.9 in turn has better
performance than Windows2003. The average
difference in throughput between FreeBSD4.9 and
Redhat9.0 is around 300 bytes while the average
difference in throughput between FreeBSD4.9 and
Windows2003 is around 150 bytes.
0
1
2
3
4
5
6
7
8
9
0 256 512 768 1024 1280 1536
Payload (bytes)
Throughput (Mbps
)
FreeBSD 4.9
RedHat 9.0
Windows2K3
Figure 14. Direct link UDP throughput
Figure 15 shows the throughput of UDP for packet
sizes 32 bytes – 64 KB. A very high overlap in
throughput performance is observed for all the three
OS. The differences between the throughputs of these
operating systems are negligible and are of the order of
100 bytes.
0
2
4
6
8
10
0
8192
16384
24576
32768
40960
49152
57344
65536
Payload (bytes)
Throughput (Mbps
)
FreeBSD 4.9
RedHat 9.0
Windows2K3
Figure 15. Direct link UDP throughput global view
6.Conclusions and future works
Experimental results shows us that the Redhat9.0
implementation of the IPv6 protocol stack has overall
good performance under Loop Back test bed. The
performance of IPv6 with FreeBSD4.9 deserves the
next higher overall performance. Windows2003 has
comparatively poor performance under Loop Back test
bed. The poor performance of Windows2003 IPv6
stack under Loop Back test bed is due to the
fragmentation of packets greater than 1440 bytes.
Fragmentation of packets occurs at 16384 bytes for
Redhat9.0 and at 14436 bytes for FreeBSD4.9. With
Direct Link test bed, all the three operating systems
have closer performance. However, Windows2003
seems to have better performance with Direct Link test
bed. Further investigation revealed that Windows OS is
favored by the Windows Direct architecture that
switches to a direct path to the physical hardware based
on per connection basis during data transmission. Also,
fragmentation may be better handled with
Windows2003 and we plan to do more tests to confirm
this. We also plan to extend our work to evaluate a
real-time IPv6 application using the same test-beds
discussed here, to further investigate the QoS features
of IPv6 by various implementations.
7.References
[1] S. Deering, R. Hinden. Internet Protocol, Version 6
(IPv6) Specification. RFC 1883, December 1995.
[2] Christian Huitema, “IPv6 The New Internet Protocol”,
Second edition, 1997, pp. 9-9.
[3] D.Lee et al. “The Next Generation of the Internet:
Aspects of the Internet Protocol Version 6”, IEEE Network,
vol. 12, no. 1, Jan./Feb. 1998, pp. 28-33.
[4] IPng Working Group Web Site.
http://playground.sun.com/pub/ipng/html/ipngmain.html
[5] Microsoft Research IPv6
http://www.research.microsoft.com/msripv6
[6] Draves, Richard P, et al. “Implementing IPv6 for
Windows NT”, Proceedings of the 2
nd
USENIX Windows
NT Symposium, Seattle WA, Aug 3-4, 1998.
[7] Peter Ping Xie, “Network Protocol Performance
Evaluation of IPv6 for Windows NT”, Master Thesis,
California Polytechnic State University, San Luis Obispo,
June 1999.
[8] Seiji Ariga et al. “Performance Evaluation of Data
Transmission Using IPSec over IPv6 Networks”, INET 2000
Proceedings, Japan, July 18
th
, 2000.
[9] Ettikan, “Application Performance Analysis in Transition
Mechanism from IPv4 to IPv6”, Faculty of Information
Technology, Jalan Multimedia, Multimedia University, June
2001.
[10] Ettikan, “IPv6 Dual Stack Transition Technique
Performance Analysis: KAME on FreeBSD as the case”,
Faculty of Information Technology, Jalan Multimedia,
Multimedia University, October 2000.
[11] Sherali Zeadally, Ioan Raicu. “Evaluating IPV6 on
Windows and Solaris”, IEEE Internet Computing, Volume 7,
Issue 3, May-June 2003, pp 51 – 57.
Proceedings of the Third International Conference on Information Technology and Applications (ICITA’05)
0-7695-2316-1/05 $20.00 © 2005 IEEE