Fast IP Routing

greydullNetworking and Communications

Oct 30, 2013 (3 years and 8 months ago)

50 views

1

AC_055_2000

© 2000, Cisco Systems, Inc.

Fast IP Routing

Axel Clauberg

Consulting Engineer

Cisco Systems

Axel.Clauberg@cisco.com

2

AC_055_2000

© 2000, Cisco Systems, Inc.

Agenda


The Evolution of IP Routing


Transmission Update: 10GE


Router Architectures


So, it‘s all just speed ?

3

AC_055_2000

© 2000, Cisco Systems, Inc.

The Evolution of IP Routing

4

AC_055_2000

© 2000, Cisco Systems, Inc.

Heard around the corner ?


IP Routers are slow, sw
-
based


IP Routers cause high latency


IP Routers are undeterministic


IP Routers do not support QoS


5

AC_055_2000

© 2000, Cisco Systems, Inc.

WAN Customer Access Speed
Evolution


Late 1980s:

9.6 Kb/s .. 64 Kb/s


Early 1990s:

64 Kb/s .. 2 Mb/s


Late 1990s:

2 Mb/s .. 155 Mb/s


Early 2000s:

155 Mb/s .. 10 Gb/s

6

AC_055_2000

© 2000, Cisco Systems, Inc.

Backbone Evolution


Late 1980s: 56/64 Kb/s


Early 1990s: 1.5/2
Mb/s


Mid 1990s: 34 Mb/s,
155 Mb/s


Late 1990s: 622 Mb/s,
2,5 Gb/s


Early 2000s: 10 Gb/s,
40 Gb/s


Late 1980s: 10 Mb/s


Early 1990s: 100 Mb/s
(FDDI)


Mid 1990s: 155 Mb/s
(ATM)


Late 1990s: nx FE, 155
Mb/s, 622 Mb/s, GE


Early 2000s: 10 Gb/s,
n x 10 Gb/s

WAN

Campus

7

AC_055_2000

© 2000, Cisco Systems, Inc.

Transmission Update: 10GE

8

AC_055_2000

© 2000, Cisco Systems, Inc.

Lower Cost and Overhead

MAN/WAN IP Transport
Alternatives

IP

ATM

Optical

B
-
ISDN

IP

Optical

IP

SONET/SDH

Optical

ATM

SONET/SDH

IP

Optical

Multiplexing, Protection and Management at every Layer

IP over
ATM

IP over SDH

IP over
Optical

IP

Ethernet

Optical

IP over
Ethernet

GE


10GE

9

AC_055_2000

© 2000, Cisco Systems, Inc.

Ethernet Scaling History


1981: Shared 10 Mbit


1x


1992: Switched 10 Mbit

10x


1995: Switched 100 Mbit

100X


1998: Switched 1 Gigabit

1000X


200x: Switched 10 Gigabit 10000X

10

AC_055_2000

© 2000, Cisco Systems, Inc.

Moving the Decimal Point: 10 GbE Performance

and Scalability

1996

1997

1998

1999

2000

1 Gbps

100 Mbps

10 Gbps

10 Gbps

Ethernet

Gigabit Ethernet

Fast Ethernet

Fast EtherChannel

Gigabit EtherChannel

STM
-
64

2001

2002

10 GbE IEEE
802.3ae
Standard


LAN applications


Metro applications


WAN applications

11

AC_055_2000

© 2000, Cisco Systems, Inc.

Why 10 Gigabit Ethernet


Aggregates Gigabit Ethernet segments


Scales Enterprise and Service Provider LAN
backbones


Leverages installed base of 250 million
Ethernet switch ports


Supports all services (packetized voice and
video, data)


Supports metropolitan and wide area networks


Faster and simpler than other alternatives


12

AC_055_2000

© 2000, Cisco Systems, Inc.

IEEE Goals for 10 GbE

(Partial List)


Preserve 802.3 Ethernet frame format


Preserve minimum and maximum frame size of
current 802.3 Ethernet


Support only full duplex operation


Support 10,000 Mbps at MAC interface


Define two families of PHYs

LAN PHY operating at 10 Gbps

Optional WAN PHY operating at a data rate
compatible with the payload rate of OC
-
192c/SDH VC
-
4
-
64c


13

AC_055_2000

© 2000, Cisco Systems, Inc.

IEEE 802.3ae Task Force Milestones

1999

2001

2002

2000

PAR

Drafted

PAR

Approved

802.3ae

Formed

First

Draft

Working

Group

Ballot

LMSC

Ballot

Standard

HSSG= Higher Speed Study Group

PAR= project authorization request

802.3ae= the name of the project and the name of the sub
-
committee of IEEE 802.3 chartered with writing the
10GbE Standard

Working group ballot= task force submits complete draft to larger 802.3 committee for technical review and ballot

LMSC: LAN/MAN Standards Committee ballot. Any member of the superset of 802 committees may vote and
comment on draft

HSSG

Formed

First 10GE
deliveries

14

AC_055_2000

© 2000, Cisco Systems, Inc.

10 Gigabit Ethernet Media Goals

1300 nm Laser

CDWM (4x2.5)

1300 nm Laser

standard reach

Media Type

Type

1550 nm Laser

extended reach

40
-
100 km

std/dispersion free fiber

single mode fiber

2
-
10 km


multimode fiber

300 m

200 m

ribbon multimode fiber

Max Distance

780 nm VCSEL

multichannel

15

AC_055_2000

© 2000, Cisco Systems, Inc.

IEEE Status


802.3ae Meeting 10.
-
14. Juli 2000


75% Consensus

1550nm Transceiver 40 Km @ SMF

1300nm Transceiver 10 Km @ SMF


No Consensus yet

Multimode Support


300m mit 62.5µ 160/500 Mhz*Km MM


50µ 2000/500 MHz*Km MM

16

AC_055_2000

© 2000, Cisco Systems, Inc.

Router Architectures

17

AC_055_2000

© 2000, Cisco Systems, Inc.

Components


Memory Architecture


Interconnect


Forwarding Engine



Scalability


Stability


Queueing / QoS

18

AC_055_2000

© 2000, Cisco Systems, Inc.

Basic Design


Data are of random sizes


Arrival is async, unpredictable, independantly on i/f


Data have to be buffered


TCP/IP traffic is bursty, but short
-
term congestion
only

Router

...

...

Inputs

Outputs

Forwarding

Engine

Route

Processor

Buffer

Memory

Interfaces

19

AC_055_2000

© 2000, Cisco Systems, Inc.

How much buffers ?


Rule of Thumb: RTT x BW

(Villamizer & Song, High Performance TCP in ANSNET,
1994)


STM
-
16 @ 200 ms: ~ 60 MB buffering
capacity

20

AC_055_2000

© 2000, Cisco Systems, Inc.

How to Buffer ?


SRAM

Fast, Power
-
hungry, Density 8 Mb
-
> 16
Mb, Simple Controller Design


DRAM / SDRAM

Slower, Less Power, Density 64 Mb
-
>
256 Mb, Complex Controller Design

21

AC_055_2000

© 2000, Cisco Systems, Inc.

Interconnect


Switch Fabric / Crossbar


Shared Memory


Variations

22

AC_055_2000

© 2000, Cisco Systems, Inc.

Switch Fabric / Crossbar


Packet forwarding
decision done on each
linecard


Ingress and Egress
Buffering on Linecards


Possible Problem: Head
of Line Blocking


Solution: VOQ


Line

Card 0




Switch

Fabric




Scheduler

Line

Card 1

Line

Card N

Line

Card 0

Line

Card 1

Line

Card N

RP

RP

Ingress Line Cards

Egress Line Cards

23

AC_055_2000

© 2000, Cisco Systems, Inc.

Linecard in Detail

Physical

Layer

(Optics)

Layer 3

Engine

Fabric

Interface

RX

TX

CPU

To Fabric

From Fabric




Switch

Fabric




Scheduler



HOL Blocking can occur when packet cannot flow off transmit linecard



Packet will be buffered on receiving linecard



Packet blocks other packets to other linecards



Solution: Virtual Output Queues, one per egress linecard

24

AC_055_2000

© 2000, Cisco Systems, Inc.

Receive Line Card

Transmit

Line Card

Group of 8 CoS


Queues


Per Interface

(M
-
DRR)

Input

Ports

Output

Ports

Crossbar Switch Fabric

W
-
RED

CAR

CEF



Virtual

Output

Queues

DRR

GSR Queuing Architecture

25

AC_055_2000

© 2000, Cisco Systems, Inc.

Shared Memory Architecture

Physically Centralized


One large memory
system, data passing
through it


Simple memory
management


High speed memory


Simple Linecards


Needs SRAM for high
speeds




Interconnects

&

Forwarding

Engine







2.5Gbps

2.5Gbps

2.5Gbps

2.5Gbps

Line Cards 1
-
8

Memory Controller

40

G

26

AC_055_2000

© 2000, Cisco Systems, Inc.

Shared Memory Architecture

Distributed


Memory distributed over
linecards


Memory controller treats sum
of pieces as shared memory


Packet forwarding decision
in central engine(s)


Difficult to maximize
interconnect efficiency

Egress line cards simply
request packets from shared
memory

Causes Head of Line (HOL)
blocking and high latency,
worsening under moderate
-
to
-
heavy system load or with
multicast traffic




Memory

Controller

&

Forwarding

Engine(s)

Memory

System

2.5Gbps

2.5Gbps

Memory

System

2.5Gbps

2.5Gbps

Line Cards 1
-
8

27

AC_055_2000

© 2000, Cisco Systems, Inc.

Switch Fabric vs. Shared Memory


Shared Memory requires only half the
buffer space


HOL Blocking in Shared Memory,
especially for Multicast


Involvement of distributed shared memory
causes more points of failure

28

AC_055_2000

© 2000, Cisco Systems, Inc.

Forwarding Engine


Classifying the packet

IPv4, IPv6, MPLS, ...


Packet validity (TTL, length, ...)


Next Hop


Basic Statistics


Optional:


Policing, Extended Statistics, RPF check
(security, Multicast), QoS, Tunnel, ...


Distributed vs. Central

29

AC_055_2000

© 2000, Cisco Systems, Inc.

Central Forwarding ?


IP Longest match

Hash vs. TCAM vs.
Tree Lookup


Tree Lookup requires high number of
routing table lookups

Need SRAM

Danger to run out of SRAM

Forwarding speed dependant on depth of
routing table

30

AC_055_2000

© 2000, Cisco Systems, Inc.

Distributed Forwarding


One copy of forwarding info per
linecard


Parallel processing without sync or
communication between linecards


Able to use TCAMs and SDRAMs



31

AC_055_2000

© 2000, Cisco Systems, Inc.

So, it’s all just speed ?

32

AC_055_2000

© 2000, Cisco Systems, Inc.

So, it’s just
speed ?


Services

IP Multicast

IP QoS

Security


IPv6


MPLS


Manageability


Availability


Investment protection

33

AC_055_2000

© 2000, Cisco Systems, Inc.

Interdomain Multicast

Campus Multicast

Multicast Solutions

End
-
to
-
End Architecture


End Stations (hosts
-
to
-
routers):

IGMP


Switches (Layer 2 Optimization):

IGMP Snooping


Routers (Multicast Forwarding
Protocol):

PIM Sparse Mode


Multicast routing across domains

MBGP


Multicast Source Discovery

MSDP with PIM
-
SM

ISP B

Multicast Source

Y

ISP A

Multicast Source

X

ISP B

DR

RP

RP

DR

DR

IGMP

PIM
-
SM

CGMP

MBGP

MSDP

ISP A

34

AC_055_2000

© 2000, Cisco Systems, Inc.

Summary


IP Routers have evolved during the past
years


Line rate up to 10 Gb/s


Crossbar architectures with distributed
forwarding seem to scale better than
shared memory architectures


Services remain the most decisive factor

35

AC_055_2000

© 2000, Cisco Systems, Inc.

Outlook


10 Gb/s Interfaces supported in 2000

10GE, STM
-
64/OC
-
192


High density of 10 Gb/s interfaces soon in
a PoP


Next step will be STM
-
256/OC
-
768 = 40
Gb/s


Will these routers be „Palm
-
Size“ ?

Probably not...

www.cisco.com