Panel #2) The Need for Speed – Beyond 100GbE - Ethernet Alliance

wartrashyΔίκτυα και Επικοινωνίες

26 Οκτ 2013 (πριν από 4 χρόνια και 2 μήνες)

129 εμφανίσεις

© 2013 Ethernet Alliance

1

Moderator: Scott Kipp, President of Ethernet Alliance, Principle Engineer, Brocade

Panelist #1: Alan
Weckel
, Vice President,
Dell’Oro

group

Panelist #2: Dr. Jeffery J. Maki, Distinguished Engineer, Juniper

Panelist #3: Dr. Gordon
Brebner
, Distinguished Engineer, Xilinx










Need for Speed: Beyond 100GbE


2

© 2013 Ethernet Alliance

© 2012 Ethernet Alliance

Agenda


Introductions: Scott
Kipp,
Moderator


Panelist #1:
Alan
Weckel
,


10
, 40 and 100GbE Deployments in the Data Center


Panelist #2:
Dr.
Jeffery
J. Maki,


Stepping
Stones to Terabit
-
Class Ethernet


Panelist #3:
Dr. Gordon
Brebner
,


Technology
Advances in 400GbE Components


Q&A


2:40


Live Broadcast from IEEE 802.3 Meeting in
Orlando from John D’Ambrosia


Update on 400GbE Call For Interest


3

© 2013 Ethernet Alliance

Disclaimer


The views WE ARE expressing in this
presentation are our own personal views
and should not be considered the views or
positions of the Ethernet Alliance.


4

© 2013 Ethernet Alliance

Bandwidth Growth

Increased #

of

Users

Increased
Access

Rates
and
Methods

Increased
Services

+

+

=

Bandwidth
Explosion

Everywhere

Source: nowell_01_0911.pdf citing Cisco Visual Networking Index (VNI) Global IP Traffic Forecast, 2010

2015,
http://www.ieee802.org/3/ad_hoc/bwa/public/sep11/nowell_01_0911.pdf


More Devices

More Internet
Users

More Rich Media
Content

Key
Growth
Factors

Speed
Increasing

Broadband

2010
-

7Mbps

2015


28 Mbps

15B Devices

In 2015

2010
-

1 Minute video

2015


2 hour HDTV Movie

3B Users

In 2015

5

© 2013 Ethernet Alliance

Bandwidth Growth
Vs

Ethernet Speeds


IP Traffic is growing ~ 30%/year


If 400GbE is released in 2016, Ethernet speeds will
grow at about 26%/year

0
200
400
600
800
1000
1200
1400
1600
Ethernet Speed
Internet Traffic
Ethernet Speed (Gb/s)

Internet traffic normalized

to 100 in 2010

Internet traffic
would grow ~10X by
2019 at 30%/year

Ethernet speeds
to grow 4X by
2016 at 26%/year

6

© 2013 Ethernet Alliance

Ethernet Optical Modules

XENPAK

XPAK

X2

300 Pin
MSA

100G





10G





1G

1995 2000 2005 2010 2015


Standard Completed

40G

100GbE

40GbE

Data Rate and Line Rate (b/s)

Key:

Ethernet

Standard
Released

Module
Form Factor
Released

GbE

CFP

QSFP+

SFP

GBIC

10GbE

SFP+

XFP

CFP2

QSFP28

CFP4

CXP

7

© 2013 Ethernet Alliance

Ethernet Speeds 2010
-
2025

Key:

Ethernet

Speeds


Ethernet
Electrical

Interfaces


Hollow
Symbols =
predictions


Stretched
Symbols =
Time
Tolerance


1T




100G




10G

400G

40G

4x10G

10X10G

2010 2015 2020 2025


Standard Completed

100GbE

10X10G

40GbE

4X10G

Data Rate and Line Rate (b/s)

16x25G

400GbE

16X25G

4x25G

100GbE

4X25G

8X50G

400GbE

8X50G

400GbE

4X100G

100GbE

1X100G

TbE

10X100G

nX100G

1.6TbE

16X100G

If Ethernet line rates doubles the line rate every
3 years at 26% CAGR, then 400GbE would come
out in 2016 and
TbE

would come out in 2020.
Something will have to change.

8

© 2013 Ethernet Alliance

Ethernet Success


Ethernet has been extremely successful at
lowering the price/bit of bandwidth


If the cost of a new speed/technology is too
high, then it is not widely deployed


Technology needs to be ripe for picking


400GbE is ripe with 100GbE technology


TbE

isn’t ripe and a revolutionary breakthrough
would be needed to get it before 2020


This panel will look at how high speeds of
Ethernet are being deployed and the
technology that is leading to the next
generation of Ethernet

© 2013 Ethernet Alliance

9

10, 40 and 100GbE Deployments in the
Data Center

Alan Weckel

Vice President, Data Center Research

Dell’Oro Group

10

© 2013 Ethernet Alliance

10

© 2013 Ethernet Alliance

Introduction


Progress on server migration from 1 GbE to
10 GbE



10G Base
-
T update



Data center networking market update



40 GbE and 100 GbE market forecasts

11

© 2013 Ethernet Alliance

11

© 2013 Ethernet Alliance

Overview


Dell’Oro Group is a market research firm that
has been tracking the Ethernet Switch and
Routing markets on a quarterly basis since
1996



We also track the SAN market, Optical
market, and most Telecom equipment
markets



We produce quarterly market share reports
that include port shipments as well as market
forecasts

12

© 2013 Ethernet Alliance

12

© 2013 Ethernet Alliance


Petabytes per Second Shipped
per Year

Data Center Bandwidth Shipping



Ethernet Switching

13

© 2013 Ethernet Alliance

13

© 2013 Ethernet Alliance


Percent of Server Shipments

Switch Attach Rate on Servers

10 GbE

1 GbE

40 GbE

14

© 2013 Ethernet Alliance

14

© 2013 Ethernet Alliance


Port Shipments in Thousands

Data Center Port Shipments


10 G Base
-
T Port Shipments

10G Base
-
T controller

and adapter ports

10G Base
-
T switch ports

15

© 2013 Ethernet Alliance

15

© 2013 Ethernet Alliance


Port Shipments in Millions

Data Center Port Shipments


Ethernet Switching

16

© 2013 Ethernet Alliance

16

© 2013 Ethernet Alliance


Port Shipments in Millions

Data Center Port Shipments


Ethernet Switching

17

© 2013 Ethernet Alliance

17

© 2013 Ethernet Alliance

Summary


Ethernet Switches will be responsible for the
majority of 40 GbE and 100 GbE port
shipments over the next five years



Form
-
factor and cost driving 40 GbE over
100 GbE



10 GbE server access transition is key to
higher speed adoption

© 2013 Ethernet Alliance

18

Stepping Stones to Terabit
-
Class Ethernet:

Electrical Interface Rates and

Optics Technology Reuse

Jeffery J. Maki

Distinguished Engineer, Optical

Juniper Networks, Inc.

19

© 2013 Ethernet Alliance

19

© 2013 Ethernet Alliance

100G

20

© 2013 Ethernet Alliance

CFP, CFP2 and CFP4 for

SMF or MMF Applications

CFP

CFP2

CFP4

CFP MSA Form Factors:

http://www.cfp
-
msa.org/

Optical Connector


LC Duplex (depicted)


MPO

Courtesy of

TE Connectivity

21

© 2013 Ethernet Alliance

CFP

CFP2

CFP4

Module Electrical Lane
Capability

12x10G

electrical

lanes

10x10G or 8x25G

electrical

lanes

4x25G

electrical

lanes

CAUI
-
4 for 4x25G

CPPI & CAUI for 10x10G

CAUI
-
4 for 4x25G

CAUI for 10x10G

22

© 2013 Ethernet Alliance

CFP, CFP2, and CFP4 for
100G Ethernet SMF PMD

Transmit side only depicted.

Current Options


Up to 10 km:
100GBASE
-
LR4



Up to 40 km:
100GBASE
-
ER4

Gear Box

1295.56 nm

1300.05 nm

1304.58 nm

1309.14 nm

Gear Box

1295.56 nm

1300.05 nm

1304.58 nm

1309.14 nm

CFP

CFP2

CFP4

4
λ

on LAN WDM

LAN WDM

LAN WDM

23

© 2013 Ethernet Alliance

23

© 2013 Ethernet Alliance

400G

24

© 2013 Ethernet Alliance

Projection of Form Factor
Evolution to 400G

CD
-
CFP

CFP4

CFP4

CFP4

CFP4

400G

CD
-
CFP2

16x25G

electrical

lanes

8x50G

electrical

lanes

speculation

defensible

CD
-
CFP4

4x100G

electrical

lanes

CFP

CFP2

CFP4

100G

Roman Numerals

XL = 40

C = 100

CD = 400

25

© 2013 Ethernet Alliance

Likely MSA Activity


CFP MSA
http://www.cfp
-
msa.org/



CD
-
CFP: Current CFP needs revamping to support 16 x 25G


CD
-
CFP2: Current CFP2 is ready for 8 x 50G


CD
-
CFP4: Unclear


New CDFP MSA
http://www.cdfp
-
msa.org/


High
-
density form factor supporting 16 x 25G


From slide 26 of
http://www.ieee802.org/3/cfi/0313_1/CFI_01_0313.pdf


26

© 2013 Ethernet Alliance

400G Optics Requirements


First
-
generation transceivers have to be
implementable that meet and eventually
do better than these requirements


Size (Width):


82 mm (CFP width, ~4 x CFP4)


Cost:


4 x CFP4


Power:


24 W (4 x 6 W power profile of CFP4)


Improved bandwidth density transceivers
will need higher rate electrical
-
lane
technology


50G


100G

27

© 2013 Ethernet Alliance

How 400G Ethernet Can
Leverage 100G Ethernet

CFP4
-
LR4

CFP4
-
LR4

CFP4
-
LR4

CFP4
-
LR4

CFP4
-
LR4

CFP4
-
LR4

CFP4
-
LR4

CFP4
-
LR4

CFP4
-
LR4

CFP4
-
LR4

Duplex Single
-
Mode
Fiber Infrastructure

100G Ethernet up to 10 km

400G Ethernet up to 10 km

Parallel Single
-
Mode Fiber Infrastructure

Only 8
Fibers
Used

28

© 2013 Ethernet Alliance

Possible SMF Ethernet Road
Map: 100G, 400G, 1.6T

4 x
100GBASE
-
LR4

or

“400GBASE
-
PSM4”

CD
-
CFP4(LC
)

CFP4(LC)

CFP4(LC)

CFP4(LC)

CD
-
CFP(MPO
)

400GBASE
-
???

CD
-
CFP2(LC
)

CFP4(LC)

4 x 400GBASE
-
???

or

“1600GBASE
-
PSM4”

CD
-
CFP4(LC)

(High
-
Density
100GE)

Early Adopter 400G

Mature 400G

Early Adopter 1.6T

Parallel Single Mode, 4 Lanes (PSM4)

4,
Tx

Fibers and
4,
Rx Fibers

1x12 MPO Connector

CD
-
CFP2(MPO
)

CD
-
CFP4(LC)

CD
-
CFP4(LC)

CD
-
CFP4(LC)

29

© 2013 Ethernet Alliance

Early Adopter 400G using
SMF

Structured Cabling

Technology Reuse:

4 x 100GBASE
-
LR4

Parallel SMF:

“400GBASE
-
PSM4”

Courtesy of
Commscope

30

© 2013 Ethernet Alliance

Early Adopter 400G using
MMF

Structured Cabling

Technology Reuse:

4 x 100GBASE
-
SR4

Parallel MMF:

“400GBASE
-
SR16”

Parallel
Multi
-
Mode


100GBASE
-
SR4, 4 x 25G optical lanes:


4,
Tx

Fibers and 4, Rx Fibers using

1x12 MPO



“400GBASE
-
SR16”, 16 x 25G optical lanes:

16, TX Fibers and 16, Rx Fibers using 2x16 MPO

Courtesy of
Commscope

31

© 2013 Ethernet Alliance

2 x 16 MPO

MMF Breakout Cables


Enabling 400G Adoption

1 x 12 (8 used) MPO

1 x 12 (8 used) MPO

1 x 12 (8 used) MPO

1 x 12 (8 used) MPO

Courtesy of
USConec

2 x 16 MMF MT ferrule

32

© 2013 Ethernet Alliance

100G Can Build 400G at

the Cost of 4 x 100G

Technology Reuse:

4 x 100GBASE
-
SR4

Parallel MMF:

“400GBASE
-
SR16”

Technology Reuse:

4 x 100GBASE
-
LR4

Parallel SMF:

“400GBASE
-
PSM4”

33

© 2013 Ethernet Alliance


Early Adopter PMD


Parallel Fiber, SMF or MMF


Leverage of mature PMD from previous speed of
Ethernet


Planned obsolescence


Implementation (with MPO connector) persists as
high
-
density support of previous speed of Ethernet
(e.g., 4 x 100G)


Mature PMD


SMF: Duplex SMF cabling (e.g., with LC duplex
connector)


MMF: Lower fiber count MMF cabling

Ethernet PMD Maturity &
Possible Obsolescence

34

© 2013 Ethernet Alliance

SMF Density Road Map

Front
-
Panel

Bandwidth

Density

(Relative)

100G

400G

1.6T

CFP(LC)

CFP2(LC)

CFP4(LC)

CFP4(LC)

4 x

or

CD
-
CFP(MPO
)

CD
-
CFP2(LC
)

CD
-
CFP4(LC
)

CD
-
CFP4(LC
)

4 x

CD
-
CFP2(MPO
)

CD
-
CFP2(MPO
)

1

2

4

8

16

Port Bandwidth

(mature)

(early adopter)

(mature)

(mature)

(early

adopter)

(early adopter)

35

© 2013 Ethernet Alliance

Summary


Form
-
factor road map for bandwidth
evolution


Early adopter 400G Ethernet by reusing 100G
module and parallel cabling, SMF or MMF


Need for a new, 2 x 16 MMF MT ferrule


Possible common module for 400G Ethernet
and high
-
density (4
-
port) 100G Ethernet


Need for new electrical interface definitions
supporting lane rates at


50G


100G

© 2013 Ethernet Alliance

36

Gordon Brebner

Distinguished Engineer

Xilinx, Inc.

Technology Advances in 400GbE
Components

37

© 2013 Ethernet Alliance

400GbE PCS/MAC


Expect first: 16
PCS
lanes,
each at 25.78125
Gbps


Glueless

interface to optics


Possible
re
-
use of the 802.3ba
PCS


Other options possible for PCS, maybe native FEC



Later: 8 lanes, each at 51.56Gbps


Or 4 lanes with 2 bits/symbol at 56Gbaud (e.g. PAM4)



Packet size 64 bytes to 9600 bytes



Use 100GbE building blocks where possible

38

© 2013 Ethernet Alliance

Silicon technology


Technology nodes (silicon feature size)


130nm, 65nm, 40nm, 28/32nm, 20/22nm, 14/16nm



Application
-
Specific Integrated Circuit (ASIC)


Fixed chip


Increasingly expensive: need high volumes


Best suited to post
-
standardization Ethernet



Field Programmable Gate Array (FPGA)


Programmable logic chip


Suitable for prototyping and medium volumes


Best choice for pre
-
standardization Ethernet

39

© 2013 Ethernet Alliance

400GbE line/system bridge

500G


Interlaken

40 x 12.5G

or

48 x 10G

SERDES



Bridge

logic

400GbE



PMA/PCS

CDFP

or

4xCFP4


O
ptical

16 x 25G

SERDES

400GbE



MAC

Wide parallel data path between blocks

ASIC or FPGA chip

Line side

System side

40

© 2013 Ethernet Alliance

MAC rate = Width x Clock

400
Gbps

and 1
Tbps

Ethernet MAC options

MAC rate

Silicon node

Technology

Data

path w
idth

Clock frequency

100
Gbps

45,

40nm

ASIC

160 bits

644 MHz

100
Gbps

45,

40nm

FPGA

512 bits

195 MHz

400
Gbps

28,

2
0nm

ASIC

400 bits

1 GHz

400
Gbps

28,

20nm

FPGA

1024 bits

1536 bits

400 MHz

267 MHz

1
Tbps

20, 14nm

ASIC

1024 bits

1 GHz

1
Tbps

20
, 14
nm

FPGA

2048 bits

2560 bits

488 MHz

400 MHz

41

© 2013 Ethernet Alliance

Multiple Packets/Word


Up to 512
-
bit, only one packet completed


Just need to deal with EOP then SOP in word



Beyond 512
-
bit, multiple packets completed


Need to add parallel packet processing


Must deal with varying EOP and SOP positions

Bus width

Max packets

Max EOPs

512

2

1

1024

3

2

1536

4

3

512 *
n

n
+1

n

42

© 2013 Ethernet Alliance

400GbE CRC Example


All Ethernet packets carry Cyclic Redundancy
Code (CRC) for error detection


Computed using CRC
-
32 polynomial


Critical function within Ethernet MAC



Requirements


Computed at line rate


Deal with multiple packets in wide data path


Economical with silicon resources

43

© 2013 Ethernet Alliance

400GbE CRC Prototype


Xilinx Labs research project


Modular: built out of 512
-
bit 100G units


Computes multiple CRCs per data path word


Targeting 28nm FPGA (
X
ilinx Virtex
-
7 FPGAs)

N
-
bit data
path
partitioned
into 512
-
bit
sections

512
-
bit unit
CRC results
combined
to get final
CRC results

44

© 2013 Ethernet Alliance

400GbE CRC Prototype


Results:








1024
-
bit width is feasible for 400GbE


Other widths:


Less challenging clock frequencies


Demonstrate scalability beyond 400GbE

Data bus

word size

1024
-
bit

1536
-
bit

2048
-
bit

Max
clock frequency
(MHz)

400

381

326

Maximum l
ine rate (
Gbps
)

409

585

668

Latency (ns
)

17.5

18.4

21.5

FPGA resources (slices)

2,888

4,410

5,719

45

© 2013 Ethernet Alliance

Conclusions


Can anticipate 400GbE PCS/MAC standard



Ever
-
increasing rates mean ever
-
wider
internal data path width in electronics


Leading to multiple packets per data word



Possible to prototype pre
-
standard
PCS/MAC using today’s FPGA technology



Demonstrated modular Ethernet CRC block
based on 100GbE units


Silicon resource scales linearly with line rate