Nexus 7000 virtual Port-Channel

thoughtlessskytopNetworking and Communications

Oct 29, 2013 (3 years and 5 months ago)

170 views

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

1

Roberto Mari

Technical Marketing Engineer

Data Center Business Unit

November 2009

version
1.1

Nexus 7000

virtual Port
-
Channel


Best Practices & Design Guidelines

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

2

Agenda


Feature Overview & Terminology


vPC Design Guidance & Best Practices


Building a vPC domain


Attaching to a vPC domain


Layer 3 and vPC


Spanning Tree Recommendations


Data Center Interconnect (& Encryption)


HSRP with
vPC


vPC and Services


vPC latest
e
nhancements


ISSU


Convergence and Scalability


vPC Hands
-
on Lab Information


Reference Material

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

4


Allow a single device to use a
port channel across two upstream
switches


Eliminate STP blocked ports


Uses all available uplink
bandwidth


Dual
-
homed server operate in
active
-
active mode


Provide fast convergence upon
link/device failure


Reduce CAPEX and OPEX


Available on current and future
hardware for M1 and D1
generation cards.

Logical Topology without vPC

Logical Topology with vPC

Feature Overview & Terminology

vPC Definition

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

5


vPC peer


a vPC switch, one of a pair


vPC member port



one of a set of ports
(port channels) that form a vPC


vPC



the combined port channel between
the vPC peers and the downstream device


vPC peer
-
link



Link used to synchronize
state between vPC peer devices, must be
10GbE


vPC peer
-
keepalive link



the keepalive
link between vPC peer devices, i.e., backup
to the vPC peer
-
link


vPC VLAN



one of the VLANs carried
over the peer
-
link and used to
communicate via vPC with a peer device.


non
-
vPC VLAN



One of the STP VLANs
not carried over the peer
-
link


CFS



Cisco Fabric Services protocol, used
for state synchronization and configuration
validation between vPC peer devices

vPC

vPC peer

non
-
vPC
device

vPC peer
-
keepalive
link

vPC
member
port

vPC

vPC
member
port

CFS protocol

vPC peer
-
link

Feature Overview & Terminology

vPC Terminology

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

6

Building a vPC Domain

Configuration Steps

Following steps are needed to build a vPC (Order does Matter!)

1.
Configure globally a vPC domain on both vPC devices

2.
Configure a Peer
-
keepalive

link on both vPC peer switches (make sure is operational)

NOTE:

When a vPC domain is configured the
keepalive

must be

operational to allow a
vPC domain to successfully form.

3.
Configure (or reuse) an interconnecting port
-
channel between the vPC peer switches

4.
Configure the inter
-
switch channel as Peer
-
link on both vPC devices (make sure is
operational)

5.
Configure (or reuse) Port
-
channels to dual
-
attached devices

6.
Configure a unique logical vPC and join port
-
channels across different vPC peers


vPC

vPC member port

vPC peer
-
keepalive link

vPC peer
-
link

Standalone
Port
-
channel

vPC peer

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

7

Building a vPC Domain

Peer Link


Definition:


Standard 802.1Q Trunk


Can Carry vPC and non vPC
VLANs
*


Carries Cisco Fabric Services messages (tagged as
CoS
=4
for reliable communication)


Carries flooded traffic from a vPC peer


Carries STP BPDUs, HSRP Hellos, IGMP updates, etc.


Requirements:


Member ports must be
10GE

interfaces one of the
N7K
-
M132XP
-
12 modules


Peer
-
link are point
-
to
-
point. No other device should be inserted
between the vPC peers.


Recommendations (strong ones!)


Minimum 2x 10GbE ports on separate cards for best
resiliency.


Dedicated 10GbE ports (not shared mode ports)


*
It is Best Practice to split vPC and non
-
vPC
VLANs on different Inter
-
switch Port
-
Channels.

vPC peer
-
link

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

8


Common Nexus 7000 configuration:

1x

10G
,
7x

1G

cards


vPC recommendation is 2
10G

cards


Potential problem occurs if Nexus 7000 is
L3

boundary with
single
10G

card


Use Object Tracking Feature available in 4.2


More information from
CCO
:


http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_2/nx
-
os/interfaces/configuration/guide/if_vPC.html#wp1529488

Building a vPC Domain

Peer Link with Single 10G Module

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

9

Scenario
:


vPC deployments with a single N7K
-
M132XP
-
12 card, where core and peer
-
link interfaces are localized on the same
card.


This scenario is vulnerable to access
-
layer isolation if the 10GE card fails on
the primary vPC.

vPC Object Tracking Solution:


Leverages object tracking capability in
vPC (new CLI commands are added).


Peer
-
link and Core interfaces are
tracked as a list of boolean objects.


vPC object tracking suspends vPCs on
the impaired device, so traffic can get
diverted over the remaining vPC peer.

e1/…

e1/…

e1/…

e1/…

e1/…

e1/…

e1/…

e1/…

vPC

Primary

e2/…

e2/…

vPC

Secondary

vPC PL

vPC PKL


L3

L2

rhs
-
7k
-
1(config
-
vpc
-
domain)# track <object>

Building a vPC Domain

Peer Link with Single 10G Module


Object Tracking

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

11

Building a vPC Domain

Peer
-
Keepalive (1 of 2)


Definition:


Heartbeat between vPC peers


Active/Active (no Peer
-
Link) detection


Messages sent on 2 second interval


3 second hold timeout on peer
-
link loss


Fault Tolerant terminology is specific to VSS and deprecated in
vPC.


Packet Structure:


UDP message on port 3200, 96 bytes long (32 byte payload),
includes version, time stamp, local and remote IPs, and domain ID.


Keepalive messages can be captured and displayed using the
onboard Wireshark Toolkit.


Recommendations:


Should be a dedicated link (1Gb is adequate)


Should NOT be routed over the Peer
-
Link


Can optionally use the mgmt0 interface (along with management
traffic)


As last resort, can be routed over L3 infrastructure

vPC peer
-
keepalive link

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

12

Building a vPC Domain

Peer
-
Keepalive (2 of 2)

Cautions/Additional Recommendations:


When using supervisor management interfaces to carry the vPC peer
-
keepalive, do not connect them back to back between the two switches.


Only one management port will be active a given point in time and a
supervisor switchover may break keep
-
alive connectivity


Use the management interface only if you have an out
-
of
-
band
management network (management switch in between).

vPC1

vPC2

vPC_PL

Management
Network

Standby Management
Interface

Active Management
Interface

Management
Switch

vPC_PK

vPC_PK

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

13


Definition:


Port
-
channel member of a vPC peer.


Requirements:


Configuration needs to match other vPC
peer’s member port config.


In case of inconsistency a VLAN or the
entire port
-
channel may suspend (i.e.
MTU mismatch).


Number of member ports on both vPC
peers is not required to match.


Up to 8 active ports between both vPC
peers (16
-
way port
-
channel can be build
with multi
-
layer vPC)

vPC
member
port

vPC
member
port

Building a vPC Domain

vPC Member Port

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

14


vPC works seamlessly in any VDC based environment.




One vPC domain per VDC is supported, up to the maximum number of
VDCs supported in the system.


It is still necessary to have a separate vPC peer
-
link and vPC Peer
-
Keepalive Link infrastructure for each VDC deployed.

Can vPC run between VDCs on the same switch?


This scenario should technically work, but it is NOT officially supported
and has not been extensively tested by our QA team.


Could be useful for Demo or hands on, but It is
NOT recommended

for
production environments. Will consolidate redundant points on the same
box with VDCs (e.g. whole aggregation layer on a box) and introduce a
single point of failure.


ISSU will NOT work in this configuration, because the vPC devices can
NOT be independently upgraded.

Building a vPC Domain

VDC Interaction

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

15

Agenda


Feature Overview & Terminology


vPC Design Guidance & Best Practices


Building a vPC domain


Attaching to a vPC domain


Layer 3 and vPC


Spanning Tree Recommendations


Data Center Interconnect (& Encryption)


HSRP with
vPC


vPC and Services


vPC latest
e
nhancements


ISSU


Convergence and Scalability


vPC Hands
-
on Lab Information


Reference Material

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

16

Attaching to a vPC domain

The One and Only Rule…

ALWAYS

dual attach
devices to a vPC
Domain!!!

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

17


Definition:


Port
-
channel for devices for devices dual
-
attached to
the vPC pair.


Provides local load balancing for port
-
channel
members


STANDARD 802.3ad port channel


Access Device Requirements


STANDARD 802.3ad capability


LACP Optional


Recommendations:




Use LACP when available for better failover and mis
-
configuration protection

vPC
member
port

vPC

Regular

Port
-
channel
port

Attaching to a vPC Domain

IEEE 802.3ad and LACP

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

18

*
VLAN that is NOT part of any vPC and not present on vPC peer
-
link

Attaching to a vPC Domain

”My device can’t be dual attached!”

Recommendations (in order of preference):

1.

ALWAYS

try to
dual attach

devices using vPC (not applicable for routed links).


PROS:

Ensures minimal disruption in case of peer
-
link failover and consistent behavior with vPC dual
-
active scenarios. Ensures full redundant active/active paths through vPC.


CONS:

None

2.
If (1) is not an option


connect the device via a vPC attached access switch (could use VDC to create a
“virtual access switch”).


PROS:

Ensures minimal disruption in case of peer
-
link failover and consistent behavior with vPC dual
-
active scenarios. Availability limited by the access switch failure.


CONS:
Need for an additional access switch or need to use one of the available VDCs. Additional
administrative burden to configure/manage the physical/Virtual Device

3.
If (2) is not an option


connect device directly to (primary) vPC peer in a non
-
vPC VLAN
*

and provide
for a separate interconnecting port
-
channel between the two vPC peers.


PROS:

Traffic diverted on a secondary path in case of peer
-
link failover


CONS:
Need to configure and manage additional ports (i.e. port
-
channel) between the Nexus 7000
devices.

4.
If (3) is not an option


connect device directly to (primary) vPC peer in a vPC VLAN


PROS:

Easy deployment


CONS:
VERY BAD.

Bound to vPC roles (no role preemption in vPC) , Full Isolation on peer
-
link failure
when attached vPC toggles to a secondary vPC role.

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

19

Attaching to a vPC Domain

vPC and non
-
vPC VLANs (i.e. single attached .. )

Orphan
Ports

Orphan
Ports

S

S

S

S

P

P

P

P

1. Dual Attached

2. Attached via VDC/Secondary Switch

3. Secondary ISL Port
-
Channel

4. Single Attached to vPC Device

Primary vPC

Secondary vPC

S

P

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

20

*

Run the same STP mode as the vPC domain. Enable portfast/port type edge on host facing ports

Attaching to a vPC Domain

”My device only does STP!”

Recommendations (in order of preference):

1.

ALWAYS

try
dual attach

devices using vPC


PROS:

Ensures minimal disruption in case of peer
-
link failover and consistent behavior with
vPC dual
-
active scenarios. Ensures full redundant active/active paths through vPC.


CONS:

None

2.
If (1) is not an option


connect the device via two independent links using STP. Use non
-
vPC VLANs ONLY on the STP switch.
*


PROS:

Ensures minimal disruption in case of peer
-
link failover and consistent behavior with
vPC dual
-
active scenarios. Ensures full redundant Active/Active paths on vPC VLANs.


CONS:

Requires an additional STP port
-
channel between the vPC devices. Operational
burden in provisioning and configuring separate STP and vPC VLAN domains. Only
Active/Standby paths on STP VLANs.

3.
If (2) is not an option


connect the device via two independent links using STP. (Use vPC
VLANs on this switch)


PROS:

Simplify VLAN provisioning and does not require allocation of an additional 10GE
port
-
channel.


CONS:

STP and vPC devices may not be able to communicate each other in certain failure
scenarios (i.e. when STP Root and vPC primary device do not overlap). All VLANs carried
over the peer
-
link may suspend until the two adjacency forms and vPC is fully
synchronized".

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

21

S

S

S

P

P

P

1. All devices Dual Attached via vPC

2. Separate vPC and STP VLANs

3. Overlapping vPC and STP VLANs

Primary vPC

Secondary vPC

Primary STP Root

Secondary STP Root

S

P

SR

PR

PR

SR

SR

PR

vPC Design principles

Attaching to a vPC Domain
-

vPC and non
-
vPC
VLANs

(STP/vPC Hybrid)

Non vPC port
-
channel

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

22


Multi
-
Layer vPC can join 8 active
ports port
-
channels in a unique 16
-
way port
-
channel
*


vPC peer side load
-
balancing is
LOCAL to the peer


Each vPC peer has only 8 active
links, but the pair has 16 active load
balanced links

Nexus
7000

Nexus
5000

*
Possible with any device supporting
vPC/MCEC and 8
-
way active port
-
channels

16
-
way port
channel

Attaching to a vPC Domain

16
-
way Port
-
Channel (1 of 2)

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

23


16 active ports between 8
active port
-
channel devices
and 16 active port
-
channel
devices?


vPC peer side load
-
balancing
is LOCAL to the peer


Each vPC peer has only 8
active links, but the pair has 16
active load balanced links to
the downstream device
supporting 16 active ports


D
-
series N7000 line cards will
also support 16 way active
port
-
channel load balancing,
providing for a potential 32
way vPC port channel!


Nexus
7000

Nexus
5000

Nexus 5000 16
-
port port
-
channel
support introduced in 4.1(3)N1(1a)
release

16
-
port port
-
channel

Attaching to a vPC Domain

16
-
way Port
-
Channel (2 of 2)

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

24

Agenda


Feature Overview & Terminology


vPC Design Guidance & Best Practices


Building a vPC domain


Attaching to a vPC domain


Layer 3 and vPC


Spanning Tree Recommendations


Data Center Interconnect (& Encryption)


HSRP with
vPC


vPC and Services


vPC latest
e
nhancements


ISSU


Convergence and Scalability


vPC Hands
-
on Lab Information


Reference Material

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

25

Router

7k1

7k2

Switch

Po1

Po2


Use separate L3 links to hook up routers to a vPC domain is still standing.


Don’t use L2 port channel to attach routers to a vPC domain unless you can
statically route to HSRP address


If both, routed and bridged traffic is required, use individual L3 links for routed
traffic and L2 port
-
channel for bridged traffic

Router

Switch

L3 ECMP

Po2

Layer 3 and vPC

Recommendations

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

26

vPC view

Layer 2 topology

Layer 3 topology

Port
-
channel looks like
a single L2 pipe.
Hashing will decide
which link to chose

Layer 3 will use ECMP
for northbound traffic

7k1

7k2

R

7k1

7k2

R

7k vPC

R

R could be any router,
L3 switch or VSS
building a port
-
channel

Layer 3 and vPC

What can happen… (1 of 3)

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

27

1)
Packet arrives at R

2)
R does lookup in routing table and sees 2
equal paths going north (to 7k1 & 7k2)

3)
Assume it chooses 7k1 (ECMP decision)

4)
R now has rewrite information to which
router it needs to go (router MAC 7k1 or
7k2)

5)
L2 lookup happens and outgoing
interface is port
-
channel 1

6)
Hashing determines which port
-
channel
member is chosen (say to 7k2)

7)
Packet is sent to 7k2

8)
7k2 sees that it needs to send it over the
peer
-
link to 7k1 based on MAC address

R

7k1

7k2

S

Po1

Po2

Layer 3 and vPC

What can happen… (2 of 3)

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

28

9)
7k1 performs lookup and sees that it
needs to send to S

10)
7k1 performs check if the frame came
over peer link & is going out on a vPC.

11)
Frame will only be forwarded if outgoing
interface is NOT a vPC
or

if outgoing
vPC doesn’t have active interface on
other vPC peer (in our example 7k2)


R

7k1

7k2

S

Po1

Po2

Layer 3 and vPC

What can happen… (3 of 3)

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

29

Agenda


Feature Overview & Terminology


vPC Design Guidance & Best Practices


Building a vPC domain


Attaching to a vPC domain


Layer 3 and vPC


Spanning Tree Recommendations


Data Center Interconnect (& Encryption)


HSRP with
vPC


vPC and Services


vPC latest
e
nhancements


ISSU


Convergence and Scalability


vPC Hands
-
on Lab Information


Reference Material

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

30

Spanning Tree Recommendations

Overview


STP Interoperability


STP Uses:


Loop detection (failsafe to vPC)


Non
-
vPC attached
device


Loop management
on
vPC addition/removal


Requirements:


Needs to remain enabled, but doesn’t dictate vPC member
port state


Logical ports still count, need to be aware of number of
VLANs
/port
-
channels deployed!


Best Practices:


Not recommended to enable Bridge Assurance feature on
vPC channels (i.e. no STP “network” port type). Tracked by
CSCsz76892
.


Make sure
all switches

in you layer 2 domain are running
with Rapid
-
PVST

or MST (
IOS

default is non
-
rapid
PVST
+),
to avoid
slow STP convergence
(30+
secs
)


Remember to configure
portfast

(edge port
-
type) on host
facing interfaces to avoid
slow STP convergence
(30+
secs
)


vPC

vPC

STP is running to manage
loops outside of vPC’s
direct domain, or before
initial vPC configuration

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

31

Spanning Tree Recommendations

Port Configuration Overview

Aggregation

Access

Data Center Core

B

L

R

N

E

BPDUguard

Loopguard

Rootguard

Network port

Edge or portfast port type

-

Normal port type

B

R

R

N

N

-

-

-

-

-

-

-

-

R

R

R

R

R

R

-

-

B

E

B

B

E

B

E

Layer 3

Layer 2 (STP + Rootguard)

Layer 2 (STP + BPDUguard)

L

E

Secondary

Root

HSRP

STANDBY

Primary

Root

HSRP

ACTIVE

E

-

Primary

vPC

Secondary

vPC

vPC

Domain

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

32

Agenda


Feature Overview & Terminology


vPC Design Guidance & Best Practices


Building a vPC domain


Attaching to a vPC domain


Layer 3 and vPC


Spanning Tree Recommendations


Data Center Interconnect (& Encryption)


HSRP with
vPC


vPC and Services


vPC latest
e
nhancements


ISSU


Convergence and Scalability


vPC Hands
-
on Lab Information


Reference Material

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

33

Long Distance

DC 1

DC 2

CORE

AGGR

ACCESS

Server Cluster

CORE

AGGR

ACCESS

Server Cluster

Key Recommendations




vPC Domain id for facing vPC layers should be different



No Bridge Assurance on interconnecting
vPCs



BPDU

Filter on the edge devices to avoid
BPDU

propagation



No
L3

peering between
DCs

(i.e.
L3

over vPC)

vPC domain
10

vPC domain
20

vPC domain
21

vPC domain
11

Rootguard

B

F

N

E

BPDUguard

BPDUfilter

Network port

Edge or portfast port type

-

Normal port type

R

E

E

-

-

-

-

-

-

-

-

-

-

F

F

F

F

-

-

-

-

-

-

-

B

B

N

N

N

N

N

N

N

N

R

R

-

R

R

R

R

R

R

Data Center Interconnect

Multi
-
layer vPC for Agg and DCI

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

34

Nexus 7010

Nexus 7010

DC
-
1

DC
-
2

Nexus 7010

Nexus 7010

vPC

vPC

Data Center Interconnect

Encrypted Interconnect

CTS Manual Mode
(802.1AE 10GE line
-
rate
encryption)

No ACS is required

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

35

Agenda


Feature Overview & Terminology


vPC Design Guidance & Best Practices


Building a vPC domain


Attaching to a vPC domain


Layer 3 and vPC


Spanning Tree Recommendations


Data Center Interconnect (& Encryption)


HSRP with
vPC


vPC and Services


vPC latest
e
nhancements


ISSU


Convergence and Scalability


vPC Hands
-
on Lab Information


Reference Material

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

36


Support for all FHRP protocols
in Active/Active mode with vPC


No additional configuration
required


Standby device communicates
with vPC manager produces to
determine if vPC peer is
“Active” HSRP/VRRP peer


General HSRP best practices
still applies.


When running active/active
aggressive timers can be
relaxed (i.e. 2
-
router vPC
case)

L3

L2

HSRP/VRRP
“Standby”:
Active for
shared L3 MAC

HSRP/VRRP
“Active”:
Active for
shared L3 MAC

HSRP with vPC

FHRP Active/Active

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

37

L2
/
L3

Aggregation

ACTIVE HSRP

STANDBY HSRP

GW

GW

GW

VLAN 100

VLAN 200

VLAN 100, 200

L3 CORE

Cautions:


Not
recommended using HSRP link tracking in a vPC configuration


Reason: vPC will not forward a packet back on a vPC once it has
crossed the peer
-
link, except in the case of a remote member port
failure

HSRP with vPC

Do NOT use Object Tracking

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

38


Use an OSPF point
-
to
-
point adjacency (or equivalent L3 protocol)
between the vPC peers to establish a L3 backup path to the Core
through in case of uplinks failure


A single point
-
to
-
point VLAN/SVI will suffice to establish a L3
neighborship.


L3

L2

OSPF

Primary

vPC

Secondary

vPC

OSPF

OSPF

VLAN 99

HSRP with vPC

L3 Backup Routing

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

39

Scenario
:


Provide L2/L3 interconnect between
L2 Pods, or between L2 attached
Datacenters (i.e. sharing the same
HSRP group).


A vPC domain without an active
HSRP instance in a group would not
able to forward traffic.

Multi
-
layer vPC with single HSRP:


L3 on the N7K supports
Active/Active on one pair, and still
allows normal HSRP behavior on
other pair (all in one HSRP group)


L3 traffic will run across Intra
-
pod
link for non Active/Active L3 pair



Active

Standby

Listen

Listen

HSRP with vPC

Dual L2/L3 Pod Interconnect

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

40

Agenda


Feature Overview & Terminology


vPC Design Guidance & Best Practices


Building a vPC domain


Attaching to a vPC domain


Layer 3 and vPC


Spanning Tree Recommendations


Data Center Interconnect (& Encryption)


HSRP with
vPC


vPC and Services


vPC latest
e
nhancements


ISSU


Convergence and Scalability


vPC Hands
-
on Lab Information


Reference Material

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

41



vPC and Services

Catalyst 6500 Services Chassis w. Services VDC Sandwich

Two Nexus 7000 Virtual Device Contexts used to “sandwich”
services between virtual switching layers


Layer
-
2 switching in Services Chassis with transparent
services


Services Chassis provides Etherchannel capabilities for
interaction with vPC


vPC running in both VDC pairs to provide Etherchannel for
both inside and outside interfaces to Services Chassis

Design considerations:


Access switches requiring services are connected to sub
-
aggregation VDC


Access switches not requiring services may be connected to
aggregation
VDC


May be extended to support multiple virtualized service
contexts by using multiple
VRF

instances in the sub
-
aggregation
VDC

Design Cautions:


Be aware of the Layer 3 over vPC design caveat. If Peering at
Layer 3 is required across the two vPC layers an alternative
solution should be explored (i.e. using STP rather than vPC to
attach service chassis)

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

42

Agenda


Feature Overview & Terminology


vPC Design Guidance & Best Practices


Building a vPC domain


Attaching to a vPC domain


Layer 3 and vPC


Spanning Tree Recommendations


Data Center Interconnect (& Encryption)


HSRP with
vPC


vPC and Services


vPC latest
e
nhancements


ISSU


Convergence and Scalability


vPC Hands
-
on Lab Information


Reference Material

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

43

Several enhancements to vPC:


vPC Object Tracking


vPC Peer
-
Gateway


vPC Delay Restore


Multi
-
layer vPC with single HSRP group


vPC
unicast

ARP handling


vPC Exclude Interface
-
VLAN


vPC single attached device Listing


vPC Convergence and Scalability

For more details:


4.2
Release
Notes

http
://www.cisco.com/en/US/docs/switches/datacenter/sw/4_2/nx
-
os/release/notes/42_nx
-
os_release_note.html#wp218085


vPC Latest Enhancements

Summary

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

44

vPC PL

vPC PKL


L3

L2

Scenario
:


Interoperability with non
RFC

compliant features of some NAS devices
(i.e.
NETAPP

Fast
-
Path or EMC IP
-
Reflect)


NAS device may reply to traffic using
the MAC address of the sender device
rather than the HSRP gateway.


Packet reaching vPC for the non local
Router MAC address are sent across the
peer
-
link and can be dropped if the final
destination is behind another vPC.

vPC Peer
-
Gateway Solution:


Allows a vPC switch to act as the
active gateway for packets addressed
to the peer router MAC (
CLI

command
added in the vPC global config)


Local Routing for peer
router

mac Traffic

N7k(config
-
vpc
-
domain)# peer
-
gateway

vPC Latest Enhancements

vPC Peer
-
Gateway for NAS interoperability

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

45

Agenda


Feature Overview & Terminology


vPC Design Guidance & Best Practices


Building a vPC domain


Attaching to a vPC domain


Layer 3 and vPC


Spanning Tree Recommendations


Data Center Interconnect (& Encryption)


HSRP with
vPC


vPC and Services


vPC latest
e
nhancements


ISSU


Convergence and Scalability


vPC Hands
-
on Lab Information


Reference Material

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

46


ISSU is still the recommended system
upgrade in a multi
-
device vPC environment


vPC system can be independently upgraded
with
no disruption to traffic
.


Upgrade is serialized and must be run one at
the time (i.e. config lock will prevent
synchronous upgrades)


Configuration is locked on “other” vPC peer
during ISSU.



Begin

End

Caveats

4.1(x)

4.2(x)

None

4.2(x)

4.1(x)

None

4.1(3)

4.1(3)

4.2(1)

4.1(3)

4.2(1)

4.2(1)

In
-
Service Software Upgrade (ISSU)

vPC System Upgrade/Downgrade

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

47

Agenda


Feature Overview & Terminology


vPC Design Guidance & Best Practices


Building a vPC domain


Attaching to a vPC domain


Layer 3 and vPC


Spanning Tree Recommendations


Data Center Interconnect (& Encryption)


HSRP with vPC


vPC latest
e
nhancements


ISSU


Convergence and Scalability


vPC Hands
-
on Lab Information


Reference Material

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

48

vPC Peer Link LACP
Channel (2x10 GigE)

vPC Peer
-
Keepalive (GigE)

L2/L3
Aggregation

Nexus 7000 vPC

L3 Core

Nexus 7000

L2 Access

Nexus 5000

N7K
-
1

N7K
-
2

Po10

20 flows @1000
pps

20 flows @1000
pps

20 flows @1000
pps

Po20

Po160

16
-
way port
-
channel

4
-
way port
-
channel

OSPF

OSPF

4.2(1) vPC Enhancements

Convergence Topology

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

49

vPC on Nexus 7000

Convergence Numbers

Failover case

Failure Topology

Convergence Time

Failure

Restoration

Failure of
secondary vPC
peer*

4.1(4)

North
-
Bound: ~
700 ms

South
-
Bound:
~2.5 sec

4.1(4)

North
-
Bound: ~
3 sec

South
-
Bound:

~3.4 sec


4.2(1)

North
-
Bound:
~50
ms
.

South
-
Bound:
~100 ms

4.2(1)

North
-
Bound:
100


900 ms

South
-
Bound:
1.2
-
2 s

Failure of a
primary vPC peer*


4.1(4)

North
-
Bound:
~150 ms

South
-
Bound:
~3 sec

4.1(4)

North
-
Bound:
~4.5
secs

South
-
Bound:
~5
secs


4.2(1)

North
-
Bound: ~
50 ms

South
-
Bound:
~100 ms

4.2(1)

North
-
Bound:
~400 ms
-
1.5 s

South
-
Bound: ~
1.5 s

Failover of the
vPC Peer Link

4.1(4)

North
-
Bound:
~1.3 s

South
-
Bound:

~1.8 s

4.1(4)

North
-
Bound:
~900 ms

South
-
Bound:

up to 10+ s (
CSCsz88998
)

4.2(1)

North
-
Bound:
100
-
300 ms

South
-
Bound:

50
-
500 ms

4.2(1)

North
-
Bound:

150
-

900 ms

South
-
Bound:
~ 900 ms

1.5 s


NOTE
: Convergence numbers may vary depending on the specific configuration (i.e. scaled
number of VLANs/SVIs or HSRP groups) and traffic patterns (i.e. L2 vs L3 flows).

P

S

P

S

P

S

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

50

vPC on Nexus 7000

Scalability Number Improvements

Release

Supported Scalability

4.1(5)

192 vPC’s (2
-
port) with the following,

200 VLANs

200 HSRP Groups

40K MACs & 40K ARPs

10K (S,G) w. 66 OIFs (L3 sources)

3K (S,G) w. 34 OIFs (L2 sources)

Latest

Ankara
4.2(1)

256
vPC’s

(4
-
port)
with the following,

260
VLANs


200
SVI
/
HSRP

Groups

40k

MACs

&
40K

ARPs


10K

(
S,G
) w. 66
OIFs

(
L3

sources)


3K

(
S,G
) w.

64
OIFs

(
L2

sources)

NOTE
:

Supported numbers of
VLANs
/
vPCs

are NOT related to an hardware or software limit but reflect what
has been currently validated by our QA. The
N7k

BU is planning to
continuously increase these numbers

as
soon as new data
-
points become available.

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

51

Agenda


Feature Overview & Terminology


vPC Design Guidance & Best Practices


Building a vPC domain


Attaching to a vPC domain


Layer 3 and vPC


Spanning Tree Recommendations


Data Center Interconnect (& Encryption)


HSRP with
vPC


vPC and Services


vPC latest
e
nhancements


ISSU


Convergence and Scalability


vPC Hands
-
on Lab Information


Reference Material

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

52

vPC Hands
-
on Lab Information

On Demand vPC Lab Overview

Pod 1

Pod 2

Pod 3

Pod 4

Pod 5

Pod 6

N7K
-
1

POD 1
-
2 VPC


N7K
-
2

POD 1
-
2 VPC


N7K
-
4

POD 3
-
4 VPC


N7K
-
3

POD 3
-
4 VPC


N7K
-
8

POD 5
-
6 VPC


N7K
-
7

POD 5
-
6 VPC


Pod 1

Pod 2

N7K
-
Aggr

N7K
-
Aggr


Instructor
-
led hands
-
on lab
introducing the vPC (virtual Port
-
channel) feature for the Nexus 7000.


Participants exposed to the
configuration of vPC with NX
-
OS.


Lab needs to be manually booked
through Nexus 7000 TMEs.

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

53

vPC Hands
-
on Lab Information


vPC Lab Logistics and Timing


The vPC Laboratory consists of 6 independent
PODs
.



A group of 2 students is assigned to each Pod.



Each student will configure a vPC peer device.



PODs

are logically independent. Two adjacent
PODs

are physically
bound to the same Nexus. Virtual Device Contexts (
VDCs
) are used to
define logically independent devices on the same Nexus 7010 box.



The vPC Lab session is expected to be completed in around two hours.


© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

54

Agenda


Feature Overview & Terminology


vPC Design Guidance & Best Practices


Building a vPC domain


Attaching to a vPC domain


Layer 3 and vPC


Spanning Tree Recommendations


Data Center Interconnect (& Encryption)


HSRP with
vPC


vPC and Services


vPC latest
e
nhancements


ISSU


Convergence and Scalability


vPC Hands
-
on Lab Information


Reference Material

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

55

L2/L3 Aggregation


Nexus 7000 vPC

L3 Core

L2 Access


6500 VSS

E1/25

Te2/2/1

E1/26

Te1/2/1

Po10

vPC Peer Link LACP
Channel (2x10 GigE)

vPC Peer
-
Keepalive (GigE)

Po100

VSS VSL Channel
(2x10 GigE)

N7K
-
1

N7K
-
2

6K
-
2

6K
-
1

Po100

Physical

Logical

Reference Material

vPC/
VSS

Interop

Test Details

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

56


The following scenarios were tested:


VSS

and vPC member failover and convergence


Dual active scenarios and behavior


Best practice guidelines for STP,
L3

(NSF), Multicast


Catalyst 6500/Nexus 7000 interoperability:


Multiple ports per chassis act as one larger ether
-
channel

Reference Material

vPC/
VSS

Interop

Test Details

© 2009 Cisco Sy stems, Inc. All rights reserv ed.

Cisco Conf idential

57


Enterprise Solutions Engineering:

http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DC_3_0/DC
-
3_0_IPInfra.html


Implementing Nexus 7000 in the Data Center Aggregation
Layer with Services
:

https://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/nx_7000_dc.html


Configuration Guide for Object Tracking Feature:

http://www.cisco.com/en/US/partner/docs/switches/datacenter/sw/4_2/nx
-
os/interfaces/configuration/guide/if_vPC.html#wp1530133


vPC white Paper:

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/white_paper_c11
-
516396.html


Reference Material

Other Solution Tests and Recent vPC Documentation