Implement MPLS VPNs

munchdrabNetworking and Communications

Oct 30, 2013 (3 years and 10 months ago)

96 views

How to Secure Your Job:
Implement MPLS VPNs

roger@gottsponer.ch


NANOG24 / Miami, Florida

February 10
-
12, 2002

2

Agenda


The Idea behind this Talk


Lessons Learned


The Service Architecture


The Use of Route Reflectors


Baby Jumbo Frames


The LDP
-
ID


Traceroute is no longer Your Friend


Some Security Considerations


no Religious Wars please :
-
)

3

Agenda


The Idea behind this Talk


Lessons Learned


The Service Architecture


The Use of Route Reflectors


Baby Jumbo Frames


The LDP
-
ID


Traceroute is no longer Your Friend


Some Security Considerations


no Religious Wars please :
-
)

4

The Idea behind this Talk


This talk should show some lessons we learned when

implementing and running a real life MPLS VPN network



Nextra Switzerland is running a national MPLS VPN

network since 2.5 years (Q3 1999)


about 30 PoPs, Cisco based



The Nextra Group is running a pan
-
European MPLS VPN

network since about one year (Q1 2001)


present in most European countries


provides seamless VPN services between autonomous Nextra
companies



I will not discuss whether MPLS is a good thing or not :
-
)

5

Agenda


The Idea behind this Talk


Lessons Learned


The Service Architecture


The Use of Route Reflectors


Baby Jumbo Frames


The LDP
-
ID


Traceroute is no longer Your Friend


Some Security Considerations


no Religious Wars please :
-
)

6

Lessons Learned

A reload a day

keeps the TAC away










just kidding…

7

Agenda


The Idea behind this Talk


Lessons Learned


The Service Architecture


The Use of Route Reflectors


Baby Jumbo Frames


The LDP
-
ID


Traceroute is no longer Your Friend


Some Security Considerations


no Religious Wars please :
-
)

8

Service Architecture


Is entirely based on MPLS VPN ‘Routing Realms’



Each Added Value Service has its own Routing Realm/VPN



Each Customer has several Routing Realms/VPNs



Customers are given access to Services by
interconnecting ‘Customer’ VPNs and ‘Service’ VPNs



Limited Set of Building Blocks


Full Mesh Routing Realm


Hub/Spoke Realm


(Point
-
Point Realm)


Inter
-
Realm ‘Conduits’


Inter
-
Realm ‘Adapters’

9

Lego
-
Brick #1: Full
-
Mesh VPN

10

Lego
-
Brick #2: Hub & Spoke VPN

Hub

Spoke

Spoke

Spoke

11

Lego
-
Brick #3: Conduits

Resource

X

Mbps

12

Lego
-
Brick #4: Adapters

Resource

13

The Four Network Building Blocks

14

Multiple VPNs per Customer

15

Managed Firewall IAS for PN

Internet

Nextra Managed Firewall

16

Standard VPN IAS

Internet

Customer Managed Firewall

17

Redundant Load Sharing PN

POP

Redundant

Access

HSRP

18

Disaster Recovery VPN

Backup

Computing Center

(IP: 10.5.5.0/24)

POP A

Primary

Computing Center

(IP: 10.5.5.0/24)

POP B

Fast (?)

Re
-
routing

19

PN Dial Backup

ISDN

VPDN Home Gateway

20

PartnerNet

Customer

A

Customer

B

PartnerNet

21

Agenda


The Idea behind this Talk


Lessons Learned


The Service Architecture


The Use of Route Reflectors


Baby Jumbo Frames


The LDP
-
ID


Traceroute is no longer Your Friend


Some Security Considerations


no Religious Wars please :
-
)

22

The Use of Route
-
Reflectors


You do not want to maintain a full VPNv4
-
iBGP mesh


Use at least two VPNv4
-
RRs for redundancy reasons



Do not use the IPv4
-
RRs as VPNv4 Route
-
Reflectors


IPv4 Routing instabilities could affect VPN routing


If PE reloads it will get all 100’000 IPv4 routes first. So it will take
several minutes until the VPN routes are restored


If RRs are separated, both feeds are done simultaneously


So you need at least four RRs
--
> what about core routers?



Use global
-
local
-
massive
-
extreme unique RDs


A different RD per VRF, not per VPN


otherwise RR will break iBGP load
-
sharing



Check next
-
hops of BGP routes you get from the RR :
-
)

23

Agenda


The Idea behind this Talk


Lessons Learned


The Service Architecture


The Use of Route Reflectors


Baby Jumbo Frames


The LDP
-
ID


Traceroute is no longer Your Friend


Some Security Considerations


no Religious Wars please :
-
)

24

Baby Jumbo Frames


MPLS labels are 4 Bytes long


Available MTU on a link gets decreased by x*4 Bytes


Due to Path
-
MTU
-
Discovery, the sender will detect this


Path
-
MTU
-
Discovery seems to be broken sometimes


Customer will blame you, not the broken PMTUD


“everything worked fine before we moved to your MPLS network!”


Workaround:


allow 1500+(x*4) Bytes packets over Ethernet (Baby Jumbo Frames)


Special config command:
tag
-
switching mtu 1508


oversized packets only allowed if oversize is caused by MPLS labels


Before you migrate to MPLS, make sure all your routers,
interfaces and switches support Baby Jumbo Frames!

25

Agenda


The Idea behind this Talk


Lessons Learned


The Service Architecture


The Use of Route Reflectors


Baby Jumbo Frames


The LDP
-
ID


Traceroute is no longer Your Friend


Some Security Considerations


no Religious Wars please :
-
)

26

The LDP
-
ID


After a crash, a P router stopped forwarding any traffic...


interface loopback99

description test test test

ip address 222.222.222.222 255.255.255.255



Like in OSPF, LDP uses highest loopback address as ID


Unlike in OSPF, LDP
-
ID has to be reachable


LDP
-
ID not reachable


no LDP TCP
-
session


no labels!



So always configure the LDP
-
ID manually!

27

Agenda


The Idea behind this Talk


Lessons Learned


The Service Architecture


The Use of Route Reflectors


Baby Jumbo Frames


The LDP
-
ID


Traceroute is no longer Your Friend


Some Security Considerations


no Religious Wars please :
-
)

28

Traceroute is no longer Your Friend

Traceroute stops at PE router !

29

A Multi ISP Environment

ISP A

ISP B

Trans it ISP

Traceroute stops at PE router !

30

Customer Complains about Strange Traceroute

CPE
-
Router#traceroute www.cisco.com


1 accesslink.company.com (10.150.73.17) 0 msec 4 msec 0 msec

<
--

PE (Europe)

2 r16b01
-
pos0
-
0
-
0.bb.nextra.net (141.22.5.2)
104 msec 104 msec 100 msec
<
--

P (Europe)

3 r08b02
-
pos1
-
0
-
0. bb.nextra.net (141.22.1.6) 104 msec 104 msec 104 msec
<
--

P (Europe)

4 r08b01
-
fe4
-
1
-
0. bb.nextra.net (141.22.5.49) 148 msec 200 msec 200 msec
<
--

P (Europe)

5 r10b01
-
pos4
-
0
-
0. bb.nextra.net (141.22.5.34) 160 msec 200 msec 200 msec
<
--

PE (USA)

6 Pos4
-
0.GW8.NYC4.ALTER.NET (157.130.21.37) 100 msec 100 msec 100 msec

7 172.ATM2
-
0.XR2.NYC4.ALTER.NET (146.188.180.62) 104 msec 104 msec 104 msec

8 188.at
-
1
-
0
-
0.TR2.NYC8.ALTER.NET (152.63.21.106) 108 msec 104 msec 104 msec

9 124.at
-
6
-
0
-
0.TR2.SAC1.ALTER.NET (152.63.6.14) 180 msec 176 msec 180 msec

10 190.ATM6
-
0.GW8.SJC2.ALTER.NET (152.63.52.181) 184 msec 188 msec 184 msec

11 cisco.customer.alter.net (157.130.200.30) 188 msec 188 msec 188 msec

12 pigpen.cisco.com (128.107.240.170) 188 msec 184 msec 184 msec

13 * www.cisco.com (198.133.219.25) 184 msec *

31

What a Professional Troubleshooter Told Us :
-
)


Hi,


I think I can explain what is happening. Any IP packets received with IP
options that require processing are process switched. That means that the
packet is sent to the Route Processor for forwarding, rather than switched
directly using CEF. If the RP is busy, then this may add to the latency.


The traceroute command uses IP options in the header, so these packets
will be process switched. Any other traffic, such as http, ftp etc will be
switched using CEF. So when they run the traceroute command, they are
seeing the latency of the packet being forwarded through the RP, rather
than the true latency of the packets being fast switched by CEF.


Regards,


Your professional troubleshooter

32

The Real Reason


PE sends back ICMP Time Exceeded via appropriate
Routing Table


P router has no routing information!


Only information he has is receiving LSP, but LSPs are unidirectional


So P router sends ICMP Time Exceeded via receiving LSP


ICMP packet travels through the MPLS backbone to the other PE,
which then has the routing information to send the ICMP packet back
through another LSP to the sender


So each traceroute probe travels through the whole MPLS backbone
twice


TTL is high!



Customer can accept this because his productive traffic
is not affected

33

But How to Explain this to a Customer?

CPE
-
Router#traceroute www.cisco.com


1 accesslink.company.com (10.150.73.17) 0 msec 4 msec 0 msec

<
--

PE (Europe)

2 r16b01
-
pos0
-
0
-
0.bb.nextra.net (141.22.5.2) 104 msec 104 msec 100 msec

<
--

P (Europe)

3 r08b02
-
pos1
-
0
-
0. bb.nextra.net (141.22.1.6)
15 msec 12 msec 22 msec

<
--

P (Europe)

4 r08b01
-
fe4
-
1
-
0. bb.nextra.net (141.22.5.49)
16 msec 17 msec 14 msec

<
--

P (Europe)

5 r10b01
-
pos4
-
0
-
0. bb.nextra.net (141.22.5.34) 160 msec 200 msec 200 msec
<
--

PE (USA)

6 Pos4
-
0.GW8.NYC4.ALTER.NET (157.130.21.37) 100 msec 100 msec 100 msec

7 172.ATM2
-
0.XR2.NYC4.ALTER.NET (146.188.180.62) 104 msec 104 msec 104 msec

8 188.at
-
1
-
0
-
0.TR2.NYC8.ALTER.NET (152.63.21.106) 108 msec 104 msec 104 msec

9 124.at
-
6
-
0
-
0.TR2.SAC1.ALTER.NET (152.63.6.14) 180 msec 176 msec 180 msec

10 190.ATM6
-
0.GW8.SJC2.ALTER.NET (152.63.52.181) 184 msec 188 msec 184 msec

11 cisco.customer.alter.net (157.130.200.30) 188 msec 188 msec 188 msec

12 pigpen.cisco.com (128.107.240.170) 188 msec 184 msec 184 msec

13 * www.cisco.com (198.133.219.25) 184 msec *

BUG!

34

Agenda


The Idea behind this Talk


Lessons Learned


The Service Architecture


The Use of Route Reflectors


Baby Jumbo Frames


The LDP
-
ID


Traceroute is no longer Your Friend


Some Security Considerations


no Religious Wars please :
-
)

35

Configuration Integrity


Sometimes config lines for a vrf routing enviroment

appear on the global routing process level


Some buggy software versions


Typo in vrf name


Copy
-
Paste works too fast :
-
)



This can be very very very dangerous



Sometimes someone puts a interface into a wrong VRF :
-
(



The problem is not when you break connectivity (easy to
spot) but when you add too much connectivity



Sanity Check Tools!

36

Configuration Integrity


How it should look like


router bgp 88


[usual iBGP stuff]


no auto
-
summary


!


address
-
family ipv4 vrf green


redistribute connected


redistribute rip


neighbor 211.27.213.9 remote
-
as 65500


neighbor 211.27.213.9 activate


default
-
information originate


no auto
-
summary


no synchronization


exit
-
address
-
family


What I found on a router


router bgp 88


redistribute rip


default
-
information originate

37

Configuration Integrity


How it should look like


router rip


version 2


no auto
-
summary


!


address
-
family ipv4 vrf blue


version 2


redistribute bgp 88 metric 3


network 211.27.213.0


distribute
-
list 22 in


no auto
-
summary


exit
-
address
-
family


What I found on a router


router rip


redistribute bgp 88 metric 3


network 211.27.213.0


Imagine what happens if you use
addresses from the 62/8 block!


Another reason for good filtering


(Preconfigure all possible redistri
-
butions with route
-
map “deny
-
all”)

38

Secure Your PE VTYs!

line vty 0 4


access
-
class 99 in


login authentication bbmethod


access
-
list 99 permit 10.25.25.5 0.0.0.0
<
--

NMS box

access
-
list 99 permit 61.8.0.0 0.0.7.255
<
--

BB
-
links



What happens, if customer sends hostroute

for your NMS box?



He will be able to telnet to your PE router!

39

Secure Your PE VTYs!



only use hostroutes for NMS


RIP, OSPF, EBGP has better admin distance than IBGP:
-
(



put hostroutes to Null0 for NMS into all VRFs


how to manage the CPEs?



filter routing updates from customer


do not accept all internal address ranges (NMS, BB)


secure, but could impact customer connectivity

40

Secure Your PE VTYs!


use better ACLs to secure the VTYs


interface loopback0

ip address 1.1.1.1 255.255.255.255


access
-
list 199 permit 10.25.25.5 0.0.0.0 1.1.1.1 0.0.0.0

access
-
list 199 permit 61.8.0.0 0.0.7.255 1.1.1.1 0.0.0.0


access
-
list 199 permit 10.25.25.5 0.0.0.0 61.8.0.0 0.0.7.255

access
-
list 199 permit 61.8.0.0 0.0.7.255 61.8.0.0 0.0.7.255



needs entries for BB interface addresses as telnet

destination (hop
-
to
-
hop telnet)


as BB interface addresses and loopback addresses are

not reachable within the vrfs, this should be secure


needs a nice IP addressing scheme (or a clever config generation tool)

41

That’s it!

Thank you for your attention

Now, it is time for



questions


comments


corrections


flames

How to Secure Your Job:
Implement MPLS VPNs

roger@gottsponer.ch


NANOG24 / Miami, Florida

February 10
-
12, 2002