Supporting BGP Backup Paths in NX-OS

Author
Carole Warner Reece
Architect

I have been looking at using a service provider MPLS VPN to back up an enterprise MAN connections where the core routers are Nexus 7000s (N7Ks).

Basic Premise

An organization called Gamma has two main sites, DC and HQ, in the same city. The two divisions are in different buildings connected with a 10GE fiber and EIGRP. They use Nexus 7000s (N7Ks) as the core devices, and one N7K at each of the main sites is a BGP peer with the service provider to provide a backup path to the 10GE fiber link. The DC-02 and HQ-02 routers each send an aggregate address for it’s site subnets to the SP.

Gamma uses the 10GE link as the primary path between their sites. Gamma has been supporting this preference by simply using the more EIGRP specific routes over the BGP summaries.
2013_02_12_nx-os-bgp-backdoor

However, as the number of networks at each site has grown, now Gamma wants to summarize the networks between each site with EIGRP summary addresses to reduce the complexity and length of the routing table. They would still like to use BGP with the service provider to back up the 10GE fiber link. The issue is the internal EIGRP summary route has a distance of 90, while the external BGP route has a distance of 20.

Typical Process

In Cisco IOS, this issue is easily overcome with the network address backdoor command in BGP. Here is a quick recap of BGP backup networks: The network address backdoor command can be used under the BGP routing process to mark routes received from BGP as ‘back-up’ routes to the same routes received from an IGP process. The backdoor BGP routes will be treated as a locally assigned network, except BGP updates do not advertise this network. The result is that if a router learns a backdoor route from both BGP and an IGP, the IGP route will be placed in the routing table even though it has a higher admin distance.

With the network backdoor under IOS, the BGP table would look like this:

HQ-02# sh ip bgp
BGP table version is 43, local router ID is 10.13.24.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
 r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

 Network          Next Hop           Metric LocPrf Weight Path
r> 10.12.0.0/16    192.168.203.165                       0 64323 64666 i
r  10.13.0.0/16    192.168.203.165                       0 64323 64666 i
r>                 0.0.0.0                            32768 i
HQ-02#

NX-OS Complication

However, NX-OS does not currently support the network address backdoor command. Since some organizations may want to use N7K as their enterprise core, I thought I would describe another way to support BGP as a backup route that works in NX-OS.

My plan to support the BGP as a back up route is to use the distance command under one of the routing process. Since Gamma only has two devices running BGP, and many more running EIGRP, it makes most sense to adjust the distance on the two BGP processes.

Also, Cisco already tweaks the admin distance in NX-OS:

Route Source Default Distance – IOS Default Distance – NX-OS
Connected Interface 0 0
EIGRP Summary Route 5 5
External BGP 20 20
Internal EIGRP 90 90
External EIGRP 170 170
Internal BGP 200 200
Local BGP 200 220

Configuration Updates for Gamma

Since the Gamma BGP routers are already redistributing BGP routes into EIGRP, Gamma decides to set the external BGP distance to 165. This value will be greater than the internal EIGRP admin distance of 90. (The EIGRP summary route for the remote division will appear as an EIGRP internal route on the HQ and DC routers.) The 165 value for the BGP route will be lower than the redistributed BGP route, which has an admin distance of 170.

First the EIGRP summary routes will be configured on HQ-01 and DC-01 as shown:

! DC-01
interface ethernet1/8
 ip summary-address eigrp 100 10.12.0.0/16
!

. . .

! HQ-01
interface ethernet1/8
 ip summary-address eigrp 100 10.13.0.0/16
!

The distance command will be configured HQ-02 and DC-02  as shown:

! DC-02
router bgp 64666
 address-family ipv4 unicast
 distance 165 200 220
!

. . .

! HQ-02
router bgp 64666
 address-family ipv4 unicast
 distance 165 200 220
!

Typical State

When the link between HQ and DC is up, HQ-01 and HQ-02 will have 10.12.0.0/16 as an EIGRP internal route. Similarly, HD-01 and DC-02 will have 10.13.0.0/16 as an EIGRP internal route.

For example, here is the routing table entry for 10.13.0.0 on HD-02 when the link between HQ and DC is down:

!
HD-02# sh ip ro 10.13.0.0
Routing entry for 10.13.0.0/16
 Known via "eigrp 100", distance 90, metric 156160, type internal
 Redistributing via eigrp 100
 Last update from 10.12.248.1 on Ethernet3/26, 00:00:14 ago
 Routing Descriptor Blocks:
 * 10.12.248.1, from 10.12.248.1, 00:00:14 ago, via Ethernet3/26
    Route metric is 156160, traffic share count is 1
    Total delay is 5100 microseconds, minimum bandwidth is 100000 Kbit
    Reliability 255/255, minimum MTU 1500 bytes
    Loading 1/255, Hops 1

HD-02#

Backup State

If the link between HQ and DC goes down, then the BGP route from the SP will valid in the BGP table and the routing table.

HQ-02# sh ip bgp
BGP table version is 20, local router ID is 10.13.24.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
 r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

 Network          Next Hop          Metric LocPrf Weight Path
r> 10.12.0.0/16    192.168.90.77          0            0 64323 i
*> 10.13.0.0/16    192.168.90.77                       0 64323 64666 i
HQ-02#

The BGP route will be redistributed at HQ-02 and DC-02 into EIGRP so that all devices will have connectivity through the backup path through the SP.

For example, here is the routing table entry for 10.13.0.0 on HD-01 when the link between HQ and DC is down:

!
HD-01# sh ip ro 10.13.0.0
Routing entry for 10.13.0.0/16
 Known via "eigrp 100", distance 170, metric 96000
 Tag 64323, type external
 Redistributing via eigrp 100
 Last update from 10.12.248.2 on Ethernet3/26, 00:00:24 ago
 Routing Descriptor Blocks:
 * 10.12.248.2, from 10.12.248.2, 00:00:24 ago, Ethernet3/26
    Route metric is 96000, traffic share count is 1
    Total delay is 200 microseconds, minimum bandwidth is 28169 Kbit
    Reliability 255/255, minimum MTU 1500 bytes
    Loading 1/255, Hops 1
    Route tag 64323

HD-01#

Summary

Typically, Cisco recommends using the BGP backdoor command instead of adjusting the BGP admin distance as discussed in a BGP Case Study http://www.cisco.com/en/US/tech/tk365/technologies_tech_note09186a00800c95bb.shtml#bgpbackdoor
However, if the network core is NX-OS, you can adjust the admin distance to create a BGP backup route.

— cwr

__________________________________________________________________________

If you would like some additional on NX-OS, you may want to review the following articles:

One response to “Supporting BGP Backup Paths in NX-OS

  1. With as simple a network as this … why don’t they do a floating static from HQ-02 DC-02 with a higher admin distance the EIGRPs ?
    If the 10G link goes down .. there is only one – static – other way to get across the fjord .

Leave a Reply

 

Nick Kelly

Cybersecurity Engineer, Cisco

Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.