Supporting BGP Backup Paths in NX-OS

I have been looking at using a service provider MPLS VPN to back up an enterprise MAN connections where the core routers are Nexus 7000s (N7Ks).

Basic Premise

An organization called Gamma has two main sites, DC and HQ, in the same city. The two divisions are in different buildings connected with a 10GE fiber and EIGRP. They use Nexus 7000s (N7Ks) as the core devices, and one N7K at each of the main sites is a BGP peer with the service provider to provide a backup path to the 10GE fiber link. The DC-02 and HQ-02 routers each send an aggregate address for it’s site subnets to the SP.

Gamma uses the 10GE link as the primary path between their sites. Gamma has been supporting this preference by simply using the more EIGRP specific routes over the BGP summaries.

However, as the number of networks at each site has grown, now Gamma wants to summarize the networks between each site with EIGRP summary addresses to reduce the complexity and length of the routing table. They would still like to use BGP with the service provider to back up the 10GE fiber link. The issue is the internal EIGRP summary route has a distance of 90, while the external BGP route has a distance of 20.

Typical Process

In Cisco IOS, this issue is easily overcome with the network address backdoor command in BGP. Here is a quick recap of BGP backup networks: The network address backdoor command can be used under the BGP routing process to mark routes received from BGP as ‘back-up’ routes to the same routes received from an IGP process. The backdoor BGP routes will be treated as a locally assigned network, except BGP updates do not advertise this network. The result is that if a router learns a backdoor route from both BGP and an IGP, the IGP route will be placed in the routing table even though it has a higher admin distance.

With the network backdoor under IOS, the BGP table would look like this:

HQ-02# sh ip bgp
BGP table version is 43, local router ID is 10.13.24.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
 r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

 Network          Next Hop           Metric LocPrf Weight Path
r> 10.12.0.0/16    192.168.203.165                       0 64323 64666 i
r  10.13.0.0/16    192.168.203.165                       0 64323 64666 i
r>                 0.0.0.0                            32768 i
HQ-02#

NX-OS Complication

However, NX-OS does not currently support the network address backdoor command. Since some organizations may want to use N7K as their enterprise core, I thought I would describe another way to support BGP as a backup route that works in NX-OS.

My plan to support the BGP as a back up route is to use the distance command under one of the routing process. Since Gamma only has two devices running BGP, and many more running EIGRP, it makes most sense to adjust the distance on the two BGP processes.

Also, Cisco already tweaks the admin distance in NX-OS:

Route Source	Default Distance – IOS	Default Distance – NX-OS
Connected Interface	0	0
EIGRP Summary Route	5	5
External BGP	20	20
Internal EIGRP	90	90
External EIGRP	170	170
Internal BGP	200	200
Local BGP	200	220

Configuration Updates for Gamma

Since the Gamma BGP routers are already redistributing BGP routes into EIGRP, Gamma decides to set the external BGP distance to 165. This value will be greater than the internal EIGRP admin distance of 90. (The EIGRP summary route for the remote division will appear as an EIGRP internal route on the HQ and DC routers.) The 165 value for the BGP route will be lower than the redistributed BGP route, which has an admin distance of 170.

First the EIGRP summary routes will be configured on HQ-01 and DC-01 as shown:

! DC-01
interface ethernet1/8
 ip summary-address eigrp 100 10.12.0.0/16
!

. . .

! HQ-01
interface ethernet1/8
 ip summary-address eigrp 100 10.13.0.0/16
!

The distance command will be configured HQ-02 and DC-02  as shown:

! DC-02
router bgp 64666
 address-family ipv4 unicast
 distance 165 200 220
!

. . .

! HQ-02
router bgp 64666
 address-family ipv4 unicast
 distance 165 200 220
!

Typical State

When the link between HQ and DC is up, HQ-01 and HQ-02 will have 10.12.0.0/16 as an EIGRP internal route. Similarly, HD-01 and DC-02 will have 10.13.0.0/16 as an EIGRP internal route.

For example, here is the routing table entry for 10.13.0.0 on HD-02 when the link between HQ and DC is down:

!
HD-02# sh ip ro 10.13.0.0
Routing entry for 10.13.0.0/16
 Known via "eigrp 100", distance 90, metric 156160, type internal
 Redistributing via eigrp 100
 Last update from 10.12.248.1 on Ethernet3/26, 00:00:14 ago
 Routing Descriptor Blocks:
 * 10.12.248.1, from 10.12.248.1, 00:00:14 ago, via Ethernet3/26
    Route metric is 156160, traffic share count is 1
    Total delay is 5100 microseconds, minimum bandwidth is 100000 Kbit
    Reliability 255/255, minimum MTU 1500 bytes
    Loading 1/255, Hops 1

HD-02#

Backup State

If the link between HQ and DC goes down, then the BGP route from the SP will valid in the BGP table and the routing table.

HQ-02# sh ip bgp
BGP table version is 20, local router ID is 10.13.24.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
 r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

 Network          Next Hop          Metric LocPrf Weight Path
r> 10.12.0.0/16    192.168.90.77          0            0 64323 i
*> 10.13.0.0/16    192.168.90.77                       0 64323 64666 i
HQ-02#

The BGP route will be redistributed at HQ-02 and DC-02 into EIGRP so that all devices will have connectivity through the backup path through the SP.

For example, here is the routing table entry for 10.13.0.0 on HD-01 when the link between HQ and DC is down:

!
HD-01# sh ip ro 10.13.0.0
Routing entry for 10.13.0.0/16
 Known via "eigrp 100", distance 170, metric 96000
 Tag 64323, type external
 Redistributing via eigrp 100
 Last update from 10.12.248.2 on Ethernet3/26, 00:00:24 ago
 Routing Descriptor Blocks:
 * 10.12.248.2, from 10.12.248.2, 00:00:24 ago, Ethernet3/26
    Route metric is 96000, traffic share count is 1
    Total delay is 200 microseconds, minimum bandwidth is 28169 Kbit
    Reliability 255/255, minimum MTU 1500 bytes
    Loading 1/255, Hops 1
    Route tag 64323

HD-01#

Summary

Typically, Cisco recommends using the BGP backdoor command instead of adjusting the BGP admin distance as discussed in a BGP Case Study http://www.cisco.com/en/US/tech/tk365/technologies_tech_note09186a00800c95bb.shtml#bgpbackdoor
However, if the network core is NX-OS, you can adjust the admin distance to create a BGP backup route.

— cwr

__________________________________________________________________________

If you would like some additional on NX-OS, you may want to review the following articles:

• NX-OS QoS Hints and Quirks
• Deny Equals Permit in NX-OS QoS ACLs
• QoS on the Nexus 5000/2000 – Part 1
• (Configuring) QoS on the Nexus 5000/2000 – Part 2
• Trust Boundaries and QoS on the Nexus 5000/2000 – Part 3