I have been thinking about best practices for configuring Nexus 7000 switches, and most recently about Layer 2 best practices. One of my customers wanted to set his root bridge and secondary bridge for his VLANs to be his N7Ks. This is a best practice recommended by Cisco. With virtual Port-Channels (vPCs), you should configure the spanning tree root and secondary roots for the member VLANs to be on the N7Ks. Cisco also recommends that you match the primary root bridge and the vPC primary.
However, I noticed that the “Data Center Aggregation Layer Design and Configuration with Cisco Nexus Switches and Virtual PortChannels” document suggested that you “Configure the spanning tree root and secondary root priorities as usual.” That started me thinking about what was usual and unusual about configuring spanning tree roots. The Cisco NX-OS/IOS STP Comparison wiki suggests using the spanning-tree vlan vlan-id root [primary | secondary] command to configure the root and secondary root.
I looked at the command references for IOS and NX-OS, and found there were some differences between what the Cisco IOS and NX-OS documentation state the spanning-tree vlan root macro will do. My concern is that you could run into a scenario where you set one N7K for primary root successfully, but when you set the secondary root it would stay on some older IOS device. Obviously it was time for some lab testing.
Background
So what is the difference in what the macro is supposed to do? According to Cisco IOS LAN Switching Command Reference, the spanning-tree vlan root primary initially alters a device’s bridge priority to 8192. If the device does not become the root then the bridge priority is changed to 100 less than the bridge priority of the current bridge. (Lowest priority is best for becoming the VLAN root.) If the switch does not become the root, an error results. The spanning-tree vlan root secondary alters a device’s bridge priority to 16384.
However, Cisco Nexus 7000 Series NX-OS Layer 2 Switching Comamnd Reference says the spanning-tree vlan root primary initially alters a device’s bridge priority to 24576. If the device does not become the root then the bridge priority is changed to 4096 less than the bridge priority of the current bridge. If the switch does not become the root, an error results. The spanning-tree vlan root secondary alters a device’s bridge priority to 28762.
Since the spanning-tree vlan root primary uses information from the current root beidge to form a new root bridge, setting the primary should work out ok. However, since the secondary macro does not involve any adjustment against other switches, there could be issues.
Lab Connectivity
I had a pair of N7Ks and a 6500 for the lab, connected with 3 trunk trunks as on the following diagram:
I configured VLAN 101 on all three devices, and the default STP status made 65K the root bridge:
65K(config)#do sh span vlan 101
VLAN0101
Spanning tree enabled protocol ieee
Root ID Priority 32869
Address 0017.0fae.0140
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32869 (priority 32768 sys-id-ext 101)
Address 0017.0fae.0140
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 15
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Gi2/1 Desg FWD 4 128.129 P2p
Gi2/2 Desg FWD 4 128.130 P2p
65K(config)#do sh run | inc span
spanning-tree mode pvst
spanning-tree extend system-id
65K(config)#
. . .
N7K11(config)# sh span vlan 101
VLAN0101
Spanning tree enabled protocol rstp
Root ID Priority 32869
Address 0017.0fae.0140
Cost 4
Port 131 (Ethernet1/3)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32869 (priority 32768 sys-id-ext 101)
Address 0024.f714.c242
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth1/3 Root FWD 4 128.131 P2p Peer(STP)
Eth2/1 Altn BLK 2 128.257 P2p
N7K11(config)#
. . .
N7K12(config)# sh span vlan 101
VLAN0101
Spanning tree enabled protocol rstp
Root ID Priority 32869
Address 0017.0fae.0140
Cost 4
Port 131 (Ethernet1/3)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32869 (priority 32768 sys-id-ext 101)
Address 0022.5579.f742
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth1/3 Root FWD 4 128.131 P2p Peer(STP)
Eth2/1 Desg LRN 2 128.257 Dispute P2p
N7K12(config)#
Catalyst 6500 and SPANNING-TREE VLAN ROOT PRIMARY
I first set up the 6500 as the root bridge, in case I was adding N7Ks to an existing environment. I found that the 6500s use the same vlan root convention as the NX-OS devices:
65K(config)#span vlan 101 root primary
65K(config)#do sh span vlan 101
VLAN0101
Spanning tree enabled protocol ieee
Root ID Priority 24677
Address 0017.0fae.0140
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 24677 (priority 24576 sys-id-ext 101)
Address 0017.0fae.0140
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 300
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Gi2/1 Desg FWD 4 128.129 P2p
Gi2/2 Desg FWD 4 128.130 P2p
65K(config)#do sh run | inc span
spanning-tree mode pvst
spanning-tree extend system-id
spanning-tree vlan 101 priority 24576
65K(config)#
I looked it up, and indeed the Catalyst 6500 Release 12.2SX Software Configuration Guide used the same 24576 (primary) and 28762 (secondary) convention as NX-OS. This is NOT the IOS convention.
NX-OS and SPANNING-TREE VLAN ROOT SECONDARY
I then tested the sapnning-tree vlan secondary macro – it simply applies the default priority value, and does not notice if a device become the secondary root or not. First I configured N7K12, then N7K11:
N7K12(config)# spanning vlan 101 root second
N7K12(config)# sh run spann
spanning-tree vlan 101 priority 28672
N7K12(config)# sh spann vlan 101
VLAN0101
Spanning tree enabled protocol rstp
Root ID Priority 24677
Address 0017.0fae.0140
Cost 4
Port 131 (Ethernet1/3)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 28773 (priority 28672 sys-id-ext 101)
Address 0022.5579.f742
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth1/3 Root FWD 4 128.131 P2p Peer(STP)
Eth2/1 Desg FWD 2 128.257 P2p
N7K12(config)#
. . .
N7K11(config)# spanning vlan 101 root second
N7K11(config)# sh run spa
spanning-tree vlan 101 priority 28672
N7K11(config)# sh span vlan 101
VLAN0101
Spanning tree enabled protocol rstp
Root ID Priority 24677
Address 0017.0fae.0140
Cost 4
Port 131 (Ethernet1/3)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 28773 (priority 28672 sys-id-ext 101)
Address 0024.f714.c242
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth1/3 Root FWD 4 128.131 P2p Peer(STP)
Eth2/1 Altn BLK 2 128.257 P2p
N7K11(config)#
Since N7K11 has a higher MAC address, it will not replace N7K12 as the secondary root.
Making the SPANNING-TREE VLAN MACROs Work
However, if you apply the spanning-tree vlan root primary macro twice, first to the secondary N7K, and then to the primary N7K, two applications of the macro will configure the appropriate primary and secondary devices. First I set up N7K12:
N7K12(config)# spanning-tree vlan 101 root primary
N7K12(config)# sh run spanning-tree
spanning-tree vlan 101 priority 20480
N7K12(config)# sh spann vlan 101
VLAN0101
Spanning tree enabled protocol rstp
Root ID Priority 20581
Address 0022.5579.f742
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 20581 (priority 20480 sys-id-ext 101)
Address 0022.5579.f742
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth1/3 Desg FWD 4 128.131 P2p Peer(STP)
Eth2/1 Desg FWD 2 128.257 P2p
N7K12(config)#
. . .
N7K11(config)# sh spann vlan 101
VLAN0101
Spanning tree enabled protocol rstp
Root ID Priority 20581
Address 0022.5579.f742
Cost 2
Port 257 (Ethernet2/1)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 28773 (priority 28672 sys-id-ext 101)
Address 0024.f714.c242
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth1/3 Desg FWD 4 128.131 P2p Peer(STP)
Eth2/1 Root FWD 2 128.257 P2p
N7K11(config)# sh run spann
spanning-tree vlan 101 priority 28672
N7K11(config)#
As expected, N7K12 became the root bridge. Second step, I set up N7K11:
N7K11(config)# sh run spann
spanning-tree vlan 101 priority 28672
N7K11(config)# spanning-tree vlan 101 root primary
N7K11(config)# sh run span
spanning-tree vlan 101 priority 16384
N7K11(config)# sh spanning-tree vlan 101
VLAN0101
Spanning tree enabled protocol rstp
Root ID Priority 16485
Address 0024.f714.c242
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 16485 (priority 16384 sys-id-ext 101)
Address 0024.f714.c242
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth1/3 Desg FWD 4 128.131 P2p Peer(STP)
Eth2/1 Desg FWD 2 128.257 P2p
N7K11(config)#
. . .
N7K12(config)# sh spanning-tree vlan 101
VLAN0101
Spanning tree enabled protocol rstp
Root ID Priority 16485
Address 0024.f714.c242
Cost 2
Port 257 (Ethernet2/1)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 20581 (priority 20480 sys-id-ext 101)
Address 0022.5579.f742
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth1/3 Desg FWD 4 128.131 P2p Peer(STP)
Eth2/1 Root FWD 2 128.257 P2p
N7K12(config)#
. . .
65K(config)#do sh span vlan 101
VLAN0101
Spanning tree enabled protocol ieee
Root ID Priority 16485
Address 0024.f714.c242
Cost 4
Port 129 (GigabitEthernet2/1)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 24677 (priority 24576 sys-id-ext 101)
Address 0017.0fae.0140
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 300
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Gi2/1 Root FWD 4 128.129 P2p
Gi2/2 Altn BLK 4 128.130 P2p
65K(config)#
Success! N7K11 is the root bridge, N7K12 is the secondary root bridge.
(A slightly different process would be to configure the spanning-tree vlan root primary macro on the alternate device first, and manually set a lower priority on the device you would like to be the root brudge.)
What to Remember
When you are working with a macro, you should look up the expected behavior on all devices in your network, NX-OS, IOS, and Catalyst IOS. If different conventions are used, you may need to apply any macros more ‘creatively’. By creatively, I mean in the order you need to get your desired results, in this case where two sequenced applications of spanning-tree vlan root primary are better than simply configuring spanning-tree vlan root primary and spanning-tree vlan root secondary. Finally, verifying your results afterwards is also a really good idea!
— cwr
ps – I plan to post additional NX-OS best practices in later articles, so check back later, or subscribe to the NetCraftsmen blog feed!
I tend to not use the macros as they only look a the network right now, I prefer manually setting the primary to 0 and secondary to 4096 to help protect against altered priorities on new switches joining the network. Obviously it is no guarantee that root placement won’t change when a new switch is installed but it does further lower the risk.
Thanks for your feedback!
Carole
Excellent article, Carole! Very timely for me, and thanks to your info, I am going to change things a bit.