Configuring Back-to-Back vPCs on Cisco Nexus Switches

Author
Carole Warner Reece
Architect

Recently I was talking to one of my co-workers about connecting back-to-back virtual Portchannels (vPCs) from one pair of Nexus 7000s to another pair of  Nexus 7000s (or 5000s). For example, you might have a data center with lots of VLANs, and decide that while you are waiting for TRILL and before you implement FabricPath, you will minimize STP with vPCs. The back-to-back vPCs will give you a loopless design.

Cisco mentions back-to-back vPCs in the Cisco Data Center Interconnect Design and Implementation Guide

http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns949/ns304/ns975/data_center_interconnect_design_guide.pdf
under the “2 Sites vPC-vPC Case Study”, and also calls them multi-layer vPCs in their recent configuration guides.

I decided put together a set of configurations on paper to see what they would look like on N7Ks. Here is the layout:

Back-to-Back vPCs on Cisco Nexus Switches

In my example, 7K-A & 7K-B are at the core layer, 7K-C and 7K-D are at the distribution or access layer. I used different PortChannel numbers mostly to show you could, and I thought it might help illustrate the different devices. I chose different interface numbers for the same reason.

Here are the slightly commented config fragments I put together:

. . .
hostname 7K-A
!
feature lacp
feature vpc
! enable the VLANs needed
vlan 1-50
!
! enable ports that will be used for peer link w/ 7K-B
interface ethernet 7/1-2
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 20 mode active
exit
!
! enable peer keepalive (in this case, use separate VRF)
vrf context pkal
interface ethernet 8/1
vrf member pkal
ip address 172.23.145.17/30
no shut
!
! configure VPC switch domain
vpc domain 1
peer-keepalive destination 172.23.145.18 source 172.23.145.17 vrf pkal
!
! configure the peer link
interface port-channel 20
vpc peer-link
exit
!
! Configure the interface for the port channel to 7K-C off of the vPC
interface ethernet 7/9
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 50 mode active
exit
interface port-channel 50
vpc 50
exit
!
. . .
hostname 7K-B

!
feature lacp
feature vpc
! enable the VLANs needed
vlan 1-50
!
! enable ports that will be used for peer link to 7K-A
interface ethernet 8/1-2
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 20 mode active
exit
!
! enable peer keepalive (in this case, use separate VRF)
vrf context pkal
interface ethernet 9/1
vrf member pkal
ip address 172.23.145.18/30
no shut
!
! configure VPC switch domain
vpc domain 1
peer-keepalive destination 172.23.145.17 source 172.23.145.18 vrf pkal
!
! configure the peer link
interface port-channel 20
vpc peer-link
exit
!
! Configure the interface for the port channel 7K-D off of the vPC.
interface ethernet 8/9
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 50 mode active
exit
interface port-channel 50
vpc 50
exit
. . .
!
. . .
hostname 7K-C
!
feature lacp
feature vpc
! enable the VLANs needed
vlan 1-50
!
! enable ports that will be used for peer link
interface ethernet 3/1-2
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 10 mode active
exit
!
! enable peer keepalive (in this case, use separate VRF)
vrf context pkal2
interface ethernet 4/1
vrf member pkal2
ip address 172.23.146.33/30
no shut
!
! configure VPC switch domain
! note the 7K-C / 7K-D vPC topology has a different Domain ID than the 7K-A / 7K-B vPC topology
vpc domain 3
peer-keepalive destination 172.23.146.34 source 172.23.146.33 vrf pkal2
!
! configure the peer link
interface port-channel 10
vpc peer-link
exit
!
! Configure the interface for the port channel to 7K-A off of the vPC.
interface ethernet 3/10
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 40 mode active
exit
interface port-channel 40
vpc 40
exit
. . .
!
. . .
hostname 7K-D
!
feature lacp
feature vpc
! enable the VLANs needed
vlan 1-50
!
! enable ports that will be used for peer link
interface ethernet 8/1-2
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 10 mode active
exit
!
! enable peer keepalive (in this case, use separate VRF)
vrf context pkal
interface ethernet 9/1
vrf member pkal2
ip address 172.23.146.34/30
no shut
!
! configure VPC switch domain

! note the 7K-C / 7K-D vPC topology has a different Domain ID than the 7K-A / 7K-B vPC topology
vpc domain 3
peer-keepalive destination 172.23.146.33 source 172.23.146.34 vrf pkal2
!
! configure the peer link
interface port-channel 10
vpc peer-link
exit
!
! Configure the interface for the port channel to the remote data center off of the vPC.
interface ethernet 8/9
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 40 mode active
exit
interface port-channel 40
vpc 40
exit
. . .
!

— cwr

_____________________________________________________________________________________________

If you would like some additional details, the following references should be helpful:

Configuring vPCs from Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide, Release 5.x

Virtual PortChannels: Building Networks without Spanning Tree Protocol

14 responses to “Configuring Back-to-Back vPCs on Cisco Nexus Switches

  1. very nice….I think you may want to add "Domain 2" to the middle of the diagram….
    thanks for sharing!
    Mike

  2. Hi Mike –

    I don’t think "Domain 2" is needed in the diagram. A vPC domain is just an arbitrary number assigned to a pair of switches that share vPC information. I was trying to show two domains (1 and 3), and that they do not need to be sequential.

    Carole

  3. I notice that you write ‘or 5000s’, well it doesn’t work todo the same configuration in Nexus 5000 since it doesn’t support creating more VRF’s for the peer keepalive (at least not in the 4.2(1)N2(1) that I have been working with. You should update the text either by removing the ‘or 5000s’ or mention which NX-OS version that is needed to enable this in a Nexus 5000 (if possible at all) or multiple configurations depending on hardware platform.

  4. Going back and looking at this again, I may have been referencing it shows VPC Domain 3 in the drawing, but shows VPC Domain 2 in the configurations…..I think that is what I meant to correct….
    Thanks, Mike

  5. Hi,
    Thanks carole. Very nice and clean representation. Well at that level we all know that Nexus is very keen and choosy when it comes to version and hardware specifications. I think all we need to know is the actual concept, configs can be taken anytime from cisco design guide lines as noone on earth can remember the encyclopaedia for the whole set of commands related to NS-OS. Thanks

  6. Very instructive.

    I think you need to update the diagram to reflect the configuration. 7K-C and 7K-D are shown in the diagram to be in VPC Domain 3, but the configuration files for them show VPC Domain 2 being configured.

  7. Hi Ron –

    Thanks, I missed the mismatch on the diagram and the configs. I updated the 7K-C & the 7K-D configs to show domain 3, which was my original intent…

    Carole

  8. Mike – I may have finally figured out your comment – I just updated the configs so the 7K-C and 7K-D configs showed domain 3…

    Carole

  9. Hi Carole..

    Have a question here.. how do you combine 4 nexus 5000s and hook up with 2 fex 2000s

  10. Hello All,

    I’ve been attempting this configuration using sets of Nexus 7k’s and the port-channel does not come up. My scenarios is connecting 2 data centers via 2-10gig dark fiber links.

    After opening a TAC request, I’m told that a total of 4 links are required with 2 separate vPC’s connecting the 4 N7K switches in the traditional sense of using vPC.

    Have you tested this configuration to work? My only difference is the vPC number is identical on both sides.

    -Brian

  11. Hi Carole,
    This is a great writeup and will come into use in some of my deployments. I’m assuming that a key point here is that you do indeed have to use different VPC domains, correct?

    Thanks for sharing!

  12. what happens, if traffic is received from another vPC member link say a VM via UCS via FI-A under 7K-C, now 7K-C’s link to 7K-A 3/10 breaks, so we’ve extended the vlan across the vpc peer-link, frame is sent across this link, here my understanding would be VSL bit is set to prevent a loop and the upstream B2B vPC member (40) link is no longer valid to exit the traffic flow?

  13. Ignore previous answered in the N7K vPC best practices guide, seems the data-plane gets re-programmed in this failure scenario to allow this flow:

    “The only exception to this rule occurs when vPC member port goes down. vPC peer devices exchange memberport
    states and reprogram in hardware the vPC loop avoidance logic for that particular vPC. The peer-link is then
    used as backup path for optimal resiliency. Traffic need not ingress a vPC member port for this rule to be
    applicable.”

  14. I am implementing the same topology in 2 Pair of 3548 Nexus switch and getting blocking port channel which is configured for back to back vPC at one end .
    My topology is

    4500 Series CORE-SW-01 (As Primary Root ) —–NEXUS SW01 & Nexus SW02–in single port channel at Core and vPC port at both Nexus switch

    4500 Series CORE-SW-02 (As Secondary Root ) —–NEXUS SW03 & Nexus SW04–in single port channel at Core and vPC port at both Nexus switch

    and after deploying the same topology as shown in above diagram

    and getting block at NEXUS SW03 & Nexus SW04 –port channel 40

    PVST is running in both Core Switch while RSTP is running all four Nexus switch

    Kindly suggest
    Ganesh

Leave a Reply

 

Nick Kelly

Cybersecurity Engineer, Cisco

Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.