Why vPC UCS to Nexus?

Author
Peter Welcher
Architect, Operations Technical Advisor

For a change of pace, this blog poses a thought exercise (teaser?), a small item I ran into while considering whether or not to use a vPC to connect a UCS chassis to a pair of Nexus switches. I’ll give you the diagram and scenario, and then my answer. I’m hoping this will give your Layer 2 and Layer 3 skills a little exercise along the way. For me, it confirms that Cisco validated designs usually have a reason for how they do things!

The Setting

The consulting customer had two UCS chassis and two NetApp devices to connect to two Nexus switches. Each of the four systems to connect up had 4 x 10 Gbps interfaces. The UCS chassis would be doing NFS to the NetApp devices for storage, since NFS has worked well for the site historically. 

Testing showed that single-homing the NetApp boxes worked well, with a NetApp rapid failover mechanism protecting against loss of a single Nexus. However, there was some desire to align the design with the Cisco FlexPod validated design. So the design evolved to dual-homing the 2 UCS and the 2 NetApp chassis via two links to each Nexus switch. Small detail: the design was a collapsed core, and Nexus 7K was playing the role of the Nexus 5K’s in the standard FlexPod design. 

The UCS devices and the NetApp boxes will be in different VLANs (hardly necessary, but planned). 

The Diagram

The following diagram illustrates the proposed design (using vPC for the uplinks).

The Question

The question came up of whether it was necessary / appropriate / useful to vPC the 4 links from each chassis. Using end host mode or MAC pinning mode could more or less imitate the traffic patterns that would occur with vPC. And not doing vPC would be a bit less technically complex, avoiding using the new / recent vPC functionality. Yet the FlexPod approach calls for using vPC.

Aside from high level “design philosophy”, is there a technical advantage to using vPC in this setting? What does using vPC add? 

Peeking

No peeking! My answer is below. 


The Answer

What’s your FHRP going to be? Whether HSRP, VRRP, or GLBP, your traffic will be sent to a virtual MAC address associated with the FHRP default gateway for the end system.

In a non-vPC setting, end host mode (or whatever) hashing randomness is likely to send about 50% of your traffic up the “wrong” link to get to the virtual MAC address. GSLB will just do that with more vMAC addresses involved. Such mis-guided traffic will then have to cross the peer link (which might be just a trunk if there’s no vPC being used). 

When you  do vPC, FHRP spoofing means that whichever uplink gets used, the receiving Nexus will forward it. Less / no cross-link traffic. That’s a win.

Bonus Round

What changes if we put the UCS and the NetApp devices into the same VLAN? Is vPC useful? 

Answer: I’m coming up with “maybe not for UCS to NetApp traffic, and back again”. But: what about traffic to users, who will be in a different VLAN? Then the same argument applies. 

Hope you enjoyed this teaser!

If I missed something,  I’m sure you’ll be delighted to tell the world all about it in a comment. Be polite or I won’t let it see the light of day! 🙂

Leave a Reply

 

Nick Kelly

Cybersecurity Engineer, Cisco

Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.