Working With Cisco Nexus 5K Switch Profiles

Author
Carole Warner Reece
Architect

[Guest article from my friend Dominic Basta. Thanks again, Dominic!]

I thought I’d share my experiences working with switch profiles on Cisco Nexus5000 switches. Cisco suggest using switch profiles for the following reasons:

  • Allows configurations to be synchronized between switches.
  • Merges configurations when connectivity is established between two switches.
  • Provides control of exactly which configuration gets synchronized.
  • Ensures configuration consistency across peers through merge and mutual‐exclusion checks.
  • Provides verify and commit semantics.
  • Supports configuring and synchronizing port profileconfigurations.
  • Provides an import command to migrate existing vPC configurations to a switch profile.

I decided to test setting Switch Profiles up to provide a single point of configuration change for our bowtie setup Nexus 5548s and 2232 FEX.  The switch profiles would protect us from vPC misconfigurations and insure configuration redundancy across the parent switches.

If you have a symmetric network, using port profiles to configure port-channels and the physical member ports for vPCs can be a very good idea.

Here is the basic design topology of the data center row:

2012_04-30_n5k_test_bed

BASIC CONCEPTS

There are a couple of basic concepts to know before you start configuring the switch profile.

  1. Switch Profiles can only support a configuration sync across the Mgmt0 interfaces.  For this test I cabled each Mgmt0 interface on both test-5k-sw1 and test-5k-sw2 together, you can however sync across a true out of band management network as long as the Mgmt0 interfaces are Layer 2 Adjacent.
  2. You need to pick a name to use for your switch profile. For test purposes, we will use “test-sync”.  The name is case sensitive as well so if you get cute with the naming convention changing it later requires deleting the switch profile and recreating it.  In theory that sounds easy but anything configured via the Switch Profile and sync’d to the second switch will now be deleted and all of your configuration lost.  I have not found a way to “stamp” down the configuration into the local switches running configuration…that would be a nice addition.
  3. Swith profiles are NOT supported on the Nexus 7000 switches.

SWITCH PROFILE CONFIGURATION

The first step is to enable CFSoIP communication over the Mgmt0 interfaces:

Test-5k-sw1# configure terminal
Test-5k-sw1(config)# cfs ipv4 distribute
Test-5k-sw1(config)# exit
Test-5k-sw1#

NOTE: The cfs ipv4 distribute command enables CFSoIP communication over the Mgmt0 interfaces. During my first setup I didn’t issue the command, and while I was able to ping across the Mgmt0 interfaces, the switches will not sync.

The next step is to configure a switch profile with a name and a peer destination IP address. Creating the switch profile is quite easy, instead of using the config t command you use the config sync command.

Test-5k-sw1# config sync
Test-5k-sw1(config-sync)# switch-profile test-sync
Test-5k-sw1(config-sync-sp)# sync-peers destination 10.0.0.2

NOTE: You must create the switch profile with the same name on each switch, and the switches must configure each other as a peer across the mgmt0 interface. When connectivity is established between switches with the same active switch profile, the switch profiles are synchronized.

After both peers are configured and synchronized, you enter the shared commands for both switches on one switch in the switch profile. It works similar to configuring the commands from the the regular configure terminal mode but none of the commands you enter are placed or “committed” into the running configuration until you enter the commit command.  For example:

Test-5k-sw1# config sync
Test-5k-sw1(config-sync)# switch-profile test-sync
Test-5k-sw1(config-sync-sp)# interface port-channel20
Test-5k-sw1(config-sync-sp)# speed 1000
Test-5k-sw1(config-sync-sp)# interface Ethernet1/1
Test-5k-sw1(config-sync-sp)# speed 1000
Test-5k-sw1(config-sync-sp)# channel-group 20
NOTE: The commands themselves have to pass a Mutual Exclusion Check which means the switch profile commands are checked against the local switch and the peer switch.  Normally this will not be an issue if you have only configured the new switches via the switch profile, but if you have issued commands directly to either the local switch or peer switch via the configuration terminal…you will get used to failing Mutual Exclusion Checks.

One thing to recognize is that as soon as a commit is successful, the switch saves the configuration buffer and drops you out of the switch profile configuration mode.  So if you needed to add more configurations, you will need to re-enter the switch profile.  I’ve found that when working with switch profiles I like to use the verify and the show switch-profile name buffer commands fairly often:

Test-5k-sw1(config-sync-sp)# verify
Verification Successful
Test-5k-sw1(config-sync-sp)#

NOTE: The verify command verifies the commands in the switch profile buffer can be added to both the local and peer switches.  The verify command does not drop you out of the switch profile, so you can basically use it after every command to make sure you will not later fail the Mutual Exclusion Checks.

Test-5k-sw1(config-sync-sp)# show switch-profile test-sync buffer
switch-profile : test-sync 
----------------------------------------------------------
Seq-no Command
----------------------------------------------------------
1      interface port-channel20
1.1      speed 1000
2      interface Ethernet1/1
2.1      speed 1000
2.2      channel-group 20

After you finish configuring and verifying all the commands you want to synch between peers, you need to commit the commands:

Test-5k-sw1(config-sync-sp)# verify
Verification Successful
Test-5k-sw1(config-sync-sp)# commit
Commit Successful
Test-5k-sw1(config-sync)# exit
Test-5k-sw1#

CONCLUSION

I found using the switch profile is quite easy but a bit quirky at times.  We have since moved the switch profile into production with a few issues, but for the most part it does ease the pain of configuring both N5Ks.

I believe that configuring a switch profile on an existing terminal-configured N5K isn’t something to taken lightly.  I have found that using the import command to import an existing interface configuration appears to fail more times than work for reasons that are often hard to diagnose.

However, in our secondary data center the 5548s and 2232s were configured from the start with the switch profiles and switch profiles work VERY well there. I  am hopeful that in a few releases to come the rest of the quirks in switch profiles will be worked out.

REFERENCES

Here is a good document to use as a reference if you want to start your own lab switch:
Configuring Switch Profiles

One response to “Working With Cisco Nexus 5K Switch Profiles

  1. Hi Carol,
    very nice & interesting article. I´ve also started to configure our N5k with switch profiles and it works very well for all kind of shared configs between the Nexus switches.
    I have on question regarding your vPC setup between N5k´s and the N7k´s: Why do you use 2 vPC´s (for each N7k) instead of one vPC between both N7k? Is there a reason for that? (vPC peer link failure scenarios on N7k or something like that?)

    Thanks, Diemtar

Leave a Reply

 

Nick Kelly

Cybersecurity Engineer, Cisco

Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.