Working With Cisco Nexus 5K Switch Profiles

[Guest article from my friend Dominic Basta. Thanks again, Dominic!]

I thought I’d share my experiences working with switch profiles on Cisco Nexus5000 switches. Cisco suggest using switch profiles for the following reasons:

• Allows configurations to be synchronized between switches.
• Merges configurations when connectivity is established between two switches.
• Provides control of exactly which configuration gets synchronized.
• Ensures configuration consistency across peers through merge and mutual‐exclusion checks.
• Provides verify and commit semantics.
• Supports configuring and synchronizing port profileconfigurations.
• Provides an import command to migrate existing vPC configurations to a switch profile.

I decided to test setting Switch Profiles up to provide a single point of configuration change for our bowtie setup Nexus 5548s and 2232 FEX.  The switch profiles would protect us from vPC misconfigurations and insure configuration redundancy across the parent switches.

If you have a symmetric network, using port profiles to configure port-channels and the physical member ports for vPCs can be a very good idea.

Here is the basic design topology of the data center row:

BASIC CONCEPTS

There are a couple of basic concepts to know before you start configuring the switch profile.

1. Switch Profiles can only support a configuration sync across the Mgmt0 interfaces.  For this test I cabled each Mgmt0 interface on both test-5k-sw1 and test-5k-sw2 together, you can however sync across a true out of band management network as long as the Mgmt0 interfaces are Layer 2 Adjacent.
2. You need to pick a name to use for your switch profile. For test purposes, we will use “test-sync”.  The name is case sensitive as well so if you get cute with the naming convention changing it later requires deleting the switch profile and recreating it.  In theory that sounds easy but anything configured via the Switch Profile and sync’d to the second switch will now be deleted and all of your configuration lost.  I have not found a way to “stamp” down the configuration into the local switches running configuration…that would be a nice addition.
3. Swith profiles are NOT supported on the Nexus 7000 switches.

SWITCH PROFILE CONFIGURATION

The first step is to enable CFSoIP communication over the Mgmt0 interfaces:

Test-5k-sw1# configure terminal
Test-5k-sw1(config)# cfs ipv4 distribute
Test-5k-sw1(config)# exit
Test-5k-sw1#

NOTE: The cfs ipv4 distribute command enables CFSoIP communication over the Mgmt0 interfaces. During my first setup I didn’t issue the command, and while I was able to ping across the Mgmt0 interfaces, the switches will not sync.

The next step is to configure a switch profile with a name and a peer destination IP address. Creating the switch profile is quite easy, instead of using the config t command you use the config sync command.

Test-5k-sw1# config sync
Test-5k-sw1(config-sync)# switch-profile test-sync
Test-5k-sw1(config-sync-sp)# sync-peers destination 10.0.0.2

NOTE: You must create the switch profile with the same name on each switch, and the switches must configure each other as a peer across the mgmt0 interface. When connectivity is established between switches with the same active switch profile, the switch profiles are synchronized.

After both peers are configured and synchronized, you enter the shared commands for both switches on one switch in the switch profile. It works similar to configuring the commands from the the regular configure terminal mode but none of the commands you enter are placed or “committed” into the running configuration until you enter the commit command.  For example:

Test-5k-sw1# config sync
Test-5k-sw1(config-sync)# switch-profile test-sync
Test-5k-sw1(config-sync-sp)# interface port-channel20
Test-5k-sw1(config-sync-sp)# speed 1000
Test-5k-sw1(config-sync-sp)# interface Ethernet1/1
Test-5k-sw1(config-sync-sp)# speed 1000
Test-5k-sw1(config-sync-sp)# channel-group 20

One thing to recognize is that as soon as a commit is successful, the switch saves the configuration buffer and drops you out of the switch profile configuration mode.  So if you needed to add more configurations, you will need to re-enter the switch profile.  I’ve found that when working with switch profiles I like to use the verify and the show switch-profile name buffer commands fairly often:

Test-5k-sw1(config-sync-sp)# verify
Verification Successful
Test-5k-sw1(config-sync-sp)#

NOTE: The verify command verifies the commands in the switch profile buffer can be added to both the local and peer switches.  The verify command does not drop you out of the switch profile, so you can basically use it after every command to make sure you will not later fail the Mutual Exclusion Checks.

Test-5k-sw1(config-sync-sp)# show switch-profile test-sync buffer
switch-profile : test-sync 
----------------------------------------------------------
Seq-no Command
----------------------------------------------------------
1      interface port-channel20
1.1      speed 1000
2      interface Ethernet1/1
2.1      speed 1000
2.2      channel-group 20

After you finish configuring and verifying all the commands you want to synch between peers, you need to commit the commands:

Test-5k-sw1(config-sync-sp)# verify
Verification Successful
Test-5k-sw1(config-sync-sp)# commit
Commit Successful
Test-5k-sw1(config-sync)# exit
Test-5k-sw1#

CONCLUSION

I found using the switch profile is quite easy but a bit quirky at times.  We have since moved the switch profile into production with a few issues, but for the most part it does ease the pain of configuring both N5Ks.

I believe that configuring a switch profile on an existing terminal-configured N5K isn’t something to taken lightly.  I have found that using the import command to import an existing interface configuration appears to fail more times than work for reasons that are often hard to diagnose.

However, in our secondary data center the 5548s and 2232s were configured from the start with the switch profiles and switch profiles work VERY well there. I  am hopeful that in a few releases to come the rest of the quirks in switch profiles will be worked out.

REFERENCES

Here is a good document to use as a reference if you want to start your own lab switch:
Configuring Switch Profiles