This blog is a quick note about an easily misunderstood set of switch commands, Cisco Traffic Storm Control. The commands are very useful, and work. However, they do seem to be commonly misunderstood — or else the documentation is wrong. Part of the confusion may be due to different behavior on the large switches compared to the smaller Cisco switches. I get asked about this a lot when teaching the Nexus classes. I’ve seen Networkers slides that seem to think storm control behaves differently.
I’m going by the documentation here, I don’t have easy access to a 6500 or Nexus 7000 for lab testing (most N7K’s tend to be in production).
The traffic storm control command(s) are still very useful for mitigating the effects of a Spanning Tree loop. My co-workers tell me that you do want hardware-based storm control, for example by the time a 6500 Sup 2 (MSFC2) notices it should be doing software-based storm control, it is already toast. Toast = CPU spun up, stops doing BPDU’s, stops sending UDLD so peers errdisable connections, etc.
Traffic storm control is most useful in Cisco 6500 and Nexus 7000 switches. The documentation for the two models matches. (One suspects the source code is rather similar too.)
Where does the confusion arise?
Well, the manual says “Traffic storm control (also called traffic suppression) allows you to monitor the levels of the incoming broadcast, multicast, and unicast traffic over a 1-second interval. During this interval, the traffic level, which is a percentage of the total available bandwidth of the port, is compared with the traffic storm control level that you configured. When the ingress traffic reaches the traffic storm control level that is configured on the port, traffic storm control drops the traffic until the interval ends.“
It goes on to specify the syntax, which is to configure an interface with:
storm-control {broadcast | multicast | unicast} level percentage[.fraction]
The standard example is:
storm-control broadcast level 40
storm-control multicast level 40
storm-control unicast level 40
The problem comes about in that people think they get different thresholds for each of the three types of traffic: broadcast, multicast, unicast. WRONG! First hint: the thresholds in the example are all 40.
Now read the syntax introduction carefully: “Traffic storm control uses a bandwidth-based method to measure traffic. You set the percentage of total available bandwidth that the controlled traffic can use. Because packets do not arrive at uniform intervals, the 1-second interval can affect the behavior of traffic storm control.“
(I put the key words in bold characters.) Each time you enter a storm-control command, you are adding to the flavors or types of controlled traffic. The threshold is the same threshold, which is applied to the sum total of the controlled traffic.
The manual goes on to provide examples of how this works. It starts with
•
If you enable broadcast traffic storm control, and broadcast traffic exceeds the level within the 1-second interval, traffic storm control drops all broadcast traffic until the end of the interval.
This is what we all would probably expect, either way we interpret the operation of storm control. The manual goes on with:
•If you enable broadcast and multicast traffic storm control, and the combined broadcast and multicast traffic exceeds the level within the 1-second interval, traffic storm control drops all broadcast and multicast traffic until the end of the interval.
•If you enable broadcast and multicast traffic storm control, and broadcast traffic exceeds the level within the 1-second interval, traffic storm control drops all broadcast and multicast traffic until the end of the interval.
•If you enable broadcast and multicast traffic storm control, and multicast traffic exceeds the level within the 1-second interval, traffic storm control drops all broadcast and multicast traffic until the end of the interval.
This makes it fairly clear that the aggregate of all the controlled types is what is being measured against the threshold — and that there is only one threshold.
The Nexus 7000 command reference pretty much clarifies it: “Enter the storm-control level command to enable traffic storm control on the interface, configure the traffic storm-control level, and apply the traffic storm-control level to all traffic storm-control modes that are enabled on the interface. Only one suppression level is shared by all three suppression modes. For example, if you set the broadcast level to 30 and set the multicast level to 40, both levels are enabled and set to 40.“
Unfortunately, “both levels” sounds like two different levels, each set to 40 — unfortunate wording. The other documentation and the command behavior makes much more sense if the one and only threshold level is being set to 40.
The blog at http://blog.ipexpert.com/2010/03/15/old-ccie-myths-storm-control/ tackles testing this, albeit on a small Catalyst switch. The testing methodology at the end of the blog unfortunately turns only one of broadcast and multicast storm control on at a time, which seems to me to defeat the purpose. My tentative conclusion: it seems likely the behavior of small Catalyst switches may differ from that of the Catalyst 6500 and Nexus 7000. The small switch documentation is a bit ambiguous either way.
Invitation (Testing Challenge?)
If you have lab tested this, or if you are inspired to go run a test, please add a comment to this blog on your test methodology and findings — or a link to your blog about it. I searched but am not seeing anything definitive!
The only reliable way to test this and set it to rest is to perform very precise traffic generation with multiple levels of different traffic types and measure received frames on another port.
To do so, one would need a device like Spirent SmartBits (or similar), which wasn’t available to me when I wrote the blog you reference.
You raise a valid point however that there is only one threshold, not multiple ones. However, my tests (done for the blog and otherwise) did show that there are indeed multiple levels. I.e. if you set limit to 0 for certain traffic kind, that traffic WILL be dropped, regardless of you using higher threshold value for another traffic type. Reading your article I understood that higher value would be used as a threshold – again, this is not something I observed in my testing.
-Marko.
Wow, that was a fast comment. Thanks, Marko!
I’m not too worried about precision, and I liked Marko’s testing concept in general in his blog that I referenced. I do think the smaller switches have multiple levels, which his tests (first part of his blog) confirmed as far as I’m concerned — maybe I could have made that more clear. My concern is that the documentation for the bigger switches reads a bit differently, and I think it may behave differently. Probably for some hardware limitation or coding reason. Anyway, only testing with the bigger switches can confirm or deny that.
Hi
i have a lot of old c2950 in my network i set
storm-control broadcast level pps 200 150
storm-control multicast level pps 2000 1500
storm-control unicast level 20.00
storm-control action shutdown
if and i see a lot of time my interface its shutdown couse of action
if i take off the action shutdown my interface will be alive if level pps its higher ?
thank you
The smaller switches behave differently than the 6500, with individual levels of control for the flavors of traffic. If you’re constantly having problems, you probably need higher thresholds (but not too high). I wouldn’t want to ever do shutdown in response to high traffic levels, that’s overkill. The idea is to drop the excessive traffic.
I tested this on a Nexus 6000 switch. I am pretty sure, there are 3 different threshold levels. I tested the following scenario : Set the unicast broadcast level at 1% and left the broadcast & multicast levels at 100% (default value). In case of unicast, it did not allow more than 1% traffic to pass, whereas in case of broadcast traffic, all traffic passed.
You seem to be right about “unfortunate wording” but it seems Command reference guide got it wrong & not the other way round.