Have you ever accidentally left debug enabled on a Cisco network device? I have, many years ago. I had been doing some troubleshooting and was interrupted by someone with a question. Auto-logout closed my session and by the time I returned to what I was doing, I’d forgotten where I was. An hour or so later, an associate asked why remote access was down for a few seconds periodically. So the debug affected network performance. We plotted the ping round trip times, which were enlightening. Can you figure it out from the graph?
The ping packets are buffered by the router during periods when the CPU is busy handling the debug output. When the debug output is over, the ping packets are immediately processed. The ramp is dictated by the number of seconds that ping packets were buffered. The period between buffering events is how often the debug runs.
This type of behavior is more prevalent in software-based routers – the CPU is making the forwarding decisions as well as processing the requested debugging information. Adding ‘no logging console’ and ‘logging buffered’ helps reduce the load because the CPU doesn’t have to send debug output to the console port and can buffer the debug output. In the graphic above, the period is 90 seconds, which corresponds to the IGRP update timer (that indicates how long ago it happened).
Back to my original though – how do you know that debug has not been left enabled on some of your network devices? After we recently found a device with debug enabled, I wrote a quick NetMRI script to do a ‘show debug’ and create an issue if it is found enabled on any device. Running this script once a week will let us know if debug is left enabled and allow us to fix it if there’s no need for a long-running debug session. The result will be more efficient network device operation. The first time I ran the script, it found four devices with debug enabled.
While debug isn’t a configuration setting, it is part of the operational configuration of a device and can have a big effect on its performance. I think of it as part of configuration compliance. Now I’m satisfied that debug is not accidentally left enabled in the network.
-Terry
########################################################################### ## Export of Script: Debug Check ## Script-Level: 3 ## Script-Category: Uncategorized ########################################################################### Script: Debug Check Script-Description: Check that debug is not enabled on Cisco devices ########################################################################### ## Export of Script: Debug Check ## Script-Level: 3 ## Script-Category: Uncategorized ########################################################################### Script-Filter: $Vendor eq "Cisco" and $sysDescr like /IOS/ ######################################################################### Action: Show Debug Action-Description: Execute 'show debug', then check that there was no output. Action-Commands: show debug Output-Triggers: Process Debug ######################################################################### Trigger: Process Debug Trigger-Description: Match output that contains 'debugging is on' Trigger-Template: debugging is on Trigger-Commands: SET: $found_debug = "yes" Output-Triggers: IssueDebugEnabled ######################################################################### Issue: IssueDebugEnabled Issue-ID: DebugEnabled Issue-Severity: Warning Issue-Filter: $found_debug eq 'yes' Issue-Description: Debug is enabled on a Cisco device. Issue-Details: Host $IPAddress Name $Name ######################################################################### ## End of Script ## #########################################################################
_____________________________________________________________________________________________
Re-posted with Permission
NetCraftsmen would like to acknowledge Infoblox for their permission to re-post this article which originally appeared in the Applied Infrastructure blog under http://www.infoblox.com/en/communities/blogs.html
