Cisco Debug Enabled?

Author
Terry Slattery
Principal Architect

Have you ever accidentally left debug enabled on a Cisco network device?  I have, many years ago.  I had been doing some troubleshooting and was interrupted by someone with a question.  Auto-logout closed my session and by the time I returned to what I was doing, I’d forgotten where I was.  An hour or so later, an associate asked why remote access was down for a few seconds periodically.  So the debug affected network performance.  We plotted the ping round trip times, which were enlightening.  Can you figure it out from the graph?

Ping-Times

The ping packets are buffered by the router during periods when the CPU is busy handling the debug output.  When the debug output is over, the ping packets are immediately processed.  The ramp is dictated by the number of seconds that ping packets were buffered.  The period between buffering events is how often the debug runs.

This type of behavior is more prevalent in software-based routers – the CPU is making the forwarding decisions as well as processing the requested debugging information.  Adding ‘no logging console’ and ‘logging buffered’ helps reduce the load because the CPU doesn’t have to send debug output to the console port and can buffer the debug output. In the graphic above, the period is 90 seconds, which corresponds to the IGRP update timer (that indicates how long ago it happened).

Back to my original though – how do you know that debug has not been left enabled on some of your network devices?  After we recently found a device with debug enabled, I wrote a quick NetMRI script to do a ‘show debug’ and create an issue if it is found enabled on any device.  Running this script once a week will let us know if debug is left enabled and allow us to fix it if there’s no need for a long-running debug session.  The result will be more efficient network device operation.  The first time I ran the script, it found four devices with debug enabled.

While debug isn’t a configuration setting, it is part of the operational configuration of a device and can have a big effect on its performance.  I think of it as part of configuration compliance.  Now I’m satisfied that debug is not accidentally left enabled in the network.

-Terry

###########################################################################
## Export of Script:  Debug Check
## Script-Level: 3
## Script-Category: Uncategorized
###########################################################################

Script:
 Debug Check

Script-Description:
 Check that debug is not enabled on Cisco devices

###########################################################################
## Export of Script: Debug Check
## Script-Level: 3
## Script-Category: Uncategorized
###########################################################################

Script-Filter:

 $Vendor eq "Cisco"   and
 $sysDescr like /IOS/

#########################################################################
Action:
 Show Debug
Action-Description:
 Execute 'show debug', then check that there was no output.
Action-Commands:
 show debug
Output-Triggers:
 Process Debug

#########################################################################
Trigger: Process Debug
Trigger-Description:
 Match output that contains 'debugging is on'

Trigger-Template:
 debugging is on
Trigger-Commands:
 SET: $found_debug = "yes"
Output-Triggers:
 IssueDebugEnabled

#########################################################################
Issue: IssueDebugEnabled

Issue-ID: DebugEnabled
Issue-Severity:    Warning
Issue-Filter:
 $found_debug eq 'yes'
Issue-Description:
 Debug is enabled on a Cisco device.
Issue-Details:
 Host    $IPAddress
 Name    $Name

#########################################################################
##                            End of Script                            ##
#########################################################################

_____________________________________________________________________________________________

Re-posted with Permission 

NetCraftsmen would like to acknowledge Infoblox for their permission to re-post this article which originally appeared in the Applied Infrastructure blog under http://www.infoblox.com/en/communities/blogs.html

infoblox-logo

Leave a Reply

 

Nick Kelly

Cybersecurity Engineer, Cisco

Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.