VoIP Troubleshooting Visibility

Terry Slattery
Principal Architect

VoIP troubleshooting is a pretty interesting problem. (See my earlier blog entry regarding my talk at VoiceCon Fall 2007.) There are a large number of tools that look at packets on the wire, or look at the RTCP (Real Time Control Protocol) stream from certain phones, or gather data from the call controllers. While many of these approaches can identify that a problem exists, I’ve not encountered any, other than NetMRI, that also look at the infrastructure devices (routers and switches) to try to identify the real source of problems.

While it is somewhat useful to identify that a problem exists, it is still a lot of work to identify the source of the problem. For example, knowing that high packet loss occurrs on a particular set of voice calls is useful. However, identifying the source of the packet loss requires more work. It could be due to a bad link, duplex mismatch, using the wrong codec on a call leg, or improper QoS configuration. If the call path is long, there are many points to check. Each check will take several minutes to several hours to manually collect and analyze the data at each point in the path necessary to validate the correct operation or that a potential source of the problem has been detected. Multiply by the number of points in the path and you have a lengthy and tedious troubleshooting session.

What’s needed is something akin to the analysis computer in modern automobiles. In the network, the infrastructure’s configuration should be regularly validated against the design policies to verify that configurations contain the correct QoS configuration and default duplex settings. You can think of this as the “seatbelt not buckled” warning.

The network’s operational parameters should be checked to make sure that things like duplex settings negotiated properly. Another operational parameter is to verify that the QoS queue bandwidth is not exceeded, resulting in dropped packets, all because a high bandwidth codec was inadvertently used or that too many concurrent calls are occurring. Warnings on operational exceptions are like the “check engine” light — something is amiss in the system’s operation.

While it may sound like an advertisement, network analysis and improving network efficiency like that described above is our goal in creating NetMRI. I’ve not seen anything in the market that performs the same breadth of analysis in one package as exists in NetMRI. If you have, please leave me a comment.



Re-posted with Permission 

NetCraftsmen would like to acknowledge Infoblox for their permission to re-post this article which originally appeared in the Applied Infrastructure blog under http://www.infoblox.com/en/communities/blogs.html


Leave a Reply


Nick Kelly

Cybersecurity Engineer, Cisco

Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.


Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.


John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.