Device Real-Time Displays

Author
Terry Slattery
Principal Architect

I’ve been doing a number of device configuration changes as a result of the things that NetMRI and syslog are reporting in a customer network. Both NetMRI and syslog have rather long reporting cycles before I know that something was fixed, so I’ve been thinking about a device real-time display as a nice function to have.

NetMRI is not unique in its data collection. Network management systems tend to using a polling mechanism to collect a lot of the data that is displayed. The collection tends to be on a periodic basis which is often dependent on how many devices need to be polled and the performance of the NMS system. The end result is that it can take a long time to update a system’s status after a configuration change is made.

Syslog also has a long time delay, but because of a different reason. I’ll see messages in syslog, make a change, then I have to watch syslog and wait long enough to satisfy myself that the cause of a given message has been fixed.

In both cases, the NMS doesn’t know that changes are being made to the network and that it should update a given device’s data more quickly than normal. In fact, I’ve never heard of an NMS that adapts its polling interval based on configuration changes that it detects in the network.

The result is that I’ve been thinking that network management systems should incorporate a real-time device viewer that allows me to see the changes I’m making and their impact on the device’s operation. Most network management systems only provide a real-time viewer for interface performance traffic. There are a lot of other data points that would be useful to display, either as graphs or showing the the current state of some operational variable.
However, network devices contain a lot of data, especially if you include large tables like routing, ARP, neighbor discovery, and switch forwarding tables. Polling all this data every few seconds could create a large load on both the NMS and the device. There are several ways to reduce the load. One way to reduce the load is to only poll for a sub-set of all the data. This is a viable approach because I am typically only interested in a few data points. A User Interface that allows me to select the data to be displayed could be used by the NMS to determine the data to poll, reducing the load on the NMS and the device. Ideally, the views of most or all of this data would be collected together on one display, providing me with a device dashboard. For example, I may want to have interface configuration and performance combined with HSRP configuration on one page.

Here are some examples of things that I would want to track…

  • Interface performance and status. Watch interface status (up/down), duplex, utilization, and error counts. When the duplex setting is changed, how do the counter rates change? Some of these values would be best displayed as graphs, perhaps in the style of sparklines.
  • HSRP and STP root bridge. Show which device is active and which is standby, if a standby router is known. For STP, show the root bridge and number of switches in the spanning tree. Is the root bridge the same as the HSRP active router for each VLAN that has both configured?
  • QoS traffic class drops. Show a plot of the drops in each traffic class within a QoS policy. This would be great if done using a graphing tool, perhaps with sparklines, like mentioned above.
  • Environmental parameters. Watch the temperature in a room go up or down as the air conditioning is turned off and on. Or watch power supply fluctuations as PoE devices turn on at the start of the day or in response to an EnergyWise automation system.
  • Device operational parameters. Variables like CPU utilization, free memory, buffer misses, and backplane utilization may be good factors to display for a new device as part of a network benchmarking program.

Selecting a sub-set of all available device data monitoring sources would provide me with a neat way to look only at those variables that of interest to me at a given time. If I can save the view and use it later, I can then create views that augment my other network troubleshooting tools.
Imagine being able to select the Interface performance and QoS views to be display simultaneously on a router. I could quickly determine whether QoS is working as desired for a given traffic class, knowing that the interface traffic load is typical for the interface in question. Not seeing QoS drops? Is any traffic transiting the interface? Is the interface congested (QoS is only engaged when the interface is congested)?

With some careful UI work on top of a good polling engine, a real-time device viewer could be a pretty interesting and useful tool. It could also wind up being a pretty cool display in the NOC, with changing displays of device metrics.

-Terry

_____________________________________________________________________________________________

Re-posted with Permission 

NetCraftsmen would like to acknowledge Infoblox for their permission to re-post this article which originally appeared in the Applied Infrastructure blog under http://www.infoblox.com/en/communities/blogs.html

infoblox-logo

Leave a Reply

 

Nick Kelly

Cybersecurity Engineer, Cisco

Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.