Link Utilization Analysis

Author
Terry Slattery
Principal Architect

I’m sure that most of you have some sort of network monitoring system in place and that you’ve looked at utilization of your important links. I was looking at a set of links recently that had some common characteristics as well as one unexpected attribute.  The utilization charts below are for Gigabit interfaces in the core of a network.  In the first image, you’ll note the common work-day traffic signature that looks like a double peak of 24-32Mbps, with the valley between the peaks occurring around lunch time.  No such signature occurs on Saturday and Sunday.

The second thing you should note is the evening automated backups and other back-office workload that occurrs every day, starting in the evening and running into the early morning hours of the next day. Depending on the extent of the data being copied, the evening traffic load may be even higher than the daily workload.  In fact, at startup, the evening network traffic peaks regularly at around 30Mbps before dropping back to 16Mbps.  I’ll bet that this traffic is primarily one or more backups and that the both the sending and receiving systems have free memory available to buffer the data, resulting in higher network demands.  But after the process has run a short time, the physical I/O constraints of the disk systems lower the overall throughput to 16Mbps (about 1.6MB/s).

Picture_202_work_day_traffic

The next utilization chart is more interesting because it doesn’t have the same clear signatures during the daytime.  You have to look more closely to see the night-time network traffic, which occurs after midnight each day.  You also have to be careful in examining this chart because the magnitude of the traffic is higher.  What is interesting to me about this chart is the extent of the peaks and the plateau of overall traffic at around 100Mbps on each Sunday. While this link is 1Gbps, the sites which are using it are connected by 100Mbps circuits.

Picture_201_utilization

Without flow data to correlate with the traffic utilization graphs, it is difficult to tell what traffic is traversing the link and whether higher peaks should be expected.  If there is primarily one site performing backups or some other activity, then I would be satisfied in accepting that the peak traffic would be around 100Mbps.  But if more than one site were attempting to use the link, and I expected the overall traffic load to be greater than 100Mbps, then I’d want to start further investigations.  Is something else keeping the load at 100Mbps?  Why am I not seeing the characteristic initial traffic burst that preceeds the longer term flow, like we observed for the evening transfers in the first graph?

My concern is that something in the network is limiting the throughput to 100Mbps.  If I had flow data available, I’d be able to determine who were the top sources and destinations and conduct further investiations.

Analysis like this is useful when validating a Service Level Agreement (SLA).  If a Gigabit path was contracted, but for some other reason is restricted to 100Mbps, then I’d want to know about it.  Perhaps the carrier has an internal limitation?  Or, more likely, the application just can’t run any faster than 100Mbps throughput and the link has plenty of reserve capacity.  You can’t tell from these charts which case applies without correlating other data.

-Terry

_____________________________________________________________________________________________

Re-posted with Permission 

NetCraftsmen would like to acknowledge Infoblox for their permission to re-post this article which originally appeared in the Applied Infrastructure blog under http://www.infoblox.com/en/communities/blogs.html

infoblox-logo

Leave a Reply

 

Nick Kelly

Cybersecurity Engineer, Cisco

Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.