You’ve heard it before: “The application is slow!” It’s impacting the business and is frustrating for employees. Both application and network teams say they can’t find problems in their respective areas.
Is there any possibility of resolution? You’ve tried everything. Now what?
Let’s look at another case in the NetCraftsmen archives of “Impossible IT Challenges.”
The IT Challenge – Poor Enterprise App Performance
In this case, the CIO of a large enterprise requested our assistance to identify the cause of poor application performance within the organization. The core business applications relied on the transfer of many large image files, significantly loading the servers and network. The poor performance had existed for many months.
Diagnosing Problem #1 – The Metro Ethernet Configuration
We deployed a team of three architects who used NetCraftsmen’s network analysis toolkit to perform a complete network assessment. The team examined the applications, network performance, and network configuration. We found two significant problems, both of which involved network congestion.
The network architecture served several large offices from two data centers, all within a few miles of each other. The two data centers were connected by two 1 Gbps metro ethernet circuits. Interactive applications would routinely access massive data sets from either data center, requiring the data to be quickly copied across the links. This is where the performance problem manifested itself.
The initial review didn’t find anything remarkable. However, a detailed investigation of the data center interconnecting routers identified excessive packet buffering on the link configurations. The buffering interacted with the network protocols in a way that significantly reduced application throughput.
The immediate solution was to reduce the packet buffering to better match the router’s multi-gigabit LAN with the 1 gigabit WAN link’s ability to forward packets between data centers. This is like matching the flow rate of a fire hose with that of a garden hose to avoid overflow (i.e., packet loss). The long-term solution was already in progress: acquire higher capacity circuits between the data centers.
Diagnosing Problem #2 – Interface Overload
NetCraftsmen’s network data collection tools then identified a second problem: several data center switch interfaces showed high input packet loss, which is an extremely rare situation. Upon investigation, we found that the core business servers had all been connected to an adjacent set of switch interfaces. The concentration of network traffic overloaded the switch, causing the excessive packet loss.
The solution was to distribute the servers across a broader range of interfaces on multiple switches which spread the load and eliminated the packet loss. Application performance improved again. Spreading the load had the added benefit of increasing application resilience since the original switch was a single point of failure.
Problem Solved
The combination of the two problems had created the perfect storm with regard to application slowness. Our architects applied their years of experience and knowledge of network protocols to identify and correct two significant problems that had been hurting business performance for months.
Solving the Most Difficult Network Challenges
Does your organization have what seems like “impossible” network challenges or is your network not performing the way it should? The NetCraftsmen team’s in-depth understanding of networks and network protocols allows us to tackle those seemingly unsolvable problems.
We bring to bear our tools, processes, and expertise to thoroughly review your network’s design, configuration, and operation and share our learnings with your network staff so that they’re better able to support your business applications.
Let’s start a conversation! Contact us to see how NetCraftsmen experts can help ensure your IT infrastructure gets healthy and stays healthy.