A recent blog contrasted routers and switches. This blog contains some related thoughts and is essentially Part 2 of the prior blog.
In some recent design situations, I’ve spent some time looking at routers and firewalls, looking for some fairly hefty performance characteristics. That’s where it became quite clear (if it wasn’t clear enough already), that router and firewall performance costs a lot more, compared to switches. As covered in the previous blog.
This blog discusses the related topic of finding the router or firewall model with the right performance, for situations where more than relatively low-cost speed is needed.
Router Throughput
I keep encountering people who seem to think that since their Cisco ISR 4K router has a couple of 1 Gbps interfaces, it will do 1 Gbps of GigE to GigE forwarding (routing). Bzzzt, wrong! Probably not, depending on the model! The standard consulting answer applies: “it depends” and “don’t assume.”
This is not criticizing Cisco, other vendors do the same thing. Cisco’s data sheets are fairly candid on throughput. You do have to read them and understand what the performance numbers mean.
One place this comes up is in IPsec / IWAN / SD-WAN design. The Cisco ISR 4K and ASR 1K routers offer a variety of performance and price points. You do need to pick the right one, considering both the packet forwarding throughput and IPsec throughput, and what other loads you might be imposing on the router. This can be a bit of an art. You need to factor in other features you use that might load up a router (NAT, for instance).
You also need to allow for about 2x to 4x growth in traffic as well.
You DO have a real capacity planning process, don’t you? That is, tracking actual vs. projected traffic for about the last 3+ years. My experience is that most sites do not have solid capacity planning and trending information, including historical data. I’ll spare you why I think that is the case; some of it is tools, some of it is Ops staff ensuring all devices and interfaces are monitored, and some of it is methodology and capturing / improving your own forecasts.
<<Must Stay on Main Topic…>>
The Cisco licensing approach with the ISR 4K models allow some flexibility and is an interesting approach. If you get the base performance right, then buying a license allows doubling the throughput in some ISR 4K models, or different increases in the ASR 1K series (model-specific, see the datasheets for details). Some ASR models’ throughput is also capped by the CPU choice. For IWAN, you need to also keep an eye on the IPsec throughput, which will likely push you to a larger router model.
Getting Practical about Throughput Numbers
Forwarding performance is what I like to call a two-way number. It is the total workload a device can forward. Or as some would say, it is the sum of the “gazintas” (“goes into’s” with a NYC accent).
Question: So if you have a router rated at 1 Gbps forwarding performance, can it keep a 1 Gbps Internet link full?
Answer: NO. If the outbound traffic, which came in from some or all the other interfaces, is 1 Gbps, and the inbound is 1 Gbps. That’s 1 + 1 = 2 Gbps, which is more than the device is rated for.
Question: So, can I do 500 Mbps routing between LAN ports, plus 500 Mbps each way on the Internet link?
Answer: NO. That’s 500 in LAN1 + 500 in LAN2 + 500 in Internet = 1500 Mbps, > 1 Gbps. Buy a router rated at 2 Gbps (or bigger, to allow for traffic growth, etc.).
Firewall Throughput
This also comes up when designing for a large or shared WAN service with firewall. Prices (Cisco or Palo Alto) start to get “interesting” when you’re looking at multiples of 10 Gbps throughput, as in “full college education” or “major luxury auto” types of list prices.
With firewalls, you generally have more factors to consider: tiers of services you might or might not license or use. The more you have the firewall doing, the lower the performance, usually. If you’ve been buying firewalls for a while, you expect that. Historically, CheckPoint firewalls had a reputation for bogging down if you turned all the features on.
The reason for the performance drop-off is that a general-purpose CPU has to do most of the work. The nature of the firewall’s traffic analysis is generally not amenable to offload to a specialized chipset and code.
I’ll refer you to Table 2 in the “Cisco Next Gen Firewall” datasheet for an example. I was a bit surprised on how hard it was to find this from the Cisco NGFW glossy product pages, but Google Search for “Cisco NGFW datasheet” worked.
Do your homework, check the specs, and maybe divide the performance spec by 2x for some headroom / safety margin.
Virtual Devices
Yet another place you will encounter the need to be careful about performance is with virtual routers, e.g. the Cisco CSRv, which I happen to like for “real routing in the cloud,” albeit with a performance cap.
The CSRv is formally known as the CSR 1000v, but most people are confused or put off by the “1000v.” That leads them to think it’s somehow tied to the virtual switch, or if they are among the large 1000v-loathing community, turns them off. Could we please call it the CSRv instead – far better branding?
Virtual device performance depends on the hardware, or virtual slice of the hardware, provided to the router virtual machine. Cisco has published specs. Performance varies with Cloud Provider due to the CPU and other capacity they give the CSRv virtual machine.
If the CSRv specs say that CSRv on AWS doesn’t have enough throughput, you might consider scale-out using multiple virtual routers, each handling a portion of the total traffic. This scales to some extent.
Network function virtualization (NFV) is a similar setting; virtual routers (firewalls, etc.) running on a server or multi-purpose platform you own. The Cisco 5000 “Enterprise Network Compute System” is one such platform. Cisco’s marketing makes it fairly clear that you’re trading performance for the flexibility and agility of NFV. Well, agile as long as you have a pool of licenses or rapid way to acquire them to spin up virtual devices with.
One concern with virtual devices is ensuring they get sufficient capacity of shared resources (CPU, RAM, internal bus, NIC I/O). If they’re sharing limited resources with an occasionally performance hogging VM, you may get intermittent performance problems. Fun to troubleshoot! One of life’s trade-offs.
Does It Add Up?
It also became clear that for routers and firewalls, the numbers don’t add up. I’ll go on to explain that, and no, I’m not accusing vendors of anything. My intent here is to alert readers to something they need to pay attention to.
Let’s pick on the Cisco NGFW 2140 model, although I’ve noticed the same thing with Palo Alto firewalls. The top performance number in the NGFW datasheet is 20 Gbps, but only 8.5 with firewall + AVC. That’s the maximum forwarding throughput.
Coming back to the 2130 and 2140 NGFW’s, they come with “integrated I/O” consisting of “12 x 10M/100M/1GBASE-T Ethernet interfaces (RJ-45), 4 x 10 Gigabit (SFP+) Ethernet interfaces.”
Do the math. The sum of the input wire speeds (ignoring fine details) is 12 + 40 = 52 Gbps. The performance specs mean you’d best not be pumping a full 10 Gbps of firewall + AVC in even one of those interfaces.
What I want to caution about here is assuming that a given router or firewall can do wire speed on all the interfaces the vendor put in the box.
That leaves me wondering, why all the interfaces? My guess is that some people like to physically segment (legacy / strict interpretation of PCI or government requirements?). I personally have been leaning towards “firewall on a stick” trunking to a switch, since it saves having to manually re-cable anything. But that’s a matter of taste, and local security rules. Clear “inside” and “outside” interfaces also work, and are easily understood.
Is there an intent to deceive? I’m not a vendor mind-reader, nor am I about to state an opinion on that topic. Granting the benefit of the doubt, it may just be about customer expectations and adding ports being relatively inexpensive.
Right-Sizing Your Router or Firewall
In conclusion, it is advisable to wade through those datasheet performance numbers and apply a grain of salt (or a larger amount of salt), depending on your trust in your vendor’s numbers. If you aren’t comfortable doing that, get some help with it.
I’d also advise documenting what your requirements are, what features you need, and what kind of performance you need. Then, check out the math – not using interfaces that’ll put you over the specified performance.
It might also be wise to run your thoughts by a couple of outside parties, either savvy peers, vendors, and even a savvy consultant. They might catch a math error or a “gotcha” you missed. Better to do so before placing the order than to get it wrong. Most consultants, VAR’s, and Cisco SE’s don’t want you buying the wrong box and then being unhappy with it.
I’ll note NetCraftsmen is a consulting firm that happens to also be a VAR, both for Cisco and other product lines. We try to prioritize services and looking out for the customer’s interests.
Comments
Comments are welcome, both in agreement or constructive disagreement about the above. I enjoy hearing from readers and carrying on deeper discussion via comments. Thanks in advance!
—————-
Hashtags: #router #firewall #Cisco
Twitter: @pjwelcher
NetCraftsmen Services
Did you know that NetCraftsmen does network /datacenter / security / collaboration design / design review? Or that we have deep UC&C experts on staff, including @ucguerilla? For more information, contact us at info@ncm2020.ainsleystaging.com.