Load Balancing and Cisco Express Forwarding (CEF) with Parallel Paths

Author
Carole Warner Reece
Architect

One of my friends asked me recently if he had too much redundancy. He was concerned about potential issues in having multiple equal cost paths from his Unified Communications Manager to his IP phones, and between IP phones in different locations. (In his case, he is running EIGRP and has up to 4 equal cost paths.)

I told him that his VoIP traffic should be fine with the Cisco Express Forwarding (CEF) enabled on his Layer 3 switches. By default, his switches use CEF’s per-destination load balancing across his multiple parallel paths. CEF tracks paths based on both the source and destination address of the two devices on the ends of the path. Each one-way path between two IP nodes is an IP session, and each session is logically identified as a unique source-destination address pair. Therefore, the session from a remote VoIP phone to the Unified Call Manager would always follow one unidirectional path, and the path from one VoIP phone to a remote VoIP phone would always follow one unidirectional path. This preserves packet ordering for packets within a flow by ensuring that all packets within a single IP source/destination flow are sent on the same route.

With CEF load balancing, the session-to-path assignment is done using a hash function that considers the source and destination IP addresses and, in recent releases of Cisco IOS, a unique hash ID that randomizes the assignment across the end-to-end path. For all sessions being forwarded, each active path carries a share of the number of sessions.

Note: Because per-destination CEF load balancing depends on the statistical distribution of traffic, the load sharing across the links becomes more effective as the number of source-destination pairs increase.

Enabling CEF with Infrastructure Enhancements

I then started wondering about whether there were quirks for enabling CEF by platform. (I sort of remembered some issues in the past.) However, after looking up CEF features, I found that most IOS releases now support CEF infrastructure enhancements. In addition, IPv4 fast switching has been removed from the IOS. For these and later Cisco IOS releases as shown in the table below, the switching path options are Cisco Express Forwarding switched or process switched. As best I can tell, CEF is enabled by default on all these platforms, and can be re-enabled with the ip cef orthe ip cef distributed global configuration.

Cisco IOS Release Platforms Supported
12.2(25)SE
(use global command ip cef [distributed] to re-enable if needed)
Catalyst 2970 series switches Catalyst ME 3400
Catalyst 3500 series switches
Catalyst 3750 series switches
12.2(25)SG
(use global command ip cef [distributed] to re-enable if needed)
Catalyst 4500 series switches Catalyst 4900 series switches
12.2(28)SB Cisco 7200 series routers
Cisco 7301 series routers
Cisco 7304 series routers
Cisco 10000 series routers
12.2(33)SRA
(use global command ip cef [distributed] to re-enable if needed)
Cisco 7600 series routers
12.2(33)SXH
(use global command ip cef to re-enable if needed)
Catalyst 6500 series switches
12.4(20)T
(use global command ip cef to re-enable if needed)
Cisco 800 series routers
Cisco 1700 series routers
Cisco 1800 series routers
Cisco 2600 series routers
Cisco 2800 series routers
Cisco 3200 series routers
Cisco 3600 series routers
Cisco 3700 series routers
Cisco 3800 series routers
Cisco 7200 series routers
Cisco 7400 series routers
Cisco 8850 series routers
Cisco AS5000 series universal gateways
15.0
(use global command ip cef to re-enable if needed)
Cisco 1900 series routers
Cisco 2900 series routers Cisco 3900 series routers

Commands to Use to Check CEF Status:

  • To verify the CEF is running on the device, use the show ip cef command.
  • To verify the state of CEF on the device, use the show cef state command.
  • To verify the CEF configuration of an interface, use the show cef interface type number command.
  • To verify that the routing table includes all the parallel paths, use the show ip route network command.
  • To determine which path a session between a source and destination address is assigned, use the show ip cef exact-route source-ip-address destination-ip-address command.

__________________________________________________________________________________________________

More on Cisco Express Forwarding

Some useful resources on CEF include:

2 responses to “Load Balancing and Cisco Express Forwarding (CEF) with Parallel Paths

  1. excellent summary, this information cleared my questions.

    Thank you so much Carole warner reece 🙂

  2. Nothing is said about dissimilar latency parallel paths, Does CEF takes this into account?

Leave a Reply

 

Nick Kelly

Cybersecurity Engineer, Cisco

Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.