I’ve been testing some end-to-end VRF-Lite configurations, and I’ve been thinking about whether the customer edge (CE) device should run one or more VRFs. Like many design choices in networking, the decision to run VRFs on your CE devices depends on your application and your network.
Although the names Multi-VRF CE and VRF-Lite have been linked for awhile, I think the way the terms are now used are different.
My summary of the difference is these two definitions:
- Multi-VRF CE on the edge router gives you the opportunity in a MPLS-VPN environment to extend some provider edge (PE) functions to the CE. The CE router can maintain separate VRF tables so that different routing decisions can be made. Multi-VRF CE can allow a service provider to share a managed CE device with multiple customers, perhaps in a multi-tenant building. So for diverse customers with multiple VPNs, this can be a good choice. Some reference documents include Designing MPLS Extensions for Customer Edge Routers, OSPF Support for Multi-VRF on CE Routers, Multi-VRF Support (VRF Lite), …
- VRF-Lite supports VRFs on devices without MPLS. An End-to-End VRF-Lite design supports network virtualization without ever using MPLS in the network. Some good references include the “Path Isolation Deploying VRF-Lite End-to-End” section of Network Virtualization–Path Isolation Design Guide, and Network Virtualization–Services Edge Design Guide
So I’ve been thinking about a simple shared services scenario running an End-to-End VRF-Lite design , where a small service provider or large organization is providing access to the Internet, as well as on-network Voice over IP (VoIP) service between some locations. In this scenario, there will be two VRFs, Internet and Voice offered to multiple locations. Is it a better design choice to run VRFs on the CE in this scenario? Some additional details may help influence the decision.
In the environment, the provider is running only VRF-Lite across their backbone, and is not using BGP. The backbone routing protocol is EIGRP, and the customer sites have non-overlapping IP address allocations. The network connects each remote campus or site as ‘customers’ to a pair of CE routers, and the CEs to PE routers in the core. Two of the PE routers connect to a cloud of VoIP servers – such as a Unified Communication Manager, a Unity server, … Two of the PE routers connect to the Internet. The provider allocates IP addressing to the customers, with network 10.0.0.0/9 is reserved for data, and network 10.128.0.0/9 is reserved for voice. (By convention, the voice service is 10.d + 128.x.x, where “d” is the block assigned for data service.) Each customer runs EIGRP to interconnect their campus locations. ME-3400s were used for the CEs, 7600s were used for the PEs.
In the diagram, the Voice VRF is shown in blue, the Internet VRF is in Red. The global routing for the infrastructure management is in black.
The 10.8.0.0/16 addresses are allocated for the Internet infrastructure, and the 10.136.0.0/16 addresses are allocated for the voice infrastructure, with the 10.136.248.0/248 addresses allocated for the network management infrastructure. (Note: We reserved a block of the Voice infrastructure addresses for network management, since we wanted them to have more protection than the Internet infrastructure.)
CE Running as a Fusion Router
In this article, I will look at one connectivity option for the CEs for CustomerA, where the CEs are NOT running VRFs. The CE router is acting as a ‘fusion router‘, fusing the customer routing with the shared services VRFs. The Customer network is running EIGRP 1 internally. VRFs run only on the PEs, and dynamic routing is running between the CEs and the customer.
Note: I will focus on CE-A1 and PE-1 configurations. The diagram shows the VLANs used for the network virtualization.
This design will require three main configuration tasks:
- Connect shared services VRFs with the routing process on the CEs
- Connect the CE with the customer routing process
- Establish the fusion mechanism – apply the appropriate filtering and route distribution between the routing processes on the CE
1. Connecting the Shared Services VRFs with the CEs
The backbone VRF configuration is straight-forward, and the same on all devices supporting the VRFs:
! PE-1 ip vrf Internet
rd 8:4 ! ip vrf Voice
rd 136:2 ! router eigrp 248 network 10.136.248.0 0.0.7.255 no auto-summary ! address-family ipv4 vrf Voice
network 10.136.0.0 0.0.255.255
no auto-summary
autonomous-system 136
exit-address-family ! address-family ipv4 vrf Internet
network 10.8.0.0 0.0.255.255
no auto-summary
autonomous-system 8
exit-address-family !
The CEs will use three EIGRP routing processes to connect to the PEs to segment the traffic:
! CE-A1 router eigrp 248 network 10.136.248.0 0.0.7.255 no auto-summary ! router eigrp 136
network 10.136.4.140 0.0.0.3
no auto-summary ! router eigrp 8
network 10.8.4.140 0.0.0.3
no auto-summary
!
Note that the basic routing processes on the CE routers looks matches up with the global routing process (supporting the management traffic) and address families (supporting the Voice VRF and the Internet VRF) configured for the PEs.
We can confirm that the CE routers see routes from the PEs:
! CE-A1#sh ip ro eigrp 88
10.0.0.0/8 is variably subnetted, 12 subnets, 5 masks D 10.8.0.0/16 [90/3072] via 10.8.4.141, 00:06:57, Vlan13
D*EX 0.0.0.0/0 [170/281856] via 10.8.4.141, 00:06:57, Vlan13
CE-A1# CE-A1#sh ip ro eigrp 216
10.0.0.0/8 is variably subnetted, 12 subnets, 5 masks
D 10.128.0.0/9 [90/3072] via 10.136.4.141, 00:08:33, Vlan12
CE-A1# CE-A1#show ip route eigrp 248 10.0.0.0/8 is variably subnetted, 12 subnets, 5 masks D 10.136.248.0/21 [90/3072] via 10.136.250.141, 00:00:26, Vlan11 CE-A1# !
We see that the PE routers are sending to the CEs summary routes for the infrastructure addressing, a summary route for the VoIP services, and a default route to the Internet services.
2. Connecting the CEs with the Customer
Next we need to connect the routing process for CE and the customer.
! CE-A1 router eigrp 1 network 10.100.1.248 0.0.0.3 network 10.216.250.148 0.0.0.3 network 10.228.1.248 0.0.0.3 !
! Customer router eigrp 1 network 10.0.0.0 !
We can confirm that the CE routers see some routes from the customer:
!
CE-A1#show ip route eigrp 1
10.0.0.0/8 is variably subnetted, 44 subnets, 10 masks
D 10.129.1.128/30 [90/130816] via 10.100.1.249, 00:05:19, Vlan8
[90/130816] via 10.100.1.249, 00:05:19, Vlan136
D 10.1.1.0/25 [90/130816] via 10.228.1.249, 00:05:19, Vlan136
D 10.1.1.0/24 [90/130816] via 10.100.1.249, 00:05:19, Vlan8
D 10.66.1.1/32 [90/130816] via 10.228.1.249, 00:05:19, Vlan136
[90/130816] via 10.100.1.249, 00:05:19, Vlan8
D 10.67.1.1/32 [90/130816] via 10.228.1.249, 00:05:19, Vlan136
[90/130816] via 10.100.1.249, 00:05:19, Vlan8
D 10.117.32.0/25 [90/130816] via 10.228.1.249, 00:05:19, Vlan136
D 10.111.0.0/16 [90/130816] via 10.100.1.249, 00:05:20, Vlan8
D 10.100.4.0/24 [90/130816] via 10.228.1.249, 00:05:19, Vlan136
D 10.100.0.0/16 [90/28416] via 10.100.1.249, 00:05:20, Vlan8
. . .
!
We see that the customer routers are currently sending many routes. We prefer that the customer routers just send summaries for the live networks, because the CE routers do not need to know the details of the customer routing process. We will manage this in the next task.
3. Fusing the Routing Processes on the CEs
Now we need to establish the appropriate filtering and route distribution between the routing processes on the CEs. Prefix-lists, distribute-lists, and route-maps will be our tools for fusing the processes.
Routing with the Customer
Looking first at the routes sent to and received from the customer, we only want to
- send a summary route for the VoIP services to the customer
- send a default route to the Internet services to the customer
- receive the summarized customer routes from the customer
! define the default route to the Internet ip prefix-list All-Data description Only send default route to customer
ip prefix-list All-Data seq 5 permit 0.0.0.0/0
! route-map All-Data-out permit 10
match ip address prefix-list All-Data
! ! ! define the summary route for the VoIP services ip prefix-list All-Voice description Only send 10.216.0.0/16 prefix to customer
ip prefix-list All-Voice seq 5 permit 10.128.0.0/9
! route-map All-Voice-out permit 10
match ip address prefix-list All-Voice
! ! ! define the summarized routes for the Customer's allocated Data addresses ip prefix-list CustA-Data description Only accept summarized routes from customer
ip prefix-list CustA-Data seq 5 permit 10.100.0.0/16
ip prefix-list CustA-Data seq 10 permit 10.117.0.0/16
ip prefix-list CustA-Data seq 15 permit 10.118.0.0/16 le 17
ip prefix-list CustA-Data seq 20 permit 10.119.0.0/16
! ! define the summarized routes for the Customer's allocated VoIP addresses ip prefix-list CustA-Voice description Only accept summarized routes from customer
ip prefix-list CustA-Voice seq 5 permit 10.228.0.0/16
ip prefix-list CustA-Voice seq 10 permit 10.245.0.0/16
ip prefix-list CustA-Voice seq 15 permit 10.246.0.0/16 le 17
ip prefix-list CustA-Voice seq 20 permit 10.247.0.0/16
! router eigrp 1 ! pull in the summary route for the VoIP services redistribute eigrp 136 metric 100000 100 255 1 1500 route-map All-Voice-out
! pull in the default route for the Internet service redistribute eigrp 8 metric 100000 100 255 1 1500 route-map All-Data-out
network 10.100.1.248 0.0.0.3 network 10.216.250.148 0.0.0.3 network 10.228.1.248 0.0.0.3 ! send only the summary route for the VoIP services distribute-list prefix All-Voice out Vlan216
! send only the default route for the Internet service distribute-list prefix All-Data out Vlan88
! accept only the summarized routes for the Customer's data devices distribute-list prefix CustA-Data in Vlan8
! accept only the summarized routes for the Customer's VoIP devices distribute-list prefix CustA-Voice in Vlan136
We can confirm that the CE routers now only sees the summary routes from the customer on the correct interfaces:
! CE-A1#show ip route eigrp 1 | inc Vlan8
D 10.119.0.0/16 [90/130816] via 10.100.1.249, 00:25:08, Vlan8
D 10.100.0.0/16 [90/28416] via 10.100.1.249, 00:25:08, Vlan8
D 10.118.0.0/16 [90/130816] via 10.100.1.249, 00:25:08, Vlan8
D 10.117.0.0/16 [90/130816] via 10.100.1.249, 00:25:08, Vlan8
C 10.100.1.248/30 is directly connected, Vlan8 CE-A1# CE-A1#show ip route eigrp 1 | in Vlan136
C 10.228.1.248/30 is directly connected, Vlan136 D 10.246.0.0/16 [90/130816] via 10.228.1.249, 00:25:04, Vlan136
D 10.228.0.0/16 [90/28416] via 10.228.1.249, 00:25:04, Vlan136
D 10.247.0.0/16 [90/130816] via 10.228.1.249, 00:25:04, Vlan136
D 10.245.0.0/16 [90/130816] via 10.228.1.249, 00:25:04, Vlan136
CE-A1# !
We verify that the customer now sees the routes for the Voice and Internet services:
!
Cust1#show ip route | include EX
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
D EX 10.128.0.0/9
D*EX 0.0.0.0/0 [170/53760] via 10.100.1.250, 00:02:00, FastEthernet0/0.8
Cust1#
Routing with the PE
We need to configure redistribution of the appropriate routes into EIGRP 8 and EIGRP 136:
! CE-1 ! define route-maps using the summarized routes from the Customer's devices ! route-map CustA-Voice-in permit 10
match ip address prefix-list CustA-Voice-in
! route-map CustA-Data-in permit 10
match ip address prefix-list CustA-Voice-in
! router eigrp 136
redistribute eigrp 1 metric 10000 100 255 1 1500 route-map CustA-Voice-in
network 10.136.4.140 0.0.0.3 network 10.228.1.248 0.0.0.3 no auto-summary ! router eigrp 8
redistribute eigrp 1 metric 10000 100 255 1 1500 route-map CustA-Data-in
network 10.8.0.0 0.0.255.255 network 10.100.1.248 0.0.0.3 no auto-summary !
We can verify that the PEs now see the redistributed summary routes from the customer:
! PE-1#show ip route vrf Voice | in Vlan12
D EX 10.247.0.0/16 [170/281856] via 10.136.4.142, 00:10:06, Vlan12
C 10.136.4.140/30 is directly connected, Vlan12 L 10.136.4.141/32 is directly connected, Vlan12 D EX 10.228.0.0/16 [170/281856] via 10.136.4.142, 00:10:06, Vlan12
D 10.228.1.248/30 [90/3072] via 10.136.4.142, 00:37:06, Vlan12 D EX 10.246.0.0/16 [170/281856] via 10.136.4.142, 00:10:06, Vlan12
D EX 10.245.0.0/16 [170/281856] via 10.136.4.142, 00:10:06, Vlan12
PE-1# PE-1#show ip route vrf Internet | in Vlan13
C 10.8.4.140/30 is directly connected, Vlan13 L 10.8.4.141/32 is directly connected, Vlan13 D EX 10.100.0.0/16 [170/281856] via 10.8.4.142, 00:15:14, Vlan13
D EX 10.119.0.0/16 [170/281856] via 10.8.4.142, 00:15:14, Vlan13
D EX 10.117.0.0/16 [170/281856] via 10.8.4.142, 00:15:14, Vlan13
D EX 10.118.0.0/16 [170/281856] via 10.8.4.142, 00:15:14, Vlan13
PE-1#!
We can also verify that the customer has connectivity to devices in the two service VRFs:
! Cust1#ping 10.136.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.136.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms Cust1# Cust1#ping 1.1.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms Cust1#
Conclusion
In this design, the CE routers selectively fuse the VRF and customer routing processes. The CE router can support multiple customers. The CE router will advertise the summary routes it receives from each customer to the PE.
One drawback to this configuration is that there are a lot of small pieces to configure to manage two redistribution points into multiple routing processes, so it is conceptually a bit complex to configure.
Stay tuned – my next article CE Design Options When Using VRF-Lite End-to-End (Part 2) will look at running the VRFs on the CE routers.
____________________________________________________________________________________________________________
More on VRF-Lite
Other recent NetCraftsmen blogs on VRF-Lite include:
Using VRF-Lite, EIGRP, and Static Routes
How Carole,
Can you post the full configuration of CE Running as a Fusion Router section? Also I need to use BGP between CE and PE, what are your thoughts on that?
Thanks,
Joe Jones
Hi Carole,
Can you email me or port the full config of the CE routers of your example "CE Running as a Fusion Router"? Also I need the BGP peer with the PE(telco).
Thanks Joe