I’ve been working with two customers that I’d describe as “rural fiber network providers”. Their designs and business objectives differ somewhat. (For more about one, Google “Michael Minnich” and “fiber farms” — and thanks to Mike for getting me and Chesapeake Netcraftsmen involved in his project, and to Brian / Scott and BCC for their project. They’re the entrepreneurs, putting their credit at risk to make their vision happen!)
The rough idea is that in rural areas, bandwidth is still costly. In one case, T1’s cost over $3000 monthly. That creates an economic opportunity. There’s demand for faster speeds, perhaps Gigabit per second or better. And if you can keep the costs down, you can provide that bandwidth competitively. “Trade your costly T1 in for 10 Gbps fiber connectivity” kind of gets your attention from a sales perspective! Of course, with competition, the existing carrier might reduce prices, and that has to be factored in.
The business model entails somehow funding installation of fiber, preferably in a County core or some controlled geography, to keep route miles down. The fiber can be buried, or space rented on telephone poles. Licensing and other barriers exist as well. (But if you’re going to provide service to the County government, maybe that helps with barriers?)
In one case, relatively few fiber strands were pulled. That fits a shared services model. In another, more fiber was hung (incremental costs of more strands being relatively low), and fiber provided directly to some end customers. With fewer fibers, the up front cost is a little lower, but when the fiber runs out, higher speeds or DWDM gear might be needed — a pay me now / pay me later situation.
Enough of the business case, I want to get technical here.
Network Requirements:
- Simple technology
- Low cost
- High stability
In both cases, the guys who’d be operating the network aren’t shabby technically, but the KISS (“Keep It Simple Sam”) principle applies. If you only have a few VPNs or customers needing segmentation, MPLS VPN might be technical overkill.
So my design approach was to leverage Cisco VRF-Lite. While Cisco is perceived by some as having high cost, the gear is really stable and well-supported, and offers a plethora of technical alternatives for solving real problems and dealing with the unexpected (design-wise). And the support is great! The lower end (small) Cisco switches have some pretty nifty capabilities too. We worked with the enterprise product line, mostly (3560-E, 4900M, or ME-3400 2CS and 12 CS), since most of the Cisco Metro product line is an order of magnitude shorter on bandwidth. Nice equipment, but if I can use the enterprise gear to go 10x faster … As long as MPLS and some of the Metro Ethernet features aren’t absolutely required, that might make sense for you.
The cheapest approach, and the one Cisco gear fits best perhaps in terms of capabilities, is inexpensive Layer 2 switches feeding a core MPLS VPN. Adding routing capabilities (IOS image) to small switches to support routing and VRF Lite is not cheap. We opted to do this, at least for the provider core, to provide far greater stability, by isolating Layer 2 spanning tree domains to the edge, e.g. customer buildings. That protects the “ring around the county”. And trades some up front cost for lower OpEx (operating costs) later. Plus stability leads to higher customer satisfaction, which is pretty important too!
The plan is to look at things again as these networks grow. If they get to the point of interconnecting county-sized networks, then maybe MPLS VPN might have a role in the core. If you’ve only got 10 VPN customers, VRF’s might do the trick. They’re a little clunkier to configure, but much simpler in terms of lower tech level routing and show commands. And in this setting they just don’t need to scale very far.
In one case, the shared network is to provide central managed Cisco voice services and also Internet access. Two VRFs isolate those.
In the other case, the shared network provides segmented connectivity (VPNs) to the County government, including Public Safety, and to the County schools, also shared Internet connectivity. So a couple of VRFs covers that. The school service offering is being provided as e-rate services, which apparently apply.
Carole Warner Reece has written a blog article about some unicast routing issues and VRF Lite deployment, along with sample configurations see https://netcraftsmen.com/blogs/entry/vrf-lite-eigrp-and-static-routes.html.
I’ll be doing a follow-on blog, since in one of these networks, the school system is using 3Com IP PBXs now, and they apparently do small conference calls using IP multicast. (They dislike Mitel rather strongly, I’m not sure why. And like Cisco UC features, but can’t afford the price they were quoted.) So I got the chance to work with Multicast VRF Lite, which is kind of neat!
For the technical blog about IP Multicast VRF Lite (multicast in a VRF), see https://netcraftsmen.com/blogs/entry/ip-multicast-in-a-vrf.html.