SD-Access IP Pools

This blog is part of a series covering various SD-Access topics.

Previous blogs in this series:

This blog focuses on IP Pools. Not on how to create them in DNA Center (“DNAC”), but on planning them across multiple sites.

Cisco’s deployment documentation covers the How-To part quite adequately. You can create multiple global pools and then carve them up per-site for a different purpose (data, wireless, etc.). When you are fabric provisioning switches from a site, you supply IP pools by name called for in dialog boxes. I think it is helpful to create and give suitable names to all the IP pools to assist in the later match-up process. See below for more about this.

What Cisco’s documentation doesn’t do (as far as I can see) is provide a list of all the possible IP pools DNAC might use in one place, plus discussion of how big they should be, so that one can step back and plan addressing from that list. With one site, you can pretty much make it up as you go. With 10, 20, 50 sites, that’s maybe not such a good idea. That’s my starting assumption, anyway!

This blog can be viewed as a progress report. I’m trying something out, based on what I think I know / can verify and based on lab work. It may have errors or have overlooked something that applies to your situation. In short, user beware, and please let me know if you spot any errors.

The intent is, if nothing else, to get you thinking about this topic, if it is relevant to your present or future SD-Access design.

Naming Pools

Site-VN name or purpose is a good naming convention for IP pools. Example: NYC-STAFFVN. For a pool that is not per-VN, an example might be NYC-FABRICAPs. Or NYC-FABRIC-APS. (There’s no charge for extra hyphens!)

I am allergic to mixing hyphens and underscores in names; it creates potential errors. (I’m a big fan of just removing underscore from keyboards.) However, some neutral separator is needed here for readability.

You probably want to establish a case convention: all upper, all lower, or something, to avoid configuration mis-matches.

A Technicality

Technically, you create one or more global pools in DNAC, and you then create the name and reserve sub-pools for various uses. This blog will just refer to IP pools, with the understanding that we’re focused more on the sub-pools that get reserved.

A Disclaimer

Disclaimer: I personally have always really liked summarizable addressing, with ONE summarizable block per site. That bias may permeate this blog!

Why summarizable? That way, you have some location information for troubleshooting. It is handy for the address to tell you something like site, without having to consult a per-/24 lookup spreadsheet.

I sometimes see summarizable addressing in the field. There seems to be more anti-summarizable. I’ve seen highly fragmented use of the address space, with /24s handed out sequentially across sites, tracked in IPAM or a spreadsheet. Maybe that started historically, and nobody has had the opportunity or desire to change it. (See also, “technical debt.”)

There’s no way you can efficiently do lookups with a small per-site XLS printout of per-site address blocks when each sites/24s (or whatever size blocks) are essentially random. Hopefully, your IPAM tool facilitates IP lookups and has good descriptions of what each IP block is for. I’m not sure that is a realistic hope, based on my small sample size, however.

The real cost here is that user and security analysis done subsequently involves a bit more effort to figure out where the endpoint(s) are. Compared to, say, actually recognizing some addresses (main campus, datacenter, etc.).

Let’s carry that a little further and then change the subject. I’ve had success with campus or other equipment replacement by using a repeatable addressing pattern. Things like each building get a block of some size, within that floors are blocks of some other size, and per-floor the first /24 is VoIP, the second is WLAN, etc. That sort of thing, where a per-site pattern means you can quickly take an address and not only determine the site or building, but the floor, purpose, or other useful info.

We might refer to the site-first approach as “site. VN”. As in 10.site.VN.x if that fits your needs. Caution: you can blow through the entire 10.0.0.0/8 in a hurry that way!

Anyway, read onwards to see where that takes us. Let’s see if I can persuade you that there’s something useful lurking here!

Summarizability by VN

In some recent discussions, it has become clear that you might want to think about doing “VN.site.” If you can summarize the addressing by VN, that lets you write IP ACL-based micro-segmentation policy on a firewall or other device that does not do pxGrid / SXP. That is, a firewall external to the campus fabric(s), like an edge or datacenter entry firewall. Since SGs in a VN are all in the same subnet, you’d lose micro-segmentation. But the macro-segmentation might be all you really need with well-chosen VNs.

Do recall my previous advice however: be stingy with VNs, as they currently require extra effort and manual “glue” configuration on fusion firewalls or routers.

What IP Pools Will DNA Center Want?

The following is my current list of IP pools I’ve encountered in SD-Access / DNA Center lab work. It attempts to be comprehensive. No guarantees! Once this is published, no doubt the next release of DNAC will add more pools – Murphy’s Law. And note, there may be some I have not yet encountered in the DNAC GUI, for whatever reason.

The IP pools come in two forms: per-VN and site-wide.

Per-VN pools:

Users or devices (laptop, up to N personal devices, desk IP phone, or other networked devices. Wired or wireless. Printers, etc. What else?)
Wireless users and devices, if using different addressing than wired addressing. I personally intend to keep it simple since SDA lets us treat wireless very much like wired.
IP multicast loopback pool.
(Spare per-VN pools. Because requirements will change over time.)

Site-wide pools:

LAN Automation
Fabric APs
Border pool for BNs
Infra_VN pool
(Spare pools for future needs)

Comments:

It is rather inefficient to assume all VN pools are the same size. That way, you end up multiplying your worst case by the number of VNs. I started with that for simplicity. It soon ate most of network 10. That’s a non-starter!
Because sites will likely vary in the mix of users and devices across VNs, do you really want to consider adjusting VN pool sizes to site-specific granularity? That sounds labor-intensive! Plus, if you get it wrong, it could get complicated.

The multicast loopback pool could be small; it needs to allow 1 per VN per device, as far as I can tell.

LAN Automation and IP Pools

LAN Automation provides a way to discover and manage devices in DNAC while provisioning them with standardized underlay connectivity.

The minimal IP pool for LAN automation is a /25. The pool should be “significantly larger” than the number of devices to be discovered. The devices to be discovered must all be at one site. Since CDP is used, I presume there should not be crosslinks to another site.

You will need routing to this pool from DNAC (and back). Of course, the pool must be unique.

For LAN Automation, you give it a global pool to use. It then automatically sub-divides it and uses it for three things:

Pool for a temporary DHCP server for devices
Pool for underlay link addressing
Pool for assigning a loopback per device

See the LAN Automation Deployment Guide link under References below for more details of this, plus other information, including how to do LAN Automation to discover and manage devices relatively quickly.

The Fine Manual says you can re-use a pool for more than one LAN Automation discovery. In that case, the pool should be at least a /24. I do not think this is a good idea. The point is that the LAN Automation pool is used for temporary DHCP addressing plus permanent addresses. When the LAN Automation completes, the devices will be using some of the addresses permanently. The DHCP block will not be in use but cannot be reclaimed, as far as I can see.

Links receive /30 addressing, so 4 IP addresses are consumed per link. Devices also get assigned a loopback. Another LAN automation sub-block is used for temporary DHCP addressing. You need to factor all this in when sizing the LAN automation pool.

If you plan on doing LAN Automation, you might add this to the IP pool spreadsheet discussed below. Note that you might want the pool might be out of a different address block, like 172.16.0.0/16. It can be convenient to have a clear distinction between underlay and overlay addressing.

In particular, if you’re doing SDA Transit, you can summarize both at the datacenter BNs. The overlay just needs to know to go to the datacenter BNs to get to the underlay, and vice versa. In fact, default is all the sites overlays generally need to have from the datacenter BNs. In effect, the underlay is “out there with the rest of the world.”

This alternative means having a pool for all underlay addresses and then assigning (reserving) per-site blocks out of that global pool. That sounds to me like another spreadsheet tab and then IPAM entries, tracking which underlay pool is at which site.

This is where having a regular topology at sites can help. If you have 1 BN or 2 BNs and all the access switches attach to them, then the calculation is simple. Don’t forget to allow for a crosslink when you have two BNs, connecting them.

Thus for 1 BN, you would have N links to access switches, where N is how many there are.

For 2 BNs, you would have 2 x N + 1: two links to each access switch, plus one crosslink between the BNs.

A Calculating Approach

I wanted to get my arms around the addressing implications and the likely network 10 consumption. So I built a spreadsheet. I’m going to describe it roughly so as to not propagate any bad assumptions or spreadsheet formula errors.

The spreadsheet works in terms of number /24 blocks needed.

Input: approximate number of users at the site. (For site-wide VNs, anyway.)

Per VN:

Calculate 4.25 times that, divided by approximately 250 usable addresses per /24, and round up. The 4 is for the user, 2 personal devices, and a desk phone. The .25 is for printers and shared devices. That gives the number of users/device /24 blocks per VN. Alternatively, use the number of active ports, and adjust if moving to IP phones, merging wired / WLAN, or otherwise increasing the number of addresses needed.
If you later (or upfront) create a VN for IoT devices, is that going to be enough addresses? Something to consider, hard to predict. IOT could be a lot of devices.
I don’t feel the need to address WLAN users separately, and it just complicates the calculations. So, I assumed wireless users are the same as users on switch ports. No /24 blocks needed. It does mean the per-VN address pool(s) may have to be bigger to accommodate wired + WLAN users.
The IP multicast loopback pool uses one per VN per device, so 1 /24 is enough unless the Site is large. Per VN.
Spare pools per VN, I added enough /24s to add up to the next power of 2, since that summarizes well. Unless that got ridiculously large. YMMV. It’s your spreadsheet!
Add it all up, multiple by the number of VNs and spare VNs (which I also rounded up to the nearest power of 2). This is Subtotal #1.

Overall, not per-VN:

LAN Automation. See above.
Fabric AP addressing: 1 /24 unless there are more than 250 APs in a large site. They all get addresses, one per AP.
Border pool: addressing for the BNs. You’ll use a /30 for each VN, tunnels for SDA Transit addressing, etc. One /24 is probably plenty.
INFRA_VN addressing: 1 /24.
Spare site-wide pools: add enough to hit power of two.
Add them all up. This is Subtotal #2.

To finish, add the two subtotals.

For example, for 125 or 250 users, I ended up with 72 x /24s needed. Rounding up to 128, that’s a /17, which is a lot. Squeezing that down a little gets you to 64. For bigger, I end up with a /16 per Site.

The major contributing factor is that you’re leaving room for growth in the number of VNs, plus how many /24s per VN. In my case, we plan 3 VNs for now, with 5 for growth. That leads to 5 VNs time 8 /24s per VN or 40 /24s consumed right there, not being immediately used.

More recently, I changed over to the VN.site approach. I took one big block and carved it up for VN.site, then about five smaller blocks for the other pools. That works out pretty cleanly (and more simply) – but it sure consumes a lot of address space!

To deal with address sprawl, I / we then took that approach and did it for two sizes, with /20 and /22 per-VN addressing depending on site size. That helped with address consumption.

This is a work in progress, so there’s a pending discussion about maybe doing /21 and /23 (to reduce wasting address space due to many small sites of around 64 users).

Summarizability

The other major contributing factor is rounding up to a power of two. That can consume a lot of address space.

In DNAC, IP pools must be summarizable blocks. You do not actually put site pools in. But you don’t want to just grab the next 72 /24s in a row because the sub-blocks may not be summarizable. That’s why I put in rounding up.

One alternative would be to allocate just what you need right now, with a little padding to powers of two. And plan on doing similarly when more VNs drive the need for more IP pools. The drawback here is, you won’t have one tidy IP block per Site.

There are other alternatives, like carving up a couple of big blocks by sites, e.g., one block for the per-VN stuff and one for the per-site stuff. But simple address summarization goes out the window with that.

To sum up, I’m still grappling with how to do this cleanly. The above provides a fair amount of room for growth but cannot possibly anticipate all future needs.

Note: If you err on the low side and need more addresses for a VN, you can add more pools to it. From my perspective, it would be preferable for these to come from spare space within a site block.

Update

Allocating ¼ or ½ of 10.0.0.0/8 just did not seem wise.

We decided to allocate per-VN blocks “just in time” for two sizes, large and small. That allows for learning/changes as we go, plus new requirements. For instance, fabric site-wide user VNs require a user count suitably scaled up, but one or more IOT VNs might have many more devices per site, requiring larger blocks.

So we’re starting out with 4 blocks for the initial 4 VNs, with one perhaps being a /24 per site for guests (actually a couple of different guest VNs for sites belonging one of two management/financial groupings for <<reasons>>, knowing that no site will be both flavors). The other VNs are user-based, so /20 or /22 covers approx. 4000 or 1000 users, phones, etc., in the biggest VN at any site.

Anything bigger will have to be handled as a likely single exception, rather than burning up address space for a block to cover “very large” sites.

I hope there’s a hint there at an approach you might use. YMMV, of course.

Next Steps

The next step is to analyze the sites where SDA is to be deployed and classify them as to size.

That’s the other reason I built a spreadsheet. Once I got the formulas and layout, I copied the first tab and plugged in different numbers of users to see what resulted as far as /24 consumption.

I am currently working with a customer-provided spreadsheet, with per-site info: # of closets, biggest current closet switch stack and the number of active ports, the total number of active ports at the site, and total standalone switches or stacks (devices needing addressing). APs we’ll deal with separately. It’s a lot of work gathering accurate data for that!

I’m hoping that data will fall into 2 or 3 size “buckets.” We can then label each site as to size and start assigning small and large blocks of addresses, whatever size those turn out to be. Based on the per-VN per-site spreadsheet for IP pools for that size.

Replacing all the equipment at once isn’t going to happen (budget, staffing). So I imagine we will do something, and then perhaps make a mid-course correction.

Other considerations:

Having a good site by site inventory of switches and ports helps.
Rule #1 in Excel is never aggregate cells, treat it like a flat database, one row per site/closet/switch capturing number of ports and their speeds. That facilitates rolling up data by site. Color code all you want, but don’t merge cells.
With COVID, figuring out real port usage is harder, since people are doing WFH and the office switch ports are inactive. The flip side of that: having people absent from the site does help when doing equipment replacement and migration to SDA. Institutional knowledge helps here.
Outdoor wireless for large numbers (for distanced gatherings) may be a new Thing in some cases.

Conclusion

I hope this discussion proves useful in your SD-Access address planning!

References

Disclosure Statement