I’ve been trying to blog about NSX, DFA, and ACI for a while. I’ll blame Cisco — they’re now coming out with additional info. And I kept coming up with more Things I Don’t Know. Hence, deferred blogging, to avoid putting an ignorant foot in my mouth. I’m now equipped to place a smart foot in my mouth instead! That is, this will be my best effort to describe the new technology. This particular blog will necessarily be an brief overview. In subsequent blogs I hope to describe and contrast aspects of the three technologies in the title: NSX, DFA, and ACI. If you’re not familiar with those three acronyms, well, read on. That’s one goal for this particular blog, providing an overview of those three technologies.
Some Commentary
The timing seems perfect. I can now react to Greg Ferro’s (@etherealmind, PacketPushers.net) 12/12/2013 Network Computing blog Cisco ACI: Proceed At Your Peril, at http://www.networkcomputing.com/data-center/cisco-aci-proceed-at-your-peril/240164699, and Joe Onisick’s long response. (And no, we are not now calling Greg “Mr. Grouchy”, he has some somewhat valid concerns, although I have a somewhat different perspective. I’m generally positive and excited about ACI. I can’t wait to see how this plays out and how well Cisco executes on it.)
There’s a recent Cisco blog, triggered by Greg (or not), claiming lots of early interest: Cisco ACI: In the market, building momentum, at http://blogs.cisco.com/datacenter/cisco-aci-in-the-market-building-momentum-full-suite-on-schedule-for-q2-2014/. I’ve heard anecdotal information about one large firm I was a bit surprised to find exhibiting strong early interest.
For those not tracking my history with this, the Tech Field Day #TFD brought me to the Cisco ACI launch, at which we podcasted new raw opinions. Since then I (and many others, I suspect) have been searching for crumbs of information about ACI. I’ve also been looking for serious details about DFA since CiscoLive in June. Gideon Tam (@mfmahler) was kind enough to tweet links to relevant Cancun CiscoLive presentations. They provided some good initial information.
Last week I had the good fortune to get a full day of ACI info, plus some DFA info, via Cisco datacenter PVT webexes for partners. A *LOT* of info. No, I can’t explain about 6 hours of presentations in this blog. And there are still some areas where I think Cisco has yet to provide some of the details. Lack of info tends to make all of us nervous. I’m extending some trust here, because I see some very bright people and a lot of R&D energy pointed in one direction.
In this case, we seem to have an orchestrated increasing flow of information as ACI delivery gets closer. And it sounds like a lot of related items will be falling into place over the first half of 2014. Exciting! Cisco may be doing this to avoid overloading media with details, it may be because there are some surprises left to be announced.
My guess: Cisco is gradually laying its cards on the table, saving some hole cards. (But I’m terrible at poker.) I do get the sense that there is a long-orchestrated set of steps coming to fruition here, with some pieces from inside Cisco as well as from the former Insieme. OnePK may be a general play by Cisco that is also an enabler for what the ACI team needs to do. One way of perceiving DFA is as an evolutionary step in thinking about spine and leaf forwarding, well on the path towards ACI. It might also be a hedge that can serve a different market than ACI (see below).
One of the parts that Greg was concerned about was several parts that have to all work together — software and hardware integration. Well, yes, there’s still a chance of problems there. And there will inevitably be some early bugs, since software is involved. But well-defined API’s can certainly help with system integration. Any chip-related delays Insieme had might (or might not) have had, might also have allowed more testing and refinement of GUI, everything but the chips and OS code that have yet to appear. The Cisco blog above mentions an APIC simulator becoming available in 2014. It sounds similar to the UCS emulator Cisco has made to developers. Getting your GUI into a lot of hands is one way to (a) test it thoroughly, and (b) rapidly build peoples’ skills at using the GUI.
I do agree with Greg that few will jump right in and flip their datacenter to ACI. There will be pilots and incremental migration. However, some of my initial questions vanished when it became clear that ACI supports doing traditional networking, or all ACI policy-based networking, or a hybrid model.
And yes, there will need to be some cultural changes. Some of the biggest power of ACI might tie to things like meaningful DNS names for servers and VMs. So datacenters will need to be doing some planning to take full advantage of ACI moving forward. Joe Onisick had a good description of the UCS equivalent: people starting by naming their profiles for specific server instances, and then starting to abstract and generalize (my words, not his).
One of the gaps, for me, is that I still haven’t gotten a good description of how configuration details get instantiated in devices. Although there is a clear mechanism that has been described for 3rd party policy to configuration template add-ons, something that sounded a bit Puppet-like. The point may be to focus discussion on the policy discussion rather than on the details of how devices actually get configured. So we’ll just quietly defer that topic for a later blog.
A Useful Picture
To describe the three virtualization schemes, we have to recognize they have different goals and perspectives. My analogy for this is a well-known New Yorker cartoon/poster, the View of the World from 9th Avenue. A low-res version is at http://upload.wikimedia.org/wikipedia/en/4/4d/Steinberg_New_Yorker_Cover.png (link only: potential copyright concerns).
We’ll see how this is relevant in a moment — bear with me.
What is NSX?
For those who haven’t been following along, NSX is VMware / Nicira’s virtualization play. It offers the ability to instantiate and connect included virtual router / firewall / NAT functionality, as well as virtual machines (VMs). NSX uses VXLAN overlays between hypervisors to connect virtual devices at L2, overlaying a L2 or L3 physical network. It also provides routing to the external physical world.
For more info, see my blog Good Links, 10/14/2013 at https://netcraftsmen.com/blogs/entry/good-links-10-14-2013.html. Ivan Pepelnjak, Brad Hedlund, and Scott Lowe’s presentation on ipspace.net is one of the best resources I’ve seen so far. It is at http://demo.ipspace.net/get/NSX%20Architecture.pdf. The NSX Virtualization Design Guide is also interesting reading, at http://www.vmware.com/files/pdf/products/nsx/vmw-nsx-network-virtualization-design-guide.pdf.
NSX seems to me to be a very VM-centric approach. It makes it fairly easy to deploy virtual devices and connect them. That’s one of the aspects I don’t like as much, see VMware Product Walkthroughs, at http://www.vmwarewalkthroughs.com/. Great idea, by the way. But I came away with the reaction: cool, I can stand up a router, I have to create a virtual switch to connect it up — oh, I’m doing the virtual equivalent of mounting a router in a rack, then cabling it to a switch, and creating VLANs. That’s faster than physical. But not automated (yet?). And arguably a lot more clicking and details than I think we ought to have to do. Maybe templates will be coming to expedite things? It’s early days yet, and integration with cloud management tools is another piece of the puzzle.
The flavor of NSX came across to me as a bit like: NSX can connect all your VMs and virtual appliances, with the physical network relegated to just providing L2 or L3 connectivity. And oh, by the way, if you have physical appliances, NSX can connect them up too. (And I hear a little “but why would you want to?” in the back of my head.)
And here’s an (attempted) picture of that:
What is DFA?
DFA stands for Dynamic Fabric Automation. It appears to be a “semi-classic Nexus” centric approach, perhap intended as a hedge should ACI have problems, perhaps intended for those who don’t buy into ACI, or whose hardware doesn’t support it. DFA appears to be a bit more of a software integration play, leveraging DCNM, UCS Director, and 1000v if available. It also however leverages some new code / protocols in the Nexus 7700 hardware to offload some of the work. Specifically, FabricPath as tunneling (more or less), plus BGP and an address family for host routes, informing leaf switches how to do L3 forwarding on a per-host and context basis. More about this in later blogs.
I currently view DFA as a mostly Cisco datacenter hardware-centric approach, with some potential for automation and integration with VMware. And by the way, it may remove the need for the overlays NSX uses. Simplifying the potential issues of managing overlays and underlays and trying to tie problems in the overlay back to problems in the underlay.
This sort of looks like:
What is ACI?
ACI is the big launch from Cisco and Insieme as of 11/6/2013. ACI stands for Application Centric Infrastructure.
ACI is a policy-oriented approach, described as trying to bridge the language gap between developers and networking / datacenter technical staff. With automated provisioning per policy and templates. I see this as a way for Cisco to escape from “MQC hell”, where every time you wanted to do security, QoS, or other service functions, you’d have to describe essentially the same traffic flows for each function. Instead, ACI will let you define an application relationship and all the services it needs in one place. With RBAC (Role Based Access) so that different teams can configure their parts of the policy, if that is desired. And (eventually), policy based on DNS name or VM name awareness. (Will we need structured naming conventions? Stay tuned!)
ACI also has some aspects intended to provide very finely grained manageability, including the idea of activating counters and being able to see which hop is losing packets (or duplicating them). ACI also appears to be intended to work with orchestration software that drives the vSphere side of things. (Note to self: fill in some of the details on that!)
ACI is going to require writing policy rules so as to leverage pattern recognition and abstraction. Those with microscopic CLI and ACL vision are going to hate it. Those who want to lighten the configuration burden may come to appreciate the approach.
The hardware in ACI can terminate VLANs and VXLAN at the edge switches. Although ACI will apparently use a variant form of VXLAN internally in its switching fabric. The impact? It looks to me like this means ACI can play well with VMware NSX as well as vSphere, since every edge switch can be a hardware-based high performance VXLAN gateway (VTEP).
Maybe ACI provides more balance? And via API’s, a management product teaming approach to managing the datacenter?
Life Log
You can probably see why I have not considered a career in graphic arts.
Twitter: @pjwelcher
Impressive Logos
I feel that I need to point out that I wrote two articles about Cisco ACI. One highlighting the positive aspects of ACI and another considering the negatives. I even specifically highlighted the first article in the introduction. This is also known as balanced journalism where both sides of a debate are considered.
Like you, I feel that more of the ACI story will appear over time and the change and value it will bring to networking is enormous. After all, Cisco ACI, VMware NSX and all the other SDN solutions are getting close to delivering the SDN concepts I’ve been talking about for quite some time.
Unfortunately, I’m not able to access much information about ACI at this time. The Cisco website has registration walls across most of the ACI/APIC information and I’m not able to access useful information to update my knowledge. Hopefully we will get access to more information soon and see what there is to get excited about.
Liked your article a lot. One thing, you don’t mention Cisco XNC / OpenDaylight work which is also SDN controllers and applications. I think this product it’s equally important as ACI since many customers will be considering which one to choose. They are all good solutions
greg
Thanks, Greg. I was just taking the rare opportunity to give you a hard time. I did see your responses to Joe about pro/con articles and probably should have taken the time to track down the positive one. Good point about public info re ACI, I was feeling the same frustration over DFA — vague announcements about forthcoming wonderful at CiscoLive2013 then nothing much technical for months. Also good point re the SDN side, that’s another piece of ACI where what I’ve seen to far is a bit vague for me — degree of integration with SDN tools, degree of reliance on them to make policy turn into configuration. I create a policy, I plug in a box, presto, configuration appears in the boxes? Some details are presently missing there.
Hard time accepted!!
Anthony Burke @pandom_ was kind enough to point out via Twitter that: templates , blueprints and then catalogues can be made with vCloud director or vCAC to solve those issues 🙂 good post.
Thanks for pointing that out! I’m glad to know that NSX can be supplemented with those other products to do that, I hadn’t made that connection.
Pete,
Excellent post and thank you for the honorable mention! I just wanted everyone to know that it was Ivan Pepelnjak who really made the VMware NSX webinar a success. I highly recommend that any vendor with a networking related product consider hiring Ivan for a webinar.
One other thing.. with VMware NSX you can also point-and-click to deploy Firewalls and Load Balancers in seconds/minutes. And of course all of this point-and-click activity is actually just a human interface to the NSX API. Just as Anthony Burke said, you can take off the shelf, open source, or home grown cloud management software and point it at the very same NSX API (eg. vCAC, OpenStack). Any configuration you can point-and-click can also be defined in policy based templates at the CMS. So that CMS can auto-deploy a full L2-L7 virtual network for any application, in seconds/minutes, on any network, including (but not limited to) Cisco Nexus 7K/5K/2K, UCS, Nexus 7700, Nexus 9000, and so on. VMware NSX can sit side-by-side the existing physical workloads you show in the diagrams, and if necessary bridge or route those physical hosts into the NSX virtual networks with, you guessed it, point-and-click or API 🙂
Cheers,
Brad
Thanks for the comment, Brad. Yes, Ivan is a rock star, not only of the climbing variety!
Re the CMS side, good points. Appreciated.
Hi Pete,
Great article and it is exciting times at Cisco indeed! Just wanted to add the DFA is not just for the Nexus 7700, but also the Nexus 7000 with F3 modules, which are now shipping, will be able to participate in a DFA network when the software is released later this year. Nexus 7700 and Nexus 7000s can participate in a DFA network today as a spine node with shipping 6.2 code. I am looking forward to the rest of your series!
Ron, thanks! I had thought N7K as well but couldn’t find anything in print confirming that. I believe I also checked re F3 availability and didn’t find much. Good to hear that investment in N7K supported, albeit with upgrade to F3 linecard.
Pete, don’t you think that customers may get confused by the two architectures offered by Cisco (DFA and ACI)?
Mario: I agree, Cisco needs to position the hardware and the software solutions carefully and well. They have not done that well so far. Which switches play with DFA, and which with ACI? I find myself wondering if DFA is suffering from muffled publicity so as to not detract from ACI, or perhaps for other internal reasons, like marketing debating and politics. Or is it a hedge that Cisco is not talking about unless ACI develops problems? I believe I’ve heard that DFA is alive and well inside Cisco, but that message does not seem to be being clearly transmitted to the outside world.
That’s why I tackled all three in this blog (and intend to do more once I find some blogging time). I think seeing how DFA works is not only interesting but suggestive for what ACI is doing "under the hood", albeit with tweaks to protocols Cisco already has. With the policy APIC aspect being the biggest innovation. Well, the hardware approach too.
Thanks Pete,
and do you think that VMW may have a strategic advantage vs cisco becouse of its dominance in virtualized environments?
I see NSX as more of a cloud play, for the heavily virtualized datacenter.
Its weaknesses seem to me to be that it may be thin on the networking and routing protocols side, and the gatewaying may impose performance limitations to the physical side. No doubt they’ll work to improve those, and it’s early days yet. If the physical side consists of a powerful blade chassis that IS the datacenter, with a router and firewall for external connectivity, and a building core switch to get to wired and wireless users, maybe the datacenter can be mostly virtual. I’ll note a UCS chassis can hold a large number of VMs, and the number is increasing. With built-in networking and SAN.
Looking at the landscape, server admins tend to be interested in about 1% of what a Cisco router does, or often 10% of what an F5 SLB/ADC does, and that’s perhaps good enough for small to medium sized shops. From a business point of view, cost and "good enough" (and server admins carrying more weight than network admins at business mgmt levels?) might be determining factors? Especially if the ease of virtual network offsets that and provides agility? That’s the pessimistic or anti-Cisco view, anyway. It may play out that way in some organizations.
Unfortunately "good enough" may not be in some ways, e.g. how do you manage say 100 virtual firewalls?
If you’re dropping packets due to limitations of a virtual router or firewall, maybe you turned on a feature that slows it down, how do you even see that? (I’ve had enough quality time with physical IPS, WebSense, and Brand X firewalls: trying to determine if they were the source of slowness.)
The other weakness in NSX is not being able to tie overlay tunnel problems to underlay physical link or switch problems. Part of the ACI approach is the ability to do that well, with what looks to be very helpful troubleshooting tools.
With either, other considerations are cost and degree of vendor lockin.