I’m writing this at and after the Cisco / Insieme #ACI (Application Centric Infrastructure) launch today in New York City. There’s much to cover! Recall that in some prior blogs I threw out a bunch of guesses in a prior blog, and a couple of things I hoped for. I’m amused that some of them appear to actually be part of the announcements, or unnannounced intent. The announcements today were broad in coverage and to some extent short in detail. In particular, ACI details are still emerging.
This will be a two part (at least) blog, with this one focusing on the hardware / optics side of announcements. It is already long enough, just covering that material. I hope to garner a bit more ACI information today and follow up with what I can in a Part 2 soon, focusing on ACI.
As more comes out, we may all be amazed at the breadth of the vision. Some of the key Insieme personnel are … visionary, intense! With UCS and now perhaps ACI, Cisco may be demonstrating that there is virtue in allowing a new technology to settle a little, to have more time to analyze what is key in a market or technology. They have been participating: OnePK to provide agility, ability to change direction quickly, OpenDaylight for cloud automation support, 1000v as early service-chaining and to build skills working with the lead hypervisors and learn how to provide consistent functionality across hypervisors. And it appears that ACI now their attempt to define the market and stake out a position, with much more vision that the innovation around the edges that some other network device vendors have produced.
My wishlist that I’m referring to includes:
- Packaging up a policy or dynamic template describing an application’s deployment needs and perhaps our policy. That is apparently present, incorporating security and QoS as well.
- Reducing complexity — for policy (and automation?), who cares about the IP, the TCP/UDP port, the VLAN — just make it work. Per some informal (i.e. bar) discussions sounded like this may be part of the intent. Deployment patterns for policy, rather than specifics.
- Some way to do roughly what VMware vShieldEdge and NSX do / promise to do, as far as cloning a vApp group of VMs, i.e. a service / application deployment pattern, but without the NAT!
A good starting point is what was announced. I was a bit surprised there wasn’t any one summary presented listing all the new stuff! The announcements are definitely a big deal, a bid to manage and automate the whole datacenter and cloud, from network to server, application, storage, and security, encompassing both physical and virtual. I’ve read the various blogs to try to pull together a comprehensive list.
Here’s my own list of the major announcement items, culled from various sources:
- Cisco acquisition of the remaining shares of Insieme. (Not a big surprise!)
- New hardware: Nexus 9000, featuring “the 5 P’s” (port density, low power, price point, performance, programmability)
- Insieme ASIC: ACI chips on line cards (when they ship). Or ability to do OpenFlow.
- APIC = Application Policy Infrastructure Controller
- Optics: 40 Gbps bi-dir, using 2 strands MMF and LC connectors, for 10 G to 40 G cabling re-use.
- Ecosystem of partners, excited large-scale customers (see the whole set of ACI Solution documents, which are partner whitepapers about product integration, etc.)
- Application Virtual Switch (follow-on to 1000v, providing consistent cross-hypervisor support for ACI control and management)
- That’s just the tip of the iceberg. As you read the following links (or deeper in this blog) you’ll see the complexity and magnitude of what was announced.
Related Links
For additional information, unfiltered through my brain, see also the following URLs:
- http://online.wsj.com/article/PR-CO-20131106-911147.html?dsk=y
- http://gigaom.com/2013/11/06/cisco-sdn-news-new-switches-now-a-controller-later-and-insieme-comes-home
- http://www.networkworld.com/news/2013/110613-cisco-insieme-275666.html
- http://www.networkcomputing.com/data-center/cisco-unveils-insieme-sdn-platform-new-s/240163611
- http://www.cisco.com/en/US/products/ps13386/index.html
- http://www.cisco.com/en/US/solutions/ns340/ns517/ns224/ns945/app-centric-infrastructure.html (or http://www.cisco.com/go/aci)
Initial Impressions
- Cisco is pre-announcing its controller and architecture, in effect staking out position and mindshare.
- I’d like to hear approximately what it’ll cost.
- Concerning reliability, I hear that Cisco and some large beta customers are testing now. So hopefully any egregious bugs will be fixed by the time the rest of us see it.
- The policy-driven approach to automate and facilitate app deployment does require cultural / operations changes.
- Who will provide the policy info? For major apps, the vendor, probably. For home-grown apps, it’ll have to become another contractual deliverable. Is there a ideal world versus reality gap lurking here? I’ve certainly seen enough apps where getting security port info has been painful.
- The story sounds good. I of course want more details. What will it do when installing a new app per profile? How well will it play with VMware and other hypervisors? What will the tie-in be?
- The new hardware is pretty impressive. (See below.)
- The 40 G bidir optics also look quite useful.
Let’s look at some of the most noteworthy details, easy ones first…
The Nexus 9K Hardware
Nexus 9508 takes up to 6 fabric modules at 5.12 Tbps each forwarding, directly connected to line cards.
The three line cards available are:
- 36 x 40 G QSFP+, non-blokcing, no ACI mode upgrade
- 48 x 1/10 SFP/SFP+ + 4 x 40 Gbps QSFP+ card, supports N2K FEX, direct 10 G copper or optical transceivers, can be used in ACI leaf configurations
- 48 x 100M/1/10 GBase-T + 4 x 40 Gbps QSFP+, for mid-row use, supports, can be used in ACI leaf configurations
This switch can be a big edge switch, leveraging N2K FEX. Or an aggregation layer switch. It seems positioned as a much faster 6500 replacement in these roles. Another use case is leaf or spine in a leaf-spine CLOS tree, possibly with the 9300 platform as leaf nodes. At least they didn’t call that “SPLINE” the way Arista did. (The TFD crew seems to not like that term.)
A diagram shows up to 12 spine 9500 switches, and claims support for scaling to 200,000 x 10 G servers or more. (With FEX and some FEX oversubscription, I imagine.)
One NX-OS binary supports the N9K switches. NX-OS Plus is required to support the ASIC controller.
See the datasheet (link above) for details. And you can play the game, “here’s the impressive long list of features, but what did they leave OUT?”
The Nexus 9300 is a ToR switch, with 2 models:
- The 9296PX is 2 RU high, with 48 fixes 10 Gbps SFP+ ports and 12 x 40 Gbps QSFP+ ports in an uplink module.
- The 93128TX is 3 RU high, with 96 fixed 1/10 GBase-T and 8 x 40 Gbps QSFP+ ports in the same uplink module which contains 12 x 40 Gbps ports
The Nexus 9300 also supports N2K FEX.
For those paying attention, the N9K series supports VXLAN. And 100 G line cards are coming, with adequate fabric capacity to support them.
One aspect not prominent in the announcements was positioning and pricing. How are these different than the Nexus 7K/5K in terms of features? If the price per port is lower, is there some displacement of the earlier Nexus models (“virtual end of life”) coming? And while one can run them using NX-OS, the ACI chipset and central APIC controller mode is the really interesting alternative.
The N9K is a mix of merchant silicon and customer ACI logic (per line card).
40 G BiDir Optics
The Cisco QSFP 40 Gbps BiDirectional (“BiDi”) transceiver uses duplex LC connectors for 40 G over MMF at short distances. The impact is that you can use your 10 G MMF fiber plant in the datacenter for 40 G connections, as is, at distances up to 100 meters over OM3 fiber. The cost is apparently about twice that of a short distance 10 G SFP+. The transceiver apparently uses two wavelengths on each strand (one for each direction) forming two 20 Gbps channels.
Application Virtual Switch
The Cisco Application Virtual Switch (AVS) isn’t hardware, but appears more straightforward to explain than ACI and profiles. Its product page is http://www.cisco.com/en/US/products/ps13463/index.html.
“Based on the highly successful Cisco Nexus 1000V virtual switch, AVS provides feature support for the ACI application policy model, full switching capabilities, and more advanced telemetry features.”
The role of AVS appears to be a 1000v footprint for each hypervisor, that ties it back into the Cisco ACI system. Part 2 will cover the telemetry aspect.
One item that caught my eye: “Compared to other hypervisor-based virtual switches, AVS provides cross-consistency in features, management, and control through Application Policy Infrastructure Center (APIC), rather than through hypervisor-specific management stations. As a key component of the overall ACI framework, AVS allows for intelligent policy enforcement and optimal traffic steering for virtual applications.”
That is, AVS provides a hypervisor-neutral set of functionality. That seems like it may appeal to those trying to avoid being locked into one hypervisor vendor, and looking for cloud portability perhaps as well. (From what I hear, VMware pricing has incented many customers to hedge their bets for fear of hypervisor lock-in subjecting them to future licensing fee shock.)
About Life Log
The trip to NYC for the ACI Launch provided quite a bit of fun and intellectual stimulation. I’d like to thank Cisco, especially Amy Lewis (@CommsNinja), and Tech Field Day in the person of Tom Hollingsworth (@networkingnerd) (http://networkingnerd.net/), for inviting me! And the rest of the TFD folks, for the interesting discussions!
Twitter: @pjwelcher