Automated IOS Updates

Author
Terry Slattery
Principal Architect

You have a lot of switches and routers, more than you care to count. The IOS in each of them is now old and needs updating. You’re not looking forward to handling each one with a manual process. What are your options? Well, if you did five of them a day, you could do 25 per week. Ten a day would double that volume. Garnering support from a couple of team mates could get you to 75 to 150 per week. Even still, doing the upgrades is a painful process.

How do we make the task less painful? A good way to tackle something big is to decompose it into a set of smaller problems that can be independently tackled. If you have 500 network devices and a three-tier design, you may find that you still have about 200 edge switches of the same model, perhaps even with different IOS versions loaded. Here, you may need to inventory which devices have a given IOS version. NetMRI produces a useful inventory in that it shows the device models and the list of IOS versions running on all devices of that model. Other products have similar inventory reports. Using one of these reports, you can decide which devices to upgrade first. But that doesn’t remove the need to automate the upgrade process. Of course, having a standard network design for different classes of device makes it much easier to plan and implement your upgrades.

Automating the IOS upgrade process isn’t simple. There are a number of critical factors that should be taken into account.

  1. Is there enough flash memory for the upgrade?
  2. Can the new IOS fit into free space in flash, or must you delete the old version first?
  3. If there isn’t enough free memory and more than one IOS version is in flash, which version in flash should be deleted? Delete the oldest one, or the one that’s not being used? We’ve seen situations in which the oldest version in flash was the active running version. If you delete one of the two, is there then sufficient free memory in flash? Or should you make everything consistent and remove all IOS versions from flash and install the new version, giving you a consistent installation across all devices?
  4. Save the old IOS version prior to deleting it from flash, if you don’t already have a copy, in case you need to do a roll-back to the prior version.
  5. Save the current configuration to enable a roll-back. This is important for those upgrades in which the configuration will be re-written by the new IOS.
  6. Do a few ‘show’ commands to capture the state of key functions within the device, such as HSRP, routing protocols, number of routes, STP state, STP root bridge.
  7. Do the upgrade.
  8. Save the upgraded configuration.
  9. Execute the same ‘show’ commands as above and compare the results, identifying any devices that have not resumed the correct operational state.

Doing this much planning and preparation takes time. Fortunately, a lot of the preparation can be applied to multiple device classes.

The flexibility of doing all of the above functions dictates the need for a scripting tool like the one built into NetMRI. At CiscoLive, one of the Infoblox SEs told me that he was working on a script to help a customer do IOS upgrades on a large switched infrastructure. By incrementally testing the upgrade script on one device, then a few devices, then applying it across the network, it is not difficult to make sure that it works reliably. The result is a tool that saves a lot of customer time, employee stress, and implements the upgrades much faster than would be possible using manual methods.

How does this benefit the business? Automating IOS upgrades makes it easier to deploy new IOS features. An agile business is one that is leading its competition. Studies have found that the leading companies typically have a commanding lead over the other businesses in its category, so anything a business can do to be more agile than its competitors is beneficial to that business.

-Terry

_____________________________________________________________________________________________

Re-posted with Permission 

NetCraftsmen would like to acknowledge Infoblox for their permission to re-post this article which originally appeared in the Applied Infrastructure blog under http://www.infoblox.com/en/communities/blogs.html

infoblox-logo

Leave a Reply

 

Nick Kelly

Cybersecurity Engineer, Cisco

Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.