We recently had a near-disaster occur at a customer. A 6500 10/100 ethernet blade had died and was replaced. When the replacement blade was installed, the configuration for its interfaces disappeared from the configuration! The techs couldn’t figure out what happened. They saved the current config as part of their normal operations, but now the old configs were missing from both the running and saved configurations. With the interface configs now missing, they called for assistance from the network engineering group.
If you want a good thought exercise, stop reading now, try to figure out what happened, then read the solution below.
The person handling the call from the technicians took a quick look and saw that both the running and saved configurations were now missing for all interfaces on that blade. He immediately thought of the NetMRI configuration repository and found the configuration archive going back many months. Looking through the saved configs, he found the most recent config that had the affected interfaces. The config was quickly exported to his laptop and he pasted the configs into the 6500.
To his surprise, the configs didn’t immediately work. (Have you figured out why?)
The 10/100 blade had been replaced with a 10/100/1000 blade. Not a problem, it would handle all the interface speeds and had more modern QoS support. A win all the way around, right?
The new blade’s interfaces were GigabitEthernet while the old interfaces were FastEthernet. The interface names in the original config didn’t match the new blade, so they were not set. When the techs saved the running config, it over-wrote the saved copy. The only other backup config was in NetMRI. When the engineer handling the call saw the configuration problem, it was easy for him to edit the interface configs to change the name and load them into the 6500. Problem solved!
There’s another critical component to this story. It is important to have a configuration repository that automatically discovers devices and collects their configurations. It is easy to forget to add a new device to the configuration collector, especially if the organization doesn’t have strong procedures in place for deployment of new network devices. One of the things that I like about NetMRI is that it automatically discovers new devices and will automatically begin collecting data, including the configurations (provided that it has the proper access credentials). Another factor is for the configuration collector to save all the configurations that it collects. It doesn’t take much disk space to save every copy of a configuration throughout the history of a device. Automatic functions like those in this example make it easier to run a network.
-Terry
_____________________________________________________________________________________________
Re-posted with Permission
NetCraftsmen would like to acknowledge Infoblox for their permission to re-post this article which originally appeared in the Applied Infrastructure blog under http://www.infoblox.com/en/communities/blogs.html
