NCCM: Verify Configuration Changes

Author
Terry Slattery
Principal Architect

I’ve been doing a configuration change on over 500 devices of a customer’s network, and using NetMRI to install the change was extremely easy.  Below is the script that I used, and I thought it would be useful to describe how it works.   The overview of the process is that the script connects to each device in a work list, learns the desired MPLS VPNs that exist on the device, then adds a pair of import/export statements to the vrf configuration.  When it has finished the configuration for all the matching vrfs on a device, it does an ‘end’ and ‘write mem’ to save the modified configs.  Finally, I wanted to verify that the desired outcome was achieved, which was that a new route would be visible within the selected MPLS VPNs.  If the route didn’t exist, I wanted to receive a notification of the failure so that I could correct it.

The script I used is shown in sections below.  First, NetMRI automatically performed a login on each device.  Then the Action section of the script starts the process by doing a ‘show run‘, and entering config mode.  It then passes control to the Trigger: ProcessVRFs section to process the output of the ‘show run‘ command.

Script-Filter:
$Vendor eq "Cisco"   and
$sysDescr like /IOS/
Script-Variables:
$DEBUG word "set to 'debug-on' or 'debug-off'" string
#########################################################################
Action:
Show Run
Action-Description:
Execute 'show run', then enter config mode
Action-Commands:
show run
config t
Output-Triggers:
ProcessVRFs
SaveConfig

The Trigger: ProcessVRFs section of the script contains a Trigger-Variables definition to match the vrf name and a Trigger-Template that matches any command in the output that looks like ‘ip vrf testXX’, where XX is a two-digit number (each d matches one digit). [Note: It should be clear to you from this example that naming standards are good – it makes automated configuration updates much easier to implement.] The Trigger-Commands section then cycles through each of the vrf names, issuing the commands to configure the route-targets and exit vrf config mode. A sleep action is performed after each set of configuration commands and then the route is checked. Note that the variable $success is set to ‘no’ for each vrf that is processed, prior to handing control to the Output-Triggers, which are CheckVRFRoute and IssueRouteFailed. Because CheckVRFRoute gets control next, it parses the output of the ‘show ip route vrf’ command.

#########################################################################
Trigger: ProcessVRFs
Trigger-Description:
For each 'ip vrf testXX', add the import/export statements
Trigger-Variables:
$vrfName/testdd/ string
Trigger-Template:
ip vrf [[$vrfName]]
Trigger-Commands:
ip vrf $vrfName
 route-target export 10:6
 route-target import 6:10
exit
SLEEP: 10
do show ip route vrf $vrfName
SET: $success = "no"
Output-Triggers:
CheckVRFRoute
IssueRouteFailed

The Trigger: CheckVRFRoute section checks that the desired route exists in each vrf by matching the pattern defined in the Trigger-Template section. If the template matches the output from the ‘show ip route vrf’ command from the ProcessVRFs Trigger section, the desired route exists and the script sets the variable $success = “yes”.

#########################################################################
Trigger: CheckVRFRoute
Trigger-Description:
Verify that the route to the hub now exists in the VRF.
Trigger-Template:
10.10.10.0/28
Trigger-Commands:
SET: $success = "yes"

The Trigger: SaveConfig section exits config mode and saves the config after the ProcessVRFs section has finished adding the import/export commands to the vrfs.

#########################################################################
Trigger:
SaveConfig
Trigger-Description:
All VRFs have been processed, save the updated config.
Trigger-Filter:
$DEBUG eq "debug-off"
Trigger-Commands:
end
wr mem

Finally, the Issue: IssueRouteFailed section gets control. It includes an Issue-Filter that will only create an issue if the variable $success is set to ‘no’, which indicates that the desired route was not found by the CheckVRFRoute Trigger section. By creating an issue only for failures, I had an easy way to run the script on hundreds of devices and get a notification only if a device failed to show the desired route.

#########################################################################
Issue: IssueRouteFailed
Issue-ID: testVRFFailed
Issue-Severity:Error
Issue-Description:
The route to the services subnet does not appear in the test vrf.
Issue-Filter:
$success eq "no"
Issue-Details:
Host$IPAddress
Name$Name
VRF$vrfName

The script took only a few minutes to check over 500 devices, making my weekend change window a relaxing event rather a stressful day driving the CLI on a lot of devices.  [Note: Just glue each of the above sections together to recreate the script.] There are several other ways that this script could have been created. For example, the SLEEP could have been moved outside the loop where each vrf is modified, but that would have required an additional Trigger section.

-Terry

_____________________________________________________________________________________________

Re-posted with Permission 

NetCraftsmen would like to acknowledge Infoblox for their permission to re-post this article which originally appeared in the Applied Infrastructure blog under http://www.infoblox.com/en/communities/blogs.html

infoblox-logo

Leave a Reply

 

Nick Kelly

Cybersecurity Engineer, Cisco

Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.