Testing Egress Marking in NX-OS QoS – Part 2

Author
Carole Warner Reece
Architect

This is part 2 of my testing of egress marking on NX-OS, the first part is discussed in Testing Egress Marking in NX-OS QoS. The good news is that egress marking does indeed work. With the topology in my first test, I was unable to make egress marking work on a Nexus 7000 running NX-OS 5.2(4) because I had used too simplistic a test bed.

I’ll show you my latest lab results.

Flawed Test Topology

The issue in the first set of tests was with the SVI used to simulate one class of traffic. The flawed lab topology looked like this:

2012_09_11_nx-os-egress-policy-map

My outbound marking test focused on applying a policy to Eth 2/10 on N7K-1. The test was flawed in that it was trying to mark traffic sourced from the SVI (this was the return ping traffic.)  After my friend Jeremy mentioned that egress marking worked for him based on ACLs, I went back to the lab.

Working Test Topology

The revised lab topology looked like this:

2012_09_18_NX-OS-Egress-Policy-Map

I reused the same ACL, class-maps, and policy-map structure from the first test:

! QoS Trust Boundary Access-Lists
!
ip access-list QOS-VOICE
 permit ip any 10.120.11.0 0.0.0.255
 permit ip 10.120.11.0 0.0.0.255 any
!
! identify the voice traffic to be marked
class-map type qos match-any IN-VOICE
 description Voice/VoIP/IPT
 match access-group name QOS-VOICE
!

! identify marked traffic
class-map type qos match-all DSCP46
 match dscp 46
class-map type qos match-all DSCP1
 match dscp 1
!
! used to mark traffic from 65K on N7K-2
policy-map OUT-MARKING
  description Outbound classification/marking policy for trust boundaries.
  class IN-VOICE
    set dscp ef
!

!
! used to remark traffic on N7K-1
policy-map type qos CHANGE-DSCP
 class DSCP46
   set dscp 1
!

!
! used to verify traffic markings on N7K-2 and N7K-1
policy-map type qos IN-DSCP
 class DSCP46
   set dscp 46
 class DSCP1
   set dscp 1
!

Note: I did rename the old IN-MARKING policy-map to OUT-MARKING.

Applying the Policy-Maps on N7K-2

My first test was to see if outbound marking on the N7K-2 worked for traffic from 65K to the N7K-1. I applied the IN-MARKING policy map outbound, and the IN-DSCP policy inbound on Eth 2/10. I also applied the IN-DSCP policy inbound of Eth2/10 on N7K-1

! N7K-2
interface eth 2/10
 ...
 service-policy output OUT-MARKING
 service-policy input IN-DSCP
!
! N7K-1
interface eth 2/10
 ...
 service-policy input IN-DSCP
!

Verifying Outbound Marking

Next, a quick ping test from 65K to 10.12.2.1 on N7K-1. This worked fine, the outbound policy marked the traffic.

65K#ping 10.120.11.6

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.120.11.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms
65K#

. . .
N7K-2(config-if)# sh pol int e2/10

. . .

Service-policy (qos) output:   OUT-MARKING
 SNMP Policy Index:  285212996

 Class-map (qos):   IN-VOICE (match-any)

 Slot 1
 5 packets  590 bytes
 5 minute offered rate 0 bps

 Aggregate forwarded :
 5 packets  590 bytes
 Match: access-group QOS-VOICE
 5 packets
 set dscp 46

. . .
! looks fine
!
N7K-1# sh pol int e2/10


Global statistics status :   enabled

Ethernet2/10

 Service-policy (qos) input:   IN-DSCP
 SNMP Policy Index:  285212992

 Class-map (qos):   DSCP46 (match-all)

 Slot 2
 5 packets  590 bytes
 5 minute offered rate 0 bps

 Aggregate forwarded :
 5 packets  590 bytes
 Match: dscp 46
 set dscp 46

 Class-map (qos):   DSCP1 (match-all)

 Aggregate forwarded :
 0 packets
 Match: dscp 1
 set dscp 1

. . .
! looks fine

Testing Outbound Marking for Traffic from N7K-2
Next, what if the traffic is sourced from N7K-2?
 

!
N7K-2(config)# ping 10.120.11.6 sou 10.120.11.2 count 22
PING 10.120.11.6 (10.120.11.6) from 10.120.11.2: 56 data bytes
64 bytes from 10.120.11.6: icmp_seq=0 ttl=254 time=1.073 ms
64 bytes from 10.120.11.6: icmp_seq=1 ttl=254 time=0.725 ms
64 bytes from 10.120.11.6: icmp_seq=2 ttl=254 time=0.739 ms
64 bytes from 10.120.11.6: icmp_seq=3 ttl=254 time=0.515 ms
64 bytes from 10.120.11.6: icmp_seq=4 ttl=254 time=0.979 ms
64 bytes from 10.120.11.6: icmp_seq=5 ttl=254 time=0.629 ms
64 bytes from 10.120.11.6: icmp_seq=6 ttl=254 time=0.74 ms
64 bytes from 10.120.11.6: icmp_seq=7 ttl=254 time=0.621 ms
64 bytes from 10.120.11.6: icmp_seq=8 ttl=254 time=0.72 ms
64 bytes from 10.120.11.6: icmp_seq=9 ttl=254 time=0.627 ms
64 bytes from 10.120.11.6: icmp_seq=10 ttl=254 time=0.626 ms
64 bytes from 10.120.11.6: icmp_seq=11 ttl=254 time=0.697 ms
64 bytes from 10.120.11.6: icmp_seq=12 ttl=254 time=0.628 ms
64 bytes from 10.120.11.6: icmp_seq=13 ttl=254 time=0.626 ms
64 bytes from 10.120.11.6: icmp_seq=14 ttl=254 time=1.275 ms
64 bytes from 10.120.11.6: icmp_seq=15 ttl=254 time=0.625 ms
64 bytes from 10.120.11.6: icmp_seq=16 ttl=254 time=0.724 ms
64 bytes from 10.120.11.6: icmp_seq=17 ttl=254 time=0.693 ms
64 bytes from 10.120.11.6: icmp_seq=18 ttl=254 time=0.626 ms
64 bytes from 10.120.11.6: icmp_seq=19 ttl=254 time=0.625 ms
64 bytes from 10.120.11.6: icmp_seq=20 ttl=254 time=0.626 ms
64 bytes from 10.120.11.6: icmp_seq=21 ttl=254 time=0.627 ms

--- 10.120.11.6 ping statistics ---
22 packets transmitted, 22 packets received, 0.00% packet loss
round-trip min/avg/max = 0.515/0.716/1.275 ms
N7K-2(config-if)# sh pol int e2/10

. . .

Service-policy (qos) output:   OUT-MARKING
 SNMP Policy Index:  285212996

 Class-map (qos):   IN-VOICE (match-any)

 Slot 1
 5 packets  590 bytes
 5 minute offered rate 0 bps

 Aggregate forwarded :
 5 packets  590 bytes
 Match: access-group QOS-VOICE
 5 packets
 set dscp 46

. . .
! ah, traffic sourced from the N7K-2 is NOT marked
N7K-1# sh pol int e2/10

. . .

Global statistics status :   enabled

Ethernet2/10

 Service-policy (qos) input:   IN-DSCP
 SNMP Policy Index:  285212992

 Class-map (qos):   DSCP46 (match-all)

 Slot 2
 5 packets  590 bytes
 5 minute offered rate 0 bps

 Aggregate forwarded :
 5 packets  590 bytes
 Match: dscp 46
 set dscp 46

 Class-map (qos):   DSCP1 (match-all)

 Aggregate forwarded :
 0 packets
 Match: dscp 1
 set dscp 1
. . .

Key point – traffic sourced from the local device was not marked in the local device’s outbound policy map.

Testing Egress Re-Marking on N7K-1
My final test was to verify out-bound re-marking on the N7K-1 using the return traffic from N5K-1.

!
N7K-1(config-if)# int e2/10
N7K-1(config-if)# ser out CHANGE-DSCP
N7K-1(config-if)# no ser in IN-DSCP

. . .
65K#ping
Protocol [ip]:
Target IP address: 10.120.11.5
Repeat count [5]: 66
Datagram size [100]:
Timeout in seconds [2]:
Extended commands [n]:
Sweep range of sizes [n]:
Type escape sequence to abort.
Sending 66, 100-byte ICMP Echos to 10.120.11.5, timeout is 2 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (66/66), round-trip min/avg/max = 1/1/4 ms
65K#

. . .
N7K-1(config-if)# sh pol int e2/10

. . .
Global statistics status : enabled

Ethernet2/10

Service-policy (qos) output:   CHANGE-DSCP
 SNMP Policy Index:  285213050

 Class-map (qos):   DSCP46 (match-all)

 Slot 2
 66 packets  7788 bytes
 5 minute offered rate 0 bps

 Aggregate forwarded :
 66 packets  7788 bytes
 Match: dscp 46
 set dscp 1
. . .

! outbound marking based on DSCP values appears to be working
N7K-2(config-if)# sh pol int e1/10

. . .
Global statistics status : enabled

Ethernet1/10

 Service-policy (qos) input: IN-DSCP

 Class-map (qos): DSCP46 (match-all)

 Slot 2
 5 packets 590 bytes
   5 minute offered rate 0 bps

 Aggregate forwarded :
 5 packets 590 bytes
 Match: dscp 46
 set dscp 46

 Class-map (qos): DSCP1 (match-all)

 Slot 2
 66 packets 7788  bytes
  5 minute offered rate 0 bps

 Aggregate forwarded :
 66 packets 7788 bytes
 Match: dscp 1
 set dscp 1
. . .
! yes, N7K-1 is remarking the traffic based on the DSCP values

The egress policy on N7K-1 worked successfully – ping traffic that was marked outbound from N7K-2 was re-marked outbound from N7K-1.

Summary

When you test QoS in the lab, be mindful of where the traffic is sourced.

— cwr

_____________________________________________________________________________________________

If you would like some additional on NX-OS QoS, you may want to review the following articles: 

Leave a Reply

 

Nick Kelly

Cybersecurity Engineer, Cisco

Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.