(Configuring) QoS on the Nexus 5000/2000 – Part 2

This is the second article on my series on QoS on the Nexus 5000 (N5K) and Nexus 2000 (N2K) for medical grade networks. The first article was QoS on the Nexus 5000/2000 – Part 1. (I apologize it took me so long to get back to this series!)

In the previous article, I discussed some MQC background as well as key aspects of QoS on Nexus 5000/2000 switches. I also discussed queuing and queue structure on Nexus 5000 and 2000 switches. (I don’t mind if you want to go review that article before going on…)

I also provided a brief outline of the six steps to configure QoS on a Nexus 5000/2000 switch:

1. Configuring global traffic classification based on COS to support the N2K
2. Configuring more specific traffic classification for interfaces based on DSCP or ACLs to support the non- N2K ports with the standard medical grade traffic classifications
3. Marking COS based on qos-groups for traffic sent to N2K ports
4. Configuring egress scheduling and queuing based on qos-groups
5. Establishing the system qos configuration
6. Applying a traffic classification policy based on DSCP to support the non- N2K ports with the standard medical grade traffic classifications

For this article I want to build on the outlined steps and provide some samples for configuring medical grade QoS on the Nexus 5000/2000. In my scenario, I want to classify traffic based on DSCP when possible, and will use COS when I must.

I will be using the following QoS Classifications for Medical Grade Networks:

The example environment includes Nexus 5548s as well as Nexus 2232s. The Nexus 5000/2000 switches will be the trust boundary for edge devices and will follow the same ACL configuration guidelines defined elsewhere in the enterprise.

 Configuring Global Traffic Classification Based on COS
At this time the Nexus 2148, Nexus 2232, and Nexus 2248 modules can only support CoS based traffic classification. Therefore, I use a global classification policy based on COS that is applied globally all N5K/N2K ports using a type qos policy and type QOS class-maps as follows:

! To support Nexus 2000s, classify traffic by COS.
! This classification will be for global COS to qos-group mapping.
class-map type qos match-all COS1-GLOBAL
 match cos 1
class-map type qos match-all COS2-GLOBAL
 match cos 2
class-map type qos match-all COS3-GLOBAL
 match cos 3
class-map type qos match-all COS4-GLOBAL
 match cos 4
class-map type qos match-all COS5-GLOBAL
 match cos 5
class-map type qos match-all COS6-GLOBAL
 match cos 6
! The default class already matches COS 0.
!
! Assign each global COS class to a qos-group in a qos policy-map
policy-map type qos CLASSIFY-GLOBAL
 class COS6-GLOBAL
 set qos-group 3
 class COS5-GLOBAL
 set qos-group 5
 class COS4-GLOBAL
 set qos-group 4
 class COS3-GLOBAL
 set qos-group 3
 class COS2-GLOBAL
 set qos-group 3
 class COS1-GLOBAL
 set qos-group 2
! The default class is assigned to qos-group 0 by default.

Note: Although the N5548 supports user configured qos-group 1, I deliberately do not use qos-group 1 because the Nexus 5010 and 5020 switches hard code qos-group 1 for FCoE traffic.

As needed, I also hard code COS values for traffic received from the host ports on the N2K:

!
interface “interface n2k/mod/port[-port]
 untagged cos 0
! set the CoS value to 0 for traffic received on the host interfaces
!interface “interface n2k/mod/port,n2k/mod/port
!  untagged cos 5
! selectively apply a CoS value for voice/UC servers as needed
end

 Configuring Traffic Classification Based on DSCP
For the ingress N5K ports, I define more specific classes that match on DSCP by using ACLs to match medical grade traffic. Here are some examples:

ip access-list QOS-NETWORK-CONTROL
 remark Network protocol traffic
 permit pim any any
 permit tcp any any eq bgp
 permit tcp any eq bgp any
!
ip access-list QOS-SIGNALING
 remark Deny voice, video, and audio streams expressly NOT related to
 remark QoS call signaling functions.
 remark SCCP and Secure SCP
 permit tcp any any eq 2000
 permit tcp any any eq 2443
 ...
 remark H323 Q931 and H225 Call Signaling
 permit udp any any eq 1719
 permit tcp any any eq 1720
!
ip access-list QOS-VOICE
 remark Phone media traffic between valid endpoints (IP phones, IPTgateways,
 remark & IPT servers)
 permit udp 10.16.0.0 0.15.255.255 range 16384 32768 10.16.0.0 0.15.255.255 range 16384 32768
!
ip access-list QOS-CLINICAL-LIFE-CRITICAL
 remark Clinical Life Critical traffic between associated clinical subnets.
 remark This ACL deliberately left blank - Reserved for Future Use
!
ip access-list QOS-MULTIMEDIA-CONFERENCING
 remark Multimedia Conferencing traffic between sanctioned applications and
 remark associated TCP/UDP ports.
 !
 remark Traversal Calls
 permit udp any gt 1023 10.96.0.0 0.15.255.63 gt 1023
 permit udp 10.96.0.0 0.15.255.63 gt 1023 any gt 1023
!
ip access-list QOS-REAL-TIME-INTERACTIVE
 remark Real-Time Interactive traffic (i.e. Immersive TelePresence)
 permit udp 10.96.0.64 0.15.255.63 gt 1023 10.96.0.64 0.15.255.63 gt 1023
!
ip access-list QOS-MULTIMEDIA-STREAMING
 remark Multimedia Streaming traffic from sanctioned streaming audio/video endpoints
 remark at the access layer.
 permit udp 10.96.0.128 0.15.255.63 gt 1023 10.96.0.128 0.15.255.63 gt 1023
!
ip access-list QOS-LOW-LATENCY-DATA
 remark Latency sensitive Data application traffic
 remark This ACL deliberately left blank - Reserved for Future Use
!
ip access-list QOS-NETWORK-MANAGEMENT
! Might want to list out net-management subnets
! if high-volume telnet application traffic is observed.
 remark Utilities for access to network devices.
 permit tcp any any eq telnet
 permit tcp any eq telnet any
 permit tcp any any eq 22
 permit tcp any eq 22 any
 permit udp any any eq syslog
 permit udp any eq syslog any
 permit tcp any any eq tacacs
 permit tcp any eq tacacs any
 ...
!
ip access-list QOS-HIGH-THROUGHPUT-DATA
 remark High-throughput data traffic (FTP, etc.)
 remark This ACL deliberately left blank - Reserved for Future Use
!
ip access-list QOS-LOW-PRIORITY-DATA
 remark Low priority (i.e. Scavenger) traffic from non-business oriented applications
! Add additional internal networks as necessary
 remark --- External Networks ---
! Add external networks as necessary
 remark ------
 permit ip any any
!

Class-maps will use these ACLs:

!
class-map type qos match-any IN-NETWORK-CONTROL
 description Network Control
 match access-group name QOS-NETWORK-CONTROL
!
class-map type qos match-any IN-VOICE
 description Voice/VoIP/IPT
 match access-group name QOS-VOICE
!
class-map type qos match-any IN-CLINICAL-LIFE-CRITICAL
 description Clinical Life Critical
 match access-group name QOS-CLINICAL-LIFE-CRITICAL
!
class-map type qos match-any IN-MULTIMEDIA-CONFERENCING
 description Multimedia Conferencing
 match access-group name QOS-MULTIMEDIA-CONFERENCING
!
class-map type qos match-any IN-REAL-TIME-INTERACTIVE
 description Real-Time Interactive (i.e. Immersive TelePresence)
 match access-group name QOS-REAL-TIME-INTERACTIVE
!
class-map type qos match-any IN-MULTIMEDIA-STREAMING
 description Multimedia Streaming
 match access-group name QOS-MULTIMEDIA-STREAMING
!
class-map type qos match-any IN-SIGNALING
 description Call Signaling
 match access-group name QOS-SIGNALING
!
class-map type qos match-any IN-LOW-LATENCY-DATA
 description Low-Latency Data (i.e. Scavenger Class)
 match access-group name QOS-LOW-LATENCY-DATA
!
class-map type qos match-any IN-NETWORK-MANAGEMENT
 description Network Management
 match access-group name QOS-NETWORK-MANAGEMENT
!
class-map type qos match-any IN-HIGH-THROUGHPUT-DATA
 description High-throughput data traffic (FTP, etc.)
 match access-group name QOS-HIGH-THROUGHPUT-DATA
!
class-map type qos match-any IN-LOW-PRIORITY-DATA
 description Low priority data applications (i.e. non-business apps)
 match access-group name QOS-LOW-PRIORITY-DATA
!

These classes are assigned to a qos-group and marked with a DSCP value using a policy-map. The policy-map will later be applied to the N5K interfaces. Note that multiple traffic classes are assigned to the same qos-group:

policy-map type qos IN-MARKING-5K
 description Inbound classification/marking policy for trust boundaries.
 class IN-VOICE
 set dscp ef
 set qos-group 5
 class IN-CLINICAL-LIFE-CRITICAL
 set dscp cs5
 set qos-group 5
!
 class IN-MULTIMEDIA-CONFERENCING
 set dscp af41
 set qos-group 4
 class IN-REAL-TIME-INTERACTIVE
 set dscp cs4
 set qos-group 4
!
class IN-NETWORK-CONTROL
 set dscp CS6
 set qos-group 3
! 
 class IN-MULTIMEDIA-STREAMING
 set dscp af31
 set qos-group 3
 class IN-SIGNALING
 set dscp cs3
 set qos-group 3
 class IN-LOW-LATENCY-DATA
 set dscp af21
 set qos-group 3
 class IN-NETWORK-MANAGEMENT
 set dscp cs2
 set qos-group 3
!
 class IN-HIGH-THROUGHPUT-DATA
 set dscp af11
 set qos-group 2
 class IN-LOW-PRIORITY-DATA
 set dscp cs1
 set qos-group 2
!
 class CLASS-DEFAULT
 set dscp default
end

This service-policy will be applied all the physical interfaces and port-channel interfaces to classify traffic based on DSCP except the FEX host interface range Ethernet 100/1/1- 199/x/x. The more specific IN-MARKING-5K interface policy will take precedence over the global CLASSIFY-GLOBAL policy.

Note: For a portchannel member, the service-policy needs to be configured under the interface port-channel, not to individual channel ports.

interface “ethernet mod/port[-port]” or “port-channel #”
! Use a range of interfaces if possible.
! Use a range of interfaces if possible.
 service-policy input IN-MARKING-5K
!
end

 Marking COS Based on QOS-Groups for Traffic Sent to N2K Ports
Since the N2K only supports COS based QOS, a policy is needed to establish the COS value of traffic from the N5K ports. This traffic has been marked with a DSCP value, and placed in a qos-group. The GLOBALNETWORKPOLICY is a global policy that marks COS values based on existing qos-groups, allowing traffic from the N5K to be queued and scheduled on the N2K.

The following type network-qos configuration commands support the global policy that marks COS values based on existing qos-groups.

!
! Set CoS based on qos-groups for traffic sent from the N5K to the N2K
class-map type network-qos COS5
 match qos-group 5
class-map type network-qos COS4
 match qos-group 4
class-map type network-qos COS6-3-2
 match qos-group 3
class-map type network-qos COS1
 match qos-group 2
!
policy-map type network-qos GLOBALNETWORKPOLICY
 class type network-qos COS5
 set cos 5
 mtu 9216
 class type network-qos COS4
 set cos 4
 mtu 9216
 class type network-qos COS6-3-2
 set cos 3
 mtu 9216
 class type network-qos COS1
 set cos 1
 mtu 9216
!

Note: the “mtu 9216” command is used to set the jumbo mtu size globally. To verify this setting on a N5K, you need to use the “show queuing interface ethX/Y” command because the standard “show interface ethX/Y” will fib and incorrectly show a MTU of 1500.

 Configuring Egress Scheduling and Queuing Based on QOS-Groups
The egress queue structures for the Nexus 5000/Nexus 2232/Nexus 2248 consists of 6 hardware queues for data plane traffic. All traffic in one of the six qos-groups is sent to a queue. If you add a system class, a queue is assigned to the class. Bandwidth is not automatically dedicated to user-defined system classes so must be manually configured. For this environment, we will be using 5 of the hardware queues.

Note: There are only 2 hardware queues for data plane traffic on the Nexus 2148.

The following type queuing configuration commands support the transmit queue assignments and bandwidth allocations:

class-map type queuing QUEUING-COS5
 match qos-group 5
class-map type queuing QUEUING-COS4
 match qos-group 4
class-map type queuing QUEUING-COS6-3-2
 match qos-group 3
class-map type queuing QUEUING-COS1
 match qos-group 2
! a class-default is predefined which matches qos-group 0
!
!
policy-map type queuing QUEUING-GLOBAL
 class type queuing class-default
 bandwidth percent 25
 class type queuing class-fcoe
 bandwidth percent 0
!
! Before you can successfully allocate bandwidth to a class you must
! first reduce the default bandwidth configuration on class-default
! and class-fcoe
!
 class type queuing QUEUING-COS5
 priority
 bandwidth percent 20
! one class in each policy map can have strict priority set on it
 class type queuing QUEUING-COS4
 bandwidth percent 10
 class type queuing QUEUING-COS6-3-2
 bandwidth percent 40
 class type queuing QUEUING-COS1
 bandwidth percent 5
!

 Establishing the System QoS Configuration
The global system qos configuration on a Nexus 5000/2000 switch is used for the N2K ports.

!
system qos
 service-policy type qos input CLASSIFY-GLOBAL
!

The input qos policy allows the CoS based classification to be applied globally to support the N2K ports.

Follow these guidelines when globally establishing the system qos configuration on a Nexus 5000 switch.

!
system qos
 service-policy type queuing output QUEUING-GLOBAL
  ! The output queueing policy determines the bandwidth allocation for both
  ! N5k interfaces and N2K host interfaces.
 service-policy type queuing input QUEUING-GLOBAL
  ! The input queuing policy determines how bandwidth is shared for the N2K
  ! uplink in the  direction from N2K to N5k.
 service-policy type network-qos GLOBALNETWORKPOLICY
  ! The network-qos policy allows queuing and scheduling of traffic
  ! from the N5K to the N2K.
!

The global system qos configuration on a Nexus 5000/2000 switch is used unless service policies are configured at the interface level.

Note: Since they are more specific, an interface-level policy always takes precedence over system class configuration or the default configuration.

Applying a traffic classification policy based on DSCP to support the non- N2K ports
The last step is to apply the interface level QoS configuration based on DSCP.

!
interface “ethernet mod/port[-port]” or “port-channel #”
  ! Use a range of interfaces if possible.
 service-policy input IN-MARKING-5K
!
end

The more specific service-policy on the physical interface is used to classify traffic based on DSCP on each non-FEX interface.

Note: For a portchannel member, the service-policy needs to be configured under the interface port-channel, not to individual channel ports.

I will discuss a sample configuration for Trust Boundaries in my Trust Boundaries and QoS on the Nexus 5000/2000 – Part 3 article in this series.

— cwr

_________________________________________________________________________________________

More on Medical Grade Networks
For more information on medical grade networks, you may want to review the following articles:

• QoS on the Nexus 5000/2000 – Part 1
• Trust Boundaries and QoS on the Nexus 5000/2000 – Part 3
• Designing a Medical Grade Network
• Healthcare Network Pain: Causes and Treatments
• Medical Grade Network Design and Operation

More on Nexus 5000
For more information on Nexus 5000 designs, you may want to review the following articles: 

• Configuring Back-to-Back vPCs on Cisco Nexus Switches
• The Nexus 5000 and Nexus 2000: Tidbits and Gotchas
• Designing with the Nexus 5000 / 2000
• QoS on the Nexus 5000/2000 – Part 1
• Trust Boundaries and QoS on the Nexus 5000/2000 – Part 3