This is the second article on my series on QoS on the Nexus 5000 (N5K) and Nexus 2000 (N2K) for medical grade networks. The first article was QoS on the Nexus 5000/2000 – Part 1. (I apologize it took me so long to get back to this series!)
In the previous article, I discussed some MQC background as well as key aspects of QoS on Nexus 5000/2000 switches. I also discussed queuing and queue structure on Nexus 5000 and 2000 switches. (I don’t mind if you want to go review that article before going on…)
I also provided a brief outline of the six steps to configure QoS on a Nexus 5000/2000 switch:
- Configuring global traffic classification based on COS to support the N2K
- Configuring more specific traffic classification for interfaces based on DSCP or ACLs to support the non- N2K ports with the standard medical grade traffic classifications
- Marking COS based on qos-groups for traffic sent to N2K ports
- Configuring egress scheduling and queuing based on qos-groups
- Establishing the system qos configuration
- Applying a traffic classification policy based on DSCP to support the non- N2K ports with the standard medical grade traffic classifications
For this article I want to build on the outlined steps and provide some samples for configuring medical grade QoS on the Nexus 5000/2000. In my scenario, I want to classify traffic based on DSCP when possible, and will use COS when I must.
I will be using the following QoS Classifications for Medical Grade Networks:
The example environment includes Nexus 5548s as well as Nexus 2232s. The Nexus 5000/2000 switches will be the trust boundary for edge devices and will follow the same ACL configuration guidelines defined elsewhere in the enterprise.
Configuring Global Traffic Classification Based on COS
At this time the Nexus 2148, Nexus 2232, and Nexus 2248 modules can only support CoS based traffic classification. Therefore, I use a global classification policy based on COS that is applied globally all N5K/N2K ports using a type qos policy and type QOS class-maps as follows:
! To support Nexus 2000s, classify traffic by COS. ! This classification will be for global COS to qos-group mapping. class-map type qos match-all COS1-GLOBAL match cos 1 class-map type qos match-all COS2-GLOBAL match cos 2 class-map type qos match-all COS3-GLOBAL match cos 3 class-map type qos match-all COS4-GLOBAL match cos 4 class-map type qos match-all COS5-GLOBAL match cos 5 class-map type qos match-all COS6-GLOBAL match cos 6 ! The default class already matches COS 0. ! ! Assign each global COS class to a qos-group in a qos policy-map policy-map type qos CLASSIFY-GLOBAL class COS6-GLOBAL set qos-group 3 class COS5-GLOBAL set qos-group 5 class COS4-GLOBAL set qos-group 4 class COS3-GLOBAL set qos-group 3 class COS2-GLOBAL set qos-group 3 class COS1-GLOBAL set qos-group 2 ! The default class is assigned to qos-group 0 by default.
Note: Although the N5548 supports user configured qos-group 1, I deliberately do not use qos-group 1 because the Nexus 5010 and 5020 switches hard code qos-group 1 for FCoE traffic.
As needed, I also hard code COS values for traffic received from the host ports on the N2K:
! interface “interface n2k/mod/port[-port] untagged cos 0 ! set the CoS value to 0 for traffic received on the host interfaces !interface “interface n2k/mod/port,n2k/mod/port ! untagged cos 5 ! selectively apply a CoS value for voice/UC servers as needed end
ip access-list QOS-NETWORK-CONTROL remark Network protocol traffic permit pim any any permit tcp any any eq bgp permit tcp any eq bgp any ! ip access-list QOS-SIGNALING remark Deny voice, video, and audio streams expressly NOT related to remark QoS call signaling functions. remark SCCP and Secure SCP permit tcp any any eq 2000 permit tcp any any eq 2443 ... remark H323 Q931 and H225 Call Signaling permit udp any any eq 1719 permit tcp any any eq 1720 ! ip access-list QOS-VOICE remark Phone media traffic between valid endpoints (IP phones, IPTgateways, remark & IPT servers) permit udp 10.16.0.0 0.15.255.255 range 16384 32768 10.16.0.0 0.15.255.255 range 16384 32768 ! ip access-list QOS-CLINICAL-LIFE-CRITICAL remark Clinical Life Critical traffic between associated clinical subnets. remark This ACL deliberately left blank - Reserved for Future Use ! ip access-list QOS-MULTIMEDIA-CONFERENCING remark Multimedia Conferencing traffic between sanctioned applications and remark associated TCP/UDP ports. ! remark Traversal Calls permit udp any gt 1023 10.96.0.0 0.15.255.63 gt 1023 permit udp 10.96.0.0 0.15.255.63 gt 1023 any gt 1023 ! ip access-list QOS-REAL-TIME-INTERACTIVE remark Real-Time Interactive traffic (i.e. Immersive TelePresence) permit udp 10.96.0.64 0.15.255.63 gt 1023 10.96.0.64 0.15.255.63 gt 1023 ! ip access-list QOS-MULTIMEDIA-STREAMING remark Multimedia Streaming traffic from sanctioned streaming audio/video endpoints remark at the access layer. permit udp 10.96.0.128 0.15.255.63 gt 1023 10.96.0.128 0.15.255.63 gt 1023 ! ip access-list QOS-LOW-LATENCY-DATA remark Latency sensitive Data application traffic remark This ACL deliberately left blank - Reserved for Future Use ! ip access-list QOS-NETWORK-MANAGEMENT ! Might want to list out net-management subnets ! if high-volume telnet application traffic is observed. remark Utilities for access to network devices. permit tcp any any eq telnet permit tcp any eq telnet any permit tcp any any eq 22 permit tcp any eq 22 any permit udp any any eq syslog permit udp any eq syslog any permit tcp any any eq tacacs permit tcp any eq tacacs any ... ! ip access-list QOS-HIGH-THROUGHPUT-DATA remark High-throughput data traffic (FTP, etc.) remark This ACL deliberately left blank - Reserved for Future Use ! ip access-list QOS-LOW-PRIORITY-DATA remark Low priority (i.e. Scavenger) traffic from non-business oriented applications ! Add additional internal networks as necessary remark --- External Networks --- ! Add external networks as necessary remark ------ permit ip any any !
Class-maps will use these ACLs:
! class-map type qos match-any IN-NETWORK-CONTROL description Network Control match access-group name QOS-NETWORK-CONTROL ! class-map type qos match-any IN-VOICE description Voice/VoIP/IPT match access-group name QOS-VOICE ! class-map type qos match-any IN-CLINICAL-LIFE-CRITICAL description Clinical Life Critical match access-group name QOS-CLINICAL-LIFE-CRITICAL ! class-map type qos match-any IN-MULTIMEDIA-CONFERENCING description Multimedia Conferencing match access-group name QOS-MULTIMEDIA-CONFERENCING ! class-map type qos match-any IN-REAL-TIME-INTERACTIVE description Real-Time Interactive (i.e. Immersive TelePresence) match access-group name QOS-REAL-TIME-INTERACTIVE ! class-map type qos match-any IN-MULTIMEDIA-STREAMING description Multimedia Streaming match access-group name QOS-MULTIMEDIA-STREAMING ! class-map type qos match-any IN-SIGNALING description Call Signaling match access-group name QOS-SIGNALING ! class-map type qos match-any IN-LOW-LATENCY-DATA description Low-Latency Data (i.e. Scavenger Class) match access-group name QOS-LOW-LATENCY-DATA ! class-map type qos match-any IN-NETWORK-MANAGEMENT description Network Management match access-group name QOS-NETWORK-MANAGEMENT ! class-map type qos match-any IN-HIGH-THROUGHPUT-DATA description High-throughput data traffic (FTP, etc.) match access-group name QOS-HIGH-THROUGHPUT-DATA ! class-map type qos match-any IN-LOW-PRIORITY-DATA description Low priority data applications (i.e. non-business apps) match access-group name QOS-LOW-PRIORITY-DATA !
These classes are assigned to a qos-group and marked with a DSCP value using a policy-map. The policy-map will later be applied to the N5K interfaces. Note that multiple traffic classes are assigned to the same qos-group:
policy-map type qos IN-MARKING-5K description Inbound classification/marking policy for trust boundaries. class IN-VOICE set dscp ef set qos-group 5 class IN-CLINICAL-LIFE-CRITICAL set dscp cs5 set qos-group 5 ! class IN-MULTIMEDIA-CONFERENCING set dscp af41 set qos-group 4 class IN-REAL-TIME-INTERACTIVE set dscp cs4 set qos-group 4 ! class IN-NETWORK-CONTROL set dscp CS6 set qos-group 3 ! class IN-MULTIMEDIA-STREAMING set dscp af31 set qos-group 3 class IN-SIGNALING set dscp cs3 set qos-group 3 class IN-LOW-LATENCY-DATA set dscp af21 set qos-group 3 class IN-NETWORK-MANAGEMENT set dscp cs2 set qos-group 3 ! class IN-HIGH-THROUGHPUT-DATA set dscp af11 set qos-group 2 class IN-LOW-PRIORITY-DATA set dscp cs1 set qos-group 2 ! class CLASS-DEFAULT set dscp default end
This service-policy will be applied all the physical interfaces and port-channel interfaces to classify traffic based on DSCP except the FEX host interface range Ethernet 100/1/1- 199/x/x. The more specific IN-MARKING-5K interface policy will take precedence over the global CLASSIFY-GLOBAL policy.
Note: For a portchannel member, the service-policy needs to be configured under the interface port-channel, not to individual channel ports.
interface “ethernet mod/port[-port]” or “port-channel #” ! Use a range of interfaces if possible. ! Use a range of interfaces if possible. service-policy input IN-MARKING-5K ! end
Marking COS Based on QOS-Groups for Traffic Sent to N2K Ports
Since the N2K only supports COS based QOS, a policy is needed to establish the COS value of traffic from the N5K ports. This traffic has been marked with a DSCP value, and placed in a qos-group. The GLOBALNETWORKPOLICY is a global policy that marks COS values based on existing qos-groups, allowing traffic from the N5K to be queued and scheduled on the N2K.
The following type network-qos configuration commands support the global policy that marks COS values based on existing qos-groups.
! ! Set CoS based on qos-groups for traffic sent from the N5K to the N2K class-map type network-qos COS5 match qos-group 5 class-map type network-qos COS4 match qos-group 4 class-map type network-qos COS6-3-2 match qos-group 3 class-map type network-qos COS1 match qos-group 2 ! policy-map type network-qos GLOBALNETWORKPOLICY class type network-qos COS5 set cos 5 mtu 9216 class type network-qos COS4 set cos 4 mtu 9216 class type network-qos COS6-3-2 set cos 3 mtu 9216 class type network-qos COS1 set cos 1 mtu 9216 !
Note: the “mtu 9216” command is used to set the jumbo mtu size globally. To verify this setting on a N5K, you need to use the “show queuing interface ethX/Y” command because the standard “show interface ethX/Y” will fib and incorrectly show a MTU of 1500.
Configuring Egress Scheduling and Queuing Based on QOS-Groups
The egress queue structures for the Nexus 5000/Nexus 2232/Nexus 2248 consists of 6 hardware queues for data plane traffic. All traffic in one of the six qos-groups is sent to a queue. If you add a system class, a queue is assigned to the class. Bandwidth is not automatically dedicated to user-defined system classes so must be manually configured. For this environment, we will be using 5 of the hardware queues.
Note: There are only 2 hardware queues for data plane traffic on the Nexus 2148.
The following type queuing configuration commands support the transmit queue assignments and bandwidth allocations:
class-map type queuing QUEUING-COS5 match qos-group 5 class-map type queuing QUEUING-COS4 match qos-group 4 class-map type queuing QUEUING-COS6-3-2 match qos-group 3 class-map type queuing QUEUING-COS1 match qos-group 2 ! a class-default is predefined which matches qos-group 0 ! ! policy-map type queuing QUEUING-GLOBAL class type queuing class-default bandwidth percent 25 class type queuing class-fcoe bandwidth percent 0 ! ! Before you can successfully allocate bandwidth to a class you must ! first reduce the default bandwidth configuration on class-default ! and class-fcoe ! class type queuing QUEUING-COS5 priority bandwidth percent 20 ! one class in each policy map can have strict priority set on it class type queuing QUEUING-COS4 bandwidth percent 10 class type queuing QUEUING-COS6-3-2 bandwidth percent 40 class type queuing QUEUING-COS1 bandwidth percent 5 !
! system qos service-policy type qos input CLASSIFY-GLOBAL !
The input qos policy allows the CoS based classification to be applied globally to support the N2K ports.
Follow these guidelines when globally establishing the system qos configuration on a Nexus 5000 switch.
! system qos service-policy type queuing output QUEUING-GLOBAL ! The output queueing policy determines the bandwidth allocation for both ! N5k interfaces and N2K host interfaces. service-policy type queuing input QUEUING-GLOBAL ! The input queuing policy determines how bandwidth is shared for the N2K ! uplink in the direction from N2K to N5k. service-policy type network-qos GLOBALNETWORKPOLICY ! The network-qos policy allows queuing and scheduling of traffic ! from the N5K to the N2K. !
The global system qos configuration on a Nexus 5000/2000 switch is used unless service policies are configured at the interface level.
Note: Since they are more specific, an interface-level policy always takes precedence over system class configuration or the default configuration.
Applying a traffic classification policy based on DSCP to support the non- N2K ports
The last step is to apply the interface level QoS configuration based on DSCP.
! interface “ethernet mod/port[-port]” or “port-channel #” ! Use a range of interfaces if possible. service-policy input IN-MARKING-5K ! end
The more specific service-policy on the physical interface is used to classify traffic based on DSCP on each non-FEX interface.
Note: For a portchannel member, the service-policy needs to be configured under the interface port-channel, not to individual channel ports.
I will discuss a sample configuration for Trust Boundaries in my Trust Boundaries and QoS on the Nexus 5000/2000 – Part 3 article in this series.
More on Medical Grade Networks
For more information on medical grade networks, you may want to review the following articles:
- QoS on the Nexus 5000/2000 – Part 1
- Trust Boundaries and QoS on the Nexus 5000/2000 – Part 3
- Designing a Medical Grade Network
- Healthcare Network Pain: Causes and Treatments
- Medical Grade Network Design and Operation