|QoS for the Cisco 6500 (Revisited)|
|Friday, 04 November 2005 14:55|
Interest in QoS has been picking up lately. This may reflect the increasing numbers of IP phone deployments. Also the fact that folks have gotten their networks upgraded or other projects completed, and are ready to attempt new things. In any case, it seems like it's time for another article on QoS!
Pete has written several prior articles and presentations about QoS. See his articles and blogs at at Pete's Articles and Blogs. Some previous QoS articles:
The first of these somewhat addresses the statement, "I have lots of bandwidth to spare in my campus, so I don't need QoS there." Our answer to that is "You might need it, and until you look for drops, you won't know. Or things might work fine until you have a busy network day, when your Voice over IP (VoIP) call quality degrades intermittently." In short, QoS in the campus is about insurance -- insuring you've done your best to avoid dropping VoIP UDP packets.
The second of these documents talks about what you can do in the 6500. Since then, things have changed. Among them, Pete's perspective on 6500 QoS has changed.
The last of the above articles covers the Modular QoS CLI (Command Line Interface), or MQC. That's a hierarchical acronym (one acronym nested within another)!
QoS for 6500 -- The Big PictureIf you're used to the Modular QoS CLI (MQC) and Class-Based Weighted Fair Queuing (CBWFQ) for IOS, you'll probably happily set out to configure your MSFC1 or MSFC2 or Sup720. You may or may not even end up thinking you've got QoS working. There's a gotcha here.
The 6500 does QoS in three places:
This trichotomy is more obvious when you're working with hybrid IOS (CatOS + IOS for MSFC). The CatOS-brain / IOS-brain duality is pretty obvious when you've got two sets of configuration commands! But when you configure QoS under native IOS, for example with the newer Sup32 or Sup720 engines, you're further from the hardware and so you may not realize what's going on. The line card part is less conspicuous, because it is invisible to the user.
Why do you need to know this? This is important because most of the traffic is multi-layer switched ("hardware switched"), and therefore handled by the PFC logic. The MSFC never even sees that traffic. So if you don't set up PFC-based QoS, you're missing most of the traffic that matters.
There's another factor to take into account. What you can configure for QoS depends on the type of interface you'll be applying the QoS policy to.
Quick QoS ReviewSee my Quality of Service (QoS) Seminar PDF for an overview with slides.
QoS is about getting various kinds of traffic to share a congested link while receiving the sort of treatment they require. QoS is partly about insuring that applications (including Voice over IP, VoIP) receive what they need, even if congestion is only occasional and short-lived.
QoS uses markings in L2 or L3 headers to specify what sort of traffic the packet is, and, indirectly, how to handle it. These marking are usually set at the edge or "Trust Boundary". The process of doing this is known as classification, and may involve processing of access lists or other rules specifying what traffic gets which marking. Downstream devices can then use the markings to determine appropriate handling of the frame or packet.
Markings include Class of Service (CoS, Cisco's term for 3 bits in the L2 frame header), and DSCP or Differentiated Services Code Point, which is the high-order 6 bits in the TOS field of the IP header. IP Precedence ("IPP") is an older scheme, basically the lead 3 bits of DSCP. See the above seminar for diagrams of this.
The modular QoS CLI interface is the way to configure QoS on most Cisco devices. Lately, we have found that we describe and think about QoS in terms of the MQC operations. Unfortunately, only some of this carries over to the 6500. (There are actually three QoS command sets for the 6500: MQC which supports most QoS function on Cisco devices; MLS QoS which supports 6500 hardware specific QoS features; and port based CLI for some specific LAN interface queueing functions.) Nonetheless, we'll use MQC as our common starting point for our discussion.
MQC involves three sets of commands, based around the class-map, policy-map, and the service-policy.
You use class-map commands to tell the router or switch how to recognize (or "classify") various kinds of traffic. The class-map assigns a name to the class of traffic, for use in further configuring that one router or switch. Once you've set up classes, you use some of them to build a policy with the policy-map command. The policy lists the relevant classes, and tells the router or switch what to do with each class of traffic. The service policy then tells the router or switch which policy to use on which interface, and whether it is used for inbound or outbound traffic.
The rest of QoS to some extent boils down to what you can do to the various classes of traffic. The choices are:
For examples of MQC, see the above seminar.
Weighted Random Early Detection (WRED) is a queue-management technique that attempts to automatically avoid congestion by selective random drops. It works well on higher speed links with many flows. It is arguably rather preferably to the other alternative, tail drops. It should not be used with UDP traffic (which doesn't slow down in response to a drop) or with priority traffic (where the point is trying to avoid drops).
There are also additional operations you might use on slow WAN interfaces: Link Fragmentation and Interleaving (LFI), and header compression. There's a lot more to QoS, for example Call Admission Control, but the above should be enough to get you started. See also Pete's previous articles and seminars for more info. Our focus here needs to stay on the 6500, which is quite a broad topic in itself.
If you're trying to understand PFC QoS for the 6500, it may be helpful to start by reading the documentation for CatOS, even if you do native IOS. The advantage to doing this is when you read the native IOS documentation on "PFC QOS", you'll be able to see how the IOS commands map to the underlying PFC capabilities that are more directly visible via the CatOS CLI.
Since there's so much there, we'll have to focus on essentials. The Cisco configuration guides lately provide a lot of good information, but sometimes fail to distinguish key commands from "knerd knobs". For the PFC QoS, the following are some of the key things to consider:
First, you have to enable QoS, for example with the "mls qos" configuration command on Native IOS. When you do so, egress traffic is mapped to queues based on an internal DSCP value and any changes you have made to the default DSCP to queue assignments. Just doing this gives preferential treatment to voice traffic marked with IP precedence 5 or DSCP value EF (Expedited Forwarding). How many queues there are, and what types they are (strict priority or not) depends on the line card.
The internal DSCP value is based on your inbound trust settings and the line card type. We generally trust received DSCP values, which is true for OSM WAN ports and FlexWAN ports. 6500 LAN ports default to untrusted CoS, and untrusted DSCP. If the switch is an access switch at the trust boundary, we use inbound QoS policies to classify and mark traffic, which specifies the internal DSCP value. If you trust inbound CoS bits, the CoS bits are copied or mapped to the internal DSCP field associated with the frame or packet. Upon output, the internal DSCP settings are copied to the IP header. If the outbound frame is being given a trunking header (ISL or 802.1Q) then the appropriate CoS bits are also set.
If you trust CoS (only), you can get the benefit of some inbound queues and WRED-like behavior, which may help if the inbound port is experiencing traffic bursts.
When you are doing PFC QoS on LAN ports and on the OSM WAN ports, your classification ability is somewhat limited compared to full Cisco IOS. Filtering or MQC match criteria can only be:
The outbound policy actions are also somewhat restricted. They depend on the interface or port. We cover this in the next section below. Other actions not familiar from MQC include trust. The inbound policy can specify whether CoS, IPP, or DSCP is trusted, or not. You can also specify inbound micro-policer (per-flow policing).
Inbound PFC QoS can be port-based or VLAN-based. If you mix the two, you'll be at risk of missing some inbound ports.
Mapping commands specify how to turn CoS or IPP into DSCP, or vice versa. The DSCP to CoS mapping is used, for example, to determine how to set outbound CoS bits based on the internal DSCP value.
QoS Feature Support TableThe following table represents summary information gleaned from documentation and extensive lab testing by Carole. She put a lot of effort into this, and certainly has my thanks for doing so!
Disclaimer: this table reflects our best effort to provide you with accurate information. Bear in mind that some of this does not seem to be clearly documented by Cisco and so may be subject to change.
The Flex WAN capabilities are basically full IOS MQC capabilities for a high-speed WAN interface. The new Shared Interface Process (SIP) line cards have similiar QoS capabilities as the FlexWAN, but we have not tested them.
Sample 6500 QoS ConfigurationHere's a sample configuration, straight from a test lab. This was from a 6500 running native IOS with a Sup720.
mls qos map cos-dscp 0 8 16 24 32 46 48 54
class-map match-all policeclass
match access-group name policetest
class-map match-all voice
match ip dscp ef
police 1000000 31250 31250 conform-action transmit exceed-action drop
set dscp ef
ip address 22.214.171.124 255.255.255.248
mls qos trust dscp
service-policy output qospolicy
ip access-list extended policetest
deny ip any any dscp ef log
permit ip any any
We provide show command samples in case you are not familiar with these commands. The main verification command remains "show policy interface".
6500#show mls qos
QoS is enabled globally
Microflow policing is enabled globally
QoS ip packet dscp rewrite enabled globally
Qos trust state is DSCP on the following interfaces:
Gi1/1 Gi1/2 Gi1/3 Gi1/4 Gi1/5 Gi1/6 Gi1/7 Gi1/8 Gi1/9 Gi1/10
Gi1/11 Gi1/12 Gi1/13 Gi1/14 Gi1/15 Gi1/16 Gi1/17 Gi1/18 Gi1/19 Gi1/20
Gi1/21 Gi1/22 Gi1/23 Gi1/24 Gi5/1 Gi5/2
Vlan or Portchannel(Multi-Earl) policies supported: Yes
Egress policies supported: Yes
----- Module  -----
QoS global counters:
Total packets: 1730211
IP shortcut packets: 0
Packets dropped by policing: 0
IP packets with TOS changed by policing: 0
IP packets with COS changed by policing: 1729894
Non-IP packets with COS changed by policing: 0
MPLS packets with EXP changed by policing: 0
More useful data comes from "show mls qos ip":6500#show mls qos ip
QoS Summary [IP]: (* - shared aggregates, Mod - switch module)
Int Mod Dir Class-map DSCP Agg Trust Fl AgForward-By AgPoliced-By
Gi1/9 5 Out voice 46 7 -- 0 1064027990 0
Gi5/2 5 Out voice 46 3 -- 0 0 0
All 5 - Default 0 0* No 0 313359032612 0
Note that the "show mls qos ..." commands only show information that applies to PFC QoS, and not to QoS services performed on line cards such as the FlexWAN.
Your comments, questions, and suggestions for future articles are of course welcome! See below to decipher Pete's email address.
Carole Warner Reece (CCIE #5168) is also a Senior Consultant with Chesapeake NetCraftsmen. She has worked with a wide variety of products and technologies in complex environments, and has broad experience in network design, consulting, and project implementation, as well as in developing network training materials. She can be reached at cwr <at> netcraftsmen <dot> net.