Configuring Campus QoS

Home

Resources

Archived Articles

Sunday, 05 August 2001 21:00

QoS in the Campus

Introduction

Last month's article was about QoS in the Campus. It explained why QoS is necessary and appropriate in the campus. The article can be found online at QoS in the Campus.

In this month's article, we'll take a look at configuring QoS in the Catalyst 6500 switches, since that's where most of the switch QoS features are available.

Recall that in 802.1q frames can be trunked (where there a bytes with a VLAN id and CoS bits) or in the native VLAN with no 802.1q header. With Cisco ISL trunking, frames must be trunk encapsulated if the receiving port is set to trunking, and normally encapsulated (ARPA or 802.3 with 802.2 or SNAP) if the receiving port is not trunking. For a review of trunking, see Switching: Trunks and Dynamic Trunking Protocol (DTP).

Recall that the Layer 2 switching engine can only work with CoS values in the frame header. The three CoS bits can only be carried in trunk-encapsulated frames. And the CoS is set on the egress (exit) interface.

Also please recall that the Layer 3 switching engine can manipulate both the IP ToS field (IP Precedence or Diff Serve settings), and the Layer 2 CoS field. The packet has to be processed by the PFC (policy feature card) to alter the ToS field. The outbound interface has to be using trunking encapsulation (ISL or 802.1q) or the CoS cannot be written to it. The PFC2 does not mark IPX or MAC traffic, just IP traffic.

The configuration options seem to fall naturally into the following capabilities:

Layer 2: Configuring inbound port internal marking of CoS
Layer 2: Configuring inbound port scheduling and congestion avoidance (drop/WRED thresholds)
Layer 3: Configuring marking and policing using access lists (ACL's)
Layer 2: Configuring outbound port scheduling and congestion avoidance
Miscellany, Show Commands, RSVP, COPS, etc.

We start by enabling QoS in the switch:

set qos enable

Trust

We then specify our trust policy.

Trust refers to whether or not we believe CoS (Layer 2 trunking frame header bits) or IP Precedence or Diff Serve (DSCP) bits (Layer 3 IP packet ToS byte settings) that were set by the station attached to a port.

Do you trust your PC users and applications to do the right thing as far as CoS? Or do you expect some to try to optimize their performance at the expense of others' performance? I already have anecdotal evidence there are "greedy" applications out there, ones which use inappropriate IP Precedence settings so as to boost performance. Do you want your Web browser competing seriously with VoIP traffic?

The simplest form of trust is per-port, with a Layer 2 switching engine. Configure:

set port qos mod_num/port_num trust {untrusted | trust-cos | trust-ipprec | trust-dscp}

With a Layer 3 switching engine, trust can be per-VLAN rather than per-port. This is configured with:

set port qos mod_num/port_num {port-based | vlan-based}

To specify the port default CoS value (for untrusted or unmarked frames), configure:

set port qos mod_num/port_num cos cos-value

Some Details of Trust

The documentation has flow charts showing how trust configuration operates. See http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/sw_6_3/confg_gd/qos.htm#xtocid127713 .

My short text version of how the 6500 switches handle trust follows. (Hey, it beats tracing flow charts!)

Note that "apply the port CoS" means that an internal data field is set appropriately, to carry the CoS information. If the outbound port is a trunking port, the desired CoS will be written to the trunk header. Otherwise, the information is solely used within that one switch.

Layer 2, trust based on ingress port

If the port is untrusted, apply the default port CoS and switch the frame.
If the frame is not trunked, apply the port CoS but continue with the next step.
If the port is set to trust IP Precedence, use the IP Precedence (apply the Precedence as internal CoS label) and switch the frame.
If the port is set to trust DSCP, do similarly.
If the port is set to trust CoS, queue the frame, using receive queues and thresholds based on the hardware and configured drop thresholds (see the previous article).

Layer 3 trust decision

First determine what internal DSCP value to use:

If there is an ACL on the interface, use it. Otherwise use the default ACL (which matches all traffic).
If the packet matches a permit entry in the ACL, then process as follows based on the entry matched.

The ACL entry matched specifies a type of traffic, and what to trust about packets matching the entry.

If the packet is IP protocol, and if the ACL entry says to trust DSCP, then use the received DSCP value.
If the packet is IP, and if the entry says to trust IP Precedence, then use the IP Precedence value to set DSCP internally.
Otherwise, and for non-IP packets, if the entry says to trust received CoS or port CoS, then use that value for DSCP internally.
If the packet is untrusted, use the DSCP setting from the ACL entry.

Then check policing for conformance with specified bandwidth policy:

After this, if the ACL entry did not specify policing, derive CoS from DSCP via mapping table and label internally.
If the ACL entry did specify policing, but the traffic is within profile, do the same (derive CoS and label internally).
If the ACL entry said to police but the traffic is out of profile, either drop the packet or use the mark down DSCP value, whichever is configured in the policing rule.

QoS Trust Access Lists

A switch variant of the Cisco IOS named IP or IPX or MAC access-lists is used with the Layer 3 switching QoS. Switch ACL entries are retained in an edit buffer while you build the ACL. You can view the contents with a show command, and make insertions and changes using line numbers. You then commit the ACL, upon which it is applied to the running configuration. You can, however, go back later and edit it further. If you don't like the ACL you're building, you can discard the edit buffer contents with

rollback qos acl {acl_name | all}

Here is a sample of the syntax for an ACL entry:

set qos acl ip acl_name {dscp dscp | trust-cos | trust-ipprec | trust-dscp} [microflow microflow_name ] [aggregate aggregate_name] tcp src_ip_spec [operator port | range port port] dest_ip_spec [operator port | range port port][established] [precedence precedence | dscp-field dscp] [before editbuffer_index | modify editbuffer_index ]

The entry could also instead use udp, icmp, igmp, or other layer 4 protocols in place of the "tcp" (obviously, the options that apply vary with the choice here). The words microflow and aggregate apply to score-keeping for policing: what bucket of traffic this traffic counts toward. The words before and modify are for editing ACL's in the edit buffer (the index is the line number).

When you look at this rather long command, the thing to pick up is that you're going to build up a list of access control entries or ACE's. Each ACE specifies what to trust (trust-cos, trust-ipprec, trust-dscp, or don't trust and use the specified DSCP setting). From the keyword "tcp" onwards (up to "before"), the syntax is that of an IOS device extended IP ACL. You put in source and destination IP addresses. You can put in source and destination ports, either with an operator (eq, lt, gt, neq) or as a range. You can also match only established traffic (i.e. packets after the initial SYN packet) or packets containing certain IP Precedence or DSCP values.

The conclusion to reach is that you can be as precise as you wish about what traffic gets marked with what bits. Here's some examples of entries in a hypothetical ACL:

set qos acl ip IP_ACL_1 trust-ipprec microflow MFLOW aggregate AGGIE tcp any any
set qos acl ip IP_ACL_1 trust-ipprec microflow MFLOW aggregate AGGIE udp any any
set qos acl ip IP_ACL_1 trust-ipprec microflow MFLOW aggregate AGGIE icmp any any echo

I've put names in CAPITAL letters here to make them stand out. I personally prefer to always, always use lower case so I don't encounter case-sensitivity surprises.It's very easy to think you're classifying traffic, and due to a case typo, not be giving traffic the desired priority.

If you don't want all that complexity, you can also modify the default ACL (which matches all traffic), to specify what marking to use with all (other) traffic. Syntax:

set qos acl default-action ip {dscp dscp | trust-cos | trust-ipprec | trust-dscp}[microflow microflow_name ] [aggregate aggregate_name]

I won't give an example since this is basically the same as before, with name "default-action".

Did I mention that you can use MAC addresses for marking? Sample syntax:

On a PFC:
set qos acl mac acl_name {dscp dscp | trust-cos} [aggregate aggregate_name] src_mac_spec dest_mac_spec [ethertype] [before editbuffer_index | modify editbuffer_index ]
With PFC2:
set qos acl mac acl_name aggregate aggregate_namesrc_mac_spec dest_mac_spec [ ethertype] [before editbuffer_index | modify editbuffer_index ]

(Note what's different/missing in the second of these. Why is this?)

If you're going to use these ACL's, as usual you'll need to know how to apply them to interfaces, or remove them from interfaces. The commands:

set qos acl map acl_name { mod_num/port_num | vlan}
clear qos acl map acl_name {mod_num/port_num | vlan | all}

Here's what that ends up looking like in practice:

set qos acl map IP_ACL 2/1
set qos acl map IP_ACL 4

The first of these applies the map to port 2/1. The second applies it to vlan 4.

With Layer 2 switching, you can still do MAC-based CoS:

set qos mac-cos dest_MAC_addr VLAN cos_value

This tells the switch what COS value to use for all traffic to the specified MAC address in the specified VLAN.

Policing

Policing provides a token bucket scheme with timing value 0.25 milliseconds. The bandwidth rules for policing can be set up to apply to microflows or to aggregate flows. You can have up to 63 microflow rules and 1023 aggregate policing rules. The PFC supports policing with two rules for traffic in or exceeding specification. The PFC2 allows you to specify two rates (normal and excess), with three policies (traffic within normal spec, traffic over normal but under excess, and traffic over excess).

To specify the rules, we create COS ACL entries. The granularity of the bandwidth specified has certain fixed settings. See just below the start of the following link for a table of settings: http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/sw_6_3/confg_gd/qos.htm#xtocid1277143 .

Here's the syntax for the variants of the policer command:

set qos policer microflow microflow_name {rate rate} {burst burst} {drop | policed-dscp}
With PFC or PFC2:

set qos policer aggregate aggregate_name {rate rate} {burst burst} {drop | policed-dscp}
With PFC2:
set qos policer aggregate aggregate_name {rate rate} policed-dscp {erate erate} {drop | policed-dscp} burst burst

For example:

set qos policer microflow MICRO rate 1000 burst 10000

The burst setting should be at least the rate divided by 4000 or you won't be able to sustain the specified rate.

Microflow policing normally only applies at Layer 3. To also police bridged traffic, configure

set qos bridged-microflow-policing {enable | disable} vlan

Thresholds

You can map different COS values to different drop thresholds. The settings depend on port type. For details, see http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/sw_6_3/confg_gd/qos.htm#xtocid1277168 .

Sample syntax and example:

set qos map 2q2t tx q# thr# cos coslist
set qos map 2q2t tx 1 1 cos 0,1

Policy

The 6500 QoS code supports COPS (for bulk configuration of devices in a standard way). See http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/sw_6_3/confg_gd/qos.htm#xtocid1277182 . You either believe in COPS or you don't. Basically, you configure local versus server-driven policy, roles (capabilities) for various ports, and what the address of the COPS server (Policy Decision Point) is, domain name, and timers.

RSVP is also supported. RSVP for the campus includes a DSBM, the Designated Subnet Bandwidth Manager, if you choose to use this mechanism. The switch can be configured to participate in the DSBM election. COPS can be used to provide the RSVP policy. For details, see http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/sw_6_3/confg_gd/qos.htm#xtocid1277196 .

Statistics

If you configure it, QoS statistics can be exported from the switch. This is separate from TopN and Netflow data export and has no interaction with these other features. See http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/sw_6_3/confg_gd/qos.htm#xtocid12771105 .

With QoS statistics export, you can configure a periodic interval for data export. Data collection however occurs periodically at a fixed interval. The data collected is per-port and per-aggregate policer utilization data. The data is sent as UDP packets for external post-processing. The data consists of input and output packet and bytes counts. Aggregate statistics show the allowed and excess packet counts.

Miscellany

You can also map received COS values to specified internal DSCP values. See http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/sw_6_3/confg_gd/qos.htm#xtocid1277173 . IP Precedence can be similarly mapped. Examples:

set qos cos-dscp-map 60 61 50 40 33 20 13 8
set qos ipprec-dscp-map 60 61 50 40 33 20 13 8

(This shows the DSCP values for each COS or IP Precedence value 0-7).

You can also map internal DSCP values to egress COS values. Example:

set qos dscp-cos-map 20-25:4 33-36:3

A similar map sets policed DSCP markdown values (i.e. what to internally mark excess frames with).

Show Commands

The following pretty much speak for themselves.

show port capabilities
show port qos mod/port
show port qos
show qos policer {config | runtime} {microflow | aggregate | all}
show qos acl info {acl_name | all} editbuffer [editbuffer_index]
show qos acl info {acl_name | all}
show qos acl info default-action {ip | ipx | mac | all}
show qos mac-cos {dest_mac [vlan] | all}
show qos bridged-packet-policing {config | runtime} vlan
show qos info config {1p1q4t rx | 1p1q0t rx | 1p2q2t tx | 2q2t tx | 1p3q1t tx}
show qos maps {config | runtime} [cos-dscp-map | ipprec-dscp-map | dscp-cos-map | policed-dscp-map]
show qos info [runtime | config]
show qos statistics { mod[/port] | l3stats | aggregate-policer [policer_name]}
show qos statistics export info

show mac [utilization] [mod[/port]

show qos policy-source
show port cops [mod [/port]]
show cops info

show qos rsvp info
show qos rsvp flow-info
show port rsvp [mod [/port]

In Conclusion

The Cisco program QPM might help you configure QoS. It now does support COPS. Third party tools generally support Cisco devices (although the question will always remain: how well? And with what delay on new features?)

Dr. Peter J. Welcher (CCIE #1773, CCSI #94014) is a Senior Consultant with Chesapeake NetCraftsmen. NetCraftsmen is a high-end consulting firm and Cisco Premier Partner dedicated to quality consulting and knowledge transfer. NetCraftsmen has eleven CCIE's (4 of whom are double-CCIE's, R&S and Security). NetCraftsmen has expertise including large network high-availability routing/switching and design, VoIP, QoS, MPLS, network management, security, IP multicast, and other areas. See http://www.netcraftsmen.net for more information about NetCraftsmen. Pete's links start at http://www.netcraftsmen.net/about-us/bios/staff-articles-and-blogs/pete-welcher.html . New articles will be posted under the Articles link. Questions, suggestions for articles, etc. can be sent to This e-mail address is being protected from spambots. You need JavaScript enabled to view it .

Chesapeake NetCraftsmen