Internet Engineering Task Force (IETF) B. Briscoe, Ed. Request for Comments: 6789 BT Category: Informational R. Woundy, Ed. ISSN: 2070-1721 Comcast A. Cooper, Ed. CDT December 2012 Congestion Exposure (ConEx) Concepts and Use Cases
AbstractThis document provides the entry point to the set of documentation about the Congestion Exposure (ConEx) protocol. It explains the motivation for including a ConEx marking at the IP layer: to expose information about congestion to network nodes. Although such information may have a number of uses, this document focuses on how the information communicated by the ConEx marking can serve as the basis for significantly more efficient and effective traffic management than what exists on the Internet today. Status of This Memo This document is not an Internet Standards Track specification; it is published for informational purposes. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are a candidate for any level of Internet Standard; see Section 2 of RFC 5741. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc6789.
Copyright Notice Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1. Congestion . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2. Congestion-Volume . . . . . . . . . . . . . . . . . . . . 5 2.3. Rest-of-Path Congestion . . . . . . . . . . . . . . . . . 6 2.4. Definitions . . . . . . . . . . . . . . . . . . . . . . . 6 3. Core Use Case: Informing Traffic Management . . . . . . . . . 7 3.1. Use Case Description . . . . . . . . . . . . . . . . . . . 7 3.2. Additional Benefits . . . . . . . . . . . . . . . . . . . 9 3.3. Comparison with Existing Approaches . . . . . . . . . . . 9 4. Other Use Cases . . . . . . . . . . . . . . . . . . . . . . . 11 5. Deployment Arrangements . . . . . . . . . . . . . . . . . . . 12 6. Experimental Considerations . . . . . . . . . . . . . . . . . 13 7. Security Considerations . . . . . . . . . . . . . . . . . . . 14 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 14 9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 15 10. Informative References . . . . . . . . . . . . . . . . . . . . 15 RFC3168]. As described in Figure 1, congestion control currently relies on the transport receiver detecting these 'Congestion Signals' and informing the transport sender in 'Congestion Feedback Signals'. The sender is then expected to reduce its rate in response.
This document provides the entry point to the set of documentation about the Congestion Exposure (ConEx) protocol. It focuses on the motivation for including a ConEx marking at the IP layer. (A companion document, [CONEX-ABS], focuses on the mechanics of the protocol.) Briefly, the idea is for the sender to continually signal expected congestion in the headers of any data it sends. To a first approximation, the sender does this by relaying the 'Congestion Feedback Signals' back into the IP layer. They then travel unchanged across the network to the receiver (shown as 'IP-Layer-ConEx-Signals' in Figure 1). This enables IP-layer devices on the path to see information about the whole-path congestion. ,---------. ,---------. |Transport| |Transport| | Sender | . |Receiver | | | /|___________________________________________| | | ,-<---------------Congestion-Feedback-Signals--<--------. | | | |/ | | | | | |\ Transport Layer Feedback Flow | | | | | | \ ___________________________________________| | | | | | \| | | | | | | ' ,-----------. . | | | | | |_____________| |_______________|\ | | | | | | IP Layer | | Data Flow \ | | | | | | |(Congested)| \ | | | | | | | Network |--Congestion-Signals--->-' | | | | | Device | \| | | | | | | /| | | `----------->--(new)-IP-Layer-ConEx-Signals-------->| | | | | | / | | | |_____________| |_______________ / | | | | | | |/ | | `---------' `-----------' ' `---------' Figure 1: The ConEx Protocol in the Internet Architecture One of the key benefits of exposing this congestion information at the IP layer is that it makes the information available to network operators for use as input into their traffic management procedures. A ConEx-enabled sender signals expected whole-path congestion, which is approximately the congestion at least a round-trip time earlier as reported by the receiver to the sender (Figure 1). The ConEx signal is a mark in the IP header that is easy for any IP device to read. Therefore, a node performing traffic management can count congestion as easily as it might count data volume today by simply counting the volume of packets with ConEx markings.
ConEx-based traffic management can make highly efficient use of capacity. In times of no congestion, all traffic management restraints can be removed, leaving the network's full capacity available to all its users. If some users on the network cause disproportionate congestion, the traffic management function can learn about this and directly limit those users' traffic in order to protect the service of other users sharing the same capacity. ConEx- based traffic management thus presents a step change in terms of the options available to network operators for managing traffic on their networks. The remainder of this document explains the concepts behind ConEx and how exposing congestion can significantly improve Internet traffic management, among other benefits. Section 2 introduces a number of concepts that are fundamental to understanding how ConEx-based traffic management works. Section 3 shows how ConEx can be used for traffic management, discusses additional benefits from such usage, and compares ConEx-based traffic management to existing traffic management approaches. Section 4 discusses other related use cases. Section 5 briefly discusses deployment arrangements. Section 6 suggests open issues that experiments in the use of ConEx could usefully be designed to answer. The final sections are standard RFC back matter. The remainder of the core ConEx document suite consists of: [CONEX-ABS], which provides an abstract encoding of ConEx signals, explains the ConEx audit and security mechanisms, and describes incremental deployment features; [CONEX-DESTOPT], which specifies the IPv6 destination option encoding for ConEx; [TCP-MOD], which specifies TCP-sender modifications for use of ConEx; and the following documents, which describe some feasible scenarios for deploying ConEx: [CONEX-DEPLOY], which describes a scenario around a fixed broadband access network; [CONEX-MOBILE], which describes a scenario around a mobile communications provider; [CONEX-DATA], which describes how ConEx could be used for performance isolation between tenants of a data centre.
Section 2.4. Bauer09]. The definition used for the purposes of ConEx is expressed as the probability of packet loss (or the probability of packet marking if ECN is in use). This definition focuses on how congestion is measured, rather than describing congestion as a condition or state.
Figure 1 indicate the congestion along a whole path from transport source to transport destination. Therefore, if a measurement point detects both of these signals, it can subtract the level of ECN (upstream congestion) from the level of ConEx (whole path) to derive a measure of the congestion that packets are likely to experience between the monitoring point and their destination (rest-of-path congestion). A measurement point can calculate this measurement in the aggregate, across all flows. A network monitor can usually accurately measure upstream congestion only if the traffic it observes is ECN-capable. [CONEX-ABS] further discusses the constraints around the network's ability to measure upstream and rest-of-path congestion in these circumstances. However, there are a number of initial deployment arrangements that benefit from ConEx but work without ECN (see Section 5). Section 3.1).
Rest-of-path congestion (or downstream congestion): The congestion a flow of traffic is expected to experience on the remainder of its path. In other words, at a measurement point in the network, the rest-of-path congestion is the congestion the traffic flow has yet to experience as it travels from that point to the receiver. Upstream congestion is usually expressed as a dimensionless percentage. Upstream congestion: The accumulated congestion experienced by a traffic flow thus far, relative to a point along its path. In other words, at a measurement point in the network, the upstream congestion is the accumulated congestion the traffic flow has experienced as it travels from the sender to that point. At the receiver, this is equivalent to the end-to-end congestion level that (usually) is reported back to the sender. This is usually expressed as a dimensionless percentage. Network operator (or provider): Operator of a residential, commercial, enterprise, campus, or other network. User: The contractual entity that represents an individual, household, business, or institution that uses the service of a network operator. There is no implication that the contract has to be commercial; for instance, the users of a university or enterprise network service could be students or employees who do not pay for access, but may be required to comply with some form of contract or acceptable use policy. There is also no implication that every user is an end user. Where two networks form a customer-provider relationship, the term "user" applies to the customer network. [CONEX-ABS] gives further definitions for aspects of ConEx related to protocol mechanisms.
operator seeks a way to manage traffic such that the traffic that contributes more to congestion bears more of the brunt of the management. Assuming ConEx signals are visible at the IP layer, the network operator can accomplish this by placing a congestion policer at an enforcement point within the network and configuring it with a traffic management policy that monitors each user's contribution to congestion. As described in [CONEX-ABS] and elaborated in [CongPol], one way to implement a congestion policer is in a similar way to a bit-rate policer, except that it monitors congestion-volume (based on IP-layer ConEx signals) rather than bit rate. When implemented as a token bucket, the tokens provide users with the right to cause bits of congestion-volume, rather than to send bits of data volume. The fill rate represents each user's congestion-volume quota. The congestion policer monitors the ConEx signals of the traffic entering the network. As long as the network remains uncongested and users stay within their quotas, no action is taken. When the network becomes congested and a user exhausts his quota, some action is taken against the traffic that breached the quota in accordance with the network operator's traffic management policy. For example, the traffic may be dropped, delayed, or marked with a lower QoS class. In this way, traffic is managed according to its contribution to congestion -- not some application- or flow-specific policy -- and is not managed at all during times of no congestion. As an example of how a network operator might employ a ConEx-based traffic management system, consider a typical DSL network architecture (as elaborated in [TR-059] and [TR-101]). Traffic is routed from regional and global IP networks to an operator-controlled IP node, the Broadband Remote Access Server (BRAS). From the BRAS, traffic is delivered to access nodes. The BRAS carries enhanced functionality, including IP QoS and traffic management capabilities. By deploying a congestion policer at the BRAS location, the network operator can measure the congestion-volume created by users within the access nodes and police misbehaving users before their traffic affects others on the access network. The policer would be provisioned with a traffic management policy, perhaps directing the BRAS to drop packets from users that exceed their congestion-volume quotas during times of congestion. Those users' apps would be likely to react in the typical way to drops, backing off (assuming at least some use TCP), and thereby lowering the users' congestion-volumes back within the quota limits. If none of a user's apps responds, the policer would continue to increase focused drops and effectively enforce its own congestion control.
LEDBAT], that provide ways for bulk-transfer applications to rapidly yield when interactive applications require capacity (thereby "scavenging" remaining bandwidth). With a congestion policer in place as described in Section 3.1, users of these protocols will be less likely to run afoul of the network operator's traffic management policy than those whose bulk-transfer applications generate the same volume of traffic without being sensitive to congestion. In short, two users who produce similar traffic volumes over the same time interval may produce different congestion-volumes if one of them is using a scavenger transport protocol and the other is not; in that situation, the scavenger user's traffic is less likely to be managed by the network operator. ConEx-based traffic management also makes it possible for a user to control the relative performance among its own traffic flows. If a user wants some flows to have more bandwidth than others, it can reduce the rate of some traffic so that it consumes less congestion- volume "budget", leaving more congestion-volume "budget" for the user to "spend" on making other traffic go faster. This approach is most relevant if congestion is signalled by ECN, because no impairment due to loss is involved and delay can remain low.
These approaches share bit rate instant by instant; however, heavy users are active at every instant, whereas light users only occupy their share of the link occasionally. Volume-based approaches measure the overall volume of traffic a user sends (and/or receives) over time. Users may be subject to an absolute volume cap (for example, 10GB per month) or the "heaviest" users may be sanctioned in some other manner. Many providers use monthly volume limits, and count volume regardless of whether the network is congested or not, creating the potential for over- or under-constraining problems, as with the original rate-based approaches. ConEx-based approaches, by comparison, only react during times of congestion and in proportion to each user's congestion contribution, making more efficient use of capacity and more proportionate management decisions. Unlike ConEx-based approaches, neither rate-based nor volume-based approaches provide incentives for applications to use scavenger transport protocols. They may even penalize users of applications that employ scavenger transports for the large amount of volume they send, rather than rewarding them for carefully avoiding congestion while sending it. While the volume-based approach described in "Comcast's Protocol-Agnostic Congestion Management System" [RFC6057] aims to overcome the over-/under-constraining problem by only measuring volume and triggering traffic management action during periods of high utilization, it still does not provide incentives to use scavenger transports, because congestion-causing volume cannot be distinguished from volume overall. ConEx provides this ability. Application-based approaches use deep packet inspection or other techniques to determine what application a given traffic flow is associated with. Network operators may then use this information to rate-limit or otherwise sanction certain applications, in some cases only during peak hours. These approaches suffer from being at odds with IPsec and some application-layer encryption, and they may raise additional policy concerns. In contrast, ConEx offers an application-agnostic metric to serve as the basis for traffic management decisions. The existing types of approaches share a further limitation that ConEx can help to overcome: performance uncertainty. Flat-rate pricing plans are popular because users appreciate the certainty of having their monthly bill amount remain the same for each billing period, allowing them to plan their costs accordingly. But while flat-rate pricing avoids billing uncertainty, it creates performance uncertainty: users cannot know whether the performance of their
connections is being altered or degraded based on how the network operator is attempting to manage congestion. By exposing congestion information at the IP layer, ConEx instead provides a metric that can serve as an open, transparent basis for traffic management policies that both providers and their customers can measure and verify. It can be used to reduce the performance uncertainty that some users currently experience, without having to sacrifice their billing certainty. Section 3.2 explains how operators can use ConEx-based traffic management to encourage use of scavenger transport protocols, which significantly improves the performance of interactive applications while still allowing heavy users to transfer high volumes. Here we explain how this can also benefit network operators. Today, when loss, delay, or average utilization exceeds a certain threshold, some operators just buy more capacity without attempting to manage the traffic. Other operators prefer to limit a minority of heavy users at peak times, but they still eventually buy more capacity when utilization rises. With ConEx-based traffic management, a network operator should be able to provision capacity more efficiently. An operator could benefit from this in a variety of ways. For example, the operator could add capacity as it would do without ConEx, but deliver better quality of service for its users. Or, the operator could delay adding capacity while delivering similar quality of service to what it currently provides.
Figure 1) how the sender piggy-backs ConEx signals on normal data packets to re-insert feedback about packet drops (and/or ECN) back into the IP layer. In the case of TCP, [TCP-MOD] proposes the required sender modifications. ConEx works with any TCP receiver as long as it uses SACK (selective acknowledgment), which most do. There is a receiver optimisation [TCPM-ECN] that improves ConEx precision when using ECN, but ConEx can still use ECN without it. Networks can make use of ConEx even if the implementations of some of the transport protocols on a host do not support ConEx (e.g., the implementation of DNS over UDP might not support ConEx, while perhaps RTP over UDP and TCP will). On the network side, the provider solely needs to place ConEx congestion policers at each ingress to its network in a similar arrangement to the edge-policed architecture of Diffserv [RFC2475]. A sender can choose whether to send packets that support ConEx or packets that don't. ConEx-enabled packets bring information to the policer about congestion expected on the rest of the path beyond the policer. Packets that do not support ConEx bring no such information. Therefore, the network will tend to conservatively rate-limit non-ConEx-enabled packets in order to manage the unknown risk of congestion. In contrast, a network doesn't normally need to rate-limit ConEx-enabled packets unless they reveal a persistently high contribution to congestion. This natural tendency for networks to favour senders that provide ConEx information reinforces ConEx deployment.
Feasible initial deployment scenarios exist for a broadband access network [CONEX-DEPLOY], a mobile communications network [CONEX-MOBILE], and a multi-tenant data centre [CONEX-DATA]. The first two of these scenarios are believed to work well without ECN support, while the data center scenario works best with ECN (and ECN can be deployed unilaterally). The above gives only the most salient aspects of ConEx deployment. For further detail, [CONEX-ABS] describes the incremental deployment features of the ConEx protocol and the components that need to be deployed for ConEx to work. CONEX-DESTOPT]) the right ones? o Is it possible to combine techniques for distinguishing self- congestion from shared congestion with ConEx-based traffic management such that users are not penalized for congestion that does not impact others on the network? Are other techniques needed? o In practice, how does traffic management using ConEx compare with traditional techniques (Section 3.3)? Does it give the benefits claimed in Sections 3.1 and 3.2? o Approaches are proposed for congestion policing of ConEx traffic alongside existing management (or lack thereof) of non-ConEx traffic, including UDP traffic [CONEX-ABS]. Are they strategy- proof against users selectively using both? Are there better transition strategies? o Audit devices have been designed and implemented to assure ConEx signal integrity [CONEX-ABS]. Do they achieve minimal false hits and false misses in a wide range of traffic scenarios? Are there new attacks? Are there better audit designs to defend against these?
o If ECN deployment remains patchy, are the proposed initial ConEx deployment scenarios (Section 5) still useful enough to kick-start deployment? Is auditing effective when based on loss at a primary bottleneck? Can rest-of-path congestion be approximated accurately enough without ECN? Are there other useful deployment scenarios? ConEx is intended to be a generative technology that might be used for unexpected purposes unforeseen by the designers. Therefore, this list of experimental considerations is not intended to be exhaustive. CONEX-ABS].
[Bauer09] Bauer, S., Clark, D., and W. Lehr, "The Evolution of Internet Congestion", 2009. [CONEX-ABS] Mathis, M. and B. Briscoe, "Congestion Exposure (ConEx) Concepts and Abstract Mechanism", Work in Progress, October 2012. [CONEX-DATA] Briscoe, B. and M. Sridharan, "Network Performance Isolation in Data Centres using Congestion Exposure (ConEx)", Work in Progress, July 2012. [CONEX-DEPLOY] Briscoe, B., "Initial Congestion Exposure (ConEx) Deployment Examples", Work in Progress, July 2012. [CONEX-DESTOPT] Krishnan, S., Kuehlewind, M., and C. Ucendo, "IPv6 Destination Option for ConEx", Work in Progress, September 2012. [CONEX-MOBILE] Kutscher, D., Mir, F., Winter, R., Krishnan, S., Zhang, Y., and C. Bernardos, "Mobile Communication Congestion Exposure Scenario", Work in Progress, July 2012.
[CongPol] Briscoe, B., Jacquet, A., and T. Moncaster, "Policing Freedom to Use the Internet Resource Pool", ReArch 2008 hosted at the 2008 CoNEXT conference , December 2008. [LEDBAT] Shalunov, S., Hazel, G., Iyengar, J., and M. Kuehlewind, "Low Extra Delay Background Transport (LEDBAT)", Work in Progress, September 2012. [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., and W. Weiss, "An Architecture for Differentiated Services", RFC 2475, December 1998. [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition of Explicit Congestion Notification (ECN) to IP", RFC 3168, September 2001. [RFC6057] Bastian, C., Klieber, T., Livingood, J., Mills, J., and R. Woundy, "Comcast's Protocol-Agnostic Congestion Management System", RFC 6057, December 2010. [TCP-MOD] Kuehlewind, M. and R. Scheffenegger, "TCP modifications for Congestion Exposure", Work in Progress, May 2012. [TCPM-ECN] Kuehlewind, M. and R. Scheffenegger, "More Accurate ECN Feedback in TCP", Work in Progress, July 2012. [TR-059] Anschutz, T., Ed., "DSL Forum Technical Report TR-059: Requirements for the Support of QoS-Enabled IP Services", September 2003. [TR-101] Cohen, A., Ed. and E. Schrum, Ed., "DSL Forum Technical Report TR-101: Migration to Ethernet-Based DSL Aggregation", April 2006.
http://bobbriscoe.net/ Richard Woundy (editor) Comcast 1701 John F Kennedy Boulevard Philadelphia, PA 19103 US EMail: email@example.com URI: http://www.comcast.com Alissa Cooper (editor) CDT 1634 Eye St. NW, Suite 1100 Washington, DC 20006 US EMail: firstname.lastname@example.org