Obsoleted by: 1819
Network Working Group CIP Working Group Request for Comments: 1190 C. Topolcic, Editor Obsoletes: IEN-119 October 1990 Experimental Internet Stream Protocol, Version 2 (ST-II) Status of this Memo This memo defines a revised version of the Internet Stream Protocol, originally defined in IEN-119 , based on results from experiments with the original version, and subsequent requests, discussion, and suggestions for improvements. This is a Limited-Use Experimental Protocol. Please refer to the current edition of the "IAB Official Protocol Standards" for the standardization state and status of this protocol. Distribution of this memo is unlimited. 1. Abstract This memo defines the Internet Stream Protocol, Version 2 (ST-II), an IP-layer protocol that provides end-to-end guaranteed service across an internet. This specification obsoletes IEN 119 "ST - A Proposed Internet Stream Protocol" written by Jim Forgie in 1979, the previous specification of ST. ST-II is not compatible with Version 1 of the protocol, but maintains much of the architecture and philosophy of that version. It is intended to fill in some of the areas left unaddressed, to make it easier to implement, and to support a wider range of applications.
1.1. Table of Contents Status of this Memo . . . . . . . . . . . . 1 1. Abstract . . . . . . . . . . . . . . . 1 1.1. Table of Contents . . . . . . . . . . . 2 1.2. List of Figures . . . . . . . . . . . . 4 2. Introduction . . . . . . . . . . . . . . 7 2.1. Major Differences Between ST and ST-II . . . . 8 2.2. Concepts and Terminology . . . . . . . . . 9 2.3. Relationship Between Applications and ST . . . . 11 2.4. ST Control Message Protocol . . . . . . . . 12 2.5. Flow Specifications . . . . . . . . . . . 14 3. ST Control Message Protocol Functional Description . 17 3.1. Stream Setup . . . . . . . . . . . . . 18 3.1.1. Initial Setup at the Origin . . . . . . . 18 3.1.2. Invoking the Routing Function . . . . . . 19 3.1.3. Reserving Resources . . . . . . . . . . 19 3.1.4. Sending CONNECT Messages . . . . . . . . 20 3.1.5. CONNECT Processing by an Intermediate Agent . . 22 3.1.6. Setup at the Targets . . . . . . . . . 23 3.1.7. ACCEPT Processing by an Intermediate Agent . . 24 3.1.8. ACCEPT Processing by the Origin . . . . . . 26 3.1.9. Processing a REFUSE Message . . . . . . . 27 3.2. Data Transfer . . . . . . . . . . . . . 30 3.3. Modifying an Existing Stream . . . . . . . . 31 3.3.1. Adding a Target . . . . . . . . . . . 31 3.3.2. The Origin Removing a Target . . . . . . . 33 3.3.3. A Target Deleting Itself . . . . . . . . 35 3.3.4. Changing the FlowSpec . . . . . . . . . 36 3.4. Stream Tear Down . . . . . . . . . . . . 36 3.5. Exceptional Cases . . . . . . . . . . . 37 3.5.1. Setup Failure due to CONNECT Timeout . . . . 37 3.5.2. Problems due to Routing Inconsistency . . . . 38 3.5.3. Setup Failure due to a Routing Failure . . . 39 3.5.4. Problems in Reserving Resources . . . . . . 41 3.5.5. Setup Failure due to ACCEPT Timeout . . . . 41 3.5.6. Problems Caused by CHANGE Messages . . . . . 42 3.5.7. Notification of Changes Forced by Failures . . 42 3.6. Options . . . . . . . . . . . . . . . 44 3.6.1. HID Field Option . . . . . . . . . . . 44 3.6.2. PTP Option . . . . . . . . . . . . . 44 3.6.3. FDx Option . . . . . . . . . . . . . 45 3.6.4. NoRecovery Option . . . . . . . . . . 46 3.6.5. RevChrg Option . . . . . . . . . . . 46 3.6.6. Source Route Option . . . . . . . . . . 46 3.7. Ancillary Functions . . . . . . . . . . . 48 3.7.1. Failure Detection . . . . . . . . . . 48 220.127.116.11. Network Failures . . . . . . . . . . 48 18.104.22.168. Detecting ST Stream Failures . . . . . . 49 22.214.171.124. Subset . . . . . . . . . . . . . 51
3.7.2. Failure Recovery . . . . . . . . . . . 51 126.96.36.199. Subset . . . . . . . . . . . . . 55 3.7.3. A Group of Streams . . . . . . . . . . 56 188.8.131.52. Group Name Generator . . . . . . . . 57 184.108.40.206. Subset . . . . . . . . . . . . . 57 3.7.4. HID Negotiation . . . . . . . . . . . 58 220.127.116.11. Subset . . . . . . . . . . . . . 64 3.7.5. IP Encapsulation of ST . . . . . . . . . 64 18.104.22.168. IP Multicasting . . . . . . . . . . 65 3.7.6. Retransmission . . . . . . . . . . . 66 3.7.7. Routing . . . . . . . . . . . . . . 67 3.7.8. Security . . . . . . . . . . . . . 67 3.8. ST Service Interfaces . . . . . . . . . . 68 3.8.1. Access to Routing Information . . . . . . 69 3.8.2. Access to Network Layer Resource Reservation . 70 3.8.3. Network Layer Services Utilized . . . . . . 71 3.8.4. IP Services Utilized . . . . . . . . . 71 3.8.5. ST Layer Services Provided . . . . . . . 72 4. ST Protocol Data Unit Descriptions . . . . . . . 75 4.1. Data Packets . . . . . . . . . . . . . 76 4.2. ST Control Message Protocol Descriptions . . . . 77 4.2.1. ST Control Messages . . . . . . . . . . 79 4.2.2. Common SCMP Elements . . . . . . . . . 80 22.214.171.124. DetectorIPAddress . . . . . . . . . 80 126.96.36.199. ErroredPDU . . . . . . . . . . . . 80 188.8.131.52. FlowSpec & RFlowSpec . . . . . . . . 81 184.108.40.206. FreeHIDs . . . . . . . . . . . . 84 220.127.116.11. Group & RGroup . . . . . . . . . . 85 18.104.22.168. HID & RHID . . . . . . . . . . . . 86 22.214.171.124. MulticastAddress . . . . . . . . . . 86 126.96.36.199. Name & RName . . . . . . . . . . . 87 188.8.131.52. NextHopIPAddress . . . . . . . . . . 88 184.108.40.206. Origin . . . . . . . . . . . . . 88 220.127.116.11. OriginTimestamp . . . . . . . . . . 89 18.104.22.168. ReasonCode . . . . . . . . . . . . 89 22.214.171.124. RecordRoute . . . . . . . . . . . 94 126.96.36.199. SrcRoute . . . . . . . . . . . . 95 188.8.131.52. Target and TargetList . . . . . . . . 96 184.108.40.206. UserData . . . . . . . . . . . . 98 4.2.3. ST Control Message PDUs . . . . . . . . 99 220.127.116.11. ACCEPT . . . . . . . . . . . . . 100 18.104.22.168. ACK . . . . . . . . . . . . . . 102 22.214.171.124. CHANGE-REQUEST . . . . . . . . . . 103 126.96.36.199. CHANGE . . . . . . . . . . . . . 104 188.8.131.52. CONNECT . . . . . . . . . . . . . 105 184.108.40.206. DISCONNECT . . . . . . . . . . . . 110 220.127.116.11. ERROR-IN-REQUEST . . . . . . . . . . 111 18.104.22.168. ERROR-IN-RESPONSE . . . . . . . . . 112 22.214.171.124. HELLO . . . . . . . . . . . . . 113 126.96.36.199. HID-APPROVE . . . . . . . . . . . 114 188.8.131.52. HID-CHANGE-REQUEST . . . . . . . . . 115
184.108.40.206. HID-CHANGE . . . . . . . . . . . . 116 220.127.116.11. HID-REJECT . . . . . . . . . . . . 118 18.104.22.168. NOTIFY . . . . . . . . . . . . . 120 22.214.171.124. REFUSE . . . . . . . . . . . . . 122 126.96.36.199. STATUS . . . . . . . . . . . . . 124 188.8.131.52. STATUS-RESPONSE . . . . . . . . . . 126 4.3. Suggested Protocol Constants . . . . . . . . 127 5. Areas Not Addressed . . . . . . . . . . . . 131 6. Glossary . . . . . . . . . . . . . . . 135 7. References . . . . . . . . . . . . . . . 143 8. Security Considerations. . . . . . . . . . . 144 9. Authors' Addresses . . . . . . . . . . . . 145 Appendix 1. Data Notations . . . . . . . . . . 147 1.2. List of Figures Figure 1. Protocol Relationships . . . . . . . . . 6 Figure 2. Topology Used in Protocol Exchange Diagrams . . 16 Figure 3. Virtual Link Identifiers for SCMP Messages . . 16 Figure 4. HIDs Assigned for ST User Packets . . . . . 18 Figure 5. Origin Sending CONNECT Message . . . . . . 21 Figure 6. CONNECT Processing by an Intermediate Agent . . 22 Figure 7. CONNECT Processing by the Target . . . . . . 24 Figure 8. ACCEPT Processing by an Intermediate Agent . . 25 Figure 9. ACCEPT Processing by the Origin . . . . . . 26 Figure 10. Sending REFUSE Message . . . . . . . . . 28 Figure 11. Routing Around a Failure . . . . . . . . 29 Figure 12. Addition of Another Target . . . . . . . . 32 Figure 13. Origin Removing a Target . . . . . . . . 34 Figure 14. Target Deleting Itself . . . . . . . . . 35 Figure 15. CONNECT Retransmission after a Timeout . . . . 38 Figure 16. Processing NOTIFY Messages . . . . . . . . 43 Figure 17. Source Routing Option . . . . . . . . . 47 Figure 18. Typical HID Negotiation (No Multicasting) . . . 60 Figure 19. Multicast HID Negotiation . . . . . . . . 61 Figure 20. Multicast HID Re-Negotiation . . . . 62 Figure 21. ST Header . . . . . . . . . . . . . 75 Figure 22. ST Control Message Format . . . . . . . . 77 Figure 23. ErroredPDU . . . . . . . . . . . . . 80 Figure 24. FlowSpec & RFlowSpec . . . . . . . . . . 81 Figure 25. FreeHIDs . . . . . . . . . . . . . . 85 Figure 26. Group & RGroup . . . . . . . . . . . . 85 Figure 27. HID & RHID . . . . . . . . . . . . . 86 Figure 28. MulticastAddress . . . . . . . . . . . 86 Figure 29. Name & RName . . . . . . . . . . . . 87 Figure 30. NextHopIPAddress . . . . . . . . . . . 88
Figure 31. Origin . . . . . . . . . . . . . . 88 Figure 32. OriginTimestamp . . . . . . . . . . . 89 Figure 33. ReasonCode . . . . . . . . . . . . . 89 Figure 34. RecordRoute . . . . . . . . . . . . . 94 Figure 35. SrcRoute . . . . . . . . . . . . . . 95 Figure 36. Target . . . . . . . . . . . . . . 97 Figure 37. TargetList . . . . . . . . . . . . . 97 Figure 38. UserData . . . . . . . . . . . . . . 98 Figure 39. ACCEPT Control Message . . . . . . . . . 101 Figure 40. ACK Control Message . . . . . . . . . . 102 Figure 41. CHANGE-REQUEST Control Message . . . . . . 103 Figure 42. CHANGE Control Message . . . . . . . . . 105 Figure 43. CONNECT Control Message . . . . . . . . . 109 Figure 44. DISCONNECT Control Message . . . . . . . . 110 Figure 45. ERROR-IN-REQUEST Control Message . . . . . . 111 Figure 46. ERROR-IN-RESPONSE Control Message . . . . . 112 Figure 47. HELLO Control Message . . . . . . . . . 113 Figure 48. HID-APPROVE Control Message . . . . . . . 114 Figure 49. HID-CHANGE-REQUEST Control Message . . . . . 115 Figure 50. HID-CHANGE Control Message . . . . . . . . 117 Figure 51. HID-REJECT Control Message . . . . . . . . 119 Figure 52. NOTIFY Control Message . . . . . . . . . 121 Figure 53. REFUSE Control Message . . . . . . . . . 123 Figure 54. STATUS Control Message . . . . . . . . . 125 Figure 55. STATUS-RESPONSE Control Message . . . . . . 126 Figure 56. Transmission Order of Bytes . . . . . . . 147 Figure 57. Significance of Bits . . . . . . . . . . 147
+--------------------+ | Conference Control | +--------------------+ | +-------+ +-------+ | | Video | | Voice | | +-----+ +------+ +-----+ +-----+ Application | Appl | | Appl | | | SNMP| |Telnet| | FTP | ... | | Layer +-------+ +-------+ | +-----+ +------+ +-----+ +-----+ | | | | | | | V V | | | | | ------------ +-----+ +-----+ | | | | | | PVP | | NVP | | | | | | +-----+ +-----+ + | | | | | \ | \ \ | | | | | +-----|--+-----+ | | | | | Appl.|control V V V V V | ST data | +-----+ +-------+ +-----+ | & control| | UDP | | TCP | ... | | Transport | | +-----+ +-------+ +-----+ Layer | /| / | \ / / | / /| |\ / | +------+--|--\-----+-/--|--- ... -+ / | | \ / | | | \ / | / | | \ / | | | \ +----|--- ... -+ | ----------- | \ / | | | \ / | | | V | | | V | | | +------+ | | | +------+ | +------+ | | | SCMP | | | | | ICMP | | | IGMP | | Internet | +------+ | | | +------+ | +------+ | Layer | | | | | | | | | V V V V V V V V V +-----------------+ +-----------------------------------+ | STream protocol |->| Internet Protocol | +-----------------+ +-----------------------------------+ | \ / | | \ / | | X | ------------ | / \ | | / \ | VV VV +----------------+ +----------------+ | (Sub-) Network |...| (Sub-) Network | (Sub-)Network | Protocol | | Protocol | Layer +----------------+ +----------------+ Figure 1. Protocol Relationships
2. Introduction ST has been developed to support efficient delivery of streams of packets to either single or multiple destinations in applications requiring guaranteed data rates and controlled delay characteristics. The motivation for the original protocol was that IP   did not provide the delay and data rate characteristics necessary to support voice applications. ST is an internet protocol at the same layer as IP, see Figure 1. ST differs from IP in that IP, as originally envisioned, did not require routers (or intermediate systems) to maintain state information describing the streams of packets flowing through them. ST incorporates the concept of streams across an internet. Every intervening ST entity maintains state information for each stream that passes through it. The stream state includes forwarding information, including multicast support for efficiency, and resource information, which allows network or link bandwidth and queues to be assigned to a specific stream. This pre-allocation of resources allows data packets to be forwarded with low delay, low overhead, and a low probability of loss due to congestion. The characteristics of a stream, such as the number and location of the endpoints, and the bandwidth required, may be modified during the lifetime of the stream. This allows ST to give a real time application the guaranteed and predictable communication characteristics it requires, and is a good vehicle to support an application whose communications requirements are relatively predictable. ST proved quite useful in several early experiments that involved voice conferences in the Internet. Since that time, ST has also been used to support point-to-point streams that include both video and voice. Recently, multimedia conferencing applications have been developed that need to exchange real-time voice, video, and pointer data in a multi-site conferencing environment. Multimedia conferencing across an internet is an application for which ST provides ideal support. Simulation and wargaming applications  also place similar requirements on the communication system. Other applications may include scientific visualization between a number of workstations and one or more remote supercomputers, and the collection and distribution of real-time sensor data from remote sensor platforms. ST may also be useful to support activities that are currently supported by IP, such as bulk file transfer using TCP. Transport protocols above ST include the Packet Video Protocol (PVP)  and the Network Voice Protocol (NVP) , which are end-to-end protocols used directly by applications. Other transport layer protocols that may be used over ST include TCP , VMTP , etc. They provide the user interface, flow control, and packet ordering. This specification does not describe these higher layer protocols.
2.1. Major Differences Between ST and ST-II ST-II supports a wider variety of applications than did the original ST. The differences between ST and ST-II are fairly straight forward yet provide great improvements. Four of the more notable differences are: 1 ST-II is decoupled from the Access Controller (AC). The AC, as well as providing a rudimentary access control function, also served as a centralized repository and distributor of the conference information. If an AC is necessary, it should be an entity in a higher layer protocol. A large variety of applications such as conferencing, distributed simulations, and wargaming can be run without an explicit AC. 2 The basic stream construct of ST-II is a directed tree carrying traffic away from a source to all the destinations, rather than the original ST's omniplex structure. For example, a conference is composed of a number of such trees, one for traffic from each participant. Although there are more (simplex) streams in ST-II, each is much simpler to manage, so the aggregate is much simpler. This change has a minimal impact on the application. 3 ST-II defines a number of the robustness and recovery mechanisms that were left undefined in the original ST specification. In case of a network or ST Agent failure, a stream may optionally be repaired automatically (i.e., without intervention from the user or the application) using a pruned depth first search starting at the ST Agent immediately preceding the failure. 4 ST-II does not make an inherent distinction between streams connecting only two communicants and streams among an arbitrary number of communicants. This memo is the specification for the ST-II Protocol. Since there should be no ambiguity between the original ST specification and the specification herein, the protocol is simply called ST hereafter. ST is the protocol used by ST entities to exchange information. The same protocol is used for communication among all ST entities, whether they communicate with a higher layer protocol or forward ST packets between attached networks. The remainder of this section gives a brief overview of the ST Protocol. Section 3 (page 17) provides a detailed description of the operations required by the protocol. Section 4 (page 75) provides descriptions of the ST Protocol Data Units exchanged
between ST entities. Issues that have not yet been fully addressed are presented in Section 5 (page 131). A glossary and list of references are in Sections 6 (page 135) and 7 (page 143), respectively. This memo also defines "subsets" of ST that can be implemented. A subsetted implementation does not have full ST functionality, but it can interoperate with other similarly subsetted implementations, or with a full implementation, in a predictable and consistent manner. This approach allows an implementation to be built and provide service with minimum effort, and gives it an immediate and well defined growth path. 2.2. Concepts and Terminology The ST packet header is not constrained to be compatible with the IP packet header, except for the IP Version Number (the first four bits) that is used to distinguish ST packets (IP Version 5) from IP packets (IP Version 4). The ST packets, or protocol data units (PDUs), can be encapsulated in IP either to provide connectivity (possibly with degraded service) across portions of an internet that do not provide support for ST, or to allow access to services such as security that are not provided directly by ST. An internet entity that implements the ST Protocol is called an "ST Agent". We refer to two kinds of ST agents: "host ST agents", also called "host agents" and "intermediate ST agents", also called "intermediate agents". The ST agents functioning as hosts are sourcing or sinking data to a higher layer protocol or application, while ST agents functioning as intermediate agents are forwarding data between directly attached networks. This distinction is not part of the protocol, but is used for conceptual purposes only. Indeed, a given ST agent may be simultaneously performing both host and intermediate roles. Every ST agent should be capable of delivering packets to a higher layer protocol. Every ST agent can replicate ST data packets as necessary for multi-destination delivery, and is able to send packets whether received from a network interface or a higher layer protocol. There are no other kinds of ST agents. ST provides applications with an end-to-end flow oriented service across an internet. This service is implemented using objects called "streams". ST data packets are not considered to be totally independent as are IP data packets. They are transmitted only as part of a point-to-point or point-to-multi- point stream. ST creates a stream during a setup phase before data is transmitted. During the setup phase, routes are selected and internetwork resources are reserved. Except for explicit changes to the stream, the routes remain in effect until the stream is explicitly torn down.
An ST stream is: o the set of paths that data generated by an application entity traverses on its way to its peer application entity(s) that receive it, o the resources allocated to support that transmission of data, and o the state information that is maintained describing that transmission of data. Each stream is identified by a globally unique "Name"; see Section 184.108.40.206 (page 87). The Name is specified in ST control operations, but is not used in ST data packets. A set of streams may be related as members of a larger aggregate called a "group". A group is identified by a "Group Name"; see Section 3.7.3 (page 56). The end-users of a stream are called the "participants" in the stream. Data travels in a single direction through any given stream. The host agent that transmits the data into the stream is called the "origin", and the host agents that receive the data are called the "targets". Thus, for any stream one participant is the origin and the others are the targets. A stream is "multi-destination simplex" since data travels across it in only one direction: from the origin to the targets. A stream can be viewed as a directed tree in which the origin is the root, all the branches are directed away from the root toward the targets, which are the leaves. A "hop" is an edge of that tree. The ST agent that is on the end of an edge in the direction toward the origin is called the "previous-hop ST agent", or the "previous-hop". The ST agents that are one hop away from a previous-hop ST agent in the direction toward the targets are called the "next-hop ST agents", or the "next-hops". It is possible that multiple edges between a previous-hop and several next-hops are actually implemented by a network level multicast group. Packets travel across a hop for one of two purposes: data or control. For ST data packet handling, hops are marked by "Hop IDentifiers" (HIDs) used for efficient forwarding instead of the stream's Name. A HID is negotiated among several agents so that data forwarding can be done efficiently on both a point-to-point and multicast basis. All control message exchange is done on a point-to-point basis between a pair of agents. For control message handling, Virtual Link Identifiers are used to quickly dispatch the control messages to the proper stream's state machine.
ST requires routing decisions to be made at several points in the stream setup and management process. ST assumes that an appropriate routing algorithm exists to which ST has access; see Section 3.8.1 (page 69). However, routing is considered to be a separate issue. Thus neither the routing algorithm nor its implementation is specified here. A routing algorithm may attempt to minimize the number of hops to the target(s), or it may be more intelligent and attempt to minimize the total internet resources consumed. ST operates equally well with any reasonable routing algorithm. The availability of a source routing option does not eliminate the need for an appropriate routing algorithm in ST agents. 2.3. Relationship Between Applications and ST It is the responsibility of an ST application entity to exchange information among its peers, usually via IP, as necessary to determine the structure of the communication before establishing the ST stream. This includes: o identifying the participants, o determining which are targets for which origins, o selecting the characteristics of the data flow between any origin and its target(s), o specifying the protocol that resides above ST, o identifying the Service Access Point (SAP), port, or socket relevant to that protocol at every participant, and o ensuring security, if necessary. The protocol layer above ST must pass such information down to the ST protocol layer when creating a stream. ST uses a flow specification, abbreviated herein as "FlowSpec", to describe the required characteristics of a stream. Included are bandwidth, delay, and reliability parameters. Additional parameters may be included in the future in an extensible manner. The FlowSpec describes both the desired values and their minimal allowable values. The ST agents thus have some freedom in allocating their resources. The ST agents accumulate information that describes the characteristics of the chosen path and pass that information to the origin and the targets of the stream. ST stream setup control messages carry some information that is not specifically relevant to ST, but is passed through the interface to the protocol that resides above ST. The "next
protocol identifier" ("NextPcol") allows ST to demultiplex streams to a number of possible higher layer protocols. The SAP associated with each participant allows the higher layer protocol to further demultiplex to a specific application entity. A UserData parameter is provided; see Section 220.127.116.11 (page 98). 2.4. ST Control Message Protocol ST agents create and manage a stream using the ST Control Message Protocol (SCMP). Conceptually, SCMP resides immediately above ST (as does ICMP above IP) but is an integral part of ST. Control messages are used to: o create streams, o refuse creation of a stream, o delete a stream in whole or in part, o negotiate or change a stream's parameters, o tear down parts of streams as a result of router or network failures, or transient routing inconsistencies, and o reroute around network or component failures. SCMP follows a request-response model. SCMP reliability is ensured through use of retransmission after timeout; see Section 3.7.6 (page 66). An ST application that will transmit data requests its local ST agent, the origin, to create a stream. While only the origin requests creation of a stream, all the ST agents from the origin to the targets participate in its creation and management. Since a stream is simplex, each participant that wishes to transmit data must request that a stream be created. An ST agent that receives an indication that a stream is being created must: 1 negotiate a HID with the previous-hop identifying the stream, 2 map the list of targets onto a set of next-hop ST agents through the routing function, 3 reserve the local and network resources required to support the stream,
4 update the FlowSpec, and 5 propagate the setup information and partitioned target list to the next-hop ST agents. When a target receives the setup message, it must inquire from the specified application process whether or not it is willing to accept the stream, and inform the origin accordingly. Once a stream is established, the origin can safely send data. ST and its implementations are optimized to allow fast and efficient forwarding of data packets by the ST agents using the HIDs, even at the cost of adding overhead to stream creation and management. Specifically, the forwarding decisions, that is, determining the set of next-hop ST agents to which a data packet belonging to a particular stream will be sent, are made during the stream setup phase. The shorthand HIDs are negotiated at that time, not only to reduce the data packet header size, but to access efficiently the stream's forwarding information. When possible, network-layer multicast is used to forward a data packet to multiple next-hop ST agents across a network. Note that when network-layer multicast is used, all members of the multicast group must participate in the negotiation of a common HID. An established stream can be modified by adding or deleting targets, or by changing the network resources allocated to it. A stream may be torn down by either the origin or the targets. A target can remove itself from a stream leaving the others unaffected. The origin can similarly remove any subset of the targets from its stream leaving the remainder unaffected. An origin can also remove all the targets from the stream and eliminate the stream in its entirety. A stream is monitored by the involved ST agents. If they detect a failure, they can attempt recovery. In general, this involves tearing down part of the stream and rebuilding it to bypass the failed component(s). The rebuilding always occurs from the origin side of the failure. The origin can optionally specify whether recovery is to be attempted automatically by intermediate ST agents or whether a failure should immediately be reported to the origin. If automatic recovery is selected but an intermediate agent determines it cannot effect the repair, it propagates the failure information backward until it reaches an agent that can effect repair. If the failure information propagates back to the origin, then the application can decide if it should abort or reattempt the recovery operation.
Although ST supports an arbitrary connection structure, we recognize that certain stream topologies will be common and justify special features, or options, which allow for optimized support. These include: o streams with only a single target (see Section 3.6.2 (page 44)), and o pairs of streams to support full duplex communication between two points (see Section 3.6.3 (page 45)). These features allow the most frequently occurring topologies to be supported with less setup delay, with fewer control messages, and with less overhead than the more general situations. 2.5. Flow Specifications Real time data, such as voice and video, have predictable characteristics and make specific demands of the networks that must transfer it. Specifically, the data may be transmitted in packets of a constant size that are produced at a constant rate. Alternatively, the bandwidth may vary, due either to variable packet size or rate, with a predefined maximum, and perhaps a non-zero minimum. The variation may also be predictable based on some model of how the data is generated. Depending on the equipment used to generate the data, the packet size and rate may be negotiable. Certain applications, such as voice, produce packets at the given rate only some of the time. The networks that support real time data must add minimal delay and delay variance, but it is expected that they will be non-zero. The FlowSpec is used for three purposes. First, it is used in the setup message to specify the desired and minimal packet size and rate required by the origin. This information is used by ST agents when they attempt to reserve the resources in the intervening networks. Second, when the setup message reaches the target, the FlowSpec contains the packet size and rate that was actually obtained along the path from the origin, and the accrued mean delay and delay variance expected for data packets along that path. This information is used by the target to determine if it wishes to accept the connection. The target may reduce reserved resources if it wishes to do so and if the possibility is still available. Third, if the target accepts the connection, it returns the updated FlowSpec to the origin, so that the origin can decide if it still wishes to participate in the stream with the characteristics that were actually obtained.
When the data transmitted by stream users is generated at varying rates, including bursts of varying rate and duration, there is an opportunity to provide service to more subscribers by providing guaranteed service for the average data rate of each stream, and reserving additional network capacity, shared among all streams, to service the bursts. This concept has been recognized by analog voice network providers leading to the principle of time assigned speech interpolation (TASI) in which only the talkspurts of a speech conversation are transmitted, and, during silence periods, the circuit can be used to send the talkspurts of other conversations. The FlowSpec is intended to assist algorithms that perform similar kinds of functions. We do not propose such algorithms here, but rather expect that this will be an area for experimentation. To allow for experiments, and a range of ways that application traffic might be characterized, a "DutyFactor" is included in the FlowSpec and we expect that a "burst descriptor" will also be needed. The FlowSpec will need to be revised as experience is gained with connections involving numerous participants using multiple media across heterogeneous internetworks. We feel a change of the FlowSpec does not necessarily require a new version of ST, it only requires the FlowSpec version number be updated and software to manage the new FlowSpec to be distributed. We further suggest that if the change to the FlowSpec involves additional information for improved operation, such as a burst descriptor, that it be added to the end of the FlowSpec and that the current parameters be maintained so that obsolete software can be used to process the current parameters with minimum modifications.
**** **** * * ST Agent 1 * * +---+ * *------- o ---------* *-------+ B | * * * * +---+ * * **** +---+ * * | | | * * | | A +---------* * o ST Agent 3 | | * * | +---+ * * | * * *** * * * * +---+ * * ST Agent 2 * *-------+ C | * *------- o --------* * +---+ * * * * **** * * * * +---+ * * +---+ | E +--------* *-------+ D | +---+ * * +---+ *** Figure 2. Topology Used in Protocol Exchange Diagrams **** ST Agent 1 **** * +--+---14--- o -----15--+----+--44---+---+ * | +-+--11--- -----16--+-+ * | B | * | | * * |+-+--45---+---+ * | | * *++* +---+ * | | * 34 ||32 | +----4----+--+ | * || | A +----6----+----+ * o ST Agent 3 | +----5----+---+ * | +---+ * | * | 33 * | * ST *+* * | * Agent * | * * | * 2 -----24-+--+ * +---+ * +--+--23--- o -----25-+-----+--54---+ C | * * -----26-+---+ * +---+ **** -----27-+-+ | * * | | * +---+ * | | * +---+ | E +---74---+-+ +-+--64---+ D | +---+ * * +---+ *** Figure 3. Virtual Link Identifiers for SCMP Messages
3. ST Control Message Protocol Functional Description This section contains a functional description of the ST Control Message Protocol (SCMP); Section 4 (page 75) specifies the formats of the control message PDUs. We begin with a description of stream setup. Mechanisms used to deal with the exceptional cases are then presented. Complications due to options that an application or a ST agent may select are then detailed. Once a stream has been established, the data transfer phase is entered; it is described. Once the data transfer phase has been completed, the stream must be torn down and resources released; the control messages used to perform this function are presented. The resources or participants of a stream may be changed during the lifetime of the stream; the procedures to make changes are described. Finally, the section concludes with a description of some ancillary functions, such as failure detection and recovery, HID negotiation, routing, security, etc. To help clarify the SCMP exchanges used to setup and maintain ST streams, we have included a series of figures in this section. The protocol interactions in the figures assume the topology shown in Figure 2. The figures, taken together, o Create a stream from an application at A to three peers at B, C and D, o Add a peer at E, o Disconnect peers B and C, and o D drops out of the stream. Other figures illustrate exchanges related to failure recovery. In order to make the dispatch function within SCMP more uniform and efficient, each end of a hop is assigned, by the agent at that end, a Virtual Link Identifier that uniquely (within that agent) identifies the hop and associates it with a particular stream's state machine(s). The identifier at the end of a link that is sending a message is called the Sender Virtual Link Identifier (SVLId); that at the receiving end is called the Receiver Virtual Link Identifier (RVLId). Whenever one agent sends a control message for the other to receive, the sender will place the receiver's identifier into the RVLId field of the message and its own identifier in the SVLId field. When a reply to the message is sent, the values in SVLId and RVLId fields will be reversed, reflecting the fact the sender and receiver roles are reversed. VLIds with values zero through three are received and should not be assigned in response to CONNECT messages. Figure 3 shows the hops that will be used in the examples and summarizes the VLIds that will be assigned to them.
Similarly, Figure 4 summarizes the HIDs that will eventually be negotiated as the stream is created. **** ST Agent 1 **** * +>+--1200-> o -------->+--->+-3600->+---+ * ^ * * * | B | * | * * +->+-6000->+---+ * | * *+** +---+ * | * ^ | +-------->+-->+ * | | A | * * o St Agent 3 | +-------->+-->+ * ^ +---+ * | * | 4801 * | * *+* * V * ST Agent 2 * ^ * +---+ * +>+--2400-> o ------->+->+->+-4800->+ C | **** * | * 4801 +---+ * | * +---+ * V * +---+ | E +<-4800--+<-+->+-4800->+ D | +---+ * * 4801 +---+ *** Figure 4. HIDs Assigned for ST User Packets Some of the diagrams that follow form a progression. For example, the steps required initially to establish a connection are spread across five figures. Within a progression, the actions on the first diagram are numbered 1.1, 1.2, etc.; within the second diagram they are numbered 2.1, 2.2, etc. Points where control leaves one diagram to enter another are identified with a continuation arrow "-->>", and are continued with "[a.b] >>-->" in the other diagram. The number in brackets shows the label where control left the earlier diagram. The reception of simple acknowledgments, e.g., ACKs, in one figure from another is omitted for clarity. 3.1. Stream Setup This section presents a description of stream setup assuming that everything succeeds -- HIDs are approved, any required resources are available, and the routing is correct. 3.1.1. Initial Setup at the Origin As described in Section 2.3 (page 11), the application has collected the information necessary to determine the
participants in the communication before passing it to the host ST agent at the origin. The host ST agent will take this information, allocate a Name for the stream (see Section 18.104.22.168 (page 87)), and create a stream. 3.1.2. Invoking the Routing Function An ST agent that is setting up a stream invokes a routing function to find a path to reach each of the targets specified in the TargetList. This is similar to the routing decision in IP. However, in this case the route is to a multitude of targets rather than to a single destination. The set of next-hops that an ST agent would select is not necessarily the same as the set of next hops that IP would select given a number of independent IP datagrams to the same destinations. The routing algorithm may attempt to optimize parameters other than the number of hops that the packets will take, such as delay, local network bandwidth consumption, or total internet bandwidth consumption. The result of the routing function is a set of next-hop ST agents and the parameters of the intervening network(s). The latter permit the ST agent to determine whether the selected network has the resources necessary to support the level of service requested in the FlowSpec. 3.1.3. Reserving Resources The intent of ST is to provide a guaranteed level of service by reserving internet resources for a stream during a setup phase rather than on a per packet basis. The relevant resources are not only the forwarding information maintained by the ST agents, but also packet switch processor bandwidth and buffer space, and network bandwidth and multicast group identifiers. Reservation of these resources can help to increase the reliability and decrease the delay and delay variance with which data packets are delivered. The FlowSpec contains all the information needed by the ST agent to allocate the necessary resources. When and how these resources are allocated depends on the details of the networks involved, and is not specified here. If an ST agent must send data across a network to a single next-hop ST agent, then only the point-to-point bandwidth needs to be reserved. If the agent must send data to multiple next- hop agents across one network and network layer multicasting is not available, then bandwidth must be reserved for all of them. This will allow the ST agent to
use replication to send a copy of the data packets to each next-hop agent. If multicast is supported, its use will decrease the effort that the ST agent must expend when forwarding packets and also reduces the bandwidth required since one copy can be received by all next-hop agents. However, the setup phase is more complicated. A network multicast address must be allocated that contains all those next-hop agents, the sender must have access to that address, the next-hop agents must be informed of the address so they can join the multicast group identified by it (see Section 22.214.171.124 (page 86)), and a common HID must be negotiated. The network should consider the bandwidth and multicast requirements to determine the amount of packet switch processing bandwidth and buffer space to reserve for the stream. In addition, the membership of a stream in a Group may affect the resources that have to be allocated; see Section 3.7.3 (page 56). Few networks in the Internet currently offer resource reservation, and none that we know of offer reservation of all the resources specified here. Only the Terrestrial Wideband Network (TWBNet)  and the Atlantic Satellite Network (SATNET)  offer(ed) bandwidth reservation. Multicasting is more widely supported. No network provides for the reservation of packet switch processing bandwidth or buffer space. We hope that future networks will be designed to better support protocols like ST. Effects similar to reservation of the necessary resources may be obtained even when the network cannot provide direct support for the reservation. Certainly if total reservations are a small fraction of the overall resources, such as packet switch processing bandwidth, buffer space, or network bandwidth, then the desired performance can be honored if the degree of confidence is consistent with the requirements as stated in the FlowSpec. Other solutions can be designed for specific networks. 3.1.4. Sending CONNECT Messages A VLId and a proposed HID must be selected for each next-hop agent. The control packets for the next-hop must carry the VLId in the SVLId field. The data packets transmitted in the stream to the next-hop must carry the HID in the ST Header. The ST agent sends a CONNECT message to each of the ST agents identified by the routing function. Each CONNECT message contains the VLId, the proposed HID (the HID Field option bit
must be set, see Section 3.6.1 (page 44)), an updated FlowSpec, and a TargetList. In general, the HID, FlowSpec, and TargetList will depend on both the next-hop and the intervening network. Each TargetList is a subset of the received (or original) TargetList, identifying the targets that are to be reached through the next-hop to which the CONNECT message is being sent. Note that a CONNECT message to a single next-hop might have to be fragmented into multiple CONNECTs if the single CONNECT is too large for the intervening network's MTU; fragmentation is performed by further dividing the TargetList. If multiple next-hops are to be reached through a network that supports network level multicast, a different CONNECT message must nevertheless be sent to each next-hop since each will have a different TargetList; see Section 126.96.36.199 (page 105). However, since an identical copy of each ensuing data packet will reach each member of the multicast group, all the CONNECT messages must propose the same HID. See Section 3.7.4 (page 58) for a detailed discussion on HID selection. In the example of Figure 2, the routing function might return that B is reachable via Agent 1 and C and D are reachable via Agent 2. Thus A would create two CONNECT messages, one each for Agents 1 and 2, as illustrated in Figure 5. Assuming that the proposed HIDs are available in the receiving agents, they would each send a responding HID-APPROVE back to Agent A. Application Agent A Agent 1 Agent 2 1.1. (open B,C,D) V 1.2. +-> (routing to B,C,D) V 1.3. +->(reserve resources from A to Agent 1) | V 1.4. | +-> CONNECT B --------->> | <RVLId=0><SVLId=4> | <Ref=10><HID=1200> V 1.5. +->(reserve resources from A to Agent 2) V 1.6. +-> CONNECT C,D ------------------>> <RVLId=0><SVLId=5> <Ref=15><HID=2400> Figure 5. Origin Sending CONNECT Message
3.1.5. CONNECT Processing by an Intermediate Agent An ST agent receiving a CONNECT message should, assuming no errors, quickly select a VLId and respond to the previous-hop with either an ACK, a HID-REJECT, or a HID-APPROVE message, as is appropriate. This message must identify the CONNECT to which it corresponds by including the CONNECT's Reference number in its Reference field. Note that the VLId that this agent selects is placed in the SVLId of the response, and the previous-hop's VLId (which is contained in the SVLId of the CONNECT) is copied into the RVLId of the response. If the agent is not a target, it must then invoke the routing function, reserve resources, and send a CONNECT message(s) to its next-hop(s), as described in Sections 3.1.2-4 (pages 19- 20). Agent A Agent 1 Agent B [1.4] >>-> CONNECT B -------->+--+ <RVLId=0><SVLId=4> | V 2.1. <Ref=10><HID=1200> | (routing to B) | V 2.2. V +->(reserve resources from 1 to B) 2.3. +<- HID-APPROVE <------+ V 2.4. <RVLId=4><SVLId=14> +-> CONNECT B ---------->> <Ref=10><HID=1200> <RVLId=0><SVLId=15> <Ref=110><HID=3600> Agent A Agent 2 Agent C [1.6] >>-> CONNECT C,D ------>+-+ <RVLId=0><SVLId=5> | V 2.5. <Ref=15><HID=2400> | (routing to C,D) | V 2.6. V +-->(reserve resources from 2 to C) 2.7. +<- HID-APPROVE <------+ | V 2.8. <RVLId=5><SVLId=23> | +-> CONNECT C ---------->> <Ref=15><HID=2400> | <RVLId=0><SVLId=25> | <Ref=210><HID=4800> | | Agent D V 2.9. +->(reserve resources from 2 to D) V 2.10. +-> CONNECT D ---------->> <RVLId=0><SVLId=26> <Ref=215><HID=4800> Figure 6. CONNECT Processing by an Intermediate Agent
The resources listed as Desired in a received FlowSpec may not correspond to those actually reserved in either the ST agent itself or in the network(s) used to reach the next-hop agent(s). As long as the reserved resources are sufficient to meet the specified Limits, the copy of the FlowSpec sent to a next-hop must have the Desired resources updated to reflect the resources that were actually obtained. For example, the Desired bandwidth might be reduced because the network to the next-hop could not provide all of the desired bandwidth. Also, the delay and delay variance are appropriately increased, and the link MTU may require that the DesPDUBytes field be reduced. (The minimum requirements that the origin had entered into the FlowSpec Limits fields cannot be altered by the intermediate or target agents.) 3.1.6. Setup at the Targets An ST agent that is the target of a CONNECT, whether from an intermediate ST agent, or directly from the origin host ST agent, must respond first (assuming no errors) with either a HID-REJECT or HID-APPROVE. After inquiring from the specified application process whether or not it is willing to accept the connection, the agent must also respond with either an ACCEPT or a REFUSE. In particular, the application must be presented with parameters from the CONNECT, such as the Name, FlowSpec, Options, and Group, to be used as a basis for its decision. The application is identified by a combination of the NextPcol field and the SAP field in the (usually) single remaining Target of the TargetList. The contents of the SAP field may specify the "port" or other local identifier for use by the protocol layer above the host ST layer. Subsequently received data packets will carry a short hand identifier (the HID) that can be mapped into this information and be used for their delivery. The responses to the CONNECT message are sent to the previous- hop from which the CONNECT was received. An ACCEPT contains the Name of the stream and the updated FlowSpec. Note that the application might have reduced the desired level of service in the received FlowSpec before accepting it. The target must not send the ACCEPT until HID negotiation has been successfully completed. Since the ACCEPT or REFUSE message must be acknowledged by the previous-hop, it is assigned a new Reference number that will be returned in the ACK. The CONNECT to which the ACCEPT or REFUSE is a reply is identified by placing the CONNECT's Reference number in the LnkReference field of the ACCEPT or REFUSE.
Agent 1 Agent B Application B 3.1. (proc B listening) [2.4] >>-> CONNECT B ---------->+------------------+ <RVLId=0><SVLId=15> | | 3.2. <Ref=110><HID=3600> V (proc B accepts) 3.3. +<- HID-APPROVE <--------+ | <RVLId=15><SVLId=44> | <Ref=110><HID=3600> V 3.4. (wait until HID negotiated) <---+ V 3.5. <<--+<- ACCEPT B <-----------+ <RVLId=15><SVLId=44> <Ref=410><LnkRef=110> Agent 2 Agent C Application C 3.6. (proc C listening) [2.8] >>-> CONNECT C ---------->+------------------+ <RVLId=0><SVLId=25> | | 3.7. <Ref=210><HID=4800> V (proc C accepts) 3.8. +<- HID-APPROVE <--------+ | <RVLId=25><SVLId=54> | <Ref=210><HID=4800> V 3.9. (wait until HID negotiated) <---+ V 3.10. <<--+<- ACCEPT C <-----------+ <RVLId=25><SVLId=54> <Ref=510><LnkRef=210> Agent 2 Agent D Application D 3.11. (proc D listening) [2.10] >>-> CONNECT D ---------->+------------------+ <RVLId=0><SVLId=26> | | 3.12. <Ref=215><HID=4800> V (proc D accepts) 3.13. +<- HID-APPROVE <--------+ | <RVLId=26><SVLId=64> | <Ref=215><HID=4800> V 3.14. (wait until HID negotiated) <---+ V 3.15. <<--+<- ACCEPT D <-----------+ <RVLId=26><SVLId=64> <Ref=610><LnkRef=215> Figure 7. CONNECT Processing by the Target 3.1.7. ACCEPT Processing by an Intermediate Agent When an intermediate ST agent receives an ACCEPT, it first verifies that the message is a response to an earlier CONNECT. If not, it responds to the next-hop ST agent with an ERROR-IN- REPLY (LnkRefUnknown) message. Otherwise, it responds to the next-hop ST agent with an ACK, and propagates