RFC 1190
Experimental Internet Stream Protocol: Version 2 (ST-II)
the ACCEPT message to the previous-hop along the same path
traced by the CONNECT but in the reverse direction toward the
origin. The ACCEPT should not be propagated until all HID
negotiations with the next-hop agent(s) have been successfully
completed.
The FlowSpec is included in the ACCEPT message so that the
origin and intermediate ST agents can gain access to the
information that was accumulated as the CONNECT traversed the
internet. Note that the resources, as specified in the
FlowSpec in the ACCEPT message, may differ from the resources
that were reserved by the agent when the CONNECT was
Agent A Agent 1 Agent B
+<-+<- ACCEPT B <-------<< [3.5]
V | <RVLId=15><SVLId=44>
4.1. (wait for ACCEPTS) V <Ref=410><LnkRef=110>
4.2. V +-> ACK --------------->+
4.3. (wait until HID negotiated)<-+ <RVLId=44><SVLId=15>
V <Ref=410>
4.4. <<--+<-- ACCEPT B <---------+
<RVLId=4><SVLId=14>
<Ref=115><LnkRef=10>
Agent A Agent 2 Agent C
+<-+<- ACCEPT C <------<< [3.10]
| | <RVLId=25><SVLId=54>
| V <Ref=510><LnkRef=210>
4.5. | +-> ACK --------------->+
| <Ref=510>
| <RVLId=54><SVLId=25>
|
| Agent D
V
+<-+<- ACCEPT D <------<< [3.15]
V | <RVLId=26><SVLId=64>
4.6. (wait for ACCEPTS) V <Ref=610><LnkRef=215>
4.7. V +-> ACK --------------->+
4.8. (wait until HID negotiated)<-+ <RVLId=64><SVLId=26>
V <Ref=610>
4.9. <<--+<- ACCEPT C <----------+
<RVLId=5><SVLId=23> |
<Ref=220><LnkRef=15>|
V
4.10. <<--+<- ACCEPT D <----------+
<RVLId=5><SVLId=23>
<Ref=225><LnkRef=15>
Figure 8. ACCEPT Processing by an Intermediate Agent
originally processed. However, the agent does not adjust the
reservation in response to the ACCEPT. It is expected that any
excess resource allocation will be released for use by other
stream or datagram traffic through an explicit CHANGE message
initiated by the application at the origin if it does not wish
to be charged for any excess resource allocations.
3.1.8. ACCEPT Processing by the Origin
The origin will eventually receive an ACCEPT (or REFUSE or
ERROR-IN-REQUEST) message from each of the targets. As each
ACCEPT is received, the application should be notified of the
target and the resources that were successfully allocated along
the path to it, as specified in the FlowSpec contained in the
ACCEPT message. The application may then use the information
to either adopt or terminate the portion of the stream to each
target. When ACCEPTs (or failures) from all targets have been
received at the origin, the application is notified that stream
setup is complete, and that data may be sent.
Application A Agent A Agent 1 Agent 2
+<-- ACCEPT B <--------<< [4.4]
| <RVLId=4><SVLId=14>
V <Ref=115><LnkRef=10>
5.1. +--> ACK ----------------->+
| <RVLId=14><SVLId=4>
V <Ref=115>
5.2. +<-- (inform A of B's FlowSpec)
| +<-- ACCEPT C <----------------<< [4.9]
| | <RVLId=5><SVLId=23>
| V <Ref=220><LnkRef=15>
5.3. | +--> ACK ------------------------->+
| | <RVLId=23><SVLId=5>
| V <Ref=220>
5.4. +<-- (inform A of C's FlowSpec)
| +<-- ACCEPT D <----------------<< [4.10]
| | <RVLId=5><SVLId=23>
| V <Ref=225><LnkRef=15>
5.5. | +--> ACK ------------------------->+
| | <RVLId=23><SVLId=5>
| V <Ref=225>
5.6. +<-- (inform A of D's FlowSpec)
V
5.7. (wait until HIDs negotiated)
V
5.8. (inform A open to B,C,D)
Figure 9. ACCEPT Processing by the Origin
There are several pieces of information contained in the
FlowSpec that the application must combine before sending data
through the stream. The PDU size should be computed from the
minimum value of the DesPDUBytes field from all ACCEPTs and the
protocol layers above ST should be informed of the limit. It
is expected that the next higher protocol layer above ST will
segment its PDUs accordingly. Note, however, that the MTU may
decrease over the life of the stream if new targets are
subsequently added. Whether the MTU should be increased as
targets are dropped from a stream is left for further study.
The available bandwidth and packet rate limits must also be
combined. In this case, however, it may not be possible to
select a pair of values that may be used for all paths, e.g.,
one path may have selected a low rate of large packets while
another selected a high rate of small packets. The application
may remedy the situation by either tearing down the stream,
dropping some participants, or creating a second stream.
After any differences have been resolved (or some targets have
been deleted by the application to permit resolution), the
application at the origin should send a CHANGE message to
release any excess resources along paths to those targets that
exceed the resolved parameters for the stream, thereby reducing
the costs that will be incurred by the stream.
3.1.9. Processing a REFUSE Message
REFUSE messages are used to indicate a failure to reach an
application at a target; they are propagated toward the origin
of a stream. They are used in three situations:
1 during stream setup or expansion to indicate that there
is no satisfactory path from an ST agent to a target,
2 when the application at the target either does not
exist does not wish to be a participant, or wants to
cease being a participant, and
3 when a failure has been detected and the agents are
trying to find a suitable path around the failure.
The cases are distinguished by the ReasonCode field and an
agent receiving a REFUSE message must examine that field in
order to determine the proper action to be taken. In
particular, if the ReasonCode indicates that the CONNECT
message reached the target then the REFUSE should be propagated
back to the origin, releasing resources as appropriate along
the way. If the ReasonCode indicates that
the CONNECT message did not reach the target then the
intermediate (origin) ST agent(s) should check for alternate
routes to the target before propagating the REFUSE back another
hop toward the origin. This implies that an agent must keep
track of the next-hops that it has tried, on a target by target
basis, in order not to get caught in a loop.
An ST agent that receives a REFUSE message must acknowledge it
by sending an ACK to the next-hop. The REFUSE must also be
propagated back to the previous-hop ST agent. Note that the ST
agent may not have any information about the target in
Appl. Agent A Agent 2 Agent E
(proc E NOT listening)
1. (add E)
2. +----->+-> CONNECT E ---------->+->+
<RVLId=23><SVLId=5> | |
<Ref=65> V |
3. +<-- ACK <---------------+ |
<RVLId=5><SVLId=23> V
4. <Ref=65> (routing to E)
V
5. (reserve resources 2 to E)
V
6. +--> CONNECT E --------->+
<RVLId=0><SVLId=27> |
<Ref=115><HID=4600> |
V
7. +<-+<- REFUSE B <-----------+
| | <RVLId=27><SVLId=74>
| | <Ref=705><LnkRef=115>
| V <RC=SAPUnknown>
8. | +-> ACK ---------------->+
| | <RVLId=74><SVLId=27> |
| V <Ref=705> |
9. | (free link 27) V
10. V (free link 74)
11. +<- REFUSE B <-----------+
| <RVLId=5><SVLId=23> |
| <Ref=550><LnkRef=65> V
12. | <RC=SAPUnknown> (free resources 2 to E)
V
13. +-> ACK --------------->+
| <RVLId=23><SVLId=5> |
| <Ref=550> V
14. V (keep link 23 for C,D)
15. (keep link 5 for C,D)
V
16. (inform application failed SAPUnknown)
Figure 10. Sending REFUSE Message
the TargetList. This may result from interacting DISCONNECT
and REFUSE messages and should be logged and silently ignored.
If, after deleting the specified target, the next-hop has no
remaining targets, then those resources associated with that
next-hop agent may be released. Note that network resources
may not actually be released if network multicasting is being
Appl. Agent A Agent 2 Agent 1 Agent 3 Agent B
1. (network from 1 to B fails)
2. (add B)
3. +-> CONNECT B ----------------->+
<RVLId=0><SVLId=6> |
<Ref=35><HID=100> |
3. +<- HID-APPROVE <---------------+
<RVLId=6><SVLId=11> |
<Ref=35><HID=100> V
4. (routing to B: no route)
V
5. +<-+-- REFUSE B ----------------+
| | <RVLId=6><SVLId=11>
| | <Ref=155><LnkRef=35>
| V <RC=NoRouteToDest>
6. | +-> ACK -------------------->+
| | <RVLId=11><SVLId=6> V
7. | V <Ref=155> (drop link 6)
8. V (drop link 11)
9. (find alternative route: via agent 2)
10. (resources from A to 2 already allocated:
V reuse control link & HID, no additional resources required)
11. +-> CONNECT B -------->+->+
<RVLId=23><SVLId=5>| |
<Ref=40> V |
12. +<- ACK <--------------+ |
<RVLId=5><SVLId=23> V
13. <Ref=40> (routing to B: via agent 3)
V
14. +-> CONNECT B -->+
15. <RVLId=0><SVLId=24> +-> CONNECT B --------->+
<Ref=245><HID=4801> V <RVLId=0><SVLId=32> |
16. +<- HID-APPROVE -+ <Ref=310><HID=6000> |
<RVLId=24><SVLId=33> |
<Ref=245><HID=4801> V
17. +<- HID-APPROVE --------+
<RVLId=32><SVLId=45>|
<Ref=310><HID=6000> V
18. (ACCEPT handling follows normally to complete stream setup)
Figure 11. Routing Around a Failure
used since they may still be required for traffic to other
next-hops in the multicast group.
When the REFUSE reaches a origin, the origin sends an ACK and
notifies the application via the next higher layer protocol
that the target listed in the TargetList is no longer part of
the stream and also if the stream has no remaining targets. If
there are no remaining targets, the application may wish to
terminate the stream.
Figure 10 illustrates the protocol exchanges for processing a
REFUSE generated at the target, either because the target
application is not running or that the target application
rejects membership in the stream. Figure 11 illustrates the
case of rerouting around a failure by an intermediate agent
that detects a failure or receives a refuse. The protocol
exchanges used by an application at the target to delete itself
from the stream is discussed in Section 3.3.3 (page 35).
3.2. Data Transfer
At the end of the connection setup phase, the origin, each target,
and each intermediate ST agent has a database entry that allows it
to forward the data packets from the origin to the targets and to
recover from failures of the intermediate agents or networks. The
database should be optimized to make the packet forwarding task
most efficient. The time critical operation is an intermediate
agent receiving a packet from the previous-hop agent and
forwarding it to the next-hop agent(s). The database entry must
also contain the FlowSpec, utilization information, the address of
the origin and previous-hop, and the addresses of the targets and
next-hops, so it can perform enforcement and recover from
failures.
An ST agent receives data packets encapsulated by an ST header. A
data packet received by an ST agent contains the non-zero HID
assigned to the stream for the branch from the previous-hop to
itself. This HID was selected so that it is unique at the
receiving ST agent and thus can be used, e.g., as an index into
the database, to obtain quickly the necessary replication and
forwarding information.
The forwarding information will be network and implementation
specific, but must identify the next-hop agent or agents and their
respective HIDs. It is suggested that the cached information for
a next-hop agent include the local network address of the next-
hop. If the data packet must be forwarded to multiple next-hops
across a single network that supports multicast, the database may
specify a single HID and may identify the next-hops by a (local
network) multicast address.
If the network does not support multicast, or the next-hops are on
different networks, then the database must indicate multiple
(next-hop, HID) tuples. When multiple copies of the data packet
must be sent, it may be necessary to invoke a packet replicator.
Data packets should not require fragmentation as the next higher
protocol layer at the origin was informed of the minimum MTU over
all paths in the stream and is expected to segment its PDUs
accordingly. However, it may be the case that a data packet that
is being rerouted around a failed network component may be too
large for the MTU of an intervening network. This should be a
transient condition that will be corrected as soon as the new
minimum MTU has been propagated back to the origin. Disposition
by a mechanism other than dropping of the too large PDUs is left
for further study.
3.3. Modifying an Existing Stream
Some applications may wish to change the parameters of a stream
after it has been created. Possible changes include adding or
deleting targets and changing the FlowSpec. These are described
below.
3.3.1. Adding a Target
It is possible for an application to add a new target to an
existing stream any time after ST has incorporated information
about the stream into its database. At a high level, the
application entities exchanges whatever information is
necessary. Although the mechanism or protocol used to
accomplish this is not specified here, it is necessary for the
higher layer protocol to inform the host ST agent at the origin
of this event. The host ST agent at the target must also be
informed unless this had previously been done. Generally, the
transfer of a target list from an ST agent to another, or from
a higher layer protocol to a host ST agent, will occur
atomically when the CONNECT is received. Any information
concerning a new target received after this point can be viewed
as a stream expansion by the receiving ST agent. However, it
may be possible that an ST agent can utilize such information
if it is received before it makes the relevant routing
decisions. These implementation details are not specified
here, but implementations must be prepared to receive CONNECT
messages that represent expansions of streams that are still in
the process of being setup.
To expand an existing stream, the origin issues one or more
CONNECT messages that contain the Name, the VLId, the FlowSpec,
and the TargetList specifying the new target or targets. The
origin issues multiple CONNECT messages if
either the targets are to be reached through different next-hop
agents, or a single CONNECT message is too large for the
network MTU. The HID Field option is not set since the HID has
already been (or is being) negotiated for the hop;
consequently, the CONNECT is acknowledged with an ACK instead
of a HID-REJECT or HID-APPROVE.
Application Agent A Agent 2 Agent E
1. (open E)
2. V (proc E listening)
3. +->(routing to E)
V
4. +-> (check resources from A to Agent 2: already allocated,
V reuse control link & HID, no additional resources needed)
5. +-> CONNECT E --------->+->+
<RVLId=23><SVLId=5> | V
6. <Ref=20> V (routing to E)
7. +<- ACK <---------------+ V
<RVLId=5><SVLId=23> +->(reserve resources 2 to E)
<Ref=20> V
8. +-> CONNECT E --------->+
<RVLId=0><SVLId=27> |
<Ref=230><HID=4800> |
9. +<- HID-APPROVE <-------+
<RVLId=27><SVLId=74>|
<Ref=230><HID=4800> V
10. (proc E accepts)
11. (wait until HID negotiated)
V
12. +<-+<- ACCEPT E <----------+
V | <RVLId=27><SVLId=74>
13. (wait for ACCEPTS) V <Ref=710><LnkRef=230>
14. V +-> ACK --------------->+
15. (wait until HID negotiated)<-+ <RVLId=74><SVLId=27>
V <Ref=710>
16. +<- ACCEPT E <-------+
| <RVLId=5><SVLId=23>
V <Ref=235><LnkRef=20>
17. +-> ACK ------------>+
| <RVLId=23><SVLId=5>
V <Ref=235>
18. +<-(inform A of E's FlowSpec)
V
19. +<-(wait for ACCEPTS)
V
20. +<-(wait until HID negotiated)
V
21. (inform A open to E)
Figure 12. Addition of Another Target
An ST agent that is already a node in the stream recognizes the
RVLId and verifies that the Name of the stream is the same. It
then checks if the intersection of the TargetList and the
targets of the established stream is empty. If this is not the
case, then the receiver responds with an ERROR-IN-REQUEST with
the appropriate reason code (RouteLoop) that contains a
TargetList of those targets that were duplicates; see Section
4.2.3.5 (page 106).
For each new target in the TargetList, processing is much the
same as for the original CONNECT; see Sections 3.1.2-4 (pages
19-20). The CONNECT must be acknowledged, propagated, and
network resources must be reserved. However, it may be
possible to route to the new targets using previously allocated
paths or an existing multicast group. In that case, additional
resources do not need to be reserved but more next-hop(s) might
have to be added to an existing multicast group.
Nevertheless, the origin, or any intermediate ST agent that
receives a CONNECT for an existing stream, can make a routing
decision that is independent of any it may have made
previously. Depending on the routing algorithm that is used,
the ST agent may decide to reach the new target by way of an
established branch, or it may decide to create a new branch.
The fact that a new target is being added to an existing stream
may result in a suboptimal overall routing for certain routing
algorithms. We take this problem to be unavoidable since it is
unlikely that the stream routing can be made optimal in
general, and the only way to avoid this loss of optimality is
to redefine the routing of potentially the entire stream, which
would be too expensive and time consuming.
3.3.2. The Origin Removing a Target
The application at the origin specifies a set of targets that
are to be removed from the stream and an appropriate reason
code (ApplDisconnect). The targets are partitioned into
multiple DISCONNECT messages based on the next-hop to the
individual targets. As with CONNECT messages, an ST agent that
is sending a DISCONNECT must make sure that the message fits
into the MTU for the intervening network. If the message is
too large, the TargetList must be further partitioned into
multiple DISCONNECT messages.
An ST agent that receives a DISCONNECT message must acknowledge
it by sending an ACK back to the previous-hop. The DISCONNECT
must also be propagated to the relevant next-hop ST agents.
Before propagating the message, however, the TargetList should
be partitioned based on next-hop ST
agent and MTU, as described above. Note that there may be
targets in the TargetList for which the ST agent has no
information. This may result from interacting DISCONNECT and
REFUSE messages and should be logged and silently ignored.
If, after deleting the specified targets, any next-hop has no
remaining targets, then those resources associated with that
next-hop agent may be released. Note that network resources
may not actually be released if network multicasting is being
used since they may still be required for traffic to other
next-hops in the multicast group.
Application Application
Agent A Agent 1 Agent 2 Agent B C
1. (close B,C ApplDisconnect)
V
2. +->+-+-> DISCONNECT B ----->+
3. | | <RVLId=14><SVLId=4>+-+-> DISCONNECT B ------>+
| | <Ref=25> | | <RVLId=44><SVLId=15>|
| V <RC=ApplDisconnect>| | <Ref=120> |
4. | (free A to 1 resrc.) | V <RC=ApplDisconnect> |
5. | V (free 1 to B resrc.) |
6. | +<- ACK <--------------+ V
7. | | <RVLId=4><SVLId=14>| +<- ACK <---------------+
| V <Ref=25> | | <RVLId=15><SVLId=44>|
8. | (free link 4) V | <Ref=120> |
9. | (free link 14) V |
10. | (free link 15) V
11. | (inform B that stream closed ApplDisconnect)
12. | (free link 44)
V
13. +<-+-+-> DISCONNECT C ---------->+
14. | | <RVLId=23><SVLId=5> +-+-> DISCONNECT C ------>+
| | <Ref=30> | | <RVLId=54><SVLId=25>|
| V <RC=ApplDisconnect> | | <Ref=240> |
15. | (keep A to 2 resrc for | V <RC=ApplDisconnect> |
16. | data going to D,E) | (free 2 to C resrc.) |
| V |
17. | +<- ACK <-------------------+ V
18. | | <RVLId=5><SVLId=23> | +<- ACK <---------------+
| V <Ref=30> | | <RVLId=25><SVLId=54>|
19. | (keep link 5 for D,E) V | <Ref=240> |
20. | (keep link 23 for D,E) V |
21. | (free link 25) V
22. | (inform C that stream closed ApplDisconnect>)
23. V (free link 54)
24. (inform A closed to B,C ApplDisconnect)
Figure 13. Origin Removing a Target
When the DISCONNECT reaches a target, the target sends an ACK
and notifies the application that it is no longer part of the
stream and the reason. The application should then inform ST
to terminate the stream, and ST should delete the stream from
its database after performing any necessary management and
accounting functions.
3.3.3. A Target Deleting Itself
The application at the target may inform ST that it wants to be
removed from the stream and the appropriate reason code
(ApplDisconnect). The agent then forms a REFUSE message with
itself as the only entry in the TargetList. The REFUSE is sent
back to the origin via the previous-hop. If a stream has
multiple targets and one target leaves the stream using this
REFUSE mechanism, the stream to the other targets is not
affected; the stream continues to exist.
An ST agent that receives such a REFUSE message must
acknowledge it by sending an ACK to the next-hop. The target
is deleted and, if the next-hop has no remaining targets, then
the those resources associated with that next-hop agent may be
released. Note that network resources may not actually be
released if network multicasting is being used since they may
still be required for traffic to other next-hops in the
multicast group. The REFUSE must also be propagated back to
the previous-hop ST agent.
Agent A Agent 2 Agent E
1. (close E ApplDisconnect)
V
2. +<- REFUSE E --+
| <RVLId=27><SVLId=74>
| <Ref=720>
V <RC=ApplDisconnect>
3. +<-+-> ACK ------>+
| | <RVLId=74><SVLId=27>
4. V V <Ref=720>
5. +<-+<- REFUSE E --+ (prune allocations)
| | <RVLId=5><SVLId=23>
| | <Ref=245>
| V <RC=ApplDisconnect>
6. | +-> ACK ------>+
| | <RVLId=23><SVLId=5>
| V <Ref=245>
7. V (prune allocations)
8. (inform application closed E ApplDisconnect)
Figure 14. Target Deleting Itself
When the REFUSE reaches the origin, the origin sends an ACK and
notifies the application that the target listed in the
TargetList is no longer part of the stream. If the stream has
no remaining targets, the application may choose to terminate
the stream.
3.3.4. Changing the FlowSpec
An application may wish to change the FlowSpec of an
established stream. To do so, it informs ST of the new
FlowSpec and the list of targets that are to be changed. The
origin ST agent then issues one or more CHANGE messages with
the new FlowSpec and sends them to the relevant next-hop
agents. CHANGE messages are structured and processed similarly
to CONNECT messages. A next-hop agent that is an intermediate
agent and receives a CHANGE message similarly determines if it
can implement the new FlowSpec along the hop to each of its
next-hop agents, and if so, it propagates the CHANGE messages
along the established paths. If this process succeeds, the
CHANGE messages will eventually reach the targets, which will
each respond with an ACCEPT message that is propagated back to
the origin.
Note that since a CHANGE may be sent containing a FlowSpec with
a range of permissible values for bandwidth, delay, and/or
error rate, and the actual values returned in the ACCEPTs may
differ, then another CHANGE may be required to release excess
resources along some of the paths.
3.4. Stream Tear Down
A stream is usually terminated by the origin when it has no
further data to send, but may also be partially torn down by the
individual targets. These cases will not be further discussed
since they have already been described in Sections 3.3.2-3 (pages
33-35).
A stream is also torn down if the application should terminate
abnormally. Processing in this case is identical to the previous
descriptions except that the appropriate reason code is different
(ApplAbort).
When all targets have left a stream, the origin notifies the
application of that fact, and the application then is responsible
for terminating the stream. Note, however, that the application
may decide to add a target(s) to the stream instead of terminating
it.
3.5. Exceptional Cases
The previous descriptions covered the simple cases where
everything worked. We now discuss what happens when things do not
succeed. Included are situations where messages are lost, the
requested resources are not available, the routing fails or is
inconsistent.
In order for the ST Control Message Protocol to be reliable over
an unreliable internetwork, the problems of corruption,
duplication, loss, and ordering must be addressed. Corruption is
handled through use of checksumming, as described in Section 4
(page 76). Duplication of control messages is detected by
assigning a transaction number (Reference) to each control
message; duplicates are discarded. Loss is detected using a
timeout at the sender; messages that are not acknowledged before
the timeout expires are retransmitted; see Section 3.7.6 (page
66). If a message is not acknowledged after a few retransmissions
a fault is reported. The protocol does not have significant
ordering constraints. However, minor sequencing of control
messages for a stream is facilitated by the requirement that the
Reference numbers be monotonically increasing; see Section 4.2
(page 78).
3.5.1. Setup Failure due to CONNECT Timeout
If a response (an ERROR-IN-REQUEST, an ACK, a HID-REJECT, or a
HID-APPROVE) has not been received within time ToConnect, the
ST agent should retransmit the CONNECT message. If no response
has been received within NConnect retransmissions, then a fault
occurs and a REFUSE message with the appropriate reason code
(RetransTimeout) is sent back in the direction of the origin,
and, in place of the CONNECT, a DISCONNECT is sent to the
next-hop (in case the response to the CONNECT is the message
that was lost). The agent will expect an ACK for both the
REFUSE and the DISCONNECT messages. If it does not receive an
ACK after retransmission time ToRefuse and ToDisconnect
respectively, it will resend the REFUSE/DISCONNECT message. If
it does not receive ACKs after sending NRefuse/ NDisconnect
consecutive REFUSE/DISCONNECT messages, then it simply gives up
trying.
Sending Agent Receiving Agent
1. ->+----> CONNECT X ------>//// (message lost or garbled)
| <RVLId=0><SVLId=99>
V <Ref=1278><HID=1234>
2. (timeout)
V
3. +----> CONNECT X ------------>+
4. | <RVLId=0><SVLId=99> +----> CONNECT X ----------->+
| <Ref=1278><HID=1234> V <RVLId=0><SVLId=1010> |
5. | //<- HID-APPROVE <----------+ <Ref=6666><HID=6666> V
6. | <RVLId=99><SVLId=88> +<- HID-APPROVE <---------+
V <Ref=1278><HID=1234> <RVLId=1010><SVLId=1111>
7. (timeout) <Ref=6666><HID=6666>
V
8. +----> CONNECT X ------------>+
<RVLId=0><SVLId=99> |
<Ref=1278><HID=1234> V
9. +<-+<- HID-APPROVE <----------+
| <RVLId=99><SVLId=88>
V <Ref=1278><HID=1234>
(cancel timer)
Figure 15. CONNECT Retransmission after a Timeout
3.5.2. Problems due to Routing Inconsistency
When an intermediate agent receives a CONNECT, it selects the
next-hop agents based on the TargetList and the networks to
which it is connected. If the resulting next-hop to any of the
targets is across the same network from which it received the
CONNECT (but not the previous-hop itself), there may be a
routing problem. However, the routing algorithm at the
previous-hop may be optimizing differently than the local
algorithm would in the same situation. Since the local ST
agent cannot distinguish the two cases, it should permit the
setup but send back to the previous-hop agent an informative
NOTIFY message with the appropriate reason code (RouteBack),
pertinent TargetList, and in the NextHopIPAddress element the
address of the next-hop ST agent returned by its routing
algorithm.
The agent that receives such a NOTIFY should ACK it. If the
agent is using an algorithm that would produce such behavior,
no further action is taken; if not, the agent should send a
DISCONNECT to the next-hop agent to correct the problem.
Alternatively, if the next-hop returned by the routing function
is in fact the previous-hop, a routing inconsistency has been
detected. In this case, a REFUSE is sent back to
the previous-hop agent containing an appropriate reason code
(RouteInconsist), pertinent TargetList, and in the
NextHopIPAddress element the address of the previous-hop. When
the previous-hop receives the REFUSE, it will recompute the
next-hop for the affected targets. If there is a difference in
the routing databases in the two agents, they may exchange
CONNECT and REFUSE messages again. Since such routing errors
in the internet are assumed to be temporary, the situation
should eventually stabilize.
3.5.3. Setup Failure due to a Routing Failure
It is possible for an agent to receive a CONNECT message that
contains a known Name, but from an agent other than the
previous-hop agent of the stream with that Name. This may be:
1 that two branches of the tree forming the stream have
joined back together,
2 a deliberate source routing loop,
3 the result of an attempted recovery of a partially
failed stream, or
4 an erroneous routing loop.
The TargetList is used to distinguish the cases 1 and 2 (see
also Section 4.2.3.5 (page 107)) by comparing each newly
received target with those of the previously existing stream:
o if the IP address of the targets differ, it is case 1;
o if the IP address of the targets match but the source
route(s) are different, it is case 2;
o if the target (including any source route) matches a
target (including any source route) in the existing
stream, it may be case 3 or 4.
It is expected that the joining of branches will become more
common as routing decisions are based on policy issues and not
just simple connectivity. Unfortunately, there is no good way
to merge the two parts of the stream back into a single stream.
They must be treated independently with respect to processing
in the agent. In particular, a separate state machine is
required, the Virtual Link Identifiers and HIDs from the
previous-hops and to the next-hops must be different, and
duplicate resources must be reserved in both the agent and in
any next-hop networks. Processing is the same for a deliberate
source routing loop.
The remaining cases requiring recovery, a partially failed
stream and an erroneous routing loop, are not easily
distinguishable. In attempting recovery of a failed stream, an
agent may issue new CONNECT messages to the affected targets;
for a full explanation see also Section 3.7.2 (page 51),
Failure Recovery. Such a CONNECT may reach an agent downstream
of the failure before that agent has received a DISCONNECT from
the neighborhood of the failure. Until that agent receives the
DISCONNECT, it cannot distinguish between a failure recovery
and an erroneous routing loop. That agent must therefore
respond to the CONNECT with a REFUSE message with the affected
targets specified in the TargetList and an appropriate reason
code (StreamExists).
The agent immediately preceding that point, i.e., the latest
agent to send the CONNECT message, will receive the REFUSE
message. It must release any resources reserved exclusively
for traffic to the listed targets. If this agent was not the
one attempting the stream recovery, then it cannot distinguish
between a failure recovery and an erroneous routing loop. It
should repeat the CONNECT after a ToConnect timeout. If after
NConnect retransmissions it continues to receive REFUSE
messages, it should propagate the REFUSE message toward the
origin, with the TargetList that specifies the affected
targets, but with a different error code (RouteLoop).
The REFUSE message with this error code (RouteLoop) is
propagated by each ST agent without retransmitting any CONNECT
messages. At each agent, it causes any resources reserved
exclusively for the listed targets to be released. The REFUSE
will be propagated to the origin in the case of an erroneous
routing loop. In the case of stream recovery, it will be
propagated to the ST agent that is attempting the recovery,
which may be an intermediate agent or the origin itself. In
the case of a stream recovery, the agent attempting the
recovery may issue new CONNECT messages to the same or to
different next-hops.
If an agent receives both a REFUSE message and a DISCONNECT
message with a target in common then it can release the
relevant resources and propagate neither the REFUSE nor the
DISCONNECT (however, we feel that it is unlikely that most
implementations will be able to detect this situation).
If the origin receives such a REFUSE message, it should attempt
to send a new CONNECT to all the affected targets. Since
routing errors in an internet are assumed to be temporary, the
new CONNECTs will eventually find acceptable routes to the
targets, if one exists. If no further routes exist after
NRetryRoute tries, the application should be
informed so that it may take whatever action it deems
necessary.
3.5.4. Problems in Reserving Resources
If the network or ST agent resources are not available, an ST
agent may preempt one or more streams that have lower
precedence than the one being created. When it breaks a lower
precedence stream, it must issue REFUSE and DISCONNECT messages
as described in Sections 4.2.3.15 (page 122) and 4.2.3.6 (page
110). If there are no streams of lower precedence, or if
preempting them would not provide sufficient resources, then
the stream cannot be accepted by the ST agent.
If an intermediate agent detects that it cannot allocate the
necessary resources, then it sends a REFUSE that contains an
appropriate reason code (CantGetResrc) and the pertinent
TargetList to the previous-hop ST agent. For further study are
issues of reporting what resources are available, whether the
resource shortage is permanent or transitory, and in the latter
case, an estimate of how long before the requested resources
might be available.
3.5.5. Setup Failure due to ACCEPT Timeout
An ST agent that propagates an ACCEPT message backward toward
the origin expects an ACK from the previous-hop. If it does
not receive an ACK within a timeout, called ToAccept, it will
retransmit the ACCEPT. If it does not receive an ACK after
sending a number, called NAccept, of ACCEPT messages, then it
will replace the ACCEPT with a REFUSE, and will send a
DISCONNECT in the direction toward the target. Both the REFUSE
and DISCONNECT will identify the affected target(s) and specify
an appropriate reason code (AcceptTimeout). Both are also
retransmitted until ACKed with timeout ToRefuse/ ToDisconnect
and retransmit count NRefuse/NDisconnect. If they are not
ACKed, the agent simply gives up, letting the failure detection
mechanism described in Section 3.7.1 (page 48) take care of any
cleanup.
3.5.6. Problems Caused by CHANGE Messages
An application must exercise care when changing a FlowSpec to
prevent a failure. A CHANGE might fail for two reasons. The
request may be for a larger amount of network resources when
those resources are not available; this failure may be
prevented by requiring that the current level of service be
contained within the ranges of the FlowSpec in the CHANGE.
Alternatively, the local network might require all the former
resources to be released before the new ones are requested and,
due to unlucky timing, an unrelated request for network
resources might be processed between the time the resources are
released and the time the new resources are requested, so that
the former resources are no longer available. There is not
much that an application or ST can do to prevent such failures.
If the attempt to change the FlowSpec fails then the ST agent
where the failure occurs must intentionally break the stream
and invoke the stream recovery mechanism using REFUSE and
DISCONNECT messages; see Section 3.7.2 (page 51). Note that
the reserved resources after the failure of a CHANGE may not be
the same as before, i.e., the CHANGE may have been partially
completed. The application is responsible for any cleanup
(another CHANGE).
3.5.7. Notification of Changes Forced by Failures
NOTIFY is issued by a an ST Agent to inform upsteam agents and
the origin that resource allocation changes have occurred after
a stream was established. These changes occur when network
components fail and when competing streams preempt resources
previously reserved by a lower precedence stream. We also
anticipate that NOTIFY can be used in the future when
additional resources become available, as is the case when
network components recover or when higher precedence streams
are deleted.
NOTIFY is also used to inform upstream agents that a routing
anomaly has occurred. Such an example was cited in Section
3.5.2 (page 38), where an agent notices that the next-hop agent
is on the same network as the previous-hop agent; the anomaly
is that the previous-hop should have connected directly to the
next-hop without using an intermediate agent. Delays in
propagating host status and routing information can cause such
anomalies to occur. NOTIFY allows ST to correct automatically
such mistakes.
NOTIFY reports a FlowSpec that reflects that revised guarantee
that can be promised to the stream. NOTIFY also
identifies those targets affected by the change. In this way,
NOTIFY is similar to ACCEPT. NOTIFY includes a ReasonCode to
identify the event that triggered the notification. It also
includes a TargetList, rather than a single Target, since a
single event can affect a branch leading to several targets.
NOTIFY is relayed by the ST agents back toward the origin,
along the path established by the CONNECT but in the reverse
direction. NOTIFY must be acknowledged with an ACK at each
hop. If intermediate agent corrects the situation without
causing any disruption to the data flow or guarantees, it can
choose to drop the notification message before it reaches the
origin. If the originating agent receives a NOTIFY, it is then
expected to adjust its own processing and data rates, and to
submit any required CHANGE requests. As with ACCEPT, the
FlowSpec is not modified on this trip from the target back to
the origin. It is up to the origin to decide whether a CHANGE
should be submitted. (However, even though the FlowSpec has
not been modified, the situation reported in the
Application Agent A Agent 1 Agent B
1. (high precedence request preempts 10K of
the stream's original 30Kb bandwidth
allocated to the hop from 1 to B)
|
V
2. +<------+-- NOTIFY -------------+
| | <RVLId=4><SVLId=14>
| | <Ref=150>
| V <FlowSpec=20Kb,...><TargList=B>
3. | +-> ACK --------------->+
| <RVLId=14><SVLId=4>
V <Ref=150>
4. (inform application)
....
5. change(FlowSpec=20Kb,...)
V
6. +---------> CHANGE B ---------->+
7. <RVLId=14><SVLId=4> +--> CHANGE B ------------>+->+
<Ref=60> | <RVLId=44><SVLId=15> | |
<FlowSpec=20Kb,...> V <Ref=160> | |
8. +<- ACK ----------------+ <FlowSpec=20Kb,...> | |
<RVLId=4><SVLId=14> V |
9. <Ref=60> +--- ACK ------------------+ |
<RVLId=15><SVLId=44> |
<Ref=160> V
... perform normal ACCEPT processing ... <-----+
Figure 16. Processing NOTIFY Messages
notify may have prevented the ST agents from meeting the
original guarantees.)
3.6. Options
Several options are defined in the CONNECT message. The special
processing required to support each will be described in the
following sections. The options are independent, i.e., can be set
to one (1, TRUE) or zero (0, FALSE) in any combination. However,
the effect and implementation of the options is NOT necessarily
independent, and not all combinations are supported.
3.6.1. HID Field Option
The sender of a CONNECT message may or not specify an HID in
the HID field. If the HID Field option of the CONNECT message
is not set (the H bit is 0), then the HID field does not
contain relevant information and should be ignored.
If this option is set (the H bit is 1), then the HID field
contains a relevant value. If this option is set and the HID
field of the CONNECT contains a non-zero value, that value
represents a proposed HID that initiates the HID negotiation.
If the HID Field option is set but the HID field of the CONNECT
message contains a zero, this means that the sender of that
CONNECT message has chosen to defer selection of the HID to the
next-hop agent (the receiver of a CONNECT message). This
choice can allow a more efficient mechanism for selecting HIDs
and possibly a more efficient mechanism for forwarding data
packets in the case when the previous-hop does not need to
select the HID; see also Section 4.2.3.5 (page 105).
Upon receipt of a CONNECT message with the HID Field option set
and the HID field set to zero, a next-hop agent selects the HID
for the hop, enters it into its appropriate data structure, and
returns it in the HID field of the HID-APPROVE message. The
previous-hop takes the HID from the HID-APPROVE message and
enters it into its appropriate data structure.
3.6.2. PTP Option
The PTP option (Point-to-Point) is used to indicate that the
stream will never have more than a single target. It
consequently implies that the stream will never need to support
any form of multicasting. Use of the PTP option may thus allow
efficiencies in the way the stream is built or is
managed. Specifically, the ST agents do not need to request
that the intervening networks allocate multicast groups to
support this stream.
The PTP option can only be set to one (1) by the origin, and
must be the same for the entire stream (i.e., propagated by ST
agents). The details of what this option does are
implementation specific, and do not affect the protocol very
much.
If the application attempts to add a new target to an existing
stream that was created with the PTP option set to one (1), the
application should be informed of the error with an ERROR-IN-
REQUEST message with the appropriate reason code. If a CONNECT
is received whose TargetList contains more than a single entry,
an ERROR-IN-REQUEST message with the appropriate reason code
(PTPError) should be returned to the previous-hop agent (note
that such a CONNECT should never be received if the origin both
implements the PTP option and is functioning properly).
As implied in the last paragraph, a subsetted implementation
might choose not to implement the PTP option.
3.6.3. FDx Option
The FDx option is used to indicate that a second stream in the
reverse direction, from the target to the origin, should
automatically be created. This option is most likely to be
used when the TargetList has only a single entry. If used when
the TargetList has multiple entries, the resulting streams
would allow bi-directional communication between the origin and
the various targets, but not among the targets. The FDx option
can only be invoked by the origin, and must be propagated by
intermediate agents.
This option is specified by inclusion of both an RFlowSpec and
an RHID parameter in the CONNECT message (possibly with an
optional RGroup parameter).
Any ST agent that receives a CONNECT message with both an
RFlowSpec and an RHID parameter will create database entries
for streams in both directions and will allocate resources in
both directions for them. By this we mean that an ST agent
will reserve resources to the next-hop agent for the normal
stream and resources back to the previous-hop agent for the
reverse stream. This is necessary since it is expected that
network reservation interfaces will require the destination
address(es) in order to make reservations, and because all ST
agents must use the same reservation model.
The target agent will select a Name for the reverse stream and
return it (in the RName parameter) and the resulting FlowSpec
(in the RFlowSpec parameter) of the ACCEPT message. Each agent
that processes the ACCEPT will update its partial stream
database entry for the reverse stream with the Name contained
in the RName parameter. We assume that the next higher
protocol layer will use the same SAP for both streams.
3.6.4. NoRecovery Option
The NoRecovery option is used to indicate that ST agents should
not attempt recovery in case of network or component failure.
If a failure occurs, the origin will be notified via a REFUSE
message and the target(s) via a DISCONNECT, with an appropriate
reason code of "failure" (i.e., one of DropFailAgt,
DropFailHst, DropFailIfc, DropFailNet, IntfcFailure,
NetworkFailure, STAgentFailure, FailureRecovery). They can
then decide whether to wait for the failed component to be
fixed, or drop the target via DISCONNECT/REFUSE messages. The
NoRecovery option can only be set to one (1) by the origin, and
must be the same for the entire stream.
3.6.5. RevChrg Option
The RevChrg option bit in the FlowSpec is set to one (1) by the
origin to request that the target(s) pay any charges associated
with the stream (to the target(s)); see Section 4.2.2.3 (page
83). If the target is not willing to accept charges, the bit
should be set to zero (0) by the target before returning the
FlowSpec to the origin in an ACCEPT message.
If the FDx option is also specified, the target pays charges
for both streams.
3.6.6. Source Route Option
The Source Route Option may be used both for diagnostic
purposes, and, in those hopefully infrequent cases where the
standard routing mechanisms do not produce paths that satisfy
some policy constraint, to allow the origin to prespecify the
ST agents along the path to the target(s). The idea is that
the origin can explicitly specify the path to a target, either
strictly hop-by-hop or more loosely by specification of one or
more agents through which the path must pass.
The option is specified by including source routing information
in the Target structure. A target may contain zero or more
SrcRoute options; when multiple options are present, they are
processed in the order in which they occur. The parameter code
indicates whether the portion of the path contained in the
parameter is of the strict or loose variety.
Since portions of a path may pass through portions of an
internet that does not support ST agents, there are also forms
of the SrcRoute option that are converted into the
Application Agent A Agent 2 Agent 3 Agent B
1. (open B<SR=2,3>)
2. V (proc B listening)
3. (source routed to 2)
V
4. (check resources from A to Agent 2: already allocated,
V reuse control link & HID, no additional resources needed)
5. +-> CONNECT B<SR=2,3>->-+-+
<RVLId=23><SVLId=5> | |
6. <Ref=50> V |
7. +<- ACK ----------------+ |
<RVLId=5><SVLId=23> |
<Ref=50> V
8. (source routed to 3)
V
9. (reserve resources 2 to 3)
V
10. +-> CONNECT B<SR=3> ---->+
<RVLId=0><SVLId=24> |
<Ref=280><HID=4801> V
11. +<- HID-APPROVE <--------+
<RVLId=24><SVLId=33> |
<Ref=280><HID=4801> |
V
(routing to B)
V
(reserve resources from 3 to B)
V
12. +-> CONNECT B ---------->+
<RVLId=0><SVLId=32> |
<Ref=330><HID=6000> V
13. +<- HID-APPROVE <--------+
<RVLId=32><SVLId=45> |
<Ref=330><HID=6000> V
14. (proc B accepts)
V
... perform normal ACCEPT processing ... <-----+
Figure 17. Source Routing Option
corresponding IP Source Routing options by the ST agent that
performs the encapsulation.
The SrcRoute option is usually selected by the origin, but may
be used by intermediate agents if specified as a result of the
routing function.
For example, in the topology of Figure 2, if A wants to add B
back into the stream, its routing function might decide that
the best path is via Agent 3. Since the data is already being
multicast across the network connected to C, D, and E, the
route via Agent 3 might cost less than having A replicate the
data packets and send them across A's network a second time.
3.7. Ancillary Functions
There are several functions and procedures that are required by
the ST Protocol. They are described in subsequent sections.
3.7.1. Failure Detection
The ST failure detection mechanism is based on two assumptions:
1 If a neighbor of an ST agent is up, and has been up
without a disruption, and has not notified the ST agent
of a problem with streams that pass through both, then
the ST agent can assume that there has not been any
problem with those streams.
2 A network through which an ST agent has routed a stream
will notify the ST agent if there is a problem that
affects the stream data packets but does not affect the
control packets.
The purpose of the robustness protocol defined here is for ST
agents to determine that the streams through a neighbor have
been broken by the failure of the neighbor or the intervening
network. This protocol should detect the overwhelming majority
of failures that can occur. Once a failure is detected,
recovery procedures are initiated.
3.7.1.1. Network Failures
In this memo, a network is defined to be the protocol
layer(s) below ST. This function can be implemented in a
hardware module separate from the ST agent, or as software
modules within the ST agent itself, or as a combination of
both. This specification and the robustness protocol do not
differentiate between these alternatives.
An ST agent can detect network failures by two mechanisms;
the network can report a failure, or the ST agent can
discover a failure by itself. They differ in the amount of
information that ST agent has available to it in order to
make a recovery decision. For example, a network may be
able to report that reserved bandwidth has been lost and the
reason for the loss and may also report that connectivity to
the neighboring ST agent remains intact. In this case, the
ST agent may request the network to allocate bandwidth anew.
On the other hand, an ST agent may discover that
communication with a neighboring ST agent has ceased because
it has not received any traffic from that neighbor in some
time period. If an ST agent detects a failure, it may not
be able to determine if the failure was in the network while
the neighbor remains available, or the neighbor has failed
while the network remains intact.
3.7.1.2. Detecting ST Stream Failures
Each ST agent periodically sends each neighbor with which it
shares a stream a HELLO message. A HELLO message is ACKed
if the Reference field is non-zero. This message exchange
is between ST agents, not entities representing streams or
applications (there is no Name field in a HELLO message).
That is, an ST agent need only send a single HELLO message
to a neighbor regardless of the number of streams that flow
between them. All ST agents (host as well as intermediate)
must participate in this exchange. However, only agents
that share active streams need to participate in this
exchange.
To facilitate processing of HELLO messages, an
implementation may either create a separate Virtual Link
Identifier for each neighbor having an active stream, or may
use the reserved identifier of one (1) for the SVLId field
in all its HELLO messages.
An implementation that wishes to send its HELLO messages via
a data path instead of the control path may setup a separate
stream to its neighbor agent for that purpose. The HELLO
message would contain a HID of zero, indicating a control
message, but would be identified to the next lower protocol
layer as being part of the separate stream.
As well as identifying the sender, the HELLO message has two
fields; a HelloTimer field that is in units of milliseconds
modulo the maximum for the field size, and a
(next page on part 3)
