This section outlines known issues and incompatibilities with RTP and RTCP extensions when multiple media types are used in a single RTP session. Future extensions to RTP and RTCP need to consider, and document, any potential incompatibilities.
The RTP retransmission payload format [RFC 4588
] can operate in either SSRC-multiplexed mode or session-multiplexed mode.
In SSRC-multiplexed mode, retransmitted RTP packets are sent in the same RTP session as the original packets but use a different SSRC with the same RTCP Source Description (SDES) CNAME. If each endpoint sends only a single original RTP stream and a single retransmission RTP stream in the session, this is sufficient. If an endpoint sends multiple original and retransmission RTP streams, as would occur when sending multiple media types in a single RTP session, then each original RTP stream and the retransmission RTP stream have to be associated using heuristics. By having retransmission requests outstanding for only one SSRC not yet mapped, a receiver can determine the binding between the original and retransmission RTP streams. Another alternative is the use of different RTP payload types, allowing the signalled "apt" (associated payload type) parameter [RFC 4588
] of the RTP retransmission payload format to be used to associate retransmitted and original packets.
Session-multiplexed mode sends the retransmission RTP stream in a separate RTP session to the original RTP stream, but using the same SSRC for each, with the association being done by matching SSRCs between the two sessions. This is unaffected by the use of multiple media types in a single RTP session, since each media type will be sent using a different SSRC in the original RTP session, and the same SSRCs can be used in the retransmission session, allowing the streams to be associated. This can be signalled using SDP with the BUNDLE grouping extension [RFC 8843
] and the Flow Identification (FID) grouping extension [RFC 5888
]. These SDP extensions require each "m=" line to only be included in a single FID group, but the RTP retransmission payload format uses FID groups to indicate the "m=" lines that form an original and retransmission pair. Accordingly, when using the BUNDLE extension to allow multiple media types to be sent in a single RTP session, each original media source ("m=" line) that is retransmitted needs a corresponding "m=" line in the retransmission RTP session. If there are multiple media lines for retransmission, these media lines will form an independent BUNDLE group from the BUNDLE group with the source streams.
An example SDP fragment showing the grouping structures is provided in Figure 1
. This example is not legal SDP, and only the most important attributes have been left in place. Note that this SDP is not an initial BUNDLE offer. As can be seen in this example, there are two bundle groups -- one for the source RTP session and one for the retransmissions. Then, each of the media sources is grouped with its retransmission flow using FID, resulting in three more groupings.
a=group:BUNDLE foo bar fiz
a=group:BUNDLE zoo kelp glo
a=group:FID foo zoo
a=group:FID bar kelp
a=group:FID fiz glo
m=audio 10000 RTP/AVP 0
m=video 10000 RTP/AVP 31
m=video 10000 RTP/AVP 31
m=audio 40000 RTP/AVPF 99
m=video 40000 RTP/AVPF 100
m=video 40000 RTP/AVPF 100
The RTP payload format for generic Forward Error Correction (FEC), as defined in [RFC 5109
] (and its predecessor, [RFC 2733
]), can either send the FEC stream as a separate RTP stream or send the FEC combined with the original RTP stream as a redundant encoding [RFC 2198
When sending FEC as a separate stream, the RTP payload format for generic FEC requires that FEC stream to be sent in a separate RTP session to the original stream, using the same SSRC, with the FEC stream being associated by matching the SSRC between sessions. The RTP session used for the original streams can include multiple RTP streams, and those RTP streams can use multiple media types. The repair session only needs one RTP payload type to indicate FEC data, irrespective of the number of FEC streams sent, since the SSRC is used to associate the FEC streams with the original streams. Hence, it is RECOMMENDED
that the FEC stream use the "application/ulpfec" media type in the case of support for [RFC 5109
] and the "application/ parityfec" media type in the case of support for [RFC 2733
]. It is legal, but NOT RECOMMENDED
, to send FEC streams using media-specific payload format names (e.g., using both the "audio/ulpfec" and "video/ulpfec" payload formats for a single RTP session containing both audio and video flows), since this (1) unnecessarily uses up RTP payload type values and (2) adds no value for demultiplexing because there might be multiple streams of the same media type).
The combination of an original RTP session using multiple media types with an associated generic FEC session can be signalled using SDP with the BUNDLE extension [RFC 8843
]. In this case, the RTP session carrying the FEC streams will be its own BUNDLE group. The "m=" line for each original stream and the "m=" line for the corresponding FEC stream are grouped using the SDP Grouping Framework, using either the [RFC 5956
] or, for backwards compatibility, the FEC grouping [RFC 4756
]. This is similar to the situation that arises for RTP retransmission with session-based multiplexing as discussed in Section 6.1
The [RFC 5576
] defines an SDP extension (the "FEC" semantic of the "ssrc-group" attribute) to signal FEC relationships between multiple RTP streams within a single RTP session. This cannot be used with generic FEC, since the FEC repair packets need to have the same SSRC value as the source packets being protected. There existed a proposal (now abandoned) for an Uneven Level Protection (ULP) extension to enable transmission of the FEC RTP streams within the same RTP session as the source stream [FEC-Src-Multiplexing
When the FEC is sent as a redundant encoding, the considerations in Section 6.3
The RTP payload format for redundant audio [RFC 2198
] can be used to protect audio streams. It can also be used along with the generic FEC payload format to send original and repair data in the same RTP packets. Both are compatible with RTP sessions containing multiple media types.
This payload format requires each different redundant encoding to use a different RTP payload type number. When used with generic FEC in sessions that contain multiple media types, this requires each media type to use a different payload type for the FEC stream. For example, if audio and text are sent in a single RTP session with generic ULP FEC sent as a redundant encoding for each, then payload types need to be assigned for FEC using the audio/ulpfec and text/ ulpfec payload formats. If multiple original payload types are used in the session, different redundant payload types need to be allocated for each one. This has potential to rapidly exhaust the available RTP payload type numbers.