tech-invite   World Map     

IETF     RFCs     Groups     SIP     ABNFs    |    3GPP     Specs     Glossaries     Architecture     IMS     UICC    |    search

RFC 3984


Pages: 83
Top     in Index     Prev     Next
 

RTP Payload Format for H.264 Video

Part 1 of 3, p. 1 to 31
None       Next RFC Part

Obsoleted by:    6184


Top       ToC       Page 1 
Network Working Group                                          S. Wenger
Request for Comments: 3984                               M.M. Hannuksela
Category: Standards Track                                 T. Stockhammer
                                                           M. Westerlund
                                                               D. Singer
                                                           February 2005


                   RTP Payload Format for H.264 Video

Status of This Memo

   This document specifies an Internet standards track protocol for the
   Internet community, and requests discussion and suggestions for
   improvements.  Please refer to the current edition of the "Internet
   Official Protocol Standards" (STD 1) for the standardization state
   and status of this protocol.  Distribution of this memo is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (2005).

Abstract

   This memo describes an RTP Payload format for the ITU-T
   Recommendation H.264 video codec and the technically identical
   ISO/IEC International Standard 14496-10 video codec.  The RTP payload
   format allows for packetization of one or more Network Abstraction
   Layer Units (NALUs), produced by an H.264 video encoder, in each RTP
   payload.  The payload format has wide applicability, as it supports
   applications from simple low bit-rate conversational usage, to
   Internet video streaming with interleaved transmission, to high bit-
   rate video-on-demand.

Table of Contents

   1.  Introduction..................................................  3
       1.1.  The H.264 Codec.........................................  3
       1.2.  Parameter Set Concept...................................  4
       1.3.  Network Abstraction Layer Unit Types....................  5
   2.  Conventions...................................................  6
   3.  Scope.........................................................  6
   4.  Definitions and Abbreviations.................................  6
       4.1.  Definitions.............................................  6
   5.  RTP Payload Format............................................  8
       5.1.  RTP Header Usage........................................  8
       5.2.  Common Structure of the RTP Payload Format.............. 11
       5.3.  NAL Unit Octet Usage.................................... 12

Top      ToC       Page 2 
       5.4.  Packetization Modes..................................... 14
       5.5.  Decoding Order Number (DON)............................. 15
       5.6.  Single NAL Unit Packet.................................. 18
       5.7.  Aggregation Packets..................................... 18
       5.8.  Fragmentation Units (FUs)............................... 27
   6.  Packetization Rules........................................... 31
       6.1.  Common Packetization Rules.............................. 31
       6.2.  Single NAL Unit Mode.................................... 32
       6.3.  Non-Interleaved Mode.................................... 32
       6.4.  Interleaved Mode........................................ 33
   7.  De-Packetization Process (Informative)........................ 33
       7.1.  Single NAL Unit and Non-Interleaved Mode................ 33
       7.2.  Interleaved Mode........................................ 34
       7.3.  Additional De-Packetization Guidelines.................. 36
   8.  Payload Format Parameters..................................... 37
       8.1.  MIME Registration....................................... 37
       8.2.  SDP Parameters.......................................... 52
       8.3.  Examples................................................ 58
       8.4.  Parameter Set Considerations............................ 60
   9.  Security Considerations....................................... 62
   10. Congestion Control............................................ 63
   11. IANA Considerations........................................... 64
   12. Informative Appendix: Application Examples.................... 65
       12.1. Video Telephony according to ITU-T Recommendation H.241
             Annex A................................................. 65
       12.2. Video Telephony, No Slice Data Partitioning, No NAL
             Unit Aggregation........................................ 65
       12.3. Video Telephony, Interleaved Packetization Using NAL
             Unit Aggregation........................................ 66
       12.4. Video Telephony with Data Partitioning.................. 66
       12.5. Video Telephony or Streaming with FUs and Forward
             Error Correction........................................ 67
       12.6. Low Bit-Rate Streaming.................................. 69
       12.7. Robust Packet Scheduling in Video Streaming............. 70
   13. Informative Appendix: Rationale for Decoding Order Number..... 71
       13.1. Introduction............................................ 71
       13.2. Example of Multi-Picture Slice Interleaving............. 71
       13.3. Example of Robust Packet Scheduling..................... 73
       13.4. Robust Transmission Scheduling of Redundant Coded
             Slices.................................................. 77
       13.5. Remarks on Other Design Possibilities................... 77
   14. Acknowledgements.............................................. 78
   15. References.................................................... 78
       15.1. Normative References.................................... 78
       15.2. Informative References.................................. 79
   Authors' Addresses................................................ 81
   Full Copyright Statement.......................................... 83

Top      ToC       Page 3 
1.  Introduction

1.1.  The H.264 Codec

   This memo specifies an RTP payload specification for the video coding
   standard known as ITU-T Recommendation H.264 [1] and ISO/IEC
   International Standard 14496 Part 10 [2] (both also known as Advanced
   Video Coding, or AVC).  Recommendation H.264 was approved by ITU-T on
   May 2003, and the approved draft specification is available for
   public review [8].  In this memo the H.264 acronym is used for the
   codec and the standard, but the memo is equally applicable to the
   ISO/IEC counterpart of the coding standard.

   The H.264 video codec has a very broad application range that covers
   all forms of digital compressed video from, low bit-rate Internet
   streaming applications to HDTV broadcast and Digital Cinema
   applications with nearly lossless coding.  Compared to the current
   state of technology, the overall performance of H.264 is such that
   bit rate savings of 50% or more are reported.  Digital Satellite TV
   quality, for example, was reported to be achievable at 1.5 Mbit/s,
   compared to the current operation point of MPEG 2 video at around 3.5
   Mbit/s [9].

   The codec specification [1] itself distinguishes conceptually between
   a video coding layer (VCL) and a network abstraction layer (NAL).
   The VCL contains the signal processing functionality of the codec;
   mechanisms such as transform, quantization, and motion compensated
   prediction; and a loop filter.  It follows the general concept of
   most of today's video codecs, a macroblock-based coder that uses
   inter picture prediction with motion compensation and transform
   coding of the residual signal.  The VCL encoder outputs slices: a bit
   string that contains the macroblock data of an integer number of
   macroblocks, and the information of the slice header (containing the
   spatial address of the first macroblock in the slice, the initial
   quantization parameter, and similar information).  Macroblocks in
   slices are arranged in scan order unless a different macroblock
   allocation is specified, by using the so-called Flexible Macroblock
   Ordering syntax.  In-picture prediction is used only within a slice.
   More information is provided in [9].

   The Network Abstraction Layer (NAL) encoder encapsulates the slice
   output of the VCL encoder into Network Abstraction Layer Units (NAL
   units), which are suitable for transmission over packet networks or
   use in packet oriented multiplex environments.  Annex B of H.264
   defines an encapsulation process to transmit such NAL units over
   byte-stream oriented networks.  In the scope of this memo, Annex B is
   not relevant.

Top      ToC       Page 4 
   Internally, the NAL uses NAL units.  A NAL unit consists of a one-
   byte header and the payload byte string.  The header indicates the
   type of the NAL unit, the (potential) presence of bit errors or
   syntax violations in the NAL unit payload, and information regarding
   the relative importance of the NAL unit for the decoding process.
   This RTP payload specification is designed to be unaware of the bit
   string in the NAL unit payload.

   One of the main properties of H.264 is the complete decoupling of the
   transmission time, the decoding time, and the sampling or
   presentation time of slices and pictures.  The decoding process
   specified in H.264 is unaware of time, and the H.264 syntax does not
   carry information such as the number of skipped frames (as is common
   in the form of the Temporal Reference in earlier video compression
   standards).  Also, there are NAL units that affect many pictures and
   that are, therefore, inherently timeless.  For this reason, the
   handling of the RTP timestamp requires some special considerations
   for NAL units for which the sampling or presentation time is not
   defined or, at transmission time, unknown.

1.2.  Parameter Set Concept

   One very fundamental design concept of H.264 is to generate self-
   contained packets, to make mechanisms such as the header duplication
   of RFC 2429 [10] or MPEG-4's Header Extension Code (HEC) [11]
   unnecessary.  This was achieved by decoupling information relevant to
   more than one slice from the media stream.  This higher layer meta
   information should be sent reliably, asynchronously, and in advance
   from the RTP packet stream that contains the slice packets.
   (Provisions for sending this information in-band are also available
   for applications that do not have an out-of-band transport channel
   appropriate for the purpose.)  The combination of the higher-level
   parameters is called a parameter set.  The H.264 specification
   includes two types of parameter sets: sequence parameter set and
   picture parameter set.  An active sequence parameter set remains
   unchanged throughout a coded video sequence, and an active picture
   parameter set remains unchanged within a coded picture.  The sequence
   and picture parameter set structures contain information such as
   picture size, optional coding modes employed, and macroblock to slice
   group map.

   To be able to change picture parameters (such as the picture size)
   without having to transmit parameter set updates synchronously to the
   slice packet stream, the encoder and decoder can maintain a list of
   more than one sequence and picture parameter set.  Each slice header
   contains a codeword that indicates the sequence and picture parameter
   set to be used.

Top      ToC       Page 5 
   This mechanism allows the decoupling of the transmission of parameter
   sets from the packet stream, and the transmission of them by external
   means (e.g., as a side effect of the capability exchange), or through
   a (reliable or unreliable) control protocol.  It may even be possible
   that they are never transmitted but are fixed by an application
   design specification.

1.3.  Network Abstraction Layer Unit Types

   Tutorial information on the NAL design can be found in [12], [13],
   and [14].

   All NAL units consist of a single NAL unit type octet, which also
   co-serves as the payload header of this RTP payload format.  The
   payload of a NAL unit follows immediately.

   The syntax and semantics of the NAL unit type octet are specified in
   [1], but the essential properties of the NAL unit type octet are
   summarized below.  The NAL unit type octet has the following format:

      +---------------+
      |0|1|2|3|4|5|6|7|
      +-+-+-+-+-+-+-+-+
      |F|NRI|  Type   |
      +---------------+

   The semantics of the components of the NAL unit type octet, as
   specified in the H.264 specification, are described briefly below.

   F: 1 bit
      forbidden_zero_bit.  The H.264 specification declares a value of
      1 as a syntax violation.

   NRI: 2 bits
      nal_ref_idc.  A value of 00 indicates that the content of the NAL
      unit is not used to reconstruct reference pictures for inter
      picture prediction.  Such NAL units can be discarded without
      risking the integrity of the reference pictures.  Values greater
      than 00 indicate that the decoding of the NAL unit is required to
      maintain the integrity of the reference pictures.

   Type: 5 bits
      nal_unit_type.  This component specifies the NAL unit payload type
      as defined in table 7-1 of [1], and later within this memo.  For a
      reference of all currently defined NAL unit types and their
      semantics, please refer to section 7.4.1 in [1].

Top      ToC       Page 6 
   This memo introduces new NAL unit types, which are presented in
   section 5.2.  The NAL unit types defined in this memo are marked as
   unspecified in [1].  Moreover, this specification extends the
   semantics of F and NRI as described in section 5.3.

2.  Conventions

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in BCP 14, RFC 2119 [3].

   This specification uses the notion of setting and clearing a bit when
   bit fields are handled.  Setting a bit is the same as assigning that
   bit the value of 1 (On).  Clearing a bit is the same as assigning
   that bit the value of 0 (Off).

3.  Scope

   This payload specification can only be used to carry the "naked"
   H.264 NAL unit stream over RTP, and not the bitstream format
   discussed in Annex B of H.264.  Likely, the first applications of
   this specification will be in the conversational multimedia field,
   video telephony or video conferencing, but the payload format also
   covers other applications, such as Internet streaming and TV over IP.

4.  Definitions and Abbreviations

4.1.  Definitions

   This document uses the definitions of [1].  The following terms,
   defined in [1], are summed up for convenience:

      access unit: A set of NAL units always containing a primary coded
      picture.  In addition to the primary coded picture, an access unit
      may also contain one or more redundant coded pictures or other NAL
      units not containing slices or slice data partitions of a coded
      picture.  The decoding of an access unit always results in a
      decoded picture.

      coded video sequence: A sequence of access units that consists, in
      decoding order, of an instantaneous decoding refresh (IDR) access
      unit followed by zero or more non-IDR access units including all
      subsequent access units up to but not including any subsequent IDR
      access unit.

      IDR access unit: An access unit in which the primary coded picture
      is an IDR picture.

Top      ToC       Page 7 
      IDR picture: A coded picture containing only slices with I or SI
      slice types that causes a "reset" in the decoding process.  After
      the decoding of an IDR picture, all following coded pictures in
      decoding order can be decoded without inter prediction from any
      picture decoded prior to the IDR picture.

      primary coded picture: The coded representation of a picture to be
      used by the decoding process for a bitstream conforming to H.264.
      The primary coded picture contains all macroblocks of the picture.

      redundant coded picture: A coded representation of a picture or a
      part of a picture.  The content of a redundant coded picture shall
      not be used by the decoding process for a bitstream conforming to
      H.264.  The content of a redundant coded picture may be used by
      the decoding process for a bitstream that contains errors or
      losses.

      VCL NAL unit: A collective term used to refer to coded slice and
      coded data partition NAL units.

   In addition, the following definitions apply:

      decoding order number (DON): A field in the payload structure, or
      a derived variable indicating NAL unit decoding order.  Values of
      DON are in the range of 0 to 65535, inclusive.  After reaching the
      maximum value, the value of DON wraps around to 0.

      NAL unit decoding order: A NAL unit order that conforms to the
      constraints on NAL unit order given in section 7.4.1.2 in [1].

      transmission order: The order of packets in ascending RTP sequence
      number order (in modulo arithmetic).  Within an aggregation
      packet, the NAL unit transmission order is the same as the order
      of appearance of NAL units in the packet.

      media aware network element (MANE): A network element, such as a
      middlebox or application layer gateway that is capable of parsing
      certain aspects of the RTP payload headers or the RTP payload and
      reacting to the contents.

         Informative note: The concept of a MANE goes beyond normal
         routers or gateways in that a MANE has to be aware of the
         signaling (e.g., to learn about the payload type mappings of
         the media streams), and in that it has to be trusted when
         working with SRTP.  The advantage of using MANEs is that they
         allow packets to be dropped according to the needs of the media
         coding.  For example, if a MANE has to drop packets due to
         congestion on a certain link, it can identify those packets

Top      ToC       Page 8 
         whose dropping has the smallest negative impact on the user
         experience and remove them in order to remove the congestion
         and/or keep the delay low.

   Abbreviations

      DON:        Decoding Order Number
      DONB:       Decoding Order Number Base
      DOND:       Decoding Order Number Difference
      FEC:        Forward Error Correction
      FU:         Fragmentation Unit
      IDR:        Instantaneous Decoding Refresh
      IEC:        International Electrotechnical Commission
      ISO:        International Organization for Standardization
      ITU-T:      International Telecommunication Union,
                  Telecommunication Standardization Sector
      MANE:       Media Aware Network Element
      MTAP:       Multi-Time Aggregation Packet
      MTAP16:     MTAP with 16-bit timestamp offset
      MTAP24:     MTAP with 24-bit timestamp offset
      NAL:        Network Abstraction Layer
      NALU:       NAL Unit
      SEI:        Supplemental Enhancement Information
      STAP:       Single-Time Aggregation Packet
      STAP-A:     STAP type A
      STAP-B:     STAP type B
      TS:         Timestamp
      VCL:        Video Coding Layer

5.  RTP Payload Format

5.1.  RTP Header Usage

   The format of the RTP header is specified in RFC 3550 [4] and
   reprinted in Figure 1 for convenience.  This payload format uses the
   fields of the header in a manner consistent with that specification.

   When one NAL unit is encapsulated per RTP packet, the RECOMMENDED RTP
   payload format is specified in section 5.6.  The RTP payload (and the
   settings for some RTP header bits) for aggregation packets and
   fragmentation units are specified in sections 5.7 and 5.8,
   respectively.

Top      ToC       Page 9 
       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |V=2|P|X|  CC   |M|     PT      |       sequence number         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                           timestamp                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |           synchronization source (SSRC) identifier            |
      +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
      |            contributing source (CSRC) identifiers             |
      |                             ....                              |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 1.  RTP header according to RFC 3550

   The RTP header information to be set according to this RTP payload
   format is set as follows:

   Marker bit (M): 1 bit
      Set for the very last packet of the access unit indicated by the
      RTP timestamp, in line with the normal use of the M bit in video
      formats, to allow an efficient playout buffer handling.  For
      aggregation packets (STAP and MTAP), the marker bit in the RTP
      header MUST be set to the value that the marker bit of the last
      NAL unit of the aggregation packet would have been if it were
      transported in its own RTP packet.  Decoders MAY use this bit as
      an early indication of the last packet of an access unit, but MUST
      NOT rely on this property.

         Informative note: Only one M bit is associated with an
         aggregation packet carrying multiple NAL units.  Thus, if a
         gateway has re-packetized an aggregation packet into several
         packets, it cannot reliably set the M bit of those packets.

   Payload type (PT): 7 bits
      The assignment of an RTP payload type for this new packet format
      is outside the scope of this document and will not be specified
      here.  The assignment of a payload type has to be performed either
      through the profile used or in a dynamic way.

   Sequence number (SN): 16 bits
      Set and used in accordance with RFC 3550.  For the single NALU and
      non-interleaved packetization mode, the sequence number is used to
      determine decoding order for the NALU.

   Timestamp: 32 bits
      The RTP timestamp is set to the sampling timestamp of the content.
      A 90 kHz clock rate MUST be used.

Top      ToC       Page 10 
      If the NAL unit has no timing properties of its own (e.g.,
      parameter set and SEI NAL units), the RTP timestamp is set to the
      RTP timestamp of the primary coded picture of the access unit in
      which the NAL unit is included, according to section 7.4.1.2 of
      [1].

      The setting of the RTP Timestamp for MTAPs is defined in section
      5.7.2.

      Receivers SHOULD ignore any picture timing SEI messages included
      in access units that have only one display timestamp.  Instead,
      receivers SHOULD use the RTP timestamp for synchronizing the
      display process.

      RTP senders SHOULD NOT transmit picture timing SEI messages for
      pictures that are not supposed to be displayed as multiple fields.

      If one access unit has more than one display timestamp carried in
      a picture timing SEI message, then the information in the SEI
      message SHOULD be treated as relative to the RTP timestamp, with
      the earliest event occurring at the time given by the RTP
      timestamp, and subsequent events later, as given by the difference
      in SEI message picture timing values.  Let tSEI1, tSEI2, ...,
      tSEIn be the display timestamps carried in the SEI message of an
      access unit, where tSEI1 is the earliest of all such timestamps.
      Let tmadjst() be a function that adjusts the SEI messages time
      scale to a 90-kHz time scale.  Let TS be the RTP timestamp.  Then,
      the display time for the event associated with tSEI1 is TS.  The
      display time for the event with tSEIx, where x is [2..n] is TS +
      tmadjst (tSEIx - tSEI1).

         Informative note: Displaying coded frames as fields is needed
         commonly in an operation known as 3:2 pulldown, in which film
         content that consists of coded frames is displayed on a display
         using interlaced scanning.  The picture timing SEI message
         enables carriage of multiple timestamps for the same coded
         picture, and therefore the 3:2 pulldown process is perfectly
         controlled.  The picture timing SEI message mechanism is
         necessary because only one timestamp per coded frame can be
         conveyed in the RTP timestamp.

         Informative note: Because H.264 allows the decoding order to be
         different from the display order, values of RTP timestamps may
         not be monotonically non-decreasing as a function of RTP
         sequence numbers.  Furthermore, the value for interarrival
         jitter reported in the RTCP reports may not be a trustworthy
         indication of the network performance, as the calculation rules

Top      ToC       Page 11 
         for interarrival jitter (section 6.4.1 of RFC 3550) assume that
         the RTP timestamp of a packet is directly proportional to its
         transmission time.

5.2.  Common Structure of the RTP Payload Format

   The payload format defines three different basic payload structures.
   A receiver can identify the payload structure by the first byte of
   the RTP payload, which co-serves as the RTP payload header and, in
   some cases, as the first byte of the payload.  This byte is always
   structured as a NAL unit header.  The NAL unit type field indicates
   which structure is present.  The possible structures are as follows:

   Single NAL Unit Packet: Contains only a single NAL unit in the
   payload.  The NAL header type field will be equal to the original NAL
   unit type; i.e., in the range of 1 to 23, inclusive.  Specified in
   section 5.6.

   Aggregation packet: Packet type used to aggregate multiple NAL units
   into a single RTP payload.  This packet exists in four versions, the
   Single-Time Aggregation Packet type A (STAP-A), the Single-Time
   Aggregation Packet type B (STAP-B), Multi-Time Aggregation Packet
   (MTAP) with 16-bit offset (MTAP16), and Multi-Time Aggregation Packet
   (MTAP) with 24-bit offset (MTAP24).  The NAL unit type numbers
   assigned for STAP-A, STAP-B, MTAP16, and MTAP24 are 24, 25, 26, and
   27, respectively.  Specified in section 5.7.

   Fragmentation unit: Used to fragment a single NAL unit over multiple
   RTP packets.  Exists with two versions, FU-A and FU-B, identified
   with the NAL unit type numbers 28 and 29, respectively.  Specified in
   section 5.8.

   Table 1.  Summary of NAL unit types and their payload structures

      Type   Packet    Type name                        Section
      ---------------------------------------------------------
      0      undefined                                    -
      1-23   NAL unit  Single NAL unit packet per H.264   5.6
      24     STAP-A    Single-time aggregation packet     5.7.1
      25     STAP-B    Single-time aggregation packet     5.7.1
      26     MTAP16    Multi-time aggregation packet      5.7.2
      27     MTAP24    Multi-time aggregation packet      5.7.2
      28     FU-A      Fragmentation unit                 5.8
      29     FU-B      Fragmentation unit                 5.8
      30-31  undefined                                    -

Top      ToC       Page 12 
      Informative note: This specification does not limit the size of
      NAL units encapsulated in single NAL unit packets and
      fragmentation units.  The maximum size of a NAL unit encapsulated
      in any aggregation packet is 65535 bytes.

5.3.  NAL Unit Octet Usage

   The structure and semantics of the NAL unit octet were introduced in
   section 1.3.  For convenience, the format of the NAL unit type octet
   is reprinted below:

      +---------------+
      |0|1|2|3|4|5|6|7|
      +-+-+-+-+-+-+-+-+
      |F|NRI|  Type   |
      +---------------+

   This section specifies the semantics of F and NRI according to this
   specification.

   F: 1 bit
      forbidden_zero_bit.  A value of 0 indicates that the NAL unit type
      octet and payload should not contain bit errors or other syntax
      violations.  A value of 1 indicates that the NAL unit type octet
      and payload may contain bit errors or other syntax violations.

      MANEs SHOULD set the F bit to indicate detected bit errors in the
      NAL unit.  The H.264 specification requires that the F bit is
      equal to 0.  When the F bit is set, the decoder is advised that
      bit errors or any other syntax violations may be present in the
      payload or in the NAL unit type octet.  The simplest decoder
      reaction to a NAL unit in which the F bit is equal to 1 is to
      discard such a NAL unit and to conceal the lost data in the
      discarded NAL unit.

   NRI: 2 bits
      nal_ref_idc.  The semantics of value 00 and a non-zero value
      remain unchanged from the H.264 specification.  In other words, a
      value of 00 indicates that the content of the NAL unit is not used
      to reconstruct reference pictures for inter picture prediction.
      Such NAL units can be discarded without risking the integrity of
      the reference pictures.  Values greater than 00 indicate that the
      decoding of the NAL unit is required to maintain the integrity of
      the reference pictures.

      In addition to the specification above, according to this RTP
      payload specification, values of NRI greater than 00 indicate the
      relative transport priority, as determined by the encoder.  MANEs

Top      ToC       Page 13 
      can use this information to protect more important NAL units
      better than they do less important NAL units.  The highest
      transport priority is 11, followed by 10, and then by 01; finally,
      00 is the lowest.

         Informative note: Any non-zero value of NRI is handled
         identically in H.264 decoders.  Therefore, receivers need not
         manipulate the value of NRI when passing NAL units to the
         decoder.

      An H.264 encoder MUST set the value of NRI according to the H.264
      specification (subclause 7.4.1) when the value of nal_unit_type is
      in the range of 1 to 12, inclusive.  In particular, the H.264
      specification requires that the value of NRI SHALL be equal to 0
      for all NAL units having nal_unit_type equal to 6, 9, 10, 11, or
      12.

      For NAL units having nal_unit_type equal to 7 or 8 (indicating a
      sequence parameter set or a picture parameter set, respectively),
      an H.264 encoder SHOULD set the value of NRI to 11 (in binary
      format).  For coded slice NAL units of a primary coded picture
      having nal_unit_type equal to 5 (indicating a coded slice
      belonging to an IDR picture), an H.264 encoder SHOULD set the
      value of NRI to 11 (in binary format).

      For a mapping of the remaining nal_unit_types to NRI values, the
      following example MAY be used and has been shown to be efficient
      in a certain environment [13].  Other mappings MAY also be
      desirable, depending on the application and the H.264/AVC Annex A
      profile in use.

         Informative note: Data Partitioning is not available in certain
         profiles; e.g., in the Main or Baseline profiles.
         Consequently, the nal unit types 2, 3, and 4 can occur only if
         the video bitstream conforms to a profile in which data
         partitioning is allowed and not in streams that conform to the
         Main or Baseline profiles.

      Table 2.  Example of NRI values for coded slices and coded slice
      data partitions of primary coded reference pictures

      NAL Unit Type     Content of NAL unit              NRI (binary)
      ----------------------------------------------------------------
       1              non-IDR coded slice                         10
       2              Coded slice data partition A                10
       3              Coded slice data partition B                01
       4              Coded slice data partition C                01

Top      ToC       Page 14 
         Informative note: As mentioned before, the NRI value of non-
         reference pictures is 00 as mandated by H.264/AVC.

      An H.264 encoder SHOULD set the value of NRI for coded slice and
      coded slice data partition NAL units of redundant coded reference
      pictures equal to 01 (in binary format).

      Definitions of the values for NRI for NAL unit types 24 to 29,
      inclusive, are given in sections 5.7 and 5.8 of this memo.

      No recommendation for the value of NRI is given for NAL units
      having nal_unit_type in the range of 13 to 23, inclusive, because
      these values are reserved for ITU-T and ISO/IEC.  No
      recommendation for the value of NRI is given for NAL units having
      nal_unit_type equal to 0 or in the range of 30 to 31, inclusive,
      as the semantics of these values are not specified in this memo.

5.4.  Packetization Modes

   This memo specifies three cases of packetization modes:

      o Single NAL unit mode
      o Non-interleaved mode
      o Interleaved mode

   The single NAL unit mode is targeted for conversational systems that
   comply with ITU-T Recommendation H.241 [15] (see section 12.1).  The
   non-interleaved mode is targeted for conversational systems that may
   not comply with ITU-T Recommendation H.241.  In the non-interleaved
   mode, NAL units are transmitted in NAL unit decoding order.  The
   interleaved mode is targeted for systems that do not require very low
   end-to-end latency.  The interleaved mode allows transmission of NAL
   units out of NAL unit decoding order.

   The packetization mode in use MAY be signaled by the value of the
   OPTIONAL packetization-mode MIME parameter or by external means.  The
   used packetization mode governs which NAL unit types are allowed in
   RTP payloads.  Table 3 summarizes the allowed NAL unit types for each
   packetization mode.  Some NAL unit type values (indicated as
   undefined in Table 3) are reserved for future extensions.  NAL units
   of those types SHOULD NOT be sent by a sender and MUST be ignored by
   a receiver.  For example, the Types 1-23, with the associated packet
   type "NAL unit", are allowed in "Single NAL Unit Mode" and in "Non-
   Interleaved Mode", but disallowed in "Interleaved Mode".
   Packetization modes are explained in more detail in section 6.

Top      ToC       Page 15 
   Table 3.  Summary of allowed NAL unit types for each packetization
   mode (yes = allowed, no = disallowed, ig = ignore)

      Type   Packet    Single NAL    Non-Interleaved    Interleaved
                       Unit Mode           Mode             Mode
      -------------------------------------------------------------

      0      undefined     ig               ig               ig
      1-23   NAL unit     yes              yes               no
      24     STAP-A        no              yes               no
      25     STAP-B        no               no              yes
      26     MTAP16        no               no              yes
      27     MTAP24        no               no              yes
      28     FU-A          no              yes              yes
      29     FU-B          no               no              yes
      30-31  undefined     ig               ig               ig

5.5.  Decoding Order Number (DON)

   In the interleaved packetization mode, the transmission order of NAL
   units is allowed to differ from the decoding order of the NAL units.
   Decoding order number (DON) is a field in the payload structure or a
   derived variable that indicates the NAL unit decoding order.
   Rationale and examples of use cases for transmission out of decoding
   order and for the use of DON are given in section 13.

   The coupling of transmission and decoding order is controlled by the
   OPTIONAL sprop-interleaving-depth MIME parameter as follows.  When
   the value of the OPTIONAL sprop-interleaving-depth MIME parameter is
   equal to 0 (explicitly or per default) or transmission of NAL units
   out of their decoding order is disallowed by external means, the
   transmission order of NAL units MUST conform to the NAL unit decoding
   order.  When the value of the OPTIONAL sprop-interleaving-depth MIME
   parameter is greater than 0 or transmission of NAL units out of their
   decoding order is allowed by external means,

   o  the order of NAL units in an MTAP16 and an MTAP24 is NOT REQUIRED
      to be the NAL unit decoding order, and

   o  the order of NAL units generated by decapsulating STAP-Bs, MTAPs,
      and FUs in two consecutive packets is NOT REQUIRED to be the NAL
      unit decoding order.

   The RTP payload structures for a single NAL unit packet, an STAP-A,
   and an FU-A do not include DON.  STAP-B and FU-B structures include
   DON, and the structure of MTAPs enables derivation of DON as
   specified in section 5.7.2.

Top      ToC       Page 16 
      Informative note: When an FU-A occurs in interleaved mode, it
      always follows an FU-B, which sets its DON.

      Informative note: If a transmitter wants to encapsulate a single
      NAL unit per packet and transmit packets out of their decoding
      order, STAP-B packet type can be used.

   In the single NAL unit packetization mode, the transmission order of
   NAL units, determined by the RTP sequence number, MUST be the same as
   their NAL unit decoding order.  In the non-interleaved packetization
   mode, the transmission order of NAL units in single NAL unit packets,
   STAP-As, and FU-As MUST be the same as their NAL unit decoding order.
   The NAL units within an STAP MUST appear in the NAL unit decoding
   order.  Thus, the decoding order is first provided through the
   implicit order within a STAP, and second provided through the RTP
   sequence number for the order between STAPs, FUs, and single NAL unit
   packets.

   Signaling of the value of DON for NAL units carried in STAP-B, MTAP,
   and a series of fragmentation units starting with an FU-B is
   specified in sections 5.7.1, 5.7.2, and 5.8, respectively.  The DON
   value of the first NAL unit in transmission order MAY be set to any
   value.  Values of DON are in the range of 0 to 65535, inclusive.
   After reaching the maximum value, the value of DON wraps around to 0.

   The decoding order of two NAL units contained in any STAP-B, MTAP, or
   a series of fragmentation units starting with an FU-B is determined
   as follows.  Let DON(i) be the decoding order number of the NAL unit
   having index i in the transmission order.  Function don_diff(m,n) is
   specified as follows:

      If DON(m) == DON(n), don_diff(m,n) = 0

      If (DON(m) < DON(n) and DON(n) - DON(m) < 32768),
      don_diff(m,n) = DON(n) - DON(m)

      If (DON(m) > DON(n) and DON(m) - DON(n) >= 32768),
      don_diff(m,n) = 65536 - DON(m) + DON(n)

      If (DON(m) < DON(n) and DON(n) - DON(m) >= 32768),
      don_diff(m,n) = - (DON(m) + 65536 - DON(n))

      If (DON(m) > DON(n) and DON(m) - DON(n) < 32768),
      don_diff(m,n) = - (DON(m) - DON(n))

   A positive value of don_diff(m,n) indicates that the NAL unit having
   transmission order index n follows, in decoding order, the NAL unit
   having transmission order index m.  When don_diff(m,n) is equal to 0,

Top      ToC       Page 17 
   then the NAL unit decoding order of the two NAL units can be in
   either order.  A negative value of don_diff(m,n) indicates that the
   NAL unit having transmission order index n precedes, in decoding
   order, the NAL unit having transmission order index m.

   Values of DON related fields (DON, DONB, and DOND; see section 5.7)
   MUST be such that the decoding order determined by the values of DON,
   as specified above, conforms to the NAL unit decoding order.  If the
   order of two NAL units in NAL unit decoding order is switched and the
   new order does not conform to the NAL unit decoding order, the NAL
   units MUST NOT have the same value of DON.  If the order of two
   consecutive NAL units in the NAL unit stream is switched and the new
   order still conforms to the NAL unit decoding order, the NAL units
   MAY have the same value of DON.  For example, when arbitrary slice
   order is allowed by the video coding profile in use, all the coded
   slice NAL units of a coded picture are allowed to have the same value
   of DON.  Consequently, NAL units having the same value of DON can be
   decoded in any order, and two NAL units having a different value of
   DON should be passed to the decoder in the order specified above.
   When two consecutive NAL units in the NAL unit decoding order have a
   different value of DON, the value of DON for the second NAL unit in
   decoding order SHOULD be the value of DON for the first, incremented
   by one.

   An example of the decapsulation process to recover the NAL unit
   decoding order is given in section 7.

      Informative note: Receivers should not expect that the absolute
      difference of values of DON for two consecutive NAL units in the
      NAL unit decoding order will be equal to one, even in error-free
      transmission.  An increment by one is not required, as at the time
      of associating values of DON to NAL units, it may not be known
      whether all NAL units are delivered to the receiver.  For example,
      a gateway may not forward coded slice NAL units of non-reference
      pictures or SEI NAL units when there is a shortage of bit rate in
      the network to which the packets are forwarded.  In another
      example, a live broadcast is interrupted by pre-encoded content,
      such as commercials, from time to time.  The first intra picture
      of a pre-encoded clip is transmitted in advance to ensure that it
      is readily available in the receiver.  When transmitting the first
      intra picture, the originator does not exactly know how many NAL
      units will be encoded before the first intra picture of the pre-
      encoded clip follows in decoding order.  Thus, the values of DON
      for the NAL units of the first intra picture of the pre-encoded
      clip have to be estimated when they are transmitted, and gaps in
      values of DON may occur.

Top      ToC       Page 18 
5.6.  Single NAL Unit Packet

   The single NAL unit packet defined here MUST contain only one NAL
   unit, of the types defined in [1].  This means that neither an
   aggregation packet nor a fragmentation unit can be used within a
   single NAL unit packet.  A NAL unit stream composed by decapsulating
   single NAL unit packets in RTP sequence number order MUST conform to
   the NAL unit decoding order.  The structure of the single NAL unit
   packet is shown in Figure 2.

      Informative note: The first byte of a NAL unit co-serves as the
      RTP payload header.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |F|NRI|  type   |                                               |
      +-+-+-+-+-+-+-+-+                                               |
      |                                                               |
      |               Bytes 2..n of a Single NAL unit                 |
      |                                                               |
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :...OPTIONAL RTP padding        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 2.  RTP payload format for single NAL unit packet

5.7.  Aggregation Packets

   Aggregation packets are the NAL unit aggregation scheme of this
   payload specification.  The scheme is introduced to reflect the
   dramatically different MTU sizes of two key target networks:
   wireline IP networks (with an MTU size that is often limited by the
   Ethernet MTU size; roughly 1500 bytes), and IP or non-IP (e.g., ITU-T
   H.324/M) based wireless communication systems with preferred
   transmission unit sizes of 254 bytes or less.  To prevent media
   transcoding between the two worlds, and to avoid undesirable
   packetization overhead, a NAL unit aggregation scheme is introduced.

   Two types of aggregation packets are defined by this specification:

   o  Single-time aggregation packet (STAP): aggregates NAL units with
      identical NALU-time.  Two types of STAPs are defined, one without
      DON (STAP-A) and another including DON (STAP-B).

   o  Multi-time aggregation packet (MTAP): aggregates NAL units with
      potentially differing NALU-time.  Two different MTAPs are defined,
      differing in the length of the NAL unit timestamp offset.

Top      ToC       Page 19 
   The term NALU-time is defined as the value that the RTP timestamp
   would have if that NAL unit would be transported in its own RTP
   packet.

   Each NAL unit to be carried in an aggregation packet is encapsulated
   in an aggregation unit.  Please see below for the four different
   aggregation units and their characteristics.

   The structure of the RTP payload format for aggregation packets is
   presented in Figure 3.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |F|NRI|  type   |                                               |
      +-+-+-+-+-+-+-+-+                                               |
      |                                                               |
      |             one or more aggregation units                     |
      |                                                               |
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :...OPTIONAL RTP padding        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 3.  RTP payload format for aggregation packets

   MTAPs and STAPs share the following packetization rules:  The RTP
   timestamp MUST be set to the earliest of the NALU times of all the
   NAL units to be aggregated.  The type field of the NAL unit type
   octet MUST be set to the appropriate value, as indicated in Table 4.
   The F bit MUST be cleared if all F bits of the aggregated NAL units
   are zero; otherwise, it MUST be set.  The value of NRI MUST be the
   maximum of all the NAL units carried in the aggregation packet.

      Table 4.  Type field for STAPs and MTAPs

      Type   Packet    Timestamp offset   DON related fields
                       field length       (DON, DONB, DOND)
                       (in bits)          present
      --------------------------------------------------------
      24     STAP-A       0                 no
      25     STAP-B       0                 yes
      26     MTAP16      16                 yes
      27     MTAP24      24                 yes

   The marker bit in the RTP header is set to the value that the marker
   bit of the last NAL unit of the aggregated packet would have if it
   were transported in its own RTP packet.

Top      ToC       Page 20 
   The payload of an aggregation packet consists of one or more
   aggregation units.  See sections 5.7.1 and 5.7.2 for the four
   different types of aggregation units.  An aggregation packet can
   carry as many aggregation units as necessary; however, the total
   amount of data in an aggregation packet obviously MUST fit into an IP
   packet, and the size SHOULD be chosen so that the resulting IP packet
   is smaller than the MTU size.  An aggregation packet MUST NOT contain
   fragmentation units specified in section 5.8.  Aggregation packets
   MUST NOT be nested; i.e., an aggregation packet MUST NOT contain
   another aggregation packet.

5.7.1.  Single-Time Aggregation Packet

   Single-time aggregation packet (STAP) SHOULD be used whenever NAL
   units are aggregated that all share the same NALU-time.  The payload
   of an STAP-A does not include DON and consists of at least one
   single-time aggregation unit, as presented in Figure 4.  The payload
   of an STAP-B consists of a 16-bit unsigned decoding order number
   (DON) (in network byte order) followed by at least one single-time
   aggregation unit, as presented in Figure 5.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                      :                                               |
      +-+-+-+-+-+-+-+-+                                               |
      |                                                               |
      |                single-time aggregation units                  |
      |                                                               |
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 4.  Payload format for STAP-A

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                      :  decoding order number (DON)  |               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
      |                                                               |
      |                single-time aggregation units                  |
      |                                                               |
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 5.  Payload format for STAP-B

Top      ToC       Page 21 
   The DON field specifies the value of DON for the first NAL unit in an
   STAP-B in transmission order.  For each successive NAL unit in
   appearance order in an STAP-B, the value of DON is equal to (the
   value of DON of the previous NAL unit in the STAP-B + 1) % 65536, in
   which '%' stands for the modulo operation.

   A single-time aggregation unit consists of 16-bit unsigned size
   information (in network byte order) that indicates the size of the
   following NAL unit in bytes (excluding these two octets, but
   including the NAL unit type octet of the NAL unit), followed by the
   NAL unit itself, including its NAL unit type byte.  A single-time
   aggregation unit is byte aligned within the RTP payload, but it may
   not be aligned on a 32-bit word boundary.  Figure 6 presents the
   structure of the single-time aggregation unit.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                      :        NAL unit size          |               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
      |                                                               |
      |                           NAL unit                            |
      |                                                               |
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 6.  Structure for single-time aggregation unit

Top      ToC       Page 22 
   Figure 7 presents an example of an RTP packet that contains an STAP-
   A.  The STAP contains two single-time aggregation units, labeled as 1
   and 2 in the figure.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                          RTP Header                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |STAP-A NAL HDR |         NALU 1 Size           | NALU 1 HDR    |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                         NALU 1 Data                           |
      :                                                               :
      +               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |               | NALU 2 Size                   | NALU 2 HDR    |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                         NALU 2 Data                           |
      :                                                               :
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :...OPTIONAL RTP padding        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 7.  An example of an RTP packet including an STAP-A and two
                 single-time aggregation units

Top      ToC       Page 23 
   Figure 8 presents an example of an RTP packet that contains an STAP-
   B.  The STAP contains two single-time aggregation units, labeled as 1
   and 2 in the figure.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                          RTP Header                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |STAP-B NAL HDR | DON                           | NALU 1 Size   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      | NALU 1 Size   | NALU 1 HDR    | NALU 1 Data                   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
      :                                                               :
      +               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |               | NALU 2 Size                   | NALU 2 HDR    |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                       NALU 2 Data                             |
      :                                                               :
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :...OPTIONAL RTP padding        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 8.  An example of an RTP packet including an STAP-B and two
                 single-time aggregation units

5.7.2.  Multi-Time Aggregation Packets (MTAPs)

   The NAL unit payload of MTAPs consists of a 16-bit unsigned decoding
   order number base (DONB) (in network byte order) and one or more
   multi-time aggregation units, as presented in Figure 9.  DONB MUST
   contain the value of DON for the first NAL unit in the NAL unit
   decoding order among the NAL units of the MTAP.

      Informative note: The first NAL unit in the NAL unit decoding
      order is not necessarily the first NAL unit in the order in which
      the NAL units are encapsulated in an MTAP.

Top      ToC       Page 24 
       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                      :  decoding order number base   |               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
      |                                                               |
      |                 multi-time aggregation units                  |
      |                                                               |
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 9.  NAL unit payload format for MTAPs

   Two different multi-time aggregation units are defined in this
   specification.  Both of them consist of 16 bits unsigned size
   information of the following NAL unit (in network byte order), an 8-
   bit unsigned decoding order number difference (DOND), and n bits (in
   network byte order) of timestamp offset (TS offset) for this NAL
   unit, whereby n can be 16 or 24.  The choice between the different
   MTAP types (MTAP16 and MTAP24) is application dependent: the larger
   the timestamp offset is, the higher the flexibility of the MTAP, but
   the overhead is also higher.

   The structure of the multi-time aggregation units for MTAP16 and
   MTAP24 are presented in Figures 10 and 11, respectively.  The
   starting or ending position of an aggregation unit within a packet is
   NOT REQUIRED to be on a 32-bit word boundary.  The DON of the
   following NAL unit is equal to (DONB + DOND) % 65536, in which %
   denotes the modulo operation.  This memo does not specify how the NAL
   units within an MTAP are ordered, but, in most cases, NAL unit
   decoding order SHOULD be used.

   The timestamp offset field MUST be set to a value equal to the value
   of the following formula: If the NALU-time is larger than or equal to
   the RTP timestamp of the packet, then the timestamp offset equals
   (the NALU-time of the NAL unit - the RTP timestamp of the packet).
   If the NALU-time is smaller than the RTP timestamp of the packet,
   then the timestamp offset is equal to the NALU-time + (2^32 - the RTP
   timestamp of the packet).

Top      ToC       Page 25 
       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      :        NAL unit size          |      DOND     |  TS offset    |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  TS offset    |                                               |
      +-+-+-+-+-+-+-+-+              NAL unit                         |
      |                                                               |
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 10.  Multi-time aggregation unit for MTAP16

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      :        NALU unit size         |      DOND     |  TS offset    |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |         TS offset             |                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
      |                              NAL unit                         |
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 11.  Multi-time aggregation unit for MTAP24

   For the "earliest" multi-time aggregation unit in an MTAP the
   timestamp offset MUST be zero.  Hence, the RTP timestamp of the MTAP
   itself is identical to the earliest NALU-time.

      Informative note: The "earliest" multi-time aggregation unit is
      the one that would have the smallest extended RTP timestamp among
      all the aggregation units of an MTAP if the aggregation units were
      encapsulated in single NAL unit packets.  An extended timestamp is
      a timestamp that has more than 32 bits and is capable of counting
      the wraparound of the timestamp field, thus enabling one to
      determine the smallest value if the timestamp wraps.  Such an
      "earliest" aggregation unit may not be the first one in the order
      in which the aggregation units are encapsulated in an MTAP.  The
      "earliest" NAL unit need not be the same as the first NAL unit in
      the NAL unit decoding order either.

Top      ToC       Page 26 
   Figure 12 presents an example of an RTP packet that contains a
   multi-time aggregation packet of type MTAP16 that contains two
   multi-time aggregation units, labeled as 1 and 2 in the figure.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                          RTP Header                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |MTAP16 NAL HDR |  decoding order number base   | NALU 1 Size   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  NALU 1 Size  |  NALU 1 DOND  |       NALU 1 TS offset        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  NALU 1 HDR   |  NALU 1 DATA                                  |
      +-+-+-+-+-+-+-+-+                                               +
      :                                                               :
      +               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |               | NALU 2 SIZE                   |  NALU 2 DOND  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |       NALU 2 TS offset        |  NALU 2 HDR   |  NALU 2 DATA  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
      :                                                               :
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :...OPTIONAL RTP padding        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 12.  An RTP packet including a multi-time aggregation
                  packet of type MTAP16 and two multi-time aggregation
                  units

Top      ToC       Page 27 
   Figure 13 presents an example of an RTP packet that contains a
   multi-time aggregation packet of type MTAP24 that contains two
   multi-time aggregation units, labeled as 1 and 2 in the figure.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                          RTP Header                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |MTAP24 NAL HDR |  decoding order number base   | NALU 1 Size   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  NALU 1 Size  |  NALU 1 DOND  |       NALU 1 TS offs          |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |NALU 1 TS offs |  NALU 1 HDR   |  NALU 1 DATA                  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
      :                                                               :
      +               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |               | NALU 2 SIZE                   |  NALU 2 DOND  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |       NALU 2 TS offset                        |  NALU 2 HDR   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  NALU 2 DATA                                                  |
      :                                                               :
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :...OPTIONAL RTP padding        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 13.  An RTP packet including a multi-time aggregation
                  packet of type MTAP24 and two multi-time aggregation
                  units

5.8.  Fragmentation Units (FUs)

   This payload type allows fragmenting a NAL unit into several RTP
   packets.  Doing so on the application layer instead of relying on
   lower layer fragmentation (e.g., by IP) has the following advantages:

   o  The payload format is capable of transporting NAL units bigger
      than 64 kbytes over an IPv4 network that may be present in pre-
      recorded video, particularly in High Definition formats (there is
      a limit of the number of slices per picture, which results in a
      limit of NAL units per picture, which may result in big NAL
      units).

   o  The fragmentation mechanism allows fragmenting a single picture
      and applying generic forward error correction as described in
      section 12.5.

Top      ToC       Page 28 
   Fragmentation is defined only for a single NAL unit and not for any
   aggregation packets.  A fragment of a NAL unit consists of an integer
   number of consecutive octets of that NAL unit.  Each octet of the NAL
   unit MUST be part of exactly one fragment of that NAL unit.
   Fragments of the same NAL unit MUST be sent in consecutive order with
   ascending RTP sequence numbers (with no other RTP packets within the
   same RTP packet stream being sent between the first and last
   fragment).  Similarly, a NAL unit MUST be reassembled in RTP sequence
   number order.

   When a NAL unit is fragmented and conveyed within fragmentation units
   (FUs), it is referred to as a fragmented NAL unit.  STAPs and MTAPs
   MUST NOT be fragmented.  FUs MUST NOT be nested; i.e., an FU MUST NOT
   contain another FU.

   The RTP timestamp of an RTP packet carrying an FU is set to the NALU
   time of the fragmented NAL unit.

   Figure 14 presents the RTP payload format for FU-As.  An FU-A
   consists of a fragmentation unit indicator of one octet, a
   fragmentation unit header of one octet, and a fragmentation unit
   payload.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      | FU indicator  |   FU header   |                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
      |                                                               |
      |                         FU payload                            |
      |                                                               |
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :...OPTIONAL RTP padding        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 14.  RTP payload format for FU-A

Top      ToC       Page 29 
   Figure 15 presents the RTP payload format for FU-Bs.  An FU-B
   consists of a fragmentation unit indicator of one octet, a
   fragmentation unit header of one octet, a decoding order number (DON)
   (in network byte order), and a fragmentation unit payload.  In other
   words, the structure of FU-B is the same as the structure of FU-A,
   except for the additional DON field.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      | FU indicator  |   FU header   |               DON             |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
      |                                                               |
      |                         FU payload                            |
      |                                                               |
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :...OPTIONAL RTP padding        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 15.  RTP payload format for FU-B

   NAL unit type FU-B MUST be used in the interleaved packetization mode
   for the first fragmentation unit of a fragmented NAL unit.  NAL unit
   type FU-B MUST NOT be used in any other case.  In other words, in the
   interleaved packetization mode, each NALU that is fragmented has an
   FU-B as the first fragment, followed by one or more FU-A fragments.

   The FU indicator octet has the following format:

      +---------------+
      |0|1|2|3|4|5|6|7|
      +-+-+-+-+-+-+-+-+
      |F|NRI|  Type   |
      +---------------+

   Values equal to 28 and 29 in the Type field of the FU indicator octet
   identify an FU-A and an FU-B, respectively.  The use of the F bit is
   described in section 5.3.  The value of the NRI field MUST be set
   according to the value of the NRI field in the fragmented NAL unit.

   The FU header has the following format:

      +---------------+
      |0|1|2|3|4|5|6|7|
      +-+-+-+-+-+-+-+-+
      |S|E|R|  Type   |
      +---------------+

Top      ToC       Page 30 
   S: 1 bit
      When set to one, the Start bit indicates the start of a fragmented
      NAL unit.  When the following FU payload is not the start of a
      fragmented NAL unit payload, the Start bit is set to zero.

   E: 1 bit
      When set to one, the End bit indicates the end of a fragmented NAL
      unit, i.e., the last byte of the payload is also the last byte of
      the fragmented NAL unit.  When the following FU payload is not the
      last fragment of a fragmented NAL unit, the End bit is set to
      zero.

   R: 1 bit
      The Reserved bit MUST be equal to 0 and MUST be ignored by the
      receiver.

   Type: 5 bits
      The NAL unit payload type as defined in table 7-1 of [1].

   The value of DON in FU-Bs is selected as described in section 5.5.

      Informative note: The DON field in FU-Bs allows gateways to
      fragment NAL units to FU-Bs without organizing the incoming NAL
      units to the NAL unit decoding order.

   A fragmented NAL unit MUST NOT be transmitted in one FU; i.e., the
   Start bit and End bit MUST NOT both be set to one in the same FU
   header.

   The FU payload consists of fragments of the payload of the fragmented
   NAL unit so that if the fragmentation unit payloads of consecutive
   FUs are sequentially concatenated, the payload of the fragmented NAL
   unit can be reconstructed.  The NAL unit type octet of the fragmented
   NAL unit is not included as such in the fragmentation unit payload,
   but rather the information of the NAL unit type octet of the
   fragmented NAL unit is conveyed in F and NRI fields of the FU
   indicator octet of the fragmentation unit and in the type field of
   the FU header.  A FU payload MAY have any number of octets and MAY be
   empty.

      Informative note: Empty FUs are allowed to reduce the latency of a
      certain class of senders in nearly lossless environments.  These
      senders can be characterized in that they packetize NALU fragments
      before the NALU is completely generated and, hence, before the
      NALU size is known.  If zero-length NALU fragments were not
      allowed, the sender would have to generate at least one bit of
      data of the following fragment before the current fragment could
      be sent.  Due to the characteristics of H.264, where sometimes

Top      ToC       Page 31 
      several macroblocks occupy zero bits, this is undesirable and can
      add delay.  However, the (potential) use of zero-length NALUs
      should be carefully weighed against the increased risk of the loss
      of the NALU because of the additional packets employed for its
      transmission.

   If a fragmentation unit is lost, the receiver SHOULD discard all
   following fragmentation units in transmission order corresponding to
   the same fragmented NAL unit.

   A receiver in an endpoint or in a MANE MAY aggregate the first n-1
   fragments of a NAL unit to an (incomplete) NAL unit, even if fragment
   n of that NAL unit is not received.  In this case, the
   forbidden_zero_bit of the NAL unit MUST be set to one to indicate a
   syntax violation.



(page 31 continued on part 2)

Next RFC Part