The MBMS Quality of Experience (QoE) metrics feature is optional for both MBMS streaming server and MBMS Client, and shall not disturb the MBMS service. An MBMS Server that supports the QoE metrics feature shall activate the gathering of client QoE metrics with SDP as described in
clauses 8.3.2.1 and
8.4.2 and via the reception reporting procedure as described in
clause 9.4. Alternatively, QoE activation can be done with OMA-DM as described in
clause 8.3.2.2. An MBMS Client supporting the feature shall perform the quality measurements in accordance to the measurement definitions, aggregate them into client QoE metrics and report the metrics to the specified server using the content reception reporting procedure. The way the QoE metrics are processed and made available is out of the scope of the present document.
An MBMS Client should measure the metrics at the transport layer after FEC decoding (if FEC is used), but may also do it at the application layer for better accuracy.
The measurement period for the metrics is the whole streaming duration and the measurement resolution of each reported metrics value is defined by the
"Measure-Resolution" field. The measurement period may be less than the session duration, because of late joiners or early leavers. The measurement period shall not include any voluntary event that impacts the actual play, such as pause, or any buffering or freezes/gaps caused by them.
The following metrics in
Table 8.4.2-1 shall be derived by the MBMS Client implementing QoE:
All media metrics are only applicable to at least one of audio, video, speech and timed text media types, and are not applicable to other media types such as synthetic audio, still images, bitmap graphics, vector graphics, and text.
Corruption duration, M, is the time period from the NPT time of the last good frame before the corruption, (since the NPT time for the first corrupted frame cannot always be determined) or the start of the measurement period (whichever is later)to the NPT time of the first subsequent good frame or the end of the measurement period (whichever is sooner). A corrupted frame may either be an entirely lost frame, or a media frame that has quality degradation and the decoded frame is not the same as in error-free decoding. A good frame is a
"completely received" frame X that, either:
-
it is a refresh frame (does not reference any previously decoded frames AND where none of the subsequently decoded frames reference any frames decoded prior to X); or
-
does not reference any previously decoded frames; or
-
only references previously decoded "good frames".
"Completely received" means that all the bits are received and no bit error has occurred.
Corruption duration, M, in milliseconds can be calculated according to the derivation of good frames as below:
-
A good frame can be derived by the client using the codec layer, in which case the codec layer signals the decoding of a good frame to the client. A good frame could also be derived by error tracking methods, but decoding quality evaluation methods shall not be used. An error tracking method may derive that a frame is a good frame even when it references previously decoded corrupted frames, as long as all the referenced pixels for generating the prediction signal were correctly reconstructed when decoding the reference frames. A decoding quality evaluation method may derive that a frame is a good frame even one or more pixels of the frame have not been correctly reconstructed, as long as the decoding quality is considered by the method as acceptable. Such a frame is not a good frame according to the definition above, which shall be strictly followed.
-
In the absence of information from the codec layer, a good frame should be derived according to N, where N is optionally signalled from MBMS streaming server (via SDP) to the MBMS Client and represents the maximum duration, in presentation time, between two subsequent refresh frames in milliseconds. After a corrupted frame, if all subsequent frames within N milliseconds in presentation time have been completely received, then the next frame is a good frame.
-
N is not signalled, then it defaults to ¥ (for video) or to one frame duration (for audio).
The optional parameter D is defined to indicate which of options a) and b) is in use. D is signalled from the client to the server. When D is equal to "a", option a) shall be in use, and the optional parameter T shall be present. When D is equal to "b", option b) shall be in use and the optional parameter T shall not be present.
The optional parameter N as defined in point b is used with the
"Corruption_Duration" parameter. The optional parameter T is defined to indicate whether the client uses error tracking (when T is equal to
"On") or not (when T is equal to
"Off"). T is signalled from the client to the server.
The syntax for D, N to be included in the
"att-measure-spec" (
clause 8.3.2.1) is as follows:
-
D = "D" "=" "a" / "b"
-
N = "N" "=" 1*DIGIT
In MBMS reception reporting will be done only once at the end of streaming, hence all the occurred corruption durations are summed up over each resolution period of the stream and stored in the vector TotalCorruptionDuration. The unit of this metrics is expressed in milliseconds. For each resolution duration the number of individual corruption events are summed up and stored in the vector NumberOfCorruptionEvents. These two vectors are reported by the MBMS Client as part of the reception report (
clauses 9.4.6 and
9.5.3).
Rebuffering is defined as any stall in playback time due to any involuntary event at the client side.
The syntax for the metric
"Rebuffering_Duration" for the QoE-Feedback header is as defined in
clause 8.3.2.1.
Rebuffering starts at the NPT time of the last played frame before the occurrence of the rebuffering.
In MBMS reception reporting will be done only once at the end of streaming, hence all the occurred rebuffering durations are summed up over each resolution period of the stream and stored in the vector TotalRebufferingDuration. The unit of this metrics is expressed in seconds, and can be a fractional value. The number of individual rebuffering events for each resolution duration are summed up and stored in the vector NumberOfRebufferingEvents. These two vectors are reported by the MBMS Client as part of the reception report (
clauses 9.4.6 and
9.5.3).
Initial buffering duration is the time from receiving the first RTP packet until playing starts.
The syntax for the
"Initial_Buffering_Duration" is as defined in
clause 8.3.2.1.
The metric value indicates the initial buffering duration where the unit of this metrics is expressed in seconds, and can be a fractional value. There can be only one measure and it can only take one value.
"Initial_Buffering_Duration" is a session level parameter. This value is reported by the MBMS Client as part of the reception report (
clauses 9.4.6 and
9.5.3).
The metric
"Successive_Loss" indicates the number of RTP packets lost in succession (excluding FEC packets) per media channel.
The syntax for the metrics
"Successive_Loss" is as defined in
clause 8.3.2.1.
In MBMS reception reporting will be done only once at the end of streaming, hence all the number of successively lost RTP packets are summed up over each resolution period of the stream and stored in the vector TotalNumberofSuccessivePacketLoss. The unit of this metric is expressed as an integer equal to or larger than 0. The number of individual successive packet loss events over each resolution duration are summed up and stored in the vector NumberOfSuccessiveLossEvents. The number of received packets is also summed up over each resolution duration and stored in the vector NumberOfReceivedPackets. These three vectors are reported by the MBMS Client as part of the reception report (
clauses 9.4.6 and
9.5.3).
Frame rate and frame rate deviation indicates the playback frame rate information. Frame rate deviation happens when the actual playback frame rate during a measurement period is deviated from a pre-defined value.
The actual playback frame rate is equal to the number of frames played during the resolution period divided by the time duration, in seconds, of the actual measurement. For the last measurement period in the session this time duration might be shorter than the configured measurement resolution (see
clause 8.3.2.1 for the definition of the measurement resolution).
The parameter FR that denotes the pre-defined frame rate value is used with the
"Framerate_Deviation" parameter in the
"3GPP-QoE-Metrics" attribute. The value of FR shall be set by the server. The syntax for FR to be included in the
"att-measure-spec" (
clause 8.3.2.1) is as follows:
-
FR = "FR" "=" 1*DIGIT "." 1*DIGIT
The syntax for the metrics
"Framerate" and
"Framerate_Deviation" is defined in
clause 8.3.2.1
The metric
"Framerate" indicates the actual playback frame rate. It is expressed in frames per second, and can be a fractional value..
For the Metrics-Name
"Framerate_Deviation", the value field indicates the frame rate deviation value that is equal to the pre-defined frame rate minus the actual playback frame rate. This metric is expressed in frames per second, and can be a fractional value, and can be negative.
The frame rate and the frame rate deviations for each resolution period are stored in the vectors Framerate and FramerateDeviation and the vectors are reported by the MBMS Client as part of the reception report (
clauses 9.4.6 and
9.5.3).
Jitter happens when the absolute difference between the actual playback time and the expected playback time is larger than a pre-defined value, which is 100 milliseconds. The expected time of a frame is equal to the actual playback time of the last played frame plus the difference between the NPT time of the frame and the NPT time of the last played frame.
The syntax for the metric
"Jitter_Duration" is defined in
clause 8.3.2.1.
In MBMS reception reporting will be done only once at the end of streaming, hence all the Jitter_Durations are summed up over each resolution duration and stored in the vector TotalJitterDuration. The unit of this metrics is expressed in seconds, and can be a fractional value. The number of individual events over the resolution duration are summed up and stored in the vector NumberOfJitterEvents. These two vectors are reported by the MBMS Client as part of the reception report (
clauses 9.4.6 and
9.5.3).
Content access/switch time is the time that elapses between the initiation of a content request/switch by the user and up to the time when the first packet of the content or media stream is received.
The syntax for the metric
"Content_Access_Time" is defined in
clause 8.3.2.1.
The metric value indicates the content access/switch time and the unit of this metrics is expressed in seconds, and can be a fractional value. There can be only one measure and it can only take one value.
"Content_Access_Time" is a session level parameter. This value is reported by the MBMS Client as part of the reception report (
clauses 9.4.6 and
9.5.3).
The Network_Resource identifies the cell which has been used during each measurement resolution duration. There may be many measurement resolution durations in a reception report for a session, each of which identified with a cell identity in which the measurement was performed.
The syntax for the metric
"Network_Resource" is as defined in
clause 8.3.2.1.
In GERAN and UTRAN, the cell is identified by the Cell Global Identity (as described in
TS 23.003), which is a concatenation of MCC, MNC, LAC and CI. It shall be coded as a text string as follows: Starting with the most significant bit, MCC (3 digits), MNC (2 or 3 digits depending on MCC value), LAC (4 hexadecimal digits) and CI (4 hexadecimal digits).
In E-UTRAN, the cell is identified by the E-UTRAN Cell Global Identification (ECGI) (as described in
TS 36.331) which is a concatenation of the PLMN Identifier (PLMN-Id) and the E-UTRAN Cell Identity (ECI). The PLMN identifier consists of MCC and MNC. It shall be coded as a text string as follows: starting with the most significant bit, MCC (3 digits), MNC (2 or 3 digits depending on MCC value) and ECI (7 hexadecimal digits). The reported ECGI shall be the identity of the MBMS cell (see
TS 36.300), which could be the Primary Cell (PCell), the Secondary Cell (SCell), or the configurable SCell, if Carrier Aggregation
TS 36.300 is employed in the E-UTRAN.
Only one cell shall be reported per measurement resolution duration, even if more than one cell has been used during a measurement resolution duration or if reception is done simultaneously from several cells.
The cells used for all the corresponding measurement durations are stored in the vector networkResourceCellId. If the cell identifier value in the vector for a resolution period is unchanged from the previous value, it is allowed to put the value
"=" in the vector to indicate this. The vector is reported by the MBMS Client as part of the reception report (
clauses 9.4.6 and
9.5.3).
The average codec bit rate is the bit rate used for coding
"active" media information during the measurement resolution period.
For audio media
"active" information is defined by frames containing audio. If the audio codec uses silence frames (SID-frames), these frames are not counted as
"active", and the SID-frames and the corresponding DTX time periods are excluded from the calculation. Thus for audio media the average codec bit rate can be calculated as the number of audio bits received for
"active" frames, divided by the total time, in seconds, covered by these frames. The total time covered is calculated as the number of
"active" frames times the length of each audio frame.
For non-audio media the average codec bi trate is the total number of media bits played out during the measurement resolution period, divided by the length of the playout period. The playout period length is normally equal to the length of the measurement resolution period, but if rebuffering occurs the playout period will be shorter (i.e. any rebuffering time shall be ignored when calculating the codec bit rate).
The syntax for the metric
"Average_Codec_Bitrate" is defined in
clause 8.3.2.1.
The average codec bit rate value for each measurement resolution period shall be stored in the vector AverageCodecBitrate. The unit of this metrics is expressed in kbit/s and can be a fractional value. The vector is reported by the client as part of the reception report (
clauses 9.4.6 and
9.5.3).
The codec information metrics contain details of the media codec used during the measurement resolution period. If the codec information is changed during the measurement resolution period, the codec information valid when each measurement resolution period ends shall be reported. The unit of this metric is a string value. No
"white space" characters are allowed in the string values, and shall be removed if necessary.
For audio media the codec information contains the audio codec type, represented as in an SDP offer, for instance
"AMR-WB/16000/1".
For video media, the codec information contains the video codec type, represented as in an SDP offer, for instance
"H263-2000/90000". Furthermore, the video profile and level used, as well as the image size used shall be reported. For instance
"profile=0;level=45" for the profile and level information and
"176x144" for the image size. In some cases the profile and level is reported together, for instance
"profile-level-id=42e00a". Note that the image size reported for each measurement resolution period shall be the one actually used, not the maximum size allowed by the SDP negotiation.
For timed text media, the codec information contains the text encoding, represented as in an SDP offer, for instance
"3gpp-tt/1000".
The syntax for the metric
"Codec_Info",
"Codec_ProfileLevel" and
"Codec_ImageSize" are defined in
clause 8.3.2.1.
The codec info, profile/level and codec image size value for each measurement resolution period shall be stored in the vectors CodecInfo, CodecProfileLevel and CodecImageSize respectively. If the metric values in these vectors for a measurement resolution period are unchanged from the previous values in the respective vector, it is allowed to put the value
"=" in the vector to indicate this. The CodecInfo, CodecProfileLevel and CodecImageSize vectors are reported by the client as part of the reception report (
clauses 9.4.6 and
9.5.3).
The metric
"Object_Loss" indicates the number of objects lost in a FLUTE session during a resolution period.
The syntax for the metric
"Object_Loss" is as defined in
clause 8.3.2.1.
The number of lost objects are summed up over each resolution period of the session and stored in the vector numberOfLostObjects. The unit of this metric is expressed as an integer equal to or larger than 0. The number of received objects is also summed up over each resolution duration and stored in the vector NumberOfReceivedObjects. These two vectors are reported by the MBMS Client as part of the reception report (
clauses 9.4.6 and
9.5.3).
The elements of the distribution of the metric
"Distribution_of_Symbol_Count_Underrun" are calculated by subtracting the total number of source symbols, from the number of received symbols for a failed block in a failed object. The range of values of the distribution are limited to the range of interest through top and bottom range parameters. Values greater than the top of the range are reported as the maximum value. Values lower than the bottom of the range are reported as the minimum value.
Reported values may also be grouped in bins. The size of the bins used for collecting statistics are specified through a bin size parameter. The first bin starts at the bottom of the range. The last bin must include the top of the range. Collection bins are adjacent.
The range of file sizes considered for calculating the metric can also be restricted through optional minimum, and maximum file size parameters.
The distribution is reported per measurement duration as a string list of (bin lower bound, number of occurrences) pairs, with each pair corresponding to a single entry. The bin lower bound uniquely identifies a bin by providing the lowest value of the range of each bin. The bin lower bound, and number of occurrences are both integer values. When reporting the bin lower bound and number of occurrences pairs, the following string format shall be used:
"(bin lower bound, number of occurrences)", where the parentheses represent the delimiter and the comma separates the bin lower bound (integer range of values) and number of occurrences (positive integer range of values).
When it contains entries for a single measurement duration, the vector SymbolCountUnderrun is a string vector where the entries are listed sequentially, without a space character between adjacent entries of the overall set. The set of entries is delimited by curly brackets:
"{" at the beginning and
"}" at the end. Bins with zero occurrences are omitted from the list. Values greater than the top of the range are reported within the bin containing the top of the range. Values lower than the bottom of the range are reported within the first bin containing the bottom of the range.
When SymbolCountUnderrun contains information for multiple measurement durations, it shall comprise a sequence of curly bracket delimited entries, with adjacent members of the sequence separated by one or more space characters. If the number of occurrences for all bins equals zero for a particular measurement duration within the SymbolCountUnderrun, the string
"{}" shall be used to signal that event.
The following example shows a scenario whereby the reported distribution of the symbol count underrun comprises the entries of a single measurement duration, and for which bins -9, -4 and 0 have occurrences 2, 6 and 4, respectively. The vector SymbolCountUnderrun is given as:
SymbolCountUnderrun =
"{(-9,2)(-4,6)(0,4)}"
The next example shows a scenario whereby the reported distribution comprises entries of multiple measurement durations. In the first measurement duration, bins -3, -2 and -1 have occurrences 1, 3 and 5, respectively, and there are no occurrences in the next two measurement durations. In this case, the vector SymbolCountUnderrun is given as:
SymbolCountUnderrun =
"{(-3,1)(-2,3)(-1,5)} {} {}"
The top, bottom, and bin size of the distribution range are provided through the optional parameters T, B, and S respectively. The syntax for T, B, and S to be included in the
"att-measure-spec" (see
clause 8.3.2.1) is as follows:
-
T = "T" "=" {+/-} 1*DIGIT
-
B = "B" "=" {+/-} 1*DIGIT
-
S = "S" "=" 1*DIGIT
The default value of the top of the range, in case the T parameter is omitted, is 0. The default value of the bottom of the range, in case the B parameter is omitted, is -10. The default value of the bin size is 1.
The minimum and maximum file sizes considered for calculating the metric are provided through the optional parameters Y and Z, respectively. The syntax for Y and Z to be included in the
"att-measure-spec" (see
clause 8.3.2.1) is as follows:
-
Y = "Y" "=" 1*DIGIT
-
Z = "Z" "=" 1*DIGIT
The default value of the minimum file size is 0, and the default value of the maximum file size is infinity.