To enable haptics enhanced media experiences, the following gaps and characteristics, based on the use-cases and relevant 3GPP services collected in this document, have been identified:
-
3GPP media services currently do not support any interoperable haptics media formats or codecs.
-
Haptic interoperable services are hindered by the current prevalence of proprietary formats.
-
A codec supporting haptics media in both parametric and PCM formats, as well as providing a parametric mezzanine format to and from which other parametric haptics formats could be converted without loss, would be required to address the use-cases.
-
Lossless compression was not identified as a need for open loop haptics enhanced media services.
-
The bandwidth required to support haptics-enhanced media services varies based on the number of haptics channels (up to 32 or 64), the haptics media format (time sampled versus parametric), and on the density of the channel. A compressed format is deemed suitable.
-
In addition, the traffic characteristics of a haptics media stream in its parametric form differs substantially from any timed sampled media (haptics, audio, video), with bursts of data being followed by silent unit periods which can be longer than few seconds.
-
The QoE of haptics-enhanced media services may be substantially impacted by the asynchronicity of haptics with other media and may trigger the end-user to turn-off haptics media rendering. Handling of asynchronicity delays in 3GPP services will need to be carefully considered.
The following technical aspects would need to be addressed to ensure the interoperability of haptics-enhanced media in 3GPP services:
-
Defining the support for a top-level haptics media type and sample entry (e.g., in TS 26.143, TS 26.244, TS 29.514).
-
Defining the support for an interoperable haptics media formats and codecs in streaming, and real-time communication architecture specifications, along with suitable constraints (e.g. number of channels, number of tracks, parametrics and/or PCM formats) in the service specifications (e.g. TS 26.264, TS 26.511) and in UE capability specifications (e.g. TS 26.119, TS 26.114).
-
For services based on split rendering architecture (e.g. TS 26.565 and TS 26.567), in addition to the support for an interoperable haptics media formats and codecs, the support for negotiating split haptic media capabilities needs to be defined and are for further study. Use of non-3GPP haptics codecs is out of scope of this document.
-
Defining the support in TS 26.119 of the scene description extensions for haptics, and the support of scene description extension for interactivity at the node level, to support haptics media integration within a Scene Description.
-
Defining the support for haptics media packaging in the file format; in transport protocols such as DASH, RTP, and webRTC, as well as signaling and protocol configurations.
-
Assessing more precisely if the 5QIs and QoS parameters currently used in media services are sufficient to address the desired QoE in active and passive haptics media enhanced experiences.
From the collected scenarios and analysis of the identified 3GPP services the following requirements for a new haptics media codec have been identified to ensure interoperability across devices and services:
-
The support of a standardised compression format to provide haptics enhanced media experience.
-
The support of both parametric and time sampled signals.
-
The provision of a parametric mezzanine format to and from which other parametric formats may be converted without loss.
-
The support for compression and decompression of one or more haptics channels.
From the identified candidate technologies studied in this report, MPEG Haptics defined in
[7] fulfills these potential requirements.
The following requirements have also been identified to support a haptics media codec:
-
File format, DASH and RTP support for the haptics media content and stream.
-
Further evaluation on the potential need for enhanced handling of asynchronicity and multi-flow synchronisation for haptics enhanced media services containing one or more haptics channels or effects
-
Definition of QoS requirements for haptics enhanced media services.
From the identified candidate technologies studied in this document, ISO/IEC 23090-32
[25], the IETF
"RTP Payload for Haptics" [29] fulfill these requirements.