JTC 1/SC 29/WG 7 (MPEG-3DGH) has completed the development of the MPEG Haptics Coding standard at the April 2023 meeting and is pending publication as ISO/IEC 23090-31 [7]. In addition to a codec and bitstream format, this MPEG Haptic coding standard supports multiple haptic modalities and specifies a human readable JSON coded representation format (HJIF) for descriptive and quantized media formats. Haptic primitive from AHAP and IVS parametric signals can be losslessly converted to and from HJIF. The HJIF data model is a flexible hierarchical structure that can describe one or more channels, allowing for mono, stereo or multi-channel haptics media experiences, targeting one or more actuators or devices. Application areas especially targeted for the use of MPEG Haptics coding include media streaming and broadcast, immersive applications and XR services, real time communications, as well as tactile communication, the latter being out of scope of this study. During the standardization process, MPEG also worked on a reference and conformance software available in ISO/IEC 23090-33 [45] that was publicly released. An in-depth evaluation of the performances of this reference codec was conducted with both objective metrics (PSNR) and subjective tests MUlti Stimulus test with Hidden Reference and Anchor (MUSHRA) as defined in ITU-R recommendation BS.1534-3 [43] for different target bitrates and different input test streams. Three representative sets of test streams were provided by different companies and corresponded to market needs: two sets for vibrotactile signals (short effects and long effects) and one set for kinaesthetic signals (including force signals, acceleration, or movement). For a total of 43 test streams. The objectives performances reported by MPEG in [44] are given in Figure 7.1.1.1-1. On the left the PSNR is given for three configurations of the MPEG encoder CRM3.2 considering PCM input signals (C2V: vectorial encoding, C2W: wavelet encoding, C2VWR: hybrid encoding). On the right the histogram of the bit-rate is depicted for parametric transcoding of the .ivs and .ahap parametric input streams.

The transcoding of parametric input content is usually around 1-4 kbps (It consists in transcoding without loss the input parametric file to the MPEG parametric format which is then binarized). The results on PCM input data show that signals lossy encoded with the 2 kbps target bitrate present perceptible distortions with an average PSNR of 24db. Signals encoded with a 8 kbps target bitrates show some distortions, but not annoying, with an average PSNR of 39.17db. Finally signals encoded with a 16kpbs bitrate have no perceptible distortion with an average PSNR of 45.62db Thus, 8kbps is considered as a reasonable average bitrate for MPEG-Haptic encoding. Considering an average input bitrate being 128kbp (8kHz sampling for 16bits samples), it leads to a compression ratio of 16 per channel. Subjective performances, performed by three independent laboratories and two reference haptic devices, confirmed the above results. The MUSHRA score was higher than 94 (maximum is 100) for bitrates of 8kbps and higher.

Figure 7.1.1.2-1 represents the MPEG haptics codec architecture. Media formats supported by the MPEG Haptics representation and coding includes AHAP, HJIF, IVS parametric media formats, as well as the WAV time sampled media format. The synthesizer is not defined by the standard and is illustrated in this Figure to highlight how the MPEG Haptics codec can be integrated with a renderer.

MPEG Scene Description [31] is used in IBACS [23] and in [SR-MSE] via MeCAR [32]. To address use-cases in clause 5.4 and 5.5 and these 3GPP services, Haptics media may be integrated with a scene description. MPEG Scene Description provides extensions for the support of haptics media defined in [7]. Two extensions have been defined MPEG_haptic and MPEG_haptic_material. The first one associates a haptic media stream on a set of nodes and defines every haptic media object. The second describes the texture-based haptic media data when used (as a 2D map associated to an object). The trigger/action mechanism of MPEG Scene Description can be used to associate and play haptic media effects on objects and/or avatars with specified location and playback type. In particular, the relation between a node in the scene and haptic media data is established through haptic actions. Interactive haptic media feedback is produced by defining behaviors with triggers (e.g collisions, proximity, etc.) and haptic actions. For each node in a haptic action, the associated Haptic media data is defined either through a reference to an element or through a MPEG_haptic_material attached to a mesh of the node. When a haptic action is triggered, the associated haptic media data is rendered according to the properties specified in the action.

The IEEE Standard for Haptic Codecs for the Tactile Internet was developed in 2016 and approved in 2024 [30]. This standard specifies 3 different codecs (no-delay kinaesthetic codec, delay-robust kinaesthetic codec, and a tactile codec) supporting only a time-sampled media format (PCM) and no parametric media formats. Some performances of the IEEE Haptic Codecs were documented in [2] in relation to closed loop haptic systems (with a reliability of 99,999%), which are out of scope of this study. Performance of the IEEE Haptic Codecs for open loop haptic system are for further study. Further, with no support for parametric media formats, these codecs could not be used as mezzanine format nor interoperable formats with parametric haptics media source content widely used in use cases of clause 5.