Content for  TR 26.998  Word version:  18.0.0

In clause 4.2 of TR 26.928, quality experience for XR is summarized. In order to provide the feeling of presence in immersive scenes, this clause provides a summary. TR 26.928 has some focus on VR and HMDs.

In TR 26.928, some high-level statements on experience KPIs for AR are provided. To achieve Presence in Augmented Reality, seamless integration of virtual content and physical environment is required. Like in VR, the virtual content has to align with user's expectations. For truly immersive AR and in particular MR, it is expected that users cannot discern virtual objects from real objects. Also relevant for VR and AR, but in particular AR, is not only the awareness for the user for the environment. This includes, safe zone discovery, dynamic obstacle warning, geometric and semantic environment parsing, environmental lighting and world mapping. Based on updated information, Table 4.5.2-1 provides new KPIs with focus on AR and in particular glasses. For some background and additional details refer for example to [10], [11], [49], [50], and [51].

Building on top of the architectures introduced in clause 4.2 in this document as well as the latency considerations in TR 26.928, Figure 4.5.3-1 provides a summary of different latencies involved networked AR services. Based on TR 26.928 as well as Table 4.5.2-1, two relevant latency requirements for adequate user experience matter:

• motion-to-photon latency being less 20ms, but preferably even single digit latency below 10ms.
• pose-to-render-to-photon latency: as small as 50-60ms

It is important to note that the motion-to-photon latency is primarily a function of the device implementation as it is basically covered within the AR runtime. What matters and is relevant is the time used to provide the pose information from the AR runtime to the renderer and the renderer using this pose to generate the displayed media. Final pose correction to the latest pose may always be done in the AR runtime. Figure 4.5.3-1 provides different latency critical uplink and downlink operations, depending on where the rendering is done, locally, in the edge or in the cloud. If done in the edge or cloud, rendered data needs to be delivered in low-latency and high-quality over the network. The typical operations in this case include:

• pose detection in the UE
• sending the pose through a 5G uplink network to the edge of cloud.
• rendering the scene in the edge or cloud
• compressing and encrypting the rendered scene and delivering to the UE
• decrypting and decompressing the rendered scene
• composition of the scene
• applying the latest pose in the pose correction and display the immersive media.

Note that Figure 4.5.3-1 also adds buffers that are typically handled by the AR Run time, namely eye and depth as well as sound buffers.

It is ultimately relevant that in case of networking the rendering loop, the processes in the loop are executed such that the end-to-end latency requirements for the pose-to-render-to-photon latency are ensured. Clearly the "closer" the rendering happens at the AR UE, the easier it is to meet latency requirements. However, with proper support of 5G system and media functionalities, these networked AR challenges are solved. This is subject of the remaining discussion of this report. With reference to TR 26.928, other types of latencies impact the user experience, for example when used for cloud gaming, interactive scenes or in case of real-time network-based processing of sensor data. These aspects are not specific to AR but are also relevant. Some more details are provided in clause 6 for the different scenarios.