illustrates the procedure diagram for 5G immersive media downlink streaming using a STAR-based UE when all essential AR/MR functions in a UE are available without an assist by an edge.
Prerequisites and assumptions:
The AR/MR Scene Manager includes immersive media rendering and scene graph handling functionalities.
The Media Player includes immersive content delivery and immersive media decoding functionalities.
The AR/MR Application in the UE is run by the user.
The STAR UE initialises AR registration (starts analysing the surroundings where a user/UE is located), it namely:
captures its surroundings via camera(s)
analyses where the device is located
registers the device into the analysed surroundings.
AR/MR Application and AR/MR Application Provider have exchanged some information, such as device capability or content configuration, for content rendering. The exchange procedures for device capability and content configuration are FFS.
AR/MR Application Provider has established a Provisioning Session and its detailed configurations has been exchanged.
AR/MR Application Provider has completed to set up ingesting immersive contents.
The scene content is ingested by the 5GMSd AS.
Service Announcement is triggered by AR/MR Application. Service Access Information including Media Client entry or a reference to the Service Access Information is provided through the M8d interface.
Desired media content is selected.
Optionally, the Service Access information is acquired or updated.
The AR/MR Application initializes the Scene Manager with the entry point (full scene description) URL.
The Media Client establishes the transport session for receiving the entry point (scene description).
The Media Client requests and receives the full scene description.
The entry point (scene description) is processed.
The AR/MR Scene Manager requests the creation of a new AR/MR session from the AR Runtime.
The AR Runtime creates a new AR/MR session.
AR Media Delivery Pipeline, steps 11~23 requests, receives and renders scenes and scene updates:
The Media Client and/or AR/MR Scene Manager notifies the necessary QoS information required to the Media Session Handler.
The Media Session Handler shares the information with the 5GMSd AF, in some cases including desired QoS information. Based on existing provisioning by the AR/MR Application Provider, the 5GMSd AF may request QoS modifications to the PDU sessions.
Steps 13~15 establish the transport sessions, receives, and process the delivery manifests:
For the required media content, the Media Client establishes the transport session(s) to acquire delivery manifest(s) information.
The Media Client requests and receives the delivery manifest(s) from the 5GMSd AS.
The Media Client processes the delivery manifest(s). It determines for example the number of needed transport sessions for media acquisition. The Media Client is expected to be able to use the delivery manifest(s) information to initialize the media pipelines for each media stream.
The AR/MR Scene Manager and Media Client configures the rendering and delivery media pipelines.
The Media Client establishes the transport session(s) to acquire the media content.
Media session loop, steps 18~23 provide the latest pose information, request, receive and render the media objects of the immersive scene:
The latest pose information is acquired by the AR/MR Scene Manager and shared to the Media Client.
The Media Client requests the immersive media data according to the delivery manifest processed, possibly taking into account pose information (e.g., viewport dependent streaming).
The Media Client receives the immersive media data and triggers the media rendering pipeline(s), including the registration of AR content into the real world accordingly.
The Media Client decodes and processes the media data. For encrypted media data, the Media Client may also perform decryption.
The Media Client passes the media data to the AR/MR Scene Manager.
The AR/MR Scene Manager renders the media, and passes the rendered media to the AR Runtime, which performs further processing such as registration of the AR content into the real world, and pose correction.