Content for TR 26.996 Word version: 18.1.0
6.1.2 Further tests
6.1.2.1 ISAR tests at 384 and 512 kbps
6.1.2.1.1 Overview
6.1.2.1.2 Test plan
6.1.2.1.3 Test conditions
6.1.2.1.4 Requirements and Objectives
6.1.2.1.5 Test results
6.1.2.2 Overall conclusion
...
...
6.1.2 Further tests p. 19
6.1.2.1 ISAR tests at 384 and 512 kbps p. 19
6.1.2.1.1 Overview p. 19
In addition to the selection tests evaluating the performance of the single ISAR candidate solution for IVAS described above, analogous experiments were carried out with the selected ISAR solution operated at 384 and 512 kbps. These experiments were provided by a company contribution and described below. The original contribution is found for reference in the electronic attachment of this TR.
6.1.2.1.2 Test plan p. 19
The test plan for the listening experiments of the ISAR solution for IVAS, operated at 384 and 512 kbps (here referred to as low-rate (LR)) is identical with the test plan of the selection tests described in clause 6.1.1.1 with the exception that conditions with the two lower ISAR bit rates are added instead of the 768 kbps condition. The experiments were in only a single instance, i.e., without duplication by a second lab.
The experiments and corresponding input formats are listed in Table 6.1-11.
| Exp | Input format | Source material | Listening environment | Bitrates kbps | Listening Lab |
|---|---|---|---|---|---|
| BS1534-1-LR | SBA (HOA3) | Generic Audio | Headphones | IVAS: 512, CuT: 384
IVAS: 512, CuT: 512 | Dolby |
| BS1534-2-LR | Multi-channel 7.1+4 | Generic Audio | Headphones | IVAS: 512, CuT: 384
IVAS: 512, CuT: 512 | Dolby |
| BS1534-3-LR | Objects (ISM-4) | Generic Audio | Headphones | IVAS: 512, CuT: 384
IVAS: 512, CuT: 512 | Dolby |
| BS1534-4-LR | MASA (2 TC) | Generic Audio | Headphones | IVAS: 512, CuT: 384
IVAS: 512, CuT: 512 | Dolby |
6.1.2.1.3 Test conditions p. 19
A description of the test conditions of all experiments is given in Table 6.1-12.
| Condition | Description |
|---|---|
| c01 (REF) | Hidden reference: Native coding system (IVAS@512kbps rendered to post renderer pose) |
| c02 (LP7) | LP7 anchor: Hidden reference, 7Khz LP filtered |
| c03 (0DOF) | 0-DOF native transcoding reference (IVAS@512kbps binaurally rendered to pre-renderer pose, IVAS stereo coded@256kbps) |
| c04 (CuT1) | 3-DOF system 1 under test (IVAS@512kbps split-rendered with ISAR operating at 512kbps) |
| c05 (CuT2) | 3-DOF system 2 under test (IVAS@512kbps split-rendered with ISAR operating at 384kbps) |
6.1.2.1.4 Requirements and Objectives p. 20
See clause 6.1.1.3.
6.1.2.1.5 Test results p. 20
6.1.2.1.5.1 Experiment BS1534-1-LR: SBA (HOA3) p. 20
Provided below is the result plot and Tables with statistical analysis results for the BS1534-1-LR experiment.
| Mean Diff. (c03 - c04) | Stdev Diff. | SEMD | t | Prob. | ToR |
|---|---|---|---|---|---|
| -18.3021 | 14.2564 | 1.4550 | -12.5784 | 1.0000 | Pass |
| Mean Diff. (c03 - c05) | Stdev Diff. | SEMD | t | Prob. | ToR |
|---|---|---|---|---|---|
| -18.6354 | 14.3455 | 1.4641 | -12.7280 | 1.0000 | Pass |
6.1.2.1.5.2 Experiment BS1534-2-LR: Multi-Channel 7.1.4 p. 21
Provided below is the result plot and Tables with statistical analysis results for the BS1534-2-LR experiment.
Figure 6.1-10: Results of BS1534-2-LR test for Multi-Channel 7.1.4 input audio
(⇒ copy of original 3GPP image)
(⇒ copy of original 3GPP image)
| Mean Diff. (c03 - c04) | Stdev Diff. | SEMD | t | Prob. | ToR |
|---|---|---|---|---|---|
| -18.4479 | 4.4506 | 0.4542 | -40.6130 | 1.0000 | Pass |
| Mean Diff. (c03 - c05) | Stdev Diff. | SEMD | t | Prob. | ToR |
|---|---|---|---|---|---|
| -17.3542 | 5.0304 | 0.5134 | -33.8016 | 1.0000 | Pass |
6.1.2.1.5.3 Experiment BS1534-3-LR: ISM-4 p. 22
Provided below is the result plot and Tables with statistical analysis results for the BS1534-3-LR experiment.
| Mean Diff. (c03 - c04) | Stdev Diff. | SEMD | t | Prob. | ToR |
|---|---|---|---|---|---|
| -15.0938 | 10.4419 | 1.0657 | -14.1629 | 1.0000 | Pass |
| Mean Diff. (c03 - c05) | Stdev Diff. | SEMD | t | Prob. | ToR |
|---|---|---|---|---|---|
| -13.6771 | 9.9202 | 1.0125 | -13.5085 | 1.0000 | Pass |
6.1.2.1.5.4 Experiment BS1534-4-LR: MASA p. 23
Provided below is the result plot and Tables with statistical analysis results for the BS1534-4-LR experiment.
| Mean Diff. (c03 - c04) | Stdev Diff. | SEMD | t | Prob. | ToR |
|---|---|---|---|---|---|
| -13.0104 | 5.7647 | 0.5884 | -22.1132 | 1.0000 | Pass |
| Mean Diff. (c03 - c05) | Stdev Diff. | SEMD | t | Prob. | ToR |
|---|---|---|---|---|---|
| -11.8958 | 5.6819 | 0.5799 | -20.5134 | 1.0000 | Pass |
6.1.2.2 Overall conclusion p. 23
Conclusion of the low-rate experiments is that the ISAR solution for IVAS, when operated at 384 and 512 kbps, respectively, still clearly meets the requirement that it shall be no worse than the 0-DOF transcoding reference system. This is true across all tested immersive input audio formats, SBA, Multi-Channel 7.1.4, ISM-4 and MASA. It can generally be observed that the achievable quality is even clearly better whereby a quality level in the 'excellent' range close to the quality of the native IVAS coding reference system is achieved. In contrast, the 0-DOF transcoding alternative offers substantially lower quality.
The objective to provide a quality level as close as possible to the native coding reference system is met in the sense that the quality score of the split rendering system is in the high 'excellent' range, which indicates only very minor audible differences.
Despite the lower bit rates, this conclusion is the same as for the highest tested bit rate of 784 kbps.
