For the purposes of the present document, the following abbreviations apply: AMR-NB (or AMR) AMR-WB ANOVA ASY BLER CDF CMR COND CN CQ CRC DCH DL DMOS DPCH DTCH Eb/No EID FER GAL GQ HM HT HSDPA/EUL IA IP ITU-T JBM LAB LM LT MAC MANOVA Log-MAP MOS NB PC PDCP PDU Pa PL plc RC PS RB RAB RCV RLC ROHC RRM RTCP RTP SYM TB size TF ToC TrCH TTI UDP UE UL UM UMD US VOIP VQ WB XMIT

The performance of default speech codecs (AMR-NB and AMR-WB) for packet switched conversational multimedia [12, 19] was characterised over DCH channels and over HSDPA/EUL radio channels. The testing over DCH channels was carried out from October 2003 until February 2004. Further subjective testing was carried out from June until October 2007 in order to characterize the performance over HSDPA/EUL radio channels. The main purpose of the latter testing was to evaluate and verify adequate performance of the AMR-NB and AMR-WB speech codecs used as defined in IMS Multimedia Telephony TS 26.114 with a specific focus on jitter buffer management.

The tests over DCH channels were separated into two phases: Phase 1 considered the default speech codecs AMR-NB [13] and AMR-WB [14] in various operating conditions. Phase 2 considered also several other codecs including ITU-T codecs G.723.1 [5], G.729 [4], G.722 [6] and G.711 [3]. In Phase 1, France Telecom R&D acted as host laboratory. The subjective testing laboratories were ARCON for the North American English language, France Telecom R&D for the French language and NTT-AT for the Japanese language. Phase 1 tests consisted of 24 test conditions both for the AMR codec (modes 6.7 and 12.2 kbit/s) and the AMR-WB codec (modes 12.65 and 15.85 kbit/s) with error conditions covering both IP packet loss of 0% and 3% and radio conditions with 10-2, 10-3 and 5 10-4 BLER (Block Error Rate). End-to-end delays of 300 and 500 ms were covered. Robust Header Compression (ROHC), an optional UMTS functionality, was included for some test cases for AMR-WB. Three types of background noise were used: car, street and cafeteria. In Phase 2, France Telecom R&D acted as host and listening laboratory. Two languages were used (French and Arabic). The following codecs were tested: AMR-NB (modes 6.7 and 12.2 kbit/s), AMR-WB (modes 12.65 and 15.85 kbit/s), ITU-T G.723.1 (mode 6.4 kbit/s), ITU-T G.729 (mode 8 kbit/s), ITU-T G.722 (mode 64 kbit/s) and ITU-T G.711 (64 kbit/s). Transmission error conditions covered IP packet loss of 0% and 3%. Siemens provided the real time air interface simulator for the Phase 1. France Telecom provided the IP core network simulator and terminal simulator used in both experiments Phase 1 and Phase 2. IPv6 was employed in the testing. (IPv6 is fully simulated over the radio interface. The CN simulator employs IPv4 but since the only impact is a marginal difference in the end-to-end delay - of the order of ~16 ìs - the use of a particular IP-version in CN part has no impact on the performance results.) These tests were the first ever conversational tests conducted in any standardization body. Performance evaluation consisted of assessment of 5 aspects: 1) voice quality, 2) difficulty of understanding words, 3) quality of interaction, 4) degree of impairments, and 5) global communication quality. A 5-category rating scale was used for each aspect. Dynastat performed the global analysis for Phases 1 and 2. The results are contained in Clause 7.

These listening-only tests characterized the performance of AMR-NB and AMR-WB speech codecs over HSDPA/EUL channels when conducting buffer adaptation to the network delay variations using a simple jitter buffer management (JBM) algorithm. The tests focused on the effect of channel errors and channel jitter to speech quality instead of the impact of overall end-to-end delay in speech conversation. The end-to-end delay impact was considered separately by conducting a delay analysis on the whole processed test material. The subjective listening-only tests were conducted in Finnish and Swedish languages at Nokia and Ericsson, respectively. The tests consisted of eight different channel conditions in clean speech and in background noise conditions. AMR-NB was tested in 12.2 and 5.9 kbit/s modes, and AMR-WB at 12.65 kbit/s. The outstanding issue was to evaluate the performance of adaptive JBM operation in HSDPA/EUL channel conditions. The applied adaptive jitter buffer was a simple implementation conducting buffer adaptation mainly during discontinuous transmission, i.e. speech pauses, and not using any time scaling operation. A non-implementable fixed jitter buffer with the full a priori knowledge on the channel characteristics was used as a reference. Although the average end-to-end delays of both adaptive and fixed jitter buffers were the same, the number and locations of jitter buffer induced frame losses were different depending on the channel conditions. The results are contained in Clause 8.A. A program of Conversation Tests was organized to evaluate the performance of AMR-NB and AMR-WB for UMTS over HSDPA/EUL. Three test labs were contracted to conduct the conversation tests and deliver raw voting data to Dynastat, the Global Analysis Lab (GAL), for processing and statistical analysis. Three conversation tests were conducted in each of three test labs. The test labs were FTRD, testing in the French language, BIT, testing in the Chinese language, and Dynastat, testing in North American English. Each of the three conversation tests involved a different speech codec:

• Exp.1 - AMR operating at 5.9k bps
• Exp.2 - AMR operating at 12.2k bps
• Exp.3 - AMR-WB operating at 12.65k bps

The experiments were conducted according to specifications contained in the ITU-T Recommendation for Conversation Testing, P.805 [20]. Alcatel-Lucent provided the network impairment simulation test-bed. The raw voting data for each test lab and each Experiment was delivered to the GAL. The GAL conducted statistical analyses on the raw voting data and the results of those analyses are contained in Clause 8.B.