Journal of Computer Networks and Communications

Volume 2013 (2013), Article ID 405858, 10 pages

http://dx.doi.org/10.1155/2013/405858

Research Article

Obtaining More Realistic Cross-Layer QoS Measurements: A VoIP over LTE Use Case

F. Javier Rivas, Almudena Díaz, and Pedro Merino

Department of Languages and Computer Science, University of Malaga, UMA, Campus Teatinos, 29071 Malaga, Spain

Received 12 December 2012; Revised 4 June 2013; Accepted 9 July 2013

Academic Editor: Maria G. Martini

Copyright © 2013 F. Javier Rivas et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

We introduce a real-time experimentation testbed in this paper which enables more realistic analysis of quality of service (QoS) in LTE networks. This testbed is envisioned for the improvement of QoS and quality of experience (QoE) through the experimentation with real devices, services, and radio configurations. Radio configurations suggested in the literature typically arise from simulations; the testbed provides a real and controlled testing environment where such configurations can be validated. The added value of this testbed goes a long way not only in the provision of more realistic results but also in the provision of QoS and QoE cross-layer measurements through the correlation of information collected at different layers: from service and IP levels to radio and protocol parameters. Analyzing the interlayer dependencies will allow us to identify optimal settings for the radio access network and service parameters. This information can be used to suggest new cross-layer optimizations to further improve quality of experience of mobile subscribers. As a use case, we examine VoIP service over LTE, which is currently an open issue.

1. Introduction

The enhancement of QoS in a sustainable manner is a critical goal for network operators as management tasks are becoming increasingly complex. Although some initial efforts have been carried out by the standardization bodies, there is still a significant gap to be covered in QoS and also in QoE optimization. Actually, current efforts towards improving QoS and QoE are typically based on estimations derived from costly drive test campaigns. Furthermore, involvement of human expertise is required to manually tune network configurations. On the other hand, most of the service and network configurations available in the literature are derived from simulations [1–6]. As is widely known, in the process of modeling communication systems to simulate them, some details may be missed, and thus, misleading results may be derived. For example, it is very common to find that the consumption of control resources is ignored when evaluating different scheduling methods. In this context, providing optimized network configurations based on measurements obtained directly from the subscribers' terminals and correlated with the information collected at the network will pave the way for a reduction of costs and more accurate tuning of network operation from the point of view of the QoS perceived by final users. Moreover, as stated by standards organizations (SDOs) or alliances with the participation of network operators such as NGMN [7], the optimization of QoS still requires “real” developments to further study the direction in which to move forward.

In this paper, we propose the use of an experimental testbed [8] implemented by our research group to carry out specific long term evolution (LTE) experiences in a real context and to extract the correlation between LTE radio configurations and QoS parameters perceived at the application level. The execution of exhaustive measurements campaigns using this testbed will enable the identification of specific performance counters, correlations between them, and use cases for QoS and QoE optimization in LTE networks.

The focus of the paper is on VoIP calls over LTE, which pose new challenges over previous technologies. In LTE, voice calls are now delivered through an all-IP network (VoIP) instead of a circuit switched one, which means that voice has to compete for bandwidth with other services provided in the network. It is vital to at least guarantee the same QoE for VoIP calls that was available in pre-LTE technologies such as global system mobile communications (GSM) and universal mobile telecommunications (UMTS). This will be required to avoid significant impact on customers, who will demand a good service in all-IP mobile networks. Due to situations like this, the testbed has been conceived to validate the performance of the network configurations and problems presented in the research literature. Specifically for VoIP service we have correlated layer 1 and layer 2 LTE radio parameters with IP performance parameters and mean opinion score (MOS) measurements base on perceptual evaluation of speech quality (PESQ) algorithm.

The organization of the paper is as follows. Section 2 introduces the necessity of obtaining performance measurements which capture the QoS and QoE as perceived by final users and the new challenges presented in the provision of voice call services in LTE. This section provides also a brief state of the art on VoIP over LTE and some noteworthy LTE configurations proposed in the literature to optimize its performance. Section 3 presents the testbed and the configuration under which the measurements were collected. Section 4 provides the results obtained during the analysis of IP performance parameters and Section 5 provides their correlation with LTE parameters. Finally, in Section 6, we present conclusions and future work.

2. Voice Calls over LTE: A Regular Data Service, the Same Quality as before

Third generation partnership project (3GPP) has standardized two solutions for the deployment of voice call service over LTE. The first is circuit-switched fallback (CSFB) [9], which implies a shift of the user equipment (UE) access from LTE to 2G/3G during a voice call. The second is VoLTE [10] which on the contrary is based on IP multimedia subsystem (IMS) and does not require the use of legacy technologies. Another alternative available in the market is voice over LTE via generic access (VoLGA) [11]. That specification has been developed by the VoLGA forum, based on the existing 3GPP Generic Access Network (GAN) standard [12]. CSFB and VoLGA provide interim solutions for early LTE deployments, while VoLTE offers a long term opportunity for mobile operators. VoLTE allows integrating voice and Internet services and delivering new multimedia services in a permanent environment. This will enable the exploitation of the potential offered by mature LTE networks. In this context, VoLTE has emerged as the preferred solution by carriers and the GSM association (GSMA) is developing a specification for delivering integrated telephony services over LTE.

Specifically, the GSMA defines in [10], the minimum mandatory set of features that a wireless device and a network should implement to support a high quality IMS-based telephony service over LTE radio access. The scope provided by GSMA is shown in Figure 1.

Figure 1: Scope of VoIP deployment over LTE by GSMA PRD IR.92 v6.0.

Standardization bodies are confident about the necessity of introducing specific LTE configurations for the deployment of VoIP service. In [10] GSMA proposes a list of LTE configurations which we aim to extend with new ones obtained and validated in the testbed proposed.

As stated in the 3GPP initiative, the multiservice forum has already demonstrated successful VoLTE calls, and also multi media telephony (MMTel) services [13]. During these tests, equipment from 19 manufacturers was used. The tests performed focused on validating the interoperability between the interfaces defined in the 3GPP technical specifications.

Network operators have also performed testings experiments for quality of service (QoS) measuring. For example, different performance metrics such as latency or throughput are evaluated for the TeliaSonera network in [14]. However, our work aims to go a step beyond and not only measure the performance of interfaces and terminals using individual metrics but also correlate all these measurements with specific LTE parameters in order to identify optimum configurations.

2.1. Some Considerations about the Transport of VoIP over LTE

As proposed by GSMA in [10] session initiation protocol (SIP) is the protocol used to register UE in the IMS server. real-time transport protocol (RTP) and user datagram protocol (UDP) are the protocols recommended to voice transport, and RTP control protocol (RTCP) to provide link aliveness information, while the media are on hold.

The most restrictive performance indicators for interactive real-time services such as VoIP are the end-to-end delay and jitter. The maximum allowed one-way delay for voice service is 300 ms as stated in [15], with a recommended value lower than 150 ms. The low latency of LTE access (20–30 ms) reduces the end-to-end delay obtained in previous cellular technologies. However, LTE radio bearers do not employ fixed delay. Instead, fast retransmissions are used to repair erroneous transmissions, and uplink and downlink transmissions are controlled by schedulers. Consequently, LTE transmissions introduce jitter, which implies that UE must implement efficient dejitter buffers. The minimum performance requirements for jitter buffer management of voice media are described in [16]. In Section 4, IP parameters will be analyzed in more detail, while in this section, we will continue with the introduction of some concepts which will identify exactly what is standardized in VoIP over LTE, what is not, and how to generate specific configurations to improve the performance of VoIP over LTE.

The objective of radio resource management (RRM) procedures in LTE is to ensure an efficient use of the resources [17]. RRM algorithms at the eNodeB involve functionalities from layer 1 to layer 3. Admission control mechanisms, QoS management, and semipersistent scheduling are deployed at layer 3, while hybrid adaptive repeat and reQuest (HARQ) management, dynamic scheduling, and link adaptation are in layer 2 and channel quality indicator (CQI) manager and power control in layer 1.

3GPP specifies RRM signaling, but the actual RRM algorithms are not provided [17]. The combination of radio bearers that a UE must support for voice over IMS profile is defined in [18] Annex B. Concretely, the voice traffic requires a guaranteed bit rate (GBR) bearer, as described in [19]. The network resources associated with the evolved packet system (EPS) bearer supporting GBR must be permanently allocated by admission control function in the eNodeB at bearer establishment. Reports from UE, including buffer status and measurements of UE’s radio environment, must be required to enable the scheduling of the GBR as described in [20]. In uplink, it is the UE’s responsibility to comply with GBR requirements.

In the following subsections, we will analyze separately those parameters and configurations which have been identified in the current state of the art as optimized LTE solutions for VoIP.

2.1.1. Quality Class Indicator

The characteristics of the bearers are signalized with a QoS class identifier (QCI). The QCI is a pointer to a more detailed set of QoS attributes, including layer 2 packet delay budget, packet loss rate, and scheduling priority. As defined in [21], QCI 1 is intended for conversational voice.

2.1.2. RLC Mode Configuration

As specified in [10], the unacknowledged mode (UM) should be configured at radio link control (RLC) layer for EPS bearers with QCI 1 to reduce traffic and latency.

2.1.3. DRX Mode

Support of LTE discontinuous reception (DRX) methods for both UE and network is mandatory to reduce power consumption on mobile devices. The idea behind DRX methods is that the terminal pauses the monitorization of control channels during some periods of time, allowing it to turn the radio off. DRX parameters can be tuned depending on radio resource control (RRC) status or service. Decisions about when the radio should be activated again can be based on QoS indicators. The simulations conducted in [1] give, for VoIP applications, a potential saving of about 60 percent.

2.1.4. Compression

In order to optimize, radio resources the UE and the network must support robust header compression (RoHC) to minimize the size of IP packets during VoIP calls [10]. As we have already said, the use of UM at RLC and reduced sequence number sizes also decrease overhead. The reduction of packet size will enable the improvement of coding efficiency which is especially important for uplink scenarios and to improve the quality of data connection in areas with poor coverage.

2.1.5. Semipersistent Scheduling

The mechanisms involved in packet scheduling at the eNode-B are three: dynamic packet scheduling, link adaptation, and hybrid adaptive repeat and request management (H-ARQ). Semipersistent scheduling significantly reduces control channel overhead for applications that require persistent radio resource allocations such as VoIP. There are many different ways in which semi-persistent scheduling can configure persistent allocations.

Simulated results obtained in [2] show that in uplink direction, semi-persistent scheduling can support higher capacity than dynamic scheduling while at the same time guaranteeing VoIP QoS requirement, but with the cost of sacrificing some statistical multiplexing gains from HARQ. As dynamic scheduling is already needed for other services, they suggest using dynamic scheduling by default for VoIP. As an exception, semi-persistent scheduling should be applied for some VoIP users only in situations where the signaling load becomes too high.

2.1.6. Admission Control

Admission control mechanisms determine whether a new evolve packet system (EPS) bearer request should be admitted by checking that QoS requirements of at least all the bearers with high priority are fulfilled. Specific rules and algorithms for admission control are not specified by 3GPP. An interesting technique is proposed by AT&T Labs Research in [3] based on an intelligent blocking algorithm (IBA) for the admission process of VoIP calls subject to individual customers' blocking objectives and which is only invoked when the total bandwidth in use is close to its engineering limit. The algorithm has also been validated with simulations.

2.1.7. Packet Bundling in Downlink

Traffic generated by voice codecs can be very bursty due to the nature of the service, that is, silences in the conversations, and the nature of the codecs, for example, traffic from the adaptive multi-rate (AMR) codec. The combination between VoIP traffic patterns with dynamic packet scheduling can lead to inefficient resource utilization that might be improved by the use of packet bundling in the downlink. Several simulation studies have been done for both universal terrestrial radio access network (UTRAN) [4] and evolved universal terrestrial radio access (E-UTRAN) [5] probing that dynamic packet bundling approaches improve VoIP services.

2.1.8. RLC Segmentation and TTI Bundling in Uplink

UE has limited transmission power, and at the edge of LTE cells, there is a high probability of obtaining an error in the transmission of VoIP packets because the device is not able to gather enough energy during one transmission time interval (TTI) (1 ms) to send the packet. In the case of unsuccessful transmission, HARQ retransmissions are required, which will imply the introduction of 8 ms delay per retransmissions. A large number of retransmissions will involve an intolerable increase in the delay for a conversational voice service and reduction of the transmission efficiency.

The conventional approach used to reduce delays and improve the coverage at the cell edge is RLC segmentation. It consists of the segmentation of service data unit (RLC SDU) and their transmission in consecutive TTIs. However, this is not an optimum solution from the point of view of the overhead introduced in the control signaling and the increase of the vulnerability of packet loss due to HARQ feedback errors. The idea behind TTI bundling is that for a given transport block a fixed number of transmissions is done in consecutive TTIs without waiting for the HARQ feedback. The eNodeB sends the corresponding HARQ feedback only when it has received the whole bundle of transmissions. This approach reduces the amount of HARQ feedback significantly [6]. The L2 header overhead is also reduced because there is a lower need for segmentation, as well as the signaling overhead required for uplink grants (i.e., resource allocations) because a single grant is required for each TTI bundle.

3. Testbed Configuration for VoIP Testing

We have composed an experimental testbed [8] with the aim of providing a realistic test scenario where previous and new radio configurations could be deployed. Additionally, it is possible to analyze their interactions and to verify cross-layer performance of Internet applications and services over LTE. Moreover, the testbed can be used to reproduce, in a controlled environment, behaviors captured in field test campaigns [8].

The testbed includes an LTE test base station from AT4 wireless which provides high performance protocol and radio capabilities behaving as an actual LTE radio access network (RAN), as shown in Figure 2. It also includes features such as emulation of channel propagation that allows modeling fading and additive white gaussian noise impairments, in addition to a high degree of configurability of the LTE stack and logging functions. The LTE test base station supports the connection of real LTE terminals and the transport of IP traffic generated by commercial applications installed on them. The non access stratum (NAS) signalling exchange is provided by a core network emulation. Although the effect of core network transportation is also important for QoS, it has not been analyzed in the present work because of the focus on RAN and will be addressed in the future. The mobile terminals also incorporate advanced monitoring software [22, 23]. Finally, the testbed includes postprocessing tools which enable the testing and identification of IP connectivity issues and LTE mismatches through the correlation of logs collected at different points as shown in Figure 2.

Figure 2: Testbed configuration.

During the experiment, VoIP calls are initiated by a commercial VoIP client running in a laptop. The laptop uses the Samsung GT-B3730 USB LTE modem connected, via a radio frequency (RF) wire, to the E2010 eNodeB emulator from AT4 wireless. The emulator is connected to the Internet and to a local Asterisk server via a proprietary data gateway. The core network is not presented in the current version of the testbed. In this work, we focus on the study of radio access interface performance from the point of view of QoS and QoE perceived at the user equipment (UE). In order to automate the establishment of the calls, the Asterisk server is configured to provide a callback service, so that this service reproduces a 30-second recording each time a call is received in a preconfigured VoIP extension. Records have been extracted from audio samples provided in the (international telegraph union telecommunication standardization sector (ITU-T) recommendation P.501 to speech quality evaluation on telephone networks. The codec used during the transmission is the G.711 because it is well known and its constant bit rate eases the initial analysis of the impact in throughput and similar metrics. Other codecs such as AMR (codec recommended for VoLTE profile in [10]) or SILK (Skype) will be addressed in future experiments.

Wireshark is used to capture the IP traffic on both sides. The emulator also provides low level EUTRAN traces that are valuable for detailed examination of behaviors of interest. The postprocessing of the results collected is carried out using a tool developed in our research group which, among other things, obtains delay, packet losses, jitter, and MOS values of the VoIP calls. perceptual evaluation of speech quality (PESQ) algorithm defined in [24] is used to calculate MOS. PESQ analyzes relative degradation between the original and the received voice signals. In order to apply the algorithm, both waveforms have to be provided as input. The source signal is directly fed to the server, but for comparison purposes, we obtain it from the generated IP traffic to isolate it from server encoding effects, whereas the received voice is reconstructed from the IP traffic recorded by Wireshark at the destination end.

Table 1 contains an example configuration deployed in the LTE test base station.

Table 1: Resource scheduling and radio frequency configuration in the LTE test base station.

Parameter
Configuration

MIMO configuration
2 × 2

Channel bandwidth
10 MHz

Reference signal power
−60 dBm/15 kHz

Noise power
−67 to −73 dBm/15 KHz

Max HARQ retransmissions
3

RLC transmission mode
UM

RLC sequence number size
5

PDCP discard policy
No discard

PDCP sequence number size
7

Peak PDSCH bandwidth
120 Kbps

Resource allocation
Periodic, 5 ms

Modulation
16QAM

MCS (mod.& coding index)
13

PHICH duration
Normal

PHICH resources
1/6

Number of PDCCH symbols
1

Specific aggregation level
2

Fading profile
EPA5

Different fading and noise propagation conditions have been applied. Multipath fading conditions are typically experienced in mobile environments as a result of the user mobility. A typical environment with low delay spread is represented with the EPA5 profile as defined in [25]. EPA stands for Extended Pedestrian A channel model, which contains 7 channel taps with an average delay spread of 45 ns and a maximum tap delay of 410 ns. The EPA5 profile has an associated maximum Doppler frequency of 5 Hz, and the associated tap delay and relative power is shown in Table 2.

Table 2: Extended Pedestrian A channel model.

Excess tap delay (ns)
Relative power (dB)

0
0.0

30
−1.0

70
−2.0

90
−3.0

110
−8.0

190
−17.2

410
−20.8

As demonstrated in the table, the signal to noise ratio (SNR) has been swept in a range of 7 to 13 dB to analyze the results under moderate packet loss conditions. As the VoIP service has real-time requirements, the RLC layer is configured to operate in unacknowledged mode (UM), that does not retransmit. However, although the RLC UM does not retransmit unconfirmed data, the hybrid automatic repeat request (HARQ) at medium access control (MAC) level provides convenient fast retransmission with incremental redundancy. Thus, even in the presence of a moderate Physical downlink shared channel (PDSCH) bLock error rate (BLER), a higher layer protocol data unit (PDU), will only be lost if the maximum number of HARQ retransmissions is reached at MAC level.

4. Analysis of IP Performance

The aim of this paper is not only to introduce a reference framework for measurements of IP services deployed over LTE but also the importance of cross-layer results in the context of the VoIP service. Using the testbed described in the previous section, a campaign of experiments have been carried out to obtain the results referred to. Also cross-layer correlations between LTE mechanism and IP performance are explained.

4.1. IP Parameters

As we have stated in previous sections, some variable bit rate codecs are expected to be used in VoIP over LTE however, in this approach we have chosen the G.711 codec to compare because it is a standard and widely studied codec with constant bit rate (CBR). G.711 codec has a 64 kbps voice bandwidth. The constant sampled rate of 20 ms and the fixed 160 bytes of the payload plus 40 bytes of IP/UDP/RTP header produce a flow with a bandwidth of 80 kbps at the IP level. At the radio access a peak PDSCH bit rate of 120 kbps has been scheduled to provide enough throughput headroom. We have analyzed the IP bandwidth of the flow received by the mobile device during 10 consecutive VoIP calls in the worst-case scenario, that is, the scenario with 7 dB of SNR. The call length is 30 seconds. Results are depicted in Figure 3(a). The instantaneous evolution of the IP bandwidth for the 10 calls is compared with the nominal 80 kbps constant bit rate generated by the source. The IP bandwidth fluctuations obtained at the destination are caused by lost and delayed packets, which cause instantaneous decrements of the received bit rate. Successful retransmissions generate bandwidth peaks one second after the decrement, because the calculation is made averaging the received packets during the last second. Although we have used the LTE same fading profile and nominal signal to noise ratio in all the calls, different instantaneous results have been obtained because of the random nature of the fading and noise generators.

(a)

(b)

(c)

(d)

Figure 3: IP performance parameters analysis. (a) Instant IP bandwidth measurements during 10 VoIP calls in an EPA5 LTE scenario with a 7 dB of SNR. (b) Packet losses per VoIP call for different levels of SNR in an EPA5 scenario. (c) Instant jitter measurements during 10 VoIP calls in a EPA5 LTE scenario with a 7 dB of SNR. (d) Mean Jitter per VoIP call for different levels of SNR.

A packet loss rate close to 0% and a jitter lower than 2 ms can provide good quality VoIP calls, even comparable with a public switched telephone network (PSTN) call. However most codecs used in the VoIP service are not tolerant of higher packet losses. For the “standard" G.711 codec or the G.729 codec, a 1% packet loss rate significantly degrades a call [26]. In Figure 3(b), we depict packet losses obtained during sessions where different SNR were configured. We can see that packet losses are higher than 1% only for VoIP call with the lower configured SNR, 7 dB.

The interarrival jitter is calculated as defined in [27] using the IP traces captured at the mobile subscriber terminal. Each RTP packet contains a timestamp which reflects the sampling instant of the first octet in the RTP data packet. The instantaneous variation of the delay is obtained by comparing the elapsed time between two received packets with the difference between their timestamp. The jitter is then derived applying a filter to the instantaneous delay variation. In Figure 3(c), we observe the temporal evolution of instant jitter during VoIP calls conducted in the scenario configured with 7 dB of SNR, while Figure 3(d) shows the mean jitter obtained in all the scenarios. This is a traditional analysis based on only IP parameters, which is very useful to characterize the performance of the service under study. However, it is not enough for the adaptation and optimization of the service to the underlying transport technology. This can be better appreciated by observing Figure 3(d). Concretely, it can be seen that despite the delay variations introduced by HARQ retransmissions, the average jitter is kept in only a few ms, although eventually the instantaneous delay may vary in the order of tens of ms. However, to obtain a better comprehension of VoIP performance over LTE, it is necessary to monitor low level parameters and correlate them with IP parameters. In the following section, we will analyze the correlations between parameters monitored at different LTE layers.

5. Cross-Layer Measurement Analysis

In this section, a further analysis of the results obtained in previous experiments is provided. Specifically, we will present a correlation of the SNR configured in the experiments with different IP and RF measured parameters. In addition, we will also depict the mapping of voice quality measurements. Known functions (linear, polynomial, exponential, and logarithmic functions) have been applied to obtain the correlation between the parameters. To retrieve the degree of correlation, the coefficient of determination has been calculated. ranges from 0 (indicating the absence of a systematic correlation) to 1 (indicating a perfect correlation).

5.1. HARQ and Packet Losses

Figure 4(a) shows the effect of the SNR on the packet loss rate. We have represented the mean value, as well as the minimum and the maximum packet loss rate to illustrate the maximum variability for a given SNR value. It must be noted that although in some points the slope seems to change, these effects may appear because of the randomness of the propagation conditions, these magnitudes require large statistical analysis and in a limited set of experiments small deviations may appear in the results.

(a)

(b)

(c)

(d)

(e)

Figure 4: Cross-layer measurements and correlations. (a) Packet Loss versus SNR. (b) Packet Loss versus PDSCH BLER. (c) MOS versus SNR. (d) PESQ MOS versus packet lost. (e) PESQ MOS versus PDSCH BLER.

We have also compared the PDSCH BLER with the IP packet loss rate. The PDSCH BLER represents the ratio of correctly acknowledged transport blocks to the total number of transmitted transport blocks. As the SNR decreases, the effect of the noise makes the PDSCH BLER increase. In Figure 4(b), we correlate the PDSCH BLER with the packet loss rate. In absence of HARQ, the PDSCH BLER should match the packet loss rate, but HARQ reduces the rate of packet losses at the cost of additional use of PDSCH resources to allocate retransmissions. As we are operating in relatively ideal conditions, the relation between the BLER and the effective bandwidth reduction is approximately linear as a single retransmission will succeed typically. In worse SNR conditions, the ratio of lost packets rate to PDSCH BLER would be even further reduced at the expense of a more noticeable impact on the bandwidth.

Other useful magnitudes are also related with SNR and parameters. The CQI is a magnitude reported by a mobile device, with a configurable periodicity, that provides an estimation of the instantaneous quality of the channel. The larger the reported CQI, the higher the coding rate (lower redundancy) that can be used for transmission. We have verified that the reported average CQI decreases consistently as the noise increases, and in future work, we will provide detailed results of the mapping of PDSCH BLER to reported CQI for different cell configurations and propagation conditions. Furthermore, we will also analyze different CQI adaptive schedulers, that will react to the instantaneous received CQI to provide appropriate resource allocations.

In general, low values have been obtained (0.7 or less) with the equations used, which exposes the complexity of the relationships between the cross-layer parameters analyzed. In future work, we will apply objective-driven simulations [28] to obtain more accurate patterns between cross-layer parameters under study.

5.2. Voice Quality Measurements

In this section, we provide the voice quality results associated to the former experiments. VoIP voice quality has been calculated using the objective PESQ algorithm standardized by ITU-T. The PESQ algorithm outcome provides a quality metric mapped to the MOS. The algorithm requires injecting a known speech signal into the system under test, and the degraded output signal is compared with the original (reference) input. Values between 4 and 4.5 represent toll quality, which is the typical quality offered by the PSTN, while values below 3.5 are often considered by some users. In Figure 4(c), we can see that an SNR of 8 dB or higher produces a an average MOS higher than 4, whereas an SNR of 7 dB results in an MOS of below 3.5 for some calls.

Although the mapping of MOS to packet losses has been analyzed in the literature, it is also represented in Figure 4(d) to verify that a linear estimation can be derived. Particularly, it can be verified that for packet losses close to 0%, the maximum quality is reported by PESQ, whose maximum output is 4.5. It must be noted that the PESQ algorithm does not consider factors such as the end to end delay that affect the subjective quality and are considered in other methods such as the E-model.

In Figure 4(e), the mapping of MOS to the PDSCH BLER is represented. Although it will depend on the cell configuration, in future work, we will consider using delay aware quality algorithms such as E-model to review the relation between MOS and PDSCH BLER, as the delay introduced by HARQ retransmissions could have also an impact on quality depending on the configuration and the end to end delay budget.

6. Conclusions

In this paper, we have provided a detailed state of the art of LTE improvements for enhanced VoIP support. It has also been identified that standardization organizations, alliances, and network operators demand more realistic measurements to improve the QoS and QoE of LTE deployments in general and VoIP over LTE in particular. To that end, we have proposed an experimental testing setup based on a real-time implementation of the LTE radio access where it is possible to test LTE protocol settings, commercial devices, and real applications.

We have also shown the results obtained by the provision of relevant reference performance measurements carried out for a specific configuration at different levels of the communication, covering LTE physical level, IP performance and voice quality evaluation. Moreover, the measurements are correlated to check the consistency of the results and extract the relationships between them. For example, the number of configured HARQ retransmissions has impact on the tradeoff between maximum delay and packet losses. These results will also complement the limited nonsimulated results currently available in the scientific literature.

During the experiments, it has been identified a noticeable variability of the results because of the use of realistic impairments such as fading and noise. Real-time test environment where the experiments can be largely repeated, as the proposed one, will definitely contribute to improve the statistical relevance of the results with shorter test times.

For future work we plan to extend the number of fading profiles, increase the parameters logged at the emulated environment, and automate the execution of the tests to increase the number of VoIP calls under study, which will improve statistical validity of the results. With these automation features, we will apply the presented methodology to validate the proposals identified in the summarized state of the art. The testbed will be also extended with a core network which will enable to evaluate the impact of the transport in the evolved packet core (EPC) and study the performance of mobility procedures such as S1- and X2-based handovers.

Acknowledgments

This work has been funded by the Government of Andalusia under Grant ITC-20111046, the Spanish Ministry of Innovation and Science under Grant IPT-2011-1034-370000 and FEDER from the European Commission.

References

C. S. Bontu and E. Illidge, “DRX mechanism for power saving in LTE,” IEEE Communications Magazine, vol. 47, no. 6, pp. 48–55, 2009.

D. Jiang, H. Wang, E. Malkamaki, and E. Tuomaala, “Principle and performance of semi-persistent scheduling for VoIP in LTE system,” in Proceedings of the International Conference on Wireless Communications, Networking and Mobile Computing (WiCOM '07), pp. 2861–2864, September 2007.

X. Mang, Y. Levy, C. Johnson, and D. Hoeflin, “Admission control for VoIP calls with heterogeneous codecs,” in Proceedings of the 18th Annual Wireless and Optical Communications Conference (WOCC '09), pp. 1–4, May 2009.

O. Fresan, T. Chen, K. Ranta-aho, and T. Ristaniemi, “Dynamic packet bundling for VoIP transmission over Rel'7 HSUPA with 10ms TTI length,” in Proceedings of the 4th IEEE International Symposium on Wireless Communication Systems (ISWCS '07), pp. 508–512, October 2007.

J. Puttonen, T. Henttonen, N. Kolehmainen, K. Aschan, M. Moisio, and P. Kela, “Voice-over-IP performance in UTRA long term evolution Downlink,” in Proceedings of the IEEE 67th Vehicular Technology Conference (VTC Spring '08), pp. 2502–2506, May 2008.

R. Susitaival and M. Meyer, “LTE coverage improvement by TTI bundling,” in Proceedings of the IEEE 69th Vehicular Technology Conference (VTC Spring '09), esp, April 2009.

NGMN, “NGMN top OPE recommendations version 1.0.21,” Tech. Rep., NGMN Alliance, 2010.

A. Diaz, P. Merino, and F. J. Rivas, “Test environment for QoS testing of VoIP over LTE,” in Proceedings of the IEEE Network Operations and Management Symposium (NOMS '12), pp. 780–794, 2012.

3GPP, “Digital cellular telecommunications system (Phase 2+), Universal Mobile Telecommunications System (UMTS), Circuit Switched (CS) fallback in Evolved Packet System (EPS), stage 2,” TS 23.272.

GSMA, “Permanent Reference Document (PRD) IP Multimedia Subsystem (IMS) profile for voice and SMS,” GSMA IR92, 2012.

V. Forum, “Voice over LTE via generic access, requirements specification, phase1,” TS 1.4.0, 2010.

ETSI, “Digital cellular telecommunications system (Phase 2+), Generic Access Network (GAN), stage 2,” TS 43.318, 2012.

M. Forum, “MSF VoLTE interoperability event 2011. Multivendor testing in global LTE and IMS Networks,” WhitePaper, 2011.

Epitiro, “LTE “Real World” Performance Study. Broadband and voice over LTE, (VoLTE) quality analysis: TeliaSonera,” Tech. Rep., Epitiro, 2011.

3GPP, “Digital cellular telecommunications system (Phase 2+), Universal Mobile Telecommunications System (UMTS), LTE, Services and service capabilities,” TS 22.105.

3GPP, “Universal Mobile Telecommunications System (UMTS); LTE; IP Multimedia Subsystem (IMS); Multimedia telephony; Media handling and interaction,” TS 26.114.

A. T. Harri Holma, LTE for UMTS: Evolution to LTEAdvanced, chapter 8, Wiley, 2011.

3GPP, “LTE; Evolved Universal Terrestrial Radio Access (E-UTRA); Radio Resource Control (RRC); Protocol specification,” TS 36.331.

3GPP, “LTE; General Packet Radio Service (GPRS) enhancements for Evolved Universal Terrestrial Radio Access Network (E-UTRAN) access,” TS 23.401.

3GPP, “LTE; Evolved Universal Terrestrial Radio Access (E-UTRA) and Evolved Universal Terrestrial Radio Access Network (E-UTRAN); Overall description; stage 2,” TS 36.300.

3GPP, “Digital cellular telecommunications system (Phase 2+); Universal Mobile Telecommunications System (UMTS); LTE; Policy and charging control architecture,” TS 23.203.

A. Díaz, P. Merino Gomez, and F. Rivas Tocado, “Mobile application profiling for connected mobile devices,” IEEE Pervasive Computing, vol. 9, no. 1, pp. 54–61, 2010.

A. Alvarez, A. Diaz, P. Merino, and F. J. Rivas, “Field measurements of mobile services with android smartphones,” in Proceedings of the IEEE Consumer Communications and Networking Conference (CCNC '12), pp. 105–109, 2012.

I. T. U. -T, “Perceptual evaluation of speech quality (PESQ): an objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs,” Recommendation P.862.

3GPP, “Evolved Universal Terrestrial Radio Access (EUTRA); User Equipment (UE) conformance specification; radio transmission and reception; part 1: conformance testing,” Tech. Rep. 36.521-1, TS.

O. Heckmann, The Competitive Internet Service Provider: Network Architecture, Interconnection, Traffic Engineering and Network Design, Wiley, 2007, Edited by, D. Hutchison.

H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, RTP: A Transport Protocol For Real-Time Applications, Internet Engineering Task Force, RFC.

A. Díaz, P. Merino, and A. Salmerón, “Obtaining models for realistic mobile network simulations using real traces,” IEEE Communications Letters, vol. 15, no. 7, pp. 782–784, 2011.