This paper deals with the resource allocation problem aimed at maximizing users’ perception of quality in wireless channels with timevarying capacity. First of all, we model the subjective qualityaware scheduling problem in the framework of Markovian decision processes. Then, given that the obtaining of the optimal solution of this model is unachievable, we propose a simple scheduling index rule with closedform expression by using a methodology based on Whittle approach. Finally, we analyze the performance of the achieved scheduling proposal in several relevant scenarios, concluding that it outperforms the most popular existing resource allocation strategies.
Undoubtedly, the use of mobile Internet applications has notably increased over the last years, which has led to the growth of the demand for wireless bandwidth. Hence, one of the fundamental challenges that network providers nowadays face is how to efficiently share radio resources among users’ traffic flows. In wireless links channel capacity evolves over time due to the intrinsic degradations of this medium, which has motivated the investigation of scheduling problems in timevarying channels.
Traditional scheduling strategies for resource allocation are oriented to objective quality parameters such as delay. Nevertheless, considering the importance and the necessity of network resource allocation for maximizing users’ subjective quality of service or quality of experience (QoE) [
Thus, motivated by the necessity of obtaining an implementable QoEaware scheduler in wireless channels with timevarying capacity, in this paper we aim at characterizing in closedform a novel channelaware scheduler for the problem of maximizing users’ perceived quality. We focus on a scenario where traffic flows arrive and depart upon service completion.
Channelaware or opportunistic schedulers give priority to users in good channel conditions. Although several channelaware strategies exist in the literature, Max Rate and Proportional Fair [
The best contribution found in the field of optimizing delay in a timevarying context is found in [
Furthermore, even though a few QoE and channelaware scheduling proposals exist [
On the other hand, it is worth mentioning the work carried out in [
Since the stochastic and dynamic resource allocation problem of subjective quality maximization in channels with timevarying capacity is analytically and computationally unfeasible for finding an optimal solution, the main objective of this work is to design a simple and tractable heuristic priority scheduling rule using analytical tools that have had a great contribution in the optimization area.
Thus, the main contribution of this paper is threefold.
Firstly, we propose a QoEaware MDP model in a timevarying channel context.
Secondly, in order to achieve a tractable solution for the aforementioned QoE and channelaware model, we focus on designing a simple heuristic index rule using Whittle approach.
Thirdly, although for many years exponential flow size distributions have been considered for traffic modelling in order to simplify the resolution of scheduling optimization problems, as a step forward, in this work we take into account size distributions that better capture the real world patterns.
The remainder of the paper is organized as follows. First of all, we characterize the QoEaware scheduling problem for the timevarying channel context in Section
We analyze the problem of maximizing average QoE in timevarying channels. Even though this study is applicable to any timevarying channel context, we focus on wireless networks. In particular, we centre on a wireless downlink data channel in a single cell system. In this way, at the beginning of each transmission time interval (TTI), the scheduler located in the base station makes decisions in order to choose a user traffic flow to transmit.
Each user
We consider a system with a fixed number of users, without arrivals of new users. This assumption simplifies the mathematical model. However, being aware of the impact of arrivals on the performance of scheduling, we will analyze the performance of scheduling strategies in the presence of arrivals.
Flows are characterized by their random size
We focus on the important class of size distributions with a decreasing hazard rate, particularly we assume Pareto distributed flow sizes. It is known that Internet traffic flow sizes are properly modelled by means of Pareto distributions [
Note that, even though we use Pareto distributed flow sizes, the results provided are valid for any size distribution with a decreasing hazard rate.
The time needed to transmit to channel all the bits from a user flow has a direct impact on user’s perceived quality. We consider that this delay is the main cause of subjective quality distortion. Nowadays, most of the Internet traffic is transported over TCP, which implements the packet recovery mechanisms to avoid application level losses. In TCPbased services, the predominant source of losses at the application is packets arriving later than their playout time. Hence, it is generally useful to provide a delaydriven QoEawareness.
This way, we quantify QoE by a delay dependent Mean Opinion Score (MOS) [
As shown in QoE studies [
We illustrate an example of the explained behavior in Figure
MOS versus delay.
The channel quality associated to a user
The objective of the scheduler is finding a policy
Delay dependency: the time that a flow remains in the system,
QoEawareness: the delaydependent MOS function that characterizes user subjective quality is known.
Channelawareness: channel quality indicator (CQI) information sent from mobile users is used as user’s instantaneous channel condition information.
Nonanticipation: similarly to most current IP systems, flow size is unknown. However, existent nonanticipating sizebased disciplines make use of flow attained service [
Preemption: we assume that the server is preemptive, that is, at every decision epoch, it is permitted to suspend the service of a flow whose transmission is unfinished.
Single service: only the transmission of a flow is allowed in each TTI.
In this section we formulate the problem described in Section
The action space,
Unfinished states: all the states in which flow transmission is uncompleted belong to this group. These states have three dimensions, denoted as
Reward states: these
Final state: the chain of states ends in the absorbing state
where
where
Thus, the dynamics of user
Figure
A part of peruser state diagram.
Besides, the optimization problem (
Nonetheless, optimally solving problem (
In this part we provide a Whittle index rule type approximate solution to problem (
Note that we are only interested in the indices of
Let us define serving set
Problem (
Due to the complexity of our model, proving the indexability of Problem (
Following Whittle index definition in [
Whittle index,
For any state
From the definition of reward and work, respectively, we have
By substituting the expressions (
Nevertheless, in order to obtain an analytically tractable Whittle index expression for (
In such situation, aimed at achieving the characteristics of the optimal policy and the subsequent active set building evolution we will use an algorithm called AdaptiveGreedy, shortly
(1)
(2)
(3)
(4)
(5)
We have performed several numerical experiments for
The Whittle index value depends on attained service, delay, and channel state. Moreover, as can be seen in graphs from Figures
For the same channel condition and the same delay level, Whittle index values are decreasing with attained service:
For the same channel condition and the same attained service level, Whittle index values are decreasing with delay:
In a better channel condition, for the same attained service level and the same delay level, the Whittle index value is higher:
Opposite to the previous channelaware work [
Dependency on channel condition probabilities: hile for channel condition
Whittle index values obtained from
Whittle index values obtained from
Once we analyzed the properties of the Whittle index, we set out to define the structure of the active set in order to obtain an analytically and computationally tractable Whittle index expression. In such a way, to derive a closedform characterization of the Whittle index of
Nonetheless, in the best channel the active set is totally specified since all the states that compute in the index are passive, because of nonexisting channel improvement and being in the future attained service equal or higher and delay higher. As shown in Proposition
The Whittle index for Problem (
It is known that for
Besides, as stated before, obtaining a computationally tractable Whittle index expression for any channel condition is not possible. In such situation, in order to achieve a tractable Whittlebased QoEaware index rule for timevarying channels, we propose a Whittlebased approximate heuristic in the next subsection.
In the following we propose an approximate solution for the QoEaware Whittle index in a timevarying channel context (
First of all, we will determine the structure of the active set. For the Whittle index computation of
On the one hand, we assume that for any
On the other hand, we assume
Then, suppose that the aforementioned
In such a way, applying the previous simplifications, we obtain the Whittle index approximation presented in Proposition
The formulation of the ASPIM index is:
If we analyze expression (
Hence, a user in its best channel state with nonnull instantaneous normalized MOS function has priority over a user which is not in its best channel condition. Thus, we summarize the proposed QoEaware Whittlebased ASPIM index rule in Definition
The ASPIM index rule consists of: at every decision slot,
serving the user in its best channel condition with nonnull instantaneous normalized MOS function with the highest value of
if there is no user in its best channel condition with nonnull instantaneous normalized MOS function, serving the user with the highest value of
So, if channel condition is not the best, the index value is the ratio of the multiplication between the actual service completion and the instantaneous normalized MOS function respect to the expected potential improvement of the completion probability.
Therefore, in the context of timevarying channels aimed at maximizing average MOS, we have proposed ASPIM scheduling algorithm, which is a simple and tractable QoEaware, channelaware, and sizebased index rule. Nevertheless, in order to verify the correct behavior of this heuristic, we analyze its performance in the subsequent section.
In this section, we evaluate the performance of the proposed ASPIM index rule (see proposal in Definition
Max rate (MR): this opportunistic policy consists in serving the user with the best channel capacity.
Proportional fair (PF): this channelaware discipline serves the user with the highest ratio between instantaneous transmission rate and current attained throughput.
Cost and attained service dependant
GM [
In case of ties, we use random tiebreaking rule.
Concerning traffic flows in the system, we use the Pareto size distribution given in (
Referring to network characteristics, we use transmission rates that depend on CQIs used in real wireless networks, which are associated with 4G modulation and coding schemes. We show the mapping between CQI values and transmission rates in Table
CQIs and corresponding transmission rates (Mbps).
CQI  0  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15 



0  4.2  6.72  8.4  11.256  16.8  21.84  25.2  26.88  33.6  44.68  50.4  53.76  67.2  75.8  80.64 
Moreover, we consider different network loads in order to analyze the behavior of scheduling strategies under different network conditions. Flow arrival rate determines network load,
Regarding QoE aspects, as typically utilized in perceived quality studies [
We have implemented the whole network environment for the scheduling of network resources in MATLAB. In relation to the performed experiments, for each scenario, for each combination of scheduling discipline and network load, we have carried out a set of 10 rounds of 10000 s length simulations. These rounds differ in the randomly pregenerated traces (input vector) of sizes and arrivals. Thus, not only the overall average MOS values will be provided but we will also include their 95% confidence intervals.
Besides, in order to guarantee that the obtained performance results are generally valid, we will analyze different simulation scenarios. We have chosen five relevant settings, which differ in QoE, size, channel, or/and cost characteristics. The parameters of these scenarios are summarized in Table
Parameter set in the experimental scenarios.
Scenario  MOS  Size  Channel 


1 


CQI = {3, 5}  1 




2 


CQI = {3, 5}  1 




3 


CQI = {3, 9}  1 




4 


CQI = {3, 5}  1 




5 


see Table 


 





We have carefully selected the scenarios used in simulations. First of all, we define a setting (Scenario 1) in which the values of all the parameters are typical or standard; the values of these parameters introduce low error in the approximations considered for our ASPIM scheduler proposal. Then, in order to evaluate worst performance cases for ASPIM, we have chosen three scenarios (Scenario 2, Scenario 3, and Scenario 4) in which the error of the approximation used for ASPIM is high due to a different factor (gradient of MOS, rate and size, resp.).
Apart from that, it is worth mentioning that for obtaining the ASPIM index rule that we have considered an isolated MDP model per user. Hence, the achieved solution is based on a unique user type or single class. However, it would be interesting to examine the proposal when there are different types of users, which is analyzed in Scenario 5. Furthermore, this last setting resembles a real 4G wireless network. Note that, even though this technology allows the simultaneous transmission of multiple flows per TTI, we assume that a single flow transmits in each TTI.
Next we describe the results we have achieved in the aforementioned scenarios.
In this first family of simulations we consider a basic or typical scenario, which takes into account the equiprobable channel case and mediumsized selfsimilar flows. Figure
In order to justify the previous performance results we now analyze delay and MOS statistics for the highest network load considered. In such a way, as regarded in Figure
Statistics in Scenario 1: delay PDF (a), MOS PDF (b), and MOS CCDF (c).
What is previously concluded about delay statistics has a direct reflection on QoE statistics. This way, as can be seen in the PDF of MOS presented in Figure
This scenario reflects the case of users with low tolerance to delay, in which the MOS function in the QoE degradation range shows a great slope. Thus, in this setting the error of the MOS approximation (
During the simplification process carried out for obtaining the ASPIM index rule in Section
Besides, it is known that for a distribution with a decreasing hazard rate such as Pareto that the gradient of completion probability increases as long as the mean size decreases, which makes the error of the
This last scenario reflects a real 4G wireless network context, but with the simplification that a single user transmits in each TTI. To that end, we use CQI traces obtained from a systemlevel radio access simulator [
Channel state probabilities in Scenario 5.
CQI  0  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15 



0.28  0.12  0.09  0.08  0.08  0.08  0.07  0.06  0.05  0.04  0.03  0.01  0.009  0.0005  0.0003  0.0002 
Under this mixture of classes and realistic channel models, in Figure
Concerning class behaviour, we show perclass results in Figures
This paper goes into detail about obtaining a scheduling algorithm aimed at maximizing users’ perception of quality in channels with timevarying capacity. Furthermore, this work considers traffic flows with nonexponential size distributions, contrary to previous approaches that assume memoryless distributions.
As first contribution, we provide a MDPbased model for the scheduling problem presented. This model combines QoEawareness, sizeawareness and channelawareness. Nevertheless, since this model can not be solved either analytically or computationally, from the proposed MDP model we derive a simple, tractable and implementable wellperforming scheduling index rule by applying a methodology based on Whittle approach.
The proposed scheduling solution still combines QoE, size and channelawareness. Our scheduling proposal gives priority to users in their best channel condition that are out of the service unavoidable range. As verified from simulation results, the proposed scheduling strategy shows suitable subjective quality performance in a wide range of scenarios, including the case of a simplified 4G network with heterogeneous users.
Therefore, the results of this work present a relevant mathematical basis for developing scheduling algorithms aimed at maximizing QoE in current and future networks with timevarying channel capacity. This way, the proposed channelaware ASPIM discipline will be useful for network operators in order to guarantee to their customers service satisfaction in timevarying wireless networks. Apart from that, the approximations and simplifications used in Whittle method are applicable to any area when the gradient of system components is considerably small among consecutive decision slots.
As future research, we will extend our work to a multiuser approach, considering the simultaneous transmission of multiple flows per TTI.
Suppose that states
Referring to work elements, using (
Analogously, for reward elements, using (
Substituting (
And therefore, for the undiscounted case, the
The authors declare that there is no conflict of interests regarding the publication of this paper.
The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007–2013) under Grant agreement 284863 (FP7 SEC GERYON).