UE-Initiated Cell Reselection Game for Cell Load Balancing in a Wireless Network

,


Introduction
Over a few decades, wireless data traffic has been increased exponentially.To cope with the traffic demands, various efforts have been made in many directions to increase network capacity [1].For example, MIMO is adopted [2] in a based station and user equipment (UE), small cells are deployed [3], heterogeneous wireless networks are integrated [4], and new spectrums are secured [5].
Among those, increasing cell density is an intuitive way to increase spectral efficiency.Operators deploy base stations (i.e., cells) and configure their capacities based on the expected traffic demands in an area.However, since the traffic demands vary widely in time and space, a mismatch between the deployed resources and the actual traffic demands happens frequently over time.Therefore, balancing loads among cells become one of the important issues in a wireless network to increase resource efficiency.
In a wireless network, cell loads can be controlled at two different phases.The first phase is when a UE accesses a wireless network.A UE needs to be associated with a cell to receive a service from a network.Unlike conventional SINRbased association, cell loads can be balanced if a UE knows the loads of neighboring cells and selects the least loaded cell.The load-aware cell association is a mapping problem that finds pairs of cells and UEs optimizing system performance index [6,7].However, the channel condition between a cell and a UE is dynamic, and activities of UEs are also dynamic.Therefore, load unbalance problem occurs even though UEs select serving cells based on the cell loads when they first access a network.Therefore, at the second phase, UEs served by an overloaded cell needs to be transferred to underloaded cells to balance loads among cells.In this paper, we focus on the second phase of cell load balancing.
In [8,9], an overloaded cell borrows resources from lightly loaded neighboring cells to resolve cell load unbalance problem.However, channel borrowing method limits the amount of radio resource that can be reused because of the cochannel interference problem.
Basically, to balance loads among cells, some UEs in a congested cell must change their serving cells while the other UEs in the same cell remain in the cell.To achieve this goal, transmission power control methods are proposed, which adjust transmit power of a set of cells according to the loads of the cells [10,11].An overloaded cell decreases its coverage by reducing transmit power while underloaded neighboring cells increase their transmit power to expand cell coverage.However, since transmit power of a cell controls the physical cell boundary, coverage holes may occur if the transmit powers of relevant cells are not adjusted precisely.On the other hand, mobility load balancing (MLB) is proposed to control the logical cell boundary by manipulating a parameter called cell individual offset (CIO) [12].The CIO determines the coverage areas between a pair of cells.If we denote CIO between a cell  and a cell  by CIO , , by reducing CIO , , the coverage of a cell  is reduced while that of cell  increases correspondingly.Various methods are proposed to adjust CIOs between a congested cell and its neighboring cells [13,14].
MLB methods proposed in the literature are driven by a system.In other words, cells or a system controller determines the time to redistribute loads among cells and UEs that need to change serving cells.Generally, cells and a controller exchange necessary information periodically to achieve load balancing.UEs passively follow the decisions made by cells or a controller in a system.However, as the service quality that users expect from a wireless network increases, users play more active role in a cell reselection process.If a cell is congested, the service level provided to UEs associated with the cell decreases.Since there are a few available cells around a user, a user changes its serving cell if the quality of experience (QoE) perceived in the current serving cell is unsatisfactory.
UEs in a congested cell reselect cells to increase their own QoEs without any coordination among the UEs in the cell.Therefore, if too many UEs change serving cells, the congested cell can be underloaded rapidly.However, since the locations of UEs in the same cell are similar, it is very likely that they select the same cell as their new serving cell.Then, the newly selected cell becomes congested right after it accommodates the UEs in congested neighboring cells, which drive UEs in the cell to change serving cells accordingly.If the process repeats itself, a system becomes unstable and the system resource efficiency decreases.Therefore, the decisions made by UEs need to be coordinated by a system to achieve system stability.Since a cell load management method should not restrict the scalability of a system, the coordination must be arranged in a distributed manner without any message exchanges among UEs.
In this paper, we design a UE-initiated cell reselection policy for cell load balancing by adopting the minority game (MG) model.The MG model guarantees that the number of players who win the game converges to a prespecified value in a macroscopic level, even though players act selfishly without knowing the actions taken by the other players [15,16].By adopting the MG model, we devise a cell reselection rule.Each UE selects one of two actions-changing its serving cell or staying at the current serving cell-selfishly to maximize its QoE according to the previous game results.A system specifies a performance goal before operating.In this paper, we use the degree of cell load balance as the system goal.The degree of cell load balance changes by the selections made by UEs; a system monitors the degree of cell load balance periodically and informs UEs whether the degree of cell load balance is acceptable by a system or not.Since the feedback information affects the decisions made by UEs, this manifests an implicit cooperation among UEs even if they decide their actions only with their local information.Through the implicit cooperation among UEs, the number of UEs choosing to change serving cells is determined so that the degree of cell load balance converges to the target value, which avoids wide fluctuation of cell loads.Since it is not necessary for each UE to know the selections made by other UEs, our cell reselection policy does not incur signaling overhead among UEs.
The rest of the paper is organized as follows.In Section 2, we review related works on cell load balancing methods.In Section 3, we detail a UE-initiated cell reselection policy.We evaluate the performance of the proposed method through simulations in Section 4. Section 5 concludes the paper.

Related Works
Various technical approaches haven been taken for cell load balancing by reselecting pairs of UEs and their serving cells.Heuristic methods are proposed in [17][18][19].In [17], a ripple algorithm is adopted to resolve load unbalance problem.The authors in [18] propose a heuristic algorithm to solve a load balancing problem in HetNets which are known to be NP-hard.In [19], dynamic hysteresis adjustment method is proposed to harmonize the handover parameter optimization and load balancing for an LTE self-organizing network.
In [20][21][22], learning methods are used to bias cells adaptively in a distributed manner.In [20], Q-learning is used to determine the best step size used for CIO adjustment between a congested cell and its neighboring cells.In [21], a reinforcement Q-learning algorithm is proposed to resolve cell congestion problem by predicting the load status of each cell.By taking a statistical learning approach, the authors in [22] propose a cell range expansion method for LTE heterogeneous networks.Stochastic geometry is also used to model and analyze a load balancing algorithm [23,24].Unlike the optimization methods that maximize the utility function for the current network configurations, stochastic geometry performs optimization over the average utility.
Game theory is applied to cell load balancing in [25][26][27].Using game models, interactive decision-making processes are devised.However, in these works, actions to be taken by other players are predicted by the perfect deductive rationality of each player.In other words, it is assumed that each player has enough information and makes decisions based on precise logical reasoning.Therefore, large signaling overhead is incurred for each player to acquire enough information.
We adopt MG to devise a cell reselection policy.However, unlike the noncooperative and cooperative game theories, MG does not assume that players acts fully rationally.Instead of a deductive reasoning, players in a MG uses inductive reasoning to anticipate the actions taken by the other players.In addition, each player in MG need not know the actions taken by other players.The only information shared by all players is a result of game which is binary information.In addition, our method differs from above cell load balance methods in that the cell reselection is made not by a system but by UEs.

Cell Reselection Game
3.1.Minority Game.The minority game model explains the phenomenon that even if each player behaves selfishly to maximize its profit, the number of winners converges to the half of the total number of players with small fluctuations around the equilibrium.A MG is played repeatedly at discrete iteration steps.Let  be the number of players participating in a MG.Generally,  is supposed to be an odd number.When a game begins, each player selects one group between two groups (say, group 0 and group 1) based only on its strategy and a past game history.Each player maintains the most recent  game results.
Before joining a game, each player has a set of strategies.Each strategy is associated with a score.The score represents the appropriateness of the strategy and is updated according to a game result.Whenever each player plays a game, it selects a strategy that has the highest score and decides a group based on the past history and the selected strategy.
Then, the players in a group whose number of players is minor win the game.A result of a game is encoded by one bit which represents the minor group and is shared among all the players so that they can update the game history and scores of strategies.A player updates each score for each strategy regardless of whether a strategy is used to choose a group or not.
Since each player does not know the groups that other players select, there is no coordination among the players when they decide on a group.However, as the game is played repeatedly, implicit cooperation among players emerges by the game result shared among all the players, which converges the number of players in a minority group to /2.In [28], the properties of the minority game are investigated analytically.The authors derive an exact solution of the model and show that not only the individual utility but also the global efficiency improves.It was shown in [29] that MG gives a near-to-optimal solution when multiple selfish agents contend for common resources.

System Model.
We consider an LTE network integrated with SDN [30,31].In Notations section, we summarize the notations used in the paper.Figure 1 shows a system model.We assume that cell reselection game is played at discrete iteration steps.
We assume that each UE  selects a cell  to be associated with based on the downlink (DL) data rate ( , ) that  can provide .If  , is larger than QoE requirement   ,  remains associated with .If we denote the transmission power of a cell  and the channel gain between  and  by   and  , respectively, SINR at  from  becomes where  0 is the power of the white Gaussian noise.The amount of resources  allocated to  depends on the type of scheduler that  uses.In this paper, we assume that all cells employ the proportional fair scheduler.Then, from [32], the number of resources called RB (Resource Block) in the LTE specifications allocated to  by  is given as where  , is the maximum number of RBs that  has and   is the set of UEs associated with .|  | represents the cardinality of the set   .Then, if we denote the bandwidth of a RB as  RB ,  , is given as Thus,  , can be increased if other UEs in a congested cell  change their serving cells or  can find a cell that can provide higher DL data rate by compensating for the decrease in SINR with the increase in  , .UEs whose  , is smaller than   take part in the cell reselection game.Since UEs cannot exchange information for cooperation, each UE must determines autonomously whether to change serving cell or not.We assume that each UE makes the decision once per each game step.
An SDN controller manages a set of cells denoted by .At the end of each game step, an SDN controller collects cell load information from each cell  in  and calculates cell load balance index.If we denote the load of a cell  at the end of the th step by   (), the cell load balance index at the end of the th step is defined as As  lb () becomes smaller, it means that the loads among cells are more balanced.
The load of a cell can be defined in a few ways [33,34].However, to focus on the cell reselection process, we use the intuitive definition of a cell load which is the RB utilization ratio.From (3), the number of RBs to satisfy  , ≥   is given as Therefore, a cell load is defined as However, we note that any other definition for a cell load can easily replace (6).We assume that time is synchronized among all UEs and an SDN controller.We also assume that the backhaul networks connecting UEs and eNBs is congestion free.

Initialization.
Before participating in a cell reselection game, each UE initializes their strategy tables.Each UE  has a set of binary actions {−1, 1}.The action "−1" encodes the determination that a UE chooses to stay at the current serving cell, while the action "1" means that a UE determines to change its serving cell.A strategy table defines a mapping between a game history and a recommended action.In other words, the th strategy table of a UE ( , ) has 2  entries.Each entry corresponds to each instance of the past  game history and specifies an action to be taken if a past game history is given.Namely, Since the size of game history is , there can be 2 2  strategy tables.Each UE  selects a set of   = { ,1 , . ..,  , , . . .,  ,|  | } strategy tables randomly from 2 2  possible strategy tables and maintains   throughout the game once it is selected.
Each strategy table  , is associated with a score  , .The initial score for each  , is randomly chosen from [0, 1] according to the uniform distribution and  , is updated every time a game result is determined.

Cell Reselection Game.
At the beginning of each game step , UEs whose  , is less than   take part in a game.A UE playing a cell reselection game chooses the strategy table that has the highest score and uses it to determine its action.In other words, the best strategy of a UE  is determined as Then,  selects an action   () according to  , * and ℎ  , where ℎ  is the game history at the beginning of the th step.If we denote the action specified in the strategy table  , when the game history is ℎ  by   ( , , ℎ  ),  selects an action as At the end of the th step, an SDN controller calculates  lb () and compares it with a threshold value   .Then, an SDN controller broadcasts a binary information () to UEs who played the game during the th step.The binary information () means  lb () ≥   if () = 1.Since the cell load balance index is higher than   , more UEs need to change their serving cells.By disseminating () = 1, an SDN controller encourages UEs to change their serving cell.On the contrary, () = 0 means  lb () <   . lb () <   implies that cell loads are balanced.Therefore, an SDN controller inspires UEs to stay by sending () = 0.
Since all systems broadcast system information periodically, the increase in signaling overhead by adding a binary feedback () to the system information is trivial.In addition, the signaling overhead of the proposed method is not related to the number of UEs playing a game.
After receiving (),  updates the history information as follows: also updates the scores of strategy tables as follows: , ( + 1) =  , () − (2 () − 1)   () , ∀ ∈   .(11) We note that all the scores for all the members in   are updated regardless of whether a strategy table  , is used to determine   () or not.The same procedure repeats whenever UEs play a game.

Performance Evaluation
In this section, through simulations, we evaluate the performance of the proposed method by comparing it with the method in which UEs select new serving cells selfishly (henceforth, we will call this method as NonCo).For simulation studies, we construct a 3-tiered cell topology using  = 37 cells.In Figure 2, we depict the simulation topology.Except the cells located at the third tier, all the cells have 6 neighboring cells.The cells are homogeneous in that they have the same system parameters.We configure the system parameters according to the recommendations by 3GPP [35].The intercell distance is set to 500 m.We assume the frequency reuse factor is one.The system bandwidth is set to 5 MHz, which is translated to 25 RBs.According to the LTE specification, the bandwidth of a RB is configured as 180 KHz.The transmit power of a cell is set to 46 dBm.The antenna height and antenna gain of each cell are set to 32 m and 15 dBi, respectively.Those of each UE are set to 1.5 m and 2 dBi, respectively.To model the path loss from a cell to a UE, we applied the model 128.1 + 37.6 log(max(,  ref )), where  is the distance between a sender and a receiver in km and  ref is the path-loss reference distance which is 0.035 km.We adopt a lognormal shadow fading model with mean zero and standard deviation of 8 dB. 0 is set to −111.45 dBm.We set the minimum coupling loss to 70 dB.Initially, we overloaded 7 cells { 0 ,  7 ,  9 ,  11 ,  13 ,  15 ,  17 } by deploying 200 UEs in each of these cells while making the other 30 cells underloaded by deploying 20 UEs in each of them.The locations of UEs in each cell are randomly selected according to the uniform distribution.We configure  = 10 and |  | = 2 for all UEs.Since the amount of UEs joining a game depends on   , we experimentally configure   according to   .For   = 256 Kbps and 512 Kbps,   is configured as 0.25 and 1.3, respectively.We investigate how the proposed method balances cell loads by varying   .
Figure 3 shows the cell load variations of initially congested cells over time and Figure 4  loads fluctuate more widely when NonCo is used than when the proposed method is used.Since UEs selfishly select new serving cells when NonCo is used, the load of a congested cell can be released rapidly.However, it is more likely that the lightly loaded cell  is overloaded after it accommodates UEs associated with its neighboring cells.Then, the DL rates of UEs in  will decrease, which makes UEs in  attempt to change serving cells.If this process repeats itself, the load of a cell oscillates over time.When the proposed method is used, even though UEs selfishly select whether to change their serving cells or not using its strategy tables, the cooperation among UEs is manifested.Therefore, the number of UEs who choose to change serving cells is controlled to make  lb () converge to   in a distributed manner without message exchanges among UEs.
Figure 5 shows the utility of a system over time.The system utility is defined as ∑ ∈ ∑ ∈  log( , ).We can observe in Figure 5 that the proposed method can increase the system utility, which means that the proposed method can use system resources more efficiently.In addition, as   becomes large, the system utility fluctuates widely when NonCo is used while the proposed method maintains the system utility at a stable level because the proposed method prevents UEs from changing serving cells unnecessarily.Accordingly, as shown in Figure 6, the proportion of UEs whose data rates are less than   becomes smaller when the proposed method is used than when NonCo is used.
To analyze the effect of the proposed method on the performance of UEs, we investigate  , and the outage probability of UE.The outage probability is defined as the probability of  , >   .From [36], the outage probability of a UE  associated with a cell  is given as follows: where  , = 2   / , − 1.
Figure 7 shows the cumulative proportion of UEs according to  , when   is 256 Kbps.The two methods can improve the initial distribution.They can decrease the proportion of UEs whose DL data rates are less than   while increasing the proportion of UEs whose DL data rates are larger than 710 Kbps.However, the proposed method improves NonCo.For example, the proportion of UEs whose DL data rates are less than 256 Kbps is 0.32 initially.NonCo decreases the proportion to 0.25 and the proposed method decreases the proportion further to 0.22.
Figure 8 shows the cumulative proportion of  , when   = 256 Kbps.We can observe that the proposed method can decrease the outage probability by the implicit cooperation among UEs competing system resources.

Conclusions
In this paper, we proposed a UE-initiated cell reselection policy for load balancing.Inspired by the minority game theory, we design a cell reselection strategy that each UE uses whenever it needs to decide whether to change its serving cell or to stay at the current serving cell.Each UE makes a decision selfishly to maximize its profit irrespective of the decisions made by other UEs.However, the minority game played among UEs drives the number of UEs who select to change serving cells to a stable state where the cell load balance index of a system converges to a prespecified value.In addition, the proposed method does not require any signaling messages among UEs except the binary feedback from an SDN controller.Therefore, the signaling overhead incurred by the proposed method is trivial.Through extensive simulation studies, we showed that the proposed method increases not only the system performance, such as the load balance level among cells and system utility, but also the performance of UEs in terms of the DL data rate and the outage probability.
Even though the minority game converges the number of players who win a game to a given value, the number of players winning a game fluctuates around a predefined target value.Therefore, as a future work, we will devise an algorithm that controls the fluctuation to guarantee a system performance index used to determine a game result.As another future work, we will extend our method to the environment where there is no entity such as an SDN controller that provides a global information so that our method works in a fully distributed manner.

Figure 3 :
Figure 3: Cell loads variations of initially congested cells over time when   = 256 Kbps and   = 0.25.

Figure 4 :Figure 5 :
Figure 4: Variation of cell load balance index over time when   = 256 Kbps and   = 0.25.

𝜑
, : SINR at a UE  from a cell   , : Th en u m b e ro fR B sa l l o c a t e dt oU E by a cell   , : Downlink data rate provided to UE  by a cell    : L o a do fac e l l  lb (): Cell load balance index at the end of the th step   : Q o Et h r e s h o l do faU E   : C e l ll o a db a l a n c et h r e s h o l d : Th es i z eo fg a m eh i s t o r y   : Set of strategy tables of a UE   , : Th eth strategy table of a UE   , : S c o r ev a l u eo f ,   ( , , ℎ  ): Action specified in  , given game history of ℎ  (): Result of game played at the th step  , : O u t a g ep r o b a b i l i t yo faU E in a cell .
Proportion of UEs whose DL rate is smaller than   .