Dynamic Price Competition between a Macrocell Operator and a Small Cell Operator: A Differential Game Model

An economic model was analyzed where a new supplier implements the technology of the small cells and positions itself as an incumbent service provider. This provider performs a dynamic reuse of resources to compete with the macrocells service provider. Themodel was analyzed using game theory as a two-stage game. In the first stage, the service providers play a Stackelberg differential game where the price is the control variable, the existing provider is the leader, and the new supplier is the follower. In the second stage, users’ behavior ismodeled using an evolutionary game that allows predicting the population changeswith variable conditions. This paper contributes to the implementation of new technologies in the market of mobile communications through analysis of competition between the new small cell service providers (SSPs) and the existing service providers along with the users’ behavior of mobile communications. The result shows that users get a better service, SSP profits are guaranteed, and SSP entry improves users’ welfare and social welfare.


Introduction
Mobile communications have experienced an enormous growth and this tendency still continues as the number of connected users is higher every day.Cisco reports that the number of connected mobile devices will be around 5500 million in 2020 and that 70% of the world's population will be connected [1].The traffic of mobile data generated will increase by 800% and the bandwidth demand from users will grow accordingly.The present times show a picture where mobile communications and their associated services have become an everyday need which expresses the users' need to be connected all the time, everywhere with better and faster connections.
This growth has made mobile communications a very attractive market for providers who wish to make their way and satisfy the great demand required by users.Due to the limitations of the radio spectrum and the unavailability of licenses for new bands, new service providers should seek to implement new technologies which allow them a greater market impact.The service providers (SP) are faced with the need to confront this growth in the number of users and their bandwidth demand and this explains their constant quest for innovation which enables total connection everywhere.Despite their efforts, many users experience low reception in indoor environments.This is due to the present mobile network model, also called macrocell, whose signal will become significantly lessened by factors such as distance, climate, and obstacles.
In order to solve the issues related to the increase of mobile data and of the lessening of the signal in indoor environments, various technologies are being developed and integrated within the present model of mobile communications such as Cognitive Radio Networks and Heterogeneous Nets.This paper is focused on the solutions provided by HetNets and, more specifically, on one of its key elements: the small cells technology: micro-, pico-, and femtocells [2].This technology has been developed and deployed over the latest years making use of small stations connected to the Internet able to capture the signal of users and route the calls towards the mobile network [3], thus achieving significant improvements in data speed, availability, and coverage [4,5].The integration of this technology is feasible from a technical point of view as far as it incorporates improvements to the network, but significant challenges are still pending for its full deployment.One of them is a feasibility study which 2 Wireless Communications and Mobile Computing makes it attractive for service providers in economic terms by considering the necessary improvements in infrastructures [6,7] and the added value compared with the rest of providers who would not implement this technology [8].
The limitations and challenges to the development success of a Hetnet are discussed in [9], where a theoretical model was proposed to show the effectiveness of incorporating the HetNets into the existing network model.In [5,10] a study was carried out to find out which economic incentive would obtain a macrocells service provider (MSP) that implements the femtocells service.This model shows the feasibility of implementing the service for an existing SP but does not incorporates the entry of a new SP and the competition in the market.In addition, the behavior of users over time and the dynamic reuse of resources are not studied.Authors in [11] proposed an economic model that motivates that a small cell service provider (SSP) leases part of its resources to a MSP, and in this way the MSP increases the capacity of its networks.This model makes a dynamic control of the resources that the MSP leases, which allows using the resources of the small sell network more efficiently.The model also takes into account the evolutionary users' behavior regarding which network is connected.On the other hand, in our paper a new service provider (SP) implements the technology of small cells to compete with the MSP for the users with a dynamic price control variable, the SSP resources vary over time, and the evolutionary users' behaviors regarding subscribing the SP are analyzed.There are models which study the entry of a new SP as in [12], where the effects of the entry of an SP that uses femtocells to compete with the MSP are analyzed.The results show that all system agents improve their welfare with the implementation of femtocells technology, but as it is a static model, it does not consider the evolutionary behavior of users or the dynamic reuse of resources.Additionally, in [13] a similar model is studied where the interactions between a MSP, a SSP, and users are considered as a dynamic three-stage game, but in this model there is no reuse of existing resources.There are a lot of papers that analyze economic models that allow the integration of small cell technology in existing mobile telephony networks; in all of them it is concluded that there is an incentive for service providers to improve the quality of service and increase the capacity of the network as well as the spectral efficiency of the transmission channel.
The main contributions of the paper are as follows: (i) An economic model is developed for analyzing the implementation of the small cells technology and the effects in the market of the incorporation of a new provider offering new technologies for mobile communications.(ii) An alternative is proposed for the deployment of mobile networks, which allows increasing the density of users using small cells by reusing the excess of bandwidth of the clients of the Internet Server Provider (ISP) service.(iii) A dynamic model is analyzed which allows delving into the evolution of the users' behavior and the competition between the SPs when the resources vary.
(iv) A dynamic reuse of resources is employed in order to use the bandwidth not used by the user of the Internet service of the ISP more efficiently.
(v) It is demonstrated that the users improve the quality of service obtained due to the fact that the new technology in the market increases the resources and improves the efficiency; in addition the price competition between SPs reduces the prices charged to the users.All these things improve the users' welfare.
(vi) The analysis demonstrates the viability of the entry of the SSP.In addition, it is shown how profits increase when the resources increase and decrease when the MSP's spectral efficiency decreases.
The paper is structured as follows.The model is described in Section 2. In Section 3, the game analysis is performed.The numerical results and discussion of scenarios are shown in Section 4. Finally, Section 5 draws the conclusions.

Model Description
Two operators that provide fixed wireless service (MSP and SSP) and a set of  users are considered, as shown in Figure 1.The MSP is a conventional operator and owns a set of BS, each serving a macrocell that provides full coverage on the service area.The SSP deploys a radio access network (RAN) consisting only of small cells reusing the resources of an Internet Service Provider (ISP) to provide the mobile service.The coverage areas of the small cells are disjoint, included in the service area of the MSP and covering only a fraction of the latter.In the sequel, to simplify notation, it is considered without any loss of generality that the RAN of the MSP is composed of a single macrocell.Both SPs compete to serve the users inside the small cells.
The bandwidth available to the MSP is denoted by   ().The SSP deploys a total of  small cells.It refers to the th small cell as   and the available bandwidth is obtained by reusing the resources of the ISP to provide the service in the following way: given that the ISP can offer a bandwidth of   and the clients of the Internet service only uses   (1 −   ()), the remaining bandwidth can be used to provide the service of mobile communications, that is,    () =     (), where   () is the bandwidth fraction available shown in Figure 2. The area of the macrocell not overlapped with the coverage area of any small cell is referred to as  0 , where   0 = 0.While the MSP holds a license to exploit a spectrum band, the SSP does not hold such a license, but a generic authorization for providing wireless communications services.

Users.
Assuming that there are  users distributed thought the coverage area of the MSP.Users make their subscription decision according to the expected utility and independently from one another.To determine with which SP the users subscribe, the user's utility proposed in [14] is used which integrates the following: (i) The perceived rate reflects the fact that the higher rate the user is allocated, the greater the utility is; it is obtained in the same way as in [13][14][15], where the perceived rate depends directly on the perceived spectral efficiency and on the amount of bandwidth subscribed, in such a way that the perceived rate by the MSP in   is       , where     is the perceived spectral efficiency by a user  from a MSPs' BS, normalized in [0, 1], and the perceived rate by the SSP is       .However, given that the area of the small cells is relatively small, we can assume that     = max(   ) = 1, and therefore, the perceived rate by the SSP can be simplified to   ∀  .

Wireless Communications and Mobile Computing
(ii) The amount of bandwidth subscribed by each user with the MSP and SSP is   () and   (), respectively.
(iii) The payment for the (maximum) achievable rate  affects the utility through a negative exponential function (i.e.,  −⋅ ).This is a similar effect to the one achieved by a quasilinear utility and a budget constraint, where the payment for the rate is linear (i.e., − ⋅ ).Although the latter is a more common model in network economics, it is argued that our proposal reflects more realistically how the spectrum scarcity faced by a service provider is translated to the user.The payment made by the users for the amount of bandwidth subscribed per time unit with the MSP and SSP is   ()  () and   ()  (), respectively.
The utility of a user that subscribes to the MSP or to the SSP is, respectively, Given that we consider rational users, they subscribe the bandwidth that maximizes their utility: therefore the user's utility given that they make an optimal decision of bandwidth is where  *   and  *  denote the maximum utility of the users that are in   and subscribe to the MSP and SSP, respectively.Lastly, the utility perceived by the users who do not subscribe the service is  *  = 0, which is consistent with a user subscribing zero bandwidth.
We define    () and    () as the population ratio that subscribes to the MSP and SSP, respectively, in the small cell   , so the number of users subscribing to MSP in   is   ()   () and the SSP is   ()   (), where   are the users in   ,  = ∑    .In addition, the bandwidth demanded cannot be greater than the bandwidth available, where the bandwidth demanded to the MSP and the SSP is, respectively, The profits of SPs over a time horizon T are defined as where  − is the discount rate, which influences in the future payments [16,17].The SPs compete against each other to determine the dynamic control strategy in equilibrium, that is,  *  () and  *  (), given the profits obtained in ( 8) and (9).

Game Analysis
The interactions between the SPs and users are analyzed using game theory [18], as a two-stage dynamic game, shown in Figure 3.In the first stage, the SPs play a Stackelberg differential game, where the control variable is the price [19,20].Each SP posts a price per nominal-data-rate unit and time unit, which is   () for the MSP and   () for the SSP, where the MSP is the leader in the price choice and the SSP in the follower.In the second stage, the users inside the small cells can subscribe to either MSP's or SSP's service and pay for a nominal data rate, which is equal to the bandwidth allocated by the SP [5].The behavior of the users that play an evolutionary game to choose which SPs they subscribe is modeled using the replicator dynamic [21,22], shown in Figure 3.The two-stage game is solved using backward induction to guarantee the perfect subgame equilibrium [18].

Stage II: Evolutionary Game.
In the second stage, the decision that users should take is to which SP they have to subscribe, knowing in advance the prices announced by the SPs (see Figure 3).An evolutionary game is proposed that allows reaching solutions in equilibrium given that players

STAGE I: Stackelberg Differential Game
The MSP announces p m (t) The leader makes his optimal choice of price, anticipating the follower's best response (p * s (p m , t)).
The follower makes his optimal choice of price according to the leader's optimal choice.(p m (t)).

STAGE II: User Evolution Game
Users choose the service provider to subscribe, we model the strategy learning of users by using replicator dynamic.  () are the users' utilities perceived for the strategies defined in (3) and (4), respectively, while  *   () = 0. (iv) Replicator dynamic: it models the evolutionary behavior of the population among its different strategies over time.The Replicator Dynamic [21,22] is defined as follows: where  ∈ S,  is the learning rate of the population, which controls the frequency of strategy adaptation for service selection and   ( the replicator dynamics within a small cells have the limitations of coverage and resources of the SPs shown in (6).This differential equation indicates how the population evolves along the time horizon and given the initial state of the population, allowing making predictions about the future behavior of the population.The population will evolve to the Evolutionary Stable Strategy (ESS), which is a stationary state where the population shares will not change [18,21].

Stage I: Stackelberg Differential Game.
In the first stage, SPs anticipate the evolutionary behavior of the population (see Figure 3); based on this, the SPs will determine the dynamic prices  *  () and  *  ().To analyze the decision making, a Stackelberg differential game with an open-loop control [19] is formulated, where the MSP is the leader and the SSP is the follower.The SSP optimal control problem is defined as The MSP optimal control problem, given the behavior of the SSP, is   (0) =  0 .

Wireless Communications and Mobile Computing
The open-loop Stackelberg differential game is presented as the two optimal control problems with restrictions described in expressions ( 12) and ( 13), represented in the Lagrange form.To solve them, the Pontryagin Maximum Principle is used [23,24], which provides us with the necessary optimization conditions for the optimal control problem given that it takes into account the effects of the choosing the price by the SPs and generating a first immediate effect through the instantaneous value of the profits of SP and a second effect on the variation of the population state.
First, for the Lagrange problem of the follower, the Hamiltonian function is defined as follows: where   () is the costate variable associated with the state in the  state in time when it moves along the optimal trajectory, where  ∈ S.
The optimal control strategy  *  () of the original problem (10) also maximizes the corresponding Hamiltonian function [23] However, (15) it is only optimal if the multiplier vector is defined to reflect the marginal impact of the state vector on the profits of the SSP where H *  is obtained from (15).In comparison with the SSP, the Hamiltonian function of the MSP takes into account the maximization of the profits of the SSP, the dynamics of the variables of the costates of the MSP and the variation of the costates.The MSP Hamilton function is defined as where   () is the costate variable associated with the state in the  state when it moves along the optimal trajectory and   is a variable associated with the effect of the variation of the population of the SSP on the MSP.Applying the Pontryagin Maximum Principle in the same way as with the SSP, the following optimization conditions are necessary to find the optimal control strategies of the original problem  ( The first step is to solve the optimal prices that the service providers announce, ( 15) and ( 18).These prices are based on the state of the population (  ()) and the costates (  ,   , and   ).The optimal prices must be replaced in the optimization conditions of the Pontryagin Maximum Principle (( 10), ( 16), (19), and ( 20)), obtaining the following system of differential equations: This problem is a two-point boundary value problem (TPBVP), where the initial variables of the state of the population ( 0 ) and the final variables of the costates of the population   () = 0 ∀ ∈  = {1, 2, . . ., , , } and   () = 0 ∀ ∈  = {1, 2, . . ., , , } are known.Solving the TPBVP [25], the optimal vectors of the states and costates are obtained along the time horizon, that is,  *  (),  *  (),  *  (), and  *  ().Replacing these optimal vectors in the prices that were obtained from solving (15) and ( 18), the prices in equilibrium announced by SPs are obtained.

Results and Discussion
In this section some results are presented in order to illustrate the capabilities of our model and analysis and to provide an insight into the system behavior.In order to quantify the viability of the model, the users' welfare (UW) and social welfare (SW) functions are used.The UW is defined as the aggregate utility of all the users obtained along the time horizon.It allows us to quantify the welfare of the entire population of users and is obtained as where      () *   () and      *   () are a the utilities that users who subscribe receive.The SW is defined as the aggregated utility of all the users and SPs and is obtained as The numerical resolution of the dynamic problem of decision making was made using the function "BVP4C" of MATLAB [26,27], which allows solving the system of differential equations of (21) given the initial state of the population and the final state of the costates.The scenario was evaluated along a time horizon of [0-100] with a jump of ℎ = 0.01 and the parameters were varied to know the effect that has the spectral efficiency, the bandwidth of the SPs, and the dynamic reuse of the resources of the SSP.The results are obtained assuming a small cell network that covers 100% of the BS area of the MSP.Given that the coverage areas of the small cells are disjoint   = ∑     .The system parameters values are those shown in Table 1. 4, 5, and 6 show the effect of the spectral efficiency obtained with the MSP by all users (  ) on the scenario, evaluating the evolutionary behavior of users ( *  (),  *  (), and  *  ()), the dynamics of prices in equilibrium ( *  () and  *  ()), the total profits of the SPs, the SW, and UW.The results were obtained for the following parameters in each of the small cells: there are   = 80 users, all the users inside the small cells perceive on average the same spectral frequency with the BS of MSP, the learning rate of the population is  = 0.68, the bandwidth available for the SSP is    =   = 10.4MHz in all small cells, and the initial state of the population is  *   (0) = 0.1,  *   (0) = 0.1, and  *   (0) = 0.8 ∀.

Effect of Spectral Efficiency. Figures
In Figures 4(a) and 4(b) the evolution of MSP and SSP prices over the time horizon for different values of   is shown and the following is observed: (i) Only the time interval [0, 10] is represented graphically, because at time 10 the population almost reaches the ESS, that is, the population is in a stationary strategy and its decisions do not change, and this makes the prices remain constant in the interval .
(ii) The lower   , the lower the price announced by the MSP because it has to compensate for the low spectral efficiency that users receive.On the other hand, the lower the spectral efficiency, the higher the price that the SSP can announce, because users perceive a better utility with him.
(iii) The prices increase with the time, because users that do not subscribe to the service ( *   (0) = 0.8) choose to subscribe to the SPs, and given that the subscribed users increase the price announced by the SPs can be higher (there are the same resources and a greater demand).
In Figures 5(a) and 5(b) the evolutionary behavior of the subscribing population is shown with the MSP and SSP, respectively.It is observed that the users always preferred the service and therefore they will evolve over time until they subscribe to a SP; besides, depending on the spectral efficiency perceived, they will prefer to subscribe with one or the other SP, in such a way that the higher   is, the more the users prefer to subscribe with the MSP.Only the time values of [0-10] are shown because at that time the population reaches or approaches the ESS; therefore, the population is in a stationary strategy.
In Figure 6 the profits from the perspective of the SPs and the users' and social welfare are shown.It is observed that the profits of the MSP increase as   increases and that the SSP's profits decrease in the same amount.The users are benefited if   increases, as shown in the UW, which also increases the SW.Additionally, it is expected that the SW, UW, and the profits of the SPs will increase if the initial state of the population of users who do not subscribe the service is lower, given that 80% of users start not to subscribe.

Effect of the Available
Bandwidth of the SSP.In Figures 7, 8, and 9 the effect that the available resources of the SSP have on the scenario is shown, evaluating the evolutionary behavior of the users, the dynamics of equilibrium prices, the total profits of the SPs, the SW, and the UW.The results were obtained for the following parameters in each of the small cells: there are   = 80 users, who perceive on average the same spectral efficiency    =   = 0.8 bits/s/Hz, the initial state of the population is  *   (0) = 0.1,  *   (0) = 0.1, and  *   (0) = 0.8 ∀, and the learning rate of the population is  = 0.68.
In Figures 7(a  (i) Only the time interval [0, 10] is represented graphically, because at time 10 the population arrives or approaches the ESS; that is, the population is in a stationary strategy and its decisions do not change, and this makes the prices remain constant in the interval [10-100].(ii) The higher   is, the lower the price the SPs announces, because the SSP has more resources to compete and this means that both SPs have to lower prices.(iii) The prices increase with the time, this is because the number of users subscribed also increases with the time, and it is needed to satisfy a greater bandwidth demand with the same resources.
In Figures 8(a) and 8(b) the evolutionary behavior of the subscribing population with the MSP and SSP, respectively, is shown.It was observed that the users always preferred the service and therefore they will evolve over time until they subscribe an SP; in addition, the higher the available resources of the SSP, the higher the users who want to subscribe with it.At  = 10 the population arrives or approaches the ESS.
In Figure 9 the profits are shown from the perspective of the SPs, and users' and social welfare are evaluated.It is observed that the profits of the SSP increase with   and that the MSP's profits diminish in the same amount.If resources in the scenario increase, the SW and UW increase.Additionally, it is expected that SW, UW, and the SP's profits increase if the initial state of the population of users that do not subscribe to the service is lower, given that 80% of users do not subscribe.

Effect of Dynamic Reuse of the Resources of SSP.
In this section, approaching the available resources of the network from a traffic study at the University of Washington [28], the bandwidth available to service the SSP at each instant of time is shown in Figure 10 [29].
In Figures 11(a) and 11(b) the equilibrium prices for the MSP and SSP, respectively, are shown, considering that the SSP's bandwidth available varies.It is observed that the SSP prices are inversely proportional to the available bandwidth and the prices are displaced 0.8 due to the spectral efficiency that users perceive.It is also observed that when the learning rate is low ( = 0.1), it evolves more smoothly; i.e., users do not learn as quickly as resources vary.
In Figures 12(a) and 12(b) the evolution of the distribution of the population for the MSP and SSP, respectively, is shown.It is observed that the population of the SSP has a direct relation to resources; that is, the higher the resources of the SSP, the higher the population that subscribes to the SSP.On the other hand, the higher the resources of the SSP, the lower the population that subscribes to the MSP.It is also observed that when the learning rate is low ( = 0.1), the distribution of the population varies more slowly; that is, the population do not learn as fast as the resources vary.
As there is a dynamic reuse of resources by the SSP, its profits will increase as the available resources increase and the user's learning rate increases, given that this will allow the fact that the decisions of the users adjust more quickly to variations of resources.Given the existence of a dynamic reuse of resources of the SSP there will be more resources in the model, which lowers the SSP's prices and increases the SW and the UW.

Conclusions
This paper proposes a business model where a service provider implements small cells technology (SSP) and competes against the existing macrocell provider or MSP.The limitations of this technology were taken into accounts such as limited availability and coverage, dynamic reutilization of resources, and the decisions of users and service providers, while taking into consideration the influence of each provider over the decisions of their competitor.
Game theory enabled us to predict the behavior of users and providers on the basis that users and providers take the decisions that suit them best and allowed us to know the effects of a new provider on the market of mobile communications, which are as follows: (i) The users get a better service due to the fact that the SSP forces the MSP to lower prices, as the SSP increases the spectrum efficiency of users and the resources available in the scenario.In addition, all users would prefer to subscribe to the service, and they will adapt their decisions to subscribe.
(ii) The SSP goes into the communication market well aware that their profits are guaranteed.This is because the SSP is offering better spectrum efficiency and has

Figure 3 :
Figure 3: Hierarchical dynamic game framework for pricing and SP selection.
) and 7(b) the behavior of the prices of the MSP and SSP are shown along the evaluated time horizon ( = 10) as a function of   and the following is observed:

Figure 4 :Figure 5 :
Figure 4: SP's prices on the time horizon as a function of   .
These demands are limited by the available bandwidth of the corresponding operator,   ()   () *  () ≤   () and   ()   () *  () ≤    ().From these it is obtained that The SPs compete in price for the users.Each SP posts a price per nominal-data-rate unit and time unit,   (),   (), ∀ ∈ T, in order to maximize its profits over the time horizon, T = [0, ].The instantaneous profits of the SP are defined as the income minus the costs in a time instant, where the income are given by the amount that users pay for all the demanded bandwidth, and the costs are assumed to be zero.The instantaneous profits of SPs are defined as   (  ,   , ) =   ()   (  ,   , ) ,   (  ,   , ) =   ()   (  ,   , ) .