Resource Scheduling and Energy Cooperation in HetNet with Cross-Layer Interference Constraints

Facing more and more severe global warming problems, renewable energy, as an alternative to traditional fossil fuels, is attracting more and more attentions due to its capability of reducing carbon emission. ,is paper considers two-tier HetNets with orthogonal frequency division multiple access (OFDMA), where the macro base station (MBS) is powered by power grid and small base stations (SBSs) have hybrid energy supplies. ,rough smart grid, SBSs can share their renewable energy with each other. We consider the problem of cross-layer interference caused by spectrum reuse, the burst of user data, and the randomness of renewable energy arrivals. ,rough energy cooperation, this paper investigates maximizing the time-average energy efficiency of SBSs. Based on user data queue and SBS energy queue, the optimal problem is decoupled into two subproblems by Lyapunov optimization: resource allocation subproblem and energy scheduling and energy cooperation subproblem. By solving two subproblems, the online solution to the optimization problem is obtained.,rough theoretical analysis and simulations, both user data queues and energy queues have an upper bound, the network is stable, and the proposed algorithm performs better than the basic algorithm without energy cooperation.


Introduction
With the dramatic increase of smart devices, a high burden will be posed on the fifth-generation (5G) wireless networks, which are expected to provide user equipment (UE) with ubiquitous service. To achieve higher capacity and better coverage, more base stations (BSs) are deployed, which means higher energy consumption. As claimed in [1], energy consumption of information and communication technology (ICT) occupies 10% of world's total energy consumption. Concerned about the environment and energy saving, operators are exploring energy harvesting technology swhich is able to harvest renewable energy source (RES) by solar panels and wind turbines.
It is widely known that, in order to meet the requirements of 5G wireless networks, the heterogeneous network (HetNet), where various small cells are densely underlaid in a macro cellular network, is a promising technique to achieve more spectrum-efficient and energy-efficient communications. HetNets significantly boost the overall spectral efficiency and energy efficiency through a full spatial resource reuse. However, the severe intralayer interference among dense macro base stations (MBSs) and cross-layer interference between small cell base stations (SBSs) and MBSs are challenging the successful commercial operations of HetNets. In recent years, a great emphasis has been given on advanced techniques that take interference control, resource allocation, and self-organization into account to enhance both the spectral efficiency and energy efficiency [2].
On the other hand, the rapid developments in energy harvesting technologies have made the dream of self-sustaining devices and BSs potentially possible. As such, energy harvesting is highly desirable for prolonging the battery life and improving the energy efficiency of networks, especially for HetNets with low-power SBSs [3,4]. Reference [5] summarized recent contributions in energy harvesting wireless communications and, reference [6] presented an overview of the past and recent developments in energy harvesting wireless communications and highlighted a number of possible future research directions. Particularly, for energy supply by RES, some works considered the solely RES powered communication networks [7][8][9]. However, owing to the dynamics of renewable energy generation and limited capacity of energy storage, sole RES may not guarantee enough power supplies for all kinds of BSs. Hence, HetNets are more likely to adopt hybrid energy supplies with both traditional power grid and RES [3][4][5], and the works in [10][11][12][13] focused on resource allocation in hybrid energy powered cellular networks, where the power grid is needed to provide reliable energy to BSs.
en it is noted that, with its ongoing paradigm shift from traditional grid to smart grid, the power grid also provides new opportunities for BSs' cost saving. Since smart grid enables both two-way information and energy flows between the grid and end users, the energy cooperation in cellular networks which allows the BSs to trade and share their unevenly harvested energy to support nonuniform wireless traffic in a cost-effective way is emerging [14]. Reference [15] studied energy group buying with load sharing, where two network operators are aggregated as a single group to make the day-ahead and real-time energy purchase and their BSs share the wireless traffic at the same time. Reference [16] proposed a hybrid energy sharing framework for cellular network, where a combination of physical power lines and energy trading with other BSs using smart grid is used. In [17], to minimize the on-grid power consumption, the BS operation and the power distribution were jointly optimized. Reference [18] presented a contemporary review of recent breakthroughs on the utilization, redistribution, trading, and planning of energy harvested in future wireless networks interoperating with smart grids. In a word, the above works in [14][15][16][17] on energy cooperation and resource allocation emphasized on energy planning and/or energy trading through smart grid, while the traffic coordination or resource allocation was performed on a large timescale.
In this work, we focus on the small-timescale resource scheduling and energy cooperation in a HetNet powered by hybrid energy. Particularly, the random renewable energy arrivals, the inherent cross-layer interference constraint, and the burst of user data in the two-tier HetNet are considered. With the objective of maximizing the time-average energy efficiency of SBSs, the transmission power allocation, the subcarrier assignment, the energy scheduling, and energy cooperation among SBSs are jointly optimized. e main contributions of this paper are summarized as follows: (1) Firstly, considering a two-tier downlink HetNet with hybrid energy supplies, we jointly optimize the resource allocation, energy scheduling, and energy cooperation with the burst of user data, the crosslayer interference constraints, and the random renewable energy arrivals.
(2) Secondly, a stochastic optimization problem that maximizes the time-average energy efficiency is modelled for tradeoff between user throughput improvement and on-grid energy consumption reduction. Leveraging Lyapunov optimization technique, an online algorithm that decides transmission power allocation, subcarrier assignment, discharging and charging of the storage devices, energy cooperation among SBSs, and on-grid energy consumption is developed after several decompositions. (3) irdly, both theoretical analysis and simulation results are presented to show the stability and superiority of the proposed online resource scheduling and energy cooperation algorithm. e rest of this paper is organized as follows. In Section 2, we illustrate the system model that contains network model, traffic model, data queue model, energy supply and demand model, and energy queue model. Section 3 shows the formulation of the stochastic optimization problem and its transformation based on Lyapunov optimization method. An online algorithm for transmission power allocation, subcarrier assignment, energy scheduling, and energy cooperation is developed in Section 4. Performance analysis is given in Section 5. Section 6 presents simulation results and discussions. Finally, Section 7 concludes the whole work.

Network Model.
We consider the downlink transmission of an OFDMA two-tier HetNet system as shown in Figure 1. MBS of the first tier provides basic coverage and SBSs of the second tier enhance the capacity of the whole network. In this system, MBS is powered by power grid, while SBSs are powered by hybrid energy supply containing both power grid and RES. SBSs are connected through smart grid and they can exchange renewable energy through smart grid, so that the unbalanced renewable energy sources can be effectively utilized to match with the uneven user data.
In the system, one MBS with M active MUEs corresponds to K SBSs, and each SBS has S k active SUEs. Here, MUEs, SBSs, and SUEs are denoted as M � 1, 2, · · · , M { } indexed by m, K � 1, 2, · · · , K { } indexed by k, and S � S 1 , S 2 , · · · , S k indexed by s k , respectively. All SBSs only provide service to the preregistered SUEs as SBSs have a closed access. We assume that all SBSs are connected to the MBS via high speed backhaul with negligible delay. To match with the real-world scenario of suburban areas, we consider sparse deployed small cells. So we could ignore the co-tier interference between SBSs.
As wireless channels and renewable energy arrivals change dynamically, the transmission between BSs and UEs could be regarded as slot-based transmission. Meanwhile, during each time slot, the wireless channels and renewable energy arrivals remain stable. We consider the problem of a fixed scheduling time containing T time slot, denoted as T � 1, 2, · · · , T { }.

Traffic Model.
is paper considers the downlink transmission of a two-tier HetNet with orthogonal frequency division multiple access (OFDMA). In the network, the total bandwidth B is equally divided into N subcarriers, denoted as N � 1, 2, · · · , N { }. To avoid intracell interference, at time slot t, subcarrier n can be allocated to zero or one SUE in each small cell. e allocation policy of subcarrier n at time slot t is denoted as X k,s k ,n (t), whose value represents that subcarrier n is allocated to SUE s k in small cell k if it equals 1 and otherwise if it equals 0. en, we have S k s k �1 X k,s k ,n (t) ≤ 1, ∀k, n, t.
(1) P S k,s k ,n (t) is used to denote the power allocated by SBS k to SUE s k on subcarrier n. Limited by hardware, each SBS k has its energy consumption upper bound, denoted as P max k . Depending on subcarrier allocation and power allocation, the total energy consumption P k (t) of SBS k at time slot t cannot exceed the aforementioned upper bound. is restriction is denoted as where P c k is the static energy consumption of SBS k related to cooling and baseband processing.
For wireless channel, h S k,s k ,n (t) denotes the channel gain of SUE s k from SBS k on subcarrier n, h SM k,m,n (t) denotes the interference channel gain of MUE m served by MBS from small cell k on subcarrier n, and h MS k,s k ,n (t) denotes the interference channel gain of SUE s k in small cell k from MBS on subcarrier n. Generally, the channel gain contains two parts: one is the large-scale fading component with path loss and shadowing, and the other is the small-scale fading component with multipath Rayleigh fading. P M m,n (t) is used to denote the power allocated by MBS to MUE m on subcarrier n at time slot t. So the received SINR of SUE s k served by SBS k and occupying subcarrier n at time slot t is denoted as where δ 2 is the background noise. Meanwhile, the achievable rate on subcarrier n of SUE s k served by SBS k at time slot t is defined as R S k,s k ,n (t) � X k,s k ,n (t)log 2 1 + SINR S k,s k ,n (t) .
To reuse subcarrier n of MBS to transmit information, the subcarrier n has a tolerable interference level I n , meaning that the interference on subcarrier n cannot exceed the interference threshold. en, we have K k�1 S k s k �1 X k,s k ,n (t)P S k,s k ,n (t)h SM k,m,n (t) ≤ I n .

Data Queue Model.
Each SBS has an independent buffer for every one of its preregistered SUEs. e arriving data is first stored in the buffer and then transmitted to the corresponding SUE. We use A k,s k (t) to denote the arriving data for SUE s k at time slot t, where SUE s k is severed by SBS k. e arriving data of SUE s k in SBS k can be modelled as an independently and identically distributed (i.i.d) process with a mean of E[A k,s k (t)] � λ k,s k . At each time slot, the arriving data of SUE s k cannot exceed the maximum arriving data A max k,s k . Let J k,s k (t) denote the data backlog of SUE s k in SBS k at time slot t. Based on the admitted data rate and service rate of each SUE, the data backlog of SUE s k in SBS k at time slot t can be updated as where [x] + � max x, 0 { }. We first give the definition of queue stability.

Mathematical Problems in Engineering
Definition 1 (see [19]). When a discrete-time process Depending on rate stability theorem [20], the queue Moreover, when all individual queues of the network are strongly stable, a network is strongly stable.
According to the stability theorem, the queue J k,

Energy Supply and Demand Model.
We use E k (t) to represent the renewable energy collected by SBS k at time slot t, which is based on the i.i.d stochastic process. It should be noted that there is no correlation between renewable energy generations among SBSs. In this paper, the collected renewable energy is firstly used by the SBS and then the remaining energy is stored in the storage device. At time slot t, there are two cases considering the relationship between the energy consumption P k (t) of SBS k and the renewable energy arrival E k (t) of SBS k.
At time slot t, the renewable energy collected by SBS k cannot satisfy its energy consumption. For the insufficient part, it is denoted as P k (t) � (P k (t) − E k (t)) + , which can be satisfied by three ways: (1) Obtained from its storage device.
At time slot t, SBS k gets O k,k (t) from its storage device for the insufficient part of renewable energy. (2) Energy cooperation among SBSs.
Due to the energy cooperation between SBSs, SBS k can use the surplus renewable energy of energy taken from other SBSs has upper bounds, and the maximum energy available from SBS k ′ is O max ,e k′,k < ∞. (3) Getting from the power grid.
When the renewable energy getting from its storage devices and other SBSs cannot meet the energy demand, the SBS k can get energy from the power grid. e amount of energy getting from power gird is expressed as G k (t).
By the above three ways, the insufficient energy of SBS k can be satisfied, which means At time slot t, the renewable energy collected by SBS k can satisfy or exceed its energy consumption. e renewable energy collected is first used to meet its energy consumption. For the remaining renewable energy, which is denoted as E k (t) � (E k (t) − P k (t)) + , there are two applications: (1) Sharing with other SBSs. e renewable energy sharing from SBS k to SBS

Energy Queue Model.
At each time slot t, the remaining renewable energy of the storage device of SBS k is denoted by F k (t). Since the energy changes of the storage device at each time slot are mainly the charging process and the discharging process, we can describe this as an energy queue whose updating process is At each time slot t, the discharging of storage device can neither be more than its remaining energy F k (t) nor exceed the maximum discharging limit O max k . erefore, we get the following constraint: Similarly, at each time slot t, the charging of storage device can neither exceed the remaining battery capacity E k (t) nor exceed the maximum charging limit Y max According to (10), by taking iterated expectation and using telescoping sums over t � 0, 1, . . . , T − 1, we get We further divide T on both sides of (13), where T ⟶ + ∞. According to the stability theory, the energy For clarity, the important notations of this paper are listed in Table 1.

Problem Formulation.
Taking into account the random arrivals of each SUE's traffic data and each SBS's renewable energy, our optimization goal is to minimize time-average on-grid energy consumption while maximizing the timeaverage throughput of all SUEs. At the same time, it is also needed to meet the network stability constraints, the subcarrier allocation constraints, and the cross-layer interference constraints.
en, we formulate the following optimization problem: where and it is SUE s k 's time-average service rate, and and it is SBS k's time-average on-grid energy consumption. We use ι 1 to denote the optimal objective value of P1. In P1, the control variables X k,s k ,n (t), P S k,s k ,n (t), O k,k (t), and O k ′ ,k (t) denote the subcarrier assignment, transmission power control on each subcarrier, SBS's discharge control, and energy sharing, respectively. ϕ and ψ are nonnegative constants, which are used to balance the throughput and ongrid energy consumption. C1 indicates that one subcarrier can be allocated to at most one SUE in one SBS. C2 indicates an index of subcarrier allocation. C3 is the transmission power consumption constraint of SBSs. Constraint C4 is the tolerable cross-layer interference for each subcarrier occupied by SUEs. C5 is the stability constraint of data and energy queues. C6 is the compensation component of the insufficient renewable energy. C7 is the allocation of extra renewable energy for each SBS. C8 and C9 are discharging and charging constraints of each SBS, respectively. C10 is the renewable energy sharing constraint.

Problem Transformation.
It is observed that, in P1, the decision variables X k,s k ,n (t) , P S k,s k ,n (t) , O k,k (t) , and O k′,k (t) are coupled together. We decompose the optimization problem P1 into joint resource allocation and energy scheduling by the Lyapunov optimization method in each time slot. e design principle of Lyapunov optimization method is to minimize the upper bound of Lyapunov drift minus objective function greedily without knowing the profile of stochastic events. F(t) � F k (t) and J(t) � J k,s k (t) are used to represent the energy queue and the data queue, respectively. en, we use Q(t) � F(t), J(t) { } to denote all queues in the network system. At each slot t, we define the Lyapunov function as where θ is a perturbation factor, which ensures that there is enough energy in the energy queue. Without loss of generality, we assume that all queues are empty when t � 0 such that L(Q(t)) � 0. e Lyapunov function L(Q(t)) is a scalar measure of network congestion. Intuitively, if L(Q(t)) is small, then all queues are small; and if L(Q(t)) is large, then at least one queue is large. us, by minimizing the Lyapunov drift (i.e., by minimizing a difference in the Lyapunov function from one time slot to the next), the data queues and energy queues can be stabilized.
By using expression in (17), the Lyapunov drift can be written as and, to solve optimization problem P1, we now use the driftplus-penalty minimization method [21], where a control policy that solves P1 is obtained by minimizing the upper bound of the following drift-plus-penalty term: where V is a nonnegative control variable that represents the relative importance of minimizing the queue length and maximizing the objective function. According to [21], when

Mathematical Problems in Engineering
V is sufficiently large, the optimization algorithm approaches the optimal policy. e upper bound of the drift-plus-penalty can be derived as follows. (19) is

Lemma 1. The upper bound of drift-plus-penalty in
The proof of Lemma 1 is shown in Appendix A. By Lemma 1, we can transform the optimization problem P1 into minimizing the right-side term of (20) at each time slot t. us, we will decompose the optimization problem P1 and present an online dynamic control algorithm for the resource allocation and the energy scheduling and energy cooperation, respectively.

Online Solution
e proposed problem is solved online based on two subproblems. e resource allocation subproblem determines the subcarrier assignment and the transmission power allocation on each subcarrier. e energy scheduling and energy cooperation subproblem determines the discharging and charging, the energy cooperation, and on-grid power consumption of each SBS. Solving the above two subproblems in each time slot can finally determine the online resource allocation scheme and the energy scheduling and the energy cooperation scheme, so as to maximize the timeaverage energy efficiency of SBSs.

Resource Allocation Subproblem.
e resource allocation subproblem jointly considers transmission power control and subcarrier allocation. At the beginning of each time slot, we can solve the resource allocation subproblem if we get the subcarrier status and queue information. e resource allocation subproblem is given below: Problem P2 is an NP-hard mixed-integer programming problem. To solve this problem, we extend the binary variable X k,s k ,n (t) to the continuous variable X k,s k ,n (t) ∈ [0, 1]. At the same time, we introduce a new variable P S k,s k ,n (t) � X k,s k ,n (t)P S k,s k ,n (t). Ignoring the static power consumption, problem P2 can be rewritten as Vϕ + J k,s k (t) X k,s k ,n (t) × log 2 Since the optimization problem P3 is a convex optimization problem, it can be solved in polynomial time. Here, we consider using the Lagrange dual method to solve problem P3. e Lagrange function is Vϕ + J k,s k (t) X k,s k ,n (t) × log 2 where α � α k , β � β n , ζ � ζ k,n are Lagrange multipliers for constraints C 3 , C 4 , C 1 , respectively. Further, we have the dual function as follows: (26) e dual problem (26) can be further decomposed into K × N independent subproblems, making each SBS independently decide on each subcarrier. erefore, problem (26) is equivalent to where L k,n (t) X k,s k ,n (t) , P S k,s k ,n (t) , α, β, ζ � S k s k �1 Vϕ + J k,s k (t) X k,s k ,n (t) × log 2 1 + P S k,s k ,n (t)h S k,s k ,n (t) X k,s k ,n (t) P M m,n (t)h MS k,s k ,n (t) + δ 2 For the solution of L k,n (t), we first introduce the Lagrange multiplier τ k,s k ,n for the constraint C 5 . en, we find the partial differential function of L k,n (t) with respect to P S k,s k ,n (t). en, we have zL k,n − τ k,s k ,n P S k,s k ,n (t) zP S k,s k ,n (t) � X k,s k ,n (t) Vϕ + J k,s k (t) ln 2 × h S k,s k ,n (t) X k,s k ,n (t)Z k,s k ,n (t) + P S k,s k ,n (t)h S k,s k ,n (t) where Z k,s k ,n (t) � P M m,n (t)h MS k,s k ,n (t) + δ 2 .
In order to obtain the optimal transmission power control for the resource allocation subproblem, the above problem (27) must satisfy its Karush-Kuhn-Tucker (KKT) conditions, which are denoted as zL k,n − τ k,s k ,n P S k,s k ,n (t) zP S k,s k ,n (t) � 0, τ k,s k ,n, P S k,s k ,n (t) � 0, P S k,s k ,n (t) ≥ 0, τ k,s k ,n ≥ 0.
(30) erefore, the optimal P S * k,s k ,n (t) can be obtained by setting the partial differential function to be 0. en, the specific optimal transmission power allocation is as follows: (31) We use the subgradient method to update the Lagrange multipliers α k and β n , which are updated as where ρ ε 1 and ρ ε 2 are the step sizes of iteration ε. Based on the above optimal transmission power allocation results, we can find the optimal subcarrier assignment. Decomposing the dual problem (27) into S k subproblems, for each subproblem, we have where L k,s k ,n (P) � Vϕ + J k,s k (t) X k,s k ,n (t) Subcarriers will be assigned to SUEs with the largest L k,s k ,n (P), which means where L k,n * s ,n (P) > 0. erefore, the subcarrier assignment decision is as follows: Besides, we also use the subgradient method to update the Lagrange multiplier ζ k,n , which is updated as where ρ ε 3 is the step size of iteration ε.

Energy Scheduling and Energy Cooperation
Subproblem. e energy scheduling and energy cooperation subproblem jointly considers the control of SBS's discharging, charging, energy cooperation, and on-grid energy consumption. At each time slot t, SBS k solves the following problem based on local information for energy scheduling and energy cooperation: s.t. C 7 , C 8 , C 9 , C 10 , Mathematical Problems in Engineering 9 According to constraint (8) and K can be further written as P5: min e optimal solution to the above problem is Y * k (t), O * k,k (t), and O * k,k ′ (t). e energy G * k (t) of SBS k getting from the grid can be expressed as Since P5 is a linear programming problem, it can be solved directly by linear programming tools.

Online Algorithm.
As presented in the above subsections, the formulated time-average delay-aware and interference-aware energy efficiency problem is solved distributively through several decompositions. Firstly, we use Lyapunov optimization method to decompose the optimal problem into two subproblems at each time slot, that is, resource allocation subproblem and energy scheduling and energy cooperation subproblem. Secondly, the resource allocation subproblem is solved through Lagrange dual decomposition. en, the dual problem is decomposed into K × N independent subproblems for optimal power allocation and S k independent subproblems for optimal subcarrier assignment. Meanwhile the energy scheduling and energy cooperation subproblem is decomposed into K independent subproblems for the optimal discharging, charging, energy cooperation, and on-grid energy consumption.
Based on the above solution, we further present the distributed algorithm for joint resource scheduling and energy cooperation in interference-constrained HetNets powered by hybrid energy, which is shown in Algorithm 1.

Performance Analysis
hold, the following relationships have been established: are satisfied, the renewable energy collected by the storage device satisfies F k (t) ∈ [0, F max k ] at any time slot.
Proof. See Appendix C.
□ Lemma 2. Assume that the coverage range of the network is Λ. As the data arriving process J k,s k (t) , the channel gain H(t), and the renewable energy generation E k (t) all follow i.i.d processes, a random control strategy ω is implemented. en, where ς is a finite positive number. When the constraint of P1 is satisfied, it can be proved by Carathéodory's theorem. Please refer to the specific proof in [22,23].

Theorem 3 (a) e upper bound of the time-average data queue is
(b) e time-average energy efficiency of the proposed algorithm satisfies Proof. See Appendix D.

Simulation Results
In this section, the simulation results of the proposed algorithm are given. We consider a hybrid network that consists of one MBS and K � 3 SBSs, where the coverage radius of MBS is 500m and the coverage radius of SBS is 30m. e MBS has 10 MUEs and each SBS has 2 SUEs. In this paper, we consider wireless channels with small-scale Rayleigh fading component and a large-scale path loss. Every SUE has a buffer to its related SBS, where the buffer size J max k,s k � 500. At each time slot, SBS k receives burst data of its corresponding SUE s k which follows a Poisson distribution with average rate λ k,s k � 15. SBS k receives renewable energy which follows a Poisson distribution with average rate λ k � (5.5 J/s) [9,24]. e other parameters are listed in Table 2.
Firstly, we demonstrate the stability of network. We choose T � 1000, V � 10, ϕ � 4, ψ � 0.5; the trend of each queue is shown in Figure 2. As the same types of queues follow a similar trend, we only show the trends of SUE1 of SBS1, SUE2 of SBS2, and SUE3 of SBS3 in the data queue J and show the trends of SBS1, SBS2, and SBS3 in the energy queue F. From Figure 2, all data queues are strictly less than the maximum data buffer size, which is in line with eorem 3 (a); and all the energy queues do not exceed the capacity of the storage devices, which is in line with eorem 2.
When T � 1000, V � 10, ϕ � 4, ψ � 0.5, λ k,s k � 5, we give the relationship between the long-term admitted rate A k,s k and the service rate R k,s k for all SUEs. As shown in Figure 3, each SUE's admitted rate is less than the service rate, which means that the actual queues are stable. Figure 4 shows the trend of time-average objective value with parameter ψ when V and ϕ are fixed. For each ψ, we take T � 1000. With the increase of ψ, which means the emphasis on time-average on-grid energy consumption, the time-average objective values decrease. On the other hand, Figure 5 shows the trend of time-average objective value with parameter ϕ when V and ψ are fixed. For each ϕ, we take T � 1000. As ϕ increases, which means that the timeaverage service rate weighs more, the time-average objective value increases. e results of both Figures 4 and 5 agree with eorem 3 (b).   Figure 6 compares the proposed algorithm with a basic algorithm. To highlight the proposed joint resource allocation and energy cooperation algorithm, in the basic algorithm, all SBSs are supplied by hybrid energy and perform energy scheduling noncooperatively. Besides, the resource allocation in the basic algorithm is optimized with the proposed algorithm in this work. It is shown that, at V � 10, the value of the objective function of the proposed algorithm is about 7% higher than that of the basic algorithm without energy cooperation, which demonstrates that the proposed algorithm has a better time-average energy efficiency than that of the basic algorithm. Moreover, as shown in Figures 4  and 5, the tradeoff factors ϕ and ψ also greatly affect the value of the objective function.

Conclusion
In this paper, we consider the resource scheduling and energy cooperation for a two-tier downlink HetNet, which is powered by hybrid energy supplies. By jointly adjusting the transmission power allocation, the subcarrier assignment, the discharging and charging, the energy cooperation, and the on-grid energy consumption, we can maximize the timeaverage energy efficiency of SBSs in an online manner. Compared with the other works, this paper considers the burst of user data, the cross-layer interference from SBSs to MBS, and the scheduling of random renewable energy sources. rough theoretical analysis and simulations, we prove the stability and superiority of the proposed algorithm.  Depending on the update process of energy queue F k (t), we have By employing above inequalities, we can prove Lemma 1.

B. Proof of Theorem 1
In order to minimize the target value of the energy scheduling problem (36), we can infer that Y * k (t) � 0 when

(B.2)
For the second case, in order to minimize the objective function, we have O * k,k (t) � 0. Bringing θ � Vψ + O max k to (B.1), we get

C. Proof of Theorem 2
At the first time slot 0, the constraint 0 ≤ F k (0) ≤ F max k holds, since SBS K has no operation. We consider that there are four cases for the energy F k (t) in storage devices.
So, according to the energy update process of storage device, Based on the above proofs, F k (t + 1) ∈ [0, F max k ] is still satisfied when F k (t) ∈ [0, F max k ] is satisfied.

D. Proof of Theorem 3
Using the random control strategy ω, we have en, eorem 3 (a) is proved. Moreover, the above (D.1) can be rewritten as (D.6) In conclusion, eorem 3 (b) is proved.
Data Availability e data in the simulations used to support the findings of this study are included within the article. e common data are provided in Table 2. For specific simulation, the data are illustrated in the context. All the results can be redone according to the provided data.

Conflicts of Interest
e authors declare that they have no conflicts of interest.