We study how to utilize network coding to improve the throughput of secondary users (SUs) in cognitive radio networks (CRNs) when the channel quality is unavailable at SUs. We use a two-dimensional multiarmed bandit (MAB) approach to solve the problem of SUs with network coding under unknown channel quality in CRNs. We analytically prove the asymptotical-throughput optimality of the proposed two-dimensional MAB algorithm. Simulation results show that our proposed algorithm achieves comparable throughput performance, compared to both the theoretical upper bound and the scheme assuming known channel quality information.
With the fast development of the technology of software-defined radios [
Recently, network coding [
Nevertheless, in CRNs, similar to the idle duration, the channel quality may also be unknown to SUs. For example, when the SUs choose a new channel, the quality of this channel can not be obtained immediately. Moreover, the channel quality itself may be changing dynamically due to the complex spectrum environment. Unfortunately, most existing works (e.g., [
In this paper, we propose a two-dimensional multiarmed bandit (MAB) approach to combat the problem of SUs with network coding under unknown channel quality in CRNs. Specifically, we develop a two-dimensional MAB algorithm that sequentially chooses the idle duration and block size jointly. With the help of MAB, the exploration and exploitation on the idle duration-block size pair can ensure the asymptotical-throughput optimality of the proposed algorithm. Unlike previous studies, our proposed algorithm does not need the channel quality in advance.
The novelties of this paper are as follows: The problem of SUs with network coding under unknown channel quality in CRNs is defined. A novel two-dimensional MAB algorithm will be proposed that chooses the idle duration and block size jointly before data transmission. The proposed algorithm is able to properly handle the case when channel quality cannot be obtained. The optimality of the proposed two-dimensional MAB algorithm will be mathematically analyzed, and the algorithm will be theoretically proven as asymptotical throughput optimal. The performance of the proposed algorithm is compared to the theoretical upper bound and the scheme assuming known channel quality information, in terms of throughput achieved.
The rest of the paper is organized as follows. In Section
Several previous studies have demonstrated the various benefits of network coding in CRNs from different aspects. Jin et al. [
On the other hand, our work can be classified as the study of learning the primary user environment in CRNs, since we try to utilize network coding for SUs’ data transmission without knowing the idle duration and channel quality information. Among the works about the channel access in CRNs, there are several studies employing MAB to solve the learning-based dynamic spectrum access problem from a sequential decision aspect. Shu and Krunz [
In this section, we will first introduce the system and network model used in our work and then present the problem formulation.
We consider a CR network where secondary users (SUs) want to achieve better performance by utilizing network coding. Consider that there are one PU channel and multiple SUs. Without loss of generality, we assume that there are
Time is slotted and synchronized among the channel and multiple SUs. We note that, in our work, one time slot consists of both the time of channel sensing and the time of transmitting one data packet. The slot structure is shown in Figure
SU’s slot structure for channel sensing and data transmission.
In this work, we focus on systematic network coding (SNC) [
According to several previous studies [
In this study, we aim to provide a general sensing/transmitting mechanism for SUs with SNC under unknown channel quality in CRNs. The sequential sensing/transmitting problem is to determine the idle duration of the channel and the block size jointly, without knowing future channel states, for SUs to improve the throughput. We model the problem into a nonstationary multiarmed bandit (MAB) problem. Since there is no gain if no data is transmitted in one time slot, we employ several continuous time slots spent for sensing the channel as our study unit, which is called as a round in this paper. After a round, the SU sender may or may not transmit data, according to the sensing result and previous gains. Note that the transmission time slots are not counted in a round. The strategy for sensing/transmitting is composed of many sequential rounds of sensing.
A sensing/transmitting strategy
Let
In the modeled two-dimensional MAB problem for SUs with SNC, each idle duration is considered as an arm of a gamble machine and the block size is the coin the gamble machine bets on that arm. The reward is defined as the number of successfully delivered data packets. Typically, the gambler’s performance is measured in terms of regret in the MAB problem. It is defined as the difference between the expected return of the gambler’s actions and that of the static optimal strategy. In this study, we aim to design a strategy
The regret in our application is the difference between the expected total number of delivered packets using our proposed algorithm and that using the static optimal strategy over lifetime
In this section, we focus on developing a MAB-based algorithm for SUs with network coding under unknown channel quality in CRNs. Specifically, we propose a two-dimensional MAB algorithm to sequentially choose the idle duration
Inspired by the approach in [
Generally speaking, the SUs choose the strategy vector
In step 1, the calculation of
In steps 2 and 3, the SUs execute the transmission according to the selected
Calculate the strategy probability distribution: Choose the strategy ( Get the scaled output For Update all the weights as
In this part, we analyze the performance of our two-dimensional MAB algorithm in terms of regret. In our application, the regret is the difference between the number of successfully delivered data packets using the static optimal idle duration-block size pair and that using our two-dimensional MAB algorithm.
For any
The detailed proof is provided in the Appendix.
Based on the above regret analysis, we prove that Algorithm
Algorithm
According to Theorem
In Theorem
In this section, we evaluate the performance of our proposed two-dimensional MAB algorithm via simulations. Note that, when channel quality is known in advance, the optimal coding block size of SNC can be determined according to the current idle duration [
Besides, we define a new metric named “utilization,” which is the ratio of the performance of our two-dimensional MAB algorithm to that of “Full Information + OSNC” or “MAB + OSNC.” This metric can not only precisely characterize the spectrum utilization of SUs in CRNs but also show the throughput performance of our proposed algorithm under unknown channel quality. Lastly, we present the practical regrets of our two-dimensional MAB algorithm to validate the regret bound analysis.
In the simulation, we fix the number of SU receivers
To simulate a trace of nonstationary idle durations, we mix several different probability distributions. In this work, we only present two representative cases due to space limitation. One consists of a Poisson distribution
Figure
Performance comparison under
Performance of different schemes under different channel qualities
Relative utilization of two-dimensional MAB Algorithm
Moreover, from Figure
Figure
Performance comparison under
Performance of different schemes under different channel qualities
Relative utilization of two-dimensional MAB algorithm
Figure
Performance under
Performance comparison under nonstationary idle duration III
Performance comparison under nonstationary idle duration IV
Relative utilization under nonstationary idle durations III and IV
To validate the regret bound analysis, we conduct simulations on the practical regrets achieved by the proposed two-dimensional MAB algorithm, compared with the corresponding theoretic bound. Specifically, we fix the number of SU receivers
Figures
Practical regrets of two-dimensional MAB algorithm.
Nonstationary idle duration I
Nonstationary idle duration II
In this paper, we investigate how to utilize network coding to improve the throughput performance of SUs in CRNs when the PU channel quality information is not available. We formulate the problem of SUs with network coding under unknown channel quality as a two-dimensional MAB problem. We propose a two-dimensional MAB algorithm that chooses the idle duration and block size jointly. The performance of the proposed algorithm is analyzed in terms of regrets and analytically proven asymptotically throughput optimal, compared to the static optimal scheme. Our extensive simulation results show that the throughput performance of our proposed algorithm is close to the theoretic bound and the network coding algorithm assuming known channel quality information, by achieving up to 87% utilization.
Define
Define
For the lower bound, according to the definitions, we have
For the upper bound,
Thus,
Summing for
Combining the lower bound and the upper bound, we have
For any fixed
Applying this bound, with probability at least
By doing some transpositions and using the fact that
If we set
Block size determined at round
Estimated idle duration
Average sensing time
Maximum idle duration
Set of idle duration and block size pairs
Set of SU receivers
SUs’ strategy at round
SUs’ decision to access the channel or not at round
Gain for strategy
Total gain over a fixed strategy
Total gain over chosen strategies up to round
The authors declare that there is no conflict of interests regarding the publication of this paper.
This work is supported by the National NSF of China under Grants no. 61371124 and no. 61472445 and the NSF of Jiangsu Province under Grant no. BK20140076.