A PSD Maps Estimation Algorithm for Underlay Cognitive Radio Networks Based on the Convolutional Neural Network

In the underlay cognitive radio networks (CRNs), the power spectral density (PSD) maps play a foundational role in detecting the idle radio resources. However, it is hard to get a high-accurate PSD map estimation result because of the complicated radio environment. For this reason, we propose a novel convolutional neural network- (CNN-) based PSD map estimation algorithm named map reconstruction CNN (MRCNN). Using the CNN to estimate PSD maps for underlay CRNs has not been reported until now. First, on the basis of the proposed color mapping process, we transform the PSD map estimation task to the image reconstruction task. Then, we train the MRCNN to learn the radio environment characteristics from the training data, rather than making direct biased or imprecise wireless environment hypotheses as in the conventional methods. We utilize the extracted knowledge in the training process to reconstruct the PSD map images. As demonstrated in the simulations, the proposed MRCNN method has a better PSD map estimation performance than the conventional methods.


Introduction
e cognitive radio (CR) technology is a promising technology to solve the spectrum resource scarcity problem. On the basis of sensing and adjusting the working parameters, the secondary users or unlicensed users can dynamically access the spectrum holes, i.e., the unused radio resources, which are initially owned by the primary users or licensed users [1,2].
In the underlay cognitive radio networks (CRNs), the unlicensed users are allowed to share the same bands with the licensed users under the constraint of the transmitting power [3]. ere are mainly two approaches to meet the power constraint [4]. As for the first approach, in order to make the interference to the licensed users under a certain threshold, the unlicensed users spread their power over a wide range of spectrum, as shown in Figure 1. On the basis of the ultrawide band (UWB) technology, this approach is usually applied for short-range communications [4]. As for the second approach, the interference temperature is used to limit the transmitting power. In this method, the unlicensed user is allowed to work in the licensed spectrum with a higher transmitting power, provided that the accumulated interferences from the unlicensed users are under a preselected threshold. e underlay cognitive radio utilizes the idle radio resources in the spectral and spatial domains. To detect the spectrum holes, the power spectral density (PSD) maps play an important role in the underlay CRNs. e PSD map is a vivid description of the licensed users' power distribution in space and frequency. On the basis of the PSD maps, the unlicensed users can avoid the power-intensive areas and make full use of the spectrum holes [5]. In addition, the unlicensed users also decide their transmitting power based on the PSD maps.
In the PSD map estimation task, the general setup includes the sensing unlicensed users and the working licensed users. e unlicensed users work cooperatively with each other and estimate the PSD maps of the whole region on the basis of their power spectrum sensing results. e estimation of the radio environment characteristics is the core factor for the PSD map estimation task. However, tradition methods usually make direct assumptions about the radio environment. For example, the inverse distance weighted (IDW) method assumes that the radio attenuation only depends on the distance between the receiving users and the transmitting users [6]. It is not appropriate for the radio environment in the urban region, where there is severe shadow fading effect caused by the buildings, etc.
Deep learning method is a promising method to solve the above biased assumption problem in traditional PSD map estimation algorithms [7,8]. By learning the radio propagation features from the training data, we can estimate the PSD maps of the target region without any radio environment assumptions. Estimating the PSD maps is a regression task in deep learning. e convolutional neural network (CNN) is a powerful neural structure to solve the regression problem [9]. However, using CNNs to estimate PSD maps for underlay cognitive radio networks has not been reported until now.
In this paper, we propose a convolutional neural network-based PSD map estimation algorithm named map reconstruction CNN (MRCNN) algorithm. Firstly, we propose a mathematical model for the PSD map estimation task. To solve the above mathematical problem, we design a color mapping process to convert it into an image reconstruction task. en, on the basis of the autoencoders and the fully convolutional network, we design the MRCNN framework to learn the radio propagation characteristics from the training data and reconstruct the PSD map images. Finally, as illustrated by the simulation results, the MRCNN algorithm outperforms the traditional methods in the PSD map estimation task.
We summarize the original contributions as follows: (1) On the basis of the proposed color mapping process, the original PSD map estimation task is converted into an image reconstruction task. en, the deep learning technology can be applied to solve the PSD map estimation problem. (2) On the analogy of autoencoders and the fully convolutional network, we design a map reconstruction CNN training framework, which compresses the PSD map images and extracts concise and meaningful representations of the wireless radio environments. (3) e MRCNN method intelligently learns the radio propagation features from the training data rather than the conventional algorithms, which make direct imprecise hypotheses about the radio environment features. e proposed algorithm has a better PSD map estimation performance as demonstrated in the simulations.
e structure of this paper is organized as follows. We introduce the related works of the PSD map estimation task in Section 2. e mathematical model is described in Section 3. Section 4 introduces the map reconstruction convolutional neural network. Section 5 shows the simulation results, and Section 6 concludes the paper.

Related Works
To estimate the PSD maps for underlay CRNs, traditional algorithms use the prior knowledge or hypotheses of the wireless environment in the target area. Conventional algorithms include the spatial interpolation algorithms and the basis expansion algorithm.
As for the spatial interpolation algorithms, there are mainly two methods. e first method is the inverse distance weighted (IDW) interpolation [6,10], which assumes that the radio propagation loss only relates to the distance between the receiving users and the transmitting users. We use (1/d idw ) p id to denote the power value of the inverse distance. p id controls the influence of the receiving users upon the interpolation locations. To estimate the PSD maps, we need to preselect certain p id to carry out the IDW interpolation algorithm. However, the IDW method is irrelevant to the realistic physical phenomena. It is difficult to determine whether preselected p id is appropriate or not [6].
Regarding the second spatial interpolation algorithm, i.e., the Kriging interpolation [11,12], it assumes that the values in the interpolation locations are weighted linear accumulations of the receiving PSD of the licensed users based on the semivariogram functions, e.g., the exponential semi-variogram, the linear semi-variogram, etc. e semi-variogram functions can be regarded as the hypotheses of the spatial features for the wireless radio environment in the target area, which quantify the relationships between the interpolation distance and the average PSD maps values of different locations.
As for the basis expansion (BP) algorithm, it constructs the basis expansion model on the basis of the sparsity in spatial and spectral domains [13,14]. According to the wireless environment in the target region, the BP algorithm directly adopts an approximate radio propagation model to estimate the PSD maps. In addition, the basis expansion algorithm usually assumes that the licensed users lie in several candidate positions [15,16]. e above conventional PSD map estimation methods are all on the basis of the radio environment assumptions, which may be effective in the simple wireless environment. However, the real radio environment is quite complicated. It is difficult to build a formulaic representation of the wireless features because of the complex and random factors of the radio environment, e.g., the radio propagation loss, the multipath effect, the shadow fading effect, etc. Using the imprecise hypotheses to model the radio environment leads to the inaccurate PSD map estimation results. Deep learning is a powerful method to solve the PSD map estimation problem [7]. We adjust the parameters of the deep neural networks to obtain the implicit representations of the radio environment features by learning from the training PSD map dataset. On the basis of the neural network-based extracted features, we can get a high-accurate PSD map estimation result, which is the core innovation of this paper.

PSD Map Model
We assume that the target area S is a square region. ere are N U unlicensed users and N L licensed users, which are uniformly distributed in S.
denote the locations and the PSD received by the unlicensed users.
denote the locations and the PSD transmitted by the licensed users. We use P (p,q)⟶(x,y) to denote the wireless propagation function from the licensed users to the unlicensed users. en, we list the corresponding relationships in equation (1), where σ is the standard deviation of the additive white Gaussian noise (AWGN) in S.
In the general settings of the PSD map estimation task, are unknown. In addition, the radio propagation function P (p,q)⟶(x,y) is also unknown, which is the core factor in estimating the PSD maps of the target region. In our proposed MRCNN algorithm, we adjust the parameters of the convolutional neural network to obtain the implicit representations of the radio propagation function P (p,q)⟶(x,y) by learning from the training PSD map dataset.
Equation (2) shows the PSD map estimation model. On the basis of the known PSD relationships (equation (1)), our goal is to estimate the PSD maps in the target area, i.e., the PSD at every position in S:

Color Mapping
Process. e PSD map estimation task expressed in equation (2) can be regarded as a regression problem in deep learning. To use the deep learning-based methods, we design the following color mapping process to convert the PSD map estimation task into the image reconstruction task.
We divide the target area into N × N grids. In addition, we assume that there is up to one user (a licensed user or an unlicensed user) in each grid. e PSDs sensed by the unlicensed user are normalized, and we color the PSD maps according to the PSD values, i.e., we uniformly map different PSD values to different colors, as shown in Figure 2.
In Figure 2, the colored grids denote the positions where there are unlicensed users. e bigger the power value, the brighter the color in each grid. In addition, the white grids in Figure 2 denote the positions where there are no unlicensed users. e core problem of the PSD map estimation task is to estimate the PSD values in these white grids.
On the basis of the above color mapping process, we convert the PSD map estimation task to the image reconstruction task, i.e., reconstructing the complete PSD map images based on the incomplete PSD map images.

Map Reconstruction CNN Algorithm.
e key factor for estimating the PSD maps is to make full use of the radio environment characteristics of the target area. We propose a map reconstruction CNN algorithm to extract the helpful radio environment characteristics from the training data and estimate the PSD maps for area S. e sketch map of the MRCNN is shown as follows.
In Figure 3, we use I c and I e to denote the complete PSD map images from the training data and their corresponding estimation results, respectively. In addition, S(·) is a sampler used to produce I i , i.e., the incomplete PSD map images. e CNN in Figure 3 is our proposed deep neural network, which estimates the PSD maps for the training data, i.e., CNN(I i ) � I e .
ere are two processes in the proposed algorithm: the training process and the testing process. As for the training process, the complete PSD map images from the training dataset are sampled by S(·), i.e., S(I c ) � I i . We reconstruct the incomplete images through the convolutional neural network. en, we measure the gaps between the complete images and the reconstructed images. On the basis of the above gaps, we train the convolutional neural network to bridge the gap through the training iterations. As for the testing process, we input the incomplete PSD maps of the target region S to the well-trained neural network from the training process. en, we can get the estimation results from the outputs of CNN.
To estimate the high-accurate PSD maps, there are 4 factors in our proposed MRCNN: the sampler S(·), the objective function, the convolutional neural network, and the training data. S β (·)which randomly and independently sets the pixels to zero, denoting the grids where there are no unlicensed users. β is the proportion of the incomplete grids. To enhance the estimation ability for different numbers of unlicensed users, we assume that β is uniformly distributed in [α, 1]. In addition, if S is the urban area with streets and buildings and the unlicensed users are uniformly distributed outside the buildings, we use masks to shade the buildings to zero and retain the grids, which stand for the streets. en, we can use the above S β (·) to produce the incomplete images. (2) Regarding the objective function, it measures the gap between I c and I e . e objective function plays a guiding role in improving the estimation ability of the deep neural network. In our proposed MRCNN method, the objective function is shown as follows: In the MRCNN algorithm, we regard the white squares in the incomplete images as the image noise and use the total variation (TV) norm as an image denoising regularization term [17]. λ is a preselected hyperparameter for the TV norm.
(3) As for the convolutional neural network, we design it in the light of the fully convolutional network [18] and the autoencoders [19], as shown in Figure 4. On the analogy of the fully convolutional network [18], we design the map reconstruction CNN to extract helpful knowledge of the target area from the training images. e proposed structure of the neural network is based on the convolution layer, which owns the translation invariance property for images of different sizes [18]. e basic components (e.g., convolution, activation functions, etc.) operate on local input regions and depend only on relative spatial coordinates. e fully connected layers are not recommended for the PSD map estimation task because of their fixed dimensions, which are not flexible for images of different sizes.
On the analogy of the autoencoders, the MRCNN has a symmetrical structure including an encoder and a decoder. e encoder extracts the latent characteristics of the input images, i.e., I c , and compresses them to the low-dimensional feature maps, which contain the implicit representations of the radio propagation features [19,20]. en, the decoder decompresses the low-dimensional feature maps and reconstructs the PSD map images. In the encoder-decoder process, the deep neural network not only compresses the image information but also obtains a semantically helpful representation. To avoid overfitting in the training process, we use the data augmentation method. We transform the complete PSD maps I c from the training data through three sequential operations: image transposing, horizontally flipping, and vertically flipping. e above transformation makes the training dataset 8 times larger than the original set. In addition, the sampler S β (·), which randomly and independently sets the pixels to zero, can be regarded as the regularization technique to improve the performance and robustness of the convolutional neural networks, according to [21]. (4) Regarding the training data, the training PSD map images should be independent and identically distributed as the maps in the target area [7]. us, we should collect them in advance in the target area or in the regions, which share the same radio environment characteristics as the target area. To construct the complete PSD map image dataset for training, we should collect the spectrum of all grids in the target region. Setting the spectrum sensing devices to several vehicles is a feasible way. Rather than making direct biased or imprecise wireless environment hypotheses as in the conventional methods, the core idea of our proposed deep learning-based algorithm is to learn the radio propagation features from the complete PSD map training set. us, the collection of the complete PSD maps is vital and we need to collect them in advance. In addition, if the radio environment characteristics of the target region change, we should update the complete PSD map training dataset. Although the collection of the complete PSD maps can be realized, it should be noted that the collecting process is timeconsuming, especially for the large target regions. Compared with the traditional PSD map estimation methods (e.g. the IDW, etc.) which make direct assumptions about the radio environment features, our proposed MRCNN algorithm has better PSD map estimation results because it is a data-driven method, which works on the basis of the training dataset, rather than the direct assumptions. However, a data-driven method also means that it is highly data-dependent. If we need just a rough estimation result, the traditional PSD map estimation methods are acceptable, which are not data-dependent. However, the conventional methods are all based on the preselected hypotheses or parameters, i.e., preselected parameter-dependent. Choosing a proper parameter is also a difficult thing. For example, the IDW method is irrelevant to any realistic physical phenomenon. It is difficult to determine whether preselected p id of the IDW method is appropriate or not [6]. e PSD maps are powerful tools to improve the utilization efficiency of the radio resources in the underlay cognitive radios. e PSD map estimation results do not need to be superprecise, but accurate enough to reveal the spectrum holes, which is our original goal in the PSD map estimation task. us, besides the training set collected from the target area, it is also allowed to use the simulated training images or the dataset previously collected from the area with similar wireless environment. For example, if the target area is the urban region with the quasiflat terrain, we can simulate the training data on the basis of the Okumura-Hata radio propagation model or/ and other similar models. In addition, if the target area is an urban residential block, we can adopt the training set which is collected in advance from the similar blocks, e.g., the adjacent blocks of the target area, etc.

Simulation Settings.
In the simulations, the target area S is divided into 48 × 48 grids. We assume that the signals attenuate on the basis of the inverse polynomial law [13], i.e., P loss � min 1, (d/d 0 ) −α . P loss is the radio propagation loss in the target area. We use d to denote the distance between the transmitting licensed users and the receiving unlicensed users. d 0 and α are preselected parameters, which are related to the wireless radio environment in the target area. e variance of the additive white Gaussian noise is known in the target area. As for the shadow fading effect in S, a log normal  Figure 4: e neural structure of the MRCNN.   distribution model with 0 mean value is adopted, and σ 2 fd denotes its variance.
As for the testing data, we assume that α � 2, d 0 � 2, and σ 2 fd � 1 for S. In addition, there are two licensed users located at grids (10, 29) and (43, 23) in S, respectively. eir working spectrums are shown in Figure 5, centered at 25 MHz and 75 MHz. e unlicensed users are uniformly distributed in the target region, and they account for 20% grids of S.
Regarding the training data, we simulate 30,000 complete PSD map images on the basis of the same inverse polynomial law model but with different preselected parameters. We simulate 30,000 samples from 3 sets of parameters: α � 3, d 0 � 2, and σ 2 fd � 1; α � 1, d 0 � 2, and σ 2 fd � 0.5; and α � 2, d 0 � 1, and σ 2 fd � 1. Each set includes 10,000 training data. e number of licensed users in each image is selected from 1 to 5 randomly, and we normalize their working power to 1 W. To produce the incomplete PSD map images, we use the sampler S β (·) introduced in Section 4.2 and assume that α � 0.2, according to the unlicensed users' distribution in the testing data.
e training parameters of our proposed MRCNN algorithm are as follows: we use the Adam algorithm with 0.0001 learning rate; the batch size is 32; and λ � 0.001 in the objective function. e change of the Euclidean distance between I c and I e is shown in Figure 6, which helps us to monitor the training process of the MRCNN method. 2000 training iterations achieve sufficient convergence for the MRCNN method in our simulations. e proposed method is compared with the conventional Kriging and IDW interpolation algorithms. e Kriging with the linear and exponential semi-variograms is denoted as L-Krig and E-Krig, respectively. As for the IDW interpolation, we assume that p id � 3.

Simulation Results.
In this section, we define 4 indicators to test the PSD map estimation performance of MRCNN: (1) the visual inspection of the estimated images; (2) the PSD estimation performance for licensed users; (3) the estimation deviation over different numbers of unlicensed users; and (4) the estimation ability for different sizes of images.

e Visual Inspection of the Estimated Images.
We use the well-trained CNN to reconstruct the incomplete PSD map images and observe the reconstruction performance, which is a qualitative and intuitive testing way. e image reconstruction results are shown in Figures 7 and 8.
As shown in Figures 7 and 8, the proposed MRCNN method has better PSD map image reconstruction results than the traditional interpolation algorithms, especially in the region near the licensed users.

e PSD Estimation Performance for Licensed Users.
To test the estimation ability for the licensed spectrum resources, we compare the MRCNN-based method with the real working bands of the licensed users. We assume that the spectrums of the licensed users are centered at 25 MHz and 75 MHz, respectively. ey are represented by the black line in Figure 9. e MRCNN method outperforms the Kriging and IDW methods, as shown in the figure above. e biased or inappropriate hypotheses of the wireless environment features, i.e., the semi-variograms in Kriging and the power value of the inverse distance in IDW, lead to the gap between the real PSD and their estimation results. It is not an accurate way to use the above assumptions to represent the complicated radio environment characteristics. e MRCNN has a better spectrum estimation performance because we learn the features from the training process on the basis of the deep neural networks rather than directly making assumptions about the features.

e Estimation Deviation over Different Numbers of Unlicensed Users.
To measure the estimation deviation, we define the average reconstruction deviation A rd in equation   (4), which measures the gap between the reconstructed and true PSD map images. We select q � 10 frequency points randomly within the spectrum of the licensed users. e superscript i denotes the ith frequency point.
We estimate the PSD maps on the basis of the spectrum measurements from the unlicensed users. us, the average reconstruction deviation A rd is related to the number of the unlicensed users. In Figure 10, the MRCNN algorithm has a better image reconstruction performance than the IDW and the Kriging methods. e deviation of the MRCNN method decreases with the increase of the unlicensed users, which is caused by the fact that more sensing users bring more information of the radio environment.
As for the IDW interpolation, compared with the MRCNN, it has a relatively worse image reconstruction performance because of the biased power value hypothesis. However, the IDW interpolation is irrelevant to the realistic physical phenomena. It is hard to determine whether the preselected power value is proper or not. As for the two Kriging methods, the curves of their estimation deviation rise, as shown in Figure 10. With the increase of the measurement data, the real features of the radio environment show growing differences from the inaccurate semi-variogram hypotheses. e average reconstruction deviation increases because the more the data received by the unlicensed users, the larger the gaps between the Kriging's assumptions and the true wireless environment in the target area S.

e Estimation Ability for Different Sizes of Images.
We train our proposed neural network with a certain size of training data, i.e., 48 × 48 grids as described in the above simulation settings. en, we use the well-trained neural network to test the estimation performance for different sizes of incomplete PSD map images.
As verified by the above three simulations, the MRCNN method provides more accurate estimation performance than the traditional methods. In this section, we test the flexibility and adaptability of the MRCNN-based algorithm for different sizes of PSD map images. To test the trained neural network, we use different sizes of incomplete PSD maps, which share the same radio environment model as that of the training data. We suppose that there are 20%  Figure 11: e estimation results for different sizes of incomplete PSD map images. From the first row to the third row: the incomplete PSD map with 20% grids of unlicensed users, the estimated PSD maps from MRCNN algorithm, and the real PSD maps in the target region. unlicensed users in each testing PSD map. e simulation results are shown in Figure 11.
As shown in Figure 11, although the neural network is trained with a fixed size of images, the proposed algorithm performs well for different sizes of input PSD maps. e simulation results in Figure 11 are caused by the fact that the MRCNN is based on the fully convolutional structure, which operates on local input regions and depends only on relative spatial coordinates. e above testing results verify the flexibility and adaptability of our proposed algorithm.

Conclusion
In this paper, we propose a novel convolutional neural network-based PSD map estimation algorithm named map reconstruction CNN. e proposed method extracts helpful information of the environment characteristics from the training PSD map images, rather than making direct biased or imprecise wireless environment hypotheses as in the conventional methods. e MRCNN method utilizes the extracted features to reconstruct the PSD map images. As verified in the simulations, the proposed MRCNN algorithm has a better PSD map estimation performance than the conventional methods. In our future research, we will focus on the extension of the MRCNN to more actual and timevarying wireless environments. In addition, the collection of the complete PSD maps is time-consuming. Extending our proposed algorithm to smaller training set or using incomplete PSD maps as the training set is also an important direction in our future research.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that there are no conflicts of interest regarding the publication of this paper.