Mathematical Problems in Engineering
Volume 2018 (2018), Article ID 9547934, 9 pages
https://doi.org/10.1155/2018/9547934

Research Article

An Adaptive Gradient Projection Algorithm for Piecewise Convex Optimization and Its Application in Compressed Spectrum Sensing

Tianjing Wang,¹ Hang Shen,² Xiaomei Zhu,² Guoqing Liu,¹ and Hua Jiang¹

¹School of Physical and Mathematical Science, Nanjing Tech University, Nanjing 211816, China
²College of Computer Science and Technology, Nanjing Tech University, Nanjing 211816, China

Correspondence should be addressed to Tianjing Wang; wangtianjing@njtech.edu.cn

Received 18 November 2017; Revised 20 March 2018; Accepted 2 April 2018; Published 14 May 2018

Academic Editor: Paula L. Zabala

Copyright © 2018 Tianjing Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Signal sparse representation has attracted much attention in a wide range of application fields. A central aim of signal sparse representation is to find a sparse solution with the fewest nonzero entries from an underdetermined linear system, which leads to various optimization problems. In this paper, we propose an Adaptive Gradient Projection (AGP) algorithm to solve the piecewise convex optimization in signal sparse representation. To find a sparser solution, AGP provides an adaptive stepsize to move the iteration solution out of the attraction basin of a suboptimal sparse solution and enter the attraction basin of a sparser solution. Theoretical analyses are used to show its fast convergence property. The experimental results of real-world applications in compressed spectrum sensing show that AGP outperforms the traditional detection algorithms in low signal-to-noise-ratio environments.

1. Introduction

The marked advances in signal processing in recent years have been driven by the emergence of new signal models and their applications. Signal sparse representation is an effective model for solving real-world problems, such as brain signal processing [1], face recognition [2], compressed spectrum sensing [3], and singing voice separation [4].

Given a signal , signal sparse representation aims to identify the sparsest solution from an underdetermined linear system , where is a full row rank matrix. The sparsity of a solution can be measured by -norm, which leads to the following optimization problem

Unfortunately, problem (1) is NP-hard [5]. Many methods (e.g., the greedy algorithm [6], the -norm minimization [7], the -norm minimization [8, 9], and the Bayesian method [10]) have been used to find sparse solutions. Because -norm is the most effective measurement of the sparsity, some researchers are interested in -norm minimization

The piecewise convex optimization (2) can be solved using the existing algorithms, including the Focal Underdetermined System Solver (FOCUSS) [11], the Affine Scaling Transformation (AST) method [12], the Iteratively Reweighted minimization (IRL1) [13], and the Iteratively Thresholding Method (ITM) [14]. The solutions they obtain, however, may be suboptimal sparse solutions.

In this paper, we propose a novel Adaptive Gradient Projection (AGP) algorithm for the piecewise convex optimization (2). This algorithm moves the iteration solution out of the attraction basin of a suboptimal sparse solution and finds a sparser solution in another attraction basin. The convergence analysis reveals that AGP performs better than AST when finding the global optimal sparse solution. The experimental results show that the detection performances of compressed spectrum sensing based on AGP are greatly improved compared to other algorithms.

The remainder of the paper is organized as follows. In Section 2, we derive an Adaptive Gradient Projection algorithm that can find a sparser solution than AST. Section 3 presents the application of AGP to compressed spectrum sensing. The detection performances based on AGP are compared to the traditional spectrum sensing method. Finally, conclusions are presented in Section 4.

2. Adaptive Gradient Projection Algorithm for Piecewise Convex Optimization

2.1. Description of Affine Scaling Transformation Method

For convenience, problem (2) can be rewritten as

Note that the objective function is nondifferentiable with zero components. AST uses an affine scaling transformation to solve problem (3). For the th iteration, it defines a symmetric scaling matrix and a scaled variable

Thus, problem (3) in is transformed to the problem in

where .

Given a search direction , the new solution is

where is an identity matrix, is a Moore-Penrose pseudoinverse matrix, and is a stepsize.

Using a fixed stepsize , AST is summarized as

The convergence theorem of AST is as follows.

Theorem 1. Starting from an initial point , AST generates a sequence converging to a sparse solution of problem (3).

From (9), we see that some small entries of iteration solution converge to zero, because they are sequentially compressed by the scaling elements in . Thus, a sequence of iteration solutions of AST will converge to a sparse solution , which may be close to . However, this solution may be not the sparsest solution of problem (3). Making the iteration solution enter the attraction basin of other sparse solution is very important to reduce the effect of the initial point. Furthermore, Theorem 1 shows that AST obtains within an infinite number of iterations, which affects the convergence rate. How to enhance the convergence speed of AST is another problem to be solved.

2.2. Derivation of Adaptive Gradient Projection Algorithm

To solve the above two problems, we first consider the convergence process of AST if an iteration solution has some zero entries.

Lemma 2. Given a block matrix , we get

where is a Schur complement.

Lemma 3. If with zero entries is a solution of problem (3), then these zero entries will not change in the remaining iterations.

For simplicity, let the front entries of be zero (i.e., ), where , . Then, can be computed as follows:

where . Partitioning , we calculate in (11) to be

and its Moore-Penrose pseudoinverse matrix is

where

Because is full row rank, is also full rank, which has an invertible submatrix. For convenience, assume that is an invertible submatrix. Otherwise, we partition to get an invertible submatrix according to Lemma 2. Defining , , , , we get

where , , , , .

Substituting (12) and (15) into (13), we have

where , . Equation (11) then becomes

The front entries of in (17) are still zero, and the reverse cannot occur in the remaining iterations.

Lemma 3 implies that an unknown sparse solution can be identified in smaller and smaller subspace. It motivates us to accelerate convergence by sequentially shrinking the solving range. Meanwhile, the choice of subspace cannot be limited by the initial point; that is, the iteration solution is able to enter the subspace that does not contain the initial point. AST chooses a fixed stepsize, so it cannot move the iteration solution from one octant to another distant octant. These solutions concentrate in the attraction basin of the suboptimal sparse solution. Moving the iteration solutions out of the current attraction basin is a goal of the Adaptive Gradient Projection algorithm.

To find the search direction at an iteration solution , we define the gradient

where

and represents that the partial derivative is not computed as , and the corresponding component is deleted from . Definition (18) avoids the nondifferentiable problem of -norm minimization such that the gradient can be calculated in subspace. Then, we provide the algorithm derivation below.

Initially, we reset an initial point as

where the index set records the locations of entries as . For example, if , , then and .

For the th iteration, let and ; the gradient and the search direction are

where records the locations of entries as , the column vectors of are selected from due to , and is an identity matrix. In the span space of , the new solution is defined due to (6):

where is a stepsize.

The major challenge in solving problem (3) is to identify the most appropriate subspace where the sparse solution locates. Equation (22) states that is important to find an appropriate subspace; thus, a function with respect to is defined to investigate the property of the stepsize

Figure 1(b) shows that the piecewise convex function has nonunique minimum points that correspond to the zero entries of . We can compute these extreme points.

(a)

(b)

Figure 1: Function curves of with respect to . (a) Fixed stepsize selected by AST, marked as “+.” (b) Adaptive stepsize selected by AGP, marked as “.”

Without loss of generality, let every entry of equal to zero (i.e., ). Then, the extreme point is computed as

where and . An adaptive stepsize is chosen to find the minimal objective function value

and the new solution is written as

The minimum point , marked as “” in Figure 1(b), is determined by comparing three extreme values. Using the adaptive stepsize to obtain an iteration solution is beneficial to accelerate convergence. However, AST cannot obtain a minimum point at the search direction in (6), and the corresponding point is marked as “+” in Figure 1(a). Using the fixed stepsize makes the iteration solutions gather in the adjacent region of the initial point. On the other hand, we set as , where is a threshold. AGP then determines multiple zero entries at each iteration so that it can quickly identify subspace where the sparse solution locates.

After some entries of become zero, the set of their indices is deleted from (i.e., ). We define that represents the entries of located in that are assigned to . Thus, (26) can be restated as

For example, if and , then and . The iterations are performed until the needed support set is determined.

Returning the final solution to -dimensional space is convenient to obtain the sparse solution . By presetting , then represents that the entries of located in that are assigned to . For example, if and , then .

As discussed above, AGP is summarized as follows.

Algorithm 4 (Adaptive Gradient Projection Algorithm).  ⁠
Step 1 (initialization). Given a threshold , compute an initial point and set an index set and the iteration index .
Step 2 (iteration)

 while  Stopping criterion not met  do

 Compute and due to (21), Choose an adaptive stepsize by

 Compute a solution , Update the solution , Increase iteration index .

 end while

Step 3 (solution). An optimal sparse solution .

2.3. Convergence Analysis

The convergence property of AGP is discussed as follows.

Lemma 5. AGP can determine at least one zero entry at each iteration.

Due to (26), there are three cases that AGP determines zero entry at each iteration. The first case is that one entry of is set to zero, if moves from one octant to the coordinate surface of another distant octant. In the second case, more than one entry of become zero when the new iteration solution exactly locates on the coordinate axis of another distant octant. In the third case, if locates on the side of the coordinate axis of another distant octant, we set as . Therefore, AGP can determine at least one zero entry at each iteration.

Theorem 6. Let with nonzero entries be a sparse solution of problem (3). The number of iterations of AGP is no more than .

If has nonzero entries, then the locations of zero entries need to be determined. According to Lemma 5, AGP obtains multiple zero entries at each iteration, and these zero entries do not change in the remaining iterations based on the definition in (20). Therefore, the number of iterations of AGP is no more than .

Remark 7. Theorem 6 shows that AGP obtains a sparse solution within a finite number of iterations, while Theorem 1 shows that AST requires an infinite number of iterations to obtain a sparse solution. In theory, the number of iterations of AGP is less than AST.

Figure 2 gives an example to show the improved convergence performance of AGP compared to AST. There are three sparse solutions , where is the global optimal sparse solution. To clearly display the iteration process of AGP, we return all to three dimensional space, which form a sequence , where represents that the entries of located in which are assigned to .

(a)

(b)

(c)

(d)

Figure 2: Iteration processes of AST and AGP. (a) Sequence solved by AST converges to . (b) Sequence solved by AGP converges to . (c) Iteration process of AST. (d) Iteration process of AGP.

Starting from , a sequence solved by AST converges to in Figure 2(a). Figure 2(c) shows that all iteration solutions concentrate in the attraction basin of in the contour map. The second iteration solution of AGP in Figure 2(b), however, moves away from and reaches . In Figure 2(d), the iteration solution moves out of the attraction basin of and enters the attraction basin of , in which the adaptive stepsize plays an important role. This example verifies that AGP can find a sparser solution than AST by calculating the minimum point of the search direction at each iteration.

2.4. Comparison of Convergence Performance

To compare the convergence performance of AST and AGP, we give an experiment by the following -norm problem

There exist two sparse solutions and , where is the global optimal sparse solution. We choose as an initial point. Comparing the results in Table 1 with Table 2, we see that AGP quickly finds in smaller and smaller subspace, while AST limited by the initial point just obtains the suboptimal solution . On the other hand, the computing times of AST and AGP are 0.0063 s and 0.0089 s, respectively. AGP costs some time to find an adaptive stepsize at each iteration, but it can obtain the global minimizer. Obviously, it is more important to find the global optimal sparse solution of problem (3).

Table 1: List of the partial iteration solutions solved by AST.

1
(0.0339, 0.5510, 0.0625, 0.3716, 0.0024, 0.4874, 0.3344, 0.0860)

3
(−0.0000, 0.7245, 0.0007, 0.2784, −0.0000, 0.4747, 0.3310, 0.0017)

5
(−0.0000, 0.8575, 0.0000, 0.1115, −0.0000, 0.4334, 0.3910, 0.0000)

7
(−0.0000, 0.9379, 0.0000, 0.0094, −0.0000, 0.4084, 0.4276, 0.0000)

9
(−0.0000, 0.9453, 0.0000, 0.0000, −0.0000, 0.4061, 0.4310, 0.0000)

Table 2: List of the partial iteration solutions solved by AGP.

1
(0.0030, 0.6181, 0, 0.3752, −0.0115,0.5101, 0.2972,0.0410)

2
(−0.0005, 0.6860, 0.3288, 0.0017, 0.4870, 0.3128)

3
(−0.0070, 1.1946, 0.0229, 0.7034, 0.0010)

4
(1.2000, 0.7000)

3. Application of Adaptive Gradient Projection Algorithm in Compressed Spectrum Sensing

The Compressive Spectrum Sensing (CSS) is considered for this study, because it performs the same tasks as signal sparse representation. In [15, 16], the model of CSS is formulated as follows:

where is a measurement, is a Primary User (PU) signal, is a Gaussian noise, is a Secondary User (SU) received signal, and is a Gaussian random matrix. Assume that and can be represented on the discrete cosine basis , (i.e., and ), where and are spectrum coefficients. Let ; the model in (30) can be reformulated as

CSS intends to reconstruct from , so the reconstruction error is used to evaluate the reconstruction performance.

Corresponding to and in Figure 3(a), Figure 3(b) shows that has nonzero entries, while is distributed throughout the frequency domain. The reconstructed spectrum coefficient is obtained by solving the following problem

where . After 21 iterations, solved by AST in Figure 3(d) is unsatisfactory, while solved by AGP converges to after 17 iterations. The corresponding computing times of AST and AGP are 0.0154 s and 0.0227 s, respectively. The reconstructed signals and are shown in Figure 3(c), and the reconstruction errors are 27.22% and 9.99%, respectively. At the cost of a little computing time, the reconstruction performance of AGP is improved compared to AST in noise environment, so AGP exhibits better performance of noise suppression than AST.

(a)

(b)

(c)

(d)

Figure 3: Reconstruction of the SU received signal using AST and AGP, respectively. (a) PU signal and SU received signal . (b) Coefficients and . (c) Reconstructed SU received signals solved by AST and AGP. (d) Reconstructed coefficients solved by AST and AGP.

Note that the characteristic of noise suppressing can greatly improve the detection performance of CSS, especially when the number of nonzero entries is not given in advance. We can reconstruct the SU received signal by choosing some larger nonzero entries of with a threshold . Setting , Figure 4(b) shows the denoising spectrum coefficients are sparser than the coefficients in Figure 3(d). The corresponding reconstructed SU received signals are shown in Figure 4(a). The reconstruction errors reduce to 21.71% and 8.50%. Meanwhile, the variance of white noise reduces to .

(a)

(b)

Figure 4: SU received signals are reconstructed with the denoising spectrum coefficients solved by AST and AGP, respectively. (a) Reconstructed SU received signals based on AST and AGP. (b) Denoising spectrum coefficients solved by AST and AGP.

Next, we consider the detection performance using the reconstructed SU signal with denoising coefficient. Let be a false alarm probability and be a judgment threshold. Then, a binary hypothesis testing problem is used to determine whether the PU is present

where denotes the energy of the detection signal . The Monte Carlo experiments are performed to test the detection probability , where is the number of accurately detecting PU signal. Given , , Figure 5 shows that the Energy Detection (ED) method [17] exhibits high mistaken probabilities in low signal-to-noise-ratio (SNR) environments. However, the detection probabilities of AST, AGP, IRL1, and ITM are greatly enhanced, when the reconstructed SU received signals are solved by the denoising spectrum coefficients. For example, when SNR equals to −5 dB, the detection probabilities using AST, AGP, IRL1, and ITM improve by 75.47%, 79.25%, 58.49%, and 47.17% compared with ED. Furthermore, when SNR changes from −15 dB to −1 dB, AGP shows better detection performance than AST, IRL1, and ITM because of its improved reconstruction performance. Figure 6 shows the corresponding computing time of five reconstruction algorithms, in which the time consumption of AGP increases at most 43.66% and 23.39% compared with ITM and AST. Spending a little more computing time, AGP can attain the best result of spectrum sensing especially in low SNR environment.

Figure 5: Comparison of the detection performance using ED, ITM, AST, AGP, and IRL1 for different SNRs.

Figure 6: Comparison of the computing time using ED, ITM, AST, AGP, and IRL1 for different SNRs.

Note that the sparsity of the spectrum coefficient vector has impact on the reconstruction performance of -norm minimization. Given SNR is −7 dB, Figure 7 displays that the detection probabilities of AST, AGP, IRL1, and ITM descend when K changes from 2 to 10. This is because the measurement vector in (32) is not able to attain the whole information of a SU received signal when K is larger than 6. Therefore, the unsatisfactory reconstruction results of -norm minimization greatly affect the detection performance of CSS via AST, IRL1, and ITM, while the performance degradation of CSS via AGP is slower than that of the three reconstruction algorithms. Meanwhile, AST, AGP, IRL1, and ITM cost more computing time to find the sparse solution in Figure 8. The above results reveal that the detection performance of CSS needs to be improved when the number of measurement is insufficient. ED determines the state of PU by measuring the energy of a SU received signal, so the property of the sparsity has little impact on its detection result and computing time.

Figure 7: Comparison of the detection performance using ED, ITM, AST, AGP, and IRL1 for different sparsity of the spectrum coefficient vector.

Figure 8: Comparison of the computing time using ED, ITM, AST, AGP, and IRL1 for different sparsity of the spectrum coefficient vector.

4. Conclusions

Signal sparse representation has become a fundamental tool that is embedded into various application systems. One of its fundamental problems is finding a sparse coefficient. In this paper, we develop a novel AGP algorithm to solve the -norm minimization. Theoretical analysis demonstrates that AGP can find a sparser solution than AST, because it avoids the iteration solutions concentrating in the attraction basin of a suboptimal sparse solution. Applying AGP to compressed spectrum sensing, it can obtain the better detection performance than ED, AST, IRL1, and ITM by spending a little more computing time. Future research will extend AGP to more scenarios.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This research was supported by the National Natural Science Foundation of China (nos. 61501224, 61501223, and 61502230), the Natural Science Foundation of Jiangsu Province (no. BK20150960), and the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (no. 17KJB510024).

References

Y. Li, Z. L. Yu, N. Bi, Y. Xu, Z. Gu, and S.-I. Amari, “Sparse representation for brain signal processing: a tutorial on methods and applications,” IEEE Signal Processing Magazine, vol. 31, no. 3, pp. 96–106, 2014.

Y. Wong, M. T. Harandi, and C. Sanderson, “On robust face recognition via sparse coding: The good, the bad and the ugly,” IET Biometrics, vol. 3, no. 4, pp. 176–189, 2014.

J. Yin, J. Sun, and X. Jia, “Sparse Analysis Based on Generalized Gaussian Model for Spectrum Recovery with Compressed Sensing Theory,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 8, no. 6, pp. 2752–2759, 2015.

T.-S. T. Chan and Y.-H. Yang, “Informed group-sparse representation for singing voice separation,” IEEE Signal Processing Letters, vol. 24, no. 2, pp. 156–160, 2017.

B. K. Natarajan, “Sparse approximate solutions to linear systems,” SIAM Journal on Computing, vol. 24, no. 2, pp. 227–234, 1995.

D. L. Donoho, Y. Tsaig, I. Drori, and J.-L. Starck, “Sparse solution of underdetermined systems of linear equations by stagewise orthogonal matching pursuit,” Institute of Electrical and Electronics Engineers Transactions on Information Theory, vol. 58, no. 2, pp. 1094–1121, 2012.

S. S. Chen, D. L. Donoho, and M. A. Saunders, “Atomic decomposition by basis pursuit,” SIAM Review, vol. 43, no. 1, pp. 129–159, 2001.

C.-B. Song and S.-T. Xia, “Sparse signal recovery by ℓq minimization under restricted isometry property,” IEEE Signal Processing Letters, vol. 21, no. 9, pp. 1154–1158, 2014.

F. Y. Wu and F. Tong, “Gradient optimization p-norm-like constraint LMS algorithm for sparse system estimation,” Signal Processing, vol. 93, no. 4, pp. 967–971, 2013.

H. S. Mousavi, V. Monga, and T. D. Tran, “Iterative convex refinement for sparse recovery,” IEEE Signal Processing Letters, vol. 22, no. 11, pp. 1903–1907, 2015.

I. F. Gorodnitsky and B. D. Rao, “Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm,” IEEE Transactions on Signal Processing, vol. 45, no. 3, pp. 600–616, 1997.

B. D. Rao and K. Kreutz-Delgado, “An affine scaling methodology for best basis selection,” IEEE Transactions on Signal Processing, vol. 47, no. 1, pp. 187–200, 1999.

E. J. Candès, M. B. Wakin, and S. . Boyd, “Enhancing Sparsity by Reweighted ℓ₁ Minimization,” Journal of Fourier Analysis and Applications, vol. 14, no. 5-6, pp. 877–905, 2008.

Y. She, “Thresholding-based iterative selection procedures for model selection and shrinkage,” Electronic Journal of Statistics, vol. 3, pp. 384–415, 2009.

D. Romero and G. Leus, “Wideband spectrum sensing from compressed measurements using spectral prior information,” IEEE Transactions on Signal Processing, vol. 61, no. 24, pp. 6232–6246, 2013.

S. Ren, Z. Zeng, C. Guo, and X. Sun, “A Low Complexity Sensing Algorithm for Wideband Sparse Spectra,” IEEE Communications Letters, vol. 21, no. 1, pp. 92–95, 2017.

S. Atapattu, C. Tellambura, H. Jiang, and N. Rajatheva, “Unified Analysis of Low-SNR Energy Detection and Threshold Selection,” IEEE Transactions on Vehicular Technology, vol. 64, no. 11, pp. 5006–5019, 2015.