In this paper, using the near orthogonal property, we analyze the performance of greedy block coordinate descent (GBCD) algorithm when both the measurements and the measurement matrix are perturbed by some errors. An improved sufficient condition is presented to guarantee that the support of the sparse matrix is recovered exactly. A counterexample is provided to show that GBCD fails. It improves the existing result. By experiments, we also point out that GBCD is robust under these perturbations.
National Natural Science Foundation of China115260811160113411671122U1404603Henan Normal Universityqd14142Henan Province17A1100081. Introduction
Greedy block coordinate descent (GBCD) algorithm was presented by [1] for direction of arrival (DOA) estimation. In the work of [1], the DOA estimation is treated as the multiple measurement vectors (MMV) model that recovers a common support shared by multiple unknown vectors from multiple measurements. The authors provided a sufficient condition, based on mutual coherence, to guarantee that GBCD exactly recover the nonzero supports with noiseless measurements.
Recently, the work of [2] discussed the following method: (1)minXX2,1s.t.Y^=AX+N,A^=A+E,with inputs Y^∈Rm×L and A^∈Rm×N. N denotes the measurement noise and E denotes the system perturbation. The perturbations E and N are quantified with the following relative bounds:(2)NFYF≤ε,E2KA2K≤εA,where A2(K) and YF are nonzero. Here, A2(K) denotes the largest spectral norm taken over all K-column submatrices of A. Throughout the paper, we are only interested in the case where ε and εA are far less than 1. In (1), X is a K-group sparse matrix; that is, it has no more than K nonzero rows, and X2,1=∑i=1Nxi2, xi is the ith row of X. It is assumed that all columns of A^ are normalized to be of unit-norm [3]. Both Y=AX and A are totally perturbed in (1). This case can be found in source separation [4], radar [5], remote sensing [6], and countless other problems. In addition, the total perturbations have also been discussed in [7–9].
One of the most commonly known conditions is the restricted isometry property (RIP). A matrix A satisfies RIP of the order K if there exists a constant δ∈(0,1) such that (3)1-δh22≤Ah22≤1+δh22for all K-sparse vector h. In particular, the minimum of all constants δ satisfying (3) is called the restricted isometry constant (RIC) δK.
There are many papers [8, 10–14] discussing the sufficient condition for orthogonal matching pursuit (OMP) that is one of the widely greedy algorithms for sparse recovery. In [3], using the near orthogonality property, the authors improved the sufficient condition of OMP. As cited in [3], the near orthogonality property can further develop the orthogonality characterization of columns in A; it will play a fundamental role in the study of the signal reconstruction performance in compressed sensing. In the noiseless case, the work of [15] analyzed the performance of GBCD using near orthogonality property and improved the results in [2].
In this paper, under the total perturbations, we use near orthogonality property to improve the theoretical guarantee for the GBCD algorithm. In [2], the authors stated that δK+1<1/(K+1) is a sufficient condition for GBCD. We improve this condition to δK+1<(4K+1-1)/2K. We also present a counterexample to show that GBCD fails. The example is superior to that in [2]. Under the total perturbations, the robustness of GBCD is shown by experiments.
Now we give some notations that will be used in this paper. ai denotes the ith column of a matrix A. A′ denotes the transpose of A. IM denotes an M×M identity matrix. The symbol vec denotes the vectorization operator by stacking the columns of a matrix one underneath the other. The cardinality of a finite set Γ is denoted by |Γ|. Let Ω≔{1,2,…,N}. Γc=Ω∖Γ={i∣i∈Ω, and i∉Γ}. The support of X is denoted by supp(X) (supp(X)={i∣xi≠0}). A2(K) denotes the largest spectral norm taken over all K-column submatrices of A. Let A2,∞ denote the maximum l2 norm of the rows of A. We write AΓ for the column submatrix of A whose indices are listed in set of Γ and XΓ for the row submatrix of X whose indices are listed in the set Γ. ei∈RN denotes the ith unit standard vector.
2. Problem Formulation
Analogous to [1], (1) can be rewritten as(4)minX12Y^-A^XF+λX2,1.
Assume that Γ≔supp(X). Obviously, |Γ|=K. The objective function in (4) can be written as (5)FX=GX+HX,where G(X)=(1/2)Y^-A^XF2=(1/2)vec(Y^)-IL⊗A^vec(X)F2 with ⊗ denoting the Kronecker product and H(X)=λX2,1=λ∑i=1Nxi2. Combining the quadratic approximation for G(X) and standard BCD algorithm, the solution to the ith subproblem can be given by a soft-thresholding operator. The authors in [1] only update the block that yields the greatest descent distance. Now, we list GBCD algorithm (Algorithm 1).
(7) Choose the index i0 such that comp(i0)=max(comp)
(8) X(n)←[x1(n-1);…;xi0-1(n-1);
(9) xi0(n);xi0+1(n-1);…;xN(n-1)]
(10) n←n+1
(11) End Repeat
Suppose that A satisfies the Kth order RIC δK∈(0,1). Recall that X has no more than K nonzero rows. According to the fact XF2=∑i=1Lxi22, we can obtain (6)1-δKXF2≤AXF2≤1+δKXF2from (3).
Combining Lemma 2.4 in [3] and (6), we have (7)AΓ′AXF2=∑i=1LAΓ′Axi22(8)≥∑i=1L1-δKAxi22(9)=1-δK∑i=1LAxi22(10)=1-δKAXF2.
Lemma 1 (near orthogonality property, see [3]).
Let u and v be two orthogonal sparse vectors with supports Tu and Tv fulfilling Tu∪Tv≤K. Suppose that A satisfies RIP of order K with RIC δK. Then we have (11)cos∡Au,Av≤K,where ∡(Au,Av) denotes the angle between Au and Av.
Lemma 2 (see [3]).
Under the same assumptions as in Lemma 1, we have (12)Au,Av≤δAu2Av2.
Lemma 3.
For finite sets Γ- and Γ~, let supp(X-)=Γ- and supp(X~)=Γ~. Here, Γ-∩Γ~=∅, and Γ-∪Γ~≤K. If A satisfies the RIP condition (3) with δK∈(0,1), then we have (13)AX-,AX~F≤δKAX-FAX~F.
Proof.
Note that the Frobenius norm of A is derived from the Frobenius inner product. (14)AX-,AX~F=∑i=1LAx-i,Ax~i(15)≤δK∑i=1LAx-i2Ax~i2(16)=δK∑i=1LAx-i2Ax~i22(17)≤δK∑i=1LAx-i22∑i=1LAx~i22(18)=δKAX-FAX~F,where (15) and (17) follow from Lemma 2 and Cauchy-Schwarz inequality, respectively.
3. RIP Based Recovery Condition
In this section, we firstly present the upper bound of the noise matrix -EX+N and provide the recovery condition for GBCD.
Lemma 4 (see [2]).
Suppose that A^ satisfies the Kth order RIC δ^K∈(0,0.5). Then we have (19)-EX+NF<εA1/3-1+1/3εA+εY^F1-ε.
According to steps (7) and (8) of Algorithm 1, at the nth iteration, GBCD can obtain a correct index if (20)maxi∈ΓXin-Xin-12>maxj∈ΓcXjn-Xjn-12.
Theorem 5.
Consider model (4). Let t0=mini∈Γxi2. If the matrix A^ satisfies RIP of order K+1 with(21)δ^K+1<4K+1-12K,(22)t0>1+δ^K+1+Kε01-δ^K+1-1-δ^K+1Kδ^K+1,where ε0=εA/1/3-1+1/3εA+ε(Y^F/(1-ε)), then GBCD can exactly recover the support set Γ.
Proof.
Consider n=1. The initial value is X(0)=0. In order to guarantee that GBCD selects a correct index i0∈Γ, combining step (4) of Algorithm 1 and (20), we should verify the following inequality: (23)maxi∈Γxi1-xi02(24)=maxi∈Γpi0pi02max0,pi02-λβ2(25)>maxj∈Γcxj1-xj02(26)=maxj∈Γcpj0pj02max0,pj02-λβ2.If pj(0)2-λβ≤0(j∈Γc), the right-hand-side is 0. Then inequality (26) holds. Thus, we only consider pj(0)2-λβ>0. Using Remark 1 in [2], inequality (26) is true when (27)maxi∈Γpi02>maxj∈Γcpj02.
Now, it is sufficient to verify (27). Let us construct an upper bound for maxj∈Γcpj(0)2. By step (3) of Algorithm 1, we have (28)maxj∈Γcpj02(29)=maxj∈Γcxj0-βa^j′A^X0-Y^2(30)=maxj∈Γcβa^j′Y^2(31)=βA^Γc′A^X-EX+N2,∞(32)≤maxj∈Γcβa^j′A^X2+βA^Γc′2,∞-EX+NF(33)=βmaxj∈ΓcA^ej,A^XF+βA^Γc′2,∞-EX+NF(34)≤βδ^K+1A^XF+βε0,where (32) is from the property of norm and (34) follows from each column of A^ which is of unit-norm, Lemmas 3 and 4.
To prove (27), we only need to prove (35)maxi∈Γpi02>βδ^K+1A^XF+βε0.We then go on to show by contradiction that (35) is true. For all i∈Γ, assume that (36)pi02≤βδ^K+1A^XF+βε0.Then we have (37)PΓ0F=∑i∈Tpi022≤βKδ^K+1A^XF+ε0.
Using the triangle inequality, we can get (38)PΓ0F=XΓ0-βA^Γ′A^X0-Y^F=βA^Γ′Y^F(39)≥βA^Γ′A^XF-βA^Γ′-EX+NF(40)≥β1-δ^K+1A^XF-1+δ^K+1ε0,where (40) is from (10) and the property of norm.
After straightforward manipulations, we have (41)PΓ0F≥βKδ^K+1A^XF+ε0+β1-δ^K+1-Kδ^K+1︸≥0A^XF-β1+δ^K+1+Kε0(42)≥βKδ^K+1A^XF+ε0+β1-δ^K+1-Kδ^K+11-δ^K+1XF-β1+δ^K+1+Kε0(43)>βKδ^K+1A^XF+ε0+K-11+δ^K+1ε0+K-Kε0(44)≥βKδ^K+1A^XF+ε0,where (41) follows from (21) and (43) follows from XF≥Kt0 and (22).
Obviously, (44) contradicts (37), so this fact guarantees (27).
Assume that GBCD always picks up indices from the support Γ for n≤k(k≥1 is an integer). Consider n=k+1. In order to prove that GBCD can choose a correct index i0∈Γ, analogous to [2], inequality (46) should be verified.(45)maxi∈Γpin-1-xin-12(46)>maxj∈Γcpjn-1-xjn-12.
Combining step (3) of Algorithm 1 with (46) yields (47)maxi∈Γa^i′A^Xn-1-Y^2(48)>maxj∈Γca^j′A^Xn-1-Y^2.
It is sufficient to prove that (48) holds. Note that supp(X(n-1))⊆Γ; we have (49)maxj∈Γca^j′A^Xn-1-Y^2(50)≤δ^K+1A^Xn-1-XF+ε0.Now, we only need to prove (51)maxi∈Γa^i′A^Xn-1-Y^2(52)>δ^K+1A^Xn-1-XF+ε0.
We then show that (52) is true by contradiction. For all i∈Γ, assume that (53)a^i′A^Xn-1-Y^2≤δ^K+1A^Xn-1-XF+ε0.
Using the definition of Frobenius norm, we have (54)A^Γ′A^Xn-1-Y^F=∑i∈Γa^i′A^Xn-1-Y^22(55)≤Kδ^K+1A^Xn-1-XF+ε0.Combining X(n-1)-XF≥t0, (21), and (22), we have (56)A^Γ′A^Xn-1-Y^F(57)≥1-δ^K+1A^Xn-1-XF-1+δ^K+1ε0(58)=Kδ^K+1A^Xn-1-XF+ε0+1-δ^K+1-Kδ^K+1A^Xn-1-XF-1+δ^K+1+Kε0(59)>Kδ^K+1A^Xn-1-XF+ε0,where (59) follows from (60)1-δ^K+1-Kδ^K+1A^Xn-1-XF(61)>1+δ^K+1+Kε0.
This contradicts (53). Thus, (48) is true.
Remark 6.
The weaker the RIC bound is, the less required number of measurements we need, and the improved RIC results can be used in many CS-based applications [16]. In the work of [2], the authors provided that the condition for GBCD is δK+1<1/(K+1). Obviously, it is smaller than the bound (4K+1-1)/2K in (21).
4. The Counterexample
Consider the measurements (62)Y^=A^-EX+N=A^X-EX+N.In this section, giving a matrix A^, whose RIC is a slight relaxation of 1/K+1, we will verify that GBCD can fail to recover the support of sparse matrix from (62).
Let(63)X=22t022t022t022t0⋮⋮22t022t000K+1×2,E=000⋯00.500⋯0000⋯0⋮⋮⋮⋯⋮000⋯0K+1×K+1,N=0024t024t000⋮⋮0022ϵ22ϵK+1×2,where supp(X)={1,2,…,K}=Γ, Γc={K+1} and t0/ϵ>1+1/K (the value of ϵ is far less than 1; this is reasonable).
The matrix A^ is constructed as(64)A^=a00⋯0s0a0⋯0s00a⋯0s⋮⋮⋮⋮⋮⋮000⋯as000⋯01K+1×K+1,where (65)s=δK,a=1-δ2.
Set (66)δ=1-ϵ/t0K+1.
The eigenvalues λii=1K+1 of A^′A^ are (67)λi=1-δ2,1≤i≤K-1,λK=1-δ,λK+1=1+δ.
Thus, the RIC of A^ is δ-K+1(A^)=δ.
Recall that condition (27) is the criterion of recovery for GBCD. Note that pi(0)2=βa^i′Y^2. One can obtain (68)maxi∈Γa^i′Y^2=22t0a2,22t0a22=a2t0.On the other hand, we have (69)maxj∈Γca^j′Y^2=22t0Kas+22ϵ,22t0Kas+22ϵ2=Kast0+ϵ.
It can be derived that (70)Kast0+ϵ-t0a2=t0aKs-a2+ϵ(71)=t01-δ2Kδ-1+δ2+ϵ(72)>0,where (71) and (72) follow from (65) and (66).
It is obviously in contradiction to (27). Thus, GBCD fails to recover support Γ.
Remark 7.
In the work of [2], the authors presented a matrix A- whose RIC is δK+1(A-)=1/K-ϵ2/K(t02K+ϵ2). They showed that the GBCD algorithm fails when using A- as measurement matrix. After a simple calculation, we can get (73)1-ϵ/t0K+1<1K-ϵ2Kt02K+ϵ2.Thus, our result improves this existing result.
5. Experimental Results
In this section, under the total perturbations, we test the performance of the GBCD algorithm for solving the DOA estimation problem.
Consider K narrowband far-field point source signals impinging on an m-element uniform linear array. The steering vector of the matrix A is (74)ai=1e-jπcosθi-1⋯e-jm-1πcosθi-1′,where 1≤i≤N. L is the number of snapshots.
Using the sparse optimization approach in [1], the DOA estimation problem can be rewritten as model (1). Then the aim is hence to find out which row of the matrix X is nonzero, that is, the support of the matrix X.
Analogous to the simulation of [1], we have the following assumptions:
The number of the array elements is m=11.
The number of snapshots is L=200.
The grid spacing is 1∘ from 0∘ to 180∘. Then N=181.
Five (K=5) uncorrelated signals impinge from θl1=30∘, θl2=80∘, θl3=100∘, θl4=120∘, and θl5=145∘.
Both the signals and the noise are white and follow Gaussian distributions. The power of nonzero entries of X is σ2, and the power of each entry of N is σN2.
Use the following SNR1 and SNR2 to measure noises E and N, respectively: (75)SNR1=10log10σ2σN2.(76)SNR2=AFEF.
Define the root mean square error (RMSE) of 500 Monte Carlo trials as the performance index: (77)RMSE=∑i=1500∑k=1Kθ^lki-θlk2500K,where θ^lk(i) is the estimate of θlk at the ith trial.
Figure 1, fixing matrix E, describes the performance of GBCD. The results show that RMSE decreases as SNR1 increases. Figure 2, fixing matrix N, describes the performance of GBCD. The results show that RMSE decreases as SNR2 increases. Thus, the performance of GBCD still is robust under the total perturbations.
For SNR2 = 10. The RMSE of GBCD versus input SNR1.
For SNR1 = 2. RMSE of GBCD versus input SNR2.
6. Conclusion
In this paper, using the near orthogonality property, we provide a recovery condition for GBCD under the total perturbations. A counterexample is presented to show that GBCD fails. By experiments, we point out that GBCD is robust under the total perturbations.
Conflicts of Interest
The author declares that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported by National Natural Science Foundation of China (nos. 11526081, 11601134, 11671122, and U1404603), the Scientific Research Foundation for Ph.D. of Henan Normal University (no. qd14142), and the Key Scientific Research Project of Colleges and Universities in Henan Province (no. 17A110008).
WeiX.YuanY.LingQ.DOA estimation using a greedy block coordinate descent algorithm201260126382639410.1109/TSP.2012.2218812MR3006427LiH.FuY.HuR.RongR.Perturbation analysis of greedy block coordinate descent under RIP20142155185222-s2.0-8489614137010.1109/LSP.2014.2307116ChangL.-H.WuJ.-Y.An improved RIP-based performance guarantee for sparse signal recovery via orthogonal matching pursuit20146095702571510.1109/TIT.2014.2338314MR3252415BlumensathT.DaviesM.Compressed sensing and source separationProceedings of the International Conference on Independent Component Analysis and Signal Separation (ICA '07)2007341348HermanM. A.StrohmerT.High-resolution radar via compressed sensing20095762275228410.1109/TSP.2009.2014277MR26418232-s2.0-66849132093FannjiangA. C.StrohmerT.YanP.Compressed remote sensing of sparse objects20103359561810.1137/090757034MR2736022Zbl1201.450172-s2.0-78651591980HermanM. A.StrohmerT.General deviants: an analysis of perturbations in compressed sensing20104234234910.1109/jstsp.2009.20391702-s2.0-77949674991DingJ.ChenL.GuY.Perturbation analysis of orthogonal matching pursuit201361239841010.1109/TSP.2012.2222377MR30091332-s2.0-84871800065LiB.ShenY.WuZ.LiJ.Sufficient conditions for generalized Orthogonal Matching Pursuit in noisy case20151081111232-s2.0-8490772111610.1016/j.sigpro.2014.09.006ZhangT.Sparse recovery with orthogonal matching pursuit under RIP20115796215622110.1109/TIT.2011.2162263MR2857968WenJ.ZhuX.LiD.Improved bounds on restricted isometry constant for orthogonal matching pursuit20134923148714892-s2.0-8488905325610.1049/el.2013.2222WuR.HuangW.ChenD.-R.The exact support recovery of sparse signals with noise via orthogonal matching pursuit20132044034062-s2.0-8487496886210.1109/LSP.2012.2233734MoQ.ShenY.A remark on the restricted isometry property in orthogonal matching pursuit20125863654365610.1109/TIT.2012.2185923MR2924391WangJ.ShimB.On the recovery limit of sparse signals using orthogonal matching pursuit20126094973497610.1109/TSP.2012.2203124MR29605782-s2.0-84865209376LiH.MaY.LiuW.FuY.Improved analysis of greedy block coordinate descent under RIP20155164884902-s2.0-8492474756210.1049/el.2014.3756SongC. B.XiaS. T.LiuX. J.Improved analysis for subspace pursuit algorithm in terms of restricted isometry constant201421111365136910.1109/LSP.2014.23367332-s2.0-84904708801