This paper establishes new sufficient conditions on the restricted isometry property (RIP) for compressed sensing with coherent tight frames. One of our main results shows that the RIP (adapted to D) condition δk+θk,k<1 guarantees the stable recovery of all signals that are nearly k-sparse in terms of a coherent tight frame D via the l1-analysis method, which improves the existing ones in the literature.
National Natural Science Foundation of China1150144061673015621730201. Introduction
Compressed sensing (CS) has received much recent attention in many fields, for example, information science, electrical engineering, and statistics [1–10]. A key problem in CS is to recover a nearly sparse signal from considerably fewer linear measurements. Typically, it takes the following model:(1)y=Φf+z,where y∈Rm is a vector of observed measurements, f∈Rn is an unknown signal needed to be estimated, Φ∈Rm×n is a given measurement matrix, and z∈Rm is a vector of measurement errors. Considering the fact that f is sparse or nearly sparse in terms of an orthogonal basis, one straightforward approach is to find the sparsest solution of (1) by l0 minimization. However, it is well known that solving an l0 minimization problem directly is NP-hard in general and thus is computationally infeasible for even moderate size setting [11].
To efficiently estimate f in the high-dimensional setting, the most popular strategy is to replace the l0 norm with its closest convex surrogate, the l1 norm, which leads to the following l1 norm minimization:(2)minf∈Rnf1subject toΦf-y2∈B,wheref1=∑i=1nfi and B is a bounded set determined by the noise structures. It is obvious that (2) is a convex optimization problem and thus can be solved efficiently in polynomial time. Therefore, the l1 minimization method (2) has been widely used in compressed sensing and other related problems.
There are a number of practical applications in signal and image processing point to problems where signals are not sparse in terms of an orthogonal basis but in terms of an overcomplete and tight frame (see [12–16], and the references therein). In such contexts, the signal f can be expressed as f=Dx, where x∈Rd is a sparse or nearly sparse vector. One natural way to recover f is first solving the l1 minimization problem (2) with the decoding matrix ΦD instead of Φ to find the sparse transform coefficients x^ and, then, reconstructing the signal f by a synthesis operation, that is, f^=Dx^. This is the so-called l1-synthesis or synthesis based method. Since the entries of ΦD are correlated when D is highly coherent, ΦD may no longer satisfy the standard restricted isometry property (RIP [1]) and the mutual incoherence property (MIP [17]) which are commonly used in the standard CS framework. Thus, it would be difficult to characterize the theoretical performance of l1-synthesis method under the CS framework.
An alternative to the l1-synthesis method is the l1-analysis method, which finds the estimator f^ directly by solving the following l1 minimization problem:(3)minf∈RnD∗f1subject toΦf-y2∈B.It has been shown in [14] that there is a remarkable difference between the two typical methods despite their apparent similarity. To investigate the theoretical performance of the l1-analysis method, Candès et al. in [13] introduced the definition of D-RIP: A measurement matrix Φ is said to satisfy the restricted isometry property adapted to D (abbreviated D-RIP) with constant δk if(4)1-δkDx22≤ΦDx22≤1+δkDx22holds for every vector x∈Rd that is k-sparse. Note that it is a natural generalization of the RIP introduced in [1]. Similarly, it is also computationally difficult to verify the D-RIP for a given deterministic matrix. But as discussed in [13], the matrices which satisfy the standard RIP requirements will also satisfy the D-RIP requirements. Many previous works have tried to derive sufficient conditions on δck(c>0) for stable recovery of nearly sparse (in terms of D) signals via l1-analysis. Candès et al. first presented conditions δ2k<0.08 and δ7k<0.6 [13]. Then, the conditions 9δ2k+4δ4k<5 and δ2k<0.4931 were used in [18] and [19], respectively. In the recent literature, S. Lin and J. Lin [19] extended the notion of restricted orthogonality constant (ROC) used in standard CS to the setting of CS with coherent tight frames. The D-restricted orthogonality constant (D-ROC) of order k1, k2, θk1,k2 is defined to be the smallest positive number satisfying(5)ΦDx1,ΦDx2-Dx1,Dx2≤θk1,k2x12x22for every x1 and x2 such that x1 and x2 are k1-sparse and k2-sparse, respectively. With this new notion, they extended some sufficient conditions which appeared in standard CS to the setting of CS with coherent tight frames, such as δk+1.25θk,k<1, δ8/7k+θ8/7k,8/7k<1, and δ1.25k+θk,1.25k<1. Moreover, they also obtained that δk<0.307, which is the first sufficient condition on δk, is sufficient for l1-analysis to guarantee the stable recovery of nearly k-sparse (in terms of D) signals. In a recent paper [20], the condition δk<0.307 was improved to δk<1/3.
Along the lines of [8, 19], we establish in this paper more relaxed RIP conditions for stable recovery of nearly sparse (in terms of D) signals from incomplete and contaminated data. Specifically, the main contribution of this paper is to show that, under the RIP condition δk+θk,k<1, any signal that is nearly sparse in terms of D can be recovered stably from its noisy measurements by solving the l1-analysis problem (3). To show these new conditions, in Section 2, we shall introduce a key technique tool which is an extension of Lemma 5.1 in [8] and also state two lemmas that appeared in [8]. In Section 3, we will establish our new results by use of some proof ideas for the standard CS in [8]. Furthermore, we will show that the condition δk+θk,k<1 is mostly weaker than the best known condition δk<1/3.
2. Preliminaries
In this section, we first state two useful lemmas that appeared in [8], which reveal the relationship between D-RIC and D-ROC, and the relationship between D-ROCs of different orders, respectively.
Lemma 1.
For positive integers k1, k2≤d, we have (6)θk1,k2≤δk1+k2.
Lemma 2.
For any b≥1 and positive integers k1, k2 such that bk2 is an integer, we have (7)θk1,bk2≤bθk1,k2.
In the following, we will introduce and prove a key technical tool, which will be very useful for proving our main results.
Lemma 3.
Let k1 and k2 and η≥0. Suppose c1, c2∈Rd and c1 is a k1-sparse vector. If c21≤ηk2 and c2∞≤η, then we have(8)ΦDc1,ΦDc2-Dc1,Dc2≤θk1,k2c12·ηk2.
Proof.
We shall prove it by mathematical induction. Suppose the size of support of c2 is l, that is, c20=l. For l≤k2, by the definition of θk1,k2, we have(9)ΦDc1,ΦDc2-Dc1,Dc2≤θk1,k2c12c22≤θk1,k2c12lc2∞2≤θk1,k2c12·ηk2.Thus, (8) holds for l≤k2.
For the case l>k2, we first assume that (8) holds for l-1. The following discussion will use the same argument as in [8]. But for completeness, we will include the sketch. Now for l, we write c2 as c2=∑i=1lαiΔi, where α1≥α2≥⋯≥αl>0, and Δii=1l are “indicator vectors” with different supports. A vector is called an “indicator vector” if it has only one nonzero entry and the value is either 1 or -1. Since ∑i=1lαi≤ηk2≤l-1η, the set (10)Ω≜1≤j≤l-1:αj+αj+1+⋯+αl≤l-jηis not empty. Now we choose the largest element j∈Ω, which means(11)αj+αj+1+⋯+αl≤l-jη,αj+1+αj+2+⋯+αl>l-j-1η.Define(12)βω=∑i=jlαil-j-αω,j≤ω≤l,rω=βω∑i=jlβi∑i=1j-1αiΔi+∑i=j,i≠ωlαiΔi,j≤i≤l.It is not hard to check that ∑ω=jlrω=c2, ∑i=jl=l-j∑i=jlβi. Similar to the proof of lemma 5.1 in [8], we also have(13)βω≥βj>0,j≤ω≤l,rω1≤βω∑i=jlβiηk2,rω∞≤βω∑i=jlβiη.From the definition of rω, we obtain that rωis (l-1)-sparse. Finally, using the induction assumption, we get(14)ΦDc1,ΦDc2-Dc1,Dc2≤∑ω=jlΦDc1,ΦDrω-Dc1,Drω≤∑ω=jlθk1,k2c12βω∑i=jlβiηk2=θk1,k2c12·ηk2,which arrives to the conclusion of Lemma 3.
Remark 4.
When D is an identity matrix and c1, c2 have disjoint supports, Lemma 3 has essentially the same result as lemma 5.1 in [8].
3. Improved RIP Conditions
We now consider the stable recovery of nearly sparse (in terms of D) signals via the l1-analysis method (3). We will present some new RIP conditions under two bounded noise settings: B1(ε)={r:r2≤ε}andB2(ε)={r:D∗Φ∗r∞≤ε}.Throughout the paper, x[k]is the vector with all but the k largest absolute entries of x set to zero, and x-[k]=x-x[k]. The following theorem represents our main result.
Theorem 5.
Let D be a given tight frame, and Ca,b,k=max2k-a/ab,2k-a/a. If the measurement matrix Φ satisfies the D-RIP condition with(15)δa+Ca,b,kθa,b<1for some positive integers a and b with 1≤a≤k, then the solution f^ to (3) obeys
When f is exact k-sparse in terms of D and no noise is present, the solution f^ to (3) is equal to f; that is to say, the recovery is exact.
Remark 7.
As reported in [8], when D=I, the bound 1 is sharp in the sense that, for any ε>0, the D-RIP condition δa+Ca,b,kθa,b<1+ε does not guarantee such exact recovery. But it is still open whether this bound is also sharp when D is not identity matrix. We leave it to the interested readers.
Proof.
Let h=f^-f, where f is the original signal and f^ is the solution to (3). As noted in [15, 18], different from the proof in [8] for standard compressed sensing, we need to develop bounds on D∗h2 instead of h2. We write D∗h=∑i=1dμiΔi, where μ1≥μ2≥⋯≥μd≥0 and {Δi}i=1d are indicator vectors (we have mentioned in the proof of Lemma 3) with different support. In the following, we will use some proof ideas from [8].
By the fact that f^ is a minimizer of (3), we can easily get the following inequality (see [5, 6, 19]):(18)∑i=k+1dμi≤∑i=1kμi+2D∗f-k1.Thus,(19)D∗h-a∞=μa+1≤∑i=1aμia≤∑i=1aμia+2D∗f-k12k-a,(20)D∗h-a1=∑i=a+1kμi+∑i=k+1dμi≤∑i=a+1kμi+∑i=1kμi+2D∗h-k1.Note that(21)∑i=a+1kμi≤k-aμa+1≤k-aa∑i=1aμi;∑i=1kμi=∑i=1aμi+∑i=a+1kμi≤∑i=1aμi+k-aa∑i=1aμi=ka∑i=1aμi.It then follows with (20) that(22)D∗h-a1≤k-aa∑i=1aμi+ka∑i=1aμi+2D∗h-k1=2k-aaD∗ha1+2D∗h-k1.Applying Lemma 3 with η=D∗ha1/a+2D∗h-k1/2k-a, k1=a and k2=2k-a yield (23)ΦDD∗ha,ΦDD∗h-a-DD∗ha,DD∗h-a≤θa,2k-a2k-aD∗ha2×D∗ha1a+2D∗h-k12k-a.Hence, (24)Φh,ΦDD∗ha=ΦDD∗ha,ΦDD∗ha+ΦDD∗h-a,ΦDD∗ha≥1-δaDD∗ha22-θa,2k-a2k-aD∗ha2×D∗ha1a+2D∗h-k12k-a+DD∗ha,DD∗h-a.By plugging the equality(25)DD∗ha,DD∗h-a=DD∗ha,h-DD∗ha=DD∗ha,DD∗h-DD∗ha22=Dha22-DD∗ha22into the above inequality, we obtain(26)Φh,ΦDD∗ha≥Dha22-δaDD∗ha22-θa,2k-a2k-aD∗ha2×D∗ha1a+2D∗h-k12k-a≥1-δa-θa,2k-a2k-aaD∗ha22-θa,2k-aD∗ha22D∗h-k12k-a.On the other hand, for l2 bounded noise, we first have(27)Φh,ΦDD∗ha≤Φh2ΦDD∗ha2≤2ε1+δaDD∗ha2≤2ε1+δaD∗ha2,where we have used the following inequality:(28)Φh2≤Φh-f2+f-Φh2≤2ε.Therefore,(29)D∗ha2≤2ε1+δa1-δa-θa,2k-a2k-a/a+θa,2k-a1-δa-θa,2k-a2k-a/a2D∗h-k12k-a.It is known thath22=D∗h22; thus, now we shall boundD∗h2. By lemma 5 in [7], we have(30)∑j=r+1mcj2≤r∑i=1rci2r+λr2,if c1≥c2≥⋯≥cm≥0, and ∑i=1rci+λ≥∑i=r+1mci with λ≥0. Then, set λ=2D∗h-k1 in (30) and combine it with (18); we further get (31)D∗h2=∑i=1kμi2+∑i=k+1dμi2≤∑i=1kμi2+k∑i=1kμi2+2D∗h-k1k2≤2∑i=1kμi2+2D∗h-k1k≤2ka∑i=1aμi2+2D∗h1k≤2ε21+δak/a1-δa-θk,2k-a2k-a/a+θa,2k-a1-δa-θa,2k-a2k-a/a22k-a+2k·D∗h-k1,where the last inequality we have used is (29). From Lemma 2, it is not hard to get (32)θa,2k-a≤2k-aminb,2k-aθa,minb,2k-a≤max2k-ab,1θa,b=a2k-aCa,b,kθa,b.Therefore, (33)h2≤2ε21+δak/a1-δa-Ca,b,kθa,b+2D∗f-D∗fk1×2kCa,b,kθa,b1-δ-Ca,b,kθa,b2k-a+1k,which arrives to the conclusion.
Forl∞bounded noise, we first have (34)D∗Φ∗Φh∞≤D∗Φ∗y-Φh^∞+D∗Φ∗Φh-y∞≤2ε.Hence,(35)Φh,ΦDD∗ha=D∗Φ∗Φh,Dha≤D∗Φ∗Φh∞Dha1≤2εaDha2.Then, the following proof is essentially the same, where we only need to replace (27) with (35).
To be noted, if we choosea=b=kin Theorem 5, we can naturally get the following result.
Corollary 8.
If the measurement matrix satisfies δk+θk,k<1, then the solution f^ of (3) satisfies (16) and (17). In particular, if the original signal f is exact k-sparse in terms of D, the recovery is exact in the noiseless case.
Remark 9.
It is obvious that the obtained condition δk+θk,k<1 is weaker than δk+1.25θk,k<1,δ8/7k+θ8/7k,8/7k<1, and δ1.25k+θk,1.25k<1 which were used in [15].
Since most of the sufficient recovery conditions in the literature are based on D-RIC δ alone, it would be interesting to compare these conditions with δk+θk,k<1. For this, we shall present the following lemma which provides the D-ROC θ in terms of D-RIC δ.
Lemma 10.
Let D be a given tight frame; then, D-ROC θk,k and D-RIC δk of the measurement matrix Φ satisfy (36)θk,k≤2δk,when k is even,k≥2,2kk2-1δk,when k is odd,k≥3.
Proof.
The proof is similar to the proof of lemma 3.1 in [8]. For simplicity, we only present the proof sketch in the following. For two k-sparse (in terms of D) signals f1, f2∈Rn, we can write Df1 and Df2 as (37)Df1=∑i∈T1ciei,Df2=∑i∈T2diei,where ci>0, di>0, T1 is the support of D∗f1, T2 is the support of D∗f2, and ei is the vector whoseith entry equals ±1 and all the other ones equal zero.
Case 1 (k≥2 is even). Without loss of generality, suppose f1 and f2 are normalized such that Df12=Df22=1. Divide T1 and T2 into two subsets such that T1=T11∪T12, T2=T21∪T22, and T11,T12,T21,T22 are disjoint and Tij≤k/2 for i,j∈1,2. Denote (38)Df1i=∑i∈T1iciei,Df2=∑i∈T2idiei,i=1,2.Then, by the definition of D-RIP (4), we have(39)ΦDf1,ΦDf2=∑i,j=12ΦDf1i,ΦDf2j=14∑i,j=12ΦDf1i+ΦDf2j22-ΦDf1i-ΦDf2j22≤14∑i,j=121+δkDf1i+Df2j22-1-δkDf1i-Df2j22=14∑i,j=124δkDf1i22+Df2j22+4Df1i,Df2j=∑i,j=12δkci2+di2+∑i,j=12Df1i,Df2j=2δk+Df1,Df2.Similarly, we have(40)ΦDf1,ΦDf2≥-2δk+Df1,Df2.Then, from (39) and (40), we have (41)ΦDf1,ΦDf2-Df1,Df2≤2δkDf12Df22.Thus,θk,k≤2δk.
Case 2 (k≥3 is odd). Without loss of generality, suppose |T1|=k,|T2|=k, and ci,di might be 0 for i∈T1∪T2. Also we can assume Df1 and Df2 are normalized such that Df12=∑i∈T1ci2=k-1/k+1 and Df22=∑i∈T2di2=k+1/k-1. Then, we have (42)4k-1k-12k-1k-32ΦDf1,ΦDf2=4k-1k-12k-1k-32Φ∑i∈T1ciei,Φ∑i∈T2diei=∑A⊆T1,A=k+1/2B⊆T2,T2=k-1/2Φ∑i∈Aciei+∑i∈Bdiei22-Φ∑i∈Aciei-∑i∈Bdiei22≤∑A⊆T1,A=k+1/2B⊆T2,T2=k-1/22δk∑i∈Aci2+∑i∈Bdi2+4∑i∈A∑j∈Bci,dj=2δkk-1k-12kk-12∑i∈T1ci2+k-1k-32kk+12∑j∈T2dj2+4k-1k-12k-1k-32∑i∈T1∑j∈T2ci,dj=2δkk-1k-12k-1k-32kk-1/2∑i∈T1ci2+kk+1/2∑j∈T2dj2+4k-1k-12k-1k-32∑i∈T1∑j∈T2ci,dj=4k-1k-12k-1k-322kk2-1δkDf12Df22+Df1,Df2.Similarly, we have(43)4k-1k-12k-1k-32ΦDf1,ΦDf2≥4k-1k-12k-1k-32-2kk2-1δkDf12Df22+Df1,Df2.Then, from (42) and (43), we have (44)ΦDf1,ΦDf2-Df1,Df2≤2kk2-1δkDf12Df22,which implies θk,k≤2k/k2-1δk.
The following results can be directly obtained from the above Theorem 5 and Lemma 10.
Corollary 11.
For some integer k≥2, if δk<1/3, then we have (45)θk,k+δk<1,when k is even;θk,k+δk<13+2k3k2-1<1+13k2,when k is odd.
Remark 12.
As stated in [20], δk<1/3 is the best condition for stable recovery of nearly sparse signal in terms of D via l1-analysis. Based on Corollary 11, our presented condition θk,k+δk<1 is mostly weaker than δk<1/3. Basically, there are several benefits to weaken the D-RIP condition. For example, using a standard covering argument as in [21], it is easy to show that, for any positive integer m<n and 0<t<1, the D-RIC δm of a Gaussian or Bernoulli random measurement matrix satisfies(46)Pδk<t≥1-212epmtmexp-nt216-t348.
Note that θk,k+δk<1 is implied by δ2k+δk<1 which is further implied by the conditions δk<0.4 and δ2k<0.6. Hence, using (46) and following the discussion of Section IV in [8], the number of measurements n should satisfy (47)n≥162klogpk+4.6-logε2to ensure the condition δk<1/3 holds with probability at least 1-ε. Similarly, δk+δ2k<1 holds with probability at least 1-ε if the number of measurements n satisfies n≥maxn1,n2 with (48)n1=115.4klogpk+4.4-logε4guaranteeing δk<0.4 with probability at least 1-ε/2, and with(49)n2=111.1klogpk+3.3-logε/42guaranteeing δ2k<0.6 with probability at least 1-ε/2. Therefore, for large k and p, the size requirement to ensure θk,k+δk<1 is less than 71.2% (115.4/162) of the corresponding size requirement to ensure δk<1/3. This clearly demonstrates the advantage of our presented condition θk,k+δk<1 over the best known condition δk<1/3.
4. Conclusion
Under the framework of CS with coherent tight frames, we present in this paper some improved RIP conditions for stable recovery of nearly sparse (in terms of D) signals via l1-analysis method, which are weaker than the existing ones. Although only convex optimization method is considered here, it would be also interesting to relax the RIP conditions for nonconvex optimization method. It is known that standard lq(0<q<1) minimization method could recover conventional nearly sparse signals stably under weaker RIP conditions than standard l1 minimization method [22–24]. As such, one may make effort to weaken the RIP condition for nonconvex lq(0<q<1)-analysis method, thus facilitating the further use of nonconvex analysis based method for more practical CS scenarios.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported by Natural Science Foundation of China under Grant nos. 11501440, 61673015, and 62173020.
CandèsE. J.TaoT.Decoding by linear programming200551124203421510.1109/TIT.2005.858979MR2243152CandèsE. J.RombergJ.TaoT.Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information200652248950910.1109/TIT.2005.862083MR2236170DonohoD. L.Compressed sensing20065241289130610.1109/TIT.2006.871582MR22411892-s2.0-33645712892CandesE. J.TaoT.Near-optimal signal recovery from random projections: universal encoding strategies?200652125406542510.1109/TIT.2006.885507MR2300700Zbl1309.940332-s2.0-33947416035CaiT. T.WangL.XuG.New bounds for restricted isometry constants20105694388439410.1109/TIT.2010.2054730MR28073352-s2.0-77955767521CaiT. T.XuG.ZhangJ.On recovery of sparse signals via ℓ1 minimization20095573388339710.1109/TIT.2009.2021377MR2598028CaiT. T.ZhangA.Sharp RIP bound for sparse signal and low-rank matrix recovery2013351749310.1016/j.acha.2012.07.010MR3053747CaiT. T.ZhangA.Compressed sensing and affine rank minimization under restricted isometry201361133279329010.1109/TSP.2013.2259164MR30703212-s2.0-84879055175GrasmairM.Non-convex sparse regularisation20103651192810.1016/j.jmaa.2009.09.055MR25850712-s2.0-73449088688LustigM.SantosJ. M.LeeJ. H.DonohoD. L.PaulyJ. M.Application of compressed sensing for rapid MR imagingProceedings of the SPARS2005Rennes, FranceNatarajanB. K.Sparse approximate solutions to linear systems199524222723410.1137/S0097539792240406MR1320206Zbl0827.680542-s2.0-0029291966RauhutH.SchnassK.VandergheynstP.Compressed sensing and redundant dictionaries20085452210221910.1109/TIT.2008.920190MR24508582-s2.0-43749118418CandèsE. J.EldarY. C.NeedellD.RandallP.Compressed sensing with coherent and redundant dictionaries2011311597310.1016/j.acha.2010.10.002MR2795875EladM.MilanfarP.RubinsteinR.Analysis versus synthesis in signal priors200723394796810.1088/0266-5611/23/3/007MR23299262-s2.0-34249671661LinJ.LiS.ShenY.New bounds for restricted isometry constants with coherent tight frames201361361162110.1109/TSP.2012.2226171MR30183162-s2.0-84872338171LinJ.LiS.Sparse recovery with coherent tight frames via analysis Dantzig selector and analysis LASSO201437112613910.1016/j.acha.2013.10.003MR3202305DonohoD. L.EladM.TemlyakovV. N.Stable recovery of sparse overcomplete representations in the presence of noise200652161810.1109/TIT.2005.860430MR2237332Zbl1288.940172-s2.0-33144483155LiuY.MiT.LiS.Compressed sensing with general frames via optimal-dual-based ℓ1-analysis20125874201421410.1109/TIT.2012.2191612MR2943084LiS.LinJ.Compressed sensing with coherent tight frames via ℓq-minimization for 0 < q ≤ 120148376177710.3934/ipi.2014.8.7612-s2.0-84906907180ZhangR.LiS.Optimal D-RIP bounds in compressed sensing201531575576610.1007/s10114-015-4234-4MR3333169BaraniukR.DavenportM.DeVoreR.WakinM.A simple proof of the restricted isometry property for random matrices200828325326310.1007/s00365-007-9003-xMR24533662-s2.0-55649115527ChartrandR.Exact reconstruction of sparse signals via nonconvex minimization200714107077102-s2.0-3454872443710.1109/LSP.2007.898300SunQ.Recovery of sparsest signals via ℓq-minimization201232332934110.1016/j.acha.2011.07.001MR2892737SongC.-B.XiaS.-T.Sparse signal recovery by ℓq minimization under restricted isometry property20142191154115810.1109/LSP.2014.23232382-s2.0-84902190186