AAA Abstract and Applied Analysis 1687-0409 1085-3375 Hindawi Publishing Corporation 482357 10.1155/2013/482357 482357 Research Article A Sharp RIP Condition for Orthogonal Matching Pursuit Dan Wei Pereverzyev Sergei V. School of Mathematics and Statistics Guangdong University of Finance & Economics Guangzhou 510320 China gdufs.edu.cn 2013 19 11 2013 2013 03 07 2013 14 09 2013 05 10 2013 2013 Copyright © 2013 Wei Dan. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

A restricted isometry property (RIP) condition δK+KθK,1<1 is known to be sufficient for orthogonal matching pursuit (OMP) to exactly recover every K-sparse signal x from measurements y=Φx. This paper is devoted to demonstrate that this condition is sharp. We construct a specific matrix with δK+KθK,1=1 such that OMP cannot exactly recover some K-sparse signals.

1. Introduction

Compressive sampling (compressed sensing, CS) is known as a new type of sampling theory that one can reconstruct a high dimensional spare signal from a small number of linear measurements at the sub-Nyquist rate . Nowadays, the CS technique has attracted considerable attention from across a wide array of fields like applied mathematics, statistics, and engineering, including signal processing areas such as MR imaging, speech processing, and analog to digital conversion. The basic problem in CS is to reconstruct the unknown sparse signal x from measurements: (1)y=Φx, where Φ is an M×N(MN) sampling matrix. Suppose Φ=(Φ1,Φ2,,ΦN), where Φi denotes the ith column of Φ. Throughout the paper, we will assume that the columns of Φ are normalized; that is, Φi2=1 for i=1,2,,N.

It is well understood that under some assumptions on the sampling matrix Φ, the unknown sparse signal x can be reconstructed by solving the l0-minimization problem: (2)minx0subjecttoy=Φx, where x0 denotes the number of nonzero entries of x. We say a signal x is K-sparse when x0K.

However, the optimization problem is NP-hard, so one seeks computationally efficient algorithms to approximate the sparse signal x, such as greedy algorithm, l1 minimization, and lp(0<p<1) minimization .

Orthogonal matching pursuit (OMP), which is a canonical greedy algorithm, has receive much attention in solving the problem (2), due to its ease of implementation and low complexity. Algorithm 1 can be described below. Until recently, many popular generalizations of OMP are introduced, for example, OMMP and KOMP; for details, see [7, 8].

<bold>Algorithm 1: </bold>Orthogonal matching pursuit—OMP (<inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M27"><mml:mi mathvariant="normal">Φ</mml:mi><mml:mo>,</mml:mo><mml:mi>y</mml:mi></mml:math></inline-formula>).

Input: Sampling matrix Φ, observation y

Output: Reconstructed sparse vector x* and index set

INITIALIZATION: Let the index set Ω0= and the residual r0=y. Let the iteration counter t=1.

IDENTIFICATION: Choose the index i subject to |ΦiTrt-1|>maxji|ΦjTrt-1|.

UPDATE: Add the new index i to the index set: Ωt=Ωt-1i, and update the signal and the residual

xt|Ωt=arg  minz||y-ΦΩtz||2,xt|Ω-t=0;

rt=y-Φxt.

If rt=0, stop the algorithm. Otherwise, update the iteration counter t=t+1 and return to Step IDENTIFICATION.

The mutual incoherence property (MIP) introduced in  is an important tool to analyze the performance of OMP. The MIP requires the mutual coherence μ of the sampling matrix Φ to be small, where μ is defined as (3)μ=maxij|ΦiTΦj|. Tropp has shown that the MIP condition (2K-1)μ<1 is sufficient for OMP to exactly recover every K-sparse signal . This condition is proved to be sharp in .

The restricted isometry property (RIP) is also widely used in studying a large number of algorithms for sparse recovery in CS, which is introduced in . A matrix Φ satisfies the RIP of order K with the restricted isometry constant (RIC) δK if δK is the smallest constant such that (4)(1-δK)x22Φx22(1+δK)x22 holds for all K-sparse signal x. A related quantity of the restricted orthogonality constant (ROC) θK,K is defined as the smallest quantity such that (5)|Φx,Φx|θK,Kx2·x2 holds for all disjoint support K-sparse signal x and K-sparse signal x. It is first shown by Davenport and Wakin that the RIP condition (6)δK+1<13K can guarantee that OMP will exactly recover every K-sparse signal . The sufficient condition is then improved to δK+1<1/(1+2K) , δK+1<1/2K , δK+1<1/(1+K) , and δK+KθK,1<1 [15, 16]. By contrast, Mo and Shen have given a counterexample, a matrix with δK+1=1/K where OMP fails for some K-sparse signals . The main result of this note is to show that the sufficient RIP condition (7)δK+KθK,1<1 is sharp for OMP.

2. Main Result Theorem 1.

For any given positive integer K1, there exist a K-sparse signal x and a matrix Φ with the restricted isometry constant (8)δK+KθK,1=1 for which OMP fails in K iterations.

Proof.

For any given positive integer K1, let (9)Φ=(Φij)(2K-1)×(2K-1), where (10)Φij={0(i<j),2K2K-1·(-ii(i+1))(i=j),2K2K-1·1i(i+1)(i>j).

By simple calculation, we can get (11)Φj22=2K2K-1·(j2j(j+1)+i=j+12K-11i(i+1))=2K2K-1·(jj+1+1j+1-1j+2++12K-1-12Kjj+1+1j+1-1j+2)=1,Φl,Φj=2K2K-1·(-jj(j+1)+i=j+12K-11i(i+1))=2K2K-1·(-1j+1+1j+1-1j+2++12K-1-12K-1j+1+1j+1-1j+2)=-12K-1 for any integers 1l<j2K-1.

Thus, for any index set Λ whose cardinality is K, we have (12)ΦΛTΦΛ=(1-12K-1-12K-1-12K-1-12K-11). It is obvious that the eigenvalues {λi}i=1K of ΦΛTΦΛ are (13)λ1==λK-1=1+12K-1,λK=1-K-12K-1. Therefore, the restricted isometry constant δK of Φ is (K-1)/(2K-1).

Now, we turn to calculate the restricted orthogonality constant θK,1. In view of (11), we may, without loss of generality, assume that x=(x1,,xK,0,,0)T and x=(0,,0,xK+10,,0)T. We have (14)θK,1=max|Φx,Φx|x2·x2=max|Φx,ΦK+1|x2=max12K-1·|i=1Kxi|x2=K2K-1. The last equality holds when x1==xK. It is easy to check that (15)δK+KθK,1=K-12K-1+K·K2K-1=1.

Let x=(1,,1K,0,,0)TR2K-1; we have (16)|Sj|=|Φx,Φj|=K2K-1,j{1,2,,2K-1}. This implies that OMP fails in the first iteration. The proof is complete.

3. Discussion

In this paper, we showed that the RIP condition δK+KθK,1<1 is sharp for orthogonal matching pursuit to exactly recover every K-sparse signal x from measurements y=Φx. It is worth discussing the relations between our sharp RIP condition and that in two relative papers [10, 17]. First of all, it follows from the facts that δK<(K-1)μ and θK,1<Kμ that the sharp RIP condition δK+KθK,1<1 in this paper is weaker than the sharp MIP condition (2K-1)μ<1 in . Moreover, our result is also stronger than the previous RIP condition. The condition δK+KθK,1<1 in this paper is necessary and sufficient for OMP, while the previous necessary RIP condition δK<1/K in  is not sufficient. Therefore, the result in the paper may guide the practitioners to apply OMP properly in sparse recovery.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant nos. 11271060, U0935004, U1135003, 11071031, and 11290143, 11101096), the Guangdong Natural Science Foundation (Grant no. S2012010010376), and the Guangdong University and Colleges Technology Innovation Projects (Grant no. 2012KJCX0048).

Candès E. J. Romberg J. Tao T. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information IEEE Transactions on Information Theory 2006 52 2 489 509 10.1109/TIT.2005.862083 MR2236170 Donoho D. L. Compressed sensing IEEE Transactions on Information Theory 2006 52 4 1289 1306 10.1109/TIT.2006.871582 MR2241189 Xu Z. Compressed sensing: a survey Scientia Sinica Mathematica 2012 42 9 865 877 10.1360/012011-1043 Donoho D. L. Elad M. Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization Proceedings of the National Academy of Sciences of the United States of America 2003 100 5 2197 2202 10.1073/pnas.0437847100 MR1963681 Saab R. Yılmaz Y. Sparse recovery by non-convex optimization—instance optimality Applied and Computational Harmonic Analysis 2010 29 1 30 48 10.1016/j.acha.2009.08.002 MR2647010 Tropp J. A. Greed is good: algorithmic results for sparse approximation IEEE Transactions on Information Theory 2004 50 10 2231 2242 10.1109/TIT.2004.834793 MR2097044 Maleh R. Improved RIP analysis of orthogonal matching pursuit 2011, http://arxiv.org/abs/1102.4311 Xu Z. The performance of orthogonal multi-matching pursuit under RIP 2012, http://arxiv.org/abs/1210.5323 Donoho D. L. Huo X. Uncertainty principles and ideal atomic decomposition IEEE Transactions on Information Theory 2001 47 7 2845 2862 10.1109/18.959265 MR1872845 Cai T. T. Wang L. Xu G. Stable recovery of sparse signals and an oracle inequality IEEE Transactions on Information Theory 2010 56 7 3516 3522 10.1109/TIT.2010.2048506 MR2799010 Candes E. J. Tao T. Decoding by linear programming IEEE Transactions on Information Theory 2005 51 12 4203 4215 10.1109/TIT.2005.858979 MR2243152 Davenport M. A. Wakin M. B. Analysis of orthogonal matching pursuit using the restricted isometry property IEEE Transactions on Information Theory 2010 56 9 4395 4401 10.1109/TIT.2010.2054653 MR2807336 Huang S. Zhu J. Recovery of sparse signals using OMP and its variants: convergence analysis based on RIP Inverse Problems 2011 27 3 14 035003 10.1088/0266-5611/27/3/035003 MR2772522 Liu E. Temlyakov V. N. The orthogonal super greedy algorithm and applications in compressed sensing IEEE Transactions on Information Theory 2012 58 4 2040 2047 10.1109/TIT.2011.2177632 MR2951314 Dan W. Wang R. H. Robustness of orthogonal matching pursuit under restricted isometry property Science China Mathematics 2013 56 10.1007/s11425-013-4655-4 Wang J. Shim B. Improved recovery bounds of orthogonal matching pursuit using restricted isometry property 2012, http://arxiv-web3.library.cornell.edu/abs/1211.4293v1 Mo Q. Shen Y. A remark on the restricted isometry property in orthogonal matching pursuit IEEE Transactions on Information Theory 2012 58 6 3654 3656 10.1109/TIT.2012.2185923 MR2924391