Approximately Normalized Iterative Hard Thresholding for Nonlinear Compressive Sensing

The nonlinear compressive sensing (NCS) is an extension of classical compressive sensing (CS) and the iterative hard thresholding (IHT) algorithm is a popular greedy-type method for solving CS. The normalized iterative hard thresholding (NIHT) is a modification of IHT and is more effective than IHT. In this paper, we propose an approximately normalized iterative hard thresholding (ANIHT) algorithm for NCS by using the approximate optimal stepsize combining with Armijo stepsize rule preiteration. Under the condition similar to restricted isometry property (RIP), we analyze the condition that can identify the iterative support sets in a finite number of iterations. Numerical experiments show the good performance of the new algorithm for the NCS.

There is another greedy method, iterative hard thresholding (IHT) algorithm for problem (4), which was proposed by

Algorithm
In this section, we will present the approximately normalized iterative hard thresholding (ANIHT) algorithm for (2) and then analyze its convergence properties.Denote For  ∈ R  and set  ⊆ R  , projector onto  is   () ≜ arg min ∈ ‖ − ‖.Note that the projection onto sparse set , written as   (⋅), sets all but  largest absolute value components of  to zero.The definition of -stationarity was proposed in [17] based on the notion of fixed-point equation.
Note that  * ∈  is the stationary point of problem (2) if and only if [17] where M  () denotes the th largest element in absolute value of  ∈  and In NIHT for CS [16], to guarantee the objective function a sufficient decrease per iteration, the authors added a stepsize strategy based on restricted isometry property (RIP) [1].In ANIHT for NCS (2), we use the approximately optimal stepsize to accelerate the convergence and use Armijo-type stepsize and make a sufficient decrease of the objective function directly without RIP.Here (⋅) Γ is the subvector (submatrix) obtained by discarding all but the elements (columns) in Γ.The framework of ANIHT is described as follows.
), and   =  0  ; else compute and Γ +1 = supp( +1 ), where   =  ( This stepsize is in accordance with the optimal stepsize in NIHT for CS in [16].Furthermore, by Assumption 4, when  is relatively small, the error introduced in the linear approximation is small; then the objective function decreases if the support set is not changed. (ii) Armijo-type stepsize rule in Step 3 makes the choice of stepsize and support set adaptively and a sufficient decrease of the objective function meanwhile per iteration.It is well defined by Lemma 6.
The following assumptions are chosen to ensure the descent property (14) of the objective function (1/2)‖()‖ 2 .Assumption 3.There exists a constant We also need the assumption that the Jacobian (⋅) of residual (⋅) is restricted Lipschitz continuous on R  . where Since ANIHT algorithm generates monotonically decreasing function values, then ‖(  )‖ ≤ ‖( 0 )‖ for all .Direct calculation yields that ( It follows from (15) that which completes the proof.
Therefore,   is well defined.
Proof.According to the computation in Step 2, we have which implies that that is, (ii) any accumulation point of {  } is the stationary point of (2).
We are now ready to show that under suitable conditions the support set of a point is identified in a finite number of iterations.We can easily verify that if ‖ * ‖ 0 = , then the support set of  in a sufficiently small neighborhood of  * is identified.For ‖ * ‖ 0 < , we introduce the concept of strict complementarity to identify the support set.

Numerical Experiments
In this part, sensor localization problem and phase retrieval problem will be stimulated.In both examples, the stop criteria will be set as ‖((  ) ⊤ (  )) Γ  ‖ ≤  2 , where  2 is pretty small in different cases or the maximum iterative times being equal to 5000.
Sensor localization problem can be described as follows: given  known anchors  1 ,  2 , . . .,   ∈ R  , the purpose is to find a sensor  ∈ R  satisfying where   ,  = 1, . . ., , is the noise (which obeys the normal distribution with zero expectation and  2 0 variance here).The problem of finding an  ∈ R  satisfying above equalities is the same as finding an optimal solution to the optimization problem (4) with () = ∑  =1 (‖ −   ‖ 2 −   ) 2 .We first For each value of  ( = 1, 2, . . ., 10), we ran ANIHT algorithm from 100 different and randomly generated initial data sets.The numbers of runs of 100 in which the methods found the "correct" solution are given in Table 1.Here, the "correct" solution  (‖‖ 0 ≤ ) means that () ≤ ( orig ), or  and  orig are very close, say As can be clearly seen by the results in the table, the ANIHT performs well in terms of the success probability.For more details, when the "true" solutions are quite sparse compared to the dimension , ANIHT can almost recover all the "ture" solutions, while the performance is becoming worse as  rises.
Then we run ANIHT algorithm in a higher dimensional data set, where  = 2,  = ⌈0.01⌉,and  = 100, 200, . . ., 2000.For each data set, we run 40 times and record the average results (in which the unsuccessful recoveries are expelled).Figure 1 shows the performance of ANIHT when addressing this problem.
Phase retrieval is to recover a signal from the magnitude of its Fourier transform, or of any other linear transform.Due to the loss of Fourier phase information, the problem is generally ill-posed.The phase retrieval problem can be described as follows: given  known measurement vectors  1 ,  2 , . . .,   ∈ R  , the purpose is to reconstruct a signal  ∈ R  satisfying      ⟨,   ⟩      2 +   =   ,  = 1, . . ., , where   is the th column of the general matrix or the discrete Fourier transform (DFT) and   ,  = 1, . . ., , is the noise (which obeys the normal distribution with zero expectation and  2 0 variance here).Also this problem is equivalent to recover an optimal solution to the optimization problem (4) with () = ∑  =1 ( ⊤    −   ) 2 , where   =    ⊤ ,  = 1, . . ., .
There are some other methods for sparse phase retrieval and the codes are available.So we can compare our ANIHT algorithm with them.We first compare the ANIHT algorithm with the partial sparse-simplex method (PSS) and greedy sparse-simplex (GSS) method in [17] with  = 80,  = 120, and  = 2, 3, . . ., 9 which is identical to those in [17].The true vector  orig and the measurement vectors  1 ,  2 , . . .,   are generated as that produced in sensor localization problem;  is designed as following MATLAB codes: For each value of  ( = 2, 3, . . ., 9), we ran ANIHT algorithm from 100 different and randomly generated initial data sets.
Mathematical Problems in Engineering 7 The numbers of runs of 100 in which the methods found the "correct" solution are given in Table 2.As can be clearly seen by the results in the table, the ANIHT outperforms PSS and GSS in terms of the success probability.What is more, the data in the row with  0 = 0 are higher than those with  0 = 0.01 and  0 = 0.1.
We also compare our ANIHT algorithm with GESPAR in [19] to recover a signal from the magnitude of its Fourier transform.Namely, it is to find a real-valued discrete time signal  ∈ R  from its magnitude-squared value of an  point discrete Fourier transform (DFT): We denote by  the DFT matrix; then elements   =  −2i(−1)(−1)/ and  = || 2 , where | ⋅ | 2 denotes the element-wise absolute-squared value.We get  by the pseudo MATLAB codes:  = abs (fft ( orig )) .∧ 2 +  0 * randn (, 1) .
To see the accuracy of the solutions and the speed of these two methods, we run the two methods for  increasing from 512 to 3072 and keeping  = 2,  = 1%.We also test them under noiseless and two noise levels,  0 = 0.01 and  0 = 0.1.From Table 3, we can see that ANIHT outperforms GESPAR in terms of both average CPU time and average relative error for large  ( ≥ 2048).

Conclusion
Nonlinear CS (NCS) not only is of academic interest but also might be important in many real-world applications when the measurements cannot be designed to be perfectly linear.In this paper, we have proposed an ANIHT algorithm for NCS and studied its convergence.We have showed that any accumulation point of the algorithm is the stationary point.The support set of the sequence can be identified with the assumption of nondegeneracy and strict complementarity of stationary point.The numerical experiments show that ANIHT algorithm is effective for NCS.In the future, we will further consider other methods for nonlinear least square problem to improve the rate of convergence, such as L-M method or cubic regularization methods [20].

Figure 1 :
Figure 1: Average results over 40 simulations with different noise.

Table 1 :
The success numbers over 100 runs under different noise.

Table 2 :
The success numbers of four methods over 100 runs under different noise.