Subband Adaptive Filtering with l1-Norm Constraint for Sparse System Identification

This paper presents a new approach of the normalized subband adaptive filter (NSAF) which directly exploits the sparsity condition of an underlying system for sparse system identification.The proposed NSAF integrates a weighted l 1 -norm constraint into the cost function of the NSAF algorithm. To get the optimum solution of the weighted l 1 -norm regularized cost function, a subgradient calculus is employed, resulting in a stochastic gradient based update recursion of the weighted l 1 -norm regularized NSAF. The choice of distinct weighted l 1 -norm regularization leads to two versions of the l 1 -norm regularized NSAF. Numerical results clearly indicate the superior convergence of the l 1 -norm regularized NSAFs over the classical NSAF especially when identifying a sparse system.


Introduction
Over the past few decades, the relative simplicity and good performance of the normalized least mean square (NLMS) algorithm have made it a popular tool for adaptive filtering applications.However, its convergence performance is significantly deteriorated in case of correlated input signals [1,2].As a popular solution, adaptive filtering in the subband has been recently developed, which is referred to as subband adaptive filter (SAF) [3][4][5][6][7].Its distinct feature is based on the property that the LMS-type adaptive filters converge faster for white input signals than colored ones [1,2].Thus, carrying out a prewhitening on colored input signals, it results in the accelerated convergence compared to the LMS-type adaptive filters.Recently, the use of multiple-constraint optimization criteria into formulation of a cost function has resulted in the normalized SAF (NSAF) with its computational complexity close to that of the NLMS algorithm [6,7].
In the context of a system identification, the unknown system to be identified is sparse in common scenarios, such as echo paths [8] and digital TV transmission channels [9].Namely, the unknown system consists of many near-zero coefficients and a small number of large ones.However, the adaptive filtering algorithms suffer from poor convergence performance in case of identifying the sparse system [8].Indeed, the capability of the NSAF is faded in a sparse system identification scenario.To deal with this issue, a variety of proportionate adaptive algorithms have been presented for NSAF, which utilize proportionate step sizes to distinct filter taps [10][11][12].However, these algorithms have not exploited the sparsity condition of an underlying system.
Recently, motivated by compressive sensing framework [13,14] and the least absolute shrinkage and selection operator (LASSO) [15], a number of adaptive filtering algorithms which make use of the sparsity condition of an underlying system have been developed [16][17][18][19][20].The core idea behind this approach is to incorporate the sparsity condition of underlying system by imposing an a sparsity-inducing constraint term.Adding the sparsity constraint using  0 or  1norm constraint to the cost function makes the least relevant weights of the filter shrink to zeros.However, to the best of the author's knowledge, adaptive filtering in subband which exploits the sparsity condition has not been studied yet.
With regard, this paper presents a novel approach of the sparsity-regularized NSAFs, which incorporates the sparsity condition of the system directly into the cost function via a sparsity-inducing constraint term.This is carried out by regularizing a weighted  1 -norm of the filter weights estimate 2 Mathematical Problems in Engineering to the cost function.Considering the two choices of the weighted  1 -norm regularization, two stochastic gradientbased  1 -norm regularized NSAF algorithms are derived.First, the  1 -norm NSAF ( 1 -NSAF) is obtained by utilizing the identity matrix as a weighting matrix.Second, the reweighted  1 -norm NSAF ( 1 -RNSAF) which uses the current estimate of the system as a weighted  1 -norm is developed.Through numerical simulations, the resultant sparsity-regularized NSAFs have proven their superiority over the classical NSAFs, especially when the sparsity of the underlying system becomes severe.The remainder of the paper is organized as follows.Section 2 introduces the classical NSAF, followed by the derivation of the proposed sparsity-regularized NSAFs in Section 3. Section 4 illustrates the computer simulation results and Section 5 concludes this study.

Conventional NSAF
Consider a desired signal () that arises from the system identification model where w ∘ is a column vector for the impulse response of an unknown system that we wish to estimate, V() accounts for measurement noise with zero mean and variance  2 V , and u() denotes the 1 ×  input vector, Figure 1 shows the structure of the NSAF, where the desired signal () and input signal () are partitioned into  subbands by the analysis filters  0 (),  1 (), . . .,  −1 ().The resulting subband signals,   () and   () for  = 0, 1, . . .,  − 1, are critically decimated to a lower sampling rate commensurate with their bandwidth.Here, the variable  to index the original sequences and  to index the decimated sequences are used for all signals.Then, the decimated filter output signal at each subband is defined as  , () = u  ()w(), where u  () is 1 ×  row vector, such that and w() = [ 0 (),  1 (), . . .,  −1 ()]  denotes an estimate for w ∘ with length .Thus the decimated subband error signal is given by where  , () =   () is the decimated desired signal at each subband.
In [6], the authors have formulated the Lagrangianbased multiple-constraint optimization problem, which is formulated as where   for  = 0, 1, . . .,  − 1 denote the Lagrange multipliers.Solving the cost function ( 5), the update recursion of the NSAF algorithm is given by [6,7].Consider where  is the step-size parameter.
Mathematical Problems in Engineering 3

Derivation of the Proposed Algorithm.
To reflect the sparsity condition of the true system, that is, w ∘ , a weighted  1 -norm of the filter weight estimate is regularized on the cost function of the NSAF, which is given by where ‖Πw( + 1)‖ accounts for the weighted  1 -norm of the filter weight vector w( + 1) and is written as where Π is a  ×  weighting matrix whose diagonal elements are   and other elements are equal to zero, and   ( + 1) denotes the th tap weight of w( + 1), for  = 0, 1, . . .,  − 1.In addition,  is a positive value parameter which plays a role in compromising the error related term and the weighted  1 -norm regularization in the right-hand side of (7).
To find the optimal weight vector w( + 1) which minimizes the cost function (7), the derivative of ( 7) with respect to w( + 1) is taken and set to zero.Note that the weighted  1 -norm regularization term, that is, ‖Πw( + 1)‖ 1 , is not differentiable at any point in case   ( + 1) = 0. To address this issue, a subgradient calculus [21] is carried out.
Consequently, combining ( 12) and ( 15), the update recursion of the sparsity-regularized NSAF is given by where  is the step-size parameter.

Determination of the Weighted 𝑙 1 -Norm Regularization.
Here, by choosing the weighting matrix Π, two versions of the sparsity-regularized NSAF are developed.First, the use of the identity matrix as the weighting matrix, that is, Π = I  , results in the following update recursion: which is referred to as the  1 -norm NSAF ( 1 -NSAF) as an unweighted case.The  1 -NSAF uniformly attracts the tap coefficients of w() to zero.The zero attraction process leads to the improved convergence of the  1 -NSAF when the majority of entries of a system are zero; that is, a system is sparse.
Second, to approximate the actual sparsity condition of an underlying system, that is,  0 -norm of the system, the weights of Π are chosen inversely proportional to magnitude of the actual coefficients of the system as given by where   denotes the th coefficients of the system, that is, w ∘ .However, since the actual coefficients of the system are unavailable, the estimate of the current filter weights is utilized instead of the actual weights, which is referred to as the reweighting scheme [22], as follows: where   () denotes the th tap weight of the w() and  is a small positive value to avoid singularity when |  ()| = 0.Then, the weighting matrix Π consists of the values of   () as the th diagonal elements and has a time-varying feature.Finally, the update recursion is given by where u  () = u  ()Π and the vector division operation in last term accounts for a componentwise division.Then, this recursion is called the reweighted  1 -norm NSAF ( 1 -RNSAF) Table 1 lists the number of multiplications and divisions of the NSAF [6],  1 -NSAF, and  1 -RNSAF per iteration.As shown in Table 1, the use of  1 -norm constraint leads to an acceptable increase in computation.

Numerical Results
The performance of the proposed sparsity-regularized NSAFs is validated by carrying out computer simulations in a system identification scenario in which the unknown channel is randomly generated.The lengths of the unknown system are  = 128 and 512 in experiments where  of them are nonzero.The nonzero filter weights are positioned randomly and their values are taken from a Gaussian distribution N(0, 1/).Here,  = 4 is used in the simulations except Figure 5 in which various  values are considered.The adaptive filter and the unknown system are assumed to have the same number of taps.The input signals are obtained by filtering a white, zero-mean, Gaussian random sequence through a first-order system () = 1/(1−0.9−1 ).The signalto-noise ratio (SNR) is calculated by where () = u()w ∘ .The measurement noise V() is added to () such that SNR = 10, 20, and 30 dB.In order to compare the convergence performance, the normalized mean square deviation (MSD), is taken and averaged over 50 independent trials.The cosinemodulated filter banks [23] with the subband number of  = 4 are used in the simulations.The prototype filter of length  = 32 is used.For comparison purpose, the proportionate NSAF (PNSAF) [12] is considered, which has been developed for sparse system identification.The step-size is set to  = 0.5 for SAF algorithms except the PNSAF where the step sizes  = 0.6 (Figure 2) and  = 0.65 (Figure 6) are chosen to achieve similar steady-state MSD with the  1 -RNSAF for comparison purpose.For the  1 -RNSAF,  = 0.01 is chosen.In addition,  = 0.05 is used for the PNSAF.The  values are obtained by repeated trials to minimize the steady-state MSD.
Figure 2 shows the normalized MSD curves of the NLMS, NSAF,  1 -NSAF, and  1 -RNSAF, in cases of  = 4 and SNR = 30 dB.For the  1 -NSAF and  1 -RNSAF,  = 3×10 −5 is chosen.As shown in Figure 2, the not only  1 -RNSAF outperforms the conventional NLMS, NSAF, PNSAF, and  1 -NSAF, but also the  1 -NSAF has better performance than other conventional algorithms, in terms of the convergence rate and the steadystate misalignment.
In Figure 3, to verify the effect of  on convergence performance, the normalized MSD curves of the  1 -RNSAF for different  values are illustrated, in case of  = 4 and SNR = 30 dB.For different  values ( = 1×10 −4 , 1×10 −5 , 5× 10 −5 , and 1 × 10 −6 ), the  1 -RNSAF is not excessively sensitive to .The analysis of an optimal  value remains a future work.
Next, the performance of the proposed  1 -norm regularized NSAFs is compared to the original NSAF under different SNR conditions.Figure 4 depicts the normalized MSD curves of the NSAF,  1 -NSAF, and  1 -RNSAF under SNR = 10 and 20 dB, respectively.The  value for the  1 -NSAF and  1 -RNSAF is set to 5 × 10 −5 .It is clear that both the  1 -NSAF and  1 -RNSAF are superior to the NSAF under several SNR cases.Furthermore, the  1 -RNSAF performs well compared to  1 -NSAF.In Figure 5, the convergence properties of the NSAF and  1 -RNSAF are compared under various sparsity conditions of an underlying system.With the same length of the system, that is,  = 128, different sparsity conditions ( = 4, 8, 16, and 32) are considered under SNR = 30 dB.The value of  is set to 3 × 10 −5 for the  1 -RNSAF.Figure 5 shows that the NSAF is independent of the sparsity condition.On the other hand, the results indicate that the more sparse the underlying system, the better the  1 -RNSAF.
The comparison of performance of the NSAF,  1 -NSAF, and  1 -RNSAF with a long system, here, the filter length  = 512, is presented in Figure 6.For the  1 -NSAF and  1 -RNSAF,  = 5×10 −5 is chosen.A similar result of Figure 2 is observed in Figure 6.
Finally, the tracking capabilities of the algorithms of a sudden change in the system are tested for  = 4 and SNR = 30 dB. Figure 7 shows the results when the unknown system is right-shifted for 20 taps.Same value of  in Figure 2 is used.As can be seen, the  1 -NSAF and  1 -RNSAF keep track of weight change without losing the convergence rate nor  the steady-state misalignment compared to the conventional NLMS, NSAF, and PNSAF.To be specific, the  1 -RNSAF achieves better performance than the  1 -NSAF in terms of both convergence rate and steady-state misalignment.

Conclusion
A new family of the NSAFs which takes into account the sparsity condition of an underlying system has been presented by incorporating a weighted  1 -norm constraint of filter weights in the cost function.The update recursion is obtained by employing subgradient calculus on the weighted  1 -norm constraint term.Subsequently, two sparsity regularized NSAFs, that is, the unweighted  1 -NSAF and  1 -RNSAF have been developed.The numerical results indicate that the proposed  1 -NSAF and  1 -RNSAF achieve highly improved convergence performance over the conventional algorithms for sparse system identification.

Figure 1 :
Figure 1: Subband structure with the analysis filters and synthesis filters and the subband desired signals, subband filter outputs, and subband error signals.
(: filter length, : number of subbands, and : length of the analysis filters and synthesis filters).