This paper presents a novel subband adaptive filter (SAF) for system identification where an impulse response is sparse and disturbed with an impulsive noise. Benefiting from the uses of l1-norm optimization and l0-norm penalty of the weight vector in the cost function, the proposed l0-norm sign SAF (l0-SSAF) achieves both robustness against impulsive noise and remarkably improved convergence behavior more than the classical adaptive filters. Simulation results in the system identification scenario confirm that the proposed l0-norm SSAF is not only more robust but also faster and more accurate than its counterparts in the sparse system identification in the presence of impulsive noise.

1. Introduction

Adaptive filtering algorithms have gained popularity and proven to be efficient in various applications such as system identification, channel equalization, and echo cancellation [1–4]. The normalized least mean square (NLMS) algorithm has become one of the most popular and widely used adaptive filtering algorithms because of its simplicity and robustness. Despite these advantages, the use of NLMS has been limited since it converges poorly for correlated input signals [2]. To address this problem, various approaches have been presented, such as the recursive least squares algorithm [2], the affine projection algorithm [2], and subband adaptive filtering (SAF) [5–9]. Among these, the SAF algorithm allocates the input signals and desired response into almost mutually exclusive subbands. This prewhitening characteristic of SAF allows each subband to converge almost separately so that the subband algorithms obtain faster convergence behavior. On the basis of these characteristics, Lee and Gan proposed a normalized SAF (NSAF) algorithm in [8, 9]. This work improves the convergence speed, while using almost the same computational complexity as the NLMS algorithm. However, the NSAF still suffers from the degradation of convergence performance in cases when an underlying system to be identified is sparse such as network echo path [10], underwater channel [11], and digital TV transmission channel [12]. Motivated by the proportionate step-size adaptive filtering [13, 14], the proportionate NSAF (PNSAF) has been presented to combat poor convergence in sparse system identification [15]. However, it does not exploit the sparsity condition itself. Moreover, the NSAF and PNSAF algorithms are highly sensitive to impulsive interference, leading to deteriorated convergence behavior. Impulsive interference exits in various applications such as acoustic echo cancellation [16], network cancellation [17], and subspace tracking [18].

To address the robustness issue, the sign SAF (SSAF) [19] has been developed based on the l1-norm optimization, making it robust against impulsive interference. However, its use is limited in case of sparse system identification. Moreover, the SSAF converges poorly and fails to accelerate the convergence rate with the number of subbands.

In recent years, motivated by compressive sensing framework [20, 21] and the least absolute shrinkage and selection operator (LASSO) [22], a variety of adaptive filtering algorithms which incorporate the sparsity of a system have been developed unlike the proportionate adaptive filtering approach [23–27]. Along this line, the SAF with the l1-norm penalty has been recently presented as an alternative for incorporating the sparsity of a system [28]. In particular, the l0-norm of a system is able to represent the actual sparsity [24–26]. In this paper, a l0-norm constraint SSAF (l0-SSAF) is presented, aiming at developing a sparsity-aware SSAF. With this in mind, by integrating the l0-norm penalty of the current weight vector into the l1-norm optimization criterion, the l0-SSAF benefits both superior convergence for sparse system identification and robustness against impulsive noise. In addition, the l0-SSAF is derived from a l1-norm optimization of the a priori error instead of the a posteriori error used in the SSAF. Thus, there is no need to approximate the a posteriori error with the a priori error to derive the update recursion of the l0-SSAF. Simulation results show that the l0-SSAF is superior to the conventional SAFs in identifying a sparse system in the presence of severe impulsive noise.

The remainder of the paper is organized as follows. Section 2 introduces the classical SAFs, followed by the derivation of the proposed l0-SSAF algorithm in Section 3. Section 4 illustrates the computer simulation results and Section 5 concludes this study.

2. Conventional SAFs

Consider a desired signal d(n) that arises from the system identification model
(1)d(n)=u(n)w∘+v(n),
where w∘ is a column vector for the impulse response of an unknown system that we wish to estimate, v(n) accounts for measurement noise with zero mean and variance σv2, and u(n)=[u(n)u(n-1)⋯u(n-M+1)] is a 1×M input vector.

Figure 1 shows the structure of the NSAF, where the desired signal d(n) and output signal u(n) are partitioned into N subbands by the analysis filters H0(z),H1(z),…,HN-1(z). The resultant subband signals, di(n) and yi(n) for i=0,1,…,N-1, are critically decimated to a lower sampling rate commensurate with their bandwidth. Here, the variables n to index the original sequences and k to index the decimated sequences are used for all signals. Then, the decimated desired signal and the decimated filter output signal at each subband are defined as di,D(k)=di(kN) and yi,D(k)=ui(k)w(k), where ui(k) is the input data vector for the ith subband such that
(2)ui(k)=[ui(kN),ui(kN-1),…,ui(kN-M+1)]
and w(k)=[w0(k),w1(k),…,wM-1(k)]T denotes an estimate for w∘. Then, the decimated subband error vector is given by
(3)ei,D(k)=di,D(k)-yi,D(k)=di,D(k)-ui(k)w(k).

Subband structure used in the proposed SAF.

In [8], the authors have presented that the update recursion of the NSAF algorithm is given by
(4)w(k+1)=w(k)+μ∑i=0N-1uiT(k)∥ui(k)∥2ei,D(k),
where μ is a step-size parameter. Then, the estimation error in all the N subbands, that is, eD(k)=[e0,D(k),…,eN-1,D(k)]T, can be written in a compact form as
(5)eD(k)=dD(k)-U(k)w(k),
where the N×M subband data matrix U(k) and the N×1 desired response vector d(k) are given by
(6)U(k)=[u0(k),u1(k),…,uN-1(k)]T,dD(k)=[d0,D(k),d1,D(k),…,dN-1,D(k)]T.

More recently, the SSAF [19] has been obtained from the following optimization criterion:
(7)minw(k+1)tto.∥dD(k)-U(k)w(k+1)∥1subjectto∥w(k+1)-w(k)∥22≤μ2,
where ∥·∥1 denotes the l1-norm and μ2 is a parameter which prevents the weight coefficient vectors from abrupt change. Using Lagrange multipliers to solve the constrained optimization problem and utilizing the accessible eD(k) instead of unavailable a posteriori error, that is, dD(k)-U(k)w(k+1), the update recursion of the SSAF is formulated as
(8)w(k+1)=w(k)+μUT(k)sgn[eD(k)]∑i=0N-1ui(k)uiT(k)+δ,
where δ is a regularization parameter and sgn(·) denotes the sign function, where sgn[eD(k)]=[sgn(e0,D(k)),…,sgn(eN-1,D(k))]T.

Our objective is to cope with the sparsity of an underlying system while inheriting robustness from the l1-norm optimization criterion. Our approach is to find a new weight vector, w(k+1), that minimizes the l1-norm of the a priori error vector with the l0-norm penalty of the current weight vector w(k) as follows:
(9)minwJ(k)≜minw[∥eD(k)∥1+γ∥w(k)∥0],
where ∥·∥0 denotes the l0-norm and γ(>0) is a regularization parameter which governs the compromise between the effect of the l0-norm penalty term and the error vector related term. Note that the a priori error eD(k) is used unlike the SSAF, leading to no approximation of the a posteriori error with the a priori error.

Taking derivative of J(k), with respect to w(k), it leads to
(10)∇w(k)J(k)=-UT(k)sgn(eD(k))+γ∂∥w(k)∥0∂w(k)≜-UT(k)sgn(eD(k))+γfβ(w(k)),
where fβ(w(k))≜[fβ(w0(k)),fβ(w1(k)),…,fβ(wM-1(k))]T. To avoid a nonpolynomial hard problem from the l0-norm minimization, the l0-norm penalty is often approximated as follows [29]:
(11)∥w(k)∥0≈∑i=0M-1(1-e-β|wi(k)|),
where the parameter β plays a role adjusting the degree of zero attraction. A mth component of the gradient for (11) is given by
(12)∂∥w(k)∥0∂wm(k)=fβ(wm(k))=βsgn[wm(k)]e-β|wm(k)|∀0≤m≤M-1.
To reduce the computational cost in (12), the first-order Taylor series expansion of the exponential function is employed:
(13)e-β|x|≈{1-β|x|,|x|≤1β0,elsewhere.
Then, a gradient (12) is computed as
(14)fβ(wm(k))={β2wm(k)+β,-1β≤wm(k)<0β2wm(k)-β,0<wm(k)≤1β0,elsewhere.
Finally, the update recursion of the l0-SSAF is given by
(15)w(k+1)=w(k)+μUT(k)sgn(eD(k))-κfβ(w(k)),
where μ is the step-size parameter and κ=μγ.

4. Simulation Results

To validate the performance of the proposed l0-SSAF, computer simulations are carried out in a system identification scenario in which the unknown system is randomly generated. The length of the unknown system is M=128, where S of them are nonzero. The nonzero filter weights are positioned randomly and their values are taken from a Gaussian distribution N(0,1/S). Here, the sparse systems of the sparsity S=4,8,16,32 are considered. The adaptive filter and the unknown system are assumed to have the same number of taps. The input signals u(n) are obtained by filtering a white, zero mean, Gaussian random sequence through a first-order system,
(16)G1(z)=11-0.9z-1,
or a second-order system,
(17)G2(z)=1+0.5z-1+0.8z-21-0.9z-1.

A measurement noise v(n) with white Gaussian distribution is added to the system output y(n) such that the signal-to-noise ratio (SNR) is 20 dB, where the SNR is defined as
(18)SNR=10log10(E[y2(n)]E[v2(n)]),
where y(n)=u(n)w∘. An impulsive noise z(n) is added to the system output y(n) with the signal-to-interference ratio (SIR) of −30 or −10 dB correspondingly. The impulsive noise is modeled by a Bernoulli-Gaussian (BG) distribution [16], which is obtained as the product of a Bernoulli distribution and a Gaussian one; that is, z(n)=ω(n)η(n), where ω(n) is a Bernoulli process with a probability mass function given by P(ω)=1-Pr for ω=0 and P(ω)=Pr for ω=1. In addition, η(n) is an additive white Gaussian noise with zero mean and variance ση2=1000σy2. Here, Pr=0.01 is used. In order to compare the convergence performance, the normalized mean square deviation (NMSD),
(19)NormalizedMSD=E[∥w∘-w(k)∥2∥w∘∥2],
is taken and averaged over 50 independent trials. The cosine-modulated filter banks [30] with the subband numbers of N=4 are used in the simulations. The prototype filter of length L=32 is used. The parameters used in simulations are as follows: NSAF (μ=0.0005 or 0.0001), SSAF (μ=0.0005, δ=0.001), PNSAF (μ=0.0005, ρ=0.01), and l0-SSAF (μ=0.0003, β=20). The γ of the l0-norm SSAF is obtained by repeated trials to minimize the steady-state NMSD. We use the input signals generated by G1(z) and G2(z) for Figures 2–7 and Figures 8 and 9, respectively.

NMSD learning curves of the NSAF, PNSAF, SSAF, and l0-SSAF algorithms [N=4, SIR=-30 dB, input: Gaussian AR(1) with pole at 0.9].

NMSD learning curves of the l0-SSAF algorithm with various γ values [N=4, SIR=-30 dB, input: Gaussian AR(1) with pole at 0.9].

NMSD learning curves of the NSAF, PNSAF, SSAF, and l0-SSAF algorithms [N=4, SIR=-10 dB, input: Gaussian AR(1) with pole at 0.9].

NMSD learning curves of the NSAF, PNSAF, SSAF, and l0-SSAF algorithms for different sparsity (S = 8, 16, 32) [N=4, SIR=-30 dB, input: Gaussian AR(1) with pole at 0.9].

NMSD learning curves of the NSAF, PNSAF, SSAF, and l0-SSAF algorithms with difference values of β (β = 1, 20, 50, 100) [N=4, SIR=-30 dB, input: Gaussian AR(1) with pole at 0.9].

NMSD learning curves of the NSAF, PNSAF, SSAF, and l0-SSAF algorithms in case of a time-varying unknown system (N=4). The system is right-shifted for 20 taps at 20000th iteration [N=4, SIR=-30 dB, input: Gaussian AR(1) with pole at 0.9].

NMSD learning curves of the NSAF, PNSAF, SSAF, and l0-SSAF algorithms [N=4, SIR=-30 dB, input: Gaussian AR(2,2)].

NMSD learning curves of the NSAF, PNSAF, SSAF, and l0-SSAF algorithms [N=4, SIR=-10 dB, input: Gaussian AR(2,2)].

Figure 2 shows the NMSD learning curves of the NSAF, PNSAF, SSAF, and l0-norm SSAF algorithms in the case of SIR=-30 dB. For the l0-SSAF, γ=5×10-5 is chosen. Compared to the conventional SAF algorithms, the proposed l0-SSAF yields remarkably improved convergence performance in terms of the convergence rate and the steady-state misalignment.

In Figure 3, to verify the effect of γ on convergence performance, the NMSD curves of the l0-SSAF for different γ values are illustrated in the case of SIR=-30dB. With different γ values (γ=3×10-4,1×10-4,7×10-5, and 5×10-5), the l0-SSAF is not excessively sensitive to γ. The analysis for an optimal γ value remains a future work.

Figure 4 illustrates the NMSD learning curves of the NSAF, PNSAF, SSAF, and l0-norm SSAF algorithms under SIR=-10dB. The same γ value with Figure 2 is chosen. In the figure, similar results shown in Figure 2 are observed.

Figure 5 depicts the NMSD learning curves of the NSAF, PNSAF, SSAF, and l0-SSAF algorithms for difference sparsity. Here, S=8, 16, 32 were chosen. The same parameters as in Figure 2 are used. As can be seen, the more sparse the system, the better the convergence performance of the l0-SSAF.

Figure 6 shows the NMSD learning curves of the NSAF, PNSAF, SSAF, and l0-SSAF algorithms with difference values of β in the case of S=4. The values of β = 1, 20, 50, 100 were used. Also, the same step-size parameter μ=0.0003 is chosen. In the figure, it is apparent that the larger the value of β, the higher the steady-state MSD. However, the optimal value of β remains a future issue.

Next, the tracking capabilities of the algorithms to a sudden change in the system are tested for SIR=-30dB. Figure 7 shows the results in case when an unknown system is right-shifted for 20 taps. The same value of γ of Figure 2 is used. The figure shows that the l0-SSAF keeps track of weight change while achieving a faster convergence rate and a low steady-state misalignment compared to the conventional SAF algorithms.

Finally, Figures 8 and 9 show the simulation results with the different input signal generated by G2(z) for SIR=-30dB and -10dB, respectively. The same parameters of all SAF algorithms in Figure 2 are chosen in Figures 8 and 9. We can see similar results in previous figures, implying the capability of the l0-norm SSAF over the classical SAF algorithms for different input signal.

5. Conclusion

This paper has proposed a robust and sparse-aware SSAF algorithm which incorporates the sparsity condition of a system into the l1-norm optimization criterion of the a priori error vector. By utilizing the l0-norm penalty of the current weight vector and approximating it to avoid a nonpolynomial hard problem, the update recursion of the proposed l0-norm SSAF is obtained while reducing the computational cost using Taylor series expansion. The simulation results indicate that the proposed l0-SSAF achieves highly improved convergence performance over the conventional SAF algorithms where a system is not only sparse but also disturbed with impulsive noise.

Conflict of Interests

The author declares that there is no conflict of interests regarding the publication of this paper.

HaykinS.SayedA. H.DinizP. S. R.SondhiM. M.The history of echo cancellationGilloireA.VetterliM.Adaptive filtering in subbands with critical sampling: analysis, experiments, and application to acoustic echo cancellationde CourvilleM.DuhamelP.Adaptive filtering in subbands using a weighted criterionPradhanS. S.ReddV. U.A new approach to subband adaptive filteringLeeK. A.GanW. S.Improving convergence of the NLMS algorithm using constrained subband updatesLeeK. A.GanW. S.Inherent decorrelating and least perturbation properties of the normalized subband adaptive filterDuttweilerD. L.Proportionate normalized least-mean-squares adaptation in echo cancelersLiW.PreisigJ. C.Estimation of rapidly time-varying sparse channelsSchreiberW. F.Advanced television systems for terrestrial broad-casting: some problems and some proposed solutionsGayS. L.Efficient, fast converging adaptive filter for network echo cancellation1Proceedings of the 32nd Asilomar Conference on Signals, Systems & ComputersNovember 1998Pacific Grove, Calif, USA3943982-s2.0-0032269759DengH.DoroslovačkiM.Improving convergence of the PNLMS algorithm for sparse impulse response identificationAbadiM. S. E.Proportionate normalized subband adaptive filter algorithms for sparse system identificationVegaL. R.ReyH.BenestyJ.TressensS.A new robust variable step-size NLMS algorithmYangZ.ZhengY. R.GrantS. L.Proportionate affine projection sign algorithms for network echo cancellationLiaoB.ZhangZ. G.ChanS. C.A new robust Kalman filter-based subspace tracking algorithm in an impulsive noise environmentNiJ.LiF.Variable regularisation parameter sign subband adaptive filterDonohoD. L.Compressed sensingCandèsE. J.RombergJ.TaoT.Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency informationTibshiraniR.Regression shrinkage and selection via the lassoChenY.GuY.HeroA. O.Sparse LMS for system identificationProceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '09)April 2009Taipei, Taiwan312531282-s2.0-7034921642710.1109/ICASSP.2009.4960286GuY.JinJ.MeiS.l0 norm constraint LMS algorithm for sparse system identificationJinJ.GuY.MeiS.A stochastic gradient approach on compressive sensing signal reconstruction based on adaptive filtering frameworkEksiogluE. M.TancA. K.RLS algorithm with convex regularizationKalouptsidisN.MileounisG.BabadiB.TarokhV.Adaptive algorithms for sparse system identificationChoiY.-S.Subband adaptive filtering with l1-norm constraint for sparse system identificationBradleyP. S.MangasarianO. L.Feature selection via concave minimization and support vector machinesProceedings of the International Conference on Machine Learning (ICML '98)19988290VaidyanathanP. P.