Estimating Frequency by Interpolation Using Least Squares Support Vector Regression

Discrete Fourier transform(DFT-) based maximum likelihood (ML) algorithm is an important part of single sinusoid frequency estimation. As signal to noise ratio (SNR) increases and is above the threshold value, it will lie very close to Cramer-Rao lower bound (CRLB), which is dependent on the number of DFT points. However, its mean square error (MSE) performance is directly proportional to its calculation cost. As a modified version of support vector regression (SVR), least squares SVR (LSSVR) can not only still keep excellent capabilities for generalizing and fitting but also exhibit lower computational complexity. In this paper, therefore, LS-SVR is employed to interpolate on Fourier coefficients of received signals and attain high frequency estimation accuracy. Our results show that the proposed algorithm can make a good compromise between calculation cost and MSE performance under the assumption that the sample size, number of DFT points, and resampling points are already known.


Introduction
Estimating frequency of a single sinusoid based on discrete Fourier transform (DFT) has attracted considerable attentions for many decades.Rife and Boorstyn exploited the relationship of maximum likelihood estimator (MLE) to DFT and proposed a frequency domain periodogram (FDP) algorithm having two stages: coarse search and fine search [1].In order to reduce the calculation cost, the frequency step of coarse search is appropriately broadened.As a result, the absolute frequency value is usually not in the DFT points.Hence, a fine search will be necessary for higher frequency estimation accuracy.
A great deal of fine search algorithms has erupted mainly from two sides: the dichotomous search and interpolation on Fourier coefficients.During the former, an iterative binary search for the true signal frequency has been presented, which is particularly suited for digital signal processing (DSP) implementation [2].In [3], the same authors have proposed a number of hybrid estimators that combine the dichotomous search with various interpolation techniques in order to reduce the computational complexity at the expense of acquisition range; and other modified dichotomous search frequency estimators have been addressed in [4][5][6].In the latter, complex Fourier coefficients have been utilized to interpolate the true signal frequency between the maximum and the second highest bin [7].However, it has been shown to have a frequency dependent performance [8].Two improved estimators have been proposed, which were implemented iteratively [9,10].Rational combination of three spectrum lines (RCTSL) has been employed as the fine estimation because of its constant combinational weights in least square approximation [11].Other methods used for interpolation include Lagrange interpolator [12], L-filter DFT [13], nonlinear filter [14], Kaiser window [15], trigonometric polynomial interpolator [16], and narrowband approximation [17].However, all of the DFT-based algorithms attain better mean square error (MSE) performance at the expense of calculation cost.
As the concrete implementation of statistical learning theory (SLT) and structure risk minimization (SRM) principle, SVR overcomes the overfitting and local minimum problems currently existing in artificial neural network (ANN).Least squares support vector regression (LS-SVR) has the improvements of support vector regression (SVR): inequality constraint is substituted by equality one; a squared 2 Mathematical Problems in Engineering loss function is taken for the error variable.Consequently, its good generalizing and learning capabilities are exploited for interpolation on Fourier coefficients, and a novel DFTbased frequency estimation algorithm adopting LS-SVR for fine search is proposed.Our results show that under the assumption that the sample size, number of DFT points, and resampling points are already known, the proposed algorithm can derive low calculation cost while still keeping good MSE performance.
The remainder of this paper is organized as follows.Section 2 concretely describes the process of our algorithm.In Section 3, interpolation by LS-SVR is analyzed, and its parameters are selected properly to improve generalization capability.Section 4 shows the results of simulations and experiments.The paper is finally concluded in Section 5.

The Proposed Algorithm
Here,  > 0,  ∈ [−0.5, 0.5), and  ∈ [−, ) are the amplitude, deterministic but unknown frequency, and initial phase, respectively;   is an independent complex additive white Gaussian noise (AWGN) with zero mean and variance  2 ;  is the sample size.

An Interpolation by LS-SVR in DFT-Based Frequency Estimation Algorithm.
The key idea proposed in [1] is exploited and a two-step algorithm including coarse search and fine search is proposed.At first, FDP algorithm is employed and the number of DFT points is extended to  by appending ( − ) zeros.Now, the frequency step is 1/, and where (  ) = ∑ −1 =0    −2   ,   = /.For the sake of convenience of fast Fourier transform (FFT) calculation,  is usually set to 2, 4, 8, . ... Secondly, a training set  = {(  ,   ) |  = 1, . . ., } is constructed and LS-SVR is utilized to fit it, where   =  − 1 and   = ( − 1).Because   is a complex series and LS-SVR is only suitable for real number,   is divided into real and image part.
Ŷreal = ( ŷreal 1 , . . ., ŷreal 2−1 ) T = ( real ( 1 ), . . .,  real ( 2−1 )) T is defined as LS-SVR's output of Y real , and by substituting ( 10) into (9), we have where element of matrix Q  with 2 − 1 rows and  columns, T , and is an all-zero matrix with 2 − 1 rows and  − 1 columns.From ( 11), the LS-SVR process can be regarded as a linear interpolator whose output is the linear weighted sum of all inputs, where the weighted coefficients are dependent on 11), ŷreal  at the different moments keep the same value obviously, which means the LS-SVR process can not fit Y real accurately.Hence,  is set as large as possible. = 0.1,  = 0,  = 32,  = 2,  = 8, and ℎ = 1 are set and the number of Monte Carlo experiments is 10000.Figure 1 illustrates the impact of  on the estimation performance, which is consistent with all analyses above.When  is larger than 100, its MSE performance will almost not increase anymore, so  = 100 is set.
At the same time, according to ( real  ,  real  ) = exp[−( − ) 2 /ℎ 2 ] and the definitions of   and    in ( 8) and (11), it is concluded that So when where   is (, )th element of matrix  with 2 − 1 rows and  columns.It means now the LS-SVR process can only make ensure the accuracy of spectrum of original signals but not interpolate effectively.On the contrary, as a result of lim ℎ → +∞   = 1, lim ℎ → +∞    = 1, now lim ℎ → +∞   = 1/, which means that now the LS-SVR process can also not fit Y real accurately.Everything is as in Figure 1 except that  = 100; Figure 2 illustrates the impact of ℎ on the estimation performance when ℎ is 0.1, 1, 10, and 100, respectively, which is consistent with all analyses above.So ℎ = 1 is set.
Assuming that the received signal is   =   +   , where   = exp[(2 + )],   are signal and noise components of   , the true interpolation value of spectrum of   at point At the same time, the estimating interpolation value of spectrum of   at point   is attained through LS-SVR.Consider As a result, the error between ( 14) and ( 15) is where the noise item /]} is still an independent complex AWGN.Its mean and variance are listed as follows: Then, the mean and variance of (  ) are Everything is as in Figure 1 except that  = 100,  = 4, and SNR = 0 dB; the values of ( 19 T . ( The values of (20) are very small, which means when SNR is higher than threshold, the estimating value through LS-SVR and true value of spectrum of   at point   are almost the same.It validates the validity of interpolation in our proposed algorithm.

Simulations and Experiments
The proposed algorithm entitled LS-SVR is compared with other fives: FDP proposed in [1]; dichotomous search (DS) proposed in [2]; modify dichotomous search (Modify DS) proposed in [4]; iterative interpolation on Fourier coefficients (IIFC) proposed in [9]; RCTSL proposed in [11].Analyzing these five algorithms, it is known that the MSE performances of FDP and RCTSL algorithms will be improved as their DFT points increase, and the MSE performances of DS, Modified DS, and IIFC algorithms will be improved as their iterative numbers increase.In order to compare these algorithms' calculation cost under nearly the same precondition,  = 32 in FDP,  = 2,  DS = 20 in DS,  M-DS = 20 in Modified DS,  IIFC = 10 in IIFC,  = 8 in RCTSL,  = 2, and  = 8 in LS-SVR are set to keep their MSE performance almost the same.

Mean Performance.
Everything is as in Figure 1 except that  = 100.Figure 3 illustrates the mean of these six algorithms, while SNR is −4 dB and 0 dB, respectively.Obviously, when SNR is low, the unbiased performances of Modified DS and IIFC algorithms are worse than the other four.

Frequency Estimation Performance.
As in Figure 3, Figure 4 illustrates the MSE curves of these six algorithms versus SNR.From Figure 4, the threshold values of DS, Modified DS, and IIFC algorithms are higher than other three ones.And, these six algorithms' MSEs are kept almost the same when they are all above their threshold values, where LS-SVR one is the best one.

Impact of DFT Points 𝐾.
As in Figure 3, the MSE curves of LS-SVR algorithm versus SNR are plotted in Figure 5, while  is , 2, 4, and 8, respectively.Now, the sample size and interpolation times are fixed.From Figure 5, when  = , its MSE value will not approach Cramer-Rao lower bound (CRLB) as SNR increases, which means now the LS-SVR algorithm is ineffective; when  = 2, 4, and 8, their MSE values are nearly the same, and the  = 2 condition is a little better than the two others.The reason is to  with a fixed value.From Figure 7, its threshold value will decrease and MSE performance will be improved as  increases.

Calculation Cost.
Calculation cost is an important factor to impact the real time of frequency estimation algorithm.The calculation cost of these six algorithms is analyzed through calculating the operation number of their real multiplication and real addition under the same parameter setting as Figure 3.
The DFT value with  points can be derived by fast Fourier transform (FFT), that is, log 2 /2 complex multiplication and log 2  complex addition.Besides, 1 complex multiplication needs 4 real multiplications and 2 real additions; 1 complex addition needs 2 real additions.In order to decrease the calculation cost, FDP algorithm also integrates coarse and fine search, where the point of coarse search is 2.Contemporarily, 1 modular operation needs 2 real multiplications and 1 real addition; 1 comparing operation needs 1 real addition.Consequently, the operation numbers of real multiplication  FDP and real addition  FDP of FDP algorithm are The coarse search of DS algorithm adopts FDP one with  = 2 points, so it needs 4log 2  + 8 real multiplication and 6log 2  + 10 − 1 real addition.Also, 1 iteration needs  complex multiplication,  − 1 complex addition, 4 real additions, and 1 modular operation.Accordingly, the operation numbers of real multiplication  DS and real addition  DS of DS algorithm are The coarse search of Modified DS algorithm adopts FDP one with  =  points, so it needs 2log 2  + 2 real multiplication and 3log 2  + 2 − 1 real addition.Also, 1 iteration needs 2 complex multiplication, 2 − 2 complex addition, 6 real additions, and 2 modular operations.Accordingly, the operation numbers of real multiplication  M-DS and real addition  M-DS of Modified DS algorithm are The coarse search of IIFC algorithm adopts FDP one with  =  points, so it needs 2log 2  + 2 real multiplication and 3log 2  + 2 − 1 real addition.Also, 1 iteration needs 2 complex multiplication, 2 − 2 complex addition, 6 real additions, and 2 modular operations.Accordingly, the operation numbers of real multiplication  IIFC and real addition  IIFC of IIFC algorithm are The coarse search of RCTSL algorithm adopts FDP one with  = 4 points, so it needs 8log 2  + 24 real multiplication and 12log 2 +32−1 real addition.Also, it needs 7 real multiplications and 4 real additions.Accordingly, The coarse search of LS-SVR algorithm adopts FDP one with  = 2 points, so it needs 4log 2  + 8 real multiplication and 6log 2  + 10 − 1 real addition.Also, if , , and  are fixed, matrices Q and Q  are already known, so the value of matrix  with 2 − 1 rows and  columns can be calculated.As a result, it only has to add the calculation cost for (11) and comparison of 2 − 1 real number.It is worth stressing that ( 11) is used for real and image part, respectively.Accordingly, the operation numbers of real multiplication  LS-SVR and real addition  LS-SVR of LS-SVR algorithm are As in Figure 7, the operation numbers of real multiplication and real addition of these six algorithms are listed in Table 1.Additionally,  = 32 is taken, for example, to compare.From Table 1, the calculation cost of FDP one is much larger than the other five; second larger is Modified DS one and the next is DS one.The calculation cost of LS-SVR one is between IIFC and RCTSL one, and the distances between each of two are very small.

Conclusions
The MSE performance and the calculation cost in DFT-based frequency estimation algorithm are a pair relationship of interaction and interrestriction.The better its MSE performance is, the higher its calculation cost is.However, very high calculation cost will restrict the algorithm's real time capacity.So, how to compromise between these two factors is a hot and key point.
This paper utilizes LS-SVR's good generalizing and fitting capabilities while keeping lower computational complexity and proposes a LS-SVR-based interpolation frequency estimation algorithm.Firstly, the sparser DFT points are learned and the continuous spectrum with less fitting errors is predicted.Then, because any output of LS-SVR is the linear weighted sum of all inputs, LS-SVR is used as a linear interpolator to resample the DFT points densely and decrease the calculation cost.At last, the resampling DFT points between the left and right of its maximum value are searched to derive the frequency estimation value.
In order to compare the calculation cost of the proposed and other DFT-based algorithms, the parameters of all these algorithms are set to ensure nearly the same precondition.Stressed that, the sample size, number of DFT points, and resampling points are already known.Our results show that the proposed algorithm can make a good compromise between MSE performance and calculation cost under the assumption mentioned above.
At the same time, the proper ranges of LS-SVR's parameters are obtained.As a next step, how to select them exactly is an important research point.