Journal of Applied Mathematics

Volume 2014 (2014), Article ID 694053, 8 pages

http://dx.doi.org/10.1155/2014/694053

Research Article

Data Filtering Based Recursive Least Squares Algorithm for Two-Input Single-Output Systems with Moving Average Noises

Xianling Lu,1,2 Wei Zhou,² and Wenlin Shi²

¹Key Laboratory of Advanced Process Control for Light Industry, Ministry of Education, Jiangnan University, Wuxi 214122, China

²School of Internet of Things Engineering, Jiangnan University, Wuxi 214122, China

Received 26 September 2013; Revised 11 February 2014; Accepted 27 February 2014; Published 26 March 2014

Academic Editor: Hak-Keung Lam

Copyright © 2014 Xianling Lu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

This paper studies identification problems of two-input single-output controlled autoregressive moving average systems by using an estimated noise transfer function to filter the input-output data. Through data filtering, we obtain two simple identification models, one containing the parameters of the system model and the other containing the parameters of the noise model. Furthermore, we deduce a data filtering based recursive least squares method for estimating the parameters of these two identification models, respectively, by replacing the unmeasurable variables in the information vectors with their estimates. The proposed algorithm has high computational efficiency because the dimensions of its covariance matrices become small. The simulation results indicate that the proposed algorithm is effective.

1. Introduction

Studies on identification methods have been active in recent years [1–3]. The recursive least squares algorithm is a popular and important identification method for many different systems [4–6]. Recently, Wang and Ding presented an input-output data filtering based recursive least squares parameter estimation for CARARMA systems [7]; Wang et al. proposed a data filtering based recursive least squares algorithm for Hammerstein systems using the key-term separation principle [8]; and Ding and Duan presented a two-stage parameter estimation algorithm for Box-Jenkins systems [9]. Hu proposed an iterative and recursive least squares estimation algorithm for moving average systems [10].

The filtering technique has received much attention in the field of system identification [7, 11, 12] and signal processing [13, 14]. For example, Xie et al. studied recursive least squares parameter estimation methods for nonuniformly sampled systems based on data filtering [11]; Wang et al. discussed filtering based recursive least squares algorithm for Hammerstein nonlinear FIR-MA systems [12]; Wang proposed a filtering and auxiliary model-based recursive least squares identification algorithm for output error moving average systems [15]; Shi and Fang developed a recursive algorithm for parameter estimation by modifying the Kalman filter-based algorithm after designing a missing output estimator [16]; and Wang et al. derived a hierarchical generalized stochastic gradient algorithm and a filtering based hierarchical stochastic gradient algorithm to estimate the parameter vectors and parameter matrix of the multivariable colored noise systems by using the hierarchical identification principle [17].

For several decades, multiple-input single-output systems [18] or multiple-input multiple-output systems [19, 20] have attracted researchers’ attention, but most of the work focused on the single-input single-output systems [21]. For example, Li proposed parameter estimation for Hammerstein controlled autoregressive moving average systems based on the Newton iteration [22]. Yao and Ding derived a two-stage least squares based iterative identification algorithm for controlled autoregressive moving average (CARMA) systems; the basic idea is to decompose a CARMA system into two subsystems and to identify each subsystem, respectively [23]. This paper considers the identification problems of two-input single-output controlled autoregressive moving average systems by using input-output data filtering and derives a data filtering based recursive least squares method. The proposed algorithm has high computational efficiency because the dimensions of its covariance matrices become small. Although this paper focuses on two-input single-output systems, the proposed method can be extended to multiple-input single-output systems.

The rest of the paper is organized as follows. Section 2 proposes a data filtering based recursive least squares algorithm for a two-input single-output system with moving average noise. Section 3 introduces the recursive extended least squares algorithm for comparison. In Section 4, we give an example to prove the effectiveness of the proposed algorithm. Finally, concluding remarks are given in Section 5.

2. Data Filtering Based Recursive Least Squares Algorithm

Consider the two-input single-output system, described by the following controlled autoregressive moving average model, depicted in Figure 1:

where are the input sequences of the system, is the output sequence of the system, is a white noise sequence with zero mean and variance , and , , , and are the polynomials in the unit backward shift operator [i.e., ] and defined by

Assume that the degrees , , , and are known and , , and and for .

Figure 1: The two-input single-output system with moving average noise.

Define the parameter vector and the information vector as

The goal of this paper is to apply the data filtering technique and to develop a new recursive least squares for estimating the system parameters.

If we use the rational fraction (a liner filter) to filter the input-output data, we can get a simple “equation error model” which is easy to identify, then the recursive least squares algorithm can be applied. Because is unknown, we use its estimate to filter the input-output data [7]. The identification method based on this approach will be referred to as the data filtering based recursive least squares (F-RLS) method.

For the model in (1), define the filtered inputs and , the filtered output , and the filtered information vector as

Dividing both sides of (1) by gives

It can be written as

This filtered model is an equation error model and can be rewritten in a vector form

Define the inner variable:

For two identification models (7) and (8), we can obtain the following recursive least squares algorithm for computing the estimates and of and :

Note that the filtered input , the filtered input , and the filtered output are all unknown because of the unknown polynomial and the unmeasurable noise term in the information vector and are unknown. So it is impossible to implement the algorithm in (9)–(14). The solution we adopted here is to replace the unknown variables with their estimates according to the auxiliary model identification idea [24–26].

From (1), we get

Substituting (8) into the above equation, we get

Replacing on the right-hand side of (15) with its estimate , the estimate can be computed by . Let be the estimate of and construct the estimate of as

From (8), we have . Replacing and with and , the estimate can be computed by .

Using the parameter estimates of the noise model,

to construct the estimate of ,

Filter , , and with to get the estimates of , , and as follows:

From the above equations, we can recursively compute , , and by the following equations:

Construct the estimate of the :

Replacing the unknown information vector in (9)–(11) with , in (9) with , in (12)–(14) with , and the unknown noise terms in (12) with , we obtain the data filtering based recursive least squares (F-RLS) algorithm for estimating the parameter vectors and for the two-input single-output system [1]:

The data filtering based recursive least squares algorithm has high computational efficiency because the dimensions of its covariance matrices become small and can generate more accurate parameter estimation. To initialize the algorithm, we take

The steps involved in the F-RLS algorithms are listed as follows.

(1)Set , , for .(2)Let ; set the initial values of the parameter estimation vectors and the covariance matrices according to (39), and , , , , for .(3)Collect the input–output data , , and and construct the information vectors by (36), by (26), and by (33).(4)Compute by (34), the gain vector by (31) and the covariance matrix by (32).(5)Update the parameter estimate by (30).(6)Compute by (35), by (27), by (28), and by (29).(7)Compute the gain vector by (24) and the covariance matrix by (25).(8)Update the parameter estimate by (23).(9)Increase by 1; go to Step .

3. The RELS Algorithm

To show the advantages of the algorithm we proposed, we give the recursive extended least squares (RELS) algorithm for comparison.

Let be the estimate of . Based on the identification model in (16), the unknown variables in the information vector are replaced with their estimates , so we can obtain the following recursive extended least squares algorithm for identifying the parameter vector :

In this RELS algorithm, the forgetting factor used is 1.

4. Example

Consider the following example:

The inputs , are taken as two uncorrelated persistent excitation signal sequences with zero mean and unit variance, as a white noise sequence with zero mean and variance and , and the corresponding noise-to-signal ratio are and , respectively. Applying the RELS and the F-RLS algorithms to estimate the parameters of the system, the parameter estimates and their errors are shown in Tables 1 and 2, and the estimation errors versus are shown in Figure 2 with .

Table 1: The parameter estimates and their errors (, ).

Algorithms

(%)

F-RLS
100
0.49783
0.82362
0.47259
0.33889
0.45701
0.67571
−0.31256
10.87062

200
0.53237
0.83678
0.42607
0.30153
0.50390
0.70802
−0.26142
13.33636

500
0.51680
0.83056
0.43167
0.29518
0.50208
0.66555
−0.35787
6.59957

1000
0.50285
0.81789
0.42489
0.28951
0.50703
0.61872
−0.35247
4.41100

2000
0.50405
0.81391
0.42283
0.27871
0.49694
0.62908
−0.38038
3.56288

3000
0.50565
0.80834
0.41436
0.29579
0.49903
0.62586
−0.39109
2.37173

RELS
100
0.53926
0.86216
0.45846
0.28535
0.44404
0.76552
−0.23374
18.60645

200
0.54959
0.85622
0.40695
0.27414
0.46667
0.76896
−0.23790
18.13522

500
0.51636
0.83069
0.42841
0.27674
0.48978
0.67084
−0.38132
6.46314

1000
0.50859
0.81827
0.42018
0.27842
0.50739
0.62544
−0.36367
4.24112

2000
0.50723
0.81444
0.42236
0.27730
0.49418
0.62936
−0.38827
3.47473

3000
0.50794
0.80920
0.41212
0.28907
0.49876
0.62707
−0.39464
2.48087

True values
0.50000
0.80000
0.40000
0.30000
0.50000
0.60000
−0.40000

Table 2: The parameter estimates and their errors (, ).

Algorithms

(%)

F-RLS
100
0.48293
0.79642
0.41760
0.30069
0.49608
0.58878
−0.40118
1.99087

200
0.51101
0.81127
0.40743
0.30352
0.50298
0.61790
−0.37052
2.81593

500
0.50157
0.81012
0.40710
0.29963
0.50121
0.61021
−0.39941
1.17016

1000
0.49914
0.80525
0.40512
0.29771
0.50204
0.60182
−0.38316
1.35563

2000
0.50007
0.80429
0.40469
0.29566
0.49964
0.60499
−0.39422
0.78466

3000
0.49969
0.80258
0.40296
0.29869
0.49998
0.60404
−0.40137
0.43055

RELS
100
0.51462
0.82354
0.41087
0.30152
0.48869
0.64030
−0.14920
18.69550

200
0.51683
0.81951
0.40067
0.29934
0.49279
0.63819
−0.18953
15.98366

500
0.50478
0.81270
0.40486
0.29674
0.49758
0.61544
−0.33064
5.07263

1000
0.50338
0.80708
0.40371
0.29683
0.50111
0.60612
−0.34207
4.35710

2000
0.50298
0.80552
0.40417
0.29637
0.49871
0.60680
−0.37711
1.81881

3000
0.50223
0.80344
0.40231
0.29826
0.49969
0.60581
−0.38867
0.98907

True values
0.50000
0.80000
0.40000
0.30000
0.50000
0.60000
−0.40000

Figure 2: The estimation errors versus ().

From Tables 1 and 2 and Figure 2, we can draw the following conclusions.

(i)The parameter estimation errors become (generally) smaller and smaller with the data length increasing. This shows that the proposed algorithm is effective.(ii)The F-RLS algorithm is more accurate than the RELS algorithm. This means that the proposed F-RLS algorithm has better identification performance compared with the RELS algorithm.(iii)The parameter estimates given by the F-RLS algorithm converge fast to their true values compared with the RELS algorithm.(iv)The F-RLS algorithm has a higher computational efficiency than the RELS algorithm because the dimensions of its covariance matrices are smaller than those of the covariance in the RELS algorithm.

5. Conclusions

The data filtering based recursive least squares algorithm for the two-input single-output system with moving average noise is proposed by means of the data filtering technique. Compared with the recursive least squares algorithm, the proposed algorithms can require less computational load and can give more accurate parameter estimates compared with the recursive extended least squares algorithm. The proposed method can be extended to nonuniformly sampled systems and nonlinear systems. The convergence analysis of the proposed filtering based algorithm is worth further studies. The proposed method can combine the multi-innovation identification methods [27–36], the hierarchical identification methods [37–42], the auxiliary model identification methods [43–47], the iterative identification methods [48, 49], and other identification methods [50–53] to study identification and adaptive control problems for linear or nonlinear, single-rate or dual-rate, and scalar or multivariable systems [54–61].

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This work was supported by the Fundamental Research Funds for the Central Universities (no. JUSRP21129) and a Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD).

References

F. Ding, System Identification—New Theory and Methods, Science Press, Beijing, China, 2013.

W. Xiong, W. Fan, and R. Ding, “Least-squares parameter estimation algorithm for a class of input nonlinear systems,” Journal of Applied Mathematics, vol. 2012, Article ID 684074, 14 pages, 2012.

H. Zhang and F. Ding, “On the Kronecker products and their applications,” Journal of Applied Mathematics, vol. 2013, Article ID 296185, 8 pages, 2013.

L. Ljung and T. Sodestrom, Theory and Practice of Recursive Estimation, MIT Press, Cambridge, Mass, USA, 1987.

F. Ding, X. M. Liu, H. B. Chen, and G. Y. Yao, “Hierarchical gradient based and hierarchical least squares based iterative parameter identification for CARARMA systems,” Signal Process, vol. 97, pp. 31–39, 2014.

F. Ding, “Combined state and least squares parameter estimation algorithms for dynamic systems,” Applied Mathematical Modelling. Simulation and Computation for Engineering and Environmental Systems, vol. 38, no. 1, pp. 403–412, 2014.

D. Wang and F. Ding, “Input-output data filtering based recursive least squares identification for CARARMA systems,” Digital Signal Processing, vol. 20, no. 4, pp. 991–999, 2010.

D. Wang, F. Ding, and Y. Chu, “Data filtering based recursive least squares algorithm for Hammerstein systems using the key-term separation principle,” Information Sciences, vol. 222, pp. 203–212, 2013.

F. Ding and H. H. Duan, “Two-stage parameter estimation algorithms for Box-Jenkins systems,” IET Signal Processing, vol. 7, no. 8, pp. 646–654, 2013.

Y. B. Hu, “Iterative and recursive least squares estimation algorithms for moving average systems,” Simulation Modelling Practice and Theory, vol. 34, pp. 12–19, 2013.

L. Xie, H. Yang, and F. Ding, “Recursive least squares parameter estimation for non-uniformly sampled systems based on the data filtering,” Mathematical and Computer Modelling, vol. 54, no. 1-2, pp. 315–324, 2011.

Z. Wang, Y. Shen, Z. Ji, and R. Ding, “Filtering based recursive least squares algorithm for Hammerstein FIR-MA systems,” Nonlinear Dynamics, vol. 73, no. 1-2, pp. 1045–1054, 2013.

M. S. Ahmad, O. Kukrer, and A. Hocanin, “Recursive inverse adaptive filtering algorithm,” Digital Signal Processing, vol. 21, no. 4, pp. 491–496, 2011.

Y. Shi and H. Fang, “Kalman filter-based identification for systems with randomly missing measurements in a network environment,” International Journal of Control, vol. 83, no. 3, pp. 538–551, 2010.

D. Q. Wang, “Least squares-based recursive and iterative estimation for output error moving average systems using data filtering,” IET Control Theory & Applications, vol. 5, no. 14, pp. 1648–1657, 2011.

Y. Shi and H. Fang, “Kalman filter-based identification for systems with randomly missing measurements in a network environment,” International Journal of Control, vol. 83, no. 3, pp. 538–551, 2010.

D. Wang, T. Shan, and R. Ding, “Data filtering based stochastic gradient algorithms for multivariable CARAR-like systems,” Mathematical Modelling and Analysis, vol. 18, no. 3, pp. 374–385, 2013.

Y. Liu, Y. Xiao, and X. Zhao, “Multi-innovation stochastic gradient algorithm for multiple-input single-output systems using the auxiliary model,” Applied Mathematics and Computation, vol. 215, no. 4, pp. 1477–1483, 2009.

F. Ding, Y. Liu, and B. Bao, “Gradient-based and least-squares-based iterative estimation algorithms for multi-input multi-output systems,” Proceedings of the Institution of Mechanical Engineers. Part I: Journal of Systems and Control Engineering, vol. 226, no. 1, pp. 43–55, 2012.

F. Ding, “Coupled-least-squares identification for multivariable systems,” IET Control Theory & Applications, vol. 7, no. 1, pp. 68–79, 2013.

F. Ding, “Two-stage least squares based iterative estimation algorithm for CARARMA system modeling,” Applied Mathematical Modelling, vol. 37, no. 7, pp. 4798–4808, 2013.

J. Li, “Parameter estimation for Hammerstein CARARMA systems based on the Newton iteration,” Applied Mathematics Letters, vol. 26, no. 1, pp. 91–96, 2013.

G. Yao and R. Ding, “Two-stage least squares based iterative identification algorithm for controlled autoregressive moving average (CARMA) systems,” Computers & Mathematics with Applications, vol. 63, no. 5, pp. 975–984, 2012.

F. Ding and T. Chen, “Combined parameter and output estimation of dual-rate systems using an auxiliary model,” Automatica, vol. 40, no. 10, pp. 1739–1748, 2004.

F. Ding and T. Chen, “Parameter estimation of dual-rate stochastic systems by using an output error method,” IEEE Transactions on Automatic Control, vol. 50, no. 9, pp. 1436–1441, 2005.

F. Ding and J. Ding, “Least-squares parameter estimation for systems with irregularly missing data,” International Journal of Adaptive Control and Signal Processing, vol. 24, no. 7, pp. 540–553, 2010.

F. Ding and T. Chen, “Performance analysis of multi-innovation gradient type identification methods,” Automatica, vol. 43, no. 1, pp. 1–14, 2007.

F. Ding, P. X. Liu, and G. Liu, “Multiinnovation least-squares identification for system modeling,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 40, no. 3, pp. 767–778, 2010.

F. Ding, P. X. Liu, and G. Liu, “Auxiliary model based multi-innovation extended stochastic gradient parameter estimation with colored measurement noises,” Signal Processing, vol. 89, no. 10, pp. 1883–1890, 2009.

F. Ding, “Several multi-innovation identification methods,” Digital Signal Processing, vol. 20, no. 4, pp. 1027–1039, 2010.

F. Ding, “Hierarchical multi-innovation stochastic gradient algorithm for Hammerstein nonlinear system modeling,” Applied Mathematical Modelling, vol. 37, no. 4, pp. 1694–1704, 2013.

F. Ding, H. Chen, and M. Li, “Multi-innovation least squares identification methods based on the auxiliary model for MISO systems,” Applied Mathematics and Computation, vol. 187, no. 2, pp. 658–668, 2007.

L. Han and F. Ding, “Multi-innovation stochastic gradient algorithms for multi-input multi-output systems,” Digital Signal Processing, vol. 19, no. 4, pp. 545–554, 2009.

D. Wang and F. Ding, “Performance analysis of the auxiliary models based multi-innovation stochastic gradient estimation algorithm for output error systems,” Digital Signal Processing, vol. 20, no. 3, pp. 750–762, 2010.

L. Xie, Y. J. Liu, H. Z. Yang, and F. Ding, “Modelling and identification for non-uniformly periodically sampled-data systems,” IET Control Theory & Applications, vol. 4, no. 5, pp. 784–794, 2010.

Y. Liu, L. Yu, and F. Ding, “Multi-innovation extended stochastic gradient algorithm and its performance analysis,” Circuits, Systems, and Signal Processing, vol. 29, no. 4, pp. 649–667, 2010.

F. Ding and T. Chen, “Hierarchical least squares identification methods for multivariable systems,” IEEE Transactions on Automatic Control, vol. 50, no. 3, pp. 397–402, 2005.

F. Ding and T. Chen, “Hierarchical identification of lifted state-space models for general dual-rate systems,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 52, no. 6, pp. 1179–1187, 2005.

J. Ding, F. Ding, X. P. Liu, and G. Liu, “Hierarchical least squares identification for linear SISO systems with dual-rate sampled-data,” IEEE Transactions on Automatic Control, vol. 56, no. 11, pp. 2677–2683, 2011.

Y. Liu, F. Ding, and Y. Shi, “Least squares estimation for a class of non-uniformly sampled systems based on the hierarchical identification principle,” Circuits, Systems, and Signal Processing, vol. 31, no. 6, pp. 1985–2000, 2012.

Z. Zhang, F. Ding, and X. Liu, “Hierarchical gradient based iterative parameter estimation algorithm for multivariable output error moving average systems,” Computers & Mathematics with Applications, vol. 61, no. 3, pp. 672–682, 2011.

D. Wang, R. Ding, and X. Dong, “Iterative parameter estimation for a class of multivariable systems based on the hierarchical identification principle and the gradient search,” Circuits, Systems, and Signal Processing, vol. 31, no. 6, pp. 2167–2177, 2012.

F. Ding, Y. Shi, and T. Chen, “Auxiliary model-based least-squares identification methods for Hammerstein output-error systems,” Systems & Control Letters, vol. 56, no. 5, pp. 373–380, 2007.

F. Ding and T. Chen, “Identification of dual-rate systems based on finite impulse response models,” International Journal of Adaptive Control and Signal Processing, vol. 18, no. 7, pp. 589–598, 2004.

F. Ding and Y. Gu, “Performance analysis of the auxiliary model-based least-squares identification algorithm for one-step state-delay systems,” International Journal of Computer Mathematics, vol. 89, no. 15, pp. 2019–2028, 2012.

F. Ding and Y. Gu, “Performance analysis of the auxiliary model-based stochastic gradient parameter estimation algorithm for state-space systems with one-step state delay,” Circuits, Systems, and Signal Processing, vol. 32, no. 2, pp. 585–599, 2013.

Y. Gu and F. Ding, “Auxiliary model based least squares identification method for a state space model with a unit time-delay,” Applied Mathematical Modelling, vol. 36, no. 12, pp. 5773–5779, 2012.

F. Ding, X. Liu, and J. Chu, “Gradient-based and least-squares-based iterative algorithms for Hammerstein systems using the hierarchical identification principle,” IET Control Theory & Applications, vol. 7, no. 2, pp. 176–184, 2013.

F. Ding, “Decomposition based fast least squares algorithm for output error systems,” Signal Processing, vol. 93, no. 5, pp. 1235–1242, 2013.

F. Ding, G. Liu, and X. P. Liu, “Partially coupled stochastic gradient identification methods for non-uniformly sampled systems,” IEEE Transactions on Automatic Control, vol. 55, no. 8, pp. 1976–1981, 2010.

Y. Liu, J. Sheng, and R. Ding, “Convergence of stochastic gradient estimation algorithm for multivariable ARX-like systems,” Computers & Mathematics with Applications, vol. 59, no. 8, pp. 2615–2627, 2010.

F. Ding, G. Liu, and X. P. Liu, “Parameter estimation with scarce measurements,” Automatica, vol. 47, no. 8, pp. 1646–1655, 2011.

F. Ding, P. X. Liu, and H. Yang, “Parameter identification and intersample output estimation for dual-rate systems,” IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans 38, vol. 38, no. 4, pp. 966–975, 2008.

F. Ding, T. Chen, and Z. Iwai, “Adaptive digital control of Hammerstein nonlinear systems with limited output sampling,” SIAM Journal on Control and Optimization, vol. 45, no. 6, pp. 2257–2276, 2007.

J. Zhang, F. Ding, and Y. Shi, “Self-tuning control based on multi-innovation stochastic gradient parameter estimation,” Systems & Control Letters, vol. 58, no. 1, pp. 69–75, 2009.

D. Wang and F. Ding, “Least squares based and gradient based iterative identification for Wiener nonlinear systems,” Signal Processing, vol. 91, no. 5, pp. 1182–1189, 2011.

D. Q. Wang and F. Ding, “Hierarchical least squares estimation algorithm for Hammerstein-Wiener systems,” IEEE Signal Processing Letters, vol. 19, no. 12, pp. 825–828, 2012.

D. Wang, F. Ding, and L. Ximei, “Least squares algorithm for an input nonlinear system with a dynamic subspace state space model,” Nonlinear Dynamics, vol. 75, no. 1-2, pp. 49–61, 2014.

F. Ding, X. P. Liu, and G. Liu, “Identification methods for Hammerstein nonlinear systems,” Digital Signal Processing, vol. 21, no. 2, pp. 215–238, 2011.

Y. Liu, F. Ding, and Y. Shi, “An efficient hierarchical identification method for general dual-rate sampled-data systems,” Automatica, vol. 50, no. 3, pp. 962–973, 2014.

X. L. Luan, S. Y. Zhao, and F. Liu, “H-infinity control for discrete-time markov jump systems with uncertain transition probabilities,” IEEE Transactions on Automatic Control, vol. 58, no. 6, pp. 1566–1572, 2013.