Journal of Applied Mathematics

Volume 2013 (2013), Article ID 318786, 8 pages

http://dx.doi.org/10.1155/2013/318786

Research Article

Recursive Identification for Dynamic Linear Systems from Noisy Input-Output Measurements

Dan Fan and Kueiming Lo

School of Software, Tsinghua University, Tsinghua National Laboratory for Information Science and Technology, Beijing 100084, China

Received 7 March 2013; Accepted 13 April 2013

Academic Editor: Xiaojing Yang

Copyright © 2013 Dan Fan and Kueiming Lo. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Errors-in-variables (EIV) model is a kind of model with not only noisy output but also noisy input measurements, which can be used for system modeling in many engineering applications. However, the identification for EIV model is much complicated due to the input noises. This paper focuses on the adaptive identification problem of real-time EIV models. Some derivation errors in an accuracy research of the popular Frisch scheme used for EIV identification have been pointed out in a recent study. To solve the same modeling problem, a new algorithm is proposed in this paper. A Moving Average (MA) process is used as a substitute for the joint impact of the mutually independent input and output noises, and then system parameters and the noise properties are estimated in the view of the time domain and frequency domain separately. A recursive form of the first step calculation is constructed to improve the calculation efficiency and online computation ability. Another advantage of the proposed algorithm is its applicableness to different input processes situations. Numerical simulations are given to demonstrate the efficiency and robustness of the new algorithm.

1. Introduction

In the field of engineering, modeling is an essential issue. In most cases, the systems are modeled by stochastic models in which the input signals are assumed to be measured exactly and all the disturbed noises are added to the output signals; that is, only the output measurements are noisy. These models are called “errors-in-equation models.” However, there are always signals beyond our control that also affect the input of the systems; some of them cannot be included in the output noises. Therefore, it is also necessary to consider the modeling problem for those systems with noisy input-output measurements, especially when we concern the actual physical laws of the process rather than the prediction of the future behaviour [1]. This kind of model whose input and output measurements are both containing noise is called “errors-in-variables (EIV) model [2].”

The identification of EIV models has received a lot of attention during the past decades. By far, EIV models have been used in numerous applications, such as the modeling problems in econometrics, computer vision, biomedicine, chemical and image reconstruction, spectrum estimation, speech analysis, noise cancelation, and digital communications [3–8].

In EIV models, the noise in input measurements cannot be equivalent to the output error, which makes the identification of EIV models much more difficult. The identifiability of EIV dynamic models was analyzed in [9, 10]. It is pointed out that EIV dynamic models cannot be uniquely identified from the second-order properties [9]. Thus specific prior knowledge is needed to achieve the identifiability. Once the identifiability is established, estimation algorithms can be developed [10]. Owing to the noisy input measurements in the EIV models, the standard least squares method for errors-in-equation models cannot yield consistent estimates anymore. To overcome this problem, a bias-compensated least squares (BCLSs) principle was proposed in [4]. On the basis of BCLS principle, various algorithms have been developed, such as the Frisch scheme-based algorithms [7], the KL algorithm [8], ECLS [9], BELS [10], and others in [11–15].

Although there are such a number of approaches for identifying different EIV models, the convergence of the algorithms has always been a difficulty. Only a few literatures have tried to solve this problem [12, 15, 16]. In [16], the accuracy of Frisch scheme for EIV identification was analyzed, in which the estimates of the system parameters as well as the noise variance were both proved asymptotically Gaussian distributed by linearizing three primary equations in this scheme. This conclusion can be perceived as the theoretical support of the Frisch scheme-based algorithms. Based on this work, continued extensions and real applications have sprung up recently [17–20], which reaffirms the value of this particular analysis result. However, the analysis in [16] needs a condition that the estimates of the parameters are close to their true values, which is not clear how to be guaranteed. A counterexample that could not converge was present in [21], and in addition, some derivation errors of [16] were found and discussed at the same time. Furthermore, another method was provided to identify the EIV model in [21]. But comparing to the model concerned in [16], due to the difficulty of the identification problem, the one considered in [21] was a simpler one with a stronger condition that the input and output noise processes had the same variance, which has been hampering its application in some degree.

The purpose of this paper is to consider how to avoid the restrict condition in [21]; in other words, we are trying to solve the same modeling problem as in [16], that is, to propose an identification algorithm for the modeling of dynamic EIV systems with independent input and output noises to estimate both the unknown system parameters and the noise signals. In order to achieve this purpose, we used a two-step method: in Step 1, the original model is rewritten into another form to get the system parameters in the time domain; in Step 2, the noise variances are calculated in the frequency domain. Moreover, the recursive form of the proposed method will be presented to improve its operational efficiency and enhance its online applicability.

The structure of the paper is as follows. In Section 2, the concerned model is described in detail. The new identification algorithm is presented in Section 3. Some simulations are given in Section 4 to illustrate the identification accuracy, the convergence rate, and the antinoise performance. Finally, conclusion remarks are given in Section 5.

2. Problem Formulation

A basic dynamic EIV system is shown in Figure 1.

Figure 1: A basic errors-in-variables system.

Unlike the normal errors-in-equation model, as mentioned before, the EIV model has noise in both input measurements and output measurements. The immeasurable true input and output processes and are linked by a dynamic system, which can be a linear or a nonlinear system in different applications. So far, most of the related studies are focused on the linear systems, which is also the focus of research in this paper. The model is considered here as

where

are the polynomials in the backward shift operator . The are the unknown system parameters to be identified, while the measured variables and are disturbed by the unknown noises and . Thus the input and output measurements are

After introducing the notations

the EIV system can be described as the following model:

To ensure the identifiability, we list some assumptions first.

The EIV system is asymptotically stable, which means that there is no zero of inside the unit circle. The noises and are mutually independent and also independent of the true input and output signals and . and are white noises with zero mean and independent variances and .

The problem we need to solve is to estimate the system parameter vector with the help of the measured regressor vector . Furthermore, considering that a noise process can be described by the mean and variance, to identify the zero-mean input and output noises is simplified to identify the variances. Therefore, except for the system parameters, we also want to estimate the output and input noise variances and . In the following section, we will give an algorithm with two independent steps to fulfill the two aspects of the estimate requirements.

3. Identification Algorithms

As mentioned before, the identification for EIV system is much more difficult because the input and output noises are unknown. For the EIV system described in Section 2, to overcome the influence of the input noise, we will use another MA process as a substitute for the joint impact of the mutually independent input and output noises, as two mutually independent sequences of independent random variables can be represented as an MA process which has the same spectra with the two jointly sequences [22]. Then the system can be modified as an ARMAX model, and what need to do is changed to estimate the system parameters of the new model and to determine the variances of the input/output noises in terms of . Thus a two-step recursive estimation algorithm can be constructed to identify the system parameters and the noise variances and , respectively.

Step  1. for the time , we used the obtained estimation of to estimate the parameters and get the current estimates and .

Step  2. These results are utilized to calculate the estimates of the noise variance and .

In the following, we will give the algorithm followed by proof.

Step 1   (estimation for the unkown system parameter ). For convenience, denote the last two terms of (5) by , that is,

where and are mutually independent with

Introduce an process

where is white noise with

It can be shown that we can find a pair of and such that and have the same spectra [22], which means that can be represented by in (8) as

The and are intermediate variables.

For the new model (10), denote a new parameter vector and a new regressor vector by

and then the EIV system (5) can be rewritten as

In this step, we will give a recursive algorithm to identify (13).

The covariance matrix of the regressor vector and output variables is denoted by

For convenience, introduce

Assume that the input is a stationary process; in the calculation, we can use the algebra means and instead of the mathematical expectations and in (14), as by ergodicity, we have

Lemma 1 (Matrix Inversion Formula [23]). For the matrices , , and , the inverse matrix of is

where .

Theorem 2. For system (13), under the assumptions (A1)–(A3), the parameter vector can be estimated recursively as follows with a large and arbitrary :

where .

Proof. Like the least squares methods, we use the covariance matrix for help. By multiplying with to the system model (13), we have

with assumptions (A2) and (A3) which can be rewritten as

Replacing and with and in (15) and (16), respectively, as mentioned before, one has

We note that (15) and (16) imply

Then on condition that is reversible, is estimated as

Using (23), we can calculate the vector . But we note that there is an inverse operation of at each recursive step, which is a very time-consuming process. To avoid the inversing, introduce

and apply the Matrix Inversion Formula in Lemma 1 to (24); taking , we have

Moreover, by (26) it is clear that

Taking (27) into (23), we have

Noting that in (12) is unknown, cannot be constructed directly. This problem can be solved in the similar way as for RPLR (recursive pseudolinear regression) algorithm [22], that is, to form a substitute for as

where , . The proof of the substitution’s correctness is omitted (see details in [22]).

Then using defined by (29) to replace the in (26) and (28), we get Theorem 2 easily.

Theorem 2 gives a recursive algorithm to get the estimate of : At time , we store only the finite-dimensional information . At time , it is updated using (23), (27), and (29), which is done with a given fixed amount of operations, making it a high operational efficiency and suitable for online applications. Since is obtained, obviously can be easily got by (11).

Next we go to estimate the noise properties, which will be helpful in real applications such as the cascade system modeling.

Step 2 (estimation for the noise variances and ). To estimate the noise variances , , we need to find the relationship between in (6) and in (8).

We know that the spectrum of a signal is the Fourier transform of its covariance function as

where

Since and have the same spectra, that is,

this means that for all ,

Thus we can find the relationship between , , and by using the covariance functions and .

At step , an estimate of can be used as

We switched to the notation , rather than , to account for certain differences due to recursive step .

Introduce

where

are the transformation of the parameters in .

Then according to (A2), (A3), (6), (8), and (34), we have

Similarly we have

Then it is clear that for ,

is a set of linear equations about the unknown and , in which is known at time , and can be estimated by:

by taking in Theorem 2 as the estimate of .

Then the estimates and for the output/input noise variances and can be calculated by (37)–(40).

4. Simulation Examples

This section addresses some numerical evaluation of the identification algorithm presented in this paper. Matlab 7.7 is used to do the simulations. To demonstrate its validness to various EIV systems, we have chosen different signal processes as the true input variables in each case: in Case A, a zero-mean Gaussian process is used; in Case B, a sawtooth signal is applied; in Case C, it is an ARMA process. The noise processes and in these cases are mutually uncorrelated white noise signals with zero-mean. The robustness of the algorithm is also tested, which is shown in Case C.

Case A. First we examine how well the algorithm works for systems with Gaussian input. Consider an EIV dynamic system with and

It is easy to get the system as follows, which is denoted by System 1:

Let the input signal be a zero-mean Gaussian process whose variance equals 1. Let the noise signals and be mutually uncorrelated white noise signals with , , which means a strong noise environment for the system.

The system is simulated for steps. Calculation results are listed in Table 1, where the calculation error is defined by the standard deviation. Figures 2 and 3 show the system parameter and the noise variances estimates separately. Solid lines indicate the true values and dashed lines denote the corresponding estimates. Noting that the vertical coordinate scopes are very small in both figures, it can be seen easily that the estimates are converging fast to the true parameters.

Table 1: Estimation results of System 1.

Parameter
True value
Estimation

0.30

0.20

0.50

Figure 2: Estimation result of in System 1.

Figure 3: Estimation result of and in System 1.

Case B. Consider another system, System 2, with sawtooth input:

in which , and

This is the counterexample which was presented in [21] to show the unconvergency of the Frisch-based method analyzed in [16]. We use our proposed algorithm to identify this system under the same conditions; that is, the input sawtooth signal’s amplitude equals 1 and its frequency is 10 Hz; the noises’ variances are

The simulation results of the system parameters and the noise variances are all displayed in Figure 4. The true values and estimates are also denoted by solid lines, and dotted lines respectively. We can see that the algorithm has an even better performance for this kind of EIV system. All the estimates converge to their corresponding true values consummately.

Figure 4: Estimates of System 2 by the new recursive method.

Case C. The proposed algorithm in this paper is valid not only for EIV models with iid random input process but also for those whose input signals are more general such as the ARMA process. In order to examine it, we have considered the following system:

where

In this system, the input process is an ARMA process:

with being a zero-mean iid Gaussian process whose variance is

The noise signals and are mutually uncorrelated white noise signals with

The estimation results are shown in Figures 5 and 6. Similarly, the true values are marked by solid lines and the estimates of parameters are marked by dotted lines. We can see that for ARMA input process, the estimation can still converge to the true parameters quickly.

Figure 5: Estimation result of in System 3 with ARMA input process.

Figure 6: Estimation of and in System 3 with ARMA input process.

In order to verify the robustness against noise, we further did several experiments with different signal-to-noise ratios (SNRs) in System 3. The noise processes used in the experiments are shown by the first three columns in Table 2. Column 1 is the signal variables used to generate the input signals; columns 2 and 3 are the variables of input noise and output noise.

Table 2: Comparison of the estimations for different SNRs.

Average error

1
0.05
0.05
0.0025

1
0.2
0.2
0.0039

1
0.2
0.5
0.0040

1
0.5
0.2
0.0049

1
0.5
0.5
0.0052

1
1
1
0.0086

1
1
2
0.0064

The average estimation error of System 3 (including the system parameter estimation and the noises estimation) is listed in column 4. It is clear that the performance of the proposed algorithm is keeping good when the noises increase (even when the noises are equal or larger than the input signals).

5. Conclusions

This paper discussed the identification problem of dynamic errors-in-variables (EIV) systems. EIV model is very useful and has a wide engineering application range such as the modeling of cascade system, and camera calibration. In comparison with the usual errors-in-equation models, EIV model has a more troublesome noise problem with the input measurements being disturbed. Since several severe errors in the previous analysis of the attractive Frisch scheme identification approaches have been presented in the recent studies, we developed an adaptive algorithm to solve the same modeling problem. For the dynamic EIV model with mutually independent input and output noises, this two-step algorithm can not only estimate the system parameter vector as well as the noise variances with greater accuracy but also reduce the computational complexity significantly due to its recursive form. It has been shown by several simulation results that the presented algorithm demonstrates, as shown in the numerical simulations, a great accuracy, fast convergence speed, and good antinoise performance. Theoretical analysis of the proposed algorithm and the identification of some more complicated model such as EIV nonlinear models will be considered in future work.

Acknowledgments

This work is supported by the funds NSFC61171121 and NSFC60973049, the Science Foundation of Chinese Ministry of Education, and China Mobile 2012.

References

T. Söderström, “Errors-in-variables methods in system identification,” Automatica, vol. 43, no. 6, pp. 939–958, 2007.

R. J. Adcock, “Note on the method of least squares,” Analyst, vol. 4, pp. 183–184, 1877.

L. Ng and V. Solo, “Errors-in-variables modeling in optical flow estimation,” IEEE Transactions on Image Processing, vol. 10, no. 10, pp. 1528–1540, 2001.

W. X. Zheng and C. B. Feng, “Unbiased parameter estimation of linear systems in the presence of input and output noise,” International Journal of Adaptive Control and Signal Processing, vol. 3, no. 3, pp. 231–251, 1989.

R. Diversi, “Bias-eliminating least-squares identification of errors-in-variables models with mutually correlated noises,” International Journal of Adaptive Control and Signal Processing, 2012.

R. Diversi, R. Guidorzi, and U. Soverini, “Identification of ARMAX models with noisy input and output,” World Congress, vol. 18, no. 1, pp. 13121–13126, 2011.

S. Beghelli, R. P. Guidorzi, and U. Soverini, “The Frisch scheme in dynamic system identification,” Automatica, vol. 26, no. 1, pp. 171–176, 1990.

K. V. Fernando and H. Nicholson, “Identification of linear systems with input and output noise: the Koopmans-Levin method,” IEE Proceedings D, vol. 132, no. 1, pp. 30–36, 1985.

J. C. Agüero and G. C. Goodwin, “Identifiability of errors in variables dynamic systems,” Automatica, vol. 44, no. 2, pp. 371–382, 2008.

W. X. Zheng, “A bias correction method for identification of linear dynamic errors-in-variables models,” IEEE Transactions on Automatic Control, vol. 47, no. 7, pp. 1142–1147, 2002.

B. D. O. Anderson and M. Deistler, “Identifiability in dynamic errors-in-variables models,” Journal of Time Series Analysis, vol. 5, no. 1, pp. 1–13, 1984.

H.-F. Chen and J.-M. Yang, “Strongly consistent coefficient estimate for errors-in-variables models,” Automatica, vol. 41, no. 6, pp. 1025–1033, 2005.

M. Ekman, “Identification of linear systems with errors in variables using separable nonlinear least-squares,” in Proceedings of the 16th Triennial World Congress of International Federation of Automatic Control (IFAC '05), pp. 815–820, Prague, Czech Republic, July 2005.

K. Mahata, “An improved bias-compensation approach for errors-in-variables model identification,” Automatica, vol. 43, no. 8, pp. 1339–1354, 2007.

H. J. Palanthandalam-Madapusi, T. H. van Pelt, and D. S. Bernstein, “Parameter consistency and quadratically constrained errors-in-variables least-squares identification,” International Journal of Control, vol. 83, no. 4, pp. 862–877, 2010.

T. Söderström, “Accuracy analysis of the Frisch scheme for identifying errors-in-variables systems,” IEEE Transactions on Automatic Control, vol. 52, no. 6, pp. 985–997, 2007.

R. Diversi and R. Guidorzi, “A covariance-matching criterion in the frisch scheme identification of MIMO EIV models,” System Identification, vol. 16, no. 1, pp. 1647–1652, 2012.

T. S. Söderström, “System identification for the errors-in-variables problem,” Transactions of the Institute of Measurement and Control, vol. 34, no. 7, pp. 780–792, 2012.

T. Söderström, “Extending the Frisch scheme for errors-in-variables identification to correlated output noise,” International Journal of Adaptive Control and Signal Processing, vol. 22, no. 1, pp. 55–73, 2008.

M. Hong and T. Söderström, “Relations between bias-eliminating least squares, the Frisch scheme and extended compensated least squares methods for identifying errors-in-variables systems,” Automatica, vol. 45, no. 1, pp. 277–282, 2009.

D. Fan and G. Luo, “Frisch Scheme identification for Errors-in-Variables systems,” in Proceedings of the 9th IEEE International Conference on Cognitive Informatics (ICCI '10), pp. 794–799, Beijing, China, July 2010.

L. Ljung, System Identification-Theory for the User, Prentice Hall, 2nd edition, 1999.

G. W. Stewart, Introduction to Matrix Computations, Academic Press, New York, NY, USA, 1973.