The Hierarchical Iterative Identification Algorithm for Multi-Input-Output-Error Systems with Autoregressive Noise

This paper considers the identification problem of multi-input-output-error autoregressive systems. A hierarchical gradient based iterative (H-GI) algorithm and a hierarchical least squares based iterative (H-LSI) algorithm are presented by using the hierarchical identification principle. A gradient based iterative (GI) algorithm and a least squares based iterative (LSI) algorithm are presented for comparison. The simulation results indicate that the H-LSI algorithm can obtain more accurate parameter estimates than the LSI algorithm, and the H-GI algorithm converges faster than the GI algorithm.


Introduction
System identification studies mathematical models of dynamic systems by fitting experimental data to a suitable model structure [1,2].Many practical systems have multiple inputs and multiple outputs such as chemical processes [3,4], automation devices [5][6][7], and network communication engineering [8][9][10].For decades, much research has been performed on the multivariable systems [11,12], and some typical approaches for the parameter estimation of the multivariable systems have been reported [13], such as the canonical approach [14], the iterative methods [15,16], and the least squares methods [17].Recently, Panda and Vijayaraghavan adopted the sequential relay feedback test to estimate the parameter of the linear multivariable systems [18].Jafari et al. presented an iterative least squares algorithm to identify the multivariable nonlinear systems with colored noises [19].
The multivariable systems contain both parameter vectors and parameter matrices, and the systems inputs and system outputs are relevant and coupled [20][21][22].For the sake of reducing the computational complexity, the hierarchical identification principle is utilized to transform a complex system into several subsystems and then to estimate the parameter vector of each subsystem [23,24], respectively.In this literature, Schranz et al. proposed a feasible hierarchical identification process for identifying the viscoelastic model of respiratory mechanics [25].Xu et al. developed the parameter estimation for dynamical response signals [26,27].
The iterative methods have been widely applied in identifying the parameters of linear or nonlinear systems [28][29][30].Many iterative algorithms for system identification are based on the gradient method [31]and the least squares method [32][33][34][35].The basic idea of iterative methods is to update the parameter estimates using batch data.
This paper focuses on the parameter estimation for output-error autoregressive (OEAR) systems using the hierarchical identification principle and the iterative identification principle and presents a hierarchical gradient based iterative (H-GI) algorithm and a hierarchical least squares based iterative (H-LSI) algorithm.The key is to decompose a multi-input OEAR system into two subsystems and then to identify each subsystem.The work in [36,37] discussed the single-input single-output systems, but many practical systems have multiple inputs and multiple outputs with the development of industrial technology.Compared with the work in [36,37], this paper discusses the parameter estimation for multi-input OEAR systems and the presented H-LSI algorithm can achieve higher estimation accuracy than the LSI algorithm, and the H-GI algorithm also can achieve higher estimation accuracy than the GI algorithm.

Complexity
The rest of this paper is organized as follows.Section 2 gives some definitions and the identification model of multiinput OEAR systems.Section 3 presents a gradient based iterative algorithm and a least squares based iterative algorithm for multi-input OEAR systems.Section 4 derives a hierarchical gradient based iterative algorithm.Section 5 derives a hierarchical least squares based iterative algorithm.Section 6 provides two illustrative examples to demonstrate the effectiveness of the proposed algorithms.Finally, concluding remarks are given in Section 7.

The Problem Formulation
Let us define some notation.

Symbols: meaning
1  : an  dimensional column vector whose entries are all 1  0 : a large positive constant, for example,  0 = 10 6 X  : the transpose of the vector or matrix X fl :  defined as   max [X]: the maximum eigenvalue of the symmetric real matrix X.
Consider the following multi-input-output-error type models: where () ∈ R is the system output,   () ∈ R,  = 1, 2, . . ., , are the system inputs, and () ∈ R is the colored noise with zero mean.  () and   () are polynomials in the unit backward shift operator  −1 , and Assume that the orders   are known, () = 0,   () = 0, and () = 0 as  ⩽ 0. The colored noise () can be fitted by a moving average process or an autoregressive process or an autoregressive moving average process where V() is the white noise with zero mean and (), () are polynomials in the unit backward shift operator  −1 : This paper considers the colored noise to be an autoregressive process, so the models in (1) can be taken as the multi-input OEAR systems.Define the intermediate variables: ()   () ,  = 1, 2, . . ., .
From ( 4) and ( 7), we have The output () in (1) can be written as Define the parameter vectors as According to the above definitions, ( 8) and ( 9) can be written as Equation ( 14) is the identification model of the multi-input OEAR system.

The Gradient Based and Least Squares Based Iterative Algorithm
Consider the data { 1 (),  2 (), . . .,   (), ()} from  = −+ 1 to  =  and define quadratic criterion function as Let  = 1, 2, 3, . . .be an iteration variable and θ () be the estimate of  at iteration .Minimizing () by using the negative gradient search, we can obtain where   () > 0 is an iterative step-size.Because the information vector () contains the unknown variables   ( − ) and ( − ), we use the estimates x,−1 ( − ) and ŵ−1 ( − ) at iteration  − 1 to replace the unknown variables   (−) and (−); we can obtain the gradient based iterative (GI) algorithm for estimating the parameter vector  of the multi-input OEAR systems: ( The convergence rate of the GI algorithm is slow.To improve the convergence speed, we derive a least squares based iterative (LSI) identification algorithm.Minimizing () and letting the derivative of () with respect to  be zero give the LSI identification algorithm for the multi-input OEAR systems:

The Hierarchical Gradient Based Iterative Algorithm
Define intermediate variables: Using the hierarchical identification principle, the multiinput OEAR system in ( 14) can be decomposed into two fictitious subsystems: Next, we identify the parameters  and c of each subsystem in ( 21) and ( 22), respectively.Define quadratic criterion functions as Let θ () and ĉ () be the estimates of  and c at iteration .
Using the negative gradient search and minimizing  1 (, c) and  2 (, c), we can obtain Here,  1 () and  2 () are the iterative step-sizes or convergence factors.Substituting ( 19) into ( 24) and ( 20) into ( 25), we can obtain The parameter estimates θ () and ĉ () cannot be computed by ( 26) and ( 27 From ( 12) and ( 14 Replacing (), () in ( 26) and ( 27) with their estimates φ () and ψ (), replacing c in (26) with its estimate ĉ−1 (), and replacing  in (27) with its estimate θ (), we have In order to guarantee the convergence of θ () and ĉ (), a conservative choice is At last, we can summarize the hierarchical gradient based iterative parameter estimation (H-GI) algorithm for estimating  and c of the multi-input OEAR systems: The steps of computing the parameter estimates θ () and ĉ () for the multi-input OEAR systems are as follows.

The Hierarchical Least Squares Based Iterative Algorithm
The H-GI algorithm can produce higher parameter estimation accuracy compared with the GI algorithm, but it 6 Complexity converges slowly.In order to solve this short board, we derive a hierarchical least squares based iterative algorithm for the multi-input OEAR systems.
Minimizing  1 (, c) and letting the partial derivative of  1 (, c) with respect to  be zero and minimizing  2 (, c) and letting the partial derivative of  2 (, c) with respect to c be zero, respectively, we can obtain the least squares estimate θ(): Inserting ( 19) into ( 44) and ( 20) into (45) gives The above estimates θ() and ĉ() are impossible to compute, since the right-hand side of ( 46) contains the unknown parameter vector c and the unknown information vectors () and () and the right-hand side of (47) also contains the unknown parameter vector  and the unknown information vectors () and ().We solve this difficulty by replacing (), () with their estimates φ (), ψ () and replacing c in (46) and  in (47) with their estimates ĉ−1 () and θ ().Then, we can summarize the hierarchical least squares based iterative (LSI) algorithm of estimating the parameter vectors  and c as follows: The procedure for computing the parameter estimation θ () and ĉ () is as follows.

Example
Example 1.Consider the following two-input OEAR system: The inputs { 1 (),  2 ()} are taken as two persistent excitation signal sequences with zero mean and unit variance and {V()} as a white noise sequence with zero mean and variance  2 = 0.20 2 .Take the data length  = 2000, applying the LSI algorithm and the H-LSI algorithm to estimate the parameters of this example system.The parameter estimates and their errors of the LSI algorithm are shown in Table 1, the parameter estimates and their errors of H-LSI algorithm are shown in Table 2, and the parameter estimation errors of the LSI and H-LSI algorithms versus  are shown in Figure 1.From the simulation results in Tables 1 and 2 and Figure 1, we can draw the following conclusions.
(i) The estimation errors given by the LSI algorithm and H-LSI algorithm become smaller and smaller as iteration variable  increases.
(ii) Under the same noise variance, the estimation errors given by the H-LSI algorithm are lower than that given by the LSI algorithm.(57) The simulation conditions are the same as that of Example 1, and the noise variance  2 = 0.20 2 .Take the data length  = 3000.Applying the GI algorithm and the H-GI algorithm to estimate the parameters of this example system, the simulation results are shown in Tables 3-5 and Figure 2.
From the simulation results in Tables 3-5 and Figure 2, we can draw the following conclusions.(i) Under the same noise variance and data length, the H-GI algorithm has less estimation errors than the GI algorithm.This shows that the H-GI estimation algorithm can obtain more accurate estimates than the GI algorithm.
(ii) As the iteration variable  increases, the H-GI parameter estimates are very close to their true values.
(iii) The proposed H-GI algorithm requires more iterations than the H-LSI algorithm to achieve almost same estimation accuracy.

Conclusions
Combining the iterative technique and the hierarchical identification principle, a H-GI algorithm and a H-LSI algorithm are derived for identifying the multi-input OEAR systems.

Figure 1 :
Figure 1: The GI estimation errors  versus  with different  2 .

Figure 2 :
Figure 2: The GI and H-GI estimation errors  versus .

Table 1 :
The LSI parameter estimates and errors for Example 1.

Table 3 :
The H-LSI parameter estimates and errors for Example 2.

Table 4 :
The H-GI parameter estimates and errors for Example 2.

Table 5 :
The H-GI parameter estimates and errors for Example 2. The estimation accuracy of the H-LSI algorithm is close to their true values; this indicates that the proposed algorithm can effectively identify the multiinput OEAR systems.