Least-Squares Based and Gradient Based Iterative Parameter Estimation Algorithms for a Class of Linear-in-Parameters Multiple-Input Single-Output Output Error Systems

Theidentification

As a basic class of multivariable systems, multiple-input single-output (MISO) systems have lots of applications in industrial processes.Several works on MISO system identification have been reported [33].For example, in order to improve the convergence rate, Liu et al. developed a stochastic gradient algorithm for MISO systems using the multi-innovation theory [34].The least-squares methods can also be found in the literature.
Recently, Wang and Tang studied the identification algorithms for a class of linear-in-parameters single-input singleoutput (SISO) systems with colored noises using the recursive least-squares method [35].In this work, we extend these results from SISO systems into a class of linear-in-parameters MISO systems with the colored noises shown in Figure 1 [36,37].Consider where () ∈ R is the system output, {  () ∈ R,  = 1, 2, . . ., } are the system inputs, and V() ∈ R is the stochastic white noise with zero mean.  () and () are polynomials, of known orders (  ,   ), in the unit backward shift operator  −1 , and defined by The linear-in-parameters multiple-input single-output output error moving average systems.
,   , and  T  are the unknown parameters to be estimated.The superscript T denotes the matrix/vector transpose.It is worth noting that the models in (1) include but are not limited to linear MISO systems; that is, when   and   (  ()) are defined by system (1) denotes a nonlinear MISO system.On the basis of the iterative algorithms for linear-inparameters SISO systems [37,38], this paper develops the least-squares based and gradient based iterative identification algorithms to improve the parameter estimation accuracy for a class of linear-in-parameters MISO output error moving average systems.Compared with the gradient based iterative algorithm, the least-squares based iterative algorithm can provide more accurate parameter estimates.
The remainder of this paper is organized as follows.Section 2 introduces the identification model.Section 3 derives the least-squares based iterative algorithm.Section 4 proposes a gradient based iterative algorithm.Section 5 presents an illustrative example to show the effectiveness of the algorithms.Finally, concluding remarks are offered in Section 6.

The Identification Model
Let us define some symbols.The symbol I  denotes an identity matrix of order ; 1  denotes an -dimensional column vector whose elements are 1;  max [X] and X −1 represent the maximum eigenvalue and the inverse of the square matrix X.
Define the intermediate variables as follows: Define the parameter vectors as follows: and define the information vectors as follows: Then we can express (5) as and system (1) can be rewritten as Equation ( 9) is the identification model of system (1), and parameter vector  contains all the parameters of the system.

The Least-Squares Based Iterative Algorithm
Consider the newest  data from  −  + 1 to  and define the quadratic criterion function as follows: By minimizing () and letting the derivative of () with respect to  be zero, we can obtain the least-squares estimate of  as The above estimate θ() is impossible to implement due to the unknown noise-free outputs   (−) and unmeasurable noise items V( − ) in ().Here, the difficulties are solved by using the iterative identification technique [38]: let  = 1, 2, . . .be the iterative variable, and let θ, () and θ () be the iterative estimates of   and  at iteration , replace the unknown items Let θ, () and θ () be the estimates of   and  at iteration , let x, () and V () be the estimates of   () and V() at iteration .Replacing ( − ) in (11) with its corresponding estimate φ ( − ), we can obtain the following least-squares based iterative algorithm for MISO systems in (1) (the MISO-LSI algorithm for short) into: φ, () = [−x , ( − 1) , −x , ( − 2) , . . ., −x , ( −   ) , x, ( − ) = φT , ( − ) θ, () , The steps of computing θ () involved in the algorithm are summarized as follows.

The Gradient Based Iterative Algorithm
By minimizing () through the negative gradient search, we obtain the following recursive relation of computing the estimate of  at iteration : where   () is the step-size or the convergence factor to be given later.The same difficulties arise in that the noise-free outputs   ( − ) in  s () and the noise items V( − ) in  n () of () on the right-hand side of ( 24) are unknown.

Example
Consider the following nonlinear multiple-input singleoutput simulation system: Here, the inputs { 1 ()} and { 2 ()} are taken as uncorrelated persistent excitation signal sequences with zero means and unit variances and {V()} as a white noise sequence with zero mean.
Using  =  = 1000 data and applying the MISO-GI algorithm in ( 25)- (32) and the MISO-LSI algorithm in ( 13)- (19) to estimate the parameters of this nonlinear system, the parameter estimates of each algorithm and their errors with noise variance  2 = 0.50 2 are shown in Table 1; the parameter estimation errors  := ‖ θ () − ‖/‖‖ versus  of each algorithm are illustrated in Figure 2. We also investigate the performance of two algorithms under a relatively high noise level with noise variance  2 = 1.00 2 , and the corresponding simulation results are illustrated in Table 2 and Figure 3.
From the simulation results in Tables 1 and 2 and Figures 2 and 3, we can draw the following conclusions.
(i) The parameter estimation errors are getting smaller as the iterative variable  increases.(ii) Both algorithms can produce highly accurate parameter estimates under different noise variances.(iii) The MISO-LSI algorithm converges faster than the MISO-GI algorithm does; however, due to the use of a batch of data, the MISO-LSI algorithm involves many matrix computations, resulting in the high computational complexity.One possible solution for reducing the computational load of the MISO-LSI algorithm with large  is using the decomposition technique [27], which is widely adopted in the leastsquares based iterative algorithms.