Least Squares Based and Two-Stage Least Squares Based Iterative Estimation Algorithms for H-FIRMA Systems

This paper studies the identification of Hammerstein finite impulse response moving average (H-FIR-MA for short) systems. A new two-stage least squares iterative algorithm is developed to identify the parameters of the H-FIR-MA systems. The simulation cases indicate the efficiency of the proposed algorithms.


Introduction
System modeling [1][2][3][4][5] and parameter estimation [6][7][8][9][10] are basic for controller design [11,12].Nonlinear Hammerstein model identification has received much attention due to its ability to describe a wide class of nonlinear systems and has extensive applications in many engineering problems [13,14].The Hammerstein models are special class of nonlinear systems; the nonlinear block is usually static nonlinearity and is followed by a linear system [15].For example, Wang et al. discussed the identification problem for a Hammerstein nonlinear system with a dynamic subspace state space [16]; Greblicki investigated a class of continuous time Hammerstein system identification [17].
There are a lot of research topics about linear or nonlinear system identification [18,19] and control [20,21].For example, Ding et al. derived the gradient search based and the Newton based identification methods for Hammerstein systems [22]; Wang and Ding proposed a hierarchical least squares identification method for Hammerstein-Wiener systems by using the hierarchical identification principle and the auxiliary model identification idea [23]; Based on the data filtering technique and the key-term separation principle, Wang et al. investigated a filtering based recursive least squares identification algorithm for Hammerstein output error moving average systems [24].The proposed algorithm can identify not only the system model parameters but also the noise model parameters and the internal variables.
The iterative algorithm is one of the basic methods for system analysis and synthesis, and nonlinear optimization [25][26][27][28].In [29], Wang and Ding presented a gradient based and least squares based iterative identification algorithms for Wiener systems through the use of the hierarchical identification principle.In [30], Ding et al. discussed the Newton iterative identification algorithm of a class of Wiener nonlinear systems with moving average noises from inputoutput measurement data.Li et al. derived iterative parameter identification methods for nonlinear functions [31].Pan et al. proposed a digital image correlation using iterative least squares and pointwise least squares for displacement field and strain field measurements [32].In the field of control, Zhang et al. applied the iterative algorithm to the predictive control field [33,34].
Recently, the multistage identification strategy is widely applied to the system identification field [35,36].For example, Ding and Duan studied a new-type two-stage least squares based iterative algorithm for identifying the system model parameters and the noise model parameters [37].
The main concern of this paper is to investigate the parameter identification problem of Hammerstein finite impulse response moving average (H-FIR-MA) systems.The memoryless polynomial input nonlinearity is followed by a linear dynamical system, as is explained in Figure 1.Both the least squares iterative and the two-stage least squares iterative algorithms are proposed to estimate the parameters of the H-FIR-MA systems.
The layout of this paper is organized as follows.Section 2 describes the identification model of H-FIR-MA systems.Section 3 provides the least squares iterative algorithm for the H-FIR-MA systems.Section 4 introduces the two-stage least squares iterative algorithm for the H-FIR-MA systems.In Section 5, we apply the proposed algorithms to an example to illustrate their implementation.Finally, concluding remarks are offered in Section 6.

System Description and Identification Model
Some notation is given.‖X‖ 2 = tr[XX  ] stands for the norm of a matrix X; " := " or " := " expresses that " is defined as "; I represents the identity matrix of appropriate sizes and 1  is defined as an -dimensional identity column vector.
Consider an H-FIR-MA system, which is described by where {()} and {()} are the input and output sequences of the systems, {V()} is an uncorrelated stochastic noise sequence with zero mean and variance Assume that the orders   ,   , and  are known in ( 1) and ( 4) and () = 0, () = 0, () = 0, and V() = 0 for  ≤ 0. In order to get unique parameter estimates, here we let  0 = 1 [24].The item () in ( 1) is chosen as the key term; substituting (4) into (1) gets Define the parameter vector Θ and the information vector () as follows: From ( 5), we obtain the following identification model: Define the cost function: In what follows, we derive the algorithms for identifying the H-FIR-MA system using the least squares and two-stage least squares iterative estimation algorithms.
Consider the data from  = 1 to  =  ( ≥ ) and define the stacked information matrices Φ(), the stacked output vector Y(), and the stacked white noise vector V() as Hence, (7) can be rewritten as According to the estimation model in (12), the cost function in ( 8) can be written as To minimize  1 (Θ), letting its partial derivative of  1 (Θ) with respect to Θ be zero, we have It is impossible to obtain the estimate Θ, because the information matrix Φ() (i.e., () in ( 6)) contains the unmeasurable inner variables ( − ) and the noise terms V( − ).Here we adopt the auxiliary model idea and the hierarchical identification principle: let  = 1, 2, 3 . . .be iteration variable, let Θ be the iterative estimate of Θ at iterative  and û ( − ), and let V ( − ) be the  iterative estimates of ( − ) and V( − ).We replace ( − ) and V( − ) in (6) with their estimates and obtain the estimates φ () and Φ () as follows: Replacing Φ() in (14) with Φ () and combining (10) and (15), we can obtain the LSI estimation algorithm of identifying Θ for the H-FIR-MA system as follows [27]: The computation procedures of the LSI algorithm in ( 16)-( 22) are summarized as follows.
Step 5. Increase  by 1 and jump to Step 2.

The Two-Stage Least Squares Iterative Estimation Algorithm
Here, we derive a two-stage least squares iterative (TS-LSI) estimation algorithm for the H-FIR-MA system.From ( 5) and ( 6), we can obtain the following identification model: Mathematical Problems in Engineering Define the information vector Ψ() and f(()) as Ψ () = [ ( − 1) ,  ( − 1) , . . .,  ( −   ) , Define two intermediate variables  1 () := () − f(()) and  2 () := () − Ψ  (); then the system in ( 23) can be decomposed into two "suppositional" subsystems: The estimates of two "suppositional" subsystems in ( 25) can be obtained by minimizing the cost function: Two intermediate variables can be rewritten as From ( 25), we have According to the estimation model in (31), the cost function in (26) can be written as To minimize  2 (), let its partial derivative of  2 () with respect to  be zero: From (34), the least squares estimate of the parameter vector  can be expressed as Here, put ( 29) into ( 35) and (35) gives In accordance with the same derivation process of θ, we can easily get the estimation formula of However, ( 36) and ( 37) contain the unknown parameter  and , respectively, it is impossible to θ and γ.According to the method in Section 3, we can summarize the two-stage least squares iterative estimation algorithm for estimating  and  of the H-FIR-MA systems as follows: The computation procedures of the TS-LSI algorithm in (38)-(46) are summarized as follows.
Step 6. Increase  by 1 and jump to Step 2.
Take two different data lengths  = 3000 and  = 7000.

The parameter values
are estimated using the two different methods described in the paper, namely, the LSI and the TS-LSI methods in Sections 3 and 4. We apply the LSI method to estimate the parameters of this case; the parameter estimation with different data length and noise variances  2 are shown in Tables 1 and 2, and the estimation errors  versus iteration  are shown in Figures 1 and 2, where  := ‖ Θ − Θ‖/‖Θ‖.Similarly, the parameter estimation and estimation errors  of the TS-LSI method with different data length and noise variances  2 are shown in Tables 3 and 4 and Figures 3 and 4.
From the simulation results in Tables 1-4 and Figures 1-4, we can draw the following conclusions.
(1) The parameter estimation errors given by the LSI and TS-LSI algorithms become small as iterations increase.
(2) The parameter estimation errors given by the LSI and TS-LSI algorithms become closer to their true values with the data length  increasing.(3) It is easy to see that a high noise level results in a low consistence rate of the parameter estimates to the true parameters for both of the proposed algorithms.All in all, this shows that the proposed algorithms are effective.

Conclusions
The LSI and the TS-LSI identification algorithms are developed for H-FIR-MA systems.The simulation results indicate that the proposed algorithms can obtain highly accurate parameter estimates and fast convergence rate and illustrate the proposed algorithms' performance.Compared with other methods, the LSI and TS-LSI methods must compute the

( 4 )
When the data length goes to infinity, the estimation errors converge to zero.The simulations of results in Tables 1-4 and Figures1-4indicate that the proposed algorithms based iterative algorithm should stop for about  = 3 ∼ 4. The fluctuation of the estimation errors is caused for large  due to the stationary of noise.