An Improved Method for Stochastic Nonlinear System’s Identification Using Fuzzy-Type Output-Error Autoregressive Hammerstein–Wiener Model Based on Gradient Algorithm, Multi-Innovation, and Data Filtering Techniques

This paper proposes an innovative identification approach of nonlinear stochastic systems using Hammerstein–Wiener (HW) model with output-error autoregressive (OEA) noise. Two fuzzy systems are suggested for the identification of the input and output nonlinear blocks of a proposed model from given input-output data measurements. In this work, the need for the commonly used assumptions including well-known structure of input and/or output nonlinearities and/or reversible nonlinear output is eliminated by replacing the intermediate variables and noise with their estimates. Four parametric estimation algorithms to identify the proposed fuzzy-type stochastic output-error autoregressive HW (FSOEAHW) model are derived based on backpropagation algorithm and multi-innovation and data filtering identification techniques. The proposed algorithms are improved backpropagation gradient (IBPG) algorithm, multi-innovation IBPG (MIIBPG) algorithm, a data filtering IBPG (FIBPG) algorithm, and a multi-innovation-based FIBPG (MIFIBPG) algorithm. The convergence of the parameter estimation algorithms is studied. The effectiveness of the proposed algorithms is shown by a given simulation example.

All physics systems are nonlinear, and it is natural to use nonlinear model to describe such systems. erefore, nonlinear identification and control techniques have received, recently, more attention, since linear identification and control-based approaches are widely developed and are becoming mature. One class of such nonlinear modeling system is so-called block-oriented models that can be represented in various configurations where linear dynamic blocks and nonlinear static or dynamic subsystems are cascaded. e Hammerstein (H) model (static nonlinear block followed by a dynamic linear one) and the Wiener (W) system (a linear dynamic subsystem followed by a static nonlinear block) are the basic class of the cascaded systems which are widely used in many industrial practice engineering applications [14][15][16][17][18][19][20][21][22] and, therefore, the modeling approaches of such class of block-oriented models have received great attention for many years [23][24][25][26][27][28][29][30][31][32][33][34][35][36]. Hammerstein and Wiener systems are combined together to produce more complex subcategories, namely, Hammerstein-Wiener (HW) model (a linear block is cascaded between two nonlinear subsystems) and a Wiener-Hammerstein (WH) system (a nonlinear block is embedded between two linear blocks). Particularly, in many cases, the real system contains both actuator and sensor nonlinearities. en, it is appropriate to consider the input nonlinear block as actuator nonlinearity and the output nonlinear subsystem as sensor or/and process nonlinearity. e HW model has several advantages: (i) It has a physical view of the nonlinear characteristics of the real system, which is important in the analysis, monitoring, diagnosis, and control of the system. (ii) e HW dynamics are mainly produced by a linear subsystem. en, developed linear approaches could be used. (iii) When the output nonlinearity has an inverse function, the linear control techniques can be easily applied with desired performances.
is block-oriented model form is perfectly used in different fields, such as electrical, mechanical, hydraulic, and chemical fields [37][38][39][40][41][42][43][44][45][46]. Consequently, the identification of HW models has been an active research topic nowadays. In literature, different methods are proposed and can be roughly divided into different categories: the recursive, the iterative, the blind, the subspace, the frequency domain, the overparameterization, decomposition methods, and so on. e basic idea of the abovementioned methods is that the model parameters are approximated either by constructing hybrid model of nonlinear and linear parts (e.g., overparameterization, subspace, decomposition, and blind methods [36,[43][44][45][46]) or by separated steps where the estimation input and output nonlinear subsystems parameters and the dynamic linear part ones are established based on the unmeasured intermediate variables estimation step (such as frequency domain, iterative, recursive, stochastic, correlation methods, and a special input-based one [35,[47][48][49][50][51]).
However, the common representation of the abovementioned works is that the process noise (i.e., noise given between the linear dynamic part and the nonlinear block) or output disturbances (i.e., measurement noise given after nonlinear output block) have not been considered, which is not the case in practical physical process. en, it is more evident to consider some stochastic disturbances. Along these lines, a very few papers that deal with stochastic H-W model have been proposed in the literature. For example, in [43][44][45][46][47][48][49], the authors treated a particular structure of H-W presenting some forms of measurement noise. For H-W with process noise, some papers were proposed in [49][50][51][52][53]. Esmaeilani et al. and Wills et al. [54][55][56] have proposed identification methods for an H-W system's class presenting measurement and process noises.
It should be noted that the aforementioned approaches' applications are severely limited due to some problems. In fact, correspondent-cited ideas are restrictive to polynomial forms of the input and/or output nonlinear blocks or wellknown input and/or output nonlinear characteristics (like dead zone or backlash) but with unknown parameters. erefore, if the nonlinearity is not continuous or not in the polynomial form or with unknown characteristics, the algorithms do not give satisfactory performances. Moreover, the redundancy problem producing an oversizing dimension of HW's matrix parameters is another loophole of previously mentioned methods. For compensation of these shortcomings, artificial intelligent systems such as neural networks and fuzzy systems can be explored to model the H-W model owing to their universal approximation property and their ability to model a given nonlinear function to any arbitrary accuracy. Considering that the H-W structure should be ensured, the identification problem of neural network or fuzzy-type H-W is different (where the modeling of the two nonlinear blocks and the dynamic linear part are altogether needed) from that of traditional neural network and fuzzy system which focuses only on the global data process nonlinear transformation. Nowadays, only scattered works were reported in the identification of H-W model based on neural networks and fuzzy systems. In [57,58], a multistage approach is proposed to establish a recurrent neural network type H-W model based on an active region boundary initialization, a frequency domain eigensystem realization, and least squares and recursive recurrent learning algorithms. A neuro-fuzzy-type H-W system was presented in [59] using two-stage input signal. For the same form of system, Jia et al. and Li et al. [60,61] combined special input signal and correlation technique to identify separately and nonrecursively system parameters. However, most of the above-cited algorithms are based on some restrictive conditions, especially the prior knowledge of some parameters and the output nonlinearity's invertibility assumption to obtain intermediate variable's estimate, which is not always the case. Even in [62] a backpropagation gradient algorithm is used to estimate jointly unknown parameters and intermediate variables of neural network type H-W system with polynomial form of input and output nonlinear blocks.
It is clear that there are some problems that need to be addressed in nonlinear process identification using H-W model to achieve satisfactory results. e first one is how to consider stochastic nonlinear process disturbances in the H-W model description. e second is how to identify H-W model without any prior system knowledge and using, only, input and output measurement. e third is how to acquire the powerful identification aptitude of the H-W model with minimum attractive theoretical assumptions. e last is to identify the learning algorithm that can be used to achieve good performance.
To deal with the above-described issues, this paper presents a novel modeling and parameter identification method for the identification of nonlinear stochastic process described by fuzzy-type output-error autoregressive H-W (FSOEAHW) model. us, compared with the existent studies, the main contributions of this paper are as follows.
e first originality lies in the proposed fuzzy-type H-W scheme. In fact, the linear part is considered as a discrete transfer function and an output-error autoregressive mathematical model describes the process disturbance. Moreover, two fuzzy models are designed to describe the input and output nonlinearities of H-W model with only input and output measurement knowledge. As a result, the proposed model can not only describe the dynamics of such nonlinear system operating in stochastic environment and avoid the encountered input and output nonlinear blocks 2 Complexity restrictions based on particular or basis functions forms but also eliminate the reversibility assumption of the static output part based on internal variable estimates without parameter model oversizing problem. e second involvement is related to the proposed parameter estimation algorithms. e major idea is to combine a feedback propagation gradient algorithm with a multiinnovation and data filtering technique for the fuzzy-type H-W model identification. e proposed identification algorithms are based on input/output measurement and the approximation of all internal variables resulting from the preceding corresponding parameter estimates. In this instance, four algorithms are suggested: an improved backpropagation gradient (IBPG) algorithm, a multi-innovation improved backpropagation gradient (MIIBPG) algorithm for improving the convergence rate through the multi-innovation identification technique, a data filtering IBPG (FIBPG) algorithm, and a multi-innovation FIBPG (MIFIBPG). e remainder of this paper is organized as follows. e considered system is described in Section 2. In Section 3, the FSOEAHW parameter estimation algorithms, IBPG, MIIBPG, FIBPG, and MIFIBPG, are developed and their convergence analysis is studied. Section 4 presents simulation results. Some conclusions are offered in Section 5.
In that system, u k and y k are system input and output, respectively. y 1 k , x k , and h k are internal variables. e discrete linear transfer function G(.) is surrounded by an input static nonlinear block f 1 (.) and an output static nonlinear block f 2 (.). It is assumed that the measured output y k contains an unknown additive noise component w k described by an autoregressive mathematical model. v k is a white noise with zero mean and unknown variance σ 2 . e first unmeasurable intermediate variable h k is the output of input nonlinear block and it is expressed by the following equation: (1) e linear dynamic part is given by the following expression: e second unmeasurable intermediate variable can be written as follows: e output of the nonlinear output block is expressed as follows: Noise w k is given by We Given equations (4) and (5), the system's output is expressed as e objective of this paper is to give the following: (i) A good description of an unknown nonlinear system operating in a stochastic environment using HW model despite the presence of disturbance and the great model complexity, with minimum attractive theoretical assumptions that are not always valid, especially the rigorous restrictions related to the nonlinear output block's reversibility and wellknown input and/or output nonlinear blocks characteristics (ii) A suitable approximation of the above-described system such as the sum of square residual term errors E (given as follows) which should be reduced as possible using adequate algorithm where y k is a priori estimated output. To achieve the abovementioned objectives and based on the universal approximation properties of fuzzy systems, we propose using two fuzzy models to describe the input and output nonlinearities. nonlinear functions. Better modeling requires not only approximating the nonlinear function accurately but also simplifying the identification process. In the literature, several researches are restrictive to a polynomial form of the input and/or output nonlinear blocks or well-known input and/or output nonlinear characteristics (such as dead zone or backlash) but with unknown parameters [41][42][43][44][45][46][47][48][49][50][51][52][53][54][55][56][57][58][59][60][63][64][65][66][67][68][69][70][71][72][73][74]. erefore, if the nonlinearity is not continuous or not in the polynomial form or with unknown characteristic, the algorithms do not give a satisfactory performance. For compensation of the aforementioned shortcomings encountered in the existent structure of Hammerstein-Wiener model, in this paper, we devoted two fuzzy models to describe the input and output nonlinearities of H-W model with only input and output measurement knowledge.

Fuzzy-Type Stochastic Output-Error Autoregressive
In this section, we propose developing a modeling approach of a nonlinear dynamic process operating in the stochastic environment identification based on fuzzy model. According to the previous section, we propose a new fuzzytype stochastic output-error autoregressive H-W (FSOEAHW) model given in Figure 2. It consists of two static nonlinear blocks described by two independent fuzzy systems: a dynamic linear block and autoregressive noise block. en the first fuzzy system's output h k can be formulated as follows: Giving that W 1 n (n � 1, 2, . . . , N 1 ) is the n th fuzzy rule's consequence and N 1 is the total rule number, is n th Gaussian membership function where o 1 n and σ 1 n present, respectively, the correspondent center and width.
It should be noted that the membership functions can be of several shapes such as triangular, trapezoidal, and Gaussian. e only condition that must be fulfilled is that it must be in the interval [0, 1]. In the literature, Gaussian shape is commonly used because of its simplicity, its smoothness, and nonzero at all points. It is defined by only two parameters (the center and the width) and it is a continuously differentiable function [75][76][77]. e second unmeasurable intermediate variable of the FSOEAHW can be written as follows: where θ l � [a 1 , . . . , a n a , b 1 , . . . , b n b ] and φ l k � [− x k− 1 , . . . , − x k− n a , h k− 1 , . . . , h k− n b ] are, respectively, the vector parameters and the observation vector of the linear dynamic part. e output of the second fuzzy system y 1 k is expressed by where T are, respectively, the vector parameter and the observation vector of the output nonlinear block. W 2 m is the m th consequence fuzzy rule and N 2 is the correspondent total rule number.
Noise w k is expressed by (5). us, the system output can be rewritten as or, equivalently, it can be written in the following matrix form: Input static nonlinear block Output static nonlinear block Linear dynamic block Figure 1: SOEAHW model.

Complexity
Note that all vector parameters and observation vectors (φ 1 (u k ), φ 2 (x k ), and φ k ) are unknown. e problem then consists in developing a novel identification algorithm estimating the unknown parameters and variables using the input measurement and output measurement u k and y k that respect the objectives cited in the previous paragraph. Founded on the estimated model parameters, the mathematical model could be built identifying a given practical stochastic nonlinear dynamical system.

FSOEAHW Parameter Estimation
In this section, we suggest using a backpropagation gradient algorithm to estimate unknown parameters and unmeasured variable. e gradient algorithm is an important tool in the linear and nonlinear problems in which a modification of parameter estimates is reached using the negative gradient direction of the criterion function. Today and in the subject of identification and control, different recursive and iterative gradient algorithms are proposed [16,17,[78][79][80][81][82][83][84]. Stochastic gradient algorithm is a basic recursive identification algorithm which is used to study different types of systems such as multivariable systems [85,86] and nonlinear block-oriented systems [16,17,26,87]. In [88], an iterative gradient parameter estimation for output-error autoregressive systems using hierarchical principle has been presented. Yu et al. [31] used a gradient-based backpropagation algorithm to identify Hammerstein neural network type system. An equal algorithm has been extended by [62] for neural network type H-W system identification. e weakness of the gradient-based methods lies in the fact that estimation accuracy is not good enough for precision control purpose and the convergence rate is very slow. To improve these drawbacks, different approaches have been proposed in the literature. Among them, the multi-innovation technique is one of the most popular techniques which can improve the parameter estimation quality [89]. It consists of parameter estimation based not only on the current data but also on previous finite data at each iteration. Different papers are presented, in the literature, for the identification of different classes of systems like bilinear-inparameter systems [90], multivariable linear systems [91,92], block-oriented systems [93][94][95], and so on.
Another meaningful way for improving parameter estimation is manipulating the powerful data filtering technique. e focal idea based on identification algorithm is to generate system parameter estimates using a special filter to filter the measurement data and then identify the filtered system model and the filtered noise model. In these regards, the structure of the system to identify will be changed without eliminating noise from data or changing the relationship between variables. is technique has shown the effectiveness in the identification of different types of disturbed systems such as those considered in [70,92,[96][97][98][99][100][101][102][103][104].
In pursuit, we present four estimation algorithms to give the unknown parameters of the proposed FSOEAHW model. e first algorithm is the improved backpropagation gradient (IBPG) that utilized the backpropagation-based gradient algorithm. e second algorithm, namely, MIIBPG, employs a multi-innovation technique and IBPG algorithm for improving the convergence rate through the multi-innovation identification technique. Lastly, and for the same purpose, a data filtering-based IBPG (FIBPG) algorithm and a multiinnovation-based FIBPG (MIFIBPG) algorithm are proposed.

Improved Backpropagation Gradient (IBPG) Algorithm.
Let h k, t , x k, t , and y 1k, t be the estimated variables, respectively, of intermediate variables h k , x k , and y 1k at iteration step t as follows: where the estimated vector parameters at recursive step t are First fuzzy system Second fuzzy system Linear dynamic part Figure 2: Fuzzy-type stochastic output-error autoregressive Hammerstein-Wiener (FSOEAHW) model.

Complexity 5
As a result, the a priori estimated output y k can be expressed by the following equivalent adjustable model: where To give w k , y 1 k is replaced by its estimate y 1 k,t , which leads to the following expression: Using equation (16), we can define an estimation error term as Using backpropagation gradient algorithm, each unknown parameter in the FSOEHW model is updated according to adjustment formula given by the minimization of the following quadratic error function with respect to this parameter: By recursive application of the chain rule, all unmeasurable variables should be calculated first based on the FSOEHW parameter values before adjustment (i.e., the parameters resulting from the previous adjustment step t-1).
en each unknown parameter is adjusted according to the following expression: where η α is the learning rate of parameter updates. Parameters α t and α t− 1 are, respectively, values of parameter α after and before each adjustment step. α represents one of a tuning FSOEAHW's set of parameters Applying IBPG algorithm, the adjustment equations of the first fuzzy model parameters are given as follows: where 6 Complexity e linear dynamic part parameters are adjusted according to equations (28) and (29): Finally, the parameter adjustment equations of the second fuzzy system and the additive noise can be obtained by the following formulas: e difference between the proposed algorithm (IBPG) and a classical backpropagation gradient algorithm (BPG) lies in the second terms of equations (25)- (27), (30), and (31) which are omitted in the BPG case. In fact, our algorithm is inspired by [15,24]. e unmeasured variables h k− j,t , (j � 1, . . . , n b ), and x k− i,t (i � 1, . . . , n a ) are recalculated at each recursive step t based on a i,t− 1 , b j,t− 1 , σ 1 n,t− 1 , o 1 n,t− 1 , and W 1 n,t− 1 (i � 1, . . . , n a , j � 1, . . . , n b , n � 1, . . . , N 1 ), resulting from the just previous recursive step because the considered system is supposed to be invariant and the linear part is dynamic.
en, considering the above-cited terms is recommended.
e following summarizes the IBPG identification procedure of FSOEAHW model: Step 1: initialize the FSOEAHW parameters randomly. Fix the learning rates; t � 1.
At each iteration t and for each sample k, repeat the following steps: Step 2: for input u(k − j), calculate h k− j,t (j � 1, . . . , n b ) using equation (13).
Step 6: if the stopping criteria E given by (7) and the parameter variations (|α t − α t− 1 |) are less than fixed small values, then stop; else, go to step 2; t � t + 1.

IBPG Algorithm Convergence Analysis.
e adjustment procedures of FSOEAHW parameters are based on the learning rate choice (η α ). Too small η α guarantees convergence but with slow training speed, whereas too big η α guides to parametric divergence. In this section, eorem 1 gives a selecting approach of a convenient η α .

Theorem 1.
e asymptotic convergence of the IBPG algorithm is guaranteed if each learning rate η α of each correspondent adjustable parameter α is chosen to satisfy where P α, max � max k |zy k,t /zα t− 1 |.
Proof. See Appendix A. algorithms such as gradient algorithm and least-square method in its recursive and iterative form [33,[105][106][107][108][109][110][111][112][113]. Particularly, it is well known that recursive gradient algorithm presents a slow convergence rate compared to other estimation approaches [113]. Many factors contribute to this disadvantage, principally its dependence on current data u k , y k only. As a matter of fact, at each recursive step, gradient algorithm does not use previous data u k− z , y k− z 1 ; z 1 � 1, 2, . . . . It does not have the capacity to use the available data in the same step.
To overcome this problem, we propose in this section using multi-innovation IBPG (MIIBPG) algorithm based on previous finite data; that is, the MIIBPG algorithm uses the current and the previous data at each iteration t, which can improve the parameter estimation accuracy. e elementary idea is to expand the error term e 1 k,t given by (8) and denoted scalar innovation to an innovation vector (called multi-innovation) [109,111,113].
Define an innovation vector E 1L k,t as where L is positive integer denoted innovation length and e 1 k− l,t � y k− l − y k− l,t is the l th error scalar term at time k − l (l � 0, 1, . . . , L) expressed by According to the recursive gradient algorithm minimizing the cost function given by equation (39), the MIIBPG technique provides a set of parameters updating equations listed in the following: where 8 Complexity e MIIBPG identification algorithm can be implemented based on the following steps: Step 1: initialize the parameters of the linear dynamic parts of the two fuzzy systems and of the output noise randomly. Fix the learning rates and the innovation length L; t � 1. At each iteration t and for each sample k, repeat the following steps: Step 2: for l � 0, . . . , L − 1, and for given input u(k − j − l), calculate h k− j− l,t , x k− i− l,t , y 1 k− l,t , y k− l,t , and e 1 k− l,t (j � 1, . . . , n b ) using equations (13)- (16) and (18) replacing k with k-l.
Step 4: calculate noise's estimate by equation (17) replacing k with k-l.
Step 5: if the stopping criteria E given by (7) and the parameter variations (|α t − α t− 1 |) are less than fixed small values, then stop; else, go to step 2; t � t + 1.

MIIBPG Algorithm Convergence Analysis.
is section surmises the convergence analysis of MIIBPG algorithm which can be applied to estimate a nonlinear stochastic system that can be illustrated by the proposed FSOEAHW model. e convergence properties of the MIIBPG algorithm are introduced by the following theorem.

Theorem 2.
e asymptotic convergence of MIIBPG algorithm is guaranteed if the learning rate η α of the corresponding adjustable parameter α is chosen to satisfy where

Data Filtering-Based Improved Backpropagation Gradient (FIBPG) Algorithm.
e data filtering technique in system identification is used to deal with the parameter estimation issues of systems disturbed by colored noises. Specifically, the elementary idea is to use a linear filter to filter input-output data so that the original systems with colored noises are transformed into new ones with white noises.
en the system's structure is transformed to be simpler without changing the relationship between the system inputs and outputs. Owing to the advantages of the data filtering technique, it has been widely applied for different system identification and parameter estimation. For example, each of recursive, iterative, and hierarchical least-square algorithms is combined with data filtering technique in the identification problem of output-error autoregressive linear systems [114], two-input single-output controlled autoregressive moving average linear systems [115], Hammerstein finite impulse response systems with moving average output noise [116], and a multivariable box Jenkins-like system [117]. Similarly, a gradient algorithm in its recursive and iterative form is used with data filtering technique for identification of some classes of linear and nonlinear systems such as linear multivariable autoregressive moving average system [101], a state-space linear system in its observability canonical form disturbed by colored noise [118], and a block-oriented system like Hammerstein finite impulse response system with moving average output noise [119].
Stirred by the above description, we propose in this section combining IBPG algorithm (given in Section 3.1) with a data filtering technique to improve the accuracy rate of IBPG algorithm. In our case, we propose introducing a linear filter C(z) to filter a measurement data in the purpose of parameter and intermediate variables estimation based on two criteria.
We define, respectively, the filtered intermediate variable y 1f k and the filtered output y f k as We have c 0 � 1. Multiplying both sides of (11) by C(z) and using (10), we obtain us, the filtered output can be written in matrix form as In that, we can write the filtered observation vector as follows: Consequently, the filtered system output can be defined as e objective, now, is to develop an FSOEAHW parameters estimation approach based on gradient algorithm, using a filtered data measurement and minimizing a filtered estimation error given by where the filtered estimated output y f k,t can be expressed by the equivalent adjustable model: where φ 2f,t− 1 (x k,t ) will be expressed later. It represents the estimated filtered observation vector in which the estimated intermediate variable x k,t is given by (14).
In reality, the quadratic criterion J 2 k,t contains the unmeasurable output noise vector w k , so the estimation of unknown parameter c p , p � (1, 2, . . . , n c ), cannot be implemented. en, we propose replacing w k in (62) by its estimated w k given by equation (64). Afterward, the new quadratic criterion functions will be defined as where Consequently, the estimation equation of parameter c p , p � (1, 2, . . . , n c ), can be inferred using gradient algorithm minimizing the criteria J 2 k,t as follows: e estimation procedure of unknown parameters set β is derived using the gradient method minimizing the quadratic criterion function defined as follows: where y f k,t and y f k,t are expressed, respectively, by where We have erefore, the rest of parameters are adjusted using the following equations with c 0,t � 1: where To summarize the FIBPG algorithm, we give the following steps: Step 1: initialize the parameters of the linear dynamic parts of the two fuzzy systems and the output noise parameters randomly. Fix the learning rates; t � 1. At each iteration t and for each sample k, repeat the following steps: Step 2: for input u(k − p − j), calculate h k− p− j,t (j � 1, . . . , n b , p � 0, . . . , n c ), x k− p− i,t , y 1 k− p,t , and y k− p,t using, respectively, equations (13)- (16) replacing k with k-p.
Step 7: if the stopping criteria E given by (7) and the parameter variations (|α t − α t− 1 |) are less than fixed small values, then stop; else, go to step 2; t � t + 1.

FIBPG Convergence Analysis.
is subsection aims at the convergence analysis of the FIBPG which can be applied to estimate the parameters of a nonlinear system that can be described by the FSEOFHW model. e convergence properties of the FIBPG algorithm are determined in the following theorem.

Multi-Innovation and Data Filtering-Based Improved
Backpropagation Gradient (MIFIBPG) Algorithm. Similar to Section 3.2 and to improve the convergence accuracy, we can combine multi-innovation approach with data filtering technique and then we obtain a multi-innovation FIBPG (MIFIBPG) algorithm.
Define E 2L k,t and E 1fL k,t as follows: where Like the previous section and referring to MIIBPG algorithm, initially the vector parameter c is adjusted in order to calculate the innovation vector error E 1fL k,t . It is updated according to formula (91) by minimizing the quadratic criterion given by the following equation: e adjustment equations of all other parameters are gotten using the MIFIBPG algorithm minimizing the cost function J 1fL k,t � (1/2)E T 1fL k,t E 1fL k,t . ey are listed as follows: Complexity 13 where where 14 Complexity where e theorem's proof is similar to those given in Appendices B and C for the MIIIBPG and FIBPG algorithms convergence study.

Example
In this section, we achieve an illustration simulation in order to test the performance and the efficiency of the developed algorithms.
erefore, we consider a nonlinear stochastic system given by stochastic output-error Hammerstein-Wiener mathematical model in which its blocks are represented using the following expressions: A set of 9000 random values between 0 and 4 are used as input u k . v k is a white noise sequence with zero mean and noise-to-signal ratio as c ns � �������������� var(v k )/var(y 1k ) × 100 � 13%. e input u k and the corresponding output y k are presented in Figure 3.

Parametric Estimation Using IBPG Algorithm.
We propose estimating the FSOEAHW parameters using the IBPG algorithm described in Section 3.1.
us, we demonstrate the estimation results by giving the evolution curves of the intermediate variables and output estimates h k,t , x k,t , y 1k,t , and y k,t in Figure 4.
In addition, Figure 5 represents the evolution curves of the estimation error e 1 k,t and the parametric distance In Table 1, we present some obtained numerical values of vector parameter estimates θ t and the corresponding parametric error for t � 0, 2000, 4000, 6000, and 9000. In addition, to evaluate the estimation quality, we depict in Table 2 the estimation errors (e h � ‖h − h‖/‖h‖, e x � ‖x − x‖/‖x‖, e y 1 � ‖y 1 − y 1 ‖/‖y 1 ‖, e w � ‖w − w‖/‖w‖, and e y � ‖y − y‖/‖y‖).
As is noticed in the above-presented simulation results, a satisfactory estimation is achieved with the proposed IBPG algorithm despite the existence of the noise acting on the process output without restrictive assumptions. Furthermore, the parameter estimation errors gradually become, iteratively, smaller. us, the parameter estimates are closer to the true values and the fuzzy models generate a satisfactory estimation accuracy of the nonlinear input and output blocks.
For comparison purpose, we propose giving simulation results in which the input and output nonlinear parts are estimated by two independent polynomial functions expressed by the following equations: where χ n (n � 1: N 1 ) and β m (m � 1: N 2 ) are constant parameters. en the corresponding output-error autoregressive HW (OEAHW) system parameters are χ n , a i , b j , c p , and β m (n � 1: 3, i � 1: 2, j � 1: 2, p � 1, m � 1: 3). e proposed IBPG algorithm is adopted for parameter adjustment based on updating equations given in Appendix D. It should be noticed that OEAHW is inspired from [24], where the IBPG algorithm is used to estimate a polynomial Wiener model.
In this sense, Figure 6 illustrates the evolution curves of the estimation and parametric errors for the OEAHW and FSOEAHW models. Table 3 presents the OEAHW quality estimation.
e obtained results confirm that FSOEAHW model generates a more satisfactory estimation accuracy compared to the OEAHWM model with polynomial form. Due to the ability of the fuzzy systems for providing a good approximation of the two nonlinear parts, the proposed FSOEAHW reaches a good performance with a comparatively small amount of calculation and the resulting estimated model can capture systems dynamics correctly.

Parametric Estimation Using MIIBPG Algorithm.
Using the MIIBPG algorithm for FSOEAHW parameter identification, the vector parameter estimates θ t and their corresponding parametric errors δ t (t � 0, 2000, 4000, 6000, 9000) are given in Table 4 for L � 2,   Complexity become smaller with the iterative variable and innovation length increasing. In addition, the developed MIIBPG algorithm drives to have higher performance than the standard IBPG algorithm. In fact, the system parameter estimates converge to their true values and the estimation errors decrease to smaller values. In this direction, the          proposed multi-innovation approach can estimate the parameters effectively and increasing the innovation length can improve parameter and variable estimation accuracy and accelerate the convergence rate because the algorithm uses more information in each iteration. Hence, the proposed algorithm is effective for the FSOEAHW model identification.

Parametric Estimation Using FIBPG Algorithm.
In this section, the efficiency of the FIBPG algorithm compared to the IBPG algorithm is well noticed by visualizing the evolutions curves of the estimation error variance σ 2 e 1 k,t and a parametric error δ t given, respectively, in Figures 9 and 10. In fact, a smooth and a more precise parametric convergence is ensured.
is is more approved by examining Tables 6 and 7 that show the estimation accuracy of FSOEAHW model.

Parametric Estimation Using MIFIBPG Algorithm.
In this part, we exploit the MIFIBPG algorithm to estimate the parameters of the FSOEAHW. us, Tables 8 and 9 present the parameter estimates and their estimation errors for L � 2, 5, and 7. For the same innovation length values, Figures 11   and 12 give the estimation error variance σ 2 e 1 k,t and the parametric distance δ t . e proposed multi-innovation approach combined with data filtering technique can estimate the parameters effectively and smoothly and increasing the innovation length gives a good parameter estimation accuracy and accelerates the convergence rate and then improves the gradient algorithm drawbacks.  Table 8: e vector parameter estimates θ t and their parametric errors δ t for the FSOEAHW model using MIFIBPG algorithm with L � 2, 5, and 7.

Conclusion
is paper deals with modeling and identification of SOEAHW systems based on an artificial intelligence technique. Two fuzzy models with adjustable parameters are used to identify the input and output nonlinear blocks. e output nonlinearity may be noninvertible. A fuzzy model having as input the linear dynamic output estimate approximates it. e four given parameter estimation algorithms (IBPG, MIIBPG, FIBPG, and MIFIBPG) are based on recursive gradient algorithm, multi-innovation technique, and data filtering approach. ey have convergence property and validity for online implementation. Simulation results validate the effectiveness of the proposed algorithms. However, the drawbacks of the proposed algorithms lie, particularly, in the updated parameter initialization and the learning rate choice. In fact, it is well known that adjustment equations based on gradient algorithm depend directly on these parameter choices and these have a crucial effect on the convergence and accuracy. e work presented in this paper can be extended to other classes of stochastic nonlinear multivariable systems with colored noise. Furthermore, we can suggest proposing an adaptive fuzzy control of nonlinear industrial process (such as practical hydraulic process) based on the presented SOEAHW model. Finally, a more suitable approach for an appropriate learning rate's research and other optimization techniques can be proposed to give better performance.

B: MIIBPG Algorithm's Convergence Proof
In this case, we define a Lyapunov function as (B.1) e alternation of V k due to the training process is hence where the error difference Δe 1 k− l can be computed by