Adaptive Gradient-Based Iterative Algorithm for Multivariable Controlled Autoregressive Moving Average Systems Using the Data Filtering Technique

The identification problem of multivariable controlled autoregressive systems with measurement noise in the form of the moving average process is considered in this paper. The key is to filter the input–output data using the data filtering technique and to decompose the identification model into two subidentification models. By using the negative gradient search, an adaptive data filtering-based gradient iterative (F-GI) algorithm and an F-GI with finite measurement data are proposed for identifying the parameters of multivariable controlled autoregressive moving average systems. In the numerical example, we illustrate the effectiveness of the proposed identification methods.


Introduction
Parameter estimation plays an important role in system control [1][2][3][4], system analysis [5][6][7][8], and signal processing [9][10][11][12][13].Parameter estimation is significant in system modeling [14,15].Multi-input multi-output systems widely exist in industrial control areas, which are also called multivariate systems or multivariable systems [16][17][18].They are more complex in model structures than single-input single-output systems and always have high dimensions and numerous parameters, which make the parameter estimation more difficult.In this literature, Ding et al. proposed a filtering decomposition-based least squares iterative algorithm for multivariate pseudolinear ARMA systems [19].Ma et al. studied the parameter estimation problem of multivariate Hammerstein systems and presented a modified Kalman filter-based recursive least squares algorithm to give the parameter estimates [20].Pan et al. proposed a filteringbased multi-innovation extended stochastic gradient algorithm for multivariable systems [21].
The data filtering technique is an important approach in system identification [22] and state estimation.Chen and Ding applied the data filtering technique to identify the multi-input and single-output system based on the maximum likelihood recursive least squares algorithm [23].Mao et al. derived an adaptive filtering-based multi-innovation stochastic gradient algorithm for the input nonlinear system with autoregressive noise [24].They introduced a linear filter to filter the input and output signals and decomposed the identification model into two subidentification models (i.e., a noise model and a system filtered model), which can improve the convergence rate and computation efficiency [25].The identification methods can be applied to many areas [26][27][28][29].
The gradient search is useful for identification as an optimization method [30,31].Many gradient-based algorithms, including the stochastic gradient algorithms [32][33][34] and the gradient-based iterative algorithms, have been developed using the multi-innovation identification theory, the maximum likelihood estimation methods [35,36], the key-term separation principle [37,38], and the data filtering theory.
This paper uses the hierarchical identification principle to study the data filtering-based iterative identification methods for a multivariable controlled autoregressive moving average (M-CARMA) system.The basic idea is to introduce a linear filter to decompose the original identification model into two subidentification models and then obtain the parameter estimates using the negative gradient search.The main contributions are as follows: (i) A filtering-based gradient iterative (F-GI) algorithm is proposed using the data filtering technique and the gradient search.
(ii) A filtering-based gradient iterative algorithm with finite measurement data is developed to obtain the parameter estimates.
The layout of the remainder of this paper is as follows.Section 2 derives the identification model for the M-CARMA system.In Section 3, we derive a data filteringbased gradient iterative algorithm based on the data filtering technique.A filtering-based gradient iterative algorithm with finite measurement data is developed to estimate the unknown parameters in Section 4. A numerical example is shown in Section 5 to illustrate the benefits of the proposed methods in this paper.Finally, some concluding remarks are given in Section 6.

The Problem Formulation
Some notation is introduced for convenience: θ t denotes the estimate of θ at time t; "A ≕ X" or "X ≔ A" stands for" A is defined as X"; the symbol I (I n ) represents an identity matrix of appropriate size (n × n); the symbol 1 n represents an n-dimensional column vector whose elements are 1; z denotes a unit forward shift operator like zx t = x t + 1 and z −1 x t = x t − 1 ; the superscript T symbolizes the vector/matrix transpose; and the norm of a matrix X is defined by ∥X∥ 2 ≔ tr XX T .
The following multivariable controlled autoregressive moving average system in Figure 1 is considered, where u t ∈ ℝ r is the system input vector, y t ∈ ℝ m is the system output vector, v t ∈ ℝ m is a white noise vector with zero mean, A z and B z are the matrix polynomials in the unit backward shift operator z −1 , and D z is the polynomial in z −1 .
Assume that the orders n a , n b , and n d are known, and u t = 0, y t = 0, and v t = 0 for t ≤ 0. The intermediate variable is defined as The system information vector φ s t , the noise information vector ψ t , the system parameter matrix θ s , and the noise parameter vector θ n are defined as Equations ( 3) and (1) can be written as Equation ( 5) is the noise identification model.For the M-CARMA system in (1), choose the polynomial L z ≔ 1/D z as a filter.Define the filtered input vector u f t , the filtered output vector y f t , and the filtered information vector φ f t as Figure 1: A multivariable controlled autoregressive moving average system.
2 Complexity Multiplying both sides of (1) by L z obtains or Then we have Equations ( 10) and ( 5) form the filtered identification models of the M-CARMA system.

The F-GI Algorithm
In this section, a linear filter L z is applied to deal with the moving average noise.A gradient-based iterative identification algorithm is proposed for M-CARMA systems by using the data filtering technique [60][61][62][63].
Considering the newest p data from i = t − p + 1 to i = t, the stacked filtered output matrix Y f p, t , the stacked filtered information matrix Φ f p, t , the stacked noise vector W p, t , and the stacked noise information matrix Φ n p, t are defined as Define a quadratic criterion function: Let k = 1, 2, 3, … be an iterative variable.Let θ s,k t and θ n,k t be the estimates of θ s and θ n at iteration k.Minimizing J 1 θ s and J 2 θ n and using the negative gradient search will give the following iterative relations for obtaining the parameter estimates of θ s and θ n : where μ 1,k t ≥ 0 and μ 2,k t ≥ 0 are the iterative step size or the convergence factor.However, the difficulty is that the noise information matrix Φ n p, t (i.e., ψ t ) contains the unmeasured vector v t − i .So the gradient-based iterative algorithm in ( 13) and ( 14) cannot give the parameter estimate θ n,k t directly.The solution is to use the hierarchical identification principle and to replace the unknown variable v t − i with its corresponding estimates υ k−1 t − i at iteration k − 1, and to define the estimate of ψ t as Replacing t in (6) with t − i gives Replacing θ s in (17) with θ s,k−1 t obtains the estimate of w t − i at iteration k: From (6), we have Replacing θ s , θ n , and ψ t − i with θ s,k t , θ n,k t , and ψ k t − i obtains the iterative estimate of v t − i at iteration k: Then, using ŵk t to construct the iterative estimate of W p, t at iteration k gives Using Dk t, z to filter y t and u t gives the filtered estimates ŷf,k t and ûf,k t of y f t and u f t : Furthermore, we use ŷf,k t to construct the estimate of Y f p, t , use ŷf,k t and ûf,k t to construct the estimate of φ f t , and use φ f,k t to construct the estimate of Φ f p, t at iteration k: From the above derivation, we can summarize a filteringbased multi-innovation gradient iterative identification algorithm: The identification steps of the algorithm in ( 25), ( 26), ( 27), ( 28), ( 29), ( 30), ( 31), ( 32), ( 33), ( 34), ( 35), ( 36), ( 37), (38), (39), and (40) to compute θ s,k t and θ n,k t are listed as follows: (1) Set the initial values: let t = 1, give the data length p, and give a small positive number ε. Set the initial values θ s 0 = 1 mn a +rn b ×m /p 0 , θ n 0 = 1 n d /p 0 , p 0 = 10 6 .
(2) Collect the input-output data u t and y t and construct φ s t using (33).
( The flowchart of computing θ s,k t and θ n,k t from the F-GI algorithm is shown in Figure 2.

The F-GI Algorithm with Finite Measurement Data
Consider the data from t = 1 to t = L and define the stacked filtered output matrix Y f L , the stacked filtered information matrix Φ f L , the stacked noise vector W L , and the stacked noise information matrix Φ n L as Collect u(t) and y(t) Construct ψ k (t) and ϕ f,k (t) and compute ω(t)  The two gradient criterion functions are defined as Similarly, minimizing J 3 θ s and J 4 θ n , we can derive a filtering-based gradient iterative (F-GI) algorithm with the data length L for the M-CARMA system: The identification steps of the F-GI algorithm with finite measurement data in ( 43), ( 44), ( 45), ( 46), ( 47), ( 48), ( 49), ( 50), ( 51), ( 52), ( 53), ( 54), ( 55), ( 56), (57), and (58) to compute θ s,k and θ n,k are listed as follows.
(9) Compute υ k t using (55).The flowchart of computing θ s,k and θ n,k from the F-GI algorithm with finite measurement data is shown in Figure 3.
From Tables 1-3 and Figures 4 and 5, we can draw the following conclusions.
(1) The parameter estimation errors obtained by the presented algorithms gradually become smaller with the iterative variable k increasing.Thus, the proposed algorithms for M-CARMA systems are effective.
(2) The system parameter estimates converge to their true values with the increasing of the data length.
(3) Under the same data length, a smaller noise variance leads to higher parameter estimation accuracy and a faster convergence rate.7 Complexity (4) A longer data length L leads to a smaller estimation error under the same noise level.

Conclusions
An F-GI algorithm and an F-GI algorithm with finite measurement data are proposed for identifying the multivariable controlled autoregressive system with measurement noise in this paper.The linear filter is introduced to filter the inputoutput data, and the hierarchical identification principle is applied to decompose the identification model into two subidentification models.The simulation results show that the proposed algorithms can generate accurate estimates.The proposed approaches in the paper can combine other mathematical tools [64][65][66][67][68][69] and statistical strategies [70][71][72][73][74][75] to study the performances of some parameter estimation algorithms and can be applied to other multivariable systems with different structures and disturbance noises and other literature [76][77][78][79][80][81][82][83][84][85][86] such as system identification [87][88][89][90][91][92].

Figure 3 :
Figure 3: The flowchart of computing the F-GI parameter estimates with finite measurement data.
, and  n,k (t) ˆˆF igure 2: The flowchart of computing the F-GI parameter estimates.