Multi-Innovation Stochastic Gradient Parameter and State Estimation Algorithm for Dual-Rate State-Space Systems with d-Step Time Delay

*is paper presents a multi-innovation stochastic gradient parameter estimation algorithm for dual-rate sampled state-space systems with d-step time delay by the multi-innovation identification theory. Considering the stochastic disturbance in industrial process and using the gradient search, a multi-innovation stochastic gradient algorithm is proposed through expanding the scalar innovation into an innovation vector in order to obtain more accurate parameter estimates. *e difficulty of identification is that the information vector in the identificationmodel contains the unknown states.*e proposed algorithm uses the state estimates of the observer instead of the state variables to realize the parameter estimation. *e simulation results indicate that the proposed algorithm works well.


Introduction
e mathematical model can represent the basic features of the system, and system identification applies the statistical methods to set up the mathematical models of dynamic systems from available data [1][2][3][4]. ere exist some identification methods for state-space models with and without state-delay [5,6], such as the recursive least squares (RLS) algorithm and the stochastic gradient (SG) algorithm. e SG algorithm is used in adaptive control because of its small computation. Recently, Chen et al. presented an Aitken based the modified Kalman filtering stochastic gradient algorithm for dual-rate nonlinear models [7]. Many identification methods have been developed for linear stochastic systems [8][9][10][11], bilinear stochastic systems [12][13][14], and nonlinear systems with colored noises [15][16][17].
In system identification, some algorithms concentrate on reducing the calculation amount and improving the recognition accuracy [18][19][20]. Since the gradient optimization method only requires calculating the first-order derivative, its computation is small [21]. However, the calculation accuracy of the gradient algorithm is low [22]. In order to improve the calculation accuracy, many improved gradient algorithms have been proposed. Although these improved algorithms can improve the parameter estimation accuracy, the computational complexity is large. e innovation is effective information to improve the parameter estimation accuracy [23,24]. It can promote the convergence of the algorithm in the recursive process. In order to improve the estimation accuracy through using more innovation, the multi-innovation theory has been used in system recognition.
e stabilities and identification of time-delay systems have drawn a great deal of attention of many researchers in system control and system analysis [25]. In industrial processes and control systems, time delays are difficult to avoid due to material transmission and signal interruptions. e time delay makes it difficult for the control system to respond to the input changes in time [26]. In addition, the time delay can cause instability and unsatisfactory performance of the controlled process. e recognition of time-delay systems has been a hot topic [27]. For example, Sanz et al. studied an observation and stabilization of LTV systems with time-varying measurement delay [28]. Li et al. discussed the local discontinuous Galerkin method for reaction-diffusion dynamical systems with time delays [29].
is paper studies identification problems of a dual-rate state-space model with d-step time delay. e main contributions of this paper are as follows. e input-output representation is derived from a canonical state-space model of the state-delay system for the identification through eliminating the state variables in the systems, to derive a joint parameter and state estimation algorithm by means of the multi-innovation identification theory and the state observer for reducing the computational burden and improving the parameter estimation accuracy and the convergence speed.
is paper is organized as follows: Section 2 gives the canonical state-space model for state-delay systems; Section 3 introduces the identification model; Section 4 presents a combined multi-innovation stochastic gradient (MISG) parameter and state estimation algorithm; Section 5 provides an illustrative example; and finally, we offer some concluding remarks in Section 6.

The Canonical State-Space Model for State-Delay Systems
Let us introduce some symbols. e relation A ≔ X or X ≔ A means that A is defined as X; I (I n ) stands for an identity matrix of appropriate size (n × n); z denotes a unit forward shift operator: zx(k) � x(k + 1); T represents the matrix/ vector transpose; θ(k) is the estimate of θ at time k; 1 n means that an n × 1 vector whose elements are all unity; E denotes the expectation operator; adj[X] stands for the adjoint matrix of the square matrix X: adj[X] � det[X]X −1 ; det[X] � |X| represents the determinant of the square matrix X. Consider the following state-space system with d-step state-delay: A ≔ 0 1 0 · · · 0 0 0 1 ⋱ ⋮ ⋮ ⋮ ⋱ ⋱ 0 0 0 · · · 0 1 a n a n−1 a n−2 · · · a 1 where x(k) ∈ R n is the system state vector, u(k) ∈ R is the system input, y(k) ∈ R is the system output, v(k) ∈ R is a random noise with zero mean, and A ∈ R n×n , B ∈ R n×n , f ∈ R n , and c ∈ R 1×n are the system parameter matrices/ vectors. Assume that (c, A) is observable and u(k) � 0 and y(k) � 0 for k ≤ 0. e system matrices/vectors A, B, and f are the unknown parameters to be estimated from the inputoutput data u(k), y(k) . If we remove Bx(k − d) in equation (1), then it becomes the conventional standard state-space model. (1) and (2), if the state vector x(k) is known, the system matrix/vector (A, b) is easy to identify. is paper considers the case that the state x(k) is completely unavailable. e objective is to propose new methods for jointly estimating the unknown states and parameters from the measurement data u(k), y(k): k � 1, 2, . . . and to study the performance of the proposed methods.

Remark 1. For the system in
proposed based on the identification models from observation data [34,35] such as the gradient algorithms, the least squares algorithms, and the Newton algorithms [36].

Remark 2.
is is the identification model of the dual-rate state-space system with d-step state-delay. e information vector φ(kτ) consists of the state vector x(kτ − i), the input u(kτ − i), and the correlated noise w(kτ − i), and the parameter vector θ consists of the parameters a i , b i , and f i of the state-space model in (1) and (2).

Remark 3.
In what follows, a SG algorithm is derived for the state-space system with colored noise. Furthermore, a MISG algorithm is presented to reduce the computational burden and enhance the parameter estimation accuracy. A simulation example is provided to evaluate the estimation accuracy and the computational efficiency of the proposed algorithms.

The Parameter and State Estimation Algorithm
is section derives a multi-innovation stochastic gradient algorithm to estimate the parameter vector θ in (11) and uses the observer to estimate the state vector x(kτ) of the system.

e SG Algorithm.
Defining and minimizing the cost function, and using the gradient search principle, we may obtain a stochastic gradient algorithm: where 1/r(kτ) is the step-size or convergence factor. e choice of r(kτ) guarantees that the parameter estimation error converges to zero. However, difficulties arise in that the information vector φ(kτ) contains the unknown state vector . . , n + d) and the SG algorithm in (13) and (14) cannot compute the estimate of θ in (11). e approach here is to replace the unknown Based on the identification model in (11), we can obtain the following stochastic gradient parameter estimation algorithm for estimating θ: When d ≥ n, we have

e MISG Algorithm.
In order to improve the accuracy of the SG algorithm, we extend the SG algorithm and derive a multi-innovation stochastic gradient algorithm by expanding the innovation length. Define an innovation vector:

Remark 4.
e flop number is used to measure the calculation efficiency (calculation amount) of a complex algorithm. e total number of four floating-point operations required by an algorithm is defined as its calculation amount. Based on this, the calculation efficiency of the algorithm is evaluated as a benchmark, and an efficient and economical algorithm is sought. e calculation method is necessary to analyze the performance of the proposed algorithm.
e computational efficiency of the MISG and the SG algorithms is shown in Tables 1-3. Total floating-point operation (flop) numbers of the MISG and the SG algorithms are N 1 � (n 2 + n + n g )(4p + 3) and N 2 � 7(n 2 + n + n g ), respectively. e difference between the MISG algorithm and the SG algorithm is N 1 − N 2 � n 2 + n + n g (4p + 3) − 7 n 2 + n + n g � (4p − 4) n 2 + n + n g > 0. (40) us, the SG algorithm has smaller computational efforts than the MISG algorithm.

Example
Consider the following dual-rate time-delay system with τ � 2: e parameter vector to be identified is In simulation, the input u(t) { } is taken as an uncorrelated persistent excitation signal sequence with zero mean and unit variance, and v(t) { } as a white noise sequence with zero mean and variances σ 2 � 0.30 2 and σ 2 � 1.00 2 . e parameter estimation based MISG algorithm in (24)-(34) to estimate the parameter vector θ and the state estimation algorithm in (35)- (39) to estimate the state vector x(t) of this example system are applied. e parameter estimates and their estimation errors are shown in Tables 4 and 5 and the parameter estimation errors δ versus t are shown in Figures 1 and 2 with p � 1, 2, 5, respectively, and the state estimates x 1 (t) and x 2 (t) versus t are shown in Figures 3 and 4.
From Tables 4 and 5 and Figures 1-4, we can draw the following conclusions: 6 Complexity Table 1: e computational burden of the MISG algorithm.
Algorithms   (1) e parameter estimates converge fast to their true values for large p, see Tables 4 and 5 (2) e MISG algorithm with p ≥ 2 has higher accuracy than the SG algorithm, see Figures 1 and 2 (3) e parameter estimation errors given by the MISG algorithm become smaller with the data length t and the innovation length p increasing, see Tables 4 and 5 and

Conclusions
is study has taken up a category of state-space models with state time delay as the research background and accordingly developed two folds of solutions for the model identification (parameter and state estimation in specific). e theoretical analysis has proved that the estimates converge to the real value under the condition of continuous excitation in modelling. e algorithms used in this paper can be applied to hybrid switched impulsive power networks and uncertain    8 Complexity chaotic nonlinear systems with time delay [54][55][56][57] and can be applied to other literature studies [58][59][60][61][62][63][64]. e simulation case study has demonstrated that the proposed algorithms/ procedures are effective and efficient in design and implementation.

Data Availability
No data were used to support this study.

Conflicts of Interest
e authors declare that there are no conflicts of interest regarding the publication of this paper.