An Intelligent Calculation Method of Volterra Time-Domain Kernel Based on Time-Delay Artificial Neural Network

To solve the problems of high complexity and low accuracy in Volterra time-domain kernel calculation of a nonlinear system, this paper proposes an intelligent calculation method of Volterra time-domain kernel by time-delay artificial neural networks (TDANNs) and also designs a root mean square error (RMSE) index to choose the neuron number of the network input layer. Firstly, a three-layer TDANN is designed according to the characteristics of the Volterra model. Secondly, the relationship between parameters of TDANN and Volterra time-domain kernel is analyzed, and then three-order expressions of Volterra timedomain kernel are derived.*e calculation of Volterra time-domain kernel is completed by network training. Finally, it is verified by a nonlinear system. Simulation results indicate that compared with traditional methods, the new method has higher accuracy, and it can realize the batch calculation of Volterra kernel, which not only improves the calculation efficiency but also provides accurate data for fault diagnosis based on Volterra kernel in further research work.


Introduction
As the transfer function of a linear system, Volterra kernel (including Volterra kernel time-domain kernel and Volterra kernel frequency-domain kernel) does not depend on the input and output of the system, which can characterize the essential properties of the nonlinear system and explain the special phenomena of the nonlinear system. It has advantages of clear physical meaning and rich information, which is widely used in electronic engineering, mechanical and electrical engineering, control engineering, and other fields [1][2][3][4][5][6].
At present, the research and application of Volterra kernel have developed rapidly; especially the nonlinear spectrum research based on the generalized frequency response function (GFRF) and the output spectrum response function (OFRF) has been gradually applied to the fields of feature extraction, mechanism analysis, and fault diagnosis [7][8][9][10][11]; however, the complexity and accuracy of Volterra kernel calculation are the key factors to its application. Until now, the calculation of Volterra kernel is still difficult, and the main calculation methods of Volterra kernel are the recursive method and identification method. e recursive algorithm is to substitute the relevant parameters of the system into the recursive equation to calculate the Volterra kernel. Billings and Tsang [12] adopted recursive algorithm to obtain Volterra kernels of each order for a nonlinear system. Jing et al. [13] established a description function from low-order Volterra kernel to high-order Volterra kernel to calculate the Volterra kernel of each order based on the nonlinear differential equation. Billings and Peyton Jones [14] adopted the harmonic probing method to derive the recursive algorithm of Volterra frequency-domain kernel. Swain and Billings [15] adopted recursive algorithm to calculate the Volterra kernel of an MIMO system. e research studies mentioned above calculated the Volterra kernel based on a system mathematical model, and the calculation accuracy depends entirely on the accuracy of the model. Obviously, recursive algorithm is not suitable for the system with uncertain mathematical model. e identification algorithm is a "black box" operation, which does not rely on the mathematical model of the system; it only relies on the input and output data of the system. e Volterra kernel of each order can be calculated by adjusting the parameters according to the error between the true output and the estimated output. Toker and Emara-Shabaik [16] proposed LMS identification algorithm by using pseudorandom multilevel sequence excitation, which can avoid matrix inversion and improve calculation speed and accuracy. Sigrist et al. [17] adopted the improved LMS method to identify the first two-order Volterra time-domain kernel. Asyali and Juusola [18] expanded the Volterra kernel by the Meixner function to identify the Volterra model of a nonlinear system with time delay. Prazenica and Kurdila [19] proposed multiwavelet function to identify Volterra time-domain kernel, and the first three-order Volterra timedomain kernel function of a nonlinear system was obtained. e identification algorithm can realize the adaptive calculation, but this method calculated one Volterra kernel value at one time, which cannot achieve batch calculation and cannot meet the needs of fault diagnosis based on data. erefore, there are at least two shortcomings in the traditional calculation of Volterra kernel: (1) the calculation efficiency and process are low and complex, respectively; (2) due to the approximate processing in the calculation process, the error between the estimated value and the true value is large.
In order to solve the problems of calculation methods mentioned above, a new intelligent calculation method is urgently needed. Artificial neural network (ANN) is a parallel distributed information processing network. e neurons in the network have strong ability of nonlinear approximation and fitting, which can approach any nonlinear function [20][21][22]. In recent years, to simplify Volterra kernel calculation, a small number of researchers have tried to adopt ANN to calculate the Volterra kernel. Wray and Green [23] and Stegmayer et al. [24] adopted the weight coefficient of ANN to express the Volterra kernel of each order, and it was also verified by a nonlinear system. In references [25,26], a Volterra kernel identification method based on time-delay neural network (TDANN) was proposed, and the relationship between the parameters of network and Volterra kernel function was derived, which successfully identified the first three-order Volterra kernel of nonlinear unsteady aerodynamic loading. Although ANN adopted in above research studies can realize fast and accurate calculation of Volterra kernel, there are still some disadvantages, for example, the number of input nodes in the network structure was set by experience, and the analysis of input node number impact on the calculation accuracy was lacking; furthermore, the network structure was established with great randomness and arbitrariness, which has no adaptability.
e main contributions of this paper are as follows. (1) Based on the research studies by the authors of [23][24][25][26], the impact of input node number and the memory length of TDANN network on the accuracy and calculation speed is analyzed and studied. (2) Root mean square error (RMSE) is introduced as the reference of neuron node number selection to realize network structure adaptive adjustment, which can avoid the network structure selection blindness. After obtaining the optimal structure, only the input and output of the system were needed to realize the intelligent calculation of the Volterra kernel. (3) e proposed method was verified by a nonlinear system. e reason why it was called "intelligent calculation" is that compared with the traditional calculation method, the whole calculation process does not need human's participation, which can avoid the complex calculation process. In Section 2, the mathematical model of Volterra time-domain kernel is provided. Section 3 introduces the concept of TDANN. In Section 4, the relationship between the Volterra kernel model and TDANN model is analyzed, and the explicit expression of Volterra time-domain kernel is obtained. In Section 5, the proposed method is simulated and verified by a nonlinear model. e conclusions are summarized in Section 6.

Volterra Time-Domain Kernel Model
For any continuous time invariant nonlinear system, if the system input satisfies then, the output of system can be expressed by Volterra series [27][28][29], namely, where y(t) is output of the system, u(t) is input of the system, and h n (τ 1 , τ 2 , . . . , τ n ) is the n-order Volterra timedomain kernel or generalized frequency response function. e n-order Volterra kernel model can be obtained by truncating (2), which is shown as Discretize equation (3) to obtain the discrete Volterra kernel model, which is shown as 2 Mathematical Problems in Engineering For a system with memory length M, equation (4) can be changed to Generally speaking, the energy components of Volterra kernels above three-order are very small, and it is also not easy to visualize the graphs over three dimensions [30]. erefore, this paper only calculates the first three-order Volterra kernel of the system.

Time-Delay Artificial Neural Networks (TDANNs)
As shown in Figure 1, the artificial neural network is composed of many neurons which are related to each other to achieve efficient data processing. Such a network is also known as feedforward multilayer perceptron. Artificial neural network is generally composed of an input layer, hidden layer, and output layer. A simple three-layer network can realize the approximation of any nonlinear function. A mapping of data from the input layer to the output layer is called training. In order to achieve accurate approximation, several training iterations are required. In this process, the network constantly updates the weights and thresholds between different neurons. According to the error between true value and calculation result, the weights and thresholds are modified in turn. After updating several times, the calculation result is closer to the true value, and the optimal weight and threshold can also be obtained. ANN is a typical MIMO neural network with M inputs and N outputs. e selection of input and output of a neural network depends on the characteristics of the specific system. For the Volterra kernel model, from equation (5), it can be seen that the Volterra kernel model with memory length . For a single-input single-output (SISO) system, how to construct dataset with M+1 inputs and 1 output is important. e specific process is shown in Figure 2.
e artificial neural network takes M + 1 delay neurons as the input layer and 1 output neuron as the output layer, which is called time-delay artificial neural network (TDANN). Because the network adopts forward time delay, it has the function of memory. e structure of TDANN is shown in Figure 3.
In Figure 3, ω 11 , φ 12 , . . . , ω Mp are weights between the input layer and hidden layer; b 1 , b 2 , . . . , b p are thresholds on the hidden layer; ω 1 , ω 2 , . . . , ω p are weights between the hidden layer and output layer; and b 0 is the threshold on the output layer.

Equation Derivation of Volterra Kernel Based on TDANN
According to equation (3), the first three-order Volterra kernel model with memory length M can be shown as For the TDANN model shown in Figure 3, the selection of activation function and number of neurons in the hidden layer directly affects the accuracy of output. In this paper, sigmoid function is chosen as the activation function of neurons in the hidden layer, which is shown as Mathematical Problems in Engineering e number of hidden neurons has a significant impact on the performance of the network. If the number of neurons is too small, the calculation accuracy is low. If the number of neurons is too large, the overfitting phenomena will occur. So, according to the Volterra kernel characteristics, this paper adopts root mean square error (RMSE) as an evaluation index to determine the reasonable number of hidden neurons, which is shown as where y(j) is the true value of output at time j; y(j) is the predicted value of output at time j; and RMSE represents the standard deviation of error samples between the estimated value and the true value, which reflects the dispersion degree of sample. e smaller the RMSE is, the better the approximation effect is.
φ(x) is chosen as activation function between neurons in the input layer and hidden layer, and pureline function is chosen as activation function between neurons in the hidden layer and output layer; then, the j-th output y j can be expressed as where M is the memory length of the network; ω ij is the weight between the i-th neuron in the input layer and the jth neuron in the hidden layer; u i is the i-th input; and b j is the threshold of j-th neuron in the hidden layer. e output of TDANN can be expressed by where p is the number of neurons, ω j is the weight between the j-th neuron in the hidden layer and output layer, and b 0 is the threshold on the output layer. Substituting equation (9) into equation (10) and expanding φ(x) with Taylor series, equation (12) can be obtained.
where φ (n) (b j )stands for the n-th derivative of φ(x) at b j . Because only the first three-order Volterra kernel values are considered in this paper, equation (11) can be translated into  Figure 2: e process of dataset construction.

Input layer
Hidden layer  Mathematical Problems in Engineering Expanding n and i in equation (13), equation (14) can be obtained.
Comparing equation (6) and equation (14), the first three-order Volterra time-domain kernel calculation equations can be obtained, which are shown as follows: It can be seen from equations (15)-(21) that the first three-order Volterra time-domain kernel expressions are completely composed of weights and thresholds of TDANN. We substitute the optimal weights and thresholds into the equations above to calculate the Volterra kernel of each order. Compared with the traditional calculation method, the new method only needs the input and output of system to train the network and does not need manual participation, which can realize the intelligent calculation of Volterra kernel.
Mathematical Problems in Engineering 5

Simulations
According to reference [31], the dynamic model of the nonlinear oscillator is shown in Figure 4. e nonlinear differential equation of the model can be expressed as follows: where m is the mass of the oscillator; c is the damping coefficient of the system; k 1 and k 2 are linear stiffness parameters and k 3 is nonlinear stiffness parameter; u(t) is external force applied to the system, which is the input of the system; and y(t) is the displacement of the vibrator, which is the output of the system. e parameters are setting the input as u(t) � 50cos(0.4πt).

e Effect of Volterra Memory Length M on System
Output. From Figures 2 and 3, it can be seen that the memory length M not only affects the neuron number of the input layer but also affects the composition of dataset. If M is too small, the nonlinear correlation between neurons is not high. If M is too large, overfitting phenomena may occur. So, it is very important to choose M reasonably to ensure the output accuracy. In order to facilitate the analysis, this paper makes the neuron number of the input layer equal to the hidden layer, namely, M + 1 � p. e influences of M on the network output are tested, which are shown in Figures 5-7 and Table 1. e specific effects of memory length M on system output are shown in Table 1.
From Figures 5-7 and Table 1, it can be seen that M has a great effect on the accuracy of system output. At the beginning, when M is gradually increasing, the system output error tends to decrease due to the increase of correlation degree between neurons in the network. However, with the increase of M, the degree of correlation between neurons is increasing, but the overfitting phenomena occurred, so the system output error shows oscillation. In addition, with the increase of M, the structure of network becomes more complex, and the time cost of network training and testing is longer. Furthermore, based on the RMSE index, the memory length M is selected to be 8.

e Calculation of Volterra Kernel.
When M � 8, the neuron number of the input layer is 9, and the neuron number of the hidden layer is 9. After TDANN training, the optimal weight of different neurons between the input layer and hidden layer is c m u (t) e optimal threshold of neurons in the output layer is e first three-order Volterra time-domain kernel can be obtained by substituting the parameters above into equations (15)- (20), and the results are shown in Figures 8-10.
In order to prove that the identified kernels hold the correct dynamics, predicted responses can be obtained by the first three-order Volterra time-domain kernels, which are shown in Figure 11.
From Figure 11, it can be seen that the predicted response using first-order kernel is similar to the system input, which stands for the linear system response, and the predicted response using first two-order kernels begin to appear as nonlinearity components; furthermore, with the order of kernel increasing, the nonlinear components are more and more obvious and also close to the real response.

e Robustness of the Proposed Method.
In order to demonstrate the robustness of the proposed method, Gaussian noise is added to the system input; the first threeorder Volterra kernels are calculated by TDANN, and then the system output estimated value can be obtained by equation (6), which is shown in Figure 12. From Figure 12, it can be seen that due to the influence of noise, the estimated output value of the system is in fluctuation state at beginning, and the errors between the estimated value and the true value are relatively large, but after about 2 s, the estimated value begins to approach the true value, and after 4.7 s, the error between the estimated value and the true value is getting smaller and smaller. So, it can be demonstrated that the proposed method has robustness.

e Accuracy Comparison with Traditional Algorithm.
In order to verify the accuracy of Volterra kernel calculated by TDANN, the first three-order Volterra time-domain kernels of the nonlinear system are calculated by TDANN, recursive algorithm in reference [32], and variable step size least mean square (VSSLMS) adaptive identification algorithm in reference [33], respectively. en, three different output results are obtained by equation (6). e accuracy of three methods is shown in Figures 13 and 14.
From Figures 13 and 14, it can be seen that among three methods, the accuracy of the TDANN method is highest, so each order of Volterra time-domain kernel (as shown in Figures 7-9) calculated by TDANN is closest to the true value of Volterra time-domain kernel. In the VSSLMS identification algorithm, all kinds of parameters have been adjusted to the best, but the identification accuracy is not ideal for the system; two reasons can account for such phenomena. Firstly, the adjustment of parameters in the identification algorithm is based on experience, which is subjective. Secondly, the accuracy of the identification algorithm may be affected by the data type. So, the accuracy of identification algorithm is the lowest. It is worth to notice that in order to further enhance the accuracy, TDANN can easily realize the above three-order Volterra time-domain kernel calculation, while it is difficult for others to calculate the Volterra time-domain kernel above three-order.   Mathematical Problems in Engineering calculation of multiple Volterra kernels, which can improve the calculation speed and efficiency, while with identification algorithm, one operation can only calculate one Volterra kernel value. In order to demonstrate that TDANN can reduce the computational complexity, the recursive algorithm in reference [32] and the identification algorithm in reference [33] are adopted to randomly calculate several first three-order Volterra kernels of the system. e time consumption of three methods is shown in Table 2.

e Computation
It can be seen that the identification algorithm consumed the most time, and TDANN consumed the least time. e time consumption of identification algorithm, recursive algorithm, and TDANN is 1325.25 ms, 559.2 ms, and 366.96 ms, respectively. Compared with identification algorithm and recursive algorithm, the calculation speed of TDANN is increased by 72.31% and 34.38%, respectively. erefore, the method proposed in this paper reduced the complexity of calculation, which can realize the online calculation and detection of the system.

Conclusions
To solve the problems of large computation amount and low accuracy in the conventional method to calculate the Volterra time-domain kernel of the nonlinear system, this paper proposed a novel method to intelligently calculate Volterra time-delay kernel by TDANN; by comparing the Volterra kernel model with the TDANN model, the connection of the two models has been found, and the complex Volterra timedomain kernel calculation was transformed into the optimal parameter calculation of neural network; the novel method has been well verified on a nonlinear system. e main achievements can be summarized as follows. Compared with recursive algorithm, the TDANN method has three contributions: Firstly, it can avoid the complex modeling operation of the system; only with input and output of the system, the first three-order Volterra timedomain kernel can be calculated, so it is more adaptable. Secondly, it transforms the complex calculation into the optimal parameter solution of the network, and the Volterra time-domain kernel can be calculated through network training, so it is more intelligent. irdly, it can adjust the network structure according to the output of the network to ensure the calculation result is closer to the true value, so it is more adaptive.
Compared with the identification method, the TDANN method can not only achieve batch calculation at one time but also reduce the time consumption, so the calculation efficiency is high, which provides the possibility of online calculation.
However, the three-order Volterra kernels calculated by TDANN are only parts of the whole Volterra kernel information because the value of M cannot be too large. How to balance the amount of Volterra kernel information and time consumption is the future work.

Data Availability
No data were used to support this study.