EstimatingWeak Pulse Signal in Chaotic Backgroundwith Jordan Neural Network

In target estimating sea clutter or actual mechanical fault diagnosis, useful signal is often submerged in strong chaotic noise, and the targeted signal data are difficult to recover. Traditional schemes, such as Elman neural network (ENN), backpropagation neural network (BPNN), support vector machine (SVM), and multilayer perceptron(MLP-) based model, are insufficient to extract the weak signal embedded in a chaotic background. To improve the estimating accuracy, a novel estimating method for aiming at extracting problem of weak pulse signal buried in a strong chaotic background is presented. Firstly, the proposed method obtains the vector sequence signal by reconstructing higher-dimensional phase space data matrix according to the Takens theorem. +en, a Jordan neural network(JNN-) based model is designed, which can minimize the error squared sum by mixing the single-point jump model for targeting signal. Finally, based on short-term predictability of chaotic background, estimation of weak pulse signal from the chaotic background is achieved by a profile least square method for optimizing the proposed model parameters. +e data generated by the Lorenz system are used as chaotic background noise for the simulation experiment. +e simulation results show that Jordan neural network and profile least square algorithm are effective in estimating weak pulse signal from chaotic background. Compared with the traditional method, (1) the presented method can estimate the weak pulse signal in strong chaotic noise under lower error than ENN-based, BPNN-based, SVM-based, and -ased models and (2) the proposed method can extract the weak pulse signal under a higher output SNR than BPNN-based model.


Introduction
As early as the end of the 19th century, the French Poincare found that the solution of the three-body problem could be random within a certain range when he studied the threebody problem, but this discovery did not arouse widespread concern among scholars. It was not until 1963 that Lorenz, an American meteorologist, published a paper [1] that gave the initial condition parameters for generating chaos in differential equations, which were called Lorenz equations. With the continuous in-depth study of chaos, the characteristics and scientific concepts of chaos gradually come into people's vision. Chaotic phenomena can be described as random phenomena in a deterministic system, which is a complex behavior generated by a nonlinear dynamic system. Moreover, chaos has three characteristics: sensitivity to initial value, short-term predictability, and intrinsic randomness.
Pulse signal is a kind of discrete signal, which is usually submerged by noise and not easy to detect. It is widely existed in communication engineering, biomedicine, and industrial automatic control. erefore, the prediction of the pulse signal from chaotic noise has certain practical significance. Experts and scholars at home and abroad have also studied the prediction of the pulse signal in chaotic noise and obtained a lot of research results, such as Volterra filter, correlation detection in time domain, spectral analysis in frequency domain, chaotic Duffing oscillator [2], local linear [3][4][5], multiple kernel extreme learning machine [6], and Kalman filter , extend Kalman filter (EKF) [7], wavelet transform and nonlinear autoregressive model, etc.
At present, experts and scholars have proposed many methods to predict chaotic time series and extract impulse signal from chaotic noise [8,9]. Jiang Xiangcheng proposed the Volterra adaptive method to predict chaotic time series [10]. e improved local projection algorithm proposed by Han Min is a nonlinear chaotic noise reduction method, which combines wavelet analysis with local projection to estimate noisy chaotic signals [11]. Wang Shiyuan et al. proposed the fractional order maximum correlation entropy algorithm [12].
Neural network is a model that imitates the information processing mode of the human nervous system. It is the abstraction and simplification of human neural network. Neural networks can approximate and model the processes of nonlinear dynamic systems [13]. Neural network is divided into forward network, feedback network, random neural network, and competitive network. Because of the rich diversity of neural networks, different neural network to build models of nonlinear systems is also being studied. For example, radial quantity function (RBF) [14,15], support vector machine (SVM) [16], least squares support vector machine (LS-SVM) [17], backpropagation neural network (BPNN) [13], fuzzy neural network (FNN) [18], recursive neural network (RNN) [19,20], multilayer perceptron (MLP), nonlinear regression model (NAR), recursive predictor (RPNN), and ESN [21]. Chandra used the method of memetic cooperative coevolution to train the Elman recurrent neural network on the problem of grammatical inference [22]. Two years later, Chandra used Elman neural network to characterize chaotic time series and added cooperation and competition into them, so as to achieve the purpose of prediction [20].
is paper uses the Jordan network, a neural network established by Jordan in 1986 [23]. Both the Jordan neural network and the Elman neural network (ENN) are cyclic neural networks. e difference is that the Elman neural network feeds the output value of the hidden layer at the previous moment back to the feedback layer, while the Jordan neural network feeds the output value of the output layer at the previous moment back to the feedback layer, which can be used to represent the dynamic system.
In this paper, the Jordan neural network is used to fit the chaotic background and estimate the pulse signal from its residuals. Because the pulse signal is very weak, the characteristics of chaotic time series are mainly reflected in the chaotic background. Firstly, the observed signal was reconstructed, then the Jordan neural network and the single-point pulse signal model were established, the weight and bias of the Jordan neural network were optimized by the gradient descent algorithm, and then the amplitude of the pulse signal was estimated by the profile least square method. e remainder of the paper is organized as follows. Section 2 gives a brief background and related work. Section 3 presents and analyzes this problem. Section 4 estimates the weak pulse signal from a chaotic background by Jordan neural network. Section 5 presents 3 experimental simulations results and discussion. Finally, conclusions and discussion on future work are provided in Section 6.

Background and Related Work
Our work mainly involves two aspects: the application of feedback neural network and the application of feedback neural network in time series prediction.

Application of Feedback Neural Network.
Chaotic time series is a kind of irregular motion in deterministic system and a special time series. However, the traditional time series prediction models-MA, ARMA, and ARIMA-and other models are not ideal for chaotic time series prediction. e applications for chaotic time series prediction are wide and range from financial prediction to weather prediction [20].
Recursive neural networks are used in a wide range of applications, such as weather prediction [20], unbalanced fault diagnosis [24], and detection of network malware [25]. Classification is also an important application of recursive neural network. Recently, recursive neural networks have been widely integrated into convolutional neural networks. Epstein integrates the recursive neural network into the convolutional neural network for image classification. Experiments show that the model is more robust to category-independent attributes [26]. Not only is the wide application range of neural network reflected in its high frequency of use in different fields but also it can be flexibly combined with other networks, models, and methods. Slawek Smyl found that the preprocessing parameters were actually updated formulas from some models of the exponential smoothing family. erefore, it proposes a hybrid prediction method in [27], which mixes the exponential smoothing model with the advanced long and short-term memory neural network in a common framework. e advantages of this hybrid prediction method are that the exponential smoothing equation enables the method to effectively capture the major components of a single sequence, while neural networks allow for nonlinear trending and cross-learning.

Application of Feedback Neural Network in Time-Series
Prediction. Liu et al. combined the dual stage two-phase model with the feedback neural network to make a longterm prediction of multiple time series in [28]. And it has been successfully used to develop widely used expert or intelligent systems. In [29], Wei Wu used ARIMA model, Elman neural network, and Jordan neural network, respectively, to predict the nonlinear time series data of human brucellosis in China, and the results showed that the feedback neural network was more accurate than the traditional ARIMA model. In [30], Zhihao Wang proposed a heart sound signal recovery method based on the long and short time memory prediction model based on the structure of circular neuron network. is method can not only restore incomplete or disturbed signals, but also have filtering effect. e damaged or incomplete heart sound signal has a good recovery effect, which has a promoting effect on medical research. Zhang proposed a new nonstationary time series modeling optimization algorithm in [31]. e algorithm uses moving window and exponential decay weight to eliminate the influence of historical gradient. e regretful boundary of the new algorithm is analyzed to ensure the convergence of the calculation. e simulation results of this algorithm on the data set of short-term power load are better than the existing optimization algorithm.

Problem Analysis
is paper analyzes the weak pulse signal which is fixed chaotic noise. Let t be a time parameter. We can assume that observation sequence, expressed as x(t), contains chaotic noise c(t), white noise signal n(t), and weak pulse signal s(t). So we have a formula as follows: (1) Pulse signal can be boiled down by periodic signal: where a � a 1 , a 2 , · · · , a k Τ , where a i (i � 1, 2, · · · , k) is the amplitude of the weak pulse signal s i (t)(i � 1, 2, · · · , k) and T i is the periodic of s i (t)(i � 1, 2, · · · , k). Estimating weak pulse signal from x(t), we can consider that estimating weak pulse signal s(t) in chaotic background is to estimate the value of a. en, our ultimate goal turns to estimating a.

Phase Space Reconstruction.
Chaotic time series is a univariate or multivariable time series produced by a dynamical system. Phase space reconstruction theorem is mapping a chaotic time series to high-dimensional space. According to the Takens theorem of embedding [32], there is a correlation between one component of the dynamical system and the other components.
For an observed chaotic signal x(t), t � 1, 2, · · · , n { }, the embedding dimension m and the time delay τ could be obtained by using Cao's method [33], and a phase point in the reconstructed phase space can be expressed as en there is a smooth mapping, g: R m ⟶ R, which provides the relationship between X(t) and y(t), denoted as y(t) � g(X(t))(t � n 1 , n 1 + 1, · · · , n − 1). To predict chaotic time series is to find the function g or the approximate function of g. Remark 1. In mathematics, Takens embedding theorem gives the conditions under which a chaotic dynamical system can be reconstructed from a sequence of observations of the state of a dynamical system. e reconstruction preserves the properties of the dynamical system that do not change under smooth coordinate changes, but it does not preserve the geometric shape of structures in phase space. It provides the conditions under which a smooth attractor can be reconstructed from the observations made with a generic function.
Cao's method is a practical method which determines the minimum embedding dimension from a scalar time series. e average ratio of Euclidean distance of a vector with embedded dimension m to Euclidean distance with embedded dimension m + 1 is denoted as E (m). e ratio of E (m + 1) to E (m) stops changing when m is greater than some value m 0 if the time series comes from an attractor. en m 0 + 1 is the minimum embedding dimension we look for.

Jordan Neural Network.
Jordan combined the Hopfield network storage concept and distributed parallel processing theory to create a new circular neural network in 1986 [23]. Jordan neural network is made of 4 parts: input layer, hidden layer, output layer, and context layer, respectively. ere is a first-order delay operator in the connection from the output layer to the context layer, so that the context layer stores the information of the output layer.
ere are two types of activation functions in the neural network: linear activation function and nonlinear activation function. Sigmoid nonlinear activation function which is expressed as f(x) � 1/1 + e − x is used at the hidden layer and the linear function is used at the output layer in this paper. Jordan neural network topology is shown in Figure 1.
In order to express clearly and write easily, the following signs will be introduced: So the Jordan neural network is obtained: To better understand equation (5), we provide a detailed diagram of the first three steps of the Jordan neural network in Figure 2.
e following equation is obtained by the Takens theorem of embedding: where x(t + 1) is the fitting value of the Jordan neural network at time t. And the smooth mapping g can be expressed as in Output layer

Hidden layer
Input layer

Model for Estimating the Pulse Signal.
According to the Takens theorem of embedding, the model of pulse signal, and Jordan neural network, we can obtain the following equation for estimating the pulse signal: en, we can deduce the following equation with unknown parameters involving only g(·) and a: Profile least-square method [34] is a basic idea for fitting a semiparametric model. It is semiparametrically efficient for parametric components, and the nonparametric components are fitted if the parametric components were known. Details of the profile least-square estimation are given in the next section.

Profile Least Square Algorithm.
Profile least square algorithm is usually divided into two steps: first, estimating the nonparametric function with a given a, resulting in g(X(t)) and second, estimating the unknown parameter a with the estimated function g(X(t)).
(A) Estimation of g(X(t)) with Given a. . From equation (10), given the initial value a (null vector), the nonparametric function g(X(t)) is obtained by minimizing the residual of fitting: Gradient descent algorithm is used to update the weights of Jordan neural network to achieve the purpose of optimization. e basic idea of gradient descent algorithm is the linear approximation by firstorder Taylor expansion. erefore, the partial derivatives of the weights and the bias that need to be optimized in this paper are, respectively, calculated as follows: Input: X (t) Complexity e updates of weights and the bias are equal to the sum of the corresponding old one and the product of the partial derivative and the learning rate. (B) Estimating a for Given g(X(t)). For the given g(X(t)), the key to estimate a is to minimize the following equation: a � arg min a n− 1 e method of estimating a is Newton-Raphson iteration method.
Assuming that R(a) � n− 1 t�n 1 [x(t + 1) − g(X(t))] 2 and a 1 is the minimum solution of R(a), so we have (R(a 1 )) ′ � 0. en there is the Taylor expansion at the point a (0) near a 1 : e first derivative and the second derivative of R(a) are obtained with respect to the estimated parameters a: 6 Complexity You obtain the following equation from Sections 4.1 through 4.3: It can be seen that equation (16) contains the first derivative and the second derivative of g(X(t)) with respect to a, and the first derivative is e component form of the first derivative in equation (18) can be written as the following vector form: e second derivative of g(X(t)) with respect to a is Complexity Similarly, the component form of the second derivative in equation (20) can be written as the following vector form: where S(t) is expressed as

Procedure of Algorithm.
We now outline the Algorithm 1.

Remark 2.
Because the mathematical expression of the Jordan neural network is very complex, the difference is used to solve the approximate first and second derivatives for simplifying the calculation when the profile least square is used for estimation. e solution process of difference is as follows: We take 3 points (a 1 , a 2 , a 3 ) near a and compute the first derivative (R(a)) ′ and the second derivative (R(a)) ″ of the function R(a), where (R(a)) ′ is a vector of the same form as a. (R(a)) ″ is a matrix with k rows and k columns.

Experimental Simulations
In order to verify the validity and feasibility of the proposed prediction model, three simulation experiments were carried 8 Complexity out. e experimental data in this paper were generated by Lorenz dynamic system and realized by R language. e codes of experiments are uploaded to GitHub website, link is https://github.com/ling-xiu/jordan/blob/master/An application of Jordan neural network. e mean square error(MSE) is used to measure the recovery accuracy of the model: where e equation of Lorenz dynamic system is in the following form: where σ � 10, b � 8/3, and r � 28. Let x, y, and z be the time parameters that are measured according to discretization and the discretization step size is 0.01. We consider the length of data which is used to solve numerical equation (26) by the fourth-order Runge-Kutta method is 10000. We select the first component of equation (26) as the data of chaotic background noise x(t). With complex autocorrelation and the method of Cao, the embedding dimension can be determined as m � 6 and the delay time can be determined as τ � 7. We normalize the original data sets and there are two kinds of normalized equations. Different neural network structures correspond to different normalization equations in this paper: where

x(t) is the original data and z(t) is the normalized data sets. E(x(t)) is the mean of x(t) and σ x(t) is the standard deviation of x(t).
Absolute error (AE) and absolute percentage error (APE) are used to measure the error, and they are SNR input defines the ratio of the variance of s(t) to the variance of the sum of c(t) and n(t).

Example 1: Prediction of Pulse Signals with Different SNRs.
Chaotic time series c(t) is generated by the Lorenz dynamic system and the index t ranges from 3000 to 7000 for ensuring the chaotic nature of the data. Pulse signal s(t) is generated by equation (2) and the unknown parameters T i and a are assumed as shown in Table 1. e experimental data consists of c(t), s(t), and white noise sequences n(t) with a mean of zero.
To demonstrate the stability of the model presented in this paper, we performed Monte Carlo simulations consisting of 50 runs on the same data and the experimental results were averaged.
From Table 1 and Figure 3, we can see that the convergence rate is very fast, and the iteration times of the 12 sets of data in the experiment can all reach the given accuracy within 20 times. Second, both the absolute error (AE) and the absolute percentage error (APE) of the amplitude of the impulse signal are very small. e mean square error (MSE) of the recovered signal is less than 10 − 5 . As can be seen from Table 2, the model proposed in this paper is very stable. For these 12 sets of data, no difference in the amplitude of the recovered pulse signal was found in either one experiment or 50 Monte Carlo simulation experiments. e Input: Observation sequence x(t), the initial value of a(a (0) ), precision (0.00001). Output: a (1) Begin While (a (1) − a (0) is greater than the given precision); Step 1: For given a (0) , update the series x(t) by equation (9).
Step 2: e phase point data set X(t) is obtained by normalizing and reconstructing x(t) with the embedding dimension and the time delay.
Step 3: Based on (A) in Section 4.3.1 above, the Jordan neural network was trained to use the phase point data set X(t).
Step 4: a (1) is updated using the difference. End while Obtain the optimal a (1) to achieve the accuracy. End ALGORITHM 1: Jordan neural network estimation algorithm.

Example 2: Prediction Comparison of Different Models.
In this experiment, different models were used to fit the same set of observed data, which had an amplitude of 2.5 and a period of 400. e SNR of the observed data is − 36.42872. e chaotic noise and original observation signal are shown in Figure 4. e models that fit the reconstructed original data are Jordan neural network (JNN), Elman neural network (ENN), backpropagation neural network (BPNN), support vector machine (SVM), and multilayer perceptron (MLP).
From Table 3 and Figure 5, the Jordan neural network has the smallest absolute error between the estimated amplitude and the real value, while the MLP has the largest absolute error. It can be seen from in Figures 6(a), 6(c), and 6(e) that the fitted value of the original signal by Jordan neural network is significantly closer to the real value than support vector machine (SVM) and multilayer perceptron (MLP). It can be seen from the prediction error graph of Figures 6(b), 6(d), and 6(f ) that the prediction error of Jordan neural network is relatively large at the time point with pulse signal, and the error value is close to the amplitude of the pulse signal, while the time point error of the rest without pulse signal is relatively small. Support vector machine and multilayer perceptron have large prediction error.

Example 3: Jordan Neural Network Compared with
Backpropagation Neural Network. SNR output is the signal-tonoise ratio after extracting the pulse signal. SNR output defines the ratio of the variance of s(t) to the variance of residual: Figure 7 takes the input SNR as the horizontal axis and the output SNR as the vertical axis. e experimental data are 12 sets of data from experiment 1, and then the backpropagation neural network estimation is carried out on the basis of experiment 1. It can be seen from Figure 7 that the output SNR increases with the increase of the input SNR, and they are proportional to each other. And with the increase of the input SNR, the increase of the output SNR slows down. It can also be seen from the figure that the output SNR of the Jordan neural network is 10-20db higher than the output SNR of the backpropagation neural network.

Discussion.
is section reports the performance of the profile method combined with Jordan neural network in estimation of pulse signals against chaotic background. e purpose of the experiments was to evaluate if JNN can maintain quality in prediction performance when compared to conventional methods. e results have shown that the method proposed in this paper not only solves the weak pulse signal in chaotic signal well, but also has good stability. We evaluate the application breadth of the proposed model by comparing the experimental data of different SNR. Table 1 shows that the    Within this range, the model proposed in this paper can effectively estimate the pulse signal amplitude, and its absolute error (AE) can be as low as 0.00112 and as high as 0.05, and its MSE is also below 10 − 5 . e results report the estimation of pulse signals by different neural networks under chaotic noise background, as well as the mean amplitude and standard deviation (std) of 50 test runs in Table 3. We can also see from Table 3 that the two minimum standard deviations are 0.00421 and 0.00531, which are, respectively, from the Jordan neural network and the Elman neural network. We know that both the Jordan neural network and the Elman neural network are feedback neural networks. e difference between them is that the context layer connects to a different layer. Compared with [35], we found that the Jordan neural network could better fit the time series than the Elman neural network.

Complexity
A major innovation of this method is to use Jordan neural network to fit the chaotic noise background and then estimate the impulse signal in the chaotic noise by combining the profile method. Compared with other neural networks, this method is more accurate and the overall training time is shorter.

Conclusions
In this paper, we are interested in weak pulse signal in chaotic background. Based on the short-term predictability and Takens theorem, we provide an algorithm for directly estimating the problems; that is, the Jordan neural network is used to fit the chaotic time series, and the one-step prediction error was obtained. en, starting from the error, the single-point jump signal model was connected, and the amplitude of the pulse signal was estimated by the profile least square method, so as to achieve the prediction effect of the pulse signal under the chaotic background. e following conclusions can be drawn from the experimental results: the model proposed in this paper can predict the weak pulse signal in the chaotic background, and it can be seen from the results of experiment 1 that the prediction accuracy is high. It can be seen from the results of experiment 2 that the Jordan neural network is significantly better than other feedforward neural networks in fitting the nonlinear dynamic system, and the absolute percentage error of the Jordan neural network is the smallest. In future work, we can try to improve in two aspects, so that it has the best generalization performance in most cases, and apply the method to solve the relevant practical problems. Firstly, we can try to estimate the amplitude of the impulse signal without knowing the period of the impulse signal. Secondly, the Jordan neural network can be trained by other optimization methods, and the Jordan neural network can be improved.

Data Availability
No data were used to support this study.