Incomplete Phase Space Reconstruction Method Based on Subspace Adaptive Evolution Approximation

,


Introduction
In recent years, industrial disasters and accidents occurred frequently, the meteorological and hydrological conditions were complicated and changeable, and financial markets fluctuated drastically.These phenomena often contain chaotic characteristics [1,2], and prediction [3] for these phenomena is imminent.For a long time, there was no scientific tool for handling this issue, because the changing mechanisms of characteristics in these phenomena were not understood very well.Hence, aiming at the chaotic characteristics, some scholars worked with structures and made a lot of new researches on the prediction of chaotic time series [4][5][6][7][8].
To study and deal with the measurement data of chaotic system, Kennel et al. presented the reconstruction method of phase space system.Two parameters, the embedding dimension  and delay time , needed to be determined before the phase space reconstruction [9,10].At present, time delay selection methods that are commonly used in the chaotic short-term prediction mainly include autocorrelation method [11], mutual information method [12], and singular value fraction method [13].Calculating methods of embedding dimension mainly include saturated correlation dimension [14], false nearest neighbors method [15], and Cao's method [16].Hu and Chen put forward the C-C method [17], which can simultaneously estimate the delay time  and embedding dimension  with the correlation integral.Autocorrelation method extracts only linear correlation degree between time series, which is hard to be applied to high-dimensional chaos system and nonlinear dynamical system.Mutual information method, which can determine the optimal delay time by calculating the first minimal value of mutual information function, is a nonlinear analysis method, but it cannot avoid massive calculation and cannot satisfy the requirement of complicated space division.It is difficult to determine the threshold of the singular value, because the singular value fraction method is largely affected by noise.When the embedded dimension with the saturated correlation dimension is calculated, the main question is to choose the different neighborhood radius.The radius selection has certain randomness, and the result will be in large deviation with improper choice, because of the influence of noise in the data and excessive concentration of the data.The determination of threshold has very strong subjectivity when we use false nearest neighbors method to determine the embedding dimension.There is no objective standard to determine the threshold value, especially for the experimental data, which may get a wrong result.Cao's method, an improved false nearest neighbors method, can effectively distinguish random signals and deterministic signals, and embedding dimension can be obtained through a less amount of data.C-C method is based on the statistical theory, so  cannot be precisely determined.
Researches have showed that different phase space reconstruction methods get different  and .Moreover, the same chaotic time series with the same kind of method in different times may get different  and .There is no phase space reconstruction that can obtain complete and independent phase space.After phase space reconstruction, prediction model is often established through the functional approximation method.
The prediction model based on phase space reconstruction has been used to adopt the functional approximation method based on the neural network [18][19][20][21], which has strong nonlinear fitting capability and can approximate any complex nonlinear relationships.However, since neural network is only suitable for approximation of a deterministic system, it is difficult to guarantee the time-varying system performance and ensure its generalization performance in other untrained neighborhood.Meanwhile, the prediction effect of neural network is not good, because the chaotic time series is a complex nonlinear uncertain system.
In this study, we introduce Kalman filtering to neural network model [22], inspired by Kalman iteration and Bucy and Sunahara's nonlinear extended Kalman filtering theory [23].The subspace approximation of neural network based on the nonlinear extended Kalman filtering (EKF) has a function which is dynamic evolution approximation from one neighborhood to another.Therefore, we can constitute a phase space by choosing a kind of phase space reconstruction method, and the space may be incomplete, not separate, and can be seen as a subspace of the ideal phase space.On this basis, we put forward adaptive neural network model based on nonlinear Kalman filtering and finally realize the subspace approximation of dynamic evolution system.In addition, we simulate wind speed series in Wulong city using the proposed method.By comparing with BP neural network prediction model, the results show that our method possesses higher prediction accuracy.
The paper is organized as follows.Section 2 discusses about the subspace approximation of phase space reconstruction.In Section 3, we describe the neural network model based on nonlinear Kalman filtering.Section 4 uses practical examples and series tests to verify the proposed method, while Section 5 contains the conclusions of the present work.

Subspace Approximation of Phase Space Reconstruction
Reconstructing phase space by chaos theory needs to identify the chaos of time series.Single variable time series can be reconstructed into a phase space by Takens' embedding theorem in phase space reconstruction [24,25]; that is, the original dynamical system can be restored in the sense of topological equivalence as long as the embedding dimension is sufficiently high.For the observed time series (1), (2), . . ., (), after time delay reconstruction by Takens embedding theorem, it will receive a set of space vector X () = { () ,  ( + ) , . . .,  ( + ( − 1) )} ,  = 1, 2, . . .,  =  − ( − 1) .
After phase space reconstruction, the data space is Accordingly, we acquire where  is a single-valued function.Then, we have However, it cannot be really obtained as the data are often limited.Hence, f : R  → R can only be constituted by limited measurement data, making f sufficiently approximate to , consequently we can get a nonlinear prediction model.This paper employs the neural network to predict chaotic series.However, the neural network cannot readily handle the inconsistency of the phase space reconstruction because of uncertain nonlinear chaotic time series.Therefore, it is crucial to adaptively construct subspace to approximate chaotic series through the incomplete phase space.The feature of adaptive subspace approximation is that it can add new data in real time and forget old data in the process of training.Consequently, weights and thresholds of the neural network are continuously modified to realize the dynamic evolution modeling.

Neural Network Model Based on Nonlinear Kalman Filtering
Kalman filtering has good adaptability.It can dynamically update and forecast the system information in real time with limited data.However, it cannot be readily used for complicated nonlinear model.Meanwhile, the extended Kalman filtering (EKF) is a kind of effective method to handle nonlinear filtering.The mathematical model of EKF is as follows: where W  and V  are independent, zero mean, and Gaussian random processes with covariance matrices Q and R, respectively.The statistical properties are as follows: EKF spreads nonlinear functions (⋅) and ℎ(⋅) to Taylor series around filtering value X and predicted value X−  , respectively, only retaining the first-order information.Hence, the linearization model of the nonlinear system is obtained, and then we can obtain the EKF formula in nonlinear system by basic equations of Kalman filtering.
Given a forward network with  layers, the numbers of neurons in each layer are   ( = 1, 2, . . ., ).Suppose that input layer is the first layer and output layer is the th layer.The weights of the th layer neurons are    ( = 1, 2, . . .,  −1 ;  = 1, 2, . . .,   ).In order to convert the calculation of connection weights    in the above problem into filter recursive estimation form, we let all of the network weights constitute the state vector where state vector W consists of all of the weights according to the linear array, and its dimension is as follows: Then the state equation and measurement equation of the system can be expressed as where Y  is the expected output, X  is the input vector, and Y  is the actual output.The measurement noise V  is assumed to be additive, white, and Gaussian, with zero mean and with covariance matrix defined by Suppose that the output of the th node for the th layer in the th iteration is From ( 10) and ( 12), we have (3) Measurement update (correct) R  are process noise covariance and measurement noise covariance, respectively, H  is the Jacobian matrix of observable model, θ−  is the optimal predictive value for step  according to step  − 1, and θ is the optimal filter estimate for step .

Assume that
Accordingly, the measurement equation may also be expressed as The Jacobian matrix of the function ℎ(⋅) is described by Similarly, all thresholds of the network constitute the state vector where the dimension is Suppose that W and b are both state variable; that is, the state vector composed of weights and thresholds is described by Kalman filtering algorithm on training weights and thresholds of the neural network is as in Table 1.

Determining of Embedding Dimension and Delay Time.
One of the most popular chaos logistic mapper is selected as the study object.Logistic equation is The related time series are produced according to (20).It is a chaotic system when  = 4. Assume that initial value of series is 0.1, and 4000 points are calculated.The first 1000 points are eliminated as transition phenomenon, leaving the remaining 3000 points to reconstruct phase space.Before the phase space reconstruction, we determine the embedding dimension  and delay time .A comparison among several methods is present in Table 2.
Obviously, the optimal embedding dimension and delay time are generally different by different methods of phase space reconstruction.
In order to verify the fact that data at different time will obtain different embedding dimension  and delay time  with the same phase space reconstruction method, we have the following experiment.
Apparently, the data during different time periods will acquire different embedding dimension and delay time by using the same phase space reconstruction method.

Wind Speed Chaotic Series Forecasting Simulation.
Analysis about the chaotic characteristics of wind speed in the process of wind power generation has been presented in a related article [26].We record one of the wind speed data every 10 minutes, and 150 groups of wind speed data in Wulong city are used to simulate experiments in our study.We obtain the corresponding  and  by different phase space reconstruction methods, as shown in Table 4.
Various combinations are present in Table 5.
Wind speed prediction [27,28] of chaotic time series about neural network model usually extracts phase space reference points as the BP neural network training samples on the basis of phase space reconstruction.We establish the neural network model based on nonlinear Kalman filtering, including two parts: predict wind speed and constantly modify weights and thresholds of the neural network by Kalman recursion.In this paper, BPNN model in the same structure is employed to forecast wind speed time series, in order to illustrate the validity of EKFNN on predicting the chaotic time series.The same 150 groups of wind speed data are used to simulate experiments.The predicted curves and error curves are shown in Figures 1, 2, 3, 4, and 5.
Comparisons among several models in four indices are present in Table 6.
We list 12 groups, a total of 2 hours of wind speed forecasting results in two methods, under the same phase space reconstruction.Compare the prediction performance in the next 10 min, 20 min, 30 min, and up to, 120 min.Comparisons among several prediction results in two methods are present in Table 7.
Figures 1-5 show that relative error of wind speed prediction by EKF neural network is much smaller than that by BP neural network, through observing the future wind speed prediction of 150 groups.As can be seen in Table 6, the prediction effects are largely different by different kinds of phase space reconstruction methods.Four performance indices, which are Mean Absolute Error (MAE), Mean Relative Error (MRE), Mean Square Error (MSE), and Sum of Squared Error (SSE), of EKF neural network, are also far less than those of corresponding general neural network.
Apparently, EKF neural network can solve the inconsistency problem of phase space reconstruction and approximate chaotic time series well through subspace.The neural network model based on EKF has outstanding adaptability, so it can predict the wind speed chaotic time series with higher precision, compared with BP neural network.
Furthermore, we can conclude that in Table 7, prediction accuracy of EKF neural network is higher than that of BP neural network, by comparing the prediction performance of wind speed in the next 10 min, 20 min, 30 min, and up to, 120 min.It demonstrates that EKF neural network model, which has better dynamic adaptability, can better the prediction of wind speed time series with nonlinear chaotic characteristics.Therefore, the proposed phase space reconstruction method of the adaptive evolution approximation in this paper is an effective approach.

Conclusion and Further Work
The phase space reconstruction cannot meet characteristics of the completeness and independence, and the results with different reconstruction methods are obviously inconsistent.The reconstructed phase space is a subspace of the ideal space.If a subspace approximation can make the real-time dynamic evolution, then the initial constructed phase space, for which the evolution is adaptive subspace approximation, can finally approximate to the ideal phase space much better.
In this paper, neural network model based on nonlinear Kalman filter is established, by dynamic adaptivity of nonlinear Kalman filter.The model will add new samples in real time and gradually eliminate previous data, as a moving samples window, and the evolution of the training sample continually updates weights and thresholds of the neural network.As a result, adaptive subspace approximation is implemented by reconstructed incomplete phase space.
The optimized plan, which combines the nonlinear Kalman filter with neural network, sufficiently utilizes the nonlinear approximation capability of neural network and dynamic adaptive ability of real-time update correction of nonlinear Kalman filter.Consequently, it can realize subspace adaptive evolution approximation and solve the inconsistency problem of phase space reconstruction.Therefore, it is a nice direction in research into chaotic prediction.Future research can be performed in a number of areas.It provides a good technical support in studying problems of meteorology, hydrology, and finance fields.

1 (
b) Relative wind speed error

Table 1 :
Extended Kalman filtering neural network algorithm.

Table 2 :
Comparison among phase space reconstruction methods.

Table 3 :
Parameters of the same phase space reconstruction during different time periods.

Table 4 :
Comparison among phase space reconstruction methods.

Table 5 :
Various combinations on two forecasting methods.

Table 6 :
Different wind speed model index.
MAE, MRE, MSE, and SSE are Mean Absolute Error, Mean Relative Error, Mean Square Error, and Sum of Squared Error, respectively.

Table 7 :
Observed wind speed data and predicted data.