^{1}

^{1}

^{2}

^{1}

^{1}

^{1}

^{2}

This paper discusses the prediction of time series with missing data. A novel forecast model is proposed based on max-margin classification of data with absent features. The issue of modeling incomplete time series is considered as classification of data with absent features. We employ the optimal hyperplane of classification to predict the future values. Compared with traditional predicting process of incomplete time series, our method solves the problem directly rather than fills the missing data in advance. In addition, we introduce an imputation method to estimate the missing data in the history series. Experimental results validate the effectiveness of our model in both prediction and imputation.

The subjects of time series prediction have sparked considerable research activities, ranging from short-range-dependent series to long-range-dependent series [

Two distinct processes of incomplete time series prediction.

Traditional process

Our process

The issue of modeling incomplete time series is interpreted as classification of data with missing features in this paper. We use the optimal hyperplane of the classification to determine the prediction values. A similar approach has been applied to prediction of complete data [

Compared with traditional imputation methods, different samples can be selected to calculate the absent data in the history series using our model, which ensures the imputing accuracy of missing data.

The rest of this paper is organized as follows. Section

We start by formalizing the problem of incomplete time series. Assume that a time series with missing data is given as

Predicting technologies usually establish regression models by

The implementation process of our model starts by dividing the sample set

We construct two classes of incomplete data

This model can also be used to predict the missing data of incomplete time series. The imputing samples, taken from the sample set, also fall on the hyperplane. Therefore the missing values can be estimated in the same way as the prediction values. The implementation process of our model is shown in Figure

The implementation process of our model.

In the process of imputation, each missing data can be estimated by all the samples containing it in

In the previous discussion, the issue of modeling incomplete time series is interpreted as classification of data with missing features. In this section, we review the theory of max-margin classification of data with missing features proposed by Chechik [

Assume a set of samples

The problem of classification can be interpreted as to find an optimal hyperplane with the max-margin framework. In the case of classification of incomplete data, the instance margin treating the margin of each instance in its own relevant subspace is defined as

In our model, the hyperplane of classification of data with missing data is used to compute the estimation values. Both predicting samples and imputing samples satisfy (

Suppose a test sample

The simplification of (

Sometimes, analytical solution is meaningless or nonexistent. We need to get numerical solution of our model by iterative algorithms [

The iterative equation of

Therefore, the estimation values are calculated by our model effectively. Numerical solution is more complicated, but applicable in every case.

In conclusion, we have introduced the establishment and solution of our model. The key idea is to first identify a hyperplane of classification of data with missing features by incomplete time series. Then, the hyperplane is used to calculate the estimation values in predicting and imputing samples. Figure

The algorithms of our model for prediction and imputation.

Incomplete time series prediction algorithm

Incomplete time series imputation algorithm

To check the validity of our model, four experiments are conducted in this section. Firstly, the prediction performance of our model is evaluated in test A. Given that conventional imputation methods usually perform distinctly when incomplete time series are missing discretely and continuously, we examine the imputation performance of our model in two missing modes in test B and test C, respectively. The performance of our model is compared with that of RGGB and other two classical imputation methods: Mean and KNN. Finally, we verify the prediction performance of incomplete time series imputed by different models in test D.

The time series used in the experiments are Mackey-Glass time series and Henon time series. Mackey-Glass time series is generated by the chaotic equation

By contrast, Henon time series has a higher volatility. The dimension of the sample set

MSE (Mean Squared Error) and MAE (Mean Absolute Error MAE) are used to evaluate the performance of the experiments. All the results are obtained by repeating the algorithms 10 times.

In this test, continuous 115 data of Mackey-Glass time series with the missing level from 3% to 18% are used to construct the initial sample set,and the next 65 data are for testing the prediction performance of our model. The prediction results are shown in Figure

Prediction results of our model in Mackey-Glass time series.

From Figure

Prediction performance of our model in Mackey-Glass time series.

Prediction results of our model in Henon time series.

Prediction performance of our model in Henon time series.

The continuous 115 data of Mackey-Glass time series and Henon time series with discrete missing data are used as the experimental data in this test. The imputation results of different models are shown in Tables

Imputation results of Mackey-Glass time series with discrete missing data.

Missing level | Our | Mean | KNN | RGGB | ||||

MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |

4–6% | 0.0716 | 0.0626 | 0.0628 | 0.0545 | 0.0728 | 0.0602 | 0.0692 | 0.0594 |

7–9% | 0.0783 | 0.0744 | 0.0809 | 0.0657 | 0.0823 | 0.0612 | 0.0840 | 0.0771 |

10–12% | 0.0912 | 0.0865 | 0.0969 | 0.0833 | 0.0795 | 0.0669 | 0.0979 | 0.0857 |

13–15% | 0.1064 | 0.0933 | 0.1010 | 0.0836 | 0.1086 | 0.1050 | 0.1242 | 0.1098 |

Imputation results of Henon time series with discrete missing data.

Missing level | Our | Mean | KNN | RGGB | ||||

MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |

4–6% | 0.2385 | 0.1999 | 0.8352 | 0.6791 | 0.4624 | 0.4602 | 0.6683 | 0.5592 |

7–9% | 0.4223 | 0.3120 | 0.9954 | 0.8148 | 0.6707 | 0.5534 | 0.7888 | 0.6156 |

10–12% | 0.6461 | 0.4422 | 1.3951 | 1.2638 | 0.6746 | 0.6193 | 0.8321 | 0.6378 |

13–15% | 0.6787 | 0.5261 | 1.4655 | 1.3114 | 0.7868 | 0.6467 | 1.2035 | 0.7639 |

From Tables

Imputation performance of our model in Henon time series with the missing level of 10%.

Figure

We evaluate the performance of different imputation methods by incomplete time series with continuous missing data in the same way. Set the maximum length of continuous missing data

Imputation results of Mackey-Glass time series with continuous missing data.

Missing level | Our | Mean | KNN | RGGB | ||||

MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |

4–6% | 0.0741 | 0.0604 | 0.1044 | 0.0941 | 0.0846 | 0.0676 | 0.1296 | 0.1013 |

7–9% | 0.0831 | 0.0691 | 0.1718 | 0.1374 | 0.1299 | 0.1081 | 0.1531 | 0.1320 |

10–12% | 0.0832 | 0.0748 | 0.1791 | 0.1508 | 0.1091 | 0.0853 | 0.2020 | 0.1553 |

13–15% | 0.1082 | 0.0814 | 0.1951 | 0.1526 | 0.1347 | 0.1119 | 0.2220 | 0.1843 |

Imputation results of Henon time series with continuous missing data.

Missing level | Our | Mean | KNN | RGGB | ||||

MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |

4–6% | 0.2409 | 0.1966 | 0.9633 | 0.8316 | 0.5449 | 0.4247 | 0.7294 | 0.5216 |

7–9% | 0.3945 | 0.3391 | 1.0269 | 0.8957 | 0.5450 | 0.5022 | 0.8275 | 0.5777 |

10–12% | 0.5472 | 0.4904 | 1.1343 | 0.9802 | 0.6777 | 0.5297 | 1.1462 | 0.7090 |

13–15% | 0.8340 | 0.6190 | 1.3937 | 1.3101 | 0.7895 | 0.6711 | 1.2137 | 0.7657 |

Tables

Imputation performance of our model in Mackey-Glass time series with the missing level of 10%.

There are three sets of continuous missing data in Figure

The prediction performance of incomplete time series imputed by different models in test B and test C is evaluated in this test. We also use the next 65 data to test the prediction performance. The error-tolerant BP algorithm is used to build the predicting model. The prediction results are shown in Figures

Prediction results in Mackey-Glass time series imputed by different methods.

Prediction results in Henon time series imputed by different methods

From Figures

Learning and prediction of incomplete data are still pervasive problems, although extensive studies have been conducted to improve the efficiency of data acquisition and transmission [

This work was supported by the National Natural Science Foundation of China under the project Grants nos. 60573125, 90820306, and 60873264. The authors would like to thank the anonymous reviewers in MPE for helpful suggestions and corrections.