Crude oil is the most important nonrenewable energy resource and the most key element for the world. In contrast to typical forecasts of oil price, this study aims at forecasting the demand of imported crude oil (ICO). This study proposes different single stage and two-stage hybrid stages of forecasting models for prediction of ICO in Taiwan. The single stage forecasting modeling includes multiple linear regression (MLR), support vector regression (SVR), artificial neural networks (ANN), and extreme learning machine (ELM) approaches. While the first step of the two-stage modeling is to select the fewer but more significant explanatory variables, the second step is to generate predictions by using these significant explanatory variables. The proposed two-stage hybrid models consist of integration of different modeling components. Mean absolute percentage error, root mean square error, and mean absolute difference are utilized as the performance measures. Real data set of crude oil in Taiwan for the period of 1993–2010 and twenty-three associated explanatory variables are sampled and investigated. The forecasting results reveal that the proposed two-stage hybrid modeling is able to accurately predict the demand of crude oil in Taiwan.
Natural resources are often classified into two groups: renewable and non-renewable resources. Renewable energy is the one which comes from natural resources such as sunlight, wind, and rain, and it is naturally replenished. It is reported that about 16% of global final energy consumption comes from renewable energy. The share of renewable energy in global electricity generation is around 19% [
The relationship between economic growth and oil consumption was addressed [
The study estimated the short-run and long-run elasticities of demand for crude oil in Turkey for the period of 1980 to 2005 [
In addition to forecasting the oil demand or consumption, several forecasting methods were applied to predict different types of energies. For example, regression modeling was employed to forecast the coal, oil, and gas electricity requirement [
Extreme learning machine (ELM) proposed by Huang et al. [
In June 1946, Chinese Petroleum Corp. (CPC) was funded, and the headquarters was set up in Taipei under the direction of the Ministry of Economic Affairs. With service facilities covering the whole nation, its operations today include the import, exploration, development, refining, transport, marketing, and sale of petroleum and natural gas. CPC’s total capital stands at NT$130.1 billion, and its total revenues in 2011 amounted to NT$1.03 trillion.
Since crude oil is extremely important for development of Taiwan’s economy, the predictions of the demand of imported crude oil are a must. Accordingly, this study is aimed at proposing single and two-stage forecasting techniques to predict the demand of imported crude oil in Taiwan. The single stage forecasting modeling includes the support vector regression (SVR), artificial neural networks (ANN), extreme learning machine (ELM), and multiple linear regression (MLR) approaches. The two-stage models combine the two modeling components. The first component of the model uses its own feature to capture the significant explanatory variables. Then, the second component generates the predictions based on these explanatory variables. In this study, the combinations of MLR and SVR (i.e., refer to MLR-SVR), MLR and ANN (i.e., refer to MLR-ANN), and MLE and ELM (i.e., refer to MLR-ELM) are used as the two-stage models.
Real data are sampled for the period of 1993–2010 for the ICO in Taiwan. According to the suggestion [
The contents of this study are organized as follows. The following section introduces the proposed forecasting techniques. Section
This study considers MLR, SVR, ANN, ELM, and their hybrid modeling schemes as possible forecasting models for import demand of crude oil in Taiwan. These forecasting techniques are introduced in the subsequent sections.
The multiple linear regression analysis is the procedure by which an algebraic equation is formulated to estimate the value of a dependent variable
To identify significant independent variables, the backward elimination, forward selection, or stepwise regression procedures can be applied. The backward elimination procedure begins with the model which includes all of the available explanatory variables, and successively deletes one variable at a time to the model in such a way that at each step, the variable deleted is the variable contributing the least to the prediction of dependent variable at that step. On the contrary, the forward selection procedure begins with the constant model that includes no explanatory variable, and successively adds one variable at a time to the model in such a way that, at each step, the variable added is the variable contributing the most to the prediction at that step. The stepwise regression procedure is an admixture of the backward elimination procedure and the forward selection procedure. This selection procedure builds a sequence of models and at each step deletes or adds an explanatory variable according to some selection criterions such as coefficient of partial correlation or error sum of squares reduction.
While support vector machine (SVM) is one of the most powerful techniques in machine learning areas [
Statistical learning theory and structural risk minimization principle have provided a very effective framework for development of support vector regression [
Typical regression modeling obtains the coefficients through minimizing the square error, which can be considered as empirical risk based on loss function. The
When empirical risk and structure risk are both considered, the SVR can be setup to minimize the following quadratic programming problem:
The general form of the SVR-based regression function is described as follows [
Artificial neural networks, originally derived from neurobiological models, are massively parallel, computer-intensive, and data-driven algorithmic systems composed of a multitude of highly interconnected nodes, known as neurons as well. Mimicking human neurobiological information-processing activities, each elementary node of a neural network is able to receive an input single from external sources or other nodes and the algorithmic procedure equipped in each node is sequentially activated to locally transforming the corresponding input single into an output single to other nodes or environments.
It was indicated that knowledge is not stored within individual processing units, but is represented by the strength between units [
For ANN modeling, the relationship between output
Accordingly, the ANN model in (
ELM randomly selected the input weights and analytically determined the output weights of SLFNs. One may randomly choose and fix the hidden node parameters which are the key principle of the ELM. After randomly choosing the hidden nodes parameters, SLFN becomes a linear system where the output weights of the network can be analytically determined using simple generalized inverse operation of the hidden layer output matrices [
In general, the concept of ELM is similar to that of the random vector functional-link (RVFL) network where the hidden neurons are randomly selected. However, the main difference between ELM and RVFL is the characteristics of hidden neuron parameters. In ELM, all the hidden node parameters are randomly generated independently of the target functions and the training patterns [
Consider
The first step of ELM algorithm is to randomly assign input weight
Recent research indicates that hybrid systems which are integrated with several standard ones can help to achieve a better performance for some applications. For example, the hybrid modeling applications have been reported in forecasting [
In the first stage of hybrid modeling, more significant variables are selected using MLR with forward selection, backward elimination or stepwise regression, say,
To show the effectiveness of the proposed hybrid modeling, the real data, from the years 1993 to 2010, were sampled for the ICO from Bureau of Energy in Taiwan [
Meaning of the influential variables for ICO model building.
Variable | Meaning |
---|---|
|
Imported crude oil |
|
Gross domestic product |
|
Consumer price index |
|
Personal disposable income |
|
Average temperature |
|
Average sunshine per day (hours) |
|
Average electricity households |
|
Average electricity price |
|
National income |
|
Population |
|
Foreign trade total |
|
Wholesale prices |
|
Consumer index |
|
GNP deflators |
|
Foreign exchange reserves |
|
Total primary energy supply |
|
Total final consumption |
|
Total domestic consumption |
|
Energy productivity |
|
Energy intensity |
|
Energy consumption of energy intensive industries |
|
Value-added of energy intensive industries |
|
Dependence on imported energy |
|
Electricity average load |
Figure
Historical yearly data of ICO in Taiwan.
Our numerical results reveal that all the values of VIFs of independent variables are greater than 10 except the variables of
Pearson correlations for pairs of variables.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
1.00 | |||||||||||||||||||||||
|
|
1.00 | ||||||||||||||||||||||
|
|
|
1.00 | |||||||||||||||||||||
|
0.31 | 0.27 | 0.32 | 1.00 | ||||||||||||||||||||
|
0.29 | 0.26 | 0.30 | 0.00 | 1.00 | |||||||||||||||||||
|
|
|
|
0.31 | 0.35 | 1.00 | ||||||||||||||||||
|
0.33 | 0.45 | 0.31 | −0.02 | 0.13 | 0.37 | 1.00 | |||||||||||||||||
|
|
|
|
0.32 | 0.30 |
|
0.30 | 1.00 | ||||||||||||||||
|
|
|
|
0.34 | 0.38 |
|
0.30 |
|
1.00 | |||||||||||||||
|
|
|
|
0.09 | 0.16 |
|
0.43 |
|
|
1.00 | ||||||||||||||
|
|
|
|
0.04 | 0.16 |
|
0.55 |
|
|
|
1.00 | |||||||||||||
|
|
|
|
0.27 | 0.26 |
|
0.45 |
|
|
|
|
1.00 | ||||||||||||
|
0.19 | 0.18 | 0.21 | 0.45 | −0.02 | 0.20 | −0.49 | 0.22 | 0.23 | −0.13 | −0.26 | 0.18 | 1.00 | |||||||||||
|
|
|
|
0.13 | 0.40 |
|
0.57 |
|
|
|
|
|
−0.22 | 1.00 | ||||||||||
|
|
|
|
0.31 | 0.38 |
|
0.25 |
|
|
|
|
|
0.17 |
|
1.00 | |||||||||
|
|
|
|
0.30 | 0.39 |
|
0.27 |
|
|
|
|
|
0.15 |
|
|
1.00 | ||||||||
|
|
|
|
0.30 | 0.38 |
|
0.25 |
|
|
|
|
|
0.16 |
|
|
|
1.00 | |||||||
|
0.11 | 0.24 | 0.09 | −0.24 | −0.31 | 0.11 | 0.75 | 0.08 | 0.00 | 0.37 | 0.54 | 0.24 | −0.53 | 0.30 | −0.02 | −0.01 | −0.02 | 1.00 | ||||||
|
−0.09 | −0.22 | −0.07 | 0.25 | 0.34 | −0.08 | −0.72 | −0.06 | 0.02 | −0.35 | −0.52 | −0.22 | 0.53 | −0.27 | 0.05 | 0.03 | 0.05 |
|
1.00 | |||||
|
|
|
|
0.25 | 0.44 |
|
0.38 |
|
|
|
|
|
−0.01 |
|
|
|
|
0.07 | −0.05 | 1.00 | ||||
|
|
|
|
0.27 | 0.33 |
|
0.34 |
|
|
|
|
|
0.07 |
|
|
|
|
0.12 | −0.10 |
|
1.00 | |||
|
|
|
|
0.27 | 0.38 |
|
0.25 |
|
|
|
|
|
0.20 |
|
|
|
|
−0.01 | 0.04 |
|
|
1.00 | ||
|
|
|
|
0.35 | 0.35 |
|
0.30 |
|
|
|
|
|
0.16 |
|
|
|
|
0.06 | −0.03 |
|
|
|
1.00 | |
|
0.91 | 0.84 | 0.92 | 0.19 | 0.41 | 0.90 | 0.06 | 0.92 | 0.92 | 0.81 | 0.68 | 0.84 | 0.23 | 0.82 | 0.94 | 0.94 | 0.94 | −0.13 | 0.15 | 0.90 | 0.90 | 0.95 | 0.93 | 1.00 |
Collinearity diagnosis for MLR modeling.
Variable |
|
|
|
|
|
|
---|---|---|---|---|---|---|
VIF | 1.52 | 2.04 | 3.91 | 1.99 | 4.36 | 1.49 |
MLR model for ICO in Taiwan.
Variables | Estimated |
Standard |
|
---|---|---|---|
Constant | −11862349.62 | 853356.39 | <0.01** |
Average electricity price ( |
−75695.53 | 28564.49 | 0.02** |
Dependence on imported energy ( |
124776.53 | 8782.77 | <0.01** |
The regression coefficients in the MLR model indicate that the higher the average electricity price is, the lower the imported crude oil is. On the contrary, the higher the dependence on imported energy is, the lower the imported crude oil is.
For ANN modeling, since the backpropagation neural network (BPNN) structure has been widely used [
After applying ANN to ICO data, we have obtained the {23-22-1} topology with a learning rate of 0.01 which provides the best result. The
As mentioned earlier, the most important ELM parameter is the number of hidden nodes and that ELM tends to be unstable in single run forecasting [
A rational strategy for a hybrid modeling is to use the fewer but more informative variables, which were selected by the first stage of modeling approaches, as the inputs for the second stage of classifier approaches. Accordingly, in this study, the significant variables selected, that is, average electricity price (
After completing the first stage of hybrid modeling, the ANN topology settings can be established. This study has found that the {2-3-1} topology with a learning rate of 0.01 provides the best result for the hybrid model. The network topology with the minimum testing RMSE is also considered as the optimal network topology. For the MR/SVR hybrid modeling, the parameters of
In this study, we consider the forecasting accuracy measures of MAPE, MSE, and MAD to address the forecasting performance for the five different approaches, ANN, SVR, ELM, MRSEL,
Various forecasting models’ accuracy measures for ICO.
MAPE | RMSE | MAD | |
---|---|---|---|
Single stage models | |||
ANN | 15.681 | 56190.676 | 52155.891 |
SVR | 22.304 | 77973.147 | 76318.645 |
MRSEL | 11.753 | 48139.849 | 39431.691 |
ELM | 10.174 | 43189.042 | 33271.068 |
Proposed hybrid models | |||
MRSEL-ANN | 7.302 | 33875.635 | 24572.516 |
MRSEL-SVR | 9.385 | 35523.521 | 32465.307 |
MRSEL-ELM | 7.094 | 28902.330 | 23271.608 |
In comparison to the single stage and our proposed hybrid models in Table
Improvement of the proposed models in comparison with the single stage models.
Models | MAPE (%) | RMSE (%) | MAD (%) |
---|---|---|---|
Proposed hybrid |
|||
ANN | 53.43 | 39.71 | 52.89 |
SVR | 67.26 | 56.55 | 67.80 |
MRSEL | 37.87 | 29.63 | 37.68 |
ELM | 28.23 | 21.56 | 26.14 |
Proposed hybrid |
|||
ANN | 40.15 | 36.78 | 37.75 |
SVR | 57.92 | 54.44 | 57.46 |
MRSEL | 20.15 | 26.21 | 17.67 |
ELM | 7.76 | 17.75 | 2.42 |
Proposed hybrid |
|||
ANN | 54.76 | 48.56 | 55.38 |
SVR | 68.19 | 62.93 | 69.51 |
MRSEL | 39.64 | 39.96 | 40.98 |
ELM | 30.27 | 33.08 | 30.05 |
Figure
Plot of actual ICO values for the last four years and the forecasts by using four single-stage models.
Plot of actual ICO values for the last four years and the forecasts by using three hybrid models.
Plot of actual ICO values for the last years and the forecasts by using MR, ANN, and the hybrid MR-ANN model.
Plot of actual ICO values for the last years and the forecasts by using MR, SVR, and the hybrid MR-SVR model.
Plot of actual ICO values for the last years and the forecasts by using MR, ELM, and the hybrid MR-ELM model.
As shown in Figures
Oil is not only used to make gas for cars, but for heating homes, producing electricity, making plastics, and other commodities. Oil and its byproducts are ingrained into almost every culture in the world. Therefore, the accurate prediction of the demand of ICO is very important for the economic development of a country.
Because it is difficult to fully capture the characteristics of the real ICO data, the two hybrid prediction models are then proposed to forecast the demand of ICO in Taiwan. Based on our numerical results, it is found that the proposed hybrid approaches are more accurate than the established single-stage ones. The modeling procedures and results of this work may provide a guidance to develop forecasting models for other energies.
Besides, there are many other two-stage hybrid forecasting models that have been proposed and applied in various fields [
The authors declare that there is no conflict of interests regarding the publication of this paper.
This research was supported in part by the National Science Council of Taiwan, Grants NSC102-2221-E-030-019 and NSC 102-2118-M-030-001.