A Hybrid Model of EMD and PSO-SVR for Short-Term Load Forecasting in Residential Quarters

Short-term load forecasting plays a vital role in the daily operational management of power utility. To improve the forecasting accuracy, this paper proposes a hybrid EMD-PSO-SVR forecastingmodel for short-term load forecasting based on empirical mode decomposition (EMD), support vector regression (SVR), and particle swarm optimization (PSO), also considering the effects of temperature, weekends, and holidays. EMD is used to decompose the residential electric load data into a number of intrinsic mode function (IMF) components and one residue; then SVR is constructed to forecast these IMFs and residual value individually. In order to gain optimization parameters of SVR, PSO is implemented to automatically perform the parameter selection in SVR modeling. Then all of these forecasting values are reconstructed to produce the final forecasting result for residential electric load data. Compared with the results from the EMD-SVRmodel, traditional SVRmodel, and PSO-SVRmodel, the result indicates that the proposed EMD-PSO-SVR model performs more effectively and more stably in forecasting the residential short-term load.


Introduction
Considering that electricity cannot be stored, thus the accurate forecasting of electric load has a significant effect on reliability of power systems and the economic development of society.Particularly the short-term load forecasting plays a vital role in the daily operational management of power utility, such as energy transfer scheduling, unit commitment, and load dispatch [1,2].As an integral part of the daily operational management of power utility, accuracy prediction of the short-term electric load in residential quarters is of great significance to urban power network planning and the electric power market operating.The overestimation will raise the operating cost while underestimation will lead to electricity shortage [3].Because of the limited capacities on the user side, the load characteristics have less smoothing effect.In addition, the residential load is influenced by many factors, such as holidays, weekends, and temperature [4,5].All of these factors lead to the load of residential quarters with more variability, higher randomness, and lower similarities in history load curves.Thus, the short-term load forecasting for residential quarters is a complex task and worthy of further studies.
In the literature, many researchers have proposed various methodologies to improve the short-term load forecasting accuracy during the past decades.These methods can be classified into two categories, namely, the classical statistical methods and artificial intelligence (AI) based algorithms.The classical statistical methods mainly include multiple regression [6], autoregressive integrated moving average (ARIMA) [7] or autoregressive moving average (ARMA) [8], and Kalman filter techniques [9].Though these methods can provide some valuable improvements in terms of forecasting accuracy, most of these models are linear predictors, which have difficulties in forecasting the hard nonlinear behavior of electricity load series.To overcome the limitations of the linear models and account for the nonlinear patterns existing in real cases, AI techniques, such as artificial neural network (ANN) [10], fuzzy logic [11], and echo state network (ESN) [12], have been introduced into load forecasting.One of the most well-known models in the category of AIs is ANN, which is an effective way to deal with the complex nonlinear 2 Mathematical Problems in Engineering problem.However, the problems of ANN in overlearning, connection weight estimations, and the need of a large number of data pieces for system training make it difficult to apply for some short-term prediction.Considering support vector regression (SVR) employs the structural risk minimization principle to minimize an upper bound of the generalization errors, rather than minimizing the training errors used by ANNs, SVR is shown to be very resistant to the overlearning problem [13][14][15] and has been widely and successfully applied to many forecasting problems, particularly in terms of electric load forecasting.For example, Wang et al. [16] proposed a SVR model with the differential evolution (DE) algorithm to choose the appropriate parameters for the annual electric load forecasting of Beijing in China.Ceperic et al. [17] presented a generic strategy for short-term load forecasting based on SVR.Since the accuracy and stability of SVR largely depend on the selection of the kernel parameter and the penalty parameter, the parameter selection of SVR becomes a key problem.For now, the grid algorithm and the genetic algorithm are widely used in optimization methods, while the grid algorithm is time consuming and has poor performance.The genetic algorithm is less time consuming but has a troublesome operation [18].Compared with both the grid algorithm and the genetic algorithm, particle swarm optimization (PSO) has some significant advantages, such as simple operation, good optimization performance, and quick convergence, and it does not require evolution operators such as crossover and mutation.
However, several previous studies [19,20] using PSO-SVR model for short-term load forecasting usually directly applied the original electric load data to construct forecasting models.Because of the nonlinearity and nonstationarity in residential load data, it is difficult to describe the moving tendency of electric load and to improve the forecast accuracy.To establish a suitable and effective forecasting model, the original data features of the residential shortterm electric load need to be fully considered and analyzed.Despite the wavelet transform (WT) becoming a de facto standard for the analysis of nonlinear and nonstationary signals, the time consuming for computing and failure in achieving fine resolutions in both time domain and frequency domain simultaneously present a major barrier for the analysis of short-term load time series.Therefore, empirical mode decomposition (EMD), as a special adaptive and direct data processing method developed especially for dealing with nonlinear and nonstationary data, is used to decompose the original load data into a set of intrinsic mode function (IMF) components and one residue, which can improve the accuracy of forecasting [21][22][23].As EMD can decompose data into a number of independent components, some researchers developed several kinds of hybrid forecasting methods by combining EMD with forecasting models to achieve better performance in various fields of signal processing, short-term electric loading, and traffic engineering [24].For example, Wang et al. [25] proposed a novel model of EMD and Elman neural network to forecast wind speed and concluded that the proposed model was suitable for wind speed prediction.Chen et al. [26] proposed an EMD approach combined with an artificial neural network for tourism demand forecasting.
Zhu et al. [27] presented a hybrid method based on EMD and SVM for short-term electronic load forecasting.To date, short-term load forecasting in residential quarters still remains insufficiently researched in the literature.
Considering the nonlinear and nonstationary characteristics of residential electric load data and the divide and conquer principle, a novel hybrid model based on the EMD and the PSO-SVR is proposed to forecast short-term load in residential quarters in this paper.EMD is used to decompose the electric load data into a number of intrinsic mode function (IMF) components and one residue; then SVR is constructed to forecast these IMFs and residual value individually.In order to gain optimization parameters of SVR, PSO is implemented to automatically perform the parameter selection in SVR modeling.Then all of these forecasting values are reconstructed to produce the final forecasting result for residential electric load data.The proposed model is compared with the single SVR, the hybrid models of EMD-SVR and PSO-SVR, and it is shown that the EMD-PSO-SVR model performs more effectively and more stably and yields more accurate results.
The rest of this paper is organized as follows: Section 2 describes EMD, SVR, and PSO methodologies.Section 3 presents the detailed modeling steps of the EMD-PSO-SVR model.Section 4 presents the numerical results from real data and compares the performance of the proposed hybrid model with other alternative models.Section 5 draws the conclusion of this article.

Empirical Mode Decomposition. EMD, first proposed by
Huang et al. [28], is a novel empirical analysis tool used for processing nonlinear and nonstationary datasets.The main idea of EMD is to decompose the nonlinear and nonstationary time series into a sum of several simple intrinsic mode function (IMF) components and one residue with individual intrinsic time scale properties.Each IMF represents a kind of natural oscillatory mode and has to satisfy the following two conditions: (1) in the whole data series, the number of extreme values and the number of the zero crossings must either be equal or differ at the most by one; (2) at any point, the average of the envelope constructed by the local minimum values and maximum values is zero [29][30][31][32][33].
Let () be a given original short-term load time series; then the detailed steps of EMD calculation can be described as follows [34].
Step 1. Determine all the local extremes of the original electrical load series ().
Step 2. Calculate the upper envelope  up (), which can be derived by connecting all the local maxima using a cubic spline line.Similarly, the lower envelope  low () also can be obtained.And then the average envelope () based on the upper and lower envelopes can be calculated as Step 3. Calculate the first difference  1 () between the original series data () and the mean envelope (): Step 4. Check whether  1 () satisfies the two requirements of IMF.If  1 () is an IMF, then  1 () is denoted as the first IMF  1 () and replaced () with the residue  1 (): Otherwise, if  1 () is not an IMF, replace () with  1 () and repeat Steps 2-3 until the termination criterion is satisfied.For convenience, we use the standard deviation (SD) as the termination criterion and it is defined as where  is the length of the signal;  represents the number of iterative calculations and  is the terminated parameter, which is usually determined according to the actual application.According to [2],  is assumed in the range of 0.2 to 0.3 in this paper.
After the EMD calculation, the original time series data () can be decomposed into summing up all the IMF components and a residue as follows: where   () ( = 1, 2, . . ., ) is the IMF in different decomposition and   () is the residue after  numbers of IMFs are derived.By this process, each IMF is independent and specific for expressing the local characteristics of the original time series data.

Support Vector Regression.
SVR, an extension of support vector machine (SVM), is a machine learning method based on statistics, which was proposed by Drucker et al. [35].
The basic idea of the method can be described as follows.
Assuming that a set of data {(  ,   ),  = 1, 2, . . ., }, where   ∈   is a -dimensional input vector and   ∈  is a corresponding target output.Define a nonlinear mapping function to map the input data into the high dimensional feature space.Then in the high dimensional feature space, there theoretically exists a linear function  to formulate the nonlinear relationship between input and output data.Such a linear function, namely, SVR function, can be defined as where  and  are the coefficients that can be adjusted.The empirical risk is defined as follows: where ∑  =1   (  , (  )) is called  insensitive loss function, it is utilized to obtain an optimum hyperplane which divides the training data into two linearly separable subsets with maximum separation distance.In fact, SVR is an optimizing problem in whose objective function is: The first term of ( 8) measures the flatness of the function.The second term penalizes training errors of () and  by using the  insensitive loss function. specifies the trade-off between the empirical risk and the model flatness.Training errors above  are denoted as  *  while below - are denoted as   .After solving the optimization problem the parameters vector of ( 6) can be found as in the following: where   and  *  are the Lagrangian multipliers, which are obtained by solving a quadratic program.Finally, the SVR regression function is obtained: where (  ,   ) is called Kernel function and must satisfy Mercer's theorem (Vapnik, 1995 [13]).In the feature space, the kernel function can be described as (  ,   ) = (  ) ⋅ (  ).
Among the three typical kernel functions, that is, Gaussian radial basis function (RBF), polynomial function, and linear function, the Gaussian RBF is the most widely used kernel function.It can be defined as (  ,   ) = exp(−‖  −   ‖2 2 ), where  denotes the width of the RBF.Considering that the RBF kernel is not only easy to implement, but also a proper tool for dealing with nonlinear problems, we thus employ the RBF kernel in this study.

Particle Swarm Optimization. PSO is a population-based stochastic optimization algorithm proposed Kennedy and
Eberhart [36].Inspiration for optimization procedure is found in social behaviors of bird groups and fish schools.In a PSO system, each particle is a potential solution in searching hyperspace with a randomized velocity for the optimal position to land.When the search is complete, the best position of the system can be found by adjusting the direction of each particle towards its own best location and towards the best particle of the swarm at each generation.
In this study, PSO is used to optimize the parameters in the SVR model.The detailed description is listed as follows.

End
Evaluating the EMD-PSO-SVR hybrid model Step 1 (data preparation).Input original data for forecasting.
Step 2 (particle initialization).Initialize a population of particles with random positions and velocities in the hyperspace.
Step 3. It is to train the SVR model with training samples data and to evaluate each particle fitness value of PSO for the SVR.
Step 4. The velocity and position of each particle are updated until the stopping condition is satisfied.
Step 5. Build and retrain the SVR forecasting model based on the optimal parameters.

The Proposed EMD-PSO-SVR Model
As mentioned above, the residential electric load data series has the typical characteristics of higher nonlinearity and nonstationary, which leads to insuperable difficulties in the load forecasts.For this reason, the proposed EMD-PSO-SVR model is employed according to the principle of decomposition and ensemble.The framework of an EMD-PSO-SVR model is demonstrated in Figure 1.
The detailed calculation steps are illustrated as follows.
Step 1 (data decomposition by EMD).Use EMD to decompose the original load data into a finite set of IMFs and one residue.We assume that the residue C n is also considered to be a IMF.
Step 2 (PSO-SVR forecasting model construction).Use the SVR to build a forecasting model for each obtained IMF and residue; consequently, the corresponding forecasting values for all IMF and residual components are acquired from the forecasting model.PSO is applied in the proposed SVR model to optimize parameter selection.
Step 3 (data reconstruction and evaluating).Aggregate the forecasting results in Step 2 to attain the final forecasts for the original time series.Then compare the performance capability of the EMD-PSO-SVR hybrid model according to several criteria with the traditional SVR model, PSO-SVR model, and EMD-SVR model.
Through these procedures, distinct information scales in the original load data can be determined through EMD.Moreover, each IMF exhibits similar frequency characteristics, which allows the hybrid model to decrease the complexity of the SVR model and further increase the SVR forecasting efficiency and accuracy.

The Original Electric Load Data.
To investigate the performance of the proposed EMD-PSO-SVR model, we select two different representative residential quarters for shortterm forecasting.The load data are collected from State Grid Handan Electric Power Company in China.The actual daily peak load data from May 1, 2014, to October 31, 2014, are used, for a total of 184 data points, as illustrated in Figure 2, with Case I and Case II representing each of the two residential quarters, respectively.These datasets will also be divided into training group and testing group.The first 138 data points (75% of the total dataset) are used as the training group and the remaining 46 data points are used as the testing group, which will be described as in Figure 2.
Figure 2 shows that the short-term load fluctuates severely.From Figure 2, we cannot see any apparent regularity of the load series.Thus, EMD is employed to decrease the nonstationary characteristic of the original dataset, which will simplify the forecasting task into several simple forecasting subtasks.

Results after EMD of the Electric Load Data.
After being decomposed by EMD, the two residential electric load data series are decomposed into five independent IMFs and one residue component, respectively, as illustrated in Figure 3 (Case I and Case II).The decomposition results have simpler frequency components and thus will be easier and more accurate for forecasting.In terms of the divide and conquer concept, the decomposed variables, including each IMF and the residual components, will be used in SVR model construction.

Forecasting Results with EMD-PSO-SVR Model.
The decomposed IMFs components and one residue for each residential quarter from the previous step are used in PSO-SVR model construction.It should be noted that the residential short-term load is influenced by many environmental factors such as the weather and the development of economy.
In fact, weather and day type may be the most important factors in residential short-term load forecasts.Therefore, when constructing the short-term load forecasting model, in addition to taking the IMFs and residue (  ) as the important input variables of PSO-SVR model, the daily maximum temperature (  max ), the daily minimum temperature (  min ), weekends (  ), and holidays (  ) are also considered as the input variables for the PSO-SVR forecasting model.Specifically, the original residential load data are divided into two datasets: the training dataset (138 data points, from May 1 to September 15) and the testing dataset (46 data points, from September 16 to October 31).In the process of SVR model construction, the PSO algorithm is selected to determine the optimal parameters of SVR model.The optimal parameters of each IMF and the residue are selected based on PSO-SVR by cross validation to establish shortterm load forecasting models.The results of the final optimal parameters are presented in Table 1.
At the reconstruction step, each PSO-SVR is applied to forecast the corresponding datasets, and the final prediction of the original electric load data is obtained by combining the forecasting results of all datasets.Figure 4 shows the forecasting results for the original electric load data based on the EMD-PSO-SVR model.

Comparison of Forecasting Results.
To test the forecasting capability of the proposed EMD-PSO-SVR model, four different statistical metrics, namely, the root mean square error (RMSE), the mean absolute error (MAPE), the mean absolute percentage error (MAPE), and the variance of absolute percentage error (VAPE), are selected to evaluate the forecasting accuracy.The definitions of RMSE, MAE MAPE, and VAPE are described as [1,2,24,34] where   and   are the th predicted and actual values, respectively, and  is the total number of predictions.
For the purpose of comparison, the single SVR model, the PSO-SVR model, and the EMD-SVR model are employed.The forecasting results of the proposed EMD-PSO-SVR model, the SVR model, the PSO-SVR model, and the EMD-SVR model for the two selected residential quarters are shown in Figure 5.
It can be seen from Figure 5 that the EMD-PSO-SVR model gives acceptable predictions most of the time for Case I, and the forecasting curve of the EMD-PSO-SVR model fits better than other alternative models.For Case II, similarly results can also be found, which means that the forecasting curve of the proposed model also fits better than others.
The comparison forecasting results in Case I and Case II are summarized in Table 2.The proposed model is compared    The proposed model shows the higher forecasting accuracy in terms of all the evaluation criteria.In view of the model effectiveness and efficiency on the whole, we can conclude that the proposed model is quite competitive against other compared models.In other words, the hybrid model leads to better accuracy and statistical interpretation.
Several observations can be noticed from above discussion.
(1) Through the comparison among all involved models, it can be found that the proposed EMD-PSO-SVR model outperforms other alternative models in terms of prediction performance.
(2) The results that both EMD-PSO-SVR is superior to the PSO-SVR and EMD-SVR is superior to the SVR model illustrate that EMD is useful to improve the performance of forecasting models.
(3) The improved PSO-SVR models are better than the single SVR model.The main reason for this is that PSO has strong global search capability and can improve greatly the accuracy of SVR reliability prediction.
(4) All the hybrid models have higher accuracy than single SVR model.
Overall, the proposed EMD-PSO-SVR model achieves better results than the alternative models consistently in terms of the minimum statistical values of the performance indicators, meaning that the EMD-PSO-SVR model has the best performance.Thus, it can be concluded that the EMD-PSO-SVR is effective and suitable and can enhance the forecasting ability.
Furthermore, to verify the significance of the accuracy of the EMD-PSO-SVR model, the forecasting accuracy comparisons in both cases among four models are conducted by a nonparametric statistical test, namely, a Wilcoxon's signed rank test (details of the Wilcoxon signed-ranks test can be found in Diebold and Mariano [37]), and the test results are shown in Table 3.It can be seen Table 3 that the values of the four measures for the proposed model are significantly different from that of other models.Since the proposed  method yielded the smallest RMSE, MAE, MAPE, and VAPE values in this study, as well as the best scores on the two-tailed Wilcoxon signed-ranks test for those two measures, it can be concluded that the prediction performance of the proposed EMD-PSO-SVR model is significantly superior to that of the other involved models with regard to forecasting electric load in the residential quarters.

Conclusions
Accurate short-term load forecasting in the residential quarter can be very useful and important for the efficient operation and management of power systems.There has been increasing attention given to finding an effective model to address the problem of the residential electric load forecasting in terms of nonlinear and nonstationary characteristics.For this purpose, a hybrid EMD-PSO-SVR model, which combines the empirical mode decomposition, particle swarm optimization, and the support vector regression, is proposed and developed to forecast the residential electric load data.Through empirical comparison of the SVR model and the PSO-SVR model, the proposed EMD-PSO-SVR model is superior to other models on several performance indicators.The results show that the hybrid EMD-PSO-SVR model can be an effective and significant tool for the residential shortterm load forecasting.
In the future study, the EMD-PSO-SVR model can be used for other residential quarters for short-term electric load forecasting, other factors including social factors and residential habits in electricity usage also can be combined in the forecasting model, and other powerful artificial intelligence techniques can also be introduced the hybrid model to further improve the forecasting accuracy.

Figure 1 :
Figure 1: The flowchart of the hybrid EMD-PSO-SVR model.

Figure 4 :
Figure 4: Forecasting results of the original electric load datasets based on the EMD-PSO-SVR model.

Figure 5 :
Figure 5: Comparison of the original data and the forecasting results with different models.

Table 1 :
Results of the final optimal parameters.Figure 2: The daily peak load data from May 1, 2014, to October 31, 2014.

Table 2 :
Summary of results of different forecasting models.