Short-Term Load Forecasting Based on Wavelet Transform and Least Squares Support Vector Machine Optimized by Fruit Fly Optimization Algorithm

. Electric power is a kind of unstorable energy concerning the national welfare and the people’s livelihood, the stability of which is attracting more and more attention. Because the short-term power load is always interfered by various external factors with the characteristics like high volatility and instability, a single model is not suitable for short-term load forecasting due to low accuracy. In order to solve this problem, this paper proposes a new model based on wavelet transform and the least squares support vector machine (LSSVM) which is optimized by fruit fly algorithm (FOA) for short-term load forecasting. Wavelet transform is used to remove error points and enhance the stability of the data. Fruit fly algorithm is applied to optimize the parameters of LSSVM,avoidingtherandomnessandinaccuracytoparameterssetting.Theresultofimplementationofshort-termloadforecasting demonstratesthatthehybridmodelcanbeusedintheshort-termforecastingofthepowersystem.


Introduction
Power load forecasting is an important part of management modernization of electric power systems, which has attracted more and more attentions from the academic circle and the practice.Power load forecast with high precision can ease the contradiction between power supply and demand, providing a solid foundation for the stability and reliability of the power grid.It can avoid the waste of resources in the process of grid scheduling and improve the economic benefit.Thus, improving load forecasting methods and the accuracy of prediction constantly is of great important significance to formulate the economic and better power generation plan, reduce spinning reserve capacity, analyse power market demand, and so forth.However, the power load will be influenced by many factors, so features like irregularity and linear independence do exist, which result in the difficulty in making accurate predictions to the power load.
At present, the methods for load forecasting can be divided into two parts: classical mathematical statistical methods and approaches based on artificial intelligence.Most load forecasting theories are based on time series analysis and autoregression models, including vector autoregression model (VAR) and autoregressive moving average model (ARMA) [1][2][3][4][5].Time series smoothness prediction methods are criticized by researchers for their weakness of nonlinear fitting capability.With the development of the electricity market, the requirement of high accuracy load forecasting is more strict and efficient.So artificial intelligence, which includes neural network [6][7][8], grey model [9,10], and support vector machine [11], gains more and more attention from scholars.Due to the strong self-learning, self-adapting ability, and nonlinear change character, artificial neural network is widely used in prediction field.Kandil et al. [12] applied artificial neural network (ANN) for short-term load forecasting without the use of load history and only temperature was used, and it was proved to be effective.After that, BPNN [13] and GRNN [14] are used in improving ANN model for excellent predictive abilities.The shortcomings of neural networks are the requirement of a large amount of data, a long time solving which may cause easily falling into local optimum.
Support vector machine (SVM) is a new statistical learning method [15].Compared with other machine learning methods, SVM implement the structural risk minimization principle to minimize an upper bound on the generalization error, rather than employing the empirical risk minimization principle to minimize the training error, and it gives SVMs better generative performance.As an extension of SVM, least squares support vector machine (LSSVM) transforms the second optimal inequality constraints problem in original space into equality constraints' linear system in feature space through nonlinear mapping [16], which improved the speed and accuracy of convergence.Different parameters selection in LSSVM (Kernel parameter and penalty factor) would have big influences on the fitting accuracy and generalization ability; inappropriate parameter selection may lead to the limitation of the performance of LSSVM.However, it is possible to employ an optimization algorithm to obtain an appropriate parameter combination.Particle swarm optimization model [17], genetic algorithm model [18], and Firefly Algorithm [19] model are all proposed in parameter optimization for LSSVM; for example, particle swarm optimization finds overall optimal value by following the currently found optimal values.But premature convergence and easily falling into local optimum restrict the application of the particle swarm optimization.Thus, this paper puts forward fruit fly optimization model to optimize the parameters of LSSVM.Fruit fly optimization algorithm was proposed by Pro.Pan [20] in Taiwan in 2012.The FOA has the advantages of being easy to understand due to the shorter program code compared with other optimization algorithms and reaching a better global optimal solution.Li et al. [21] applied the FOA to optimize the parameters of GRNN in order to forecast the annual power load.Zheng et al. [22] proposed a novel fruit fly algorithm for the semiconductor final testing scheduling problem.Pan et al. [23] developed an improved FOA algorithm for continuous function optimization problems.
The wavelet transform (WT) is a recently developed mathematical tool for signal analysis.It has been applied successfully in astronomy, data compression, signal and image processing, earthquake prediction, and so on [12].The combination of WT and LSSVM is widely used in forecasting fields [24,25].This paper proposes WT to reprocess the data and improve its reliability.In order to enhance the accuracy of load forecasting, WT-FOA-LSSVM is put up with, and the examples demonstrate the effectiveness of the model.
The rest of the paper is organized as follows: Section 2 provides some basic theoretical aspects of WT, LSSVM, and FOA and gives a brief description about WT-FOA-LSSVM model; in Section 3, an experiment study is put forward to prove the efficiency of the proposed model; Section 4 is the conclusion of this paper.components which are related to the small-scale space in the signal.Figure 1 is a wavelet decomposition tree showing the decomposition process.

WT-FOA-LSSVM Model
In order to overcome the high redundancy situation which exists in continuous wavelet transform, it captures both frequency and location information in temporal resolution.We propose the discrete wavelet transform (DWT) in this paper, and it is defined as follows: where the scale factor  = 1/2  ,  ∈ , and the displacement factor  = /2  ,  ∈ .In general,  , () represents the discrete transform.Using DWT in the signal (), the formula can be shown as In this paper, the original load signal is proposed to be decomposed into an approximation component and some detail components.The approximation presents the main fluctuation of the load and the details to contain the spikes and stochastic volatilities.A suitable number of levels can be decided by comparing the similarity between the approximation and the original signal.

Least Squares Support Vector Machine.
LSSVM is an extension of the standard support vector machine (SVM), proposed by Suykens and Vandewalle [26].It transforms the inequality constraints of traditional SVM into equality constraints and considers sum squares error loss function as the loss experience of the training set, which transforms solving quadratic programming problems into solving linear equations problems [27].The training set is set as {(  ,   ) |  = 1, 2, . . ., }, in which   ∈   is the input data and   ∈   is the output data.(⋅) is the nonlinear mapping function which transfers the samples into a much higher dimensional feature space (  ).Establish the optimal decision function in the high-dimensional feature space: where () is mapping function;  is weight vector;  is constant.
Using the principle of structural risk minimization, the objective optimization function is as follows: Its constraint condition is in which  is the penalty coefficient and   is slack variable.Define the Lagrange function to solve the problem: where Lagrange multiplier   ∈ R. According to the Karush-Kuhn-Tucker (KKT) conditions, , ,   ,   are taken as partial derivatives and required as zero.Consider According to (7), the optimization problem can be transformed into solving linear problem, which is shown as follows: Solve formula (8) to get  and ; then the LSSVM optimal linear regression function is According to Mercer condition, (,   ) = ()  ⋅(  ) is kernel function.In this paper, set radial basis function (RBF) as kernel function which is shown in the following equation: where  2 is the width of kernel function.

Fruit fly group
Fruit fly 1 Fruit fly 2 Fruit fly 3 (X, Y) Figure 2: Food finding iterative process of a fruit fly swarm.
From the problems of training LSSVM, kernel parameter  2 and penalty parameter  are generally set based on experience, which leads to the existence of randomness and inaccuracy in the application of the LSSVM algorithm.To solve the problem, the paper uses fruit fly optimization algorithm to optimize these two parameters to improve the prediction accuracy of LSSVM.

Fruit Fly Optimization Algorithm.
Fruit fly optimization algorithm is a kind of intelligent optimization algorithms based on fruit fly foraging behaviours proposed by Pan [20] in 2012.The basic concept of FOA is that fruit fly perceives food concentration according to its position, and then it will move to the site of maximum or minimum concentration by comparing flavor concentration; finally the objective function extreme value can be obtained through repeated iterations of food concentration.Food finding iterative process of fruit fly swarm is shown in Figure 2.
According to the food finding characteristics of fruit fly swarm, the fruit fly optimization algorithm can be divided into following steps: (1) Randomly initialize the fruit fly swarm location ( axis,  axis).
(2) Give the random flight direction and the distance for food finding of an individual fruit fly by using olfactory: (3) Calculate the distance between the origin and each individual fruit fly position (Dist), and then calculate the value of flavor concentration (); it is the reciprocal of distance: (4) Put the value of flavor concentration  into its fitness function, and then get the flavor concentration of the individual fruit fly location (Smell).
(5) Find out the individual fruit fly with minimal smell concentration among the fruit fly swarm: (6) Retain the best flavor concentration and its ,  coordinates, and then the fruit flies fly to the position by using vision.Enter iterative optimization to repeat steps (2)- (5).When the fitness value reaches target set, or the iterative number reaches the maximal iterative number, the circulation stops.Update the information as follows: Smell best = best Smell  axis =  (best index)  axis =  (best index) .(3) LSSVM Optimized by Fruit Fly Optimization Algorithm.Calculate the distance   between the origin and each individual fruit fly  according to formulas (11) and (12) and the smell concentration judgment value   according to formula (13).In the FOA-LSSVM program, the parameters [,  2 ] of LSSVM model are represented by [(, 1), (, 2)], and we set  = 20 * (, 1) and  2 = (, 2) for LSSVM model training.According to the electric load forecasting result, the value of fitness function can be calculated.In this paper, we employ the mean absolute percentage error (MAPE) as the fitness function, and the formula is as follows: where   represents the actual value at period ; ŷ is the forecasting value at period  and  is the number of forecasting periods.
If the maximum iterative number  max = 100 or MAPE < 0.01%, stop the iterative process and output the best values of  and  2 .
(4) Forecast Using Least Squares Support Vector Machine.Put the optimal parameter value obtained from step (3) in the least squares support vector machine and do the forecast.Finally, get the forecasting load value.From Figure 4, it is clear that the major fluctuation of A1 shows high similarity to the original load data S.The detail component D1 is excluded from the original data to ensure the stability of the input data.So, A1 is selected as the input in proposed model.

Selection of Input.
Human activities are always disturbed by many external factors and then the power load is affected.So, some effective features are considered as input features.In this paper, the input features are discussed as follows.The first feature is the highest temperature and the lowest temperature.Temperature is one of these effective features.In [27][28][29], temperature was considered as an essential input feature and the forecasting results were accurate enough.So, the highest and lowest temperatures are taken into consideration.The second feature is weather conditions.The weather conditions are divided into four types: sunny, cloudy, overcast, and rainy.For different weather conditions, we set different weights: {sunny, cloudy, overcast, and rainy} = {0.9,0.7, 0.5, 0.2}.The third feature is days type.For different days type, the electric power consumption is different.Figure 5 shows the load data from June 10, 2013, to June 16, 2013, among which Wednesday is dragon festival in China.From Figure 5, we can see that the mean power load of Wednesday is higher than other days and different days type has different curve features.So, we assign values to days type in Table 1.

Parameters Setting of Comparison Models.
In this paper, we introduce five other models, WT-LSSVM, least squares support vector machine optimized by fruit fly optimization algorithm (FOA-LSSVM), least squares support vector machine optimized by particle swarm optimization algorithm (PSO-LSSVM), least squares support vector machine, and the BP neural network, to make a comparison with the proposed model.Referring to some of the relative literature [21,27], the parameters of the comparison models are set as shown in Table 2.

Model Performance Evaluation.
To examine the performance of model, the relative error (RE), the mean absolute percentage error (MAPE), the mean square error (MSE), and the mean absolute error (MAE) are proposed to measure the forecast accuracy.The formulas are as follows:  where   represents the actual value at period ; ŷ is the forecasting value at period  and  is the number of forecasting periods.

Analysis of Forecasting Results
. The program in this paper is run in MATLAB R2011b under the XP system.Table 3 shows the short-term electric load forecasting results of the WT-FOA-LSSVM, WT-LSSVM, FOA-LSSVM, PSO-LSSVM, LSSVM, and BPNN models.Figures 6, 7, and 8 present the comparisons of the forecasting results between the proposed model and the others.Figures 9,10,and 11 show the comparisons of relative errors between the proposed model and the others.The relative error ranges [−3%, 3%] and [−1%, 1%] are always considered as a standard to assess  a result, the MAPE, MSE, and MAE of the WT-FOA-LSSVM are all smaller than those of the WT-LSSVM, so we can conclude that the parameter optimization to LSSVM is essential in the forecasting model.Besides, the MAPE, MSE, and MAE of the WT-LSSVM are all smaller than those of FOA-LSSVM, PSO-LSSVM, LSSVM, and BPNN, indicating the preprocessing of load data is useful for a better performance and higher forecasting accuracy.At the same time, the MAPE, MSE, and MAE of the FOA-LSSVM are all smaller than those of PSO-LSSVM, LSSVM, and BPNN, and it is presented that the optimization result of the fruit fly optimization algorithm is efficient.So, we can conclude that the stability and forecasting accuracy of the proposed model is better than the comparison models, and it is worth of being widely used in the short-term load forecasting.

Conclusion
To strengthen the stability and economy of the grid and avoid the waste in grid scheduling, it is essential to improve the forecasting accuracy.Because the short-term power load is always interfered by various external factors with the characteristics like high volatility and instability, the high accuracy of load forecasting should be taken into consideration.Based on the features of load data and the randomness of the LSSVM parameters setting, we propose the model based on wavelet transform and least squares support vector machine optimized by fruit fly optimization algorithm.To validate the proposed model, four other comparison models (FOA-LSSVM, PSO-LSSVM, LSSVM, and BPNN) are employed to compare the forecasting results.Example computation results show that the relative errors of WT-FOA-LSSVM model are all in the range [−3%, 3%], and the MAPE, MSE, and MAE are all smaller than the others.In addition, the fruit fly optimization algorithm is easy to understand and operate, so it is applied widely in parameters optimization.The hybrid model can be effectively used in the short-term load forecasting on power system.

4 .
The Introduction of WT-FOA-LSSVM.Flowchart of the WT-FOA-LSSVM model is shown in Figure3, and the detailed processes are as follows.(1)Data Preprocessing Using DWT.Decompose the load signal into the approximation A1 and the details D1, and select A1 as the training data and testing data.

3. 1 .
Data Preprocessing.This paper chooses the 24-hour power load data from May 1, 2013, to July 23, 2013, in Shanxi province for model checking.In this paper, we select 1986 pieces of load data from May 1 to July 22 as training set and 24 pieces of load data of July 23 as testing set.In order to eliminate the effects of random fluctuations of load data, we decompose the original load data S into the approximation component A1 and detail component D1 through one-level DWT, as shown in Figure 4.

Figure 4 :Figure 5 :
Figure 4: Original load signal and its approximation component and detail component decomposed by DWT.

Table 1 :
The weights of days type.

Table 2 :
Parameters of models mentioned in the paper.

Table 3 :
Actual load and forecasting results.