In IaaS (infrastructure as a service) cloud environment, users are provisioned with virtual machines (VMs). To allocate resources for users dynamically and effectively, accurate resource demands predicting is essential. For this purpose, this paper proposes a self-adaptive prediction method using ensemble model and subtractive-fuzzy clustering based fuzzy neural network (ESFCFNN). We analyze the characters of user preferences and demands. Then the architecture of the prediction model is constructed. We adopt some base predictors to compose the ensemble model. Then the structure and learning algorithm of fuzzy neural network is researched. To obtain the number of fuzzy rules and the initial value of the premise and consequent parameters, this paper proposes the fuzzy

In cloud computing [

It stands to reason that resource provisioning in the cloud environment is influenced directly by performance predictions [

The key contributions of this paper are listed as follows:

We make analysis of the characters of user demands and preferences. Corresponding models and solutions are researched.

Self-adaptive cloud resource demands prediction algorithm using fuzzy neural network is proposed. Besides historical data, the base predictors’ output results are adopted by FNN with different weight.

Fuzzy

The learning algorithm of fuzzy neural network is optimized with self-adjusting learning rate and momentum weight, which improves the robustness and the real-time performance.

To evaluate the prediction algorithm, some statistic indexes are introduced to compare with other algorithms, MSE (mean squared error), MAE (mean absolute error), and PRED(

Researches on resource demand prediction are mainly focused on how to save energy [

There are some methods predicting resource demands based on duration time. Reference [

To balance the performance and the system cost, some researches make efforts to maximize the system utility. In [

As workload has an obvious nonlinear feature, many machine-learning algorithms have also been used to support its prediction. Neural network (NN) is introduced as a prediction method [

Fuzzy neural network is a combination of a fuzzy logic system and a neural network. It keeps the merit of each [

According to the literature reviewed above, FNN is used in many areas for predicting and the performance is fine. Unfortunately, in cloud resource demands predicting area, few researches have adopted FNN as predicting method. In this paper, we adopt FNN with self-adjusting learning rate and momentum weight as the core of the prediction system. To improve the performance, we use ensemble model and clustering algorithm to optimize FNN prediction system. Before the data is sent to the predictor, some preparation work needs to be done.

Before prediction of user demands, we firstly analyze the user requests, including the utilization data structure, content, and number of historical resources. By analyzing the historical data, we may draw conclusions about user preference, demand description, and so forth. The short-term versus long-term and the fluctuation period versus flat period prediction are separately treated. Fluctuation-threshold (

The overview of the prediction system.

As depicted in Figure

In user long-term resources requirements, there are some characters, as we summarize in the following:

Based on the characters of flat period, the second moving average (SMA) [

The model of sliding window.

The predicted output value

The

In function (

In formula (

Then the total number of the resources requested by all the

From the analysis above, we can see that the prediction value at the time

Exponential moving method (EMA) is an effective method for short-term prediction and particularly suitable for time series prediction of the nonseasonal effect owing to its quick responsiveness and weight decreases with time passed. Predicted values are calculated using smoothing constant

In (

The EMA method gives a higher weight to the later measure value and lower weight to the earlier measure value. So the EMA method is able to response rather quickly to the fluctuations in a short-term demand and workload conditions [

Though we give different prediction methods according to the wiggle levels, it is difficult to know or identify the boundary of different wiggle levels in the overall situation. In this section, we define “fluctuation-threshold (

The above procedure can be illustrated by Pseudocode

Resource Demands are experiencing a fluctuation period

Resource Demands are experiencing a flat period

Resource demands is intervenient flatness and fluctuation

In order to predict the resource demands accurately and effectively, a prediction method with different individual base prediction models ensemble and fuzzy neural network is proposed in this section. With different base prediction models, different demands occasions can be estimated accordingly, and the most likely future outcome is able to be predicted. With the results of base predictors, the fuzzy neural network tends to present better predicting performance. To improve the learning ability, we use self-adjusting learning rate and momentum weight to optimize the learning procedure. Clustering algorithm is adopted to initialize the fuzzy inference rules of the FNN. The introduction of fuzzy neural network promises the robustness and accuracy of the prediction system.

The core of the prediction model adopts a two-level structure, as shown in Figure

The structure of ensemble model and FNN based prediction model.

As we know, diversity is necessary for the survival and evolution of species ensemble model. So as for the performance of the prediction models, it is important to introduce the diversity to the prediction ensemble model. To guarantee the prediction performance; the base prediction models should be firstly selected. Besides the prediction models mentioned in Section

Autoregression model (ARM) is one of the linear models that are used for estimating the relationships between one dependent output variable

Trend seasonality model (TSM) represents a regularity that repeats periodically, which can be modeled by low order polynomials. To measure how the general cycle affects data value, we calculate a series of periodic indicators. Seasonality indicator demonstrates the offset between certain period average value and the overall average value. To get an accurate estimation of the indicator, each periodic value is calculated and compared with the total average value. The seasonal indicator

By judging whether the monitoring data crosses over the moving average, moving average method (MAM) predicts the future trend.

In fuzzy neural network, we use neural network to evolve the fuzzy inference rules. We consider a multi-input single-output (MISO) fuzzy model which consists of

In the rules,

To give a full description to the FNN without loss of generality, we construct the FNN as shown in Figure

The architecture of FNN.

Layer 1 is known as the input layer. Nodes in this layer transmit the input data to the next layer directly. Input data is abstracted as

Layer 2 is known as the fuzzification layer. In this layer, every node performs the calculation of a Gaussian membership function and specifies the degree to which the given input

Here,

Layer 3 is the fuzzy inference layer. The fuzzified results of the individual scalar functions of each input data are aggregated. All potential rules of the input data are formulated by applying fuzzy intersection, which means product of data. Thus, a product operation denoted as

The output of layer 3 represents the firing strength of the rules.

Layer 4 is the normalization layer. Each node in layer 4 is labeled with

The output of this layer is known as normalized firing strength.

Layer 5 and layer 6 are known as the defuzzification layers. In layer 5, every node in this layer is labeled with

Here

In layer 6, there is only one node labeled with

After the structure of FNN is constructed, we use error back propagation method to adjust the parameters

In the defuzzification layer (layer 5), the evolution of error is expressed as follows:

In layer 4, the differential error of node

In layer 3,

In layer 2,

The update parameters are

In (

In addition, we adopt a “progressive-increase” and “conservative-decrease” method to adjust the learning rate

The variable

In FNN, the construction of fuzzy if-then rules is difficult. The improper rule set may result in bad prediction results. Recently, a number of different approaches have been used for designing fuzzy if-then rules based on clustering, gradient algorithms [

By analyzing the membership degree of sample data, the fuzzy

The optimization objective function of FCM algorithm is

In the above equation,

The membership degree matrix can be calculated by the following equation:

From the discussion above, we can see that FCM is sensitive to isolated data. As the sum of the membership degree has to be 1, the results may be not good if the sample data is not ideal. Besides, the cluster centroids of FCM are initialized stochastically. If the initial value is not properly selected, the convergence may be affected and local convergence may happen. Thus, FCM relies on initial centroids greatly. Moreover, the diversity of membership function may lower the convergence speed.

To improve the FCM algorithm, we introduce the subtractive clustering as a complement. Subtractive clustering is unsupervised clustering, in which the number of clusters for input data points is determined by the clustering algorithm. The subtractive clustering does not need to define the number of the clusters. The results may be used for initializing the centroids of FCM algorithm. It assumes that each data is a potential cluster centroid. Based on the data density index of the potential centroid data, we select the data that has the highest density as the centroid. The procedure is concluded as follows.

Calculating the density index of each data

The clustering radius

Data beyond the radius affects little to the density index. We firstly choose the data

We use the following equation to modify the density index of each data:

s.t.

By calculating the density index of the remaining data, the next centroid is obtained. If the constraint equation (

Here,

This section introduces a clustering algorithm that combines FCM with subtractive clustering method. We obtain the cluster centroids and number through subtractive cluster. This can effectively improve the convergence speed of FCM and the probability of local convergence is decreased.

The procedure can be described as follows.

Set the parameters, including neighborhood radius

Calculate the number and centroids of clusters through subtractive method.

Use (

Verify if the termination constraints are achieved. If

Update

To evaluate the performance of the prediction system, we use a series of metrics [

MAE is the criterion of measuring the mean deviation between the prediction output and the actual output. MAE can be calculated by the following function:

MSE represents the energy of the mean error. MSE can be expressed as in the following:

PRED(

The index PRED

To optimize the performance of the resource demands prediction system, we introduce the feedback control [

In this section, experiments are conducted to validate the proposed prediction method. When we predict the fine-grained resource demands, the method of each kind of resource is similar to others. Here we do not distinguish resource type, and we use network traffic as the representation. From [

The training results and error of ESFCFNN.

The comparison of actual value and prediction output of ESFCFNN

The error curve of ESFCFNN

The test data is used to verify the prediction method. The test results are shown in Figure

The test results and error of ESFCFNN.

The actual output and prediction output of ESFCFNN training procedure

The normalized error curve of ESFCFNN training procedure

The prediction error using different methods is compared with each other and the results are depicted in Figure

Normalized prediction error using different methods.

According to the main criteria we defined from formulas (

Prediction error indexes comparison of different methods.

MAE

MSE

SSE

PRED

Figure

The comparison of the training performance of different learning algorithms.

Training procedure without self-adjusting learning rate

Training procedure of FNN with self-adjusting learning rate and momentum weight

The performance is much better after using self-adjusting learning rate and momentum weight. Moreover, the convergence speed after using self-adjusting learning rate is improved. The training error falls down to 0.05 within 10 steps when using self-adjusting learning rate. However, if the self-adjusting learning rate algorithm is not adopted, the convergence speed is slowed down. More than 85 steps are needed before error falls down to 0.05. From above analysis, we can see that the performance of FNN with self-adjusting learning rate and momentum weight is improved greatly.

In Figure

The performance without clustering.

The convergence procedure

The training error

To improve the performance of resource provision and resource utilization, this paper proposed a cloud resource demands self-adaptive predicting method using ensemble model and subtractive-fuzzy clustering based fuzzy neural network, which is called ESFCFNN for short. We discuss the structure of the prediction system. Users’ preferences are analyzed to reduce the amount of calculation. Then the base prediction model is introduced into the system. The results are sent to FNN with self-adjusting learning rate and momentum weight as the inputs. To optimize the convergence performance of FNN, fuzzy-subtractive clustering algorithm is proposed. Fuzzy-subtractive algorithm is composed with fuzzy

Though the method this paper proposes is promising in improving the performance, the system is complex. As we can see, there are two prediction layers. The time delay may be increased. In future, the improvement of efficiency is the main point of the research. We would also test the method in the real cloud computing system in future.

The authors declare that there is no conflict of interests regarding the publication of this paper.

This work is supported by the Weapon Equipment Pre-research Program under Grant no. 9140A04030213JB34001.