Dynamic Placement of Virtual Machines with Both Deterministic and Stochastic Demands for Green Cloud Computing

. Cloud computing has come to be a significant commercial infrastructure offering utility-oriented IT services to users worldwide. However,datacentershostingcloudapplicationsconsumehugeamountsofenergy,leadingtohighoperationalcostandgreenhouse gasemission.Therefore,greencloudcomputingsolutionsareneedednotonlytoachievehighlevelserviceperformancebutalso tominimizeenergyconsumption.Thispaperstudiesthedynamicplacementofvirtualmachines(VMs)withdeterministicand stochasticdemands.InordertoensureaquickresponsetoVMrequestsandimprovetheenergyefficiency,atwo-phaseoptimization strategyhasbeenproposed,inwhichVMsaredeployedinruntimeandconsolidatedintoserversperiodically.Basedonanimproved multidimensionalspacepartitionmodel,amodifiedenergyefficientalgorithmwithbalancedresourceutilization(MEAGLE)and alivemigrationalgorithmbasedonthebasicset(LMABBS)are,respectively,developedforeachphase.Experimentalresultshave shownthatunderdifferentVMs’stochasticdemandvariations,MEAGLEguaranteestheavailabilityofstochasticresourceswitha definedprobabilityandreducesthenumberofrequiredserversby2.49%to20.40%comparedwiththebenchmarkalgorithms.Also, thedifferencebetweentheLMABBSsolutionandGurobisolutionisfairlysmall,butLMABBSsignificantlyexcelsincomputational efficiency.


Introduction
Cloud computing is a new paradigm for the dynamic provisioning of computing services supported by state-of-the-art data centers.With the rapid development of cloud computing, the data centers are growing at an unprecedented rate.However, an average data center consumes as much energy as 25,000 households [1] and it was estimated that the energy consumption from data centers would have accounted for about two percent of the worldwide energy consumption by 2020 [2].High power consumption not only produces huge operational cost for data centers, but also significantly contributes to the growing environmental issues of global warming.Therefore, green cloud computing is driven to achieve high level service performance and to minimize energy consumption.
Virtual machines are attractive to modern cloud data centers because they can significantly promote the efficiency and flexibility of resource management helping to cut back the power consumptions [3][4][5].However, such an incentive highly relies on a well-designed VM placement scheme [6,7].When multiple VMs run on the same physical host or rack, they share all the physical resources.According to how resource-intensive applications are, users may customize different types of VMs such as CPU-intensive, memoryintensive, and bandwidth-intensive VMs.If all VMs in a cloud data center have the same intensive resource, VMP is then reduced to the classical one-dimensional VM placement problem.But a cloud data center often has VMs with different resource intensities [8], and, thus, an efficient multidimensional algorithm is required for the VMs placement.
In cloud data centers the VM placement problem (VMP) has been tackled in both research work [7,[9][10][11][12][13][14][15][16][17] and commercial products [18,19].In most of the research work, VMP is formulated as a multidimensional bin packing problem, aiming to minimize the number of required servers and  with constraints from various management fields [4], such as the resource demands of VMs, security/isolation policies (e.g., ensuring that certain VMs will not run on the same server), and compatibility/location policies (e.g., restrict the placement of certain VMs to certain servers).Variants of the classic packing algorithms such as First Fit (FF) and First Fit Decreasing (FFD) [20] are often used to solve VMP.In particular, Li et al. [17] proposed an energy efficient algorithm with balanced resource utilization.However, in these studies, the VMs' demands are assumed to be deterministic for all resources and these algorithms are called deterministic resource algorithms.
Recent studies [11,[21][22][23] indicate that VMs' demands for some resources like network bandwidth are stochastic and highly volatile.Regarding the placement of VMs with stochastic demands, the deterministic resource algorithm can be adopted by using the mean or maximum of the demand as its estimated value.But this naive approach may lead to either poor user experience or resource waste.Wang et al. [24] and Breitgand and Epstein [25] assumed the VMs' bandwidth demand to be normally distributed.They described VMP as a stochastic bin packing problem and used improved bin packing heuristics to solve it.Nevertheless, only the network resources with stochastic demands were considered in their study while system resources with deterministic demands were not included.
In reality, the VMs' demands for all resources have different characteristics.For some resources, such as CPU and memory, VMs' demands are constant or have a very small fluctuation and, thus, can be considered as deterministic.For other resources, such as network bandwidth, VMs' demands are time varying and have high volatility, and, thus, can be considered as stochastic.Yet, little work has focused on the placement of VMs with both deterministic and stochastic demands.Jin et al. [26] considered both kinds of demands and proposed a polynomial time algorithm to solve the multidimensional stochastic VM placement problem.In their work, the VM deployment requests were given beforehand and the VMs were placed in a static manner.Practically, VM requests dynamically arrive at the cloud data center.
Taking into account VMs' deterministic and stochastic demands, this paper studies the VMP problem in a practical dynamic cloud environment.The paper is organized as follows.Section 2 formulates the instant-deployment problem for VMs and the periodic-consolidation problem for servers.Section 3 presents an improved multidimensional space partition model.Section 4 describes the execution procedures of the proposed two-phase optimization strategy, and two developed algorithms, respectively, for VM deployment and server consolidation based on the model in the previous section.In Section 5, the algorithms are validated by simulation experiments.Section 6 concludes the paper with a summary of results and future work directions.

Problem Formulation
Assume that servers are homogeneous in a cloud data center and there are sufficient server resources to meet all the VM requests.For VMs' demands, the server resources fall into two types: deterministic resources and stochastic resources.Denote  the set of deterministic resources and  the set of stochastic resources.Servers should supply fixed-value deterministic resources required by VMs and provide an availability guarantee of stochastic resources with probability at least 1 − , where  is called target overflow probability specified in the service level agreements (SLAs).
Let the time-slots be of equal length Δ.The deployment and removal of VMs only occur at each time instant [27].As shown in Figure 1, VMs, on arrival, will be placed at the next time instant.In order to ensure high responsiveness, Δ needs to be properly chosen.
For a cloud provider, quick response to the incoming VM requests is crucial for a high service quality.Thus, an instantdeployment strategy is adopted for VMs.However, VMs are being continuously removed over time, as shown in Figure 1.Over a long time period, the infrastructure may be brought to a poor state (e.g., there are only a few VMs in some servers).Therefore, it is necessary to implement server consolidation periodically.

The Instant-Deployment for VMs.
A newly arriving VM is denoted by V  .The strategy is to select a suitable physical machine (PM) to deploy V  in runtime.VM migration is not involved.Considering VMs' deterministic and stochastic demands, a placement scheme is regarded as feasible only if it satisfies the following conditions: (1) for each deterministic resource of a server, the server's capacity should not be exceeded by the total demand of all the VMs on the server; (2) for each stochastic resource of a server, the probability that the server's capacity is exceeded is no larger than .
Denote VM V  's demand for a deterministic resource  as    and server ℎ's corresponding capacity as C  ℎ .Condition (1) can be represented as Condition (2) can be represented as ℎ is the set of VMs running on server ℎ.
The VM instant-deployment problem is to find a VM placement scheme for a newly arriving VM V  , which satisfies the above two conditions and minimizes the number of required servers.The most effective way to reduce power consumption is to decrease the total number of required servers.Given the overhead of live migration, the objective is to minimize the number of migrations, in addition to the number of required servers.
In (3),  1 defines the number of required servers after the server consolidation.In (4),  2 defines the number of migrations needed during the server consolidation.Constraints ( 5)- (7) define the implicit relations between the decision variables.Constraint (8) states each VM ( ∈ ()) is placed on a single server.Constraint (9) states that for each deterministic resource of a server, the total demand of all the VMs on the server must not exceed the server's capacity.Constraint (10) states that for each stochastic resource of a server, the demands of all the VMs on the server should be satisfied with the given probability 1−.Constraints ( 11)-( 13) define the domain of the variables.

An Improved Multidimensional Space Partition Model
3.1.The Model for Deterministic Resources.Most existing algorithms (such as Next Fit and First Fit) simply perform the placement computation by comparing the VM's demand with the server's available capacity, without considering the balanced utilization of multidimensional resources.However, as long as an arbitrary dimensional resource of a server is exhausted, even when all other dimensional resources are sufficient, no more VMs can be placed on the server.In order to avoid this and reduce resource waste, Li et al. [17] proposed a multidimensional space partition model and based on the model presented an energy efficient algorithm with balanced resource utilization (EAGLE for short).Figure 2 shows the two-dimensional space partition model.In the figure, the vertical and horizontal axes, respectively, stand for the utilization ratios of resource 1 and resource 2 in server ℎ; the point E(1.0, 1.0) indicates the two types of resources are exhausted, while the point O(0, 0) means server ℎ is idle.
In the partition model, there are three domains: the AD domain which implies all resources are nearly exhausted, the FD domain which indicates there is an obvious disequilibrium of multidimensional resource utilization, and the SD domain which represents the balanced case.As shown in Figure 2, the unit square is partitioned into three parts by three quarter circles.The AD domain is one quarter circle with center E and radius  0 ( 0 ∈ [0, 1.0]).The FD domain is divided into two subparts by two symmetrical quarter circles, with the same radius  0 ( 0 ∈ [0, 1.0]).The point O 1 is the center of the red quarter circle and the point O 2 is the center of the green quarter circle.The SD domain is bounded by the three quarter circles and square edges.
Generally, in the -dimensional space partition model, one point is corresponded to each usage state, expressed as US h = ( 1 ℎ ,  2 ℎ , . . .,   ℎ , . . .,   ℎ ).  ℎ represents the utilization ratio of resource  in server ℎ.Given US h , the parameters  0 and  0 , the domain determination function f can be defined as Denote V  as a VM to be deployed.According to EAGLE in [17], for each running PM ℎ, firstly compute the posterior usage state, expressed as PS h,v k , which refers to the new usage state when VM V  is placed on it, then determine the domain f(PS h,v k ) for each PM ℎ, which has sufficient resources to host VM V  , and finally select a PM Host(V  ) for VM V  by following the steps below.
(1) Check whether  AD is empty, where  AD is a PM set in which each PM ℎ satisfies f(PS h,v k ) = AD; (2) if  AD is nonempty, then let Host(V  ) = ℎ AD , where ℎ AD ∈  AD is the PM with the highest overall resource utilization, otherwise check whether  SD is empty, where  SD is a PM set in which each PM ℎ satisfies f(PS h,v k ) = SD and go to the next step; (3) if  SD is nonempty, then let Host(V  ) = ℎ SD , where ℎ SD ∈  SD is the PM with the most balanced utilization of multidimensional resource, otherwise turn on a PM ℎ New and let Host(V  ) = ℎ New .
By comparing the experimental results of EAGLE and First Fit, Li et al. [17] validated that EAGLE work efficiently on the balanced utilization of multidimensional resources, and, thus, reduce the number of required servers.Besides, they analyzed the impact of model parameters ( 0 and  0 ) on the algorithm performance and got that the least number of servers is required when  0 is equal to 0.8 and  0 is equal to 0.1.In this paper, it is adopted that  0 = 0.8 and  0 = 0.1.

An Improved Model for Both Deterministic and Stochastic
Resources.As with [17], this work focuses on the VMP with multidimensional resources and aims to reduce the number of required servers.However, different from [17], VMs' stochastic demands and periodic server consolidation are taken into account in this paper.Considering these characteristics, a new partition is presented for the dimensional resource usage state space.
For a PM ℎ with PS h,v k ≤ 0.2 and f(PS h,v k ) = SD, it can be regarded as a candidate for the subsequent server consolidation, thus the criterion for selecting Host(V  ) from these PMs should focus on migration cost reduction.In view of this, the original SD domain is partitioned and then a new domain LD is obtained where the utilization ratio is less than 0.2 for all resources.Besides, for a usage state US h ≥ 0.8, it takes on a high and balanced utilization of -dimensional resources, which is the feature of states in the AD domain.So these states are incorporated into the AD domain and then a new domain HD is obtained where the utilization ratio is higher than 0.8 for all resources.Figure 3 shows the improved two-dimensional space partition model.The improved model includes four domains as follows.
(i) Unbalanced utilization domain (NBD): this domain indicates that there is an obvious disequilibrium of multidimensional resource utilization, which should be avoided.(ii) Balanced utilization domain (BD): this domain represents the balanced case.(iii) High utilization domain (HD): this domain implies that all resources are nearly exhausted, which is an ideal case for all PMs.(iv) Low utilization domain (LD): this domain indicates that the utilization ratio is very low for all resources.The PMs whose corresponding usage state lies in this domain may be candidates for the subsequent server consolidation.
For the -dimensional space partition model, according to US h , the parameters  0 and  0 , the domain determination function f  can be defined as In related studies, the stochastic resources refer primarily to the network bandwidth.Moreover, it is shown in [22,24] that the VMs' bandwidth demands approximately follow a normal distribution.So this paper assumes VM V  's demand    ( ∈ ) follows a normal distribution (   , (   ) 2 ).Based on this assumption, for the stochastic resource , the upper confidence limits of the total demand of all the VMs on the server ℎ can be calculated as where Φ −1 (1 − ) is the quantile of (0, 1) at probability .
Thus,   ℎ can be defined as When the server consolidation is triggered, an optimal scheme can be obtained by solving the model in Section 2.2.According to formula (17), the deterministic form equivalent with the chance constraint (10) can be derived as follows:

Strategy and Algorithms
Based on the equivalent form, the Model 0 can be transformed into a deterministic model called Model 1, which includes (3)-( 9), ( 11)- (13), and (19).Model 1 is a multiobject integer programming model.From a service provider's perspective, minimizing the number of required servers can reduce more cost than minimizing the number of migrations, so the two objectives can be arranged in order of importance, given that, the Lexicographic method [28] is adopted to solve Model 1.However, the method is only applied to solve a small-scale problem of server consolidation.As the scale of the problem gets larger, it may happen that the general solver such as Gurobi and Cplex cannot get a feasible solution in a reasonable time.Thus, a heuristic algorithm of live migration is designed in Section 4.3.

A Modified EAGLE Algorithm for VM Deployment.
Based on the improved multidimensional space partition model, a modified EAGLE algorithm (MEAGLE) is proposed.With the MEAGLE algorithm, the steps below can be followed to select Host(V  ) for VM V  .
(1) For each PM ℎ in  1 , compute the posterior usage state (2) For each PM ℎ in  fea (V  ), determine the domain f  (PS h,v k ) according to the formula (16) and PS h,v k .
(  (4) Check whether  HD is empty.If  HD is nonempty, then let Host(V  ) = ℎ HD , where ℎ HD ∈  HD is the PM with the highest overall resource utilization, otherwise go to step (5).
(5) Check whether  BD is empty.If  BD is nonempty, then let Host(V  ) = ℎ BD , where ℎ BD ∈  BD is the PM with the most balanced utilization of multidimensional resource, otherwise go to step (6).
(6) Check whether  LD is empty.If  LD is nonempty, then let Host(V  ) = ℎ LD , where ℎ LD ∈  LD is the PM with the largest number of VMs, otherwise turn on a PM ℎ New and let Host(V  ) = ℎ New .
For a PM ℎ with the usage state ( 1 ℎ ,  2 ℎ , . . .,   ℎ , . . .,   ℎ ), the overall resource utilization R ℎ and the equilibrium degree of -dimensional resource utilization D ℎ can be calculated as follows: The function JudgeMigratable (ℎ  ) in Algorithm 1 describes the procedures to determine whether a PM ℎ  is a candidate and returns a plan for VMs' migrations if so.The main points of the function are as follows.
(1) Firstly according to MEAGLE, check each VM V  on server ℎ  in the ascending order of VMs' ID, to see whether it can be migrated into a directlydeployed server, on which the original VMs need not be migrated out before VM V  's migration.
(2) If there is only one VM V  without directly-deployed servers, then go on looking for an indirectly-deployed server, into which V  can be migrated after some original VMs on it are migrated out, otherwise judge that the PM ℎ  is not a candidate.
(3) In order to keep migration cost as low as possible, three types of indirectly-deployed servers are considered: (a) a PM on which an original VM is needed to migrate out before VM V  's migration, (b) a PM on which two original VMs are needed to migrate out before VM V  's migration, and (c) a PM on which an original VM is needed to be exchanged with a VM on the other non-ℎ  PM before VM V  's migration.
(4) If there are two VMs on server ℎ  , and there exists one VM without directly-deployed and indirectlydeployed servers, then return to step (1) and check each VM again in the descending order of VMs' ID.

Simulation Configuration.
Consider a cloud data center with 200 servers with the same capacity.Each VM is assumed to have three types of resources demands: CPU, memory, and bandwidth.Set  = {CPU, memory},  = {bandwidth}, and  = 5%.The ratio of the resource demand to server capacity is employed to identify the demand intensity of each resource.
Among all resources, the resource with the largest DCR value is defined as the intensive resource and others are defined as the nonintensive resources.To emulate a typical data center setting, four groups of VMs are generated: CPU-intensive VMs, memory-intensive VMs, bandwidth-intensive VMs, and general VMs, where each group contains a quarter of all VMs.For each VM in the first three groups, the intensive resource randomly selects its DCR value from a higher range [30%, 40%] and other nonintensive resources randomly select their DCR values from a lower range [5%, 10%], which is based on the data generation method in [26].For each VM in the fourth group, all resources select their DCR value from a general range [3%, 10%] to validate the MEAGLE algorithm when  0 = 0.8.For the stochastic bandwidth demand, the selected DCR value represents the ratio between the mean of the bandwidth and the server capacity.Coefficients of variation for all VMs' bandwidth demands are set 0.2 by default.
Assume that the number of VM requests reaching the cloud data center in a period follows a Poissonian distribution with a mean .Here a period includes 60 time instants.In order to evaluate the performance of our MEAGLE and LMABBS, respectively, the experiment consists of two parts: (1) comparing the solutions of MEAGLE and EAGLE and (2) comparing the solutions of LMABBS and Gurobi.The Gurobi solutions are obtained by using Gurobi 5.5.0 to solve Model 1, which is formulated by YALMIP [29] in MATLAB environment.All experiments are performed on a computer with Intel Core i7 3.40 GHz CPU and 4 GB RAM.

Comparison between MEAGLE and EAGLE.
This subsection is conducted to evaluate the performance of MEA-GLE on the instant-deployment for VMs.Assume that there is no VM on each server at  = 0. Eighty periods are considered and 220 VMs on average are arriving in a period (i.e.,  = 220).According to the settings, a VM set is generated and called VmSet which includes all the VMs arriving during the eighty periods.The VMs are deployed instantly as soon as they arrive.For each VM V  ∈ VmSet, EAGLE, SEAGLE, and MEAGLE are, respectively, adopted to search Host(V  ).A VM's bandwidth demand is constant and taken as the mean in EAGLE while it is regarded as a random variable in SEAGLE and MEAGLE.When using SEAGLE, firstly for each PM ℎ in  1 , compute the posterior usage state PS h,v k according to formula (18) then perform the same steps as EAGLE described in Section 3.
Figure 5(a) shows the number of required servers at each time instant.The reduction of the servers at some time instants is a result of turning off the idle servers as some VMs are removed.To further validate the actual overflow probability, the following procedures are carried out.Firstly, for each VM's bandwidth demand at per time instant, we sample the trace to obtain 1000 samples of the momentary value of bandwidth consumption.Then for each packing obtained by the algorithms at per time instant and for each server in the packing, we sum up the total momentary bandwidth consumption per server and check whether Procedure MigPlan = JudgeMigratable (ℎ  ) (1) Initialize   (ℎ  ) as the set of VMs on server ℎ  at time instant  (2)  ←   (ℎ  ).size (3) for  = 1 to  do (4) According to MEAGLE, search for the Host(V  ) for V  , where Update the usage state of Host(V  ) after V  is placed on it.( 8) else (9)  ← ; (10) count ← count + 1 (11) if count > 1 / * In   (ℎ  ), there exists more than one VM without directly-deployed servers.( 13) MigPlan = []; (14) return MigPlan (15) end if (16) end if (17) end for (18) Update MigPlan (22) after two original VMs on ℎ 1 are respectively migrated into ℎ 2 and ℎ 3 , then Update MigPlan (24) else if ∃ℎ 1 , ℎ 2 ∈  1 \ {ℎ  }, V  can be placed on ℎ 1 after an original VM on ℎ 1 is exchanged with a VM on ℎ 2 , then  the server capacity constraint is broken.Figure 5(b) shows the cumulative distribution function (CDF) of server compliance with the overflow probability.From Figure 5, it can be seen, by EAGLE the number of required servers at each time instant is the least but only about 42% of servers respect the target overflow probability  = 5%.For MEAGLE and SEAGLE, almost all servers obey the target overflow probability, yet MEAGLE uses a smaller number of servers.
In order to better evaluate the performance of MEAGLE, a series of eight experiments is performed to deploy the VMs in VmSet under different coefficients of variation with respect to VMs' bandwidth demands.Here the coefficient of variation (3)  3 is 2.49%-20.40%less than  2 with the same coefficient of variation and  3 is the least when the variation coefficient is 0.01 and 0.05.From the results shown in Table 1, it can be concluded that SEAGLE and MEAGLE work well to obey the target overflow probability, while EAGLE brings a high proportion of servers violating the target overflow probability as it cannot detect the change of VM's burst level.Furthermore, though MEAGLE is slightly inferior to SEAGLE in lowering the overflow probability of servers as shown in Figure 5(b), MEAGLE uses a smaller number of servers than SEAGLE does.This is primarily due to the fact that by SEAGLE, a VM V  must not be placed on a PM ℎ with f(PS h,v k ) = FD and a new PM will be turned on if there is no proper server to host V  .VMs in our study have different resource intensities and a PM is more likely to incur an imbalanced utilization of multidimensional resources after it hosts a nongeneral VM (such as a CPU-intensive VM).Therefore, according to the procedures of SEAGLE and MEAGLE, SEAGLE will use more servers than MEAGLE obviously.

Comparison between LMABBS and Gurobi.
Within the two-phase optimization strategy, the server consolidation is implemented for every 4 periods.In order to test the performance of LMABBS,  = 11, 12, . . ., 26 is set and twenty running states are chosen for each value of .A running state for server consolidation is regarded as a problem instance and then 320 instances are generated.For each instance, the corresponding problem is solved, respectively, by the LMABBS and Gurobi.As a note, when solving large scale problems by Gurobi, one bad case that may arise is that solutions cannot be obtained due to out of memory.So the values of  in this subsection are relatively smaller than that in the previous subsection.Here set the solving time limit as 30 minutes for Gurobi.Then all instances are divided into five groups, according to all possible results ( 1 ∼  5 ) obtained by Gurobi shown in Table 2. Δ represents the variation in the number of required servers by server consolidation.The state with "the best feasible solution" represents that Gurobi cannot get an optimal solution but can get a feasible solution in 30 minutes, while the state with "no solution" means that Gurobi cannot produce a feasible solution in 30 minutes.Let   denote a set of the problem instances, where  = 1, 2, . . ., 5. Each instance in   has the same result as   if it is solved by Gurobi.
By statistical analysis on experimental results, the following points can be found out: (1) for the 80.95% instances in  1 , the LMABBS solution coincides with Gurobi solution on the number of required server and migrations; (2) by LMABBS, though there exist 19.05% instances in  1 without optimal solutions, there are still 29.17% of these instances with the minimum number of required servers (Let Ŝ1 represent the set of the 19.05% instances in  1 ); (3) in  2 , there are 74% instances for which the solutions of the two methods are identical to the number of required server and there are 60% instances for which the solutions of the two methods are identical to the number of required server and migrations; (4) for the instances in  3 and  4 , LMABBS as well as Gurobi cannot obtain a valid consolidation solution which can reduce the number of required servers; and (5) in  5 , there are 76.19% instances with valid consolidation solutions  3, where  ℎ ,   , and   , respectively, represent the ratios between the number of required servers, between the migration number, and between the computation time on the LMABBS solution and Gurobi solution.As one can see,   are both equal to 0 for some instances in Ŝ1 and  2 .This is because that the servers to turn off are chosen only from the nonbasic set when implementing server consolidation; in other words, if there exists a server in the basic set that is a candidate for server consolidation, the server will not be turned off by live migration because of high migration cost.Besides, according to the comparison results for  3 and  4 , it can be seen that LMABBS can rapidly identify a situation that the deployment of VMs on the collection of hosts is compact enough and, thus, spend little time looking for an invalid consolidation solution.Taken together, the comparison results suggest that the difference is fairly small between the number of required servers obtained by LMABBS and Gurobi and LMABBS has a significant advantage in lowering migration cost and improving the computational efficiency.

Conclusions
This paper studies the dynamic placement of VMs with deterministic and stochastic demands for green cloud computing.A two-phase optimization strategy is presented including instant VM deployment and periodic server consolidation.The innovations of the strategy can be listed as follows: (1) taking into account stochastic resources, the VMs' resource demands are described as random variables and the equivalent form of chance constraints is introduced to guarantee the availability for these resources; (2) when deploying VMs, the migration cost generated in the subsequent server consolidation is considered; and (3) given that this work focuses on the VMs with different resource intensities and the VMs with complementary demands can be placed on the same server, a temporary imbalanced utilization of multidimensional resources is allowed in our method.
For VM deployment and server consolidation, a modified EAGLE algorithm (MEAGLE) and a live migration algorithm based on the basic set (LMABBS) are proposed, respectively.The simulation results demonstrate that MEAGLE not only virtually ensures as good performance of user application as SEAGLE, but also reduces the number of required servers by 2.49% to 20.40% compared with SEAGLE.In addition, there is fairly small difference between the number of required servers obtained by LMABBS and Gurobi, but LMABBS has a computational efficiency unmatched by Gurobi.For the instances without a feasible solution in a reasonable time by Gurobi, LMABBS can obtain valid consolidation solutions and lower the number of required servers by an average of 15.98%.
For the public clouds that do not provide explicit SLAs guarantees on VM bandwidth today, the application of MEA-GLE algorithm will achieve a specified quality-of-service and then help to enhance their credibility and win more customers.Besides, the LMABBS algorithm will help to improve the efficiency of implementing server consolidation and reduce the waste of energy consumption effectively.Thus the proposed two-phase optimization strategy based on MEAGLE and LMABBS will provide a good solution to achieve high level service performance and to minimize energy consumption for green cloud computing.As a future work, we will intend to draw a specific migration plan for the implementation of server consolidation, with considering the intermediate migrations needed when a deadlock occurs in the actual migration process.

Figure 1 :
Figure 1: The deployment and removal of VMs.

Figure 3 :
Figure 3: The improved multidimensional space partition model.

4. 1 .
A Two-Phase Optimization Strategy.Figure 4(a)  shows the interactions between the two phases in optimization, which are, respectively, for VM deployment and server consolidation.At each time instant , firstly checks whether there are newly arriving VMs to be deployed; if there are, then enter into the VM deployment phase and place these VMs in the order they arrive.The procedure to deploy a VM is shown in Figure4(b), where  1 represents the set of the PMs with some VMs running.

Figure 4 :
Figure 4: The flowchart of the two-phase optimization strategy.

Figure 5 :
Figure 5: The comparisons of simulation results.
LMABBS) is proposed.Let the basic set  BS = {ℎ | f  (US h ) = HD or I ℎ = max ∈ 1 I  } and the nonbasic set  NBS =  1 \  BS , where I  indicates the number of VMs on the PM .The key idea of LMABBS is to turn off as many PMs in  NBS as possible by live migration.The algorithm is terminated when  NBS is empty or there is no server in  NBS as a candidate for server consolidation.Here a candidate means a PM on which all VMs can be migrated to other PMs.
The procedures to judge whether a PM ℎ  is a candidate for server consolidation.

Table 1 :
The simulation results with different variation coefficients.

Table 2 :
The possible results obtained by Gurobi.

Table 1 ,
where   ( = 1, 2, 3) represents the average number of required servers in a time unit and   ( = 1, 2, 3) refers to the proportion of servers violating the target overflow probability.From Table1, it can be seen, (1) with the same volatility (or coefficient of variation),  1 is significantly larger than  2 and  3 while the difference between  2 and  3 is very small; (2) as the coefficient of variation increases,  1 is rising sharply while  2 and  3 are keeping at very low level;

Table 3 :
Comparison of LMABBS solution with Gurobi solution.