^{1}

^{2}

^{3}

^{4}

^{1}

^{1}

^{1}

^{2}

^{3}

^{4}

The Cloud Computing paradigm is focused on the provisioning of reliable and scalable virtual infrastructures that deliver execution and storage services. This paradigm is particularly suitable to solve resource-greedy scientific computing applications such as parameter sweep experiments (PSEs). Through the implementation of autoscalers, the virtual infrastructure can be scaled up and down by acquiring or terminating instances of virtual machines (VMs) at the time that application tasks are being scheduled. In this paper, we extend an existing study centered in a state-of-the-art autoscaler called multiobjective evolutionary autoscaler (MOEA). MOEA uses a multiobjective optimization algorithm to determine the set of possible virtual infrastructure settings. In this context, the performance of MOEA is greatly influenced by the underlying optimization algorithm used and its tuning. Therefore, we analyze two well-known multiobjective evolutionary algorithms (NSGA-II and NSGA-III) and how they impact on the performance of the MOEA autoscaler. Simulated experiments with three real-world PSEs show that MOEA gets significantly improved when using NSGA-III instead of NSGA-II due to the former provides a better exploitation versus exploration trade-off.

Cloud Computing [

A Cloud enables for the acquisition of computing resources through different types of virtual machine (VM) instances [

Since the workloads in a Cloud are very often caused by the coexistence of applications with different computing requirements, autoscaling strategies [

Concretely, in this work, we address the autoscaling problem for PSEs, to determine the right number of each VM instance type and pricing model as well as the bid prices for the spot instances. Specifically, we rely on a formulation of our problem that involves minimizing the makespan, i.e., the total execution time of all tasks in a PSE, while also minimizing the monetary cost of this set of PSE tasks and the OOB errors, while focusing on the algorithm that best optimizes the formulated problem.

From the related works, most Cloud autoscaling strategies have been proposed for executing workflow applications and are mainly subject to deadline constraints [

A new and improved multiobjective Cloud autoscaler of the autoscaler proposed in [

The proper parameterization values for the NSGA-II and NSGA-III: we present an experimental validation demonstrating that by using NSGA-III, the performance of the MOEA autoscaler can be greatly boosted considering the results already obtained in [_{2}-norm [

Finally, Section

In this paper, we address the problem of autoscaling PSEs in public Clouds (e.g., Amazon EC2 and Google Cloud) with the help of evolutionary algorithms. This general problem involves determining the number and type of VM instances to be acquired to execute the tasks in a given PSE, and scheduling these tasks on the instances, so that the predefined optimization objectives are reached. In this respect, we consider three relevant optimization objectives: the minimization of the makespan, the minimization of the monetary cost, and the minimization of the impact of out-of-bid (OOB) errors. This problem is recognized as a multiobjective NP-hard problem [

A PSE model is a numerical experiment which involves a large number of tasks to be executed and usually needs several hours or days to be completed. In practice, when a PSE is executed, the execution, duration, and cost of the tasks differ according to the instances used to execute these tasks.

In public Clouds, instances of different types of VMs have different characteristics with respect to operating system, number of processors, processing power, memory size, memory speed, disk size, and the like. Moreover, instances of different types of VMs have different monetary cost. In this respect, instances of VMs can be acquired under different pricing models, which determine the cost of instances and also their behavior.

In this paper, we consider that instances are charged for every hour used under two possible pricing models, namely, the on-demand model and the spots model, as in real public Clouds (e.g., Amazon EC2). In the on-demand model, instances are acquired at a fixed monetary cost by the hour of computation. The monetary cost of instances acquired under this pricing model is usually higher than that of instances acquired under the spots model. For the remainder of this work, we will refer to instances acquired under the on-demand model as on-demand instances.

In the spots pricing model, the monetary cost of instances fluctuates mostly unpredictably over time and is usually lower than that of the on-demand instances. To acquire an instance under this pricing model, the user must make a bid as in a stock market, indicating the maximum amount of money he/she is willing to pay for the instance. Then, while the current monetary cost of the instance is lower than the bid of the user, the instance will remain assigned to the user. Otherwise, when the current monetary cost of the instance exceeds the bid of the user, an OOB error happens. This error forces the termination of the instance, and therefore, the abrupt termination of the execution of the tasks running on the instance. Thus, OOB errors can impact negatively on the execution of PSEs. For the remainder of this work, we will refer to the instances acquired under the spots model as spots instances.

The PSE autoscaling problem addressed here is composed of two well-defined interrelated problems. The first problem involves determining a scaling plan detailing the number and type of on-demand and spot instances to request to the Cloud provider for the next hour and also the bids corresponding to the spot instances (i.e., the virtual infrastructure setting to request for the following hour). The second problem involves scheduling the tasks of the PSE on both the on-demand and spot instances acquired.

These two problems need to be solved periodically (i.e., every one hour) during the execution of the PSE, in order to update the infrastructure setting according to the workload of the PSE (i.e., the tasks whose execution is pending) and schedule this workload on the new infrastructure setting, so that the considered optimization objectives are reached. It is important to note that the autoscaler reevaluation conditions are not the same in each autoscaling cycle. For example, the price of spot instances varies because it is subject to a supply and demand market, the Cloud provider may suddenly finish some of the acquired spot instances, and therefore, less computing power will be available, and finally, it is possible that the set of running tasks and ready-to-run tasks change (in each cycle, there may be tasks that have already been completed and/or tasks that have been canceled). In Section

This section describes the mathematical formulation of the problem [^{od}, ^{s}, and ^{b}), where

Given the set

The three objective functions considered as part of the problem are described below.

^{od} and ^{s}. Tasks are greedily scheduled using the earliest completion time (ECT) criterion, i.e., scheduling each task to the instance that promises the earliest completion time.

Formally, the makespan is computed as_{t} is the duration of the task

^{od} and ^{s} for one hour of computation:

The cost of on-demand instances is the sum of running all the on-demand instances of each type _{i}). The cost of spot instances is computed differently. The model computes such cost as the product of the number of spot instances of type

It is worth to point out that equation (

The potential impact of OOB errors is computed as the sum of the total number of virtual CPUs of type ^{th} function of

The image of the errorsImpact (·) function is [0,

The set of constraints considered as part of the problem is described below.

The constraint in equation (

_{i} is the on-demand price for the instances of type

As described in Section

To solve each of these multiobjective autoscaling problems, the MOEA autoscaler [

This step is concerned with solving the multiobjective autoscaling problem described in Section

In the MOEA autoscaler, this step is developed by applying the NSGA-II and, therefore, obtaining a Pareto set (i.e., set of nondominated solutions) where each solution represents a feasible scaling plan and is encoded like the 3-tuple (^{od}, ^{s}, and ^{b}) detailed in Section

In the study performed in this paper, this step is carried out by using the two optimization algorithms object of our analysis, i.e., NSGA-II as used in MOEA and NSGA-III [

From the Pareto set provided by the first step, one solution is selected to solve the addressed multiobjective autoscaling problem, by applying a predetermined selection criterion. This is meant in order to obtain one solution in a fully autonomous way, without relying on a human decision maker.

The selection criterion calculates the distance of each solution of the Pareto set to an ideal solution, by applying the well-known _{2}-norm [

From the solution selected by the second step, the autoscaler builds the scaling plan. Specifically, the autoscaler acquires the number of on-demand instances detailed in the solution and the number of spot instances with the corresponding bid prices detailed in the solution. Then, the autoscaler schedules the tasks in

This section describes the multiobjective optimization algorithms studied to solve the multiobjective autoscaling problem that takes place in the first step of the execution of the MOEA autoscaler.

NSGA-II [

In each generation, the algorithm selects

After the mating pool is composed, the algorithm creates an offspring population from the mating pool, by applying the known simulated binary crossover operator [

Once the offspring population is created, the algorithm combines the current and offspring populations and then selects the best _{1}, _{2}, …}. Then, each nondomination level is selected one at a time to create the new population, beginning from _{1}, until the size of the new population is equal to _{l} is not fully included in this population. In this respect, the solutions from level _{l} to level _{l − 1} are included in this population, and the _{l}.

To select the _{l}, the algorithm utilizes a selection process which considers the crowding distances of the solutions in _{l}. This process calculates the crowding distance of each solution in _{l}. Then, the process sorts the solutions in _{l} based on their distances in descending order and selects the

The algorithm ends its execution once a predefined termination criterion is achieved (i.e., a given number of evaluations for the generated solutions), providing the Pareto set (i.e., the set of nondominated solutions) of the population corresponding to the last generation.

NSGA-III [

NSGA-III has the same general behavior as NSGA-II and uses the same procedures used by NSGA-II to develop some stages of the evolutionary cycle. In this respect, NSGA-III applies the same procedure used by NSGA-II to create the initial population with

However, NSGA-III differs from NSGA-II in terms of the selection process utilized to create a new population with _{1}, _{2}, …} and then selecting the nondomination levels once at a time to build the new population, until the size of this population is greater or equal than _{l} cannot be fully included in this population, NSGA-III uses a different selection process to that used by NSGA-II to decide which _{l} will be included in this population.

To select the remaining _{l}, NSGA-III applies a selection process based on reference points. The process considers a set of reference points widely and uniformly distributed on the normalized hyperplane inherent to the optimization objectives of the problem addressed by the algorithm. Then, the process emphasizes the selection of solutions from _{l} which are associated with each of these reference points. Thus, this process promotes the selection of diverse and well-distributed nondominated solutions, with the aim of preserving the diversity and distribution of the new population.

NSGA-III considers the same termination criterion used by NSGA-II to finish its execution. After such criterion is achieved, NSGA-III provides the Pareto set of the population corresponding to the last generation.

The algorithm NSGA-III utilizes the known Das and Dennis [

This approach defines reference points on a normalized hyperplane that is equally inclined to all objective axes and has an intercept of one on each objective axis. The total number of reference points

In the algorithms NSGA-II and NSGA-III studied, each solution of the population represents a feasible scaling plan for the current autoscaling stage.

Each solution is encoded as a vector with a size equal to 3 × ^{od}, ^{s}, and ^{b}) presented in Section

The algorithms NSGA-II and NSGA-III apply the known simulated binary crossover operator [

To generate the offspring solutions _{i} is calculated by equation (_{i} is a random real number generated between 0 and 1.

The algorithms NSGA-II and NSGA-III apply the known polynomial mutation operator [

To generate a mutated value _{i} is a random real number generated between 0 and 1. The term _{i} and _{i} correspond to the lower and upper bounds of the position _{i} and _{i}) is created by the mutation operator. The term _{i} corresponds to that used in equation (

As was mentioned in Section

In the NSGA-II, the selection process considers first the nondomination level of the solutions in the combined population and then the crowding distance of these solutions. The crowding distance represents the distance of a solution to its neighboring solutions. Then, the process emphasizes the selection of nondominated solutions with larger crowding distances. Thus, the process promotes the selection of diverse nondominated solutions. However, this process does not guarantee the selection of well-distributed nondominated solutions.

In contrast to NSGA-II, in NSGA-III, the selection process considers first the nondomination level of the solutions in the combined population and then the association of these solutions to the reference points. In this sense, the process utilizes a set of well-spread reference points (i.e., a set of widely and uniformly distributed reference points). Then, the process emphasizes the selection of nondominated solutions which are associated with each of these reference points. Thus, this process promotes the selection of diverse and well-distributed nondominated solutions, in order to preserve the diversity and distribution of the new population.

By using this selection process based on well-spread reference points, the algorithm NSGA-III has the possibility of reaching better Pareto sets in terms of both diversity and distribution of the nondominated solutions, these solutions represent feasible scaling plans for the multiobjective autoscaling problem addressed in the first step of the autoscaler. This represents the rationale of applying NSGA-III to the problem at hand. From now on, we empirically evaluate this claim.

In this Section, we detail the experimental setting used to evaluate the autoscaler based on each of the studied multiobjective genetic algorithms, i.e., NSGA-II and NSGA-III. First, Section

PSEs are a very popular way of conducting simulation-based experiments, used by scientists and engineers, through which the same application code is run several times with different input parameters resulting in different output data. The first PSE in this paper consists of a classical benchmark problem [^{y} Mpa, with ^{8} Mpa. In particular, the variation of a material parameter in a parametric study may be useful to other disciplines such as design, where it is important that the specialists to know the strength, flexibility, and other characteristics of materials used in the design of certain components.

Geometry of the plane strain plate.

The second PSE is the study of the elastoplastic buckling behavior of cruciform columns [_{n} = _{n − 1} + 0.25, with _{0} = 0.5 and

Geometry of the cruciform column.

The third PSE is derived from the two above described PSE applications. We called this application as

For evaluating the performance of the autoscaler based on each studied algorithm, we have defined three PSE applications named Plate3D, Cruciform, and Ensemble, each one with three different sizes

Characteristics of each type of on-demand instance considered during the experimentation are described in Table

Characteristics of the on-demand instances considered. The data correspond to instances belonging to the US-west region.

VM type | ECUtot | ECU | Price (USD) | |
---|---|---|---|---|

t2.micro | 1 | 1 | 1 | 0.013 |

m3.medium | 1 | 3 | 2 | 0.07 |

c3.2xlarge | 8 | 28 | 3.5 | 0.42 |

r3.xlarge | 4 | 13 | 3.25 | 0.35 |

m3.2xlarge | 8 | 26 | 3.25 | 0.56 |

Spot instances used were set using actual data of Amazon EC2 spot prices corresponding to the US-west region (Oregon) in a tree-month period (between March 7^{th} and June 7^{th} of 2016.). To compute the probabilities of OOB errors, we used the first two months of data by counting the number of times a sliding time window of 1 h (minimum purchase unit in EC2) presented spot prices over the set of possible bid values in ^{b}. Then, the data corresponding to the last month of the history were kept for the experiments presented in Section

We have developed computational experiments to evaluate the autoscaler based on each of the studied algorithms, i.e, NSGA-II and NSGA-III. To perform the experiments, we run the autoscaler endowed with both optimization algorithms. In this section, we describe the procedure used to identify the best parameter setting for NSGA-III. Then, we present the parameterization for NSGA-II. Then, we describe the experimental settings utilized to evaluate the performance of the autoscaler based on each of the studied algorithms. Finally, we present and then analyze the experimental results obtained.

We developed a sensitivity analysis to identify the best parameter setting for NSGA-III, regarding each studied application and size. In this analysis, we considered 303 different parameter settings. These settings were generated by the Saltelli sampling method [

Sampling ranges for the parameters of NSGA-III.

Parameter | Description | Sampling range |
---|---|---|

Maximal number of solutions to be evaluated | [500, 20000] | |

Number of solutions of the population | [90, 200] | |

Number of reference points | [90, 200] | |

Crossover probability | [0.8, 1] | |

Crossover distribution index | [0, 60] | |

Mutation probability | [0.01, 0.6] | |

Mutation distribution index | [0, 60] |

Table

Parameter setting selected for NSGA-III regarding each application and size.

Application | Size | HV | |||||||
---|---|---|---|---|---|---|---|---|---|

Plate3D | 30 | 18400 | 92 | 91 | 0.81 | 3.8 | 0.59 | 14.6 | 0.95 |

100 | 18400 | 92 | 91 | 0.92 | 1.9 | 0.24 | 3.1 | 0.94 | |

300 | 18400 | 92 | 91 | 0.83 | 46.9 | 0.26 | 1.5 | 0.94 | |

Cruciform | 30 | 18400 | 92 | 91 | 0.90 | 20.7 | 0.45 | 5.2 | 0.81 |

100 | 18400 | 92 | 91 | 0.90 | 56.3 | 0.39 | 7.1 | 0.80 | |

300 | 18400 | 92 | 91 | 0.98 | 13.3 | 0.24 | 3.1 | 0.80 | |

Ensemble | 60 | 18400 | 92 | 91 | 0.92 | 13.3 | 0.24 | 3.1 | 0.88 |

200 | 18400 | 92 | 91 | 0.87 | 46.9 | 0.26 | 1.5 | 0.88 | |

600 | 18400 | 92 | 91 | 0.87 | 28.3 | 0.09 | 1.5 | 0.87 |

In relation to the best parameter setting for NSGA-II, regarding each studied application and size, we considered the settings recommended in [

Table

Parameter setting considered for NSGA-II regarding each application and size.

Application | Size | HV | ||||||
---|---|---|---|---|---|---|---|---|

Plate3D | 30 | 18400 | 116 | 0.88 | 18.9 | 0.55 | 10.8 | 0.87 |

100 | 18400 | 116 | 0.88 | 18.9 | 0.55 | 10.8 | 0.90 | |

300 | 18400 | 172 | 0.89 | 13.3 | 0.57 | 12.7 | 0.90 | |

Cruciform | 30 | 18400 | 116 | 0.88 | 18.9 | 0.55 | 10.8 | 0.38 |

100 | 18400 | 136 | 0.88 | 18.9 | 0.55 | 10.8 | 0.57 | |

300 | 18400 | 116 | 0.88 | 18.9 | 0.55 | 10.8 | 0.68 | |

Ensemble | 60 | 18400 | 116 | 0.88 | 18.9 | 0.55 | 10.8 | 0.72 |

200 | 18400 | 116 | 0.88 | 18.9 | 0.55 | 10.8 | 0.79 | |

600 | 18400 | 116 | 0.88 | 18.9 | 0.55 | 10.8 | 0.80 |

The autoscaler based on NSGA-II and also the autoscaler based on NSGA-III were evaluated on all the studied applications and sizes. To do that, we used the CloudSim [

As NSGA-II and NSGA-III are nondeterministic, each of the mentioned autoscalers was run 30 times on each studied application and size, with the aim of guaranteeing the statistical significance of the results. For each run, the results obtained regarding different metrics including makespan in seconds, cost in USD, and number of task failures were recorded.

In order to develop the runs of the autoscaler based on NSGA-III, we used the parameter settings described in Table

Table _{2}-norm which considers the makespan, cost, and number of task failures resulting from the experiments. This _{2}-norm jointly analyzes the metrics which are interesting for the multiobjective autoscaling problem addressed in this paper.

Results obtained from the computational experiments. The rows present the results per autoscaler regarding all the studied applications and sizes. For the average makespan, cost, number of task failures, and _{2}-norm, lower values represent better results.

Application | Size | Autoscaler | Makespan | Cost | Task failures | _{2}-norm |
---|---|---|---|---|---|---|

Plate3D | 30 | NSGA-III | 5.80 | |||

NSGA-II | 1381.02 | 0.23 | 0.44 | |||

100 | NSGA-III | 1910.03 | ||||

NSGA-II | 0.36 | 25.77 | 0.28 | |||

300 | NSGA-III | 2894.53 | ||||

NSGA-II | 0.98 | 82.17 | 0.37 | |||

Cruciform | 30 | NSGA-III | 44.75 | |||

NSGA-II | 3599.30 | 0.47 | 0.42 | |||

100 | NSGA-III | 146.33 | ||||

NSGA-II | 3602.92 | 0.91 | 0.46 | |||

300 | NSGA-III | 309.80 | ||||

NSGA-II | 5118.21 | 4.74 | 0.30 | |||

Ensemble | 60 | NSGA-III | 35.32 | |||

NSGA-II | 5504.60 | 0.69 | 0.46 | |||

200 | NSGA-III | 147.00 | ||||

NSGA-II | 6912.26 | 1.78 | 0.54 | |||

600 | NSGA-III | 372.86 | ||||

NSGA-II | 7401.15 | 4.59 | 0.40 |

From Table

Finally, in relation to the _{2}-norm, NSGA-III has obtained a much better average performance than NSGA-II in the nine study cases. This is mainly because NSGA-III outperformed NSGA-II in the nine studied cases in terms of cost and outperformed NSGA-II in seven of the studied cases in terms of makespan.

To ascertain the significance of the improvements reached by NSGA-III regarding NSGA-II, we applied a statistical significance test on the results obtained from the experiments for the _{2}-norm. Given that the _{2}-norm jointly analyzes the makespan, cost, and number of task failures resulting from the experiments, we consider that the _{2}-norm is appropriate and useful to develop the statistical significance test. In relation to the results obtained from the experiments for the _{2}-norm, each of the autoscalers was run 30 times on each studied application and size. Thus, each autoscaler obtained 30 results for the _{2}-norm regarding each studied application and size. We applied the normality Shapiro–Wilk test on the results obtained by each autoscaler regarding each studied application and size, to determine if these results follow a normal distribution or not and decide the statistical significance test to be applied. According to the Shapiro–Wilk test, which was applied with a strong confidence level of _{2}-norm, in all the studied applications and sizes.

In addition to the results presented in Table

Computation time (in seconds) required by each autoscaler regarding all the studied applications and sizes.

Application | Size | NSGA-II | NSGA-III | ||||
---|---|---|---|---|---|---|---|

Average | Max | Min | Average | Max | Min | ||

Plate3D | 30 | 6.50 | 9 | 4 | 8 | 4 | |

100 | 21.13 | 25 | 14 | 28 | 15 | ||

300 | 92.63 | 115 | 60 | 119 | 62 | ||

Cruciform | 30 | 6.14 | 10 | 5 | 7 | 5 | |

100 | 24.00 | 31 | 17 | 24.00 | 32 | 16 | |

300 | 128.14 | 154 | 82 | 121 | 75 | ||

Ensemble | 60 | 13.00 | 17 | 9 | 16 | 8 | |

200 | 65 | 39 | 64.30 | 75 | 45 | ||

600 | 235 | 187 | 209.56 | 282 | 182 |

As shown in Table

From the analysis of the results in Tables

In this section, we analyze the quality of the Pareto sets obtained by the autoscalers for each of the studied applications and sizes. We focus the attention on the Pareto sets obtained by the autoscalers during the first autoscaling stage (the first hour). This is because the following reason. For each of the studied applications and sizes, during the first autoscaling stage, the autoscalers address exactly the same multiobjective problem and generate Pareto sets for such problem. Thus, it is appropriate and valuable to compare these Pareto sets. In each of the next autoscaling stages (after the first hour), the autoscalers usually address different multiobjective problems. This is mainly because the problems inherent to each of these stages vary according to the state of the PSE’ tasks execution and the state of the virtual infrastructure. Therefore, it is not appropriate to compare the Pareto sets obtained for such problems.

To analyze the quality of the Pareto sets, we use two well-known measures which are usually utilized to evaluate Pareto sets obtained by evolutionary algorithms. One of these measures is the hypervolume [

Table

Hypervolume (average hypervolume value) and coverage (average coverage value) of the Pareto sets obtained by each autoscaler during the first autoscaling stage (first hour), for each studied application and size. For hypervolume measure, higher values represent better results. For coverage measure, lower values represent better results.

Application | Size | Hypervolume (%) | Coverage (%) | ||
---|---|---|---|---|---|

NSGA-III | NSGA-II | NSGA-III | NSGA-II | ||

Plate3D | 30 | 81 | 53.51 | ||

100 | 84 | 55.26 | |||

300 | 85 | 57.89 | |||

Cruciform | 30 | 16 | 53.57 | ||

100 | 17 | 55.95 | |||

300 | 17 | 59.52 | |||

Ensemble | 60 | 34 | 52.63 | ||

200 | 37 | 54.39 | |||

600 | 38 | 57.02 |

As shown in Table

Evolution of the hypervolume value of the Pareto sets obtained by NSGA-III and NSGA-II over the generations, for Plate3D with 30, 100, and 300 tasks.

Evolution of the hypervolume value of the Pareto sets obtained by NSGA-III and NSGA-II over the generations, for Cruciform with 30, 100, and 300 tasks.

. Evolution of the hypervolume value of the Pareto sets obtained by NSGA-III and NSGA-II over the generations, for Ensemble with 60, 200, and 600 tasks.

Table

Pareto sets obtained by NSGA-III and NSGA-II for Plate3D with 30 tasks.

Pareto sets obtained by NSGA-III and NSGA-II for Ensemble with 60 tasks.

The results detailed in Table

Cloud autoscaling involves solving two different complex optimization problems known as scaling and scheduling. The efficient management of scientific applications on the Cloud via autoscaling techniques is an important problem [

There are other works where spot instances were also considered. Among them, we can mention the work in [

There are other works that differ from ours because they were proposed for addressing workflow autoscaling [

Other works are proposed in [

On the other hand, in [

There are also two works that propose the use of the NSGA-III algorithm [

It is worth noting that, from the related works found, most of them have been proposed for workflows considering task deadlines or budget constraints, and only in two works [

Besides, another distinction is that only in two of the surveyed works [

We addressed the problem of autoscaling PSEs in public Clouds (e.g., Amazon EC2 and Google Cloud), considering that the instances of VMs can be acquired under two possible pricing models: the on-demand model and the spots model. This problem implies determining the number and type of on-demand and spot instances to be acquired for executing the tasks in a PSE, and the bid prices corresponding to the spot instances, so that the predetermined optimization objectives are reached. In this respect, three relevant optimization objectives were considered: the minimization of the makespan, the monetary cost, and the impact of OOB errors which are inherent to spot instances. To solve the resulting optimization problem, the well-known multiobjective genetic algorithms NSGA-II and NSGA-III were exploited as part of the autoscaler MOEA, which is the focus of this paper.

The autoscaler endowed with both algorithms was evaluated on three different real-world PSEs, considering three different sizes (i.e., numbers of tasks) for each PSE. Moreover, different types of on-demand and spot instances available in Amazon EC2 were considered. These different instance types have different characteristics in relation to the processing capacity and also differ regarding the monetary cost. These PSE applications and instance types were considered in order to provide diverse realistic experimental settings. Then, the performance of the autoscaler based on NSGA-III was compared in detail with that of the autoscaler based on NSGA-II.

According to the performance comparison carried out, the autoscaler based on NSGA-III has significantly outperformed the autoscaler based on NSGA-II in terms of the _{2}-norm which jointly assesses makespan, cost, and number of task failures caused by OOB errors, in all the PSE applications and sizes considered. Thus, we can mention that NSGA-III may be considered as a better alternative than NSGA-II for solving different instances of the multiobjective autoscaling problem addressed in this paper. Future work will explore other realistic scenarios, including simultaneous optimized autoscaling of PSEs belonging to different users, as well as federated Clouds in addition to single Clouds.

The spot prices data used to support the findings of this study have been deposited in the Amazon Web Services (AWS) Spot Prices Data 2016, Mendeley Data, v1 (

The authors declare that there are no conflicts of interest regarding the publication of this article.

The authors acknowledge the financial support by CONICET, grant no. 11220170100490CO, and the SIIP-UNCuyo, project no. B082.

^{th}utility