User Utility Oriented Queuing Model for Resource Allocation in Cloud Environment

. Resource allocation is one of the most important research topics in servers. In the cloud environment, there are massive hardware resources of different kinds, and many kinds of services are usually run on virtual machines of the cloud server. In addition, cloud environment is commercialized, and economical factor should also be considered. In order to deal with commercialization and virtualization of cloud environment, we proposed a user utility oriented queuing model for task scheduling. Firstly, we modeled task scheduling in cloud environment as an M / M /1 queuing system. Secondly, we classified the utility into time utility and cost utility and built a linear programming model to maximize total utility for both of them. Finally, we proposed a utility oriented algorithm to maximize the total utility. Massive experiments validate the effectiveness of our proposed model.


Introduction
Providers of cloud services usually provide different computing resources with different performances and different prices, and the requirements of users for performance and cost of resources differ greatly too.So how to allocate available resources for users to maximize the total system utilization is one of the most important objectives for allocating resources and scheduling tasks [1] and is also a research focus in cloud computing.
Traditional resource allocation models mainly focus on the response or running time, saving energy of the whole system, and fairness of task scheduling and do not take user utility into consideration [2].However, the utility of a user in cloud environment is the usage value of services or resources, and it describes how the user is satisfied with the proposed services or resources while occupying and using them [3,4].In order to maximize the total utility of all users in cloud environment, it is necessary to analyze and model user utility first and then optimize it to get a maximum [5].The modeling of user utility is very complex, as it needs a formal description considering many factors, such as the processing time that tasks have passed by [6], the ratio of finished tasks [7], the costs of finished and unfinished tasks [8], and the parallel speedup [9].
In a cloud server, requests of users, called tasks, come randomly, and a good description of these tasks is the Poisson distribution assumption.At the same time, under the commercialization constraint of the cloud environment, the utility of cloud server becomes much more important.In this paper, we formalized and quantified the problem of task scheduling based on queuing theory, divided the utility into time utility and cost utility, and proposed a linear programming method to maximize the total utility.The contributions of the paper are as follows: The rest of the paper is organized as follows.In Section 2, we review related works about resource allocation and task scheduling in cloud computing.In Section 3, we formalize the tasks in cloud environment based on the queuing theory, define a random task model for random tasks, describe our proposed user utility model, and design a utility oriented time-cost scheduling algorithm.Experiments and conclusion are given in Sections 4 and 5, respectively.

Related Works
In cluster systems that provide cloud services, there is a common agreement in researchers that the moments, when tasks come into the system, conform to the Poisson distribution, and both the intervals between two coming tasks and the serviced time of tasks are exponentially distributed.In this situation, heuristic task scheduling algorithms, such as genetic algorithm and ant colony algorithm, have better adaptability than traditional scheduling algorithms.However, the deficit of heuristic algorithms is that they have complex problem-solving process, so they can only be applied in small cluster systems.The monstrous infrastructure of cloud systems usually has many types of tasks, a huge amount of tasks, and many kinds of hardware resources, which makes heuristic algorithms unsuitable.
There are a lot of researches about resource allocation or task scheduling in cloud environment, especially for the MapReduce programming schema [10].Cheng et al. [11] proposed an approximate algorithm to estimate the remaining time (time to end) of tasks in MapReduce environment and the algorithm scheduled tasks with their remaining time.Chen et al. [12] proposed a self-adaptive task scheduling algorithm, and this algorithm computed the running progress (ratio of time from beginning to total running time) of the current task on a node based on its historical data.The advantage of [12] is that it can compute remaining time of tasks dynamically and is more suitable to heterogeneous cloud environment than [11].In addition, Moise et al. [13] designed a middleware data storage system to improve the performance and ability of fault tolerance.
Traditional task scheduling algorithms mainly focus on efficiency of the whole system.However, some researchers introduce economic models into task scheduling, and the basic idea is optimizing resource allocation by adjusting users' requirements and allocating resources upon price mechanism [14].Xu et al. [15] proposed a Berger model based task scheduling algorithm.Considering actual commercialization and virtualization of cloud computing, the algorithm is based on the Berger social allocation model and adds additional cost constraints in optimization objective.According to experiments on the CloudSim platform, their algorithm is efficient and fair when running tasks of different users.In addition, with respect to the diversity of resources in cloud environment, more researchers believe that the diversity will increase as time goes on with update of hardware resources.In order to alleviate this phenomenon and ensure quality of services, Yeo and Lee [16] found that while the resources were independently identically distributed, dropping resources that needed three times the number of minimal response time could make the whole system use less total response time and thus less energy.
The study of random scheduling began in 1966, and Rothkopf [17] proposed a greedy optimal algorithm based on the weights of tasks and expected ratios of finished time to total time.If all tasks had the same weights, then this algorithm became the shortest expected processing time algorithm.Möhring et al. [18] proved the optimal approximation for scheduling tasks with random finished time.They began with the relaxation of linear programming, studied the problem of integer linear programming for systems with homogeneous tasks, and got an approximate solution with the lower limit of the linear programming.Based on the above research, Megow et al. [19] proposed a better optimal solution with better approximation.In addition, Scharbrodt et al. [20] studied how to schedule independent tasks randomly.They analyzed the problem of scheduling  tasks on  machines randomly and gave the worst performance of random scheduling under homogeneous environment theoretically, and their result was the best among related works.
All of the above algorithms focus on the response or running time of users' requirements, saving energy of the whole system and fairness of tasks, and do not take user utility into consideration.However, utility of users is very important in cloud service systems.In order to maximize the total utility of all users in cloud environment, we analyze and model user utility first and then optimize it to get a maximal solution.
In addition, Nan et al. [21] studied how to optimize resource allocation for multimedia cloud based on queuing model, and their aim is to minimize the response time and the resource cost.However, in this paper, we deal with commercialization and virtualization of cloud environment, and our aim is maximizing utility.Xiao et al. [22] presented a system that used virtualization technology to allocate data center resources dynamically.Their aim is to minimize the number of servers in use considering the application demands and utility, whereas in this paper we aim to maximize the system's total utility under a certain cloud environment.

Queuing Model of Tasks.
In this paper, we describe randomness of tasks with the //1 model of queuing theory, and the model is illustrated in Figure 1.The model consists of one server, several schedulers, and several computing resources.When user tasks are submitted, the server analyzes and schedules them to different schedulers and adds them to local task queue of the corresponding scheduler.Finally, each scheduler schedules its local tasks to available computing resources.In Figure 1, () is the waiting time of a task in the queue and () is the running time.

Modeling Random Tasks.
In the following, we will analyze the waiting time, running time, and queue length of the proposed //1 model.
The service intensity describes the busyness of the scheduler.When  approaches zero, the waiting time of tasks is short, and the scheduler has much idle time; when  approaches one, the scheduler has less idle time, and thus tasks would have long waiting time.Generally speaking, the average arrival rate should be equal to or smaller than the average service rate, otherwise there will be more and more waiting tasks in the scheduler.
Definition 2. If we denote the expected length of tasks in a scheduler as , the expected length of tasks in queuing as   , the expected total time (including both waiting time and running time) of a task as , and the expected waiting time of a task in queuing as   , then we have the following equations according to queuing theory [17]: In addition, let   = { = } be the possibility of number of tasks in a scheduler at any moment; then, we have the following equation: If  = 0, then  0 is the possibility that all virtual machines are idle.

Time Utility of Tasks.
As we can see from Figure 1, the total time that a user takes from submitting a request to getting the result includes both waiting time () and running time ().Here, the computing resources are virtual resources managed by virtual machines.Let  be total time; then, we have In ( 4), the running time () is the sum of used time () and remaining time (); that is,  () =  () +  () . ( In order to calculate the time requirement of a task, the system needs to calculate the remaining time () and schedules () for different tasks to different virtual machines.
For analyzing the remaining time, we classified tasks into set  = {  | 1 ≤  ≤ } and nodes into set  = {  | 1 ≤  ≤ }.According to statistical computing, we can get the average executing rate of task   on node V  ; that is,  = { , | 1 ≤  ≤ , 1 ≤  ≤ }, and then the remaining time of   on V  is where  is the number of total tasks and   is the number of finished tasks.For computing intensive tasks,  is the total input data and   is the already processed input data.Schedulers schedule tasks on virtual machine resources according to their remaining time and assure all tasks are finished on time.
A task can be executed either on one virtual machine or on  virtual machines in parallel, while being divided into  subtasks.We denoted the subtask set as  = {  | 1 ≤  ≤ }.While these subtasks are executed on different virtual machines, especially different physical nodes, the communication cost increases, and we use speedup to measure the parallel performance where  1 is the time of a task in one node and   is the time of a task in  nodes.In order to make sure  ≤  0 , all subtasks run in parallel, and total time of the task is where () , is the time of subtask   on V  and max{() , } is the maximal time of all subtasks.

Cost
where  , is the unit cost of task   on node V  and () ,, is the time of subtask   on node V  .

Formalization and Optimization of User Utility
Definition 4. Let the time utility function be   and let the cost utility function be   ; then, the total utility is where  +  = 1, 0 ≤  ≤ 1 and 0 ≤  ≤ 1.
In (10), both time utility and cost utility are between 0 and 1 and  and  are the weights of time utility and cost utility, respectively.
The aim of utility oriented task scheduling is to maximize the total utility, and the constraints are expected time of tasks, expected cost, finished rate, speedup, and so on.In this paper, we classify user tasks into time sensitive and cost sensitive.
For time sensitive user tasks, change of running time for a task will affect the time utility a lot, and its definition is as follows.
Definition 5.The utility model of time sensitive user tasks is defined by the following equations: The constrains are where  is the set of subtasks for all tasks, and the aim is to maximize total utility .
For cost sensitive user tasks, change of running cost for a task will affect the cost utility a lot, and its definition is as follows.
Definition 6.The utility model of cost sensitive user tasks is defined by the following equations: The constrains are >  0 . (28) In both Definitions 5 and 6, their aims are maximizing the total utility , but the differences are the computation of   and   .Based on the above definitions, we propose a utility oriented and cost based scheduling algorithm.The details of the algorithm are as follows: (1) Analyze user type for each user and select computing equations for   and   .
(4) With the results of step (3), tag  schedulers with least waiting time.
(5) Input some data into the  schedulers and set the highest priority for these tasks.
(6) Execute the above tasks, and record the running time and cost (see Pseudocode 1).
(7) Predict running time, cost, and corresponding utility of all tasks with time and cost of results from step 6, and tag the scheduler with the maximal utility.
(8) Schedule tasks in the scheduler with maximal utility, and optimize user utility (see Pseudocode 2).( 9) Wait until all tasks finish, and record the running time, cost, and corresponding utility.There are total 20 computing nodes in our experimental environment, and each computing node starts up a virtual computing node.We start 10 schedulers, and each scheduler manages 2 virtual nodes (computing nodes).The application that we use in the experiments is WordCount.
According to (1) to (3), we computed the service intensity , the expected number of tasks in a scheduler , the expected length of queuing   , the expected finishing time of tasks , and the expected waiting time of queuing   .Figure 2 describes the expected waiting time () on each scheduler.As we can see from the figure, the waiting time from schedulers 1, 3, 5, and 7 satisfied ( 14) and ( 23), and thus we can copy and execute some subtasks (data with size 1 KB) on them.If the user task is time sensitive, then we run the task on node with faster speed; and if the user is cost sensitive, then we run the task on node with lower cost.

Experiments for Time Sensitive User Utility Model.
In order to select the parameters for time utility and cost utility functions, we normalize them first and get the following Based on running time and rate, total time, cost, and utility from schedulers 1, 3, 5, and 7, we set  = 0.7 and  = 0.3 in (10).Under constrains from (12) to (19), we compute the total utility.In Figure 4,   is the predicted time utility,   is the predicted cost utility,   is the predicted total utility,  is the actual total utility, and # is the total utility that we get by rescheduling tasks on the above 1, 3, 5, and 7 schedulers.
In Figure 4, for scheduler 1,   ,   ,   , and  are all the lowest; for scheduler 3,   is the highest,   is much lower, and   is the highest too; for schedulers 5 and 7, although their   is higher than scheduler, their   is lower than scheduler 3.According to the rule of maximizing utility, we should choose scheduler 3 as the scheduler.However, in order to further improve the total utility, we applied the proposed algorithm in Section 3.3.3.By rescheduling the tasks in queuing, we get the actual total utility # for each scheduler.In schedulers 5 and 7, # is much higher than   of scheduler 3.

Experiments for Cost Sensitive User Utility Model.
In order to select the parameters of time utility and cost utility functions for cost sensitive user tasks, we also normalize them and get the following two equations.Figure 5 describes the curves of the following two equations: From Figure 6 we can see that the predicted total utility   in scheduler 5 is the highest, and if we schedule tasks on scheduler 5, we would have the highest actual total utility #.So if user tasks have different time and cost requirements, we can choose different computing nodes to execute them and make the total utility maximal.In addition, after rescheduling the tasks, all tasks have higher actual total utility # than predicted utility   and actual total utility , which validates the effectiveness of our proposed algorithm.

Comparison Experiments.
In this experiment, we selected 10 simulating tasks and compared our algorithm with both Min-Min and Max-Min algorithms.The Min-Min algorithm schedules minimum task to the quickest computing node every time, and the Max-Min algorithm schedules maximum task to the quickest computing node every time.We implemented two algorithms for both time sensitive and cost sensitive user tasks and denoted them as MaxUtility-Time and MaxUtility-Cost.The experimental result is in Figure 7.
In Figure 7, the total utilities of MaxUtility-Time and MaxUtility-Cost algorithms are higher than the other two algorithms and are also stable; both Min-Min and Max-Min algorithms have lower total utilities, and their values fluctuate very much.Both Min-Min and Max-Min algorithms only consider the running time of tasks and ignore requirements of both time and cost, which makes them get lower total utilities and fluctuate very much.In particular, when running tasks 8, 9, and 10, total utility of the Max-Min algorithm drops quickly.The reason is that it schedules long-running tasks to computing nodes with high performance, which makes the utility very low.

Conclusion
In this paper, we introduced utility into the cloud environment, quantified the satisfaction of users to services as utility, and proposed utility oriented queuing model for task scheduling.We classified utility into time and cost utility, rescheduled tasks according to their remaining time, and minimized the total utility by constraints.With the proposed model, we can reschedule remaining tasks dynamically to get the maximum utility.We validated our proposed model by lots of experiments.

Figure 2 :
Figure 2: Expected waiting time for each scheduler.

Figure 3 :Figure 4 :
Figure 3: Time and cost utility lines for time sensitive user tasks.

Figure 5 :
Figure 5: Time and cost utility lines for cost sensitive user tasks.

Figure 6 :Figure 7 :
Figure 6: Utility distribution of cost sensitive user tasks.
Definition 3. Let  = ( , | 1 ≤  ≤ , 1 ≤  ≤ ) be the cost matrix of task   on node   , and then total cost of a task is the product of node cost and running time; that is, Utility of Tasks.In this paper, we assume that the cost rate of nodes is proportional to CPU and I/O speed, and tasks of different types consume different energy, different bandwidth, and different resource usage.So different tasks will have different cost rates.=1(, ×  () ,, ) ,

Table 1 :
if (user task is time sensitive) {select nodes with quickest speed, execute the above tasks, such that  <  0 ; } else { Select nodes with lowest cost, execute the above tasks, such that  <  0 ; } Hardware configuration parameters.