As the scale of the data centers increases, electricity
cost is becoming the fastest-growing element in their operation
costs. In this paper, we investigate the electricity cost reduction
opportunities utilizing energy storage facilities in data centers
used as uninterrupted power supply units (UPS). Its basic idea
is to combine the temporal diversity of electricity price and the
energy storage to conceive a strategy for reducing the electricity
cost. The electricity cost minimization is formulated in the
framework of finite state-action discounted cost Markov decision
process (MDP). We apply

Cloud computing is an emerging Internet-based computing paradigm which offers on-demand computing services to cloud consumers. To meet the increasing demands of computing and storage resources in cloud computing, there is an increasing trend toward large-scale data centers. As more data centers are deployed and their scale increases, energy consumption cost is becoming the fastest-growing element in their operation costs, including the computing energy cost, cooling energy cost, and other energy overheads. It has been estimated that energy consumption cost may amount to 30%–50% percentage of operation cost of large-scale data centers built by companies such as Google, Microsoft, and Facebook [

As we know, electricity cost generation depends not only on the total amount of energy consumed by the data centers, but also on the electricity price. Therefore, the electricity price is also an important factor in the electricity cost of data centers. With the development of smart grid technology which is a technology for the next generation power grid, more and more electricity markets are undergoing deregulation where the electricity market operators offer dynamic electricity rates to large industrial and commercial customers instead of traditional flat rates at the retail level. Thus, there is an opportunity for us to achieve the electricity consumption cost saving in data centers by observing and utilizing the time-varying electricity price in the deregulated electricity markets.

Normally, the UPS units may be deployed in data centers, and provide emergency energy to power them up using stored energy before the backup diesel generators (DG) can start up and operate as a secondary power source when the main power system experiences an outage. Usually, the transition from the main power system to the secondary power source takes 10–20 seconds. As an improvement of the rechargeable battery, the UPS units have enough energy storage capacity for keeping a data center working 5–30 minutes at its maximum power demand [

Based on the above two facts, the basic principle for achieving the electricity cost saving is recharging the UPS units residing in the data center when the outside electricity price is low and discharging for powering the data center when the outside electricity price is high. Hence, this paper focuses on a dynamic energy storage control strategy for reducing the electricity cost of the data centers. Dynamic energy storage control is expected to adapt the fluctuation of the electricity price and the workload by dynamically making recharge/discharge decisions for the UPS units. It aims for achieving substantial electricity cost saving without performance degradation.

In this paper, we formulate the electricity cost reduction problem utilizing energy storage facilities as the discounted cost Markov decision process. Since the statistical information about the workload arrival and the electricity price is not available, we propose an online algorithm based on

The problem of electricity cost minimization in data centers with energy storage facilities for time-varying electricity prices under deregulated electricity markets is modeled by a discounted cost Markov decision process, which achieves the cost saving by making decisions to recharge/discharge the battery.

In order to solve the optimization problem, we propose a dynamic energy storage control strategy based on the

We formulate an offline optimization problem of electricity cost minimization for obtaining the optimal offline solution as the lower bound on the performance of the online and learning theoretic problem. The offline optimization problem is solved by mapping it into a tractable mixed integer linear programming instead of nonlinear programming.

Finally, the experiments are carried out based on real workload traces and electricity price data sets to show the performance of the proposed scheme. By using the real traces that may not provably follow the Markovian assumption, the result also shows that the proposed scheme generally performs well.

The rest of the paper is organized as follows: in Section

The severe energy consumption problem in data centers has motivated many works on reducing their electricity cost. These works may be roughly categorized into two basic types of mechanisms: (1) reduce the energy consumption or improve the energy efficiency of the data centers; and (2) exploit the temporal and geographical variation of electricity prices to achieve the electricity cost saving.

Regarding the first mechanism, new hardware designs and engineering techniques such as energy-efficient chips, multicore servers [

At the data center level, dynamic cluster reconfiguration (DCR) [

The second mechanism for reducing electricity cost relies on the fact of the notable temporal and geographical variations in electricity prices. In [

In this section, we describe system architecture model for energy management in data center, present the models for battery, energy consumption, and electricity cost, as well as formulating the problem of dynamic energy storage control to minimize the expected total electricity cost.

A general system architecture model for data center with energy storage facilities, depicted in Figure

Energy management framework using energy storage facilities in data center.

The basic running process of EMS can be generally described as follows. IC periodically collects the battery level information as well as the electricity price information from the grid. The data center submits its energy demand information to IC, and ESMU uses this information to make the decision that the energy supply draws from grid or the battery. Finally, the data center can provide services using the energy managed by EMS.

In this subsection, we introduce the time-slotted system model used in this paper, and the time is divided into slots of equal duration of

From [

The energy market usually consists of Day-Ahead market and Real-Time market [

In current data centers, UPS units use lead-acid batteries typically. There are several characteristics of battery operation when using a lead-acid battery practically. For a given battery, each recharge-discharge cycle has energy loss due to AC-DC conversion, so the battery may not be completely efficient, and its performance is affected by the recharge efficiency

Let

According to inequality (

Let

Thus, the total amount of energy drawn from the grid in the slot

Define

In this paper, the goal of dynamic energy storage control is to minimize the expected total electricity cost in the data centers with energy storage facilities. Based on the above models, the problem can be formulated as follows:

According to (

In data center, the lower-level management routines like server consolidation and instantiation of new VMs may be executed. Different management routines may have different demand profiles of energy consumption. But once the lower-level management routine is given, the demand profile for the workload is determined and can be mathematically modeled. Hence, we can still apply the above mentioned model to achieve the electricity cost saving.

In this section, we will map the problem (

The energy management system in data center, as described above, can be formulated as a finite-state discrete-time MDP. In the model, let

As described in Section

Being in search of an optimal policy, the decision maker needs a facility to differentiate the desirability of possible successor states, in order to decide on the best action. A common way to rank states is by computing and using a so-called state-value function which estimates the expected discounted sum cost when starting in a specific state

Similarly, define the value of taking action

For finite MDPs, an optimal policy can be precisely defined in the following way. A policy

As seen from the above analysis, in order to minimize the expected total electricity cost, we can obtain the optimal policy by learning

In this section, we will introduce learning theoretic algorithms, namely,

According to (

observe the current state

choose action

observe the next state

update the estimate

Based on the above discussion, the estimate

Although it has been shown that the sequence

In the ASQL algorithm, let

By applying the proposed scheme, we can obtain the optimal energy storage control policy using storage facilities in data centers for electricity cost minimization. The more detailed procedures of the proposed scheme are presented in Algorithm

(1)

Initialize

Initialize learning counter

Initialize starting state

(2)

Decide to explore/exploit action with probability

Choose action

choose action

Take action

Observe the next state

Receive an immediate cost

Calculate

Calculate

Update the

+

+

Update the current state

Update learning counter

In this section, we give a lower bound on the performance of the learning theoretic problem by the optimal offline solution, which is employed as a benchmark to evaluate the optimality of the proposed learning theoretic algorithm. In order to formulate the offline optimization problem, we assume that all the future workload arrivals as well as the electricity price variations are known noncausally before the decisions of energy storage control are made. This information can be obtained from the traces of the workload and electricity price in advance. Online learning theoretic problem optimizes the expected total electricity cost over an infinite horizon while the offline solution does that over a realization of the MDP for a finite number of time slots. As previously described, an MDP realization is a sequence of state transitions of the workload, the battery energy level and the electricity price state processes for a finite number of time slots. Hence, we can optimize

From definition (

Let us define the following variables regarding recharge and discharge operations in the slot

From (

In this section, the performance of the proposed dynamic energy storage control scheme is characterized quantitatively. Real-world workload traces and electricity price data sets are employed to evaluate the performance of the proposed scheme. In the following, we elaborate on the design of the experiments and presenting the experimental results.

In the experiments, we simulated a cloud-scale data center which hosts up to

The real workload request is extracted from trace data gathered from Intel Netbatch Grid in 2012 [

Workload arrival patterns of Intel Netbatch Grid for four days.

We use real-time electricity prices at Houston obtained from the Electric Reliability Council of Texas (ERCOT), and the real-time electricity prices vary on a 15 min basis [

Real-time electricity prices at Houston from January 3 to January 6, 2013.

In the experiments, we simulated a time slotted system with slot duration of 15 minutes, that is,

In order to demonstrate the performance improvement of the proposed dynamic energy storage control algorithm, we considered the Lyapunov optimization algorithm [

In the first experiment, we intend to investigate the convergence rate and performance improvement of the proposed scheme using the real-world workload and electricity price traces. Let

In Figure

Expected total discounted electricity cost with respect to the number of iteration learning

Figure

Long-run average electricity cost with respect to the number of iteration learning

In this subsection, we further carried out an experiment in order to investigate the impact of the battery capacities of data centers by setting

In Figure

Expected total discounted electricity cost for different

Figure

Long-run average electricity cost for different

In this paper, we investigated the problem of electricity cost minimization of data centers using energy storage for time-varying electricity prices under deregulated electricity markets, which was formulated as a discounted cost Markov decision process. A dynamic energy storage control strategy based on the

The authors declare that there is no conflict of interests regarding the publication of this paper.

This work was supported by National “863” Project of China (no. 2012AA050802), the Fundamental Research Funds for the Central Universities (WK2100100021), and National Natural Science Foundation of China (no. 61174062).