Dynamic Operation Management of a Renewable Microgrid including Battery Energy Storage

In this paper, a novel dynamic programming technique is presented for optimal operation of a typical renewablemicrogrid including battery energy storage. The main idea is to use the scenarios analysis technique to proceed the uncertainties related to the available output power of wind and photovoltaic units and dynamic programming technique to obtain the optimal control strategy for a renewable microgrid system in a finite time period. First, to properly model the system, a mathematical model including power losses of the renewable microgrid is established, where the uncertainties due to the fluctuating generation from renewable energy sources are considered. Next, considering the dynamic power constraints of the battery, a new performance index function is established, where the Lagrange multipliers and interior point method will be presented for the equality and inequality operation constraints. Then, a feedback control scheme based on the dynamic programming is proposed to solve the model and obtain the optimal solution. Finally, simulation and comparison results are given to illustrate the performance of the presented method.


Introduction
Nowadays, renewable energy sources (RESs) such as wind or photovoltaics have become more wide spread due to needs for satisfying the environment concerns.On the other hand, distributed generators (DGs) like diesel engines, microturbines, and fuel cells can be used to enhance the resiliency of power system and yield other social economic benefits.Therefore, renewable microgrid is expected to play an important role in future power systems [1,2].As a key enabling element of renewable microgrids, battery energy storages make microgrid become a strong coupling system in the time domain.In this regard, the methodologies applied to operation management of a renewable microgrid are getting more complicated and challengeable; therefore there is a strong need for more reliable scheduling of energy sources in renewable microgrid including battery energy storage.
So far, many researchers have dealt with the optimal operation scheduling of energy sources in microgrids [3][4][5][6][7].Previously conventional mathematical programming such as Lagrange relaxation [8,9], lambda iteration [10,11], Newton-Raphson [12], interior point method [13], weighted minimax [14], and quadratic programming [15] have been used to determine the least cost solution.However, the conventional mathematical programming methods have major disadvantages such that they can be trapped in local optimal, exhibited sensitivity to the initial starting points.And many of the methods cannot solve the nonsmooth, convex, and nonmonotonically increasing cost functions.Recently, computational intelligence [16,17] and artificial intelligence based nonconventional methods [18] have been used to solve the optimal operation scheduling of energy sources in microgrids.Artificial intelligence based methods such as artificial neural network and computational intelligence methods such as genetic algorithm, particle swarm optimization, harmony search, simulated annealing, differential evolution, gravitational search algorithm, biogeography based optimization, bacterial foraging algorithm, ant colony optimization, cuckoo search, bat algorithm, artificial bee colony, firefly algorithm, and flower pollination algorithm have been used to solve the problem.These methods can enable us to solve the nonlinear and no-convex cost functions and can obtain nearly the global solutions.However, these methods have major disadvantages such as the evolutionary algorithms greatly depending on their parameters and having high computational time.Besides, hybrid methods which combine different algorithms have been used to solve the optimal operation scheduling of energy sources in microgrid.But, these methods usually have long computational time.Also, all previous researchers search the optimal solutions in each separated time periods.In other words, they are static optimal operation scheduling which do not consider the relationships among different time periods.Dynamic optimal operation scheduling problem is another fundamental part of the renewable microgrid operation to maintain the power balance [19][20][21][22][23][24].Among these methods, Cheng et al. [23] have used an enhanced quorum sensing based particle swarm optimization to deal with the dynamic operation optimization problem.However, it is hard to solve the complex optimization problem with high dimension variables and multiple constraints by such metaheuristic methods.Liu et al. [19,22] have used the dynamic programming algorithm for handing the dynamic economic dispatch problem.Generally, dynamic programming can be used to solve nonlinear dynamics when the problems can be discredited with time, state, and decision variable [25].In other words, dynamic programming can be used to solve the problem by dividing the decision process into different stages.Meanwhile, these stages interact and interconnect with decision variables.And the target of the dynamic programming is used to find the decision set over the whole searching trajectory.However, such methods may have some deficiencies handling large scale problems.And a challenging aspect of this method is determining which of the inequality constraints are binding at the solution.Another relevant aspect in renewable microgrid operation management should be coping with uncertainty in the renewable energy sources, load demand, and market prices [24][25][26].Similarly, power line losses should also be considered in the renewable microgrid system.There have been some researches including the power losses in a renewable microgrid [5,27,28].For example, the output power losses of distributed generators are modeled in articles [5,29].The power line losses which are calculated using Bcoefficients are considered in the economic dispatch problems [30,31].
This paper presents several contributions as follows: (1) The power losses of the renewable microgrid are included.It is noteworthy that, from an optimal operation planning point of view, such method can be optimized a renewable microgrid globally.
(2) Uncertainty in the renewable energy resources is considered.A methodology based on scenario analysis is used to proceed the uncertainties related to the available output power of wind and photovoltaic units.
(3) The dynamic performance of the battery storage system is considered.Therefore, this dynamic operation constraint makes renewable microgrid to become a strong coupling system in the time domain.Then, the Lagrange multipliers and interior point method will be presented for these operation constraints.
(4) A feedback control scheme based on the dynamic programming is proposed to solve the model and obtain the optimal solution.The advantage of the scheme is that the number of numerical operations is linear in the number of optimization parameters, which enables one to solve for a smooth input/control history.Meantime, this scheme enables one to initialize the optimization problem with a zero-initial guess.Finally, this scheme is a direct method to obtain the same results.

Description of the Renewable Microgrid
2.1.Renewable Microgrid.In this paper, the renewable microgrid system considers diesel generators, microturbines, fuel cells, wind turbines, photovoltaics, and a battery energy storage system in Figure 1.The renewable microgrid can operate in parallel to the main grid or as an island.

Active Power Balance of the Renewable Microgrid.
In general, the active power balance between electrical energy production and consumption must be met at each time interval expressed as follows [28,33]: where   (),   (),   (),   (),   (),   (),   (),   (),   () are the power set-point for the wind turbines, photovoltaics, diesel generators, microturbines, fuel cells, battery energy storage, power exchange between the main grid and renewable microgrid, controllable distribution generators, and load levels.(  ()) is the power loss of the renewable microgrid.  ,   ,   ,   ,   ,   ,   are the total number of wind turbines, photovoltaics, diesel generators, microturbines, fuel cells, battery energy storage, and load levels.In this paper, the controllable distribution generators (DGs) included diesel generators, microturbines, and fuel cells.

Estimation of Power Losses Coefficients.
When the power losses of the renewable microgrid are considered, these power losses coefficients can be calculated as follows [28,34]: where  = [  ] and  0 and  00 are the power losses coefficients.

Subsystem Model of the Renewable
Microgrid.This paper considers a renewable microgrid containing several types of devices, such as battery energy storages, distributed generators, wind turbines, and photovoltaics.A block diagram is given in Figure 1.

The Model of Battery Energy Storage
System.Battery energy storage can store energy when charging.On the other hand, it can supply energy to the load demand when discharging.
The degradation cost of battery energy storage can be calculated as follows [33]: where   is the cost coefficient which can be determined by battery energy storage charging or discharging power. , () is the permitted rate of charge during a definite period of time and  , () is the permitted rate of discharge.
The following equation can be expressed for a battery energy storage   () =   () ., () +   () ., () , where   () or   () indicate the working mode of the battery energy storage,   (),   () ∈ {0, 1}.Define   () = 1 or   () = 1 if the battery energy storage is charge or discharge.These variables satisfy The permitted rate of charge and discharge are limited by the following constrains: where   , and   , are the maximum charging and discharging rates of battery energy storage.
After charge or discharge, the state of charge of the battery energy storage can be calculated as follows [33]: where  , ,  , are the energy conversion coefficients. , is the energy capacity of the battery energy storage.On the other hand, the state of charge of the battery energy storage must be maintained in the following range.Δ is the operating time of the battery energy storage.
where  , ,  , are required limits of the state of charge.

The Model of Controllable Distributed Generators.
In this paper, the distribution generators (DGs) are included diesel generators, microturbines, and fuel cells.
(A) Diesel Generators.In general, diesel generators serve as a backup power source.Meanwhile, the operating cost of diesel generators is formulated as follows: where  , is the fuel cost of diesel generator;  , is the operating and maintenance cost;   is the start-up cost of diesel generator.The fuel cost of diesel generators is modeled as a function of their actual output power where   () is the actual output power., ,  are the fuel consumption curve fitting coefficients.
The operating and maintenance cost can be expressed as where  , is the operating and maintenance cost coefficient.Δ is the operating time of diesel generator.The start-up cost of diesel generator is relative to its operating state where  ,ℎ ,  , are the hot start-up cost and cold start-up cost. , is the cooling time of diesel generator. , ,  , are the off time and shut down time of diesel generator.For a stable operation, the actual output power of diesel generator is limited by lower and upper bounds as follows: where  , is the minimum active power of diesel generator. , is the maximum power of diesel generator.
The following constraint ensures diesel generators do not exceed their ramp limits: where   is the ramp constraint coefficient.Due to the physical constraints that state shifting can only take place after a fixed time interval, the state variable of two adjacent times is as follows: where  , ,  , are the cumulative uptime and the minimum turn on time.
(B) Fuel Cells.Fuel cell directly converts chemical energy to electrical energy by electrochemical reactions, which is one of the most promising energy conversion technologies.The operating cost of fuel cells are formulated as where  , is the fuel cost of fuel cell;  , is the operating and maintenance cost;   is the start-up cost of fuel cell.The fuel cost of fuel cells is modeled as where   is the fuel price;   is the operating time of changing two adjacent states;   is the consumption with active power for the auxiliary equipment.  is the efficiency of fuel cells.
The operating and maintenance cost can be obtained by where  , is the operating and maintenance cost coefficient.The start-up cost of fuel cells can be expressed as where  ,ℎ is the hot start-up cost of fuel cells;  ,ℎ +   is the cold start-up cost of fuel cells.   is the off time of fuel cells;   is the cooling time of fuel cells.
The actual output power of fuel cell is limited by lower and upper bounds as where  , ,  , are the minimum and maximum active power of fuel cells.
The following constraint ensures fuel cell does not exceed their ramp limits where Δ , , Δ , are the minimum and maximum ramp rate of fuel cells.
The minimum on or minimum down time constraints for fuel cells can be expressed as where  , is the cumulative uptime. , ,  , are the minimum turn on time and shut down time of fuel cells.
The number of turn on and turn off can be expressed as where  ,− () is the number of turn on and turn off in time ;  , is the maximum number of turn on and turn off.
(C) Microturbine.Microturbines have the higher power density, produce less noise, and emit much less pollutants, especially .So microturbines are effective devices of converting the fuel energy into electrical energy.Meanwhile, the operating cost of microturbines are formulated as follows: where  , is the fuel cost of microturbine;  , is the operating and maintenance cost;   is the start-up cost of microturbine.
The fuel cost of microturbines is modeled as where   is the fuel price of microturbines;   is the efficiency of microturbines.The operating and maintenance cost of microturbines can be expressed as where  , is the operating and maintenance cost coefficient of microturbines.The start-up cost of microturbines can be expressed as where  ,ℎ is the hot start-up cost of microturbines;  ,ℎ +  is the cold start-up cost of microturbines.   is the off time of microturbines;   is the cooling time of microturbines.
The actual output power of microturbines is limited by lower and upper bounds as follows: where  , ,  , are the minimum and maximum active power of microturbines.
The following constraint ensures microturbines do not exceed their ramp limits where Δ , , Δ , are the minimum and maximum ramp rate of microturbines.The minimum on or minimum down time constraints for microturbines can be expressed as where  , is the cumulative uptime;  , ,  , are the minimum turn on time and shut down time of microturbines.

The Probability Model of Wind Turbines.
Wind powers as renewable energy sources (RESs) are dependent on numerous factors such as wind velocity and efficiency.The approximate relationship between the wind power and wind speed can be expressed as where V  , V  , V  are the cut-in wind speed, cut-out wind speed, and rated wind speed.  is the rated output power of wind turbines.
Prior research has shown that the wind speed profile follows the Weibull distribution over time, which is [29,32] where positive variables  and  are the scale parameter and shape parameter, respectively.Based on the characteristic of the wind power (34) and the probability distribution function (35), the probability distribution function of wind power is obtained by where   is the array surface area;  is the efficiency of the photovoltaic in realistic reporting conditions;   is the irradiance on a surface with inclination  to the horizontal plane; ,   are the parameters that depend on inclination , reflectance of the ground , etc.
where   is the ratio of beam radiation on the tilted surface to that on a horizontal surface at any time and  0 is the solar radiation, for that day, both referring to a horizontal surface;   is ratio, diffuse radiation in hour or diffuse in day; ,  are the parameters of the daily clearness index   .Many researches have proved that cloudiness is the main factor affecting the difference between the values of solar radiation measured outside the atmosphere and on earthly surface.The daily clearness index can be obtained by [36] where   ,  0 are the ratio of the irradiance on horizontal plane and the extraterrestrial total solar irradiance.The effect of clouds on terrestrial irradiance is the daily clearness index;   can not be predicted with complete confidence; it must be treated as random variable where   ,   are the lower and upper bounds of the observed range for   ; ]  is the parameter of daily clearness index.
where ,  1 ,  are the parameters of daily clearness index.

The Model of Interaction with External Main Grid.
In this paper, the renewable microgrid is connected to the external main grid and can trade energy with the main grid.The transaction incurs the following cost to the renewable microgrid:

Start
Read system date and generate 1000 scenarios.Use the backward scenario reduction to reach the desired number of scenarios .
For scenario 2, the dynamic operation model of renewable micro grid is established by ( 52)-(56).And the standard formulation of dynamic operation model is formulated as (57)-(61).
The sequential quadratic programming is presented for the formulation(52)-(61), then the equivalent formulation with only linear and quadratic terms is obtained as (67)-(74).
The Lagrange multiplier and slacking variables are used to proceed the equalities and inequalities operation constraints, and the new formulation is obtained as (75)-(77).
The feedback control scheme based on dynamic programming which is expressed as (78)-( 94) is proposed to solve the model and obtain the optimal solution of the sub problem.
A methodology based on scenarios and policy aggregation is proposed to identify the trends and essential features on the solution of the sub problems.Then, the finally optimal solution is obtained.

End
For scenario 1, the dynamic operation model of renewable micro grid is established by ( 52)-(56).And the standard formulation of dynamic operation model is formulated as (57)-(61).
The sequential quadratic programming is presented for the formulation(52)-(61), then the equivalent formulation with only linear and quadratic terms is obtained as (67)-(74).
The Lagrange multiplier and slacking variables are used to proceed the equalities and inequalities operation constraints, and the new formulation is obtained as (75)-(77).
The feedback control scheme based on dynamic programming which is expressed as (78)-( 94) is proposed to solve the model and obtain the optimal solution of the sub problem.
The sequential quadratic programming is presented for the formulation(52)-(61), then the equivalent formulation with only linear and quadratic terms is obtained as (67)-(74).
The Lagrange multiplier and slacking variables are used to proceed the equalities and inequalities operation constraints, and the new formulation is obtained as (75)-(77).
The feedback control scheme based on dynamic programming which is expressed as (78)-( 94) is proposed to solve the model and obtain the optimal solution of the sub problem.
For scenario Ns, the dynamic operation model of renewable micro grid is The flowchart of the proposed improved dynamic programming algorithm.
Let the amount of energy be bounded by where  , is the maximum transaction limits;   ,   are the price coefficients to purchase and sell energy.

Dynamic Operation Model of the Renewable Microgrid.
Dynamic economic operation management of the renewable microgrid is to determine output power of distribution generators, in order to minimize total operation cost of the renewable microgrid and meet the dynamic operation constrains.
The total operating cost of the renewable microgrid can be defined as where () is the decision or control variables (specifically, action at state () in period ); () is the state variables of the renewable microgrid; () is the random variable such as the output power of wind turbine or photovoltaics.

Solution Methodology
Based on Section 2, dynamic operation management of the renewable microgrid can be regarded as a discrete time system under uncertainty.In order to solve the proposed problem, the random variables can be realized by the scenario analysis technique.Then, a feedback control scheme based on the dynamic programming is proposed to solve the model and obtain the optimal solution.The flowchart of the whole process is given in Figure 2.

The Scenario Analysis Technique.
In this paper, a discrete set of scenarios can be used to represent the probability realization of the output power of wind turbines or photovoltaics.On the other hand, these scenarios are generated using the Roulette wheel mechanism and Monte Carlo simulation method [19].And the probability distribution function of the random variables can be obtained by ( 36) and ( 47 V  ) 1,   = 1, . . . ,  ;  = 1, . . . ,, (62 where  is the number of random variables;   is the number of random sampling; V  is vector with dimension . According to the desired preciseness, the probability distribution functions (36) and ( 47)-( 48) are divided into   class intervals.Each class interval determines mean value   ,  = 1, . . .,   ; and each interval is associated with a probability denoted by   = 1, . . .,   .
Meanwhile, the probabilities of different intervals are normalized in which their summations become equal to unity.
Therefore, each scenario comprises a vector identifying the output power of wind turbine or photovoltaics: where     ,, ,     ,, are binary parameters indicating where   wind power interval or photovoltaic power output whether are selected in scenario.On the other hand, comparing the random number which follows the LMCS strategy and the probability   , the binary parameter can be selected.
Thus, the output power of wind turbine or photovoltaics for each scenario can be obtained by The normalized probability of each scenario can be expressed as follows: where   is the number of scenarios.

The Feedback Control Scheme Based on the Dynamic
Programming.In the following section, we formulate the dynamic operation management of renewable microgrid which satisfies dynamic operations constraints.Firstly, the Lagrange multipliers and interior point method will be presented for the equality and inequality operation constraint.Then, a feedback control scheme based on the dynamic programming is proposed to solve the model and obtain the optimal solution.

Interior Point Methods.
According to the sequential quadratic programming formulations (67), by applying the Lagrange multipliers and slacking variables to proceed the equalities and inequalities operation constraints, a new formulation is shown as where  is considered as the kth iteration.V  () is a constant.
Meanwhile, the parameters can be defined as follows: The approach in solving (z, ξ, λ, s) is given in Appendix B.

Total Algorithm Procedures.
In this paper, an algorithm is presented based on an adaptation of Mehrotra's predictorcorrector algorithm [37].Given the initial parameters  > 0,  = 0 and ( 0 ,  0 ,  0 ,  0 ) the residual terms are defined as [min (   () ,    ())] . (81) As the solution set approaches optimum point, the summation of all residual term should approach zero.
The total algorithm can be expressed as follows.

Mathematical Problems in Engineering
Step 2. Calculate the affine-scaling direction {z  , ξ , λ , s } by using the method introduced in Appendix B.
The scalars   ,   , and  can be calculated as follows: where  2 is the dimension of   .

Simulation
In this section, the renewable microgrid in the simulation is shown in Figure 1; it operates in parallel to the main power grid or as an island and comprises wind turbines with maximum power of 13kw, PV panels with maximum power of 15kw, and distributed generators such as microturbines, diesel generators, and full cells.A battery energy storage is included, bounded between 20 and 60 kWh and with maximal charge and discharge rates, respectively, 30 and -30kw.The charge and discharging efficiencies are both equal to 0.85.Tables 1-4 describe the distributed generators units parameters, based on data provided in [29,32].The simulations are carried out in the MATLAB environment on an Intel Core 2 Duo 3.00GHz running Windows 7. In the simulations study, we chose the sampling time of 1 hour.And simulations are performed over a horizon of 24 hours.The electricity usage, wind power, and solar generation data in the simulations have been provided by National Renewable Energy Laboratory.The electricity usage data is from the utility operator customers with peak usage over 90kw; the maximum power demand of load is 90.1kw.The daily electricity usage (from Miami, FI, on a certain day) is shown in Figure 3.

Scenario Generation and Reduction
Results.The approach used for generation of scenarios may have an impact on the determination of operational cost of renewable microgrid.In the study case, the dates from wind power profiles of the   management of renewable microgrid.The scenarios of wind power and photostatic chosen for this study are shown in Figures 4, 5, and 6.

The Base Case of the Dynamic Operation Management of Renewable Microgrid.
In this part, its assumed that the renewable microgrid is disconnected from the main power grid.The active power of load is supplied by the microgenerators such as microturbines, diesel generators, fuel cells, wind turbines and photovoltaics, and a battery storage system.The optimal results of the dynamic operation management of renewable microgrid are given by Figures 7 and 8.

Modes
The base case The second case Cost ($) 1.8636e+03 1.83677e+03 Figures 7 and 8 show that microturbine and fuel cell have to generate more electricity power than the diesel generators.Because the diesel generators are expensive, they are restricted to their minimum value during most hours.

The Case of the Renewable Microgrid Operates in
Parallel to the Main Power Grid.In the second case, its assumed that the renewable microgrid can operate in parallel to the main power grid.When the renewable microgrid is connected to the main power grid, its loads receive power from both the main power gird and the local distributed energy resources such as solar and wind power, distributed energy generators, and a battery storage system.The optimal results of the dynamic operation management of renewable microgrid are shown in Figures 9 and  10.
It should be noted that, in Figures 9 and 10, the main power grid has to generate more electricity power.Fuel cell and diesel generators are more expensive.
In Table 5, it is worthwhile to note that the total operation cost of second case is the lowest.In other word, the renewable microgrid which operates in parallel to the main power grid decreases the operation cost because of its loads receiving power from both the main power gird and the local distributed energy resources.

The Case of Dynamic Operation Management of Renewable Microgrid under Uncertainties.
In this case, a methodology based on scenarios analysis is used to proceed the   uncertainty of the power output of distributed renewable generators, such as wind power and photovoltaics power and it is assumed that the renewable microgrid is disconnected from the main grid.The optimal results of the dynamic economic operation optimization of renewable microgrid are shown in Figures 11 and 12.
In Table 6, it is worthwhile to note that this scheme is a direct method to obtain the optimal solution without probability.And the total operation cost of the method of proposed by this paper is the lowest.In other words, modeling the renewable microgrid system under uncertainty decreases the operation cost.This is because of considering different scenarios in the stochastic model instead of single scenario in the deterministic scheme.

Conclusions
In this paper, we introduce a novel dynamic operation management of renewable microgrid.First, to properly model the system, a mathematical model including power losses of the renewable microgrid is established.Then,   where Φ () ∈    (). () , Φ () ∈    ().  () .The modification of the dynamic programming algorithm to solve the problem can be obtained by the following.

Figure 3 :
Figure 3: Load profiles for a period of one day.

Figure 4 :
Figure 4: Renewable energy (wind power) scenarios for a period of one day.

Figure 5 :
Figure 5: Renewable energy (photovoltaics) scenarios for a period of one day.

Figure 6 :
Figure 6: The probability of renewable energy scenarios for a period of one day.

Figure 7 :
Figure 7: Optimal results of economic operation optimization of renewable microgrid.

Figure 11 :Figure 12 :
Figure 11: Optimal results of economic operation optimization of renewable microgrid under uncertainty.

Table 5 :
The comparison of the results obtained by different modes for smart microgrid.