Joint Scheduling Optimization of Virtual Power Plants and Equitable Profit Distribution Using Shapely Value Theory

The installation capacity of wind and solar photovoltaic power is continually increasing, which makes renewable energy grid connection and power generation an important link of China’s power structure optimization. A virtual power plant (VPP) is an important way to help distributed energy resource grid connection and promote renewable energy industry development. To study the economic scheduling problem of various distributed energy resources and the profit distribution problem of VPP alliance, this study builds a separate operation scheduling model for individual VPP and a joint operation scheduling model for VPP alliance, as well as the profit distribution model. The case study verifies the feasibility and effectiveness of the proposed model. The sensitivity analysis provides information about VPP decision-making in accordance with the policy environment development trend.


Introduction
With the increasing pressure of energy shortage and environmental pollution, renewable energy is strongly supported by the government of China.By the end of 2016, in China, wind power's total installed capacity had reached 169 million kW, ranking the first in the world [1,2].Because of the high transmission cost and backup service quality requirement, distributed generation (DG) has better flexibility than largescale centralized power generation.Distributed wind power and distributed solar photovoltaic power are two types of renewable DG, with the most mature technology and maximum commercial value in China, and have a broad prospect for development [3].However, wind and solar photovoltaic power generation are significantly influenced by environmental and climatic factors, which makes their output strongly volatile and uncertain and has a large influence on power grid scheduling and operation [4,5].The research conducted on this problem has mainly focused on power system day-ahead scheduling with wind and solar photovoltaic power grid connection [6,7] and ancillary service (AS) decision [8,9].
A virtual power plant (VPP) is widely studied for solving distributed renewable energy grid connection problems [10][11][12][13].A VPP can be regarded as a power resource coordination management system that combines and optimizes the energy storage system (ESS), controllable load, electric vehicle, and other distributed energy resources (DERs) based on advanced information and communication technology.It acts as a special plant that participates in power market and power grid operation [14], helps DERs realize grid connection, and improves power grid renewable energy consumption ability.Thus, a VPP is an effective system for improving energy efficiency and promoting renewable energy development.
The existing literature on VPPs has been categorized into 10 types in [15].The literature that is closely related to the theme of this paper can be categorized into the following four types: (1) Scheduling problems associated with the objective function and solution: For a VPP, the main purpose of schedule optimization is to maximize its profit.The specific objective function and constraint change with the VPP structure and different DER generation characteristics [16][17][18].The corresponding solutions for the models are based on linear [19,20] or nonlinear programing [16,21], mixed integer programming [22,23], dynamic programming [24], and decision theory [25][26][27].Multiobjective problems and the methods for solving them have also been studied [28,29].Most of the scheduling optimization research deals with day-ahead (24 h) scheduling problems to verify the effectiveness of the proposed models and the promotion effect on energy efficiency [30].Some bilevel day-and hour-ahead short-term (ultrashort-term) research [6,31] pursuits higher scheduling accuracy and reduces system peak-valley shifting and backup service requirement.Long-term research mainly aims to find the optimal ESS capacity configuration and operation scheme [32,33].
(2) Scheduling problems associated with power generation forecast and uncertainty simulation: For a VPP, uncertainty originates from power generation, load demand, and load price [15].In the power generation section, some models have been proposed to reduce wind and solar [29,34,35] power generation forecasting error.Stochastic formulation [28,29] has also been used to simulate power generation.Other literature deals with load forecasting [36,37] and market price forecasting [38,39].
(4) Scheduling problems associated with cooperation and profit distribution: The related research mainly focuses on the cooperation and joint scheduling of DER-based VPPs with the distribution network [24].The coordination control of VPPs based on a multiple-agent system has been widely studied [44,45].Reference [46] studied the VPP cooperation profit distribution problem based on the conditional value at risk and stochastic dominance constraints.
According to the above literature review, the optimization scheduling problem of a VPP has been intensively studied.The optimization model varies in objective function, constraints, solutions, and DR types.However, VPP joint scheduling and profit distribution problems have been rarely studied.This study considers the influence of government policies on system profit when building the objective function and builds an operation optimization model for VPP alliance.Currently, TOU and IL are the two DR types most widely implemented in China.DR, including TOU and IL, is applied in the model, and the Shapley value method is chosen to distribute profit to each VPP alliance member according to their marginal contribution.The main contributions of this paper are summarized as follows: (1) The optimization models for VPP operation are proposed, with the objective function of maximizing system profit.In the objective function, this paper takes government subsidy policies into consideration, to discuss its influence on VPP profit.(2) This paper studies VPP alliance operation and profit distribution problems.A profit distribution method based on the Shapley value theory is proposed, where the profit is distributed among the VPPs according to their contribution to system profit, and the solving steps are introduced.(3) Two case studies with six operation scenarios are conducted to verify the feasibility and effectiveness of the proposed scheduling model.The profit comparison shows the effectiveness of the proposed profit distribution method, and the sensitivity analysis discusses the influence of TOU price and government subsidy policies on VPP alliance profit.
The rest of this paper is structured as follows.Section 2 introduces the structure of a wind-solar-ESS VPP system considering IL response, which is subject of this paper.And then operation models of wind power, solar photovoltaic power, energy storage system, and interruptible load are introduced, as the basis of building scheduling model of both separate operation and joint operation.Based on Section 2, 3 and 4 build corresponding scheduling optimization models for VPP and VPPs alliance.Section 3 builds a separate scheduling model for the VPP considering IL, which sets maximizing VPP profit as the objective function and considers grid transaction income, reserve cost, load supply income, government subsidies, and IL cost.The model constraints include both operation constraints of VPP components and real-time power balance constraint.Section 4 presents a joint scheduling model built for VPPs alliance.A profit distribution model is built according to the Shapley value method in Section 4. The solution for the proposed model is also introduced in this section.Section 5 presents a case study conducted to verify the proposed models and analyzes the scheduling results of separate operation and joint operation, as well as the profit distribution result.The sensitivity analysis in Section 5 changes the government subsidy prices on renewable energy and TOU price to show their influence on total profit.Section 6 concludes this study.

Modeling for VPP Components
Figure 1 shows the structure of a VPP system with a control center, which contains a distributed wind power plant, solar photovoltaic power, an ESS, a load, and an IL.The solid line in the figure indicates the power flow, while the dashed-dotted line indicates the capital flow and information flow.
In this structure, the VPP control center can exchange the pretreatment information with distributed wind power generation, distributed solar photovoltaic power generation, ESS charge/discharge, load, distribution network, and electricity  market, and a stable and economical operation of the VPP can be achieved by controlling the output of the DERs.
The VPP control center collects the incomes, pays for the costs, and allocates the profit to wind power, solar photovoltaic power, ESS, and IL.As shown in Figure 1, this paper assumes that the initial investment of the facilities in a VPP is fixed, the electricity price between the grid and VPP system is a TOU price, the electricity price between the VPP system and internal load is fixed, and the IL price and government subsidies on renewable energy power generation are fixed.According to the figure, the VPP system must make maximum use of the ESS charge/discharge and IL flexibility to adjust the VPP output and weaken the impact of uncertainty.

Wind Power and Solar Photovoltaic Output Simulation
Model.This paper uses the scene simulation and reduction method to simulate the uncertainty of wind power and solar photovoltaic output [33].The output prediction correction function of wind power and solar photovoltaic power generation is expresses as We then use the interval method to simulate wind power and solar photovoltaic output.The output distribution is divided into multiple intervals, and the point value of an interval is chosen to be the expected value of the interval.When the number of partitions is large enough, the predicted output value is regarded the same as the real value.The output power is set to exist in three states-high, medium, and low-and   = 1, 2, 3 indicates the three output states.Assuming that the probability of state z is    , the output power combination of each scenario is  = {   |}  = 1, 2, . . ., , where T is the set length of a scene.Assuming the output scene probability to be  = ∏ ∈    , the wind power and solar photovoltaic power output scene is obtained.
To simplify the calculation, the number of wind power and solar photovoltaic output scenes needs to be reduced.In this paper, minimizing the Kantorovich distance between the initial and reduced scenes is set as the reduction objective.Let us assume that  = [  ] × indicates the initial scene set, where N is the number of initial scenes.  is the initial scene m at t time, and the probability of is the scene set.Then, the Kantorovich distance between the initial scene set P and reduced scene set Q is calculated as Assuming that the initial scene set P is reduced to scene set Q, with J being the deleted scene set, then the Kantorovich distance between P and Q is expressed as To calculate the probability   of scene ξ ( ∉ ), we consider the probability of   to be equal to the summation of the probability of this scene in the initial set and that of the recently deleted scene in the initial scene set.The deleted scene probability   can be calculated as Assuming D to be the number of scenes that need to be deleted, the scenario reduction can be obtained as The settings of the deleted scene number influence the reduction result.To obtain a reasonable deleted scene number, this paper uses the maximum scene reduction strategy.The scenes that are not far from o in the Kantorovich distance are considered as the reference scenes: Equation ( 6) requires that the similarity between the reserved scene set and the initial scene set is within the required range.

ESS Operation Model.
The most important feature of ESS operation is the state of charge (SOC) of the battery.SOC is calculated as To ensure the service life of ESS, SOC and charge/ discharge power should be restricted within a certain range.Then, the constraints of SOC and charge/discharge are described as  [26].Ignoring the influence of advance notice time, the IL in a VPP system, k, at time t is expressed as Simultaneously, IL scheduling should meet the IL constraints, interruption continuous-time constraints, and the total interruptible time in the contract period constraints:
The VPP operation should meet the constraints of each component, namely, (1)- (10) The power balance constraint can now be expressed as The calculation methods of the above economic factors are based on the same principle as that of the separate scheduling mode (( 12)-( 18)).Then, the joint scheduling mode can be expressed as: max  s.t.equation (1) - (10) equation (22) .(23) The proposed models could be solved by mixed integer nonlinear programming method and the simplified solution has been described in the existing literature [6].

Profit Distribution Based on Shapley
Value.The Shapley value was proposed by Shapley LS in 1953 and is used to solve the profit distribution problem in multiparty cooperation.Let us assume that several entities are involved in an economic activity and that each type of cooperation portfolio gains some profit.If the entities' economic activity is nonconfrontational, the increase in the number of entities that participates in the cooperation would not reduce the total alliance profit.In addition, the total profit reaches the maximum when all entities join the alliance.Then, the Shapley value method provides a profit distribution scheme for the alliance according to the marginal contribution of the members' entry to the total profit.Then, the average profit distribution phenomenon is effectively circumvented and the members gain their profit according to their value to the alliance.For an alliance member, the bigger the contribution to the alliance is, the more the profit distributed from the alliance is.
The cooperation among VPPs meets the above assumptions of the Shapley value.Multiple VPPs' joint scheduling profit is not less than the summation of their individual operation profits.Therefore, this paper applies the Shapley value method to the VPP alliance profit distribution problem.The contribution value of each VPP is a weighted value of several alliance portfolios probability and VPP profit in the portfolios.
Let us assume that  = {1, 2, . . ., ℎ} is the set of VPPs and that, for each member in , there is a real value function V(), where V() meets Then, [, V] is a cooperation measure of h VPPs.V is the characteristic function and, in this model, equals the objective function value.
is the income of member  ( ∈ ) gained from the cooperation portfolio with maximum profit.Then, under the cooperation profit , the profit distribution is  = ( 1 ,  2 , . . .,  ℎ ).This cooperation must meet the following constraints: where (25) indicates the relation between distributed profits and total profit and (26) indicates that the distributed profit should not be less than the original profit of the alliance member.
The profit   (V) that member l gains under  cooperation profit can be calculated as

Model Solving.
The steps for solving the scheduling model and profit distribution model are drawn in Figure 2 and are listed as follows: (1) Generate wind power and solar photovoltaic power output scenes.Case 1 (separate operation case).The VPP control center optimizes the wind power, solar power, and ESS output to reduce the power purchase cost and increase the power sale income separately.
Case 2 (operation in alliance).The VPPs form alliances and set maximizing the total profit as the optimization objective.The alliance can be formed between two VPPs or among three or more VPPs.The total profit would be distributed to the alliance participants according to the Shapley value method.

Basic
Data.An IEEE 33-point system is chosen as the simulation system.The daily internal load of the VPP is simulated based on a typical daily load curve [6].Three VPPs are considered the study subject, which contain wind power of 1.5, 3.5, and 1.5 MW; solar photovoltaic power of 2.5, 3.5, and 3.5 MW; and ESS power of 3, 3, and 4 MWh.Hereinafter, the three VPPs are represented as V1, V2, and V3.According to the wind and solar power output forecast and scene reduction method, 20 typical scenes of the three VPPs are obtained as the basic data.The IL capacity and maximum continuous time are listed in Table 1, and other parameters are listed in Table 2, where the symbol "C" refers to the storage capacity of ESS.

Scheduling Results.
Case 3 (separate scheduling result).Choosing V1 as a representative of the separate scheduling result and feature of the three VPPs, the output in 24 h of V1 is drawn in Figure 3 and the ESS charge/discharge and IL situations are drawn in Figure 4.According to Figure 1, the wind and solar photovoltaic power mainly produce output from 0:00-16:00, especially during 7:00-16:00; in contrast, the renewable power output from 16:00-23:00 is relatively small.The adjustable output, namely, ESS discharge and IL, is called at 7:00-10:00 and 16:00-20:00, where ESS discharge is called in both float load period and peak load and IL is called in the peak load period (10:00 and 18:00-20:00).
The "grid" in Figure 1 indicates the power provided by the distributed power grid; when its value is greater than zero, V1 purchases electricity from the distributed power grid and, otherwise, sells electricity to the distributed power grid.In the separate scheduling case, V1 can meet the internal    load demand and sell electricity to the distributed grid at 10:00 and during 12:00-14:00, while in other times, V1 cannot meet the internal load demand and purchases electricity from the distributed power grid.Although the TOU price during 15:00-20:00 is very high, the VPP system does not have enough adjustability to reduce the electricity purchase cost.
ESS is restrained by SOC, maximum charge and discharge power, and the climbing constraints, so the adjustability is limited.According to Figure 4, during peak load periods, ESS is only called to discharge power during 16:00-18:00 and charge power during the float load period, which is not an ideal scenario.Most of the times, 24 h ESS can meet the peakvalley shifting requirement.Therefore, IL is called to adjust the total output at 10:00 and during 17:00-20:00, which is reasonable.
Therefore, combining Figures 1 and 2, in the separate scheduling mode, the renewable power output is not enough to meet the load during 15:00-9:00 and at 11:00, and because of the TOU price, the VPP system must increase the power output and decrease the electricity purchased from the distributed power grid during peak load periods.However, the adjustability of ESS and IL is limited and the scheduling result is far from ideal.
The operation cost and income of the three VPPs in the separate scheduling case are listed in Table 3, and the operation profits of the three VPPs are calculated.Neither V1 nor V3 can gain profit from the grid transactions, and only V2 gains a little profit from the grid transactions.However, due to the uncertainty of the renewable output, V2 has the largest renewable installation capacity and, correspondingly, bears the maximum reserve cost.For adjustable power, ESS is mostly called in V2, while V2 has the same ESS capacity as that of V1.In general, in the separate scheduling mode, the operation result of V2 is better than those of V1 and V3 and the lack of adjustability significantly limits VPP profits.
Case 4 (joint scheduling case).In this case, the joint scheduling involves cooperation between two VPPs and that among three VPPs.This paper chooses the cooperation among three VPPs as a representative of the operation characteristics and compares it with the separate scheduling case.The output in 24 h of the VPP alliance is drawn in Figure 5, and the ESS charge/discharge and IL situations are drawn in Figure 6.
According to Figure 5, the overall output changes significantly when compared to the separate scheduling case, which is reflected in the "grid" curve.The VPP alliance can meet the internal load and sell electricity to the distributed power grid during 9:00-16:00, which contains the first peak load period  and part of the second peak load period of the day.During other times, the VPP alliance still purchases electricity from the distributed power grid, but the purchase rate is obviously reduced, especially during 15:00-20:00, which is the second peak load period of the day.
In Figure 6, the SOCs in the initial and end times are the same (0.2). ESS is mainly called to discharge power during 15:00-20:00 and IL is called at 10:00.Comparing Figures 4  and 6, the discharge is more concentrated in the peak load periods and mainly in the valley load period, rather than the float load period, which is much more ideal than that in the separated scheduling case.This is because the joint scheduling case provides ESS and IL with more room for adjustment to help the system adjust the overall output.

Profit Distribution.
There are five alliance types among the three VPPs: separate operation (scenario #1), one-toone alliance (scenarios #2, #3, and #4), and overall alliance (scenario #5).The profits of each VPP and VPP alliance are listed in the first seven lines in Table 4.The profits distributed to V1, V2, and V3 using the Shapley value method are listed in the second-last line in Table 4, and their power generation incomes, calculated using revenue and cost, are listed in the last line.
In Table 4, VPPs in the separate operation scenario obtain the least total profit and individual profit.The total profit in 24 h is 127 Yuan less than the second-least profit (in scenario #3).The VPPs in the overall alliance obtain the largest total profit and individual profit.The total profit of the overall alliance is 5864 Yuan more than the second-largest total profit alliance portfolio.The profits of individual VPPs increased by 2516, 1414, and 3925 Yuan when compared to those in the separate operation scenarios.
Comparing the second-last line and last line of Table 4, the profit distributed by the Shapley value method is different from that calculated using revenue and cost.This is because V2 has the maximum wind installation capacity and, with the help of ESS adjustability, can reduce the abandoned wind and gain more revenue from customers and subsidy.This result for V2 is more significant than those for V1 and V3.For VPP alliance, the total profit originates from renewable energy consumption and less power purchase in the peak load period (with high TOU price).This profit growth is caused by the common contribution of V1, V2, and V3.Hence, the Shapley value method distributes the profit to the marginal contribution of each VPP, which is much fairer than simply calculating the revenue and cost of each VPP and ignoring their contribution to the alliance.
Let us take V1 as an example for showing the profit distribution process, as listed in Table 5.The values of lines 2 and 3 in Table 5 are calculated based on those in Table 4, where the V(\1) value in the third column equals all profits of sets not containing V1 while V1 and V2 are in cooperation.Therefore, the corresponding profit value equals V3's individual operation profit.Then, lines 3 to 6 are calculated according to the Shapley value method, and the weighted profit is calculated as the profit distributed to V1.
Combining the output result and adjustable power state changes shown in Section 5.2, the joint scheduling and profit distribution method based on the Shapley value can clearly increase the profit of the alliance participants, which verifies the validity of the model proposed by this paper.3, this paper chooses government subsidy policy and TOU price as the variables and analyzes their influence on VPP alliance total profit.The sensitivity analysis is based on the {V1, V2, V3} alliance type, and the result is shown in Figure 7.According to Figure 7, the WPP and PV subsidy prices have a significant influence on the total profit of VPP alliance, peak load price and float load price have an obvious influence on the total profit of VPP alliance, and valley load and ESS subsidy prices have relatively less influence on the total profit of VPP alliance.

Sensitivity Analysis. As shown in Table
For government subsidy factors, wind power's total installation power generation capacity is less than that of solar  photovoltaic power, but its effective utilization hours are much more than those of solar photovoltaic power.Therefore, when the VPP system structure changes, the influences of these two factors may change.Currently, because of the rapid development of wind power industry and growing wind power installation generation capacity, as well as wind power consumption problems, the government is planning to reduce wind power generation subsidy.Therefore, the VPPs or VPP alliances' total profit may decrease in the future because of changes in the wind power subsidy policy.On the other hand, solar photovoltaic and ESS industries are still important supporting objects for government.The power system reform and power market require much more ESS capacity than the existing capacity to enable a safe and stable operation of the power grid, which means the government would vigorously support ESS.
The TOU price is now determined by the local government.However, with the power system reform carried out, the TOU price for VPP can be determined by the distributed network or the upper power grid in the future.The development of the power market gives TOU price more possibility of change.In addition, the case study shows the applicability of our proposed model in providing reference information for VPPs or VPP alliances in decision-making.

Conclusions
This study develops a separate operation scheduling model for a VPP and a joint scheduling optimization and profit distribution model for VPP alliance to study the collaborative cooperation problem among VPPs.The model takes VPP components, including wind power plant, solar photovoltaic, ESS, and IL operation constraints, as well as power balance constraint, into consideration and sets maximizing the total profit as the optimization objective.The case study draws the following conclusions: (1) Both the separate operation model and joint scheduling model can optimize the VPP output to the grid to increase wind power usage efficiency, control the peak-valley rate, and increase the total profit, which verifies the feasibility and effectiveness of the model.
(2) The joint scheduling provides ESS and IL with more room for adjustment to help the system optimize the overall output and gain more profit.VPPs would obtain not less profit from the cooperation operation than their individual operation, which forms the base of the collaboration among the VPPs.
(3) Using the Shapley value method to allocate the alliance profit to individual VPPs can reflect individual contributions and meet both individual rationality and overall rationality, which is important implementation method of VPP alliance.
(4) The subsidy policies may influence the total profit significantly.The development of renewable energy, ESS, and overall power structure would change the government's subsidy policies.Our proposed model can provide reference information on profit change for VPPs or VPP alliances in decision-making.
(5) With the policy support of the government policy in China, VPPs can increase their total profit by cooperating in alliance and gain more profit distribution according to their contribution.The proposed model can provide a reasonable cooperation operation and profit distribution scheme. (

Figure 1 :
Figure 1: Operation model of VPP with a control center.

( 2 )
Perform scene reduction based on Kantorovich distance to obtain typical output scenes.(3) Input the VPP parameters and obtain the VPP set Z. (4) For each VPP in Z, calculate its separate operation scheduling model and alliance joint scheduling model except for this VPP.Repeat this step until all possible alliance types are considered.(5) Allocate the total profit based on the Shapley value method.

Figure 4 :
Figure 4: V1's ESS and IL state changes in separate scheduling case.

Figure 6 :
Figure 6: VPP alliance's ESS and IL state changes in separate scheduling case.

Figure 7 :
Figure 7: Sensitivity analysis of government subsidy policy and TOU price on VPP alliance total profit.
and ESS correspond to the power resource type V = 1,2,3.The economic factors of a VPP include the government subsidies, reserve cost, power sale income from the power grid, power sale income from the load, IL cost, and power purchase cost to the power grid.The separate scheduling of the VPP sets maximizing the total profit as the optimization objective: 3.1.OptimizationObjective and Constraints.Assume the total time length is   .This paper assumes that the wind power, solar power, , as well as the power balance constraint: V=1  ,,V −  ,grid, =  load,, −  con,, .

Table 1 :
Interruptible load capacity and the maximum continuous time of the three VPPs.

Table 2 :
Parameters of the energy storage system, electricity price, and government subsidies.

Table 3 :
Operation cost and income of three VPPs in separate scheduling case (Yuan).

Table 4 :
Profits of each individual VPP and alliance and profit distribution result (Yuan).

Table 5 :
The profit distribution calculation progress of V1.
) The Shapley value in this paper focuses on operation profit and ignores the operation risk changes of VPPs.Thus, reasonable weight adjustment parameters set based on the risk would make the distribution method fairer.The day-ahead declared total output of virtual power plant  at  time  ,,V : TheactualoutputofpowerresourceV in virtual power plant  under scene  at  time Δ ,, : The deviation of the actual output from the predicted value at  time  load,, : The internal load of virtual power plant  at  time  con,, : TheILcapacityofvirtualpowerplant at  time  ,grid, : The power supplied by the power grid to virtual power plant  at  time   , : The power supplied by the power grid to the VPP alliance at  time   grid, : The total internal load of the VPP alliance at  time   con, : The total IL capacity of the VPP alliance at  time   max : The maximum charge power of ESS  con, : Power output of IL at  time   max : The maximum discharge power of ESS The reduced scene set   gov, : Government subsidy at  time   rev, : The reserve cost of the VPP alliance at  time   grid, : The transaction income of the VPP alliance with the power grid at  time   load, : The load supply income of the VPP alliance at  time   con, : The IL cost of the VPP alliance at  time  pro ,gov, : The total government subsidy to renewable energy generation and energy storage system discharge  pro ,rev, : The reserve cost of VPP grid connection  pro ,grid, : The VPP transaction income with the power grid  pro ,load, : VPP internal load supply income  Allsubsetsof that contains member   + / − : The up/down reserve price  TOU : The TOU price