Research on Power Producer ’ s Bidding Behavior Based on the Best-Response Dynamic Model

As China’s electricity market is facing many problems, the research on power producer’s bidding behavior can promote the healthy and sustainable development of China’s electricity market. As a special commodity, the “electricity” possesses complicated production process. The instable market constraint condition, nonsymmetric information, and a lot of random factors make the producer’s bidding process more complex. Best-response dynamic is one of the classic dynamic mechanisms of the evolutionary game theory, which applies well in the repeated game and strategy evolution that happen among a few bounded rational players with a quick learning capability. The best-response dynamic mechanism is employed to study the power producer’s bidding behavior in this paper, the producer’s best-response dynamicmodel is constructed, and how the producers would engage in bidding is analyzed in detail. Taking two generating units in South China regional electricity market as the example, the producer’s bidding behavior by following the producer’s best-response dynamic model is verified. The relationships between the evolutionarily stable strategy (ESS) of power producer’s bidding and the market demand, and ceiling and floor price as well as biding frequency are discussed in detail.


Introduction
In the 1970s, when ecologists Maynard Smith and Price studied the ecology evolution phenomenon, they combined the biological evolutionism with the game theory and then came up the theory of evolutionarily stable strategy (ESS) [1,2], which marked the birth of evolutionary game theory.Compared with the traditional game theory, the evolutionary game theory improves the Perfect Rationality assumption and combines the traditional game theory with the dynamic evolution process to describe how the evolutionary relationship of gaming would develop over time, which overcomes the perfect rationality paradox in both neoclassical economics and game theory.Thus, it provides a new analysis method for economic research [3].In the middle of the 1990s, Fudenberg and Levine elaborated in detail the specific content of game learning theory based on the ideas of "learning" and "evolution" [4].This theory was accepted by many economists in a very short time and achieved remarkable accomplishments in economic analysis [5,6].From the 1980s, the evolutionary game theory has been widely used in economic field, such as the change of macroeconomic system [7], enterprise competition [8], and so on.
When it comes to group decision-making, the player often observes others firstly and then learns from the game history to adjust his game strategy.The key point of evolutionary game theory is to identify the strategy adjustment path.Currently, there are three types of evolutionary decision-making mechanisms: the mechanism based on the best-response dynamic and replicate dynamic model, the mechanism based on random process or swarm intelligence optimization algorithm, and the mechanism based on neural network and reinforcement learning.Among those, the bestresponse dynamic model and the replicate dynamic model based on biological evolution are the most commonly used dynamic decision-making mechanisms [9][10][11].
Since the implementation of the Electric Power System Reformation Plan approved by the China State Council in March 2002, the electricity market reform has gained primary achievement.One outcome is that China has fully separated the power generation from the power transmission.Nowadays, the power grids in China are mainly operated by the State Grid Corporation of China (SGCC) and the China Southern Power Grid Corporation (CSPGC).The auxiliary power companies include China Power Engineering Consulting Group, China Hydropower Engineering Consulting Group, China Water Conservancy, Hydropower Construction Group, and China Gezhouba Corporation.This indicates that the separation of the assistant industry from the main power industry has been achieved to some extent [12,13].
Although China's electricity market reform has made some achievements, there still remain many problems.China's top five large-scale power generation groups and some independent power generation companies with considerable size have initially formed the oligopolistic competitive pattern on the power generation side, but the regional electricity markets located in North China, Northeast China, Northwest China, East China, and Central China are just established and currently do not have full-fledged trading rules or settlement mechanisms.Besides, some serious problems have emerged at some regional markets, and even some on-going pilot projects have been interrupted.To pursue greater profits, all the power generation companies have motives to increase the electricity price at the regional electricity market, which will naturally lead to the strategic bidding of many power generation companies.This will bring harm to the safe operation of power grid as well as the price stability [14,15].To solve those problems caused by marketoriented reform, China not only needs to prefect the trading rules and establish reasonable trading mechanisms but also needs to study the producer's bidding behavior.
Because of the specificity of electricity as a commodity and the complexity of its production, the traditional game model cannot be applied well in the electricity market.Meanwhile, the instable market constraints, information asymmetry, and plenty of random factors affect the biding process, which make this issue more complex.Therefore, in this paper, according to the actual situation of China's electricity market, the best-response dynamic model of oligopolistic power producer's bidding is constructed based on the assumption that all producers have bounded rationality, and then the relationships between the evolutionarily stable strategy (ESS) of power producer's bidding and the market demand and ceiling and floor price as well as biding frequency are discussed.
The paper is organized as follows: Section 2 establishes the best-response dynamic model of power producer's bidding; Section 3 performs the producer's bidding behavior analysis; taking two biding generating units in South China regional electricity market as an example, the empirical analysis is performed in Section 4; Section 5 concludes this paper.

Best-Response Dynamic Model of Power
Producer's Bidding 2.1.Basic Theory of Best-Response Dynamic Model.The bestresponse dynamics is usually applied in the adjustment process of a repeated game among players who have rapid learning ability and bonded rationality.The rapid learning ability means that the players can make accurate postevaluation on the results of different strategies and then adjust their strategies accordingly although their abilities of judgment and foresight are a bit poor under the complicated situation.Therefore, when the former gaming result is given, each player can identify the best-response strategy compared with the former strategies adopted by other players.
According to the theory of Fudenberg and Levine, the best-response dynamic equation of player  can be represented as [16] where    is the probability distribution of strategy adopted by player  at time ;   (  − ) is the best response of player  against his competitors' strategies; and  is the ratio of the players who adopt the best-response strategy   (  − ) in all the players at time  + 1.The rest of the players continue to choose their strategies at time .
Equation ( 1) is the general mathematical description of adjustment process of repeated game between players who have rapid learning ability and bonded rationality.

The Mechanism and Assumption of Best-Response Dynamic Model of Power
Producer's Bidding.The random bestresponse dynamic model of producer's bidding adjustment was proposed by Larson and Salant, the mechanism of which is that each producer tends to adjust his bidding strategy according to the competitors' previous bidding strategies [17].
Suppose that    represents the probability distribution of producer 's bidding at time .The probability distribution of producer 's bidding    is the arithmetic weighted mean which is calculated by  −1  and    . −1  is the probability distribution of producer 's bidding at time  − 1 and    is the correction term.   represents the best-response made by producer  when it comes to competitors.Therefore, producer  would adjust his bidding price according to where  is the ratio of the players choosing the best-response strategy, and it also represents the weight of best response.The more the profit the competitor gets in prior time, the bigger the  would become.After a round of bidding, producers could obtain the probability distribution of competitor's bidding price at this round, and then they compute the corresponding best response based on this knowledge.Some assumptions need to be made when performing the best-response dynamic model of power producer's bidding.Suppose that there are  producers competing in one regional electricity market, and the maximum and minimum market trading prices are  max and  min , respectively.At bidding time , the market demand is ; the producer's bidding prices are ordered, and the last producer's bidding price that meets the market demand is named as the market clearing price   .
The producer  adjusts the bidding price by learning the previous bidding information.There are two main factors that affect producer 's bidding price at time , which are as follows.
(1) Generating Cost   .  is a variable related to the bidding power volume (power generation)   , and suppose that where   > 0 represents the producer's capacity cost and   > 0 represents the change rate of electricity cost.
(2) Competitor's Bidding Price   − .At each round of bidding, producer  does not know the competitor's bidding price   − , but he can estimate that the competitor's bidding price obeys the probability distribution function which has density function () on interval [ min ,  max ].

Producer's Profit Function.
If a producer's bidding price is lower than or equal to the clearing price, he can sell out all the declaratory power generation at the unified clearing price.However, if the producer's bidding price is higher than the clearing price, his power generation will not be sold out, and then his profit will be equal to zero.The profit function of producer  at time  can be represented as Then, the expected profit of producer  at time  can be represented as where Pr(   ≤   ) is the probability of producer 's successful bidding; Pr(   >   ) is the possibility of bidding failure.To calculate Pr(   ≤   ) and Pr(   >   ), Pr(   <    ) and Pr(   >    ) are supposed as the possibility of producer 's bidding price higher and lower than his competitor 's bidding price, respectively.
According to [18], if producer  has a bidding failure and  producers all have successful biddings, then there are at least  producers whose bidding prices are lower than producer 's bidding price    .Hence, the possibility of producer 's failure bidding and the possibility of successful bidding can be determined as follows: So, the expected profit of producer  at time  can be calculated by

Dynamic Adjustment Process of Producer's Best Response.
After one round of bidding, the producer  can learn some information about the competitors' latest bidding price and then adjust his own bidding price, so that he can make the best response against his competitors.Suppose that  represents the bidding price adjustment of producer  at time .The adjustment mechanism is as follows: firstly, the producer  calculates his profit  −1  () according to the previous bidding price  −1  ; secondly, the producer  adjusts the previous bidding price under the scenario that other producers will not change their bidding strategies and then calculates the corresponding profit    () (this profit is not the actual profit); finally,    () is compared with  −1  ().If    () >  −1  (), the producer  can gain more profit by the bidding price adjustment, so the producer  will make the bidding price adjustment; namely,    =  −1  + ; otherwise, the producer  will not make the bidding price adjustment; namely,    =  −1  .Therefore, the bidding price adjustment process of producer  can be represented as Figure 1 shows the adjustment process of producer 's price bidding strategy, which is also called the learning process of producer 's price bidding.Once the producer determines his initial price, the expected profit can be calculated, and the bidding price adjustment can be made based on the best-response dynamic model at each price bidding round.Generator's bidding price adjustment

Journal of Applied Mathematics
Figure 1: The best-response dynamic process of producer's bidding price.

Judgment Criteria of Evolutionarily Stable Strategy for Producer's Bidding Behavior. Evolutionarily stable strategy (ESS)
is an important concept in the game learning theory, which reflects the achieved equilibrium state after the best-response dynamic adjustment process.According to the connotation of ESS, the ESS of best-response dynamic adjustment on producer's bidding price is as follows.
Suppose  is the bidding price strategy of producers,  (0 <  < 1) is the ratio of the producers who adopt the bidding strategy   (  ̸ = ), and then (1 − ) is the ratio of the producers who adopt price strategy .
If the producer's profit  satisfies (, (1 − ) +   ) > (  , (1 − ) +   ), then the bidding price strategy  is defined as one of the ESS during the best-response dynamic adjustment process.(1 − ) +   represents the distribution of producers adopting strategy  and   .
An evolutionarily stable strategy should meet the following requirements.
(1) The proportion of individuals adopting this strategy keeps constant, which means the value of  is constant.
(2) This stable state must have robustness against the slight disturbance, which means the system can automatically recover to the evolutionarily stable state from the unstable state.
Therefore, the ESS  * of best-response dynamic adjustment on producer's bidding price should satisfy the following two conditions.
(1) The profit of producer  keeps the same no matter whether the producer  adjusts the bidding price or not; namely, (2) Even though there exists a slight bidding strategy disturbance  which makes the bidding price deviated from the stable state  * , the bidding price can still go back to the stable state  * after the best-response dynamic adjustment; namely,

Producer's Bidding Behavior Analysis
The best-response dynamic model applies well in the gaming behavior that involves a few players who have strong learning ability.Meanwhile, the producers in the oligopolistic electricity power have the characteristics of small number, large scale, and strong information searching-analyzingprocessing capability.By learning the historical market information and predicting the development trend, the producer can estimate both the competitors' bidding prices and profits and then acts properly against the competitors' bidding strategies.Therefore, the best-response dynamic model of power producer's bidding can be used to study the bidding behavior of oligopolistic producer in the electricity market and the price bidding trend.Suppose that there are two exact oligopolistic producers in one regional electricity market.Due to the symmetry, this paper only needs to study one oligopolistic producer i, and the other oligopolistic producer  is the competitor of producer .These two producers have the same production capability   and cost   .Set the ceiling price and the floor price in the electricity trading market as  max =  and  min = , respectively, and the bidding price obeys the uniform distribution in interval [, ]; namely,   (   ≤    ) = (   − )/( − ).The market demand  is  =   (0 <  < 2).

Best-Response Dynamic of Two Oligopolistic Producers.
According to the above suppositions and the best-response dynamic model, the profit of producer  can be represented as When    () >  −1  (), the producer  will continue to make the adjustment on bidding price.
According to ( 9)-( 10), we can get Then, Equation ( 13) is the mathematical expression of ESS of producer 's bidding through the best-response dynamic adjustment.

The Relationship between ESS and Market Demand, the
Ceiling Price, and the Floor Price.Suppose that  is a constant, which means the bidding adjustment does not have an effect on ESS.When the ceiling price, floor price, and generating cost are given, the relationship between ESS of producer 's bidding and market demand is determined, which is shown in Figure 2.
In Figure 2, the solid line shows the relationship between  * and  when  = 90 RMB/MWh,  = 290 RMB/MWh,   = 140 RMB/MWh, and  = 0.When the ceiling price () increases to 300 RMB/MWh, the relationship between  * and  is shown as the dashed line.
Just as shown in Figure 2, when  increases,  * goes up; when  increases,  * goes up.This indicates that the producer's bidding price will become higher with the increase of electricity demand in the regional electricity market.Hence, controlling the ceiling price is an effective method to keep the oligopolistic producer's bidding price under limits.When the ceiling price is set at low level, the producer's bidding price can be controlled, while the excessive celling price cannot play a role in controlling the producer's price bidding behavior.
Meanwhile, the floor price can also affect the producer's bidding strategy, but its effect is quite special: when the market demand is small (0 <  < 1), the higher the floor price is, the higher the producer's bidding price will become; yet when the market demand is large (1 <  < 2), the higher the floor price is, the lower the producer's bidding price will become.This indicates that when the electricity demand is small, the floor price can play a role in avoiding the virulent price bidding behavior of producers; but when the electricity demand is large, the floor price will not work.

The Relationship between ESS and Bidding Frequency.
Suppose that the initial bidding price of producer  is  0  and the bidding frequency is ℎ.Then, the ESS can be represented as where  = (−(−1))/(2−)+  −2 *  , which can be derived from (13).
Then, we can get When the marker demand, ceiling price, floor price, and producer's initial bidding price are given, the relationship between bidding frequency and ESS can be discussed.Suppose  = 1,  = 90RMB/MWh,  = 290 RMB/MWh,   = 140 RMB/MWh, and  0  = 195 RMB/MWh; then we can get The relationship between ESS of producer's bidding and bidding frequency is shown in Figure 3.As shown in Figure 3, the ESS of producer's bidding will go up with the increase of bidding frequency and eventually converges to bidding price strategy  *  = (1/2)[( − ( − 1))/(2 − ) +   ] with the maximum profit.In the regional electricity market, adding the number of market trading will increase the bidding frequency, and the producer can gradually adjust his bidding price by learning market information and analyzing competitor's bidding strategy until the maximum profit can be obtained.
Moreover, lim  → ∞  *  = (1/2)[( − ( − 1))/(2 − ) +   ] = 215 RMB/MWh.This bidding price is a Nash equilibrium price of producer's bidding by using the traditional game theory, which indicates that, through the long-term and multiple bidding price adjustment, the bounded rationality producer who possesses limited information can find the optimal biding price with the maximum profit.However, in this bidding price adjustment process, every bidding price offered by the producer may not be the optimal one, which in return backs up the ESS connotation in the game learning theory; namely, the game equilibrium is the result of long-term seeking optimization of bounded-rationality participants.

Empirical Analysis
4.1.Sample Data.Two competitive generating units from South China regional electricity market are selected.Considering the limitation of essential data, the sample range includes 24 time points of relevant indicators of generating units.The cost and production capacity of these two competitive generating units are listed in Table 1, the declaratory electricity power and limited prices are listed in Table 2, and the declaratory electricity prices of generating units are listed in Table 3.

The Relationship between ESS and the Market Demand,
the Ceiling Price, and the Floor Price.In the region electricity market, the electricity demand fluctuates over time, which is shown as the increase or decrease in the declaratory electricity power at different periods of time.Just as shown in Figure 4, when the electricity demand increases, the producer's bidding price tends to go up, while the producer tends to go for the low bidding price when the electricity demand is relatively low.
When the ceiling price is set at a low level, the producers will offer a low price, which makes the overall bidding price become low; but when the ceiling price is set at a high level, the producer's optimal bidding price will rise.So, the ceiling price could not inhibit the motive of producer to raise the electricity price, which is shown in Figure 5.
The floor price will also affect the producer's bidding strategy.Due to the fact that the sample data is selected from the regional electricity market, its market demand is relatively small compared with the overall market demand.So, the multiple  of market demand to producer's production capability should be between 0 and 1.Under this situation, the higher the floor price is, the higher the producer's bidding price is, and the adjustment of producer's bidding price will be consistent with the adjustment of the floor price, which is shown in Figure 6.

The Relationship between ESS and Bidding Frequency.
According to the above analysis, the ESS of producer's bidding will go up with the increase of the bidding frequency and eventually converges into the bidding price strategy expressed as  *  = (1/2)[( − ( − 1))/(2 − ) +   ] that possesses the maximum profit.The calculation result of the relationship  The ceiling price Generating unit number 1 in power plant TG Generating unit number 1 in power plant DG between ESS and bidding frequency based on the sample data is shown in Figure 7. From Figure 7, we can see that the producer's bidding price converges to the most profitable bidding strategy  *  after a four-time adjustment.

Conclusions
The producer's bidding strategy based on the best-response dynamic mechanism is studied and the best-response dynamic model of producer's bidding behavior is constructed in this paper.Taking two generating units in South China regional electricity market as the example, the monopolistic producer's bidding behaviors are empirically studied, and some conclusions are drawn as follows.
(1) With the increase of electricity power demand, the oligopolistic producer tends to raise his bidding price.
If the bidding behavior cannot be restrained, when the market demand goes near the producer's supply capacities, all producers will raise the bidding price to a very high level.This conclusion has been proved by the power crisis in California.
(2) The ceiling price has some certain effects on inhibiting the motive of producers to raise the electricity price, and the producer's overall bidding price will go with the ceiling price.This implies that setting reasonable celling price should not only consider how to keep the producers away from raising the bidding price, but also consider how to motive the producers in terms of profit.
(3) When the electricity power demand is small, the floor price can play a role in avoiding the virulent price bidding behavior of producers.But when the  electricity demand becomes large, the floor price will not work.
(4) When the number of market trading increases, the producer's bidding frequency will increase, and the producer can gradually adjust his bidding price by learning market information and analyzing competitors' bidding strategies until the maximum profit can be obtained.

Figure 4 :
Figure 4:  The relationship between market demand and the producer's bidding strategy.

Figure 5 :Figure 6 :
Figure 5: The relationship between ceiling price and producer's bidding strategy.
unit number 1 in power plant TG, best bidding price with adjustment Generating unit number 1 in power plant TG, the maximum profit bidding strategy Generating unit number 1 in power plant DG, best bidding price with adjustment Generating unit number 1 in power plant DG, the maximum profit bidding strategy

Figure 7 :
Figure 7: The relationship between ESS and bidding frequency.

Table 1 :
The cost and production capacity of two competitive generating units.

Table 2 :
The declaratory electricity power and limited prices.

Table 3 :
The declaratory electricity price of generating units.