Bayesian Game-Theoretic Bidding Optimization for Aggregators Considering the Breach of Demand Response Resource

Abstract

Demand response (DR) aggregator controlling and aggregating flexible resource of residential users to participate in DR market will contribute the performance of DR project. However, DR aggregator has to face the risk that users may break the contract signed with aggregator and refuse to be controlled by aggregator due to the uncertainty factors of electricity consumption. Therefore, in this paper, community operator (i.e., DR aggregator) is proposed to equip auxiliary equipment, such as energy storage and gas boiler, to compensate for power shortage caused by users’ breach behavior. DR aggregated resource with different auxiliary equipment will have different characteristics, such as breach rate of DR resource. In the proposed DR framework, for selling the aggregated resource, community operator has to compete the market share with other operators in day-ahead DR market. In the competition, each operator will try its best to make the optimal bidding strategy by knowing as much information of its opponents as possible. But, some information of community operator (e.g., DR resource’s characteristic) belongs to privacy information, which is unknown to other operators. Accordingly, this paper focuses on the application of incomplete information game-theoretic framework to model the competition among community operators in DR bidding market. To optimize bidding strategy for the high profit with incomplete information, a Bayesian game approach is formulated. And, an effective iterative algorithm is also presented to search the equilibrium for the proposed Bayesian game model. Finally, a case study is performed to show the effectiveness of the proposed framework and Bayesian game approach.

1. Introduction

The fast-growing electricity demand driven by the development of social economy has made the current power grid face serious challenges, such as grid security and power supply reliability [1,2]. Since electricity cannot be largely stored, power grid has to constantly balance energy supply and demand, reducing supply when generation exceeds consumption and increasing supply when consumption exceeds generation. Once the supply is incapable of meeting the demand, blackout will happen which can be solved by building new power plants to promote the energy supply ability or reducing the energy consumption of consumers to alleviate the demand pressure [3]. Considering energy shortage periods only occur in a few time slots, it may be not very economically feasible to build a new power plant. Therefore, for this reason, demand response (DR) project has been applied extensively in demand side, including price-directed DR and incentive-directed DR [4,5]. Because there exists a lot of flexible resource in demand side, especially in residential side, hence, a well-designed DR framework via introducing market mechanism can effectively yield peak shaving capability under emergent circumstances [6]. However, since single residential consumer has limited DR resource, residential consumers are usually not eligible to directly participate in DR market [7,8]. Under the background, DR aggregator gradually appears, which is referred to as a cooperative consumer, virtual power plant and load aggregator in different contexts [7,9]. Such aggregator, who is the intermediary institution between DR purchaser and consumers, will participate in the DR bidding market with other aggregators by aggregating consumers’ DR resource. With the assistance of DR aggregator, more flexible resource in consumer side can participate in the DR project, and power grid can also economically relieve the supply pressure from consumer side [10].

In recent years, the bidding problem of DR aggregators for participation in the market has been widely investigated in the literature. Authors in [11,12] presented optimal bidding strategy in the day-ahead market via load aggregator aggregating flexible DR resource. A price-based self-scheduling mixed-integer linear programming model for an aggregator was proposed in [13], with the aim to maximize the aggregator’s payoff for participation in the day-ahead market. While in [14], an optimization model was proposed for planning and operating DR resource managed by a price-maker aggregator participating in the market, through both a deterministic and stochastic approach. Additionally, some examples on the bidding strategies of virtual power plants consisting of distributed generation and energy storage were also found in the literature [15,16,17,18]. Concretely, study [15] concentrated on a novel DR scheme that avoids the need to predict the price elasticity of demand by ranking candidate load profiles of consumers in the preference order. The bidding problem was formulated in [17,18] with reference to the day-ahead energy and spinning reserve markets. The presented problem was a nonlinear mixed-integer linear programming problem, which was solved by using a genetic algorithm. Different from above referred research, some literatures grasped the dynamic interaction characteristics of market participants in the process of strategy optimization and then introduced game theory into the DR mechanism [19,20,21,22]. Specifically, some literatures focused on the bidding strategy optimization of DR aggregator with game theory [23,24,25,26]. For example, research [24] employed non-cooperative game approach to analyze the bidding strategies of load aggregators in the day-ahead market. Authors in [25] proposed a game-theoretic approach to develop a decentralized aggregated control algorithm to seek for an optimal energy usage plan for a population of heterogeneous loads by employing potential game theory. While in [26], a Stackelberg game model was formulated between DR aggregator and electricity generators, in which DR aggregator played as the leader to make demand reduction bids, and generators played as followers to compete for maximizing their profits.

The above referred papers are all developed on basis of the complete information. However, since much information in reality is unknown to all participants in the market or many participants refuse to share their private information, consequently, each participant in the market does not have information about other opponents in terms of cost functions or payoffs. Therefore, the complete information approach is not suitable for the situation with incomplete information. Based on the background, this paper mainly concentrates on the optimization of bidding strategy for community operators (i.e., DR aggregator) in DR market considering the incomplete information. Here, bidding strategy mainly refers to planning DR resource which will be sold to DR purchaser by DR aggregator in real-time scheduling. By formulating the incomplete information game model, community operators can obtain the optimal profit in the bidding market by speculating opponents’ information. In our proposed scenario, community operator is a commercial organization who aggregates DR resource of residential community to participate in the market bidding by signing the contract with users to obtain the control of appliances. Considering DR resource’s breach due to the consumption randomness of users, some community operators will equip auxiliary equipment (e.g., energy storage or gas boiler) to compensate the breach amount to enhance DR resource quality. Here, DR resource’s breach mainly refers to breach behavior of users who have signed contract with aggregator but refuse to be in DR. Community operators with different auxiliary equipment, which can be divided into different types, will have different quality characteristics of DR resource. In order to protect the privacy of market participants, DR market will not broadcast the information about community operators’ types. While in DR market, the payoff of community operator is correlated with the price of DR resource that will be influenced by the grade of DR resource and bidding amount of market participants. Hence, each community operator can know its own payoff but cannot acknowledge other opponents’ payoffs completely because their concrete type is not public information. For that, it is necessary for a community operator to model its opponents’ expected payoffs according to the probability distribution on opponents’ types. That is, Bayesian game approach will be employed to formulate the bidding optimization model in the paper. In brief, the contributions of this paper are as follows.

• A DR framework is proposed to optimize the bidding strategy of DR resource for community operators considering the risk of DR resource’s breach.
• A Bayesian game based approach is formulated for the proposed scenario with incomplete information to promote the participants’ profit, and then an iterative algorithm is designed to obtain the bidding equilibrium among community operators.
• A case study is carried out to verify the performance and effectiveness of the proposed Bayesian game approach by simulating the bidding strategy optimization of 3 community operators.

The proposed DR framework is introduced in Section 2. In Section 3, system model is formulated. Section 4 focuses on the formulation of Bayesian game among community operators. Then, case study is presented in Section 5. Finally, this paper is concluded in Section 6.

2. Proposed DR Framework

A DR framework is proposed with N community operators and one DR purchaser, which is shown in Figure 1. DR purchaser in real power system can be grid dispatching department or energy selling company. Community operator, being as an intermediary institution, plays an important role in load aggregation for the flexible DR resource of community consumers, and then sells the resource to DR purchaser for profit. In each community, except for DR resource, some community operators have equipped auxiliary equipment to promote the quality of DR resource, such as gas boiler or energy storage.

In our proposed scenario, before conducting DR project, community operator will sign contract with users in the community to obtain the control of flexible load. According to the contract volume, community operator will compete the market share with other operators in DR bidding market. However, since energy consumption of users has a certain randomness, some users may break the contract and control the load by self. Once there exists DR resource’s breach, community operator will not complete the bidding amount and will be punished by DR purchaser. Therefore, in the bidding market, community operator must take DR resource’s breach into the consideration. As for bidding mechanism in the market, DR purchaser will broadcast market price information to all community operators. In our scenario, DR purchaser has divided DR resource into different grades according to the degree of DR resource’s breach. The price will be higher for DR resource with lower breach. Consequently, some community operators will compensate the breach amount with gas boiler or energy storage to promote the grade of DR resource. Hence, community operators can also be divided into different types according to their auxiliary equipment. After broadcasting price information, community operators tell DR purchaser if they participate in the bidding market. Furthermore, DR purchaser will gather participants’ information and broadcast to all participants. However, considering the protection of private information of participants, the broadcast information only contains the number of participants, probability distribution on community operators’ types. According to the received information, community operators will make the optimal strategy to maximize the self-profit with Bayesian game theory and then summit the bidding strategy to DR purchaser. Additionally, the above referred DR framework needs to be supported by grid’s physical platform. Therefore, we assume that: (a) Community operators and DR purchaser can communicate each other with bi-directional information network. (b) Advanced metering infrastructure is equipped in consumer side to control flexible loads and transfer data, including smart meter, bi-directional information network, and measuring-data management system.

3. System Model

Based on the proposed scenario, there are N community operators who willing to participate in DR market with the set $\mathcal{N}=\left[1,2\cdots N\right]$. DR purchaser has to purchase DR resource in $\mathcal{T}=\left[1,2\cdots T\right]$ time slots and DR resource is divided into $\mathcal{I}=\left[1,2\cdots I\right]$ grades.

3.1. Gas Boiler Model

Gas boiler can provide thermal energy demand of community users by consuming fossil fuel. In the scenario, we assume that gas boiler consumes natural gas. When consumers irregularly use electrical heating appliance for thermal demand, gas boiler can replace heating device to satisfy the same demand of users, thus reducing breach amount of DR resource. Assume that the breach amount of thermal demand in community n is $h_n^t$ in time slot t, hence, following equation has to be obeyed [27]

$$h_n^t={\eta }_b{\lambda }_{\mathrm{gas}}{r}_n^t$$

(1)

where ${\eta }_b$ is the thermal efficiency of the boiler; ${\lambda }_{\mathrm{gas}}$ is the calorific value of natural gas; $r_n^t$ is the natural gas consumption.

3.2. Energy Storage Model

Similarly, energy storage can also be employed to compensate the breach amount of DR resource. Storage will supply energy when users are unwilling to take part in DR. Assume that $So{c}_n^t$ is energy state of storage at the beginning of time slot t. Considering energy conversion losses in the operation, storage charging and discharging efficiencies are represented by ${\eta }_{\mathrm{ch}}$ and ${\eta }_{\mathrm{dis}}$, respectively. Accordingly, if energy state of storage is $So{c}_n^{t-1}$ at the beginning of time t, then storage state $So{c}_n^t$ can be calculated as [28]

$$So{c}_n^t=So{c}_n^{t-1}+{\eta }_{\mathrm{ch}}{e}_n^{\mathrm{ch},t}-1/{\eta }_{\mathrm{dis}}{e}_n^{\mathrm{dis},t}$$

(2)

where $0<{\eta }_{\mathrm{ch}},{\eta }_{\mathrm{dis}}<1$, $e_n^{\mathrm{ch},t}$ and $e_n^{\mathrm{dis},t}$ are charging and discharging energy in time slot t, respectively. Since this paper mainly concentrates on the load peak shaving with DR resource, energy storage in time slots $\mathcal{T}$ is mainly on discharging state. Therefore, the value of $e_n^{\mathrm{ch},t}$ in time slot t is 0.

3.3. Bidding Price Model

In order to maintain bidding market stability, it is necessary for DR purchaser to design a reasonable bidding price model. Assume that bidding price of grade i of DR resource is $p_i^t$ in time slot t, and bidding amount of all community operator is $x_i^t$ in time slot t for grade i. Considering that market price generally has a significant linear relation with load demand level, the bidding price can also be designed as a linear function

$$p_i^t=a_i^t{x}_i^t+b_i^t$$

(3)

where $a_i^t<0$ and $b_i^t>0$ are coefficients of grade i of DR resource. It needs to point that, when community operator n has the ith grade of DR resource, then it can participate in the bidding of corresponding grade of DR resource with other same type of operators, otherwise the bidding amount is set to 0.

3.4. DR Resource’s Breach Model

Due to the randomness of users’ daily energy consumption, it is unrealistic to assume that users will completely obey the control of community operator. It is unavoidable for users to break the contract in the real-time scheduling. Assume that bidding amount of community operator n is $x_n^t$ for one grade of DR resource, and breach amount of users in the community n is ${\delta }_n^t$. It is obvious that the value of ${\delta }_n^t$ is in $\left[0,x_n^t\right]$. This paper adopts truncated normal distribution to simulate stochastic distribution of breach amount ${\delta }_n^t$.

Assume that $\delta \left(u,{\sigma }^2\right)$, ${\delta }_{\mathrm{l}}$ and ${\delta }_{\mathrm{u}}$ are positive real numbers, hence, the distribution of $\delta$ under the condition of ${\delta }_{\mathrm{l}}\le \delta \le {\delta }_{\mathrm{u}}$ is called truncated normal distribution, which can be represented as $N\left(u,{\sigma }^2,{\delta }_{\mathrm{l}},{\delta }_{\mathrm{u}}\right)$. And the probability density function is [29]

$$f\left(u,{\sigma }^2,{\delta }_{\mathrm{l}},{\delta }_{\mathrm{u}}\right)=\left\{\begin{array}{cc}\frac{\phi \left(\frac{\delta -u}{\sigma }\right)}{\sigma \left[\Phi \left(\frac{{\delta }_{\mathrm{u}}-u}{\sigma }\right)-\Phi \left(\frac{{\delta }_{\mathrm{l}}-u}{\sigma }\right)\right]}& {\delta }_{\mathrm{l}}\le \delta \le {\delta }_{\mathrm{u}}\\ 0& {\delta }_{\mathrm{l}}>\delta ,\delta \le {\delta }_{\mathrm{u}}\end{array}\right.$$

(4)

where $\phi \left(·\right)$ and $\Phi \left(·\right)$ are probability density function and cumulative distribution function of standard normal distribution, respectively. Combining the breach amount ${\delta }_n^t$ of community operator n, it is known that $\delta ={\delta }_n^t$, ${\delta }_{\mathrm{l}}=0$ and ${\delta }_{\mathrm{u}}=x_n^t$. Accordingly, the expected breach amount in the community n can be expressed as

$$E{\delta }_n^t=u+\sigma \frac{\phi \left(\frac{-u}{\sigma }\right)-\phi \left(\frac{x_n^t-u}{\sigma }\right)}{\Phi \left(\frac{x_n^t-u}{\sigma }\right)-\Phi \left(\frac{-u}{\sigma }\right)}$$

(5)

The above analysis is only for the case of bidding amount $x_n^t>0$. When bidding amount is $x_n^t=0$, obviously, the breach amount ${\delta }_n^t$ is also equal to 0. Here, breach percentage ${\gamma }_n=E{\delta }_n^t/x_n^t$ is used to define the grade of DR resource. When breach percentage satisfies ${\gamma }_i^{min}\le {\gamma }_n\le {\gamma }_i^{max}$ (${\gamma }_i^{min}$ and ${\gamma }_i^{max}$ are upper-lower limit of ith grade of DR resource), then such DR resource is defined as ith grade.

4. Bayesian Game among Community Operators

To achieve a high profit in DR market, each community operator will participate in the bidding with other operators. Such competition among operators can be described with non-cooperative game theory. Therefore, in this section, non-cooperative game model will be firstly formulated to optimize the bidding strategy under the complete information. And then Bayesian game approach is proposed to search the optimal strategy considering the existence of incomplete information.

4.1. Community Operator’s Profit Model

Community operator’s profit mainly comes from DR purchaser, gas boiler, and energy storage. Accordingly, we can obtain the following profit model.

• Profit from DR purchaser
  From DR purchaser, community operator’s income is the earning for selling DR resource, and cost is the punishment for DR resource’s breach. According to the bidding price, when DR resource of community operator n is the ith grade, then the profit from DR purchaser can be expressed as

  $$W_n^{\mathrm{DR}}=\sum _{t=1}^T\left(p_i^t{x}_n^t-q_{i}E{\delta }_n^t-c_{\mathrm{user}}\left(x_n^t-E{\delta }_n^t\right)\right)$$

  (6)

  where $q_i$ is the punishment for ith grade of DR resource’s breach, $c_{\mathrm{user}}$ is the price paid to consumers.
• Profit from gas boiler
  From gas boiler, community operator’s income is the earning for providing heat to users, and cost is the energy cost for natural gas, investment and maintenance cost. Accordingly, profit from gas boiler can be expressed as

  $$W_n^{\mathrm{GB}}=\sum _{t=1}^T\left(s_n^{\mathrm{h}}{h}_n^t-c_{\mathrm{gas}}{r}_n^t\right)-C_n^{\mathrm{GB}}$$

  (7)

  where $s_n^{\mathrm{h}}$ is heat price charged from users; $c_{\mathrm{gas}}$ is natural gas price; $C_n^{\mathrm{GB}}$ is the investment and maintenance of gas boiler with the following formula

  $$C_n^{\mathrm{GB}}=\frac{1}{365}{k}_{\mathrm{GB}}^{\mathrm{in}}\frac{r{\left(1+r\right)}^{y_{\mathrm{GB}}}}{{\left(1+r\right)}^{y_{\mathrm{GB}}}-1}{S}_n^{\mathrm{GB}}+\sum _{t=1}^T{k}_{\mathrm{GB}}^{\mathrm{on}}{h}_n^t$$

  (8)

  where $k_{\mathrm{GB}}^{\mathrm{in}}$ and $k_{\mathrm{GB}}^{\mathrm{on}}$ are investment price and maintenance price, respectively; r is the discount rate; $y_{\mathrm{GB}}$ is the lifetime of gas boiler; $S_n^{\mathrm{GB}}$ is the capacity of gas boiler.
• Profit from energy storage
  From energy storage, community operator’s income is the earning for providing electricity to users, and cost is the electricity cost for charging storage, investment and maintenance cost. Accordingly, profit from energy storage can be expressed as

  $$W_n^{\mathrm{ES}}=\sum _{t=1}^T\left(s_n^{\mathrm{e}}{e}_n^{\mathrm{dis},t}-c_{\mathrm{ele}}{e}_n^{\mathrm{dis},t}/\left({\eta }_{\mathrm{ch}}{\eta }_{\mathrm{dis}}\right)\right)-C_n^{\mathrm{ES}}$$

  (9)

  where $s_n^{\mathrm{e}}$ is electricity price charged from users; $c_{\mathrm{ele}}$ is electricity price charged by public grid; $C_n^{\mathrm{ES}}$ is the investment and maintenance of energy storage with the following formula

  $$C_n^{\mathrm{ES}}=\frac{1}{365}{k}_{\mathrm{ES}}^{\mathrm{in}}\frac{r{\left(1+r\right)}^{y_{\mathrm{ES}}}}{{\left(1+r\right)}^{y_{\mathrm{ES}}}-1}{S}_n^{\mathrm{ES}}+\sum _{t=1}^T{k}_{\mathrm{ES}}^{\mathrm{on}}\left(e_n^{\mathrm{dis},t}+e_n^{\mathrm{dis},t}/\left({\eta }_{\mathrm{ch}}{\eta }_{\mathrm{dis}}\right)\right)$$

  (10)

  where $k_{\mathrm{ES}}^{\mathrm{in}}$ and $k_{\mathrm{ES}}^{\mathrm{on}}$ are investment price and maintenance price, respectively; $y_{\mathrm{ES}}$ is the lifetime of energy storage; $S_n^{\mathrm{ES}}$ is the capacity of energy storage.

4.2. Game Formulation with Complete Information

In the game with complete information, the grade of DR resource participating in the bidding market is well known to all other community operators. Furthermore, each community operator tries to maximize its own payoff in the bidding market by acknowledging other community operators’ strategies. Consequently, the complete information game among community operators can be formulated as follows [30].

• Players: All community operators who willing to participating in the bidding market.
• Strategies: each community changes its bidding strategy $x_n^t$ to maximize its payoff.
• Payoffs: the payoff of each community operator is defined as

  $$P_n\left(x_n,x_{-n}\right)=W_n^{\mathrm{DR}}+K_n^{\mathrm{GB}}{W}_n^{\mathrm{GB}}+K_n^{\mathrm{ES}}{W}_n^{\mathrm{ES}}$$

  (11)

  where $x_n=[x_n^1,x_n^2,\cdots x_n^T]$ is the bidding strategy set of community operator n and $x_{-n}=\left[x_1,\cdots ,x_{n-1},x_{n+1},\cdots ,x_N\right]$ is the bidding strategy set of all community operators except operator n; $K_n^{\mathrm{GB}}=1$ and $K_n^{\mathrm{ES}}=1$ represent community operator n has equipped gas boiler and energy storage, otherwise $K_n^{\mathrm{GB}}=0$ and $K_n^{\mathrm{ES}}=0$.

Community operator n is willing to schedule bidding amount of DR resource to maximize its own payoff on basis of it opponents’ strategies until no community operator wants to change. This equilibrium state is called Nash equilibrium, which can be expressed as following formula

$$P_n\left(x_n^{*},x_{-n}^{*}\right)\ge P_n\left(x_n^{},x_{-n}^{*}\right)$$

(12)

where $\left(x_n^{*},x_{-n}^{*}\right)$ is Nash equilibrium of community operators in the bidding market. Once the Nash equilibrium is reached, the payoff for any community operator n will be reduced by changing from equilibrium $x_n^{*}$.

4.3. Bayesian Game Formulation

In the above formulated game model, each community operator has the full information of other operators about DR resource’s grade and knows their payoff function completely. However, it is difficult to conduct the above complete game in reality because much information of other opponents is unknown to community operator. Consequently, the above game theory with complete information cannot be applied in such case. For that reason, Bayesian game theory is employed to describe the game behavior among community operators with incomplete information. Bayesian game, which was first proposed by Harsanyi, is an effective approach to formulate the game model with incomplete information via introducing Bayesian formula [31]. Different from the complete information game, except for three basic elements (i.e., players, strategies, and payoffs), Bayesian game has also considered other two elements, that is, the types of player and the probability distribution of the types. In this section, a detailed description about Bayesian game formulation will be given combining the proposed scenario.

Community operators with different auxiliary equipment will have different characteristic of DR resource, which can be divided into different types. Assume that the types of community operators are divided into $\mathcal{J}=\left[1,2\cdots J\right]$ kinds. Accordingly, suppose that the type space of community operator n is $J_n$ and the actual type of operator n is $j_n$. Furthermore, the type space combination for all community operators is $J=J_1\times J_2\times \cdots \times J_N$ and the actual type combination is $j=\left[j_1,\cdots ,j_n,\cdots ,j_N\right]$. Due to the lack of information about other operators’ types, each community operator will estimate the types of these operators on basis of the probability distribution of the types. Accordingly, the following formula can be employed

$$Pr\left(j_{-n}|j_n\right)=\frac{Pr\left(j_{-n},j_n\right)}{Pr\left(j_n\right)}=\frac{{\displaystyle \prod _{n=1}^N}Pr\left(j_n\right)}{Pr\left(j_n\right)}$$

(13)

where $j_{-n}=\left[j_1,\cdots ,j_{n-1},j_{n+1}\cdots ,j_N\right]$ denotes the actual type combination of other $N-1$ community operators; $J_{-n}=J_1\times \cdots \times J_{n-1}\times J_{n+1}\cdots \times J_N$ denotes the type space combination of other $N-1$ community operators; $Pr\left(j_{-n}|j_n\right)$ denotes the conditional probability for the type combination $j_{-n}$ of other $N-1$ community operators when operator n belongs to type $j_n$; $Pr\left(j_{-n},j_n\right)=Pr\left(j\right)$ denotes the joint probability for the type combination $j=\left[j_1,\cdots ,j_n,\cdots ,j_N\right]$ of all community operators. The value of probability $Pr\left(j_n\right)$ can be obtained according to the proportion of community operators with type j in all operators. For example, assume that there are total M community operators in the grid, and in these operators, m operators belong to type j. Then, if we choose any one from M operators as operator n, the probability of operator n being type j can be considered as $Pr\left(j_n\right)=m/M$. Here, it needs to note that, although there are M community operators in the grid, not all operators will participate in day-ahead market of the same day. In this paper, it is assumed that N community operators will participate in the bidding market of the same day. According to the formula (13), the incomplete information game can be divided into multiple complete information games by ascertaining the actual types of community operators. The number of complete information games is determined by the number of elements in the type space combination $J_{-n}$. Therefore, based on the payoff formulaes (11) and (13), the expected payoff of community operator n with type $j_n$ can be calculated as

$$E{P}_n\left(x_n\left(j_n\right),x_{-n}\left(j_{-n}\right)\right)=\sum _{j_{-n}\in J_{-n}}{P}_n\left(x_n\left(j_n\right),x_{-n}\left(j_{-n}\right)\right)Pr\left(j_{-n}|j_n\right)$$

(14)

where $x_n\left(j_n\right)$ denotes the bidding strategy of community operator n with type $j_n$; $x_{-n}\left(j_{-n}\right)$ denotes the bidding strategy of other community operators with type combination $j_{-n}$. Similarly, Bayesian Nash equilibrium can be defined as follows

$$E{P}_n\left(x_n^{*}\left(j_n\right),x_{-n}^{*}\left(j_{-n}\right)\right)\ge P_n\left(x_n^{}\left(j_n\right),x_{-n}^{*}\left(j_{-n}\right)\right)$$

(15)

where $\left(x_n^{*}\left(j_n\right),x_{-n}^{*}\left(j_{-n}\right)\right)$ is Bayesian Nash equilibrium. Once the equilibrium is reached, the payoff for community operator n with type $j_n$ will be reduced by changing from equilibrium $x_n^{*}\left(j_n\right)$.

Based on the formula (14), our target is to search the Bayesian Nash equilibrium to maximize the expected payoff of community operator in the bidding market. That is to solve the following optimization problem

$$\underset{x_n\left(j_n\right)}{max}E{P}_n\left(x_n\left(j_n\right),x_{-n}\left(j_{-n}\right)\right)$$

(16)

Therefore, for community operator n, under the bidding strategy $x_{-n}\left(j_{-n}\right)$ of other operators, operator n will schedule its own bidding strategy $x_n\left(j_n\right)$ to search the maximal expected daily profit. Additionally, in the process of optimization, following constraints must be obeyed in order to guarantee the validity of the solution.

• DR resource’s grade constraint
  Community operators employ gas boiler and energy storage to promote the grade of DR resource. In which, gas boiler can reduce breach amount by substituting electrical heating appliance; while energy storage can compensate breach amount via providing electricity for users. When community operator equips gas boiler or energy storage, its expected breach amount can be expressed as

  $$E{\widehat{\delta }}_n^t={\int }_0^{\Delta e}0d\delta +{\int }_{\Delta e}^{x_n^t}\left(\delta -\Delta e\right)f\left(u,{\sigma }^2,{\delta }_{\mathrm{l}},{\delta }_{\mathrm{u}}\right)d\delta$$

  (17)

  where $\Delta e$ is the compensation amount by gas boiler and energy storage, which is equal to

  $$\Delta e=K_n^{\mathrm{GB}}{h}_n^t/{\eta }_{\mathrm{e}}^{\mathrm{h}}+K_n^{\mathrm{ES}}{e}_n^{\mathrm{dis},t}$$

  (18)

  where ${\eta }_{\mathrm{e}}^{\mathrm{h}}$ is the energy efficiency ratio of electrical heating appliance. Based on formulaes (17) and (18), the breach amount will be reduced to $E{\widehat{\delta }}_n^t$ from the perspective of DR purchaser. Therefore, the breach percentage changes to ${\widehat{\gamma }}_n=E{\widehat{\delta }}_n^t/x_n^t$. When community operator n wants to participate in the bidding of ith grade of DR resource, then breach percentage ${\widehat{\gamma }}_n$ must satisfy the upper limit ${\gamma }_i^{max}$

  $$\left\{\begin{array}{c}{\widehat{\gamma }}_n=E{\widehat{\delta }}_n^t/x_n^t\hfill \\ {\gamma }_i^{min}\le {\widehat{\gamma }}_n\le {\gamma }_i^{max}\hfill \end{array}\right.$$

  (19)
• Device output constraint
  Energy output of gas boiler and energy storage have to be less than the maximal energy output of device

  $$\left\{\begin{array}{c}0\le e_n^{\mathrm{dis},t}\le e_n^{\mathrm{dis},\max }\hfill \\ 0\le h_n^t\le h_n^{max}\hfill \end{array}\right.$$

  (20)

  where $e_n^{\mathrm{dis},\max }$ and $h_n^{max}$ are the maximal energy output of energy storage and gas boiler. Additionally, the state of energy storage has to satisfy

  $$\left(1-DOD\right)S_n^{\mathrm{ES}}\le So{c}_n^t\le S_n^{\mathrm{ES}}$$

  (21)

  where $DOD$ is the maximal discharge depth of energy storage.
• Bidding amount constraint
  Each community operator makes bidding strategy according to the DR resource level in its native users. Since flexible loads only take a certain percentage of all users’ loads, the bidding amount must satisfy

  $$0\le x_n^t\le x_n^{t,max}$$

  (22)

4.4. Distribution Algorithm

Based on the formulated Bayesian game model, a distributed algorithm is proposed to search the Bayesian Nash equilibrium for community operators, which is presented in Algorithm 1. In the algorithm, $N-1$ in step 4 represents $N-1$ community operators except operator n; optimization problem in step 7 is solved by interior point method. According to the process of algorithm, we can know that each community operator with each type has to constantly readjust its bidding strategy until the Nash equilibrium is reached. The dynamic bidding-making process can be reflected with following steps:

• Community operator n with type $i_n$ initializes its bidding strategy;
• Community operator n will optimize the bidding strategy of its opponents;
• On basis of the optimal strategy of opponents, community operator n will optimize its own     bidding strategy;
• Repeat step 2 and 3 until all operators’ strategies is unchanged.

Note that, the above steps are all executed by community operator n, and other community operators will also have the same decision process.

Algorithm 1: Executed by each community operator $n\in \mathcal{N}$

5. Case Study

In this section, case study will be presented to show the performance and effectiveness of the proposed DR framework and formulated Bayesian game model. The proposed algorithm is executed by MATLAB R2012b on the personal computer with Intel(R) Core(TM) i7-7700 CPU@3.60 GHz and RAM 8.00 GB.

5.1. Simulation Parameters

Assume that grid dispatching department will purchase DR resource from community operators during 18:00–21:00, and each scheduling cycle is 15 min. Accordingly, there are 12 cycles with $\mathcal{T}=\left[1,2\cdots 12\right]$. Additionally, DR resource’s grades are divided into 3 kinds: grade 1 for ${\gamma }_1^{min}=0$ and ${\gamma }_1^{max}=3\%$; grade 2 for ${\gamma }_2^{min}=3\%$ and ${\gamma }_2^{max}=8\%$; grade 3 for ${\gamma }_3^{min}=8\%$ and ${\gamma }_3^{max}=13\%$ [32]. The price parameters broadcasted by grid dispatching department to the market is shown in

Table 1. Price parameters for different grades of DR resource.

DR Resource	18:00–19:00 (t = 1–4)	19:00–20:00 (t = 5–8)	20:00–21:00 (t = 9–12)
Grade 1	$a=-0.01$	$a=-0.03$	$a=-0.01$
	$b=1.44$	$b=1.8$	$b=1.44$
	$q=1.8$	$q=2.2$	$q=1.8$
	$c_{\mathrm{ele}}=0.3$	$c_{\mathrm{ele}}=0.5$	$c_{\mathrm{ele}}=0.3$
Grade 2	$a=-0.02$	$a=-0.04$	$a=-0.02$
	$b=1.2$	$b=1.5$	$b=1.2$
	$q=1.8$	$q=2.2$	$q=1.8$
	$c_{\mathrm{ele}}=0.3$	$c_{\mathrm{ele}}=0.5$	$c_{\mathrm{ele}}=0.3$
Grade 3	$a=-0.03$	$a=-0.05$	$a=-0.03$
	$b=0.86$	$b=1.08$	$b=0.86$
	$q=1.8$	$q=2.2$	$q=1.8$
	$c_{\mathrm{ele}}=0.3$	$c_{\mathrm{ele}}=0.5$	$c_{\mathrm{ele}}=0.3$

.

We assume that, in $\mathcal{T}=\left[1,2\cdots 12\right]$, there are 3 community operators participating in the bidding mechanism for DR market share. Considering the protection of players’ private information, DR purchaser only broadcasts the overall situation of players, such as community operators’ types and probability distribution. Here, suppose that the proposed DR project is implemented in initial stage, and there exists 3 possible types in 3 community operators: type 1 is that community operator equips gas boiler; type 2 is that community operator equips energy storage; type 3 is that community operator only has DR resource. And, the probability of community operator being each type is equal to 1/3 [33]. Actually, for each community operator, it knows its own type but doe not know its opponents’ types. Assume that the actual type of community operator 1 is type 1, community operator 2 is type 2, and community operator 3 is type 3. As for the randomness of users’ daily energy consumption, assume that $u=0$, $\sigma =0.15x_n^t$. Additionally, the maximal discharging energy in one time slot is 0.4 MWh, and the maximal percentage of heat energy demand in total energy demand varies from 20%–60% during 18:00–21:00. DR resource of 3 community operators in each time slot is shown in Figure 2.

5.2. Equilibrium Solution

According to the Algorithm 1, each community operator will optimize its own bidding strategy by estimating its opponents’ types with the probability distribution. Consequently, the bidding strategy of 3 community operators is presented in Figure 3. It shows that community operator 2 has obtained the highest market share during 18:00–21:00, while community operator 3 has the least market share in the DR bidding. From the aspect of DR resource’s grade, without the assistance of gas boiler and energy storage, community operator 3 can only participate in the bidding for the 3rd grade of DR resource. And in most of time, community operator 1 and 2 can participate in the bidding for the 2nd grade of DR resource. Since the limitation of percentage of heat demand and discharging power of energy storage, community operator 1 and 2 can participate in the bidding for the 1st grade of DR resource in only a few time slot (i.e., t = 7–8 for operator 1 and t = 1 for operator 2). In addition, in time slot t = 5–8, the bidding amount of community operator 3 only achieves 50%–60% of its maximal DR resource, while community operator 2 has achieved above 90% of its maximal DR resource. The main reason is that, since community operator 1 and 2 have equipped gas boiler or energy storage, they can compete more DR amount by promoting the grade of DR resource to reduce the punishment of breach amount. However, community operator 3 only has the original DR resource with high breach percentage, if operator 3 compete more DR amount, then it has to face the high punishment, especially in time slot 5–8.

Table 2. Profit of 3 community operators ($\times {10}^3$ RMB).

Community Operator	Operator 1	Operator 2	Operator 3
Profit from DR purchaser	31.54	36.41	9.97
Profit from gas boiler	1.23	0	0
Profit from energy storage	0	−0.63	0
Total profit	32.77	35.78	9.97

shows the optimal profit of 3 community operators. Obviously, community operator 2 has the highest total profit, while operator 3 has the lowest total profit. It shows that community operators with auxiliary equipment will have a higher profit than the profit of operators without auxiliary equipment. Furthermore, due to the high investment and maintenance cost of energy storage, community operator 2 has the negative profit from energy storage. But, in fact, community operator 2 can have a positive profit from energy storage by increasing the electricity price. As long as the electricity price is lower than the price set by public grid, users will be willing to purchase energy from energy storage. Moreover, comparing the initial profit of 3 community operators (i.e., operator 1 is $28.39\times {10}^3$ RMB, operator 2 is $30.72\times {10}^3$ RMB, and operator 3 is $7.55\times {10}^3$ RMB), one can see that after optimization with the proposed approach, each community operator will have a better performance on the daily profit. Therefore, for community operators in the DR market, they are willing to participate in the optimization of bidding strategy.

5.3. Impact of Energy Storage Capacity

From the result of Section 5.2, although community operator 1 and 2 have equipped gas boiler and energy storage, they can only participate in the 1st grade of DR resource in a few time slots. The main reason is due to the limitation of percentage of heat demand and discharging power of energy storage. Considering the percentage of heat demand in the whole demand is generally stable within a certain range, hence, it is difficult for community operator with gas boiler to further improve the grade of DR resource. However, community operator with energy storage can further improve the grade of resource by configuring larger discharging capacity. Hence, in this section, discharging power of energy storage will be discussed to analyze the influence on the bidding strategy and profit of community operators.

Total profit of 3 community operators and bidding amount of 3 grades of DR resource for different storage capacity are presented in Figure 4 and Figure 5, respectively. In the simulation, discharging energy of storage in 15 min is set to 0 MWh, 0.2 MWh, 0.4 MWh, 0.6 MWh, and the corresponding storage capacity is set to 0 MWh, 2.5 MWh, 5 MWh, 5 MWh, respectively. From Figure 4, one can see that with the increasing of storage capacity, the profit of community operator 2 is increased gradually, while the profits of community operator 1 and 3 are affected in different degree. It is obvious that with the increasing of storage capacity, the grade of DR resource has been gradually improved from 3rd grade to 1st grade. Hence, community operator 2’s profit has a sustained growth. However, since the market price of 3 grades of DR resource will be affected with the change of storage capacity, the other 2 operators’ profits will also be affected. For example, when discharging energy is changed from 0.2 MWh to 0.4 MWh, DR resource in most of time has been improved from 3rd grade to 2nd grade. Consequently, the market price for 3rd grade will be increased while for 2nd grade will be declined. Therefore, operator 1’s profit will be declined and operator 3’s profit will be increased. The bidding amount of different grades of DR resource in Figure 5 has verified the result of community operators’ profit in Figure 4. By analyzing the impact of energy storage capacity on the bidding strategy and profit, it shows that the configuration of storage capacity is an important research point for the proposed DR program, which will be our next research direction in the future.

5.4. Benefit of Auxiliary Equipment

In the above analysis, community operators have different types due to the gas boiler and energy storage. The auxiliary equipment takes an important role in the DR project. In this section, the optimal result of 3 different scenarios will be presented to compare with the above result: (a) scenario 1: community operators only have original DR resource, that is, all community operators belong to type 3; (b) scenario 2: community operators all have equipped gas boiler, that is, all community operators belong to type 1; (c) community operators all have equipped energy storage, that is, all community operators belong to type 2. In other word, the game in 3 scenarios have degenerated into complete information game. Note that, the maximal discharging energy in one time slot is 0.6 MWh for energy storage. Accordingly, the optimal results are presented as follows.

Figure 6 shows the bidding amount of 3 grades of DR resource for different scenarios. From the figure, we can see that auxiliary equipment has a great influence on the bidding strategy. Specially, when community operators do not equip any auxiliary equipment, community operators can only compete for the 3rd grade of DR resource due to the high breach amount of users. While community operators all equip energy storage, the 1st grade of DR resource takes the main market share. Furthermore, Figure 7 shows the bidding amount of 3 community operators in each time slot for scenario 1. Comparing with the result of Figure 3, one can see that the bidding amount has all reduced in each time slot. The main reason is that, since 3 operators all take part in the bidding of the same grade of DR resource, the bidding price has reduced dramatically, but the punishment is still high. Consequently, community operators have to reduce the bidding amount to guarantee the maximization of profit in scenario 1. Based on the bidding amount, profit of 3 community operators for different scenarios can be calculated, which is presented in

Table 3. Profit of 3 community operators for different scenarios ($\times {10}^3$ RMB).

Community Operator	Operator 1	Operator 2	Operator 3
Scenario 1	3.55	3.55	3.55
Scenario 2	29.68	31.27	28.93
Scenario 3	31.67	33.10	30.78
Proposed scenario	38.34	43.69	8.28

. In scenario 1, each player has the same profit due to the same bidding strategy and the profit is also the least in all scenarios. Comparing the results of scenario 1 and proposed scenario, it can be found that the profit of each community operator has been reduced dramatically, especially for operator 1 and 2. Although community operator 3 doesn’t equip any auxiliary equipment in the two cases, its profit also has been influenced, which has been reduced about 60%. Comparing the results of scenario 2 and proposed scenario, it can be found that although community operator 1 both equipped gas boiler, the profit has been reduced to some degree. The main reason is that bidding price for the 2nd grade of DR resource is influenced with the increasing of DR resource amount of the 2nd grade. Similarly, the profit of community operator 2 has also been reduced comparing the results of scenario 2 and proposed scenario. According to the above analysis, auxiliary equipment plays an important role in the aggregation of DR resource and also contributes the increase of community operators’ profit. Therefore, with the promotion of proposed DR program, community operators will gradually transform to type 1 or 2 by equipping gas boiler or energy storage. However, when all market participants have equipped auxiliary equipment, their profit will be decreased in a certain degree.

6. Conclusions

In this paper, a Bayesian game approach is proposed to optimize the bidding strategy of community operators considering the incomplete information. Community operators in DR market lacks the information about other opponents’ types, which needs to be evaluated according to the type combinations and joint probability distribution. Basically, Bayesian game model for each community operator is formulated to maximize the expected daily profit. In order to search the Nash equilibrium of the game, a distribution algorithm is proposed. Simulation results indicate that the proposed DR framework and Bayesian game approach can promote the profit of community operators and also enhance the performance of DR project. Moreover, analysis of energy storage capacity on the optimal result shows that DR market will be influenced by energy storage capacity. And Community operators’ bidding strategy and profit will have a great change when operators change the configuration of auxiliary equipment.

The results in this paper can be extended in several directions. For example, the proposed approach is only applicable for the scenario where the types of participants are discrete and finite, the following works will focus on the application of Bayesian game approach in the scenario with continuous types. In addition, time delay characteristic in the paper is not considered for gas boiler. But in practice, when gas boiler replaces heating device to satisfy users’ demand to reduce breach amount, such process will have a long time delay that has influence on DR scheduling. Therefore, it is an important research point to take the time delay characteristic of gas boiler into consideration in the scheduling of DR resource.