Online Air-Conditioning Energy Management under Coalitional Game Framework in Smart Community

Abstract

Motivated by the potential ability of air conditioning (A/C) units in demand response, this paper explores how to utilize A/C units to increase the profit of a smart community. A coalitional game between the households and the load serving entity (LSE) in a smart community is studied, where the LSE joins by selling renewable energy to householders and providing an energy saving service to them through an A/C controller. The A/C controller is designed to reduce the amount of electricity purchased from the main grid by controlling A/C units. An online A/C energy management algorithm is developed, based on Lyapunov optimization, that considers both the A/C energy consumption and the thermal comfort level of consumers. In order to quantify the contribution of A/C units, the Shapley value is adopted for distribution of the reward among the participating householders and the LSE, based on their contribution. The simulation result verifies the effectiveness of the proposed coalitional game for a smart community and the algorithm for A/C.

1. Introduction

With more electricity-consuming products coming into our daily lives, such as electrical vehicles and air conditioning (A/C) systems, load demand increases dramatically and imposes significant burdens on the existing power grid [1,2]. Along with developments in smart grid technologies—such as two-way communication networks and advanced metering infrastructure [3,4]—the control and monitoring of end use loads at the appliance level in smart households is realized [5,6]. The new information and communication technology infrastructures allow faster and more efficient communications, which offers numerous technical benefits and flexibilities and makes cooperation between utility providers and consumers possible [7,8].

The end users are indeed becoming aware of taking part in the sustainable operation of the energy system within an overall smart community strategy [9,10], rationalizing the amount of energy required by controllable loads [11], or by wisely scheduling run times of smart appliances that are likely to be shifted in time [12,13], and also turning themselves into potential carbon-free generators of energy, through the use of renewable resources [14,15]. Among the smart appliances in households, A/C has traditionally drawn more attention than other end use loads because of its ability to shift energy consumption within a certain time period by storing electricity as thermal energy [16,17]. On the other hand, A/C is one of the major consumers of energy and has a significant influence on the overall energy usage of households. Controlling the energy consumption of A/C can lead to significant energy savings for smart households [18]. The potential of A/C units for load balancing/regulation service has been evaluated in [19]. However, the load control of A/Cs adversely impacts the thermal comfort of customers. Reducing the discomfort of consumers is a top priority in the energy management of A/C units. To this end, studying the energy consumption behavior of A/C units and developing algorithms for effective control without compromising users’ comfort have become an important part of energy management research [20].

In the last few years, some studies have been devoted to the intelligent and interactive management of A/C units. Some papers need to be based on the forecast value of system variables to solve the optimization issues for A/C energy management. For example, [21] developed an innovative event-based approach to minimize the A/C’s day-ahead energy cost, which is based on the forecast information of the system. In [22], a novel intelligent residential A/C system controller is set forth to provide optimal comfort/cost trade-offs for the residents. However, the controller needs to use weather forecast information. Hong et al. [23] aimed to obtain the optimal temperature scheduling for A/Cs according to the day-ahead electricity prices and the forecasted outdoor temperature. In the load control strategy of [24] A/C temperatures were set according to the forecasted prices and outdoor temperatures 24 h in advance. Meanwhile, some other papers formulate the A/C control problem as an optimization problem with complex constraints and need to analyze a large number of historical data or adopt complex traditional algorithms, including model predictive control [25], genetic algorithms [26], and dynamic programming [27], which is of high computational complexity. For instance, a centralized optimal control algorithm with comfortable room temperature consideration is proposed in [28] by controlling the operational set-point of A/C units. However, the control algorithm relies on the population information of the room temperature, which makes it vulnerable to implementation online. Sun et al. [29] developed a methodology which combines stochastic programming and rollout techniques for controlling A/Cs in order to minimize the electricity cost. In [30], a mixed integer multi-scale stochastic optimization problem was formulated, and an algorithm based on model predictive control was proposed for the scheduling A/Cs for a home energy management system.

From the above, the existing studies on A/C energy management have fully reflected the potential ability of A/C units on the demand side. However, most of them are based on forecasting models or complex computation, which is not suitable for practical application. As for the forecasting model, due to the time-dependent uncertainties in weather condition, the unpredictable behavior of customers, and the intermittent nature of renewable energy resources [31], the forecasting error of these sources of information is relatively large [32], making the energy management of A/C much more challenging. On the other hand, the contribution of A/C in the demand response is not quantified in these researches. The householders cannot get the corresponding reward, which is disadvantageous for the long-term cooperation between householders and the load serving entity (LSE). In brief, there is a need for a cooperative strategy with a fair reward allocation scheme and a control algorithm for A/C units with low computational complexity. In this paper, motivated by A/C units’ potential in demand response, a coalitional game for a smart community and an online energy management algorithm for A/C units are proposed. In the algorithm, we tackle the A/C energy management problem with a Lyapunov optimization approach [33], which is a useful technique for solving stochastic network optimization and does not rely on any future information.

The main contributions of this paper are summarized as follows:

• A coalitional game framework is established between smart households and the LSE to improve the revenue of the community. The LSE sells renewable energy to the householders and provides energy saving service for A/C units in the household through an A/C controller. A suitable utility function is proposed to capture the benefit to the coalition. The game is economically beneficial to both the households and the LSE, with the improvement of the self-consumption for renewable energy.
• In the A/C controller provided by the LSE, an online energy management algorithm based on Lyapunov optimization is developed for A/C units to reduce the amount of electricity purchased from the main grid and further increase the revenue of the coalition. It does not rely on any future information, and could quickly make decisions under the fluctuation of weather conditions, renewable generation, load demands, and prices. The decision at each slot can be made by only using the current observations, which has high efficiency and requires small computational resources.
• The Shapley value is adopted to allocate the reward of each member in the coalition, which provides a fair and unique solution for the coalitional game. The contribution of each participating householder is quantified, which is beneficial for the long-term operation of the coalition. The simulation result verifies that a householder with a greater contribution would get more benefits.

The rest of this paper is organized as follows. In Section 2, we begin with the introduction of the system model and the description of the operation mode of the smart community considered in this study. To improve the revenue of the smart community, we propose a coalitional game and the corresponding reward allocation scheme for the participating households and LSE in Section 3. In the A/C controller of each participating household—which is provided by LSE in the game—we develop an online algorithm for A/C to solve the cost minimization problem of each household in Section 4. Simulation results based on real-world data are presented and analysed in Section 5. Finally, some concluding remarks are presented in Section 6.

2. System Model and Operation Mode

2.1. System Model

A graphical representation of the smart community is shown in Figure 1. It consists of N households that are served by an LSE. Each household is equipped with rooftop photovoltaic (PV) panels and an A/C controller which are installed and owned by the LSE. The A/C controller can execute the energy management algorithm to control the operation of the A/C unit.

The rooftop PV system consists of PV panels, inverters, and relevant filters. The output of the PV system is dynamic and difficult to predict, because it depends on the weather conditions. The household load is classified into two groups: A/C and baseline load. A/C is the single controllable appliance in this paper. The control is conducted in terms of switching A/C unit on/off states with a guarantee of thermal comfort levels that are experienced by the householders. For each household i, denote by $S_i\left(t\right)$ the output of the PV system (in units of kWh) and by $L_{i,b}$ the baseline load (in units of kWh) at time t. Define $N_i\left(t\right)=max(S_i\left(t\right)-L_{i,b},0)$ as the surplus PV energy after meeting the baseline load. The surplus PV energy $N_i\left(t\right)$ will be consumed by A/C or be sold to the main grid.

The thermal dynamic behavior of household A/C is modelled based on the equivalent thermal parameters approach [19]. As for household i, when A/C is on over the period $[t_k,t_{k+1}]$, the indoor temperature at time $t_{k+1}$ increases to $T_{k+1}$, which is given by:

$$T_{k+1}=T_0+QR-(T_0+QR-T_k)exp(-\frac{t_{k+1}-t_k}{RC})$$

(1)

When A/C is off over the period $[t_k,t_{k+1}]$, the indoor temperature at time $t_{k+1}$ drops to $T_{k+1}$, which is given by:

$$T_{k+1}=T_0-(T_0-T_k)exp(-\frac{t_{k+1}-t_k}{RC})$$

(2)

where $T_k$ is the indoor temperature at time $t_k$. $T_0$ is the temperature of the surrounding air. Q is the equivalent operational heat rates (W). R is the equivalent thermal resistances (^∘C/W). C is the equivalent heat capacities (J/^∘C) [19,34]. These thermal coefficients can be estimated with statistical and regression techniques by fitting the observed performance data to the equations [35].

2.2. Operation Mode

The LSE installs the PV panels for each householder in the smart community and makes profit by cooperating with householders or selling PV energy to the main grid. When a householder cooperates with the LSE to form a coalition, the householder could utilize its rooftop PV system to satisfy its load demand. The selling price of PV power by the LSE is the same as the price by the main grid. However, the householder will get certain rewards from the LSE, according to their contribution in the coalition. Furthermore, the LSE will help the householder reduce the electricity bill through the A/C controller. If the householder does not cooperate with the LSE, their rooftop PV energy will be totally sold to the main grid and their A/C will operate on its own.

Since the LSE offers rewards and energy saving services to the participating households in the coalition, it is reasonable to assume that the households will be interested in participating in the coalition. On the other hand, as the feed-in tariff of PV power is considerably smaller than the selling price by the main grid, the LSE will also benefit greatly from cooperation with the householders, as the coalition will promote the self-consumption of PV energy.

3. Coalitional Game for the Smart Community

Coalitional game is a branch of game theory that studies whether a group of players—i.e., the households and the LSE in this paper—can be better off if they decide to join in a coalition [36]. A coalitional game is defined by $(\mathcal{N},v)$, where $\mathcal{N}$ is the set of participating players and $v:2^{\mathcal{N}}\to \mathbb{R}$ is a function that assigns every coalition $\mathcal{M}\subseteq \mathcal{N}$ a real number which represents the rewards achieved by $\mathcal{M}$.

3.1. Proposed Coalitional Game

In this paper, the smart households and LSE in $\mathcal{N}$ form a coalition $\mathcal{M}$. The reward that the coalition of $\mathcal{M}$ can obtain is the increased profit under the coalitional game compared with that of the independent mode, under which the participating households do not cooperate with each other and operate independently. The value function $v\left(\mathcal{M}\right)$ for the coalition $\mathcal{M}$ can be defined as:

$$v\left(\mathcal{M}\right)=F_a\left(\mathcal{M}\right)-F_c\left(\mathcal{M}\right),\forall \mathcal{M}\subseteq \mathcal{N}$$

(3)

where $F_a\left(\mathcal{M}\right)$ is the total cost of all players when they do not cooperate with each other. $F_c\left(\mathcal{M}\right)$ is the overall cost of the coalition under the coalitional game. Since Equation (3) represents the revenue of the coalition $\mathcal{M}$, it can be divided in any arbitrary manner between the members. Therefore, the proposed coalitional game $v\left(\mathcal{M}\right)$ is a game with transferrable utility [37].

Theorem 1. 

Consider $\mathcal{N}=\{1,2,\cdots ,N\}$ as the set of N households in the community where the LSE provides PV panels for every householder. The reward will increase for the households as more of them agree to form a coalition $\mathcal{M}\subseteq \mathcal{N}$ with the LSE.

Proof. 

Given the fact that the feed-in-tariff of PV power is lower than the purchasing price from the main grid, the self-consumption of PV power would be beneficial for both the households and the LSE. As more households from $\mathcal{N}$ form a coalition with the LSE to consume the PV power, the electricity purchased from the main grid will reduce, and at the same time, the expected rewards of the coalition will increase. Additionally, the participating householders will get certain rewards according to their contribution to the coalition. Therefore, it would always be beneficial for each household to join the coalition in order to reap greater payments. ☐

According to Theorem 1, the proposed coalitional game will manifest itself in increased profit that can be shared among the participating households and the LSE in the coalition.

3.2. Reward Allocation Scheme

It has been demonstrated in Theorem 1 that the proposed coalition could result in increased profits for the whole group of players. In this section, we will clearly show how the cultivated rewards from the coalition should be shared among the participating players. The Shapley value is adopted to allocate the increased profits among the participating players.

The Shapley value is a solution concept that provides a unique expected payoff allocation for a given coalitional game ($\mathcal{N},v$). It describes an effective approach to the fair allocation of gains obtained by cooperation among the players of a coalitional game. The concept of the Shapley value (which was developed axiomatically by Shapley [38]) considers the relative importance of each player to the game in deciding the payoff to be allocated to the players. ${\varphi }_i\left(v\right)$ denotes the payoff to each player $i\in \mathcal{N}$. Fairness is defined as satisfying the following four axioms, which a payoff allocation scheme would reasonably be expected to satisfy.

(1) (Efficiency) The entire payoff is divided among the participating players without excess remains, which can be described as ${\sum }_{i\in \mathcal{N}}{\varphi }_i\left(v\right)=v\left(\mathcal{N}\right)$.

(2) (Symmetry) Two participants that contribute equally are rewarded equally. Let $\mathcal{S}\cap \{i,j\}=\varnothing$, if $v\left(\mathcal{S}\right)\cup \left\{i\right\}=v\left(\mathcal{S}\right)\cup \left\{j\right\}$ then ${\varphi }_i\left(v\right)={\varphi }_j\left(v\right)$.

(3) (Null player) Participants that do not contribute receive no payoff. Let $\mathcal{S}\cap \left\{i\right\}=\varnothing$, if $v\left(\mathcal{S}\right)\cup \left\{i\right\}=v\left(\mathcal{S}\right)$ then ${\varphi }_i\left(v\right)=0$.

(4) (Linearity) The total payoff rewarded for contributing to two games is the sum of the payoffs that would be awarded for contributing to each of the two games individually. If $v_1$ and $v_2$ are two value functions then ${\varphi }_i(v_1+v_2)={\varphi }_i\left(v_1\right)+{\varphi }_i\left(v_2\right)$.

The Shapley value can be shown to be the only payment distribution method that satisfies these four axioms, with the added benefit that the solution is unique. Mathematically, the Shapley value ${\varphi }_i\left(v\right)$ of a player i is given by:

$${\varphi }_i\left(v\right)=\sum _{\mathcal{S}\subset \mathcal{N}\setminus \left\{i\right\}}\frac{\left|\mathcal{S}\right|!(N-|\mathcal{S}|-1)!}{N!}[v(\mathcal{S}\cup \left\{i\right\})-v\left(\mathcal{S}\right)]$$

(4)

The Shapley value ${\varphi }_i\left(v\right)$ can be interpreted as the expected marginal contribution that player i makes to any coalition of $\mathcal{N}$, assuming all orderings are equally likely. Thus, the Shapley value takes into account all possible coalitional dynamics and negotiation scenarios among the players and comes up with a single unique way of distributing the value $v\left(\mathcal{N}\right)$ of the grand coalition among all the players [39]. In this paper, we adopt the Shapley value as the reward allocation technique for the participating householders and the LSE.

4. Online Energy Management Algorithm in the A/C (Air Conditioning) Controller

4.1. Queues of A/C

We assume that at any time t of the day, each household $i\in \mathcal{M}$, where $\mathcal{M}\subseteq \mathcal{N}$, joins the coalition and gets the energy saving service of the A/C controller. A/C is the main regulation object in the controller. The set of electricity demand for A/Cs can be described as:

$$\mathit{L}\left(t\right)\triangleq [L_1\left(t\right),L_2\left(t\right),\cdots ,L_M\left(t\right)],t\in [1,2,\cdots ,T]$$

(5)

where $\mathit{L}\left(t\right)$ is the set of electricity demand for A/Cs in each participating household at time slot t. M is the number of participating households in coalition $\mathcal{M}$. T is the number of time slots in the whole operation cycle. The electricity demand of A/C in household i needs to satisfy:

$$0\le L_i\left(t\right)\le L_{i,max}$$

(6)

where $L_{i,max}$ is the maximum demand of the A/C in household i per time slot.

The demand of A/C is randomly given in real time based on indoor temperature variation. The standard temperature of A/C is set by householders and can be denoted as $T_{i,set}$. The comfort level of householders can be evaluated through the deviation degree of indoor temperature from $T_{i,set}$. The data in the simulation is all gathered in winter and A/C is mainly for heating. When the indoor temperature is lower than $T_{i,set}$, A/C is supposed to start running to raise the indoor temperature, which dynamically generates the demand $\mathit{L}\left(t\right)$. However, as the temperature variation in an acceptable range does not influence the experience of occupants, the demand does not need to be satisfied immediately. It can be deferred with a guarantee of comfort level for householders. The set of actual electricity consumption for A/Cs can be described as:

$$\mathit{X}\left(t\right)\triangleq [X_1\left(t\right),X_2\left(t\right),\cdots ,X_M\left(t\right)],t\in [1,2,\cdots ,T]$$

(7)

where $\mathit{X}\left(t\right)$ is the set of actual electricity consumption for A/Cs in each participating household at time slot t. The actual electricity consumption of the A/C in household i needs to satisfy:

$$X_i\left(t\right)\in [0,J_i]$$

(8)

where $J_i$ is the rated electricity consumption of the A/C in household i per time slot.

The uncompleted electricity demands of A/C in household i are accumulated, and thus form the A/C queue $H_i\left(t\right)$. The queue length means the amount of electricity demand that is deferred. The set of A/C queues can be described as:

$$\mathit{H}\left(t\right)\triangleq [H_1\left(t\right),H_2\left(t\right),\cdots ,H_M\left(t\right)],t\in [1,2,\cdots ,T]$$

(9)

where $\mathit{H}\left(t\right)$ is the set of A/C queues in each participating household at time slot t. The states of A/C queues change with the random variation of indoor temperature and the execution results of the A/C controller. Specifically, the future state of $\mathit{H}\left(t\right)$ is driven by stochastic arrival $\mathit{L}\left(t\right)$ and execution process $\mathit{X}\left(t\right)$, according to the following dynamic equation.

$$H_i(t+1)=H_i\left(t\right)-X_i\left(t\right)+L_i\left(t\right),i\in [1,2,\cdots ,N]$$

(10)

4.2. Problem Formulation

As the baseline load is fixed, the corresponding electricity cost is constant. The main objective of the A/C controller is to minimize the cost generated by A/C. The electricity purchased from the main grid can be calculated as:

$$G_i\left(t\right)=max[X_i\left(t\right)-N_i\left(t\right),0]=X_i\left(t\right)[1-N_i\left(t\right)/J_i]$$

(11)

where $G_i\left(t\right)$ is the amount of electricity that needs to be purchased from the main grid for the A/C in household i. As the value of $X_i\left(t\right)$ is either 0 or $W_i$, the amount of electricity purchased from the main grid at time slot t in Equation (11) can be further converted to $X_i\left(t\right)[1-N_i\left(t\right)/J_i]$. If $X_i\left(t\right)=0$, the amount of $G_i\left(t\right)$ is 0; If $X_i\left(t\right)=J_i$, the corresponding amount is $X_i\left(t\right)-N_i\left(t\right)$.

The electricity cost generated by A/C in household i at time slot t can be calculated as:

$$F_i\left(t\right)=c\left(t\right)·G_i\left(t\right)$$

(12)

where $c\left(t\right)$ is the electricity price of the main grid.

The A/C controller is designed to lower the cost as much as possible. The online A/C energy management algorithm can be described as: at time slot t, the A/C controller receives information about $c\left(t\right)$, $N_i\left(t\right)$, and $H_i\left(t\right)$. On the premise of guaranteeing comfort level and stability of A/C queues $\mathit{H}\left(t\right)$, the A/C controller aims at minimizing the electricity cost generated by A/C by controlling the A/C operation statuses. For household i, the problem can be expressed as:

$$\begin{array}{cc}\hfill min& \underset{t\to \infty }{lim}\frac{1}{t}\sum _{\tau =0}^{t-1}{F}_i\left(\tau \right)\hfill \\ \hfill s.t.& \left(8\right),A/Cqueuesaremeanratestable\forall i,t.\hfill \end{array}$$

(13)

The A/C queue $H_i\left(t\right)$ is called mean rate stable [40] if

$$\underset{t\to \infty }{lim}\frac{\mathbb{E}\left\{\right|H_i\left(t\right)\left|\right\}}{t}=0$$

(14)

The algorithm in the A/C controller is supposed to guarantee the stability of A/C queues, in addition to minimizing the electricity cost. The thermal comfort level of householders should be ensured, which means that the A/C demands will not be put off infinitely. The stability of A/C queues can be interpreted as the principle that the A/C load demand cannot be unlimitedly accumulated, and should be responded to in a reasonable delay range.

4.3. Lyapunov Optimization

For household i, Lyapunov function $L\left(H_i\left(t\right)\right)\triangleq H_i{\left(t\right)}^2/2$ is defined as a scalar measure of the congestion in the A/C queue. The Lyapunov drift for slot t is defined as:

$$\Delta \left(H_i\left(t\right)\right)\triangleq \mathbb{E}\{L\left(H_i(t+1)\right)-L\left(H_i\left(t\right)\right)\}$$

(15)

where $\Delta \left(H_i\left(t\right)\right)$ is the variation of the Lyapunov function over a time slot, which represents the stability of the A/C queue. The variation depends on the random arrival of demands for A/C and the decisions of A/C controller.

Based on the drift-plus-penalty method [40], an online A/C energy management algorithm is designed. At every time slot, based on the information acquired by the A/C controller, the value of $\mathit{X}\left(t\right)$ is decided to minimize the drift-plus-penalty expression as follows

$$\Delta \left(H_i\left(t\right)\right)+W_i·\mathbb{E}\left\{F_i\left(t\right)\right\}$$

(16)

where $W_i$ is the weight parameter, which illustrates how much importance the householder attaches to the cost minimization. If $W_i=0$, it corresponds to the pure system stability problem by minimizing the Lyapunov drift. Minimizing $\Delta \left(H_i\left(t\right)\right)$ alone would push the A/C towards lower backlog, but would incur a large penalty on the cost. Therefore, the algorithm is designed to minimize the weighted sum of drift and penalty. $W_i$ represents the tradeoff between stabilizing A/C queues and minimizing the cost. It is set based on the A/C power level and the user preference between cost and delay.

Lemma 2. 

For any control policy that satisfies the constraints in (13), the drift-plus-penalty expression satisfies

$$\Delta \left(H_i\left(t\right)\right)+W_i·\mathbb{E}\left\{F_i\left(t\right)\right\}\le B_i+W_i·\mathbb{E}\left\{c\left(t\right)X_i\left(t\right)[1-N_i\left(t\right)/J_i]\right\}+\mathbb{E}\left[H_i\left(t\right)(L_i\left(t\right)-X_i\left(t\right))\right]$$

(17)

where the constant $B_i$ is defined as

$$B_i\triangleq \frac{{L_{i,max}}^2+{J_i}^2}{2}$$

(18)

Proof. 

A bound can be computed on the Lyapunov drift as follows:

$$\begin{array}{cc}\hfill \Delta \left(H_i\left(t\right)\right)& =\mathbb{E}\{L\left(H_i(t+1)\right)-L\left(H_i\left(t\right)\right)\}=\frac{1}{2}\mathbb{E}[H_i{(t+1)}^2-H_i{\left(t\right)}^2]\hfill \\ & =\frac{1}{2}\mathbb{E}[{(H_i\left(t\right)-X_i\left(t\right)+L_i\left(t\right))}^2-H_i{\left(t\right)}^2]\hfill \\ & \le \frac{1}{2}\mathbb{E}[L_i{\left(t\right)}^2+X_i{\left(t\right)}^2+2H_i\left(t\right)(L_i\left(t\right)-X_i\left(t\right))]\hfill \\ & =\frac{1}{2}\mathbb{E}[L_i{\left(t\right)}^2+X_i{\left(t\right)}^2]+\mathbb{E}\left[H_i\left(t\right)(L_i\left(t\right)-X_i\left(t\right))\right]\hfill \end{array}$$

(19)

and thus $B_i$ can be defined as Equation (18). Using Equations (6) and (8), we have

$$\Delta \left(H_i\left(t\right)\right)\le B_i+\mathbb{E}\left[Q_i\left(t\right)(L_i\left(t\right)-X_i\left(t\right))\right]$$

(20)

adding the cost to both sides, we thus have Equation (17). ☐

The original problem Equation (13) is transformed into the following problem by minimizing the right-hand-side of Equation (17).

$$\begin{array}{cc}\hfill min& X_i\left(t\right)\{W_i·c\left(t\right)[1-N_i\left(t\right)/J_i]-H_i\left(t\right)\}\hfill \\ \hfill s.t.& \left(8\right),\forall i,t.\hfill \end{array}$$

(21)

The above problem can be further reduced to the following simple threshold rule:

$$X_i\left(t\right)=\left\{\begin{array}{cc}0,\hfill & W_i·c\left(t\right)[1-N_i\left(t\right)/J_i]-H_i\left(t\right)>0\hfill \\ J_i,\hfill & W_i·c\left(t\right)[1-N_i\left(t\right)/J_i]-H_i\left(t\right)<0\hfill \end{array}\right.$$

(22)

It can be seen from (22) that the implementation of Lyapunov optimization is relatively simple, compared to traditional optimization algorithms. It does not need a priori statistical knowledge, and only relies on the instant information about system states at a given moment. The original complex A/C energy management problem is transformed into a linear programming problem, which largely reduces the computational complexity. Furthermore, it has no curse of dimensionality, and hence can be easily applied in extended formulations with multiple households and multiple A/C queues.

According to Equation (16), the weight parameter $W_i$ (which is set by the householder) is closely related to the deferrable degree of the A/C. With large $W_i$, householders can achieve a lower electricity bill, but will suffer from the excessive reduction of indoor temperature. With small $W_i$, the comfort level of householders is guaranteed, but the electricity cost would be relatively higher. According to the setting of $W_i$, the participating households in the community are classified into two groups: the group with higher deferrable degree A/C_h, and the group with lower deferrable degree A/C_l. A/C_h and A/C_l both cooperate with the load serving entity (LSE) to form a coalition. Thus, the set of participants in the coalition can be represented as $\mathcal{M}$={A/C_h, A/C_l, LSE}. The reward of A/C_l, A/C_h, and LSE will be allocated via the Shapley value. As for the households in A/C_h and A/C_l, their reward will be distributed according to their contributions to the consumption of PV energy. The overall flowchart of the operation strategy in the smart community is shown in Figure 2.

5. Case Study

5.1. Basic Data

In order to evaluate the effectiveness of the proposed coalitional game and online A/C energy management algorithm, a smart community including six households and an LSE was selected as the research object. The simulation was conducted in Matlab for one day, and the time resolution was 10 min. Considering the fact that the households are close to each other in geographic distance, the PV generation of these households takes the same data. Meanwhile, in order to compare the difference between A/C_h and A/C_l, the baseline load demand of these households also takes the same data. The data of PV generation and baseline load are all collected from the real-time measurements from real households, which are shown in Figure 3.

The time-of-use price $c\left(t\right)$ adopted in the simulation is shown in

Table 1. Time-of-use price.

Time	Price (RMB/kWh)
10:00–15:00, 18:00–21:00	1.37
7:00–10:00, 15:00–18:00, 21:00–23:00	0.8
23:00–7:00	0.37

.

The rated power of A/Cs in these households is set as 3 kW. According to the setting of $W_i$, the six households are classified into A/C_h and A/C_l. The specific parameters of A/Cs in these households are shown in

Table 2. A/C (air conditioning) parameters of households.

A/Ch			A/Cl
Household	Standard Temperature (${}^{\circ }$C)	$W_i$	Household	Standard Temperature (${}^{\circ }$C)	$W_i$
1	20	8	4	20	5
2	21	8	5	21	5
3	22	8	6	22	5

.

In order to verify the effectiveness of the proposed coalitional game and energy management algorithm, the independent operation mode is taken as the comparison case. The comparison of the features between the cooperative mode and independent mode is shown in

Table 3. Comparison of cooperative mode and independent mode.

Operation Mode	Householders	Utility Company
Cooperative	PV energy + A/C controller + Rewards	sell PV energy and share rewards to householders
Independent	operate on their own	sell all PV energy to the main grid

.

5.2. Analysis of Results

In the proposed coalitional game, householders can get certain rewards and the energy saving service of A/C controller. At the same time, the self-consumption of PV energy is improved, which will benefit the LSE greatly. The results of a comparison between the proposed coalitional game and independent operation mode are shown in

Table 4. Comparison of revenue between the two operation modes.

Operation Mode	Independent Mode	Cooperative Mode
Revenue (RMB)	$-248.8419$	$-178.9992$
Amount of electricity purchased from the main grid (kWh)	360.9796	274.5141

.

It can be seen from Table 4 that the revenue under the proposed game is significantly improved. The value of the coalition is just the increased profit, which can be expressed as $v\left(\mathcal{M}\right)=69.8427$. The amount of electricity purchased from the main grid is considerably reduced under the cooperative mode. The specific comparison of real-time results for purchased electricity is shown in Figure 4. It can be seen that during the most times in a day, householders need to purchase more electricity from the main grid under the independent operation mode.

The reward of the coalition is closely related to the configuration of PV generation in each household. The simulation analyses the effect of PV generation to the reward. The result of sensitivity analysis is shown in Figure 5. With the increase of PV capacity, as the amount of electricity that needs to be purchased from the main grid decreases, the reward of the coalition increases.

As for the effect of online energy management algorithm in A/C controller, the cost comparison for A/C_h and A/C_l is shown in

Table 5. Cost comparison of A/C_h (higher deferrable degree) and A/C_l (lower deferrable degree).

Group	A/Ch	A/Cl
Total cost (RMB)	88.5773	90.4218
Cost from the main grid (RMB)	95.4719	97.3211
Average backlog of A/C queues (kWh)	731.8333	439.6667

.

The simulation results in Table 5 coincide with the theoretical analysis in Section 4.3. The performance of the online energy management algorithm in the A/C controller is closely associated with the setting of weight parameter $W_i$. When $W_i$ gradually increases, the average backlog of queues would increase, and the cost would decrease. As described in Section 5.1, the only difference between A/C_h and A/C_l is the setting of $W_i$. The cost of A/C_h is lower than A/C_l, but the backlog of A/C queues in A/C_h is larger than that in A/C_l. Thus, the theoretical analysis is validated. As for the guarantee of comfort level, the comparison of the indoor temperature in these households and the outdoor temperature is shown in Figure 6.

From Figure 6, it can be seen that the lowest indoor temperature in these households is 19 ^∘C, which does not influence the comfort level of householders in winter. When the queue of an A/C has accumulated to a certain degree, or the price drops, the A/C controller will start the A/C to guarantee the comfort level and economic efficiency of the householder.

5.3. Allocation Result of Rewards

According to (4), the Shapley value of the participating players in the coalition is calculated in

Table 6. Reward of A/C_h.

$\mathit{S}\mathbf{\cup }\mathbf{\left\{}\mathit{i}\mathbf{\right\}}$	A/Ch	A/Ch, A/Cl	A/Ch, LSE	A/Ch, A/Cl, LSE
$v(\mathcal{S}\cup \{i\left\}\right)$	0	0	35.8436	69.8427
$v\left(\mathcal{S}\right)$	0	0	0	33.9991
$\frac{\left|\mathcal{S}\right|!(N-|\mathcal{S}|-1)!}{N!}$	1/3	1/6	1/6	1/3
$\frac{\left|\mathcal{S}\right|!(N-|\mathcal{S}|-1)!}{N!}[v(\mathcal{S}\cup \left\{i\right\})-v\left(\mathcal{S}\right)]$	0	0	5.9739	11.9479

,

Table 7. Reward of A/C_l.

$\mathit{S}\mathbf{\cup }\mathbf{\left\{}\mathit{i}\mathbf{\right\}}$	A/Cl	A/Ch, A/Cl	A/Cl, LSE	A/Ch, A/Cl, LSE
$v(\mathcal{S}\cup \{i\left\}\right)$	0	0	33.9991	69.8427
$v\left(\mathcal{S}\right)$	0	0	0	35.8436
$\frac{\left|\mathcal{S}\right|!(N-|\mathcal{S}|-1)!}{N!}$	1/3	1/6	1/6	1/3
$\frac{\left|\mathcal{S}\right|!(N-|\mathcal{S}|-1)!}{N!}[v(\mathcal{S}\cup \left\{i\right\})-v\left(\mathcal{S}\right)]$	0	0	5.6665	11.3330

and

Table 8. Reward of LSE.

$\mathit{S}\mathbf{\cup }\mathbf{\left\{}\mathit{i}\mathbf{\right\}}$	LSE	A/Ch, LSE	A/Cl, LSE	A/Ch, A/Cl, LSE
$v(\mathcal{S}\cup \{i\left\}\right)$	0	35.8436	33.9991	69.8427
$v\left(\mathcal{S}\right)$	0	0	0	0
$\frac{\left|\mathcal{S}\right|!(N-|\mathcal{S}|-1)!}{N!}$	1/3	1/6	1/6	1/3
$\frac{\left|\mathcal{S}\right|!(N-|\mathcal{S}|-1)!}{N!}[v(\mathcal{S}\cup \left\{i\right\})-v\left(\mathcal{S}\right)]$	0	5.9740	5.6665	23.2809

.

According to the Shapley value method, the reward of the three players is shown in

Table 9. Reward of players.

i	A/Ch	A/Cl	LSE
${\varphi }_i\left(v\right)$	17.9218	16.9995	34.9214

.

Table 9 shows that the reward of A/C_h is more than the one of A/C_l. Combined with Table 5, it can be known that the households in A/C_h adopt a larger value of $W_i$, which provides the A/C controller with larger space for regulation and thus makes greater contribution to the cost reduction of the grand coalition. Therefore, the households in A/C_h get more payment in the coalition. According to the consumption of PV energy, the reward of each household is shown in

Table 10. Reward of each household.

A/Ch		A/Cl
Household	Reward (RMB)	Household	Reward (RMB)
1	5.8976	4	5.6088
2	5.9765	5	5.6282
3	6.0476	6	5.7625

.

6. Conclusions

This paper has proposed a theoretical coalitional game approach for the cooperation between households and LSE in a smart community. In the coalitional game, householders can be considerably rewarded and acquire energy saving service from the A/C controller by purchasing PV energy from the LSE. Moreover, in the A/C controller that the LSE provides for the participating householders, an online energy management algorithm based on Lyapunov optimization is developed for the control of A/C units to reduce the amount of electricity purchased from the main grid and further increase the revenue of the coalition. This algorithm transfers the original A/C energy management problem into a linear programming problem, which effectively reduces the computational complexity. The Shapley value has been adopted to divide the reward of the coalition among the participating members, based on their contribution. Through simulation with realistic data from the residential community, we have shown that the proposed coalitional game and A/C energy management algorithm can effectively increase the profits of householders and the LSE compared to the noncooperative case.

Our results demonstrate that the proposed coalitional game has significant potential to serve as an effective means of improving the profitability of the LSE and cutting the expenses of householders. As for practical application, the operation mode based on the proposed coalitional game can be used to help community planners to improve the total revenue. Particularly, the time complexity of the proposed algorithm is low and the required computational resource is small, which makes the algorithm suitable to be integrated into the embedded system in smart households. Moreover, the algorithm can be implemented in a distributed way, which has fewer communication costs. The A/C controller based on the proposed algorithm can be used to determine the least-cost schedules of A/C while guaranteeing householders’ comfortable experiences. When setting the weight parameters in the A/C controller, householders are able to choose a larger value in order to get more payments.