Optimizing Energy Management in Microgrids Based on Different Load Types in Smart Buildings

: This paper presents an energy management strategy (EMS) based on the Stackelberg game theory for the microgrid community. Three agents or layers are considered in the proposed framework. The microgrid cluster (MGC) refers to the agent that coordinates the interactions between the microgrids and the utility grid. The microgrid agent manages the energy scheduling of its own consumers. The third agent represents the consumers inside the microgrids. The game equilibrium point is solved between different layers and each layer will beneﬁt the most. First, an algorithm performs demand response in each microgrid according to load models in smart buildings and determines the load consumption for each consumer. Then, each microgrid determines its selling price to the consumers and the amount of energy required to purchase from the utility grid to achieve the maximum proﬁt. Finally, the balance point will be obtained between microgrids by the microgrid cluster agent. Moreover, the proposed method uses various load types at different times based on real-life models. The result shows that considering these different load models with demand response increased the proﬁt of the user agent by an average of 22%. The demand response is implemented by the time of use (TOU) model and real-time pricing (RTP) in the microgrid.


Introduction
Demand-side energy management and the uncertainty of renewable energies such as solar and wind are becoming increasingly important in smart power systems [1,2]. Energy management systems (EMS) are required to realize the ability of a wide range of applications in a microgrid. The EMS can coordinate the distributed energy resources (DER) with their loads, while each DER has its own local controller [3]. The microgrids use the power supply from the grid, RESs, and their local energy storage systems to meet power demands. Moreover, microgrid clusters can make an intermediate coordination layer between the microgrids and the utility grid [4]. Therefore, the operation of the microgrid cluster can be divided into three levels of agents: the microgrid cluster, the microgrid, and the user [5].
Several research works have focused on energy management in microgrids [6]. The authors in [7] have proposed a hybrid two-stage energy management plan. The first stage is based on the day-ahead energy generation schedule market. The second stage is an intraday market that acts based on the hourly time resolution. However, the proposed method is unsuitable for energy exchange in different microgrids. In [8], a two-stage energy management model has been proposed with a different approach. The power balance is realized in the upper layer between different microgrids. In the down-stream layer, each microgrid has to provide a power balance between different consumptions.
Today, multiagent systems are used for the planning and scheduling of microgrids and consumptions based on their contributions to providing smart grid technologies such as demand response. In [9], a scheduling method has been proposed to optimize

•
A new method based on the load model of buildings in a microgrid is presented. The method is based on the Stackelberg game model between microgrids and consumers; • A demand response approach is developed to increase the consumption during light load conditions and reduce the amount of peak load regarding the amount of energy consumption and the price of the network; • Energy management and exchange in each microgrid and between the microgrid cluster agents are written as linear equations according to the principles of linearization. This issue helps to reduce the calculations and results in the global equilibrium point.
The paper is organized as follows. The theory of each agent model is explained in detail in Section 2. The model of load and the model of game theory are described in Sections 3 and 4, respectively. The case study and simulation results are presented in Section 5, and finally, the paper is concluded in Section 6.

Model Description
In this study, the modeling of microgrids and microgrid clusters is based on a game theory with one leader and several followers. The models can be scheduled independently in parallel. The microgrid cluster is based on real-time prices that are determined by the utility grid (UG). Microgrid cluster refers to the agent that coordinates the interactions between microgrids and the utility or upstream network. The microgrid agent refers to the coordinator of individual microgrids which facilitates the interactions with its downstream or domestic consumers.

Microgrid Cluster Agent Model
The microgrid cluster profit model is according to the price of buying and selling energy to the upstream network. The cost of energy for each microgrid is expressed in Equation (1): [λ s,t P s,i,t − λ b,t P b,i,t ]+ 24 ∑ t=1 [µ s,t P gs,t − µ b,t P gb,t ] (1) where i represents each microgrid, N is the number of microgrids, t is the symbol of day and night hours, λ is the price of purchasing and selling energy to microgrids, and µ is the price of buying and selling electricity to the utility grid. Some constraints must be considered in the model. One of these constraints is the power balance in each microgrid, which can be expressed as Equation (2): where BigM is a large number used to linearize the multiplication of a binary number and is a continuous variable. Purchase and sale price limits are considered as another constraint in this model, which is given in Equation (3): The price of selling energy to the utility grid is set at the highest electricity price. In contrast, the price of purchasing energy from the utility grid is set at the lowest price.

Microgrid Agent Model
The microgrid agent model is described based on the price of prosumer's energy consumption, and the profit of each agent is optimized and maximized separately. The energy balance is based on the maximum use of renewable energy. The profit model of the microgrid operator is given in Equation (4): where γ and l are the selling price of power to the consumer and the power consumption of consumers, respectively. C om represents the cost of equipment maintenance and C ex is the cost of power transmission from the grid. C co2 and C em are the cost of carbon dioxide pollution and other gas pollution such as nitrogen and sulfur, respectively. Each of the costs are calculated by the optimization of the profit function. Equation (5) represents the power balance between microgrids and consumers.
where P is the power generation of various units such as solar cells, wind turbines, and batteries. F is a binary variable to determine the state of the system, which indicates whether the microgrid is in buying or selling mode. Another constraint is applied by limitation of the power generation of gas-fueled units, which is described in Equation (6):

User or Consumer Agent Model
On the consumer's side, the profit model is based on the ratio of consumption of power to the demand purchased by consumers, which is described in Equation (7): where α and β are the profit function coefficients of consumers, which are determined according to the network conditions. The cost for each consumer is obtained from Equation (8) according to the energy purchase price.
The optimization function model of consumers is according to Equation (9): One of the most important issues is the demand response by consumers, which is considered as a hard constraint in this study (Equation (10)):

Load Model
The main idea of this paper is to model different types of loads. For this reason, different loads are considered in the optimization of consumption to make the demand response more realistic and suitable. This hypothesis makes the model more profitable for the consumer compared to that in ref. [5], which considers the demand response as a fixed coefficient of the demand amount.
Results bring more profit to the consumers and have a better effect in reducing the peak load. The implementation of this response is highly dependent on the expansion of the internet inside buildings and home appliances that can be connected to the internet.
The existence of the internet makes it possible to have control over the load models and reduce or even stop their consumption at certain hours of the day and night. It is also possible to transfer the interrupted loads to other hours of the day and night when energy consumption is lower, such as the load of refrigerators. Figure 1 shows a conceptual view of residential smart buildings and the types of loads in these buildings.
The main idea of this paper is to model different types of loads. For this reason, dif-ferent loads are considered in the optimization of consumption to make the demand response more realistic and suitable. This hypothesis makes the model more profitable for the consumer compared to that in ref. [5], which considers the demand response as a fixed coefficient of the demand amount.
Results bring more profit to the consumers and have a better effect in reducing the peak load. The implementation of this response is highly dependent on the expansion of the internet inside buildings and home appliances that can be connected to the internet.
The existence of the internet makes it possible to have control over the load models and reduce or even stop their consumption at certain hours of the day and night. It is also possible to transfer the interrupted loads to other hours of the day and night when energy consumption is lower, such as the load of refrigerators. Figure 1 shows a conceptual view of residential smart buildings and the types of loads in these buildings. The demand response can be implemented based on the price, the air temperature, etc. Then, the power consumption can be generally considered according to Equation (11): where p and q are constants that are determined by the history of consumption based on the different loads. These constants can be expressed in terms of the consumption culture and air temperature conditions. There are various loads in the buildings in the microgrid. For instance, a coefficient of price can be considered based on the demand for air conditioning loads as given in (12): According to (13), different costs can be considered per hour for the lighting loads. If the price is higher than its upper level, for example, then it is equal to k times the amount of lighting demand, if it is between the lower and upper levels of the price, it will be equal to k' times the quantity demanded, and if the price is less than the lower level, it will not react to the price. Therefore, it is possible to consider more load steps in order to achieve The demand response can be implemented based on the price, the air temperature, etc. Then, the power consumption can be generally considered according to Equation (11): where p and q are constants that are determined by the history of consumption based on the different loads. These constants can be expressed in terms of the consumption culture and air temperature conditions. There are various loads in the buildings in the microgrid. For instance, a coefficient of price can be considered based on the demand for air conditioning loads as given in (12): According to (13), different costs can be considered per hour for the lighting loads. If the price is higher than its upper level, for example, then it is equal to k times the amount of lighting demand, if it is between the lower and upper levels of the price, it will be equal to k' times the quantity demanded, and if the price is less than the lower level, it will not Energies 2023, 16, 73 6 of 17 react to the price. Therefore, it is possible to consider more load steps in order to achieve more accuracy from lighting loads. This increases the calculation complexity.
The working time of some loads, such as refrigerators, can be completely controlled by the internet. Of course, the working time shift should not be more than the maximum amount based on the price. According to (14), the total consumption should be equal to the total demand consumption for a day and a night.
Some loads are not responsive to price changes, and they do not change their consumptions. The response value of the load that causes the profit of the consumer agent is considered equal to the sum of the response values of different loads. If the amount of responsiveness is assumed to be constant, the amount of consumer benefit will be lower. Another constraint is the range of power consumption in the consumer section, which is expressed in Equation (15):

Game Model
In this study, a non-cooperative Stackelberg game is used to maximize the profit of each agent. First, the leader gives a strategy in this type of game. Then, the follower, according to the leader's strategy, gives the optimal response and passes the strategy to the leader. The optimal points of the game should be obtained for each agent, and the optimal point of the system should be determined in successive repetitions of the game. The microgrid cluster determines the prices of buying and selling energy in this system. The microgrid agents balance power based on the prices and the optimal point and the price of selling energy is determined by the consumers with the most profit. The consumer agent will respond and determine the amount of consumption to obtain the most profit based on this price. The buying and selling prices of the microgrid cluster, the selling price of energy in microgrids, the power bought and sold in microgrids, and the consumption power of consumers are determined after reaching the equilibrium point of the system. The profit of all three agents is maximized by changing the strategy; the profit of each of the factors will not increase at this point. Therefore, the method in the game includes two steps: (1) In the first layer, the microgrid cluster determines the purchase and sale prices of energy for each microgrid. Then, the microgrids determine their buying and selling power to get the most profit, and balance the power according to the prices. (2) In the second layer, each microgrid determines the prosumer's energy to consumers based on the demand of consumers and consumption history. The consumers change their consumption power by using demand response to achieve maximum profit according to the prices.
In each of these stages, if the profit difference between each agent in one iteration and the previous iteration is less than the desired threshold, the iteration will stop, and the equilibrium point of the Stackelberg game between the agents will be obtained. Figure 2 displays the different layers of the network, the power transmission path between the layers, and the game between these layers.

Simulation Results
The simulation was performed by using General Algebraic Modeling Software (GAMS) and MATLAB software. The purchase and sale prices of energy in the cluster layer of microgrids and the energy sales prices of each of the microgrids to consumers were determined in MATLAB software by using particle swarm optimization (PSO). The prices were used as inputs to the optimization problem developed in GAMS software. Figure 3 shows the real-time purchase and sale prices of energy from the upstream network.

Simulation Results
The simulation was performed by using General Algebraic Modeling Software (GAMS) and MATLAB software. The purchase and sale prices of energy in the cluster layer of microgrids and the energy sales prices of each of the microgrids to consumers were determined in MATLAB software by using particle swarm optimization (PSO). The prices were used as inputs to the optimization problem developed in GAMS software. Figure 3 shows the real-time purchase and sale prices of energy from the upstream network.

Simulation Results
The simulation was performed by using General Algebraic Modeling Software (GAMS) and MATLAB software. The purchase and sale prices of energy in the cluster layer of microgrids and the energy sales prices of each of the microgrids to consumers were determined in MATLAB software by using particle swarm optimization (PSO). The prices were used as inputs to the optimization problem developed in GAMS software. Figure 3 shows the real-time purchase and sale prices of energy from the upstream network.       Table 1 shows the specifications of production units in each microgrid.   4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Table 1 shows the specifications of production units in each microgrid.   Table 1 shows the specifications of production units in each microgrid. Two different models are considered for the game theory model in the simulation: (a) The consumer agent is based on a fixed coefficient of the total load response; (b) The consumer agent can change its energy consumption based on the type of load, the price of the load, and load response. Figure 6 shows the output of energy purchase and sale price by the microgrid cluster in MATLAB software.    First, the game is considered at the lowest level (the level of the consumer agent) based on the set prices of the microgrid contrary to ref. [1]. The amount of consumption should be determined according to the responsiveness of the load and change in consumption. The agent starts the game after determining the amount of consumption and announcing it to the microgrid agent. Then, the amount of power is requested for buying and selling with the microgrid cluster by the game and is announced to a higher level. The procedure of the game is ten iterations, if the difference in the profit value of the microgrids in two consecutive iterations is less than a certain threshold (0.1 in this program), the loop will be stopped, and the results are announced.     First, the game is considered at the lowest level (the level of the consumer agent) based on the set prices of the microgrid contrary to ref. [1]. The amount of consumption should be determined according to the responsiveness of the load and change in consumption. The agent starts the game after determining the amount of consumption and announcing it to the microgrid agent. Then, the amount of power is requested for buying and selling with the microgrid cluster by the game and is announced to a higher level. The procedure of the game is ten iterations, if the difference in the profit value of the microgrids in two consecutive iterations is less than a certain threshold (0.1 in this program), the loop will be stopped, and the results are announced. First, the game is considered at the lowest level (the level of the consumer agent) based on the set prices of the microgrid contrary to ref. [1]. The amount of consumption should be determined according to the responsiveness of the load and change in consumption. The agent starts the game after determining the amount of consumption and announcing it to the microgrid agent. Then, the amount of power is requested for buying and selling with the microgrid cluster by the game and is announced to a higher level. The procedure of the game is ten iterations, if the difference in the profit value of the microgrids in two consecutive iterations is less than a certain threshold (0.1 in this program), the loop will be stopped, and the results are announced.

Model 1
The results of the proposed model are presented assuming that the consumer agent can only change its consumption with a fixed coefficient of demand. Figures 8-10 show the variations in the amount of consumer load in the game using model 1. As can be seen from the figures, the consumers reduce the load during the hours when the prices are high from the target microgrid, and this also reduces the load at peak consumption so that they can increase their profit by responding to the load.

Model 1
The results of the proposed model are presented assuming that the consumer agent can only change its consumption with a fixed coefficient of demand. Figures 8-10 show the variations in the amount of consumer load in the game using model 1. As can be seen from the figures, the consumers reduce the load during the hours when the prices are high from the target microgrid, and this also reduces the load at peak consumption so that they can increase their profit by responding to the load.

Model 1
The results of the proposed model are presented assuming that the consumer agent can only change its consumption with a fixed coefficient of demand. Figures 8-10 show the variations in the amount of consumer load in the game using model 1. As can be seen from the figures, the consumers reduce the load during the hours when the prices are high from the target microgrid, and this also reduces the load at peak consumption so that they can increase their profit by responding to the load.   The amount of power bought from and sold to the microgrid cluster, and the amount of power produced by each microgrid is determined by the game in the microgrid layer based on the prices set by the upstream network. The consumption power is obtained from the consumer layer, while the microgrid cannot change this consumption. Each microgrid is trying to produce the maximum renewable energy to increase its profits due to the high production costs of gas units. Figures 11-13 show the variation in the production power of each microgrid and the power bought and sold with the microgrid cluster. These figures show that each microgrid uses the maximum capacity of renewable generation to increase its profit in the hours when it can use them. Additionally, during peak hours for energy supply, consumers buy energy from the upstream network. The amount of power bought from and sold to the microgrid cluster, and the amount of power produced by each microgrid is determined by the game in the microgrid layer based on the prices set by the upstream network. The consumption power is obtained from the consumer layer, while the microgrid cannot change this consumption. Each microgrid is trying to produce the maximum renewable energy to increase its profits due to the high production costs of gas units. Figures 11-13 show the variation in the production power of each microgrid and the power bought and sold with the microgrid cluster. These figures show that each microgrid uses the maximum capacity of renewable generation to increase its profit in the hours when it can use them. Additionally, during peak hours for energy supply, consumers buy energy from the upstream network. The amount of power bought from and sold to the microgrid cluster, and the amount of power produced by each microgrid is determined by the game in the microgrid layer based on the prices set by the upstream network. The consumption power is obtained from the consumer layer, while the microgrid cannot change this consumption. Each microgrid is trying to produce the maximum renewable energy to increase its profits due to the high production costs of gas units. Figures 11-13 show the variation in the production power of each microgrid and the power bought and sold with the microgrid cluster. These figures show that each microgrid uses the maximum capacity of renewable generation to increase its profit in the hours when it can use them. Additionally, during peak hours for energy supply, consumers buy energy from the upstream network.    Figure 14 shows the total power bought and sold from the microgrid cluster to the microgrids and the upstream network.    Figure 14 shows the total power bought and sold from the microgrid cluster to the microgrids and the upstream network.   Figure 14 shows the total power bought and sold from the microgrid cluster to the microgrids and the upstream network.   Figure 14 shows the total power bought and sold from the microgrid cluster to the microgrids and the upstream network.

Model 2
In this model, the consumer agent can respond appropriately according to the type of load and the purchase price of electricity from the upstream network. Figure 15 displays the change in energy purchases and sale prices to earn more profit by the game in the microgrid operators with microgrids and the upstream network. The microgrid operator raises the purchase price to encourage the microgrid to consume more energy at times when the price of purchasing energy from the upstream grid is high (gray color curve).

Model 2
In this model, the consumer agent can respond appropriately according to the type of load and the purchase price of electricity from the upstream network. Figure 15 displays the change in energy purchases and sale prices to earn more profit by the game in the microgrid operators with microgrids and the upstream network. The microgrid operator raises the purchase price to encourage the microgrid to consume more energy at times when the price of purchasing energy from the upstream grid is high (gray color curve).

Model 2
In this model, the consumer agent can respond appropriately according to the type of load and the purchase price of electricity from the upstream network. Figure 15 displays the change in energy purchases and sale prices to earn more profit by the game in the microgrid operators with microgrids and the upstream network. The microgrid operator raises the purchase price to encourage the microgrid to consume more energy at times when the price of purchasing energy from the upstream grid is high (gray color curve).     Figure 17 shows the variations in the power bought and sold from the microgrid cluster with the microgrids and the upstream network. It shows that by allowing the price change by the microgrid cluster, this agent can buy and sell more energy from the microgrid, in comparison with Figure 14.
Energies 2023, 16, x FOR PEER REVIEW 14 of 17 change by the microgrid cluster, this agent can buy and sell more energy from the microgrid, in comparison with Figure 14. One of the problems faced by the power transmission and distribution network is the supply of peak load. Table 2 shows the peak load values in each microgrid, before and after optimization. These numbers show that the demand response in model 2 has been performed in such a way that the amount of peak load has decreased, and this achievement, in addition to the cross-sectional profit for each microgrid, also prevents more investment in providing peak load.  Figure 18 displays the overall result of the game (the objective function), which shows the profit of each agent. The response of the load at different hours was better according to the price. Moreover, the profit of the consumers increases due to it being allowed to change the load based on the load types in model 2. The profit of the microgrid decreased, which was predictable due to the increase in the profit of the consumer agent. One of the problems faced by the power transmission and distribution network is the supply of peak load. Table 2 shows the peak load values in each microgrid, before and after optimization. These numbers show that the demand response in model 2 has been performed in such a way that the amount of peak load has decreased, and this achievement, in addition to the cross-sectional profit for each microgrid, also prevents more investment in providing peak load.  Figure 18 displays the overall result of the game (the objective function), which shows the profit of each agent. The response of the load at different hours was better according to the price. Moreover, the profit of the consumers increases due to it being allowed to change the load based on the load types in model 2. The profit of the microgrid decreased, which was predictable due to the increase in the profit of the consumer agent.

Conclusions
This paper presented a Stackelberg game between the agents of the network, including several microgrids that are connected by a microgrid cluster and exchange power with the upstream network. The game process was based on two separate layers. One layer modelled the interaction between each microgrid and microgrid cluster, and the equilibrium point between them. In the second layer, the game modelled the interactions between the microgrid agent and the consumer agent. In each of the layers, the follower determines the power required to achieve increased profit based on the price of the leader and announces the power to the leader. Then, the forward operator determines the balance point for buying and selling energy with the upstream grid. The responsiveness of consumers was one of the most important parts of this game, which increased the profit for each consumer and microgrid, and therefore decreased the profit of the microgrid cluster. The proposed method considers different load models in a smart building regarding a suitable demand response model compared to previous methods, which allows changing the load at different hours of the day and night. The flexible load response increased the profit of the consumer agent more with respect to the model that only changes the load by a constant factor. For future work, the load shifting between microgrids can be investigated. This can be performed by interconnecting microgrids for greater stability. Moreover, the increase in electric vehicles, the traffic of these vehicles, and their charging and discharging in different microgrids can be considered.

Conclusions
This paper presented a Stackelberg game between the agents of the network, including several microgrids that are connected by a microgrid cluster and exchange power with the upstream network. The game process was based on two separate layers. One layer modelled the interaction between each microgrid and microgrid cluster, and the equilibrium point between them. In the second layer, the game modelled the interactions between the microgrid agent and the consumer agent. In each of the layers, the follower determines the power required to achieve increased profit based on the price of the leader and announces the power to the leader. Then, the forward operator determines the balance point for buying and selling energy with the upstream grid. The responsiveness of consumers was one of the most important parts of this game, which increased the profit for each consumer and microgrid, and therefore decreased the profit of the microgrid cluster. The proposed method considers different load models in a smart building regarding a suitable demand response model compared to previous methods, which allows changing the load at different hours of the day and night. The flexible load response increased the profit of the consumer agent more with respect to the model that only changes the load by a constant factor. For future work, the load shifting between microgrids can be investigated. This can be performed by interconnecting microgrids for greater stability. Moreover, the increase in electric vehicles, the traffic of these vehicles, and their charging and discharging in different microgrids can be considered.