Coordinated Energy Scheduling of a Distributed Multi-Microgrid System Based on Multi-Agent Decisions

Regarding the different ownerships and autonomy of microgrids (MGs) in the distributed multi-microgrid (MMG) system, this paper establishes a multi-stage energy scheduling model based on a multi-agent system (MAS). The proposed mechanism enables a microgrid agent (MGA), a central energy management agent (CEMA), and a coordination control agent (CCA) to cooperate efficiently during various stages including prescheduling, coordinated optimization, rescheduling and participation willingness analysis. Based on the limited information sharing between agents, energy scheduling models of agents and coordinated diagrams are constructed to demonstrate the different roles of agents and their interactions within the MMG system. Distributed schemes are introduced for MG internal operations considering demand response, while centralized schemes under the control of the CCA are proposed to coordinate MGAs. Participation willingness is defined to analyze the MGA’s satisfaction degree of the matchmaking. A hierarchical optimization algorithm is applied to solve the above nonlinear problem. The upper layer establishes a mixed-integer linear programming (MILP) model to optimize the internal operation problem of each MG, and the lower layer applies the particle swarm optimization (PSO) algorithm for coordination. The simulation with a three-MG system verifies the rationality and effectiveness of the proposed model and method.


Introduction
Rapid technological development of demand response (DR) and distributed generation (DG) have enabled microgrids (MGs) to become more modern and more autonomous. With the presence of distributed renewable energy (DRE), energy storage systems (ESS), controllable distributed generation (CDG), and DR, MGs can be regarded as small-scaled systems that are self-interested in energy scheduling and management inside the MG [1,2]. However, due to the uncertainties of DRE and load demands and its limited energy handling capacity, it is a great challenge for a MG to deal with the energy handling issues totally independently [3]. With the trends of ever-increasing MGs that arise in the distribution network, the concept of a multi-microgrid (MMG) system is on the horizon, where multiple MGs operate in a coordinated manner to cut down the total investment and operation cost and enhance the reliability of the distributed system [4,5].
Energies 2020, 13 Coordinated energy scheduling and management in the MMG system has been an interesting research focus recently [6][7][8]. Its coordination architecture can be classified into two categories, i.e., centralized [9,10] and distributed [11,12]. In the centralized architecture, there is a central master that is in charge of accumulating data from all the individuals, performing optimization, and finally determining the actions of all units inside the MMG. Since energy scheduling strategies are based on a single master, it requires extensive information of the MGs to obtain the global optimum, which is not practical due to the different ownerships of MGs in the MMG system [13]. Considering the environment, a centralized architecture is unable to provide differentiated operating schemes with respect to each MG. Furthermore, a fully distributed coordination involves various local controllers that only dispose of a corresponding single unit. In this way, each controller in each MG is only aware of the local parameters and actions from the components, such as electricity consumers and generators, and has no idea about the neighboring MGs and system-level operation [14,15]. In addition, the generic energy planning and scheduling framework for the MG in [16] is formulated to address the problem of optimal operation with an allowable level of CO 2 . The authors in [17] model all the components in MG as autonomous individuals which optimize their behavior independently and are also able to communicate with each other in a distributed manner without a direct way. As electricity markets and distribution networks become more complex, there is a strong relationship among MGs and a tendency to have a cooperative economy in the MMG system.
As a result, some researchers devised a hybrid form of coordination known as hierarchical coordination. For instance, the authors in [18] design a hierarchical control system for a robust MG operation, where the internal model is in charge of mitigating voltage disturbances while the coordination controller is responsible for the power-sharing. The authors in [19] proposed an energy management system for a MMG organized in a bi-level hierarchical structure, where the upper level is to minimize the operation and carbon trading cost while the lower level is to achieve the self-organizing of each MG. Applications of a multi-agent system (MAS) within a MMG system have been widely proposed in [20,21]. The key advantages of a MAS are flexibility, scalability, and capability of providing the plug-and-play property to MGs. For example, the authors in [20] presented a MAS-based day-ahead energy management framework consisting of CDG agents, wind turbines agents, photovoltaic agents, demand agents, ESS agents, and a MG aggregator agent to minimize energy loss and operation cost. X. Kong et al. [21] constructed a MAS-based control structure to distribute the profits of different DER owners in the marker mechanism. MAS is also applied to the energy market of MMG under the supply-demand mismatch scenario with/without ESS in [22]. V.H Bai et al. [23] proposed a MAS-based community energy management system with the information of surplus and shortage amounts from MGs to better control the generation and adjust load demand. W. Jiang et al. [24] developed a hierarchical multi-agent-based energy management framework to maximize the local consumption of renewable energy. In [25], the authors developed a MAS-based energy market where game theory is applied for the day-ahead market while hierarchical optimization is used for the hour-ahead market in an MMG system.
The above studies proposed a variety of energy coordination mechanisms considering different entities, but they do not consider the distinctive features of limited information sharing between agents because such features raise privacy concerns. Moreover, they do not analyze the willingness of MGs to participate in the matchmaking with neighboring MGs, since MGs have different ownerships and have a self-interest in the internal operation and related costs. This paper addresses a solution to the coordinated energy scheduling problem by implementing a MAS-based environment considering the autonomy of MGs in the view of information sharing and participation initiative. Moreover, energy transmission problems can also occur in power transactions between different MGs. In this paper, we use a coordination control agent (CCA) to evaluate whether the distribution network meets the requirement under a central energy management agent (CEMA) coordination strategies by power flow calculations. In this paper, we introduce a coordinated energy scheduling method including the distributed MGA, CEMA, and CCA. We construct the models of these agents to depict the autonomy and independence of the MGA's internal operation, the data aggregation and matchmaking trade-off supported by the CEMA, and the electricity network operation maintained by the CCA. We propose a multi-stage energy scheduling framework to minimize the internal operation cost of MGs and maximize the net social benefit of the whole MMG system. In this way, each MGA only communicates with the CEMA using limited information to realize the coordination with other MGAs, which is supported by the CEMA and CCA. The optimization problems are formulated and solved by all agents using a linear solver and a modified particle swarm optimization (PSO) algorithm implemented in an iterative manner. Our contributions are as follows: We propose a novel framework for energy scheduling of a distributed MMG system, where the agents, i.e., MGA, CEMA, and CCA, work in a cooperative manner during prescheduling, coordinated optimization, and rescheduling, and participation willingness analysis (PORA) is evaluated.

2.
We adopt a novel coordinated energy scheduling method based on limited information sharing, which can not only preserve the MGA's privacy by content packaging but also facilitate promoting energy scheduling coordination within the MMG system.

3.
We introduce a novel centralized scheme under the control of the CCA for coordinated energy trade among MGs, which not only guarantees the normal operation of the network but also optimally minimizes the cost of line loss.

4.
We define the participation willingness, which is the MGA's satisfaction degree to the matchmaking scheme. It can evaluate the impact of the matchmaking scheme on MGAs.

System Framework
We consider a typical MMG system with I MGs connected to the main power grid by an electricity network. To better deal with energy scheduling issues, there are three kinds of agents providing different energy control (i.e., MGA, CCA, and CEMA). The multi-stage framework of energy scheduling with different agents is proposed in Figure 1, which is a hierarchical control with the agents having different ownerships.
• MGA: Each MGA is only responsible for the local MG, which gathers local information about the MG that is under its control and is neither aware of systematic operations nor neighboring operations. With the autonomy of MGAs, they can realize the decentralized control and independently determine whether to participate in the matchmaking based on their respective limited knowledge. • CEMA: The CEMA serves as an aggregator that can play a role in coordinating various MGs for better energy scheduling systematically. When dealing with the coordinated energy scheduling issues, the CEMA aggregates operation data submitted by all the MGAs after content packaging and determines the dynamic amount of the energy exchange among MGs. • CCA: To evaluate the feasibility and economy of the matchmaking scheme offered by the CEMA, the CCA applies power flow calculations to analyze the electricity network performance when MGs coordinate with each other.
As indicated in Figure 1, we define the operation mode 0 ≤ r ≤ 1to represent the degree of information sharing, where r = 0 means it is the fully distributed coordination implemented by each MGA, r = 1 means it is the centralized coordination implemented by CEMA, and 0 < r< 1 means it is the proposed hierarchical coordination. Our system consists of four stages.

MG Internal Energy Scheduling
We consider an MG that can consume, generate, and store electricity while being connected to the main grid. In this way, there is a natural conflict between consumers and generators. We formulate the interaction between these two entities as a Stackelberg game, where the generators are

MG Internal Energy Scheduling
We consider an MG that can consume, generate, and store electricity while being connected to the main grid. In this way, there is a natural conflict between consumers and generators. We formulate the interaction between these two entities as a Stackelberg game, where the generators are the leaders and the consumers are the followers. The generators (leaders) can predict the electricity load curve in advance and impose a set of time-varying electricity prices to minimize operation costs inside the MG. Later, the consumers (followers) reply with their best energy consumption strategies to minimize their individual costs given the assigned prices.

Electricity Consumer Model
For the consumers, we consider they are self-interested in minimizing their total cost by adjusting their optimal day-ahead electricity consumption given the prices imposed by the MG. The cost of all users inside the MG can be denoted as the sum of C e and C d . The energy payment of a consumer inside the MG depends on the imposed prices and its electricity consumption, which can be represented by: In addition, we introduce the discomfort cost to measure the discomfort experience caused by consumption adjustment, which can be derived as: where β is 0.5 in this paper. Therefore, the optimization problem can be expressed as: where µ * i is the optimal price set imposed by the i-th MG, the first constraint guarantees that the consumption is adjusted between x min ij and x max ij , respectively. The second constraint ensures the integrity of the consumption over T.

MG Operation Model
For all the generators in the i-th MG, the operation cost is: where the operation cost includes four terms, namely, C CDG , C R , C PG , and C MG . Power balance constraint: Energies 2020, 13, 4077 6 of 19 Unit output limit constraint: ESS operation constraints: where SOC i 0 and SOC i T are the initial and final SOC during the whole time slot T, respectively. Tie line exchange power limit constraint: Therefore, from the perspective of the MG operation, the optimization problem can be derived as: Since MGA is an independent individual with high autonomy, after receiving the optimal matchmaking scheme from CEMA, it can decide whether it should participate in the matchmaking based on its internal operation. We define the PW function as: where C i,n MG and C i,(n−1) MG are the operation cost of the n-th iteration and the (n-1)th iteration within the i-th MG, respectively. When PW i is 0, it means that the i-th MG is unsatisfied with the n-th matchmaking scheme. Otherwise, it means that the i-th MG is satisfied with the n-th matchmaking scheme.

MMG Coordinated Energy Scheduling
After receiving the packaged information from all MGs, the CEMA provides the most optimal coordinated energy scheduling scheme under the control of the CCA as indicated in Figure 1. With this scheme, MGs can collaborate with each other by mutually setting up the more attractive prices than the main grid, realizing energy sharing within the MMG.

CEMA Model
Due to privacy concerns, each MG adopts the content packaging method to protect its internal operation and customer data. The received information D i from the i-th MG can be derived as: Energies 2020, 13, 4077 where P i is the aggregated generated power within the i-th MG, the element of which can be formulated in (13). µ i represents the aggregated prices of the i-th MG during the whole time slot T, which can be derived as (14). P i line is the aggregated tie line power between the i-th MG and the main grid during the whole time slot T, which is expressed in (15).
The responsibility of the CEMA is to promote the matchmaking among MGs and optimize the net social benefit with the above limited MGA information. The problem of the CEMA can be formulated as: P3 : max where Q = {P,P line ,P MG } is the energy schedule set by the CEMA. C trade is the social benefit from MG coordination, where µ ikt is the price for the matchmaking between the i-th and k-th MG in the t-th time slot. W L is the cost caused by the line loss due to the matchmaking, which could be calculated by the CCA. N ik is the set of nodes between the i-th and k-th MG in the network. P min iu and P max iu are the minimum and maximum line power between the i-th node and u-th node, respectively.
The final optimal coordinated energy scheduling scheme sent back to each MGA, which can be derived as: whereP i line , is the line power between the i-th MG and main grid during the whole time slot T in the matchmaking.P i MG is the transmitted power between the i-th MG and others in the matchmaking. µ i is the matchmaking price set of the i-th MG and others.

CCA Model
The abovementioned problem is under the control of the CCA, the responsibility of which is to evaluate whether the energy exchange in the matchmaking scheme can be realized with the current network operation constraints. Thus, the above line cost W L can be formulated as: where η Loss is the cost of the unit line loss coefficient. P ik Loss,t is the total power loss caused by the energy exchange between the i-th MG and k-th MG during the t-th time slot. V u 1 t and V u 2 t are the u 1 -th and the u 2 -th node voltage, respectively. g u 1 u 2 is the line conductance between node u 1 and node u 2 . cosθ u 1 u 2 t is the power factor between node u 1 and node u 2 .
In this paper, a MG is defined as a generalized PV node and thus the power flow constraint can be written as follows: Power flow constraints: where P u 1 u 2 t and Q u 1 u 2 t are the active and reactive power of node u 1 and node u 2 during the t-th time slot, respectively. b u 1 u 2 is the line susceptance between node u 1 and node u 2 . Node voltage constraints: where V u 1 ,min and V u 1 ,max are the minimum and maximum voltage of node u 1 . Node power constraints: where P u 1 ,t and Q u 1 ,t are the active and reactive power of node u 1 , respectively. P u 1 ,min and P u 1 ,max are the minimum and maximum active power of node u 1 , respectively. Q u 1 ,min and Q u 1 ,max are the minimum and maximum active power of node u 1 , respectively. Line transmission power constraints: where P u 1 u 2 ,min and P u 1 u 2 ,max are the minimum and maximum active power of the line between node u 1 and node u 2 , respectively.

Solution to MG Internal Energy Scheduling Game
In general, the Stackelberg equilibrium (SE) can be obtained by finding the Nash equilibrium (NE) of the subgame (P1), where the consumers compete in a non-cooperative way. Thus, the NE is defined as a situation where no consumer can reduce the cost by changing their strategy. We first apply the backward induction method to solve P1 with a fixed electricity price to find the best DR of consumers. Theorem 1. In the proposed Stackelberg game within the MG, when the MGA imposes its price µ i , the optimal load demand adjusted by the j-th consumer is Proof of Theorem 1. The first and the second derivative of C i u, j with respect to x ijt is Obviously, the second derivative is greater than zero, so the C i u,j is a quasi-convex function. Thus, based on [26], there is an NE in the non-cooperative game of consumers given the prices. Let γ ijt , δ ijt and ϕ ij be the Lagrangian multipliers, where γ ijt , δ ijt are non-negative. The Lagrangian of P1 is expressed in (25). According to the KKT condition, the optimal X * ij satisfy the constraints in (26). The value of x ijt expressed in (23) is the unique Nash (Stackelberg) equilibrium. Substituting the optimal DR (23) and the constraints (26) into P2, we can rewrite P2 in (27).
We apply two linearization methods to model the internal energy scheduling optimization (IESO) problem in (27) as the mixed-integer linear programming (MILP) model, which can be solved by CPLEX [27].

1.
Linearize the product of variables Equation (1) shows that the consumer's cost is constructed using the product of two continuous variables. Therefore, the McCormick convex envelope method in [28] is applied here to approximately linearize the problem. The equation can be reshaped as: where µ min it and µ max it are the minimum and maximum price of the i-th MG during the t-th time slot, respectively.

Linearize a quadratic function
There is the piecewise linearization for the nonlinear problems caused by the quadratic function [29]. The quadratic function can be accurately approximated by a set of piecewise segments.
where f is the quadratic term in the cost of CDG C i CDG . ω is the actual value of P i CDG,t . ω ξ is the value of P i CDG,t corresponding to ξ-th point and λ ξ is the coefficient of the ξ-th point.

Coordinated Energy Scheduling Optimization (CESO) Algorithm
For the net social benefit maximization within the MMG system (i.e., P3), it is possible to obtain the optimally coordinated energy scheduling scheme using PSO. However, although PSO has the advantage of easy implementation, it can easily fall into local optima, which is caused by the premature convergence and decline in population diversity. We applied a particle swarm optimization algorithm with wavelet mutation (PSOWM) based on [30], which is introduced to mutate the particles to enhance the searching performance.
Both velocity and position are two properties of the particle. The particle r's position is defined as the energy scheduling matrix Q r in CEMA, and the velocity is defined as matrix V r . Using the historical experience of each particle, they can be updated as: where w is inertia weight factor; c 1 , c 2 are the acceleration constants; rd 1 , rd 2 are the random numbers within the range of [0, 1]; Q b,k is the historical optimal position of the r-th particle; Q gb,k is the historical optimal position in the population; and g is the function to limit the velocity within the range of [v min , v max ], which is expressed as: The particle element might mutate, which is governed by the mutation probability ρ m ∈ [0, 1]. For the selected particle Q r = [q r 1 ,q r 2 , . . . , q r D ], the resulting elementq r d is: where d ∈ 1,2, . . . ,D; D denotes the dimension of the particle; q max d and q min d are boundaries of the element; and σ is the Morlet wavelet function, which is derived as: where θ is randomly generated from [−2.5a, 2.5a]; a is the dilation parameter, which is formulated as: where g is the upper limit of a; K is the total iteration, and; λ is the shape parameter of the monotonic increasing function. Update Q and V of the another as (29)-(30) within the constraints (15), (17) With the above MILP model and PSOWM algorithm, we form a coordination diagram of agents, as shown in Figure 2, and propose a CESO algorithm in Algorithm 1 based on the framework. First, each MGA performs an internal operation optimization at the beginning of the day without communication with the others and sends the data D to the CEMA after content packaging (Steps 1-4). After that, the CEMA performs coordinated optimization under the control of the CCA based on PSOWM and returns the matchmaking schemeD (Steps 6-13). With the returned data, the MG performs internal operation optimization again (Steps 14-16) and analyses the PW (Step 17). If one of the MGAs is unsatisfied with the matchmaking scheme, this MG discards the coordination and returns the result to the CEMA to revise the matchmaking scheme (Steps 18-20). Otherwise, the CEMA accumulates the data D and performs optimization for the next iteration (Step 5). The process continues until convergence is reached.
PSOWM and returns the matchmaking scheme D (Steps 6-13). With the returned data, the MG performs internal operation optimization again (Steps 14-16) and analyses the PW (Step 17). If one of the MGAs is unsatisfied with the matchmaking scheme, this MG discards the coordination and returns the result to the CEMA to revise the matchmaking scheme (Steps [18][19][20]. Otherwise, the CEMA accumulates the data D and performs optimization for the next iteration (Step 5). The process continues until convergence is reached.

Simulation Setting
To verify the rationality and effectiveness of the proposed method, we consider the system depicted in Figure 3, which is improved from the IEEE-33 system, and Table 1 summarizes the major components of each MG. The selling price announced by the main grid is 0.84 ¥/kWh from 0:00 to 7:00 while it is 2.1 ¥/kWh from 7:00 to 24:00 according to the Tokyo Electric Power Company [31]. The CDG cost coefficients are a = 0.013 ¥/kW, b = 0.62 ¥/kW, and c = 13.4 ¥/kW. The switch status of each operation mode is shown in Table 2. We applied the robust optimization method to obtain one of the extreme boundary scenarios for simulation analysis.
The computational model is programmed in C++ by calling the commercial MILP solver ILOG CPLEX 12.5 and PSO function from MathWorks. All experiments are implemented on a personal computer, which has quad Intel Core i7 processors with CPU at 3.40 GHz and a RAM space of 8 GB.

Simulation Setting
To verify the rationality and effectiveness of the proposed method, we consider the system depicted in Figure 3, which is improved from the IEEE-33 system, and Table 1 summarizes the major components of each MG. The selling price announced by the main grid is 0.84 ¥/kWh from 0:00 to 7:00 while it is 2.1 ¥/kWh from 7:00 to 24:00 according to the Tokyo Electric Power Company [31]. The CDG cost coefficients are a = 0.013 ¥/kW, b = 0.62 ¥/kW, and c = 13.4 ¥/kW. The switch status of each operation mode is shown in Table 2. We applied the robust optimization method to obtain one of the extreme boundary scenarios for simulation analysis.     The computational model is programmed in C++ by calling the commercial MILP solver ILOG CPLEX 12.5 and PSO function from MathWorks. All experiments are implemented on a personal computer, which has quad Intel Core i7 processors with CPU at 3.40 GHz and a RAM space of 8 GB. Table 3 shows the internal costs of the MG and execution time with/without DR. It can be seen that energy scheduling considers the consumer's DR can decrease the total cost of the consumers by comparing the two main columns, especially in the MG with the high ratio of α. When α = 3.283 (i.e., MG2), the cost for consumers achieves the maximum relative reduction compared with the others. From the view of operational costs, DR can decrease the total operational cost (except MG2). In MG1 and MG3, both the operational cost and consumers' costs are decreased, which means energy scheduling that considers the interaction attracts the attention of both the consumers and MG by cutting down the costs. In MG2, although the operational cost increases slightly, the consumers' cost has a relatively higher reduction. This finding occurs because the MGA makes a better trade-off between the consumers' cost and operational cost when there is a game inside the MG. In addition, the MGA determines the internal energy scheduling strategies considering DR with a bit longer execution time, comparing with Case 1 without DR. The mean execution time of the two Cases is at the second level, wherein the mean execution time of Case 1 is 1.356 while for Case 2 it is 2.725.

Performance of the MGA
The total amount of power exchange and the utilization of renewable energy are given in Table 4. Three cases have been compared here: (1) Case 1: the distributed optimal scheduling of the MGA does not consider coordination and DR; (2) Case 2: the distributed optimal scheduling of the MGA considering DR; (3) Case 3: the optimal scheduling considering coordination and DR. It is obvious that the amount of purchasing power from the main grid has been reduced in the MG3. In particular, Case 3 reduces the amount of purchasing power by making full use of coordination among different MGAs and DR. In addition, the utilization of renewable energy has been improved in MG1 and MG3. In MG2, the utilization is slightly reduced in Case 2 but the utilization is improved in Case 3. To elaborate on the impact of coordination among MGAs on the renewable energy curtailment and power exchange with the main grid, two cases are compared: (1) Case 1: with a coordinated energy scheduling strategy provided by the CEMA; (2) Case 2: without coordinated energy scheduling. Figure 4a depicts the changes in renewable energy curtailment with/without energy coordination. It can be seen that the CEMA could promote renewable energy sharing among MGs by providing a platform for matchmaking. The result in Figure 4a shows that the total curtailment of renewable energy in MG1 is reduced by 268.27 kWh, and that of MG2 is reduced by 300.76 kWh, comparing the two situations with or without energy coordination. The lack of energy coordination by the CEMA often results in a massive curtailment of renewable resources, especially in the MG with a high ratio of internal generation and maximum load demand. From the time domain, it can be seen that the curtailment of PDER in MG2 during 9:00-17:00 is 1408 kWh when there is lack of the CEMA, while the curtailment is 1143 kWh in MG2 with the CEMA, which is reduced by 18.82%, implying that energy coordination boosts efficient utilization of renewable energy.   Figure 4b illustrates the power exchange with the main grid with/without energy coordination. From the figure, the reduction of the total amount of sold energy is 777.72 kWh in MG1, and that of MG2 is 889.06 kWh. In addition, the maximum reductions occur in the period of 1:00-7:00 and 20:00-24:00. The reduction of the total amount of buying energy is 262.22 kWh in MG3. Energy coordination helps the MG lessen the energy reliance on the main grid. From Figure 4, the reason can be explained  Figure 4b illustrates the power exchange with the main grid with/without energy coordination. From the figure, the reduction of the total amount of sold energy is 777.72 kWh in MG1, and that of MG2 is 889.06 kWh. In addition, the maximum reductions occur in the period of 1:00-7:00 and 20:00-24:00. The reduction of the total amount of buying energy is 262.22 kWh in MG3. Energy coordination helps the MG lessen the energy reliance on the main grid. From Figure 4, the reason can be explained as follows. When the load demand increases, the MG is more willing to participate in the matchmaking with other MGs due to the more attractive pricing schemes in the CEMA platform. Correspondingly, the purchased energy from the main grid decreases but the purchased energy from MGs increases. Table 5 shows the performance of the convex equivalent optimization method (CEOM) in [32,33], PSO, and PSOWM. CCEMA calculated by PSOWM is the largest, of which the value is 4758.2. The second-largest CCEMA is calculated by PSO and its value is 4725.77, which is 99.32% of the maximum CCEMA. The CCEMA of CEOM is the smallest one, which accounts for 98.22% of the maximum CCEMA. Comparing the CEOM with PSO and PSOWM, it can be found that after CEOM transforms the nonlinear problem into a linearized and convex optimization problem, the calculation time is greatly reduced, and the reduction ratio is 90.36% comparing with PSO and 90.38% comparing with PSOWM, respectively. Obviously, the calculation speed is improved but the error cannot be avoided in the transform, which could ultimately affect the optimization of the objective. PSO and PSOWM are both heuristic algorithms and will not change the original problem. The nonlinear model searches for the optimal solution on the basis of the original nonlinear model, so the calculation time of the two is longer than CEOM, but the objective value is not affected by the linearization error. But at the same time, it is difficult to ensure that the global optimal solution can be obtained by applying heuristic algorithms, so it is necessary to improve its search performance by using multiple clusters, mutations, stretching, and hybridization. PSOWM introduces mutation operation on the basis of conventional PSO. Compared with PSO, the calculation time is similar, but it can improve the final objective value to a certain extent. 1 Calculation accuracy is obtained by the ratio of optimal value to the maximum of the optimal value among the three algorithms.
Tables 6 and 7 depict the costs of different agents and the respective execution time during PORA. Within one of the intermediate iterations, the final cost of each MGA decreases compared with the initial cost during prescheduling, implying that energy coordination helps the MGA cut down on the internal cost, and consequently, MGAs are willing to accept the matchmaking scheme formulated by the CEMA. From iteration 0 to iteration 2, the social benefit calculated by the CEMA increases while the internal cost of each MG decreases. It can be seen that MGs work in a cooperative way that forms a tight association from the view of the network. When iteration = 3, the internal operation cost of MGA2 increases compared with the one in iteration 2, with which MGA2 is unsatisfied. Hence, MGA2 is unwilling to participate in the next matchmaking and regards its matchmaking with MGA1 and MGA3 as the final output, while MGA1 and MGA3 continue to derive a better matchmaking scheme without MGA2 (i.e., iteration 4 and 5). The execution time of internal energy scheduling within different MGs is at the second level, which could meet the requirement of real-time energy scheduling within the MG. Meanwhile, the execution time of coordinated energy scheduling optimization implemented by the CEMA is at the minute level, which coincides with the actual coordinated control.
I is the abbreviate of iteration while S is the abbreviate of Stage. 2 P is the abbreviate of "Prescheduling"; O is the abbreviate of "Coordinated Optimization"; R is the abbreviate of "Rescheduling"; A is the abbreviate of "Participation Analysis", where Y means MGA agree on the matchmaking while N means MGA is unsatisfied with the matchmaking.  Table 8 demonstrates the power flow distribution for all cases, where r = 0 means that there is no information sharing within the MMG system, whereas r = 1 means that there is an open system of fully shared information, and 0 < r< 1 means that there is a limited-information-sharing-based system. When r = 0, since there is no CEMA and CCA, the distributed MGAs exchange power with the main grid. This is why the utilization of the line between the MG and the main grid reaches the maximum in MGA2. In addition, the CCAs give the line loss feedback to the CEMA, which makes the line loss lower in comparison with the mode (r = 0). With the increase of information sharing, the CCA provides an optimal power flow distribution scheme to cut down the line loss. The operation costs of MGAs and the social benefit with different degrees of information sharing within the MMG system are shown in Figure 5. It can be found that with the increase of the information sharing degree, the social benefit in the MMG system increases and achieves the maximum value when r = 1, which means the CEMA is aware of the information of all the MGAs without considering the PW of MGs. At the same time, some MGAs' profits are damaged compared with other situations. Furthermore, with the increase of the data sharing degree, the profit of each MGA first increases and then decreases. It shows that there is a trade-off between distributed MGAs and the whole MMG system. with the initial cost during prescheduling, implying that energy coordination helps the MGA cut down on the internal cost, and consequently, MGAs are willing to accept the matchmaking scheme formulated by the CEMA. From iteration 0 to iteration 2, the social benefit calculated by the CEMA increases while the internal cost of each MG decreases. It can be seen that MGs work in a cooperative way that forms a tight association from the view of the network. When iteration = 3, the internal operation cost of MGA2 increases compared with the one in iteration 2, with which MGA2 is unsatisfied. Hence, MGA2 is unwilling to participate in the next matchmaking and regards its matchmaking with MGA1 and MGA3 as the final output, while MGA1 and MGA3 continue to derive a better matchmaking scheme without MGA2 (i.e., iteration 4 and 5). The execution time of internal energy scheduling within different MGs is at the second level, which could meet the requirement of real-time energy scheduling within the MG. Meanwhile, the execution time of coordinated energy scheduling optimization implemented by the CEMA is at the minute level, which coincides with the actual coordinated control. Table 8 demonstrates the power flow distribution for all cases, where r = 0 means that there is no information sharing within the MMG system, whereas r = 1 means that there is an open system of fully shared information, and 0 < r< 1 means that there is a limited-information-sharing-based system. When r = 0, since there is no CEMA and CCA, the distributed MGAs exchange power with the main grid. This is why the utilization of the line between the MG and the main grid reaches the maximum in MGA2. In addition, the CCAs give the line loss feedback to the CEMA, which makes the line loss lower in comparison with the mode (r = 0). With the increase of information sharing, the CCA provides an optimal power flow distribution scheme to cut down the line loss.

Different Degrees of Information Sharing
The operation costs of MGAs and the social benefit with different degrees of information sharing within the MMG system are shown in Figure 5. It can be found that with the increase of the information sharing degree, the social benefit in the MMG system increases and achieves the maximum value when r = 1, which means the CEMA is aware of the information of all the MGAs without considering the PW of MGs. At the same time, some MGAs' profits are damaged compared with other situations. Furthermore, with the increase of the data sharing degree, the profit of each MGA first increases and then decreases. It shows that there is a trade-off between distributed MGAs and the whole MMG system.

Conclusions
Considering the different ownerships and their autonomy of MGs, we develop a multi-stage framework for the coordinated energy scheduling problem within the MMG system, where the MGA is responsible for the internal operation, and the CEMA and the CCA are responsible for the coordination. The interaction model between agents is constructed based on the limited information sharing. In contrast to other methods, the matchmaking PW of MGs is considered in the coordination model between the CEMA and the MGA. In addition, the CCA is introduced to keep the MMG system operating within the physical network constraints and assists the CEMA in achieving global optimization from the view of the whole MMG system. The hierarchical schemes are proposed for the coordination of the MMG system with multiple stages, namely, PORA, where the upper layer is a MILP model addressing the distributed internal operation optimization and the lower layer is the energy coordination model solved by PSO. The simulation results verify the improvement effect that decreases the MG's dependence on the main grid and increases the utilization of renewable energy. The results also show that the cooperation of MGAs guided by the proposed method can yield a better trade-off between the whole MMG system and each MGA while respecting the autonomy of each MGA.
The above investigations on the proposed MAS-based coordinated energy scheduling for the MMG system reveal that the further explorations in the following directions would be worthwhile:

•
Analyze the information sharing willingness and characteristics of different types of MGs, and quantitatively characterize the degree of information sharing of microgrids; • Analyze the impact of the different degrees of information sharing between different MGs on the coordinated energy scheduling of the MMG system, and consequently propose corresponding strategies.