A Review of Microgrid Energy Management Strategies from the Energy Trilemma Perspective

Abstract

The energy sector is undergoing a paradigm shift among all the stages, from generation to the consumer end. The affordable, flexible, secure supply–demand balance due to an increase in renewable energy sources (RESs) penetration, technological advancements in monitoring and control, and the active nature of distribution system components have led to the development of microgrid (MG) energy systems. The intermittency and uncertainty of RES, as well as the controllable nature of MG components such as different types of energy generation sources, energy storage systems, electric vehicles, heating, and cooling systems are required to deploy efficient energy management systems (EMSs). Multi-agent systems (MASs) and model predictive control (MPC) approaches have been widely used in recent studies and have characteristics that address most of the EMS challenges. The advantages of these methods are due to the independent characteristics and nature of MAS, the predictive nature of MPC, and their ability to provide affordable, flexible, and secure MG operation. Therefore, for the first time, this state-of-the-art review presents a classification of the MG control and optimization methods, their objectives, and help in understanding the MG operational and EMS challenges from the perspective of the energy trilemma (flexibility, affordability, and security). The control and optimization architectures achievable with MAS and MPC methods predominantly identified and discussed. Furthermore, future research recommendations in MG-EMS in terms of energy trilemma associated with MAS, MPC methods, stability, resiliency, scalability improvements, and algorithm developments are presented to benefit the research community.

1. Introduction

The impact of energy generation through conventional fossil fuels on satisfying the increase in electricity load demand, emissions, socio-economical, environmental, and climate changes is raising alarm to look for alternative low carbon, sustainable, renewable energy sources (RES), storage systems, and networks [1]. The Net Zero goals of countries around the world, especially, the UK (target to decarbonize all sectors by 2050, and 100% zero-carbon generation by 2035) shows significant green movement towards carbon neutrality by encouraging production and utilization of power from RES [2]. The rise of RES penetration and record-breaking generation of zero-carbon power in the last few years in the UK (43% and 41% of the total in years 2020–2021) [3], and around the globe (~30% in 2020–2021) [4] shows the global trend in the transition of the energy sector. Furthermore, the awareness and importance of it can be seen around the globe through the actions, funding by respective governments, and global summits such as UN COP26 [5].

1.1. About Microgrids

The paradigm shift in the modern power system that has more renewables, distributed energy resources (DERs), flexible loads, and smart users, the smart grid (SG) concept has been gradually shifting towards the combination of microgrids (MGs) having controllable nature with several players interconnected and acting simultaneously towards achieving their objectives. MG is a combination of several system components such as RES, energy storage systems (ESS), heating and cooling systems, and local loads including active buildings (ABs) that can act as consumers/prosumers, etc. MGs can operate in both grid-connected and/or islanded modes in a controlled and coordinated way. The benefits of MGs are twofold. At the local level, the MGs enable better control and intra/interoperability of the different components in it. At the power system level, the microgrids improve reliability and provide techno-socioeconomic benefits to both end-users and energy system operators [6]. With different characteristics, control abilities, and modes of operation, MGs control and energy management (EM) is an important task. In the present work, the MGs-EMS objectives, control, and optimization approaches are presented from the energy trilemma perspective. Depending on the application, the nature of MGs varies among type (AC, DC, and hybrid), mode of operation (grid or islanded), and control architecture (centralized, decentralized, distributed, and hierarchical). An overview diagram of MGs type, mode, control, and optimization methods shown in Figure 1. Technically, MGs are classified as AC, DC, and hybrid AC-DC MGs based on the economic considerations and elements presented in them.

Although AC MGs are widely present in the energy infrastructure [7,8], low voltage (LV) DC MGs are becoming popular. An interesting recent review work [9,10] presented a comprehensive feasibility study of the massive deployment of RES in LV DC MGs and their impact on technological, and social developments through key performance indicators such as health and environment, affordable electricity rates, quality of life, community participation, etc. The key recommendations provided are related to efficient operation in both islanded and grid-connected modes, bus configurations, and voltage standardization of LV DC MGs. Furthermore, in [10], hierarchical control architecture that used different approaches in primary, secondary, and tertiary control levels discussed for LV DC MGs.

1.2. Concepts of MG Energy Management and Control

Formation of MGs is becoming more common because of their standalone/grid-connected operation, easy integrating capability to new RES, ESS, energy-positive buildings, and ability to offer services to neighbor energy systems. Therefore, EM in MGs is an important task to be performed for efficient operation as the variability and uncertainty of renewables, storage characteristics, and loads further complicate it. This can be achieved through the different control and optimization methods that provide optimal control and cost-effective utilization of MG assets. Usually, review works mostly focused on the control and optimization methods and presented from the operational point of view of MG-EMS. There is a lack of understanding of control and optimization methodologies from the perspective of the energy trilemma (flexibility, affordability, and security). Usually, energy trilemma (as sustainability, affordability, and security) perspective studies are done for the whole energy systems, network level integration of different energy systems, their planning and operation [11], and cost-efficient expansion and planning [12,13] towards indexing economic energy goals and compare energy system performance at the country level [14]. However, addressing the energy trilemma perspective in the MGs-EMS has never been presented in the literature. The flexible, affordable, and secure operation of microgrids results from an efficient EMS that is possible through advances in control and optimization approaches.

An efficient EMS reduces the system’s operational costs and avoids supply-demand imbalances. As a result, several control architectures have been developed for MG energy management. MGs-EMS with centralized, decentralized, distributed, and hierarchical control architectures that use several optimization algorithms are implemented in the literature [15,16]. In centralized architectures, there will be a single control unit that receives (sends) commands from (to) and manages the MGs components. Due to this nature, the obtained centralized control solutions are mostly feasible and optimal. However, gathering all the information and process in a single unit increases the number of control and optimization variables. This leads to an increase in complexity and requires efficient, fast solvers. Furthermore, the control system is vulnerable to single-point failures and also raises several privacy and security concerns [17]. Decentralized control architectures are introduced to avoid the shortcomings of centralized control methods by collecting the information locally and giving the control ability to each subsystem based on the global objective [18]. The local control action is taken based on the subsystem operational constraints only, and the local actions are aggregated to form the optimal solution for the whole system. Due to this, the computational burden is less on the central control unit and minimizes computational complexity. Despite the advantages, decentralized control systems also have some drawbacks such as more local units, which require more sensory equipment and infrastructure. Moreover, the global system constraints are not satisfied always, and the solution obtained may not be feasible or optimal [16]. Among the stated categories, due to the increase in the importance of privacy, flexible operation, and to improve the utilization of local generated renewable power, decentralized and distributed architectures gained prominence over centralized ones during the last few years. Along with the above methods, there have been numerous studies that use the combination of these methods and form hierarchical or non-hierarchical methods.

A computerized system consisting of numerous interactive, intelligent agents is known as a MAS. MAS methods are applied in areas such as monitoring, diagnostics, markets, protection, control and automation, energy management of SG and MGs, and their scheduling due to the characteristics and design possibility to make agents intelligent [19]. Along with the agent-based approaches, methods with predictive and feedback nature such as MPC-based methods have gained a lot of attention in recent years for MGs-EMS as they provide distributed control and avoid single-point failures in the system. This is evident in the review works and statistics mentioned in the later sections of this paper. The MAS concept is widely used for the energy systems such as MGs to develop decentralized and distributed control architectures. Recent studies that use MAS for EMS with various sources and loads are identified for islanded and grid-connected MGs [20,21], multi MGs [22], MG clusters [23], LV MG [24], DC MG [25], and hybrid AC-DC MG [26]. MAS-based consensus control approaches to balance power generation and demand in MGs are given in [27]. A comprehensive overview of technologies, techniques, and their solutions for managing RES integration, variability, and load fluctuations, while matching energy demands in smart cities MGs are presented in [28]. In [29,30], reviews of several MG structures and their EMS control methods discussed. Along with EMS, stability studies in MG and their control techniques are reviewed in [31]. The EM control and optimization of ABs that act as MGs and participate in providing services to energy networks are given in the review paper [32]. The review of hybrid renewable and heat recovery energy systems presented in [33]. The impact of distributed ESS in the MGs is reviewed in [34]. The above papers provided reviews of control and EM of MGs that gave importance to one or more of its components and focus on the specific requirements such as stability, flexibility, and reliability of the system. To achieve MG optimal operations and energy management, a wide variety of optimization algorithms were employed to solve control objectives using the MAS and MPC-based methods [6].

Table 1. Review papers for MAS and MPC-based energy management in MGs.

Review Work on	Reference	Main Contributions
MAS	Thirunavukkarasu et al., 2022 [6]	A review article comprehensively presented several hybrid, MAS, metaheuristic, and other non-conventional optimization methods that address the EMS problem in MG. The objectives of EM discussed are demand-supply forecast, unit commitment, demand management, and economic dispatch. Also, optimization of the design of MG parameters presented.
	Al-Saadi et al., 2021 [34]	Centralized, decentralized, multi-agent, and intelligent-based control strategies for distributed storage systems in MGs along with control complications and proposed solutions are reviewed in this work. Focus is given to multi-agent and RL-based control strategies.
	Bhargavi et al., 2021 [27]	Distributed consensus control approaches and their review for MGs power management using MAS are presented. The optimization approaches such as PSO, GA, FUZZY, and MPC are discussed in tertiary control for EMS of DC MGs with objectives of minimized operation costs and balanced energy storage.
	Azeroual et al., 2020 [35]	Discussed EM in MG simulation tools, especially the multi-agent platform in JADE linking with Simulink using a co-simulation platform called MACSimJX.
	Rwegasira et al., 2019 [36]	Focus on control algorithms and simulation tools for DC MG load shedding. Mainly, analyzed and stressed the importance of distributed MAS approaches in load shedding and other control operations.
	Khan et al., 2019 [19]	Efficient control and EM of a distributed grid with MAS techniques along with storage and renewable energy sources are discussed along with the protection point of view. Different control architectures and the distributed optimization techniques used to optimize operations, and maximize reliability also reviewed. MAS characteristics and advantages are presented.
MPC	Kamal and Chowdhury, 2022 [37]	From regulating voltage and managing frequency to power flow management and economic optimization, the MPC has emerged as a promising alternative to existing approaches. MPC has the predictive model that is the most precise.
	Konneh et al., 2022 [38]	The superiority of MPC to numerous strategies used to model uncertainties is highlighted in this review work for both grid-connected and islanded systems. It showed the characteristics, strengths, and weaknesses of many modeling methods of MPCs, and some of their modifications for dealing with uncertainty in MGs.
	Babayomi et al., 2022 [39]	MPC enables multivariable control of power electronic systems while addressing physical limits without requiring a cascaded structure. These characteristics result in fast control dynamic response and good performance for non-linear systems. MPC is more flexible and has multivariable and intuitive characteristics for the smart grid and microgrid systems.
	Hu et al., 2021 [40]	This work presented a complete overview of MPC in individual and interconnected MGs, including control strategies at the converter and grid levels applied to three layers of the hierarchical control architecture. This assessment demonstrated that MPC is emerging as a viable alternative to traditional approaches in voltage regulation, frequency control, power flow management, EM, and economic operation optimization.
	Tarragona et al., 2021 [41]	This study summarized the most recent enhancements to solve computational difficulties, as well as an examination of the objective functions used in each study, which were mostly focused on minimizing energy costs, peak power, and CO2 emissions. MPC is the most promising technology to reduce the running cost of the MG.
	Villalón et al., 2020 [42]	This review revealed that predictive control approaches are used on microgrids for the three control levels and with model modifications to account for uncertainties in order to enhance performance and dynamic response. For microgrid applications requiring various degrees of control, predictive control appears to be a very promising control strategy.
	Gholamzadehmir et al., 2020 [43]	The primary goal of this study was to review sophisticated control strategies and their effects on technical systems and building structures in terms of energy and cost savings. Predictive, responsive, and adaptive versus weather, users, grids, and thermal mass should be the goals of these tactics. Even though MPC is the most often used in construction, it is not ideal for systems with uncertainties and unexpected data. In order to address these problems, adaptive predictive control systems are being developed.
	Fontenot and Dong, 2019 [44]	Different kinds of control systems and optimization techniques such as rule-based control, optimal control, agent-based modeling, and MPC were compared, and concluded as MPC is the most effective for the MG systems.
	Sultana et al., 2017 [45]	MPC is gaining popularity due to its adaptability, capacity to be applied in any application regardless of field, and the availability of fast computers. The work described here will assist researchers in further exploring the versatility of this controller for design, analysis, and implementation in renewable energy systems.

summarizes the review papers published in the last few years that mainly concentrated on MAS and MPC-based EM control and optimization approaches for MGs.

The present review work related to MG-EMS, their control, and optimization is focused on energy trilemma as flexibility, security, and affordability [11] are also the major requirements of MG structure. As MGs comprise more of active, controllable, and ICT components, the study of energy management from the perspective of energy trilemma is meaningful and achieving it at the level of MGs can be propagated to the whole energy system. Hence, a study that gives a perspective of energy trilemma in MG-EMS and their insights that are beneficial to advancements in control and optimization methodologies are needed. As of the authors’ knowledge, this review is the first of its kind in presenting the MG energy management concept along with the categorization of objectives, control, and optimization methods from the energy trilemma perspective.

1.3. Objectives of EMS and Energy Trilemma

The objectives associated with MG-EMS categorized in terms of energy trilemma, i.e., the cost-efficient (affordable), flexible (sustainable), and secure system operation. Minimizing energy bills and operating costs, economic dispatch problems (EDP), and maximizing individual/cumulative benefits are some of the objectives associated with affordability. Utilization of DERs, peak load reduction, day-ahead appliance scheduling, internal energy trading, maximizing local RE utilization, energy availability, and economic emission cost reduction is useful in making the MG system more flexible and sustainable. Supply-demand balancing, minimizing load shedding, mitigating load curtailment due to attacks/faults, MG OPF/DOPF, frequency, and voltage regulation make the MG system secure. Overall, MG objectives achieved by controlling and optimizing each component’s objectives in a coordinated or cooperative way. The objective categorization in EMS of MG into flexibility, security, and affordability point of view helps in choosing the method for EM and highlights the requirements for futuristic developments to achieve specific or whole system goals. This review work provides more insights and identifies potential future research directions that should be accounted for EMS of MGs. The definitions of energy trilemma related to MG-EMS can be made as follows:

(i)

Flexibility in EMS: MGs to meet the demand using available energy resources with the goal of minimal emissions, flexible in operation considering local as well as upstream constraints from operators. In terms of control and optimization, MG changes such as adding energy sources or storage systems or any active/passive loads should accommodate by the EMS and provide flexible, scalable operation.

(ii)

Security in EMS: This category deal with various aspects/characteristics of MG components that have security-related issues, i.e., how they coordinate and make decisions without system failure. For example, switching between grid-connected and islanded modes of operation, safe and secure information exchange between different components.

(iii)

Affordability in EMS: This deal with the optimal and economical operation of all the MG components. The objectives that are related to cost come under this category. Cost-efficient and affordable operation of MGs and their components are the most addressed problems, and they include UC, EDP, cost minimization, and individual or system-level benefits maximization.

Figure 2 shows the simplest classification of the objectives under each element of trilemma based on the definition discussed above.

Factors Affecting the Energy Trilemma

MG-EMS enables flexibility in terms of the local energy generation and storage facilities, and effective management of MG components. The objectives such as day-ahead or real-time scheduling, plug-and-play capabilities of MG components, economic emission reduction, demand side management (DSM) including demand response (DR), load shedding, and self-sufficient energy nature to reduce peaks with storage systems affect the flexibility element of the trilemma. The secure operation of MG infrastructure depends on the supply-demand balance either in the islanded or grid-connected mode of operation, stability, frequency, voltage regulation, and communication among different entities. The security of MG-EMS is affected mainly by the uncertainty and variability of RES, ESS, and buildings. Furthermore, information exchange and/or delays among MG components play a role in MG system security. For example, in the MAS architecture, how an EV agent securely sends and receives information from MG energy sources and has the necessary charge for the next usage. Finally, the most addressed problems of MG-EMS were EDP, cost minimization, and benefits maximization for individual components, which are categorized as the affordability. The summary of MG components with their characteristics that affect the element of energy trilemma presented in

Table 2. Characteristic categorization of MG components.

Component	Characteristics	Affected Item of Energy Trilemma
RES (PV, wind, and other)	Uncertainty, variability	Flexibility, security
ESS	Short-team storage	Flexibility, security
EVs	Uncertainty	Security, flexibility
Loads	Controllability, uncertainty	Flexibility, affordability
Buildings (Residential, industrial, commercial)	Comfort-level, controllability, uncertainty	Flexibility, security, affordability
Fuel cells, electrolyzers, hydrogen tank	Mid and long-term storage	Affordability, flexibility, security

. Therefore, it is essential to address the control and optimization methods developed for EMS in MGs that account for energy trilemma.

The control and optimization methods that are the focus of this paper are MAS and MPC-based approaches due to their popularity among other methods for MG-EMS. As methodological advantages and capabilities of these control methods for MG-EMS, understanding them from the energy trilemma perspective will be beneficial to identify the research gaps in recent MG changes and their advancements with rapid changes in technology, digitalization, and the importance to privacy and security. The detailed energy trilemma view of MAS and MPC methods is presented in later sections. The main contributions of this review work are summarized below:

• Presenting the energy trilemma perspective of objectives and their control methods in MG-EMS.
• Understanding of trilemma perspective of MGs can be beneficial to the whole energy system.
• The significance of MAS and MPC-based methods and their challenges for achieving efficient EMS in MGs are discussed using the existing research.
• Future recommendations that include all the trilemma elements in the control and optimization methods for EMS are provided for the research community.

The organization of the paper is given below: Section 2 is the detailed methodology for selecting the research articles from the available literature. Section 3 specifically discussed MAS and MPC control, optimization methods involved in the energy management of the MGs, and the perspective from energy trilemma. Section 4 provides our recommendations and challenges for future energy systems/MGs from the understanding of existing research in terms of MAS and MPC-based methods. Section 5 presents the conclusions drawn from this review.

2. Methodology of the Review

Microgrids are a popular research topic, and significant research development has been published addressing various aspects such as generation, storage systems, and their management (RES, DERs, batteries) to energy trading using peer-to-peer methods. Around 35,000 papers have resulted from the keyword “microgrid” in Scopus showing the level of interest in this research area among the power and energy community. Many research articles (~2000) are published related to the MG, smart energy system, and EM that are evident from the databases such as IEEE and Scopus. This study conducts a systematic literature review to examine several factors, analyzes and classifies MAS, MPC-based control, and optimization methods for MGs-EMS from the perspective of energy trilemma. To do so, a relatively large number of papers dedicated to the subject are acknowledged, introducing the taxonomy of control and optimization methods for the MG system. The focus of the paper is confined to the methods that use MAS, and MPC-based approaches for EM in MGs as there is a need for an increasing trend of independent behavior in the present dynamic energy system environment. MAS approaches suitable for such kind of systems and observed increasing interest in applying MAS technologies to the current MGs scenarios as they provide features such as active/reactive, social, scalable, and decentralized/distributed control. Moreover, due to the advancement in computational abilities, communication and privacy requirements, and lack of understanding of interactions between the control and optimization methods related to “complexity”, “uncertainty”, and “forecasting”, MAS, MPC approaches seem to be beneficial for MGs-EMS. As RES introduces intermittency and uncertainty to the MGs, MPC-based methods provide better control as it involves the forecasting information. For this systematic review, the papers screened through the keywords “microgrid”, “energy management”, “control and optimization”, “MAS”, and “MPC” along with decentralized, distributed, and hierarchical control architectures. Figure 3 shows the number of publications that use the MAS and MPC-based methods for MG control. It justifies the focus of this review on MAS and MPC methods for EMS in MGs.

The essential objective of a systematic literature review is to identify, assess, and interpret specific subjects in a certain field of study [43,44]. The available databases have been browsed and an architecture has proposed for selecting the potential literature to be analyzed in detail. Moreover, the methodology of [45], known as backward and forward search, is used in some cases; to further identify suitable publications. Firstly, the language of all manuscripts is checked (English and non-English). The next level of removal is continued by screening the title of the research works. The third level is filtering the title and abstract of the papers. The fourth is to filter the remaining papers based on the full text. The final step is to categorize the total available papers as demonstrated in Figure 4. Total of 176 and 290 articles published in the last decade in the Scopus database related to MAS and MPC-based methods for MGs filtered respectively through the process shown in Figure 4.

The keywords search and screening performed as below:

• MAS-based methods: “smart energy system”, “microgrid”, “energy management”, “multi-agent systems”.
  • A total of 176 papers were found and 3 of them were omitted due to non-English language.
  • Around 35 papers removed as they are significantly non-relevant subject areas, and the remaining papers were screened based on title, abstracts, quality of paper, and relevancy to the MAS control and optimization methods. (~75 papers, shown in Figure 5).

The selected articles sorted to examine separately the MAS control methods in terms of stages, type of control (decentralized/distributed), the methodology employed, and complexity as specified below.

• Papers on “hierarchical or bilevel control and optimization” are considered.
• Papers on “decentralized/distributed control and optimization” are screened. The MAS methods that employ different optimization algorithms such as AI-based methods, conventional methods such as ADMM, consensus, primal-dual, diffusion, and metaheuristic-based methods also considered.

2.

MPC-based methods: “Model predictive control”, “microgrid”, and “optimization”.

Then, similar criteria were considered to find and systematically sort articles of MPC-based methods.

• English as the main language for the research article, 24 papers were excluded. (266 papers)
• Papers that are significantly non-relevant subject area, Scopus bronze and green category papers with no licenses, and through the title screening were excluded. (106 papers)
• Forty papers were removed because of irrelevant information in the abstract.

Around 66 articles were found by the criteria discussed above. These articles were further identified based on the MPC methods in terms of complexity, uncertainty, and forecasting system behavior.

• Papers on the “complexity” on microgrid systems [46,47,48,49,50];
• Papers on the “uncertainty” in microgrid systems [51,52,53,54,55,56,57,58,59,60,61,62];
• Papers on “forecasting” on microgrid systems [44,60,63,64,65,66,67,68,69,70,71,72,73,74,75,76].

Figure 6 shows 51 journal and 15 conference papers and their share of those journals and conference papers published on MPC-based control approaches for MG-EMS also shown. Finally, among the searched articles, the selected articles for this review analyzed and figured out how the energy trilemma perspective considered in the MGs-EMS objectives, control and optimization methods as mentioned in Section 1.

3. Control Architectures for Energy Management Systems in Microgrid

An efficient EMS of MG depends on the control architecture and optimization algorithms that are equipped to manage their assets. Centralized, decentralized, distributed, and hierarchical control schemes are widely known as MG control architectures. Decentralized and distributed architectures have several advantages over centralized control methods such as improved flexibility, reliability, robustness, and the ability to avoid whole-system failure [77]. The evolution of the MG control and research questions addressed using decentralized, distributed, hierarchical control architectures summarized in [78,79,80,81]. The selection of the control architecture depends on the type of MG, mode of operation, and user or operator requirements. A survey of cooperative control frameworks with centralized, decentralized, distributed, and hierarchical architectures, and their operations for DC MGs presented in [82,83,84]. For DC MGs, hierarchical methods with AI, MAS, and metaheuristics algorithms are preferred over other control schemes [85]. In [86], a review and comparison of distributed control methods for AC MGs and the impact of communication failure on them were discussed. An analysis of centralized and decentralized MG control from the reliability perspective is provided in [87]. A distributed hierarchical control method is developed in a coordinated manner for economic dispatch and frequency regulation in the islanded mode for AC MGs [88]. To restore frequency and voltage, consensus-based adaptive control in a distributed manner considering uncertainty and disturbances developed for islanded MGs in [89]. In [90], hierarchical methods overview for both AC and DC types of MGs are summarized. The review of the hierarchical control strategies for AC, DC, and hybrid MG architectures are presented in [91,92]. Shrivastava et al. [93] presented a review from a cyber-physical perspective of MG architectures and control using hierarchical methods. Due to the time scale differences, achieving the primary, secondary, and tertiary levels of control in MGs improves operating efficiency and offers flexibility. Usually, local voltage control is at the primary level, frequency regulation and voltage restoration are at the secondary level, and finally, energy management is dealt in the tertiary level in the hierarchical control architecture as shown in Figure 7. Moreover, a droop-free hierarchical control strategy is proposed for inverter-based AC MGs [94]. Similar to hierarchical control, bi-level optimization and distributed cooperative control models for distribution networks with multiple MGs are given in [17]. This control can provide a plug-and-play capability and less communication pressure, it is shown in Figure 7.

The flexible, affordable, secure operation of MG is achieved by efficient EMS with the above-discussed control architectures. The methods such as MAS and MPC equipped with different optimization algorithms resulted in optimal operation of MGs-EMS. In the following sections, we discuss MAS and MPC-based methods for MGs-EMS from the perspective of each element of the energy trilemma.

3.1. MAS-Based Control and Optimization Methods

Distributed coordinated control with minimal data exchange and computational burden achieved for the EMS of MGs and their components with individual objectives by intelligent and interactive MAS architecture. Autonomous agent-based methods that inherently solve global objectives are gaining interest in developing distributed architectures, where the control effort is distributed across all the components in the MGs [77]. The review of such distributed control methods with MAS is presented in [78]. MAS-based distributed control for energy management in MGs and MG communities is reviewed in [23]. Multi-agent approaches using swarm-based and game theoretic methods for multi-generation MGs are presented in [79,80]. Distributed multi-agent platform is popular for efficiently managing the secondary control level in hierarchical architecture for synchronization and communication among the power converters in autonomous MGs. The MAS-based distributed consensus control methods for balancing MGs generation and demand have the advantages of low computational power requirements and minimal information exchange among the agents [27]. In the bilevel distributed control, the upper-level layer used the MAS approach for consensus and fair utilization of MGs [22]. A 2-layer coordinated control approach with MAS for MGs where the switching control of DC devices and loads are handled as discrete events using Petri nets is given in [81]. MAS approaches are developed for energy management in MGs with RES [82], ESS [83], houses and buildings [84], and different type of load patterns [85]. These methods addressed and involved in solving several objectives such as optimal cost of operation/generation, minimizing emission costs and line losses [86], the autonomy of supply-demand balancing, ED, scheduling, user expectations (in terms of heating or cooling), peak consumption, cost-benefit ratios, etc. MAS methods for EMS in decentralized structure addressed demand management (DM) [85], ED, and UC problems for both islanded and grid-connected AC MGs [6,86]. The agent-based decentralized control methods with Stackelberg game theory algorithms identified as efficient EM solutions for MGs and MG communities that address the problem of DERs integration [87]. ED and optimal operation of RE integration of EMS using RNN-based EMS with MA-based weather forecasting technique presented in [88]. An agent-based decentralized model with robust programming based optimization for MG-EMS implemented in [89].

Various software used in implementing multi-agent approaches for solving the objectives mentioned above were MATLAB + JADE, RSCAD with ACL communication, HTTP, TCP/IP communication protocol, and fuzzy logic [90]. These methods use individual system components as agents, which can have their objective functions, able to communicate and act independently. As MG systems involve several components with different natures, bi-level or three-level hierarchical control structure is very common in recent studies. The summary of objectives and the multi-agent-based control approach for MGs-EMS is provided in

Table 3. MAS control methods for EMS in MGs and perspective of energy trilemma.

Objective	Type of MG	Methods	Key Finding(s)	Energy Trilemma Element	Reference
Minimize energy bills	DC MG	Distributed control with MAS	MAS control implementation in DC MG for load shedding	Affordability	Rwegasira et al., 2019 [36]
Minimize operating costsMinimize interruption cost, maximize reliability	MGs	MAS (distributed)	Different control and protection challenges addressed by MAS, their advantages, and various optimization methods used for EM at the MG level discussed	Affordability, flexibility	Azeroual et al., 2020 [35]
Optimal size and cost optimization	DC MG	AI-based methods including MAS for EMS	EMS in MG with residential applications using different control architectures discussed	Affordability	Ali et al., 2021 [92]
EM in MGs considering factors fatigue life (FL) and Energy Not Supply (ENS).	Multi MGs	Distributed control with MAS	Tertiary control that addresses EMS in MMGs with novel consideration of FL factor and ENS	Flexibility, security	Rashidi et al., 2021 [93]
Peak load reduction, minimize cost and maximize benefits	MG	MAS with JADE	EM in MGs through load patterns and energy availability from DERs, storage, and DR	Affordability, flexibility	Nunna and Doolla, 2013 [85]
Supply-demand balancing and Load shedding	Islanded MG	Distributed control with MAS	MAS control implementation in DC MG for load shedding	Security, flexibility	Xu et al., 2013 [94]
EDP is to Minimize cost, there by optimal operation and power allocation	MG	MAS based hierarchical control with distributed Diffusion strategy as optimization method	Proposed methods have faster convergence, higher stability, also handle change in communication topology and realize plug-and-play of DERs ensuring optimal utilization of resources	Affordability, flexibility	He et al., 2019 [95]
Maximize RE utilization and minimizing cost by reducing power import	Grid connected MG With RE, battery, and EV	RNN for optimization considering MAS architecture for MG	The proposed approach determines the optimal power values for a time horizon of one week, for wind, solar, and battery systems and the utility grid.	Flexibility, affordability	Urias et al., 2014 [88]
Maximize individual and cumulative benefits, and utilization of resources, frequency and voltage regulation	Islanded MG	MA-based Hierarchical hybrid control	The upper-level agents provide the EMS to lower-level DERs agents for implementing the power assignment. Intelligent reconfiguration strategies of operational mode are established using information fusion and Petri nets	Flexibility, affordability, Security	Dou and Liu, 2013 [86]
EM of MMGs considering optimal operation, markets, and communication	Multi MGs (MMGs)	MAS-based hierarchical, distributed method and others—a review	Uncertainty modeling of EM in MMGs is addressed	Flexibility, affordability, security	Ma et al., 2022 [96]
EM considering RES, DGs, ESS	Campus MGs	MAS and Heuristics	EMS of MGs considering several objective functions and solutions tools are discussed in detail	Flexibility, affordability, security	Muqeet et al., 2022 [97]
Cost optimization Maximization of benefits, secure communication	MG clusters	EMS with MAS-based hierarchical control using heuristics and machine learning based models	Provided control strategies with single and multi-objective optimization methods for EMS. Cloud computing and ML-based control architectures for EM and their simulation platforms	Flexibility, affordability, security	Salehi et al., 2022 [98] Rosero et al., 2021 [99]

.

The control architectures that used MAS with several optimization algorithms in solving the objectives mentioned above have numerous advantages in MGs-EMS. Analyzing them with the help of trilemma elements such as flexibility, affordability, and security and considering the MAS characteristics will provide a different perspective that can help in selecting the suitable method of control for MGs that achieve desired operation.

(i)

Flexibility in EMS of MGs through MAS: In MAS methods, the MG components are considered as individual or autonomous agents with objective functions to achieve with minimal information exchange with the neighboring agents through communication. The autonomous nature of MAS helps in adding more RES and ESS. It leads to the effective utilization of flexible, locally generated energy, and reduces grid imports and emissions. The autonomous and scalable nature of the MAS framework enhances the plug-and-play capability of MG-EMS. When the MGs are operated in power deficit mode, the flexible operation of EMS through MAS is achieved by sending control signals to the agents that can participate in optimal DSM and DR or load shedding. Similarly, in the power surplus mode, the energy availability is communicated and stored for future use to reduce peak loads in the system. As the utility grid considered as an agent, it is ready to supply whenever MG-EMS suffers from energy deficiency. This greatly enhances the system’s flexibility in handling sudden failures of MG component and satisfies the local load demand. The autonomous, scalable, sociable, and reactive abilities are the MAS characteristics that offer flexibility in the EMS of MGs.

(ii)

Affordability in EMS of MGs through MAS: The economic operation of MG is the primary goal of EMS. The consideration of local as well as global constraints in the MAS methods provides efficient EM in the MGs by solving the EDP problem. Active and reactive characteristics of MAS control provide continuous monitoring of MG systems and effectively utilize the available energy. This enhances the utilization of locally produced energy and avoids additional power trading from the grid. This reduces the cost of operation and increases affordability to the different types of consumers. The MAS control methods are popular in providing decentralized and/or distributed control architecture. Thus, these are computationally less expensive than centralized control methods. The active and reactive features of MAS enhance the affordability of the MGs. In distributed approaches, the minimum operating cost and optimal power output of all the DERs achieved with cooperative/consensus methods. In AC-MGs, consensus can be obtained through incremental cost consensus (ICC) or gradient-based distributed methods [77]. Similarly in DC-MGs and hybrid MGs, ICC approaches can be extended [91]. These are easily possible with MAS approaches as different techniques can achieve consensus between agents.

(iii)

Security in EMS of MGs through MAS: Security of supply is most important for any energy system, especially for MGs operating in the islanded mode. The loads should be supplied by the efficiently scheduled available energy resources of MGs-EMS with the amount of energy requested. To satisfy the operational constraints of MG and its components, MAS methods with intelligent and interactive communication capabilities between the agents are most suitable. For example, the MAS development using JADE provides the communication capability through the ACL (agent communication language) following the FIPA (Foundation for Intelligent Physical Agents) protocols. The possibility of single-point failure of the MG-EMS can be avoided by the decentralized/distributed architecture of MAS through communication. Moreover, most sensitive loads can be given preference at the same time while maintaining the supply-demand balance, frequency, and voltage regulation in hierarchical control that uses the MAS approach to enhance the MG-EMS security. Secure communication and avoiding communication delays are the challenging aspects of MAS methods. The uncertainty and intermittent nature of the RES, security, and health aspects of batteries/ESS are considered during the control method modeling for MG-EMS. In the case of local agent failure, the isolation of faulty component of MG is possible, which improves stability and provide secure operation of MG and its components.

Table 3 presents the works that used MAS approaches in different types of control architectures for MG-EMS. It also shows the objectives identified along with the addressed energy trilemma element. As it is clear from the table, the affordability aspect of energy trilemma is by default included in the EMS studies of any MG as the cost-optimal operation always desired. Later, the effective utilization of available energy resources, storage elements, DG, DSM, and DR are given importance. This enhances the system’s flexibility. Finally, due to the modern MG challenges such as complex dynamic behaviors of MG components, uncertain environment, multi-mode operation of MGs, supply-demand balance, and system security become more important. The security aspect of energy trilemma has gained attention in the recent MGs-EMS studies with the increase in the number of individual components and developments of communication infrastructure among them. These individual agents in the MGs have dependent and independent operating nature to supply/receive reliable, secure energy, and increased concern over secure communication. The objective functions and their problem formulations include linear, non-linear, mixed integer and other difficult characteristics which require multi-stage control and efficient optimization algorithms (for detailed mathematical formulations, kindly refer to the respective original publications). The inference made from Table 3 is that the implementation of MAS, AI-based, hybrid (two or more) control approaches for MGs-EMS is gaining popularity over the other methods. Distributed and hierarchical control architectures with these methodologies are essential to address the energy trilemma (flexibility, affordability, security) issues in modern MGs. Based on the analysis performed, the constituents of MAS for achieving energy trilemma aspects in MGs-EMS are presented in Figure 8, i.e., MAS characteristics and abilities that help in developing various control architectures to solve objectives of EMS by employing different kinds of optimization algorithms are depicted in Figure 8.

3.2. MPC-Based Control and Optimization Methods

MPC-based methods are commonly used to solve mixed integer nonlinear objectives such as the UC problem of EMS in MGs. Addressing the characteristics such as variability, MG components uncertainty, and the use of forecasting information are included in the MPC-based control methods. MPC rolling or receding horizon optimization approach is also capable of providing efficient MGs-EMS that can provide over a considered time horizon either in day-ahead or real-time scheduling.

MPC-inspired EMS used a neuro-fuzzy algorithm that accounts for RES intermittent nature in grid-connected MG with loads and PV presented in [100]. To address the uncertainty issues with RES and loads, stochastic-based EM of MG that has several control objectives is resolved using MPC rolling horizon approach. To minimize the operating costs of MGs under uncertainty in wind speed, scenario-based stochastic programming with a rolling horizon approach is presented in [101]. The rolling horizon or MPC approaches are reactive-based methodologies that can modify or update the data obtained by deterministic approaches. In [102], a scenario-based MPC proposed to minimize operating costs and overall emissions. A chance constraint MPC for a grid-connected MG consisting of a gas turbine, battery, and PVs is presented in [103]. In this, optimal scheduling considering uncertainty in the lower level, and economic operation over a long-time horizon in the higher level achieved simultaneously. In [104], MPC-based optimal control for renewable energy MGs with hybrid ESS such as hydrogen ESS, batteries, and capacitors provided to achieve economical and flexible operation. To address power quality, and unbalanced power-sharing issues, a hierarchical MPC-based method used for islanded AC MG [105]. In addition, MPC methods applied for EMS that consider battery aging were also developed [106,107]. Several MPC-based building-to-building EMs are also proposed for residential MGs with grid connection [108,109] and islanded operation of MGs [110,111,112].

In the well-known 3-layer control, the objectives of the secondary control layer such as voltage, frequency regulation, and power allocation are achieved using MPC-based mechanisms with rolling or receding optimization. Moreover, communication cost reduction obtained using self-triggering based on the predictive feature of MPC [113]. MPC is used in the optimization framework of MG to optimize the power flows and reduce the peak load ensuring the quality of service to EVs in a vehicle-to-building environment [114]. Centralized EM for a building integrated MG (BIM) using MPC with finite horizon planning optimization was developed in [115]. This algorithm optimizes the BIM operation that satisfies the indoor temperature along with BIM components and power exchange with the grid. In [116], a dual decomposition-based distributed MPC for EM in MGs networks for feasible power exchange without raising privacy concerns is presented. Similarly, a hierarchical (upper and lower layer) distributed MPC that provides high reliability, efficiency, and flexibility in its control for standalone MGs with wind, solar, and battery power systems is presented in [117]. A chance-constrained MPC for interconnected MGs accounting uncertainty to provide EM strategies presented in [118]. MGs forming also helps with service restoration. This further improves the resiliency of the distribution system [119]. MAS employed with MPC also developed to control and optimize MGs for energy allocation [120]. MPC with robust optimization for islanded MGs with intraday EM proposed in [121]. MPC and their advancements that are used to achieve objectives such as maximizing energy efficiency, managing import (export) power from (to) the grid, and economic optimization show an increasing trend in the MG-EMS application that involves distributed RES and ESS [40].

The control architectures that solve the aforementioned objectives by combining MPC with various optimization methods have several benefits in MGs-EMS. A different perspective that can help in understanding the MPC approach by analyzing its features with the help of energy trilemma elements is given below.

(i)

Flexibility in EMS of MGs through MPC: The realization of centralized, decentralized, and hierarchical control architectures for EM in MG are made possible through the MPC and their variants. MPC can effectively achieve the control and optimization objectives of EMS through the features such as a rolling or receding horizon approach, accounting uncertainty, forecasting information, and reactive (feedback) mechanism. The flexibility of EM in MGs that employed MPC-based control methods is achieved by providing the forecasting information related to the RES generations and load demand. Real-time or day-ahead scheduling is possible through the rolling horizon approach and enabling the DSM and DR techniques in control architecture reduces peak loads. The plug-and-play capability provided to the extent would not increase the computational complexity. This plug-and-play of ESS and other loads improves EMS reliability and flexibility. The flexible operation of MG can be carried out with the help of control feedback from the available generation and load demand in the islanded or grid-connected mode of operation. Delays in the energy demands of MG components are allowed within the predictive time window. This further helps in tackling flexible demand profiles in the MG-EMS.

(ii)

Affordability in EMS of MGs through MPC: The problems with the nature of mixed integer linear and nonlinear characteristics considering several operating constraints are solved effectively using MPC-based methods. The UC, EDP, and optimal operation of MG-EMS also have a similar nature and were addressed widely using MPC methods. The integration of different optimization techniques such as robust, rolling, and stochastic optimization helped in achieving affordable and optimal EM within the MGS that employed MPC-based control approaches. The self-triggering feature of MPC will reduce the communication infrastructure requirement that further helps in developing an affordable control system for MG-EMS. The energy imports and exports from the grid are continuously monitored at the control unit. This helps in achieving the affordable and economical operation of grid-connected MG.

(iii)

Security in EMS of MGs through MPC: The ability to handle any disturbances and uncertainties is the most beneficial aspect of MPC control approaches for MG-EMS. The uncertainty issue of MG-EMS to enhance security is well addressed using the MPC-based methods with robust optimization. MPC control approaches are reactive in nature and can integrate new updated or forecasted information into the EMS. This improves system security by satisfying the system and component-level constraints. MPC with a rolling horizon approach reduces forecast error for real-time online operation and provides stable operation of MG. By doing so, MPC can predict the future behavior of the system that has constraints. The decentralized/distributed architecture of MPC used to control the power flows of the complicated hybrid power system. MPC has a fast transient response since the main role of MPC is to integrate new updated data and forecast information. Additionally, giving precedence to the most sensitive loads simultaneously with preserving supply-demand equilibrium, frequency regulation, and voltage regulation in hierarchical control that employed the MPC approach improves the security of the MG-EMS. When modeling the control method for MG-EMS, the security and health aspects of batteries and ESS, as well as the uncertainty and intermittent nature of RES, are considered.

MPC uses many optimization techniques including rolling (receding) optimization, finite horizon optimization, convex programming, MILP optimization, and multivariable optimization as shown in

Table 4. MPC control approach with objectives and their relation to energy trilemma.

Objective	Feature	Type of MG	Methods	Key Finding	Energy Trilemma Element	Reference
Economical operation	Uncertainty of RES	MMGs	Distributed Robust MPC (DRMPC) With Column and constraint algorithm (C&CG)	EMS modeled as DRMPC technique has ability to balance the robustness and economy of MG operation	Affordability, security	Zhao et al., 2022 [51]
	Stability	Data center MG	Stochastic MPC (SMPC) with Rolling optimization	Able to handle negative influence of uncertain factor in achieving economic operation		Zhu et al., 2022 [122]
Minimize operational cost	Uncertainty in AC/DC loads	DC-MG	Deep-Q network-based Reinforced learning	Comparison with Stochastic MPC (reduced 41.9% operating cost)	Affordability	Thanh et al., 2022 [123]
Reduce PV curtailment, and unmet load	Reliability, forecasting uncertainties	Standalone MG	GA	Reduction in capital cost and help in downsizing the system	Affordability, flexibility, security	2021 [96]
Frequency regulation	Uncertainty in MG	MG	Adaptive Intelligent MPC (AIMPC)	Frequency deviations in a MG lower degradation tendency and lesser cost of an EV battery	Security, affordability	Khokhar and Parmar, 2022 [124]
Recovery of voltage/frequency, complexity reduction	Complexity of MG control	MG	MPC with rolling horizon approach	Reduce communication cost	Security, affordability	Dong et al., 2022 [113]
Complexity reduction in MG control	Uncertainty of battery usage	Building MGs	Hierarchical MPC (MILP formulation)	Reduces annual costs for residential and non-residential building MGs	Affordability	2020 [125]
Optimal operation considering uncertainty	Uncertainty of RES	MG	Robust MPC	Reduce operating cost, reduce peaks, and uniform grid consumption	Affordability, security	Marín et al., 2019 [126]
Increase in RE self-consumption within MG	Forecasting of load demand	MGs	MPC (Convex problem formulation)	EMS achieves reduction in cost by 30%	Affordability	Elkazaz et al., 2020 [127]
Scheduling	Multi-time scale, forecasting of RE uncertainties, load demand	Buildings	Receding horizon rule-based MILP, MPC	Optimal charge and discharging, reduced annual storage capacity loss of EV batteries by 23%	Flexibility	2022 [128]
Minimize unserved load and thermal discomfort	Uncertainty of the outdoor air temperature and solar irradiance forecasts	Buildings	MPC (MILP formulation)	Discussed impact of power flexibility in buildings	Flexibility	Wang et al., 2020 [129]
Reduce peak load and optimize power exchange	Uncertainties of power generation, operational flexibilities of EVs	Corporate building MGs	MPC with Constrained Optimization	Peak loads are reduced for building MGs	Flexibility, affordability	Ouammi, 2021 [130]
Decrease congestion and peak loads	Forecast RE generation and load	MG	Agent based, MPC, and Multi-objective optimization	Decentralized approach reduces the risk of system failure	Security, flexibility	Fontenot and Dong, 2019 [44]
Manage the data transmission in MG	Uncertainty/Forecast of the weather	MG	PSO, GA, MPC	Reduce at least 40% prediction error	Security	Yuan et al., 2019 [131]
Minimize the stochastic forecast errors of RES generation, ESS	Forecasting of the weather	MG	Hybrid MPC And Mixed-Integer Nonlinear programming (MINLP)	MG cope up with large disturbances	Security	Liu et al., 2019 [76]
Minimize rate of degradation batteries	Uncertainty of the battery	DC MG	MPC with multi-objective optimization	Reduce the reserve capacity of the grid	Affordability, security	Nair and Costa-Castelló, 2019 [132]
Minimize the prediction error	Forecasting of power demand	MG	Mixed Integer Programming (MIP) and MPC	Decrease in total cost by 22.69%	Affordability	Zhang et al., 2018 [67]
Stabilize of the grid import/export	Complexity of MG control	MG	MPC with Receding Horizon Optimization	Improve the power quality of PV power plants	Security	Lei et al., 2017 [133]

. The constituents of MPC-based control approaches for achieving the energy trilemma in MG-EMS summarized in Table 4 are illustrated in Figure 9.

In summary, the MAS and MPC frameworks are well used in control methods for MGs due to their structural features and control operations. The two mentioned control approaches have the majority share among the recent application to EMS. As discussed in the papers mentioned in Table 1, Table 3 and Table 4, the MAS framework is known for its advantages such as decentralized/distributed architecture, communication capability among the system components, and characteristics such as autonomous, social, active, proactive, and scalability. This results in developing several research works that use MAS as the main control engine in all the levels of control in MGs (such as primary, secondary, and tertiary control in MGs). At the same time, conventional control techniques such as MPC and their advancements are widely popular methods for EMS in MGs. MPC possesses a variety of beneficial traits [134,135]. It is possible to incorporate physical constraints directly and intuitively, and have an excellent dynamic, reliable control system. Direct control signals produced results in simplicity. Complex optimization problems are made easier to solve by integrating a variety of techniques into the MPC control approach. In addition, researchers working on MPC advancements in MGs must consider different factors such as RES intermittency, load-sharing accuracy, circulating currents, grid stability, etc. Finally, for the EMS in MGs with the requirements of distributive control architectures, consensus, and effective management among the MG components, plug-and-play and scalability in adding RES, ESS, etc., and services such as DSM and DR, MAS-based intelligent interactive approaches are beneficial over MPC and other methods. However, due to the adaptability, capacity to apply in any field, predictive and feedback nature, accounting uncertainty, and intuitive characteristics, MPC-based methods are advantageous in hierarchical control architectures of MGs-EMS with RES, ESS, and dynamic loads. Additionally, to overcome the pitfalls of each other, the ensemble of MAS and MPC-based methods can be a possibility to use at different levels of control in MGs.

3.3. Optimization Algorithms in the MGs-EMS

The use of the control and optimization method is based on the type of MG structure, individual entity objective functions, and their formulations. The classification of optimization techniques used in EMS includes AI-based, conventional mathematical, metaheuristic, and others. Fuzzy logic, game theory, multi-agent, neural network, and reinforcement learning methods come under the category of AI-based methods. Dynamic programming, robust programming, stochastic programming, bilevel programming, mixed integer programming, MPC-based methods are conventional approaches. Swarm intelligence, evolutionary, and heuristic approaches are of metaheuristic type. Hybrid methods and groups of two or more of the above-described types are categorized as other methods for achieving the optimization objectives of MG-EMS. In [6], the authors presented a comprehensive review of optimization techniques for addressing several control objectives in the EMS of MGs. Four major types of techniques and their subclassifications are presented in detail. Moreover, an interesting qualitative analysis was performed among methods that solve the UC, ED, forecast, and demand side management (DSM) objectives. It concluded that mixed integer programming, MAS approaches, and metaheuristics such as PSO are mostly used methods to address MGs-EMS. MAS-based methods are best for the MG to provide solutions for UC, and DSM with high efficiency, reliability, and adaptability, whereas metaheuristics such as PSO are simple and reliable, better for forecasting and ED. The ensemble of these methods collectively achieves the best results for solving EMS problems of MGs. Therefore, the idea of using hybrid approaches that include futuristic EMS requirements in a collaborative MG community with accurate forecasting, and scheduling to improve economic and computational benefits paramount in the literature.

In MAS, the optimization techniques are employed as distributed control strategies in the MGs-EMS because of the distributed nature of the agents as demonstrated in [78]. A critical overview of the distributed control systems utilizing MAS provided in [136]. The work in [86] presented a MAS-based hierarchical control for an autonomous MG intending to maintain a fixed voltage while maximizing economic and environmental benefits. To lower system peaks and integrate demand response with distributed storage, [85] suggested an energy management model employing a MAS-based system constructed in Java agent development framework (JADE). In [94], the supply-demand imbalance due to the high availability of RE is addressed by using a MAS with the distributed sub-gradient algorithm for EMS. Multi-agent-based MG supervisory controllers and their use were examined by [137]. A three-layer MAS-based controller for economic dispatch presented in [138] consists of producer, consumer, and observer agents. A MAS for DSM and schedule coordination is presented in [139].

In [17], an EMS problem divided into a UC and an optimal power flow problem. The mixed-integer non-linear problem was solved using an MPC-based approach. The suggested model’s performance assessed using the CIGRE medium-voltage benchmark. A model developed for efficient battery scheduling based on a receding control horizon is presented in [140]. The application of MPC control to solve the DSM in MG is demonstrated by [141]. Centralized and decentralized control architectures with MPC methods for EMS formulated as MINLP in grid-connected MGs proposed to solve objectives that include forecasting, DSM, and UC [6]. Among the conventional approaches, MPC and their variants always make their place in competition with modern AI-based techniques as it effectively solves UC-like problems of EMS with an MINLP nature.

Heuristics and evolutionary algorithms are popular in solving single and multi-objective optimization problems. In MGs-EMS, PSO and its variants are widely used. For optimal economic dispatch, GA are widely used evolutionary-based algorithm for EMS in MG [142]. ESS-introduced uncertainty is widely addressed by PSO-based optimization methods for EMS in MG [143]. In order to reduce operating costs and carbon emissions, a multi-objective intelligent EM control of a MG system developed in [144].

Game theory (GT) is a mathematical theory developed to investigate the rational decision-making of a decision-maker to resolve conflicts and the cooperativeness of a system to attain a common, well-defined goal. For a decentralized DSM of grid-connected home MG with EV, ESS, and RES, depending on the load demand forecasts, EMS engages in a mixed-strategy non-cooperative game until it reaches a Nash equilibrium, changing the anticipated consumption pattern to reduce the overall cost of electricity [145]. The deployment of agent-based and GT-based EM solutions for community MG has been found as the most effective approach to address the concerns of increased DERs integration into MGs. The decentralized nature of the MG EMS system benefits from the characteristics of the GT-based algorithms.

With an adaptive nature to the environment and incorporating user input into the control logic, RL-based methods have been widely used in decision-making. In [146], a thorough analysis of modeling approaches and algorithms for demand response using RL is presented. RL can coordinate with multi-agent systems that perform demand response based on generated penalties and rewards. For the rescheduling of a completely automated EMS, [147] used Q-learning. An effective scheduling and control technique using RL for heating, ventilation, and air conditioning (HVAC) systems implemented in [148]. The results demonstrated that the RL-based controller could cut energy consumption by 22% while maintaining interior thermal comfort compared to a hand-crafted baseline controller. NN, FL, and other AI techniques are also generally used in MG-EMS. The NN-based forecasting technique is typically employed to estimate the production of stochastic renewable energy sources. Forecasting and demand response models are also necessary for NN-based energy management algorithms. Additionally, NNs employed in EMS schemes that concentrate on ED and optimal operation of the integration of renewable energy [88]. An RNN-based energy management system and a multi-agent-based weather forecasting technique is presented in [88]. The coordination of various ESSs in a microgrid while taking into account an interconnected topology has also been optimized using the RL algorithm [149].

Finally, the other conventional solvers that solve mixed integer linear/nonlinear EM problems were also given in [6]. Minimizing the total operational cost as a convex optimization problem both for offline and online EMS is solved using dynamic programming [150]. MIP is widely used for solving EMS problems due to its simplicity and low computational cost. MILP is also used to manage generation and demand alongside load forecast in a rolling horizon approach [151]. The objectives in MIP are optimal scheduling, sizing, and balancing of the generation and demand of RES, ESS in MG-EMS. A stochastic MILP model for optimal EM and sizing of RE and battery presented in [152].

Energy trilemma aspect in optimization methods: As discussed earlier in Section 3.1 and Section 3.2, MAS and MPC methods have the edge over other methods to implement control and optimization architectures for MGs-EMS. The AI-based methods for EMS provide flexibility in solving plug-and-play capabilities of MGs-EMS. The algorithms that are used for EMS should be scalable for solving increase in dimensionality or non-linearity in the objective formulations. AI techniques such as RL and machine learning methods, metaheuristics that can handle such complex objective functions are gaining importance in the recent studies due to higher amount of data availability and handle the dynamic nature of the system. The computational complexity should be reduced with the help of efficient solvers and different simulation platforms. In addition to solving the optimization problems associated with EMS, the proper communication between different entities of EMS of MG is important. Such communication is useful for developing distributed architectures with minimal privacy concerns and improving security aspect of EMS.

In summary, review papers published in recent years discussed different control architectures with various optimization algorithms developed, and tools used for MGs-EMS along with key recommendations are given in

Table 5. Summary of control and optimization methods used for different MG and their EMS.

Reference	System of Study	Research Question Addressed	Discussed Control & Optimization Methods/Models/Software	Key Recommendations
Ma et al., 2022 [96]	MMG	Uncertainty modeling for EM in MMGs	Centralized, distributed, hierarchical control with: SO, RO; GT, MA, AI	Future challenges regarding MMGs planning, operation and control, integrated system EMS, increased communication, IT, and market transactions
Kamal and Chowdhury, 2022 [37]	Networked MG	Objectives from voltage and frequency control to economic OPF	Centralized, decentralized, distributed control using MPC based approaches	Predictive control with advanced mathematical modelling required to improve precise predictive model and stability
Lei et al., 2022 [153]	Standalone—Aircraft power system (APS)	Trends in optimization and EM methods for APS—Multi objective approach	MPC, AI-based methods for EMS	Along with EMS considering thermal Management System, prognostics, and health management on aircraft power system
Nafkha-Tayari et al., 2022 [154]	Virtual Power Plant (VPP)	Review of control and optimization methods, and markets	Centralized, distributed hierarchical control: SO, RO, IA, SO + IA, heuristic, Fuzzy, modified crow search (MCS), ANN, GA, Swarm-based, GT, Multi-objective optimization, auction based, MA	To develop a MAS for advanced distributed EM of a commercial VPP
Alonso-Travesset et al., 2022 [155]	MG	Optimization of DG under uncertainty: problem formulations, objective functions, optimization algorithms	LP/MILP/MINLP, heuristics, SP (2-stage, Multi-stage), RP, chance constraints, RH, MPC, GT;	Future models with additional features such as grid test bed, battery aging, DR, uncertainty, decentralization will bring more real-life settings
Muqeet et al., 2022 [97]	Campus MGs	EM methods of campus MGs with Distributed generation from PV, wind, FC, diesel generator, and ESS: objective functions, solution tools	IA (heuristics), MAS	Including of scheduling studies of DGs in MG optimization. Need for advanced approaches such as blockchain, AI, and methods in EM of MGs
Salehi et al., 2022 [98]	MG/Community MGs	Control strategies and single, multi-objective optimization methods in EM of MGs considering practical constraints	Master-slave, Peer-to-Peer control strategy: GT, Dynamic MPC, MAS Optimization: Probabilistic, and Deterministic, evolutionary, co-evolutionary approaches	Using of Deep Reinforcement Learning algorithms to improve optimization and reducing computational burden
Rosero et al., 2021 [99]	MG clusters	Cloud computing, ML based control architectures for real time EM in MGs clusters and their simulation platforms	Hierarchical control, Real time and with cloud, ML-based models (Gaussian Process), and MAS	Real time EM for interconnected MGs using cloud and ML-based architectures
Al-Saadi et al., 2021 [34]	MG with ESS	Review of various control methods (MA, RL based) to manage Distributed ESS: control complications and solutions	Centralized, decentralized, distributed control: RL-based and MA approaches	Importance to digitalization and AI-based methods in control
Rashidi et al., 2021 [93]	Multi MGs	Tertiary control review for EM in MGs with factors like Fatigue Life (FL) and energy not service: control methods and formulations review	Distributed control with MAS Software: MATLAB and Dig Silent	Consideration of more realistic model with reliability and social welfare and more MG connecting points
Topa Gavilema et al., 2021 [156]	MGs	EM of MGs with different configurations and their solutions: classification on the optimization methods used	Metaheuristics, LP/NLP, DP, Stochastic and robust programming, MPC, MAS, AI-based methods	Need of efficient methods for handling complexity arises from the interaction of MGs and the optimization of their objectives: distributed control techniques
Ouramdane et al., 2021 [15]	MGs with Vehicle-to-Grid	Overview of MG technological advancements, EMS with DERs, ESS, and mainly EV integration	Centralized, decentralized control methods: Metaheuristics, LP/NLP, DP, Fuzzy, Neural network, and MAS optimization methods	Need address challenges related to Power to Gas, scalable communication, cyber security, and the use of ML-based (AI) methods
Ali et al., 2021 [92]	DC MG	Review of DC MG architectures, control strategies and EMS for residential applications in terms of size and cost optimization	Centralized, decentralized, distributed, hierarchical control methods: Classical (LP/MILP/SO/RO/MPC and AI methods (Fuzzy/NN/EA/MAS) for EMS optimization	Hybrid methods for EMS as growth in AI and consideration of Health management strategies for components such as ESS, PVs, converters.
Azeem et al., 2021 [157]	Hybrid MG	Review of integration challenges, optimization, and control strategies in terms of efficiency, design flexibility, security, reliability, and cost effectiveness parameters	Droop control/impedance control/MPC/virtual inertia control; Optimization techniques regarding Power Flow, uncertainty, and design and topology (Heuristic, fuzzy, mathematical, swarm-based approaches, etc.)	Possible future developments regarding the hybrid MGs with DC generation, nonlinear loads, Plug in hybrid EVs
Hu et al., 2021 [40]	MG	Review of MPC based converter and grid level control strategies in Hierarchical methods for MG-EMS	Converter level and grid level MPC; MPC in 3 level control architectures of MGs	MPC advancements in future MGs
Hu et al., 2021 [158]	Residential MGs	Demand side flexibility study: co-ordination and negotiation methods classification and techniques	Centralized, decentralized, distributed (hierarchical and non-hierarchical) control architectures: Optimization: (LP/MILP/MINLP), (GA, PSO, SA, greedy algorithm), GT, MAS	Critical discussion about co-ordination and negotiation architectures and techniques (MAS, GT) and their addressing their challenges in future MGs with multiple players/agents
Anderson and Suryanarayanan, 2019 [159]	Islanded MGs	Review of planning, optimization, EM methods along with their objectives, constraints, and control variables in islanded MGs
Yamashita et al., 2020 [160]	Building MGs	Hierarchical control layers in EM of Building MGs	Deterministic, stochastic and robust, Metaheuristics, AI, MPC methods	Insights regarding building prosumers in Building MGs
Espín-Sarzosa et al., 2020 [161]	MGs	EM in MGs using centralized control architectures: main research trends	Focus on centralized control methods Optimization: Mathematical Programming (MP), computational intelligence (CI) or AI, Hybrid methods (MP and CI)	Clustering approach in centralized EMS developments
Fontenot and Dong, 2019 [44]	Building MGs	Review of modeling challenges and methods to solve using physics based, data driven models as well as control methods	Rule, optimal control (LP/NLP/MI(N)LP, DP, SP), MAS, MPC-based methods	Incorporation of advanced occupancy models, agent-based modeling, Building-2-Grid integration
García Vera et al., 2019 [90]	MGs with RES	EM of MGs with RES -optimization tools and simulation tools review	LP/NLP/MI(N)LP/DP/Stochastic and Robust programming, metaheuristics, swarm-based/MAS/MPC/AI/fuzzy-based methods	Directions on Predictive modeling with ESS
Khan et al., 2016 [162]	MGs	Review on existing optimization methods for EM in MGs	Type of optimization in EMS: LP/NLP/SP/DP; Heuristics, MAS/MPC/AI; Different platforms/tools for EM: AnyLogic, CPLEX, Dig SILENT Power Factory, GAMS, MATLAB/Simulink, MATPOWER, PSCAD/EMTDC, SIMPLORER, SCENRED and VER	Cost effective EMS for Smart Microgrid Network (SMN)

Table Legend: (C = Centralized, D = Decentralized, Di = Distributed, Hy = Hybrid, H = Hierarchical) (SO = Stochastic Optimization; RO = Robust Optimization, IA = Intelligent Algorithms (AI-based), heuristics, GA = Genetic Algorithm, Game Theory = GT, MA = Multi-Agent, AI = Artificial Intelligence, LP/MILP/MINLP = Linear Programming/Mixed Integer LP/Mixed Integer Non-LP, SP = Stochastic Programming, RP = Robust Programming, DP = Dynamic Programming ANN = Artificial Neural Network, ML = Machine Learning), RH = Rolling/Receding Horizon, DG = Distributed Generation, EM(S) = Energy Management (Systems).

below.

4. Future Research Directions and Discussion

The review presented in this paper aims to check how the energy trilemma is considered in the MPC and MAS-based EM methods of MGs. Usually, affordability, flexibility, and security are considered in different ways. The review reveals that the EM problem or the control and optimization problem was a mono-objective where a single objective function related to the operational costs of the microgrid widely addressed. In this case, one of three terms of the energy trilemma was considered, and the other two terms were included implicitly in the problem constraints or were supposed to be provided through the microgrids architecture and constituents. For example, including the energy storage or the demand-side response programs can provide flexibility to the network. Moreover, having different types of generators will prioritize the security of the supply. This security of supply can be affected by a failure of the communication system. Hence, some research papers considered these aspects by developing a centralized energy management system. When more than one element of the energy trilemma is included in energy management, the control/optimization problem becomes multi-objective. The objective function was formulated to reduce the operational costs (affordability aspects), increase the security of supply by imposing penalties in case of unmet demand (security aspects), and increase flexibility by incentivizing the utilization of locally generated and/or traded energy. The environmental dimension was considered by incentivizing the use of local renewable energy. The current review reveals that the following aspects related to EMS of microgrids require to be considered further by the research community:

Stability of microgrid: Issues related to the stability of MG affect the security of supply. Over or under frequency and voltage are considered currently through the balance of active and reactive power flows in a three-phase balanced microgrid. However, the imbalance in loads and impact of non-linear loads on the microgrid stability have been ignored in the literature. Hence, there is a need for an EMS that considers the imbalance in load and the impact of non-linear loads, and to maintain the MG stability.

Resilience of EMS against cyber-attacks: Cyber threats affect the security of the supply of MG. The focus of the current research was to reduce communication delays in distributed and decentralized architectures. However, it is essential to develop a resilient EMS that can deal with cyber-attacks in a way the energy supply will not be affected.

Scalability of EMS of a microgrid: EVs significantly affect the flexibility and considered widely in the MG-EMS studies. EVs are static energy storage systems with constraints related to their availability; for example, EVs will not be available for specific periods. Currently, the EMS of MGs charge the EV batteries during excess generation and discharge them to cover the load during generation shortage. However, it is essential to develop an MG-EMS that can consider the dynamic behavior of the EVs in the microgrid; the capacity and the stored energy are variable, and related to the drivers’ preferences/behaviors and many other parameters. The change in the number and size of EVs available at a specific time period arises scalability issues for the EMS. Considering hybrid ESS [163], battery health degradation, state-of-the-charge of batteries are also should be accounted in EMS for MGs with storage systems. These directly affect the operational costs, which in turn impact MG-EMS affordability and security.

EMS algorithm advancements: Many methods developed based on the system of systems (SoS) approaches for MGs and smart grids [164,165,166,167,168,169]. These methods are easy to implement and have supervisory control nature. However, these fail to provide optimal EMS. MAS and MPC-based methods are effective and optimal but dependent on MG complexity, architectures, or model. Therefore, combining SoS and optimal-based methods provide efficient and optimal solutions needed for MG-EMS. In addition, there is a need for AI advancement in MGs control and optimization approaches that enable EM in futuristic energy systems are required. In the MAS control framework, the agents may have objectives of conflicting nature and require efficient optimization algorithms. In solving such complex problems with conflicting objectives, the multi-objective evolutionary algorithms (MOEAs) are well known. However, the applications of the MOEAs are little-known in MGs-EMS studies. MOEAs in the control methods for MGs-EMS can be a potential application and result in a set of Pareto optimal EM solutions.

5. Conclusions

This review presented the energy trilemma perspective for MG-EMS by categorizing the objectives, control and optimization methods in terms of flexibility, affordability, and security. The focus is given to the MAS and MPC-based approaches with different control architectures for energy management in MGs. The objectives that affect each element of the energy trilemma are presented in detail. The literature was scanned for MAS and MPC-based control methods from the energy trilemma perspective, and several inferences made from the existing research. The detailed analysis of MAS and MPC approaches, optimization methods, and their constituents to achieve the energy trilemma presented in this review can provide a fresh perspective for MGs-EMS. The MAS and MPC methods are used in all types of control architectures. However, MAS control complexities rise due to interactions and market transactions [170] in interconnected MGs. Incorporating AI-based machine learning and reinforcement learning algorithms in the MAS-based methods has drawn attention recently to developing realistic EMS for MGs with ICT technologies. Though the MPC approaches are used widely in developing EMS due to their fast dynamic response and reactive features in handling uncertainties, they require high computational power. Adaptive predictive control with AI and hybrid control techniques and practical SoS methods such as logical evolution operator-based techniques (epsilon operator) are employed in MPC to decrease the computational complexity. Finally, future recommendations and analysis related to stability, scalability, and resiliency of MGs-EMS that affect flexibility, affordability, and security in addition to EMS algorithm improvements are presented to benefit the MG research community.