Review of Computational Intelligence Methods for Local Energy Markets at the Power Distribution Level to Facilitate the Integration of Distributed Energy Resources: State-of-the-art and Future Research

Abstract

The massive integration of distributed energy resources in power distribution systems in combination with the active network management that is implemented thanks to innovative information and communication technologies has created the smart distribution systems of the new era. This new environment introduces challenges for the optimal operation of the smart distribution network. Local energy markets at power distribution level are highly investigated in recent years. The aim of local energy markets is to optimize the objectives of market participants, e.g., to minimize the network operation cost for the distribution network operator, to maximize the profit of the private distributed energy resources, and to minimize the electricity cost for the consumers. Several models and methods have been suggested for the design and optimal operation of local energy markets. This paper introduces an overview of the state-of-the-art computational intelligence methods applied to the optimal operation of local energy markets, classifying and analyzing current and future research directions in this area.

1. Introduction

The power distribution system is evolving rapidly in recent years, forming the smart distribution system of the new era [1]. One of the major drivers of the changes is the massive integration of distributed energy resources (DERs) that include distributed generation (DG) units, energy storage systems (ESSs), and demand response (DR).

DG units are small generating sources connected directly to the distribution system or on the site of the consumer meter. In recent years, the penetration of renewable and non-renewable DG units is rapidly increasing worldwide, promoted and encouraged by international and national policies aiming to increase the share of renewable energy sources (RESs) and highly efficient micro-combined heat and power (CHP) units in order to reduce global greenhouse gas emissions and relieve global warming. In addition to environmental benefits, DG units offer several other advantages, including diversification of energy resources, contribution in the implementation of competitive energy policies, reduction of on-peak network operation cost, deferral of distribution network reinforcements, lower power and energy losses, lower transmission and distribution costs, and possible increase of service quality for the final customer [2]. Moreover, DG units are available in modular sizes, characterized by ease of finding installation locations for smaller generators, shorter manufacturing times and lower investment costs.

Renewable DG units, e.g., wind turbines (WT) and photovoltaic (PV) units, due to their volatile and interruptible power output, are variable (not dispatchable) generation sources, so they create security and power quality challenges for the distribution system operator (DSO). Distributed energy storage systems are used throughout power distribution network to provide flexibility and to compensate for the intermittent nature of renewable DG units, and, in this way, ESSs help increase the penetration of renewable DG units [3]. Moreover, ESSs can help increase energy efficiency and reduce electricity cost by providing energy arbitrage services, i.e., ESSs store energy during low demand (off-peak) periods when electricity cost is low and provide energy during peak demand periods when electricity cost is high [4]. Energy storage systems can also strengthen the resiliency of smart distribution network to power outages due to extreme weather events [4].

Demand response aims at changing the consumption of customers from their normal electrical consumption profile in response to changes in electricity price, or to incentives (payments) designed to cause lower electricity consumption at periods with high electricity prices [5]. The customers who participate in DR can change their consumption with three different ways: (1) reducing their consumption with load curtailment; (2) moving part of their consumption to a different time period; and (3) covering part of their consumption from energy generated locally at their site. DR offers economic advantages for both the customers and the DSO. Moreover, by reducing peak demand in long term, demand response helps reduce network investments and defer the necessity for distribution network reinforcements. A great variety of DR programs is already offered by the electric utilities to encourage and promote customer participation [5].

Innovative information and communication technologies and smart distribution systems have motivated consumers to become more active players instead of simply consuming energy. Indeed, active consumers can participate as prosumers (producers and consumers) in power generation and consumption, using their local renewable DG units, managing their consumption, and communicating with other players in the context of transactive energy and local energy markets (LEMs) at power distribution level. More specifically, in the context of transactive energy, transactions can take place among prosumers, between prosumers and distribution system operators, as well as among distribution system operators and wholesale electricity market (WEM) [6]. Mainly due to generation size restrictions, small-scale market players were formerly excluded from the energy market. Recently, this issue has been addressed and small-scale prosumers can participate in local energy markets that operate at power distribution level. As an example, KEPCO has already established LEMs that allow electricity trading among prosumers.

Computational intelligence (CI) methods are very successful in the solution of complex optimization problems. Moreover, CI methods are also very capable in forecasting complex processes. In the context of LEMs, CI methods have been successfully applied for the optimal scheduling of large-scale smart distribution networks, with complex LEMs, having many market players and numerous DERs. Moreover, CI methods have been successfully applied for solving power forecasting problems of volatile and intermittent renewable DG units, like WTs and PVs, which are then optimally scheduled within LEMs.

The way energy trading is implemented at large scale in WEM is different with the smaller-scale trading at local energy markets. Moreover, the presence of LEMs at power distribution level has a significant impact on the optimal operation of smart distribution networks. Recently, review works have analyzed the optimal operation of smart distribution networks and local energy markets. More specifically, the work [1] reviews the optimal operation of smart distribution networks and the basic schemes for active network management. The works [2] and [3] review the optimal allocation problem of DGs and ESSs, respectively, at power distribution networks. The work [5] reviews the potential and benefits of DR in smart grids. The review work [7] analyzes European regulations and identifies key enablers to promote LEMs. The work [8] reviews potential designs and market clearing methods for LEMs. The review work [9] classifies business models of LEMs.

The above bibliography review shows that there is no review paper focused exclusively on the use of innovative CI techniques for the optimal scheduling of LEMs in smart distribution networks. This paper covers this bibliography gap and introduces a taxonomy of models and CI methods for the optimal operation of LEMs in smart distribution networks, offering a unified presentation of a relatively large number of selected high quality research works [10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50]. These state-of-the-art CI methods include multi agent systems (MASs), particle swarm optimization (PSO), genetic algorithm (GA), fuzzy sets, artificial neural networks (ANNs), and support vector machines (SVMs).

The contributions of this review paper are manifold:

• It is the only review paper of the bibliography that is focused on the application of computational intelligence methods for the optimal operation of LEMs.
• It is the first paper that reviews LEM models that are optimized by CI techniques and introduces various well-designed classifications.
• It provides future research directions and sets the future research goals of regulatory framework, formal definition of the roles of local energy market players, transmission system operator (TSO) and DSO coordination, communication network and software platforms for energy trading, distribution infrastructure investments, profitability of LEMs, scalability, data privacy, and advanced models and methods for the optimal operation of LEMs.
• It serves as a guide to aid engineers and researchers on the available models and CI methods as well as the future research trends in the design and operation of LEMs.

The structure of the paper is as follows: Section 2 analyzes the attributes of LEMs operating at power distribution level. Models and CI methods for LEMs are reviewed in Section 3 and Section 4, respectively. Section 5 identifies the core contribution of the reviewed CI methods for LEMs. Section 6 provides future research directions. Section 7 summarizes the main findings and concludes the paper.

2. Local Energy Markets at Power Distribution Level

In the traditional power system, electricity is generated by few large-scale generating units, and is transferred to residential, commercial, and industrial customers through the corresponding transmission and distribution systems. The wholesale electricity market facilitates the trading of electricity in a centralized manner. The design of WEMs is not easy for two main reasons: (1) as opposed to other goods, electric energy cannot be stored economically and on a large-scale; and (2) electricity must be produced simultaneously with electricity demand, taking into account that electricity delivery is implemented according to physics laws. Countries worldwide have successfully faced these challenges by developing WEMs that permit electricity suppliers to compete to provide energy, capacity, and reliability services. Since their emergence, WEMs have evolved significantly. Indeed, new rules, market participants, technologies, software, and algorithms are added frequently [51].

The ever-increasing penetration of variable energy resources, namely large-scale and distributed renewable energy sources, has tested the WEMs worldwide. Regulators, WEM operators, electricity suppliers and other market players have all contributed with modifications to secure fairness, economic efficiency, and power system reliability.

The increasing share of fluctuating (variable) renewable energy sources at the distribution network is a challenge, because the distribution network was not originally designed to accommodate significant amounts of local DG. The increase has been accompanied and facilitated by an emerging decentralized ecosystem where, together with RES, new loads, such as electric vehicles (EV) charging infrastructure, are gaining importance [15]. These distributed ecosystems pose serious challenges to the distribution networks owned or operated by DSOs, because the elements of the distributed ecosystems can be operated by third parties, either individually or in a coordinated way. With low penetration levels, the effects of variable RESs and other DERs may be ignored. However, as the penetration levels rise, a new approach is required to integrate and manage the vast amount of DER that is expected to drive the grid in the not so distant future [52]. DER affect power distribution networks since they originate reverse power flows feeding at medium voltage (MV) and low voltage (LV) levels of the distribution network. Since the existing grid was designed for unidirectional electricity flows, there is a limited capacity to integrate intermittent generation sources. Increasing the grid hosting capacity for intermittent RES requires active, real-time, large scale integrated management of RES and other DER, i.e., it requires the distribution network to be progressively transformed into a smart distribution network. The DSOs have to optimize the operation and planning of distribution networks in the presence of increased amounts of DG.

The DERs have the potential of selling services to the power grid, but may not be able to directly do so for two main reasons: (a) their individual capacities are smaller than the required minimum, since the majority of DERs are small-size, household-level DERs; and (b) the large number of small DERs would make their management difficult even if they are allowed to participate [53]. To enable DER integration, new energy players, called aggregators, appeared in the electricity markets. An aggregator, acting toward the grid as one entity, can offer new services to the electricity market and the system operator by aggregating flexible DER involving both demand and generation resources [53]. An aggregator enters into agreements with electricity customers on access to disposing of the electricity customer flexible consumption and/or generation in the electricity market. The thriving of aggregators is facilitated by the digitalization of the economy, the advanced information and communication technologies, and the open ecosystems and platforms.

The flexibility of distributed energy resources comes in a great diversity in terms of temporal and spatial availability as well as technical and economic characteristics, which, when provided with the proper incentives, could be used by the distribution system operator as a tool in fulfilling his main role as the entity responsible for the smooth operation of the distribution network. The various distributed energy resources can offer flexibility services to the distribution system operator through aggregators in the context of local energy markets at power distribution level. This means that the role of the DSO has to be enhanced to an active manager of the distribution grid and its various DERs, which is a significant change, since the traditional role of the DSO is to passively operate, maintain, reinforce, and expand the distribution network [7]. The DSO can be also the operator of the local energy market at the power distribution level [6]. Other market designs propose aggregator to act as market operator of the local energy market [54].

The need for local energy markets has been highlighted by several researchers worldwide [55,56]. In 2001, the work [55] highlighted the need for local energy markets in Poland served mainly by the distributors. In 2006, the work [56] highlighted the need for local energy markets in Denmark to secure the large-scale integration of variable wind power into the energy system. Numerous projects worldwide have already proven the benefits of LEMs, namely increased efficiency of energy use, reduction of carbon emissions, and reduction of electricity cost [57,58]. KEPCO has already established LEMs that allow electricity trading among prosumers [42]. AEP Ohio gridSMART project uses double-auction LEM to optimally schedule participating responsive loads on four distribution feeders [59]. The European Union (EU) PowerMatcher coordination software has been installed in about one thousand houses and industries to integrate numerous producing and consuming devices in various local energy markets [59]. The main conclusions are: (1) scalability beyond one million customers is feasible; and (2) LEMs help significantly reduce peak load and wind power imbalances [59].

The “Clean Energy for All Europeans” package specifies how the EU will meet the 2030 EU climate and energy policy targets for 40% reduction in greenhouse gas emissions (from 1990 levels), 32.5% increase in energy efficiency, and 32% share for RES [60,61]. On 30 November 2016, the EU published a revised electricity market regulation and a revised electricity market directive according to which energy consumers are placed at the heart of the energy system [60]. Energy consumers can use aggregators and energy service providers [60]. Moreover, the increasing role of DSOs is recognized in the revised EU electricity market environment. The possibilities for integration of local energy markets into the EU regulatory framework are investigated in [57].

This paper introduces a taxonomy of models and CI methods for the optimal operation of LEMs in smart distribution networks, offering a unified presentation of a relatively large number of selected high quality research works [10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50]. The analysis of these works has shown that there are three different energy trading mechanisms in the LEMs:

(1)

Full peer-to-peer energy trading, where market players directly trade energy with each other without mediators [62,63].

(2)

Energy trading through a mediator, where a mediator participates in the LEM on behalf of buyers and sellers [38].

(3)

Hybrid energy trading, where buyers and sellers can trade energy either directly or through a mediator [14].

Figure 1 shows a possible future framework of WEM and LEM with mediators, DERs, and microgrids. More specifically, in this LEM, DERs (DGs, ESSs, and DR) and microgrids trade energy with the LEM operator either directly or through a mediator, which means that hybrid energy trading takes place in this LEM that is managed by the LEM operator. The DSO is between the LEM and the WEM. The DSO participates in WEM together with other DSOs, the TSO, large-scale generation companies, retailers, and large consumers.

3. Models for Local Energy Markets

In the reviewed papers [10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50], different models are considered for the studied local energy markets. In the following, the LEMs are classified according to market types, market players, types of distributed energy resources, objectives, and constraints.

3.1. Market Types

The most commonly investigated market types are the following:

(1)

A single LEM. In this case, one LEM is investigated. For example, this LEM can facilitate the energy trading of one or more microgrids, the trading of a local energy community, or the trading of participating consumers and producers. The interaction of LEM with WEM is not investigated. In some works, only the electricity price of the WEM is considered in the optimal scheduling of LEM.

(2)

LEM and WEM. In this case a single LEM is considered in combination with the interaction of LEM with WEM.

(3)

Multiple LEMs. In this case, the optimal operation of multiple LEMs is investigated.

3.2. Market Players

The most commonly investigated local market players are the following:

(1)

Sellers. These are players who sell energy to local consumers of LEM. The sellers are entities who can generate or store energy. Examples of sellers are DGs, prosumers when selling energy, battery storage systems (BSSs) and ESSs when discharging and selling energy to LEM, plug-in hybrid electric vehicles (PHEVs) when discharging, and microgrids when selling energy.

(2)

Buyers. These are players who buy energy from local generating and storage units of LEM. Examples of buyers are consumers (loads), prosumers when buying energy, ESSs and BSSs when charging and buying energy from LEM, PHEVs when charging, microgrids when buying energy, and flexible loads curtailment and adjustment.

(3)

Mediators. These players enter into agreements with electricity customers to make available in the LEM the electricity customer flexible consumption and/or generation. The mediator can also have the role of seller/buyer. In some works, the mediator can have the role of LEM operator. Examples of mediator are aggregator, auctioneer, load serving entity, retail energy broker, and LEM operator.

3.3. Types of Distributed Energy Resources

In the reviewed papers [10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50], three types of DERs are investigated:

(1)

DG units. Examples include PV, WT, CHP, fuel cell (FC), micro-turbine (MT), thermal unit (TU), hydro unit, and ground source heat pump (GSHP).

(2)

ESS units. Examples include BSS, ESS, EV, and PHEV.

(3)

DR. Examples include flexible loads curtailment and adjustment.

There are two cases depending on the number of different DERs that are investigated:

(1)

A single DER. In this case, only a single DER is investigated. Examples are [21] and [47], where the impact on LEM of EV as a single DER is investigated.

(2)

Multiple DERs. The majority of reviewed papers investigates the impact on LEM of multiple DERs, e.g., DR, FC, MT, PV, and WT [11].

3.4. Objectives

The most common objective functions are the following:

(1)

Maximization of the social welfare. The objective is to maximize the total welfare of all the players of LEM, by maximizing the profit of each local market player.

(2)

Minimization of the total operation cost. The objective is to minimize the sum of the energy cost and the system operation cost for all the local market players.

There are three cases depending on the number of objectives that are investigated and the form of the objective function:

(1)

Single objective. The most common single objective is the minimization of the total operation cost [10,11,13,15,18,23,24,25,26,27,28,30,31,34,46].

(2)

Multiple objectives with weights [19,20,32,44]. In this case, the user defines fixed weights for the individual objectives, and as a result, the objective function that is weighted sum of multiple objectives reduces to a single objective function.

(3)

Multiple objectives [12,17,33,38,39,40,41,47]. In this case, there are multiple objectives, and the proposed optimization method first computes the Pareto-optimal solutions, and, next, it computes the best-compromise solution among the set of Pareto optimal solutions.

3.5. Power Flow Constraints

Out of 41 reviewed papers, only 14 papers consider the alternating current (AC) power flow constraints at their optimization models [12,13,15,17,29,30,31,34,35,38,40,41,43,46].

3.6. Taxonomies

Table 1. Taxonomy of the reviewed energy markets.

Reference	Market Type	Local Players	Types of DERs
[10]	WEM	Sellers: microgrids	BSS, PV, TU, WT
		Buyers: loads
[11]	LEM	Sellers: DGs	DR, FC, MT, PV, WT
		Buyers: loads
		Mediators: Auctioneer, DERs agent
[12]	Pool and hybrid	Sellers: microgrids, DGs	BSS, PV, WT
		Buyers: loads
[13]	−	−	DR, ESS, FC, PV, WT
[14]	LEM	Sellers: microgrids, DGs	DG, DR
		Buyers: loads
		Mediators: microgrids agents
[15]	LEM	Sellers: PHEVs discharging, DGs	CHP, EV, FC, Hydro, PV, WT
		Buyers: PHEVs charging, loads
[16]	LEM	Sellers: PHEVs discharging	ESS, EV, TU
		Buyers: PHEVs charging, loads
		Mediator: EVs aggregator
[17]	WEM + LEM	Sellers: DGs, load curtailment by DR aggregators	DG, DR, WT
		Buyers: DR aggregators, loads
		Mediators: DR aggregators
[18]	LEM	Sellers: DGs	ESS, FC, MT, PV, WT
		Buyers: loads
[19]	LEM	Sellers: DGs	BSS, PV
		Buyers: loads
[20]	LEM	Sellers: DGs	BSS, PV
		Buyers: loads
[21]	LEM	Buyers: PHEVs charging, loads	EV
		Mediators: EV aggregators
[22]	WEM+LEM	Sellers: DR aggregator	DR
		Buyers: DR aggregator, loads
		Mediator: DR aggregator
[23]	WEM+LEM	Sellers: microgrids	DER
		Buyers: microgrids, loads
		Mediators: microgrids agents
[24]	WEM+LEM	Sellers: microgrid	ESS, MT, PV, WT
		Buyers: microgrid, loads
		Mediator: Microgrid operator
[25]	WEM+LEM	Sellers: BSSs discharging, PVs	BSS, PV
		Buyers: BSSs charging, loads
		Mediators: home agents, aggregator
[26]	LEM	Sellers: microgrids	CHP, FC, MT, PV, WT
		Buyers: microgrids
		Mediators: microgrids agents
[27]	LEM	Sellers: microgrids	BSS, DG, PV, WT
		Buyers: microgrids
		Mediators: microgrids agents
[28]	LEM	Sellers: BSSs discharging, DGs	BSS, FC, PV, WT
		Buyers: BSSs charging, loads
		Mediator: energy manager
[29]	LEM	Sellers: BSSs discharging, DGs	BSS, PV, WT
		Buyers: BSSs charging, loads
		Mediator: DSO
[30]	LEM for IES	Sellers: DGs	CHP, GB, GSHP, PV, WT
		Buyers: loads
[31]	WEM+LEM	Sellers: DGs	DG
		Buyers: loads
		Mediator: DSO
[32]	LEM	Sellers: microgrids	DG, DR, EV
		Buyers: microgrids
		Mediators: microgrids agents
[33]	LEM	Buyers: EVs charging, flexible loads	DR, EV
		Mediators: DR aggregators
[34]	LEM	Sellers: microgrids	BSS, MT, PV, WT
		Buyers: microgrids
		Mediators: microgrids aggregators
[35]	LEM	Sellers: DR aggregators	DR, PV
		Buyers: DR aggregators
		Mediators: DR aggregators
[36]	LEM	Sellers: microgrids	PV, WT
		Buyers: microgrids
		Mediators: auctioneer, microgrids managers
[37]	LEM	Sellers: DR aggregators	DR
		Buyers: DR aggregators
		Mediators: DSO, DR aggregators
[38]	WEM+LEM	Mediators: load serving entities, load aggregators	DR
[39]	LEM	Mediators: DSO, load aggregator	DR, PV
[40]	LEM	Sellers: BSSs discharging, DGs	BSS, CHP, PV, WT
		Buyers: BSSs charging, loads
[41]	LEM	Sellers: microgrids	DG, DR, ESS, PV
		Buyers: microgrids
[42]	Multiple LEMs	Sellers: prosumers, DGs	PV, WT
		Buyers: prosumers
		Mediator: auctioneer
[43]	LEM	Sellers: ESSs discharging, PVs	ESS, PV
		Buyers: ESSs charging, loads
		Mediator: aggregator
[44]	LEM	Seller: microgrid	DG, DR, ESS, PV
		Buyer: microgrid
[45]	WEM	Seller: PV	ESS, PV
[46]	LEM	Sellers: microgrids	DG, BSS, PV, WT
		Buyers: microgrids
[47]	LEM	Mediator: EVs operator	EV
[48]	WEM+LEM	Mediators: DR provider, DR aggregators	DR, ESS, PV
[49]	LEM	Sellers: DGs, prosumers	BSS, DG, PV, WT
		Buyers: loads, prosumers
[50]	LEM	Sellers: DGs, prosumers	PV, WT
		Buyers: loads, prosumers
		Mediator: retail energy broker

and

Table 2. Taxonomy of the reviewed optimization problems.

Reference	Objective	Objective Function	Power Flow Constraints
[10]	Single	Minimization of the operation cost of DERs of microgrids	No
[11]	Single	Minimization of the operation cost of DERs	No
[12]	Multiple	Maximization of the benefits of distribution system operator, microgrid, and distributed generation units	Yes
[13]	Single	Minimization of the operation cost of DERs and the load curtailment cost	Yes
[14]	Single	Finding of the optimal period to serve the loads that participate in DR	No
[15]	Single	Minimization of a cost function including the operation cost of DERs, the cost to buy energy from the pool market, and the cost of non-supplied energy	Yes
[16]	Single	Maximization of the profit of EVs aggregator participating in the reserve services market of power distribution grid	No
[17]	Multiple	Minimization of: (a) total expected cost of distribution grid, and (b) total emissions during the scheduling period	Yes
[18]	Single	Minimization of the sum of total operation and outage cost	No
[19]	Multiple	Multi-objective with weights	No
[20]	Multiple	Multi-objective with weights	No
[21]	Single	Each EV aggregator optimally allocates to each one of his PHEVs the total charging power dictated by the DSO	No
[22]	Single	Maximization of the profit of DR aggregator	No
[23]	Single	Minimization of microgrids total power transaction cost	No
[24]	Single	Minimization of the operation cost of DERs of microgrid	No
[25]	Single	Minimization of the total electricity cost of smart homes	No
[26]	Single	Minimization of the sum of generation cost by microgrids and pollution emission cost	No
[27]	Single	Minimization of operation cost of AC and DC microgrids	No
[28]	Single	Minimization of operation cost of DC residential distribution system	No
[29]	Single	Maximization of day ahead profit of DSO	Yes
[30]	Single	Minimization of the expected operation cost of the integrated power, heat, and gas local systems	Yes
[31]	Single	Minimization of operation cost of distribution network	Yes
[32]	Multiple	Multi-objective with weights	No
[33]	Multiple	Minimization of peak demand and maximization of DSO total revenue	No
[34]	Single	Minimization of the operation cost for DSO	Yes
[35]	Single	Maximization of the profit for each DR aggregator	Yes
[36]	Single	Maximization of the sum of consumers surplus and producers surplus	No
[37]	Single	Maximization of the profit of DR aggregators	No
[38]	Multiple	During peak hours, minimization of economic loss of the load serving entity and maximization of the reward of customers	Yes
[39]	Multiple	Maximization of the economic profit for interrupting industrial load, and maximization of user satisfaction for interrupting industrial load	No
[40]	Multiple	Minimization of lines congestion and minimization of network operation cost	Yes
[41]	Multiple	Minimization of expected power deficiency in case of microgrid islanding or in case of unit failure in a microgrid; minimization of load reduction; minimization of congestion	Yes
[42]	Single	Minimization of the sum of the geometric distances between local market players (prosumers and DGs) and local power trading centers	No
[43]	Single	Minimization of the weighted sum of investment, operation and maintenance cost of ESSs, plus the energy transaction cost for the lifecycle of ESSs	Yes
[44]	Multiple	Multi-objective with weights	No
[45]	Single	Maximization of benefit-cost ratio for ESS in PV power trading	No
[46]	Single	Minimization of total operating cost of microgrids	Yes
[47]	Multiple	Maximization of aggregator average profit and minimization of aggregator deviation profit	No
[48]	Single	Maximization of the profit of DR aggregators	No
[49]	Single	Maximization of the utility for the buyer and the seller of a bilateral transaction	No
[50]	Single	Maximization of the profit of retail energy broker	No

classify the reviewed energy markets and optimization problems, respectively.

4. Computational Intelligence Methods for Local Electricity Markets

The computation intelligence methods used are presented in the following subsections in alphabetical order.

Table 3. Taxonomy of the computational intelligence methods of the reviewed works.

Computational Intelligence Method	Reference
Ant Colony Optimization	[24]
Approximate Q-Learning	[35]
Artificial Bee Colony	[32]
Artificial Neural Network	[38]
Differential Evolution	[37]
Expert System	[28]
Fuzzy Sets	[17,27,44,47]
Genetic Algorithm	[16,22,25,34,36,39,43]
Geometric Clustering with Unsupervised Learning	[42]
Heuristic Optimization	[21]
Imperialist Competitive Algorithm	[18]
Multi Agent Systems	[10,14,23,41,48,49]
Particle Swarm Optimization	[11,12,13,15,26,29,30,33,37,40,46]
Reinforcement Learning	[50]
Scatter Search	[31]
Simulated Annealing	[19]
Support Vector Machines	[45]

classifies the computation intelligence methods of the reviewed works.

4.1. Ant Colony Optimization

Ant colony optimization (ACO) was introduced in the early 1990s as a novel nature-inspired algorithm to solve difficult combinatorial optimization problems [64,65]. ACO belongs to metaheuristic optimization methods that are approximate techniques applied to find good enough solutions to hard combinatorial optimization problems in a reasonable computational time. The nature inspiration of ACO is the food searching of real ants. Ants initially search randomly for food in the area around their nest. When an ant finds food, it carries some food back to the nest, and during the return trip, the ant deposits on the ground a quantity of pheromone trail that depends on the quantity and quality of found food. Pheromone trails help ants find the shortest paths between food sources and their nest. This property of real ants is exploited by ACO metaheuristic to solve complex combinatorial optimization problems [64,65].

In [24], a local energy market is combined with a wholesale electricity market. Each distributed generation unit submits energy offers to the microgrid operator, and load representatives submit load declarations for each trading interval. Then, single-sided auction is applied to find the market clearing price in every time interval. The proposed local energy market allows microgrid operator to maximize profits by optimally scheduling generation and loads. The objective of the optimization problem is to minimize the electricity production cost by optimally operating distributed energy resources using hourly day ahead scheduling as well as real time scheduling. Multi-layer ant colony optimization is proposed to solve this optimization problem. Simulation results show that in comparison with particle swarm optimization, the proposed ant colony optimization method finds an optimal solution 24 times faster at a 2.9% lower total generation cost.

4.2. Approximate Q-Learning

Reinforcement learning (RL) is a CI-based method where the learner (agent) must learn, without a supervisor, but through trial and error interactions with a dynamically changing environment. One of the classical RL algorithms is the basic Q-learning algorithm that has the disadvantage of discretizing the infinite state (search) space that implies a high computational time for learning. On the other hand, the approximate Q-learning solves this problem by using features instead of states. The feature-based approximate Q-learning algorithm provides better results, better adaptation to new environments, and reduced learning time [66].

In [35], a local energy market is considered at the low voltage power distribution level. Demand response aggregators are responsible for scheduling controllable loads. If a demand response schedule causes voltage and/or line congestion problems, the distribution system operator rejects the demand response schedule and penalizes the demand response aggregator. In the considered local energy market, the objective of each demand response aggregator is to maximize his profit and to avoid the rejection of part of his demand response schedule that implies penalties. Approximate Q-learning is proposed to optimize the scheduling of each demand response aggregator. Simulation results on a real low voltage distribution network with seven PV plants, 70 households out of which 12 households participate in demand response through four aggregators, show that the proposed approximate Q-learning method is an efficient demand response scheduling method thanks to its capability to learn to avoid paying penalties by avoiding demand response schedule rejection. Results show that the economic DR schedules are rejected by almost 20%, while the DR schedules of the proposed method are rejected at most by 0.4%.

4.3. Artificial Bee Colony

The artificial bee colony (ABC) is an evolutionary, nature-inspired, metaheuristic algorithm that solves complex optimization problems by simulating the food search of bee swarms [67,68]. The population of artificial bees is split into worker bees and non-worker bees (scouts and onlookers). The total number of food areas (possible solutions of the optimization problem) is equal to the number of employed bees. The employed bees search for food and give information to the onlookers for the quality of food. The objective of onlooker bees is twofold: to find good food areas and further search for food around the selected food areas. Those worker bees leaving non good food areas become scout bees.

In [32], a local energy market is developed to facilitate power trading among microgrids. Each microgrid has distributed generation units, electric vehicles, and elastic loads. A multi agent based transactive energy system is proposed in order to simultaneously optimize three objectives: (1) minimize energy imbalances in microgrids, (2) minimize the charging cost of electric vehicles and the operating cost of elastic loads participating in demand response, and (3) maximize the profit of electric vehicles from discharging while considering life cycle degradation cost. The formulated optimization problem is solved using artificial bee colony method. The optimization model determines, for each microgrid, the optimal scheduling of distributed generation units, electric vehicles, and elastic loads. The method is applied on the modified IEEE 37-node power distribution system that has three microgrids. Simulation results show that the proposed transactive energy market reduces the operating cost of elastic loads, reduces the reliance of microgrids on the utility grid, and reduces by 13.6% the total operating cost of microgrids thanks to the increased participation of demand response, electric vehicles, and distributed generation units.

4.4. Artificial Neural Network

An artificial neural network is a biologically inspired technology that mimics the computations and interactions of neurons within the human brain [69,70]. ANN is widely accepted as a very efficient mathematical tool for solving highly complex, nonlinear, and ill-defined, forecasting and classification problems. ANN can learn from examples. ANN is robust and fault tolerant, because it can successfully handle incomplete and noisy data. Once trained, ANN can perform, at high computational speed, generalization, i.e., classification and forecasting for unknown input data that was not used during ANN learning (training) phase.

In [38], customers are rewarded by coupons to reduce their demand during peak hours, and this is known as customer coupon demand response (CCDR). A local energy market is considered that serves a meshed secondary distribution network and the market mediators are several load serving entities (LSEs), each one having multiple load aggregators who represent load customers. The objectives of CCDR, which is executed during peak hours, are the minimization of economic loss of the LSE and the maximization of the reward of customers. Artificial neural network is proposed to estimate the new locational marginal price (LMP) for a load serving entity after implementing CCDR. The feasibility of CCDR is investigated for a real heavily-meshed secondary distribution network with 1905 buses, and 97.9 MW peak demand. Three load aggregators and eight LSEs are considered, out of which one LSE participates in CCDR. Simulation results show that CCDR helps reduce peak demand, and reduce the economic loss during peak hours, which provides 5.5% higher profits for the entire day for the participating LSE in comparison to the net profits without CCDR.

4.5. Differential Evolution

Differential evolution (DE) is a versatile and reliable method for solving large scale, constrained, multi-objective, uncertain, and complex optimization problems [71,72]. DE is a nature-inspired metaheuristic optimization method. DE belongs in the category of evolutionary algorithms, so DE is a population-based optimization algorithm. Other evolutionary algorithms recombine the solutions of a population using probabilistic rules. On the contrary, DE generates new solutions by modifying the solutions with the scaled differences of distinct and randomly selected population members.

In [37], two different optimization methods, namely differential evolution and particle swarm optimization are investigated and compared for maximizing the profit of demand response aggregators who compete to sell demand response services to the distribution system operator and offer reward to the end-users loads for changing their load profiles. A system with 60 end-users equally allocated among the available three aggregators is simulated and the results in show that differential evolution outperforms particle swarm optimization, since it provides better results for all market players, i.e., better reward to end-users, higher profit to aggregators, and 2.9% lower operational cost to the distribution system operator.

4.6. Expert System

An expert system (ES) is a powerful computer software that uses computational intelligence methodologies to simulate the decision of a human who has expertise in a specific domain [73,74]. An ES contains a knowledge base and a set of rules. The knowledge base contains the accumulated experience of the human expert. The set of rules is information in the form of if-then rules that helps apply the knowledge base to input data in order the ES to come to the right conclusion and make the appropriate decision. The capabilities of the ES can be increased by adding new information in the knowledge base or by adding new rules in the set of rules.

In [28], a local energy market is designed to minimize the total operation cost of a DC residential distribution system that is composed of homes that, in addition to loads, may have battery storage system, fuel cell, photovoltaics, and wind turbine. An optimization methodology is proposed that minimizes the total operating cost and controls the DC bus voltages to be within the specified limits. Adaptive direct current droop control is applied to regulate voltage and align with the schedule of distributed generation units. The optimization problem is solved by an expert system in combination with sequential quadratic programming approach. Simulation results on a 5-bus DC residential distribution system show that the total cost is reduced by optimally scheduling the distributed energy resources, and the DC voltage in all buses is within the allowable range thanks to adaptive direct current droop control.

4.7. Fuzzy Sets

Fuzzy sets, also known as uncertain sets, are sets whose elements have degrees of membership ranging from zero to one [75,76]. In classical set theory, the membership of each element in a set is binary, i.e., an element either does not belong to the set and it takes the binary value zero or belongs to the set and takes the binary value one. On the other hand, in fuzzy set theory, the membership of each element in a set is a real value, ranging from zero to one. The fuzzy set theory is successfully applied in a wide range of domains including pattern recognition (classification) and information processing, in which domains, information is fuzzy or incomplete, which means that sharply defined criteria are absent.

In [17], a local energy market at power distribution level and a wholesale electricity market are considered. In the local energy market, the participants are distribution system operator, distributed generators (wind and diesel generators), demand response aggregators, and large consumers. The uncertainties of wind power output and load demand are included in the scheduling of the local energy market. A stochastic multi-objective method, based on augmented ε-constraint technique, is proposed for the optimal scheduling of the energy and reserve local energy market that minimizes two conflicting objectives: a) total expected cost of distribution grid, and b) total emissions during the scheduling period. Benders decomposition is used to solve the proposed optimization problem. The stochastic multi-objective method computes the Pareto-optimal solutions. Next, fuzzy sets theory and fuzzy decision is proposed to compute the best-compromise non-dominated solution among the set of Pareto optimal solutions. Simulation results on a 41-bus power distribution network show that demand response aggregators enable the effective participation of small and medium consumers in DR programs. More specifically, DR reduces energy consumption during hours with high electricity prices, and thus DC contributes in the reduction of the operational costs. Moreover, since DR helps reduce the power from diesel generators and the power imports from the transmission grid, particularly in hours with high emission rates, emissions are reduced thanks to DR participation. In comparison to MINLP, the proposed Benders decomposition method computes the same optimal solution 155 times faster.

In [27], a local energy market is designed in order to facilitate the electricity trading between an AC and a DC microgrid so as to minimize the operation cost of these two microgrids. A methodology is developed to increase the resilience of microgrids. Fuzzy logic is proposed for the consideration of the uncertain nature of disturbance events, renewable generation, and market price. A fuzzy logic controller is proposed for the optimal operation of a battery storage system. A new mode, namely, the resilient mode of microgrids is proposed to ensure the survival of critical loads during disturbances. Two operation modes, namely, subservient and resilient, are proposed for battery storage systems to minimize operating cost and prepare microgrids for attainable islanding. Simulations have been done for normal and emergency operation mode, comparing the conventional approach (BSS is controlled by an energy management system) with the proposed approach (BSS is controlled by the proposed fuzzy logic controller). In case of normal operation mode, due to the inclusion of resilient mode, the operation cost is increased by 0.4%. In case of emergency operation mode, load shedding is reduced by 92% and operation cost is reduced by 6.3% in comparison with a conventional method.

In [44], a local energy market is formed with the objective to optimize the daily performance of a microgrid in the presence of customer demand response. The microgid has diesel generators as well as with turbine and photovoltaic units. The microgrid serves customers who can accept to participate in incentive-based demand response program, so customers can accept the interruption of part of their load. A multi-objective mathematical programming model is developed that optimizes the scheduling of microgrid, considering the uncertainties of market price, load, and PV and WT output power. Fuzzy satisfying method is proposed to solve this multi-objective optimization problem and to find the best solution from Pareto front optimal solutions. The proposed method is tested on two different microgrids. Simulation results show that the proposed method is beneficial for both the microgrid and the customers who participate in demand response. More specifically, in a small microgrid with one WT, one PV, three diesel generators, and three customers, the total daily incentive received by three customers to partially reduce their load ranges from $344.44 to $377.63 for the five different daily load profiles considered.

In [47], scheduling of an electric vehicles aggregator is studied considering the uncertainty of grid electricity prices. An interval optimization approach models grid price uncertainty, computes the upper and lower bounds of aggregator profit, and next computes the average profit and the deviation profit of the aggregator. Next, a multi-objective optimization problem is formulated with two conflicting objectives: maximization of aggregator average profit and minimization of aggregator deviation profit. The multi-objective optimization problem is solved by ε-constraint method that computes Pareto optimal solutions. Next, fuzzy satisfying method is proposed to find the best solution from Pareto optimal solutions. Simulation results show that the proposed methodology is suitable for the aggregator to efficiently schedule the charging and discharging of his electric vehicles. Moreover, the consideration of uncertainty reduces the profit of EV aggregators by 2.94% in comparison to the deterministic approach of ignoring uncertainty.

4.8. Genetic Algorithm

Genetic algorithms are general purpose, robust and practical optimization and search techniques [77,78]. GA is nature-inspired method, based on the mechanisms of natural genetics and evolution. GA is inspired from the process of natural selection that leads to the survival of the fittest members. GA belongs in the category of evolutionary algorithms, so GA is a population-based optimization algorithm. GA uses probabilistic search mechanisms to find the best solutions from a population of candidate solutions. GA uses crossover mechanism to exchange information between existing solutions to find even better solutions.

In [10], a hybrid genetic algorithm and Lagrangian relaxation method is used for the solution of the unit commitment problem of thermal units of multiple microgrids, which is part of an intelligent multi agent system capable of scheduling microgrids with multiple distributed energy resources: photovoltaics, battery storage systems, wind turbines, and thermal units.

In [16], genetic algorithm is proposed for the maximization of the profit of EVs aggregator participating in the reserve services market of power distribution network. Two electricity markets are considered: a local energy market at the power distribution level and a wholesale electricity market. Both markets have energy and reserve markets. The energy and reserve local energy markets are formed before the respective wholesale electricity markets. Simulation results indicate that there are gains for all the participants in the local energy market. More specifically, the distribution system operator reduces his network operating cost, the EVs aggregator makes profit from his participation in the reserves local energy market, and the EV owners reduce their cost to charge their EVs because they make profit when they discharge their EVs.

In [22], each customer under agreement with a demand response aggregator has a set of schedulable smart appliances. A demand response aggregator interacts with the local energy market and intelligently schedules customer smart appliances away from the peak load time, in order to reduce the peak load and offer an economic gain to the participating customers and the aggregator. To encourage customer participation in demand response and to compensate customers’ inconvenience of the aggregator controlling their smart appliances, a new pricing scheme called customer incentive pricing is introduced. A genetic algorithm is proposed for the day-ahead scheduling of the smart appliances and the determination of customer incentive pricing. The methodology is validated using real pricing data over a 24-h period as well as a large-scale system composed of 5555 residential customers with 56,642 schedulable smart appliances, which consume 11.2% of the total energy of these 5555 customers, while the rest energy is consumed by their non-schedulable loads. The genetic algorithm needs 113 min to solve this large-scale optimization problem. The results show that on a 24-h period, the profits for the aggregator are $947.90, and the average saving per customer is $0.14 ranging between $0.02 and $0.33. Moreover, thanks to demand response, the peak load of the 5555 participating customers is reduced by 12.5%. The customer incentive pricing is generally lower than the real time pricing the customer would pay otherwise.

In [25], a coordinated energy management strategy is proposed for smart homes in a neighborhood. The smart homes can have photovoltaics, energy storage systems, and controllable loads. A local energy market is developed that facilitates the trading among the smart homes. The players of the local energy market are the aggregator and the home agents, where each smart home has each own agent. An incentive is used to increase the interest of smart homes to participate in the local energy market. The objective of the coordinated energy management strategy is to minimize the electricity bills of smart homes by optimally scheduling the distributed energy resources of smart homes. Genetic algorithm is used to solve this optimization problem. Simulation results on a neighborhood with ten smart homes show that the proposed method offers benefits to the neighborhood as well as to the smart homes. With the proposed method, all smart homes with PV and BSS reduce their electricity bill. The purchased electricity from the utility grid is reduced and the PV energy usage in the neighborhood is increased. Smart homes with PV and ESS sell more energy in comparison with smart homes having only PV. In comparison to baseline operation, the proposed coordinated energy management strategy achieves 30.9% reduction in the total electricity cost of smart homes.

In [34], a local energy market is operated by the distribution system operator. The market players are multiple microgrids aggregators, where each microgrid includes loads, battery storage systems, microturbines, photovoltaics, and wind turbines. Game vectors are proposed to capture the spatio-temporal dynamic pricing behavior of each microgrid when trading in the local energy market. A two-level programming model is proposed for the operation of this local energy market. In the upper level, an optimization problem is formulated that seeks to optimize the interests of distribution system operator, i.e., minimize operation cost, minimize power loss, optimize voltage profile, and maximize his profit. In the lower level, an optimization problem is formulated that seeks to optimize the interests of each microgrid, i.e., minimize operation cost, minimize un-served load, and maximize the profit of each microgrid. A customized hierarchical genetic algorithm is proposed to solve this two-level programming problem. Simulation results on the IEEE 33-bus distribution test feeder with three microgrids show that, in comparison with the traditional centralized dispatch strategy, the proposed dynamic pricing mechanism increases the profit of DSO by 1.7% and decreases the total cost of microgrids by 6.3%.

In [36], a combinatorial double auction mechanism is introduced for energy trading among multiple microgrids in a local energy market managed by an auctioneer. According to their needs, the microgrids can buy or sell energy. The objective of the combinatorial double auction is to maximize total social welfare, which is the sum of consumers’ surplus and producers’ surplus. A hybrid method combining genetic algorithm and particle swarm optimization is proposed to find the winner of the combinatorial double auction. Simulation results show that in a system with 35 microgrids, the social welfare of the proposed combinatorial double auction is 11.4% higher than the social welfare of the traditional double auction.

In [39], incentive-based demand response is investigated for industrial users. Load aggregator manages and optimally schedules the interruptible loads, the air conditioning, and the photovoltaic systems of industrial users. The optimal scheduling of load aggregator has two objectives: maximization of the economic profit for interrupting industrial load, and maximization of user satisfaction for interrupting industrial load. This multi-objective optimization problem is solved by non-dominated sorting genetic algorithm-II. Moreover, chance constraint programming is proposed for maximizing the profits of load aggregator considering uncertainties of demand response. The method is tested for a real industrial park with five industries: one chemical fibre, one cast steel, and three food industries. The simulation results show that the proposed multi-objective optimization model effectively coordinates user satisfaction and economic profit from load interruption. More specifically, the total profits of the factories are increased by 57.4% and the total profits of the aggregator are increased by 61.4%.

In [43], a local energy market is considered for energy trading among an aggregator and customers in a rural power distribution network. The aggregator owns energy storage systems, while some of the customers own PV units. A mathematical optimization model is introduced that computes the optimal location and sizing of energy storage systems in order to minimize the weighted sum of investment, operation and maintenance cost of ESSs, plus the energy transaction cost for the lifecycle of ESSs. This optimal allocation and scheduling of ESSs has to maintain a desired level of reliability, expressed by system average interruption duration index. Genetic algorithm is proposed to solve this complex, nonlinear, and non-convex optimization problem. The method is tested on a real 109-bus rural distribution network. Simulation results show that the optimum ESSs location and sizing significantly reduces energy purchase costs, while simultaneously meets the reliability objective of system average interruption duration index (SAIDI) target. More specifically, in comparison with no PV and storage, the proposed optimum ESSs in combination with 40% PV penetration, although it has an investment cost of $293,540, it is a profitable solution because it reduces the energy transaction cost by $598,109 in five years, and additionally it improves power system reliability because it reduces SAIDI by 17.8%.

4.9. Geometric Clustering with Unsupervised Learning

The expectation and maximization (EM) is a learning algorithm that can solve non-deterministic polynomial-time hard (NP-hard) geometric clustering problems in polynomial time [79]. The EM method is an iterative algorithm to compute the maximum likelihood estimates of parameters. The EM iteration includes two steps: the expectation step and the maximization step. The expectation step calculates of expectation of the log-likelihood computed using the current estimates of parameters. The maximization calculates parameters maximizing the expected log-likelihood computed on the expectation step.

In [42], several local energy markets are operating at power distribution level. For a given number of local energy markets, a clustering problem is defined as follows: for each local energy market, the optimal location of local power trading center and the served prosumers have to be computed. This clustering problem is non-deterministic polynomial-time hard (NP-hard), and expectation and maximization based geometric clustering with unsupervised learning is proposed to solve it. Moreover, a truthful auction method is proposed to compute the winning bids that minimize the electricity cost in the local energy markets. Simulation results show that the proposed geometric clustering with unsupervised learning finds in polynomial time the optimal locations of local power trading centers, which means that the proposed method is efficient for clustering real-world distribution systems in a number of local energy markets. More specifically, in case of 15 sellers and 5 buyers, the proposed auction method is 769 times faster than the brute force auction method.

4.10. Heuristic Optimization

Heuristic optimization is a methodology to solve an optimization problem faster when classical optimization methods are too slow, or to find an approximate solution when classical methods fail to find any solution. This is achieved by compromising optimal solution for speed. A metaheuristic is a higher-level heuristic optimization method that can provide good enough solution to a complex and large-scale optimization problem, in less computational time in comparison with simple heuristic optimization methods. Heuristic methods are developed to solve specific problems using the known characteristics of each specific problem. On the other hand, metaheuristic methods, which are composed of general rules, can solve various optimization problems [80].

The work [21] proposes the coordination between distribution system operation and charging of plug-in electric vehicles across multiple aggregators. This coordination is achieved in three steps. In the first step, the charging load boundaries of each aggregator are computed, considering charging needs of each one PHEV that is controlled by this aggregator. In the second step, the distribution system operator computes the dictated charging curve for each aggregator. In the third step, each electric vehicle aggregator optimally allocates to each one of his electric vehicles the total charging power dictated by the distribution system operator, using a heuristic optimization method. Simulation results on four aggregators following different electric vehicle charging strategies indicate the importance of the coordination between distribution system operator and the four aggregators. More specifically, this charging coordination achieves electricity cost minimization and system peak load reduction. In comparison to the uncoordinated strategy, the proposed charging coordination provides three times more profits for the aggregators.

4.11. Imperialist Competitive Algorithm

The imperialist competitive algorithm (ICA) is a computational intelligence method that is used to solve different types of optimization problems [81,82]. ICA belongs in the category of evolutionary algorithms, so ICA is a population-based optimization algorithm. ICA is the computer simulation of human social evolution, while GA is the computer simulation of species biological evolution. The first step of ICA includes the random creation of an initial population (countries). The best countries with the best value of the objective function are imperialists, while the rest countries form the colonies of the colonies of the imperialists. An empire includes an imperialist and some colonies. The iterative process of ICA includes competition among the empires. The convergence (stopping) criterion is when only one empire remains.

In [18], a local energy market at the distribution level is considered. In this market, the participants are distribution system operator, distributed generators (energy storage system, fuel cell, micro turbine, phototoltaics, and wind generators), and loads. The method is designed to facilitate the energy trading in microgrids. Artificial neural network method is used to forecast the day-ahead hourly power output of photovoltaics and wind turbines. The objective is to find the optimal schedule of all distributed energy resources so as to minimize the sum of total operation and probabilistic outage cost. In order to compute the probabilistic outage cost, a two-state Markov model is used to consider the state of each component of the system (each component works properly or not), and 3000 Monte Carlo simulations are considered. Imperialist competitive algorithm is proposed to solve the microgrid scheduling problem in order to optimally coordinate the output power of distributed generations, the consumption, the energy trading between the microgrid and the main grid, and the risk of total or partial blackout. Simulation results show the importance of considering the probabilistic outage cost in the objective function, since the load outage cost becomes lower in comparison with the case of excluding the outage cost from the objective function.

4.12. Multi Agent Systems

A multi agent system is a system composed of intelligent agents that communicate each other [83,84]. An intelligent agent is a software or hardware that is within an environment and can autonomously and flexibly react to modifications in this environment. MAS is a systems integration approach to build flexible, robust, and extendable systems. Moreover, MAS is a modeling method, since MAS can represent real situations of interacting entities, and can help control how complex behaviors may take place.

The work [10] introduces an intelligent multi agent system for the scheduling of microgrids in an electric power distribution system. The microgrids have multiple distributed energy resources: battery storage systems, wind turbines, photovoltaics, and thermal units. The scheduling is implemented in three steps. In the first step, each microgrid is scheduled individually to meet its local demand. In the second step, the optimum bids of each microgrid are computed to sell power in a wholesale electricity market. In the third step, each microgrid is rescheduled to meet its demand plus the demand requested by the wholesale electricity market. The results on a power distribution system with three microgrids show that the proposed multi agent system effectively schedules the distributed energy resources of the microgrids.

In [14], an intelligent multi agent system is proposed to optimize trading among microgrids and participation of loads in demand response. A local energy market is considered that facilitates the power trading within each microgrid as well as the power trading among the microgrids and the power distribution system. Simulation results in a power distribution system with two interconnected microgrids show that the proposed MAS successfully reduces the peak load. Another conclusion is that the loads with high priority index buy power at reduced cost.

In [23], a local energy marker is considered that works in cooperation with the wholesale electricity market. The local energy market facilitates energy trading among microgrids in order to minimize microgrids total power transaction cost, which is the sum of power purchase cost, power loss cost, communication cost, and compensation cost for load shedding. Each microgrid is associated with an agent, whose role is to perform microgrid energy management. A multi agent system methodology in combination with coalitional game strategy is proposed for economic power transaction among multiple microgrids. The proposed methodology enables microgrids to form coalitions and to exchange power among them, instead of the wholesale electricity market, when this coalition helps reduce microgrids total power transaction cost. Simulation results on systems with 10 to 50 microgrids highlight the economic benefits of the proposed approach. More specifically, in comparison to the non-cooperative method, the proposed cooperative strategy significantly reduces microgrids total power transaction cost, because of the direct trading within microgrids, which requires lower cost in comparison to the power transaction with the utility grid. Another conclusion is that as the number of microgrids increases, the reduction rate of microgrids total power transaction cost also increases.

In [41], a local energy market is designed for power distribution systems composed of multiple microgrids. The objective of the local energy market is to improve the resilience of power distribution system in case of microgrid islanding or in case of unit failure in a microgrid. A resiliency-aware power management approach is proposed and a mathematical optimization model is developed with multiple conflicting objectives: minimization of expected power deficiency in case of microgrid islanding or in case of unit failure in a microgrid (this is a probabilistic objective); minimization of load reduction; and minimization of congestion. The proposed model considers the uncertainty in the estimation of expected extreme events (microgrid islanding) and unit failures. A multi agent system approach is proposed to find the optimal resource allocation of multiple microgrids in order to improve the resilience of power distribution system. More specifically, cooperative agents within microgrids negotiate to reach a Pareto optimal solution. The proposed method is tested on a power distribution system composed of three microgrids, with each microgrid being a different version of IEEE 13-bus power distribution system. Simulation results show that the proposed method reduces power deficit during expected extreme events and improves the resilience of the system, while the resiliency optimization reduces the total profit by 9.3% in comparison to the base case that optimizes the short-term profit.

In [48], a local energy market is designed to facilitate demand response of residential and industrial consumers and to provide flexibility to deal with the intermittency and volatility of renewable energy sources. Participating residential customers are represented by residential DR aggregator. Participating industrial customers, in particular cement plants and metal smelting factors that are heavy energy-intensive customers, are represented by industrial DR aggregator. A mathematical optimization problem is developed with the objective to maximize the profit of DR aggregators. Multi agent system framework is proposed to coordinate the bidding strategies of DR aggregators to maximize their profit. The method is tested on a system where the industrial DR aggregator represents eight cement plants with total demand of 184 MW and two aluminum smelting plants with total demand of 43 MW, and the residential DR aggregator represents residential customers with total demand of 10 MW. Simulation results show that the proposed method can provide flexibility for integration of increased shares of renewable power and can reduce the energy cost of consumers. Moreover, the average energy cost reduction is 34% for cement industry, 18% for aluminum industry, and 12% for residential customers.

In [49], a bilateral local energy market is proposed at power distribution level to help interested market participants maximize their utility from their participation to this market. This bilateral market, which is for near real-time application, is of interest especially for variable renewable generation units (e.g., PV and WT units) in case of imbalance to participate in the bilateral market in order to eliminate their imbalance and, thus, to avoid paying imbalance penalty. It is proposed to use discrete generation/consumption profile with four 15-min blocks, instead of traditional single generation/consumption profile with one hour duration. A multi agent system is proposed where seller and buyer agents negotiate to meet their objectives, i.e., for the seller agent (e.g., PV or WT) to maximize his revenue and to minimize his imbalance payment, while for the buyer agent (e.g., load or prosumer) to minimize his payment. Different case studies show that the proposed bilateral market helps reduce energy imbalances and increase the utility for market participants. More specifically, all four sellers have positive net revenues, while the imbalance cost the sellers have to pay to the market operator due to differences between scheduled power and actual power ranges from 12% to 35% of the net revenue.

4.13. Particle Swarm Optimization

Particle swarm optimization is a computational intelligence method able to optimize complex, large-scale, unconstrained and constrained, optimization problems [85,86]. PSO belongs in the category of evolutionary algorithms, so PSO is a population-based stochastic optimization algorithm. It iteratively solves the optimization problem by using a population of candidate solutions, called particles, and moving these particles in the search space using each particle’s best position, as well as the best known positions of other particles. In this way, the swarm moves towards the best solutions. PSO has few input parameters to regulate, it has low memory requirements, and it is easy to apply.

In [11], an innovative artificial immune system based PSO method is introduced for the optimal scheduling of multiple DERs: microturbine, fuel cell, wind turbine, photovoltaics, and demand response (load curtailment). A local energy market with continuous double auction is considered, where traders (buyers and sellers) submit their offers at any time during the trading period. Simulation results on a typical LV microgrid demonstrate that the proposed intelligent MAS effectively and profitably manages the various DERs. Moreover, the results show that the proposed method reduces the energy cost by 37% in comparison with the base case without DERs, while the PSO method reduces the energy cost by 35% in comparison with the base case.

In [12], particle swarm optimization is introduced for the optimization of a power distribution system with a microgrid and multiple distributed generation units: photovoltaics, wind turbines, and battery storage systems. Two different electricity markets are considered: pool and hybrid electricity market. More specifically, in the hybrid electricity market, the distribution system operator purchases power from the transmission grid under bilateral contract and the rest power from the microgrid and the distributed generation units under pool market. The simulation results show that in comparison with the pool market, the hybrid market model provides more benefits for the distribution system operator, the microgrid, and the distributed generation units. Moreover, the net present worth of the overall system found by the proposed PSO method is 0.09% higher than that found by GA.

In [13], signaled PSO is proposed for short-term scheduling of DER in smart power distribution networks. Signaled PSO includes an innovative mechanism, called signaling, which allows the adjustment of the velocity limits of the original PSO algorithm. Simulation results on a power distribution system with 99 DG units, 27 energy storage units, and 208 loads show that the proposed signaled PSO is superior to mixed integer non-linear programming (MINLP), GA, and other PSO variants. More specifically, in the first scenario, the signaled PSO computes the lowest total cost and MINLP computes the second lowest total cost, while GA and other PSO variants computed higher total costs than signaled PSO and MINLP. In the second scenario, MINLP computed the lowest total cost and signaled PSO computed the second lowest total cost. In comparison to MINLP, the proposed signaled PSO is 35 times faster for the first scenario and 39 times faster for the second scenario.

In [15], a modified PSO is proposed to solve the scheduling problem of distributed generation and plug-in hybrid electric vehicles in smart power distribution networks. In comparison with the standard PSO, the proposed modified PSO includes dynamic modification of the maximum and minimum velocity limits during the PSO iterations, which accelerates the execution of PSO algorithm. Uncontrolled charging and smart charging of PHEVs is studied. Moreover, two demand response programs are investigated for the PHEVs: trip reduction and trip shifting. The methodology is tested on a 33-bus power distribution network with 2000 PHEVs. The results indicate that, in comparison with MINLP methodology, the proposed modified PSO solves the scheduling problem 2700 times faster, while the value of the objective function is from 0% to 3% higher than that of MINLP.

In [26], a local energy market is formed that facilitates power trading between the microgrids as well as the distribution network. This is achieved with the formulation of an optimization model with an objective to minimize the sum of generation cost by microgrids and pollution emission cost by optimal scheduling the generated power of the distributed energy resources of the microgrids and the power transactions between the microgrids as well as the distribution network. Particle swarm optimization is proposed to solve this optimization problem. The uncertainties of load and renewable power generation are considered in the form of probability distribution functions. Ten different probability distribution functions are considered, namely, Gamma, Nakagami, generalized extreme value, Rice, t-location scale, Birnbaum-Saunders, Rayleigh, normal, Weibull, and inverse Gaussian distribution. Probabilistic analysis is used and the output results are expressed as probability distribution functions and cumulative distribution functions. Akaike’s information criterion is proposed to select the best probability distribution function. Simulation results of a power distribution network with three microgrids having different types of distributed energy resources show that in 47.6% of the cases Gamma distribution provides the best fit among the ten different investigated probability distribution functions, while the second best is Nakagami distribution that is the best in 19% of the cases.

In [29], a local energy market, operated by DSO, is designed to maximize day-ahead profit of DSO. This is achieved by optimally scheduling DSO-owned transportable battery storage systems and private-owned distributed generation units. The transportable battery storage systems can move between different nodes of power distribution network according to the optimal schedule that maximizes day-ahead profit of DSO and satisfies the imposed constraints. The proposed model is a mixed integer non-convex problem that is converted to a convex one through a two stage method. In the first stage, instantaneous transit of transportable BSSs is assumed, and the problem is reduced to a mixed integer second order cone programming, the solution of which finds the optimal buses for the transportable BSSs. In the second stage, a particle swarm optimization method finds the optimal transit times of the transportable BSSs. Simulation results on a real 41-bus distribution network shows that the proposed methodology minimizes the cost of power imported by the transmission grid, shifts BSSs charging to lowest buy price hours and schedules BSSs discharging at peak load hours, and provides local voltage control and reactive power support. In comparison to the case without BSS services, the proposed BSS services increase by 3.1% the net profit of DSO.

In [30], a local energy market is formed for the integrated energy system that is composed of power, heat, and gas local systems. A conditional value-at-risk bi-level optimization model is proposed to minimize expected operational cost for integrated energy system over a day-ahead horizon. This scheduling model considers the uncertainties in both the demand side and the supply side of the integrated energy system. The expected operational cost includes the expected overestimated and underestimated risk costs that are due to forecast errors in the demand side and/or the supply side of the integrated energy system. The bi-level optimization model is solved as follows. The upper level optimization model is solved by particle swarm optimization. The lower level optimization model is solved by interior point method. Simulation results show the efficiency of conditional value-at-risk method to assess the risk of the scheduling problem of the integrated energy system and to avoid financial losses caused by uncertainties.

In [33], the distribution system operator manages a local energy market with multiple demand response aggregators. The distribution system operator has two objectives: to minimize peak demand and to maximize his revenues. In order to achieve this objective, the distribution system operator has to offer electricity price incentives for each aggregator. To compute these incentives, two models are proposed. With the first model, the distribution system operator estimates each aggregator’s energy consumption needs using historical price and consumption data. With the second model, which is a two-stage robust optimization model, the distribution system operator computes his optimal data-driven pricing strategy, i.e., the optimal electricity price incentives for each aggregator. Hybrid particle swarm optimization with mutation is proposed to solve this two-stage robust optimization model. Simulation results of a local energy market with four demand response aggregators managing flexible loads and electric vehicles show that, in comparison to a flat rate pricing scheme, the proposed data-driven pricing strategy reduces peak demand by 25.3% and simultaneously reduces total electricity cost by 13.4% by encouraging demand response aggregators to shift their energy consumption to time periods with lower price.

In [40], a local energy market is proposed with the aim to optimize two conflicting objectives: alleviate lines congestion and minimize network operating cost. These objectives are simultaneously satisfied by optimally scheduling the distributed generation units (CHP, PV, and WT) as well as the charging and discharging of distributed battery storage systems. The uncertainties of renewable energy sources are considered through a probabilistic model. The proposed multi-objective model is a mixed integer non-linear programming problem that is solved by multi-objective particle swarm optimization that computes a set of optimal solutions, namely Pareto front. Next, fuzzy decision making is proposed to find the best solution from Pareto front. The method is tested on the modified IEEE 33-bus distribution network that has six BSSs, two CHP, two PVs, and two WT units. Simulation results show that the proposed method helps reduce lines congestion by 45.4%. Moreover, the optimal daily scheduling of battery storage systems helps reduce the curtailed energy of renewable resources.

In [46], a local energy market is considered that facilitates energy trading within microgrids and among microgrids. The microgrids can operate in radial and/or meshed configuration. Each node of each microgrid can have the following devices: either a load or a renewable distributed generation (PV or WT) in combination with a battery storage system. Each node of each microgrid has an agent that communicates only with neighboring agents and implements fully distributed control on its devices. The microgrids can belong to the same or different owners. Each microgrid simultaneously and in real-time manages its devices and its energy trading with other microgrids and the main grid. A fully distributed particle swarm optimization is proposed to solve the model predictive control-based optimization problem. The methodology has been tested on a power distribution network with four interconnected microgrids. Two of the microgrids have 38 nodes and operate in radial configuration, and the rest two microgrids have 30 nodes and operate in meshed configuration. Simulation results show that the proposed fully distributed real time optimization can be practicably applied to real-world interconnected microgrids. Moreover, power trading among microgrids reduces power bought from the HV grid by 23.9%, reduces voltage variations by 12.6%, and reduces power losses by 28.6%.

4.14. Reinforcement Learning

Reinforcement learning is a CI based method where the learner (agent) must learn, without a supervisor, but through trial and error interactions with a dynamically changing environment [87], [88]. The environment is usually modeled by a Markov decision process. The core idea of RL is that an agent, controlled by an automatic learning technique, is learning via interaction with its environment. The agent interacts with its environment and, watching the results of its actions and the rewards received, can learn to change its behavior to obtain more rewards. The objective of the agent is to find the optimal policy (optimal sequence of actions) that maximizes its expected reward.

In [50], an event-driven local energy market is designed that works as a backup market when needed (periods with energy surplus or energy shortage). This market aims at providing producers (sellers) and consumers (buyers) more energy trading alternatives. Buyers participate in the local market to buy electricity at a lower price than the utility’s price, and sellers participate in the local market to sell electricity at a higher price than through direct trading with utility. A retail energy broker acts as a mediator who facilitates the trading among buyers and sellers. The local market is designed as a Markov decision process in order to take into account the dynamics and characteristics of the market from broker’s point of view. Reinforcement learning is proposed to optimize broker’s trading strategy. More specifically, thanks to reinforcement learning, a smart trading strategy is proposed as follows: at the beginning this strategy performs moderately because it is at its learning phase. However, after some time, since more experience is obtained, the smart strategy adapts to market characteristics and exploits this knowledge to find its optimal policy and to maximize the profits of retail energy broker. Simulation results show that smart strategy with reinforcement learning provides better results in long term than a dummy strategy with objective to maximize the broker’s current profit. Moreover, in comparison to the case without LEM, the introduction of LEM increases the annual revenue of sellers by 18.1%, decreases the annual cost of buyers by 9.8%, and provides revenue for the retail energy broker that is 21.6% of the annual revenue of sellers.

4.15. Scatter Search

Scatter search (SS) is an efficient and flexible metaheuristic algorithm for solving hard optimization problems [89,90]. SS belongs in the category of evolutionary algorithms, so SS is a population-based optimization algorithm. SS uses a small population, called reference set, which is initially created non-randomly in order to as to contain diverse solutions. Next, an improvement method enhances the quality of solutions. Next, solutions with higher quality and more diversification form the updated reference set. Next, subsets of solutions of the reference set are created and combined to create new solutions with better quality, which form the new updated reference set.

In [31], a local energy market is operated by distribution system operator. The players of the local energy market are the owner of the distributed generation units and the distribution system operator. The objective of the distribution system operator is to minimize the operating cost of the distribution network. The objective of the distributed generation owner is to maximize his profit from selling energy to the distribution system operator. The above interaction between the two market players constitutes a Stackelberg leader-follower game that is modeled as a bi-level optimization problem. The upper level optimization problem corresponds to the maximization of the profit of distributed generation owner. The lower level optimization problem corresponds to the minimization of the operation cost for distribution system operator. This bi-level optimization problem is solved by scatter search that computes the optimal sizing, location, and contract pricing of distributed generation units. Simulation results on a 34-bus power distribution network show that, in comparison to memetic algorithm, the proposed scatter search provides 5.8% more profit to the owner of distributed generation units.

4.16. Simulated Annealing

Simulated annealing (SA) is a robust method for solving optimization problems in significantly reduced computational time [91,92]. SA has low memory needs, it is easy to implement, and it efficiently solves continuous and discrete optimization problems. SA method has its roots in thermodynamics and more specifically on the way the metals are cooled and annealed. The key is the gradual cooling of the liquid metal, until the system energy reaches its global minimum value. Indeed, the system energy reaches its minimum value, when the metal is slowly cooled. On the other hand, the system energy is higher if the liquid metal is quickly cooled.

In [19], a local energy market is considered, where the participants are distribution system operator, loads, and microgrids, where each microgrid corresponds to one household that has battery storage systems, photovoltaics, and loads. The energy trading takes place within the microgrids, among the microgrids, and between the microgrids and the main grid. The objective is to maximize the financial gains of a household, to fully supply all its loads, and to waste no PV energy. The optimization problem is solved by a modified simulated annealing. Different scenarios are investigated considering financial gain maximization via energy exchange: (a) with main grid, and (b) with main grid and neighbor households (microgrids). The proposed simulated annealing is evaluated in comparison to exhaustive search method, which finds the global optimum solution by considering the total search space. Simulation results show that in 9 out of 13 scenarios, the proposed simulated annealing manages to find the global optimal solution, while in the rest four scenarios the financial gain is from 0.1% to 2.0% lower than the gain computed by exhaustive search. Moreover, in comparison to exhaustive search, the simulated annealing approach is much faster, which makes it suitable for real-world large-scale applications.

4.17. Support Vector Machines

Support vector machine is a machine learning algorithm that is successfully used for classification, pattern recognition, forecasting, and regression [93,94]. In particular, for regression, a variant of SVM, called support vector regression, is used. SVM is based on statistical learning theory and the concept of empirical risk minimization, where the empirical risk is the forecasting error on the known training data, while the true risk is the forecasting error on the unknown test data in which the method will be actually tested. The core idea of SVM is to transform the nonlinear input space to a high-dimensional space and to build a hyper-plane or set of hyper-planes that can be used for classification or regression. SVM is simple to implement, fast, robust, and gives precise forecasts.

In [45], the power trading of PV plants in electricity market is studied. PV power output depends on weather conditions, so it cannot be accurately estimated. Moreover, the suppliers of renewable energy, including PV producers, should pay imbalance penalties in case of deviation between their settled energy (day-ahead dispatch) and their actual delivered energy (real time dispatch) in every hour. The prediction accuracy of SVM and ANN is analyzed for estimating PV power output. An energy storage system is proposed to operate in a coordinated manner with PV plant in order to reduce imbalance penalties. Simulation results show that the appropriate sizing of ESS can reduce imbalance penalties by up to 87% and 74%, when the PV power output is estimated by ANN and SVM, respectively. The maximum benefit to cost ratio for ANN and SVM is 1.17 and 2.32, respectively. Moreover, the ANN and SVM with ESS are approaching the no-penalty market profit of $589/h.

5. Contributions of the Reviewed Works

Table 4. Contribution of the reviewed works.

Reference	Published	Contribution
[10]	January 2011	An intelligent MAS is proposed for the scheduling and participation of multiple microgrids in a WEM. The coordinated scheduling of microgrids, using LR and GA, makes a better exploitation of multiple DERs: BSS, PV, TU, and WT.
[11]	October 2011	The optimal generation schedule of DERs is computed by an innovative artificial immune system based PSO method. Moreover, risks are considered in the auction process by applying an intelligent MAS.
[12]	January 2012	PSO is proposed for minimizing the cost of a microgrid and optimizing the production of DGs and the exchanges with the power distribution network considering two different markets: (a) pool market, and (b) hybrid market combining pool market with bilateral contracts.
[13]	June 2012	An innovative mechanism, called signaling, is proposed that allows the adjustment of the velocity limits of the PSO, and the resulting optimization methodology, called signaled PSO, is applied for short-term scheduling of DER in smart power distribution networks.
[14]	December 2012	An intelligent MAS is proposed to facilitate demand response and power trading among microgrids. A priority index is introduced based on size and number of times the load participate in demand response.
[15]	March 2013	A modified PSO, including dynamic modification of velocity limits, is proposed to solve the scheduling problem of DG and PHEVs. Smart charging, trip reduction, and trip shifting of PHEVs is investigated.
[16]	December 2013	A genetic algorithm is proposed for the maximization of the profit of electric vehicles aggregator participating in the local reserve services market of power distribution network.
[17]	February 2014	A stochastic multi-objective method is introduced for the scheduling of both energy and reserve markets considering the uncertainties of wind power output and load demand. The method includes the emission reduction objective and evaluates the DR participation in the market.
[18]	September 2014	Imperialist competitive algorithm is proposed for optimal scheduling of distributed energy resources in a microgrid and effective energy trading with the main grid. The method considers the risk of total or partial blackout and estimates the probabilistic outage cost of loads.
[19]	October 2014	It investigates a multi-objective optimization method, based on simulated annealing, to maximize the financial gains of a microgrid via energy trading not only with the main grid but also with other five neighboring microgrids.
[20]	October 2014	It investigates a multi-objective optimization method, based on a bio-inspired cognitive decision agent framework, to maximize the financial gains of microgrids energy trading and to maximize local consumption of locally produced PV energy.
[21]	December 2014	Hierarchical coordination is proposed between distribution system operation and charging of plug-in electric vehicles across multiple aggregators in order to reduce total electricity cost, reduce system peak load, and meet charging requirements of electric vehicles.
[22]	July 2015	A new pricing scheme called customer incentive pricing is introduced, in order to encourage customer participation in demand response and to compensate customers’ inconvenience of the aggregator controlling their smart appliances.
[23]	January 2016	A multi agent system in combination with coalitional game strategy is proposed for economic power transaction among multiple microgrids by simultaneously optimizing the economic efficiency of all microgrids.
[24]	February. 2016	An ant colony optimization method is proposed for optimal scheduling distributed energy resources of a microgrid. The flexibility and fast response of the method is validated by simulation and experimentation.
[25]	April 2016	A novel coordinated energy management strategy is proposed for smart homes in a neighborhood, where the objective is to minimize the electricity bills of the smart homes by optimally scheduling the distributed energy resources of smart homes.
[26]	June 2016	An optimization model is developed for optimal trading between the microgrids as well as the distribution network. The uncertainties of load and renewable power generation are considered in the form of PDFs. Akaike’s information criterion is proposed to select the best PDF.
[27]	February 2017	A methodology is developed to increase the resilience of microgrids. A fuzzy logic controller is proposed for the optimal operation of a battery storage system. A new mode, namely, the resilient mode of microgrids is proposed to ensure the survival of critical loads during disturbances.
[28]	July 2017	An expert system in combination with sequential quadratic programming approach is proposed to minimize the total operation cost of a DC residential distribution system. Adaptive direct current droop control is applied to keep voltage within limits in all DC buses.
[29]	September 2017	A methodology is proposed that optimally dispatches transportable BSSs to maximize the profit of DSO. The optimization problem is split into two sub-problems: the first is solved by MISOCP and the second is solved by PSO method.
[30]	September 2017	Conditional value-at-risk method is proposed to quantify the impact of risk due to uncertainties in both the demand and supply side of the IES. The scheduling of IES is formulated as a bi-level optimization problem and is solved by PSO in combination with IPM.
[31]	September 2017	A scatter search is proposed for optimal sizing, location, and contract pricing of distributed generation units in a local energy market managed by distribution system operator.
[32]	October 2017	A transactive local energy market is proposed with the objective to optimally integrate elastic loads with different operational preferences as well as electric vehicles buying and selling energy in economically viable way taking into account battery degradation cost.
[33]	January 2018	Inverse optimization is proposed to compute the optimal electricity prices the DSO has to offer to each demand response aggregator as incentive for the aggregator to shift energy consumption to time periods with lower prices in order for the DSO to minimize his peak demand and maximize his profit.
[34]	February 2018	Game vectors-based dynamic pricing is proposed to capture the spatio-temporal dynamic pricing behavior of each microgrid when trading in LEM. A two-level programming model, solved by customized hierarchical GA, optimizes the operation of LEM.
[35]	May 2018	Approximate Q-learning is proposed to optimize the scheduling of each DR aggregator in order to maximize his profit and to avoid the rejection of part of his DR schedule that implies penalties.
[36]	June 2018	A combinatorial double auction is introduced for energy trading among multiple microgrids. A hybrid method combining GA and PSO is proposed to find the winner of the combinatorial double auction.
[37]	July 2018	Differential evolution is proposed for maximizing the profit of DR aggregators who compete to sell DR services to the DSO and offer reward to the end-users loads for changing their load profiles.
[38]	July 2018	It investigates the feasibility of customer coupon demand response considering multiple load aggregators, energy loss of the distribution network, and the fluctuation of demand during the day.
[39]	August 2018	Non-dominated sorting genetic algorithm-II is proposed for optimal scheduling of industrial loads, by a load aggregator, with objectives to maximize user satisfaction and economic profit from load interruption
[40]	August 2018	Multi-objective PSO is proposed to optimize two conflicting objectives, i.e., to alleviate lines congestion and minimize network operating cost, which is achieved by optimal daily scheduling of BSSs and DG units.
[41]	January 2019	In power distribution system with multiple microgrids, a MAS is proposed to find the optimal resource allocation of multiple microgrids in order to improve the resilience of distribution system.
[42]	February 2019	Expectation and maximization based geometric clustering with unsupervised learning is proposed to cluster a power distribution area into a given number of local energy markets.
[43]	March 2019	GA is proposed for optimal allocation and scheduling of aggregator owned ESSs to satisfy SAIDI target and to minimize energy transaction costs as well as costs for ESSs investment, operation and maintenance.
[44]	March 2019	Fuzzy satisfying method is proposed to find the best solution from Pareto front optimal solutions that optimize the performance of a microgrid in the presence of demand response.
[45]	April 2019	A method is proposed to estimate the PV power output and to compute the optimal size of an ESS, in order to reduce imbalance penalties from power trading of PV plants in electricity market.
[46]	May 2019	A fully distributed particle swarm optimization is proposed for real-time optimal operation of interconnected microgrids in radial and/or meshed configuration.
[47]	May 2019	Fuzzy satisfying method in combination with ε-constraint technique is proposed to solve the multi-objective scheduling problem of charging and discharging of the electric vehicles of an aggregator.
[48]	May 2019	MAS framework is proposed to coordinate the bidding strategies of residential and industrial DR aggregators to maximize their profit and to provide flexibility services to compensate intermittency and volatility of RES.
[49]	June 2019	A multi agent system is proposed to manage a bilateral market, where seller and buyer agents negotiate to meet their objectives, i.e., for the seller agent to maximize his revenue and to minimize his imbalance payment, while for the buyer agent to minimize his payment.
[50]	July 2019	Thanks to its learning capability, reinforcement learning is proposed to optimize the interests of all market participants (energy broker, producers, and consumers) of an event-driven local energy market that is designed to work as a backup market when needed.

highlights the core contributions of the published works reviewed in this article in a chronological order.

6. Future Research

As has been shown in the previous sections, significant work has been done in the area of computational intelligence methods for the optimal operation of local energy markets at power distribution level [10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50]. However, there are still interesting areas for future research that need further investigation as analyzed in the following subsections.

6.1. Regulatory Framework

The regulatory framework has to define the local energy markets at power distribution level and their detailed interaction with existing wholesale electricity markets. One possible solution could be the decentralized market framework, according to which the DSO to aggregate all DERs of LEMs at power distribution level, and then, the DSO to provide a single bid to the WEM. This means that the DSO would be responsible to maintain real-time balance of supply and demand at power distribution level as well as within each LEM, based on transactions with local DERs and the WEM. In order to apply such a framework, support policies are needed for DER adoption, such as not very costly interconnection procedures and revenue opportunities for all other participants of LEMs, e.g., DER aggregators and DERs.

The regulatory framework has to ensure nondiscrimination and transparency in the operating and planning decisions of the DSO. A future regulatory framework has to remove possible barriers to market access. For example, in some countries, the minimum bid size is a barrier for aggregators of small DER units to access LEM.

There is also a need for a formal definition of distribution products and services of LEMs. This has to define the response time, direction (increase and/or decrease), and duration for each one of the grid services supplied by DERs in LEMs.

6.2. Roles of Market Players

As was shown in Section 3.2, the three main categories of market players at LEMs are the sellers, the buyers, and the mediators. There is a need for a formal definition of the roles and responsibilities of market players at LEMs. As an example, in the context of the decentralized market framework, discussed in Section 6.1, the new role of DSO may include not only the role of distribution network operator but also the role of LEMs operator. This new role of DSO will ensure that LEM operations do not overtake distribution network operations, since the DSO is a regulated entity.

6.3. TSO–DSO Coordination

In the context of the decentralized market framework, discussed in Section 6.1, there is a need for coordination and closer cooperation between the TSO and the DSO. This framework requires a shared responsibility for power system reliability between the DSO and the TSO. Since the DSO will have a very good picture of loads and DERs located at distribution network, the DSO can provide valuable information so as the TSO to be able to accurately forecast the net loads of supply substations. Moreover, the cooperation between DSO and TSO is important in the determination of DERs locations in the distribution network in combination with the safe levels of DERs participation for provision of grid services to the TSO. The enhanced cooperation between TSO and DSO will help allocate DER flexibility in order to maximize the benefit for the whole power system.

6.4. Communication Network

In order to facilitate the trading among market players in LEMs, a communication network is needed to provide the necessary information flow. Future research has to focus on the design of a secure and feasible communication network for information exchange among market players. The communication network has to be ubiquitous and has to have low latency. Moreover, future research works on LEMs have to use standards to build the communication network.

6.5. Software Platforms

In WEMs, transactions take place among entities and human operators. On the other hand, in LEMs, transactions may take place among agents, such as human operators, software platforms, and other unmanned systems. An interesting area for future research is the development of standards for the implementation of transactions among market agents. Another future research field is the development of appropriate software platforms for trading in LEMs, where the interoperability with different generations of software platforms, smart meters, and communication networks, is a key for the success of LEMs.

6.6. Distribution Infrastructure Investments

Investing in distribution infrastructure is another interesting field for future research. Distribution infrastructure investment has to be studied under a complete framework that includes LEM design, communication networks, control methods, financial incentives, and public energy policy objectives. The needs for control and communication investments of a distribution network with a low integration of DERs and no or few LEM operations are much smaller in comparison with the needs of a distribution system with much greater amounts of DERs, some of which participate in LEMs and provide flexibility services to the DSO.

6.7. Profitability of LEM

Profitability of LEM is an interesting future research field. The profitability of LEM means the profitability for all involved market players. It would be interesting to identify the minimum viable size of LEM. Current tariff structures and electricity prices in different countries may not be propitious enough to support the development and profitability of LEMs. This means that a well-designed pricing scheme is needed with the following objectives: (1) maximize market surplus for market participants, (2) provide appropriate incentives for market players to observe dispatch guidelines and make profitable investments, (3) provide transparency so that market players understand how prices are created, and (4) ensure that all sellers can recover their costs.

Since each DER has its own capabilities to offer flexibility services, a combination of pricing schemes (e.g., real time pricing, variable peak pricing, time of use pricing) has to be investigated and the impact of each combination of pricing schemes on LEM profitability has to be analyzed. Moreover, the most cost-efficient combination of DER technologies has to be identified in order to provide DER flexibility to meet specific technical requirements of the distribution system.

6.8. Power Flow Constraints

As was shown in Section 3.5, only a few research works have considered the impact of power flow constraints on the optimal operation of LEMs. The reason for not considering the impact of power flow constraints is that the main focus of these research works was the study of the optimal operation of LEMs from an economic point of view. However, power flow constraints cannot be ignored, because the physical distribution network is needed to implement energy transactions. The inclusion of power flow constraints is proposed as a future research field in order to develop models and methods that will provide viable solutions from both technical and economical point of view.

6.9. Scalability

The scalability of optimization algorithms and software platforms is proposed as a future research field. Distributed optimization algorithms for LEM optimization are more scalable. On the other hand, centralized optimization algorithms for LEM optimization are less scalable, due to the needed communication network. The scalability is also an issue in software platforms that perform trading in LEMs. The software platforms have to scale well for increased number of trading agents, and connected prosumers and smart devices.

6.10. Data Privacy

For the optimal scheduling of LEMs, existing research works assume that the market operator (e.g., the DSO) has detailed information for all the loads and DERs, which is not always realistic, due to data privacy. It is proposed as future research to develop models and methods for the optimal operation of LEMs taking into account that the market operator will have not detailed data for loads, but only aggregate data. Another future research field is the protection of market players’ data during as well as after energy trading.

6.11. Advanced Models

As a future research, it is proposed to develop advanced models that will take into account all the needs of LEMs. For example, future models have to investigate the simultaneous participation of DSO on WEM and LEM and its impact on the WEM and LEM prices. Future models have also to investigate the cooperation of TSO and DSO to manage their networks. Another interesting field is to optimally schedule LEMs in order to optimize multiple objectives for the distribution network, e.g., reliability improvement, network reconfiguration, and power loss reduction. The impact of uncertain wind power output, PV power output, electricity price, and load demand has to be analyzed using risk indicators. Another future research area is to model the use of local energy resources in order to increase the resilience of the distribution network to extreme weather conditions. Future models have to take into account social and economic factors, e.g., production and consumption profiles, customer preferences and their willingness to participate in demand response programs, and investment costs for home automation and smart metering.

6.12. Advanced Methods

A future research field is the application of stochastic programming and robust optimization methods to handle the uncertainties of electricity price, load demand, wind power output, and PV power output. Since the number of customers who would be involved in future LEMs would be very large, the design variables of the optimization problem would be too many, so distributed optimization methods and blockchain technology are proposed as future research in the area of LEMs optimization. Currently, a new generation of CI is under development that includes machine learning, big data, internet of things, and cloud computing. Consequently, a future research field is the application of the new generation of CI to the optimal operation of LEMs.

6.13. Comparison of CI Methods

LEMs at power distribution level is an emerging new field, where different market frameworks with different market players and different DERs are considered by the 41 works reviewed in this paper. There is also a big diversity of 17 different CI methods applied to optimally schedule these very different LEMs. Since different CI methods optimize different LEMs, currently it is not possible to identify the advantages and disadvantages of the different CI methods. An interesting future research field is the simultaneous comparison of many different CI methods and many different mathematical programming methods for the optimization of exactly the same LEMs using exactly the same test systems and data.

6.14. Multidisciplinary Teams

The application and advancement of CI methods necessitates the involvement of people with expertise in at least one of the following scientific fields: computer science, engineering, mathematics, physics, and operational research. Similarly, the design and operation of electricity markers requires necessitates the involvement of people with expertise in at least one of the following scientific fields: economics, electrical engineering, law, and management. Consequently, as a future research field, the collaboration of multidisciplinary teams is proposed for the optimal operation of LEMs with CI methods.

6.15. Gradual Transition

An interesting research area is the gradual transition from the current WEM framework to a modified market framework that, in addition with the trading in wholesale electricity markets, it will also facilitate the trading of DERs at the power distribution level in the context of LEMs. This gradual transition will have to remove barriers (where there are), e.g., a revised regulatory framework will have to permit aggregators trading. In order to further facilitate the development of LEMs, a modified regulatory framework and a formal definition of the role of local market players is needed, as discussed in Section 6.1 and Section 6.2, respectively.

7. Conclusions

This paper introduces a detailed description of the state-of-the-art of CI methods for LEMs at power distribution level to facilitate the integration of DERs, classifying and analyzing current and future research directions in this area. The most common LEM model has the following characteristics: (1) sellers are DGs, (2) buyers are loads, (3) mediators are aggregators, (4) DER type is PV, (5) the objective is the minimization of the total operation cost, and (6) AC power flow constraints are not included in the optimization model. The most commonly adopted CI methods for the optimal operation of LEMs are particle swarm optimization, genetic algorithm, and multi agent systems. Future research areas include regulatory framework, formal definition of the roles of local energy market players, TSO-DSO coordination, communication network and software platforms for energy trading, distribution infrastructure investments, profitability of LEMs, scalability, data privacy, and advanced models and methods for the optimal operation of LEMs.