Smart Microgrids Operation Considering a Variable Neighborhood Search: The Di ﬀ erential Evolutionary Particle Swarm Optimization Algorithm

: Increased use of renewable energies in smart microgrids (SMGs) present new technical challenges to system operation. SMGs must be self-su ﬃ cient and operate independently; however, when more elements are integrated into SMGs, as distributed energy resources (DER), traditional explicit mathematical formulations will demand too much data from the network and become intractable. In contrast, tools based on optimization with metaheuristics can provide near optimal solutions in acceptable times. Considering this, this paper presents the variable neighborhood search di ﬀ erential evolutionary particle swarm optimization (VNS-DEEPSO) algorithm to solve multi-objective stochastic control models, as SMGs system operation. The goal is to control DER while maximizing proﬁt. In this work, DER considered the bidirectional communication between energy storage systems (ESS) and electric vehicles (EVs). They can charge / discharge power and buy / sell energy in the electricity markets. Also, they have elements such as traditional generators (e.g., reciprocating engines) and loads, with demand response / control capability. Sources of uncertainty are associated with weather conditions, planned EV trips, load forecasting and the market prices. The VNS-DEEPSO algorithm was the winner of the IEEE Congress on Evolutionary Computation / The Genetic and Evolutionary Computation Conference (IEEE-CEC / GECCO 2019) smart grid competition (with encrypted code) and also won the IEEE World Congress on Computational Intelligence (IEEE-WCCI) 2018 smart grid competition (these competitions were developed by the group GECAD, based at the Polytechnic Institute of Porto, in collaboration with Delft University and Adelaide University). In the IEEE-CEC / GECCO 2019, the relative error improved between 32% and 152% in comparison with other algorithms. and J.G.-G.; analysis of results are carried out by A.B., D.A., C.C., A.G., N.B. and J.R.A.; the preparation of the original draft is done by S.R. and J.G.-G.; the review and editing is done by, D.A., C.C., A.G., D.R. and A. B.; finally, project administration is carried out by S.R.


Introduction
Decarbonization is a very important industrial and societal trend that is driving the adoption of renewable generation technologies and transportation electrification. Smart microgrids integrate elements such as EVs and generation with renewable energies [1]. In addition, microgrids are designed to encourage the participation of users, for instance, via demand response programs in competitive electricity markets [2]. Smart microgrids are systems that exhibit intelligent behavior, both at the 1. An algorithm that get good solutions for a framework (encrypted test bed by GECAD group) in the operation of smart microgrids with of energy storage systems (ESS), electric vehicles (EVs), loads with demand response, electricity markets, and renewables energies, proposed in [5,7]; 2.
An algorithm that has good solutions for a framework (encrypted test bed by GECAD group) for the treatment of problems in microgrids with high complexity due to aspects of uncertainty related to the weather conditions, load forecast, travel EVs 'and market prices, proposed in [5,7]; 3.
The novel VNS-DEEPSO algorithm is compared to the best fit of other algorithms at the international level and this one demonstrates superior performance.
The remainder of this document is as follows. Section 2 summarizes the state of the art. Section 3 presents the test case. Section 4 presents the VNS-DEEPSO algorithm. Section 5 presents further extensions of the algorithm and heuristic criteria for the initial solutions selection by the VNS-DEEPSO algorithm. Section 6 presents test results. Finally, Section 7 presents the conclusions of this work.

State of Art
We propose to carry out an evaluation of the case study based on the state of the art and unlike other research, the solution method will be evaluated taking the same case study, guaranteeing that each optimization algorithm that is carried out with its best performance (section results and discussion). In [10,11] the authors look for the interaction between providers and users to obtain economic benefits. Then, demand optimization consists of managing consumption as a function of time. In [12] the authors integrate independent networks assuming known load profiles. Moreover, in [13] generation forecasts with renewable resources are defined in discrete times, in addition DER such as ESS are distributed in the microgrid [14] and minimize investment and operational costs [13,15]. EVs have charging and discharging capability [16,17] and are managed by an aggregator that uses them as network resources along with ESS [6,7,[18][19][20]. Finally, electricity price forecasts complement the operation of the microgrid [21]. Economic data can thus be used for demand estimation [22].
The literature proposes that the inclusion of sources of uncertainty such elements include renewable energies, load forecasting, EVs and the price market; they economically and technically affect the operator decisions of the microgrid [11][12][13][14][15][16][17][18][21][22][23][24][25] (Table 1). Table 1 highlights one of the most complete works of Khodaei, 2015, Saber 2012 and Farzan 2015, which include high difficulty in the solution with uncertainty in renewable energy, demand forecast and electric vehicles. Radhakrishnan's (2016) and Di Somma (2018) present an important contribution including the uncertainty of electric markets; however, it does not include the planning of trips with electric vehicles. As a result, the Lezama and Soares case studies are the most complete and complex problems {5,7}. These models were adapted for the IEEE-WCCI 2018 and IEEE-CEC 2019/GECCO 2019 competitions (Table 1). Finally, the structure of the problem and the best proposed algorithm that maximizes the well-being of the network are detailed in materials and methods.

Materials and Methods: Case Study
The SMG case study in the real world ( Figure 1a) can be defined to consider an intermediary (microgrid aggregator). In addition, this one is based in the competition guidelines proposed in [5,7], which has the following goals: (1) Satisfy requirements from the electricity market and DER, (2) Minimize operational costs, (3) Increase revenues from electricity sales, (4) Use its own ESS and EVs (operating in V2G mode) to supply loads, (5) Establish bilateral energy contracts, (6) It does not obtain benefits for meeting the demand of EVs and users [6].
The power flow between the ESS and the EV is bidirectional, while loads can only consume energy, and generation will only be able to supply energy. In a similar way to commercial transactions, generators sell energy, loads can buy energy, and EVs, ESS, and the electricity market can buy/sell energy. Finally, all operations must be coordinated with the microgrid aggregator in an artificial problem ( Figure 1b)

Materials and Methods: Case Study
The SMG case study in the real world ( Figure 1a) can be defined to consider an intermediary (microgrid aggregator). In addition, this one is based in the competition guidelines proposed in [5,7], which has the following goals: (1) Satisfy requirements from the electricity market and DER, (2) Minimize operational costs, (3) Increase revenues from electricity sales, (4) Use its own ESS and EVs (operating in V2G mode) to supply loads, (5) Establish bilateral energy contracts, (6) It does not obtain benefits for meeting the demand of EVs and users [6].
The power flow between the ESS and the EV is bidirectional, while loads can only consume energy, and generation will only be able to supply energy. In a similar way to commercial transactions, generators sell energy, loads can buy energy, and EVs, ESS, and the electricity market can buy/sell energy. Finally, all operations must be coordinated with the microgrid aggregator in an artificial problem ( Figure 1b The microgrid aggregator schedules DER for the day-ahead market (next 24 h). For example, weather conditions to forecast renewable generation production, EV utilization, as well as market price and demand forecasts are key inputs to solve the optimization problem. Although these inputs are assumed to be highly accurate, there is high implicit uncertainty in the estimation process [6]. Therefore, optimization techniques used to solve this energy management problem must exhibit low sensitivity to these uncertainty parameters [17,18].
To incorporate uncertainty in the parameters, the competition designers used Monte Carlo simulations to generate a large number of possible scenarios based on probability distribution functions of forecast errors obtained from historical data [6,7]. A large number of scenarios increases model accuracy, but it has high computation costs due to high variations in the parameters. For this reason, in this work a reduction technique was used to keep a small number of scenarios while maintaining the main statistical features of the initial scenarios [6,17].

Structure of the Algorithm
The case study has two codes: the first one provided by the competition organizers, i.e., the performance function (a black box), and the second one provided by the participants, i.e., the optimization algorithm. In this way, the designers provided the case study data as input codes and the participants provided parameters of the optimization algorithm (the codes appear in Supplementary Materials). Then, the fitness function is a black box of the case study that limits the The microgrid aggregator schedules DER for the day-ahead market (next 24 h). For example, weather conditions to forecast renewable generation production, EV utilization, as well as market price and demand forecasts are key inputs to solve the optimization problem. Although these inputs are assumed to be highly accurate, there is high implicit uncertainty in the estimation process [6]. Therefore, optimization techniques used to solve this energy management problem must exhibit low sensitivity to these uncertainty parameters [17,18].
To incorporate uncertainty in the parameters, the competition designers used Monte Carlo simulations to generate a large number of possible scenarios based on probability distribution functions of forecast errors obtained from historical data [6,7]. A large number of scenarios increases model accuracy, but it has high computation costs due to high variations in the parameters. For this reason, in this work a reduction technique was used to keep a small number of scenarios while maintaining the main statistical features of the initial scenarios [6,17].

Structure of the Algorithm
The case study has two codes: the first one provided by the competition organizers, i.e., the performance function (a black box), and the second one provided by the participants, i.e., the optimization algorithm. In this way, the designers provided the case study data as input codes and the participants provided parameters of the optimization algorithm (the codes appear in Supplementary Materials). Then, the fitness function is a black box of the case study that limits the set

Encoding of the Fitness Function
The structure of the proposed solution will depend on the metaheuristic used. For example, for a Differential Evolution (DE), individuals will be taken; in a particle swarm optimization (PSO), there will be particles; or in a Genetic Algorithm (GA), genotypes will be taken. The solution adopted in the competition has the form presented in Table 2 [5,7]. Therefore, in Table 2, each solution is encoded as a vector consisting of six groups of variables that are repeated sequentially throughout the 24 periods (hours) of optimization. The following groups of variables are presented: (1) Active power of distributed generation (DGs) and external suppliers; (2) Binary generators; (3)  Then, the variables are continuous with limits that coincide with the associated of power or capacity limits, while binary variables are used to indicate whether a generator is connected (1) or disconnected (0). The total number of variables for a period is 142 and the variables for all periods are 3408 [6].

Mathematical Formulation of the Problem
Objective function, uncertainty, and assumptions of the model represent the structure of the problem.

Fitness Function and Uncertainty
The fitness function f′ considers the objective Z of the microgrid aggregator plus the sum of the penalties found during the evaluation of the solutions [

Encoding of the Fitness Function
The structure of the proposed solution will depend on the metaheuristic used. For example, for a Differential Evolution (DE), individuals will be taken; in a particle swarm optimization (PSO), there will be particles; or in a Genetic Algorithm (GA), genotypes will be taken. The solution adopted in the competition has the form presented in Table 2 [5,7]. Therefore, in Table 2, each solution is encoded as a vector consisting of six groups of variables that are repeated sequentially throughout the 24 periods (hours) of optimization. The following groups of variables are presented: (1) Active power of distributed generation (DGs) and external suppliers; (2) Binary generators; (3) and (4)  Then, the variables are continuous with limits that coincide with the associated of power or capacity limits, while binary variables are used to indicate whether a generator is connected (1) or disconnected (0). The total number of variables for a period is 142 and the variables for all periods are 3408 [6].

Mathematical Formulation of the Problem
Objective function, uncertainty, and assumptions of the model represent the structure of the problem.

Fitness Function and Uncertainty
The fitness function f considers the objective Z of the microgrid aggregator plus the sum of the penalties found during the evaluation of the solutions [5,26]: where X is a solution that follows the structure shown in Table 1. In this case, g i is the value of the ith construction (equality or inequality) and is a configurable penalty factor that is usually considered large. Similarly, in some parameters, uncertainty was considered to modify the value of f according to different scenarios generated by the Monte Carlo simulation. The value of f is modified with disturbances, as follows [5,26]: where δ s is the perturbation of variables and parameters in the scenario s, and F s ( X) is the objective value associated with the sampling s of Monte Carlo. Therefore, an expected average value for a given solution over the set of scenarios considered can be calculated as [5,26]: Equations (3) and (4) depend on the number of scenarios considered in the evaluation. As shown in the following paragraphs, in the optimization process the function f receives as parameter the number of scenarios that the competitor wishes to evaluate. For the final evaluation of the competition, the solutions are evaluated with a total number of 500 scenarios. The number of function evaluations is, therefore, the population size by the number of iterations by the number of scenarios [6]: The organizers of the competition (GECAD, based at the Polytechnic Institute of Porto, in collaboration with Delft University and Adelaide University) restricted the algorithm, so that a maximum of 50,000 function evaluations were allowed [6], therefore this is a restriction that significantly affects performance.

Objective Function
The objective function is given by the function [5,26]: On the one hand, operating costs are given by the function [23]: On the other hand, income is given by the function [5,26]:

Assumptions of the Model
• The cost function of distributed generation is assumed to be linear [5].

•
The microgrid aggregator can send and receive bids in the electricity price market [5].

Optimization Using VNS-DEEPSO Algorithm
Metaheuristics include a variety of strategies such as genetic algorithms, particle swarm, taboo search, swarm intelligence, scattered search, and trajectory re-chaining, among others [6,27]. In this study, the two techniques that produced the best results in the formulated problem were chosen. Next, the VNS-DEEPSO algorithm is used to solve the problem of the optimization of energy resources. VNS-DEEPSO algorithm was introduced in 2018 by this team as a hybrid technique that adapts itself to an environment with uncertainty. In this work the method is improved.

DEEPSO Algorithm
DEEPSO is a high-level hybrid technique constructed from different metaheuristics: "Evolutionary Algorithms" (EA), "Particle Swarm Optimization" (PSO) and "Differential Evolution" (DE). The advantage of these methods is that they orient the optimum in a globally correct direction. Combining these techniques aims to generate a more robust algorithm [5]. DE has many variations that seek successful self-adaptive schemes. These schemes are also involved in the start of the EPSO algorithm [5].
This algorithm included the positive effect of a probability of communication between particles, implementing the scheme of the "stochastic star". Its success strengthens the idea that a degree of control random variation is beneficial to the search of the optimum. Therefore, this assumption is further explored through some noise added to the EPSO search by embedding an operator DE in the global mechanism of the generation of new particles [5]. DEEPSO algorithm is a DE-EA-PSO hybrid without deductive demonstration, which has proven to be superior to other options [5].

VNS Algorithm
The VNS strategy consists of extending the local search to make systematic changes in the neighborhoods and solving complex problems. The algorithm explores distant neighborhoods and updates the best solution [5]. The operating structure of the algorithm has two main phases: initialization and repetition. In the initialization phase, the number of neighborhoods is defined, an initial solution is found, and the moment to stop is chosen.
In the repetition phase, a process is followed from the first until the last neighborhood. The process follows the following steps. First, a random solution of the ith neighborhood is generated, so that the solution belongs to the structures of the neighborhoods. Second, a local search method is applied using the initial solution, and the new solution is called local optimum. Third, if the local optimum solution is better than the solution found in the process, the global solution will be changed, otherwise the search continues. In this step, three situations can occur. First, the global optimum is equal to the local optimum or second, the local optimum found is worse than the global optimum. In these two cases, the algorithm looks for the next neighborhood. The last situation shows a local optimal that is better than the global optimum, then, the new global optimum will be updated [5]. This is repeated successively.

Criteria for the Selection of the VNS-DEEPSO Algorithm
Various methodologies were tested to improve the performance of the algorithm, below are presented. The Ranking Index (RI) value is calculated with the fitness Z plus the standard deviation σ (Z). This value is minimized with the following strategies.

Number of Scenarios
The maximum number of evaluations is 50,000, one proposal evaluates the sensitivity of the algorithm with the change of scenarios (Sc.) and they are changed between 10 and 500 (Sc.) to find potential Ranking Indexes (RIs) ( Table 3).

Objective Function
The objective function is divided into several objectives according to their RI percentage (Table 4). Table 4 shows that the contribution percentage of the standard deviation is 53.6% and the mean of 46.4%.

New Versions of the VNS-DEEPSO Algorithm
New algorithms are used to compare performance (Table 5). The classic VNS-DEEPSO algorithm has no variations. Some adjustments were made regarding the participation percentage between algorithms. The Simple VNS-DEEPSO algorithm eliminates mathematical functions that improve the classic algorithm. The 2VNS-DEEPSO algorithm has two VNS algorithms variations: the first VNS with penalties and the second with Ranking Index. The conditional VNS-DEEPSO algorithm evaluates some VNS executions with penalties and others without them.

Heuristic Rules Based on System Operation
The system operator studies in detail the six types of decision variables with the Simple VNS-DEEPSO algorithm and decides, for all the schedules, if it does not want load restrictions and if it wants to sell (but not to buy) from the electricity market. Table 6 presents operation criterion (OC) evaluations and RI calculations for different combinations, the classification is made for the two expected congestion intervals from 11 to 13 and from 17 to 20 h, and RI is calculated for discharging the EVs or the ESS.

1.
Operator discharges the EVs for the expected hours of congestion between 11 and 13 h.

2.
Operator discharges the ESS for the expected hours of congestion between 11 and 13 h.

3.
Operator discharges the EVs for the expected hours of congestion between 17 and 20 h.

4.
Operator discharges the ESS for the expected hours of congestion between 17 and 20 h.

Results and Discussion
To show the performance of the VNS-DEEPSO algorithm, this section presents the ranking of the teams that attended the IEEE-WCCI 2018 and CEC/GECCO 2019 competitions in the programming category of Smart Grids [5,7,23]. Table 7 presents the position, the optimization algorithm, and the RI. In terms of comparison, the relative error for the competition of the year 2018 and the year 2019 is calculated. The improvement in 2019 is more noticeable, improving the real error between 32% and 152% compared to the other algorithms. The argument of the VNS-DEEPSO algorithm, a new hybrid proposed in 2018 by the authors of this paper, is based on the use of mathematical functions that improve the performance of the algorithm. The mutation rate, communication probability and local search probability of DEEPSO presented few significant improvements, therefore their values were taken by default respectively as 0.8, 0.8 and 0.1; in contrast, the criteria presented in Section 5 showed improved algorithm performance. In addition, we take advantage of the features of VNS, because no population size and mutation ratio are required [26,27]. Regarding computation time, there is a significant restriction of a maximum of 50,000 evaluations of the objective function. For example, in the case of 500 scenarios, only 1000 evaluations can be carried out, which should be distributed between the population size and the number of iterations. Then the use of the VNS algorithm becomes paramount because it has only one particle.
However, since it is a hybrid algorithm, the percentage between VNS and DEEPSO should be adjusted. This code remained open to download and check for part of 2018 and for the competition year, 2019. In addition, all competitors devoted efforts to tune the parameters of their algorithms. In fact, they improve the performance of previous versions. Therefore, a new hybrid, the "Hybrid Levy Particle Swarm Variable Neighborhood Search Optimization" (HL_PS_VNSO), appears, which shared some characteristics of the VNS-DEEPSO 2018 algorithm, but with some improvements.
The VNS-DEEPSO 2019 algorithm was compared with several versions and a better performance was discovered for the new competition, without the use of mathematical functions. The compensation tool consisted of the use of the OC, which works in environments of uncertainty using the expected value. In other words, although there is uncertainty concerning the DER used by the system operator, some hours of congestion are expected in the network and all the DER of the microgrid should be prepared for these peak hours, which have a high probability of being more congested hours due to the penalties. An improvement to this version of the algorithm (VNS-DEEPSO 2019) is that it is not necessary to adjust the percentage between VNS and DEEPSO.
After applying the VNS-DEEPSO 2019 to the competition problem, one can see the operation of five dispatchable units, one of photovoltaic generation and one external supplier show the energy generation curve for a typical day (Figure 3a). The generators vary their connections to generate the load around 200 kW in the hours 4, 9, 13 and 21, close to 260 kW at hours 11, 14 and 15, and the valley hours show variations close to 50 kW at hours 1, 3 and 23. Regarding commercial transactions, Figure 3b depicts the two markets: the global market (color blue) and local market (color yellow). In the local market the power is sold near 10 kW while in global market, the power is sold just bellow of 100 kW with small falls at hour 3, 4, 12 and 14 (Figure 3b).
was discovered for the new competition, without the use of mathematical functions. The compensation tool consisted of the use of the OC, which works in environments of uncertainty using the expected value. In other words, although there is uncertainty concerning the DER used by the system operator, some hours of congestion are expected in the network and all the DER of the microgrid should be prepared for these peak hours, which have a high probability of being more congested hours due to the penalties. An improvement to this version of the algorithm (VNS-DEEPSO 2019) is that it is not necessary to adjust the percentage between VNS and DEEPSO.
After applying the VNS-DEEPSO 2019 to the competition problem, one can see the operation of five dispatchable units, one of photovoltaic generation and one external supplier show the energy generation curve for a typical day (Figure 3a). The generators vary their connections to generate the load around 200 kW in the hours 4, 9, 13 and 21, close to 260 kW at hours 11, 14 and 15, and the valley hours show variations close to 50 kW at hours 1, 3 and 23. Regarding commercial transactions, Figure  3b depicts the two markets: the global market (color blue) and local market (color yellow). In the local market the power is sold near 10 kW while in global market, the power is sold just bellow of 100 kW with small falls at hour 3, 4, 12 and 14 (Figure 3b). On the one hand, the 34 EVs are used to supply power to the microgrid between the hours of congestion, i.e., between hours 12, 13 and 14 (Figure 4a). On the other hand, ESS keep energy evenly between hours 1 to 10 and 18 to 24, and discharge between hours 11 and 17 (Figure 4b).  Finally, in Figure 5, the loads with demand response capability have a limit value of demand close to 0. Then, the load for users is close to 0 except for hour 6, in which the energy consumption reaches a peak near 0.015 kW. The system operator tries to avoid penalties by reducing their On the one hand, the 34 EVs are used to supply power to the microgrid between the hours of congestion, i.e., between hours 12, 13 and 14 (Figure 4a). On the other hand, ESS keep energy evenly between hours 1 to 10 and 18 to 24, and discharge between hours 11 and 17 (Figure 4b).
The VNS-DEEPSO 2019 algorithm was compared with several versions and a better performance was discovered for the new competition, without the use of mathematical functions. The compensation tool consisted of the use of the OC, which works in environments of uncertainty using the expected value. In other words, although there is uncertainty concerning the DER used by the system operator, some hours of congestion are expected in the network and all the DER of the microgrid should be prepared for these peak hours, which have a high probability of being more congested hours due to the penalties. An improvement to this version of the algorithm (VNS-DEEPSO 2019) is that it is not necessary to adjust the percentage between VNS and DEEPSO.
After applying the VNS-DEEPSO 2019 to the competition problem, one can see the operation of five dispatchable units, one of photovoltaic generation and one external supplier show the energy generation curve for a typical day (Figure 3a). The generators vary their connections to generate the load around 200 kW in the hours 4, 9, 13 and 21, close to 260 kW at hours 11, 14 and 15, and the valley hours show variations close to 50 kW at hours 1, 3 and 23. Regarding commercial transactions, Figure  3b depicts the two markets: the global market (color blue) and local market (color yellow). In the local market the power is sold near 10 kW while in global market, the power is sold just bellow of 100 kW with small falls at hour 3, 4, 12 and 14 (Figure 3b). On the one hand, the 34 EVs are used to supply power to the microgrid between the hours of congestion, i.e., between hours 12, 13 and 14 (Figure 4a). On the other hand, ESS keep energy evenly between hours 1 to 10 and 18 to 24, and discharge between hours 11 and 17 (Figure 4b).
(a) (b) Finally, in Figure 5, the loads with demand response capability have a limit value of demand close to 0. Then, the load for users is close to 0 except for hour 6, in which the energy consumption reaches a peak near 0.015 kW. The system operator tries to avoid penalties by reducing their Finally, in Figure 5, the loads with demand response capability have a limit value of demand close to 0. Then, the load for users is close to 0 except for hour 6, in which the energy consumption reaches a peak near 0.015 kW. The system operator tries to avoid penalties by reducing their consumption to 0. For future competitions, it is recommended to include a minimum operation duration as a requirement to ensure a minimum consumption value different than 0.
Energies 2019, 12, x FOR PEER REVIEW 11 of 13 consumption to 0. For future competitions, it is recommended to include a minimum operation duration as a requirement to ensure a minimum consumption value different than 0.

Conclusions
Sources of uncertainty affect the dispatch of smart microgrids. In this environment, a robust algorithm called VNS-DEEPSO was presented to provide solutions with low sensitivity to the variation of parameters due to inherent uncertainty. The algorithm was tested in the competitions developed by the groups GECAD, based at the Polytechnic Institute of Porto, in collaboration with Delft University and Adelaide University. The organizers of theses competitions provide a framework in order to test heuristic algorithms for the smart microgrids scheduling. In the proposed algorithm, heuristic rules were used based on an expected value, i.e., despite the uncertainty, the operator directs all the efforts to the expected congestion hours. The VNS-DEEPSO algorithm has the best performance when compared to versions of metaheuristic algorithms.