An Advanced Machine Learning Based Energy Management of Renewable Microgrids Considering Hybrid Electric Vehicles’ Charging Demand

Abstract

Renewable microgrids are new solutions for enhanced security, improved reliability and boosted power quality and operation in power systems. By deploying different sources of renewables such as solar panels and wind units, renewable microgrids can enhance reducing the greenhouse gasses and improve the efficiency. This paper proposes a machine learning based approach for energy management in renewable microgrids considering a reconfigurable structure based on remote switching of tie and sectionalizing. The suggested method considers the advanced support vector machine for modeling and estimating the charging demand of hybrid electric vehicles (HEVs). In order to mitigate the charging effects of HEVs on the system, two different scenarios are deployed; one coordinated and the other one intelligent charging. Due to the complex structure of the problem formulation, a new modified optimization method based on dragonfly is suggested. Moreover, a self-adaptive modification is suggested, which helps the solutions pick the modification method that best fits their situation. Simulation results on an IEEE microgrid test system show its appropriate and efficient quality in both scenarios. According to the prediction results for the total charging demand of the HEVs, the mean absolute percentage error is 0.978, which is very low. Moreover, the results show a 2.5% reduction in the total operation cost of the microgrid in the intelligent charging compared to the coordinated scheme.

1. Introduction

A renewable microgrid is a new competitive and cleaner generation of microgrids, which can make the situation for power generation low cost [1] and result in very low emissions [2], mitigating the amount of carbon emission per year [3]. In this way, different renewable energy sources, such as wind units [4], solar units, hydro units, and biomass units [5], can significantly help in achieving this goal. The results of such a new technology is reduced power losses, costs [6], voltage deviation and enhanced power quality and operation results. Solar energy is captured through some panels that are located in the microgrid, making a flat space for energy trapping in a renewable way [7]. Similarly, the wind units convert the kinetic energy of speed into electricity through some blades that are installed on top of the tower. Other than these renewable sources, the battery storages are always an inevitable part of renewable microgrids due to the highly volatile and non-dispatchable nature of the units [8]. In fact, all power produced by the renewable sources need to be purchased after generation since these units are uncontrollable in terms of power dispatch. A good solution is to store them in storage units and use the energy for peak-load hours later [9]. This is a wise policy, which not only mitigates the operating costs but also supports the ever growing progress of the renewable sources. In fact, it is not possible to think of renewable energy sources without the storage units. This shows the necessity of the research in this field specially when considering the new modern devices such as electric vehicles or automation systems. In recent years, many research works have been implemented, and a summary on the most significant topics on the energy management is given here.

Wang et al. [10] assessed the optimal energy management in renewable microgrids considering wind units, solar panels and battery. The proposed method optimizes the total operation costs by regarding the practical constraints. The role of hybrid wind-solar units on the management of a renewable microgrid is assessed in [11]. It is shown there that a higher penetration of the hybrid renewables can give promising solutions if managed properly. In [12], a stochastic method based on Monte Carlo is developed to model the uncertainty effects associated with the renewable sources in a stochastic framework. The results show that considering the uncertainty effects can bridge to a realistic solution that has considered the possible fuzziness injected into the grid. In [13], a data-driven framework is suggested for operation and management of renewable microgrids. The model uses big data for making economic and technical decisions and operating the system carefully. The same research but with a focus on the wireless sensor networks [14] is implemented in [15] in the microgrids. It is seen that a renewable microgrid is equipped with many sensors, which are always vulnerable against cyber hacking. In order to secure the decision-making process, a data intrusion detection approach based on prediction intervals is developed. This method would create a lower and an upper bound around the sample points and let the operator detect any abnormality in the model behavior. In [16], hydrogen production and thermal energy recovery of the fuel cell are investigated in a renewable microgrid. It is shown that considering the thermal aspect and hydrogen storage can enhance the efficiency of the units between 9 and 18% at peak load. In [17], a multi-objective structure is devised, which optimizes the power losses and costs in the renewable sources by optimal switching. Similar work on the effect of switching on the optimal policy and management of the microgrids is implemented in [18]. In [19], authors assessed the effect of electric vehicles on the microgrids operation. It is shown in [20] that the charging demand of electric vehicles needs to be modeled based on accurate inspection and investigation of their random behavior [21] in a complex environment [22]. While modeling the real effects of electric vehicles on the system is a necessity, finding a way to mitigate their effects and change it to opportunities for the microgrid is a research challenge [23]. In response to this need, the vehicle to grid (V2G) idea [24] is well assessed in the microgrids by the researchers to reduce the operation costs and provide a win-win game [25] for both the car owners and the microgrid operators. In this idea, the vehicle attends the charging and discharging plans based on an incentive program and thus may be regarded as mobile storage in the system. By drawing the unused stored energy of the car and injecting it into the smart grid, V2G technology is also known as “vehicle-grid integration” that can help energy supply at the peak load hours.

This paper investigates the policy development and energy management in the renewable microgrids considering wind units, solar panels, battery storage and hybrid electric vehicles (HEVs). In order to check the demand response problem in the microgrids, adjustable loads of two types including the shiftable loads and curtailable loads are modeled and considered in the model. A comprehensive model is proposed to model all components of the microgrid and optimize the total operation cost over a 24 h time horizon. The proposed method uses advanced machine learning based on support vector regression (SVR) to predict the total amount of charging demand by the electric vehicles. In contrast to the conventional machine learning methods, such as artificial neural network (ANNs), the SVR model can make a reliable prediction with high accuracy without the risk of overfitting due to the complex data [26]. Considering the highly chaotic and nonlinear nature of the data, a new optimization method based on dragonfly is introduced to optimize the setting parameters of the SVR such that more optimal hyperplanes would be attained. Due to the high complexity of the proposed discrete optimization problem, a new optimization algorithm based on the dragonfly algorithm (DA) is developed to manage the optimal scheduling of the units, loads, storages and switches in the microgrid. DA is a new heuristic optimization algorithm that mimics the fly of some insects and makes a powerful algorithm for solving the hard problems [27]. To enrich its performance, a novel self-adaptive modification method is introduced, which makes it possible for dragonflies to select the solution that best matches their structure. This would reduce the computational burden and mitigate their algorithm efforts for finding the real global solution. Considering the high penetration of HEVs in the modern microgrids, a reconfigurable structure is considered through the remotely embedded control switches to change the power flow path in the microgrid and thus avoid feeder congestion and improve the voltage profile and operation costs. The proposed model is constructed as a single-objective optimization problem [28] optimizing the cost of operation over 24 h. Therefore, the main paper contributions can be summarized as below:

• Proposing a novel operation and management framework for the reconfigurable microgrids considering the high penetration of HEVs (with two different charging schemes) and renewable energy sources;
• Developing a novel hybrid prediction model based on SVR and MDA for accurate estimation of the total charging demand of HEVs in the microgrid;
• Proposing a new optimization method based on DA and a three phase modification method to enhance its search ability for the optimization applications.

The performance of the model is assessed on an IEEE microgrids test system [29,30].

The paper structure is organized as follows: Section 2 explains the proposed machine learning based approach on SVR and modified DA (MDA). The HEVs behaviors and charging patterns are described in Section 3. In Section 4, the problem formulation addressing the comprehensive model of the microgrid is explained. The experimental results on the IEEE microgrid are discussed in Section 5. Finally, the main ideas are proposed in Section 6.

2. Machine Learning Based Policy Development and Energy Management Framework

This section is composed of two main parts; (1) a modeling tool based on SVR and (2) an optimization tool based of MDA. Each of these parts is described in the rest of the paper.

2.1. Machine Learning Based on SVR

In order to model the electric vehicles charging demand, this section explains the SVR model. SVR fundamentally is an advanced version of the traditional AI based models, which try to find the unknown relationship between the input and outputs in a system. In contrast to the neural networks (NNs), which focus on optimizing the training error even with sacrificing the generalization ability of the model, the SVR models consider the model complexity besides the training error. This means that SVR assumes a hyper plane over the model to avoid high complexity and tries to minimize the error of training as a balancing solution. To better perceive the concept, let us consider the training samples as ${\left\{(x_i,y_i)\right\}}^n$. The goal is to find the highest fitting nonlinear (or even linear) function φ (.) on the sample set such that the input and output datasets are functionalized in a higher space of dimension $\phi (.):{\mathcal{R}}^n\to {\mathcal{R}}^{n_h}$. In such a space, the linear function f exists, which learns the nonlinear relationship of the lower dimension as below [31]:

$$f(x)={\alpha }^T\phi (x)+\beta$$

(1)

where α and β are the setting constants of the model. In order to find their exact values, one needs to solve the following Equations [20]:

$$\min R_{SVR}=\frac{1}{n}{\displaystyle \sum _{i=1}^n{\Theta }_{\epsilon }}(y_i,{\alpha }^T\phi (x_i)+\beta )$$

(2)

$${\Theta }_{\epsilon }(y,f(x))=\{\begin{array}{ll}|f(x)-y|-\epsilon & ;|f(x)-y|\ge \epsilon \\ 0& ; \mathrm{Else}\end{array}$$

(3)

where $\epsilon$ is the loss function parameter, which penalizes high training errors. The above constraint problem can be converted into this combinatorial formulation as (4):

$$Mi{n}_{W,b,{\xi }^{*},\xi }{R}_{\epsilon }(\alpha ,{\xi }^{*},\xi )=\frac{1}{2}{\alpha }^T\alpha +C{\displaystyle \sum _{i=1}^n({\xi }_i^{*}+{\xi }_i)}$$

(4)

where C is the trade-off keeping parameter between the model complexity and the training error. The first term in (4) represents the inner product of the model setting parameters, which somehow shows the model density.

The constraints that are needed to be considered when solving (4) can be named as below:

$$\begin{array}{ll}{y}_i-{\alpha }^T\phi (x_i)-\beta \le \epsilon +{\xi }_i^{*}& ; i=1,2,\dots ,n\\ -y_i+{\alpha }^T\phi (x_i)+\beta \le \epsilon +{\xi }_i& ; i=1,2,\dots ,n\\ {\xi }_i^{*}\ge 0& ; i=1,2,\dots ,n\\ {\xi }_i\ge 0& ; i=1,2,\dots ,n\end{array}$$

(5)

After solving this problem, the optimal value of α is calculated as below:

$$\alpha ={\displaystyle \sum _{i=1}^n({\chi }_i^{*}-{\chi }_i)}\phi (x_i)$$

(6)

The final shape of the SVR model can be shown based on the kernel function as below:

$$\begin{array}{l}f(x)={\displaystyle \sum _{i=1}^n({\chi }_i^{*}-{\chi }_i)K(x_i,x)+\beta }\\ K(x_i,x)=\phi (x_i)\circ \phi (x_j)\end{array}$$

(7)

where ${\chi }_i^{*}/{\chi }_i$ are the dual parameters and K is the kernel function. The type of the kernel function can affect the prediction accuracy [32]. This paper uses the radial basis function due to its simplicity and high mapping features:

$$K(x_i,x_j)=\exp (-0.5{\Vert x_i-x_j\Vert }^2/{\sigma }^2)$$

(8)

where σ is the standard deviation of the kernel.

Figure 1 shows the SVR model and hyper plane parameters. In this figure, the red and black balls show the two classes of data and the goal is to decide which class a new data point will be in. To this end, SVR will try to separate such a point (classes of data) with a hyperplane. The dotted lines in the figure show the boundaries of the hyperplane, which is determined through the optimization in a higher dimensional space. More information for the interested readers can be found in [20].

Figure 1. Position of hyper parameters in support vector regression (SVR) $\omega .x+b=1$.

2.2. Improved Optimization Based on MDA

In order to make accurate predictions, the SVR setting parameters of C and σ need to get adjusted properly. This paper introduces the MDA, which is inspired by the dragonflies. DA is an evolving algorithm that works based on random flights of some insects when searching for food. This algorithm has special characteristics including the low number of setting parameters, compatible with continuous and discrete optimizations, and access to local and global operators of the search. The movement of each dragonfly is influenced by five factors, which are explained by the following [33,34].

The first one is the “separation” factor, which keeps a logical distance between the different agents as below:

$$\mu S_i=-{\displaystyle \sum _{j=1}^N\theta }-{\theta }_i$$

(9)

The second motivation is called “alignment”, which measures and adjusts the speed of an agent with others in the surrounding area:

$$\mu A_i=\frac{{\displaystyle \sum _{j=1}^N{v}_j}}{N_P}$$

(10)

The third motivation is the “cohesion”, which pretends the movement of the individuals to the center of mass population as below:

$$\mu C_i=\frac{{\displaystyle \sum _{j=1}^N{\theta }_j}}{N_P}-\theta$$

(11)

The next motivation factor is called “attraction”, which describes the way of movement of each solution in the DA toward the food position:

$$\mu F_i=F_{loc}-\theta$$

(12)

The last motivation factor is called “distraction”, which shows the defense mechanism of dragonflies from their enemies.

$$\mu E_i=E_{loc}+\theta$$

(13)

The algorithm repeats this process until the food location and solutions’ quality are improved. In each iteration, the best solution position is updated and called X_best. The enemy situation is also updated in accordance with the worst candidate in the population. All the above motivation factors are then mixed up with different weighting factors to make a united factor as below [27]:

$$\mathsf{\Delta }{\theta }_{t+1}=\left(s\times \mu S_i+a\times \mu A_i+c\times \mu C_i+f\times \mu F_i+e\times \mu E_i\right)+\varpi \theta$$

(14)

It is proved in [27] that the value of the weighting factor can be adjusted as below:

$$\begin{array}{l}s=2\times r\times \rho \\ a=2\times r\times \rho \\ c=2\times r\times \rho \\ f=2\times r\times \rho \\ e=\rho \end{array}$$

(15)

$$\varpi =0.9-Iter\times \frac{(0.9-0.4)}{MaxIter}$$

(16)

where ρ is extracted out as follows:

$$\rho =\{\begin{array}{ll}0.1-\frac{0.2\times Iter}{MaxIter}& ; if\hspace{1em}(2\times Iter)\le MaxIter\\ 0& ; else\end{array}$$

(17)

Therefore, each candidate solution is updated as below:

$${\theta }_{t+1}={\theta }_t+\mathsf{\Delta }{\theta }_{t+1}$$

(18)

In order to boost the DA performance, the adaptive formulation of modified solutions is presented in this part. At first each modification is explained:

Modification 1: This strategy adds to the diversity of the dragonflies by the use of a crossover-operator. To this end, three dissimilar solutions are taken out such that (θ₁ ≠ θ₂ ≠ θ₃ ≠ θ_i):

$$X_{mut}={\theta }_1+r_1({\theta }_2-{\theta }_3)$$

(19)

Now, three test individuals are produced as follows:

$$\begin{array}{c}{\theta }_{Test1,j}=\{\begin{array}{c}\begin{array}{cc}{\theta }_{mut,j}& r_2\end{array}\le r_3\\ \begin{array}{cc}{\theta }_{best}& otherwise\end{array}\end{array}\\ {\theta }_{Test2,j}=\{\begin{array}{c}\begin{array}{cc}{\theta }_{mut,j}& r_4\le r_5\end{array}\\ \begin{array}{cc}{\theta }_j& otherwise\end{array}\end{array}\\ {\theta }_{Test3,j}={\rho }_6{\theta }_{best}+{\rho }_7({\theta }_{best}-{\theta }_r)\end{array}$$

(20)

Modification 2: This strategy will add the population solution types by making some information changing process among the solutions as below:

$$\begin{array}{l}\begin{array}{cc}if& f_i\end{array}<f_j\\ {\theta }_{Test4,i}={\theta }_i+r_8({\theta }_i-{\theta }_j)\\ \begin{array}{cc}elseif& f_j<f_i\end{array}\\ {\theta }_{Test4,i}={\theta }_i+r_9({\theta }_j-{\theta }_i)\\ End\end{array}$$

(21)

In order to make it adaptive, every dragonfly in the population should have a weighting factor. Similarly, a success rate (SR) is assigned to each modification, which is 0.5 at the beginning. According to the objective function value, each solution would attain a weighting as below:

$$W_i=\frac{\log (f_{Np}-f_i+1)}{{\displaystyle \sum _{j=1}^{Np}\log (f_j)}}$$

(22)

Also, an accumulator is defined for each modification such that initially accum_s = 0 and it is updated as below:

$$\begin{array}{cc}accu{m}_s=accu{m}_s^{initial}+\frac{{\displaystyle \sum _{i=1}^{N_{\mathrm{mod}}}{W}_i}}{N_{\mathrm{mod}}}& \forall s\in S_{\mathrm{modification}}\end{array}$$

(23)

Also, the SR index is then updated according to the accum as below:

$$S{R}_s=(1-\theta )S{R}_s+0.142\times \frac{accu{m}_s}{M}$$

(24)

It is clear that the SR value needs to get updated per unit as below:

$$S{R}_s=\frac{S{R}_s}{{\displaystyle \sum _{k\in S_{strategy}}S{R}_k}}$$

(25)

Figure 2 shows the schematic explanation of the proposed adaptive algorithm.

Figure 2. Flowchart of the proposed modified dragonfly algorithm (MDA).

3. Charging Modeling of HEVs

The charging behavior of HEVs is an element of many factors including the market share, the state of charge (SoC), charging time and length, etc. In order to consider the HEVs’ effects in the system, all these uncertain factors need to be well-defined to inject less randomness to the system. This paper makes use of the SVR method for prediction of the overall charging demands of HEVs. From the technical point of view, HEVs can get energy from either gas or electricity. Figure 3 shows the main components in HEVs [35]. In Figure 3, the cylinder shows the fuel storage since the HEVs work partly based on electricity and partly based on the fossil fuels.

Figure 3. Main components in a typical hybrid electric vehicle (HEV).

Still, we need to find a way to understand the distribution the charging in these vehicles. To this end, two different strategies are considered. Coordinated charging and intelligent charging.

In the coordinated charging, HEVs are forced (or better to say allowed) to discharge at specific hours. These hours are based on the daily pattern life of the customers. It means that car owners can start charging after work but at off-peak hours, around 6 p.m. to 7 p.m.:

$$f(t_{start})=\frac{1}{{\gamma }_1-{\gamma }_2}\hspace{1em}{\gamma }_1\le t_{start}\le {\gamma }_2\hspace{1em};{\gamma }_1=18,{\gamma }_1=19$$

(26)

The second pattern, which may be regarded in the charging, is intelligent charging, which will let the vehicles charge according to the electrical load curve of the microgrid and bidding offers by the power companies. This strategy is modeled according to the normal distribution function as below:

$$f(t_{start})=\frac{1}{\sigma \sqrt{2\pi }}{e}^{(-\frac{1}{2}{(\frac{t_{start}-\mu }{\sigma })}^2)}\hspace{1em}\mu =1,\sigma =3$$

(27)

After knowing the starting time, the amount of charging demand by HEVs is determined according to their battery SoC. For estimating the SoC, one needs to find the average mileage passed by each vehicle. According to the statistics, the average mileage of each vehicle can be estimated as below:

$$f(m)=\frac{1}{m\sigma \sqrt{2\pi }}{e}^{\frac{-{(\ln (m)-\mu )}^2}{2{\sigma }^2}}\hspace{1em}m>0$$

(28)

Having m, the battery SoC is determined through a straightforward equation as below:

$$SOC=\{\begin{array}{cc}0& m>ER\\ \frac{ER-m}{ER}\times 100\%& m\le ER\end{array}$$

(29)

where ER is the total electric range, i.e., if mileage is more than this, the vehicle will shut down. Based on the literature, ER can take values of (20,30,49,60) depending on the car type. Having SoC and the battery capacity, the charging length can be well estimated by the following equation:

$$t_D=\frac{C_{bat}\times (1-SOC)\times DOD}{{\eta }_c\times P}$$

(30)

From Equation (30) it is seen that charging time also depends on the charger type P_c and efficiency η_c.

According to the Society of Automotive Engineers (SAE), all EVs produced by U.S. automakers must follow the SAE Surface Vehicle Recommended Practice J1772 standard. Table 1 and Table 2 provide four charger types for four different classes of HEVs in the U.S. based on the SAE standard. These data will be used further in the simulation results for making a comprehensive analysis.

Table 1. Charger type in HEVs [36].

Charger Type	Input Voltage	Maximum Power (kW)
Level 1	120 VAC	1.44
Level 2	208–240 VAC	11.5
Level 3	208–240 VAC	96
Level 3 (DC)	208–600 VDC	240

Table 2. HEV classes [36].

Class	Market Share	Min Cbat (kWh)	Max Cbat (kWh)
Micro Car	0.2	8	12
Economy Car	0.3	10	14
Mid-Size Car	0.3	14	18
SUV	0.3	19	23

The market share of HEVs is randomly determined based on their type/class in Table 2 with a normal distribution function with characteristics of mean and standard deviation as below:

$$\begin{array}{c}{\mu }_{C_{bat}}=\frac{Min{C}_{bat}+Max{C}_{bat}}{2}\\ {\sigma }_{C_{bat}}=\frac{Max{C}_{bat}-Min{C}_{bat}}{4}\end{array}$$

(31)

4. Problem Formulations

In the energy management problem, the most important task of the operator is providing a reliable and secure energy to the electrical users but with the least cost [37,38]. Therefore, the objective function is a mix of the cost of operation and technical costs as below [39,40]:

$$\begin{array}{ll}Minimize\hspace{1em}Cost=& {\displaystyle \sum _{t\in {\mathsf{\Omega }}^T}{\displaystyle \sum _{m\in {\mathsf{\Omega }}^{BG}}{\rho }_m^G{P}_{m,t}^G}}\delta +{\displaystyle \sum _{t\in {\mathsf{\Omega }}^T}{\displaystyle \sum _{m\in {\mathsf{\Omega }}^B}{\rho }_t^M{P}_{m,t}^M}}\delta +\\ & {\displaystyle \sum _{t\in {\mathsf{\Omega }}^T}{\displaystyle \sum _{m\in {\mathsf{\Omega }}^B}{\rho }_t^M{P}_{m,t}^{HEV}}}\delta +{\displaystyle \sum _{t\in {\mathsf{\Omega }}^T}{\displaystyle \sum _{mn\in {\mathsf{\Omega }}^L}{\rho }_t^M{R}_{mn}{\left(I_{mn,t}^L\right)}^2}}\delta \end{array}$$

(32)

The main operation limits of the problem that should be considered when optimizing the above cost function are as follows. In this formulation, Equations (33)–(38) belong to the DGs, (39)–(45) belong to the energy storages, (46)–(48) belong to the adjustable loads and (49)–(56) belong to the load flow and power balance limits.

-

Active and reactive power limit of DGs:

$$\underset{\_}{P_m^G}{x}_{m,t}^G\le P_{m,t}^G\le \overline{P_m^G}{x}_{m,t}^G\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BG}, \forall t\in {\mathsf{\Omega }}^T$$

(33)

$$\underset{\_}{Q_m^G}{x}_{m,t}^G\le Q_{m,t}^G\le \overline{Q_m^G}{x}_{m,t}^G\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BG}, \forall t\in {\mathsf{\Omega }}^T$$

(34)

-

Ramp up/down limits:

$$P_{m,t}^G-P_{m,t-1}^G\le R{U}_m^G\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BG}, \forall t\in {\mathsf{\Omega }}^T$$

(35)

$$P_{m,t-1}^G-P_{m,t}^G\le R{D}_m^G\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BG}, \forall t\in {\mathsf{\Omega }}^T$$

(36)

-

Minimum up and down time limit:

$$T_{m,t}^{G-on}\ge U{T}_m^G\left(x_{m,t}^G-x_{m,t-1}^G\right)\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BG}, \forall t\in {\mathsf{\Omega }}^T$$

(37)

$$T_{m,t}^{G-off}\ge D{T}_m^G\left(x_{m,t-1}^G-x_{m,t}^G\right)\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BG}, \forall t\in {\mathsf{\Omega }}^T$$

(38)

-

Charging/discharging limit of battery storage:

$$\underset{\_}{P_m^{Ch}}{y}_{m,t}^{Ch}\le P_{m,t}^{Ch}\le \overline{P_m^{Ch}}{y}_{m,t}^{Ch}\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BS}, \forall t\in {\mathsf{\Omega }}^T$$

(39)

$$\underset{\_}{P_m^{Disch}}{y}_{m,t}^{Disch}\le P_{m,t}^{Disch}\le \overline{P_m^{Disch}}{y}_{m,t}^{Disch}\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BS}, \forall t\in {\mathsf{\Omega }}^T$$

(40)

The total energy stored in the battery can be calculated as below:

$$C_{m,t}^S=C_{m,t-1}^S-P_{m,t}^{Disch}\delta /{\eta }_m^{Disch}+P_{m,t}^{Ch}\delta {\eta }_m^{Ch}\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BS}, \forall t\in {\mathsf{\Omega }}^T$$

(41)

$$\underset{\_}{C_m^S}\le C_{m,t}^S\le \overline{C_m^S}\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BS}, \forall t\in {\mathsf{\Omega }}^T$$

(42)

Moreover, it is sometimes needed to consider a minimum charging and discharging time for the storages as below:

$$T_{m,t}^{Ch}\ge C{T}_m^S\left(y_{m,t}^{Ch}-y_{m,t-1}^{Ch}\right)\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BS}, \forall t\in {\mathsf{\Omega }}^T$$

(43)

$$T_{m,t}^{Disch}\ge D{T}_m^S\left(y_{m,t}^{Disch}-y_{m,t-1}^{Disch}\right)\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BS}, \forall t\in {\mathsf{\Omega }}^T$$

(44)

It is clear that the storage unit can be in one of the charging or discharging modes at any time as below:

$$y_{m,t}^{Ch}+y_{m,t}^{Disch}\le 1\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BS}, \forall t\in {\mathsf{\Omega }}^T$$

(45)

-

Adjustable load demand:

The adjustable loads, either curtailable or shiftable, can help much to mitigate the operation cost and improve the voltage profile and are scheduled as below:

$$\underset{\_}{P_m^{AD}}{z}_{m,t}^{AD}\le P_{m,t}^D\le \overline{P_m^{AD}}{z}_{m,t}^{AD}\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BAD}, \forall t\in {\mathsf{\Omega }}^T$$

(46)

$${\displaystyle \sum _{t\in \left[{\sigma }_m,{\omega }_m\right]}{P}_{m,t}^D\delta }=E_m^{AD}\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BAD}, \forall t\in {\mathsf{\Omega }}^T$$

(47)

$$T_{m,t}^{AD-on}\ge U{T}_m^{AD}\left(z_{m,t}^{AD}-z_{m,t-1}^{AD}\right)\hspace{1em}\forall m\in {\mathsf{\Omega }}^{BAD}, \forall t\in {\mathsf{\Omega }}^T$$

(48)

Generation and demand balance:

$$\begin{array}{c}{\displaystyle \sum _{lm\in {\mathsf{\Omega }}^L}\left[P_{lm,t}^L-R_{lm}{\left(I_{lm,t}^L\right)}^2\right]}-{\displaystyle \sum _{mn\in {\mathsf{\Omega }}^L}{P}_{mn,t}^L+P_{m,t}^G-P_{m,t}^{Ch}}+P_{m,t}^{Disch}\\ +P_{m,t}^M=P_{m,t}^D\hspace{1em}\forall m\in {\mathsf{\Omega }}^B, \forall t\in {\mathsf{\Omega }}^T\end{array}$$

(49)

$${\displaystyle \sum _{lm\in {\mathsf{\Omega }}^L}\left[Q_{lm,t}^L-X_{lm}{\left(I_{lm,t}^L\right)}^2\right]}-{\displaystyle \sum _{mn\in {\mathsf{\Omega }}^L}{Q}_{mn,t}^L+Q_{m,t}^G}+Q_{m,t}^M=Q_{m,t}^D\hspace{1em}\forall m\in {\mathsf{\Omega }}^B, \forall t\in {\mathsf{\Omega }}^T$$

(50)

-

Bus voltage limit:

$$\underset{\_}{V}\le V_{m,t}\le \overline{V}\hspace{1em}\forall m\in {\mathsf{\Omega }}^B, \forall t\in {\mathsf{\Omega }}^T$$

(51)

-

Main grid power limit:

$$-\overline{P_m^M}\le P_{m,t}^M\le \overline{P_m^M}\hspace{1em}\forall m\in {\mathsf{\Omega }}^B, \forall t\in {\mathsf{\Omega }}^T$$

(52)

-

Reconfiguration using the remote switches:

Optimal switching is a strategic tool for altering the power flow path in the microgrid using the tie (normal open) switches and sectionalizing (normal closed). To this end the binary variable $w_{mn,t}^L$ is used, which can get 0 and 1 to show the closed or open status of a line:

$$0\le I_{mn,t}^L\le \overline{I^L}{w}_{mn,t}^L\hspace{1em}\forall mn\in {\mathsf{\Omega }}^L, \forall t\in {\mathsf{\Omega }}^T$$

(53)

$${\displaystyle \sum _{lm\in {\mathsf{\Omega }}^L}{w}_{lm,t}^L}=1\hspace{1em}\forall m\in \left({\mathsf{\Omega }}^{BAD}\cup {\mathsf{\Omega }}^{BCD}\right), \forall t\in {\mathsf{\Omega }}^T$$

(54)

$${\displaystyle \sum _{lm\in {\mathsf{\Omega }}^L}{\theta }_{lm,t}^L}-{\displaystyle \sum _{mn\in {\mathsf{\Omega }}^L}{\theta }_{mn,t}^L}+{\theta }_{m,t}^M={\theta }_{m,t}^D\hspace{1em}\forall m\in {\mathsf{\Omega }}^B, \forall t\in {\mathsf{\Omega }}^T$$

(55)

$$0\le {\theta }_{mn,t}^L\le \left|{\mathsf{\Omega }}^B\right|w_{mn,t}^L\hspace{1em}\forall mn\in {\mathsf{\Omega }}^L, \forall t\in {\mathsf{\Omega }}^T$$

(56)

The network reconfiguration, which is made by the use of the remotely controlled switches, can be made at any time interval as long as it meets the technical limitations of the switches. In our case study, the system can experience the reconfiguration in every hour, which means the system may get up to 24 different topologies during a day. Considering the very short time needed for a switch operation (less than a minute in the automated type), it is possible to make more reconfigurations in a day according to the automation system.

5. Simulation Results

In order to check the proposed machine learning based model performance, an IEEE microgrid test system was used and is introduced in this section [36]. The diagram of the test system is shown in Figure 4. The complete network data including the bus data and branch data are provided in the Appendix A. As it can be seen in this figure, two wind units, two solar panels and some switches are located in the microgrid. Moreover, HEVs are distributed all around the grid and asking for charging demand according to their probability distributions but equally among all buses. The tie switches are located between the buses 11–66, 15–69, 13–20, 27–54 and 39–58. Therefore, there are total 73 remote switches installed in the grid among which 68 are sectionalizing and are ready for reconfiguration maneuver if needed. The microgrid is providing an appropriate portion of its electrical load demands from the renewable energy sources of wind turbine and solar panel. Figure 5 shows the solar panel and the wind turbine output power pattern per unit. Similar patterns are considered for wind turbines with capacities of 142 and 250 kW. Also, the same power generation patterns are assumed for solar panels with capacities of 200 and 180 kW to avoid unnecessary complexities [41]. Table 3 provides raw data for the total energy charging demand of the HEVs in July 2019 recorded in California (the full dataset is for January–July 2019) [42]. These data help to have a clearer picture of the random and hard-to-predict nature of the charging demand of the HEVs in the system. As it was mentioned in Section 3, it is the role of the proposed model to determine how the total energy charging demand in the target day of operation (which of course is predicted) would be distributed according to the intelligent or coordinated charging pattern. As is shown in Figure 4, a battery storage unit is installed in the microgrid, and its characteristics are shown in Table 4. Moreover, the characteristics of the adjustable loads are shown in Table 5. It is seen that there are four adjustable loads considered in the microgrid, some of which some are shiftable, as shown by S, and some are curtailable and are shown by C. The MDA has an initial population of 40, which is upgraded after a hundred iterations.

Figure 4. Picture of the automated grid test system.

Figure 5. Wind turbine and solar panel power generation patterns.

Table 3. Historical raw data of the total energy charging demand of HEVs over a month in the microgrid.

Time (Day)	1–8
Charging demand (MWh)	2.3928	2.1667	2.2810	2.3690	2.4564	2.4876	2.2998	2.1136
Time (day)	9–16
Charging demand (MWh)	2.1185	2.1678	2.4336	2.1663	2.4215	2.1614	2.4739	2.2099
Time (day)	17–24
Charging demand (MWh)	2.1401	2.1649	2.3312	2.2661	2.2107	2.4290	2.3172	2.3010
Time (day)	25–30
Charging demand (MWh)	2.4684	2.1807	2.3955	2.3939	2.2238	2.2783

Table 4. Characteristics of the energy storage.

Storage	Bus	Capacity (kWh)	Min-Max Charging/Discharging Power (kW)	Min Charging/Discharging Time (h)
DES	15	1500	50–250	4

Table 5. Characteristics of adjustable loads (s: shiftable, c: curtailable).

Load	Type	Bus	Min-Max Capacity (kW)	Required Energy (kWh)	Initial Start/End Time (h)	Min Up Time (h)
L1	S	28	0–60	240	11–14	1
L2	S	56	0–60	240	15–19	1
L3	S	18	20–60	240	16–19	1
L4	C	35	10–40	200	1–24	24
L5	C	59	20–60	300	13–24	12

The historical data of the total energy charging demand of HEVs for half-a-year in the test system are used to train the SVR model with adjusting parameters fixed by the MDA [43]. Therefore, the total charging demand of the HEVs over 24 h is predicted using the proposed SVR-MDA based model [44]. To have a fair comparison, the prediction is made over 10 days with the use of some of the most well-known methods such as autoregressive–moving-average (ARMA), artificial neural network (ANN), SVR and SVR-DA (without modifications in MDA) [45]. Moreover, the prediction accuracy is calculated using three key criteria of mean absolute percentage error (MAPE), mean absolute relative percentage error (MARPE) and root mean square error (RMSE) as below [46]:

$${\sigma }_i\%=\frac{\left|{\tilde{y}}_i-y_i\right|}{y_i}\times 100,i=1,2,\dots ,N_v$$

(57)

$$MAPE\%=\frac{1}{N_v}{\displaystyle \sum _{i=1}^{N_v}{\sigma }_i}$$

(58)

$$RMSE=\sqrt{\frac{1}{N_v}{\displaystyle \sum _{i=1}^{N_v}{\sigma }_i^2}}$$

(59)

$$MARPE=max(100\times \frac{\left|{\tilde{y}}_i-y_i\right|}{y_i}),i=1,2,\dots ,N_v$$

(60)

where N_v shows the number of forecast samples and ${\tilde{y}}_i/y_i$ is the forecast/real value of the total charging demand sample of HEV [47]. Table 6 shows the prediction using different methods. According to these results, the proposed hybrid method based on SVR-MDA could provide much higher accuracy in terms of MAPE, MARPE and RMSE concurrently. This shows not only the high accuracy of the proposed prediction model for estimating the total charging demand of the HEVs, but also shows the appropriate search ability of the MDA by effective adjustment of the parameters in SVR. The superiority of the search optimization algorithm of MDA over the original version of DA can be deduced from the low values of MAPE, RMSE and MARPE [48].

Table 6. Forecasting total charging demand of HEVs using different methods.

Method	MAPE (%)	MARPE	RMSE
ARMA	2.08753	6.25845	2.63238
ANN	2.26421	6.56019	2.86453
SVR	1.36508	3.40989	1.45778
SVR-DA	1.16523	3.11277	1.44568
Proposed SVR-MDA Method	0.97752	1.88198	1.04481

In order to compare different charging strategies, the total load demand is distributed in both coordinated and intelligent strategies to assess their effects on the system (coinciding with the peak load hours). Figure 6 shows the HEV charging demand curves, comparatively. In order to have a better perception of the system energy demand without HEVs and DGs, Figure 7 is also shown here. From Figure 6 it is easily deduced that the intelligent charging is shifting the load demand based on the hourly peak load and price. In fact, the intelligent charging pattern considers not only the peak load hours, but also the market price, which will result in a wise and economic distribution of the power demand by these technologies. This is so important that without it, unexpected costs due to the coinciding of the HEV demand with the static load demands of the microgrid may happen. Figure 8 shows the hourly electricity market price over 24 h.

Figure 6. Comparative curves of coordinated and intelligent charging patterns.

Figure 7. Total energy consumption value in the microgrid in the base case (ignoring HEVs’ demand and DGs).

Figure 8. Hourly market price for the microgrid.

In order to assess the high search features of the proposed MDA, its performance is compared with other algorithms and the cost function results for 24 h of scheduling are given in Table 7. In order to make a fair comparison, algorithms of particle swarm optimization (PSO) [49,50], genetic algorithm (GA) [51,52] and original DA are deployed to solve the optimal energy management problem in the microgrid for 20 trails and the results of the best, worst, standard deviation and average values are provided in the table. According to these results, the proposed MDA shows more qualified results over the GA, PSO and DA in terms of finding the more optimal solution. It is seen that even the worst solution found by the MDA still is better than the best solution found by the GA. The very close value of the average index to the best solution shows the reliable optimization process [53]. The CPU time is also in seconds, which is simulated using a 2.4-GHz personal computer using MATLAB 2016.b. From the computational efforts, it is deduced that the proposed MDA could find the optimal power sharing and unit scheduling in a lower time with fewer efforts. This advocates the high efficiency and appropriate search ability of the MDA. From the standard deviation point of view, the lower value of this index for the proposed method shows the high robustness of the optimization algorithm [54]. In order to make a clear picture of the convergence capability of the different algorithms, Figure 9 is provided here. In this figure, the convergence response of the MDA, DA, PSO and GA are compared over a hundred iterations. According to the curves, the proposed MDA shows a higher performance for finding the optimal solution in the first iterations. Other algorithms have trapped in the local optima. The reliable performance and appropriate search ability can be deduced from these results as well.

Table 7. Robustness and searching features of different optimization algorithms: minimizing total microgrid cost over 20 trails for 24 h of operation.

Method	Cost ($) (×105)
	Average	Worst	Best	Standard Deviation	CPU Time (s)
GA	6.6927	7.8865	6.5364	0.1524	17.269
PSO	6.5378	7.9124	6.4473	0.1325	15.703
Original DA	6.5085	7.5295	6.4527	0.1358	14.275
Proposed MDA	6.4163	6.5946	6.2653	0.1063	13.276

Figure 9. Comparison of the convergence curves of different optimization methods for optimizing the total cost function value.

Table 8 shows the optimal output power of the solar panel, wind turbine and charging demand of the HEVs over 24 h using the proposed method. According to these results, it is seen that all renewable units are producing as the predicted results. Moreover, the HEVs are charging based on the coordinated and smart charging demands.

Table 8. Output power of wind units and solar panels along with the charging demands of HEVs.

Wind Turbine 1 (kW)	Wind Turbine 2 (kW)	Solar Panel 1 (kW)	Solar Panel 1 (kW)	HEV Charging Demand (Coordinate)	HEV Charging Demand (Intelligent)
16.898	29.7500	0	0	233.02	275.78
16.898	29.7500	0	0	106.52	289.04
16.898	29.7500	0	0	77.76	227.24
16.898	29.7500	0	0	72.00	210.00
16.898	29.7500	0	0	60.48	159.68
8.662	15.2500	0	0	50.40	136.68
16.898	29.7500	0	0	40.32	84.92
12.354	21.7500	1.6000	1.6000	28.80	53.28
16.898	29.7500	30.000	30.000	20.16	43.20
29.252	51.5000	60.200	60.200	11.52	41.74
83.070	146.250	83.600	83.600	8.64	38.86
98.548	173.5000	95.600	95.600	2.88	23.04
37.062	65.2500	191.20	191.20	1.44	20.16
22.436	39.5000	168.40	168.40	0	12.96
16.898	29.7500	63.000	63.000	0	7.20
12.354	21.7500	33.8000	33.8000	0	2.88
16.898	29.7500	4.4000	4.4000	0	0
16.898	29.7500	0	0	0	4.32
12.325	21.7000	0	0	0	17.26
16.898	29.7500	0	0	0	20.14
12.311	21.6750	0	0	113.6	50.34
12.311	21.6750	0	0	432.78	152.42
8.6620	15.2500	0	0	578.00	240.10
5.8220	10.2500	0	0	440.02	167.10

In order to make a better comparison, the microgrid hourly cost over 24 h before and after switching is provided in Figure 10. According to the results in Figure 10, the use of network reconfiguration could help significantly in reducing the losses and providing better dispatch of units [55,56]. Also, the system is experiencing lower costs in every hour, showing the necessity of switching. These results clearly prove the high quality act of the proposed model for renewable microgrids [57,58].

Figure 10. Hourly cost of the microgrid before and after optimal switching operation.

Table 9 shows the optimal scheduling of the storage unit and the adjustable loads in the microgrid. As it can be seen in this table, the battery is charging at the first hours of the day (negative values) and discharge at later hours (positive values), which would reduce the total microgrid operation cost. The same pattern is repeated in the afternoon, when the battery is charging to be able to discharge at the end of the day. It is also seen that the adjustable loads are scheduled properly according to the time slots that are available for scheduling. Finally, Table 10 shows the optimal topology of the microgrid according to the active operating switches of tie and sectionalizing. As it can be seen from these results, almost half of the switches are engaged in the scheduling problem, which can affect the optimal cost function value effectively.

Table 9. Optimal status of battery storage and adjustable loads over 24 h of scheduling.

Storage	−1	−1	−1	−1	−1	1	1	1	1	1	−1	−1	−1	−1	−1	−1	1	1	1	1	1
L1	0	0	0	0	0	0	0	0	0	0	1	1	1	1	0	0	0	0	0	0	0
L2	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	1	1	1	1	0	0
L3	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	1	1	1	0	0
L4	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1
L5	0	0	0	0	0	0	0	0	0	0	0	0	1	1	1	1	1	1	1	1	1

Table 10. Optimal switching pattern in the microgrid.

Hours	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24
Sectionalizing Switches	4	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0
	5	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0
	6	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
	7	0	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0	0	0	0	0	0
	10	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	1	0
	11	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0
	13	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0
	18	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	1	0	1	0	0
	20	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0
	23	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0
	26	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
	29	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	1	0	1	0	0	0	0	0
	30	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0
	33	1	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	1	0	0	0	0	0	0
	36	0	0	0	0	0	1	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0
	38	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
	47	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
	49	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	1	0	0	0	0	0	0	0
	52	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0
	57	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0
Tie Switches	69	1	0	1	0	1	0	1	1	1	0	0	1	1	1	0	1	1	1	1	0	1	0	1
	70	0	1	0	1	1	0	0	1	1	1	1	1	0	0	1	0	0	0	1	1	1	0	1
	71	1	1	1	1	1	1	1	0	0	1	1	0	1	1	1	0	1	1	1	1	1	1	1
	72	1	0	0	1	0	1	1	1	1	0	1	0	1	0	1	0	0	1	0	1	0	1	1
	73	1	1	1	1	0	1	0	1	0	1	1	0	1	1	1	1	1	1	1	1	1	1	0

In order to better perceive the effect of intelligent charging versus coordinated charging of the HEV power demand, Figure 11 shows the operation cost of the microgrids in the two scenarios. As it can be seen from the figure, the intelligent charging could reduce the total operation cost by shifting the HEV demand to off peak hours [59]. This can not only reduce the total microgrid costs, but can also benefit the microgrid technically by correcting the voltage profile and releasing the possible feeder congestion in the system [60].

Figure 11. Comparative operation cost of the microgrid in the coordinated and intelligent charging patterns.

6. Conclusions

This article focused on modeling the renewable microgrids and proposed a machine-learning based structure for optimal operation of the microgrids with a comprehensive model, which consists of the SVR, MDA along with smart charging of HEVs for arranging the switching patterns and managing the power dispatch of units. The simulation results on a test renewable microgrid showed that the proposed method is capable of solving the problem with high accuracy. The prediction model based on SVR-MDA could forecast the total charging demand of HEVs with higher accuracy compared to the ARMA, ANN, original SVR and SVR-DA. To this end, three indices of MAPE, RMSE and MARPE were deployed and compared for all methods. Having a practical estimation of the total charging demand of the HEVs, the proposed coordinated charging scheme can provide a more economic solution for the microgrid by shifting the demand to off peak hours. The proposed MDA could reduce the best optimal solution for the total microgrid operation cost by 4.146%, 2.823% and 2.904% compared to the GA, PSO and original DA. Moreover, it shows the proposed optimization method has a 23.124%, 15.456% and 6.998% reduction in the CPU time compared to the GA, PSO and original DA. It was seen that the wind and solar units play a critical role in mitigating the system costs when it is joint with optimal switching and reconfiguration of the feeders. Moreover, the optimal switching can help not only in reducing the costs but also in diminishing the power losses. This is due to the change of power flow in the feeders after switching, which can transfer the power flow burden from a heavy loaded feeder into another feeder with less power flow. Therefore, the power loss would be reduced in the feeders, effectively. Considering the cost objective function as the main goal to be minimized, reducing the power losses would be an indirect result, which is attained using the proposed problem formulation. Due to the single-agent structure of the proposed problem formulation, it is only suitable for the centralized frameworks and thus it is not possible to use it in a decentralized structure. It is clear that a multi-agent formulation is needed for this task, which will be addressed in future works.