# A Deep Reinforcement Learning Method for Economic Power Dispatch of Microgrid in OPAL-RT Environment

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Microgrid Model

#### 2.1. Conventional Power Generator

#### 2.1.1. Diesel Generator

#### 2.1.2. Gas Turbine Generator

#### 2.2. Renewable Power Generator and Loads

#### 2.3. BESS

#### 2.4. Electricity Trade and Power Balance

#### 2.4.1. Purchase/Sell Electricity from/to Main Grid

#### 2.4.2. Ancillary Services

#### 2.4.3. Power Balance

## 3. Proposed Methodology

#### 3.1. Deep Reinforcement Learning

#### 3.2. MDP Model

- System sate

- Action

- Transition probability

- Reward function

#### 3.3. Proposed DDPG with LSTM Method

#### 3.3.1. LSTM Network

#### 3.3.2. Procedure of DDPG with LSTM Method

Algorithm 1: Proposed DDPG with LSTM method |

1. Initialize and train the LSTM network with ${x}_{t}$ = (${D}_{t}$, ${P}_{t}^{d}$). 2. Randomly initialize evaluated actor network, $\mu \left(s|{\theta}^{\mu}\right)$, and critic network, $Q\left(s,a|{\theta}^{Q}\right)$, with weights ${\theta}^{\mu}$ and ${\theta}^{Q}$. 3. Initialize target actor and critic network ${\theta}^{{\mu}^{\prime}}$ and ${\theta}^{{Q}^{\prime}}$ with weights ${\theta}^{{Q}^{\prime}}\leftarrow {\theta}^{Q}$ and ${\theta}^{{\mu}^{\prime}}\leftarrow {\theta}^{\mu}$. 4. Initialize experience replay buffer, $B.$ 5. For episode $E$ = 1 to $M$, carry out the following. 6. Initialize the Gaussian exploration noise, ${N}_{t}\left(\mathrm{0,1}\right)$. 7. Receive the initial observation state ${s}_{1}$. 8. For time step t = 1 to $T$, carry out the following: 9. Select action ${a}_{t}=\mu \left({s}_{t}|{\theta}^{\mu}\right)$ with ${N}_{t}$ for the exploration noise. 10. Execute action ${a}_{t}$ in the system environment with the load demand forecast from the LSTM network and observe reward ${r}_{t}$ and new state ${s}_{t+1}$. 11. Store transition $({s}_{t},{a}_{t},{r}_{t},{s}_{t+1})$ into experience replay buffer $B$. 12. Sample a random minibatch of the size N transition, $({s}_{i},{a}_{i},{r}_{i},{s}_{i+1})$, from experience replay buffer $B.$ 13. Set ${y}_{i}={r}_{i}+\gamma {Q}^{\prime}\left({s}_{i+1},{\mu}^{\prime}\left({s}_{i+1}|{\theta}^{{\mu}^{\prime}}\right)|{\theta}^{{Q}^{\prime}}\right)$ 14. Update the evaluated critic network by minimizing the loss, L, as follows: $L=\frac{1}{N}{\sum}_{i}{\left({y}_{i}-Q\left({s}_{i},{a}_{i}|{\theta}^{Q}\right)\right)}^{2}$ ${\theta}^{Q}\leftarrow {\theta}^{Q}+{\gamma}^{Q}{\nabla}_{{\theta}^{Q}}L$ 15. Update the evaluated actor policy using the sampled policy gradient: ${\nabla}_{{\theta}^{\mu}}J\approx \frac{1}{N}\sum _{i}{\nabla}_{a}Q\left(s,a|{\theta}^{Q}\right){|}_{s={s}_{i},a=\mu \left({s}_{i}\right)}{\nabla}_{{\theta}^{\mu}}\mu \left(s|{\theta}^{\mu}\right){|}_{{s}_{i}}$ ${\theta}^{\mu}\leftarrow {\theta}^{\mu}+{\gamma}^{\mu}{\nabla}_{{\theta}^{\mu}}J$ 16. Softly update the target actor and critic network with the update rate, $\tau $, as follows: $\left\{\begin{array}{c}{\theta}^{{Q}^{\prime}}\leftarrow \tau {\theta}^{Q}+(1-\tau ){\theta}^{{Q}^{\prime}}\\ {\theta}^{{\mu}^{\prime}}\leftarrow \tau {\theta}^{\mu}+(1-\tau ){\theta}^{{\mu}^{\prime}}\end{array}\right.$ 17. End for. 18. End for. |

## 4. Benchmarking

- Set the initial values

- Check the purchasing price of the main grid.Check if the purchasing price of the main grid is cheaper than the average purchasing price.
- Make the decision.If the purchasing price of the main grid is cheaper than the average purchasing price, the BESS will charge at 100 kW; otherwise, the BESS will discharge at 100 kW.Check the SOC and decide the power output of the BESS.
- The SOC of the BESS is set to be 10% ≦ ${S}_{t}^{b}$ ≦ 100%. If the SOC is not within the range of the constraint conditions, the BESS does not charge or discharge; otherwise, the BESS follows the decision to charge or discharge. Moreover, the SOC of the BESS will carry on over to the next hour and proceed to the next optimization calculation.

#### 4.1. Experience-Based EMS

- Set the initial values.Import the predictive load of the microgrid, the value of PV, the value of WTG, and the power output of the BESS.
- Check the purchasing price of main grid.Examine the purchasing price of main grid, and then calculate the fuel cost of the diesel generator and that of the gas turbine generator separately. Check if the buying price of the main grid is between the fuel cost of the diesel generator and that of the gas turbine generator.
- Make the decision.If the buying price of the main grid is between the fuel cost of the diesel generator and that of the gas turbine generator or the buying price of the main grid is the most expensive, then the output powers will be shared by the gas turbine and diesel generators; otherwise, the main grid will predominantly supply the load of the microgrid.
- Output the power of generator.The results of the EMS consist of the power commands of the diesel generator, gas turbine generator and the BESS and then are sent to the individual control blocks.

#### 4.2. Newton-PSO

#### 4.3. DQN

#### 4.4. Performance Comparison

## 5. Experimentation

#### 5.1. OPAL-RT Environment and Implementation

#### 5.2. Emulation Result

## 6. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Bani-Ahmed, A.; Rashidi, M.; Nasiri, A.; Hosseini, H. Reliability Analysis of a Decentralized Microgrid Control Architecture. IEEE Trans. Smart Grid
**2018**, 10, 3910–3918. [Google Scholar] [CrossRef] - Yang, P.; Yu, M.; Wu, Q.; Hatziargyriou, N.; Xia, Y.; Wei, W. Decentralized Bidirectional Voltage Supporting Control for Mul-ti-Mode Hybrid AC/DC Microgrid. IEEE Trans. Smart Grid
**2020**, 11, 2615–2626. [Google Scholar] [CrossRef] - Li, C.; de Bosio, F.; Chen, F.; Chaudhary, S.K.; Vasquez, J.C.; Guerrero, J.M. Economic Dispatch for Operating Cost Minimization Under Real-Time Pricing in Droop-Controlled DC Microgrid. IEEE J. Emerg. Sel. Top. Power Electron.
**2017**, 5, 587–595. [Google Scholar] [CrossRef] [Green Version] - Anglani, N.; Oriti, G.; Colombini, M. Optimized Energy Management System to Reduce Fuel Consumption in Remote Military Microgrids. IEEE Trans. Ind. Appl.
**2017**, 53, 5777–5785. [Google Scholar] [CrossRef] - Zia, M.F.; Elbouchikhi, E.; Benbouzid, M.; Guerrero, J.M. Energy Management System for an Islanded Microgrid with Convex Relaxation. IEEE Trans. Ind. Appl.
**2019**, 55, 7175–7185. [Google Scholar] [CrossRef] - Paul, T.G.; Hossain, S.J.; Ghosh, S.; Mandal, P.; Kamalasadan, S. A Quadratic Programming Based Optimal Power and Battery Dispatch for Grid-Connected Microgrid. IEEE Trans. Ind. Appl.
**2018**, 54, 1793–1805. [Google Scholar] [CrossRef] - Liu, Y.; Wu, L.; Li, J. A Fast LP-Based Approach for Robust Dynamic Economic Dispatch Problem: A Feasible Region Projection Method. IEEE Trans. Power Syst.
**2020**, 35, 4116–4119. [Google Scholar] [CrossRef] - Lei, Y.; Liu, F.; Li, A.; Su, Y.; Yang, X.; Zheng, J. An Optimal Generation Scheduling Approach Based on Linear Relaxation and Mixed Integer Programming. IEEE Access
**2020**, 8, 168625–168630. [Google Scholar] [CrossRef] - Lin, C.; Wu, W.; Chen, X.; Zheng, W. Decentralized Dynamic Economic Dispatch for Integrated Transmission and Active Dis-tribution Networks Using Multi-Parametric Programming. IEEE Trans. Smart Grid
**2018**, 9, 4983–4993. [Google Scholar] [CrossRef] - Garcia-Torres, F.; Vilaplana, D.G.; Bordons, C.; Roncero-Sanchez, P.; Ridao, M.A. Optimal Management of Microgrids with External Agents Including Battery/Fuel Cell Electric Vehicles. IEEE Trans. Smart Grid
**2019**, 10, 4299–4308. [Google Scholar] [CrossRef] - Liu, C.; Ma, H.; Zhang, H.; Shi, X.; Shi, F. A MILP-Based Battery Degradation Model for Economic Scheduling of Power System. IEEE Trans. Sustain. Energy
**2023**, 14, 1000–1009. [Google Scholar] [CrossRef] - Kalakova, A.; Nunna, H.S.V.S.K.; Jamwal, P.K.; Doolla, S. A Novel Genetic Algorithm Based Dynamic Economic Dispatch with Short-Term Load Forecasting. IEEE Trans. Ind. Appl.
**2021**, 57, 2972–2982. [Google Scholar] [CrossRef] - Jordehi, A.R. A mixed binary-continuous particle swarm optimisation algorithm for unit commitment in microgrids considering uncertainties and emissions. Int. Trans. Electr. Energy Syst.
**2020**, 30, e12581. [Google Scholar] - Jordehi, A.R. An improved particle swarm optimisation for unit commitment in microgrids with battery energy storage systems considering battery degradation and uncertainties. Int. J. Energy Res.
**2021**, 45, 727–744. [Google Scholar] [CrossRef] - Ponciroli, R.; Stauff, N.E.; Ramsey, J.; Ganda, F.; Vilim, R.B. An Improved Genetic Algorithm Approach to the Unit Commit-ment/Economic Dispatch Problem. IEEE Trans. Power Syst.
**2020**, 35, 4005–4013. [Google Scholar] [CrossRef] - Abbas, G.; Gu, J.; Farooq, U.; Raza, A.; Asad, M.U.; El-Hawary, M.E. Solution of an Economic Dispatch Problem through Particle Swarm Optimization: A Detailed Survey—Part II. IEEE Access
**2017**, 5, 24426–24445. [Google Scholar] [CrossRef] - Raghav, L.P.; Kumar, R.S.; Raju, D.K.; Singh, A.R. Optimal Energy Management of Microgrids Using Quantum Teaching Learning Based Algorithm. IEEE Trans. Smart Grid
**2021**, 12, 4834–4842. [Google Scholar] [CrossRef] - Muzumdar, A.A.; Modi, C.N.; Vyjayanthi, C. Designing a Robust and Accurate Model for Consumer-Centric Short-Term Load Forecasting in Microgrid Environment. IEEE Syst. J.
**2022**, 16, 2448–2459. [Google Scholar] [CrossRef] - Trivedi, R.; Patra, S.; Khadem, S. A Data-Driven Short-Term PV Generation and Load Forecasting Approach for Microgrid Applications. IEEE J. Emerg. Sel. Top. Ind. Electron.
**2022**, 3, 911–919. [Google Scholar] [CrossRef] - Alavi, S.A.; Mehran, K.; Vahidinasab, V.; Catalao, J.P.S. Forecast-Based Consensus Control for DC Microgrids Using Distributed Long Short-Term Memory Deep Learning Models. IEEE Trans. Smart Grid
**2021**, 12, 3718–3730. [Google Scholar] [CrossRef] - Han, H.; Liu, H.; Zuo, X.; Shi, G.; Sun, Y.; Liu, Z.; Su, M. Optimal Sizing Considering Power Uncertainty and Power Supply Reliability Based on LSTM and MOPSO for SWPBMs. IEEE Syst. J.
**2022**, 16, 4013–4023. [Google Scholar] [CrossRef] - Zhao, J.; Li, F.; Mukherjee, S.; Sticht, C. Deep Reinforcement Learning-Based Model-Free On-Line Dynamic Multi-Microgrid Formation to Enhance Resilience. IEEE Trans. Smart Grid
**2022**, 13, 2557–2567. [Google Scholar] [CrossRef] - Zhang, H.; Yue, D.; Dou, C.; Hancke, G.P. PBI Based Multi-Objective Optimization via Deep Reinforcement Elite Learning Strategy for Micro-Grid Dispatch With Frequency Dynamics. IEEE Trans. Power Syst.
**2023**, 38, 488–498. [Google Scholar] [CrossRef] - Sharma, J.; Andersen, P.-A.; Granmo, O.-C.; Goodwin, M. Deep Q-Learning with Q-Matrix Transfer Learning for Novel Fire Evacuation Environment. IEEE Trans. Syst.
**2021**, 51, 7363–7381. [Google Scholar] [CrossRef] [Green Version] - Liu, Z.; Hou, L.; Zheng, K.; Zhou, Q.; Mao, S. A DQN-Based Consensus Mechanism for Blockchain in IoT Networks. IEEE Internet Things J.
**2022**, 9, 11962–11973. [Google Scholar] [CrossRef] - Xu, Y.-H.; Yang, C.-C.; Hua, M.; Zhou, W. Deep Deterministic Policy Gradient (DDPG)-Based Resource Allocation Scheme for NOMA Vehicular Communications. IEEE Access
**2020**, 8, 18797–18807. [Google Scholar] [CrossRef] - Zhang, M.; Zhang, Y.; Gao, Z.; He, X. An Improved DDPG and Its Application Based on the Double-Layer BP Neural Network. IEEE Access
**2020**, 8, 177734–177744. [Google Scholar] [CrossRef] - Ye, Y.; Qiu, D.; Sun, M.; Papadaskalopoulos, D.; Strbac, G. Deep Reinforcement Learning for Strategic Bidding in Electricity Markets. IEEE Trans. Smart Grid
**2020**, 11, 1343–1355. [Google Scholar] [CrossRef] - Lin, L.; Guan, X.; Peng, Y.; Wang, N.; Maharjan, S.; Ohtsuki, T. Deep Reinforcement Learning for Economic Dispatch of Virtual Power Plant in Internet of Energy. IEEE Internet Things J.
**2020**, 7, 6288–6301. [Google Scholar] [CrossRef] - Fang, D.; Guan, X.; Hu, B.; Peng, Y.; Chen, M.; Hwang, K. Deep Reinforcement Learning for Scenario-Based Robust Economic Dispatch Strategy in Internet of Energy. IEEE Internet Things J.
**2021**, 8, 9654–9663. [Google Scholar] [CrossRef] - Shan, X.; Xue, F. A Day-Ahead Economic Dispatch Scheme for Transmission System with High Penetration of Renewable Energy. IEEE Access
**2022**, 10, 11159–11172. [Google Scholar] [CrossRef] - Lin, F.-J.; Liao, J.-C.; Chen, C.-I.; Chen, P.-R.; Zhang, Y.-M. Voltage Restoration Control for Microgrid with Recurrent Wavelet Petri Fuzzy Neural Network. IEEE Access
**2022**, 10, 12510–12529. [Google Scholar] [CrossRef] - Fang, J.; Li, H.; Tang, Y.; Blaabjerg, F. Distributed Power System Virtual Inertia Implemented by Grid-Connected Power Con-verters. IEEE Trans. Power Electron.
**2018**, 33, 8488–8499. [Google Scholar] [CrossRef] [Green Version] - Tan, K.-H.; Lin, F.-J.; Shih, C.-M.; Kuo, C.-N. Intelligent Control of Microgrid with Virtual Inertia Using Recurrent Probabilistic Wavelet Fuzzy Neural Network. IEEE Trans. Power Electron.
**2020**, 35, 7451–7464. [Google Scholar] [CrossRef] - Gers, F.A.; Schmidhuber, J.; Cummins, F. Learning to Forget: Continual: Prediction with LSTM. Neural Comput.
**2000**, 12, 2451–2471. [Google Scholar] [CrossRef] - Khan, N.A.; Sidhu, G.A.S.; Gao, F. Optimizing Combined Emission Economic Dispatch for Solar Integrated Power Systems. IEEE Access
**2016**, 4, 3340–3348. [Google Scholar] [CrossRef] - Loyarte, A.S.; Clementi, L.A.; Vega, J.R. A Hybrid Methodology for a Contingency Constrained Economic Dispatch under High Variability in the Renewable Generation. IEEE Lat. Am. Trans.
**2019**, 17, 1715–1723. [Google Scholar] [CrossRef]

**Figure 2.**Piecewise solar power generation curve of PV system and power generation curve of WTG system.

**Figure 16.**Emulated responses of DDPG with LSTM at 7 a.m. in Case A. (

**a**) Power command of PV, WTG, and predictive load; (

**b**) power outputs of gas turbine and diesel generator; (

**c**) power flow in main grid and power output of BESS.

**Figure 17.**Emulated responses of DDPG with LSTM at 1 p.m. in Case B. (

**a**) Power command of PV, WTG, and predictive load; (

**b**) power outputs of gas turbine and diesel generator; (

**c**) power flow in main grid and power output of BESS.

**Figure 18.**Emulated responses of DDPG with LSTM at 10 p.m. in Case C. (

**a**) Power command of PV, WTG, and predictive load; (

**b**) power outputs of gas turbine and diesel generator; (

**c**) power flow in main grid and power output of BESS.

Plan | USD/kWh | Time | ||
---|---|---|---|---|

On-Peak | USD 0.207 | 13:00~17:00 | 18:00~20:00 | |

Half On-Peak | USD 0.207 | 7:00~13:00 | 17:00~18:00 | 20:00~22:00 |

Off-Peak | USD 0.06 | 0:00~7:00 | 22:00~24:00 |

Generator | P_{min} | P_{max} | a | b | c |
---|---|---|---|---|---|

Gas turbine | 60 kW | 1250 kW | 0.4969 | 0.0116 | 0.0001987 |

Diesel | 50 kW | 1250 kW | 18.3333 | 0.10157 | 0.000000661 |

Time (hr) | 0–1 | 1–2 | 2–3 | 3–4 | 4–5 | 5–6 | 6–7 | 7–8 | 8–9 | 9–10 | 10–11 | 11–12 | 12–13 | 13–14 | 14–15 | 15–16 | 16–17 | 17–18 | 18–19 | 19–20 | 20–21 | 21–22 | 22–23 | 23–24 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Cost (USD/hr) | 70.88 | 75.06 | 76.42 | 74.79 | 74.98 | 74.98 | 74.55 | 74.85 | 66.05 | 54.37 | 49.26 | 50.1 | 49.62 | 50.13 | 54.48 | 63.03 | 74.6 | 88.52 | 95.23 | 100.85 | 106.67 | 106.75 | 75.63 | 70.98 |

GT Generation (kW) | 60 | 61.6 | 60.45 | 113.2 | 107.1 | 81.2 | 62.76 | 202.75 | 224.98 | 256.88 | 247.12 | 244.03 | 252.89 | 171.9 | 195.83 | 206.64 | 215.53 | 205.09 | 199.69 | 229.16 | 232.76 | 307.54 | 64.64 | 115.36 |

DG Generation (kW) | 50 | 50 | 50 | 50.02 | 50.02 | 50.01 | 50.01 | 446.66 | 339.49 | 191.24 | 151.79 | 163.31 | 149.06 | 230.35 | 253.22 | 327.33 | 432.39 | 578.3 | 648.66 | 675.7 | 728.84 | 639.13 | 50 | 50.02 |

PV Prediction (kW) | 0 | 0 | 0 | 0 | 0 | 0 | 8.93 | 49.2 | 131.94 | 216.9 | 268.33 | 279.7 | 284 | 277.32 | 257.2 | 210.23 | 149.82 | 73.79 | 11.18 | 0 | 0 | 0 | 0 | 0 |

Grid (kW) | 759.38 | 827.98 | 851.38 | 783.64 | 792.46 | 813.56 | 818.74 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2.87 | 835.67 | 718.08 |

WTG Generation (kW) | 149.12 | 141.27 | 133.42 | 146.04 | 145.19 | 149.12 | 156.96 | 164.81 | 168.74 | 172.66 | 168.74 | 164.81 | 166.78 | 164.81 | 168.74 | 160.89 | 156.96 | 149.12 | 153.04 | 141.27 | 133.42 | 137.34 | 139.31 | 141.27 |

BESS Generation (kW) | $-$99.9 | $-$91.58 | $-$98.18 | $-$99.45 | $-$99.69 | $-$99.88 | $-$99.97 | 98.61 | 82.83 | 77.8 | 74.42 | 65.42 | 47.42 | 46.76 | 35.43 | 53.43 | 59.62 | 78.9 | 99.7 | 68.31 | $-$0.01 | $-$0.04 | 0.04 | $-$1.13 |

Electricity Price (USD/hr) | 0.06 | 0.06 | 0.06 | 0.06 | 0.06 | 0.06 | 0.06 | 0.133 | 0.133 | 0.133 | 0.133 | 0.133 | 0.133 | 0.207 | 0.207 | 0.207 | 0.207 | 0.133 | 0.207 | 0.207 | 0.133 | 0.133 | 0.06 | 0.06 |

SOC(%) | 39.99% | 49.15% | 58.97% | 68.91% | 78.88% | 88.87% | 98.87% | 89% | 80.72% | 72.94% | 65.5% | 58.96% | 54.22% | 49.54% | 46% | 40.65% | 34.69% | 26.8% | 16.83% | 10% | 10% | 10.01% | 10% | 10.11% |

Load Prediction (kW) | 918.6 | 989.27 | 997.07 | 993.45 | 995.08 | 994.01 | 997.43 | 962.03 | 947.98 | 915.48 | 910.4 | 917.27 | 900.15 | 891.14 | 910.42 | 958.52 | 1014.32 | 1085.2 | 1112.27 | 1114.44 | 1095.01 | 1086.84 | 1089.66 | 1023.6 |

Time (hr) | 0–1 | 1–2 | 2–3 | 3–4 | 4–5 | 5–6 | 6–7 | 7–8 | 8–9 | 9–10 | 10–11 | 11–12 | 12–13 | 13–14 | 14–15 | 15–16 | 16–17 | 17–18 | 18–19 | 19–20 | 20–21 | 21–22 | 22–23 | 23–24 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Cost (USD/hr) | 70.88 | 75.38 | 75.9 | 75.53 | 75.66 | 75.38 | 74.57 | 83.83 | 64.36 | 51.97 | 46.89 | 46.85 | 44.67 | 20.63 | 24.27 | 34.81 | 48.36 | 96.53 | 105.28 | 107.88 | 106.75 | 105.44 | 76.16 | 72.22 |

GT Generation (kW) | 60.01 | 68.98 | 97.3 | 61.47 | 62.24 | 61.37 | 61.71 | 258.4 | 241.33 | 212.63 | 187.19 | 186.46 | 175.19 | 225.89 | 232.71 | 229.32 | 239.94 | 242.81 | 231.65 | 241.26 | 209.01 | 221.13 | 67.41 | 99.58 |

DG Generation (kW) | 50 | 50 | 50 | 50 | 50 | 50 | 50 | 478.33 | 306.15 | 213.29 | 186.14 | 186.3 | 174.18 | 623.12 | 651.77 | 758.08 | 879.22 | 619.48 | 716.4 | 731.91 | 752.62 | 728.37 | 50.07 | 50.1 |

PV Prediction (kW) | 0 | 0 | 0 | 0 | 0 | 0 | 8.93 | 49.2 | 131.94 | 216.9 | 268.33 | 279.7 | 284 | 277.32 | 257.2 | 210.23 | 149.82 | 73.79 | 11.18 | 0 | 0 | 0 | 0 | 0 |

Grid (kW) | 759.38 | 828.71 | 816.28 | 835.91 | 837.64 | 833.52 | 819.83 | 0 | 0 | 0 | 0 | 0 | 0 | $-$500 | $-$500 | $-$500 | $-$500 | 0 | 0 | 0 | 0 | 0 | 842.63 | 752.88 |

WTG Generation (kW) | 149.12 | 141.27 | 133.42 | 146.04 | 145.19 | 149.12 | 156.96 | 164.81 | 168.74 | 172.66 | 168.74 | 164.81 | 166.78 | 164.81 | 168.74 | 160.89 | 156.96 | 149.12 | 153.04 | 141.27 | 133.42 | 137.34 | 139.31 | 141.27 |

BESS Generation (kW) | $-$99.91 | $-$99.69 | $-$99.93 | $-$99.97 | $-$99.99 | $-$100 | $-$100 | 11.29 | 99.82 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 88.38 | 0 | 0 | 0 | $-$0.04 | 0 | $-$9.76 | $-$20.23 |

Electricity Price (USD/hr) | 0.06 | 0.06 | 0.06 | 0.06 | 0.06 | 0.06 | 0.06 | 0.133 | 0.133 | 0.133 | 0.133 | 0.133 | 0.133 | 0.149 (selling) | 0.149 (selling) | 0.149 (selling) | 0.149 (selling) | 0.133 | 0.207 | 0.207 | 0.133 | 0.133 | 0.06 | 0.06 |

SOC(%) | 39.99% | 49.96% | 59.95% | 69.95% | 79.95% | 89.95% | 99.95% | 98.82% | 88.84% | 78.84% | 68.84% | 58.84% | 48.84% | 38.84% | 28.84% | 18.84% | 10% | 10% | 10% | 10% | 10% | 10% | 10.98% | 13% |

Load Prediction (kW) | 918.6 | 989.27 | 997.07 | 993.45 | 995.08 | 994.01 | 997.43 | 962.03 | 947.98 | 915.48 | 910.4 | 917.27 | 900.15 | 891.14 | 910.42 | 958.52 | 1014.32 | 1085.2 | 1112.27 | 1114.44 | 1095.01 | 1086.84 | 1089.66 | 1023.6 |

Time (hr) | 0–1 | 1–2 | 2–3 | 3–4 | 4–5 | 5–6 | 6–7 | 7–8 | 8–9 | 9–10 | 10–11 | 11–12 | 12–13 | 13–14 | 14–15 | 15–16 | 16–17 | 17–18 | 18–19 | 19–20 | 20–21 | 21–22 | 22–23 | 23–24 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Cost (USD/hr) | 70.87 | 75.43 | 76.54 | 75.37 | 75.6 | 75.37 | 74.61 | 79.66 | 78.71 | 62.43 | 53.94 | 52.98 | 53.76 | 46.44 | 50.49 | 59.85 | 71.16 | 86.77 | 177.85 | 183.04 | 83.54 | 85.8 | 75.75 | 72.53 |

GT Generation (kW) | 60 | 60 | 60 | 60 | 60 | 60 | 60 | 221.83 | 295.26 | 242.52 | 216.31 | 216.45 | 220.59 | 193.71 | 217.33 | 249.77 | 280.61 | 273.26 | 60 | 60 | 196.6 | 196.59 | 60 | 60 |

DG Generation (kW) | 50 | 50 | 50 | 50 | 50 | 50 | 50 | 475.9 | 384.33 | 285.98 | 229.08 | 219.55 | 223.24 | 176.15 | 194.29 | 252.92 | 328.41 | 490.08 | 50 | 50 | 50.01 | 50.02 | 50 | 50 |

PV Prediction (kW) | 0 | 0 | 0 | 0 | 0 | 0 | 8.93 | 49.2 | 131.94 | 216.9 | 268.33 | 279.7 | 284 | 277.32 | 257.2 | 210.23 | 149.82 | 73.79 | 11.18 | 0 | 0 | 0 | 0 | 0 |

Grid (kW) | 759.13 | 835.19 | 853.62 | 834.09 | 837.99 | 834.16 | 821.44 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 738.05 | 763.17 | 614.98 | 631.98 | 840.43 | 786.79 |

WTG Generation (kW) | 149.12 | 141.27 | 133.42 | 146.04 | 145.19 | 149.12 | 156.96 | 164.81 | 168.74 | 172.66 | 168.74 | 164.81 | 166.78 | 164.81 | 168.74 | 160.89 | 156.96 | 149.12 | 153.04 | 141.27 | 133.42 | 137.34 | 139.31 | 141.27 |

BESS Generation (kW) | $-$99.65 | $-$97.19 | $-$99.97 | $-$96.68 | $-$98.1 | $-$99.27 | $-$99.9 | 50.29 | $-$32.29 | $-$2.58 | 27.94 | 36.76 | 5.54 | 79.15 | 72.86 | 84.71 | 98.52 | 98.95 | 100 | 100 | 100 | 70.91 | $-$0.08 | $-$14.46 |

Electricity Price (USD/hr) | 0.06 | 0.06 | 0.06 | 0.06 | 0.06 | 0.06 | 0.06 | 0.133 | 0.133 | 0.133 | 0.133 | 0.133 | 0.133 | 0.207 | 0.207 | 0.207 | 0.207 | 0.133 | 0.207 | 0.207 | 0.237 (AS) | 0.237 (AS) | 0.06 | 0.06 |

SOC(%) | 39.97% | 49.68% | 59.68% | 69.35% | 79.16% | 89.09% | 99.08% | 94.05% | 97.28% | 97.53% | 94.74% | 91.06% | 90.51% | 82.6% | 75.31% | 66.84% | 56.99% | 47.09% | 37.09% | 27.09% | 17.09% | 10% | 10.01% | 11.45% |

Load Prediction (kW) | 918.6 | 989.27 | 997.07 | 993.45 | 995.08 | 994.01 | 997.43 | 962.03 | 947.98 | 915.48 | 910.4 | 917.27 | 900.15 | 891.14 | 910.42 | 958.52 | 1014.32 | 1085.2 | 1112.27 | 1114.44 | 1095.01 | 1086.84 | 1089.66 | 1023.6 |

Method | Experience-Based EMS | Newton-PSO | DQN | DDPG |
---|---|---|---|---|

Cost of Case A (USD) | 1830.78 (base) | 1770.17 ($-$3.31%) | 1769.75 ($-$3.33%) | 1752.78 ($-$4.26%) |

Cost of Case B (USD) | 1791.9 (base) | 1692.76 ($-$5.53%) | 1678.69 ($-$6.32%) | 1660.2 ($-$7.35%) |

Cost of Case C (USD) | 1973.79 (base) | 1941.28 ($-$1.65%) | 1964.38 ($-$0.48%) | 1898.49 ($-$3.81%) |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Lin, F.-J.; Chang, C.-F.; Huang, Y.-C.; Su, T.-M.
A Deep Reinforcement Learning Method for Economic Power Dispatch of Microgrid in OPAL-RT Environment. *Technologies* **2023**, *11*, 96.
https://doi.org/10.3390/technologies11040096

**AMA Style**

Lin F-J, Chang C-F, Huang Y-C, Su T-M.
A Deep Reinforcement Learning Method for Economic Power Dispatch of Microgrid in OPAL-RT Environment. *Technologies*. 2023; 11(4):96.
https://doi.org/10.3390/technologies11040096

**Chicago/Turabian Style**

Lin, Faa-Jeng, Chao-Fu Chang, Yu-Cheng Huang, and Tzu-Ming Su.
2023. "A Deep Reinforcement Learning Method for Economic Power Dispatch of Microgrid in OPAL-RT Environment" *Technologies* 11, no. 4: 96.
https://doi.org/10.3390/technologies11040096