Artificial Intelligence-Based Control and Coordination of Multiple PV Inverters for Reactive Power/Voltage Control of Power Distribution Networks
Abstract
:1. Introduction
1.1. General
1.2. Research Contribution
- Artificial Intelligence-based control and coordination;
- Provision of improved voltage controllable ratios;
- Realizes effective communication that leads to achieving minimum power losses;
- System achieves a 65% voltage controllable ratio;
- Uncontrollable voltage ratio is reduced to 0.0275.
2. Decentralized and Distributed Control Scheme
2.1. Decentralized Control Scheme
2.2. Distributed Control Scheme
3. Proposed Framework
3.1. Policy Function
3.2. Value Function
3.3. Multi Agent Actor Critic
3.4. Algorithm of Multi-Agent Actor Critic
Algorithm 1: Multi-Agent Actor Critic. |
Randomly Initialize N agents |
Initialize reply buffer D |
for episode one to the maximum episode do |
Reset network environment, get initial observation |
for step = 1 to tmax do |
Each agent selects the action |
Execute the action and get new observations o’ and the reward r from the environment |
Store transition to reply buffer |
end for |
for agent I = 1 to N do |
Sample mini batch from the buffer |
Update critic |
Update actor |
end for |
Update target network parameters of each agent |
end for |
3.5. Update Actor and Critic Network
Algorithm 2: Actor and Critic Network. |
Function: UPDATE CRITIC |
Sample Mini batch |
Calculateandby the target network |
Calculateand update critic using Adam optimizer |
End Function. |
Function: UPDATE CRITIC |
Calculatefor each agent |
Updateusing Adam optimizer |
End Function |
3.6. Flow Chart
4. Results and Discussion
4.1. Network Topology
4.2. Active Power of Integrated PVs
4.3. Reactive Power of Integrated PVs
4.4. Voltage of IEEE-33 Bus System
4.5. Losses of IEEE-33 Bus System
4.6. Comparison of Decentralized and Distributed Control Scheme
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Jamil, I.; Zhao, J.; Zhang, L.; Jamil, R.; Rafique, S.F. Evaluation of energy production and energy yield assessment based on feasibility, design, and execution of 3 × 50 MW grid-connected solar PV pilot project in Nooriabad. Int. J. Photoenergy 2017, 2017, 6429581. [Google Scholar] [CrossRef]
- Ceylan, O.; Paudyal, S.; Pisicay, I. Analysis of Local and Centralized Control of PV Inverters for Voltage Support in Distribution Feeders. In Proceedings of the 2021 IEEE Power & Energy Society General Meeting (PESGM), Washington, DC, USA, 25–29 July 2021; pp. 1–5. [Google Scholar] [CrossRef]
- Harrold, D.J.; Cao, J.; Fan, Z. Renewable energy integration and microgrid energy trading using multi-agent deep reinforcement learning. Appl. Energy 2022, 318, 119151. [Google Scholar] [CrossRef]
- Harrold, D.J.; Cao, J.; Fan, Z. Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning. Energy 2021, 238, 121958. [Google Scholar] [CrossRef]
- Iqbal, S.; Jan, M.U.; Rehman, A.U.; Rehman, A.U.; Shafique, A.; Rehman, H.U.; Aurangzeb, M. Feasibility Study and Deployment of Solar Photovoltaic System to Enhance Energy Economics of King Abdullah Campus, University of Azad Jammu and Kashmir Muzaffarabad, AJK Pakistan. IEEE Access 2022, 10, 5440–5455. [Google Scholar] [CrossRef]
- Cao, D.; Hu, W.; Zhao, J.; Huang, Q.; Chen, Z.; Blaabjerg, F. A Multi-Agent Deep Reinforcement Learning Based Voltage Regulation Using Coordinated PV Inverters. IEEE Trans. Power Syst. 2020, 35, 4120–4123. [Google Scholar] [CrossRef]
- Cao, D.; Zhao, J.; Hu, W.; Ding, F.; Huang, Q.; Chen, Z.; Blaabjerg, F. Data-Driven Multi-Agent Deep Reinforcement Learning for Distribution System Decentralized Voltage Control With High Penetration of PVs. IEEE Trans. Smart Grid 2021, 12, 4137–4150. [Google Scholar] [CrossRef]
- Gao, Y.; Wang, W.; Yu, N. Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks. IEEE Trans. Smart Grid 2021, 12, 3594–3604. [Google Scholar] [CrossRef]
- Ji, Y.; Wang, J.; Xu, J.; Fang, X.; Zhang, H. Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning. Energies 2019, 12, 2291. [Google Scholar] [CrossRef]
- Ali, K.H.; Sigalo, M.; Das, S.; Anderlini, E.; Tahir, A.A.; Abusara, M. Reinforcement Learning for Energy-Storage Systems in Grid-Connected Microgrids: An Investigation of Online vs. Offline Implementation. Energies 2021, 14, 5688. [Google Scholar] [CrossRef]
- Iqbal, S.; Xin, A.; Jan, M.U.; Abdelbaky, M.A.; Rehman, H.U.; Salman, S.; Aurangzeb, M.; Rizvi, S.A.A.; Shah, N.A. Improvement of Power Converters Performance by an Efficient Use of Dead Time Compensation Technique. Appl. Sci. 2020, 10, 3121. [Google Scholar] [CrossRef]
- Mosa, M.A.; Ali, A. Energy management system of low voltage dc microgrid using mixed-integer nonlinear programing and a global optimization technique. Electr. Power Syst. Res. 2020, 192, 106971. [Google Scholar] [CrossRef]
- Muriithi, G.; Chowdhury, S. Optimal Energy Management of a Grid-Tied Solar PV-Battery Microgrid: A Reinforcement Learning Approach. Energies 2021, 14, 2700. [Google Scholar] [CrossRef]
- Cosic, A.; Stadler, M.; Mansoor, M.; Zellinger, M. Mixed-integer linear programming based optimization strategies for renewable energy communities. Energy 2021, 237, 121559. [Google Scholar] [CrossRef]
- Iqbal, S.; Habib, S.; Khan, N.H.; Ali, M.; Aurangzeb, M.; Ahmed, E.M. Electric Vehicles Aggregation for Frequency Control of Microgrid under Various Operation Conditions Using an Optimal Coordinated Strategy. Sustainability 2022, 14, 3108. [Google Scholar] [CrossRef]
- Nakabi, T.A.; Toivanen, P. Deep reinforcement learning for energy management in a microgrid with flexible demand. Sustain. Energy Grids Netw. 2020, 25, 100413. [Google Scholar] [CrossRef]
- Kathirgamanathan, A.; Mangina, E.; Finn, D.P. Development of a Soft Actor Critic deep reinforcement learning approach for harnessing energy flexibility in a Large Office building. Energy AI 2021, 5, 100101. [Google Scholar] [CrossRef]
- Suanpang, P.; Jamjuntr, P.; Jermsittiparsert, K.; Kaewyong, P. Autonomous Energy Management by Applying Deep Q-Learning to Enhance Sustainability in Smart Tourism Cities. Energies 2022, 15, 1906. [Google Scholar] [CrossRef]
- Aurangzeb, M.; Xin, A.; Iqbal, S.; Jan, M.U. An Evaluation of Flux-Coupling Type SFCL Placement in Hybrid Grid System Based on Power Quality Risk Index. IEEE Access 2020, 8, 98800–98809. [Google Scholar] [CrossRef]
- Li, Z.; Yan, Y.; Qi, D.; Yan, S.; Wang, M. Distributed Voltage Optimization Control of BESS in AC Distribution Networks with High PV Penetration. Energies 2022, 15, 4120. [Google Scholar] [CrossRef]
- Iqbal, S.; Xin, A.; Jan, M.U.; Abdelbaky, M.A.; Rehman, H.U.; Salman, S.; Rizvi, S.A.A.; Aurangzeb, M. Aggregation of EVs for Primary Frequency Control of an Industrial Microgrid by Implementing Grid Regulation & Charger Controller. IEEE Access 2020, 8, 141977–141989. [Google Scholar] [CrossRef]
- Guo, X.; Zhao, Q.; Wang, S.; Shan, D.; Gong, W. A Short-Term Load Forecasting Model of LSTM Neural Network considering Demand Response. Complexity 2021, 2021, 5571539. [Google Scholar] [CrossRef]
- Muriithi, G.; Chowdhury, S. Deep Q-network application for optimal energy management in a grid-tied solar PV-Battery microgrid. J. Eng. 2022, 2022, 422–441. [Google Scholar] [CrossRef]
- Touzani, S.; Prakash, A.K.; Wang, Z.; Agarwal, S.; Pritoni, M.; Kiran, M.; Brown, R.; Granderson, J. Controlling distributed energy resources via deep reinforcement learning for load flexibility and energy efficiency. Appl. Energy 2021, 304, 117733. [Google Scholar] [CrossRef]
- Domínguez-Barbero, D.; García-González, J.; Sanz-Bobi, M.A.; Sánchez-Úbeda, E.F. Optimising a Microgrid System by Deep Reinforcement Learning Techniques. Energies 2020, 13, 2830. [Google Scholar] [CrossRef]
- Nespoli, A.; Ogliari, E.; Pretto, S.; Gavazzeni, M.; Vigani, S.; Paccanelli, F. Electrical Load Forecast by Means of LSTM: The Impact of Data Quality. Forecasting 2021, 3, 91–102. [Google Scholar] [CrossRef]
- Yang, B.; Sun, S.; Li, J.; Lin, X.; Tian, Y. Traffic flow prediction using LSTM with feature enhancement. Neurocomputing 2019, 332, 320–327. [Google Scholar] [CrossRef]
- Gao, G.; Wen, Y.; Tao, D. Distributed energy trading and scheduling among microgrids via multiagent reinforcement learning. IEEE Trans. Neural Netw. Learn. Syst. 2022. [Google Scholar] [CrossRef]
- Choi, Y.-A.; Park, S.-J.; Jun, J.-A.; Pyo, C.-S.; Cho, K.-H.; Lee, H.-S.; Yu, J.-H. Deep Learning-Based Stroke Disease Prediction System Using Real-Time Bio Signals. Sensors 2021, 21, 4269. [Google Scholar] [CrossRef]
Active Power (MW) | PV-1 | PV-2 | PV-3 | PV-4 | PV-5 | PV-6 | Total (MW) |
---|---|---|---|---|---|---|---|
Summer | 0.3068 | 0.3068 | 0.3970 | 0.5191 | 0.0532 | 0.0532 | 1.6361 |
Winter | 0.1744 | 0.1744 | 0.2158 | 0.3194 | 0.0226 | 0.0226 | 0.9292 |
Reactive Power (MVAR) | PV-1 | PV-2 | PV-3 | PV-4 | PV-5 | PV-6 | Total (MVAR) |
---|---|---|---|---|---|---|---|
Summer | 0.1198 | 0.7202 | −0.1307 | −0.1253 | 0.1993 | 0.1360 | 0.9193 |
Winter | 0.3390 | 0.8119 | −0.0814 | 0.0090 | 0.2071 | 0.1321 | 1.4177 |
Reactive Power (MVAR) | PV-1 | PV-2 | PV-3 | PV-4 | PV-5 | PV-6 | Total (MVAR) |
---|---|---|---|---|---|---|---|
Summer | 0.4125 | 0.4834 | −0.0133 | −0.3231 | 0.0874 | 0.1393 | 0.7866 |
Winter | 0.3474 | 0.2558 | −0.0208 | 0.0095 | 0.0194 | 0.0658 | 0.6771 |
System (Algorithm) | Voltage (p.u) | Line Losses (MW) | Average Voltage Deviation Ratio | Mean Test Voltage Controllable Ratio | Mean Test Voltage Out of Control Ratio |
---|---|---|---|---|---|
Decentralized | 0.9980 | 0.1345 | 0.01362 | 0.6850 | 0.0275 |
Distributed | 0.9972 | 0.1291 | 0.01516 | 0.6508 | 0.0523 |
System (Algorithm) | Voltage (p.u) | Line Losses (MW) | Average Voltage Deviation Ratio | Mean Test Voltage Controllable Ratio | Mean Test Voltage Out of Control Ratio |
---|---|---|---|---|---|
Decentralized | 1.003 | 0.1288 | 0.01362 | 0.6850 | 0.0275 |
Distributed | 0.9949 | 0.0488 | 0.01516 | 0.6508 | 0.0523 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Rehman, A.u.; Ali, M.; Iqbal, S.; Shafiq, A.; Ullah, N.; Otaibi, S.A. Artificial Intelligence-Based Control and Coordination of Multiple PV Inverters for Reactive Power/Voltage Control of Power Distribution Networks. Energies 2022, 15, 6297. https://doi.org/10.3390/en15176297
Rehman Au, Ali M, Iqbal S, Shafiq A, Ullah N, Otaibi SA. Artificial Intelligence-Based Control and Coordination of Multiple PV Inverters for Reactive Power/Voltage Control of Power Distribution Networks. Energies. 2022; 15(17):6297. https://doi.org/10.3390/en15176297
Chicago/Turabian StyleRehman, Anis ur, Muhammad Ali, Sheeraz Iqbal, Aqib Shafiq, Nasim Ullah, and Sattam Al Otaibi. 2022. "Artificial Intelligence-Based Control and Coordination of Multiple PV Inverters for Reactive Power/Voltage Control of Power Distribution Networks" Energies 15, no. 17: 6297. https://doi.org/10.3390/en15176297
APA StyleRehman, A. u., Ali, M., Iqbal, S., Shafiq, A., Ullah, N., & Otaibi, S. A. (2022). Artificial Intelligence-Based Control and Coordination of Multiple PV Inverters for Reactive Power/Voltage Control of Power Distribution Networks. Energies, 15(17), 6297. https://doi.org/10.3390/en15176297