Research on Long-Term Scheduling Optimization of Water–Wind–Solar Multi-Energy Complementary System Based on DDPG
Abstract
1. Introduction
- Regarding the complexity of high-dimensional uncertainty modeling:
- 2.
- Regarding the difficulty of solving in continuous action space:
2. Overall Framework
3. Hydro–Wind–Solar Scenario Generation
3.1. Hydro–Wind–Solar Complementary System Correlation Modeling
3.2. Scenario Generation for Multi-Energy Complementary Systems
3.3. Scenario Quality Evaluation
3.3.1. Correlation Metric
3.3.2. Randomness Metric
3.3.3. Volatility Metric
4. Model Construction
4.1. Objective Function
4.2. Constraints
5. Model Solution
5.1. Markov Decision Process Model Transformation
5.2. DDPG Algorithm Solution
6. Case Analysis
6.1. Dataset Analysis and Parameter Settings
6.2. Hydro–Wind Coupling Scenario Analysis
6.3. Result Analysis
6.3.1. Convergence Analysis
6.3.2. Scheduling Result Analysis
7. Conclusions
- The improved C-Vine Copula model is used to model the correlations within the hydro–wind–solar multi-energy complementary system. Combined with Latin Hypercube Sampling, the uncertainty of the system is quantified, while reducing model complexity. The resulting scenarios effectively reflect the historical runoff and the characteristic patterns of wind and solar power generation in the region.
- Compared to the DQN algorithm, the DDPG algorithm improves the model’s convergence speed and policy reward through an adaptive noise exploration mechanism and continuous action space optimization during the training phase. Additionally, through scheduling validation in typical hydrological scenarios, DDPG demonstrates stronger environmental adaptability and decision robustness in handling the complex continuous control problems of multi-energy complementary systems. Whether in the dry year scenario, where hydropower precisely compensates for wind and solar fluctuations, or in the wet year scenario, where curtailment and water level thresholds are actively optimized, DDPG maximizes the consumption of clean energy and system benefits.
8. Future Work
- The current study does not fully account for scheduling costs and economic factors. Future research may incorporate the influence of market mechanisms to enhance realism.
- This work primarily focuses on a local system and does not consider the coupling characteristics of multi-regional power grid interconnections, which could be a valuable direction for model extension.
- Future research may explore adaptive strategies for real-time dynamic scheduling to enhance the model’s practical applicability and responsiveness.
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Cao, H.; Qiu, J.; Zuo, H.-M.; Li, F.-F. A Long-Term Operational Scheme for Hybrid Hydro-Photovoltaic (PV) Systems That Considers the Uncertainties in Reservoir Inflow and Solar Radiation Based on Scenario Trees. Water Resour. Manag. 2023, 37, 5379–5398. [Google Scholar] [CrossRef]
- Zhang, Y.; Cheng, C.; Cao, R.; Li, G.; Shen, J.; Wu, X. Multivariate Probabilistic Forecasting and Its Performance’s Impacts on Long-Term Dispatch of Hydro-Wind Hybrid Systems. Appl. Energy 2021, 283, 116243. [Google Scholar] [CrossRef]
- Li, H.; Liu, P.; Guo, S.; Cheng, L.; Huang, K.; Feng, M.; He, S.; Ming, B. Deriving Adaptive Long-Term Complementary Operating Rules for a Large-Scale Hydro-Photovoltaic Hybrid Power Plant Using Ensemble Kalman Filter. Appl. Energy 2021, 301, 117482. [Google Scholar] [CrossRef]
- Mu, C.X.; Zhang, J.T.; Cheng, C.T.; Xu, Y.; Yang, Y.Q. High-dimensional Uncertainty Scenario Generation Method for Hydro-wind-solar Multi-energy Complementary System Considering Spatio-temporal Correlation. Power Syst. Technol. 2024, 48, 3614–3623. [Google Scholar] [CrossRef]
- Cao, H.; Mou, C.X.; Yang, Y.Q.; Xu, Y.; Zhang, Z.; Cheng, C.T. Long-term optimization scheduling method for hydro-wind-PV multi energy complementary systems considering multi uncertainty. Yangtze River 2024, 55, 26–34. [Google Scholar]
- Zhao, Z.P.; Yu, Z.H.; Cheng, C.T.; Wang, J.; Feng, Z.K. Multi-risk Quantification and Long-term Multi-objective Coordinative Optimal Dispatch Method for Hydro-Wind-Solar Integrated Energy Base. Autom. Electr. Power Syst. 2024, 48, 118–130. [Google Scholar]
- Yang, Z.; Liu, P.; Cheng, L.; Wang, H.; Ming, B.; Gong, W. Deriving Operating Rules for a Large-Scale Hydro-Photovoltaic Power System Using Implicit Stochastic Optimization. J. Clean. Prod. 2018, 195, 562–572. [Google Scholar] [CrossRef]
- Wen, X.; Qin, J.S.; Tan, Q.F.; Zhang, Z.Y.; Wang, Y.L. Research on stochastic optimal operation of hydro-photovoltaic complementary based on utility function of carryover stage. Water Resour. Prot. 2023, 39, 23–31+62. [Google Scholar] [CrossRef]
- Li, W.W.; Zhou, J.N.; Pei, B.L.; Zhang, Y.F. Study on long-term stochastic optimal operation of cascade reservoirs by deep reinforcement learning. J. Hydroelectr. Eng. 2023, 42, 21–32. [Google Scholar] [CrossRef]
- Jiang, W.; Liu, Y.; Fang, G.; Ding, Z. Research on Short-Term Optimal Scheduling of Hydro-Wind-Solar Multi-Energy Power System Based on Deep Reinforcement Learning. J. Clean. Prod. 2023, 385, 135704. [Google Scholar] [CrossRef]
- Ge, Y.; Xie, J.; Fu, D.; Feng, S.; Chang, J.; Song, Y. Real-Time Scheduling of Wind-Solar-Hydro Complementary System Based on Deep Reinforcement Learning. In Proceedings of the 2024 IEEE PES 16th Asia-Pacific Power and Energy Engineering Conference (APPEEC), Nanjing, China, 25–27 October 2024; pp. 1–4. [Google Scholar] [CrossRef]
- Zhang, H.; Lu, Z.; Hu, W.; Wang, Y.; Dong, L.; Zhang, J. Coordinated Optimal Operation of Hydro–Wind–Solar Integrated Systems. Appl. Energy 2019, 242, 883–896. [Google Scholar] [CrossRef]
- Lillicrap, T.P.; Hunt, J.J.; Pritzel, A.; Heess, N.; Erez, T.; Tassa, Y.; Silver, D.; Wierstra, D. Continuous control with deep reinforcement learning. arXiv 2015, arXiv:1509.02971. [Google Scholar]
Number of Groups | Reward (103) | Time (s) | ||||
---|---|---|---|---|---|---|
1 | 0.005 | 0.001 | 0.9999 | 0.1 | 0.78 | 65.12 |
2 | 0.0001 | 0.001 | 0.9999 | 0.1 | 1.02 | 43.65 |
3 | 0.0003 | 0.001 | 0.9999 | 0.1 | 1.69 | 21.06 |
4 | 0.0001 | 0.005 | 0.9999 | 0.1 | 1.98 | 15.43 |
5 | 0.0001 | 0.0001 | 0.9999 | 0.1 | 2.10 | 12.40 |
6 | 0.0001 | 0.0003 | 0.9999 | 0.1 | 1.72 | 18.72 |
7 | 0.0001 | 0.0001 | 0.9995 | 0.1 | 2.08 | 16.32 |
8 | 0.0001 | 0.0001 | 0.999 | 0.1 | 2.10 | 11.25 |
9 | 0.0001 | 0.0001 | 0.9995 | 0.5 | 2.12 | 3.15 |
10 | 0.0001 | 0.0001 | 0.9995 | 0.9 | 2.08 | 13.79 |
Model | Runoff–Wind | Runoff–PV | Wind–PV |
---|---|---|---|
C-Vine Copula | 0.031 | 0.056 | 0.038 |
The proposed model | 0.020 | 0.003 | 0.009 |
Model | Runoff | Wind | PV | |||
---|---|---|---|---|---|---|
AED | CR | AED | CR | AED | CR | |
C-Vine Copula | 1.7425 | 85.4% | 0.6582 | 80.5% | 0.2419 | 88.5% |
The proposed model | 1.1296 | 92.5% | 0.3715 | 93.8% | 0.0976 | 90.2% |
Model | Actual Power Generation (GWh) | Curtailed Power (GWh) | Curtailed Water (108 m3) | Optimization Time (s) |
---|---|---|---|---|
DDPG | 641.036 | 2.376 | 0 | 3.75 |
DQN | 638.948 | 4.176 | 0 | 6.97 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wan, Z.; Li, W.; He, M.; Zhang, T.; Chen, S.; Guan, W.; Hua, X.; Zheng, S. Research on Long-Term Scheduling Optimization of Water–Wind–Solar Multi-Energy Complementary System Based on DDPG. Energies 2025, 18, 3983. https://doi.org/10.3390/en18153983
Wan Z, Li W, He M, Zhang T, Chen S, Guan W, Hua X, Zheng S. Research on Long-Term Scheduling Optimization of Water–Wind–Solar Multi-Energy Complementary System Based on DDPG. Energies. 2025; 18(15):3983. https://doi.org/10.3390/en18153983
Chicago/Turabian StyleWan, Zixing, Wenwu Li, Mu He, Taotao Zhang, Shengzhe Chen, Weiwei Guan, Xiaojun Hua, and Shang Zheng. 2025. "Research on Long-Term Scheduling Optimization of Water–Wind–Solar Multi-Energy Complementary System Based on DDPG" Energies 18, no. 15: 3983. https://doi.org/10.3390/en18153983
APA StyleWan, Z., Li, W., He, M., Zhang, T., Chen, S., Guan, W., Hua, X., & Zheng, S. (2025). Research on Long-Term Scheduling Optimization of Water–Wind–Solar Multi-Energy Complementary System Based on DDPG. Energies, 18(15), 3983. https://doi.org/10.3390/en18153983