Next Article in Journal
Autonomous Driving Vulnerability Analysis Under Mixed Traffic Conditions in a Simulated Living Laboratory Environment for Sustainable Smart Cities
Previous Article in Journal
Environmental Implications of Reuse: A Case Study of Electrical and Electronic Devices in Slovenia
Previous Article in Special Issue
A Novel Hybrid Framework for Short-Term Carbon Emissions Forecasting in China: Aggregate and Sectoral Perspectives
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 18
Issue 1
10.3390/su18010141
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
This is an early access version, the complete PDF, HTML, and XML versions will be available soon.
Article

Multi-Agent Reinforcement Learning for Sustainable Integration of Heterogeneous Resources in a Double-Sided Auction Market with Power Balance Incentive Mechanism

by
Jian Huang
1,
Ming Yang
1,
Li Wang
1,
Mingxing Mei
1,
Jianfang Ye
1,
Kejia Liu
2,3 and
Yaolong Bo
2,3,*
1
State Grid Zhejiang Electric Power Company Lishui Power Supply Company, Lishui 323000, China
2
College of Electrical Engineering, Zhejiang University, Hangzhou 3100027, China
3
Zhejiang Key Laboratory of Electrical Technology and System on Renewable Energy, Hangzhou 310027, China
*
Author to whom correspondence should be addressed.
Sustainability 2026, 18(1), 141; https://doi.org/10.3390/su18010141
Submission received: 23 September 2025 / Revised: 13 December 2025 / Accepted: 18 December 2025 / Published: 22 December 2025
(This article belongs to the Special Issue AI-Driven Low-Carbon Sustainable Energy Systems: System Design, Computational Strategies, and Emerging Innovations)
Versions Notes

Abstract

Traditional electricity market bidding typically focuses on unilateral structures, where independent energy storage units and flexible loads act merely as price takers. This reduces bidding motivation and weakens the balancing capability of regional power systems, thereby limiting the large-scale utilization of renewable energy. To address these challenges and support sustainable power system operation, this paper proposes a double-sided auction market strategy for heterogeneous multi-resource (HMR) participation based on multi-agent reinforcement learning (MARL). The framework explicitly considers the heterogeneous bidding and quantity reporting behaviors of renewable generation, flexible demand, and energy storage. An improved incentive mechanism is introduced to enhance real-time system power balance, thereby enabling higher renewable energy integration and reducing curtailment. To efficiently solve the market-clearing problem, an improved Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (MATD3) algorithm is employed, along with a temporal-difference (TD) error-based prioritized experience replay mechanism to strengthen exploration. Case studies validate the effectiveness of the proposed approach in guiding heterogeneous resources toward cooperative bidding behaviors, improving market efficiency, and reinforcing the sustainable and resilient operation of future power systems.
Keywords: heterogeneous resources; double-sided auction market; incentive mechanism; multi-agent reinforcement learning; sustainability; renewable energy integration; resilient power systems heterogeneous resources; double-sided auction market; incentive mechanism; multi-agent reinforcement learning; sustainability; renewable energy integration; resilient power systems

Share and Cite

MDPI and ACS Style

Huang, J.; Yang, M.; Wang, L.; Mei, M.; Ye, J.; Liu, K.; Bo, Y. Multi-Agent Reinforcement Learning for Sustainable Integration of Heterogeneous Resources in a Double-Sided Auction Market with Power Balance Incentive Mechanism. Sustainability 2026, 18, 141. https://doi.org/10.3390/su18010141

AMA Style

Huang J, Yang M, Wang L, Mei M, Ye J, Liu K, Bo Y. Multi-Agent Reinforcement Learning for Sustainable Integration of Heterogeneous Resources in a Double-Sided Auction Market with Power Balance Incentive Mechanism. Sustainability. 2026; 18(1):141. https://doi.org/10.3390/su18010141

Chicago/Turabian Style

Huang, Jian, Ming Yang, Li Wang, Mingxing Mei, Jianfang Ye, Kejia Liu, and Yaolong Bo. 2026. "Multi-Agent Reinforcement Learning for Sustainable Integration of Heterogeneous Resources in a Double-Sided Auction Market with Power Balance Incentive Mechanism" Sustainability 18, no. 1: 141. https://doi.org/10.3390/su18010141

APA Style

Huang, J., Yang, M., Wang, L., Mei, M., Ye, J., Liu, K., & Bo, Y. (2026). Multi-Agent Reinforcement Learning for Sustainable Integration of Heterogeneous Resources in a Double-Sided Auction Market with Power Balance Incentive Mechanism. Sustainability, 18(1), 141. https://doi.org/10.3390/su18010141

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop