Next Article in Journal
The Effect of Initial Conditions on the Laminar Burning Characteristics of Natural Gas Diluted by CO2
Next Article in Special Issue
Emissions Trading System of the European Union: Emission Allowances and EPEX Electricity Prices in Phase III
Previous Article in Journal
Impact of the Primary Break-Up Strategy on the Morphology of GDI Sprays in 3D-CFD Simulations of Multi-Hole Injectors
Previous Article in Special Issue
Electricity Markets for DC Distribution Systems: Design Options
Article

A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids

1
School of Electronics and Information Engineering, Tongji University, Shanghai 201804, China
2
Education Technology and Computing Center, Tongji University, Shanghai 200092, China
*
Author to whom correspondence should be addressed.
Energies 2019, 12(15), 2891; https://doi.org/10.3390/en12152891
Received: 9 July 2019 / Revised: 22 July 2019 / Accepted: 23 July 2019 / Published: 26 July 2019
(This article belongs to the Special Issue Market Design for a High-Renewables Electricity System)
Decision-making of microgrids in the condition of a dynamic uncertain bidding environment has always been a significant subject of interest in the context of energy markets. The emerging application of reinforcement learning algorithms in energy markets provides solutions to this problem. In this paper, we investigate the potential of applying a Q-learning algorithm into a continuous double auction mechanism. By choosing a global supply and demand relationship as states and considering both bidding price and quantity as actions, a new Q-learning architecture is proposed to better reflect personalized bidding preferences and response to real-time market conditions. The application of battery energy storage system performs an alternative form of demand response by exerting potential capacity. A Q-cube framework is designed to describe the Q-value distribution iteration. Results from a case study on 14 microgrids in Guizhou Province, China indicate that the proposed Q-cube framework is capable of making rational bidding decisions and raising the microgrids’ profits. View Full-Text
Keywords: microgrids; continuous double auction; Q-learning algorithm; battery energy storage system, Q-cube framework; bidding strategy microgrids; continuous double auction; Q-learning algorithm; battery energy storage system, Q-cube framework; bidding strategy
Show Figures

Figure 1

MDPI and ACS Style

Wang, N.; Xu, W.; Shao, W.; Xu, Z. A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids. Energies 2019, 12, 2891. https://doi.org/10.3390/en12152891

AMA Style

Wang N, Xu W, Shao W, Xu Z. A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids. Energies. 2019; 12(15):2891. https://doi.org/10.3390/en12152891

Chicago/Turabian Style

Wang, Ning, Weisheng Xu, Weihui Shao, and Zhiyu Xu. 2019. "A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids" Energies 12, no. 15: 2891. https://doi.org/10.3390/en12152891

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop