Next Article in Journal
A Riding-through Technique for Seamless Transition between Islanded and Grid-Connected Modes of Droop-Controlled Inverters
Next Article in Special Issue
Two-Stage Multi-Objective Collaborative Scheduling for Wind Farm and Battery Switch Station
Previous Article in Journal
Impact of Generator Stroke Length on Energy Production for a Direct Drive Wave Energy Converter
Article Menu
Issue 9 (September) cover image

Export Article

Open AccessArticle
Energies 2016, 9(9), 725; doi:10.3390/en9090725

Application of a Gradient Descent Continuous Actor-Critic Algorithm for Double-Side Day-Ahead Electricity Market Modeling

School of Economics and Management, North China Electric Power University, Beijing 102206, China
*
Author to whom correspondence should be addressed.
Academic Editor: Robert Lundmark
Received: 1 June 2016 / Revised: 20 August 2016 / Accepted: 30 August 2016 / Published: 9 September 2016
(This article belongs to the Special Issue Applied Energy System Modeling 2016)
View Full-Text   |   Download PDF [5394 KB, uploaded 9 September 2016]   |  

Abstract

An important goal of China’s electric power system reform is to create a double-side day-ahead wholesale electricity market in the future, where the suppliers (represented by GenCOs) and demanders (represented by DisCOs) compete simultaneously with each other in one market. Therefore, modeling and simulating the dynamic bidding process and the equilibrium in the double-side day-ahead electricity market scientifically is not only important to some developed countries, but also to China to provide a bidding decision-making tool to help GenCOs and DisCOs obtain more profits in market competition. Meanwhile, it can also provide an economic analysis tool to help government officials design the proper market mechanisms and policies. The traditional dynamic game model and table-based reinforcement learning algorithm have already been employed in the day-ahead electricity market modeling. However, those models are based on some assumptions, such as taking the probability distribution function of market clearing price (MCP) and each rival’s bidding strategy as common knowledge (in dynamic game market models), and assuming the discrete state and action sets of every agent (in table-based reinforcement learning market models), which are no longer applicable in a realistic situation. In this paper, a modified reinforcement learning method, called gradient descent continuous Actor-Critic (GDCAC) algorithm was employed in the double-side day-ahead electricity market modeling and simulation. This algorithm can not only get rid of the abovementioned unrealistic assumptions, but also cope with the Markov decision-making process with continuous state and action sets just like the real electricity market. Meanwhile, the time complexity of our proposed model is only O(n). The simulation result of employing the proposed model in the double-side day-ahead electricity market shows the superiority of our approach in terms of participant’s profit or social welfare compared with traditional reinforcement learning methods. View Full-Text
Keywords: bidding strategy; double-side day-ahead electricity market; gradient descent continuous Actor-Critic (GDCAC) algorithm; reinforcement learning; market clearing price (MCP) bidding strategy; double-side day-ahead electricity market; gradient descent continuous Actor-Critic (GDCAC) algorithm; reinforcement learning; market clearing price (MCP)
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. (CC BY 4.0).

Scifeed alert for new publications

Never miss any articles matching your research from any publisher
  • Get alerts for new papers matching your research
  • Find out the new papers from selected authors
  • Updated daily for 49'000+ journals and 6000+ publishers
  • Define your Scifeed now

SciFeed Share & Cite This Article

MDPI and ACS Style

Zhao, H.; Wang, Y.; Guo, S.; Zhao, M.; Zhang, C. Application of a Gradient Descent Continuous Actor-Critic Algorithm for Double-Side Day-Ahead Electricity Market Modeling. Energies 2016, 9, 725.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Energies EISSN 1996-1073 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top