Deep Reinforcement Learning for Resilient Power and Energy Systems: Progress, Prospects, and Future Avenues

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

In this paper, a comprehensive review delves into the latest advancements and applications of DRL in bolstering the resilience of power and energy systems, highlighting significant contributions and key insights. However, there are some major concerns as follows:

 

1. The author should point out the difference between their work and the previous review. Clearly indicate the domain-independent innovative advance brought about by the proposed works.

2. Some of your figures need improvement, for example, please also use the same font size for each figure.

3. Illustrations and Figures: consider adding or enhancing graphical representations (such as diagrams or charts) to break down complex ideas and provide visual insights. This could make the paper more accessible to a broader audience.

4. More related works on DRL shall be discussed. Authors are suggested to review more new and relevant research to support their research contribution. Some refs could be useful, e.g., Movement-based solutions to energy limitation in wireless sensor networks: A novel deep policy gradient action quantization for trusted collaborative computation in intelligent vehicle networks. 

5. For the related DRL algorithms, the authors are suggested to list the advantages and disadvantages, via a table. At the authors' discretion.

6. The summary needs to be rewritten as it is not well structured by highlighting the importance of the study and the benefits of the work.

7. Some references have incomplete compilation, e.g., missing volume/page numbering

8. There are also some typos in this paper. Please carefully go through the manuscript to improve its presentation.

Comments on the Quality of English Language

 Moderate editing of English language required.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The paper commences with a well-structured introduction that effectively highlights the significance of resilience in power and energy systems and the categorization and description of various DRL algorithms lay a robust foundation for comprehending their applicability and also an innovative approach is taken by categorizing DRL applications into five pivotal dimensions, facilitating a methodical exploration of how DRL methodologies can effectively tackle the critical challenges within the domain of power and energy system resilience. This innovative classification approach adds clarity to the discussion. Given the increasing prevalence of extreme events and their impact on power and energy systems, the relevance of the topic is unquestionable and the paper extensively discusses previous methods and research in the field of power and energy systems resilience, providing valuable background for readers. Furthermore, the paper is well-organized into sections, which makes it easier for readers to navigate and comprehend the content and it offers insights into the challenges and limitations associated with integrating DRL into power and energy systems, offering a realistic perspective on potential obstacles in this field.

 

The paper references data only up to 2022, and given the rapidly evolving nature of the field, this may not capture the most recent trends and developments, such as Prosumer-Centric Self-Sustained Smart Grid Systems, doi: 10.1109/JSYST.2022.3156877. Additionally, the introduction uses repetitive phrases such as "power and energy systems" and "resilience," which can make the text less engaging and could be improved for readability and some sections of the paper lack smooth transitions between ideas, making it challenging to follow the flow of the paper. While the paper mentions potential pitfalls of DRL integration, it could delve deeper into these issues and offer potential solutions and the paper frequently uses complex language and jargon that might hinder the understanding of readers who are not experts in the field. There is a significant focus on weather-related events in the United States, which might not fully represent the global context and could benefit from a more international perspective. Furthermore, the citation style varies throughout the paper and should be standardized for clarity and Figure 1 is included, but its significance and implications for the research are not explained in the text. While the paper mentions existing review papers, it does not effectively connect the contributions of this work to the existing literature, missing an opportunity to provide context and relevance.

 

 

One of the significant improvements needed for the paper is in the literature review section. It should be more comprehensive in mapping existing literature, identifying gaps, and explicitly stating how this review paper intends to fill those gaps. This will help readers understand the paper's unique contributions and its context within the broader research landscape. Additionally, the paper would benefit from providing more up-to-date data and using a more accessible language to enhance readability and relevance. Furthermore, providing more figures, diagrams, and visual aids to illustrate complex concepts would improve clarity. Lastly, addressing potential ethical concerns related to AI and DRL in power systems would add depth and relevance to the discussion.

Comments on the Quality of English Language

Moderate editing of English language required

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The use of Artificial Intelligence (AI) as a tool for automation of various processes is gaining in importance every year. The increase in computer computing power and the development of various types of algorithms constituting subsets of AI mean that this type of solutions are being used more and more often and with greater success, especially in relation to complex issues with huge amounts of data to be processed, and such a challenge is certainly the issue of resilience in power and energy systems. Deep Reinforcement Learning discussed in the article seems to be a very promising tool in relation to this issue. An increasing number of publications from the last few years that deal with this problem indicate the dynamic development of this technology, and the author's attempt to systematize this knowledge should be considered valuable, especially since, based on his bibliography, it should be considered that it is he is an expert in this field. Taking into account the above statements, it should be concluded that the subject matter is very current and tailored to the profile of the MDPI Electricy magazine and presents an appropriately high substantive level.

When assessing the structure of the work, it should be considered that it is logical and well-thought-out. In individual chapters, the author discussed introductory issues, explained basic concepts and presented the basic DL algorithms currently used in the discussed application. The article is also written clearly and in good language.

The selection of literature should be considered appropriate and up-to-date, the vast majority of cited works come from recent years, which proves that the solutions presented are advanced and refined. When discussing this type of issues, it is also important that individual articles discuss specific local solutions to the problem, because in different parts of the world the problem of disruptions in the operation of the power system and its reconstruction may differ significantly, because it results from the specific structural conditions (scale of network expansion and structure energy generation) and natural (possible natural disasters).

An important element of the work is the issue of challenges and problems, as well as probable directions of development, which gives a full picture of the current state of knowledge in this topic.

Generally, I have no substantive objections to the work. I did not find any substantive errors. However, the work is very extensive, with a large amount of text and a number of issues that are not easy to understand. Therefore, the scant presence of graphics that would allow for a better illustration of the problem do not facilitate understanding of the presented issues. For example, in Chapter 3, it would be a great help to present on a time chart the process of rebuilding the energy system after a disaster and the location of the discussed DRL algorithm against this background, perhaps even with a comparison to the situation when there is no such algorithm, which would show the importance of this technique. In Chapter 2, it might be useful to have a graphic that places the DRL (or more precisely, the neural network on which it works) against the background of the entire algorithm (where and from where the input data comes in, what we get at the output, what is the feedback, etc.) .

The above remarks could improve the quality of the article, but they are not critical because I rate the article as a whole very highly.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

I encourage the authors to upload the source code and data of the paper/experiments to a public repository once that the paper will be accepted (if applicable), and to include the link of the repository in the article.

Comments on the Quality of English Language

Moderate editing of English language required.

Author Response

Reviewer 1:

I encourage the authors to upload the source code and data of the paper/experiments to a public repository once that the paper will be accepted (if applicable), and to include the link of the repository in the article.

Author’s Response: Thank you for the suggestion. However, since this is a review paper, I don’t have any source code relevant to this paper.

Reviewer 2 Report

Comments and Suggestions for Authors

The authors have addressed in detail the reviewers comments.

Author Response

Reviewer 2:

The authors have addressed in detail the reviewers comments.

Author’s Response: Thank you.