Comparative Study of Neuroevolution and Deep Reinforcement Learning for Voltage Regulation in Power Systems

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

1. (I) The references span a wide range of years (1995–2024). It is recommended to supplement with papers from the past two years on DRL applications in voltage control to enhance the cutting-edge nature of the work.

(II) The article title is somewhat lengthy; it is recommended to simplify it to: “Comparative Study of Neuroevolution and Deep Reinforcement Learning for Voltage Regulation in Power Systems”, more concise and impactful.

(III) The figures present information gaps: Figures 2–6 lack error bars or confidence intervals, failing to reflect the variability of experimental results; Figure 7 does not specify the time scale on the x-axis or the specific operating conditions for voltage fluctuations. It is recommended to enhance the completeness of the figures by adding error analysis and operational context descriptions to improve the clarity and persuasiveness of the results.

 

1. (I) The existing literature review lacks a systematic overview of research trends in neural evolutionary algorithms and deep reinforcement learning for power grid voltage control, failing to clearly identify core points of contention and technical gaps in current studies.

Suggested revisions: Include a comparative analysis of recent high-impact research findings to clearly delineate the distinctive contributions of this study from existing literature, thereby avoiding redundant arguments.

(II) The article lacks sufficient theoretical analysis of the suitability of the three algorithms, failing to thoroughly explain why these specific algorithms were chosen for comparison. It also lacks theoretical derivation demonstrating the alignment between the core principles of the algorithms and the characteristics of the voltage control problem.

Suggested revisions: Incorporate an analysis of the algorithm's theoretical adaptability to grid voltage regulation requirements to strengthen the theoretical soundness of the research design.

 

III.(I) The article's experimental setup lacks detailed specifications regarding the random load fluctuation range (80%-120%) and fault scenarios under the N-1 safety principle. Critical parameters such as fault types, occurrence probabilities, and impact scopes remain undefined, making it difficult to assess the comprehensiveness and realism of the experimental scenarios.

Suggested revisions: Supplement specific parameters and logical rationale for scenario design, and incorporate experimental validation for extreme scenarios (For example, high-proportion renewable energy fluctuations, multiple overlapping failures).

(II) The calculation details for performance metrics are insufficiently disclosed. For instance, the “control stability” metric lacks a clearly defined quantification method, and the selection of the time discount factor (γ=0.9995) for cumulative rewards lacks sensitivity analysis.

Suggested revisions: Supplement each indicator with specific calculation models, incorporate sensitivity analysis for key parameters, and validate the robustness of the results.

 

1. (I) The experimental design for comparing the article with traditional automatic voltage regulators (AVRs) is insufficiently comprehensive. It only selected four key generation nodes for voltage response analysis, failing to address comparisons of overall system metrics (such as network losses and mean voltage deviation) and neglecting differences in adaptability under varying load levels.

Suggested revisions: Expand the comparison dimensions by incorporating system-level performance metrics and results from multiple operating conditions to further highlight the algorithm's superiority.

(II) The paper does not sufficiently address the limitations of the experiment. It fails to mention the scalability test results of the algorithm in large-scale power grids (such as the IEEE 118-node system) and does not analyze the constraints imposed by computational costs on real-time control applications.

Suggested revisions: Include experimental data on algorithm scalability and quantify the computational complexity of different algorithms to provide a more comprehensive reference for practical engineering applications.

 

1. The introduction section”Conversely, during nighttime, when demand decreases and generation is minimal, transmission lines may still exhibit low active power flows but high reactive currents, leading to overvoltage events.” This sentence has a relatively complex structure, which can be improved through clearer expression to enhance its structure and wording.

Suggested revisions:”Conversely, during nighttime periods of low demand and minimal generation, transmission lines often carry limited active power but considerable reactive current, resulting in potential overvoltage conditions.” Multimodal algorithms have great inspiration and reference value for this task, such as Lightweight bilateral network of Mura detection on micro-OLED displays, Multi-task learning for hand heat trace time estimation and identity recognition, Deep soft threshold feature separation network for infrared handprint identity recognition and time estimation. If the existing research in this article does not involve multimodal algorithms, the above-mentioned papers should also be mentioned.

Author Response

Please see the attachment. Thank you for your review.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This study presents a comparative analysis of three artificial intelligence approaches: Deep Q-Learning (DQL), Genetic Algorithms (GA), and Particle Swarm Optimization (PSO) for training agents for autonomous voltage control. Here wre some important comments:

1. You provide introduction to the algorithms used, however, it is difficult to link your work with other studies since only a few are discussed in the introduction. A more extensive review of related works regarding these three AI approaches: DQL, GA, and PSO would enhance your comparison. Discuss each algorithm and highlight its pros and cons.
2. Expand your references to include additional studies related to the algorithms and their applications in voltage control.
3. In the training section, add a table summarizing the simulation and evaluation settings, including episode length, batch size, and other relevant parameters.
4. Discuss the limitations of using Deep Q-Learning and other methods, particularly in terms of computation and scalability.
5. The discussion section can be linked to the results, followed by the conclusion and recommendations for future work.
6. consider adding a summary table to explain figure 6

Author Response

Please see the attachment. Thank you for your review.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The authors tackle an interesting subject, which is neuroevolution and deep reinforcement learning algorithms for voltage control in power grids. However, certain issues need to be addressed:

1) The abstract should include the main results, quantitatively.

2) The Introduction is too brief and should be expanded in order to include more literature analysis, background, motivation, and highlight the contribution of this manuscript. In ths context, it would be useful to have a Table comparing this manuscript to previous work.

3) The Section "Problem formulation" should include at least a conceptual diagram or a flow chart, in order to be easier understood by future readers.

4) In the Methodology Section, there are only 3 formulas. A scientific manuscript is expected to have more formulas, also (wherever applicable) supported by references. For example, the formulas of the Genetic Algorithm and PSO, are missing. Also, MAPE and MAE formulas are missing.

5) In the Results Section, the use case is not described adequately. Why is it considered to be representative? There should be a Table with the main input / parameters.

6) In Figure 4, the reward per action is much different for all three methods but especially for  Deep Q. This should be explained by the authors in more detail. The same applies for the cummulative reward of Figure 6.

7) The Discussion should be extended and include the main limitations and future work.

8) Why would the authors select this kind of AI-based methodologies for voltage control, instead of conventional, model-based approaches?

Taking the aforementioned comments into consideration, a major revision of the manuscript is proposed.

Author Response

Please see the attachment. Thank you for your review.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The innovation of this paper is seriously insufficient, the algorithm still needs to be further studied, and the innovation level of the paper cannot reach the level of publication.

Reviewer 2 Report

Comments and Suggestions for Authors

The paper has now improved. Thanks to the authors. It includes more details and better clarity, and has been updated with recent relevant studies.

Reviewer 3 Report

Comments and Suggestions for Authors

The authors have addressed the comments adequately.