PowerModel-AI: A First On-the-Fly Machine-Learning Predictor for AC Power Flow Solutions

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

1.The paper should explicitly detail the activation functions, optimization algorithms, and hyperparameters (e.g., learning rates, batch sizes) used in the two-layer neural network architecture to enable exact replication of experiments. 

2.Include performance metrics (RMSE, MAE) comparisons between PowerModel-AI and traditional iterative solvers (Newton-Raphson) in terms of both speed and accuracy across all tested grid cases to better contextualize advantages.

3.Add scatter plots comparing predicted vs actual voltage magnitudes/angles across multiple grid cases, with error distribution histograms to visually demonstrate prediction accuracy patterns and outlier characteristics. 

4.Explicitly address model scalability constraints - particularly whether the shallow architecture can handle ultra-large grids.

Perhaps these questions are more difficult, I hope the authors can make effective revisions to the paper. In addition, the figures of the paper are relatively small, it is recommended to enlarge.

Author Response

General Comments

We thank the Reviewers for their comments. Importantly, we would like to point out that the key purpose of this paper is the introduction of our software and the hopeful utilization of it by the “power” community and continued development. We have provided every single piece of information used in this work in the supplementary database on Zenodo (https://zenodo.org/records/13843934),  and we have ensured that everything we did is exactly reproducible. The working directory for the PowerModel-AI app is also publicly available on GitHub (https://github.com/lanl-ansi/PowerModelsAI/tree/main).

Response to Review Comments

• The paper should explicitly detail the activation functions, optimization algorithms, and hyperparameters (e.g., learning rates, batch sizes) used in the two-layer neural network architecture to enable exact replication of experiments. 

We have this in Section 2. Table 1 and 2 provide all the information required by the Reviewer. Section 2.4 discusses them. We want to point out that every single thing done in this work (even the Jupyter notebooks used for analysis) is included as supplementary material in the zenodo database (https://zenodo.org/records/13843934). We ensured that everything in the paper is exactly replicable. It is not clear what more information the Reviewer needs to see.

• Include performance metrics (RMSE, MAE) comparisons between PowerModel-AI and traditional iterative solvers (Newton-Raphson) in terms of both speed and accuracy across all tested grid cases to better contextualize advantages.

First, we have the mean square errors (L*) in Table 1, for the performance of Model M1 (our optimal model which we have mentioned in the test becomes PowerModel-AI) compared to the test dataset. Second, we have the MAE performance metric in Figure 7 for the PowerModel-AI compared to PowerModel.jl solver for a validation set of 12000 new and independent event. We hope the Reviewer agrees that for the size and dimension of the bus cases, a statistical representation is the optimal path to show this metric. However, to drive home this point, we have also included a new table 3 to show the actual values for MAE and MSE for 12,000 independent events. We updated the text to reflect this as

“Consequently, previous and subsequent references to PowerModel-AI relate to the design of the M1 model. To further validate M1, we show in Table 3 the MAE and MSE of M1 compared to output from PowerModel.jl for 12,000 independent loading event.  The plot of the training losses for all the models discussed in this work are provided as supplementary data in the Jupyter notebooks. All the models show good convergence in the training and validation sets.”

Regarding speed, the gain in speed cannot be fully explored because in practice, and for the range of nodes (even at 500) both traditional solvers and PowerModel-AI operate almost at same speed.

 

• Add scatter plots comparing predicted vs actual voltage magnitudes/angles across multiple grid cases, with error distribution histograms to visually demonstrate prediction accuracy patterns and outlier characteristics. 

We have this in figure 5 and 6 for some nodes in some grid cases. Due to the large amount of data to showcase this, we have included plots for every single bus as supplementary material.  The supplementary material in Zenodo is well-described and this scatter plot for every single node in all buses considered are presented. This was mentioned in the both the main text as well as in the captions of Figure 5 and 6. The text in the manuscript reads (see Page 13/22 Line 348-349):

“… Similar plots for other nodes across all grids are provided as supplementary material, available as PNG files and Jupyter notebooks.”

• Explicitly address model scalability constraints - particularly whether the shallow architecture can handle ultra-large grids.

For this introductory paper, where we aim to introduce PowerModel-AI software to the public, we believe the 500-bus case is large enough to show this as we and interested scholars continue to improve this open-source software.

Regarding the shallow network, we show in section 3.11 and 3.12 that the inherent and fundamental assumption of the power flow equations guarantees the need for shallow network, which we believe is sufficient (after testing higher networks). We do not intent to make this already long paper any longer by adding justification for n-layers as we have shown that a shallow network is more than enough which also aims to avoid over or under-fitting

Additionally, working on bigger systems will be the addons to the existing version of the software (which we emphasize again in open source). But, as the Reviewer already knows, the bigger grids have problems of their own that need to be addressed even for iterative solvers.

• Perhaps these questions are more difficult, I hope the authors can make effective revisions to the paper. In addition, the figures of the paper are relatively small, it is recommended to enlarge.

We thank the reviewer for the thoughtful and intelligent comments. Regarding the figures, we have reached out to the journal, and we believe that the typesetting for the review process resulted in them reducing the page width. We are confident that, if accepted, the final print would have more legible figures as intended from our original submission.

Reviewer 2 Report

Comments and Suggestions for Authors

I noticed that this paper is basically the same as the paper published on the preprint platform, and the publication time of the paper on the preprint platform is significantly earlier than that of the paper. This needs further explanation to meet the requirements of journal publication.

1. What is the convergence process of the ML models. Adding the training process of the model can help illustrate the value of the study.
2. In Page 13, these predictions were tested on an external dataset of 12,000 data points. For a fair comparison, we focus on the worst-case predictions across the five Texas A&M synthetic grids. How does this paper overcome the impact of the worst prediction error and is there a way to improve accuracy
3. In conclusion, it minimizes the need to generate training data for every possible scenario. In fact, the paper did not reveal the benefits of this approach.

Author Response

General Comments

We thank the Reviewers for their comments. Importantly, we would like to point out that the key purpose of this paper is the introduction of our software and the hopeful utilization of it by the “power” community and continued development. We have provided every single piece of information used in this work in the supplementary database on Zenodo (https://zenodo.org/records/13843934),  and we have ensured that everything we did is exactly reproducible. The working directory for the PowerModel-AI app is also publicly available on GitHub (https://github.com/lanl-ansi/PowerModelsAI/tree/main).

Response to Review Comments

• I noticed that this paper is basically the same as the paper published on the preprint platform, and the publication time of the paper on the preprint platform is significantly earlier than that of the paper. This needs further explanation to meet the requirements of journal publication.

Yes, we had to take care of licensing procedure for PowerModel-AI software, which took a little longer than anticipated.

• What is the convergence process of the ML models. Adding the training process of the model can help illustrate the value of the study.

First, we have Figure 2 which shows the flowchart of training. We have described all the hyperparameters and justified their use in section 2. We have provided the convergence data for every grid case in the supplementary material in Jupyter notebooks (they do not need to be run as they are saved with Cache). Regarding the convergence curve, we have explicitly added a pdf file with the convergence curve for the M1 model (PowerModel-AI) as supplementary material. Also, we already provided the MSE for the test set in Table 1. We added a sentence to highlight this (see Page 11/22 Line 300-306).

“… The supplementary PDF file accompanying this work displays Figures S1 to S5, which illustrate the training (red) and validation (blue) loss convergence curves for the PowerModel-AI (M1 model) across the 14-Bus, 30-Bus, 37-Bus, 200-Bus, and 500-Bus synthetic grids. In each case, the model achieves rapid convergence—around the 10th epoch—with the validation loss beginning at a significantly lower value than the training loss.”

• In Page 13, these predictions were tested on an external dataset of 12,000 data points. For a fair comparison, we focus on the worst-case predictions across the five Texas A&M synthetic grids. How does this paper overcome the impact of the worst prediction error and is there a way to improve accuracy

As the Reviewer knows, models are developed for baselines. So, we check for the lowest performers and if those are reasonable then that becomes the favorable model. This is what we have done, and we aim to show that though this are considered the “worst case”, they are still indeed good enough prediction. We have also included all plots for all the buses in all the grids in the supplementary folder, for completeness. This we clearly mentioned in the manuscript.

• In conclusion, it minimizes the need to generate training data for every possible scenario. In fact, the paper did not reveal the benefits of this approach.

On the contrary, we did just that. We showed that this on a test case of 12,000 datapoints. We have developed criteria for on-the-fly learning, and we have the app to show that it works. It is not clear what else we have to do to prove this point. All the data are publicly available, and we hope that this is a starting point for more development. The main economic and technological benefit is that large amount of data that are not needed for prediction would not need to be stored, and prediction can be faster, especially if the software is to be used in-situ on smaller devices.

Most state-of-the-art ML models are trained on a system where data for all potential scenarios is precomputed and used during training. In this approach, minimal data is generated to initially train the model, which is then refined as needed. Our method, however, can easily handle unseen scenarios by quickly retraining the model on-the-fly with just a few new data points. We believe this point is clearly explained in the manuscript.

We are not claiming this to be the end. As a matter of fact, our conclusion section does show what we still need to do. We ask the reviewer to consider this as a “work-in-progress,” in which we believe we have made significant strides already.

Reviewer 3 Report

Comments and Suggestions for Authors

An on-the-fly machine-learning predictor for AC power flow solutions is proposed. The reviewer has some comments:

1. How did you guarantee stability for this generative AI method with impressive performance?
2. Why in Table Ⅰ, does ∆_min maintain similarity for all parameters, and ∆_max keeps changing for all the parameters? Why does ∆_max reach the maximum in 37-Bus? How did you choose these complicated hyperparameters in Table Ⅰ?
3. For a machine learning paper, the references seem old. Recently, many innovative methods have been proposed to address different challenges (doi: 10.1109/TPEL.2025.3525500, doi: 10.1109/TTE.2024. 3514657). Please discuss your innovations and contributions compared with these articles.
4. Can you also provide the training convergence curve for your neural network?
5. The fonts are too small in some figures, especially Fig.2. The presentation of the contents should be highly improved.

Comments on the Quality of English Language

The English should be further improved.

Author Response

General Comments

We thank the Reviewers for their comments. Importantly, we would like to point out that the key purpose of this paper is the introduction of our software and the hopeful utilization of it by the “power” community and continued development. We have provided every single piece of information used in this work in the supplementary database on Zenodo (https://zenodo.org/records/13843934),  and we have ensured that everything we did is exactly reproducible. The working directory for the PowerModel-AI app is also publicly available on GitHub (https://github.com/lanl-ansi/PowerModelsAI/tree/main).

Response to Review Comments

• How did you guarantee stability for this generative AI method with impressive performance?

So far, we have identified and discussed three criterions to ensure stability in Section 2.5, and our on-the-fly capability ensures that in regions of instability the model can automatically retrain itself, based on the specified criteria

• Why in Table Ⅰ, does ∆_min maintain similarity for all parameters, and ∆_max keeps changing for all the parameters? Why does ∆_max reach the maximum in 37-Bus? How did you choose these complicated hyperparameters in Table Ⅰ?

As discussed in the manuscript, the minimum was selected as 0.01 as any lower does not really make significant change (also this was what we decided to go with). The maximum is based on multiple trials using the iterative solver (PowerModel.jl) to test the limits where we achieve finite solutions for a given range of load multipliers. The ∆_max showcases the range where PowerModel.jl can produce converged results. Anything outside the selected ∆_max is not trustworthy.

• For a machine learning paper, the references seem old. Recently, many innovative methods have been proposed to address different challenges (doi: 10.1109/TPEL.2025.3525500, doi: 10.1109/TTE.2024. 3514657). Please discuss your innovations and contributions compared with these articles.

We agree that those are good research. The Authors in those publications followed a different path of reinforcement learning for a different Power problem. However, we can infer that they aimed at increasing prediction efficiency just like we are doing with on-the-fly learning and justified shallow models. Therefore, we cannot speak for these works as they are not the kind of problem we are interested in.

• Can you also provide the training convergence curve for your neural network?

Yes, we have explicitly added a pdf file with the convergence curve for the M1 model (PowerModel-AI) as supplementary material. Also, we already provided the MSE for the test set in Table 1. We added a sentence to highlight this (see Page 11/22 Line 300-306).

“… The supplementary PDF file accompanying this work displays Figures S1 to S5, which illustrate the training (red) and validation (blue) loss convergence curves for the PowerModel-AI (M1 model) across the 14-Bus, 30-Bus, 37-Bus, 200-Bus, and 500-Bus synthetic grids. In each case, the model achieves rapid convergence—around the 10th epoch—with the validation loss beginning at a significantly lower value than the training loss.”

• The fonts are too small in some figures, especially Fig.2. The presentation of the contents should be highly improved.

We have reached out to the journal, and we believe that the typesetting for the review process resulted in them reducing the page width. We are confident that, if accepted, the final print would have more legible figures as intended from our original submission.

Reviewer 4 Report

Comments and Suggestions for Authors

The paper presents PowerModel-AI, an innovative machine-learning-based approach for predicting AC power flow solutions. The authors present a learning mechanism that continuously updates the model, ensuring the accuracy of its predictions. The thorough study combines domain knowledge of power systems with machine learning (ML) principles and tests the approach on various cases. This method presents major contributions. However, the paper requires some improvements.

1. Define the contributions in the introduction
2. The units are not well treated (Mvar) and the variables are not well used in the text (vangle).
3. Use diagrams for the procedure defined from lines 180-208.
4. Include text in the beginning of a section. Example: section 3.
5. More details were expected from the results, especially from the figures related to the comparison of true vs. predicted voltage magnitudes and angles.
6. I expected values related to solving times and comparisons with different cases. 

 

Author Response

General Comments

We thank the Reviewers for their comments. Importantly, we would like to point out that the key purpose of this paper is the introduction of our software and the hopeful utilization of it by the “power” community and continued development. We have provided every single piece of information used in this work in the supplementary database on Zenodo (https://zenodo.org/records/13843934),  and we have ensured that everything we did is exactly reproducible. The working directory for the PowerModel-AI app is also publicly available on GitHub (https://github.com/lanl-ansi/PowerModelsAI/tree/main).

Response to Review Comments

• Define the contributions in the introduction

We strongly believe that we have specified the contribution in the current introduction. We outlined and discussed 4 key points that constitute the advantages of our work.

• The units are not well treated (Mvar) and the variables are not well used in the text (vangle).

Since the term Mvar and vangle are well known in the Power community, we decided to maintain this for ease and a smoother flow.

• Use diagrams for the procedure defined from lines 180-208.

Yes, the flowchart in Figure 2 and schematic overview in Figure 1 serve this purpose.

 

• Include text in the beginning of a section. Example: section 3.

This is decided by the editorial team at Energies. We used the recommended template by the Journal.

 

• More details were expected from the results, especially from the figures related to the comparison of true vs. predicted voltage magnitudes and angles.

Yes, we agree, but there is so much we can but in an already long paper. We provided more figures comparing the true and predicted quantities for every in every node in the supplementary material in our Zenodo database (https://zenodo.org/records/13843934) linked to this work. We also prepared ready-to-run Jupyter notebook with discussions to show how we arrived at the results we have.

 

• I expected values related to solving times and comparisons with different cases. 

Since each grid case is independent, it is a difficult task to find a way to compare between them. Regarding solving times, for the cases considered, there is quite similar solve times for traditional solvers and PowerModel-AI. However, we are working on this problem for bigger grids where the difference will become notable. However, there is still issues with finding solutions for bigger grids from iterative solvers and this is the first task we are working out. Since PowerModel-AI is an open-source software, updates to versions will include larger grids and compute times. This paper is a way to introduce our idea to the scholastic community and promote collaboration to improve PowerModel-AI.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

We believe that the revision of the paper has met the predetermined requirements. The conclusions are suggested to be divided into several articles. The figures are still too small.

Author Response

We believe that the revision of the paper has met the predetermined requirements. The conclusions are suggested to be divided into several articles. The figures are still too small.

We thank the Reviewer for helping us make this work better. We have received assurance from the Journal that the figures will be sized appropriately.

Reviewer 2 Report

Comments and Suggestions for Authors

No further comments

Author Response

No further comments

We thank the reviewer for helping make this paper better.

Reviewer 3 Report

Comments and Suggestions for Authors

No comments

Author Response

No comments

We thank the reviewer for helping make this paper better.

Reviewer 4 Report

Comments and Suggestions for Authors

Dear Authors

I agree with the changes.

Author Response

Dear Authors, I agree with the changes.

We thank the reviewer for helping make this paper better.