Optimizing Home Energy Flows and Battery Management with Supervised and Unsupervised Learning in Renewable Systems

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The work is very interesting and concerns important social aspects. Therefore, I think it is valuable, and the content shows that the authors have put in a lot of work into it. However, I have some minor suggestions:
1. The introduction is very nicely written, but in my opinion a bit too much. I recommend shortening it
2. Figure 1 is excessively stretched, which disturbs the proportions of the text. Also, is it made by the authors? If not, you need to provide a source
3. Line 141 – too much space before the text
4. In line 145, the authors immediately provide conclusions "It not only evaluates their effectiveness in 145 reducing energy costs, enhancing system adaptability, and responding to real-time 146 fluctuations in energy supply and demand but also emphasizes the potential of RL in 147 designing cost-based sizing and energy management Framework". Here, you should focus only on the purpose of the work
5. Figure 2 has no source, unless it is yours. In addition, the text size in the figures should be the same as the text in the manuscript - please change it everywhere
6. Line 221 - the source of these prices and the place where they are valid should be given
7. Figure 5 is difficult to read
8. The results are described a bit chaotically, which makes it difficult to interpret

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The paper presents management strategies for optimising battery energy storage in residential systems with photovoltaic (PV) power, grid interconnection, and dynamic or fixed electricity pricing. The authors used reinforcement learning (RL) and fuzzy logic control (FLC) in their analysis.

The manuscript needs to be improved, clarification and completions are necessary.

The nominal technical characteristics of the components of the residential energy system should be provided.

The authors should provide more details about the electricity market with dynamic prices. Specify the source of the electricity prices in Figure 4.

Your work would be clearer if you also provided the energy flow balance diagrams. What were the energy quantities (energy generated by the photovoltaic system; stored energy; energy exported to the grid;  energy imported from the grid; the overall energy consumption; stored energy used for consumption) during the analysis period?

How was the energy demand of the residential consumer covered from the energy sources (PV; battery; public grid)? A graphical representation would be useful.

Figure 10: The unit of measurement on the vertical axis is missing.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The paper presents a well-structured framework for evaluating reinforcement learning (RL) and fuzzy logic control (FLC) in residential energy management. However, there are several aspects that require further clarification and improvement:

 

1. The paper describes the use of the Proximal Policy Optimization (PPO) algorithm for reinforcement learning but does not provide any discussion on the training convergence. There is no mention of how many training iterations were required for the PPO agent to stabilize, nor are there any learning curves to demonstrate the evolution of the policy over time. It is important to include convergence plots (e.g., reward vs. training epochs) and clarify the criteria used to determine when the training process has converged. Additionally, details on hyperparameter tuning (e.g., learning rate, discount factor, exploration-exploitation balance) should be provided to ensure the reproducibility of results.
2. The FLC controller in this study only considers battery state-of-charge (SOC) and electricity price in decision-making, but it does not take into account household energy demand. In practical applications, demand profiles play a critical role in determining when and how much energy should be stored or discharged. The current FLC design may result in suboptimal decisions (e.g., charging the battery when household demand is high, leading to grid dependence).
3. Line 324, eq (5): This formulation may distort performance assessment since the denominator includes cost, SOC stability, and battery usage, all with different units and scales. How were these weight factors determined? Were they fine-tuned using a systematic method such as grid search or sensitivity analysis?
4. The study utilizes randomly generated price curves, but in many electricity markets, real-time price data from wholesale electricity markets (e.g., Australian NEM, PJM in the U.S., Nord Pool in Europe) is publicly available. Using real-world electricity price profiles would enhance the practical relevance and applicability of the proposed energy management strategy. The authors should clarify why they chose random price profiles instead of real-world price data.
5. In the introduction, the paper discusses MARL but lacks a reference to studies that explore MARL’s application in competition of DERs. I recommend citing: Assembly and competition for virtual power plants with multiple ESPs through a recruitment–participation approach. This paper models the aggregation and competition process of ESS and PV units, demonstrating MARL’s potential in optimizing distributed energy resource management.

 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report

Comments and Suggestions for Authors

The paper presents solid research and analysis on optimizing battery management in residential systems with integrated photovoltaic (PV) solar installations, using Reinforcement Learning (RL) and Fuzzy Logic Control (FLC). The authors conducted 24-hour simulations to evaluate different battery charge levels (State of Charge, SOC), analyze electricity demand, and assess electricity prices—both dynamic and fixed.  In the applied model, FLC allows for a quick recovery of the battery charge level but causes cost fluctuations and greater battery wear. In contrast, the RL model dynamically adjusts the charging and discharging processes, reducing costs and moderating electricity use. The results highlight that the RL model is more cost-efficient and better at preserving battery life, whereas the FLC model responds better to low charge levels. In the end, the authors suggest a more or less hybrid approach to maximize battery performance and lifespan in these photovoltaic systems. However, the authors need to better justify the use of Proximal Policy Optimization (PPO), as they mention that PPO is suitable for energy management but lack details on why it might be better than other variants like Deep Deterministic Policy Gradient (DDPG). Regarding the implementation and comparison of FLC, the predefined rules could be a limitation—using fixed rules may not be the most suitable approach in dynamic scenarios. The paper would improve by incorporating an analysis of hybrid techniques such as neuro-fuzzy systems or adapting the rules based on heuristics. Additionally, selecting only three SOC levels and two price levels might be too simplistic for a realistic analysis, especially considering how Depth of Discharge (DOD) works in batteries and how electricity prices fluctuate in energy markets. A more detailed analysis with additional classification levels could enhance the accuracy of the proposed approach.

The authors have conducted a 24-hour simulation, but to better assess the stability and robustness of the models, it would be advisable to simulate additional scenarios planning several days or even weeks. This would provide a clearer understanding of the applied models behavior and the charge-discharge cycles of the batteries over extended periods. It is crucial to consider that household energy consumption can vary significantly throughout the day, and even more from one day to another. For instance, a household's weekday consumption profile can differ greatly from its weekend consumption pattern.

Regarding the experimental validation of the analyzed systems, the paper appears to present only MATLAB simulations, but it is unclear whether real household energy consumption data was used. If not, incorporating real data in the analysis is suggested to strengthen the model's validity. It would also be beneficial to compare the results with other possible approaches such as Mixed-Integer Linear Programming (MILP) or metaheuristic algorithms. This would make the results more representative.  The paper would also improve if the authors discussed the computational cost, providing information on the training time of the RL model and the computational resources required. These parameters are crucial for evaluating the feasibility of implementing RL in real-world environments.

Finally, the paper lacks a detailed discussion on the potential real-world implementation of these two management models. Could these algorithms be integrated into real PV systems to improve energy management in smart homes? If so, how? Where would the developed software or programming be installed? Would it be integrated with other equipment such as electrical inverters, or within the Battery Management System (BMS) of the batteries? This is important, as today there is specific compatibility between electrical inverters (DC-AC) and lithium-based battery technologies.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The paper has been revised and improved.

Author Response

We sincerely appreciate your time and effort in reviewing our revised manuscript. We are pleased to hear that you find the revisions to be an improvement. Your valuable feedback has significantly contributed to enhancing the clarity, quality, and overall impact of our work.

Thank you again for your insightful comments and support throughout the review process.

Reviewer 3 Report

Comments and Suggestions for Authors

Authors have solved my issues and current version is well for acceptance.

Author Response

We sincerely appreciate your thoughtful review and positive feedback. We are glad to hear that our revisions have successfully addressed your concerns and that the current version meets the acceptance criteria.

Thank you for your time, effort, and valuable insights throughout the review process. Your comments have significantly contributed to improving the quality of our work.

Reviewer 4 Report

Comments and Suggestions for Authors

The authors have significantly improved the paper, but they have left out some paragraphs like this at the end: Funding: Please add: “This research received no external funding” or “This research was funded by NAME OF FUNDER, grant number XXX” and “The APC was funded by XXX”. Check carefully that the details given are accurate and use the standard spelling of funding agency names at https://search.crossref.org/funding. Any errors may affect your future funding.” These may be from the template, and they should be removed. The authors then add the acknowledgments for the grants.

Author Response

We appreciate the reviewer’s careful assessment and constructive feedback. We acknowledge the omission of the Funding Statement and have now included the necessary information.

we have added:
“This research received no external funding.”

Additionally, we have now reviewed and updated the acknowledgments to correctly recognize grant support where applicable.

Thank you for bringing this to our attention. We appreciate your guidance in ensuring all necessary details are accurately presented.