Reinforcement Learning: Theory and Applications in HEMS
Round 1
Reviewer 1 Report
Dear Authors,
This paper provided an in-depth review of the applications of reinforcement learning (RL) for home energy management systems (HEMS). However, there is some scope for improvement.
Please mention the full form of the abbreviation DNN on the first page.
Kindly provide a block diagram describing the home energy management system.
On page 3, second sentence, the word "win" should be replaced with "wins".
Table 2 has not been referred to in the text.
Kindly include the conclusion and future work at the end.
Presenting a short commentary on what type of RL technique to select for a given HEMS application will be a good contribution. This can be included after section 7.
Author Response
Please see the attachment.
Author Response File: 
Author Response.docx
Reviewer 2 Report
This is by far one of the highest quality papers in the energy journals I have reviewed. I have a few comments.
1. In Section 2.1, the authors focus on Zigbee. I have researched zigbee 10 years ago, and I am curious if this technology is not yet obsolete? Almost all smart homes are now using wifi for communication.
2. In Section 2.3, don't HEMS control algorithms have methods based on MPC and its variants? Some applications of MPC and its variants are described in this reference ‘Model Controlled Prediction: A Reciprocal Alternative of Model Predictive Control’. I suggest the authors cite this paper in this section and add some description related to MPC.
3. In the second page, the authors give many examples of rl, and I add a class of applications where there are many intelligent transportation algorithms based on rl, such as vehicle dispatching. I suggest the authors cite some relevant papers, such as ‘Deep dispatching: A deep reinforcement learning approach for vehicle dispatching on online ride-hailing platform’.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 3 Report
See the attachment.
Comments for author File: Comments.pdf
Author Response
Please see the attachment
Author Response File: Author Response.docx
Round 2
Reviewer 3 Report
Copyright of Figure 2 must be obtained.
Taxonomy of Deep Reinforcement Learning. Classification of all deep reinforcement learning methods that are described in this article are shown. Section 3.2 provides a description of each class. Also see [64].
Equation 7 and 8 need references.
Add a “year” column in table 1.
Author Response
I would like to thank esteemed Reviewer-3 for the comments. I have tried to address the concerns as best as I can. I hope that Reviewer-3 is satisfied with my point-wise response to each concern below (also see attachment).
"Copyright of Figure 2 must be obtained.
Taxonomy of Deep Reinforcement Learning. Classification of all deep reinforcement learning methods that are described in this article are shown. Section 3.2 provides a description of each class. Also see [64]."
I am attaching the communication with CADS at our university regarding obtaining the copyright for Figure 2. I am the author of Figure 2. Reference [64] is another reference.
"Equation 7 and 8 need references."
I thank the esteemed reviewer-3 for this observation. The reference has been added to the revised version of the manuscript.
"Add a “year” column in table 1."
As I had mentioned in my earlier response to the esteemed reviewer, with rare exceptions, all references span a short time period 2017 - 2022. Ergo, adding a new column (publication year), to Table 1 does not add any significant new information to Figure 1.
Furthermore, inserting another column to Table 1 would require me to reduce the font size which is contrary to MDPI guidelines.
However, I will make a decision depending on the other reviewers' comments. If the version (to be submitted) does not include this column, and if the esteemed reviewer-3 still wants me to add it, I will do so in the next round of review.
Author Response File: Author Response.pdf
Round 3
Reviewer 3 Report
Accept
