Simulation-Supported Testing of Smart Energy Product Prototypes

Round  1

Reviewer 1 Report

This paper presents three Smart energy products and services (SEPS) designs and their evaluation by co-simulation, scenario-based simulation and end user testing. Overall, the paper is interesting but it has several shortcomings: 1. The abstract is not in proper shape. The abstract lacks clarity and it doesnt reflect concepts of paper clearly. Further, the narrative is not contineous. 2. Sufficient details of  co-simulation are not presented. The reviewer is not convinced if this is really co-simulation, since SEPS prototype using Rapsberri for updating key indicators is not a simulation. Authors need to provide more details. 3. The introduction section is not properly written. discussion on state of the art is missing. Contributions of this paper are also not explicitly provided.

Author Response

Response to Reviewer 1 Comments

The authors sincerely appreciate your feedback and your useful comments on the manuscript. Here you will find a point-by-point response to your review, we hope to have thoroughly answered your questions.

Note: The title of the manuscript has been modified to “Simulation-supported testing of smart energy product prototypes”. Additionally, the term home energy management product (HEMP) is now used to specifically describe the type of smart product that was tested; smart energy products and services (SEPS) is still used to describe smart products in a broader sense.

Point 1: The abstract is not in proper shape. The abstract lacks clarity and it doesn’t reflect concepts of paper clearly. Further, the narrative is not continuous.

Response 1: We have taken note of this and the abstract has been rewritten to summarise the contents of the paper in a more clear and structured way.

Smart energy products and services (SEPS) have a key role in the development of smart grids, and testing methods such as co-simulation and scenario-based simulations can be useful tools for evaluating the potential of new SEPS concepts during their early development stages. Three innovative conceptual designs for home energy management products (HEMPs) – a specific category of SEPS – were successfully tested using a co-simulation environment, validating their operation using simulated production and load profiles. For comparison with reality, end user tests were carried out on two of the HEMP concepts showing mixed results for achieving a more efficient energy use, with one of the concepts reducing energy consumption by 27% and the other increasing it by 25%. The scenario-based simulations provided additional insights on these products’ performance, matching some of the general trends observed during end user tests but failing to sufficiently approximate the observed results. Overall, the presented testing methods successfully evaluated the performance of HEMPs under various use conditions and identified bottlenecks which could be improved in future designs. It is recommended that in addition to HEMPs, these tests are repeated with different SEPS and energy systems to enhance the robustness of the methods.

Point 2: Sufficient details of co-simulation are not presented. The reviewer is not convinced if this is really co-simulation, since SEPS prototype using Raspberry Pi for updating key indicators is not a simulation. Authors need to provide more details.

Response 2: There might be confusion about the experimental set-up; the co-simulation environment at AIT was used to simulate PV production and household consumption which were the main inputs for testing the physical prototypes in real time. In other words, the simulation was not of the prototypes themselves, rather it was more about modelling the energy profiles the prototypes require to operate.

In order to avoid confusion we’re now describing the tests as taking place in a simulation environment since the use of the core co-simulation features of the lab would require a comprehensive agent-based model on user behaviour and its impact on the residual load profile to upscale the real-world device to several simulated devices. This has been mentioned in the Discussion section as a next research step.

Point 3: The introduction section is not properly written. Discussion on state of the art is missing. Contributions of this paper are also not explicitly provided. 

Response 3: The introduction was streamlined to provide a more adequate context and more clearly indicate the motivation for the presented research. A brief discussion on recent examples of co-simulations and scenario-based simulations applied to smart energy systems with their corresponding references was added as well in Section 1. Additionally, some of the lessons learned from applying these testing methods can now be found on the Discussion and Conclusions section.

Reviewer 2 Report

The authors present a co-simulation approach that can be a useful tool for evaluating the potential of a given SEPS (Smart energy products and services) during its early development stages.

Recommendation: Minor revisions

The overall approach is interesting but the novelty and the progress beyond the existing similar methods is not clear.

·       In introduction section, the background information is not adequate to understand the aims and objectives of the study.

·       Provide some contrete incentives for the development of such methods.

·       A minor issue is how the provided design can affect the actual operation of the proposed devices, which is not straightforward from the provide results.

·       The scientific contribution is not evident. Please provide a concrete analysis of the proposed approach.

·       Authors should review the conclusions section and define which are the main lessons learnt for the S&T community; which is the added value of this work.

Author Response

Response to Reviewer 2 Comments

The authors sincerely appreciate your feedback and your useful comments on the manuscript. Here you will find a point-by-point response to your review, we hope to have thoroughly answered your questions.

Note: The title of the manuscript has been modified to “Simulation-supported testing of smart energy product prototypes”. Additionally, the term home energy management product (HEMP) is now used to specifically describe the type of smart product that was tested; smart energy products and services (SEPS) is still used to describe smart products in a broader sense.

Point 1: In introduction section, the background information is not adequate to understand the aims and objectives of the study.

Response 1: Based on your comment, the introduction was streamlined to provide a more adequate context and more clearly indicate the motivation for the presented research. A brief discussion on recent examples of co-simulation and scenario-based simulations applied to smart energy systems with their corresponding references was added as well in Section 1.

Point 2: Provide some concrete incentives for the development of such methods. 

Response 2: Thanks for pointing the attention to this significant topic. The testing methods we present can provide new tools that can be used to quickly and effectively develop new SEPS solutions. A mention of this has been added in the Introduction section. The success of smart energy systems requires a more active role from end-users, which will become increasingly involved through demand response strategies, energy trading, distributed energy generation, etc. SEPS can enable this process but they need to have a better connection with user expectations and needs in order to overcome the barriers in user acceptance found today.

Point 3: A minor issue is how the provided design can affect the actual operation of the proposed devices, which is not straightforward from the provided results.

Response 3: We have articulated this better in our manuscript. The end-user tests mentioned in the paper also included interviews and surveys evaluating how users responded to the SEPS during their operation, which gave us interesting insights into the their perspective on these devices. For instance, one user found that the prototype lighting can become irritating since it’s always on (especially at night). This could lead to users paying less attention to the SEPS or moving it to a location where its impact is more limited, so further designs should address this issue. While the user experience results are definitely worth looking at, due to the already significant length of the paper we decided to focus more on presenting the technical results obtained rather than on the (more subjective) feedback from users.

Point 4: The scientific contribution is not evident. Please provide a concrete analysis of the proposed approach. 

Response 4: The abstract, introduction and discussion sections have been rewritten to express this more clearly.

Smart energy systems require a more active involvement from end users which is limited by their acceptance of current SEPS, which they frequently find difficult to understand and interact with. This means that new, innovative SEPS that facilitate this role by achieving a better match with user expectations and demands need to be developed. Furthermore, the main contribution of this paper is applying simulation testing methods to innovative SEPS concepts (instead of existing ones like smart thermostats, electric vehicles, etc.) in order to improve their design and make them more effective early in their development process.

Point 5: Authors should review the conclusions section and define which are the main lessons learnt for the S&T community; which is the added value of this work.

Response 5: Thank you, we have updated the conclusion section and have improved it where necessary.

Round  2

Reviewer 1 Report

Thanks for the revisions.

Authors have satisfactorily addressed my concerns from previous review.

Now, paper can be accepted for publication.