Development and Field Demonstration of a Gamified Residential Demand Management Platform Compatible with Smart Meters and Building Automation Systems

Round 1

Reviewer 1 Report

Authors of the work “Development and Field Demonstration of a Gamified Residential Demand Management Platform Compatible with Smart Meters and Building Automation Systems” propose in this paper design, development and field demonstration of a gamified demand management platform with device-specific goals, compatible with building automation systems and smart meters.

Overall, it is a well-structured paper; the introduction section is wide, and presents the purpose of the research in detail. There is a comparison and evaluations of the proposed method, using proper figures and showing up the proofs of the experiments included in the paper.

Although the proposal is interesting and within the scope of the Energies Journal, there are different issues that should be addressed in order to improve the work.

[Minor comments]

·        Figure 3 image could be bigger.

·        Figure 1, 2 and 4 do not allow the compression of your information.

·        It is recommended that a thorough comparison be made with other related work, so that the improvement proposed in this work is evident.

The used English is correct.

Author Response

Responses to Reviewer 1 Comments:

We are thankful to Reviewer 1 for his/her recommendations. Please find below a list of the related revisions addressing the comments.

Although the proposal is interesting and within the scope of the Energies Journal, there are different issues that should be addressed in order to improve the work.

[Minor comments]

· Figure 3 image could be bigger.

Answer 1) Thank you. Figure 3 image size is increased and Figure 4 in the old version is modified, becoming a subfigure of Figure 3, to provide a comparative reflection between the sectional illustration and the implemented work.

· Figure 1, 2 and 4 do not allow the compression of your information.

Answer 2) We agree with Reviewer 2. Figure 1 in the old version is deleted. A sectional illustration is added as a subfigure of Figure 2 (the new Figure 1, as the previous figure is deleted). A new figure (Figure 2 in the revised version) is added, showing the activity cycles of the gamified platform in categories. The old Figure 4 became the subfigure of Figure 3, showing the sectional illustrations and implementation screenshots next to each other.

· It is recommended that a thorough comparison be made with other related work, so that the improvement proposed in this work is evident.

Answer 3) Thank you for this recommendation. We added a whole new section (Section 2. Related Works), mentioning 7 related studies together with the findings of 4 very recent review papers on gamified energy monitoring and management platforms and 2 studies on high resolution detailed energy consumption monitoring in buildings. The section also includes a table (Table 1 in the revised version) about the 7 key papers. The section has the paragraphs given below emphasizing improvements proposed in our study:

The related works mainly targeted consumption savings in houses, except [7] in which also demand flexibilities are aimed to be improved to achieve financial incentives. Dominantly overall consumption of the whole building or a building section is monitored, while device-level consumption monitoring is conceptually studied in [7] and demonstrated in the field in [16]. The longest trial period was 13 weeks in [15], while in most of the other works it did not exceed 1 month. A few gamification elements were used per study and impact analysis did not go beyond interviews, surveys and consumption comparisons. Moreover design and development stages were not described in details with adopted gamification methodologies.

A review of 25 studies was presented in [17], dividing gamification-based studies into two categories as digital game world dominant examples with some integration with the real world and games that are fully integrated in the real world. The same study stated that, most of the papers have “poorly described methodologies” and “limited use of validated measures to quantify outcomes”.

The review in [18] with 10 reviewed studies pointed out that, most of the studies focused on usability and game structure without any evaluation, while several studies relied on self-reported data and interviews, making it challenging to clearly evaluate the impacts of gamified solutions. It was also indicated that integration of sensors and meters can allow non-invasive data collection.

Our study addresses the gaps identified in [17] and [18] by explaining the systematic development of a fully real world integrated gamified platform in details with adopted methodologies. Moreover, detailed analysis is conducted using several measures, collected through device-level meters with high resolution (5 minutes), building automation system sensors (event-triggered instant data transfer and recording) and platform integrated digital activity logs (event-triggered instant data recording), to quantify the outcomes.

Our study contributes to the literature by exploring device-level demand flexibilities, using a wide range of gamification elements, describing systematic gamified system development in details with adopted methodologies, demonstrating in the field for 16 weeks and quantitatively analysing impacts in details to highlight correlation between gamified platform usage and load management performance.

Author Response File: Author Response.pdf

Reviewer 2 Report

This is a very interesting paper. The concept of gamification for tracking energy usage is a nice idea. The study does a small-scale evaluation as well. I would recommend accepting the paper with the following revisions:

1) The introduction is quite long. Perhaps condense it to keep the focus of the reader.

2) Include a related work section to highlight some of the relevant works.

3) Add some references that have done similar works such as

- Ankur Sial et al., Heuristics-Based Detection of Abnormal Energy Consumption, Proc. SMARTGIFT, 2018.

Ankur Sial et al., Profiling Energy Consumption in a Residential Campus. Proc. CoNEXT student workshop, 2014.

4) Some minor proof-reading can be done to improve the presentation of the paper.

5) Include a summary table showing the key papers on energy conservation thru gamification techniques.

Author Response

Responses to Reviewer 2 Comments:

We would like to thank to Reviewer 2 for his/her valuable comments and suggestions. We made several revisions, addressing the recommendations. The revisions are listed below:

This reviewer has two concerns:

1) The introduction is quite long. Perhaps condense it to keep the focus of the reader.

Answer 1) Introduction section is shortened by reducing the general comments about the literature from 36 lines to 11 lines (25 lines, approximately 20% shorter), as relevant works are highlighted in the recently added related works section (section 2).

2) Include a related work section to highlight some of the relevant works.

Answer 2) We would like to thank Reviewer 1 for this suggestion. We added a related works section, highlighting 7 relevant works, mentioning the findings of 4 recent review studies on gamified energy platform and emphasizing the importance of high resolution detailed building energy consumption monitoring citing 2 studies.

3) Add some references that have done similar works such as

- Ankur Sial et al., Heuristics-Based Detection of Abnormal Energy Consumption, Proc. SMARTGIFT, 2018.

Ankur Sial et al., Profiling Energy Consumption in a Residential Campus. Proc. CoNEXT student workshop, 2014.

Answer 3) Thank you for suggesting similar works. We added them to new “related works” section, in addition to 7 other similar works and 4 review studies.

4) Some minor proof-reading can be done to improve the presentation of the paper.

Answer 4) A thorough proof-reading is done with several minor corrections and modifications.

5) Include a summary table showing the key papers on energy conservation thru gamification techniques.

Answer 5) Thank you for this suggestion. We added a table (new Table 1) to the new “related works” section, summarizing 7 papers with considerable contributions to the related area of study.

Responses to Reviewer 1 Comments:

We are thankful to Reviewer 1 for his/her recommendations. Please find below a list of the related revisions addressing the comments.

Although the proposal is interesting and within the scope of the Energies Journal, there are different issues that should be addressed in order to improve the work.

[Minor comments]

· Figure 3 image could be bigger.

Answer 1) Thank you. Figure 3 image size is increased and Figure 4 in the old version is modified, becoming a subfigure of Figure 3, to provide a comparative reflection between the sectional illustration and the implemented work.

· Figure 1, 2 and 4 do not allow the compression of your information.

Answer 2) We agree with Reviewer 2. Figure 1 in the old version is deleted. A sectional illustration is added as a subfigure of Figure 2 (the new Figure 1, as the previous figure is deleted). A new figure (Figure 2 in the revised version) is added, showing the activity cycles of the gamified platform in categories. The old Figure 4 became the subfigure of Figure 3, showing the sectional illustrations and implementation screenshots next to each other.

· It is recommended that a thorough comparison be made with other related work, so that the improvement proposed in this work is evident.

Answer 3) Thank you for this recommendation. We added a whole new section (Section 2. Related Works), mentioning 7 related studies together with the findings of 4 very recent review papers on gamified energy monitoring and management platforms and 2 studies on high resolution detailed energy consumption monitoring in buildings. The section also includes a table (Table 1 in the revised version) about the 7 key papers. The section has the paragraphs given below emphasizing improvements proposed in our study:

The related works mainly targeted consumption savings in houses, except [7] in which also demand flexibilities are aimed to be improved to achieve financial incentives. Dominantly overall consumption of the whole building or a building section is monitored, while device-level consumption monitoring is conceptually studied in [7] and demonstrated in the field in [16]. The longest trial period was 13 weeks in [15], while in most of the other works it did not exceed 1 month. A few gamification elements were used per study and impact analysis did not go beyond interviews, surveys and consumption comparisons. Moreover design and development stages were not described in details with adopted gamification methodologies.

A review of 25 studies was presented in [17], dividing gamification-based studies into two categories as digital game world dominant examples with some integration with the real world and games that are fully integrated in the real world. The same study stated that, most of the papers have “poorly described methodologies” and “limited use of validated measures to quantify outcomes”.

The review in [18] with 10 reviewed studies pointed out that, most of the studies focused on usability and game structure without any evaluation, while several studies relied on self-reported data and interviews, making it challenging to clearly evaluate the impacts of gamified solutions. It was also indicated that integration of sensors and meters can allow non-invasive data collection.

Our study addresses the gaps identified in [17] and [18] by explaining the systematic development of a fully real world integrated gamified platform in details with adopted methodologies. Moreover, detailed analysis is conducted using several measures, collected through device-level meters with high resolution (5 minutes), building automation system sensors (event-triggered instant data transfer and recording) and platform integrated digital activity logs (event-triggered instant data recording), to quantify the outcomes.

Our study contributes to the literature by exploring device-level demand flexibilities, using a wide range of gamification elements, describing systematic gamified system development in details with adopted methodologies, demonstrating in the field for 16 weeks and quantitatively analysing impacts in details to highlight correlation between gamified platform usage and load management performance.

Author Response File: Author Response.docx