The Potential of a Mobile Augmented Reality Game in Education for Sustainability: Report and Analysis of an Activity with the EduCITY App

Round 1

Reviewer 1 Report (Previous Reviewer 1)

Comments and Suggestions for Authors

The article has been significantly improved; however, it still has grammatical, spelling, and punctuation errors. In addition, the authors should use the same font throughout the article. For example, Tables 1, 2, and 3 have a different font. Also, why is Table 3 presented in the appendix section?

Comments on the Quality of English Language

The English could be improved to more clearly express the research.

Author Response

Comments 1: The article has been significantly improved; however, it still has grammatical, spelling, and punctuation errors. In addition, the authors should use the same font throughout the article. For example, Tables 1, 2, and 3 have a different font. Also, why is Table 3 presented in the appendix section?

Response 1: Thank you for pointing this out. We agree with this comment. We have standardised all the tables. Table 3 is in the appendix because although it is very useful, it is not essential for understanding the paper. It is too extensive and detailed to be inserted into the main body of the article without interrupting the flow of the reading.

Point 1: The English could be improved to more clearly express the research.

Response 1: The text has all been re-read and improved.

Reviewer 2 Report (Previous Reviewer 2)

Comments and Suggestions for Authors

This work describes an experience of implementing a mobile augmented reality application. It shows qualitative results of the use of the app and the students' perceptions. AR used as an educational tool is a very interesting topic, however there are important things to be covered by authors yet. Below are different comments about the work.

The paragraphs that talk about Educity (before section 2.4) should appear in a discussion section (because they contrast Educity with other works, or in section 2.4 (because they describe Educity).

A hypothesis or, at least, a research question should be stated before section 3 so to present the forces that lead to the experiment authors performed. This hypothesis or research question should be linked to the first sentence in section 3 "A case study was conducted to **ascertain the perceptions of students**". This is done in lines 296 to 303. These lines should appear before section 3. Maybe, in a section on their own.

The acronym UA is presented until line 312 but is used many times before. 

Text in lines 379 to 381 should appear at the end of section 3.3, after the description of the questions. 

There are many pages talking about things that are placed in an appendix so maybe this table 3 should not be in an appendix.

The results and discussion section presents only results and authors' assumptions observed in the results, however, these elements are not contrasted to other works or other authors. This is important because this comparison will show the effectiveness of the tool and will point to the contribution of the work. This is the main thing to add to the work.

Author Response

Comments 1: The paragraphs that talk about Educity (before section 2.4) should appear in a discussion section (because they contrast Educity with other works, or in section 2.4 (because they describe Educity).

Response 1: Agree. We have already changed the text and moved the part of the text before section 2.4 and put it in that section. We also revised the text. 

Comments 2: A hypothesis or, at least, a research question should be stated before section 3 so to present the forces that lead to the experiment authors performed. This hypothesis or research question should be linked to the first sentence in section 3 "A case study was conducted to **ascertain the perceptions of students**". This is done in lines 296 to 303. These lines should appear before section 3. Maybe, in a section on their own.

Response 2: Thank you for pointing this out. The research question is in section 3 on lines 297-300. The hypothesis is in lines 300-303.

Comments 3:The acronym UA is presented until line 312 but is used many times before. 

Response 3: Thank you for pointing this out. We made that correction. The first time the acronym UA appears on lines 58 and 59, its definition is given: University of Aveiro.

Comments 4: Text in lines 379 to 381 should appear at the end of section 3.3, after the description of the questions. 

Response 4: Thank you for pointing this out. In this case, the text in lines 379 to 381 appears at this point because it serves to summarise the questionnaire used and then explain it in more detail.

Comments 5: There are many pages talking about things that are placed in an appendix so maybe this table 3 should not be in an appendix.

Response 5: Thank you for pointing this out. Table 3 is in the appendix because although it is very useful, it is not essential for understanding the paper. It is too extensive and detailed to be inserted into the main body of the article without interrupting the flow of the reading.

Comments 6: The results and discussion section presents only results and authors' assumptions observed in the results, however, these elements are not contrasted to other works or other authors. This is important because this comparison will show the effectiveness of the tool and will point to the contribution of the work. This is the main thing to add to the work.

Response 6: Thank you for pointing this out. Wereread the articles in the bibliography in order to contrast them with the results presented in these articles. Thus, the results and discussion section present the results of this study in contrast to other works or other authors.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The study has good intentions, but from my point of view, it needs major adjustments before publication. I consider that there is no traceability between the purpose of the study, the methods, and the results. Therefore, the benefits of the study to the fields are unclear.

1)      In the title, the authors propose to analyze “The impact of an Augmented Reality mobile game in education for sustainability”. Then, in the abstract, they propose to identify the “potential of the of the ‘EduCITY at the UA Campus’ mobile AR game …”. They should be consistent in the purpose: to measure the impact of to measure the potential. Additionally, the authors do not mention how to measure one variable or the other.

Furthermore, it is awkward to present the research question in the abstract.

 

2)      Section 1.1 seems unnecessary. Instead, the authors should present, in a paragraph, what the purpose of the study is. In this paragraph, the authors should clearly describe what they are going to measure, and how they will do it.

 

3)      Lines 161 to 171 (Methods and Materials) are unnecessary and should be removed from this section.

 

4)      Section 3: Materials and Methods should be reorganized. Authors should create different subsections of 1) procedure, 2), participants, and 3) instruments.

 

5)      I could not understand what variables were measured: Knowledge? If it is true, this variable should have been declared in the purpose of the study. Moreover, if the measured variable was knowledge, how the authors guarantee the absence of bias? There were no pre-tests or control treatments.

 

6)      It is well known that in terms of environmental education what matters is not so much knowledge but motivation. In this sense, I recommend that the authors add this variable to the study.

Comments on the Quality of English Language

The paper has flaws regarding grammar, spelling, and punctuation. This makes the readability of the document difficult.

Author Response

Comment 1: The study has good intentions, but from my point of view, it needs major adjustments before publication. I consider that there is no traceability between the purpose of the study, the methods, and the results. Therefore, the benefits of the study to the fields are unclear

Response 1: Thank you for pointing this out. The paper was majorly revised articulating the of the study, the methods, and the results. The benefits of the study are in conclusions chapter in lines 719-809.

Comment 2: In the title, the authors propose to analyze “The impact of an Augmented Reality mobile game in education for sustainability”. Then, in the abstract, they propose to identify the “potential of the of the ‘EduCITY at the UA Campus’ mobile AR game …”. They should be consistent in the purpose: to measure the impact of to measure the potential. Additionally, the authors do not mention how to measure one variable or the other. Furthermore, it is awkward to present the research question in the abstract.

Response 2: Thank you for pointing this out. We agree, the study is to ascertain the potential of the game integrated into the EduCITY app, not the impact. The abstract and entire text has been changed to align it with its main objective.

Comment 3: Section 1.1 seems unnecessary. Instead, the authors should present, in a paragraph, what the purpose of the study is. In this paragraph, the authors should clearly describe what they are going to measure, and how they will do it.

Response 3:Thank you for pointing this out. We clarified the purpose of the study and its objectives.

Comment 4: Lines 161 to 171 (Methods and Materials) are unnecessary and should be removed from this section.

Response 4: Thank you for pointing this out. We've already removed that part.

Comment 5: Section 3: Materials and Methods should be reorganized. Authors should create different subsections of 1) procedure, 2), participants, and 3) instruments.

Response 5: Thank you for pointing this out. Chapter 3 ‘Materials and Methods’ is organized into three sub-chapters: 3.1. Procedure of "EduCITY at UA Campus" game in the XPERiMENTA’24 activity, 3.2. Participants in the "EduCITY at UA Campus" game in the XPERiMENTA’24 activity, 3.3. Data collection instruments

Comment 6: I could not understand what variables were measured: Knowledge? If it is true, this variable should have been declared in the purpose of the study. Moreover, if the measured variable was knowledge, how the authors guarantee the absence of bias? There were no pre-tests or control treatments.

Response 6: Thank you for pointing this out. The objective of this study is to analyse the potential of the 'EduCITY at the UA Campus' mobile augmented reality game in terms of its capacity to promote education for sustainability among secondary students. To this end, data were collected through the game integrated into the app and through a questionnaire administered at the conclusion of the activity.

Comment 7: It is well known that in terms of environmental education what matters is not so much knowledge but motivation. In this sense, I recommend that the authors add this variable to the study.

Response 7: Thank you for pointing this out. We've revised the entire text and added lines 26-32, 90-95, 343-348, 90-92, 100-104, 505-507, 710-714, 763-765, 800-809.

Comments on the Quality of English Language: The paper has flaws regarding grammar, spelling, and punctuation. This makes the readability of the document difficult.

Response:Thank you for pointing this out. The text has been revised again by two different researchers and we also used the DeepL Write to revise the text.  

Reviewer 2 Report

Comments and Suggestions for Authors

This work describes an experience of implementing a mobile augmented reality application. It shows qualitative results of the use of the app and the students' perceptions. AR use as an educational tool is a very interesting topic, however, the organization of the document does not show a clear research objective. Below are different comments about the work.

Introduction needs more deepness. Authors have to state the problem that they are trying to solve. Without this, and with the contributions they list, the objectives of this work seem to be only to promote their AR game.

Introduction also needs to present an analysis of the state of the art and what is missing on it. This will help establish the problem a then, derive the contributions of the work. 

Also, a hypothesis should be stated.

The statement "The learning process becomes easier, more accurate, enjoyable, and engaging compared to traditional methods", in lines 43 and 44, needs a reference.

Define gamification before mentioning it in line 71.

Reference [11] in line 161 should be at the end of the sentence, in line 163.

Reference in line 166 should not appear there because the sentence is a conclusion of the authors and it does not appear in works [11] and [12].

Question presented in line 169 should be transformed into an hypothesis.

Do you have parental consent to publish the photo in figure 4? The students are minors.

What was the objective of making the students play the game? It was part of a class? It was part of the event? They were volonteers? Describe more the sample. 

Results presented in lines 272 to 285 should be presented in a statistical way (mean, standard deviation, etc.)

How come there are 27 students and they have 33 answers (line 365)?

There is no discussion that compares results to other works in order to know the quality of the work and its results.  

Complement conclusions with limitations of the work.

Comments on the Quality of English Language

Mostly, there are errors in conjugation.

Author Response

Comment 1:  Introduction needs more deepness. Authors have to state the problem that they are trying to solve. Without this, and with the contributions they list, the objectives of this work seem to be only to promote their AR game.

Response 1: Thank you for pointing this out. We've reformulated the introduction and added lines 26-32.

Comment 2: Introduction also needs to present an analysis of the state of the art and what is missing on it. This will help establish the problem a then, derive the contributions of the work. 

Response 2: Thank you for pointing this out. To this end, we've read more documents for the state of the art and added new chapter “Related work” and new subchapters: 2.1. Augmented Reality in education; 2.2. Mobile Augmented Reality Games; 2.3. Mobile Augmented Reality Games into education for sustainability and 2.4. The EduCITY Context. See the lines 76 – 290.

Comment 3: Also, a hypothesis should be stated.

Response 3: Thank you for pointing this out. We've reformulated the Materials and Methods chapter and added lines 299-302.

Comment 4: The statement "The learning process becomes easier, more accurate, enjoyable, and engaging compared to traditional methods", in lines 43 and 44, needs a reference.

Response 4: Thank you for pointing this out. The reference is the same as in the previous sentence, we've added it at the end (line 95).

Comment 5: Define gamification before mentioning it in line 71.

Response 5: Thank you for pointing this out. The expression mobile game learning better define what we want to express since we have added references to mobile augmented reality games. We've changed it in line 127.

Comment 6: Reference [11] in line 161 should be at the end of the sentence, in line 163.

Response 6: Thank you for pointing this out. A review of the comments from the reviewers revealed the necessity for the removal of this initial section and the subsequent reformulation of the text.

Comment 7: Reference in line 166 should not appear there because the sentence is a conclusion of the authors and it does not appear in works [11] and [12].

Response 7: Thank you for pointing this out. A review of the comments from the reviewers indicated that this initial section should be removed.

Comment 8: Question presented in line 169 should be transformed into an hypothesis.

Response 8: Thank you for pointing this out. We have included an hypothesis in the lines 301-304.

Comment 9: Do you have parental consent to publish the photo in figure 4? The students are minors.

Response 9: Thank you for pointing this out. To take part in the activity, the parents signed an informed consent form, but as there was no justification for keeping the students' faces, we added blur.

Comment 10: What was the objective of making the students play the game? It was part of a class? It was part of the event? They were volonteers? Describe more the sample. 

Response 10: Thank you for pointing this out. As part of the XPERiMENTA’24 event, an educational activity was created using the EduCITY app for secondary students. The students registered for and participated in the ‘EduCITY at the UA Campus’ game (lines 356-371).

Comment 11: Results presented in lines 272 to 285 should be presented in a statistical way (mean, standard deviation, etc.)

Response 11: Thank you for pointing this out. The results have been subjected to a rigorous review, and the incorporation of statistical analysis, as observed in the 412-419 lines, has been duly considered.

Comment 12: How come there are 27 students and they have 33 answers (line 365)?

Response 12: Thank you for pointing this out. In this question, the students were required to identify two or three of the principal concepts that they had acquired from the game. In this regard, some students identified two ideas, others three, and others just one. This resulted in a total of more than 27 answers.

Comment 13: There is no discussion that compares results to other works in order to know the quality of the work and its results.  

Response 13: Thank you for pointing this out. In this revised version of the paper, we have conducted further research into related literature in order to ensure its academic rigor. This work is along the lines 76-237.

Comment 14: Complement conclusions with limitations of the work.

Response 14: Thank you for pointing this out. We have added the limitations that go from lines 793-800.

Comments on the Quality of English Language: Mostly, there are errors in conjugation.

Response: Thank you for pointing this out. The text has been revised again by two different researchers and we also used the DeepL Write to revise the text.