Presence and Flow as Moderators in XR-Based Sustainability Education

Round 1

Reviewer 1 Report

The research presented provides indications relating to the possibility of using new digital technologies in teaching. The research is mainly focused on the correspondence between teaching objectives and learning outcomes. The only critical observation that can be made concerns the failure to take into account the study of the development of students' abilities to practice forms of critical tinking and learning when they use new types of digital education objects. The metacognitive dimension present in particular in all settings that impose themselves by their claim to be "reality" constitutes a dimension that could be explored in depth or at least detected as a research perspective

Author Response

Dear Reviewer 1,

thank you very much for your feedback on our study. We appreciate the time you invested. We have made an effort to appropriately address your comments and have created a table for this purpose. On the left side, you will find the comments of all reviewers, and on the right side, our responses. In the manuscript itself, we have highlighted in yellow all the changes we referred to in the cover letter, making it easier for you to trace the modifications. If there are further aspects we should revise, we are more than willing to do so. We are available for additional feedback at any time.

Best regards!

Author Response File: Author Response.pdf

Reviewer 2 Report

This study focuses on the use of a 3D virtual environment / VR app in sustainable education, a timely and important topic. However, the manuscript should address several issues to be considered for publication:

1.      One important issue is that the used app appears in fact to be a VR / 3D virtual application and more specifically and non-immersive one as it operates on tablets. If no AR elements are present, this aspect should be revised throughout the whole paper, including the title.

2.      Second, can you please explain why you formulated different RQs for the qualitative data analysis? Normally, overall RQs are formulated and then the appropriate data analysis methods are selected to tackle them. Can you cite other published works in Scopus-indexed high-quality journals with IF that followed this approach?

3.      The introduction is quite short and it does not establish effectively the need or gap of the current study. Moreover, it contains two questionable statements:

4.      L29 “However, the distinction between VR and AR technologies is somewhat blurry” This statement is scientifically inaccurate and should be revised. AR and VR are distinct technologies, at the opposite ends of Milgram’s continuum [1].

5.      L28 “their educational potential has not been thoroughly investigated” this statement is also problematic. Over 12 systematic reviews have been published on AR in Education [2]. Even more surveys on VR in Education have been published e.g. [3–5].

6.      An ambiguity around AR/VR is also apparent on L55-62. This paragraph should be reformulated to become clearer by not interweaving references to AR and VR. In this context, it could be useful to mention as another type of VR, namely social VR platforms or virtual worlds which is a strategic priority of European Union’s policy [6].

7.      For the better flow of the paper, I recommend explaining the materials first, hence moving section 4.2 before 4.1.

8.      The description of the used application by Greenpeace should be expanded. How does learning take place? What do teachers and students do? How do they interact with the app? Is there a tangible goal of the app use? Are there other relevant research studies with this app?

Minor issues:

L61: Hololens

References

1.          Milgram, P. et al. Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum. In Proceedings of the Proceedings of Telemanipulator and Telepresence Technologies; Das, H., Ed.; December 21 1995; pp. 282–292.

2.          Mystakidis, S. et al. A Systematic Mapping Review of Augmented Reality Applications to Support STEM Learning in Higher Education. Educ Inf Technol (Dordr) 2022, 27, 1883–1927, doi:10.1007/s10639-021-10682-1.

3.          Won, M. et al. Diverse Approaches to Learning with Immersion Virtual Reality Identified from a Systematic Review. Comput Educ 2022, 104701, doi:10.1016/j.compedu.2022.104701.

4.          Beck, D. et al. Educational Practices and Strategies with Immersive Learning Environments: Mapping of Reviews for Using the Metaverse. IEEE Transactions on Learning Technologies 2023, 1–23, doi:10.1109/TLT.2023.3243946.

5.          Pellas, N. et al. Immersive Virtual Reality in K-12 and Higher Education: A Systematic Review of the Last Decade Scientific Literature. Virtual Real 2021, 25, 835–861, doi:10.1007/s10055-020-00489-9.

6.          Hupont Torres, I. et al. Next Generation Virtual Worlds: Societal, Technological, Economic and Policy Challenges for the EU ; Publications Office of the European Union: Luxembourg (Luxembourg), 2023;

Author Response

Dear Reviewer 2,

thank you very much for your feedback on our study. We appreciate the time you invested. We have made an effort to appropriately address your comments and have created a table for this purpose. On the left side, you will find the comments of all reviewers, and on the right side, our responses. In the manuscript itself, we have highlighted in yellow all the changes we referred to in the cover letter, making it easier for you to trace the modifications. If there are further aspects we should revise, we are more than willing to do so. We are available for additional feedback at any time.

Best regards!

Author Response File: Author Response.pdf

Reviewer 3 Report

Continuing on the presented topic, I suggest to present a better connection between the research questions and the theoretical background.

It means that the author/authors must make a clear chapter of  the interpretation of the research results where the same ones will continue on the theory thesis background and on the other side the clear issues for practical operationalization on the education level context.

The interpretation is meaning to say very low quality because the mentioned issues in the paper are not connected.

In that context the conclusion must be wider also where the representation of thoughts will be continued on the research tasks.

Author Response

Dear Reviewer 3,

thank you very much for your feedback on our study. We appreciate the time you invested. We have made an effort to appropriately address your comments and have created a table for this purpose. On the left side, you will find the comments of all reviewers, and on the right side, our responses. In the manuscript itself, we have highlighted in yellow all the changes we referred to in the cover letter, making it easier for you to trace the modifications. If there are further aspects we should revise, we are more than willing to do so. We are available for additional feedback at any time.

Best regards!

Author Response File: Author Response.pdf

Reviewer 4 Report

Title: Presence and Flow as Moderators in XR-based Sustainability Education

This study aims to discuss how new technologies are revolutionizing today’s sustainable education. Examining the learning process through the lens of flow and presence has yielded some results that could contribute to advances in the field of ICT-enhanced learning.

Nevertheless, there are some issues that should be considered to further improve the manuscript and increase its reach.

-        The introductory section also needs to describe the objective of the paper, and state the methods you will use to achieve your goal. Introduction should answer the question ‘Why:’ why you choose that topic for research; why it is important; why you adopted a particular method or approach; and so on. You can also think of the Introduction as the section that points out the gap in knowledge that the rest of the paper will fill, or the section in which you define and claim your territory within the broad area of research.

-          More precise language should be used when describing, for example, knowledge, learning process, effects, learning gains …knowledge is wide term, but the authors probably do not aim to examine all types of knowledge across different taxonomy levels or ontologies….

-          Since the focus of the manuscript is to examine the learning process, what is the target, e.g., procedural knowledge, skills, attitudes, or something else?

Literature review:

Learning with VR/AR: I am missing here some of the underlying theory(s) that can best support learning with AR/VR? For example, how can this study be framed with the Activity theory and Csikszentmihalyi’s flow theory? What about Ausubel’s subsumption theory?

-           How was “knowledge” defined in this study? As I can see from the item description, it is more about the perception of knowledge rather than a knowledge itself that can be categorized into some of the known taxonomies….

-           How can "the presence" be understood through the lens of engagement theory?

-           Add more support for flow theory, specifically describing the flow dimensions relevant to this study.

Materials and methods:

-           Why does the size of focus groups vary so much – from 2 to 25 participants per group? Clarify.

-           Procedure: Did the experiment (lesson) last only 90 minutes (two periods of 45 minutes each)? How did the authors determine this time/duration to conduct the experiment and what measurable learning objectives were set for the lesson? Just name them…

-           Add section 4.5 Data analysis: and clearly justify all methods, tests, statistics and software used for analysis in this study…...

Results:

5.1.1

- in table 2 also measures for skewness and kurtosis should be given too as descriptive quantities….

5.1.3

- provide results from the Shapiro-Wilk test for all categories used in this study…

- how moderator analysis was done…explain…was it multiple regression with interaction term or one of the models developed, e.g. Hayes’ models….or any other...

- how was the “gain” calculated, e.g., knowledge, interest,…? Were the means adjusted and some corrections made (to reduce Type I error)?

- for the “knowledge” scale, there was only one item related to how students rated their knowledge… this may also be a limitation of this study….

5.2

-when reporting and interpreting the responses from the focus groups, try to be more specific, code the responses and find big ideas that can be verified.

- how did the size of the focus groups affect the responses?

- when discussing RQs, try to compare with existing literature and findings in the field….

6. this chapter should be renamed discussion, implications, limitations, and future work

- implications related to knowledge should be treated with caution since there was no knowledge test or measurement, only perception of or experience with…

- learning gain should be defined first in the method or results section, then it can be discussed accordingly...

-        The English language is sufficiently intelligible and well structured in the manuscript. The lexical sophistication is satisfactory, the text is well structured, and the syntactic forms and variety of syntactic forms are also sophisticated. The interconnectivity of text segments based on text features is present and satisfactorily indicates lexical, semantic, and argument dependencies within a text. All in all, the text is easy to read and the structure of the text well reflects the intended topic.

Author Response

Dear Reviewer 4,

thank you very much for your feedback on our study. We appreciate the time you invested. We have made an effort to appropriately address your comments and have created a table for this purpose. On the left side, you will find the comments of all reviewers, and on the right side, our responses. In the manuscript itself, we have highlighted in yellow all the changes we referred to in the cover letter, making it easier for you to trace the modifications. If there are further aspects we should revise, we are more than willing to do so. We are available for additional feedback at any time.

Best regards!

Author Response File: Author Response.pdf

Reviewer 5 Report

This paper provided a comprehensive report about the flow and presence of a VR and AR based learning application developed by Greenpeace. It was very well written, technically strong and all relevant terms were defined clearly. They full explained the study and justified their approach. The limitations of the research were noted (the lack of a control group is probably the most obvious omission of the study). 

I feel a few comments are warranted. With such rich data, considering there were focus groups held, the balance of the paper was a little disappointing. Much of the paper dealt with the background and statistical data analysis and the response to the research questions was relatively brief when reviewed in proportion to the paper as a whole. There was little of the participants in the paper. 

Additionally, I am aware of Microsoft 'Hololens' but not 'Hololenses' (L61). Even the source document listed it as 'hololens'. So this may need to be corrected? 

The bias towards male participants was explained but it would have been so much more comprehensive to have provided a balanced cohort - especially in these devices that have traditionally enjoyed more by males than females.

Overall I enjoyed this paper very much and look forward to it's publication.

Author Response

Dear Reviewer 5,

thank you very much for your feedback on our study. We appreciate the time you invested. We have made an effort to appropriately address your comments and have created a table for this purpose. On the left side, you will find the comments of all reviewers, and on the right side, our responses. In the manuscript itself, we have highlighted in yellow all the changes we referred to in the cover letter, making it easier for you to trace the modifications. If there are further aspects we should revise, we are more than willing to do so. We are available for additional feedback at any time.

Best regards!

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Dear authors,

Categorizing a mobile app with a 3D virtual environment as XR is something I have not encountered in the literature. If there is concrete evidence of other empirical studies with similar systems that have been published in respectable journals with IF accepting such classification, please add it to the appropriate section.

L65-72 contain important points. Please provide literary evidence that supports these claims.

On L75-76 you mention that the current app combines VR and AR. Can you please explain the AR component of the app? Do you classify scanning a QR code as AR?

The introduction of the paper remains blurry in the sense that the term extended reality or xreality or XR was used well before 2022 with a specific definition e.g. see 10.2139/ssrn.3300469 or 10.1201/9781003052838-17 or 10.1016/j.promfg.2018.06.054. Is this a new interpretation of the same concept? This point should be clarified.

Author Response

Dear Reviewer,

thank you very much for the feedback once again. We have made the last requested changes. We sincerely appreciate all the comments that have significantly contributed to the quality of the article and have assisted us in writing and revising it. The new changes have been highlighted in green in the document.

Best regards!

Author Response File: Author Response.pdf

Reviewer 4 Report

To the authors: As it stands now, the manuscript is much better and solid, and the authors have answered  the questions I raised correctly. Congratulations! 

The language used in the report is clear and coherent. The ideas are presented in a logical order, making it easy for the reader to follow the arguments and conclusions. The manuscript is well-edited, with minimal grammatical errors. The authors effectively use a variety of sentence structures, enhancing the overall readability of the document.The language is concise, avoiding unnecessary details and redundancies. Each sentence contributes to the overall clarity and effectiveness of the communication.In summary, the language used in the report is highly effective. It combines clarity, precision, and appropriate tone to convey complex ideas in a comprehensible manner.

Author Response

Dear Reviewer,

thank you very much for the final positive feedback. We sincerely appreciate all the comments that have significantly contributed to the quality of the article and have assisted us in writing and revising it. The new changes have been highlighted in green in the document.

Best regards!

Author Response File: Author Response.pdf

Round 3

Reviewer 2 Report

Authors addressed all remarks satisfactory and revised their work. Well done. Congratulations.

Author Response

Thank you for the positive feedback and your contribution to the quality of our article.