Review Reports - Defining XR-Specific Teacher Competencies: Extending the DigCompEdu Framework for Immersive Education

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for thinking of me as a reviewer of the article entitled “An analysis of diverse perspectives on teaching with XR resulting in concrete building blocks for XR-based educator professionalization”.

This article deals with the integration of Extended Reality (XR) technologies in education, exploring the competencies that educators need to implement these tools effectively in their pedagogical practices. Through a methodological approach that combines documentary research and expert verification, the study seeks to identify and develop concrete competency blocks that facilitate the use of XR in educational environments. Nowadays there is a great boom in this topic, as well as in the need to train teachers to use emerging technologies in their teaching. Two research questions are posed: 1) How do existing models and frameworks for digitization in education, such as the TPACK model, align with the needs and challenges of teaching with XR; and 2) what are the specific competency blocks that educators should possess from an XR perspective. These questions guide the research towards the identification of specific competencies that are essential for the effective use of XR in the classroom. The study uses a mixed approach combining desk research with expert verification. The desk research is based on a comprehensive search of existing frameworks in Google Scholar, followed by the application of exclusion criteria to ensure the relevance of the selected frameworks. Subsequently, expert verification is carried out, where the competency blocks are adjusted based on the practical experience of the participants. This approach provides a solid and valid basis for the conclusions of the study. The article presents the findings in a structured manner, beginning with an overview of existing frameworks and then delving into the specific competencies needed for teaching with XR. The results are articulated in a way that relates directly to the research questions, making it easy to understand how these findings can be applied in educational practice. However, the presentation of some data could be improved to make them more accessible to the reader. The article is well organized, with clearly defined sections that guide the reader through the research process. The writing is clear. It is an original and relevant study. In an evolving educational context, where technology plays an increasingly important role, this study addresses a significant gap in the literature on teacher readiness to use XR. By providing a practical, evidence-based framework for competency development, the study not only contributes to the field of education, but also offers valuable tools for the ongoing training of educators in the use of emerging technologies. If I may, it would be interesting to provide concrete examples of how the identified competencies can improve teaching and learning. Rating: Accept

Les doy las gracias por pensar en mí como revisor del article titulado “An analysis of diverse perspectives on teaching with XR resulting in concrete building blocks for XR-based educator professionalization”. Nos encontramos ante un artículo que trata de la integración de tecnologías de Realidad Extendida (XR) en la educación, explorando las competencias que los educadores necesitan para implementar estas herramientas de manera efectiva en sus prácticas pedagógicas. A través de un enfoque metodológico que combina investigación documental y verificación por expertos, el estudio busca identificar y desarrollar bloques de competencias concretas que faciliten el uso de XR en entornos educativos. Hoy en día hay un gran auge en esta temática, así como en la necesidad de formar a los docentes para utilizar tecnologías emergentes en su enseñanza. Se plantean dos preguntas de investigación: 1º) Cómo los modelos y marcos existentes para la digitalización en educación, como el modelo TPACK, se alinean con las necesidades y desafíos de la enseñanza con XR; y 2º), cuáles son los bloques de competencias concretas que los educadores deben poseer desde la perspectiva de XR. Estas preguntas guían la investigación hacia la identificación de competencias específicas que son esenciales para el uso efectivo de XR en el aula. Metodología El estudio utiliza un enfoque mixto que combina la investigación documental con la verificación de expertos. La investigación documental se basa en una búsqueda exhaustiva de marcos existentes en Google Scholar, seguida de la aplicación de criterios de exclusión para asegurar la relevancia de los marcos seleccionados. Posteriormente, se lleva a cabo una verificación por expertos, donde se ajustan los bloques de competencias en función de la experiencia práctica de los participantes. Este enfoque proporciona una base sólida y válida para las conclusiones del estudio. Resultados. El artículo presenta los hallazgos de manera estructurada, comenzando con una descripción general de los marcos existentes y luego profundizando en las competencias específicas necesarias para la enseñanza con XR. Los resultados se articulan de manera que se relacionan directamente con las preguntas de investigación, lo que facilita la comprensión de cómo se pueden aplicar estos hallazgos en la práctica educativa. Sin embargo, se podría mejorar la presentación de algunos datos para hacerlos más accesibles al lector. Redacción y estructura El artículo está bien organizado, con secciones claramente definidas que guían al lector a través del proceso de investigación. La redacción es clara. Es un estudio original y pertinente. En un contexto educativo en constante evolución, donde la tecnología juega un papel cada vez más importante, este estudio aborda una brecha significativa en la literatura sobre la preparación de los docentes para utilizar XR. Al proporcionar un marco práctico y basado en la evidencia para el desarrollo de competencias, el estudio no solo contribuye al campo de la educación, sino que también ofrece herramientas valiosas para la formación continua de los educadores en el uso de tecnologías emergentes. Si me lo permiten, sería interesante proporciona ejemplos concretos de cómo las competencias identificadas pueden mejorar la enseñanza y el aprendizaje. Valoración: Aceptar

Author Response

Dear Reviewer,

Thank you very much for your comments, which elaborate on the relevance of this study. This is good to hear.

Below we address the comments in your review.

Comment 1: [However, the presentation of some data could be improved to make them more accessible to the reader.]

Response 1: [We (as well as other reviewers) recognize your point, in which you suggest, that the presentation of data could be improved. One way we have tried to improve this is to include the cards of the card set in a separate appendix. In addition, those cards can now be accessed through links that lead to the cards in slide form within Google Docs. This also makes it easier for readers to print the cards themselves and apply them for their own use.]

Comment 2: [If I may, it would be interesting to provide concrete examples of how the identified competencies can improve teaching and learning.]

Response 2: [We have -as you suggested- added concrete examples of how these XR competencies can improve teaching and learning.]

Kind regards,

Reviewer 2 Report

Comments and Suggestions for Authors

The originality and relevance of the study are evident. In a constantly evolving educational context, where technology plays an increasingly important role, this study addresses a significant gap in the literature on teacher readiness to use XR. By providing a practical and evidence-based framework for competence development, the study not only contributes to the field of education, but also offers valuable tools for the ongoing training of educators in the use of emerging technologies.

If I may, I suggest explaining a little more why it is important to research teacher competencies for XR integration.

Author Response

Dear Reviewer,

Thank you for your comment indicating the relevance of our article regarding a gap in the literature on teacher readiness to use XR.

Below we address the comment in your review.

Comment 1: [If I may, I suggest explaining a little more why it is important to research teacher competencies for XR integration.]

Response 1: [In this revised version, more attention has been paid to clarifying for the reader, why it is important to research teacher competencies for XR integration in teaching anyway.]

Kind regards,

Reviewer 3 Report

Comments and Suggestions for Authors

This manuscript is located as topic in the area of interest of „Trends in Higher Education” journal.

This manuscript has a very interesting research idea. The title is clear and it is adequate to the content of the paper.

However, some significant improvements are needed to meet the standards of a quality publication.

1. Abstract - should be rewritten to highlight the methodology, results and conclusions more clearly

2. Introduction - Explain the context of the study and the state of research in the field of XR in education at the international level. The current state of the research field should be carefully reviewed and key publications cited.

3. The terms XR, VR. MR, AR should be very well defined, and the opportunities for use in education, the challenges.

4. Methods - Details could be added about the number of experts involved, their specific XR expertise and the selection criteria for them. This would improve the transparency of the validation process. In addition, the limitations of the study and possible biases should be explained

5. Figures should have a more descriptive name

6. Figures that have already been used in previous studies, e.g. should have the source specified (in the case of models)

7. Results are very long, and should be summarized better. Different sources should be compared to detect possible contradictions or common points.

8. Conclusions should be broadened and focused more on their own results and not be so general

9. The authors should consider introducing a discussion section. Authors should discuss the results and how they can be interpreted from the perspective of previous studies and of the working hypotheses.

10. The references are very weak. Newer research in the field should be used. References should be numbered in order of appearance and indicated by a numeral or numerals in square brackets—e.g., [1] not like that (1)

Good luck!

Author Response

Dear Reviewer,

Thank you very much for your comments and for your compliment about the idea of researching this being very interesting.

Below we address point by point the comments in your review.

Comment 1: [Abstract - should be rewritten to highlight the methodology, results and conclusions more clearly.]

Response 1: [In the Abstract, the methodology followed, results and conclusions have now been clarified.]

Comment 2: [Introduction - Explain the context of the study and the state of research in the field of XR in education at the international level. The current state of the research field should be carefully reviewed and key publications cited.]

Response 2: [In this current version, the background of the study has been strengthened, referring to key publications.]

Comment 3: [The terms XR, VR. MR, AR should be very well defined, and the opportunities for use in education, the challenges.]

Response 3: [Concepts are defined, substantiating what role they can play in education.]

Comment 4: [Methods - Details could be added about the number of experts involved, their specific XR expertise and the selection criteria for them. This would improve the transparency of the validation process. In addition, the limitations of the study and possible biases should be explained.]

Response 4: [The background of the experts is now explained, in which their expertise on XR in education is nuanced. The limitations and possible biases are now described in a separate Discussion section.]

Comment 5: [Figures should have a more descriptive name.]

Response 5: [The descriptions of figures are now more descriptive.]

Comment 6: [Figures that have already been used in previous studies, e.g. should have the source specified (in the case of models).]

Response 6: [The captions of the figures now refer to the authors and the key publication, in which the model was presented.]

Comment 7: [Results are very long, and should be summarized better. Different sources should be compared to detect possible contradictions or common points.]

Response 7: [The results section has been drastically shortened by not including the cards of the card set in the running text. The cards have been moved to an appendix and are now also made accessible online. The latter has the added benefit that readers can now more easily use the card set themselves as workshop materials.

The descriptions themselves have also been shortened at the Results section. In addition, the section reporting on the cards of the card set is now followed by a section on similarities and differences.]

Comment 8: [Conclusions should be broadened and focused more on their own results and not be so general.]

Response 8: [Because the previous Conclusion section was too general in nature, it has been completely rewritten. The new Conclusion section provides more concrete answers to the research questions. In addition, assertions have been substantiated with relevant literature references.]

Comment 9: [The authors should consider introducing a discussion section. Authors should discuss the results and how they can be interpreted from the perspective of previous studies and of the working hypotheses.]

Response 9: [A Discussion section has been added, especially to consider the study from the perspectives of validity and reliability.]

Comment 10: [The references are very weak. Newer research in the field should be used. References should be numbered in order of appearance and indicated by a numeral or numerals in square brackets-e.g., [1] not like that (1).]

Response 10: [In addition to correcting the notations for references to sources, these have also been supplemented with more recent research in this field.]

Kind regards,

Reviewer 4 Report

Comments and Suggestions for Authors

The article deals with an interesting topic. In my opinion, however, the article requires a comprehensive revision to be suitable for publication in a scientific journal. I also wonder if the title is fitting.

Introduction
Although there is an introduction section, the theoretical and empirical background of the study is barely explained. Actually, a chapter (Theoretical/Empirical Background) is missing here. This makes it difficult to follow the rest of the article, as concepts (such as XR, VR, etc.) are only weakly defined, and many claims are made without providing empirical evidence in the form of literature references (e.g., line 53 “novelty effect can also be deliberately used to give education a boost” → this statement needs empirical backing. Is this really the case, and what does “boost” mean here? More motivation? Better learning outcomes? There are also studies suggesting that immersive Virtual Reality, for example, might distract rather than lead to better learning outcomes → Makransky, G., Terkildsen, T. S., & Mayer, R. E. (2019).
From my perspective, a more theoretically and empirically grounded derivation of the research goals is needed, although I fundamentally find the research goals to be reasonable.
Research Method
I wonder why Google Scholar was used and not another literature database that does not follow the algorithms of Google Scholar.
The methods seem interesting to me in principle, but they are new to me. However, I find them inadequately described and potentially lacking relevant literature references. Do the results produced by these methods meet the quality criteria of qualitative research? I also miss a description of the sample (the experts). What makes the individuals experts? Their participation in the Erasmus project? How many experts were involved? How exactly were the data analyzed?
The presentation in bullet points in sections 2.1 and 2.2, as well as other sections, is more confusing to me. Would a narrative text with an illustration summarizing the approach and the individual steps not be more appropriate? I would particularly find this useful for section 2.2, as it’s hard to follow the information.
Statements like "Because we have the impression that teacher professionalization in the field of Extended Reality is still an underexposed topic..." need to be supported by references in a scientific article.
Results
The various frameworks are presented in great detail, without it being clear to me according to what criteria they were analyzed. The references to XR are vague, except for DigCompEdu.
In section 3.1.7, properties of XR are summarized. I wonder where these properties come from and what exactly is being summarized here. Is something being summarized that was mentioned earlier? As far as I understand, nothing about these properties was previously discussed. Shouldn’t these properties be presented in the theoretical part and then used to derive criteria for analyzing the frameworks? I find the decision to choose the DigCompEdu model based on this somewhat unconvincing, although I can generally understand the choice.
Section 3.2 is extremely long. I like that it goes into detail about the DigCompEdu model, but I would prefer if the tables (or rather figures?) were shown in the appendix and the results were presented more concisely. I also don’t fully understand how the content in the “tables” was generated.
Section 3.3: Here, too, I would prefer a narrative text with a more exemplary presentation. The bullet-point format is confusing to me—too detailed, in my opinion. Possibly, this could be included in the appendix. I also don’t understand: are these all the comments made, or a selection?
Why was a map (Embodied cognition) added, and why is it not presented alongside the others? Better: in the appendix!
Conclusions
I miss a structured discussion of the results. The conclusions are very general, hard to follow, and consist of a long list. As in the rest of the article, many claims are made here without being supported by literature.
References to literature in the conclusions?

Conclusions are very general, hard to follow: long list.

Comments on the Quality of English Language

Slightly informal in a few places

Author Response

Dear Reviewer,

Thank you very much for your comment and good to hear that you think, this is an interesting topic.

Below we address point by point the comments in your review.

Comment 1: [I also wonder if the title is fitting.]

Response 1: [The title has been changed to better fit what the article focuses on.]

Comment 2: [Introduction - Although there is an introduction section, the theoretical and empirical background of the study is barely explained. Actually, a chapter (Theoretical/Empirical Background) is missing here. This makes it difficult to follow the rest of the article, as concepts (such as XR, VR, etc.) are only weakly defined, and many claims are made without providing empirical evidence in the form of literature references (e.g., line 53 “novelty effect can also be deliberately used to give education a boost” → this statement needs empirical backing. Is this really the case, and what does “boost” mean here? More motivation? Better learning outcomes? There are also studies suggesting that immersive Virtual Reality, for example, might distract rather than lead to better learning outcomes → Makransky, G., Terkildsen, T. S., & Mayer, R. E. (2019).

From my perspective, a more theoretically and empirically grounded derivation of the research goals is needed, although I fundamentally find the research goals to be reasonable.]

Response 2: [In this current version, the background of this study has been described more robustly. Definitions of terms used have been added. The extent to which any novelty effect may improve learning has been nuanced.]

Comment 3 [Research Method - I wonder why Google Scholar was used and not another literature database that does not follow the algorithms of Google Scholar.]

Response 3: [Argumentation for the use of Google Scholar has been added. Indeed, this was chosen primarily because of the interest in grey literature in this rapidly developing field.]

Comment 4: [The methods seem interesting to me in principle, but they are new to me. However, I find them inadequately described and potentially lacking relevant literature references. Do the results produced by these methods meet the quality criteria of qualitative research?]

Response 4: [Both methods are now supported by relevant literature references.]

Comment 5: [I also miss a description of the sample (the experts). What makes the individuals experts? Their participation in the Erasmus project? How many experts were involved? How exactly were the data analysed?]

Response 5: [This revised version describes the background of the experts and elaborates on why they can be considered experts.]

Comment 6: [The presentation in bullet points in sections 2.1 and 2.2, as well as other sections, is more confusing to me. Would a narrative text with an illustration summarizing the approach and the individual steps not be more appropriate? I would particularly find this useful for section 2.2, as it's hard to follow the information.]

Response 6: [A narrative text with an illustration might indeed also have been an enlightening alternative way to describe the approach described in the Method section. However, since we aim at a detailed description of the process followed, we have chosen to do so point by point. We are convinced, that this will enhance repeatability. We did follow the reviewer's recommendation to convert point by point summarized text into a narrative for the section, in which the results of the expert member check are reported. There we believe this did indeed result in better readability.]

Comment 7: [Statements like “Because we have the impression that teacher professionalization in the field of Extended Reality is still an underexposed topic...” need to be supported by references in a scientific article.]

Response 7: [We have qualified the claim that teacher professionalization in the field of Extended Reality is an underexposed topic and supported it with a reference.]

Comment 8: [Results - The various frameworks are presented in great detail, without it being clear to me according to what criteria they were analysed.]

Response 8: [Each framework was analysed according to the questions, mentioned in section 2.1.3. Then the findings were described in narratives in the Results section].

Comment 9: [The references to XR are vague, except for DigCompEdu.]

Response 9: [What we have noticed during our search for frameworks focused on teaching and learning with XR is that in the process, frameworks emerge that existed before XR took off properly. In itself, this is fine, because those older frameworks appear to be quite capable of thinking about technology in education in general, and thus specifically to XR. One consequence, however, is that those references to XR are less concrete. Besides DigCompEdu, the concrete reference to XR can also be made from the CAMIL model.]

Comment 10: [In section 3.1.7, properties of XR are summarized. I wonder where these properties come from and what exactly is being summarized here. Is something being summarized that was mentioned earlier? As far as I understand, nothing about these properties was previously discussed. Shouldn't these properties be presented in the theoretical part and then used to derive criteria for analysing the frameworks?]

Response 10: [There was indeed an explanatory sentence missing about how this summary of properties of XR was worked towards to support learning with XR. That rationale has since been added. This summary comes from the analysis of the models/frameworks.]

Comment 11: [I find the decision to choose the DigCompEdu model based on this somewhat unconvincing, although I can generally understand the choice.]

Response 11: [We had set up the approach to answering our first research question in such a way, that it was quite possible, that another framework would emerge for answering the second research question. However, that turned out not to be the case: as we had expected, DigCompEdu came out on top for that after all.]

Comment 12: [Section 3.2 is extremely long. I like that it goes into detail about the DigCompEdu model, but I would prefer if the tables (or rather figures?) were shown in the appendix and the results were presented more concisely. I also don't fully understand how the content in the “tables” was generated.]

Response 12: [In the previous version, the Results section was indeed extremely long. Meanwhile, the cards of the cards set are no longer part of the running text. These have been moved to an appendix, as you also suggested. In addition, these cards are now accessible online via links, which makes it easier for readers to use them themselves if desired.

In addition, the descriptions in section 3.2 have also been shortened.

The steps taken to compile the content of the cards are described mainly in sections 2.2.3 and 2.2.4.]

Comment 13: [Section 3.3: Here, too, I would prefer a narrative text with a more exemplary presentation. The bullet-point format is confusing to me-too detailed, in my opinion. Possibly, this could be included in the appendix. I also don't understand: are these all the comments made, or a selection.]

Response 13: [The previous version indeed included all the comments made. In line with the reviewer's advice, the previous enumeration has been converted into a narrative.]

Comment 14: [Why was a map (Embodied cognition) added, and why is it not presented alongside the others? Better: in the appendix!]

Response 14: [The card on Embodied cognition was added because it did not fit sufficiently within the already existing structure of DigCompEdu. In line with the reviewer's advice, this card has been moved to the Appendix.]

Comment 15: [Conclusions - I miss a structured discussion of the results.]

Response 15: [A Discussion section has been added, mainly to consider the study from the perspectives of validity and reliability.]

Comment 16: [The conclusions are very general, hard to follow, and consist of a long list. As in the rest of the article, many claims are made here without being supported by literature.

References to literature in the conclusions? Conclusions are very general, hard to follow: long list.]

Response 16: [Because the previous Conclusion section was too general in nature, it has been completely rewritten. The new Conclusion section provides more concrete answers to the research questions. In addition, assertions have been substantiated with relevant literature references.]

Comment 17: [Comments on the Quality of English Language - Slightly informal in a few places.]

Response 17: [While re-reading our manuscript, we indeed also came across some sentences that were eligible for rewording in a more academic style. We have modified these.]

Kind regards,

Round 2

Reviewer 3 Report

Comments and Suggestions for Authors

The work has improved considerably.

Reviewer 4 Report

Comments and Suggestions for Authors

The authors have addressed the points I raised, and in some cases, implemented them well. Thank you. However, I still find the theoretical background, the conclusion, and the discussion rather weak. Additionally, I struggle to fully comprehend the extensive presentation of the models in the results section. I leave it to the editor to decide whether this article should be published in its current form in line with the journal's standards.