Evaluating the Usability and Engagement of a Gamified, Desktop, Virtual Art Appreciation Module

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The manuscript explores an important and current topic at the intersection of virtual reality, gamification, and art education.The authors designed a gamified VR art appreciation module for high school students and evaluated its effectiveness through focus group interviews and observations based on the collected data.The study is well-structured and readable, and demonstrates a thoughtful design.However, from a scientific and methodological perspective, the study has several critical limitations that limit its reach, generalizability, and rigor.

Pros:

+ Current and relevant topic - VR and gamification in education are growing fields, and their combination in art education for high school students is innovative.

+ The paper is logically organized, with well-defined research questions and a transparent explanation of the elements and flow of the qualitative research and the module's functions.

+ The authors emphasize user experience and capture rich qualitative data from real users.

+ Good integration of theory - flow theory, spatial presence, and gamification principles are discussed and connected to the findings.

 

Cons:

- The study is entirely qualitative and relies on focus group interviews and on-screen observations, which limits the ability to draw strong conclusions about effectiveness or impact.

- No control or comparison group is used – there is no baseline (e.g., a textbook art appreciation group) to which the GVR intervention is compared.

- Data analysis lacks depth and transparency in coding. The credibility of the results would be enhanced by coding them into a form that can be quantified and then inferred from a statistical basis

- The study concludes that the module improves engagement and understanding, but no direct measures of learning or retention are used – only self-perception.

- Claims about psychological immersion and flow are not supported by validated measurement tools (e.g., flow scales, presence questionnaires).

- Although gamification is a key element of the study, there is no systematic analysis of which game elements were most effective or how they influenced behavior over time (e.g., diminishing returns or reward fatigue).

 

To clarify the ideas, a few more observations, sorted by article passage:

- I miss the age of the students - it would be appropriate to explain the connection between age and year, it may not be obvious to foreign readers.

- In table 1 is a typo - 1st row contains all the data

In table 2 it is not clear which items in the second column correspond to the topic in the first column. - I recommend either separator lines or top alignment.

- The use of the TUP model is problematic - the TUP model, as originally presented by Bednarik et al. (2004), was designed as an evaluation tool for EXPERT evaluation of educational software tools by reviewers, teachers, etc. The model does not assume subjective self-evaluation, but rather an objective EXPERT assessment of the educational tool. In the article, it is used for evaluation by students. Here it would be more meaningful to use TAM, UTAUT or another similar model based on more relevant data and accepted by the community. It is obvious that in the current state of research this change may be problematic, so if possible (I cannot estimate whether it is) it is necessary to appropriately map the results to standardly used models or to make an adaptation/coding of answers.

- Line 316, 339 - what does the student assessment mean, that "the mouse sensitivity was slightly high" and "It’s the wheel in the middle of the mouse" - was it about VR? with headsets? Or was the work somehow combined?

- In conclusion, I have a little problem agreeing with the theoretical contribution. These claims are not statistically supported, they are just the authors' opinion. I am not sure whether, based on the published sequences of qualitative research, it is possible to responsibly conclude that "the study reinforces self-determination theory, revealing that VR-based learning enhances students' sense of control, enjoyment, and engagement.", while these are explicitly elements measured by quantitative instruments.

- Qualitative research in areas where objectively conducted research with appropriate quantitative methods is the standard can be enriching, as long as it represents a supplement to the content. In the case where, as the authors write, "dominant voices within the group may influence the responses of other participants, potentially compromising objectivity." it may be more about a part of the research than about the basic concept on which the conclusions are built.

- Authors have been talking a lot about gamification, which even appears in the title, but we did not know about it enough. According to the clues in the text, however, it is probably about adding and subtracting points based on answers to multiple-choice questions? Perhaps it would be better to explain the concept in more detail.

 

The manuscript addresses an engaging and innovative topic of high relevance to contemporary educational technologies. The design of a gamified VR art appreciation module tailored for high school students is timely, and the authors present their work clearly and thoughtfully. Despite these strengths, however, the scientific rigor of the study is currently insufficient for publication without substantial revisions.

To strengthen the manuscript and increase its contribution, it would be appropriate to consider:

- clarify the VR setup (e.g., whether you used a headset or a tabletop system) and describe the technical configuration.

- justify or reconsider the use of the TUP model for qualitative student-centered assessment; alternatively, consider mapping responses to more generally accepted models such as TAM, UTAUT, etc.

- avoid overinterpreting qualitative findings, especially when referring to established theories (e.g., flow, self-determination theory) unless they are supported by validated instruments.

- explain the gamification in more detail, especially what elements were used and how they align with common gamification frameworks.

- while a shift to a quantitative methodology may not be feasible at this stage, small additions (e.g., even basic post-use rating scales) and better alignment with accepted evaluation models could help bridge the gap between exploratory insights and generalizable findings.

 

Given the current state of the manuscript, I recommend a major revision. The study has clear potential, but requires both methodological and conceptual improvements to meet the expectations of rigorous empirical research.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

1. Art education emphasizes “appreciation of works”. This study uses 3D models to construct ceramic works. Can the image quality truly present the essence of the works and make them worthy of appreciation?
2. Line 144. Game elements include levels, points, badges, leaderboards, avatars, quests, social interaction, or certificates. What elements does the game in this study contain? Is it playable enough?
3. Please describe the hardware used and the control method.
4. Table 1, the content of gender is repeated with that of grade and interest in art.
5. Regarding “art interest” and “VR proficiency”, how does this study define high, medium, and low?
6. Line 169. The game system records scores, time spent on each scene, and the number of interactions with the artwork. Please present this data in the results section.
7. G3-P2 mentioned, “It makes me feel a little sick.” How severe was the subject’s motion sickness?
8. This system may motivate students to use it, but there is no evaluation of its learning outcomes.
9. Section 4.3, since this study is about education, it should be compared with traditional learning methods to show the value of this study, rather than considering the comparison with traditional learning methods as a suggestion for future research.
10. There are several literature that combine VR environment and game design with educational topics. If this study can provide more specific design suggestions for middle school students, it will help enhance the value of the results.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

I appreciate the incorporation of comments and the authors' efforts to incorporate them as effectively as possible. In the context of the edits, the strengths and weaknesses of the article can be reassessed as follows:

Pros:

• Well-edited view of qualitative data – in the current version, the authors focused on creating summary tables from the content of the interviews, which is an undeniable benefit to the article. The students' statements are authentic, diverse and thematically well categorized (technology, usability, pedagogy).
• The article provides specific design recommendations for the development of GVR systems (e.g. the importance of NPCs, mini-maps, feedback), which can be useful when designing educational digital environment.
• After incorporating the review, the statistics of scores, clicks and correlations were added, which increases the credibility of the results and makes the article a little more than a purely qualitative description, although detailed information for each student is probably not the most appropriate approach (user categorization or visualization might be more appropriate).
• The authors showed a sincere effort to respond to comments, especially regarding clarity, theoretical anchoring (e.g. adaptation of TUP) and description of gamification elements.

Cons:

• Based on the authors' explanation of their understanding of VR, I must state that I find the title of the article misleading – “VR” does not reflect the real type of technology used – the system used is a desktop simulation, not immersive VR (e.g. via HMD or spatial motion tracking). The reader would rightly expect the article to deal with immersive virtual reality, not a traditional computer screen with a mouse. I would recommend considering editing the title (or use absolutely different) and also not using abbreviations in the title – their introduction is approached when they are first used in the text.
• The article feels more like a description of an implementation than an academic study. It does not bring new theories, models or methodological breakthroughs. Questions like “was it fun?” or “is it better than a textbook?” are interesting, but insufficient for a high research standard.
• By removing references to “flow” or “self-determination theory”, the article has lost any deeper reflection on why students respond positively. The adaptation of the TUP model is partly justified by the literature, but still problematic for self-assessment by students. I would recommend mentioning / integrating pedagogical / theoretical foundations into related work as foundations for the implementation of the application.
• The excerpts from the interviews illustrate student opinions, but there is a lack of synthesis and interpretation that would take these quotes to a higher level - e.g. what does it say about gamified learning in general?
• The study is conducted at one type of school, in one region, with a narrowly selected set of students (40 people). The authors admit this, so this item should not be taken as something that can be corrected, but only as a statement.

Although the article carefully documents the use of a gamified desktop VR module in the context of art education, the research contribution remains unclearly formulated. In the current wording of the introduction and conclusion, it is not convincingly articulated, which makes this study fundamentally different from previous works in the field of VR in education. The use of gamification or the choice of age group alone is not enough to clearly define the research contribution.

For the acceptance of the article, I recommend that the introduction and conclusion passages be more significantly rephrased so that it is clear:

• Whether the benefit is a methodological innovation (e.g., combining qualitative and behavioral data in a GVR context),
• Or an application shift (e.g., proving the feasibility of low-cost desktop VR in an educational environment),
• Or a theoretical framework (e.g., a new adaptation of the TUP model for student self-reflection, if it can be substantiated), I would be very cautious here, however, because I cannot yet identify with the authors' perspective.
• Without a clear research hook, there is a risk that the article will be perceived more as a technical-descriptive case study than as an article with academic benefit.

In any case, it can be stated that the quality of the article has significantly increased and for successful publication, generalization needs to be added, the content linked to existing theories and the contribution emphasized. Also pay attention to the content, I feel that the introductory part of section 4 was omitted in the submitted version.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The authors did not specifically answer question 5. Regarding “art interest” and “VR proficiency”, how would the subjects judge their own level as high, medium or low?

Author Response

Please see the attachment

Author Response File: Author Response.docx

Round 3

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for carefully addressing previous concerns and implementing the suggested changes.
The revised manuscript demonstrates clear improvements in both methodological rigor and clarity of reporting. The theoretical foundation in SDT, flow theory, and usability models is now well articulated and relevant. The detailed description of gamification elements and their impact on engagement adds practical value. 
The article is currently suitable for publication.