Virtual Reality for Hydrodynamics: Evaluating an Original Physics-Based Submarine Simulator Through User Engagement

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

please check the attached file

Comments for author File: Comments.pdf

Author Response

Dear reviewer,

Thank you for allocating time to evaluate the manuscript, your feedback is much appreciated. All your observations have been addressed in the new version, which is now improved substantially.

• Comment 1: In the Introduction and Literature Review (Section 1–2, lines 30–179), while the background on VR in education is generally solid, the discussion lacks a clear comparative framework that distinguishes the proposed solution from existing tools, and the review does not sufficiently highlight the limitations of prior work.
• Response:  A new section named "Research Gap" has been introduced to better highlight the limitations of prior work (L66 - L133). Additionally, worth noting that Sections 1 and 2 have been reorganized and now offer additional relevant insights.

 

• Comment 2: In Methodology (Section 4, lines 301–429), the small and homogenous sample (26 students from a single institution) poses significant challenges for the generalizability of findings, yet the manuscript does not adequately address the implications of this limitation or
  potential sample bias.
• Response: This point of view has been better nuanced and enhanced in Section 5.1 while speaking about "Limits and future work" (L971 - L993).

 

• Comment 3: The Results (Section 5, lines 438–570) raise concerns about measurement validity, as key constructs such as Flow and CEGE have AVE values below accepted thresholds, and the poor discriminant validity between some factors suggests issues with the questionnaire’s ability to distinguish between closely related concepts. Furthermore, Kit indices such as SRMR and NFI indicate only marginal statistical robustness, and there is little discussion on how small sample size or missing data may affect the conclusions.
• Response:
  • We thank the reviewer for their insightful comments regarding measurement validity, particularly the AVE values, discriminant validity, model fit indices, and sample size limitations. We acknowledge that the AVE values for the Flow construct (0.407) and Core Elements of the Gaming Experience – CEGE (0.432) fall below the recommended threshold of 0.50 (Hair et al., 2019), indicating potential weaknesses in convergent validity.
  • Given the exploratory nature of this study and the novelty of the VR learning environment, we recognize the challenge in precisely defining latent constructs at this stage; thus, future iterations will involve item refinement and validation on a larger, homogenous sample.
  • Furthermore, the Fornell-Larcker criterion suggests insufficient discriminant validity, as the square root of AVE for Flow (0.638) is lower than its correlation with Affordance (0.811), which may reflect conceptual overlap between immersive affordances and flow in the interactive VR context. In response, we will revise the measurement model to better delineate these constructs.
  • Additionally, model fit indices such as SRMR (0.121) and NFI (0.509 and 0.641) fall short of ideal thresholds, suggesting the need for structural refinement and more parsimonious modeling in future analyses. While the current sample size of 26 is acceptable for exploratory PLS-SEM, we concur that it constrains generalizability and statistical power; therefore, subsequent studies will involve larger samples to enable more robust validation, subgroup analyses, and improved reliability of parameter estimates.
  • In order to highlight these points in a better and open way, we have added multiple mentions in the manuscript that acknowledge and discuss these statistical limitations (L739 - L766 & L1002 - L1021).

 

• Comment 4: In the Discussion and Conclusions (Sections 5–6, lines 572–704), claims about long-term educational impact and knowledge transfer are not empirically substantiated within the current study design, as all assessment is cross-sectional and immediate.
• Response: The overall structure of these sections has been changed and enhanced. More specifically, the manuscript now contains direct acknowledgements related to knowledge transfer as a limitation (L823 - L833 & L1022 - L1045).

 

• Comment 5: Additionally, while the manuscript acknowledges some limitations, this discussion is relatively brief and does not fully address key aspects such as the challenges of hardware adoption, potential confounding factors, or broader implementation barriers.
• Response: Both Sections 6 and 7 (Discussion and Conclusions) have been substantially enhanced. The manuscript now acknowledges more limitations including the ones mentioned in the provided feedback. While we recommend revising both Sections once again, more specific mentions can be found on the following lines: L884 - L934 & L1022 - L1045

 

Thank you for your support and guidance!

Best Regards,

Authors

Reviewer 2 Report

Comments and Suggestions for Authors

This paper presents a VR tool to enhance learning experience in under water engineering course. However, the paper's contribution is unclear, as it abruptly introduces the VR development without clearly explaining the specific gaps or limitations in existing solutions that it aims to address. 

Suggestions for Improvement: 
1. Clarify the research gap and urgency.
2. The authors should discuss on cost, scalability, and hardware limitations that could impede widespread adoption.
3. The paper lacks details on educational Impact, comparing the learning performance of the proposed VR with traditional teaching methods or other digital tools can contextualize its added value. 
4. Methodological detail should be provided on experimental procedures, control conditions, and statistical validation to enhance rigor. 

Author Response

Dear reviewer,

Thank you for allocating time to evaluate the manuscript, your feedback is much appreciated. All your observations have been addressed in the new version, which is now improved substantially.

• Comment 1: Clarify the research gap and urgency.
• Response: The manuscript is now defining and acknowledging much better both the research gap and urgency in a new dedicated section in the Introduction section named "1.2. Research Gap" (L66-L133).

 

• Comment 2: The authors should discuss on cost, scalability, and hardware limitations that could impede widespread adoption.
• Response: Sections 6 and 7 (Discussions and Conclusions) have been substantially enhanced and restructured. The mentioned limitations can now be found specifically on: L884 - L917.

 

• Comment 3: The paper lacks details on educational impact, comparing the learning performance of the proposed VR with traditional teaching methods or other digital tools can contextualize its added value.
• Response: We understand the point of view. This issue has been addressed in Section 1.3 (Aim of the Research), in which we have revised the way we present our research and better explained the paper aim (L148-L161). We have acknowledged that a comparison framework is out-of-scope for the current research paper.

 

• Comment 4: Methodological detail should be provided on experimental procedures, control conditions, and statistical validation to enhance rigor.
• Response: Section 4 (Methodology) has been broadly expanded to include more details in order to enhance rigor, we appreciate the feedback. More precisely, the following elements can be found in the new version of the manuscript:
  • The Research design subchapter containing the experimental procedures is much better defined and expanded (L523-L632);
  • The newly created Control conditions subchapter now addresses the feedback precisely (L634-L661);
  • The Statistical methods subchapter now contains more details on the statistical validation (L475 - L505)

 

Thank you for your support and guidance!

Best Regards,

Authors

Reviewer 3 Report

Comments and Suggestions for Authors

1. Abstract should be extended by the obtained results in part of the performance evaluating the proposed approach, unclear for me about targer groups (L12-L26).
2. Authors should clearly point-by-point describe the main contributions of this paper. It should somehow resonate with the title of the work (L55-L58).
3. Please reinforce the section related to methodology, right now it's not clear preprocessing aspects of the work, please explain a bit more about data which is utilized, structure of data, data distribution review (L359-L363).
4. Please specify in results section more about loss function, metrics, because I saw R2 as a metric, did you utilize something else or only R2 as a metrics (L455-L462)?

Author Response

Dear reviewer,

Thank you for allocating time to evaluate the manuscript, your feedback is much appreciated. All your observations have been addressed in the new version, which is now improved substantially.

• Comment 1: Abstract should be extended by the obtained results in part of the performance evaluating the proposed approach, unclear for me about target groups (L12-L26)
• Response: The Abstract has been revised completely. New mentions about the target group and results have been carefully included (L11 - L28).

 

• Comment 2: Authors should clearly point-by-point describe the main contributions of this paper. It should somehow resonate with the title of the work (L55-L58)
• Response: The main contributions of our paper have been better formulated in the new version of the manuscript. More precisely:
  • The research questions can now be found at L152 - L161.
  • A new sub-chapter named Research objectives and hypothesis has been included (L362 - L377) , in which the hypothesis are clearly introduced before describing the results.

 

• Comment 3: Please reinforce the section related to methodology, right now it's not clear preprocessing aspects of the work, please explain a bit more about data which is utilized, structure of data, data distribution review (L359-L363)
• Response: 
  • Section 4.2 named Research Methods has been completely revised and enhanced (L379-L473). It now contains more information about what data has been gathered.
  • Section 4.3. named Statistical methods now also contains more details about the statistical aspects related to the results described the paper.
  • Section 7. Results now contains details about the pre-processing aspects (L672-L675).
  • Additionally, Appendix 3 has been introduced to showcase the structure of data and the data distribution review.

 

• Comment 4: Please specify in results section more about loss function, metrics, because I saw R2 as a metric, did you utilize something else or only R2 as a metrics (L455-L462).
• Response: As mentioned before, Section 4.3. named Statistical methods now also contains more details about the metrics used to interpret the results.

 

Thank you for your support and guidance!

Best Regards,

Authors

Reviewer 4 Report

Comments and Suggestions for Authors

The manuscript entitled "Tailored Virtual Reality for Hydrodynamics: Evaluating an Original Physics-Based Submarine Simulator" has been reviewed. This manuscript presents the development and evaluation of a custom VR-based educational tool for enhancing learning in hydrodynamics, embedded within an engineering curriculum. Several methodological and structural issues warrant attention.

The Introduction and Methodology are overly lengthy and could benefit from conciseness and clearer subheadings.

The study sample comprises only 26 participants from a single, highly homogeneous cohort. While the authors acknowledge this limitation, the small sample size significantly undermines statistical power and generalizability. This is especially critical given the reliance on structural equation modeling and confirmatory factor analysis, which generally require larger samples for reliable parameter estimation.

The conclusions overstates the findings, given the methodological constraints. Statements such as “demonstrated that users could effectively learn…in less than one hour” should be interpreted with caution due to the lack of comparative data or long-term outcome measures.

Author Response

Dear reviewer,

Thank you for allocating time to evaluate the manuscript, your feedback is much appreciated. All your observations have been addressed in the new version, which is now improved substantially.

• Comment 1: The Introduction and Methodology are overly lengthy and could benefit from conciseness and clearer subheadings.
• Response: Both the Introduction (L31 - L161) and Methodology (L355 - 671) sections have been better organized and new subheadings have been introduced in order to enhance the clarity. While the process of making the two sections more concise is difficult at this point, as it may diminish the rigorousness of our paper, we are confident that the restructuring of the information will offer readers a much better experience than before.

 

• Comment 2: The study sample comprises only 26 participants from a single, highly homogeneous cohort. While the authors acknowledge this limitation, the small sample size significantly undermines statistical power and generalizability. This is especially critical given the reliance on structural equation modeling and confirmatory factor analysis, which generally require larger samples for reliable parameter estimation.
• Response:  We agree that the study was conducted on a cohort of 26 fourth-year engineering students from the same academic program to ensure homogeneity in prior knowledge, learning context, and technical proficiency. This decision was intentional, aiming to minimize confounding variables and allow for a focused exploratory investigation of the VR software's pedagogical efficacy. We do recommend going through the Results, Discussions and Conclusions chapters again in the new manuscript, as these statistical limitations are acknowledged much better than before. Additionally, we believe that:

  • Partial Least Squares Structural Equation Modeling (PLS-SEM), as implemented in SmartPLS, is particularly well-suited for exploratory research involving small sample sizes, non-normally distributed data, and formative or composite measurement models. Unlike covariance-based SEM (CB-SEM), which requires large samples and normally distributed variables for robust estimation, PLS-SEM offers a nonparametric, variance-based alternative that can yield meaningful results with samples as small as 20–30 observations, provided model complexity is moderate (Hair et al., 2017; Henseler et al., 2009).

  • PLS-SEM does not impose distributional assumptions and utilizes bootstrapping procedures to estimate significance, making it an appropriate choice when data normality or scale properties are not guaranteed. These features made SmartPLS the most appropriate tool for our exploratory analysis under the conditions of limited sample size and emerging constructs in a novel educational context.

  • Despite the validity constraints, PLS-SEM was selected over covariance-based SEM precisely due to its greater tolerance for small sample sizes and non-normal data (Hair et al., 2017). Furthermore, bootstrapping with 5,000 resamples was employed to enhance the reliability of estimates.

  • Nonetheless, we recognize the exploratory nature of the current findings and have clarified this in the revised manuscript. Future research will be extended to more diverse and larger samples across multiple institutions to strengthen statistical power, enable multi-group analysis, and increase generalizability of the results.

 

• Comment 3: The conclusions overstates the findings, given the methodological constraints. Statements such as "demonstrated that users could effectively learn...in less than one hour" should be interpreted with caution due to the lack of comparative data or long-term outcome measures.
• Response: Both sections - Discussions (L819 - L934) and Conclusions (L937 - L1045) - have been completely revised. All risky statements have been removed permanently.

 

Thank you for your support and guidance!

Best Regards,

Authors

Reviewer 5 Report

Comments and Suggestions for Authors

This manuscript aims to assess the effectiveness of an application in virtual and augmented reality for acquiring knowledge in hydrodynamics. This application simulates the necessary knowledge for understanding the main principles of building a small-scale submarine and experimenting with how the shape and properties of the ensemble affect the way it behaves in a simulated underwater space. This application has been tested by a limited target group of 26 students, in their 4th year at Ecole des Mines Paris, by completing all three proposed phases of the methodology.  

According to the authors, this study differs from others in the combination of realistic water simulations with intuitive, rapid skill acquisition. More specifically, this study introduces a custom-developed physics engine for hydrodynamics and abstract submarine modelling for educational purposes. 

The manuscript is well structured. The research questions are clearly defined. The background is well supported by relevant references. However, the references could be improved with more qualitative and recent ones. The methodology is concrete and clearly presented. 

However, the results are based on a very limited number of participants, and this is too risky for concluding in generalizing. This is a major issue that needs to be addressed. 

Moreover, there is a list of suggestions that will improve the quality of the manuscript:

1. Remove any abbreviations from the Abstract. Start using abbreviations from the Introduction. Use only once the full text and the abbreviation when they first meet. Fix this issue in the whole document.
2. The citations (number in brackets) should be the last item of any sentence, followed by a full stop (or a comma). Fix this issue in the whole document. Read once again the instructions to the authors. 
3. In line 31, provide a full text for STEM. 
4. In the Introduction, include qualitative and recent references. 
5. The sentence in lines 115-116 needs to be rephrased. 
6. In subsection 2.2, the authors defend that 360-degree videos enhance the immersive experience (line 143). However, this is not true, as these graphics come with no depth, and the immersion is limited. Try to rephrase.
7. Provide links as footnotes for the two multisensory virtual environments of lines 146-147. Similarly, for VR platforms of lines 191 and 195. 
8. In the Discussion, it could be interesting if the authors analyse the issue of technophobia. Moreover, an additional paragraph with the analysis of enhanced inclusion and accessibility through virtual reality in training and education could be a plus. Try to use recent references (for example, https://doi.org/10.54517/m.v5i2.2836 and similar).
9. In the Conclusion, the authors present an overview and the main findings of their research. My suggestion is to transfer the limitations to the Discussion.  

I hope that my suggestions will help improve the quality of this study.

Author Response

Dear reviewer,  

Thank you for allocating time to evaluate the manuscript, your feedback is much appreciated. All your observations have been addressed in the new version, which is now improved substantially.  

• Comment 1: Remove any abbreviations from the Abstract. Start using abbreviations from the Introduction. Use only once the full text and the abbreviation when they first meet. Fix this issue in the whole document.
• Response: All abbreviations have been removed from the Abstract (L11 - L28). The rest of the document has also been corrected as mentioned.

 

• Comment 2: The citations (number in brackets) should be the last item of any sentence, followed by a full stop (or a comma). Fix this issue in the whole document. Read once again the instructions to the authors.
• Response: We appreciate the close attention to details. This issue has been fixed throughout the paper.

 

• Comment 3: In line 31, provide a full text for STEM.
• Response: Issue has now been fixed (L32 - L33)

 

• Comment 4: In the Introduction, include qualitative and recent references. 
• Response: The Introduction section (L31 - L161) has been significantly improved. New references have been added alongside a much careful section related to the research gap and urgency.

 

• Comment 5: The sentence in lines 115-116 needs to be rephrased. 
• Response: The mentioned sentence has been rephrased (L174 - L175)

 

• Comment 6: In subsection 2.2, the authors defend that 360-degree videos enhance the immersive experience (line 143). However, this is not true, as these graphics come with no depth, and the immersion is limited. Try to rephrase.
• Response: We agree that the phrasing of that idea was not great. The idea has been rephrased completely to better illustrate our point, alongside the idea extracted from the quoted research paper (L199 - L205).

 

• Comment 7: Provide links as footnotes for the two multisensory virtual environments of lines 146-147. Similarly, for VR platforms of lines 191 and 195.
• Response: We believe that adding footnotes would not be entirely suitable unfortunately, as the platforms are not commercial applications with landing/presentation pages. However, the associated references have been brought closer to the blocks of text related to the virtual environments mentioned in the paper to enhance the overall clarity of the ideas. New text positions: L206-L207 & L251 - L254.

 

• Comment 8: In the Discussion, it could be interesting if the authors analyse the issue of technophobia. Moreover, an additional paragraph with the analysis of enhanced inclusion and accessibility through virtual reality in training and education could be a plus. Try to use recent references.
• Response: We agree and we appreciate the suggestion. We believe that introducing these ideas in the Discussions chapter makes perfect sense. These ideas have been included at the end of the section, more specifically: L918 - L934.

 

• Comment 9: In the Conclusion, the authors present an overview and the main findings of their research. My suggestion is to transfer the limitations to the Discussion.
• Response: Both mentioned chapters (Discussions and Conclusions) have been expanded in the new manuscript. The current way of structuring the limitations is the following:
  • We kept limitations related to the research experiment in the Conclusions, as we believe it builds the bridge towards future work.
  • We moved all other limitations to the Discussions chapter, as suggested.

 

Thank you for your support and guidance!  

Best Regards,

Authors

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The modification fine and ready for publication

Author Response

Dear reviewer,

Thank you for your prompt and professional feedback, much appreciated! Your insightful guidance has significantly strengthened our manuscript.

We wish you continued success in your future research endeavors.

 

Best Regards,

Authors

Reviewer 2 Report

Comments and Suggestions for Authors

In this current version, the authors try to address most problems during the first round of reviews. However, the paper's content still lacks a detailed discussion on how the result can improve the learning outcome since it only assesses the user engagement factors.

Further possible improvements could be related to the explanation of the following problem:

1. To avoid ambiguity, the title can be focused on user engagement evaluation instead of the learning outcomes.

2. It would be valuable to see longitudinal data to determine if these effects persist over time or translate into measurable learning gains. Without such evidence, asserting that improved flow correlates with better learning outcomes remains somewhat speculative, given the focus on user perceptions rather than objective performance measures.

 

Author Response

Dear reviewer,

Thank you for carefully analyzing the second version of our manuscript. Your feedback is much appreciated and shows a deep understanding of our ambitions related to the direction of our research - which is indeed more inclined towards user engagement and less on learning outcome/learning performance.

All your observations have been addressed in the new version, which is now more align with this fundamental direction.

• Comment 1: To avoid ambiguity, the title can be focused on user engagement evaluation instead of the learning outcomes.
• Response: We agree that the title can be more representative. As a result, a slight change has been applied to better express the contents of our paper (L2 - L3).

 

• Comment 2.1: It would be valuable to see longitudinal data to determine if these effects persist over time or translate into measurable learning gains.
• Response: The new manuscript version acknowledges this aspect as an important limitation of the study and an opportunity for future studies as well. More specifically, these mentions can be found at: (L773 - L779), (L911 - L914), (L967 - L970), (L980 - L983).

 

• Comment 2.2: Without such evidence, asserting that improved flow correlates with better learning outcomes remains somewhat speculative, given the focus on user perceptions rather than objective performance measures.
• Response: In order to enhance the clarity around user engagement versus learning outcomes/learning performance, we have reformulated most mentions related to learning outcomes to better illustrate that we were referring indeed to engagement and perception. More specifically: (L173 - L184), (L584 - L586), (L812 - L814). 

 

Lastly, figures and tables have been slightly improved as well.

Thank you for your support and guidance!

Best Regards,
Authors

Reviewer 3 Report

Comments and Suggestions for Authors

Thanks a lot for your cooperation and for taking into account my suggestions. I think the paper is ready to be accepted.

Author Response

Dear reviewer,

We would like to thank you as well for your prompt and professional feedback! Your insightful guidance has significantly strengthened our manuscript.

We wish you continued success in your future research endeavors.

 

Best Regards,

Authors