How Learning Environments Affect University Students’ Employability Skills Development: Students and Staff Views

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This manuscript examines the impact of 3D virtual reality (VR), 2D desktop, and lecture-based learning environments on the development of undergraduate employability skills. The topic is timely and relevant to higher education’s growing focus on digital transformation and immersive learning. However, several key areas require improvement before the paper can be considered for publication:

1. Framework- The paper primarily frames VR as a technological innovation, rather than linking it to robust pedagogical or psychological theories (e.g., experiential learning, presence theory, or constructivism). Strengthening this dimension would improve the paper’s scholarly contribution.
2. Validity - The “Employability Skills Questionnaire” was self-developed but lacks information on validation, reliability, or internal consistency. Please include Cronbach’s α or other reliability indices and discuss construct validity.

3. The student's study yielded non-significant results. These findings should not be interpreted as “equally effective.” Clarify that non-significance does not confirm equivalence, or use equivalence testing if that is the intended claim.

4. Statistical Transparency: Report confidence intervals and effect sizes alongside p-values. For small samples (n = 42 students, n = 12 staff), discuss statistical power limitations explicitly.

5. Discussion and Implications: The discussion would benefit from a more in-depth explanation of why staff perceive VR as more effective. Consider novelty, immersion, or professional experience as potential mediators. Also, expand the implications for curriculum design and digital pedagogy.

6. Limitations and Future Research: The limitations section should elaborate on sample size, generalizability, self-report bias, and novelty effects. Suggest possible directions for future longitudinal or mixed-method research.

7. Language and Style: The manuscript is clear, but specific phrasing (e.g., “Students and Staffs Views”, should be "staff") and minor redundancies in the introduction and discussion should be edited for conciseness. Ensure consistent APA referencing and remove repeated URLs.

My Overall assessment:

This paper makes a useful empirical contribution to the emerging literature on VR and employability education. However, it requires significant revisions, particularly in theoretical framing, instrument validity, and the interpretation of null findings, to achieve publishable quality.

Author Response

Comment 1: The paper primarily frames VR as a technological innovation, rather than linking it to robust pedagogical or psychological theories (e.g., experiential learning, presence theory, or constructivism). Strengthening this dimension would improve the paper’s scholarly contribution.

Response 1: The authors have included discussions as to how the research relates to the theories of experiential learning as well as more details regarding presence theory in both the introduction and discussion sections

Comment 2:  The “Employability Skills Questionnaire” was self-developed but lacks information on validation, reliability, or internal consistency. Please include Cronbach’s α or other reliability indices and discuss construct validity.

Response 2: A McDonalds Omega has now been conducted to assess the internal consistency of the questionnaire. The McDonalds Omega was chosen over a Cronbachs alpha as it gives a more accurate and flexible estimate of reliability (e.g., doesn't assume tau-equivalance). Information regarding the construct validity has also been added to the method section

Comment 3: The student's study yielded non-significant results. These findings should not be interpreted as “equally effective.” Clarify that non-significance does not confirm equivalence, or use equivalence testing if that is the intended claim.

Response 3: This has been reworded to be clearer

Comment 4: Report confidence intervals and effect sizes alongside p-values. For small samples (n = 42 students, n = 12 staff), discuss statistical power limitations explicitly.

Response 4: All p-values now have appropriate effect sizes and confidence intervals attached. The authors decided that the limitation of the sample size may not be appropriate given the sample size is equal to (in study 1) or more than (in study 2) the minimum sample recommended by the G*Power analysis. The information regarding the power analysis can be found in the participants section for both the student and staff sample

Comment 5: The discussion would benefit from a more in-depth explanation of why staff perceive VR as more effective. Consider novelty, immersion, or professional experience as potential mediators. Also, expand the implications for curriculum design and digital pedagogy.

Response 5: The discussion section has been updated to include further explanations as to why staff may perceive VR to be more effective. This includes discussions around the novelty of a VR simulation as well as how increased immersion could be involved. Further details regarding the implications of the study for curriculum design and digital pedagogy have also been discussed

Comment 6: The limitations section should elaborate on sample size, generalizability, self-report bias, and novelty effects. Suggest possible directions for future longitudinal or mixed-method research.

Response 6: The limitations section has now been expanded to include information regarding the generalisability, self-report bias and novelty effects. This includes discussions regarding the longitudinal effects. The sample size was not discussed as the authors deemed the G*Power analysis conducted for both the student and staff samples to be appropriate given the authors recruited either equal to (in study 1) or more than (in study 2) the suggested minimum sample size

Comment 7: The manuscript is clear, but specific phrasing (e.g., “Students and Staffs Views”, should be "staff") and minor redundancies in the introduction and discussion should be edited for conciseness. Ensure consistent APA referencing and remove repeated URLs.

Response 7: Edits have been made to fix grammatical mistakes (i.e., Staffs to Staff). Information in the introduction and discussion has been reviewed to try and make the information more concise. APA referencing has been checked and repeated URLs have been removed

Reviewer 2 Report

Comments and Suggestions for Authors

Thank you for submitting your article. I have gone through your paper with much interest. However, I feel that the manuscript has some major issues. The article investigates how the learning environment affects employability skills. In the introduction section, you need to establish the research gap from the point of view of the previous research effect of learning environments on employability. For example, please refer to Martínez-Argüelles et al (2023). You need to include a literature review section on the effects of learning environments research studies.

In the analysis section, you need to establish the reliability and validity of all three constructs that you have studied. Some of the items that you measured in your study might not be reliable and valid in measuring the particular construct, and thus, using the summated or mean score of all the items of the construct would not be appropriate. You might possibly get a significant effect for the relationships in your ANOVA when you drop some of the items based on reliability and validity testing. You need to perform a normality test before using ANOVA. In the discussion sections, you need to include both theoretical and practical implications of your findings. This, of course, needs to be improved after performing fresh ANOVAs with new findings that you draw with fresh scores for each construct that you derive after testing the reliability and validity.

Reference:

Martínez-Argüelles, M. J., Plana-Erta, D., & Fitó-Bertran, À. (2023). Impact of using authentic online learning environments on students’ perceived employability. Educational technology research and development, 71(2), 605-627.

Author Response

Comment 1: In the introduction section, you need to establish the research gap from the point of view of the previous research effect of learning environments on employability. For example, please refer to Martínez-Argüelles et al (2023). You need to include a literature review section on the effects of learning environments research studies Martínez-Argüelles, M. J., Plana-Erta, D., & Fitó-Bertran, À. (2023). Impact of using authentic online learning environments on students’ perceived employability. Educational technology research and development, 71(2), 605-627

Response 1: The authors have incorporated the suggestion of including the Martínez-Argüelles et al (2023) paper and included their findings in a discussion as to how different learning environments have previously been used to improve employability skills

Comment 2: In the analysis section, you need to establish the reliability and validity of all three constructs that you have studied. Some of the items that you measured in your study might not be reliable and valid in measuring the particular construct, and thus, using the summated or mean score of all the items of the construct would not be appropriate

Response 2: A McDonalds Omegas has now been included for each of the questionnaires used in both the student and staff sample. Information regarding the construct validity has also been added to the method section for the employability skills questionnaire

Comment 3: You might possibly get a significant effect for the relationships in your ANOVA when you drop some of the items based on reliability and validity testing. You need to perform a normality test before using ANOVA

Response 3: The findings from the McDonalds Omega indicated that no items needed to be removed, resulting in the ANOVA findings remaining the same. The data followed the assumptions of the ANOVA although this was not included in the writeup to keep the results section concise given the length of this section and additional McDonalds Omega analyses used

Comment 4: In the discussion sections, you need to include both theoretical and practical implications of your findings. This, of course, needs to be improved after performing fresh ANOVAs with new findings that you draw with fresh scores for each construct that you derive after testing the reliability and validity

Response 4: The applications section has been expanded to include further details regarding the theoretical and practical implications of the study. This includes more links to digital pedagogy

Reviewer 3 Report

Comments and Suggestions for Authors

Dear author(s),
The study presented has got a relevant and timely very interesting topic in higher education sector. Dual perspective idea has got wide frame enabling both students and teachers views insight.
To further improve the relevance, we are suggesting the following improvements:
-    The abstract should be summarized and note all relevant findings of the research conducted
-    Theoretical part should deal more widely with adequate learning theories that support ide that virtual reality supports learning experience to better fit the practical experience. Previous studies related to the one conducted would also add more reliability in a comparative approach.
-    Sample size should be fitted to relate to the standard frame in all groups (both students and staff) Additionally single institution sample limits generalizability of the results obtained.
-    Is should be additionally explained how and why the timing was not the same for all groups (30 mins for students vs. 10 mins for teachers)
-    The questionnaire itself should be appended and validated additionally
-    Research sessions imbalance should be explained and elaborated further (3o mins for students and 10 mins for staff)
-    Please recheck the references additionally
Kind regards,
The Reviewer

Author Response

Comment 1: The abstract should be summarized and note all relevant findings of the research conducted

Response 1: The abstract has been modified to include more details regarding the summary of the findings

Comment 2: Theoretical part should deal more widely with adequate learning theories that support that virtual reality supports learning experience to better fit the practical experience. Previous studies related to the one conducted would also add more reliability in a comparative approach

Response 2: The authors have included more details regarding appropriate learning theories. In particular, experiential learning and presence theory

Comment 3: Sample size should be fitted to relate to the standard frame in all groups (both students and staff) Additionally single institution sample limits generalizability of the results obtained

Response 3: The limitation regarding the generalisability of the sample due to conducting the study in a singular institution has been added to the limitations section. As the recruited sample size is either equal to (in study 1) or exceeds (in study 2) the minimum recommended sample for the G*Power analysis (further details can be found in the participants section in the student and staff sample) the sample size limitation was not discussed

Comment 4: It should be additionally explained how and why the timing was not the same for all groups (30 mins for students vs. 10 mins for teachers)

Response 4: To address this, further details have been added to the procedure section of the staff sample

Comment 5: The questionnaire itself should be appended and validated additionally

Response 5: Each questionnaire has been added to the appendices, with information regarding the construct validity and reliability assessed through a McDonalds Omega being included in the main text

Comment 6: Research sessions imbalance should be explained and elaborated further (3o mins for students and 10 mins for staff)

Response 6: Further explanation regarding the justification for the differences in timings has been added to the procedure section of the staff sample

Comment 7: Please recheck the references additionally

Response 7: References have been rechecked and updated appropriately

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

This revised version (V2) is more polished, more rigorous, better justified, and more aligned with journal expectations. But it still has some minor issues you may want to refine before resubmission:

1. Consistency of terminology

   Use either “desktop” or “computer,” not both.
   V2 is better but still contains occasional mixing.

2. APA inconsistencies in references

   A few minor formatting issues remain, especially in V2’s reference list.

3. The ranking ANOVA and interpretation could be more explicit for readers unfamiliar with ranking data.

4. Add one clear paragraph near the end:

   • What new knowledge does this study contribute?

   • Why VR vs desktop comparisons matter theoretically.

   • Why staff/student split is essential for the field.

5. Some repetition can still be trimmed (e.g., repeating reasons why VR is costly).
6. Proofread the manuscript. Some issues still exist, such as:
   • "A total of 42 psychology students (9 male, and 33 female)" (Should be plural).

Author Response

Comment 1: 

Use either “desktop” or “computer,” not both. V2 is better but still contains occasional mixing. 

Response 1:  

The manuscript has been edited to remove the “computer” term and make the wording more consistent 

 

Comment 2: 

A few minor formatting issues remain, especially in V2’s reference list 

Response 2: 

The authors have reviewed the reference section and modified the references and fixed minor formatting issues 

 

Comment 3: 

The ranking ANOVA and interpretation could be more explicit for readers unfamiliar with ranking data 

Response 3: 

Further details regarding the ranking ANOVA as well as its interpretations have been added to the results and discussion sections to help clarify the information 

 

Comment 4: 

Add one clear paragraph near the end. What new knowledge does this study contribute? Why VR vs desktop comparisons matter theoretically. Why staff/student split is essential for the field 

Response 4: 

A paragraph has been included in the discussion section to summarise the knowledge that this study contributes to the literature, the theoretical importance of comparing VR and desktops, as well as why it is important to investigate both students and staffs perspective for this research area 

 

Comment 5: 

Some repetition can still be trimmed (e.g., repeating reasons why VR is costly) 

Response 5: 

The manuscript has been modified to avoid repetition of information such as the statistics behind the cost and accessibility of the VR equipment 

 

Comment 6: 

Proofread the manuscript. Some issues still exist, such as "A total of 42 psychology students (9 male, and 33 female)" (Should be plural) 

Response 6: 

The manuscript has been reviewed for grammatical errors and has been updated to reflect these changes 

Reviewer 2 Report

Comments and Suggestions for Authors

Authors have addressed the comments satisfactorily.

Author Response

The authors would like to thank the reviewer for their time in improving the manuscript

Reviewer 3 Report

Comments and Suggestions for Authors

Dear author(s),

Thank you for the improvements made and reply to the review prepared. It is suggested additionally to make further, final improvements as noted below:   Please also explain additionally (through direct statistical comparison and theory-based explanation) the contradiction between the findings for students and staff. Students showed no significant differences across conditions, while staff rated VR as significantly superior on employability skills (p<.001), hedonic experience (p<.001), and presence (p=.009). Uncontrolled novelty effects confound VR benefits—the paper acknowledges only 8% of students own VR headsets, yet provides no empirical measurement of novelty perception or longitudinal evidence that effects persist. The reliance on entirely self-report questionnaires without behavioral outcome measures (e.g., mock interview performance ratings) undermines claims about "skills development." The three-item Employability Skills Questionnaire shows poor unidimensionality (hierarchical omega = 0.05), as it mixes affective and skill measures. These core methodological issues—contradictory findings, confounding factors, limitations in self-report, and weak measurement—must be substantially addressed. It is also advised to expand the staff sample to 18-20 participants (current size provides minimal statistical buffer), provide direct statistical testing of student-staff differences with theoretical explanation, add empirical novelty measurement with correlation analysis, develop or select a validated employability skills instrument with 8+ items capturing distinct competency domains, and add at least preliminary behavioral outcome data (video-recorded mock interviews rated by blinded evaluators). Please also conduct gender analysis on the unbalanced student sample (78.6% female with unexplained unequal condition allocation), explicitly bound generalizability to "UK psychology undergraduates at a single institution," and thoroughly revise the Discussion, which should also integrate Cognitive Load Theory and expertise-related metacognition literature, to explain why learning outcomes didn't differ across conditions while staff preferences did. Include a systematic thematic analysis of staff qualitative feedback, with frequencies, to address the 3:1 exposure time difference between students (30 min) and staff (10 min per condition). Finally, emphasise how conclusions reflect perceived, not demonstrated skill development, and acknowledge that transfer to real-world interviews remains unknown. Recommend a longitudinal semester-long study (in future research) to assess whether VR benefits persist beyond novelty effects.   Kind regards, The Reviewer

Author Response

Comment 1: 

Please also explain additionally (through direct statistical comparison and theory-based explanation) the contradiction between the findings for students and staff. 

Response 1: 

Further details have been added to the manuscript to highlight staffs greater knowledge of good interview practice compared to students reduced knowledge as a result of their inexperience. A statistical comparison was not conducted due to the type of designs (within subjects for a staff sample versus a between subjects design for the student sample) not being directly comparable statistically. However, to compensate for this, more details regarding the theoretical comparisons have been included.

 

Comment 2: 

Students showed no significant differences across conditions, while staff rated VR as significantly superior on employability skills (p<.001), hedonic experience (p<.001), and presence (p=.009). Uncontrolled novelty effects confound VR benefits—the paper acknowledges only 8% of students own VR headsets, yet provides no empirical measurement of novelty perception or longitudinal evidence that effects persist. 

Response 2: 

Further details on the novelty and longitudinal limitations of the study have been added to the limitations section of the manuscript as the measurement of this would require participants to be reassessed which is outside of the scope of this study  

 

Comment 3: 

The reliance on entirely self-report questionnaires without behavioral outcome measures (e.g., mock interview performance ratings) undermines claims about "skills development."  

Response 3: 

Discussions regarding the inclusion of behavioural outcomes in the form of mock interviews has been included in the future research section as this falls outside of the scope for this study 

 

Comment 4: 

The three-item Employability Skills Questionnaire shows poor unidimensionality (hierarchical omega = 0.05), as it mixes affective and skill measures. These core methodological issues—contradictory findings, confounding factors, limitations in self-report, and weak measurement—must be substantially addressed.  

Response 4: 

The discussion section has been modified to include information regarding the low unidimesnionality of this questionnaire. A suggestion has also been included that encourages future research to include an employability skills scale with greater unidimensionality,

 

Comment 5: 

It is also advised to expand the staff sample to 18-20 participants (current size provides minimal statistical buffer) 

Response 5: 

The BodySwaps simulation used for this study was only accessible to the researchers for a total of six months, with no opportunity for an extension. As this time has now elapsed, it is no longer possible to expand the staff sample 

 

Comment 6: 

provide direct statistical testing of student-staff differences with theoretical explanation 

Response 6: 

The theoretical explanation between the student and staff sample has been further expanded upon in the staff introduction section. However. a statistical comparison was not conducted due to the type of designs (within subjects for a staff sample versus a between subjects design for the student sample) not being directly comparable statistically. This has been compensated for through more details being added regarding the theoretical comparisons between these two samples

 

Comment 7: 

add empirical novelty measurement with correlation analysis, develop or select a validated employability skills instrument with 8+ items capturing distinct competency domains, and add at least preliminary behavioral outcome data (video-recorded mock interviews rated by blinded evaluators. 

Response: 

As recruitment for this project has already concluded, we are unable to administer additional instruments or collect new behavioural outcome data. Designing and validating a new measurement tool or conducting video-recorded mock interviews with blinded evaluators would require reopening recruitment and extending the study timeline, which is beyond the resources and constraints of this project. We acknowledge the value of your recommendations and agree that they would significantly strengthen future research in this area. We have therefore highlighted these points in the revised manuscript as important directions for subsequent studies

 

Comment 8: 

Please also conduct gender analysis on the unbalanced student sample (78.6% female with unexplained unequal condition allocation) 

Response 8: 

The gender distributions in the student sample are consistent with the UK university trends for Psychology students, with approximately 90% of these students being female (Calia & Kancelikak, 2023). Therefore, it was not deemed necessary to conduct a gender analysis. However, this information has been added to the manuscript for clarity 

Calia, C., & Kanceljak, D. (2023). Diversity and inclusion in UK psychology: A nationwide survey. In Clinical Psychology Forum, 1(369), 15-29. https://doi.org/10.53841/bpscpf.2023.1.369.15 

 

Comment 9: 

explicitly bound generalizability to "UK psychology undergraduates at a single institution," 

Response 9: 

The limitation of using a single institution has been added to the limitations section of the study

 

Comment 10: 

thoroughly revise the Discussion, which should also integrate Cognitive Load Theory and expertise-related metacognition literature, to explain why learning outcomes didn't differ across conditions while staff preferences did 

Response 10: 

The discussion has been expanded to include information related to both Cognitive Load Theory and Metacognition as well as how the findings of the manuscript can be explained through these theories 

 

Comment 11: 

Include a systematic thematic analysis of staff qualitative feedback, with frequencies, to address the 3:1 exposure time difference between students (30 min) and staff (10 min per condition) 

Response 11: 

The qualitative data was obtained through a short open ended feedback item at the end of the questionnaire. As a result, there is not enough information to conduct a full systematic thematic analysis using this data, with this expansion on the qualitative aspect of the study being outside of the scope of the study. However, more quotes have been included from staff to include more qualitative information in the manuscript

 

Comment 12: 

Finally, emphasise how conclusions reflect perceived, not demonstrated skill development, and acknowledge that transfer to real-world interviews remains unknown 

Response 12: 

The manuscript has been edited to reflect that the employability skills assess perceived skill development. Discussions regarding whether the findings can be applied to real world interviews has also been included.  

 

Comment 13: 

Recommend a longitudinal semester-long study (in future research) to assess whether VR benefits persist beyond novelty effects 

Response 13: 

A recommendation to investigate the research from a longitudinal perspective has been included in the limitations section of the study