Research on Learning Concentration Recognition with Multi-Modal Features in Virtual Reality Environments

Round 1

Reviewer 1 Report

Dear author(s),

 I hope you are doing well, and thank you for your submission. I appreciate the time that you put into this manuscript. Please see the following suggestions to increase the quality of this manuscript:

 General

·         Please proofread, especially for some format issues, and make sure you are using a consistent style throughout the manuscript.

Introduction

·         On Page 2, you mentioned, “there is an urgent need to explore multi-dimensional integration methods for concentration recognition.” Please provide citations to back up this.

·         On Page 2, you mentioned, “it’s urgent to explore the recognition of learning concentration in VR environments by integrating multi-dimensional and multi-type feature data.” Justify the urgency.

Materials and Methods

·         Sampling strategy is not clear. How did you recruit these participants? Were there any inclusion/exclusion criteria?

·         Add more demographic information. Who are these “41 students who major in geography from a university in Zhejiang, China”? I would also suggest including a table to summarize demographic info.

·         Page 7, provide citations and references for Brockmyer et al. and Bob et al.; also provide more detailed information about the immersion questionnaire, its reliability, and adapting process.

·         Table 1, I would suggest adding a column to include cognitive, emotional, and behavioral dimensions for the data.

Discussion and conclusion

·         Please make stronger connections among your data analysis, results, and your conclusion. It’s not clear how your conclusions follow the results.

·         Report and discuss the coefficient of determination for your correlation tests.

Best wishes,
Your peer

Author Response

Please check the attachment for details.

Author Response File: Author Response.pdf

Reviewer 2 Report

This manuscript proposed a multi-model feature integration-based learning concentration method in VR environments. By analyzing experiment results, the manuscript claims that the proposed method’s accuracy outperforms single-dimensional and single-type recognition methods.

My main comments are

1. ML methods are vaguely mentioned without detailed setup. You mentioned decision tree, random forest etc., but how are they conducted? You might use mature packages or your own code, please elaborate them and include details such as: # of epochs, how/whether they converges, learning rate, how random forest is generated, how you divide data into training, validation, test datasets etc. 

ML models are essential as they provide main results for the paper to analyze. Including those details make your manuscript more convincing.

2. Regarding 5.3.4 Model validity, I’m wondering how you do t-tests as different MLs only provide ONE F1 score data point.

3. You have 3 categories for labels (low, medium, high), how did you get precision/recall/F1-score? Usually that’s calculated by true or false Positive/Negative, which are binary. You’ll need to specify a label if you really want to. If you did, please add that. But if you did so, why you don't use binary labels?

Please also provide the label distribution of your 147 samples.

Author Response

Please check the attachment for details.

Author Response File: Author Response.pdf

Reviewer 3 Report

This manuscript describes research conducted to identify the level of learning concentration in a virtual reality (VR) environment. Learning concentration is considered an important factor affecting learning outcomes and has an impact on learners' self-regulation and teaching decisions and interventions by teachers. The results and discussions are insightful and valuable for advancing education. If the following problems can be solved (on the word file), this research may be published in MDPI Sustainability:

Comments for author File: Comments.pdf

There is some inconsistency in the tenses used in the manuscript.

Author Response

Please check the attachment for details.

Author Response File: Author Response.pdf

Reviewer 4 Report

It is a necessary to investigate learners' learning concentration recognition in learning using VR. The need for research was well described in the introduction. Overall, I think your submitted manuscript is relatively well orgarnized. However, there are somethings that need to be supplemented in order to be published in this journal. What need to be supplemented are as follows.

1. When VR is used for educational purposes, it is often aimed at improving learners' concentration, so a lot of research has been done on this, but I don't understand why you stated that there was little research.

2. Although the experiment was conducted for educational purposes, there are many ethical considerations for research using devices such as VR. For example, some people complain of dizziness or vomit after experiencing VR. In some cases, severe psychological discomfort may be reported. Therefore, it is recommended that these studies be conducted with IRB approval. However, as mentioned above, data can be collected from the subjects of the originally planned training or the participants of planned learning program. If so, please specify that, and include ethical considerations in the experiment in the description of the research procedure.

3. Please describe the characteristics of the subjects of experiment in a little more detail.

4. Please describe in detail the criteria and evidence for interpreting EEG data.

5. In t-test result, please indicate the level of significance up to 0.001,

6.While it may be significant that you explored "the cognitive, emotional and behavioral dimensions of learning concentration" through an experiment, findings that "learners with higher concentration levels achieve better learning outcomes, and learners' perceived sense of immersion is an important factor influencing their concentration” are too common sense.

7. In the discussion section, be more specific about the educational implications of your findings.

Author Response

Please check the attachment for details.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

The authors have addressed all my comments.

Reviewer 3 Report

I thank the authors for their replies, and the replies satisfy me. I would like to recommend the publication of the current manuscript in the journal.