Artificial Intelligence-Generated Content Empowers College Students’ Critical Thinking Skills: What, How, and Why

Round 1

Reviewer 1 Report (Previous Reviewer 1)

Comments and Suggestions for Authors

The study has been significantly improved through the revisions made and now demonstrates enhanced academic quality.

 

Author Response

Reviewer 1 did not provide any revision suggestions.

Author Response File: Author Response.docx

Reviewer 2 Report (New Reviewer)

Comments and Suggestions for Authors

Dear Authors,

A pertinent, clear, and well-written article that presents a good literature review and a correlational methodology.

However, it raised some questions for me:
(i) Since this article only allows for establishing associations between variables due to the data collection and, above all, data analysis methodology, we are left wondering whether it is the students' intrinsic characteristics that enable a more appropriate use of Generative AI and, consequently, better results in the variables under study, especially the improvement of self-directed learning skills, self-reflection, and critical thinking. If you had used the structural equation analysis model, you could have answered my question. However, you should warn readers that this is a study in which it is not possible to establish cause-and-effect relationships. But this aspect is only a cautionary tale.
(ii) This second aspect is more important because you only present an indicator of internal consistency (Cronbach's alpha) as an indicator of the validity of the data collection instruments used, and no indication of the construct validity in the sample to which all the multidimensional scales were administered. An instrument can be consistent and not be valid. Construct validity (since content validity is assured because you used scales already developed by other authors) was not determined. I recommend using factorial validity (exploratory factor analysis and/or confirmatory factor analysis) and only then alphas for each factor that emerges for each scale.
(iii) Another aspect concerns the interviews. You conducted five and selected randomly. Now, the interviews were to delve deeper into certain aspects, and it would have been important to have chosen your sample using some criteria. For example, positive and negative scores or other criteria (outliers), rather than just randomly selecting them. I also didn't understand what content analysis technique was used for the interview protocols and how you arrived at those results.
In summary: A good article that would be very good if you could answer and integrate all or some of the questions I raised.

Author Response

Comments 1: [Since this article only allows for establishing associations between variables due to the data collection and, above all, data analysis methodology, we are left wondering whether it is the students' intrinsic characteristics that enable a more appropriate use of Generative AI and, consequently, better results in the variables under study, especially the improvement of self-directed learning skills, self-reflection, and critical thinking. If you had used the structural equation analysis model, you could have answered my question. However, you should warn readers that this is a study in which it is not possible to establish cause-and-effect relationships. But this aspect is only a cautionary tale.]

Response 1: [In Section 5.4, “Limitations and Future Research,” I have added text noting the absence of causal-analysis techniques as a limitation of this study and recommending that future research adopt such methods to analyze the data.] 

Comments 2: [This second aspect is more important because you only present an indicator of internal consistency (Cronbach's alpha) as an indicator of the validity of the data collection instruments used, and no indication of the construct validity in the sample to which all the multidimensional scales were administered. An instrument can be consistent and not be valid. Construct validity (since content validity is assured because you used scales already developed by other authors) was not determined. I recommend using factorial validity (exploratory factor analysis and/or confirmatory factor analysis) and only then alphas for each factor that emerges for each scale.]

Response 2: [I tested the questionnaire data for common method bias using exploratory factor analysis (EFA). In Section 3.1, “Data source,” I have added the following paragraph to clarify that the influence of common method bias on the survey responses is minimal: the questionnaire employed single-factor test to assess common method bias. The researcher subjected the data from the key items and scale items to exploratory factor analysis (EFA) to determine whether a single factor dominated all items. The principal component analysis revealed that the first unrotated factor explained 23.03 % of the variance, which is below the 40 % threshold, indicating that the data were not severely affected by common method bias.

I also conducted exploratory factor analysis (EFA) on the study’s four main scales—the Intrinsic Learning Motivation Scale, the Critical Thinking Disposition Scale, the AI Dependence Scale, and the AI Literacy Scale—and reported the factor-analytic results to demonstrate that all four instruments possess sound validity.]

Comments 3: [Another aspect concerns the interviews. You conducted five and selected randomly. Now, the interviews were to delve deeper into certain aspects, and it would have been important to have chosen your sample using some criteria. For example, positive and negative scores or other criteria (outliers), rather than just randomly selecting them. I also didn't understand what content analysis technique was used for the interview protocols and how you arrived at those results.

In summary: A good article that would be very good if you could answer and integrate all or some of the questions I raised.]

Response 3: [I randomly selected five interviewees to ensure that every student profile had an equal chance of being chosen, thereby maximizing the sample’s representativeness of the broader population and reducing sampling bias. For example, this method aimed to include both students with strong and weak self-regulated learning abilities so that differences in how they use and experience AIGC could be captured and better triangulated with the survey data.]

Author Response File: Author Response.docx

Reviewer 3 Report (New Reviewer)

Comments and Suggestions for Authors

This paper addresses an important and timely topic. It is well structured, with a clear abstract, detailed literature review, and an appropriate theoretical framing using self-regulated learning and learning motivation theories. The use of both quantitative and qualitative data strengthens the study, and the findings offer relevant implications for higher education practice.

I recommend several minor revisions to improve the clarity, academic rigour and overall readability of the paper, as follows...

Clarity and Conciseness 

• The introduction could be more succinct by reducing repetition (e.g., references to previous studies on AIGC and high-order thinking appear multiple times). 

• The review of existing literature is well-cited, but in some instances the connections to the current research focus could be made more explicit, particularly regarding the distinction between frequency of AIGC use and reflective thinking.

Research Design and Methods

• The survey instrument would benefit from a brief justifcation of why self-reported data (questionnaires) were chosen as the primary quantitative method, given potential biases.

• The rationale for selecting stepwise regression (versus other models) could be better explained.

Results Presentation

• The tables could be more reader-friendly by standardising terminology.

• The findings from the interviews are insightful but might benefit from the inclusion of 1–2 representative quotes (anonymised), if available in the researchers’ data.

Discussion and Comparison with Previous Studies

• The discussion could be more balanced by addressing potential alternative interpretations of the results (e.g., whether students who are already strong critical thinkers are more likely to use AIGC reflectively, rather than reflective AIGC use leading to critical thinking).

• Some parts of the discussion slightly restate findings rather than critically analysing them.  

Limitations and Future Research

• To add to the limitations, a more detailed reflection on the limiteds sample (e.g., students from one university in central Hunan Province) would be valuable. 

Comments on the Quality of English Language

The English is generally clear but would benefit from light language polishing to improve succinctness, simplicity, flow and readability.

 

Author Response

Comments 1:

Clarity and Conciseness：The introduction could be more succinct by reducing repetition (e.g., references to previous studies on AIGC and high-order thinking appear multiple times). The review of existing literature is well-cited, but in some instances the connections to the current research focus could be made more explicit, particularly regarding the distinction between frequency of AIGC use and reflective thinking.

Response 1: In the Introduction, all text citing Kofahi & Husain (2025)—a reference previously used multiple times—was removed, as were passages citing Almelweth (2022), resulting in a more concise opening. In the Literature Review, the paraphrased content from Li Manli et al. (2025) was revised to eliminate redundancy with the Introduction. The wording was further refined to foreground the relationship between AIGC usage and reflective thinking, thereby strengthening its alignment with Research Hypothesis 1.

Comments 2:

Research Design and Methods：The survey instrument would benefit from a brief justifcation of why self-reported data (questionnaires) were chosen as the primary quantitative method, given potential biases. The rationale for selecting stepwise regression (versus other models) could be better explained.

Response 2: In Section 3.1 Data Source, a paragraph describing the common method bias test for the questionnaire has been added. In Section 3.3 Data Analysis Methods, an explanation of the principles underlying stepwise multiple regression has been included.

Comments 3:

Results Presentation：The tables could be more reader-friendly by standardising terminology. The findings from the interviews are insightful but might benefit from the inclusion of 1–2 representative quotes (anonymised), if available in the researchers’data.

Response 3: All inconsistent terminology in the text, tables, and figures was harmonized: “Usage frequency of AIGC” is now abbreviated as UF throughout, and “AIGC is used for reflection” is consistently abbreviated as AUR (replacing the previously mixed use of UR). In Section 4.4 “Analysis of the interview materials,” the content was revised to incorporate three additional verbatim quotations from the interviewees.

Comments 4:

Discussion and Comparison with Previous Studies：The discussion could be more balanced by addressing potential alternative interpretations of the results (e.g., whether students who are already strong critical thinkers are more likely to use AIGC reflectively, rather than reflective AIGC use leading to critical thinking).Some parts of the discussion slightly restate findings rather than critically analysing them.  

Response 4: In Section 5.2 “Comparison with previous studies,” a new paragraph has been added that interprets the quantitative findings from an alternative perspective: students who already possess stronger critical-thinking skills may be more inclined to employ AIGC for reflective thinking. This potential explanation is then compared with the results reported in relevant prior literature. Section 5.1 “Overview of findings” continues to serve as a concise summary of the present study’s results, synthesized from the analyses of both the questionnaire and the interview data. The critical evaluation of these results is carried out in Sections 5.2, 5.3, 5.4, and 5.5.

Comments 5:

Limitations and Future Research：To add to the limitations, a more detailed reflection on the limiteds sample (e.g., students from one university in central Hunan Province) would be valuable.

Response 5: In Section 5.4 “Limitations and Future Research,” we have added a note acknowledging that the sample is confined to a single university and is relatively small. Future studies will be designed to address this limitation.

 

Author Response File: Author Response.docx

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The abstract section should include the research topic, aim, problem, methodology, and key findings as expected in a scientific study. However, the current abstract predominantly focuses on the results and recommendations, lacking sufficient information about the research aim, core problem, and methodology. It is particularly important to briefly mention the research context, the adopted research model, and the analysis techniques within the abstract to enhance the initial impression of the study. In the keywords section, academic writing conventions require the full and explicit forms of concepts to be provided. Abbreviations (e.g., “AIGC”) are not appropriate in the keywords; such terms should first be presented in their full form and subsequently abbreviated within the main text. Regarding the methodology section, the information about data collection instruments is limited and should be elaborated in a clearly defined subsection such as “Data Collection Instruments.” Details about which scales were used, their sources or adaptation processes, the number of items, and reliability coefficients need to be systematically and comprehensively organized to strengthen the methodological rigor of the study. Furthermore, the fundamental research approach (quantitative, qualitative, or mixed) and the employed research design (e.g., correlational survey, causal-comparative, path analysis) should be explicitly stated. The methodology must be clearly defined not only in terms of instruments but also theoretically and design-wise. Therefore, the methodology section should be revised to provide clearer information about the research design. In conclusion, the manuscript addresses a timely and important topic with potential to contribute to the literature. Implementing the suggested structural and content-related improvements will enhance the scientific foundation of the study. Especially improvements in the abstract and methodology sections will significantly increase the comprehensibility of the work for both reviewers and readers.

Author Response

Comments 1:The abstract section should include the research topic, aim, problem, methodology, and key findings as expected in a scientific study. However, the current abstract predominantly focuses on the results and recommendations, lacking sufficient information about the research aim, core problem, and methodology. It is particularly important to briefly mention the research context, the adopted research model, and the analysis techniques within the abstract to enhance the initial impression of the study.

Response 1: I have rewritten the abstract of the paper according to the framework of research background, research topic, research purpose, research questions, research methods, research findings, theoretical explanations, and inspirations.

Comments 2:In the keywords section, academic writing conventions require the full and explicit forms of concepts to be provided. Abbreviations (e.g., “AIGC”) are not appropriate in the keywords; such terms should first be presented in their full form and subsequently abbreviated within the main text.

Response 2: I have changed the AIGC in the keyword to its full name: Artificial intelligence generated content。

Comments 3: Regarding the methodology section, the information about data collection instruments is limited and should be elaborated in a clearly defined subsection such as “Data Collection Instruments.”Details about which scales were used, their sources or adaptation processes, the number of items, and reliability coefficients need to be systematically and comprehensively organized to strengthen the methodological rigor of the study.

Response 3: I revised the content of the data collection section. In the text, I presented the question items of the two variables, the frequency of AIGC use and the use of AIGC for reflection, in the questionnaire. I also elaborated on the sources, number of items, and reliability coefficients of the critical thinking disposition scale, self-regulated learning ability scale, intrinsic learning motivation scale, AI dependency scale, and AI literacy scale. Meanwhile, I placed the items of these scales in the appendix following the main text of the article.

Comments 4: Furthermore, the fundamental research approach (quantitative, qualitative, or mixed) and the employed research design (e.g., correlational survey, causal-comparative, path analysis) should be explicitly stated. The methodology must be clearly defined not only in terms of instruments but also theoretically and design-wise. Therefore, the methodology section should be revised to provide clearer information about the research design.

Response 4： In both the research hypothesis section and the research method design section, additional explanatory text has been added.  In the section where Hypothesis 1 is proposed, it is theoretically argued that when college students apply AIGC to reflective thinking activities, it is possible to promote the development of their critical thinking skills, as there is a high correlation between the two. To this end, this study aims to verify this hypothesis by revealing the correlation between the two.  In the section where Hypothesis 2 is proposed, it is theoretically suggested that there may be a mediating effect among self-regulated learning ability, the application of AIGC to reflection, and critical thinking skills. In the section where Hypothesis 3 is proposed, it is theoretically posited that there may be a mediating effect among intrinsic learning motivation, the application of AIGC to reflection, and critical thinking skills. In the research method design section, the rationale for the data analysis methods used in this study is explained, including the reasons for employing correlation analysis, multiple stepwise regression analysis, and mediation effect analysis using the Process plugin in SPSS.

 

 

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

Review Report

Article title: Artificial Intelligence Generated Content Empowers College Students' Critical Thinking Skills: What, How, and Why

The study explores the relationship between the application of Artificial Intelligence Generated Content (AIGC) and critical thinking (CT). The topic of the article is difficult and should be covered in detail. The major issues with the paper are described in the following.     

1. The questionnaire method chosen by the authors to investigate this relationship is not suitable for drawing unambiguous conclusions. This method needs to be supplemented with other research methods.
2. The individual items of the questionnaire are not listed in the appendix of the manuscript.
3. The abstract does not contain the aim, methodology, or research sample, but only the conclusions.
4. The aim of the study is not stated throughout the article.
5. In Chapter 2, the authors write in several places "some argue that..." but they don't say who is claiming this and they also don't cite the source.
6. In subsections 5.2 and 5.3, results from the same research are presented twice (Avello et al., 2024), (Kofahi& Husain, 2025).
7. Table 2 should be followed by Table 3, not Table 4.
8. There are no graphically processed or evaluated results. Present the most important results in a graphical form (pie charts, etc.).
9. In subsection 5.2, the authors refer to studies, but do not cite them precisely.
10. The text does not clearly state which of the three hypotheses can be accepted or rejected.
11. The article has few citation sources.

Author Response

Comments 1: The questionnaire method chosen by the authors to investigate this relationship is not suitable for drawing unambiguous conclusions. This method needs to be supplemented with other research methods. 

Response 1: I have added an additional research method: interviews. I prepared an interview outline and conducted interviews with five randomly selected college students. In the research design section of the article, I introduced the interview outline and elaborated on the questions it contains. Additionally, in Section 4 of the article, I included an analysis of the interview materials. The Overview of findings section also incorporates content related to the interview materials. The research findings and conclusions are derived based on the analysis of both the survey questionnaires and the interview materials.

 

 

Comments 2:The individual items of the questionnaire are not listed in the appendix of the manuscript. 

Response 2: In the Materials and Methods section of the article, I added some text to introduce the survey questionnaire, including the original questions related to the frequency of AIGC use and the use of AIGC for reflection in the survey. Additionally, the items of the critical thinking disposition scale, self-regulated learning ability scale, intrinsic learning motivation scale, AI dependency scale, and AI literacy scale are included in the form of an Appendix after the main text of the article.

 

 

Comments 3: The abstract does not contain the aim, methodology, or research sample, but only the conclusions.

Response 3: I have rewritten the abstract of the paper according to the framework of research background, research topic, research purpose, research questions, research methods, research findings, theoretical explanations, and inspirations.

 

Comments 4: The aim of the study is not stated throughout the article. 

Response 4：In the final paragraph of the first section, Introduction, the article states the research purpose, which is: To this end, this study aims to reveal whether AIGC enhances college students' critical thinking skills in an open learning environment (what is it) through the analysis of survey data and interview materials, to reveal its mechanism and path (how does it work), to analyze the reasons from the perspective of self-regulated learning theory and learning motivation theory (why does it happen), and to offer suggestions on how college students can use AIGC reasonably in an open environment.

 

Comments 5:In Chapter 2, the authors write in several places "some argue that..." but they don't say who is claiming this and they also don't cite the source. 

Response 5:  There were two instances of "some argue that..." in the text, each followed by the citation of the author. I have revised these two expressions, replacing "some" with the corresponding authors of the cited works.。

 

 

Comments 6:In subsections 5.2 and 5.3, results from the same research are presented twice (Avello et al., 2024), (Kofahi& Husain, 2025). 

Response 6: During the process of translating the Chinese manuscript into English, I made an error by copying the content of Section 5.2 Comparison with previously studies into Section 5.3 Theoretical explanations, resulting in identical content in both sections. I have now re-translated the original content of Theoretical explanations from the Chinese manuscript and rewritten it in the English version.。

 

Comments 7:Table 2 should be followed by Table 3, not Table 4..

Response 7: The numerical numbering of the tables has been corrected.

 

Comments 8:There are no graphically processed or evaluated results. Present the most important results in a graphical form (pie charts, etc.)..

Response 8：I have added two figures in Section 4.3. Analysis of mediation effect of the article to illustrate the mediating effects of self-regulated learning → AIGC for reflection → critical thinking disposition and intrinsic learning motivation → AIGC for reflection → critical thinking disposition.

 

Comments 9:In subsection 5.2, the authors refer to studies, but do not cite them precisely. 

Response 9：The content of this section has been rewritten, with several additional references to studies relevant to this research. The conclusions of this study have been compared with those studies.

 

Comments 10：The text does not clearly state which of the three hypotheses can be accepted or rejected.

Response 10：In Section 5.1 Overview of findings, the text explicitly states that Hypothesis 1, Hypothesis 2, and Hypothesis 3 have been validated.

 

Comments 11：The article has few citation sources.

Response 11：I have added several research papers related to the theme of this article in the Introduction section. I have rewritten Section 5.2, "Comparison with previously studies," and conducted a more in-depth comparison between the conclusions of this study and those of previous studies. Compared to the original version, seven additional relevant references have been included.

Author Response File: Author Response.docx