Next Article in Journal
Vicarious Trauma, Rumination, and Vicarious Post-Traumatic Growth: Rethinking the Role of Clinical Supervision
Previous Article in Journal
Ethical Relativism Mediates the Association Between Openness to Experience and Negative Creativity
Previous Article in Special Issue
The Six-Facet Artificial Intelligence Literacy Questionnaire (SFAILQ): Assessing AI Literacy in Adolescents, Young Adults, and Midlife Adults
 
 
Article
Peer-Review Record

Research on the Impact of Generative Artificial Intelligence Usage Behavior on the Learning Outcomes of Higher Vocational Students

Behav. Sci. 2026, 16(7), 1166; https://doi.org/10.3390/bs16071166
by Yafeng Song *, Kangjian Zhao, Li Li and Wei Dong *
Reviewer 1:
Reviewer 2: Anonymous
Reviewer 3: Anonymous
Behav. Sci. 2026, 16(7), 1166; https://doi.org/10.3390/bs16071166
Submission received: 22 May 2026 / Revised: 2 July 2026 / Accepted: 4 July 2026 / Published: 10 July 2026
(This article belongs to the Special Issue AI Use and Academic Development)

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The manuscript deals with a timely and relevant topic on the relation between generative artificial intelligence use and learning outcomes among higher vocational students. However, the manuscript has major weaknesses conceptually, methodologically, analytically and presentationally. I do not regard the present version as fit for publication.
- Measurement of learning outcomes is very problematic. The study claims to evaluate students’ learning outcomes, but the outcomes are self-reported perceptions, not objective indicators such as grades, practical skills tests, teacher evaluations, internship performance or standardized assessments. This poses a serious risk of social desirability bias and common method bias.
- The dependent variable is thus not actual learning outcomes but perceived learning outcomes of the students. ## Title, Abstract, Discussion and Conclusion should be adjusted as appropriate.
- The regression model is too straightforward. It includes only three dimensions of GenAI use as predictors and does not control for major confounding variables such as prior academic performance, socioeconomic status, institution, major, year level, digital literacy, AI literacy, learning motivation, self-efficacy, academic engagement, teacher support, or frequency of technology use more generally.
- Sample size is large but sample composition is seriously imbalanced. The sample is heavily skewed toward first-year students and under-represents junior and senior students. This is a major limitation, since vocational learning outcomes, in particular skill application and career-related competencies, are expected to vary strongly by training stage and internship experience.
- The major category comparisons are invalid. Some categories appear to have very low sample sizes and some report zero standard deviation, indicating that the group may have very few cases or may lack sufficient variability. Running ANOVA on such uneven, and perhaps microscopic, groups is statistically dubious.
- The scale development process is not sufficiently rigorous for a paper that claims to develop and validate new measurement instruments. The pilot sample consisted of only 40 students, which is too small for robust exploratory factor analysis or scale validation.
- The authors used a principal component analysis with varimax rotation which is not the best method for validating latent constructs. If the dimensions are theoretically correlated, exploratory factor analysis using appropriate extraction methods and oblique rotation would be more defensible.
- The learning outcomes composite score is weighted with the Delphi method. The Delphi procedure is not described. 
- The dimension of GenAI usage frequency confuses simple frequency with dependence and potential over-dependence. Things like “My learning efficiency would drop significantly without GenAI tools” are measuring dependence, not frequency. Perhaps this is part of the explanation for the negative regression coefficient.
- The manuscript does not talk enough about academic integrity. Some items are using GenAI for homework, writing a thesis, writing code, and assignments so the paper should talk about plagiarism, unauthorized assistance, assessment validity, and AI policies of the institution.
- The discussion is too optimistic and sometimes speculative. The authors jump too quickly from weak correlational evidence to sweeping recommendations for colleges, teachers, and students. While these recommendations may be reasonable, they are not well supported by the current empirical design.
- The manuscript needs to be significantly shortened and reorganized. Some parts are too descriptive and the analytical contribution is somewhat limited.

Author Response

Comments 1: [The manuscript deals with a timely and relevant topic on the relation between generative artificial intelligence use and learning outcomes among higher vocational students. However, the manuscript has major weaknesses conceptually, methodologically, analytically and presentationally. I do not regard the present version as fit for publication.]

Response 1: We sincerely appreciate the reviewer’s positive comments on the topic of our study. We are greatly pleased that this research, which explores the impacts of generative artificial intelligence on vocational college students’ learning outcomes, is recognized for its practical significance and timeliness. Our attempt to develop and validate two scales respectively for measuring generative AI usage behavior and vocational students’ learning outcomes is also acknowledged as innovative and practically relevant. These encouraging remarks demonstrate the considerable research potential of our work at the intersection of vocational education and artificial intelligence.

Meanwhile, we fully take on board the reviewer’s valuable concerns regarding conceptual clarity, methodological rigor, depth of data analysis and overall presentation. We have earnestly addressed all the comments and carried out substantial revisions to respond to every raised issue. We have strived to enhance the rigor and reproducibility of this paper in terms of theoretical construction, research design and empirical analysis to meet the standards of high-quality academic journals. We welcome any further comments and suggestions. This new section can be found on page 8, starting from paragraph 4, line 368-393; page 10, starting from paragraph 2, line 439-447;page 18, starting from paragraph 3, line 16-38;page 22, starting from paragraph 2, line 179-239.The revised parts are marked in red in the modified manuscript.

 

Comments 2: [Measurement of learning outcomes is very problematic. The study claims to evaluate students’ learning outcomes, but the outcomes are self-reported perceptions, not objective indicators such as grades, practical skills tests, teacher evaluations, internship performance or standardized assessments. This poses a serious risk of social desirability bias and common method bias.The dependent variable is thus not actual learning outcomes but perceived learning outcomes of the students. ## Title, Abstract, Discussion and Conclusion should be adjusted as appropriate.]

Response 2: We sincerely thank the reviewer for pointing out this critical methodological issue. We fully agree that measuring learning outcomes solely based on students’ self-reported perceptions carries risks of social desirability bias and common method bias, which may undermine the objectivity and reliability of the research findings. We address this concern from two perspectives: empirical testing and future research directions.

In terms of empirical testing, we have added a common method bias test in the revised manuscript. Specifically, we conducted an unrotated exploratory factor analysis on all measurement items using Harman’s single-factor test. The results show that the first factor accounts for 43.12% of the total variance, which is below the 50% threshold. This indicates that common method bias is not a serious issue in this study, and the self-reported data do not substantially compromise the reliability of the main conclusions. The relevant results are presented at the beginning of Section 4 in the revised paper.

From the perspective of future research, we explicitly acknowledge the inherent limitations of self-reported data in the section titled Limitations and Future Research. We also put forward targeted improvements. To assess students’ learning outcomes more accurately, follow-up studies are recommended to adopt multi-source data and objective evaluation indicators. On the basis of student self-reports, researchers may further incorporate course grades, practical training scores, teacher evaluations, internship performance, professional skill assessments and certification results, so as to effectively reduce subjective bias caused by relying merely on self-reported data. In addition, we have revised relevant statements in the abstract to clarify that the measured outcomes refer to self-perceived learning outcomes.

Nevertheless, we acknowledge that it is practically difficult to collect the aforementioned objective indicators for this large-scale questionnaire survey. For instance, academic records are scattered across different institutions, assessment criteria vary among majors, and anonymous surveys make it impossible to match participants with their individual grades. Therefore, the self-rating scale adopted in this study remains reasonable and feasible for an exploratory research. We have clearly explained this trade-off in the revised manuscript for readers’ prudent interpretation. This new section can be found on page 13, starting from paragraph 1, line 562-570; page 22, starting from paragraph 1, line 179-198. The revised parts are marked in red in the modified manuscript.

 

Comments 3: [The regression model is too straightforward. It includes only three dimensions of GenAI use as predictors and does not control for major confounding variables such as prior academic performance, socioeconomic status, institution, major, year level, digital literacy, AI literacy, learning motivation, self-efficacy, academic engagement, teacher support, or frequency of technology use more generally.]

Response 3: We sincerely appreciate the reviewer for insightfully pointing out the issue regarding model specification. We acknowledge that the current regression model only includes the three core dimensions of generative AI usage, without controlling for potential confounding variables such as prior academic performance, socioeconomic status, institutional background, major, grade level, digital literacy, AI literacy, learning motivation, self-efficacy, academic engagement, teacher support and the general frequency of technology use. This limitation indeed weakens the causal inference of our findings.

We have adopted two solutions in the revised manuscript to address this concern. First, we clarify the exploratory nature and research scope of this study. This research aims to preliminarily examine the basic relationships between the three dimensions of generative AI usage behavior (usage frequency, usage habits and usage contexts) and learning outcomes, and it is essentially an exploratory study. Under the anonymous questionnaire survey design, it is practically challenging to collect objective indicators including students’ prior academic records, family background and perceived teacher support. Additionally, introducing excessive control variables with the current sample size (N=977) may lead to model overfitting and insufficient degrees of freedom. Accordingly, we decide to focus on the core independent variables and maintain a parsimonious model structure.

Second, we add a detailed list of control variables in the section of Limitations and Future Research. We explicitly admit the deficiency of insufficient control variables in the regression model, and enumerate key variables to be incorporated in future studies, including prior academic performance, socioeconomic status, institutional type and tier, major category, grade level, digital literacy, AI literacy, learning motivation, self-efficacy, academic engagement, teacher support and general technology usage frequency. We also suggest that future research may adopt hierarchical regression or structural equation modeling to more accurately examine the net effect of generative AI usage behavior after controlling for the above factors.

Thank you again for your constructive comments. All relevant revisions are presented in Section 5 (Discussion), specifically in the part of Limitations and Future Research. We hope these revisions fully address your concerns about the control variables in our model. This new section can be found on page 22, starting from paragraph 2, line 199-208. The revised parts are marked in red in the modified manuscript.

 

Comments 4: [Sample size is large but sample composition is seriously imbalanced. The sample is heavily skewed toward first-year students and under-represents junior and senior students. This is a major limitation, since vocational learning outcomes, in particular skill application and career-related competencies, are expected to vary strongly by training stage and internship experience.]

Response 4: We greatly appreciate the reviewer for identifying this important limitation concerning sample composition. You are absolutely correct that first-year students account for as high as 77% of the total participants, while samples of third- and fourth-year students are severely underrepresented. Given that vocational skill application and professional competence are largely developed during practical training and internships in senior grades, such skewed grade distribution may compromise the external validity of the research findings. We fully accept this comment and have implemented two corresponding revisions in the manuscript.

First, we present objective results and deliver cautious interpretations in the differential analysis. We have clearly reported the sample distribution across different grades and the differences in learning outcome scores in the results section. The analysis reveals no statistically significant differences among students of various grades in terms of knowledge mastery, competency development and overall learning outcomes. Although the unbalanced sample distribution has not led to statistically unreasonable results, we still acknowledge its potential impacts.

Second, we explicitly acknowledge this issue and propose improvement strategies in the section of Limitations and Future Research. We specifically point out the large disparity in sample size across grades and the insufficient representation of senior students. We also suggest that future studies recruit participants from vocational colleges in diverse regions, of different tiers and operating modes. Special efforts should be made to balance the proportion of students across all grades and increase the sample size of senior students, particularly those undergoing internships or approaching graduation. This will help better reflect the actual development of skill application and professional capabilities, and further improve sample representativeness as well as the external validity of research conclusions.

Additionally, we explain the practical difficulties in recruiting senior participants. Most third- and fourth-year students work off-campus for internships, making it far more difficult to distribute and collect questionnaires online compared with junior students who receive in-person courses on campus. This objective constraint is stated in the revised manuscript for readers’ careful reference.

Thank you again for your constructive comments. Relevant revisions have been made to the abstract, results discussion and limitations sections. We hope our candid discussion of limitations and targeted suggestions can address your concerns regarding sample representativeness. This new section can be found on page 14, starting from paragraph 4, line 633-636; page 22, starting from paragraph 3, line 209-220. The revised parts are marked in red in the modified manuscript.

 

Comments 5: [The major category comparisons are invalid. Some categories appear to have very low sample sizes and some report zero standard deviation, indicating that the group may have very few cases or may lack sufficient variability. Running ANOVA on such uneven, and perhaps microscopic, groups is statistically dubious.]

Response 5: We sincerely thank the reviewer for pointing out this issue regarding statistical rigor. We fully agree that the extremely small sample sizes of certain major categories and the zero standard deviation observed in several groups cast doubt on the reliability of conventional analysis of variance. We have addressed this concern from two aspects in the revised manuscript.

Firstly, we provide transparent descriptions of group composition. Among the 19 major categories classified in accordance with national standards, only one student was recruited for each of the Resource, Environment and Safety category, Public Security and Judicial category, as well as Public Administration and Service category in the sampled institutions. The zero standard deviation of these groups results from the lack of within-group variation rather than data errors. We have added relevant explanations in Section 4.2: In addition, several major categories including Resource, Environment and Safety, Public Security and Judicial, and Public Administration and Service have very small sample sizes (n < 5) with a standard deviation of 0.00. The results of inter-group comparisons should therefore be interpreted with caution. Readers can make their own judgments accordingly. Moreover, groups with inadequate samples are excluded from post-hoc multiple comparisons, and we only analyze their trends at the level of descriptive statistics.

Secondly, we clarify directions for future improvement. In the limitations section, we note that follow-up studies shall conduct supplementary sampling for underrepresented major categories to further verify the preliminary findings of this research. We believe these revisions fully respond to your concerns and enhance the transparency of our statistical analysis. This new section can be found on page 14, starting from paragraph 4, line 633-636; page 22, starting from paragraph 3, line 209-220. The revised parts are marked in red in the modified manuscript.

 

Comments 6: [The scale development process is not sufficiently rigorous for a paper that claims to develop and validate new measurement instruments. The pilot sample consisted of only 40 students, which is too small for robust exploratory factor analysis or scale validation.]

Response 6: We sincerely thank the reviewer for raising this crucial issue. We fully acknowledge that a pilot sample of 40 participants would be insufficient to conduct robust exploratory factor analysis and scale validation, which would constitute a notable methodological flaw.

We would like to clarify that this was a typographical error. In fact, we distributed 400 questionnaires for the pilot survey instead of 40. We have corrected this mistake and added detailed information at the beginning of Section 3.2.3. Prior to the formal survey, a total of 400 questionnaires were issued in the pilot phase, and 329 valid responses were collected. Using these 329 valid samples, we completed item analysis, exploratory factor analysis and reliability tests. All relevant results are fully presented in Section 3.2.3.

We understand that this error may have raised doubts about the rigor of our overall scale development process, for which we offer our sincere apologies. We hope this correction and supplementary explanation can fully address your concerns. We greatly appreciate you for identifying this significant oversight in our writing. This new section can be found on page 7, starting from paragraph 1, line 302-308; page 10, starting from paragraph 7, line 453-454. The revised parts are marked in red in the modified manuscript.

 

Comments 7: [The authors used a principal component analysis with varimax rotation which is not the best method for validating latent constructs. If the dimensions are theoretically correlated, exploratory factor analysis using appropriate extraction methods and oblique rotation would be more defensible.]

Response 7: We highly appreciate your valuable comments and fully agree with your viewpoints. Strictly speaking, more appropriate extraction methods and oblique rotation are recommended for validating latent constructs, especially when the dimensions are theoretically correlated with one another. Nevertheless, we have decided to retain principal component analysis (PCA) combined with orthogonal rotation in this study for the reasons elaborated below, and we sincerely hope for your understanding.

The primary objective of this research is not to rigorously verify the theoretical dimensional structure of latent constructs, but to extract a small number of composite indicators from multiple observed variables for subsequent regression analysis and difference tests. In this context, PCA serves as a data reduction technique designed to maximize the explained total variance, and Varimax rotation helps improve the interpretability of components. Additionally, we assumed at the research design stage that these dimensions are conceptually independent rather than highly overlapping or strongly correlated. Accordingly, the adoption of orthogonal rotation is consistent with our theoretical assumptions. For the initial development of scales or the early stage of cross-cultural validation, PCA with Varimax rotation is still widely recognized as an effective tool for dimensionality reduction.

Even so, we fully recognize your higher criteria for methodological selection. We have added relevant notes in the limitations section, suggesting that future studies may adopt exploratory factor analysis based on the common factor model together with oblique rotation to further verify the construct validity of this research. Thank you again for your rigorous review. Your suggestions are extremely valuable for improving the quality of our work. We hope that with the above explanations and supplementary discussions while keeping the original analytical methods, you can gain a more comprehensive understanding of our methodological choices. This new section can be found on page 22, starting from paragraph 1, line 196-198. The revised parts are marked in red in the modified manuscript.

 

Comments 8: [The learning outcomes composite score is weighted with the Delphi method. The Delphi procedure is not described.]

Response 8: We sincerely thank the reviewer for pointing out this issue. We fully acknowledge that the original manuscript failed to clearly describe the weighting determination method and contained an incorrect name for the adopted approach. We have carefully revised the relevant content. After re-verification, we confirm that Analytic Hierarchy Process (AHP), rather than the Delphi method, was used in this study. We have supplemented the full detailed procedures in Section 3.3 Survey Data Collection.

Specifically, we adopted the Analytic Hierarchy Process to calculate the weights for measuring the overall level of vocational college students’ learning outcomes. We invited 15 experts in vocational education with over five years of teaching or research experience to conduct pairwise comparisons among the three dimensions, namely knowledge mastery, skill application and competency development, using the Saaty 1–9 scale to evaluate their relative importance. The consistency ratio (CR) of all expert judgment matrices was below 0.1, indicating acceptable consistency. We integrated all judgment matrices via the geometric mean method and obtained the weight vector of each indicator through normalization. After rounding, the final weights for knowledge mastery, skill application and competency development were 0.2, 0.4 and 0.4 respectively.

Furthermore, we have corrected the erroneous reference to the "Delphi method" throughout the manuscript and added the specific criteria for the consistency test of judgment matrices. We hope these revisions fully address your concerns. We greatly appreciate your efforts in helping us improve the methodological rigor of this paper. This new section can be found on page 12, starting from paragraph 3, line 531-540. The revised parts are marked in red in the modified manuscript.

 

Comments 9: [The dimension of GenAI usage frequency confuses simple frequency with dependence and potential over-dependence. Things like “My learning efficiency would drop significantly without GenAI tools” are measuring dependence, not frequency. Perhaps this is part of the explanation for the negative regression coefficient.]

Response 9: We sincerely appreciate the reviewer’s perceptive observation. We fully agree that the items under the dimension of usage frequency involve minor conceptual confusion. We have accordingly proposed solutions for future research in this regard. As stated in Section 5, subsequent studies should separate usage frequency and dependence into two independent measurement dimensions. To be specific, usage frequency should focus on behavioral frequency, such as how many times the tools are used per day, while dependence should reflect cognitive and emotional reliance, for instance, whether students feel inconvenient without these tools. Such refined dimensional division can help more accurately examine their respective independent effects on learning outcomes.

Nevertheless, we would like to note that the negative coefficient of the usage frequency dimension derived from the regression results still has reference value for both theoretical and empirical research. It at least reveals that simply increasing usage frequency cannot improve learning outcomes and may even produce adverse effects. This finding also delivers meaningful implications for the practice of vocational education. We hope the above clarification fully addresses your concerns. This new section can be found on page 22, starting from paragraph 1, line 186-189. The revised parts are marked in red in the modified manuscript.

 

Comments 10: [The manuscript does not talk enough about academic integrity. Some items are using GenAI for homework, writing a thesis, writing code, and assignments so the paper should talk about plagiarism, unauthorized assistance, assessment validity, and AI policies of the institution.]

Response 10: We sincerely thank the reviewer for pointing out this important omission. We fully recognize that when generative artificial intelligence is used to assist students with assignments, thesis writing, coding and other tasks, it inevitably raises academic integrity issues including plagiarism, unauthorized assistance, assessment validity and institutional AI policies. The original manuscript indeed lacked adequate discussion on these topics.

We have added relevant content to the Future Research section in the revised paper. Specifically, we state at the end of Section 5 that this study has not fully explored academic integrity issues associated with the use of generative AI. Given that such tools can be applied to finish coursework, write papers, develop codes and complete various tasks, future research should delve into plagiarism, unauthorized help, assessment validity and relevant institutional AI policies. It is also necessary to further clarify the ethical boundaries and behavioral norms for students using generative AI. We hope these additions can fully address your concerns. This new section can be found on page 22, starting from paragraph 4, line 221-230. The revised parts are marked in red in the modified manuscript.

 

Comments 11: [The discussion is too optimistic and sometimes speculative. The authors jump too quickly from weak correlational evidence to sweeping recommendations for colleges, teachers, and students. While these recommendations may be reasonable, they are not well supported by the current empirical design.]

Response 11: We sincerely appreciate your impartial and constructive comments. We fully acknowledge that several parts of the original manuscript contained overinterpretation, and some universal recommendations were drawn directly from correlational data. Accordingly, we have systematically revised all suggestions throughout the paper to ensure that they are strictly grounded in our empirical findings and closely linked to the research results. The detailed revisions are presented as follows.

Based on the results, colleges, teachers and students need to work collaboratively with clear respective priorities to help vocational college students develop rational generative AI usage behaviors and improve their learning outcomes. For colleges, the focus should shift from simply encouraging AI use to standardizing usage scenarios and fostering good usage habits. This study reveals that generative AI usage contexts have a significantly positive correlation with learning outcomes. Therefore, rather than merely advocating the use of AI in vague terms, colleges should formulate clear behavioral guidelines for different learning scenarios. Relevant regulations can prevent the abuse of AI in inappropriate situations and maximize the positive effects of rational usage contexts on learning outcomes. Considering the notable differences in learning outcomes across various majors, colleges should avoid adopting a one-size-fits-all strategy for AI promotion. For knowledge-intensive majors, support can be prioritized for applying AI to knowledge integration and expansion. For skill-oriented majors, more attention should be paid to developing AI’s auxiliary functions in operational process simulation and case analysis. Meanwhile, major-specific prompt libraries and application templates can be established, and AI literacy can be appropriately integrated into the curriculum system (Hershkovitz et al., 2025).

For teachers, the emphasis needs to change from demonstrating tool functions to systematically guiding proper usage habits. Given the positive statistical correlation between usage habits and learning outcomes, vocational teachers, as learning facilitators, should help students understand the working mechanisms, strengths and limitations of generative AI, and cultivate their critical thinking and ethical awareness (Bower et al., 2024). Teachers can focus on key practices including prompt refinement, content verification, information screening, result comparison and tool selection. Combined with the practice-oriented features of vocational education, teachers can design targeted teaching activities based on real application scenarios of professional courses (Wang et al., 2024).

For students, it is essential to move from being able to use AI tools to using them proficiently. As direct users of generative AI, students should continuously improve their digital literacy. They need to recognize that simply increasing AI usage frequency may exert adverse impacts on learning outcomes, and refrain from cognitive offloading caused by over-reliance on generative AI for information retrieval and decision-making (Gerlich, 2025). In addition, students should enhance their ability to apply AI across diverse learning scenarios, and explore appropriate ways to utilize generative AI in knowledge acquisition, skill training, project practice and vocational simulation. Structured use of generative AI can ultimately help improve academic performance (Xu et al., 2025).

Thank you again for your valuable suggestions. We believe these revisions make the recommendations better aligned with our empirical evidence and avoid excessive speculation. This new section can be found on page 20, starting from paragraph 3, line 109-157. The revised parts are marked in red in the modified manuscript.

 

Comments 12: [The manuscript needs to be significantly shortened and reorganized. Some parts are too descriptive and the analytical contribution is somewhat limited.]

Response 12: We sincerely thank the reviewer for pointing out this structural issue. We have thoroughly revised the entire manuscript by streamlining descriptive content and adding in-depth theoretical analysis and elaboration. For instance, we have supplemented integrated explanations of the theoretical framework in Section 3.2.1 Development of the Generative AI Usage Behavior Scale. Specifically, this study systematically analyzes vocational college students’ generative AI usage behaviors by integrating Davis’s (1989) Technology Acceptance Model (TAM), Venkatesh et al.’s (2003) Unified Theory of Acceptance and Use of Technology (UTAUT), and existing assessment frameworks for AI literacy. This theoretical integration provides a more solid foundation for defining the three core dimensions, namely usage frequency, usage habits and usage contexts, and also strengthens the analytical contribution of the scale development.

With the above revisions, we have cut redundant descriptions and enhanced theoretical underpinnings and analytical depth throughout the paper. We hope these changes can optimize the overall structure, highlight the analytical value of this research, and fully address your concerns. This new section can be found on page 8, starting from paragraph 4, line 368-393. The revised parts are marked in red in the modified manuscript.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Dear Author/s;

Thank you for submitting. I was pleased and honoured to review your manuscript. Unlike some reviewers, I purely judge the paper on how it reads and what it says, rather than what I think it should be. This way, you are assured a fair review and any guidance I give will be to strengthen the paper, not my own opinions on how you could do something the way I would!!

Overall, I do agree this manuscript addresses a timely and important issue: the relationship between generative artificial intelligence usage and learning outcomes among higher vocational students. The focus on higher vocational education is a really clear strength, well done, as much of the existing literature on GenAI in education has concentrated on undergraduate or general higher education contexts. The paper usefully argues that higher vocational students differ from undergraduates in their learning characteristics, practical training needs, and employment-oriented learning outcomes. This gives the study a distinctive rationale and helps justify the development of context-specific measurement tools. I really enjoyed the conceptual ideas here.

The manuscript’s attempt to develop and validate two scales—the GenAI Usage Behavior Scale and the Higher Vocational Students’ Learning Outcomes Scale—is also valuable. The distinction between usage frequency, usage habits, and usage contexts is particularly promising, because it moves the discussion beyond whether students use GenAI and instead considers how they use it. The paper’s emphasis on usage habits, including prompt adjustment, verification of AI outputs, comparison across tools, and critical adoption of generated content, is especially relevant to current debates about responsible and effective GenAI use in education. As such, it has a strong foundation.

At the same time, respectfully, the manuscript requires revision before it is ready for publication. The central contribution is potentially strong, but the current version overstates what the design can demonstrate, underreports key methodological details, and needs clearer statistical interpretation. The paper would benefit from a more cautious framing, fuller transparency in scale validation, a better literature review, and more careful handling of the relationship between GenAI use and learning outcomes.

1. Develop further Clarity on Research Design and Causal Framing

A major developmental area concerns the manuscript’s repeated use of causal language. The paper frequently refers to the “impact” of GenAI usage behavior, “influencing mechanisms,” and the “effects” of usage frequency, habits, and contexts on learning outcomes. However, your study is based on cross-sectional questionnaire data. Although correlation and multiple regression analyses can identify statistical associations, they cannot really establish causal relationships or causal direction.

This issue is particularly important because the manuscript interprets regression coefficients as if one-unit increases in GenAI usage dimensions directly lead to increases or decreases in learning outcomes. This is not, in my opinion, warranted by the research design. For example, students with stronger learning outcomes may develop better GenAI usage habits, rather than better habits necessarily producing stronger learning outcomes. Similarly, students who struggle academically may use GenAI more frequently, which could partly explain the negative regression coefficient for usage frequency. There’s a lot of scope there.

This isn’t difficult to fix. The manuscript should therefore revise its language throughout. Terms such as “impact,” “effect,” and “influence” should be replaced with more cautious expressions such as “is associated with,” “predicts in the regression model,” or “is statistically related to.” The title, abstract, discussion, and conclusion should all be checked for causal overclaiming. If the authors wish to make causal claims, they would need a stronger design, such as a longitudinal study, intervention design, quasi-experimental approach, or the inclusion of stronger controls and causal modelling assumptions. I understand you want your work to be picked up and cited, but in essence it’s more likely to if you neutrally frame your conclusions, as people will take it more seriously.

2. Scale Development and Validation

The scale development work is really interesting, and in my view one of the most promising aspects of the paper, but it is currently underreported. The manuscript states that the two scales demonstrate good reliability and validity, and it reports high Cronbach’s alpha values, KMO values, and Bartlett test results. These are useful indicators, well done, but they are not sufficient on their own to establish strong scale validity. In my view, the paper may want to provide the full item wording for both scales, either in the main text or as an appendix. Readers need to see exactly how usage frequency, usage habits, usage contexts, knowledge mastery, skill application, and competency development were operationalized. The manuscript should also include factor loadings, cross-loadings, variance explained, item-total correlations, and the criteria used to remove items. At present, the reader is told that some items were removed because their factor loadings did not match theoretical expectations, but the evidence is not shown in enough detail. The avoidance of this seems a bit… odd, and might make the readers think something is not right. So, add greater clarity please?

A further issue concerns the pilot sample. The paper reports that a preliminary survey was conducted among 40 higher vocational students, I believe? However, it is not entirely clear whether the exploratory factor analysis was conducted on this small pilot sample or on the full formal sample? If factor analysis was performed on only 40 participants, the sample size is far too small for robust scale validation. If the full sample was used, the procedure should be clarified. Ideally, the authors should conduct exploratory factor analysis on one subsample and confirmatory factor analysis on another. This would provide much stronger evidence for the dimensional structure of the scales. In essence, explain your decision making in greater methodological detail.

3. Sampling, Representativeness, and Data Reporting

The manuscript benefits from a relatively large final sample of 977 valid responses. That’s really impressive. However, there are important reporting issues that need to be addressed. The methodology section, I believe, states that 1,000 questionnaires were distributed, while the data collection section, I think, later states that 1,329 questionnaires were collected and, confusingly, 977 valid responses were retained after excluding invalid responses. This inconsistency needs to be resolved. The authors should clarify whether 1,000 was the intended target, the minimum planned sample, or the number initially distributed. In essence, please ensure the numbers are right or double check. The paper should also provide a transparent sample flow. A short response-flow table would be useful, showing the number of questionnaires distributed or accessed, the number submitted, the number excluded, the exclusion criteria, and the final valid sample. This would improve the credibility of the data-cleaning process, and is an easy change to make. If you do not want to relabel all the tables, it’s also fine if you want to write this in a short paragraph of clarification instead – I do not want to add work to your paper.

Representativeness is another concern. The sample is heavily weighted toward first-year students, while third- and fourth-year students are underrepresented. This is fine and understandable if upper-year students were away on internships, or you just wanted to focus on the transition/starting point, but first, I might add some clarity around that – you wanted to capture them early on to understand the start points maybe. Likewise, it is a decision limits the generalizability of the findings. Your language needs to match- this is early on, vocational students for reason X and our conclusions, Y, are specific to this group. The distribution across major categories also appears uneven. Some categories seem to have very small sample sizes, which may affect the reliability of group comparisons. The authors should consider adding into the paper a report the number of students in each grade and major category and discuss the implications of unequal group sizes more explicitly.

4. Statistical Analysis and Interpretation

The statistical analysis is generally appropriate for an initial exploratory study, but several aspects require a bit more explanation. The correlation analysis shows positive relationships between GenAI usage dimensions and learning outcomes. However, the regression model shows that usage frequency becomes a negative predictor after usage habits and usage contexts are controlled. The manuscript interprets this as evidence that higher GenAI usage frequency negatively affects learning outcomes, but this interpretation is too strong.

A more cautious explanation is likely needed. The negative coefficient may indicate that frequent use without effective habits or meaningful contexts is associated with weaker outcomes. It may also reflect a suppression effect, since usage frequency is positively correlated with other GenAI usage dimensions. The authors could explore this possibility rather than presenting the result as straightforward evidence that frequent GenAI use is harmful – the data does not support that. Additional analyses could strengthen the paper, such as hierarchical regression, interaction effects between usage frequency and usage habits, nonlinear tests of frequency, or robustness checks using alternative scoring methods. However, if you do not want to do that (and I agree that its generally fine as is) then your conclusions need to be more modest here.

The differential analysis also needs refinement. The manuscript reports that grade-level differences are not statistically significant overall, but it discusses a near-significant result for skill application at p = .052 as if it were significant. This should be corrected, I believe? Unless the authors justify a different significance threshold in advance, p = .052 should not be treated as statistically significant, because it is a bit unclear why?! If post hoc comparisons are retained, the authors should explain why they are appropriate after a non-significant omnibus test. That would be a way forward, or simply clarify to a greater extent.

The major-category comparisons also require more careful treatment. Because some groups appear to have very small sample sizes, standard ANOVA assumptions may not hold. The authors should test assumptions, report effect sizes, and consider using Welch ANOVA or combining very small categories where appropriate.

5. Measurement of Learning Outcomes

The paper’s conceptualization of learning outcomes is really well aligned with vocational education, especially the inclusion of knowledge mastery, skill application, and competency development. However, the measurement appears to rely entirely on students’ self-reported perceptions. This limits the strength of the conclusions. Self-reported learning outcomes may reflect confidence, motivation, or social desirability rather than actual achievement. This limitation should be made more prominent, and indeed a more detailed limitations section is needed overall, towards the end of the paper. The manuscript should refer to “perceived learning outcomes” unless objective indicators are added. To strengthen the study, the authors could incorporate course grades, practical training scores, teacher assessments, internship evaluations, professional skill tests, or certification outcomes. Even if such data cannot be included in the present study, the limitation should be clearly acknowledged and linked to future research directions.

The weighted composite score for learning outcomes also needs fuller justification. The manuscript states that the composite score uses Delphi-derived weights, with knowledge mastery weighted at 0.2 and skill application and competency development each weighted at 0.4. However, the Delphi process is not described in sufficient detail. The authors could instead report the number of experts, number of rounds, consensus criteria, and rationale for the final weights. They should also test whether the main findings remain stable under equal weighting.

6. Review, Writing, Structure, and Presentation

The manuscript is generally well structured, but it requires careful language editing. There are recurring grammatical errors, spacing issues, awkward phrases, and occasional duplicated words. Examples include subject-verb agreement problems, punctuation errors, and phrases with multiple works in it (See: “This study.”).

These issues do not undermine the value of the research, but they do affect readability and should be addressed before publication. The abstract should be revised after the main methodological and interpretive changes are made. It currently presents the findings clearly but uses causal wording that is stronger than the design supports. The discussion section should also be tightened. Some claims are repeated, and the practical recommendations would be stronger if they were more directly linked to the study’s actual findings. The tables are useful, but the presentation should be improved. The authors should report exact p-values where appropriate, replace “p = 0.000” with “p < .001,” include effect sizes, and ensure that all table labels are clear and consistently formatted. Formula formatting should also be corrected, particularly in the regression section.

I’m also not convinced the Literature Review works as is, and I can agree that most of the statistical elements need just refining to be clearer to the reader, but the review is in my opinion too generic.

Presently, you defer to what appears to be the most common papers I’ve seen around many submissions, and often I believe to be the ones that AI itself has indexed and recommending, based on the years and clustering. Several DOIs do not work, for example the paper by Kim. What I found less helpful in reading your review is that it is in particular is lacking is a grounding in the literature landscape of Chinese HE/FE/student/university engagement with AI, which seems directly relevant to your Chinese vocational/FE angle of discussion.

Consistent with our journal policy, I do not recommend specific papers or named authors. As a researcher in the field, however, I know there’s a lot of really interesting stuff coming out of Chinese HE on AI at the moment, and I would have thought it would be logical to incorporate a dedicated subsection to move from the generic AI landscape stuff you write about, to next Chinese HE research on students/faculty and AI, and then to AI on vocational learning within the context of China, in a structured way? Otherwise the reader immediately thinks “why is this so generic” if they are read on research in these China specific areas.

What seems very popular at the moment is far ranging, but papers of late include topics as interesting as Chinese student perspectives on AI and guanxi in HE, ChatGPT in learning design for Chinese learners, utilising generative AI for Chinese student IELTS and EAL learning, Chinese learners and TAM, AI socio-emotional learning amongst Chinese students, determining factors of Chinese faculty, teachers and educators readiness to adopt generative AI, TPACK and AI in Chinese universities, Chinese student academic integrity and AI. Any/all/none of these could be considered, or an entirely specific vocational AI strand, though I am less sure of the areas of current publishing, you may want to expand further.

These are topics that may offer some direction to structure a subsection about Chinese learners more specifically around, as without this it looks like you have not reviewed the landscape, rather reviewed the topic in more general terms. I might therefore take a few days to add in a few paragraphs on the landscape of Gen AI research in China specifically, expanding in more detail, perhaps with the structure of general>HE China>HE vocational.

7. Conclusion and Recommendation

Overall, this is a promising manuscript with a relevant topic, a meaningful vocational-education focus, and a potentially useful measurement framework. Its main contribution lies in showing that the quality and context of GenAI use may matter more than simple frequency of use. This is an important message for educators and institutions seeking to integrate GenAI into vocational learning environments.

However, the paper requires major revision. The authors should substantially revise the causal framing, clarify the sample reporting inconsistency, provide fuller scale validation evidence, treat self-reported learning outcomes more cautiously, and strengthen the statistical interpretation. With these improvements, the manuscript could make a valuable contribution to research on GenAI use in vocational education.

 

Comments on the Quality of English Language

Please proof and read the discussion and narrative points further.

Author Response

Comments 1: [Thank you for submitting. I was pleased and honoured to review your manuscript. Unlike some reviewers, I purely judge the paper on how it reads and what it says, rather than what I think it should be. This way, you are assured a fair review and any guidance I give will be to strengthen the paper, not my own opinions on how you could do something the way I would!!

Overall, I do agree this manuscript addresses a timely and important issue: the relationship between generative artificial intelligence usage and learning outcomes among higher vocational students. The focus on higher vocational education is a really clear strength, well done, as much of the existing literature on GenAI in education has concentrated on undergraduate or general higher education contexts. The paper usefully argues that higher vocational students differ from undergraduates in their learning characteristics, practical training needs, and employment-oriented learning outcomes. This gives the study a distinctive rationale and helps justify the development of context-specific measurement tools. I really enjoyed the conceptual ideas here.

The manuscript’s attempt to develop and validate two scales—the GenAI Usage Behavior Scale and the Higher Vocational Students’ Learning Outcomes Scale—is also valuable. The distinction between usage frequency, usage habits, and usage contexts is particularly promising, because it moves the discussion beyond whether students use GenAI and instead considers how they use it. The paper’s emphasis on usage habits, including prompt adjustment, verification of AI outputs, comparison across tools, and critical adoption of generated content, is especially relevant to current debates about responsible and effective GenAI use in education. As such, it has a strong foundation.

At the same time, respectfully, the manuscript requires revision before it is ready for publication. The central contribution is potentially strong, but the current version overstates what the design can demonstrate, underreports key methodological details, and needs clearer statistical interpretation. The paper would benefit from a more cautious framing, fuller transparency in scale validation, a better literature review, and more careful handling of the relationship between GenAI use and learning outcomes.]

Response 1: We sincerely appreciate the reviewer’s kind recognition and encouragement regarding our work. Thank you for acknowledging that this study focuses on vocational education, a field that has received relatively little attention in research on generative AI in education, and for affirming its practical significance. The emphasis on the "usage habits" dimension of generative AI usage behavior is also recognized as highly relevant to current discussions on the responsible and effective application of AI. These positive comments confirm that our research is built upon a promising and meaningful foundation.

Meanwhile, we fully take on board your important concerns covering four aspects: insufficient evidence to support certain arguments, incomplete reporting of methodological details for scale validation, lack of clarity and prudence in the interpretation of statistical results, and inadequate coverage of local contexts in the literature review. We have taken all these comments seriously and carried out substantial revisions to address every issue raised. We have strived to bring the revised manuscript up to the journal’s publication standards in terms of theoretical construction, methodological rigor and the clarity of statistical interpretation. We welcome any further feedback. This new section can be found on page 8, starting from paragraph 4, line 368-393; page 10, starting from paragraph 2, line 439-447;page 18, starting from paragraph 3, line 16-38;page 22, starting from paragraph 2, line 179-239.The revised parts are marked in red in the modified manuscript.

 

Comments 2: [Develop further Clarity on Research Design and Causal Framing

A major developmental area concerns the manuscript’s repeated use of causal language. The paper frequently refers to the “impact” of GenAI usage behavior, “influencing mechanisms,” and the “effects” of usage frequency, habits, and contexts on learning outcomes. However, your study is based on cross-sectional questionnaire data. Although correlation and multiple regression analyses can identify statistical associations, they cannot really establish causal relationships or causal direction.

This issue is particularly important because the manuscript interprets regression coefficients as if one-unit increases in GenAI usage dimensions directly lead to increases or decreases in learning outcomes. This is not, in my opinion, warranted by the research design. For example, students with stronger learning outcomes may develop better GenAI usage habits, rather than better habits necessarily producing stronger learning outcomes. Similarly, students who struggle academically may use GenAI more frequently, which could partly explain the negative regression coefficient for usage frequency. There’s a lot of scope there.

This isn’t difficult to fix. The manuscript should therefore revise its language throughout. Terms such as “impact,” “effect,” and “influence” should be replaced with more cautious expressions such as “is associated with,” “predicts in the regression model,” or “is statistically related to.” The title, abstract, discussion, and conclusion should all be checked for causal overclaiming. If the authors wish to make causal claims, they would need a stronger design, such as a longitudinal study, intervention design, quasi-experimental approach, or the inclusion of stronger controls and causal modelling assumptions. I understand you want your work to be picked up and cited, but in essence it’s more likely to if you neutrally frame your conclusions, as people will take it more seriously.]

Response 2: We sincerely thank the reviewer for pointing out this critical issue. We fully agree that a research design based on cross-sectional questionnaire data cannot establish causal relationships. The frequent use of causal terms such as impact, effect and influencing mechanism in the original manuscript indeed goes beyond what this study can substantiate. We have systematically revised the entire paper and replaced causal wording with more prudent expressions that describe statistical associations.

The detailed revisions are as follows. First, in the abstract, the original statement "The usage frequency of generative artificial intelligence exerts a significantly negative impact on learning outcomes, while usage habits and usage contexts show markedly positive effects, with usage habits demonstrating stronger explanatory power" has been revised to "Generative AI usage frequency has a significant negative statistical correlation with learning outcomes, whereas usage habits and usage contexts are significantly and positively correlated with learning outcomes, and usage habits present the strongest explanatory power".

Second, in the regression analysis section, the original text reads: "With other variables held constant, each one-unit increase in generative AI usage frequency corresponds to a 0.137-unit decrease in learning outcomes; each one-unit increase in usage habits leads to a 0.248-unit rise in learning outcomes; and each one-unit increase in usage contexts brings about a 0.205-unit growth in learning outcomes. In terms of the direction of effects, generative AI usage frequency has a significant negative effect on vocational college students’ learning outcomes, while usage habits and usage contexts both exert significant positive effects. In terms of effect magnitude, usage habits have the greatest influence on learning outcomes, followed by usage contexts, and usage frequency has the weakest influence."

This passage has been revised to: "In terms of correlation direction, generative AI usage frequency is significantly and negatively correlated with vocational college students’ learning outcomes, which is contrary to the positive prediction proposed in Hypothesis H1. Therefore, Hypothesis H1 is not supported. Nevertheless, this negative coefficient does not mean that usage frequency itself produces adverse effects on learning outcomes. Among vocational college students, high usage frequency is often accompanied by aimless use without clear learning objectives, over-reliance on AI-generated content, and the lack of active processing and in-depth thinking. These unfavorable habits fail to translate into desirable learning results, thus leading to a negative statistical correlation. By contrast, both usage habits and usage contexts serve as significant positive predictors of learning outcomes, which supports Hypotheses H2 and H3. The results indicate that in generative AI-assisted learning, how and in what contexts the tools are used are positively correlated with learning outcomes. In terms of correlation strength, usage habits show the strongest statistical association with learning outcomes, followed by usage contexts and then usage frequency. This finding is consistent with Hypothesis H2 which assumes that usage habits have the strongest positive predictive effect, so Hypothesis H4 is supported."

Third, in the discussion section, the original sentence "Finally, this study verifies the impacts of different dimensions of generative AI usage behavior on vocational college students’ learning outcomes. Specifically, usage frequency has a significant negative impact on learning outcomes, while usage habits and usage contexts both show significant positive impacts" has been revised to "Finally, this study examines the regression predictive effects of different dimensions of generative AI usage behavior on vocational college students’ learning outcomes. In the regression model, generative AI usage frequency acts as a significant negative predictor of learning outcomes, while usage habits and usage contexts are significant positive predictors." We have also replaced strongly causal words like impact with neutral phrases such as predicts in the regression model and is statistically correlated with.

We hope these comprehensive revisions fully address your concerns regarding causal language and make the overall writing more rigorous and prudent. We fully understand that presenting conclusions in a neutral and accurate manner is fundamental for the research to be taken seriously and widely cited. Thank you again for your valuable suggestions. This new section can be found on page 1, starting from paragraph 1, line 26-32; page 18, starting from paragraph 3, line 16-38;page 20, starting from paragraph 2, line 92-98.The revised parts are marked in red in the modified manuscript.

 

Comments 3: [Scale Development and Validation

The scale development work is really interesting, and in my view one of the most promising aspects of the paper, but it is currently underreported. The manuscript states that the two scales demonstrate good reliability and validity, and it reports high Cronbach’s alpha values, KMO values, and Bartlett test results. These are useful indicators, well done, but they are not sufficient on their own to establish strong scale validity. In my view, the paper may want to provide the full item wording for both scales, either in the main text or as an appendix. Readers need to see exactly how usage frequency, usage habits, usage contexts, knowledge mastery, skill application, and competency development were operationalized. The manuscript should also include factor loadings, cross-loadings, variance explained, item-total correlations, and the criteria used to remove items. At present, the reader is told that some items were removed because their factor loadings did not match theoretical expectations, but the evidence is not shown in enough detail. The avoidance of this seems a bit… odd, and might make the readers think something is not right. So, add greater clarity please?

A further issue concerns the pilot sample. The paper reports that a preliminary survey was conducted among 40 higher vocational students, I believe? However, it is not entirely clear whether the exploratory factor analysis was conducted on this small pilot sample or on the full formal sample? If factor analysis was performed on only 40 participants, the sample size is far too small for robust scale validation. If the full sample was used, the procedure should be clarified. Ideally, the authors should conduct exploratory factor analysis on one subsample and confirmatory factor analysis on another. This would provide much stronger evidence for the dimensional structure of the scales. In essence, explain your decision making in greater methodological detail.]

Response 3: We sincerely appreciate the reviewer’s recognition and encouragement regarding our scale development work. We are greatly pleased to learn that the Generative AI Usage Behavior Scale and the Vocational College Students’ Learning Outcomes Scale are regarded as one of the most promising core contributions of this paper. Such positive feedback confirms that our research has laid a solid foundation for the development of measurement tools. Meanwhile, we fully acknowledge your concerns about the completeness of the scale validation reporting and have made substantial revisions to address all the issues raised.

First, we have supplemented the reporting of statistical indicators including factor loadings and explained the reasons for item deletion in detail. We agree that merely presenting Cronbach’s α coefficients, KMO values and Bartlett’s test results is insufficient to fully validate the scale. Therefore, we have added the following content in Section 3.2.3 of the revised manuscript:

According to the rotated component matrix, Item C1 has factor loadings of 0.581 on Factor 1 and 0.528 on Factor 3, with a difference of only 0.053, which fails to meet the basic criteria for discriminant validity. In addition, the loading of C1 on its theoretically assigned Factor 3 is not significantly higher than that on Factor 1, and both loadings exceed the acceptable threshold of 0.5, indicating no distinct primary factor for this item. This is inconsistent with other items that show high loadings on a single factor, which impairs the parsimony and discriminant validity of the factor structure. Hence, Item C1 was removed, and factor analysis was reperformed on the remaining items to verify the stability of the revised structure. Ultimately, three factors were extracted. Factor 1 consists of Items B1 to B5 with factor loadings ranging from 0.799 to 0.837; Factor 2 comprises Items A1 to A5 with factor loadings between 0.658 and 0.862; Factor 3 includes Items C2 to C6 with factor loadings from 0.526 to 0.798.

For the Vocational College Students’ Learning Outcomes Scale, the KMO value is 0.961 and the p-value of Bartlett’s Test of Sphericity is less than 0.001, indicating that the data is suitable for factor analysis. The rotated component matrix shows that Item I1 has a loading of 0.496 on its theoretically assigned Factor 1, below the acceptable threshold of 0.50, and a loading of 0.595 on Factor 2. The difference between the two values is merely 0.099, which does not satisfy the requirements for discriminant validity and contradicts the theoretical classification. Accordingly, Item I1 was deleted. Three factors were finally retained. Factor 1 contains Items I2 to I6 with factor loadings ranging from 0.598 to 0.792; Factor 2 is composed of Items H1 to H3 with factor loadings between 0.750 and 0.811; Factor 3 includes Items J1 to J3 with factor loadings from 0.717 to 0.792.*

Second, we clarify and correct the information about the pilot sample size. We sincerely apologize for a critical typographical error in the original manuscript. The statement that "a small-scale pilot survey was conducted among 40 students from a vocational college" was incorrect. In fact, we distributed 400 questionnaires for the pilot survey rather than 40. This mistake has been fully corrected in the revised version.

To specify, a total of 400 questionnaires were distributed in the pilot phase, and 329 valid responses were retrieved. We conducted item analysis, exploratory factor analysis and reliability tests based on these 329 valid samples. For the formal survey, we collected another 977 independent samples for confirmatory factor analysis and subsequent hypothesis testing. Separating the pilot sample from the formal sample effectively avoids circular reasoning caused by conducting both exploratory and confirmatory analysis on the same dataset. Relevant explanations have been added in Section 3.2.3.

We believe these supplementary reports and clarifications fully address your concerns and enhance the transparency of the entire scale development process. Thank you for helping us improve the methodological rigor of this paper. This new section can be found on page 11, starting from paragraph 2, line 475-493;page 7, starting from paragraph 1, line 302-308; page 10, starting from paragraph 7, line 453-454.The revised parts are marked in red in the modified manuscript.

 

Comments 4: [Sampling, Representativeness, and Data Reporting

The manuscript benefits from a relatively large final sample of 977 valid responses. That’s really impressive. However, there are important reporting issues that need to be addressed. The methodology section, I believe, states that 1,000 questionnaires were distributed, while the data collection section, I think, later states that 1,329 questionnaires were collected and, confusingly, 977 valid responses were retained after excluding invalid responses. This inconsistency needs to be resolved. The authors should clarify whether 1,000 was the intended target, the minimum planned sample, or the number initially distributed. In essence, please ensure the numbers are right or double check. The paper should also provide a transparent sample flow. A short response-flow table would be useful, showing the number of questionnaires distributed or accessed, the number submitted, the number excluded, the exclusion criteria, and the final valid sample. This would improve the credibility of the data-cleaning process, and is an easy change to make. If you do not want to relabel all the tables, it’s also fine if you want to write this in a short paragraph of clarification instead – I do not want to add work to your paper.

Representativeness is another concern. The sample is heavily weighted toward first-year students, while third- and fourth-year students are underrepresented. This is fine and understandable if upper-year students were away on internships, or you just wanted to focus on the transition/starting point, but first, I might add some clarity around that – you wanted to capture them early on to understand the start points maybe. Likewise, it is a decision limits the generalizability of the findings. Your language needs to match- this is early on, vocational students for reason X and our conclusions, Y, are specific to this group. The distribution across major categories also appears uneven. Some categories seem to have very small sample sizes, which may affect the reliability of group comparisons. The authors should consider adding into the paper a report the number of students in each grade and major category and discuss the implications of unequal group sizes more explicitly.]

Response 4: We sincerely thank the reviewer for acknowledging the size of our sample. We also fully recognize the prominent issues regarding the transparency of sample reporting and sample representativeness.

First, we clarify the inconsistency in the reported sample figures. We have clearly distinguished and elaborated on the procedures of the pilot survey and formal survey in Section 3.3 Survey Data Collection of the revised manuscript. In the pilot phase, we distributed 400 questionnaires at a vocational college and obtained 329 valid responses, which were used for item analysis and exploratory factor analysis. For the formal survey, we collected data from a new independent sample. A total of 1,100 questionnaires were distributed and 1,000 were retrieved. After excluding responses with uniform answers, we finally obtained 977 valid questionnaires.

Second, we address the concerns about sample representativeness. We agree that the overrepresentation of first-year students and the severe underrepresentation of senior students have limited the external validity of the research findings. We have taken two measures in the revised manuscript. On the one hand, we present objective results and deliver prudent interpretations in the results section. Section 4.2 illustrates the sample distribution across different grades, and we explicitly state that the sample mainly reflects the learning status of first-year students. Thus, cautions should be exercised when generalizing the conclusions to senior students. We have also added a note in this section: In addition, several major categories including Resource, Environment and Safety, Public Security and Judicial, and Public Administration and Service have very small sample sizes (n < 5) with a standard deviation of 0.00. The results of inter-group comparisons should therefore be interpreted with caution for readers’ reference.

On the other hand, we explicitly acknowledge this limitation in the section of Limitations and Future Research in Section 5. We point out the large disparity in sample size across grades, with a surplus of junior students and a shortage of senior ones, which may undermine the stability and generalizability of the findings. Besides, the small sample sizes of certain major categories may weaken the statistical power of inter-group comparisons. We suggest that future studies recruit participants from vocational colleges across diverse regions, tiers and institutional types, and strive to balance the proportion of students in each grade and major category. Increasing the number of senior students and participants from underrepresented majors will help improve sample representativeness and the external validity of conclusions. We also explain the practical difficulties in recruiting senior participants: most third- and fourth-year students are engaged in off-campus internships, making it far more challenging to distribute and collect online questionnaires compared with junior students who attend regular in-person courses on campus. This objective constraint is fully stated in the revised manuscript for readers’ careful interpretation.

All the above revisions are included in Sections 3.3, 4.2 and 5. We hope these clarifications and supplements fully address your concerns. This new section can be found on page 12, starting from paragraph 4, line 543-546;page 14, starting from paragraph 4, line 633-636; page 22, starting from paragraph 3, line 209-220.The revised parts are marked in red in the modified manuscript.

 

Comments 5: [Statistical Analysis and Interpretation

The statistical analysis is generally appropriate for an initial exploratory study, but several aspects require a bit more explanation. The correlation analysis shows positive relationships between GenAI usage dimensions and learning outcomes. However, the regression model shows that usage frequency becomes a negative predictor after usage habits and usage contexts are controlled. The manuscript interprets this as evidence that higher GenAI usage frequency negatively affects learning outcomes, but this interpretation is too strong.

A more cautious explanation is likely needed. The negative coefficient may indicate that frequent use without effective habits or meaningful contexts is associated with weaker outcomes. It may also reflect a suppression effect, since usage frequency is positively correlated with other GenAI usage dimensions. The authors could explore this possibility rather than presenting the result as straightforward evidence that frequent GenAI use is harmful – the data does not support that. Additional analyses could strengthen the paper, such as hierarchical regression, interaction effects between usage frequency and usage habits, nonlinear tests of frequency, or robustness checks using alternative scoring methods. However, if you do not want to do that (and I agree that its generally fine as is) then your conclusions need to be more modest here.

The differential analysis also needs refinement. The manuscript reports that grade-level differences are not statistically significant overall, but it discusses a near-significant result for skill application at p = .052 as if it were significant. This should be corrected, I believe? Unless the authors justify a different significance threshold in advance, p = .052 should not be treated as statistically significant, because it is a bit unclear why?! If post hoc comparisons are retained, the authors should explain why they are appropriate after a non-significant omnibus test. That would be a way forward, or simply clarify to a greater extent.

The major-category comparisons also require more careful treatment. Because some groups appear to have very small sample sizes, standard ANOVA assumptions may not hold. The authors should test assumptions, report effect sizes, and consider using Welch ANOVA or combining very small categories where appropriate.]

Response 5: We sincerely thank the reviewer for the detailed suggestions on the interpretation of statistical analyses. We fully agree that the explanations of regression coefficients and difference analysis results in the original manuscript were overly definitive and lacked due prudence.

Firstly, we have revised the interpretation of regression coefficients. As you pointed out, the negative coefficient of usage frequency after controlling for usage habits and usage contexts does not indicate that usage frequency itself exerts adverse effects on learning outcomes. We share this view and have thoroughly revised the relevant content. In Section 4.3.2, we replaced the original statement "usage frequency has a significant negative predictive effect on learning outcomes" with a more cautious explanation: "Nevertheless, this negative coefficient does not mean that usage frequency itself harms learning outcomes. Among vocational college students, high usage frequency is usually accompanied by unregulated use without clear learning objectives, over-reliance on AI-generated content, and the absence of active processing and in-depth thinking. These undesirable learning habits fail to produce positive learning results, thereby leading to a negative statistical correlation."

Secondly, we have corrected the presentation of the p-value in difference analysis. We acknowledge that a p-value of 0.052 should not be regarded as marginally significant. In Section 4.2, we revised the original phrase "The difference across grades is marginally significant (p=0.052)" to "The differences across grades are not statistically significant (p > 0.05)". Meanwhile, we have removed the post-hoc LSD comparisons and the corresponding claim that "First-year students score significantly higher than second-year students in skill application", so as to avoid overinterpreting non-significant findings.

Thirdly, we addressed the issues concerning small sample sizes in the comparisons across major categories. Given that these majors are conceptually distinct and cannot be combined, we retain the original classification covering 19 major categories. We added a note in Section 4.2: "In addition, several major categories including Resource, Environment and Safety, Public Security and Judicial, and Public Administration and Service have very small sample sizes (n < 5) with a standard deviation of 0.00. The results of inter-group comparisons should therefore be interpreted with caution", allowing readers to make reasonable judgments. Furthermore, in the limitations section (Section 5), we noted that the extremely small samples (n < 5) of the above-mentioned majors may violate the assumptions of standard ANOVA and undermine the stability of inter-group comparisons. We suggest that future studies collect more data from these majors to improve the reliability of research conclusions.

We hope these revisions fully address your concerns and enable more prudent and transparent interpretation of the statistical results. This new section can be found on page 18, starting from paragraph 3, line 16-38;page 14, starting from paragraph 3, line 617-621; page 14, starting from paragraph 4, line 633-636.The revised parts are marked in red in the modified manuscript.

 

Comments 6: [Measurement of Learning Outcomes

The paper’s conceptualization of learning outcomes is really well aligned with vocational education, especially the inclusion of knowledge mastery, skill application, and competency development. However, the measurement appears to rely entirely on students’ self-reported perceptions. This limits the strength of the conclusions. Self-reported learning outcomes may reflect confidence, motivation, or social desirability rather than actual achievement. This limitation should be made more prominent, and indeed a more detailed limitations section is needed overall, towards the end of the paper. The manuscript should refer to “perceived learning outcomes” unless objective indicators are added. To strengthen the study, the authors could incorporate course grades, practical training scores, teacher assessments, internship evaluations, professional skill tests, or certification outcomes. Even if such data cannot be included in the present study, the limitation should be clearly acknowledged and linked to future research directions.

The weighted composite score for learning outcomes also needs fuller justification. The manuscript states that the composite score uses Delphi-derived weights, with knowledge mastery weighted at 0.2 and skill application and competency development each weighted at 0.4. However, the Delphi process is not described in sufficient detail. The authors could instead report the number of experts, number of rounds, consensus criteria, and rationale for the final weights. They should also test whether the main findings remain stable under equal weighting.]

Response 6: We sincerely thank the reviewer for recognizing that our conceptualization of learning outcomes aligns well with the characteristics of vocational education. We also fully acknowledge the deficiencies regarding measurement approaches, weighting methods and the explanation of research limitations.

First, we address the limitations and improvements related to the measurement of learning outcomes. We agree that this study entirely adopts student self-report questionnaires to measure learning outcomes. Such measurements tend to reflect students’ self-confidence, learning motivation and social desirability bias rather than their actual knowledge mastery and skill improvement. Accordingly, we have revised the term "learning outcomes" to perceived learning outcomes in the abstract to accurately define the nature of the variable. We have also added detailed discussions in Section 5 Limitations and Future Research:

This study mainly relies on self-report questionnaires to measure learning outcomes, which is inherently subjective. The measured results may largely reflect students’ confidence, motivation and social desirability instead of their actual knowledge and skills. Future research shall update the constituent elements of generative AI usage behaviors in line with the latest developments and develop more systematic, scientific and context-adaptive assessment tools. Meanwhile, to measure students’ learning outcomes more accurately, follow-up studies should adopt multi-source data and objective evaluation indicators. On the basis of student self-reports, researchers may further incorporate course grades, practical training scores, teacher evaluations, internship assessments, professional skill tests and certification results, so as to effectively reduce subjective bias caused by single-source self-reported data.

Second, we clarify and supplement details about the weighting method. The reviewer pointed out that the original manuscript contained vague descriptions and an incorrect name for the weighting approach, where the Analytic Hierarchy Process was mistakenly referred to as the Delphi method. We have thoroughly revised this part. After re-verification, we confirm that the Analytic Hierarchy Process (AHP) was applied in this research. We have supplemented the complete operating procedures in Section 3.3 Survey Data Collection:

We adopted the Analytic Hierarchy Process to calculate indicator weights for the overall level of vocational college students’ learning outcomes. Fifteen experts in vocational education with over five years of teaching or research experience were invited to conduct pairwise comparisons among three dimensions including knowledge mastery, skill application and competency development using the Saaty 1–9 scale to assess their relative importance. The consistency ratio (CR) of all expert judgment matrices was below 0.1, indicating acceptable consistency. We integrated all judgment matrices via the geometric mean method and obtained the weight vector of each indicator through normalization. After rounding, the final weights for knowledge mastery, skill application and competency development were 0.2, 0.4 and 0.4 respectively.

Third, we conducted a robustness test with equal weights. To examine whether the regression results are sensitive to different weight settings, we performed regression analysis under the equal-weight scheme, where knowledge mastery, skill application and competency development each account for one-third of the total score. The results, as presented in Table 1, are largely consistent with those derived from the original weighted scheme with no substantial changes. Usage frequency still shows a negative correlation, while usage habits and usage contexts remain positively correlated. There are no shifts in the direction or significance level of coefficients, which proves that the core findings of this study are robust against different weight assignments.

We hope these comprehensive revisions fully address your concerns and enhance the transparency and rigor of the procedures for measuring learning outcomes and determining indicator weights. This new section can be found on page 1, starting from paragraph 1, line 26-32; page 22, starting from paragraph 1, line 179-198;page 12, starting from paragraph 3, line 531-540.The revised parts are marked in red in the modified manuscript.

 

Table 1. Regression Analysis Results of Higher Vocational Students’ GenAI Usage Behavior on Learning Outcomes.(Equal weights)

Model

Unstandardized Coefficients

Standardized Coefficients

t

Sig.

Collinearity

Statistics

B

Std. Error

Beta

Tolerance

VIF

Independent Variable

(Constant)

2.728

0.095

 

28.803

0

 

 

GenAI Usage Frequency

-0.132

0.025

-0.198

-5.309

0

0.588

1.702

GenAI Usage Habits

0.247

0.029

0.320

8.482

0

0.573

1.744

GenAI Usage Contexts

0.207

0.031

0.285

6.770

0

0.462

2.165

 

R2

0.205

 

Adjusted R2

0.203

 

F

83.715

 

P

<0.001

Dependent Variable: Learning Outcomes(Equal weights)

 

Comments 7: [Review, Writing, Structure, and Presentation

The manuscript is generally well structured, but it requires careful language editing. There are recurring grammatical errors, spacing issues, awkward phrases, and occasional duplicated words. Examples include subject-verb agreement problems, punctuation errors, and phrases with multiple works in it (See: “This study.”).

These issues do not undermine the value of the research, but they do affect readability and should be addressed before publication. The abstract should be revised after the main methodological and interpretive changes are made. It currently presents the findings clearly but uses causal wording that is stronger than the design supports. The discussion section should also be tightened. Some claims are repeated, and the practical recommendations would be stronger if they were more directly linked to the study’s actual findings. The tables are useful, but the presentation should be improved. The authors should report exact p-values where appropriate, replace “p = 0.000” with “p < .001,” include effect sizes, and ensure that all table labels are clear and consistently formatted. Formula formatting should also be corrected, particularly in the regression section.

I’m also not convinced the Literature Review works as is, and I can agree that most of the statistical elements need just refining to be clearer to the reader, but the review is in my opinion too generic.

Presently, you defer to what appears to be the most common papers I’ve seen around many submissions, and often I believe to be the ones that AI itself has indexed and recommending, based on the years and clustering. Several DOIs do not work, for example the paper by Kim. What I found less helpful in reading your review is that it is in particular is lacking is a grounding in the literature landscape of Chinese HE/FE/student/university engagement with AI, which seems directly relevant to your Chinese vocational/FE angle of discussion.

Consistent with our journal policy, I do not recommend specific papers or named authors. As a researcher in the field, however, I know there’s a lot of really interesting stuff coming out of Chinese HE on AI at the moment, and I would have thought it would be logical to incorporate a dedicated subsection to move from the generic AI landscape stuff you write about, to next Chinese HE research on students/faculty and AI, and then to AI on vocational learning within the context of China, in a structured way? Otherwise the reader immediately thinks “why is this so generic” if they are read on research in these China specific areas.

What seems very popular at the moment is far ranging, but papers of late include topics as interesting as Chinese student perspectives on AI and guanxi in HE, ChatGPT in learning design for Chinese learners, utilising generative AI for Chinese student IELTS and EAL learning, Chinese learners and TAM, AI socio-emotional learning amongst Chinese students, determining factors of Chinese faculty, teachers and educators readiness to adopt generative AI, TPACK and AI in Chinese universities, Chinese student academic integrity and AI. Any/all/none of these could be considered, or an entirely specific vocational AI strand, though I am less sure of the areas of current publishing, you may want to expand further.

These are topics that may offer some direction to structure a subsection about Chinese learners more specifically around, as without this it looks like you have not reviewed the landscape, rather reviewed the topic in more general terms. I might therefore take a few days to add in a few paragraphs on the landscape of Gen AI research in China specifically, expanding in more detail, perhaps with the structure of general>HE China>HE vocational.]

Response 7: We sincerely thank the reviewer for the detailed suggestions on language refinement, standardization of statistical reporting and localization of the literature review. We fully recognize the existing shortcomings in these aspects and have carried out systematic revisions accordingly.

First, regarding language polishing and formatting standardization. We have gone through the entire manuscript and thoroughly corrected recurring grammatical errors, subject-verb disagreement, punctuation mistakes, redundant spaces and duplicated words. In addition, we have invited peers with rich experience in academic English writing to polish the text, so as to ensure fluent and standardized expression.

Second, in terms of the formatting of statistical results. All instances of p = 0.000 in tables and the main text have been uniformly revised to p < 0.001. We have also fixed formatting errors in the formulas of regression equations to guarantee that mathematical symbols and corresponding variables are clear and legible.

Third, the structural optimization of the literature review. As pointed out, the original literature review was too generalized and lacked a systematic overview of AI research within China’s higher education and vocational education sectors. We fully accept this comment and have added content focusing on the local context of China in Section 2, which reads as follows:

Against the backdrop of China’s higher vocational education, the rapid advancement of generative artificial intelligence has drawn growing academic attention to its applications in vocational colleges. Researchers have begun to systematically explore the current status and influencing factors of vocational college students’ AI literacy. Existing studies reveal that although Chinese vocational students generally hold positive attitudes towards generative AI, their practical capabilities vary greatly. In particular, they show deficiencies in key aspects such as prompt design, result evaluation and ethical judgment (Song et al., 2026). Some studies also indicate that vocational students tend to use generative AI in a superficial manner, regarding it merely as a tool to finish specific tasks while lacking systematic and critical awareness of its use (Zou et al., 2025). Overall, research on AI literacy in China’s vocational education is developing rapidly, yet there are no authoritative standard measurement tools for assessing generative AI usage behaviors. Therefore, it is urgent to establish analytical frameworks and measurement instruments tailored to the local settings of China’s vocational and higher education, so as to meet the new requirements for talent cultivation amid digital transformation.

Fourth, the rectification of invalid DOIs. We have noticed that the DOIs of several references including the paper by Kim et al. were inaccessible. We have checked the DOI links of all references one by one, updated the invalid links and supplemented missing publication details.

We hope these comprehensive revisions fully address your concerns and bring the manuscript up to the journal’s publication standards in terms of language use, statistical norms and the quality of the literature review. This new section can be found on page 15, Table 4; page 18, starting from paragraph 2, line 9-10;page 11, starting from paragraph 2, line 478-479;page 4, starting from paragraph 2, line 154-170.The revised parts are marked in red in the modified manuscript.

 

Comments 8: [Conclusion and Recommendation

Overall, this is a promising manuscript with a relevant topic, a meaningful vocational-education focus, and a potentially useful measurement framework. Its main contribution lies in showing that the quality and context of GenAI use may matter more than simple frequency of use. This is an important message for educators and institutions seeking to integrate GenAI into vocational learning environments.

However, the paper requires major revision. The authors should substantially revise the causal framing, clarify the sample reporting inconsistency, provide fuller scale validation evidence, treat self-reported learning outcomes more cautiously, and strengthen the statistical interpretation. With these improvements, the manuscript could make a valuable contribution to research on GenAI use in vocational education.]

Response 8: We sincerely appreciate the reviewer’s full recognition and encouragement regarding our work. Thank you for acknowledging the core findings of this study. The development of the Generative AI Usage Behavior Scale and the Vocational College Students’ Perceived Learning Outcomes Scale, as well as our focus on vocational education — an area that has been relatively understudied in existing literature — are recognized for their clear practical implications and solid theoretical foundations. These positive remarks confirm that this research is built upon a promising and valuable basis.

Meanwhile, we fully take on board the key concerns raised, which cover the causal framing, inconsistent sample descriptions, insufficient evidence for scale validation, the need for a more prudent interpretation of self-reported learning outcomes, and greater clarity in explaining statistical results. We have taken all comments seriously and conducted comprehensive and substantial revisions to address every issue. Furthermore, we have polished the language and standardized the formatting throughout the manuscript, correcting grammatical errors, formula layouts and table presentations.

We have strived to bring the revised paper in line with the journal’s publication standards in terms of theoretical construction, methodological rigor and the clarity of statistical interpretation. All revisions are presented in the updated manuscript for your review. We welcome any further feedback. Once again, thank you for your time and professional guidance that have helped improve the quality of this paper.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Dear Authors,

Thank you for the opportunity to review your manuscript "Research on the Impact of Generative Artificial Intelligence Usage Behavior on the Learning Outcomes of Higher Vocational Students". The article deals with an important topic, especially in light of the rapid expansion of the use of Generative Artificial Intelligence (GenAI) tools in vocational education and training settings. The study focuses on a population that does not receive sufficient attention in the research literature, students in vocational training institutions, and thus adds an important layer to the existing knowledge. In addition, the development of two dedicated measurement tools is an important attempt to expand the toolbox of emerging research in the field. The article is based on a large and diverse sample, presents a well-organized research procedure, and produces interesting findings with potential for research and application contributions. However, there are several essential aspects that require methodological improvement and strengthening before publishing the article.

  1. The abstract is clearly written and presents the gist of the article. The literature review is comprehensive, but mostly descriptive. It is advisable to add a discussion of contradictory findings in previous studies and a clearer justification for developing the model in the review, this will strengthen the research justification.
  2. Although the article presents an interesting empirical contribution, the study does not rely on an established theoretical framework such as: TAM, UTAUT, SRL, etc. It is recommended to incorporate a clear theoretical framework that explains why different patterns of use of GenAI are expected to affect learning outcomes and how the study variables are related to each other.
  3. The development of the measurement tool is a central part of the article. The authors did conduct EFA, reliability tests, and basic validity tests. However, it is also very worthwhile to conduct a Confirmatory Factor Analysis (CFA). Without CFA, it is difficult to determine that the proposed theoretical structure is indeed valid.
  4. The article presents only four research questions. However, given an extensive literature review and a large sample, it would have been expected to present 4-5 explicit research hypotheses. For example: Intelligent use of GenAI will positively affect learning outcomes. Developing hypotheses will strengthen the research structure and anchor the findings in existing literature.
  5. Regarding the statistical analysis. Very large sample, extensive reliability and validity tests, very high Cronbach's Alpha, exploratory factor analysis (EFA), and more. It is recommended to consider using SEM (Structural Equation Modeling) to examine the model more comprehensively. Also, despite the sample size, structural equation analysis (SEM) was not performed. The results are clear and well presented, but the regression results explain only about 20% of the variance in learning outcomes (R² = 0.204), with many significant variables that were not measured. Therefore, it is recommended to discuss in more detail additional variables that may affect learning outcomes, such as: motivation, engagement in learning, self-efficacy, etc.
  6. The findings are presented in a clear and orderly manner. It seems that the most interesting finding in the article is: high frequency of use of GenAI was found to be negatively related to learning outcomes, while usage habits and usage contexts were found to be positively related. This is a very important finding, but the explanation for this finding is relatively brief. Therefore, it is recommended to develop a more in-depth discussion of this finding.
  7. Some of the limitations require a broader consideration: use of the questionnaire in self-report only, possibility of social desirability bias, limitations of generalization outside of China, underrepresentation of students in advanced years.
  8. The article does not include a visual illustration (figures and graphic models) of the research model. It is recommended to add a conceptual model and a research diagram. Such an addition will improve the clarity of the article.
  9. In general, the article is written in good English, but several linguistic and stylistic problems are evident: grammatical errors, inconsistent use of tenses, long and cumbersome sentences. For example: This This, Test results was, GenAI behavior are defined. Before publication, it is recommended to have a professional language editing by a linguistic editor. The bibliography is extensive and up-to-date, but it contains few sources from the leading literature on the subject of GenAI specifically in higher education. It is suggested to add a number of more studies directly related to higher education in order to strengthen the connection to the literature:

Carmi, G. (2025). Learning with Generative AI: An Empirical Study of Students in Higher Education. Education Sciences, 15(12), 1696. https://doi.org/10.3390/educsci15121696

Dabis, A., & Csáki, C. (2024). AI and ethics: Investigating the first policy responses of higher education institutions to the challenge of generative AI. Humanities and Social Sciences Communications, 11(1), 1–13. https://doi.org/10.1057/s41599-024-03526-z

Maxwell, D., Oyarzun, B., Kim, S., & Bong, J. Y. (2025). Generative AI in higher education: Demographic differences in student perceived readiness, benefits, and challenges. TechTrends, 69(6), 1248-1259. https://doi.org/10.1007/s11528-025-01109-6

Sergeeva, O. V., Zheltukhina, M. R., Shoustikova, T., Tukhvatullina, L. R., Dobrokhotov, D. A., & Kondrashev, S. V. (2025). Understanding higher education students’ adoption of generative AI technologies: An empirical investigation using UTAUT2. Contemporary Educational Technology, 17(2), ep571. https://doi.org/10.30935/cedtech/16039

  1. In conclusion, this is an article with high relevance, a broad data base, and a certain applied contribution. However, there are a number of theoretical and methodological weaknesses that limit the strength of its scientific contribution and require corrections before acceptance for publication.

Best of luck,

The reviewer

Comments on the Quality of English Language

Regarding English, see section 9 in the notes.

Author Response

Comments 1: [Thank you for the opportunity to review your manuscript "Research on the Impact of Generative Artificial Intelligence Usage Behavior on the Learning Outcomes of Higher Vocational Students". The article deals with an important topic, especially in light of the rapid expansion of the use of Generative Artificial Intelligence (GenAI) tools in vocational education and training settings. The study focuses on a population that does not receive sufficient attention in the research literature, students in vocational training institutions, and thus adds an important layer to the existing knowledge. In addition, the development of two dedicated measurement tools is an important attempt to expand the toolbox of emerging research in the field. The article is based on a large and diverse sample, presents a well-organized research procedure, and produces interesting findings with potential for research and application contributions. However, there are several essential aspects that require methodological improvement and strengthening before publishing the article.]

Response 1: We sincerely thank the reviewer for the in-depth comments on the significance and features of this study. We are greatly encouraged by your recognition of our three core strengths. Firstly, this research addresses the important topic of the rapid adoption of generative AI tools in vocational education and training. Secondly, focusing on vocational college students, a group that has not received adequate attention in current literature, adds valuable insights to the existing body of knowledge. Thirdly, the development of two dedicated measurement scales is regarded as a useful supplement to the toolset for emerging research in this field. These positive evaluations demonstrate that our research is built on a promising and meaningful foundation.

Meanwhile, we fully acknowledge the key methodological issues that need to be revised and improved. We have taken all your comments seriously and completed comprehensive and substantial revisions to the manuscript. We welcome any further feedback. Thank you again for your time and professional guidance, which have greatly helped enhance the quality of this paper.

 

Comments 2: [The abstract is clearly written and presents the gist of the article. The literature review is comprehensive, but mostly descriptive. It is advisable to add a discussion of contradictory findings in previous studies and a clearer justification for developing the model in the review, this will strengthen the research justification.]

Response 2: We sincerely appreciate the reviewer's compliments on the clarity of the abstract and the comprehensiveness of the literature review. We fully agree that the original literature review was largely descriptive, lacked in-depth discussion on inconsistent findings in prior studies, and failed to elaborate clearly on the theoretical basis for model development. We have made substantial revisions to Section 2 Literature Review by incorporating findings from studies with both positive and negative perspectives. The detailed revisions are as follows.

First, we added a systematic discussion on conflicting results. A new paragraph is included in Section 2.3 to sort out contradictory conclusions regarding the relationship between generative AI use and learning outcomes in existing literature:

Nevertheless, divergent views also exist in relevant research. Some scholars raise concerns about academic integrity, over-reliance on AI, as well as data privacy and security issues amid students' use of generative AI (Maxwell et al., 2025). Meanwhile, the application of generative AI may lead to cognitive offloading and mental laziness, and impair students' abilities of independent problem-solving and critical thinking (Li et al., 2026; Choudhuri et al., 2026). For instance, when generative AI is adopted for writing tasks, it improves superficial fluency yet weakens students' understanding of in-depth text structures and independent reasoning skills (Rahman et al., 2025). Furthermore, generative AI does not exert a uniformly positive effect on learning motivation. Such tools usually fail to remind users of uncertain answers, which may cause students to overestimate their own capabilities, adopt a passive learning attitude, and gradually lose intellectual curiosity and critical thinking, thereby undermining their intrinsic motivation (Abdelghani et al., 2023). For students with low self-efficacy, completing complex tasks with generative AI without independent thinking and knowledge organization may bring about favorable short-term learning performance. However, their academic performance will decline markedly without AI support and their long-term knowledge transfer ability will be impaired. This indicates that the use of AI may give rise to adaptive dependence (Li et al., 2025).

Second, we transformed the literature review from a descriptive style to a critical one. We revised the simple enumeration structure such as "Researcher A proposed... Researcher B found..." into a critical narrative framed as "Scholars holding positive views argue that... However, other studies present conflicting findings...", which enhances the analytical depth of this section.

We hope these revisions turn the literature review from descriptive to analytical, better identify research gaps and consolidate the theoretical foundation of this study. All relevant updates have been incorporated into Section 2 of the revised manuscript. This new section can be found on page 5, starting from paragraph 2, line 222-277.The revised parts are marked in red in the modified manuscript.

 

Comments 3: [Although the article presents an interesting empirical contribution, the study does not rely on an established theoretical framework such as: TAM, UTAUT, SRL, etc. It is recommended to incorporate a clear theoretical framework that explains why different patterns of use of GenAI are expected to affect learning outcomes and how the study variables are related to each other.]

Response 3: We sincerely thank the reviewer for pointing out this important theoretical deficiency. We fully agree that the original manuscript lacked a clear elaboration of the theoretical framework and failed to explain why different patterns of usage behavior were expected to exert differentiated impacts on learning outcomes. Accordingly, we have integrated classic theoretical frameworks in Section 3.2.1 Development of the Generative AI Usage Behavior Scale to strengthen the theoretical foundation of this study. The detailed revisions are presented as follows:

The Technology Acceptance Model (TAM) and the Unified Theory of Acceptance and Use of Technology (UTAUT) are classic theoretical frameworks for explaining users’ adoption and usage of information systems, which have been widely validated and applied in the field of educational technology (Daruwala, 2026). This study systematically analyzes vocational college students’ generative AI usage behaviors by combining Davis’s (1989) Technology Acceptance Model (TAM), Venkatesh et al.’s (2003) Unified Theory of Acceptance and Use of Technology (UTAUT), and existing AI literacy assessment frameworks. An evaluation framework for generative AI usage behavior is thus proposed, consisting of three dimensions: usage frequency, usage habits and usage contexts.

The framework covers the above three core dimensions. For the dimension of usage frequency, guided by the core logic of TAM that perceived usefulness drives the intensity of usage behavior and drawing on measurement approaches of existing scales, we incorporate two indicators: behavioral dependence and task-oriented usage frequency. This dimension mainly assesses how frequently students use generative AI and their degree of reliance on it in study and daily life.

The dimension of usage habits is primarily built upon the procedural competencies defined in AI literacy frameworks. It focuses on evaluating users’ behaviors during dynamic interaction with generative AI, including interaction optimization, information screening and critical adoption, so as to measure vocational college students’ practical ability to use generative AI efficiently and critically.

For the dimension of usage contexts, we adopt the construct of facilitating conditions from UTAUT as the main theoretical basis. In UTAUT, facilitating conditions are designed to measure the availability of environmental resources, technical support and system compatibility, and essentially serve as a core variable for capturing contextual differences. Considering that practical teaching runs through the entire talent cultivation process of vocational college students, this dimension divides usage contexts into learning, daily life and personal development scenarios. It aims to examine differences in usage behavior under varying levels of facilitating conditions and assess the breadth of students’ generative AI application across diverse situations.

With the above theoretical integration, we can explain from the classic perspectives of TAM and UTAUT why usage frequency, usage habits and usage contexts show differentiated correlations with learning outcomes respectively. The complete theoretical framework has been fully presented in Section 3.2.1 of the revised manuscript. We hope these revisions clarify and solidify the theoretical underpinnings of this research and fully address your concerns regarding the theoretical framework. This new section can be found on page 8, starting from paragraph 4, line 368-393.The revised parts are marked in red in the modified manuscript.

 

Comments 4: [The development of the measurement tool is a central part of the article. The authors did conduct EFA, reliability tests, and basic validity tests. However, it is also very worthwhile to conduct a Confirmatory Factor Analysis (CFA). Without CFA, it is difficult to determine that the proposed theoretical structure is indeed valid.]

Response 4: We sincerely thank the reviewer for this valuable methodological suggestion. We fully agree that while exploratory factor analysis (EFA) can preliminarily reveal the dimensional structure of a scale, it cannot fully verify the validity of the theoretical structure. Confirmatory factor analysis (CFA) plays an irreplaceable role in verifying whether the three-factor structure proposed in this study fits the empirical data well. We have addressed this concern in two aspects in the revised manuscript.

First, we explicitly acknowledge the absence of CFA in the research limitations. We have added the following statement in Section 5 Research Limitations:

Although exploratory factor analysis, reliability tests and basic validity tests have been conducted for the Generative AI Usage Behavior Scale and the Vocational College Students’ Perceived Learning Outcomes Scale, confirmatory factor analysis has not been adopted to cross-verify the robustness of the theoretical structure. To a certain extent, this weakens the evidence for the construct validity of the scales.

Second, we specify the application of CFA and structural equation modeling in the future research directions. We further note in the Future Research section of the same chapter:

Given the complex relationships among the variables in this study, structural equation modeling can be adopted for in-depth mechanism analysis in follow-up research. This approach can better address multiple causal relationships and examine potential mediating or moderating effects.

We hope this candid discussion of limitations and targeted future research directions fully addresses your concerns. All relevant revisions are included in Section 5 of the revised manuscript. This new section can be found on page 22, starting from paragraph 2, line 179-239.The revised parts are marked in red in the modified manuscript.

 

Comments 5: [The article presents only four research questions. However, given an extensive literature review and a large sample, it would have been expected to present 4-5 explicit research hypotheses. For example: Intelligent use of GenAI will positively affect learning outcomes. Developing hypotheses will strengthen the research structure and anchor the findings in existing literature.]

Response 5: We sincerely thank the reviewer for this important suggestion. We fully agree that formulating explicit research hypotheses on the basis of an extensive literature review and large sample data can greatly improve the rigor of the research framework and better link the findings to existing studies. We have added a dedicated paragraph for research hypotheses in Section 3.2.2 Development of the Vocational College Students’ Learning Outcomes Scale, which reads as follows:

Based on previous studies on generative AI usage behavior and learning outcomes, we put forward the research hypotheses of this study.

H1: Usage frequency has a significant positive predictive effect on learning outcomes.

H2: Usage habits have a significant positive predictive effect on learning outcomes.

H3: Usage contexts have a significant positive predictive effect on learning outcomes.

H4: Among all dimensions of generative AI usage behavior, usage habits exert the strongest positive predictive effect on learning outcomes.

The above hypotheses are presented in Section 3.2.2 of the revised manuscript. We believe this addition clarifies the research structure and provides clear theoretical guidance for subsequent empirical tests. This new section can be found on page 10, starting from paragraph 2, line 439-447.The revised parts are marked in red in the modified manuscript.

 

Comments 6: [Regarding the statistical analysis. Very large sample, extensive reliability and validity tests, very high Cronbach's Alpha, exploratory factor analysis (EFA), and more. It is recommended to consider using SEM (Structural Equation Modeling) to examine the model more comprehensively. Also, despite the sample size, structural equation analysis (SEM) was not performed. The results are clear and well presented, but the regression results explain only about 20% of the variance in learning outcomes (R² = 0.204), with many significant variables that were not measured. Therefore, it is recommended to discuss in more detail additional variables that may affect learning outcomes, such as: motivation, engagement in learning, self-efficacy, etc.]

Response 6: We sincerely appreciate the reviewer's recognition of our sample size, reliability and validity tests, and statistical analysis. We fully acknowledge that the explanatory power of the current regression model is limited, and structural equation modeling (SEM) was not adopted for comprehensive mechanism analysis, which constitutes a methodological limitation of this study. Accordingly, we have supplemented prospects for the application of SEM in Section 5 Research Limitations and Future Research Directions of the revised manuscript:

Secondly, in terms of model construction and variable control, the regression model adopted in this study is relatively simplistic and fails to incorporate other factors that may affect learning outcomes. Future research should thoroughly discuss and introduce key control variables such as prior academic performance, socioeconomic status, institutional background, major, grade level, teacher support, learning motivation, learning engagement and self-efficacy. This will help explore the relationship between generative AI use and learning outcomes in a more rigorous manner. In addition, given the intricate relationships among the variables in this study, structural equation modeling can be employed for in-depth mechanism analysis. It is conducive to addressing complex causal relationships and examining potential mediating and moderating effects.

We hope this frank elaboration of limitations and specific future research directions fully addresses your concerns. All relevant revisions have been incorporated into Section 5 of the revised manuscript. This new section can be found on page 22, starting from paragraph 2, line 199-208. The revised parts are marked in red in the modified manuscript.

 

Comments 7: [The findings are presented in a clear and orderly manner. It seems that the most interesting finding in the article is: high frequency of use of GenAI was found to be negatively related to learning outcomes, while usage habits and usage contexts were found to be positively related. This is a very important finding, but the explanation for this finding is relatively brief. Therefore, it is recommended to develop a more in-depth discussion of this finding.]

Response 7: We sincerely thank the reviewer for commending the clear presentation of our research findings. You also pointed out that the negative correlation between usage frequency and learning outcomes, as well as the positive correlations of usage habits and usage contexts with learning outcomes, are the most intriguing results of this paper. We fully agree that the interpretation of this core finding was overly brief and requires further elaboration. We have expanded the discussion on this result in Sections 4.3.2 and 5 of the revised manuscript, with the supplementary content as follows.

In the regression results section (Section 4.3.2), we added:

Nevertheless, this negative coefficient does not indicate that usage frequency itself exerts adverse impacts on learning outcomes. Among vocational college students, high usage frequency is often accompanied by aimless use without clear learning objectives, over-reliance on AI-generated content, and a lack of active processing and in-depth thinking. These poor learning habits fail to translate into positive academic performance, thus resulting in a negative statistical correlation.

In the discussion section (Section 5), we supplemented:

The predictive effect of generative AI usage frequency on vocational college students’ learning outcomes does not follow the assumption that more frequent use leads to better learning performance. Unstructured use or excessive reliance on generative AI may help students finish learning tasks efficiently, yet deprive them of the ability to think independently and build personal knowledge systems, which in turn hinders the improvement of learning outcomes.

In the recommendations part of the discussion, we put forward relevant suggestions based on this finding:

For students, it is essential to move from merely being able to use AI tools to using them proficiently. As direct users of generative AI, students should continuously improve their digital literacy. They need to recognize that simply increasing AI usage frequency may impair learning outcomes and avoid cognitive offloading caused by over-reliance on generative AI for information retrieval and decision-making (Gerlich, 2025).

We hope these in-depth discussions fully address your concerns and offer a more thorough interpretation of the theoretical significance and practical implications of the core findings. All revisions have been included in Sections 4.3.2 and 5 of the revised manuscript. This new section can be found on page 18, starting from paragraph 3, line 16-38;page 21, starting from paragraph 2, line 147-157. The revised parts are marked in red in the modified manuscript.

 

Comments 8: [Some of the limitations require a broader consideration: use of the questionnaire in self-report only, possibility of social desirability bias, limitations of generalization outside of China, underrepresentation of students in advanced years.]

Response 8: We sincerely thank the reviewer for this valuable suggestion. We fully recognize that this study has multiple limitations regarding measurement instruments, sample representativeness and external validity, which require more comprehensive discussion in the future research section. We have systematically supplemented relevant content in Section 5 Research Limitations and Future Research Directions of the revised manuscript, as detailed below.

First, with regard to measurement instruments. As technology evolves rapidly, generative AI tools keep expanding in categories, functions and application scenarios, which further enriches the connotation of AI usage behavior. In addition, this study mainly adopts self-report questionnaires to measure learning outcomes. This method is inherently subjective, and the results may largely reflect students’ self-confidence, motivation and social desirability rather than their actual knowledge mastery and skill improvement. Future research shall update the components of generative AI usage behavior in line with the latest developments and develop more systematic, scientific and context-adaptive assessment tools. Meanwhile, to measure learning outcomes more accurately, subsequent studies should adopt multi-source data and objective indicators. On the basis of student self-reports, researchers may further incorporate course scores, practical training results, teacher evaluations, internship assessments, professional skill tests and certification outcomes to effectively reduce biases arising from single-source self-reported data.

Second, regarding sample selection and representativeness. There is a notable imbalance in sample size across grades, with an overrepresentation of junior students and insufficient participants from senior grades. This may affect the stability and generalizability of the research findings. Furthermore, all samples are recruited from vocational colleges in China, so whether the conclusions can be applied to educational contexts outside China remains to be verified. Future research shall select participants from vocational colleges of different regions, tiers and institutional types on a larger scale. Particular attention should be paid to balancing the sample proportion across grades and recruiting more senior students, so as to improve sample representativeness and the external validity of conclusions. Cross-cultural comparative studies are also encouraged to examine the robustness of the findings across different contexts.

We hope these supplements make the discussion on research limitations and future directions more comprehensive and transparent. All revisions have been included in Section 5 of the revised manuscript. This new section can be found on page 22, starting from paragraph 1, line 179-208. The revised parts are marked in red in the modified manuscript.

 

Comments 9: [The article does not include a visual illustration (figures and graphic models) of the research model. It is recommended to add a conceptual model and a research diagram. Such an addition will improve the clarity of the article.]

Response 9: We sincerely thank the reviewer for this constructive suggestion. We fully agree that adding visual diagrams for the research design can greatly enhance the clarity and readability of the manuscript, enabling readers to intuitively understand the research framework. We have inserted relevant diagrams in Section 3.1 Research Design of the revised manuscript. This new section can be found on page 7, Figure 1. Research Design Flowchart.

 

Comments 10: [In general, the article is written in good English, but several linguistic and stylistic problems are evident: grammatical errors, inconsistent use of tenses, long and cumbersome sentences. For example: This This, Test results was, GenAI behavior are defined. Before publication, it is recommended to have a professional language editing by a linguistic editor. The bibliography is extensive and up-to-date, but it contains few sources from the leading literature on the subject of GenAI specifically in higher education. It is suggested to add a number of more studies directly related to higher education in order to strengthen the connection to the literature:

Carmi, G. (2025). Learning with Generative AI: An Empirical Study of Students in Higher Education. Education Sciences, 15(12), 1696. https://doi.org/10.3390/educsci15121696

Dabis, A., & Csáki, C. (2024). AI and ethics: Investigating the first policy responses of higher education institutions to the challenge of generative AI. Humanities and Social Sciences Communications, 11(1), 1–13. https://doi.org/10.1057/s41599-024-03526-z

Maxwell, D., Oyarzun, B., Kim, S., & Bong, J. Y. (2025). Generative AI in higher education: Demographic differences in student perceived readiness, benefits, and challenges. TechTrends, 69(6), 1248-1259. https://doi.org/10.1007/s11528-025-01109-6

Sergeeva, O. V., Zheltukhina, M. R., Shoustikova, T., Tukhvatullina, L. R., Dobrokhotov, D. A., & Kondrashev, S. V. (2025). Understanding higher education students’ adoption of generative AI technologies: An empirical investigation using UTAUT2. Contemporary Educational Technology, 17(2), ep571. https://doi.org/10.30935/cedtech/16039]

Response 10: We sincerely appreciate the reviewer’s valuable comments on the writing style, references and academic standards. We have conducted comprehensive revisions accordingly.

First, we have corrected all identified grammatical errors, tense inconsistencies and problematic complex sentences one by one. In addition, professionals with extensive experience in academic English writing have polished the full manuscript to ensure clear, professional expressions that meet the journal’s publication requirements.

Second, we have fully adopted your suggestions regarding references. We have incorporated the four key papers you recommended, namely Carmi (2025), Dabis & Csáki (2024), Maxwell et al. (2025) and Sergeeva et al. (2025), which focus on the application of generative AI in higher education. These works are cited and discussed at appropriate places in the main text. This update greatly strengthens the links between the literature review and cutting-edge research in higher education, and further consolidates the theoretical foundation of this study.

The supplemented contents are as follows:

In Section 2.3: Among relevant factors, students’ positive attitudes, satisfaction and experience in using generative AI all play a facilitating role in this process (Carmi, 2025). Meanwhile, users’ usage habits, performance expectancy, social influence and personal innovativeness are critical drivers for the adoption of generative AI (Sergeeva et al., 2025; Lin & Jiang, 2025).

Also in Section 2.3: Some studies have pointed out that students have concerns over academic integrity, excessive reliance on AI, data privacy and information security when using generative AI (Maxwell et al., 2025).

In Section 5: Meanwhile, to improve transparency, teachers should clearly specify the scope of AI use, permitted and prohibited practices, and relevant academic integrity requirements in course syllabi (Dabis & Csáki, 2024).

We believe that the targeted revisions on language and references have substantially improved the overall quality and academic rigor of the manuscript. Thank you again for your time and professional guidance. This new section can be found on page 5, starting from paragraph 2, line 232-236;page 6, starting from paragraph 1, line 246-248;page 21, starting from paragraph 1, line 143-146.The revised parts are marked in red in the modified manuscript.

 

Comments 11: [In conclusion, this is an article with high relevance, a broad data base, and a certain applied contribution. However, there are a number of theoretical and methodological weaknesses that limit the strength of its scientific contribution and require corrections before acceptance for publication.]

Response 11: We sincerely thank the reviewer for the high praise on the practical relevance, data foundation and application value of this study. We fully acknowledge that the manuscript still has several deficiencies in theoretical construction and methodology, which limit its academic contributions. We have carried out comprehensive and systematic revisions to address all the concerns raised. We have strived to bring the revised paper up to the journal’s publication standards in terms of theoretical depth, methodological rigor and academic contribution. All revisions are marked in the updated version. We welcome any further feedback. Thank you again for your time and professional guidance that have helped improve this paper.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors
  • The authors have made several revisions and have responded carefully to the reviewer’s comments. The revised manuscript appears to be more transparent regarding its limitations and the exploratory nature of the study.
  • The topic remains timely and relevant, particularly given the increasing use of generative AI in vocational education. The focus on GenAI usage behavior and students’ learning outcomes has practical and academic interest.
  • However, several major methodological concerns remain insufficiently resolved. The dependent variable still relies on self-reported perceived learning outcomes rather than objective indicators such as grades, teacher evaluations, practical skill assessments, internship performance, or standardized tests. The authors acknowledge this limitation, but the issue remains central to the validity of the study.
  • The addition of Harman’s single-factor test is useful, but it is not sufficient to fully address common method bias, especially because both independent and dependent variables are collected from the same respondents using the same questionnaire.
  • The regression model remains too limited. The authors acknowledge the absence of key control variables such as prior academic performance, socioeconomic background, institution, major, year level, digital literacy, AI literacy, motivation, self-efficacy, engagement, and teacher support. However, these variables are not incorporated into the revised analysis, which limits the strength of the empirical conclusions.
  • The sample imbalance remains a serious concern. The strong overrepresentation of first-year students limits the external validity of the findings, especially in vocational education, where learning outcomes, skill acquisition, and career-related competencies are likely to vary across training stages and internship experience.
  • The authors clarify the issue of very small major categories and zero standard deviations, but the statistical comparisons across majors remain fragile. These results should be interpreted as descriptive rather than inferential.
  • The correction regarding the pilot sample size is important and strengthens the scale development section. However, the decision to retain PCA with Varimax rotation remains methodologically debatable for validating latent constructs.
  • The clarification of the AHP procedure improves the methodological description, but the manuscript should ensure that this weighting approach is fully justified theoretically and empirically.
  • The discussion of academic integrity has been improved, but it still seems mostly placed in future research rather than integrated into the core discussion of GenAI use in assignments, thesis writing, coding, and assessment validity.
  • Overall, the manuscript has improved, but several central weaknesses remain. Many concerns are acknowledged as limitations rather than being addressed through additional analyses or stronger methodological corrections.

Author Response

Comments 1: [The authors have made several revisions and have responded carefully to the reviewer’s comments. The revised manuscript appears to be more transparent regarding its limitations and the exploratory nature of the study. The topic remains timely and relevant, particularly given the increasing use of generative AI in vocational education. The focus on GenAI usage behavior and students’ learning outcomes has practical and academic interest.]

Response 1: We sincerely appreciate the reviewer’s positive feedback on our revisions. We are particularly gratified that the revised manuscript has been recognized for its transparent discussion of research limitations and clear positioning as an exploratory study, and that the research topic closely aligns with practical demands regarding the application of generative AI in higher vocational education. These encouraging remarks affirm both the practical and academic value of this study.We thank the reviewer for devoting time and providing professional guidance to improve the quality of this paper, and we look forward to further support from the journal for this research. The revised parts are marked in red in the modified manuscript.

Comments 2: [However, several major methodological concerns remain insufficiently resolved. The dependent variable still relies on self-reported perceived learning outcomes rather than objective indicators such as grades, teacher evaluations, practical skill assessments, internship performance, or standardized tests. The authors acknowledge this limitation, but the issue remains central to the validity of the study.]

Response 2: We sincerely thank the reviewer for once again pointing out this core methodological issue. We fully acknowledge that the reliance of the dependent variable on self-reported perceived learning outcomes instead of objective indicators constitutes a critical limitation to the validity of this study. We address this concern from two perspectives as follows.

First, we elaborate on the practical obstacles to accessing objective indicators. Objective data such as students’ course grades, teacher evaluations, and skill assessment results are classified as students’ private personal information in Chinese higher vocational colleges, which researchers cannot obtain through anonymous questionnaire surveys. Requesting real-name information or retrieving academic records from educational institutions would contradict the principles of anonymous research and requirements for student privacy protection. This practical constraint is the primary reason why this study adopts self-reported perceived learning outcomes, and it represents a common challenge confronting large-scale questionnaire research within China’s vocational education sector.

Second, we explain how we have addressed this limitation within the manuscript. In the research design section, we explicitly clarify that this study measures "perceived learning outcomes" rather than objective learning outcomes. We consistently adopt this terminology throughout the abstract, main text, discussion, and conclusion to prevent conceptual confusion and overgeneralization of the findings. Furthermore, we have explicitly acknowledged this shortcoming in the section titled "Limitations and Future Research Directions", where we detail specific avenues for future research to incorporate multi-source objective data, including course grades, practical training scores, teacher assessments, internship evaluations, and professional skill examinations or certification records.

We hope the above clarification fully addresses the reviewer’s concerns. We fully recognize that this limitation weakens the robustness of our conclusions. Nevertheless, given the types of data accessible under the current practical constraints, this exploratory study on the effects of generative AI usage in vocational education still delivers exploratory value and informative implications for relevant scholarship. We once again appreciate the reviewer’s comments, which have greatly improved the methodological transparency and rigor of this manuscript. The revised parts are marked in red in the modified manuscript.

Comments 3: [The addition of Harman’s single-factor test is useful, but it is not sufficient to fully address common method bias, especially because both independent and dependent variables are collected from the same respondents using the same questionnaire.]

Response 3: We sincerely thank the reviewer for highlighting this important methodological limitation once more. We fully agree that Harman’s single-factor test only provides a preliminary diagnosis of common method bias and its cutoff criterion cannot completely rule out the risk of same-source bias, especially given that both independent and dependent variables were collected from the same respondents at a single time point. To fully address this concern, we have supplemented three layers of supporting evidence in the revised manuscript to assess the potential impact of common method bias on the robustness of our findings.

First, we deepen the interpretation of Harman’s single-factor test results. We report that the first unrotated factor accounts for only 43.12% of the total variance, well below the 50% threshold. Moreover, exploratory factor analysis reveals that all items do not converge into a single factor but distinctly load onto six theoretical constructs: three dimensions of generative AI usage behavior and three dimensions of perceived learning outcomes. This indicates that respondents could distinguish between distinct constructs when completing the questionnaire, rather than relying on undifferentiated ratings driven by consistency motivation.

Second, we provide theoretical reasoning regarding common method bias. Common method bias generally inflates main linear correlations between variables. However, the core findings of this study reveal divergent association patterns between the three independent variables (usage frequency, usage habits, usage contexts) and perceived learning outcomes: usage frequency shows a negative correlation, while usage habits and usage contexts exhibit positive correlations. Existing methodological literature confirms that common method bias can hardly systematically generate opposing correlation patterns within one dataset. This is because respondents cannot produce systematically biased estimates with opposite directional trends merely out of a desire for consistent responding. Such mixed directional relationships reduce the likelihood that common method bias serves as a dominant alternative explanation for our results. In addition, we have incorporated demographic control variables (gender and grade) into the updated regression model, which statistically partial out irrelevant variance and further mitigate distortions stemming from method bias.

Third, we explicitly acknowledge and discuss this limitation in the manuscript. We have added a dedicated subsection within “Limitations and Future Research Directions” to openly address the drawbacks of our cross-sectional single-source research design. We emphasize that causal inferences drawn from this study should be treated with caution and recommend that future research adopt multi-source data to retest the model and eliminate confounding effects from common method bias entirely.

Taken together, while common method bias cannot be completely eliminated, the cumulative evidence demonstrates that it does not pose a severe threat capable of invalidating our core conclusions. We hope this transparent discussion of limitations and multi-faceted evidence fully resolves the reviewer’s concerns. We greatly appreciate your rigorous and patient review, as your comments have rendered our arguments more prudent and our wording more objective. We invite you to examine the corresponding supplementary content in the revised manuscript and would welcome any further valuable suggestions you may provide. The revised parts are marked in red in the modified manuscript.

 

Comments 4: [The regression model remains too limited. The authors acknowledge the absence of key control variables such as prior academic performance, socioeconomic background, institution, major, year level, digital literacy, AI literacy, motivation, self-efficacy, engagement, and teacher support. However, these variables are not incorporated into the revised analysis, which limits the strength of the empirical conclusions.]

Response 4: We sincerely thank the reviewer for pointing out the insufficient control variables in the regression model once again. We fully recognize that the absence of key control variables weakens the credibility of our empirical conclusions. To address this concern, we have made substantial revisions to the regression model in the revised manuscript, as detailed below.

First, grade and gender were added as control variables to the regression model. In the regression analysis presented in Section 4.3.2, grade and gender are incorporated to eliminate potential confounding effects of demographic characteristics on the dependent variable. The updated regression outputs show that after controlling for grade and gender, the sign and statistical significance of the three core independent variables remain essentially unchanged: generative AI usage frequency retains a significant negative association (Beta = -0.122, p < 0.001), while usage habits (Beta = 0.258, p < 0.001) and usage contexts (Beta = 0.196, p < 0.001) still exhibit significant positive correlations. These updated regression results are summarized in Table 6 of the revised manuscript.

Second, we further elaborate on why other proposed variables were not included in the analysis. Measures of prior academic performance, socioeconomic status, digital literacy, AI literacy, learning motivation, self-efficacy, learning engagement, and teacher support mentioned by the reviewer were not embedded in the initial questionnaire design, making it impossible to collect such data retrospectively. Particularly under the anonymous survey framework, obtaining objective records such as students’ historical academic scores and family background information poses substantial practical barriers. We have clearly acknowledged this constraint in the section “Limitations and Future Research Directions”, where we systematically list a full set of control variables recommended for future studies, including prior academic performance, socioeconomic status, digital literacy, AI literacy, learning motivation, self-efficacy, learning engagement, teacher support, and general technology usage frequency.

We hope the supplementary analyses above adequately address the reviewer’s concerns regarding the model’s control variables and preserve the validity of our core findings. Future research can incorporate these variables at the questionnaire design stage to more comprehensively unpack the relationships between generative AI usage behaviors and learning outcomes. The revised parts are marked in red in the modified manuscript.

 

Comments 5: [The sample imbalance remains a serious concern. The strong overrepresentation of first-year students limits the external validity of the findings, especially in vocational education, where learning outcomes, skill acquisition, and career-related competencies are likely to vary across training stages and internship experience.]

Response 5: We sincerely thank the reviewer for raising the serious methodological issue of unbalanced sample distribution once again. We fully agree that learning outcomes, skill acquisition and vocational competencies vary markedly across training stages and internship experiences within the vocational education context, and the overrepresentation of first-year students severely impairs the external validity of our findings. To resolve this critical problem, we collected an additional 213 questionnaires after responding to the first round of review comments, among which 160 were completed by third-year students and the remaining 53 by first-year students. Following this supplementary data collection, the proportion of third-year respondents has risen to nearly 20%.

We hope this extra dataset sufficiently addresses the reviewer’s concerns regarding sample imbalance and external validity, and provides stronger empirical support for the robustness of our research conclusions. The revised parts are marked in red in the modified manuscript.

Comments 6: [The authors clarify the issue of very small major categories and zero standard deviations, but the statistical comparisons across majors remain fragile. These results should be interpreted as descriptive rather than inferential.]

Response 6: We highly appreciate this valuable suggestion from the reviewer. We fully acknowledge that several broad major categories only contained a handful of participants, including Resources, Environment and Safety, Public Security and Justice, as well as Public Administration and Services. Their single-digit sample sizes resulted in standard deviations of 0.00, which greatly undermined the reliability of cross-group statistical comparisons. Accordingly, we purposely recruited more respondents from these underrepresented majors in the additional 213 questionnaires to optimize the dataset for inter-group comparisons.

Specifically, after supplementary data collection, the sample size of the Resources, Environment and Safety category increased from 4 to 56; Public Security and Justice rose from 1 to 23; and Public Administration and Services expanded from 1 to 56. All previously under-sampled major groups received substantial supplementary observations. The updated means and standard deviations of learning outcome scores for each major category have been presented in Table 4 of the revised manuscript, enabling readers to assess data dispersion and inter-group discrepancies more reliably.

Furthermore, we have added relevant explanations in the section "Limitations and Future Research Directions": although sample sizes for most major categories have been improved through extra data collection, a few categories still suffer from insufficient observations. Future research should expand the pool of cooperating colleges to cover a broader range of majors and guarantee that each group meets the minimum sample size requirements for statistical tests, so as to strengthen the reliability of cross-group comparisons and the external validity of research findings.

We hope the above supplementary data collection and textual revisions fully address the reviewer’s concerns regarding the robustness of statistical comparisons across major categories. The revised parts are marked in red in the modified manuscript.

 

Comments 7: [The correction regarding the pilot sample size is important and strengthens the scale development section. However, the decision to retain PCA with Varimax rotation remains methodologically debatable for validating latent constructs.]

Response 7: We sincerely appreciate the reviewer’s professional and meticulous suggestions. We fully recognize that although principal component analysis (PCA) with Varimax orthogonal rotation is widely adopted in exploratory factor analysis (EFA), it assumes orthogonal independent factors, which often contradicts the reality that constructs are typically correlated in educational and social science research. To verify the latent structure of the measurement scale more rigorously, we replaced the original approach with a more appropriate combination of Principal Axis Factoring and Promax Rotation to re-conduct exploratory factor analysis in the revised manuscript.

Detailed revisions are presented in Section 3.2.3 of the manuscript, as outlined below: We extracted factors via Principal Axis Factoring and performed Promax Rotation to implement exploratory factor analysis on all scale items. Based on factor loadings, Item C1 exhibited severe cross-loading: its primary loading on Factor 3 was 0.441, while its cross-loading on Factor 1 reached 0.415, with a mere difference of 0.026 between the two values. This gap fell far below the discriminant validity threshold of 0.20, so Item C1 was eliminated. After removing C1, we re-ran the EFA and obtained a KMO value of 0.942 (p < 0.001). In line with the eigenvalue-greater-than-one criterion and the inflection point of the scree plot, three latent factors were retained. Collectively, these three factors explained 64.782% of the total shared variance, exceeding the recommended benchmark of 60% for social science research. The post-rotation correlation matrix showed pairwise correlation coefficients ranging from 0.539 to 0.675, all below the threshold for discriminant validity, which demonstrates that the three dimensions are interrelated yet empirically distinct. According to the Pattern Matrix, all items yielded satisfactory loadings on their respective target factors. Factor 1 comprised items B1–B5 with loadings ranging from 0.791 to 0.881; Factor 2 contained items A1–A5 with loadings between 0.563 and 0.958; Factor 3 covered items C2–C6 with loadings from 0.417 to 0.795. All retained items had primary loadings above 0.40 without substantial cross-loadings, confirming the scale’s clear three-dimensional structure and sound construct validity.

For the scale measuring vocational students’ learning outcomes, the KMO value was 0.961 and the p-value of Bartlett’s test of sphericity was less than 0.001, indicating that the dataset was suitable for factor analysis. Factor loading results revealed severe cross-loadings for Items I1 and I3. Item I1 had a primary loading of 0.272 on its target Factor 2, alongside a cross-loading of 0.460 on Factor 1; the difference of only 0.188 failed to meet the 0.20 discriminant validity standard, and the item also conflicted with its theoretical assignment, hence it was deleted. Item I3 attained a primary loading of 0.416 on Factor 2 but a cross-loading of 0.447 on Factor 1. The gap between the two loadings was far below the discriminant validity cutoff, and the item mismatched its theoretical dimension, so it was also removed. After excluding I1 and I3, we re-performed EFA, which yielded a KMO value of 0.949 (p < 0.001). Three factors were extracted in total: Factor 1 included H1, H2, H3 with loadings spanning 0.759 to 0.881; Factor 2 consisted of I2, I4, I5, I6 with loadings from 0.483 to 0.871; Factor 3 covered J1, J2, J3 with loadings ranging from 0.659 to 0.845.

All the above revisions are fully documented in Section 3.2.3 of the revised manuscript. We believe that Principal Axis Factoring paired with Promax Rotation better aligns with the empirical reality of correlated constructs in educational research and provides more credible evidence regarding the scale’s latent structure. We hope this methodological adjustment fully addresses the reviewer’s concerns. The revised parts are marked in red in the modified manuscript.

 

Comments 8: [The clarification of the AHP procedure improves the methodological description, but the manuscript should ensure that this weighting approach is fully justified theoretically and empirically.]

Response 8: We greatly appreciate this valuable suggestion from the reviewer. We fully agree that the adoption of the Analytic Hierarchy Process (AHP) requires more sufficient theoretical and empirical justification. We have supplemented relevant elaborations from both theoretical and empirical perspectives in Section 3.3 of the revised manuscript for the reviewer’s review and feedback.

Theoretically, this study employs the AHP to assign weights to evaluation indicators. This method is suitable for weight calculation within complex multi-criteria and multi-level indicator systems, as it integrates qualitative expert experience with quantitative computation, which well matches the theoretical characteristics of the multi-layer evaluation framework constructed in this research. Compared with purely objective weighting methods such as the entropy weight method and coefficient of variation method, the AHP incorporates professional judgments from domain experts and avoids the drawback that single objective approaches may overlook the practical importance of indicators. In contrast to the standalone Delphi method, the AHP restrains biases in subjective judgments through consistency tests, thus yielding more rigorous results. Evaluation of learning outcomes in vocational education inherently involves multi-dimensional constructs including knowledge, skills and competencies, whose relative importance to vocational talent cultivation objectives varies. Skill application and competency development usually carry higher weights than knowledge mastery within employment-oriented training systems. Such structural characteristics align well with the logical rationale of AHP, which captures relative importance via pairwise comparisons conducted by experts. Admittedly, we recognize the inherent subjective limitations of the AHP, which we have frankly acknowledged in the limitations section of the manuscript.

Empirically, the AHP has been widely adopted for indicator weighting in studies concerning vocational education learning outcome assessment and related measurements, such as vocational skill evaluation and comprehensive student competency assessment, laying a solid practical foundation for its application. The hierarchical indicator structure established in this study takes the comprehensive level of learning outcomes as the target layer and the three dimensions as the criterion layer, which basically satisfies the data requirements of the AHP. In addition, we invited 15 vocational education experts with over five years of teaching or research experience, whose professional backgrounds cover a full range of major categories to provide credible bases for pairwise comparisons. The consistency ratio (CR) of all experts’ judgment matrices was lower than 0.1, meeting the standard threshold for consistency testing.

We remain aware of the inherent subjectivity limitations of the AHP and have openly addressed this issue in the discussion of research limitations. Should any reasoning presented above be inappropriate, we sincerely welcome further corrections from the reviewer and will revise the manuscript accordingly. The revised parts are marked in red in the modified manuscript.

 

Comments 9: [The discussion of academic integrity has been improved, but it still seems mostly placed in future research rather than integrated into the core discussion of GenAI use in assignments, thesis writing, coding, and assessment validity.]

Response 9: We sincerely thank the reviewer for pointing out this critical issue. We fully agree that discussions on academic integrity should not be confined merely to the future research section; instead, they ought to be organically integrated into the core analytical sections of the manuscript. We have made substantial revisions to relevant discussions in the revised version, closely linking the theme of academic integrity to core GenAI application scenarios including coursework completion, thesis writing, programming, and assessment validity. This topic now runs through several key chapters such as the literature review, discussion, and practical implications, rather than serving as a peripheral remark limited to future research directions. Detailed revisions are outlined as follows:

In the literature review (Section 2.3), when sorting out inconsistent findings from existing studies, we supplemented targeted discussions on concrete academic integrity risks with the following text:

“Nevertheless, contradictory arguments have been raised in prior scholarship. Several studies identify prevalent concerns regarding academic integrity, excessive AI reliance, data privacy and security among students using generative AI (Maxwell et al., 2025). Specifically, when students directly submit AI-generated content as their own work without proper attribution for assignments, thesis writing, coding and other tasks, such conduct may amount to plagiarism or academic misconduct. Yet many students lack clear awareness of the ethical boundaries of such practices. In the absence of institutional guidelines governing AI utilization, this ambiguous grey area of AI application tends to proliferate. It not only undermines the diagnostic capacity of traditional assessments to reflect students’ true competencies, but also intensifies threats to the academic integrity framework.”

In the discussion chapter (Section 5), while analyzing how AI usage habits shape learning outcomes, we further established an intrinsic linkage between usage habits and academic integrity by adding the following passage:

“In fact, there exists an intimate correlation between AI usage habits and academic integrity. Students with sound generative AI usage habits are more inclined to question, verify, and critically examine AI-generated outputs, which enables them to maintain heightened awareness and self-discipline in compliance with academic norms when citing or adopting AI-produced content. In contrast, students lacking healthy usage habits are prone to uncritically adopt outputs generated by AI. Such habitual over-reliance may inadvertently lead to plagiarism and other forms of academic misconduct. In the absence of clear institutional regulations governing AI utilization, this ambiguous grey area of AI application tends to become prevalent. It not only intensifies threats to the academic integrity framework but also diminishes the diagnostic value of assessments for evaluating students’ genuine competencies. In other words, academic integrity and metacognitive habits are two sides of the same coin. Sound generative AI usage habits encompass not only technical competencies such as optimized human-AI interaction and information screening, but also vigilance over the legitimate boundaries of AI content as well as sound ethical judgment.”

In the practical implications subsection (Section 5), we put forward targeted measures tied to this theme with the addition of the following sentence:

“Meanwhile, to enhance transparency, instructors should explicitly articulate permitted and prohibited AI usage scenarios alongside corresponding academic integrity requirements within course syllabi (Dabis & Csáki, 2024).”

All the above revisions are fully incorporated into Sections 2.3 and 5 of the revised manuscript. We hope this integration of academic integrity considerations throughout the literature review and core discussion sections adequately addresses the reviewer’s concern. The revised parts are marked in red in the modified manuscript.

Comments 10: [Overall, the manuscript has improved, but several central weaknesses remain. Many concerns are acknowledged as limitations rather than being addressed through additional analyses or stronger methodological corrections.]

Response 10: We sincerely appreciate the reviewer’s recognition of the improvements made in the revised manuscript, and we fully understand the core concern raised: several issues have only been addressed by acknowledging limitations rather than through substantive corrections, which indeed weakens the methodological rigor of this paper. To tackle this, we have comprehensively revised the manuscript via substantive improvements including supplementary data collection, model optimization, and the addition of control variables.

We respectfully ask the reviewer to recognize the inherent constraints of this exploratory study imposed by practical conditions. The value of this research lies in establishing a referential foundation for developing measurement scales and exploring preliminary correlations regarding generative AI usage behaviors in vocational education. Its primary contribution is to offer theoretical hypotheses, measurement instruments, and empirical insights for follow-up research, rather than drawing definitive causal conclusions. We anticipate that future studies can adopt more rigorous research designs to further verify and extend our findings. Should the reviewer deem the current revisions insufficient to meet the journal’s publication criteria, we are ready to receive specific feedback and make every effort to further refine the manuscript. The revised parts are marked in red in the modified manuscript.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Dear Author/s;

Thank you for engaging with my comments. I agreed that you have made revisions consistent with my directions. I believe you have done well, and I have agreed that you have outlined the steps I suggested. Well done. I hope you felt the review was fair and constructive.

Sincerely.

Reviewer

Author Response

Dear Reviewer, We are tremendously grateful for your thorough, fair and constructive review of our manuscript. We highly appreciate your positive recognition of our revisions and acknowledge that your valuable guidance has greatly strengthened the methodological rigor and logical completeness of this paper. Your professional suggestions have provided us with clear directions to polish this exploratory research on generative AI among vocational students. We have earnestly followed all your recommended adjustments and supplemented corresponding data analyses and discussions as you advised. We sincerely thank you again for your time, careful evaluation and kind affirmation of our revision work. Best regards,   The Authors

Round 3

Reviewer 1 Report

Comments and Suggestions for Authors
  • The authors have improved the conceptual framing by clarifying that the study focuses on students’ self-reported or perceived learning outcomes rather than objective learning outcomes. This distinction is important and should be maintained consistently throughout the manuscript, including the title, abstract, discussion, and conclusion.
  • The additional data collection is a positive improvement. The larger sample strengthens the empirical basis of the study and partially addresses the previous concern regarding sample imbalance. However, the authors should carefully check the manuscript for consistency, as some sections still seem to describe grade and major imbalance as a major limitation.
  • The revised factor analysis is a significant methodological improvement. Replacing PCA with Principal Axis Factoring and Promax rotation is more appropriate for latent constructs in educational and social science research. The deletion of problematic cross-loading items also improves the construct validity of the scales.
  • The authors have also improved the regression model by adding gender and grade as control variables. However, the model remains limited because other theoretically important variables, such as prior academic performance, socioeconomic background, AI literacy, digital literacy, motivation, self-efficacy, learning engagement, teacher support, and institution-level effects, are still absent. This limitation should continue to be clearly acknowledged.
  • The discussion of common method bias has improved. Nevertheless, Harman’s single-factor test and theoretical arguments cannot fully eliminate concerns about common method bias, especially because the independent and dependent variables remain self-reported and collected from the same respondents at the same time.
  • The integration of academic integrity into the literature review and discussion is a valuable improvement. The manuscript now better addresses issues such as plagiarism, unauthorized assistance, assessment validity, and the need for institutional AI-use policies.
  • The AHP procedure is now more clearly explained and theoretically justified. However, the authors should make sure that the weighting method is not presented as fully objective, since expert-based weighting necessarily includes subjective judgment.
  • Some causal language should still be softened. Given the cross-sectional survey design, the manuscript should avoid suggesting causal “impact” or “effects” too strongly and should instead refer to associations or predictive relationships.
  • Overall, the manuscript has improved considerably. Remaining issues concern consistency, cautious interpretation, and clearer acknowledgment of methodological limitations rather than fundamental flaws requiring another major revision.

Author Response

Comments 1: [The authors have improved the conceptual framing by clarifying that the study focuses on students’ self-reported or perceived learning outcomes rather than objective learning outcomes. This distinction is important and should be maintained consistently throughout the manuscript, including the title, abstract, discussion, and conclusion.]

Response 1: We sincerely appreciate the reviewer’s reminder regarding this vital conceptual distinction. We fully acknowledge that clearly anchoring this study to perceived learning outcomes rather than generic learning outcomes constitutes a core conceptual boundary of our research. This differentiation must be consistently maintained throughout the manuscript to prevent readers from overinterpreting our empirical findings. Accordingly, we have systematically standardized relevant terminology across the entire paper, with detailed revisions outlined below:

First, we revised the abstract. All references to the dependent variable in the abstract, covering the research background, methodology, analytical results and concluding remarks, have been uniformly revised to "perceived learning outcomes".

Second, we standardized terminology across all main chapters. We meticulously reviewed and revised more than 150 instances of the phrase "learning outcomes" distributed across the Introduction, Literature Review, Methodology, Results, Discussion and Conclusion sections, replacing each with "perceived learning outcomes". Corresponding adjustments have also been made to the keywords.

Third, we updated captions and notes in all tables and the appendix. Table headers and explanatory notes for Tables 3 to 6, as well as descriptions of learning outcome items in the questionnaire appendix, have all been revised to uniformly read "perceived learning outcomes".

Fourth, we refined wording to eliminate ambiguous shorthand. Throughout the revision process, we took special care to avoid abbreviating "perceived learning outcomes" as simply "learning outcomes", completely separating the two terms to remove any potential ambiguity.

Following these comprehensive revisions, the terminology for the dependent variable is fully consistent throughout the manuscript, which fundamentally clarifies the scope of applicability of our research conclusions. We hope these thorough adjustments adequately address the reviewer’s concern about consistent conceptual wording.

The revised parts are marked in red in the modified manuscript.

 

Comments 2: [The additional data collection is a positive improvement. The larger sample strengthens the empirical basis of the study and partially addresses the previous concern regarding sample imbalance. However, the authors should carefully check the manuscript for consistency, as some sections still seem to describe grade and major imbalance as a major limitation.]

Response 2: We are truly grateful to the reviewer for recognizing our efforts in supplementary data collection. We fully agree that the substantial addition of junior-year student samples has substantially alleviated the original grade imbalance issue, which necessitates corresponding updates to the limitations section to reflect the updated sample composition. In light of this, we have implemented the following revisions in the manuscript:

First, we removed limitation statements that no longer align with the updated sample data. The original text in the limitations section stated that certain broad major categories (resources, environment and safety; public security and justice; public administration and services) contained very small sample sizes (n<5), which might weaken the statistical power of ANOVA tests and the reliability of cross-group comparisons, thereby limiting the generalizability of our findings. After supplementary sampling, each of these major groups now includes over 10 respondents, rendering the original description obsolete for the current dataset.

Second, all text addressing grade imbalance has been eliminated. We collected an additional 160 valid questionnaires from junior students, which considerably raised the proportion of junior participants and greatly balanced the grade distribution. Accordingly, we deleted all prior remarks regarding skewed grade sampling.

Third, we streamlined the discussion of sample representativeness within the future research directions. The revised version now reads: “All participants of this study were recruited from vocational colleges in China, and the generalizability of the findings to educational contexts outside China remains to be further verified.” This wording retains our cautious stance on external validity while eliminating inaccurate claims inconsistent with the updated sample structure.

All the above modifications are incorporated into Section 5 “Limitations and Future Research Directions” of the revised manuscript. We hope these adjustments ensure that our discussion of study limitations accurately matches the characteristics of the updated sample. The revised parts are marked in red in the modified manuscript.

 

Comments 3: [The authors have also improved the regression model by adding gender and grade as control variables. However, the model remains limited because other theoretically important variables, such as prior academic performance, socioeconomic background, AI literacy, digital literacy, motivation, self-efficacy, learning engagement, teacher support, and institution-level effects, are still absent. This limitation should continue to be clearly acknowledged.]

Response 3: We sincerely appreciate the reviewer’s recognition of our improvement of incorporating gender and grade as control variables in the regression model. Meanwhile, we fully acknowledge that the current model still suffers from insufficient control variables. We have supplemented clear and candid explanations regarding this issue in the revised manuscript.

Specifically, we have added the following paragraph to Section 5 “Limitations of the Study”: “Fourth, the set of control variables incorporated into the regression model is incomplete. Although gender and grade have been included as controls, the model still has notable limitations. Restricted by data accessibility and research design, several theoretically critical predictors of learning outcomes are excluded from the model, including students’ prior academic performance, socioeconomic background, artificial intelligence literacy, digital literacy, learning motivation, self-efficacy, learning engagement, teacher support, and institutional-level effects. The omission of these covariates may reduce the explanatory power of the model concerning outcome variables and introduce biases into parameter estimates. Subsequent research may embed the above variables into questionnaire designs in advance and adopt more comprehensive model specifications to delineate the relationships between generative AI usage behaviors and perceived learning outcomes with higher precision.”

We have also updated relevant statements in the “Future Research Directions” subsection as follows: “Secondly, in terms of model construction and covariate control, the regression model adopted in this study is relatively parsimonious, with only gender and grade incorporated as control variables, while other factors that may shape learning outcomes are not included. Future studies may elaborate on and introduce vital covariates such as prior academic performance, socioeconomic background, artificial intelligence literacy, digital literacy, learning motivation, self-efficacy, learning engagement, teacher support, and institutional-level factors, so as to unpack the associations between generative AI usage and perceived learning outcomes with greater methodological rigor.”

We hope that such an open discussion of limitations can help readers gain a clearer understanding of the boundary conditions for our findings, while offering explicit guidance for follow-up investigations. All relevant revisions are presented in Section 5 of the revised manuscript, and we humbly invite the reviewer to examine them.

The revised parts are marked in red in the modified manuscript.

 

Comments 4: [The discussion of common method bias has improved. Nevertheless, Harman’s single-factor test and theoretical arguments cannot fully eliminate concerns about common method bias, especially because the independent and dependent variables remain self-reported and collected from the same respondents at the same time.]

Response 4: We sincerely thank the reviewer for acknowledging our optimized discussion on common method bias. We fully concur that Harman’s single-factor test and theoretical reasoning cannot completely eliminate this methodological concern. We have supplemented relevant elaborations from two dimensions, namely study limitations and future research directions, in the revised manuscript.

In the limitations section, we have added the following text: “Third, common method bias cannot be fully ruled out. While this study has conducted preliminary statistical verification via Harman’s single-factor test and delivered corresponding theoretical discussions, both independent and dependent variables are measured through self-reported surveys collected from identical participants at a single time point. Hence, common method bias may still exist. Such bias may distort the estimated relationships among variables, and the robustness of our research conclusions requires further verification. Future research can adopt multi-source data (e.g., peer ratings, teacher assessments, or objective learning indicators), multi-wave survey designs or longitudinal research frameworks to effectively mitigate the confounding effects of common method bias.”

In the future research directions subsection, we have retained and strengthened the following statement: “Meanwhile, to measure students’ learning outcomes with higher accuracy, subsequent research should actively introduce multi-source data and objective evaluation indicators. On the basis of students’ self-reports, additional metrics including course grades, practical training scores, instructor evaluations, internship assessments, professional skill test results and vocational certification outcomes should be incorporated. These measures can substantially reduce the potential disturbance of common method bias on research conclusions and further validate the robustness of the variable associations identified in this study.”

All revisions mentioned above are available in Section 5 of the revised manuscript. We hope this candid illustration of limitations together with concrete suggestions for improvement can fully address the reviewer’s concerns regarding common method bias.

The revised parts are marked in red in the modified manuscript.

 

Comments 5: [The AHP procedure is now more clearly explained and theoretically justified. However, the authors should make sure that the weighting method is not presented as fully objective, since expert-based weighting necessarily includes subjective judgment.]

Response 5: We sincerely appreciate the reviewer for pointing out this critical methodological detail. We fully recognize that although the Analytic Hierarchy Process (AHP) can integrate expert judgments and standardize subjective biases via consistency tests, this method inherently relies on experts’ subjective experiential perceptions and should not be framed as a fully objective weighting technique. We have revised relevant descriptions in the manuscript to reflect the methodological nature of AHP more rigorously and accurately.

Detailed revisions are listed as follows:

First, in Section 3.3, we updated the original theoretical discussion of AHP to the following content: “Theoretically, the Analytic Hierarchy Process (AHP) is applicable to weight assignment for complex multi-criteria and multi-layer indicator systems. It integrates qualitative expert experience with quantitative weight calculation, which aligns well with the theoretical characteristics of the multi-level evaluation framework constructed in this study. Compared with objective weighting methods such as the entropy weight method and the coefficient of variation method, AHP allows the incorporation of domain specialists’ subjective professional judgments, thereby overcoming the drawback that objective weighting approaches may overlook the practical substantive importance of indicators. In contrast to the standalone Delphi method, AHP mitigates biases in subjective evaluations to a certain extent through consistency checks, yielding comparatively rigorous results. It is worth noting that AHP fundamentally rests on experts’ subjective judgments, and the derived weights reflect the collective perceptions of a specific expert panel regarding the relative importance of indicators at a given time point.”

Second, we added a new discussion regarding the subjectivity of AHP weighting to Section 5 “Limitations of the Study”: “Second, weight determination via the Analytic Hierarchy Process involves inherent subjectivity. This study adopts AHP to assign weights to indicators across all dimensions. While this method effectively incorporates professional insights from domain experts and restrains subjective deviations to some degree through consistency tests, the final weighting outcomes are essentially rooted in the experiential understandings of the expert panel. Such weights represent the collective evaluations of a specific group of specialists on indicator relative importance at a certain time point, rather than fully objective weight values. Different expert panels or alternative weighting algorithms may lead to divergent weight allocation schemes. Accordingly, we interpret the weighted calculation results cautiously in this research, treating the weights as quantitative reflections of aggregated expert experience within the current research context rather than absolutely objective measurement benchmarks.”

We hope these revisions clearly illustrate the inherent attributes of the AHP method and fully address the reviewer’s concerns. All relevant adjustments are incorporated into Section 3.3 and Section 5 of the revised manuscript.

The revised parts are marked in red in the modified manuscript.

 

Comments 6: [Some causal language should still be softened. Given the cross-sectional survey design, the manuscript should avoid suggesting causal “impact” or “effects” too strongly and should instead refer to associations or predictive relationships.]

Response 6:  We sincerely thank the reviewer for drawing our attention to this crucial issue once again. We fully agree that a cross-sectional survey design cannot establish causal inferences, and the entire manuscript should consistently adopt correlational or predictive wording. We have systematically screened and revised all causal statements throughout the paper, with detailed adjustments presented below:

First, in the Introduction (Section 1). The original research question containing strong causal implications has been revised to: “3. Which dimensions of generative AI usage behavior exhibit significant correlations with vocational college students’ perceived learning outcomes? If such correlations exist, what are their directional patterns?”

Second, in the Literature Review (Section 2). The positioning statement of this study is revised as follows: “Accordingly, targeting vocational college students as the research population, this study explores the correlations between generative AI usage behaviors and vocational students’ perceived learning outcomes, and further analyzes the potential linkage mechanisms underlying such relationships.”

Third, in the Results section (Section 4). Statements concerning analytical purposes and coefficient interpretation are revised respectively to: “thereby offering empirical references for an in-depth interpretation of the linkage mechanisms between generative AI usage behaviors and perceived learning outcomes.” “Building upon the correlation results, multiple linear regression analysis is further employed to examine the predictive associations between the multi-dimensional generative AI usage behaviors and vocational students’ perceived learning outcomes, as well as to compare the relative predictive contribution of each dimension.” “Nevertheless, this negative coefficient does not indicate a substantive adverse association between usage frequency itself and perceived learning outcomes.”

Fourth, in the Discussion section (Section 5). Descriptions of research objectives and core findings are updated to: “This study centers on the relationships between vocational students’ generative AI usage behaviors and their perceived learning outcomes, aiming to explore the characteristics and strength of such correlations.”“This study reveals that generative AI usage scenarios are significantly and positively correlated with perceived learning outcomes.” “It is observed that simply increasing the frequency of AI use is instead negatively correlated with perceived learning outcomes.”

In addition, we have conducted parallel screening across all other chapters to ensure that all descriptions of inter-variable relationships adopt cautious terms such as “correlation”, “predictive association” and “statistical correlation”, while eliminating causal terminology including “impact” and “effect”. All the above revisions have been implemented point-by-point in the revised manuscript. We hope these comprehensive adjustments can fully address the reviewer’s concerns regarding causal wording.

The revised parts are marked in red in the modified manuscript.

 

 

 
 
 
 
 

Author Response File: Author Response.pdf

Back to TopTop