Exploring the Use of AI to Optimize the Evaluation of a Faculty Training Program
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe theme is very relevant and interesting for the Educational Sciences domain.
The research methodology is appropriate and very well described in the article.
The references focus on the theme and are relevant for this.
The article had a very good approach for interpreting the results and discussions.
1) The theme is relevant and interesting for the Educational Sciences domain.
The teacher training program's quality is actual and a priority for educational purposes. I consider the information about the evaluation of a faculty and the AI that are involved in this process were appropriate for these issues. The evaluation of the initial training program from the university is described very well using the actual and relevant scientific literature.
2) The research methodology is appropriate and very well described in the article.
The theoretical framework describes the qualitative methods for the evaluation of the program and the possibilities of using ChatGPT. The longitudinal process of measuring (2012- 2024) is important for research methodology. The survey is well done and respects research criteria. Data analysis is included with a detailed explanation of the particular steps. In the article are numerous examples of the Ishikawa diagram of the problem.
3) The references focus on the theme and are relevant for this.
The references are selected through importance, relevance for the issues and actuality. Authors respect APA Style for citation.
4) The article had a very good approach for interpreting the results and discussions.
Discussions are centered on solutions for the science of education domain.
An action plan for teacher training is detailed.
Author Response
We sincerely thank you 1 for the positive and encouraging feedback on our manuscript. We are grateful for the recognition of the relevance of the theme, the rigor of the research methodology, the appropriateness of the references, and the clarity of the interpretation and discussion. Your comments are very much appreciated.
Reviewer 2 Report
Comments and Suggestions for AuthorsNice introduction to a very intriguing area, and the reviewer shares the interest in the aim of examining the potential of the AI chatbot ChatGPT to support human-centered tasks such as qualitative research analysis by using a case study involving an "initial university teaching training program ". The importance of teaching training is fair although several references are fairly outdated. This is why moving towards evaluation of such programs becomes important and here is also AI and chatbots introduced. Although this section is not especially rich, it opens up to next section, 1.5 on how this proposed study is using AI to optimise four main objectives of the study.
Chapter 2 presents extensively how the study setup has been made, however on p.11 (figure 1) the reviewer finds a potential typo in the figure (it says form where it probably should say from).
The problematic part reading this interesting piece was when you got to the results in section 3. Here there are a myriad of fishbone diagrams (Figure 3-6), tables (Table 1-4), and honestly it looks like figure 2 is a prompted categorised 2 by 2 matrix. Then next follows, figure 7 that summarise into the categorised dimensions, however how did you end up with these categories, can they be further motivated, and is anyone more critical than any other.
As the authors state on p.18 "The following tables 1 to 4 present the strategies proposed by ChatGPT to tackle problems 1 to 4, based on the previously identified causes". It is the same in the discussion section, which is also very much fragmented by an unreadable image (figure 8), on p.28 and the 4.1-4.4 structure that hinders a justified argumentation to be made, which in contrast to what is presented is the objective.
Rather than positioning the contribution and case experiences made, the reviewer is puzzled about what collaborative process that led to an action plan, the iterative prompting or synthesis made by the authors. The action plan as well as all four objectives of the paper is a result of a collaborative effort contextualising teaching training by running it, making an examination in chatGPT. This is why, probably also the paper is felt very fragmented and composed basically to drive an argumentation line that is loosely if all placing any attention to specific contributions scientifically or that could position the study in relation to other AI evaluation efforts in higher education.
Going through the student-based recommendations, it questionable to what extent this data is really presenting anything of novelty. The interesting aspect to this paper is the combination of and blend of using AI for pursuing a potential new way forward in teaching training evaluation. However, despite the paper includes several chatGPT outputs and lacks a path forward to anchor this to potentials or risks in examination processes it fails to convey its potential for improving program quality.
As the third concluding remark, p.31 state, "a hybrid model, pairing computational power with the refined judgment of domain experts, proved essential for producing valid, implementable enhancements; neither form of intelligence suffices on its own, but together they yield faster, fairer, and more context-aware decisions at scale". Yes, AI can provide efficiency to most process but how is this adding value, creating new dimensions for improvements, changing processes etc. questions are more after than before.. and that is a good thing so my suggestion is to pay attention to validate and reason without all tables and figures that are making it very much a prompted output paper that really bridging into something new, which it has potential of.
Author Response
We would like to warmly thank you for the careful reading of our manuscript and for the constructive comments that guided us in improving the clarity, coherence, and scholarly positioning of the paper. We particularly appreciate the observations regarding the need to update and enrich the references, the call to strengthen the section introducing AI and chatbots, and the suggestions for clarifying figures, categories, and the overall argumentative flow of the discussion. These remarks were extremely valuable and have led us to substantially revise and improve the manuscript.
In response, we carefully reviewed and updated the literature, incorporating more recent and relevant references while retaining a few seminal works that remain highly influential in the Spanish context. Section 1.5 was enriched with additional studies on AI applications in education, which we believe adds depth and relevance to the theoretical background. Methodological clarity was enhanced by revising Figure 1, correcting the detected error, and providing a complementary appendix that includes details of the process and prompts employed. To reduce fragmentation and improve readability, most diagrams and figures have been moved to the appendices, while the main text now emphasizes key findings and illustrates how raw AI outputs were refined by the expert team to validate categories and advance the analysis. We also justified more explicitly how the categories were derived and which dimensions appeared more critical, as suggested.
The discussion section has been restructured around the research questions, eliminating unnecessary fragmentation and clarifying the collaborative nature of the process between faculty experts and ChatGPT in the generation of the action plan. We further strengthened the positioning of the study by linking it with other recent research that uses ChatGPT in evaluation contexts, including examples from medical education and human resources decision-making. In this way, we aimed to better situate our contribution within ongoing international efforts in the field. Finally, we emphasized the novelty of our proposal, not only in combining human and AI perspectives for program evaluation but also in pointing toward the development of a customized GPT prototype capable of providing continuous, real-time feedback. This highlights the potential of the hybrid approach to generate context-sensitive, evidence-based solutions that reduce subjectivity and support sustainable improvements in teacher training.
We are grateful for these detailed and constructive remarks, which have undoubtedly strengthened the manuscript both conceptually and methodologically.
Reviewer 3 Report
Comments and Suggestions for AuthorsThis study presents an exploration of the use of AI to optimize the evaluation of a faculty training program, aiming to analyze survey responses to identify critical areas and recurring issues, investigate their root causes through thematic grouping and cause-and-effect diagrams, propose strategic solutions by linking issues with potential interventions, and formulate student-based recommendations to address program deficiencies, all assisted by ChatGPT.
The manuscript is well-organized and maintains a balanced approach.
However, I have few suggestions that may help improve your work. Please find them below:
The authors must provide a clearer explanation of the gap in the literature that motivated the development of this research. The novelty of the study also must be pointed.
Please check the in-text citation method, according to the template.
Please explain the notation used in tables (in the table footer)
Some solutions/actions should be pointed in Table 4, for Limited interaction spaces: Lack of collaboration in-class moments and No structured moments for peer exchange.
In discussion section, I suggest adding more explanation related to the precise ways in which your research advances or expands upon the knowledge that has been established by earlier research in the topic.
I suggest removing lines 851-852 (For research articles with several authors, a short paragraph specifying their 851 individual contributions must be provided. The following statements should be used “). Also remove unnecessary quotation marks from line 861 and 862.
The text from lines 1050-1052 does not represent a reference. Please correct.
I hope these comments help enhance your promising research.
Author Response
We sincerely thank you for the constructive and detailed comments, which have been very helpful in strengthening the manuscript. In response to the request for greater clarity regarding the research gap and the novelty of our study, we revised both the introduction and the end of the theoretical framework to explicitly highlight the lack of research on AI-assisted evaluation of initial teacher training programs in Spain, and to emphasize the contribution of our hybrid human–AI model as a novel approach. We also expanded the discussion to explain more precisely how our findings build on and extend previous work in the field.
In addition, we carefully revised all technical aspects as suggested. The citation style was checked and corrected in accordance with the template; the notation in the tables has been clarified in the footnotes; and in Table 4 we included solutions/actions for the issues related to limited interaction and lack of structured peer exchange. We also eliminated the placeholder lines regarding author contributions, removed unnecessary quotation marks, and corrected the non-reference text. These adjustments have improved the accuracy, readability, and overall coherence of the article. We are grateful for these valuable suggestions, which have contributed to enhancing both the conceptual and the formal quality of the manuscript.
Round 2
Reviewer 3 Report
Comments and Suggestions for Authorsok
