Evaluating an Artificial Intelligence (AI) Model Designed for Education to Identify Its Accuracy: Establishing the Need for Continuous AI Model Updates
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe article is of interest to educational institutions. They must certainly monitor and validate AI models, and their continuous improvement is essential in online education.
In section 6 you present the limitations and future directions well. The development of AI has been enormous in many areas. In all of them, the training of AI models with big data is fundamental. It may be interesting, in future work, to investigate how this could be useful in educational contexts. If there is a prediction for the next two years of a robot learning domestic tasks from videos, the robot teacher should be the next step. Assessing its accuracy and the need for updates will be essential.
Author Response
Dear Reviewer,
Thank you for taking the time and effort to review the manuscript. Your comments are very encouraging for us. In the future, we will investigate this AI model's real impact and how well it is assisting educational institutes. This will help us identify its accuracy and the need for the updates.
Warm regards,
Reviewer 2 Report
Comments and Suggestions for AuthorsDear authors,
Thank you for allowing me to read your manuscript "Evaluating the AI model designed for education to identify its accuracy: establishing the need for continuous AI model updates”. The article has potential as it explores an interesting topic, but to be considered for publication you should revise your manuscript based on the comments below.
Abstract. The abstract is well-structured and provides a concise summary of the study. It clearly outlines the problems, methodology, and significance of the research. You could include a brief recommendation for improving AI model accuracy.
Introduction. The introduction repeats information from the abstract without adding depth. Improve flow by starting with the problem statement, followed by existing gaps, and then introducing how this study addresses them.Instead of just stating that AI is used in education, highlight challenges in evaluating AI accuracy.
Conceptual Framework. It clearly presents the Design-Based Research approach and provides a structured explanation of research phases. The DBR diagram needs a brief explanation of each phase.
Methodology. Weel-written, it explains clearly data collection and statistical methods. I would add more details on AI model training (e.g., dataset characteristics, pre-processing). Also, explain why two experts were chosen for comparison (sample size justification).
Results & Discussion.
Well-organized presentation of statistical results. However, qualitative insight lacks, why did AI and experts disagree? The discussion could integrate theoretical perspectives on AI model reliability.
Conclusion & Implications. Lacks specific recommendations for AI model improvement. You chould discuss policy implications for educational institutions.
Limitations & Future Research. No comments.
References
Well-cited and relevant studies.
Ensure consistency in references and citation formatting.
Author Response
Comment 1: Abstract. The abstract is well-structured and provides a concise summary of the study. It clearly outlines the problems, methodology, and significance of the research. You could include a brief recommendation for improving AI model accuracy.
Response: Thank you for your feedback. We have updated our Abstract and added a brief recommendation for improving AI model accuracy. These changes are on page 1 (Abstract) and lines 22, 23, and 24.
Comment 2: Introduction. The introduction repeats information from the abstract without adding depth. Improve flow by starting with the problem statement, followed by existing gaps, and then introducing how this study addresses them. Instead of just stating that AI is used in education, highlight challenges in evaluating AI accuracy.
Response: Thank you for your suggestions. We have improved the flow of our Introduction by starting with the problem statement, followed by existing gaps, and then introducing how our study addresses them. We have also highlighted the challenges in evaluating AI accuracy. These changes are on page 2 (Introduction) and lines 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 85, 86.
Comment 3: Conceptual Framework. It clearly presents the Design-Based Research approach and provides a structured explanation of research phases. The DBR diagram needs a brief explanation of each phase.
Response: Thank you for your suggestion. To address it, we have included a brief explanation of each phase following the DBR diagram to provide greater clarity and context for the reader. These changes are on page 6 & 7 (Methods) and lines 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255,256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283.
Comment 4: Methodology. Weel-written, it explains clearly data collection and statistical methods. I would add more details on AI model training (e.g., dataset characteristics, pre-processing). Also, explain why two experts were chosen for comparison (sample size justification).
Response: Thank you for your feedback. We appreciate your suggestion to add more details on AI model training. Thus, we have included information on the dataset characteristics, pre-processing steps, and provide a justification for choosing two experts for comparison, justifying the sample size. These changes are on pages 6,7 & 8 (Methods) and lines 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255,256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 315, 316, 317, 318, 319.
Comment 5: Results & Discussion. Well-organized presentation of statistical results. However, qualitative insight lacks, why did AI and experts disagree? The discussion could integrate theoretical perspectives on AI model reliability.
Response: Thank you for your comments. To address them, we have added qualitative insights to the discussion by providing reasons behind the disagreements between the AI model and experts, thereby integrating relevant theoretical perspectives on AI model reliability to strengthen the discussion. These changes are on page 12 (Discussion) and lines 438, 439, 440, 441, 442, 443, 444, and 445.
Comment 6: Conclusion & Implications. Lacks specific recommendations for AI model improvement. You should discuss policy implications for educational institutions.
Response: Thank you for your feedback. As a recommendation for AI model improvement, we have highlighted the need for policy implications that require educational institutions to keep AI models updated to maintain accuracy and reliability. These changes are on page 13 (Discussion) and lines 498, 499, 500.
Comment 7: References Well-cited and relevant studies. Ensure consistency in references and citation formatting.
Response: Thank you for your helpful feedback. We have carefully reviewed the references and citation formatting to ensure consistency throughout the manuscript. These changes are on pages 12, 22 & 23 (Discussion and References) and lines 432, 433, 657, 658, 715, 716, and 717.
