Empowering Education with Intelligent Systems: Exploring Large Language Models and the NAO Robot for Information Retrieval

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The topic discussed in the article about the use of intelligent systems (LLMs and NAO robot) in education is relevant, especially in the context of the growing role of AI and robotics in the learning process. The study evaluates three different methods of information retrieval – traditional search engines, LLMs and NAO robot, examining them from different aspects – efficiency, engagement and perception by students. The empirical data were collected from pre-developed questionnaires. The study offers a completely feasible scenario for integrating these technologies into education, which makes it useful for teachers and researchers in the field. The study does not explicitly indicate the exact number of participants (although the results imply about 50?) or their demographic specifics, which makes it difficult to transfer the results to different educational contexts.

It presents a snapshot of the impact of these technologies, but does not consider long-term outcomes, such as effectiveness on learning outcomes or student adaptation over time. Future work could include students from different age groups and educational backgrounds to provide a more objective picture of the adoption of these technologies. Although the study examines the potential of AI and robots in education, it lacks a thorough analysis of the possible risks regarding the ethical implications of using AI in education, as well as the development of guidelines for its safe implementation.

I recommend a final section, Conclusion, to summarize the results of the study and identify tasks for future research on the topic

Author Response

Comments 1: The topic discussed in the article about the use of intelligent systems (LLMs and NAO robot) in education is relevant, especially in the context of the growing role of AI and robotics in the learning process. The study evaluates three different methods of information retrieval – traditional search engines, LLMs and NAO robot, examining them from different aspects – efficiency, engagement and perception by students. The empirical data were collected from pre-developed questionnaires. The study offers a completely feasible scenario for integrating these technologies into education, which makes it useful for teachers and researchers in the field. 

Respond 1: Thank you very much for your thorough review of our manuscript. We sincerely appreciate the time and effort you have dedicated to providing valuable feedback, as well as your kind comments on the positive aspects of our work. We have carefully considered all the suggestions and made the necessary revisions, which we believe have significantly enhanced the quality of the manuscript. Below, we address each point raised in detail.

 

Comments 2: The study does not explicitly indicate the exact number of participants (although the results imply about 50?) or their demographic specifics, which makes it difficult to transfer the results to different educational contexts.

Respond 2: Thank you very much for pointing out the omission. The number and background of the participants are mentioned at the beginning of the Methodology Section.

Comments 3: It presents a snapshot of the impact of these technologies, but does not consider long-term outcomes, such as effectiveness on learning outcomes or student adaptation over time. 

Respond 3: We thank the reviewer for highlighting this important point. In response, we have added a paragraph in the Conclusions and Future Work section, acknowledging that while our study focuses on the immediate impact of these technologies, it does not assess long-term outcomes. We also suggest directions for future research, including the exploration of how sustained interaction with these tools might influence knowledge retention, critical thinking, and independent learning.

 

Comments 4: Future work could include students from different age groups and educational backgrounds to provide a more objective picture of the adoption of these technologies. 

Respond 4: Thank you very much for the suggestion. It is something we have in mind and will investigate in a subsequent experiment, as soon as we have the opportunity.

 

Comments 5: Although the study examines the potential of AI and robots in education, it lacks a thorough analysis of the possible risks regarding the ethical implications of using AI in education, as well as the development of guidelines for its safe implementation.

Respond 5: We thank the reviewer for highlighting another important point. In response, we have expanded both the Conclusion section (old Discussion) and the subsection "Students' Perception of Robots" also in the new Conclusion section to address the ethical implications of using AI in education, including concerns related to data privacy, algorithmic bias, and the potential over-reliance on automated systems. We also discuss the need for clear guidelines and frameworks to ensure the safe and effective implementation of AI-powered tools, promoting critical thinking and responsible technology use among students.

Comments 6: I recommend a final section, Conclusion, to summarize the results of the study and identify tasks for future research on the topic

Respond 6: Thank you very much for the recommendation. The text has been restructured and now includes the Conclusions and Future Work section.

Reviewer 2 Report

Comments and Suggestions for Authors

The article is well written, but it deserves improvements, as I point out below:

1) Section 2 only focuses on reviewing the technologies used in your proposal. Instead, a table should be presented indicating related works, techniques used, population size, advantages, disadvantages, and differences with respect to your proposal.

2) Your work should consider a larger and more diverse sample, including different age groups and educational contexts, to enhance the generalizability of the findings.

3) The article should provide more detailed recommendations for addressing the technical limitations of NAO, particularly in terms of auditory recognition and responsiveness. Collaboration with robotics engineers could lead to practical solutions for improving NAO's usability in noisy environments.

4) Please explore and mention how LLMs can be integrated into classroom settings to foster critical thinking and personalized learning. This could include examples of specific pedagogical strategies or lesson plans that leverage the strengths of LLMs.

Author Response

Comment 1: The article is well written, but it deserves improvements, as I point out below:

Response 1: Thank you for your time and effort on reviewing our text, your comments are valuable and appreciated.

Comment 2: 1) Section 2 only focuses on reviewing the technologies used in your proposal. Instead, a table should be presented indicating related works, techniques used, population size, advantages, disadvantages, and differences with respect to your proposal.

Response 2: Thank you for your valuable suggestion regarding the structure of Section 2. We understand the potential benefit of including a comparative table to organize related works, techniques, and comparisons. However, constructing such a table in sufficient detail would extend beyond the scope of this manuscript, especially given the diversity and complexity of the research fields involved.

Instead, we have reorganized and reinforced Section 2. The related works have been divided into specific research sectors to improve clarity and readability. This restructuring allows for a more focused discussion on how these works relate to our proposal, with an emphasis on the distinguishing aspects of our approach. We believe this revision enhances the understanding of the background while maintaining the flow of the manuscript.

Comment 3: 2) Your work should consider a larger and more diverse sample, including different age groups and educational contexts, to enhance the generalizability of the findings.

Response 3: We appreciate your suggestion to increase the sample size and incorporate a more diverse range of participants. We agree that a larger and more varied sample would further enhance the generalizability of our findings. However, the current experiment was conducted with the available group of students, and the study has since concluded. Expanding the sample size, including participants from different age groups and educational contexts, is not feasible within the scope of this particular project.

That said, we fully recognize the importance of this consideration and plan to explore this in future research. If another opportunity arises with a larger and more diverse participant pool, we would be eager to replicate the study and investigate the broader applicability of our findings.

Comment 4: 3) The article should provide more detailed recommendations for addressing the technical limitations of NAO, particularly in terms of auditory recognition and responsiveness. Collaboration with robotics engineers could lead to practical solutions for improving NAO's usability in noisy environments.

Response 4: Thank you for raising this important point regarding the technical limitations of the NAO robot. We fully recognize that its auditory recognition and responsiveness can be hindered in noisy classroom environments, and we have pointed out these limitations in our manuscript. However, addressing these technical issues requires significant hardware and software modifications, which fall beyond the scope of our research. As the NAO robot is a commercially manufactured product, we do not have the capability to alter its internal systems or collaborate directly with robotics engineers to address these challenges.

Our goal in this study was to highlight the practical limitations of the NAO robot in real-world educational settings, with the hope that future developments by the manufacturers could address these issues. We have revised the manuscript to clarify the scope of our work in this regard and to better position these limitations as areas for future improvements in educational robotics.

Comment 5: 4) Please explore and mention how LLMs can be integrated into classroom settings to foster critical thinking and personalized learning. This could include examples of specific pedagogical strategies or lesson plans that leverage the strengths of LLMs.

Response 5: Thank you for your insightful suggestion regarding the integration of LLMs into classroom settings. In fact, our study does compare the use of search engines, LLMs, and the NAO humanoid robot as methods of retrieving cultural heritage information in primary school education. While the primary focus of this paper is on the NAO robot's role as a facilitator, LLMs are also a key part of the study. We examine their potential in fostering engagement and personalized learning, alongside other technologies.

However, we chose to limit our discussion on pedagogical strategies involving LLMs to maintain a balanced comparison between the three methods. Expanding on specific lesson plans or detailed pedagogical strategies for LLMs would broaden the scope of this study beyond its original intent. Nevertheless, we have highlighted how LLMs support knowledge retrieval and engagement, and we plan to explore their role in fostering critical thinking in future research.

Reviewer 3 Report

Comments and Suggestions for Authors

This paper offers an interesting and timely examination of how various technological tools, such as search engines, Large Language Models (LLMs), and the NAO humanoid robot, contribute to cultural heritage information retrieval in primary school education. By analyzing these three methods, the study valuable information on the impact of both embodied and non-embodied AI technologies on student engagement and learning outcomes. A notable strength of the paper is its well-presented results, which clearly show the importance of AI and robotics in the educational landscape.

Some major issues:

-the introduction must be separated into more paragraphs in order to provide a better motivation.

-In the introduction section as the last part a paragraph that clearly states the contribution of this paper must be written

-the related works could be written more clearly if the cited papers were grouped according to their contribution.

Some minor comment

-the histograms could be created using matplotlib (python)

- the images in the tables are not clear, fix the resolution of the plots (make them using python)

- the tables are not very easy to understand, make them more easy for the reader to understand.

-figures 2,3,4 have very low resolution.

-Rename the last section as “Conclusion”, and also in one paragraph

Comments on the Quality of English Language

-

Author Response

Comment 1: This paper offers an interesting and timely examination of how various technological tools, such as search engines, Large Language Models (LLMs), and the NAO humanoid robot, contribute to cultural heritage information retrieval in primary school education. By analyzing these three methods, the study valuable information on the impact of both embodied and non-embodied AI technologies on student engagement and learning outcomes. A notable strength of the paper is its well-presented results, which clearly show the importance of AI and robotics in the educational landscape.

Response 1: We sincerely thank the reviewer for their kind and encouraging comments. We are pleased to hear that the paper's examination of AI technologies in education, particularly in the context of cultural heritage information retrieval, was found to be interesting and timely. We appreciate the acknowledgment of our results and the positive impact AI and robotics can have on student engagement and learning outcomes. These comments further motivate us to continue exploring this important area of research.

 

Comment 2: -the introduction must be separated into more paragraphs in order to provide a better motivation.

Response 2: Thank you for pointing out the omission. The Introduction has been written from scratch, enriched and divided into more paragraphs.

Comment 3: -In the introduction section as the last part a paragraph that clearly states the contribution of this paper must be written

Response 3: We would like to thank the reviewer for their helpful comments on the structure and content of the Introduction. In response, we have rewritten the Introduction, separating it into more paragraphs to improve clarity and motivation. Additionally, we have included a concluding paragraph that clearly outlines the contributions of this paper. We believe these changes have made the Introduction more engaging and complete.

 

Comment 4: -the related works could be written more clearly if the cited papers were grouped according to their contribution.

Response 4: Thank you very much for pointing it out. The Related Work section has been restructured and the works have been grouped based on their contribution to a research field. The section has also been strengthened with more works.

Comment 5: -the histograms could be created using matplotlib (python)

Response 5: Thank you very much for the suggestion. It is certainly very useful and we will take it into account in continuing our research.

Comment 6: - the images in the tables are not clear, fix the resolution of the plots (make them using python)

Response 6: Thank you for pointing it out. The charts and tables have been modified to improve their appearance. The resolution and dimensions of the charts have been increased to make them easier to read.

Comment 7: - the tables are not very easy to understand, make them more easy for the reader to understand.

Response 7: Thank you very much for the comment, we understand that it is a difficult challenge for the text to display all these results and we would not like to discard any in order to simplify the display. In the tables, the analysis of the diagrams has been improved and the width of the columns has been changed to make them easier to read.

Comment 8: -figures 2,3,4 have very low resolution.

Response 8: Thank you for the guidance. Images 2, 3 and 4 are now larger in size and resolution.

Comment 9: -Rename the last section as “Conclusion”, and also in one paragraph

Response 9: Thank you for your suggestion. The last part of the text has been renamed Conclusions and Future Work. The text has been strengthened and the conclusions have been divided into sections.

Reviewer 4 Report

Comments and Suggestions for Authors

This paper titled examines the efficacy of using large language models (LLMs) and the NAO humanoid robot to facilitate information retrieval within educational settings, specifically for primary education and cultural heritage learning. It presents a comparative analysis of traditional search engines, LLMs, and the NAO robot in terms of enhancing student interaction and learning outcomes. The study employs student questionnaires to assess the impacts of each method on student engagement and learning efficiency. The strengths of the paper lie in its innovative approach to integrating advanced technologies in education and providing empirical insights into their comparative effectiveness.

Technical comments:

The authors should clarify the technical contributions and motivation of the work in the introduction to help readers understand the relevance and novelty of the study right from the beginning.

The introduction should be expanded beyond a single paragraph to adequately set the stage for the study and provide sufficient background.

In Section 3, the derivation of the three phases should be justified with educational theories or findings to establish a robust foundation for the methods used.

Detailed demographic information about the students, such as age and educational level, should be provided in Section 4. The number of students involved should also be specified to enhance the credibility of the results.

Improve the presentation of tables in Section 4 to ensure that the figures inside are clear and easily interpretable by the readers.

A conclusion section should be added to succinctly summarize the findings and implications of the study.

The literature review needs to be strengthened by including recent papers related to LLMs and their application in education. Please cite below:

https://ieeexplore.ieee.org/abstract/document/10577164

https://www.researchprotocols.org/2025/1/e63887

 

Comments on the Quality of English Language

Please proofread the paper to avoid typos.

Author Response

Comment 1: This paper titled examines the efficacy of using large language models (LLMs) and the NAO humanoid robot to facilitate information retrieval within educational settings, specifically for primary education and cultural heritage learning. It presents a comparative analysis of traditional search engines, LLMs, and the NAO robot in terms of enhancing student interaction and learning outcomes. The study employs student questionnaires to assess the impacts of each method on student engagement and learning efficiency. The strengths of the paper lie in its innovative approach to integrating advanced technologies in education and providing empirical insights into their comparative effectiveness.

Response 1: We would like to sincerely thank the reviewer for their positive feedback and thoughtful comments on our paper. We are pleased that the innovative approach and empirical insights provided in the study were appreciated, particularly the comparative analysis of traditional search engines, LLMs, and the NAO robot in educational settings. We have carefully considered all of the reviewer's comments and have made the necessary revisions to improve the clarity and depth of the paper. We believe that these edits have further strengthened the manuscript.

Comment 2: The authors should clarify the technical contributions and motivation of the work in the introduction to help readers understand the relevance and novelty of the study right from the beginning.

Response 2: We thank the reviewer for their insightful comment. In response, we have completely rewritten the Introduction to clarify both the technical contributions and the motivation of the work. We believe the revised version now provides a clearer understanding of the relevance and novelty of the study, helping readers grasp its importance right from the start.

Comment 3: The introduction should be expanded beyond a single paragraph to adequately set the stage for the study and provide sufficient background.

Response 3: We appreciate the reviewer’s suggestion regarding the structure of the Introduction. In response, we have expanded the Introduction beyond a single paragraph to provide a more detailed background and better set the stage for the study. The revised Introduction now offers a clearer and more comprehensive context for the research.

Comment 4: In Section 3, the derivation of the three phases should be justified with educational theories or findings to establish a robust foundation for the methods used.

Response 4: Thank you very much for your suggestion. Section 3 (Methodology) has been further developed and analyzed, and has also been strengthened in terms of the theoretical background in section 2.4.

Comment 5: Detailed demographic information about the students, such as age and educational level, should be provided in Section 4. The number of students involved should also be specified to enhance the credibility of the results.

Response 5: Thank you very much for pointing out the omission. The number and background of the participants are mentioned at the beginning of the Methodology Section.

Comment 6: Improve the presentation of tables in Section 4 to ensure that the figures inside are clear and easily interpretable by the readers.

Response 6: Thank you for pointing it out. The charts and tables have been modified to improve their appearance. The resolution and dimensions of the charts have been increased to make them easier to read.

Comment 7: A conclusion section should be added to succinctly summarize the findings and implications of the study.

Response 7: Thank you for your suggestion. The last part of the text has been renamed Conclusions and Future Work. The text has been strengthened and the conclusions have been divided into sections.

Comment 8: The literature review needs to be strengthened by including recent papers related to LLMs and their application in education. Please cite below:

https://ieeexplore.ieee.org/abstract/document/10577164

https://www.researchprotocols.org/2025/1/e63887

Response 8: Thank you very much for the recommendations. The Related Work section has been restructured and the works have been grouped based on their contribution to a research field. The section has also been strengthened with more works, like those recommended.

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The authors have more or less addressed the issues indicated as corrections. I have no further comments on my part.

Reviewer 3 Report

Comments and Suggestions for Authors

The authors addressed all of my comments.

Reviewer 4 Report

Comments and Suggestions for Authors

The authors have made considerable efforts to address the comments, and I am satisfied with the revised manuscript.