Empowering Education with Intelligent Systems: Exploring Large Language Models and the NAO Robot for Information Retrieval
Abstract
:1. Introduction
1.1. Study Aim and Research Focus
1.2. The Rise of Large Language Models in Education
1.3. Embodied Learning and the Role of Social Robots
1.4. Exploring the Interplay of AI and Robotics in Education
1.5. Contributions of This Paper
- Comparative analysis—it provides a systematic comparison between three different approaches for cultural information retrieval in an educational setting: traditional search engines, large language models (LLMs), and a humanoid robot (NAO) as a learning facilitator.
- Evaluation of NAO’s educational role—it explores whether NAO’s impact in the classroom extends beyond motivation and engagement, offering meaningful educational value.
- Insights on embodied vs. non-embodied AI—it examines the differences between embodied AI (NAO) and non-embodied AI (LLMs), providing insights into how these technologies influence student learning experiences.
- Empirical findings—it presents real-world data from a study involving sixth-grade students, contributing valuable empirical evidence to the discussion on AI integration in education.
2. Related Work
2.1. NAO Robot in Education and Social Interaction
2.2. NAO in Emotional Computation and Adaptive Learning
2.3. NAO in Supporting Autism Spectrum Disorder (ASD)
2.4. Large Language Models (LLMs) in Education
2.5. Limitations and Future Directions
3. Methodology
- Comparative Evaluation of Methods for Retrieving Cultural Heritage Information.
- General Comparison of Information Retrieval Methods.
- NAO-Specific Interaction.
3.1. Phase 1: Search Engines
- Introduction to search engines: students were introduced to the basics of using Google effectively, including tips on formulating search queries and evaluating the reliability of sources.
- Hands-on research activity: students conducted searches on specific topics related to Greek cultural heritage, collecting information to support their presentations.
- Group discussion: a collaborative discussion followed, during which students analyzed the effectiveness of search engines in gathering relevant and reliable information.
- Did you find the information you were looking for easily?
- Did you have trouble finding information on cultural heritage?
3.2. Phase 2: Large Language Models (LLMs)
- Introduction to LLMs: students received an overview of LLMs, their capabilities, and their potential applications in learning.
- Interactive Q&A session: students engaged with Gemini using voice commands in Greek and with ChatGPT for written queries. They compared the responses from both models, focusing on clarity, depth, and relevance.
- Cross-referencing results: students analyzed the information gathered from LLMs and compared it to the data they had collected during the first phase.
- Did each technology have knowledge of cultural heritage?
- Did you think you could use this technology without the help of the teacher?
3.3. Phase 3: NAO Robot
- Introduction to NAO: students were introduced to the features of the NAO robot and its potential as a learning assistant.
- Interactive session with NAO: students posed questions to the robot about Greek culture, observing its responses and engaging in a dialogue.
- Reflection and feedback: students shared their experiences working with the NAO robot, reflecting on how it compared to the previous methods.
- Were you satisfied with the NAO’s answers?
- Were there times when the NAO did not listen to what you were saying?
- Would you like the NAO to be smarter?
3.4. Comparative Analysis and Insights
- Which method did you find most fun?
- Which method took you the longest to do the work?
- Which method would you use again in the next exercise?
- Which method do you think should be improved?
3.5. Objectives and Outcomes
- Individual performance evaluation: to assess the performance of each method—Google’s search engine, large language models, and the NAO robot—in providing accurate and relevant information.
- Comparative analysis: to compare the three methods in terms of reliability, speed, and accuracy of responses, as well as their usability and engagement levels.
- Focus on embodied interaction: to evaluate the students’ experiences with the NAO robot, highlighting its potential as an innovative educational tool.
4. Analysis of Questionnaire Data
4.1. First Category: Comparative Evaluation of Methods for Retrieving Cultural Heritage Information
- Did you find the information you were looking for easily?
- Did you have trouble finding information about cultural heritage?
- Was the process of searching for information fun?
- Would you like to do more work with this technology?
- Do you think you can use this technology without the help of the teacher?
- Do you think this technology “knows” everything?
- Were there any questions you did not immediately find with this technology?
- Do you find it easy to interact with this technology?
- Did you spend a lot of time on each question?
- Did this technology confuse you in your search for information?
- Did each technology have knowledge of cultural heritage?
4.2. Summary of Results of First Category
4.2.1. Search Engines: High Efficiency and Usability, Low Engagement
4.2.2. LLMs: Balanced Usability and Engagement, Moderate Autonomy
4.2.3. NAO Robot: High Engagement, Challenges in Efficiency and Usability
4.2.4. Comparison Between Engagement and Efficiency Across Methods
4.2.5. Cultural Heritage Knowledge Across Methods
4.3. Second Category: General Comparison of Information-Retrieval Methods
4.4. Summary of Results of Second Category
4.5. Third Category: NAO-Specific Interaction
4.5.1. Interaction and Acoustic Skills
4.5.2. Perceptions of NAO and Robotics
4.6. Summary of Results of Third Category
5. Conclusions and Future Work
5.1. Efficiency vs. Engagement
5.2. Students’ Perception of Robots
5.3. Cultural Heritage Knowledge and Pedagogical Goals
5.4. NAO’s Acoustic Challenges
5.5. Future Orientations and Recommendations
Author Contributions
Funding
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
LLM | Large language model |
STEM | Science Technology Engineering Mathematics |
GPT | Generative Pre-trained Transformer |
AI | Artificial Intelligence |
ASD | Autism Spectrum Disorder |
References
- Company, T.B.R. Educational Robot Global Market Report 2025. Available online: https://www.researchandmarkets.com/reports/5782817/educational-robot-market-report?srsltid=AfmBOorh6s3vjcOAl70S5-Vm_iIp5HeDVQ9brvZFIxnPM7rl9ejCep8c (accessed on 17 March 2025).
- Eguchi, A.; Uribe, L. Robotics to promote STEM learning: Educational robotics unit for 4th grade science. In Proceedings of the 2017 IEEE Integrated STEM Education Conference (ISEC), Princeton, NJ, USA, 11 March 2017; pp. 186–194. [Google Scholar] [CrossRef]
- Benitti, F.B.V.; Spolaôr, N. How Have Robots Supported STEM Teaching? In Robotics in STEM Education: Redesigning the Learning Experience; Khine, M.S., Ed.; Springer International Publishing: Cham, Switzerland, 2017; pp. 103–129. [Google Scholar] [CrossRef]
- Evripidou, S.; Georgiou, K.; Doitsidis, L.; Amanatiadis, A.A.; Zinonos, Z.; Chatzichristofis, S.A. Educational Robotics: Platforms, Competitions and Expected Learning Outcomes. IEEE Access 2020, 8, 219534–219562. [Google Scholar] [CrossRef]
- Alimisis, D. Educational robotics: Open questions and new challenges. Themes Sci. Technol. Educ. 2013, 6, 63–71. [Google Scholar]
- Shamsuddin, S.; Ismail, L.I.; Yussof, H.; Ismarrubie Zahari, N.; Bahari, S.; Hashim, H.; Jaffar, A. Humanoid robot NAO: Review of control and motion exploration. In Proceedings of the 2011 IEEE International Conference on Control System, Computing and Engineering, Penang, Malaysia, 25–27 November 2011; pp. 511–516. [Google Scholar] [CrossRef]
- Amirova, A.; Rakhymbayeva, N.; Yadollahi, E.; Sandygulova, A.; Johal, W. 10 Years of Human-NAO Interaction Research: A Scoping Review. Front. Robot. AI 2021, 8, 744526. [Google Scholar] [CrossRef] [PubMed]
- Urdanivia Alarcon, D.A.; Cano, S.; Paucar, F.H.R.; Quispe, R.F.P.; Talavera-Mendoza, F.; Zegarra, M.E.R. Exploring the Effect of Robot-Based Video Interventions for Children with Autism Spectrum Disorder as an Alternative to Remote Education. Electronics 2021, 10, 2577. [Google Scholar] [CrossRef]
- Brienza, M.; Laus, F.; Guglielmi, V.; Carriero, G.; Grisolia, M.; Palermo, G.; Pierri, F.; Turi, M.; Muratori, F.; Bloisi, D.; et al. HRI-based Gaze-contingent Eye Tracking for Autism Spectrum Disorder Treatment: A preliminary study using a NAO robot. In Proceedings of the 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Busan, Republic of Korea, 28–31 August 2023; pp. 2665–2670. [Google Scholar] [CrossRef]
- Johal, W. Research Trends in Social Robots for Learning. Curr. Robot. Rep. 2020, 1, 75–83. [Google Scholar] [CrossRef]
- Ayeni, O.O.; Hamad, N.M.A.; Chisom, O.N.; Osawaru, B.; Adewusi, E. AI in education: A review of personalized learning and educational technology. GSC Adv. Res. Rev. 2024, 18, 261–271. [Google Scholar] [CrossRef]
- Atlas, S. ChatGPT for Higher Education and Professional Development: A Guide to Conversational AI; Digital Commons URI: Kingston, RI, USA, 2023. [Google Scholar]
- Bungert, K.; Bruckschen, L.; Müller, K.; Bennewitz, M. Robots in Education: Influence on Learning Experience and Design Considerations. In Proceedings of the 2020 European Conference on Education (ECE), London, UK, 16–19 July 2020; pp. 229–246. [Google Scholar] [CrossRef]
- Banaeian, H.; Gilanlioglu, I. Influence of the NAO robot as a teaching assistant on university students’ vocabulary learning and attitudes. Australas. J. Educ. Technol. 2021, 37, 71–87. [Google Scholar] [CrossRef]
- Buchem, I.; Thomas, E. A breathing exercise with the humanoid robot nao designed to reduce student stress during class: Results from a pilot study with students in higher education. In Proceedings of the 15th Annual International Conference of Education, Research and Innovation, Seville, Spain, 7–9 November 2022; pp. 6545–6551. [Google Scholar] [CrossRef]
- Baumann, A.E.; Goldman, E.J.; Meltzer, A.; Poulin-Dubois, D. People Do Not Always Know Best: Preschoolers’ Trust in Social Robots. J. Cogn. Dev. 2023, 24, 535–562. [Google Scholar] [CrossRef]
- Podpečan, V. Can You Dance? A Study of Child–Robot Interaction and Emotional Response Using the NAO Robot. Multimodal Technol. Interact. 2023, 7, 85. [Google Scholar] [CrossRef]
- Gressmann, A.; Weilemann, E.; Meyer, D.; Bergande, B. Nao Robot vs. Lego Mindstorms: The Influence on the Intrinsic Motivation of Computer Science Non-Majors. In Proceedings of the 19th Koli Calling International Conference on Computing Education Research, New York, NY, USA, 21–24 November 2019. [Google Scholar] [CrossRef]
- Valagkouti, I.A.; Troussas, C.; Krouska, A.; Feidakis, M.; Sgouropoulou, C. Emotion Recognition in Human–Robot Interaction Using the NAO Robot. Computers 2022, 11, 72. [Google Scholar] [CrossRef]
- Mutawa, A.M.; Al Mudhahkah, H.M.; Al-Huwais, A.; Al-Khaldi, N.; Al-Otaibi, R.; Al-Ansari, A. Augmenting Mobile App with NAO Robot for Autism Education. Machines 2023, 11, 833. [Google Scholar] [CrossRef]
- Berglez, D.; Kerneža, M. Integrating Generative Language Models in Lesson Planning: A Case Study; University of Maribor Press: Maribor, Slovenia, 2024; pp. 183–202. [Google Scholar] [CrossRef]
- Hang, C.N.; Wei Tan, C.; Yu, P.D. MCQGen: A Large Language Model-Driven MCQ Generator for Personalized Learning. IEEE Access 2024, 12, 102261–102273. [Google Scholar] [CrossRef]
- Burisch, C.; Bellary, A.; Breuckmann, F.; Ehlers, J.; Thal, S.C.; Sellmann, T.; Gödde, D. ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial. JMIR Res. Protoc. 2025, 14, e63887. [Google Scholar] [CrossRef] [PubMed]
- Zhang, R.; Zou, D.; Cheng, G. A review of chatbot-assisted learning: Pedagogical approaches, implementations, factors leading to effectiveness, theories, and future directions. Interact. Learn. Environ. 2023, 32, 4529–4557. [Google Scholar] [CrossRef]
- Liang, W.; Wu, Y. Exploring the use of ChatGPT to foster EFL learners’ critical thinking skills from a post-humanist perspective. Think. Ski. Creat. 2024, 54, 101645. [Google Scholar] [CrossRef]
Question | Conclusions | |
---|---|---|
Engagement and Enjoyment | NAO was overwhelmingly chosen as the most fun method (43 out of 50 students), far surpassing search engines and LLMs (3 and 4 votes, respectively). NAO’s interactive and humanoid characteristics resonate strongly with students, making it an engaging choice for activities prioritizing enjoyment. | |
Efficiency and Speed | The data on time consumption in method use reveals that students spent the least time using search engines (12), followed by LLMs (15), while the NAO robot required the most time (23). This suggests that search engines are the most time-efficient tool for retrieving information, likely due to their straightforward interface and familiar usage patterns.The NAO robot demands significantly more time. | |
Reliability and Familiarity | Search engines and LLMs were equally perceived as the most reliable methods (20 and 21 votes, respectively), with NAO trailing at 9 votes. Search engines were overwhelmingly seen as the most common method (47 votes), underscoring their familiarity and integration into students’ daily lives. Familiarity and reliability make search engines and LLMs dependable choices, while NAO might require additional training or familiarity to build trust. | |
Future Preferences | Students showed a balanced preference for using NAO (21 votes) and LLMs (18 votes) in future exercises, with search engines trailing (11 votes). While search engines remain a default choice, the novelty and interactive nature of NAO and the adaptability of LLMs appeal to students for future use. | |
Areas for Improvement | NAO was identified as the method most in need of improvement (26 votes), followed by LLMs (16 votes) and search engines (8 votes). NAO’s lower usability and efficiency likely drive this perception, indicating areas for refinement in its design and functionality. | |
Fatigue and Cognitive Load | NAO was reported as the most tiring method (30 votes), significantly more than search engines (11 votes) and LLMs (9 votes). The cognitive and physical effort required to interact with NAO may detract from its overall effectiveness, particularly for longer or more complex tasks. | |
Task Recommendations | Students recommended LLMs for difficult tasks (23 votes), followed by search engines (18 votes) and NAO (9 votes). For easier tasks, search engines and LLMs were nearly tied (17 and 26 votes, respectively), with NAO trailing (7 votes). LLMs’ adaptability and ability to handle complex queries make them the preferred choice for challenging tasks, while search engines remain a go-to option for straightforward needs. | |
Question | Graph | Conclusion |
---|---|---|
NAO’s Understanding of Human Speech | The responses indicate mixed satisfaction with NAO’s ability to understand students, with a significant portion (24) expressing dissatisfaction. This suggests that NAO’s auditory recognition system may need improvement to better meet user expectations. | |
Satisfaction with NAO’s Responsiveness | The majority of students were either somewhat or highly satisfied with NAO’s responses, indicating that while there is room for improvement, NAO generally provides acceptable answers to student queries. | |
Perceptions of NAO’s Auditory Improvements | Most students (29) perceive improvements in NAO’s auditory system, which could reflect gradual adaptation or better understanding of its limitations over time. | |
Perceptions of NAO’s Answer Quality Over Time | Responses are evenly distributed, with many students believing that NAO’s answers have room for improvement. This suggests that while NAO’s current performance is acceptable to some, others see significant potential for enhancement in its responses. | |
Enjoyment Added to Lessons by NAO’s Presence | The majority of students (35) found the interaction with NAO enjoyable, highlighting its ability to create an engaging classroom atmosphere. | |
Instances of NAO’s Inability to Listen | A high number of students (39) reported that NAO frequently failed to listen, indicating a critical area for improvement in its auditory responsiveness. | |
Instances of NAO’s Non-Responsiveness | The responses suggest that NAO occasionally fails to respond, which might disrupt the interaction flow and frustrate users. | |
NAO’s Hearing in Noisy Environments | The overwhelming majority (44) reported that NAO struggled to hear in noisy environments, emphasizing the need for better noise-cancellation features. | |
Beliefs About NAO’s Ability to Think | Most students (38) do not perceive NAO as capable of independent thought, reflecting its current limitations as a pre-programmed robot. | |
Enjoyment of Meeting NAO | The vast majority (43) enjoyed meeting NAO, showcasing its novelty and appeal as a humanoid robot in the classroom. |
Question | Graph | Conclusion |
---|---|---|
Willingness to Share Robot Interaction Experiences | Most students (31) would share their experience, suggesting that interacting with NAO is a unique and exciting event worth discussing. | |
Desire for NAO to Be Smarter | The majority (37) would prefer a smarter NAO, reflecting students’ desire for more advanced capabilities. | |
Expectations Regarding Robots Answering Questions | While many (22) expected NAO to answer questions, a significant portion (28) were either uncertain or skeptical, indicating mixed prior expectations. | |
Current Commonality of Talking to Robots | Responses suggest that robot interactions are not yet widely perceived as common, highlighting the novelty of such experiences. | |
Future Commonality of Talking to Robots | Most students (38) believe that robot interactions will become common, reflecting optimism about the future of robotics. | |
Desire for Robot Conversations to Be Common in the Future | A smaller majority (20) support the idea of frequent robot conversations, indicating some hesitation about widespread adoption. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Fragakis, N.; Trichopoulos, G.; Caridakis, G. Empowering Education with Intelligent Systems: Exploring Large Language Models and the NAO Robot for Information Retrieval. Electronics 2025, 14, 1210. https://doi.org/10.3390/electronics14061210
Fragakis N, Trichopoulos G, Caridakis G. Empowering Education with Intelligent Systems: Exploring Large Language Models and the NAO Robot for Information Retrieval. Electronics. 2025; 14(6):1210. https://doi.org/10.3390/electronics14061210
Chicago/Turabian StyleFragakis, Nikos, Georgios Trichopoulos, and George Caridakis. 2025. "Empowering Education with Intelligent Systems: Exploring Large Language Models and the NAO Robot for Information Retrieval" Electronics 14, no. 6: 1210. https://doi.org/10.3390/electronics14061210
APA StyleFragakis, N., Trichopoulos, G., & Caridakis, G. (2025). Empowering Education with Intelligent Systems: Exploring Large Language Models and the NAO Robot for Information Retrieval. Electronics, 14(6), 1210. https://doi.org/10.3390/electronics14061210