Enhancing Student Motivation and Engagement Through the Use of a Slovenian-Speaking Social Robot AlphaMini
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThis article investigates student performance within a knowledge management course under the condition of an AI robot's physical presence. The study focuses on an AI robot endowed with a more sophisticated repertoire of bodily and gestural expressiveness—an aspect that has received relatively scant attention in extant literature and thus constitutes a topic of notable scholarly relevance.
Nevertheless, the manuscript would benefit from several critical enhancements:
First, the linkage between the literature review and the proposed research hypotheses is tenuous, resulting in a weak theoretical foundation for the quantitative component of the inquiry. A more coherent integration of prior scholarship with the study's conceptual framework is essential to substantiate the hypotheses and ensure theoretical rigor.
Second, the sample exhibits a pronounced gender imbalance, with female participants constituting a substantial majority—approximately 5.8 times the number of male participants. Given established evidence that gender functions as a salient factor in technology acceptance and interaction, this asymmetry may compromise the reliability and generalizability of the findings.
Third, the disciplinary background of participants is limited to the humanities, social sciences, and management fields, with no representation from STEAM (Science, Technology, Engineering, Arts, and Mathematics) disciplines. Since prior research indicates that learners' academic domains can significantly influence their engagement with technological interventions, the absence of cross-disciplinary coverage undermines the robustness and external validity of the study.
Fourth, although the study purports to employ a mixed-methods approach, the qualitative component is underdeveloped. Key aspects such as research procedures lack clarity, collected qualitative data are not presented, and the analytical process remains inadequately described. These omissions diminish transparency and hinder the reproducibility of the research.
Fifth, there is a misalignment between the research questions and the methodological approach. The reported findings do not fully address the stated research questions. For instance, Research Question 1 (RQ1) asks: "How does the presence of the AlphaMini social robot in the classroom impact student engagement across all three dimensions (cognitive, behavioral, and emotional)?" Yet, the corresponding result—based on Pearson correlation analysis (Table 2)—merely indicates statistically significant relationships among the engagement dimensions, without elucidating the robot’s impact. Furthermore, in testing hypotheses H2–H4, the sample is divided into groups of N=22 and N=44, respectively. These subgroup sizes fall short of the thresholds required for reliable quantitative inference, thereby weakening the statistical power and credibility of the conclusions.
Sixth, while the study emphasizes the use of the AI robot within a Slovenian educational context, it fails to articulate why this specific sociocultural setting is essential or distinctive. No justification is provided for the necessity of conducting the research in Slovenia, nor is there any discussion of unique contextual factors that might render the Slovenian case particularly representative or exemplary. Consequently, the contextual specificity appears superfluous rather than integral to the research design.
In sum, while the topic holds promise, the study requires substantial revisions to strengthen its theoretical grounding, methodological rigor, and analytical coherence before it can make a compelling contribution to the field of educational technology and human-robot interaction in learning environments.
Author Response
Comments 1: First, the linkage between the literature review and the proposed research hypotheses is tenuous, resulting in a weak theoretical foundation for the quantitative component of the inquiry. A more coherent integration of prior scholarship with the study's conceptual framework is essential to substantiate the hypotheses and ensure theoretical rigor.
Response 1: We agree that the we did not sufficiently connect the literature review with the proposed hypotheses. In the revised manuscript, we have strengthened the theoretical foundation by explicitly linking prior scholar work to our conceptual framework and hypotheses. Specifically, we now emphasize that previous studies (e.g., Fridin, 2014; Belpaeme et al., 2018; Kanda et al., 2011) have consistently shown that social robots most strongly influence emotional and behavioral engagement, while cognitive effects tend to emerge more gradually. In addition, following the reviewer’s recommendation, we have also highlighted in the Results section that cognitive engagement was less pronounced than emotional and behavioral engagement, and in the Discussion we explicitly confirm this pattern by linking it to prior literature.This provides the theoretical justification for H1. Additionally, we draw on research by Woolf (2010) and Zhang & Aslan (2021), who highlighted the role of technology familiarity in shaping learner openness and engagement with robots, thereby substantiating H2–H4. These revisions ensure that our hypotheses are firmly grounded in existing theory and prior empirical findings, thereby enhancing the coherence and rigor of the quantitative component of our study.
Comments 2: Second, the sample exhibits a pronounced gender imbalance, with female participants constituting a substantial majority—approximately 5.8 times the number of male participants. Given established evidence that gender functions as a salient factor in technology acceptance and interaction, this asymmetry may compromise the reliability and generalizability of the findings.
Response 2: Indeed, the majority of participants were female, which reflects the broader enrollment structure of the programs from which our sample was drawn (educational sciences, teacher education, and management programs in Slovenia, where female participation is traditionally much higher). To support this claim, we have now added official statistics from the Statistical Office of the Republic of Slovenia (SURS). According to SURS (2023), women account for over 75% of students enrolled in education and teacher training programs and around 65% in business and administrative studies. Thus, while the gender imbalance in our sample may limit the generalizability of the findings, it mirrors the demographic composition of future educators and professionals who are most likely to interact with social robots in classroom practice. While this distribution may limit the generalizability of the findings, we consider it a strength in the context of our research focus, as it mirrors the demographic composition of future educators and professionals who are most likely to interact with social robots in classroom practice. To address this limitation, we have explicitly acknowledged the gender imbalance in the revised Conclusion and Limitations section and we caution that gender-related differences in technology acceptance should be explored more systematically in future research. Expanding the sample to include more balanced gender representation across disciplines would strengthen the external validity of the results. This was added to section Conslusion and limitation.
Comments 3: Third, the disciplinary background of participants is limited to the humanities, social sciences, and management fields, with no representation from STEAM (Science, Technology, Engineering, Arts, and Mathematics) disciplines. Since prior research indicates that learners' academic domains can significantly influence their engagement with technological interventions, the absence of cross-disciplinary coverage undermines the robustness and external validity of the study.
Response 3: We agree with the reviewer’s observation. Our sample was drawn primarily from educational sciences, management, and related fields, with no direct representation from STEAM disciplines. This limitation is a consequence of the purposeful sampling strategy, which targeted future educators and managers, as they are thematically aligned with the knowledge management content and pedagogical applications of social robots. While this approach ensured relevance to the study’s objectives, it inherently limits the external validity of our findings. To address this, we have added explicit acknowledgment in the Conclusion and Limitations section. This limitation is largely due to the purposeful sampling strategy, which focused on future educators and managers, groups that are thematically aligned with the knowledge management content and pedagogical applications of social robots. Nonetheless, we agree that disciplinary background can significantly shape how learners engage with technological interventions. To address this, we have added a statement in the Conslusion and limitation section, noting that the absence of STEAM participants restricts the external validity of our findings. Future studies should broaden the participant pool to include students from engineering, natural sciences, and other STEAM domains, where prior research suggests that engagement patterns may differ.
Comments 4: Fourth, although the study purports to employ a mixed-methods approach, the qualitative component is underdeveloped. Key aspects such as research procedures lack clarity, collected qualitative data are not presented, and the analytical process remains inadequately described. These omissions diminish transparency and hinder the reproducibility of the research.
Response 4: We agree that the qualitative component in the initial submission lacked sufficient detail and transparency. In the revised manuscript, we have substantially strengthened this part and explicitly framed it as a case study design within our mixed-methods approach. We now specify that qualitative data were collected from three complementary sources: (a) structured classroom observations recorded in field notes, (b) short post-session group discussions, and (c) open-ended responses in the online 1KA survey see Methodology section. Representative examples of student reactions and comments have been added to the Results section including observed laughter and surprise when AlphaMini spoke Slovenian, and student statements emphasizing the supportive and playful atmosphere created by the robot. We now explain that the qualitative notes and feedback were reviewed and organized according to the three engagement dimensions (behavioral, emotional, and cognitive). These observations were then compared with the survey findings to provide a fuller picture of student engagement.These revisions ensure that the qualitative dimension of our mixed-methods design is clearly described, adequately illustrated, and analytically robust.
Comments 5: Fifth, there is a misalignment between the research questions and the methodological approach. The reported findings do not fully address the stated research questions. For instance, Research Question 1 (RQ1) asks: "How does the presence of the AlphaMini social robot in the classroom impact student engagement across all three dimensions (cognitive, behavioral, and emotional)?" Yet, the corresponding result—based on Pearson correlation analysis (Table 2)—merely indicates statistically significant relationships among the engagement dimensions, without elucidating the robot’s impact. Furthermore, in testing hypotheses H2–H4, the sample is divided into groups of N=22 and N=44, respectively. These subgroup sizes fall short of the thresholds required for reliable quantitative inference, thereby weakening the statistical power and credibility of the conclusions.
Response 5: We thank the reviewer for this valuable comment. We agree that the original phrasing of RQ1 created an impression that the study could directly test the impact of the robot, which was beyond the scope of our design. To address this, we have revised RQ1 in the manuscript to focus on how student engagement is expressed across the three dimensions (cognitive, behavioral, and emotional) in the presence of the AlphaMini robot, rather than claiming to measure causal impact. This revised wording better reflects our methodological approach and the evidence we provide. We also acknowledge that the original analysis relied on Pearson correlations, which may not be optimal given the sample size and distributional assumptions. In the revised manuscript, we have therefore replaced Pearson’s test with the non-parametric Spearman rank-order correlation. The updated analysis (see Table 2) shows strong, statistically significant relationships between all three engagement dimensions. Specifically, behavioral engagement was highly correlated with both emotional engagement (ρ = .82, p < .01) and cognitive engagement (ρ = .73, p < .01), while emotional and cognitive engagement were also strongly correlated (ρ = .79, p < .01). This provides a more robust and appropriate representation of the interconnections among engagement dimensions. Regarding subgroup sizes, we agree that the division into N = 22 and N = 48 reduces statistical power and limits generalizability. We have explicitly acknowledged this limitation in the Conclusion and Limitations section and caution that these findings should be interpreted as preliminary. Nevertheless, we believe that the combination of quantitative trends and qualitative case study evidence offers meaningful insights that can inform future research with larger and more balanced samples.
Comments 6: Sixth, while the study emphasizes the use of the AI robot within a Slovenian educational context, it fails to articulate why this specific sociocultural setting is essential or distinctive. No justification is provided for the necessity of conducting the research in Slovenia, nor is there any discussion of unique contextual factors that might render the Slovenian case particularly representative or exemplary. Consequently, the contextual specificity appears superfluous rather than integral to the research design.
Response 6: In the revised manuscript, we have clarified why the Slovenian context is central to our study see Introduction. Specifically, this research was conducted as part of the national project Innovative Pedagogy 5.0, funded by the Slovenian Ministry of Education within the Recovery and Resilience Mechanism. The project’s aim is to transform teaching and learning practices by developing digital competences and integrating advanced technologies, including social robots, into Slovenian schools. We have further expanded the justification by highlighting distinctive sociocultural and policy-related factors: Policy relevance – The study directly contributes to the objectives of a government-funded initiative focused on innovation and digital transformation in education. Language and inclusion – Slovenia is a small-language country with limited availability of localized AI tools. AlphaMini’s ability to interact in Slovenian addresses equity and accessibility concerns that are highly relevant in this setting. Digital readiness and innovation culture – According to the European Commission (2024) and OECD (2022), Slovenia ranks around the EU average in digital development but has articulated ambitious goals in its Digital Slovenia 2030 Strategy, aiming to become one of the top five most digitalised European countries. AI research infrastructure – Slovenia hosts internationally recognized AI institutions, such as the UNESCO-affiliated International Research Centre on Artificial Intelligence (IRCAI) in Ljubljana and the EuroHPC Vega supercomputer in Maribor, which reinforce the country’s role as a hub for responsible and human-centric AI research. Representative value – Findings from Slovenia may serve as a reference for other small-language contexts, where similar challenges exist regarding the adoption of generative AI and social robots in education.
Reviewer 2 Report
Comments and Suggestions for AuthorsThe paper is worthy of publication with very minor additions.
The research presented in the article is very important both in its aims of making the educational process more attractive and successful by increasing motivation and engagement, and in its proposed technology to be used, i.e. ChatGPT (easily applicable to other LLMs) and a robot.
The paper is written in a clear and systematic way, and would be very interesting to a wide audience of educators.
My only recommendation is to be more detailed about the educational process and the technology used, in a few aspects as follows.
All we are told about the educational process is literally in three lines (lines 176-179 and 180-181) that tell us of its use and prior use but without any further description.
The robot used AlphaMini is quite interesting and funny but actually is greatly limited in its capabilities. I do not mean its size (of around 25 cm) but very limited graphical programming language (akin to Scratch), limited range of behaviours etc.
The characteristics that are helpful to educators and the specific use of the robot in this study should be described.
The use of ChatGPT should also be specified with some example of prompts and answers.
Maybe an example of lesson plan and lesson that used the technology should be added to the article.
Those additions, in my humble opinion, would turn the very good research into a very good article.
Author Response
Comments 1: All we are told about the educational process is literally in three lines (lines 176-179 and 180-181) that tell us of its use and prior use but without any further description.
Response 1: We agree that the original description of the educational process was too brief. In the revised manuscript, we have expanded this section to provide a clearer account of how the session was conducted. We add this in section 3.2. The process consisted of three main stages:
- Introduction – The robot was first introduced to the students, together with the topic of knowledge management and short instructions on how to interact with AlphaMini.
- Activity – Students then engaged with the robot directly. AlphaMini presented core concepts, demonstrated simple gestures, and invited students to respond and reflect. This interaction was intentionally open-ended, allowing students to explore the experience with minimal instructor intervention.
- Reflection and discussion – At the end of the session, students participated in a short group discussion about the strengths and weaknesses of using a social robot in the learning process. Their reflections provided valuable qualitative data for the study.
This extended description is now included in the Methodology section, ensuring greater transparency about the structure of the educational activity.
Comments 2:The robot used AlphaMini is quite interesting and funny but actually is greatly limited in its capabilities. I do not mean its size (of around 25 cm) but very limited graphical programming language (akin to Scratch), limited range of behaviours etc.
Response 2: We agree that AlphaMini has several limitations, including its restricted programming environment, a limited repertoire of pre-programmed behaviors, and reliance on external AI services to extend its conversational abilities. In the revised manuscript, we have explicitly acknowledged these constraints in the Conclusion and Limitations section. At the same time, we highlight that these technical limitations also shaped the design of our study. Rather than evaluating AlphaMini’s programming depth, the focus was on its social and linguistic presence in Slovenian, combined with its basic gestural and expressive features. Even with limited behavioral complexity, these functions were sufficient to elicit emotional and behavioral engagement from students. We have now clarified this point to make the scope of the study more transparent.
Comments 3:The characteristics that are helpful to educators and the specific use of the robot in this study should be described.
Response 3: In the revised manuscript, we have expanded the Methodology section to provide a clearer description of both the robot’s key characteristics and its specific use in the study. In particular, we now explain that AlphaMini’s Slovenian-language speech capabilities, expressive gestures, and supportive verbal cues were central features for educators in this context. These functions allowed the robot to introduce the topic of knowledge management, encourage participation by asking simple questions, and moderate transitions during the activity. At the end of the session, AlphaMini also facilitated reflection by inviting students to share what they found positive or challenging about the experience. This additional detail clarifies which aspects of the robot were most pedagogically useful and how they were applied during the learning process.
Comments 4:The use of ChatGPT should also be specified with some example of prompts and answers.
Response 4: In the revised manuscript, we have provided a detailed description of how ChatGPT was integrated into the study and added examples of student prompts and robot responses see Methodology section. To enable natural spoken interaction in Slovenian, we developed our own modules for speech recognition, speech detection, and speech synthesis. These modules processed students’ spoken queries in Slovenian, converted them into text, forwarded them to ChatGPT, and then vocalized the generated responses through AlphaMini. This technical pipeline ensured that all interaction was conducted in Slovenian, a feature not natively supported by the robot. Importantly, students were free to formulate their own questions without constraints. Most queries were related to management and economics, reflecting the course context. For example: Student question (spoken): “Kako lahko podjetje najbolje organizira deljenje znanja med zaposlenimi?” Robot/ChatGPT answer (spoken): “Podjetje lahko vzpostavi digitalno platformo za deljenje dokumentov, redne sestanke znanja in spodbuja kulturo sodelovanja, kjer so zaposleni nagrajeni za deljenje idej.” By developing our own Slovenian-language speech processing pipeline, we ensured that students could interact with the robot naturally in their native language, which was critical for emotional engagement and inclusivity.
Comments 5:Maybe an example of lesson plan and lesson that used the technology should be added to the article.
Response 5: In this study, however, the activity was not based on a predefined lesson plan but rather designed as an exploratory session where students themselves generated questions related to management and education. The purpose was to observe how spontaneous, student-driven interaction with AlphaMini would affect engagement, rather than to test a scripted teaching scenario. We have clarified this point in the revised Methodology section, where we now explain that the robot introduced the topic and interaction guidelines, while the core of the session consisted of students asking their own questions to AlphaMini, followed by a concluding group discussion.
Round 2
Reviewer 1 Report
Comments and Suggestions for Authors thank for the authors' detailed rejoinder to my previous review. Having re-examined the revised manuscript carefully, I note that the response concentrates almost exclusively on the quantitative component and is largely explanatory rather than remedial: no additional data have been supplied, the sample remains unchanged, and the conclusions have not been materially altered. Before the paper can be recommended for publication, I would welcome a further round of dialogue on the following points:- Research methodology (Section 3). The authors reiterate that “This study employs a case-study design, complemented by quantitative methodology and structured classroom observation.” Yet, as observed in my earlier report, the manuscript still fails to provide a rigorous account of (a) the step-by-step procedures through which the qualitative," observational data“ were collected, (b) the cleaning and analytical decisions applied to those data, and (c) the strategies employed to ensure qualitative reliability and validity. These omissions compromise the study’s methodological transparency and replicability.
- Theoretical contribution. The Discussion competently links the present findings to prior literature, but every comparison is confirmatory: no results contradict, refine, or extend previous work. Readers are therefore left asking what the study adds to existing theory or pedagogical practice. I urge the authors to distil the unique theoretical insights of the investigation and to articulate—explicitly and succinctly—how these insights advance the field. A parallel task is to demonstrate that the chosen design is not merely rougher or more abbreviated than antecedent studies, but optimally suited to the research questions.
- Alignment between results and discussion. The arguments addressing RQ1 and RQ2 are only loosely tethered to the empirical findings, probably because the quantitative and observational results are reported in skeletal form. A tighter coupling is required: each interpretative claim should be anchored in—and clearly sign-posted to—the relevant data.
Author Response
Comments 1: Research methodology (Section 3). The authors reiterate that “This study employs a case-study design, complemented by quantitative methodology and structured classroom observation.” Yet, as observed in my earlier report, the manuscript still fails to provide a rigorous account of (a) the step-by-step procedures through which the qualitative," observational data“ were collected, (b) the cleaning and analytical decisions applied to those data, and (c) the strategies employed to ensure qualitative reliability and validity. These omissions compromise the study’s methodological transparency and replicability.
Response 1: In the revised manuscript, we have clarified that the observational component was exploratory and descriptive in nature, rather than a fully structured ethnographic study. Our goal was not to develop a comprehensive qualitative coding framework, but to capture visible, immediate reactions of students to the robot interaction. To increase transparency, we now described the procedure step by step in Methodolgy section. The research team acted as non-participatory observers, in line with basic field observation practices (Spradley, 1980). Handwritten field notes were taken during the session, documenting laughter, hesitation, eye contact, spontaneous questions, and other salient behaviors. Notes from the post-session group discussion were also recorded. After the session, all notes were digitized, cleaned, and prepared: duplicates and unclear entries were removed, and related items were clustered (Miles, Huberman, & Saldaña, 2014). Data were then organized thematically around behavioral, emotional, and cognitive engagement (Braun & Clarke, 2006, 2021). We explicitly acknowledge that no formal coding software was used and that reliability was instead supported through team-based discussion and triangulation with survey results (Lincoln & Guba, 1985; O’Brien et al., 2014; Creswell & Plano Clark, 2018). This reflexive approach aligns with previous educational robotics studies that employed exploratory observation to contextualize quantitative findings (Fridin, 2014; Kory-Westlund & Breazeal, 2019). We have also added text in limitation section.
Comments 2:Theoretical contribution. The Discussion competently links the present findings to prior literature, but every comparison is confirmatory: no results contradict, refine, or extend previous work. Readers are therefore left asking what the study adds to existing theory or pedagogical practice. I urge the authors to distil the unique theoretical insights of the investigation and to articulate—explicitly and succinctly—how these insights advance the field. A parallel task is to demonstrate that the chosen design is not merely rougher or more abbreviated than antecedent studies, but optimally suited to the research questions.
Response 2: We have revised the Discussion to explicitly articulate the unique theoretical contributions of our study. Specifically, we emphasize: the role of linguistic and cultural localization (Slovenian-speaking robot) as a novel factor in fostering emotional engagement, the synergy of generative AI and embodied robotics, refining the concept of social presence, the identification of technology familiarity as a moderator of engagement, and the extension of robot-assisted learning research to abstract and higher-education contexts. We also clarify that our exploratory case study design was intentionally chosen to capture these multidimensional effects, rather than being a simplified version of previous studies.
We add the following literature that support this:
- Linguistic/cultural localization in small-language contexts enhances emotional engagement (Louie, 2021).
- Generative AI plus embodied robotics enrich social presence beyond scripted models (Pinto-Bernal, 2025).
- Prior technology familiarity moderates engagement (Sanders et al., 2017).
We emphasize that very limited prior research exists on Slovenian-speaking social robots or more generally on embodied AI in small-language contexts. This makes our case study distinctive, although we also acknowledge that findings from a relatively small and context-specific sample cannot be generalized to broader educational settings. Second, we clarify that our study focused specifically on student motivation and engagement across three dimensions (behavioral, emotional, cognitive), which has rarely been done in the context of robot-assisted learning with abstract subjects such as knowledge management. Previous studies (e.g., Fridin, 2014; Belpaeme et al., 2018; Kory-Westlund & Breazeal, 2019) have primarily focused on younger children, STEM, or language learning, while our findings extend this line of research to higher education and conceptual content. Finally, we make clear that our exploratory case study design was intentionally chosen as the most appropriate way to capture these multidimensional and context-specific phenomenon. While more structured designs and larger samples are needed in future research, our approach was optimally suited for revealing how cultural-linguistic localization, generative AI integration, and prior familiarity intersect to influence student engagement.
Comments 3:Alignment between results and discussion. The arguments addressing RQ1 and RQ2 are only loosely tethered to the empirical findings, probably because the quantitative and observational results are reported in skeletal form. A tighter coupling is required: each interpretative claim should be anchored in—and clearly sign-posted to—the relevant data.
Response 3: In the revised Discussion, we strengthened the alignment between data and interpretation. Each argument is now explicitly anchored in the quantitative results (Tables 2–4) and in the observational/discussion notes. For example, we highlight that behavioral engagement was significantly higher among students with prior robot experience (Table 3–4) and support this with representative observational quotes. This tighter coupling ensures that all interpretative claims are clearly signposted to the relevant empirical findings.
