Fostering Student Engagement and Learning Perception Through Socratic Dialogue with ChatGPT: A Case Study in Physics Education

Abstract

This classroom-based case study examines how an AI-mediated Socratic dialogue, implemented through ChatGPT, can support students’ engagement and perceived learning in undergraduate thermodynamics. Conducted in a first-year engineering physics course at a private university in northern Mexico, the activity invited small student groups to interact with structured prompts designed to promote inquiry, collaboration, and reflective reasoning about the adiabatic process. Rather than functioning as a source of answers, ChatGPT was intentionally positioned as a mediating scaffold for Socratic questioning, prompting students to articulate, examine, and refine their reasoning. A mixed-methods approach was employed, combining a 10-item Likert-scale survey with construct-level statistical analysis of two focal dimensions: perception of learning and engagement, including an exploratory comparison by gender. Results indicated consistently high levels of perceived learning and engagement across the cohort, with average scores above 4.5 out of 5. At the construct level, no statistically significant gender differences were observed, although a single item revealed higher perceived learning among female students. Overall, the findings suggest that the educational value of ChatGPT in this context emerged from its integration within a Socratic, inquiry-oriented pedagogical design, rather than from the technology alone. These results contribute to ongoing discussions on the responsible and pedagogically grounded integration of generative AI in physics education and align with Sustainable Development Goal 4 (Quality Education).

1. Introduction

The use of artificial intelligence (AI) in education is expanding the horizons of instructional design and student engagement. Particularly in STEM areas, AI technologies help identify obstacles in learning and deliver timely, customized feedback (Kamalov et al., 2023; Qusef et al., 2025; Wollny et al., 2021). These innovations are particularly relevant for promoting inclusive and high-quality learning experiences, in line with Sustainable Development Goal 4 (SDG 4) (UNESCO, 2022).

Among recent developments, generative AI models such as ChatGPT offer new possibilities for dynamic and conversational interactions that support inquiry, reflection, and critical thinking (Adıgüzel et al., 2023; Gervacio, 2024; Liu et al., 2025; Pozdniakov et al., 2024; Tang & Zhang, 2023; Wan & Chen, 2024). However, most existing applications of ChatGPT in education focus on content delivery or individual question-answering, with limited attention to collaborative, conceptually driven dialogue embedded in classroom practice (Kamalov et al., 2023; Lo, 2023; Manprisio, 2024; Pozdniakov et al., 2024).

Physics education faces persistent challenges to foster deep understanding, especially in domains involving abstract processes such as energy transformations (Neumann et al., 2013). The adiabatic process is one such topic, where students often have strong misconceptions: they confuse it with isothermal transformations or fail to link work and internal energy changes (Loverude et al., 2002).

Predominantly lecture-based instruction and procedurally focused problem-solving activities, including lectures and traditional problem sets, are often insufficient to promote conceptual coherence, particularly when phenomena are difficult to visualize or experiment with in class (McDermott, 1999; Meltzer, 2012; Redish, 1999). Inquiry-oriented pedagogical approaches, such as Socratic dialogue, have been shown to support deeper reasoning by encouraging students to articulate, question, and refine their ideas.

To address these issues, this study explores the use of ChatGPT as a Socratic tutor: not as a content-delivery tool, but as an AI-mediated facilitator that operationalizes Socratic dialogue through guided questioning and dialogic inquiry. The activity focused on helping students collaboratively make sense of adiabatic processes through structured prompts and small-group interaction. In this design, the pedagogical logic is provided by the Socratic method, while ChatGPT functions as a mediating scaffold that supports consistent and responsive questioning within the classroom context.

Our aim was to investigate students’ perceptions of learning and engagement during this dialogic learning experience, with particular attention to potential gender-related differences. As a classroom-based case study, this work does not seek to measure objective learning gains or conceptual mastery, but rather to document how students experience and interpret the integration of generative AI within an inquiry-based physics activity. By shedding light on students’ experiences, this study offers replicable design insights to inform institutional strategies for the thoughtful and pedagogically grounded integration of AI tools in higher education.

2. Materials and Methods

2.1. Research Design

This study used a classroom-based qualitative case study methodology with complementary quantitative analysis to explore how generative AI, specifically ChatGPT (OpenAI, 2025), can support students’ perceived learning and engagement during collaborative inquiry into adiabatic processes. The intervention was implemented in a first-year engineering physics course at a private university in Mexico, with 53 students participating in a guided, small-group activity mediated by ChatGPT. The pedagogical design was grounded in the Socratic method as an inquiry-oriented instructional strategy. Students worked in small teams of two to three, interacting with ChatGPT via a series of structured prompts designed to encourage guided questioning, articulation of reasoning, and metacognitive reflection, rather than direct answer provision. The activity was designed to stimulate reflection on the relationships among work, heat, pressure, volume, and internal energy under adiabatic conditions. In this design, the Socratic method provides the pedagogical framework, while ChatGPT functions as a mediating technological scaffold that operationalizes dialogic inquiry in a consistent and responsive manner within the classroom context. The study does not seek to isolate the effects of AI or pedagogy independently, but rather to examine their intentional integration in a real instructional setting. This study was guided by the following research questions, which aim to explore students’ perceptions and experiences during an AI-mediated, Socratic-style learning activity. Consistent with the exploratory nature of a case study, the research questions focus on perceived learning and engagement rather than objective learning gains. Rather than assuming positive outcomes, the questions are designed to investigate how students interacted with ChatGPT, how they engaged with the activity, and whether any gender-related patterns emerged:

• (RQ. 1) How do students perceive the use of ChatGPT as a Socratic tutor in a physics course?
• (RQ. 2) What impact does the use of ChatGPT have on student engagement during class activities?
• (RQ. 3) Can the use of ChatGPT as a Socratic tutor support more equitable learning experiences across genders in physics education?

2.2. Participants

The study involved 53 undergraduate students enrolled in a physics course at a private university in northern Mexico. Participants came from seven different engineering programs and all shared a common foundational physics curriculum, taking part in the activity during a regular in-class session. The distribution of participants by academic program is summarized in

Table 1. Participant distribution by academic program.

Academic Program	Number of Participants
Biomedical Engineering	4
Civil Engineering	5
Industrial Engineering	14
Business management Engineering	14
Administrative Mechanical Engineering	5
Automotive Engineering	3
Robotics and Mechatronics Engineering	8
Total	53

.

2.3. Procedure

Students were organized into small groups of two or three to engage collaboratively with ChatGPT through a structured prompt designed to elicit conceptual reasoning about adiabatic processes. The scenario introduced gas compression and required students to consider whether heat is transferred and which factors influence changes in internal energy and temperature.

The prompt instructed the AI to adopt a Socratic role: “Act like a Socratic instructor in thermodynamics. I am trying to understand the adiabatic process. Don’t give me direct answers—guide me with questions that help me discover the concepts for myself.” Prior to the activity, students received a brief orientation on effective interaction strategies with the AI. The objective was not to obtain correct answers, but to engage in dialogic reasoning, hypothesis generation, and self-reflection.

Students were asked to deliberate within their groups and reach consensus before submitting responses to ChatGPT. This collaborative dimension ensured that the interaction transcended individual dialogue, fostering negotiation of meaning and collective knowledge construction. ChatGPT, framed as a Socratic tutor, provided cognitive scaffolding by posing guiding questions, challenging assumptions, and prompting elaboration. Rather than delivering content, the AI supported students in identifying inconsistencies in their reasoning and refining their conceptual explanations, thereby supporting students’ regulation of their own learning.

The instructor acted as a facilitator, ensuring productive engagement with both peers and the AI by monitoring group progress, encouraging equitable participation, and supporting students in interpreting and critically evaluating AI-generated prompts. The instructor did not provide direct conceptual answers during the group–ChatGPT interactions; instead, facilitation focused on maintaining productive dialogue, for example by prompting students to justify their claims, encouraging quieter group members to contribute, and reminding groups to critically assess AI outputs. When AI responses were unclear or off-target, students were encouraged to rephrase prompts and cross-check their reasoning with course principles. The session was structured into brief phases (orientation, small-group interaction with ChatGPT, and whole-class consolidation), and concluded with a whole-class discussion aimed at situating individual and group discoveries within the broader conceptual framework of thermodynamics, thereby supporting replication in similar course settings.

2.4. Instruments and Data Collection

A mixed-methods approach was used to collect data, combining quantitative and qualitative strategies to provide a comprehensive understanding of the learning experience. Specifically, after completing the Socratic tutor activity, participants responded to a 10-item Likert-scale survey designed to measure perception of learning and engagement. Although items refer to learning, confidence, and participation, all responses are interpreted as self-reported perceptions rather than objective measures of conceptual understanding. The quantitative analysis focused on identifying overall trends and potential gender-related differences at the construct level, while open-ended reflections provided complementary qualitative insights into how students experienced the dialogic interaction. This combination of approaches ensured that the study captured not only measurable patterns but also students’ subjective perspectives, thereby strengthening the interpretive depth of the findings.

Surveys and Qualitative Data

The instruments used to explore the students’ experiences and perceptions were:

• Learning Perception Survey: A 10-item, 5-point Likert-type questionnaire adapted from the survey proposed by (Diemer et al., 2012) was used to explore the students’ experiences and perceptions. The survey items were organized into two dimensions: perception of learning (Q1–Q6) and engagement (Q7–Q10).

  Q1.

  The ChatGPT activity helped me apply course content to solve problems.

  Q2.

  The ChatGPT activity helped me learn the course content.

  Q3.

  The ChatGPT activity helped me connect ideas in new ways.

  Q4.

  The ChatGPT activity helped me participate in the course activity in ways that enhanced my learning.

  Q5.

  The ChatGPT activity helped me develop confidence in the subject area.

  Q6.

  The ChatGPT activity helped me develop skills that apply to my academic career and/or professional life.

  Q7.

  The ChatGPT activities motivated me to learn the course material more than class activities that did not use the ChatGPT.

  Q8.

  I participated more in class during the ChatGPT activities than during activities that did not use the ChatGPT.

  Q9.

  My attention to the task(s) was greater using the ChatGPT. activities.

  Q10.

  It was easier to work in a group using the ChatGPT than in other group activities.

  The survey was structured into two dimensions. The first six items (Q1–Q6) assess students’ perception of learning, while the last four items (Q7–Q10) capture their perception of engagement. These two dimensions were analyzed separately in order to identify potential differences between how students perceived the learning outcomes and how they experienced engagement during the activities.
• Qualitative Google Form: This was a digital form with four open-ended questions designed to explore students’ perceptions of the experience. This form was completed voluntarily and anonymously at the end of the activity. Qualitative responses were analyzed thematically and interpreted in relation to the two focal constructs: perception of learning and engagement. The questions were as follows:
  • How would you describe your overall experience of interacting with ChatGPT during the adiabatic process activity?
  • What differences did you notice between this activity and others you have done in previous physics classes?
  • Did you find that you were more attentive or focused during this activity, and what helped or hindered that?
  • Would you recommend this strategy to other courses or subjects? What would you change?

3. Results

Table 2. Descriptive statistics of survey responses (Q1–Q10).

Item	N	Mean	SD	SE	CI
Q1	53	4.70	0.54	0.07	0.15
Q2	53	4.75	0.52	0.07	0.14
Q3	53	4.75	0.48	0.07	0.13
Q4	53	4.77	0.42	0.06	0.12
Q5	53	4.57	0.64	0.09	0.18
Q6	53	4.58	0.63	0.09	0.18
Q7	53	4.21	0.93	0.13	0.26
Q8	53	3.75	1.18	0.16	0.32
Q9	53	3.72	1.10	0.15	0.30
Q10	53	4.26	0.84	0.11	0.23
Perception of Learning	53	4.69	0.39	0.05	0.11
Engagement	53	3.98	0.76	0.10	0.21

summarizes descriptive statistics for the full sample (N = 53). Overall, mean ratings were high across all items, particularly for Perception of Learning (Q1–Q6). By contrast, Engagement (Q7–Q10) showed comparatively lower mean values and greater variability, suggesting that engagement was more heterogeneous and context-sensitive than perceived learning in this activity.

3.1. Perception of Learning

Regarding the perception of learning, as shown in

Table 3. Descriptive statistics of survey responses (Q1–Q10) by gender.

Item	Female M	SD	SE	CI	Male M	SD	SE	CI
Q1	4.86	0.36	0.10	0.21	4.64	0.58	0.09	0.19
Q2	5.00	0.00	0.00	0.00	4.67	0.58	0.09	0.19
Q3	4.79	0.43	0.11	0.25	4.74	0.50	0.08	0.16
Q4	4.86	0.36	0.10	0.21	4.74	0.44	0.07	0.14
Q5	4.79	0.43	0.11	0.25	4.49	0.68	0.11	0.22
Q6	4.79	0.43	0.11	0.25	4.51	0.68	0.11	0.22
Q7	4.50	0.76	0.20	0.44	4.10	0.97	0.15	0.31
Q8	4.07	1.21	0.32	0.70	3.64	1.16	0.19	0.38
Q9	4.07	1.07	0.29	0.62	3.59	1.09	0.18	0.35
Q10	4.14	0.86	0.23	0.50	4.31	0.83	0.13	0.27
Perception of Learning	4.85	0.25	0.07	0.14	4.63	0.42	0.07	0.14
Engagement	4.20	0.72	0.19	0.41	3.91	0.77	0.12	0.25

, both female and male students reported consistently high scores. Female students’ means ranged from 4.79 to 5.00, with minimal variability in some items (e.g., Q2 = 5.00, SD = 0.00). Male students showed means between 4.49 and 4.74, with moderate variability (SD between 0.44 and 0.68). These results suggest that, overall, students perceived the ChatGPT activity as highly supportive of their learning, contributing to content understanding, skill development, and confidence in the subject area. Gender differences were minimal, although female students tended to report slightly higher ratings across most items.

When considered as a composite construct (Q1–Q6), Perception of Learning showed high mean values for both female and male students, reflecting a strong and consistent perception that the ChatGPT-mediated Socratic dialogue supported meaningful learning. Item-level variation was minimal, and gender differences were small, reinforcing the interpretation that perceived learning benefits were broadly shared across the sample.

3.2. Engagement

With respect to engagement, as we can see in Table 3, scores were moderately lower and showed greater variability compared to the learning dimension. Among female students, the means ranged from 4.07 to 4.50 (SD between 0.76 and 1.21), whereas the means for male students ranged from 3.59 to 4.31 (SD between 0.83 and 1.16). Items related to sustained attention (Q9) and group work (Q10) yielded lower ratings, particularly among male students, indicating a more heterogeneous perception of engagement. Nevertheless, average engagement scores for both groups remained above the scale midpoint, reflecting an overall positive evaluation of the activity in terms of participation and motivation. At the construct level (composite score Q7–Q10), no statistically significant gender differences were observed, suggesting that the AI-mediated Socratic activity supported comparable levels of engagement across genders.

When analyzed at the construct level using the composite Engagement score (Q7–Q10), both female and male students reported average values above the scale midpoint (Table 3), reflecting an overall positive evaluation of the activity in terms of participation, interest, and motivation. Importantly, no statistically significant gender differences were observed at the construct level (Table 2), suggesting that the AI-mediated Socratic activity supported broadly comparable levels of engagement across genders, despite item-level variability. Taken together, these findings indicate that while engagement was perceived as slightly more variable and context-sensitive than learning, the overall pattern supports the conclusion that ChatGPT-mediated Socratic dialogue can foster sustained student involvement and motivation in physics learning activities across diverse student groups.

3.3. Item-Level Gender Comparisons

Survey results revealed generally high ratings across all items, with mean values close to or above 4.5 for both female and male students. As illustrated in Figure 1, female students consistently rated the activity slightly higher than male students across most items. At the item level, the most pronounced difference was observed in Q2 (perceived learning), where the mean score for female students reached 5.00, compared to 4.67 for male students.

Figure 1 displays the mean scores and standard errors (Mean ± SE) for each survey item (Q1–Q10), disaggregated by gender. The overlap of error bars across most items suggests that differences were generally not statistically significant. This visual pattern supports the interpretation that perceptions of learning and engagement were broadly similar across genders, with the exception of a single item.

Statistical comparisons presented in

Table 4. Statistical comparison of survey items by gender (female $n=14$, male $n=39$). Significant differences are marked with *.

Item	Female (n)	Male (n)	df	p Value	Sig.
Q1	14	39	320	0.222	ns
Q2	14	39	350	0.0287	*
Q3	14	39	279	0.879	ns
Q4	14	39	304	0.396	ns
Q5	14	39	332	0.159	ns
Q6	14	39	326	0.208	ns
Q7	14	39	334	0.188	ns
Q8	14	39	336	0.190	ns
Q9	14	39	340	0.165	ns
Q10	14	39	241	0.490	ns
Perception of Learning	14	39	197	0.140	ns
Engagement	14	39	210	0.208	ns

confirmed this observation. With the exception of Q2, no statistically significant gender differences were detected. Female students reported significantly higher perceived learning in Q2 (“The ChatGPT activity helped me learn the course content”), with $p=0.0287$, while all remaining item-level comparisons were not statistically significant. Minor trends favoring female students were observed in items related to collaboration and conceptual understanding (Q1, Q4, Q7–Q9), whereas male students reported marginally higher scores in Q10; however, these differences did not reach statistical significance.

Given the exploratory nature of this classroom-based case study, item-level results are interpreted as descriptive trends rather than confirmatory evidence. Overall, the findings indicate that ChatGPT-mediated Socratic dialogue was perceived positively by both female and male students, supporting generally equitable perceptions of learning and engagement, with only isolated item-level differences observed.

3.4. Qualitative Google Form

The qualitative survey provided complementary insights into students’ experiences. Across responses, students consistently described the activity as novel, interactive, and cognitively demanding, particularly in comparison with traditional lecture-based instruction. Many students emphasized the importance of being required to think and articulate reasoning, rather than receiving direct answers. Representative comments included:

“I realized I had misunderstood how work affects internal energy in adiabatic processes. ChatGPT didn’t tell me the answer, but kept asking until I figured it out.”

“It was different because I had to think. ChatGPT asked questions that made me reconsider what I thought I knew. I was more alert.”

“I learned that for the process to be close to adiabatic, there must be no leakage to increase the pressure, which would raise the temperature. The piston should move fast enough so that there is no temperature exchange, and that the difference in internal energy for an adiabatic process is directly related to the work done on the system.”

Students also reported that the activity helped them clarify conceptual relationships, particularly between work, internal energy, and temperature in adiabatic processes. These qualitative responses align with the high quantitative ratings for perceived learning. Regarding engagement, several students noted that the conversational and dialogic nature of the interaction helped sustain attention, although a minority mentioned occasional distractions related to prompt formulation or interpretation of AI responses. Most participants indicated that they would recommend this strategy for other courses or subjects, particularly if complemented by additional instructor-guided discussion.

3.5. Facilitators and Obstacles of the Socratic Style

Students identified several key aspects of the ChatGPT-mediated Socratic method that enhanced their learning:

• Progressive questioning: The chained structure of the questions helped guide reasoning in incremental steps, which students found helpful for unpacking complex concepts.
• Cognitive pause: The written format allowed students to take time before responding, fostering more deliberate thinking.
• Immediate feedback: The interaction often led students to revise their ideas based on new insights triggered by the next question or comment.

These facilitators align with established principles of Socratic dialogue and inquiry-based learning, particularly scaffolding, clarification, and self-correction. Challenges were also identified. Approximately 18% of students reported moments where the AI misinterpreted their responses, disrupting conversational flow. Others found some questions too vague or repetitive, especially when prior knowledge was stronger. Despite these limitations, the overall perception of the Socratic style remained strongly positive, and a majority of students recommended the approach for use in other courses and disciplines.

4. Discussion

This study explored the potential of using ChatGPT as a Socratic tutor to support students in reasoning about the adiabatic process. The findings indicate that AI-mediated Socratic dialogue was perceived by students as supportive of their learning and engagement, and that it fostered agency and metacognitive involvement during collaborative inquiry. By encouraging students to move beyond procedural problem-solving, the AI prompted them to recognize and correct misconceptions—such as confusing isothermal and adiabatic transformations—and to develop more coherent mental models of the adiabatic process. This iterative, reflective interaction was reported by students as promoting deeper reasoning, aligning with expert-like problem-solving practices and contributing to a more meaningful and self-regulated learning experience.

These outcomes align with educational goals under SDG 4, emphasizing critical thinking and deeper understanding through innovative methods (AlAli et al., 2023; Kioupi & Voulvoulis, 2019). While the use of ChatGPT revealed some limitations, such as occasional misinterpretations by the AI, the overall findings position generative AI not merely as a mediating scaffold for pedagogical innovation when embedded within a clear instructional design.

The findings of this study highlight how AI-mediated dialogue can foster a more personalized and cognitively demanding learning environment predominantly lecture-based, procedurally focused instruction. Students reported feeling both challenged and supported, and the qualitative evidence suggests that the activity helped them recognize not only what they misunderstood but also how their reasoning had gone astray—an outcome aligned with the goals of metacognitive development (Schraw & Dennison, 1994; Sharma et al., 2024). Importantly, the contribution of this case study lies in documenting students’ perceptions of learning and engagement, showing that structured, dialogic interactions can promote epistemic agency and encourage students to take ownership of their reasoning while strengthening critical thinking skills in science education. In this sense, the present study classroom-based empirical insights to a growing body of scholarship suggesting that generative AI, when used as a facilitator of Socratic inquiry, may extend beyond content delivery to cultivate higher-order learning outcomes.

More broadly, these results reflect ongoing discussions about the transformative potential of artificial intelligence in education. AI systems have been shown to facilitate student-centered learning and adaptive assessments, as well as providing real-time feedback to improve engagement and motivation (Banik & Gullapelly, 2025; Cha & Daud, 2025; Jumah et al., 2024). At the same time, they may also expand accessibility, particularly for learners who require individualized support (Mustafa, 2024). However, the growing reliance on AI-based platforms also raises concerns about issues such as data privacy, unequal levels of digital literacy, and the potential reduction of human interaction in the classroom. These tensions underscore the need for a balanced approach in integrating AI into pedagogical practice—one that leverages its capacity to enrich learning while maintaining the social and relational dimensions that are central to education (Mustafa, 2024).

The results revealed a varied pattern of student responses to different conceptual aspects of adiabatic processes. The post-activity Likert-type survey revealed that students perceived the interaction with ChatGPT as beneficial for clarifying core concepts of the adiabatic process. Many students specifically highlighted perceived improvement in understanding of the relationship between work and internal energy, a known conceptual hurdle in thermodynamics (Loverude et al., 2002). Several responses mentioned that while students had previously memorized equations, they had lacked intuitive understanding—something the Socratic dialogue helped them articulate and refine.

However, the use of ChatGPT also revealed some limitations that merit further examination. First, the model occasionally failed to interpret students’ inputs correctly, which disrupted the dialogic flow and led to frustration. Second, the level of questioning did not always align with the students’ prior knowledge—some found the questions too simplistic or redundant, while others felt they needed more scaffolding (Fakour & Imani, 2025; Wang & Fan, 2025). This issue reflects a broader challenge in the deployment of large language models in education: their current inability to fully diagnose learners’ understanding in real time and adjust their responses accordingly (Fan et al., 2025; Tam, 2025; Zhan & Yan, 2025).

Table 5. Comparative Table of Studies on the Use of ChatGPT in Physics Education.

Reference	Pedagogical Approach	Use of ChatGPT	Main Objective	Evaluation Methodology	Key Findings	Student Perception
El Fathi et al. (2025)	Inquiry-based learning using the CILP framework	Guided assistant providing structured prompts and conceptual tasks	Enhance conceptual understanding and address misconceptions in thermodynamics	Pre- and post-tests, Likert surveys, and cluster analysis	Significant reduction of misconceptions and high student acceptance of ChatGPT	High acceptance and positive perception
This work	Socratic dialogue promoting metacognitive and inquiry-oriented reasoning through open-ended questions	ChatGPT acts as a mediating scaffold for reflective dialogue, rather than a content provider	Perception survey (learning and engagement constructs) and qualitative reflections (case study)	High levels of perceived learning and engagement; minimal gender differences at the construct level	Perceived as supportive for reflection, dialogue, and conceptual clarification	Perceived as useful and supportive for deeper understanding
Beltozar-Clemente and Díaz-Vega (2024)	Combination of generative AI and gamification to increase motivation and engagement	ChatGPT provides real-time feedback and assistance during game-based activities	Improve academic performance and student motivation in Physics 1	Comparison of exam scores and Likert-type surveys on motivation	Higher performance in the experimental group and increased interest in the subject	Highly motivated and positive perception of ChatGPT in a gamified context
Polverini and Gregorcic (2024)	Evaluation of multimodal chatbots for graph interpretation	Chatbots (free and paid versions) analyzed for interpreting kinematics graphs	Assess AI performance in interpreting visual and linguistic aspects of graphs	Kinematics Graph Test (TUG-K), comparison of language vs. vision tasks	GPT-4o performed best; language-based tasks were more accurate than vision-based ones	Student perception not reported
Fadillah et al. (2024)	Qualitative study of high school students’ experiences using ChatGPT for physics	Used as a virtual physics tutor for explaining concepts and supporting assignments	Explore how students perceive ChatGPT’s influence on learning and understanding	Open-ended surveys and qualitative content analysis	Students found ChatGPT helpful in improving physics understanding	Positive perception of ChatGPT’s educational value
Kasmaee and Mahyar (2024)	Use of chatbots as virtual teaching assistants in higher education	Chatbot responded to students’ academic questions	Evaluate effectiveness of AI teaching assistants on student learning	Pre/post-tests, academic performance analysis, and focus groups	Better academic performance and trust in the chatbot assistant among students	Trust and satisfaction in AI as a supportive teaching tool
Pavlenko and Syzenko (2024)	Perception study of Ukrainian university students using ChatGPT as a learning tool	ChatGPT used for homework help, writing tasks, and academic research	Investigate satisfaction and user experiences	Survey of 247 students across disciplines	Students expressed high satisfaction and regular use of ChatGPT	Positive attitudes and appreciation of ChatGPT’s usefulness
Mustofa et al. (2024)	Literature review and experimental use of ChatGPT in solving physics problems	ChatGPT used to solve and generate physics problems	Review benefits and challenges of AI in physics education	Literature synthesis and testing ChatGPT on physics problems	ChatGPT could solve certain physics problems and generate new ones with human-level clarity	Student perception not reported

provides a comparative overview of recent studies involving ChatGPT in physics education and outcomes. Across studies, a consistent insight is that ChatGPT’s effectiveness depends strongly on the pedagogical framework in which it is embedded. In our case, positioning ChatGPT within a Socratic, inquiry-oriented design emphasized metacognitive engagement and conceptual exploration.

To align inference with the study’s theoretical constructs, quantitative comparisons were interpreted primarily at the construct level (composite scores for perception of learning and engagement), while item-level patterns were treated as descriptive trends. This analytic choice supports more robust interpretation and mitigates the risk associated with multiple comparisons.

Student perception consistently emerges as a central metric of success. Studies such as those by (Kasmaee & Mahyar, 2024; Pavlenko & Syzenko, 2024) have shown that students find ChatGPT useful and motivating, even in contexts beyond direct instruction. Similarly, our participants appreciated the opportunity to develop their understanding through dialogue rather than passively receiving responses.

Overall, this case study adds to the growing body of research on AI in education by demonstrating that, under the right conditions, a general-purpose language model like ChatGPT can support perceived learning and engagement through dialogic inquiry and conceptual conflict.

4.1. RQ1: How Do Students Perceive the Use of ChatGPT as a Socratic Tutor in a Physics Course?

Students generally perceived the learning experience with ChatGPT as positive, engaging, and thought-provoking. Working in small groups encouraged discussion and mutual explanation of the prompts, which helped clarify misunderstandings and consolidate knowledge. The Socratic dialogue, mediated by ChatGPT, prompted students to reflect on their reasoning, make predictions, and revisit thermodynamic concepts related to the adiabatic process. Many participants reported that the AI’s persistent questioning felt different from predominantly lecture-based, procedurally focused instruction, fostering a more active engagement with the content. The collaborative setting was also positively valued by students, as it enabled them to verbalize their reasoning, challenge peers’ ideas, and build confidence in their responses before engaging with the AI tutor. This social dimension of the activity reinforces the view that learning gains emerged not solely from interaction with ChatGPT, but from the combination of dialogic AI support and peer collaboration, consistent with findings reported by (Melo-López et al., 2025).

In terms of engagement, the survey results, Table 2, show comparatively lower scores on Items Q7–Q9 (≈3.6–4.1). At the construct level, engagement remained above the midpoint for both genders, and no statistically significant gender differences were observed, indicating broadly comparable engagement patterns.

4.2. RQ2: What Impact Does the Use of ChatGPT Have on Student Engagement During Class Activities?

Among the advantages, students highlighted that ChatGPT provided immediate, non-judgmental feedback, which encouraged exploration without fear of making mistakes. The AI’s ability to ask follow-up questions in a Socratic manner pushed students to articulate their understanding rather than memorize formulas, fostering sustained cognitive engagement during the activity.

Some challenges were also reported, particularly related to moments of uncertainty or cognitive overload when the AI did not provide direct answers. Overall, these findings reinforce that AI-mediated engagement is both empowering and cognitively demanding, echoing prior research that documents increased engagement alongside occasional confusion in AI-based tutoring environments (Wang & Fan, 2025).

Importantly, when the results were interpreted at the construct level—aggregating items into Perception of Learning and Engagement—no statistically significant gender differences were observed. This analytic approach provides a more robust basis for interpretation, as it reduces the influence of item-specific variability and aligns the findings with the study’s theoretical framework. In this sense, the isolated difference observed in Q2 should be interpreted cautiously and does not outweigh the overall pattern of broadly comparable learning and engagement perceptions across genders.

4.3. RQ3: Can the Use of ChatGPT as a Socratic Tutor Support More Equitable Learning Experiences Across Genders in Physics Education?

Overall, the results suggest that ChatGPT-mediated Socratic dialogue supported broadly equitable learning experiences across genders. Although a statistically significant difference was observed in a single item (Q2), where female students reported higher perceived learning, this result warrants cautious interpretation. Because item-level comparisons are exploratory, this isolated difference should not be overgeneralized.

At the construct level, composite measures of perceived learning and engagement did not reveal large gender disparities, indicating that both female and male students experienced comparable levels of participation and perceived benefit from the activity. This pattern suggests that the AI-mediated Socratic approach may support balanced learning experiences, even in contexts where gender differences have been traditionally reported in physics education.

The tendency for female students to provide slightly higher ratings across several items, as illustrated in Figure 1, aligns with prior research indicating that dialogic and collaborative learning environments can foster inclusion and engagement among groups that are often underrepresented or less confident in STEM settings. However, the present findings do not allow firm conclusions regarding gender-specific advantages, and further research with larger samples and validated measures is required.

Prior studies have documented nuanced gender differences in attitudes toward AI, including higher levels of AI-related anxiety among women (Russo et al., 2025). Such factors may shape how students perceive and interact with AI-supported learning environments (Julien, 2024). Future work should therefore examine how AI attitudes, confidence, and anxiety interact with pedagogical design to influence equity-related outcomes.

Overall, this case study contributes preliminary evidence that embedding ChatGPT within a Socratic, inquiry-oriented framework can support generally equitable perceptions of learning and engagement across genders, while underscoring the need for continued investigation into the conditions under which AI-mediated instruction promotes inclusivity.

5. Limitations and Future Work

This study presents promising insights into the integration of ChatGPT as a Socratic tutor for facilitating students’ perceived learning and engagement in thermodynamics. However, several limitations should be acknowledged.

First, the study involved a relatively small sample size ($n=53$) drawn from a single institution, which limits the generalizability of the findings beyond the specific classroom context examined.

Second, there was no control group or pre/post conceptual test to allow for a quantitative comparison of learning gains beyond self-perception. Although a formative quiz was administered during the intervention, it was used exclusively for pedagogical feedback and was not designed or validated as a research instrument; therefore, it was not analyzed as a primary outcome measure.

Another limitation concerns the variability in how student groups interacted with ChatGPT. AI–student chat transcripts were not systematically captured or archived across groups, which precluded rigorous qualitative content analysis of dialogic interactions and reasoning trajectories. Interactions occurred on students’ own devices during in-class group work, and consistent logging at the group level was not feasible in this implementation.

Additionally, some participants reported limitations in ChatGPT’s responses, such as vagueness or occasional misinterpretations, highlighting the importance of careful prompt design and critical engagement with AI-generated content.

Future research could address these limitations by including control groups, employing validated conceptual inventories, and systematically collecting AI–student interaction data to enable fine-grained analysis of dialogic and metacognitive processes. It would also be valuable to explore the role of instructor facilitation in conjunction with AI-mediated dialogue, as well as to examine the impact of similar interventions in larger and more diverse classroom settings.

While these limitations constrain the scope of inference, they do not undermine the value of the present case study. Rather, they delineate clear directions for future work and contribute to a more transparent and responsible integration of generative AI into physics education research.

6. Conclusions

This study provides evidence from a classroom-based case study that ChatGPT, when employed as a Socratic tutor, can support students’ perceived learning and engagement in challenging areas of thermodynamics. By emphasizing dialogue, metacognitive reflection, and cognitive conflict, the intervention was perceived by students as fostering self-regulated learning and epistemic agency.

The success of this approach was not due to a single factor, but rather to a synergistic combination of pedagogical design and technological mediation: ChatGPT functioned less as a content dispenser and more as a mediating scaffold for metacognitive dialogue, helping students detect inconsistencies in their reasoning and articulate more coherent models of thermodynamic behavior.

Compared to other AI-mediated interventions—such as gamified models like (Beltozar-Clemente & Díaz-Vega, 2024) or more structured scaffolding approaches like (El Fathi et al., 2025) and those described by (Mustofa et al., 2024)—this study highlights the potential of open-ended, AI-mediated Socratic dialogue to foster epistemic agency and deep conceptual exploration, as reported by students. Participants valued the opportunity to think through problems rather than being presented with direct answers, an experience that strengthened their sense of ownership over the learning process.

At the same time, important limitations became evident. Occasional misinterpretations by the AI, mismatches in the level of questioning, and the need for better adaptive support point to the importance of refining prompt design, instructor facilitation, and interaction protocols. These challenges underscore that generative AI should not be viewed as an autonomous tutor, but as a tool whose educational value depends on thoughtful pedagogical orchestration, echoing broader concerns in the literature regarding the current limitations of large language models as real-time pedagogical agents.

Future work should explore hybrid learning environments in which AI tools like ChatGPT are integrated with peer dialogue, instructor feedback, and validated assessment strategies, allowing researchers to examine not only student perceptions but also learning outcomes more systematically. Such combinations may help mitigate the shortcomings of AI while amplifying its strengths as a dialogic support system for physics reasoning.

In conclusion, this study contributes to an emerging research agenda that positions generative AI not merely as a novel technological tool, but as a mediating catalyst for pedagogical innovation. When embedded within thoughtfully designed learning experiences and guided by instructors, models like ChatGPT can play a meaningful role in physics education by supporting students’ engagement, reflective practice, and capacity for autonomous and critical thinking, thereby contributing to inclusive and high-quality learning environments aligned with Sustainable Development Goal 4 (Quality Education), while opening new avenues for research on equitable and responsible AI integration in STEM classrooms.