Pedagogical Qualities of Artificial Intelligence-Assisted Teaching: An Exploratory Analysis of a Personal Tutor in a Voluntary Business Higher-Education Course

Featured Application

In this study, we examine custom-trained artificial intelligence (AI) teaching and learning tools based on generative pre-trained transformer (GPT) technology and their pedagogical potential to increase student engagement and satisfaction and promote the development of competencies in business schools.

Abstract

There is minimal research concerning the role of custom-trained artificial intelligence (AI) tools in higher education, with a lack of research concerning the pedagogical qualities of an AI-based personal tutor. To fill this literature gap, we examined how a custom GPT personal tutor shapes key teaching and learning qualities. Using the mixed-methods approach, we aimed to demonstrate preliminary and exploratory empirical evidence concerning the contribution of custom-trained AI tutors to building up students’ competencies. Our research analyzed the subjective assessments of students related to the GPT tutor’s contribution to improving their competencies. Both the qualitative and quantitative empirical results confirmed the positive contribution. In addition, we triangulated the results to evaluate the potential of custom-trained AI chatbots in higher education, focusing on undergraduate business courses. However, the results of this study cannot be generalized to the entire student population of business schools, since the participation in the AI-assisted tutor program was voluntary, attracting only intrinsically motivated students.

1. Introduction

Artificial intelligence (AI) has become an indispensable tool, with abilities that include generating human-sounding text and convincing images and videos, as well as assisting individuals and groups in decision making by applying critical thinking [1]. In higher education, AI tools and platforms provide instant and personalized feedback, including relevant arguments and logic, and they have the potential to provide emotional support and motivation to students [2,3]. AI systems are also beneficial for instructors, supporting them in preparing learning materials, formulating convincing arguments and teaching strategies, and easing their burden by assisting in assessments and providing support in grading different forms of assignments.

Custom-trained tutors that utilize GPT architecture, such as large language models (LLMs), are based on “vanilla” AI models, i.e., generic LLMs, and can be easily developed without specific programming or advanced technical competencies. Educators can fine-tune commercially available LLMs by using plain-language prompt engineering techniques and training them with their teaching materials, including slides, case studies, tests, and assignments. Such technology opens the door for non-technical educators to engage with the latest “frontier” of AI tools and models and apply them in the specific context of their academic teaching.

In this study, we aimed to evaluate a custom GPT tutor, developed by the authors and applied to a short academic course on the topic of blockchains in an undergraduate business school setting. Our goal was to provide a preliminary evaluation of the tutor’s pedagogical competencies and their contribution to students’ competencies, both from the students’ viewpoint.

Therefore, we set the following research questions (RQs):

• RQ1: How do students rate the perceived pedagogical quality of the custom GPT tutor?
• RQ2: To what extent does the perceived pedagogical quality predict students’ self-reported generic competencies?

Answers to these questions would inform instructors seeking to integrate AI-based tutors into business curricula without increasing their contact hours.

2. Theoretical Background

Although LLMs and other AI platforms deliver human-like writing and present convincing arguments, these responses are based on the processing of massive datasets and the production of contextually coherent outputs [1]. Therefore, these systems still cannot be evaluated as possessing independent, critical intelligence. Still, in an educational environment, AI systems are able to provide personalized feedback and create personalized content in real time.

These capabilities contribute to dynamic adjustments of the educational content and the “thinking” of alternative explanations and examples in real time, which are aligned with the individual knowledge level and the learner’s motivation [4]. When used well, AI tutors fit naturally into a student-centred approach: they support self-directed learning, increase students’ motivation, and improve their academic achievement [5]. With the real-time and personalized support, students have more autonomy [6] and an opportunity to improve their generic competences [7]. However, without human involvement, the quality and consistency of AI-based learning cannot be guaranteed.

To describe the pedagogical qualities of human involvement, we adopted the community of inquiry framework, which refers to the teaching, social, and cognitive presences [8]. To describe the pedagogical qualities of human involvement, we adopted the community of inquiry framework, which refers to the teaching, social, and cognitive presences [8].

According to Arbaugh et al. [8], teaching presence refers to human teaching competence (and the comparable human-like competence of successful AI tutors) to structure the teaching and learning content, by using scenarios and clarifying misconceptions. We implemented this feature in our GPT tutor on blockchain, utilizing highly structured teaching materials and prompting the tutor to follow the custom training structure strictly. Cognitive presence is ensured by exhibiting and encouraging reflection and reasoning by the instructor (or the AI teaching platform). We achieved this teaching competence through a custom prompting strategy, which ensured that the GPT tutor posed questions and encouraged discussions with students until they reached the desired level of understanding of blockchain concepts. The last component of the framework is related to social presence, which is shown by encouraging and motivating students, referring to previous discussions, and providing encouraging feedback and praise for the successful completion of learning tasks. An adequate prompting strategy also achieved this pedagogical feature.

By creating a ‘pedagogically aware’ GPT tutor, we hoped to achieve a highly successful teaching and learning environment. According to Kember et al. [9], it consists of active learning, constructive feedback, and critical-thinking components, which lead to behavioral, emotional, and interactive engagement [10].

Thus, students start learning through experiences, comparable to those attained in the ‘real-world’ [11], due to the AI platform functionalities, enabling the delivery of real-time, personalized feedback. It especially matters when human tutors are not immediately available or have limited time and resources to support students in extensive courses [12,13].

The described ‘optimistic’ scenario of AI-enabled academic teaching and learning corresponds to Cicchetti’s model of AI as a ‘Digital Aristotle’, empowering students through individualized tutoring. However, the same author contrasts this scenario with a more pessimistic view of AI as a threat, which commodifies students and erodes the authenticity of the learning experience [14]. Therefore, we created a custom GPT tutor using the prompting strategy (see Appendix SI), which focuses on human-centered values and provides comprehensive, contextual support, similar to that provided by human instructors.

Therefore, we expect that the well-planned and carefully prompted GPT tutor has the potential to significantly improve business students’ motivation, efficacy, and expectations, as already indicated by Gao et al. [15], as well as by Gupta et al. [16]. As the student took a short, AI-supported course, we were available for questions and comments from participants. We constantly offered ethical guidelines and the resolution of potential ethical issues, as suggested by Gupta et al. [16]. We also provided additional explanations of related concepts, in case the GPT tutor failed to deliver an adequate level of teaching performance. However, the number of cases requiring human intervention remained negligible, despite our desire to follow the advice of Pang et al. [17], who suggest that fully autonomous AI learning tools are highly efficient in summarizing but less successful than human instructors in introducing new concepts.

Potential outcomes of the AI-assisted teaching and learning process are more than promising: the extant literature reports gains in students’ domain-specific reasoning, but also more positive attitudes toward the subject, in addition to higher motivation and confidence [18,19]. However, effective implementation requires much more than access to the technology. Rytilahti and Lokkila [20] reported that without structured guidance, both learners and the chatbot can misinterpret tasks. Students should, nevertheless, be the focus of the AI implementation process. In this context, Razmerita [21] suggests the partnership between students and AI, finding that the use of platforms, such as ChatGPT, accelerates idea generation and summarization, but also carries risks of plagiarism and factual errors, which are difficult to identify with prior specialized knowledge.

Rienties et al. [16] show that students are more willing to accept AI assistants when they are older or more experienced users, have solid support from their university, and know the tool will be affordable and kept in good working order.

A handful of promising studies have begun to fill the existing literature gap concerning the pedagogical qualities of AI implementations in higher education. Li [22] used ChatGPT to run a flipped classroom and concluded that students turn in high-quality projects, grow in confidence, and become more imaginative. Likewise, Ellikkal and Rajamohan [23] report that AI-guided lessons give learners a greater sense of autonomy, competence, and connection with others—three ingredients that fuel engagement and, ultimately, better results.

In conclusion, custom-trained GPT tutors have been assessed as a promising pedagogical advancement in a variety of higher education backgrounds, including computer programming courses [24], assisting students in a general context [25], and in their graduate writing tasks [26]. These studies provide the potential of ‘pedagogically focused’ prompt engineering, which has been recognized even by one of the major providers of generative AI tools. Namely, in July 2025, OpenAI launched the Chat GPT ‘study mode’, which functions as a generalized GPT tutor, focused on critical thinking, thus confirming our initial research aims and design [27].

3. Materials and Methods

3.1. Materials

This study is based on the evaluation of a custom GPT tutor, covering the fundamentals of blockchain technology and its use in contemporary business. The GPT tutor was implemented as a custom GPT chatbot in the OpenAI ChatGPT environment. The AI tool was instructed to follow the highly structured teaching approach and apply the pedagogical principles of student-centered pedagogy. There was no programming involved in the development of the blockchain GPT tutor. The entire custom training process was completed by using plain-language prompts and our training materials.

We used the Five-Step Prompt Framework to develop the prompting strategy [28,29]. In the first step, we developed the system prompt, setting the overall aim and approach of the GPT tutor (see Appendix SI in Supplementary Materials). The second step shapes the context of responses, which is crucial for implementing the intended pedagogical approach. The exact prompt (also available in the Appendix SI) concerns the general rules for the discussion, as well as more specific elements, including learning structures, constraints and dynamics, visual elements, and tone and style.

In the third step, we provided the teaching materials, examples, and references to ensure teaching-by-example and avoid hallucinations, which can severely compromise the AI tutor’s performance in dialog and writing [30,31]. The teaching materials were highly structured, and the GPT tutor had been strictly instructed to follow this structure, with the flexibility to create relevant examples and mini-cases in real time, adapting to individual student needs.

In the last step, the GPT tutor was iteratively improved based on early user feedback. The first users were blockchain experts. We used their comments to review and improve all prompts, with the final versions being presented in the Appendix SI. Finally, we invited students to use the GPT tutor. A GPT tutor can also translate the text and write in multiple languages. However, students were explicitly asked to choose between Croatian and English after accessing the tutor for the first time.

For OpenAI Plus subscribers, the implementation of Custom GPT does not incur additional fees. We made the GPT tutor freely accessible at the following URL: https://chatgpt.com/g/g-67c6caacddb08191a3e2e91b71b3ed53-sto-je-blockchain (accessed on 1 August 2025). Access to Custom GPTs is enabled for both paying subscribers and free users, who may be ‘cut off’ after using the daily message cap. They can continue their access the next day, which effectively structures our course into ‘bite-sized chunks’ that are easily accessible to students with busy schedules and can be accessed ‘on-the-go’ using a smartphone, tablet, or other mobile device.

3.2. Methods

This study used a convergent mixed-methods design. We collected transcripts of interactions between the students and the GPT tutor. In addition, students were asked to fill in a questionnaire on perceived pedagogical quality and the perceived improvement in their generic competencies after taking a GPT tutor-led academic course on the design and application of blockchain technology. We analyzed qualitative and quantitative data and triangulated the findings from both research stages.

3.3. Participants

Out of 155 third-year undergraduates enrolled in the Business Informatics courses (both in Croatian and English) at the Faculty of Economics and Business of the University of Split (FEBT Split), fifty-one voluntarily participated in this study. All participants were self-selected and intrinsically motivated, since the blockchain mini-course was offered as an optional module, without any course credits. Most of the participants were motivated to learn about the blockchain technology, which is not covered by any of the undergraduate or graduate courses in the current FEBT curriculum. This mini-course offered content on the concepts, design, and implementation of blockchain technology.

Blockchain basics were covered in a live lecture by the course instructor during week one. No separate digital handbook was distributed before the students began using the GPT tutor. All students had completed interactions with a custom-trained GPT tutor in the next five weeks of the course.

Because the tutor was offered only as an optional supplement during the summer term, no parallel control cohort was available, and the empirical findings should be interpreted as exploratory and descriptive. Participants’ responses were anonymous, and this study was conducted following institutional ethical guidelines.

3.4. Instruments

A 19-item questionnaire collected student perceptions, with each statement rated on a five-point agreement scale (1 = Strongly Disagree, 5 = Strongly Agree). Two theoretical constructs were calculated by including 19 items:

• GPT-tutor pedagogical quality was measured by ten items, specifically adapted for this study from the pedagogically relevant dimensions of the BUS-15 Chatbot Usability Scale, previously validated in our exploratory work [32]. Items probe the clarity of explanations, conversational coherence, context tracking, timeliness, diagnostic value of feedback, knowledge-application support, and overall recommendability. The scale showed high internal consistency (α = 0.84).
• The generic competency scale consisted of nine items. Two engagement items were taken from the emotional- and skills-engagement facets of the Student Course Engagement Questionnaire [10]. In comparison, seven items were drawn from the “generic capability” subscales of the Teaching-and-Learning Environment Questionnaire [9] (critical thinking, self-managed learning, creative thinking, self-regulation, problem solving). Together, they index students’ self-reported development of higher-order skills fostered by the course; reliability was acceptable (α = 0.83).

3.5. Research Procedure

After completing the GPT-tutor module, students received an online Qualtrics survey link during a regular class session. The survey began with an informed-consent statement, followed by the instructional preamble and the 19 items. Completion time was approximately 6–8 min.

3.6. Data Analysis

Item-level missing values (<4% per scale) were handled via listwise deletion. Composite scores for the GPT-tutor quality mean and generic capabilities were computed. Internal consistency was assessed using Cronbach’s α. One-sample t-tests compared each composite mean against the neutral midpoint (3). The Pearson correlation tested the association between the two research constructs, while a simple linear regression was used to examine the perceived pedagogical qualities as a predictor of achieved competences. All statistical analyses were performed in SPSS 28.

We were not able to capture log files and perform an audit of GPT tutor accuracy. This should be done in future studies.

4. Results

4.1. Results of the Qualitative Analysis

Qualitative analysis proceeded in four sequential stages. First, transcripts of twenty GPT tutor sessions, provided by student participants in the research project, were converted to searchable text via OCR, and Croatian text was translated into English to ensure consistency in coding. Second, open coding was conducted by reading each of the 20 student–GPT tutor sessions line by line. Text segments reflecting either student behaviors or tutor actions were assigned to one or more of five a priori constructs: Engagement, Learning Environment, Teaching Presence, Social Presence, and Cognitive Presence. Third, axial coding clustered similar open codes into sub-themes under each construct; only sub-themes appearing in at least eight transcripts (≥40%) were retained. Finally, for each sub-theme, we selected illustrative excerpts that preserved the original wording and context. The following themes were identified in the student transcripts.

Student engagement: Two sub-themes characterized students’ persistence and motivation.

• Persisting through challenge appeared in 90% of transcripts. Students who scored poorly on initial quizzes immediately opted to retry, demonstrating notable behavioral persistence. One student noted, “Quiz 1 (result: 2/5)… agreed to retake the quiz a second time.”
• Mastery and completion were present in 85% of sessions. In approximately 85 percent of the transcripts, students kept working until they had the perfect score on the final quiz, as shown by the frequent evaluations in the transcripts: “FINAL QUIZ: … Result 7/7”.

Learning-environment quality: Two sub-themes captured the tutor’s responsiveness and contextualization of AI-led teaching.

• Immediate, just-in-time feedback was present in 95% of transcripts. The GPT tutor returned instant confirmation or a gentle correction (“Correct! Time for the quiz”) within seconds, so students were sure whether they were on the right learning track.
• Contextual examples appeared in 80% of sessions. The GPT tutor nudged students to apply new ideas to fresh situations: “Can you imagine another real-world example where such record-keeping would be useful?”
• Teaching presence: Two sub-themes illustrated the tutor’s instructional structure.
• Guided questioning was identified in all transcripts. Every session unfolded through guided questioning, starting with everyday scenarios (“Imagine you have bought a product online…”) and then walking the student step-by-step toward the underlying concept.
• Clarifying misconceptions was found in 60% of all analyzed sessions. In these cases, the tutor paused, clarified, and praised the corrected answer (“Bravo!”), giving students the confidence to move forward.
• Social presence: Two sub-themes reflected the tutor’s interpersonal tone.
• Encouraging tone was evident in around 75% of transcripts. Frequent positive reinforcement—“Tutor: Bravo!”—fostered a warm, conversational atmosphere.
• Personalized interaction was present in approximately 40% of sessions. While AI did not use students’ names, it did remember their previous mistakes and accordingly adapted its approach.

Cognitive presence is shaped by two sub-themes, which can be formulated in terms of ‘conceptual application’ and ‘reflective problem-solving’:

• Conceptual application was observed in 85% of transcripts. Students synthesized technical concepts in their own words, for example: “Student: Transparency, authenticity, fraud reduction.”
• Reflective problem-solving emerged in 70% of sessions when the tutor’s prompts elicited deeper reasoning:

“Tutor: Why is it useful that records cannot be changed?”

“Student: Because the system stays secure.”

Across the 20 transcripts, each construct and its sub-themes were well covered, with Teaching Presence and Learning-Environment Quality identified in almost all the chats, followed by Engagement (90%) and Cognitive Presence (85%), with Social Presence present in two-thirds of the analyzed chats.

4.2. Results of the Quantitative Analysis

Figure 1 shows how students rated each of the survey statements on a five-point Likert scale. Average scores are in the range between 3.8 and 4.4, pointing to broadly positive views. Students were most enthusiastic about how clearly the tutor explained concepts, how relevant the feedback felt, and how much it sparked their curiosity.

The error bars (±1 SD) highlight that, although most items exhibit relatively tight agreement (SD ≈ 0.4), a few prompt-engineering and self-regulated learning questions show greater dispersion, suggesting more varied student experiences in those areas.

Internal consistency was adequate for both composite variables. The 10-item GPTQual scale, describing the perceived GPT pedagogical quality, had a Cronbach’s α value of 0.844 (N = 49), and the 9-item StudCmp (describing the perceived improvement in competences) scale had a Cronbach’s α value of 0.830 (N = 51). Students rated the GPT tutor’s pedagogical quality highly (M = 4.31, SD = 0.45, range 3.10–5.00) and reported substantial capability gains (M = 4.01, SD = 0.46, range 3.00–5.00).

Two one-sample t-tests compared each composite against the neutral midpoint of 3:

• GPT pedagogical quality: t(50) = 20.67, p < 0.001, mean difference = 1.3076, 95% CI [1.1806, 1.4347], Cohen’s d ≈ 2.90;
• Student competencies: t(50) = 15.62, p < 0.001, mean difference = 1.0087, 95% CI [0.8790, 1.1384], Cohen’s d ≈ 2.19.

Both composites are significantly above neutral, with substantial effect sizes. Cohen’s d was computed as the mean difference over the sample standard deviation. For GPT pedagogical quality, d = (4.31–3)/0.45 = 2.90; for student competencies, d = (4.01–3)/0.46 = 2.19. According to conventional benchmarks, these represent considerable effects [33].

The Pearson correlation revealed a moderate-to-strong positive association between GPT pedagogical quality and Student competencies, r(49) = 0.571, p < 0.001, indicating that students who perceived higher tutor quality also reported greater gains in generic capabilities.

A simple linear regression tested whether the perceived GPT pedagogical quality predicts the achieved student competencies. The estimated OLS model had the following characteristics:

• Model fit: R = 0.571, R² = 0.326, adjusted R² = 0.312, F(1,49) = 23.72, p < 0.001;
• Coefficients: Intercept = 1.497 (SE = 0.518), t = 2.887, p = 0.006; GPTQual → StudCmp: b = 0.583 (SE = 0.120), β = 0.571, t = 4.870, p < 0.001.

Thus, perceived GPT tutor pedagogical quality accounts for 32.6% of the variance in students’ self-reported capability gains. All statistical assumptions for the OLS regression were met, including a lack of significant deviations from normality or homoscedasticity (as determined by a visual check of the residual plots).

4.3. Triangulation of Research Results

Both the transcripts and the survey results support our interpretation of the research results. Students repeatedly describe the GPT tutor as easy to approach and quick to help. Almost every transcript praises the tutor’s instant answers, and roughly 80% of the transcripts praise the practical examples used by the tutor.

The numbers support the same conclusion. On a five-point scale, students rated the tutor’s clarity, response speed, and overall usefulness at M = 4.31 (α = 0.84), well above the neutral midpoint (t = 20.67, p < 0.001, d ≈ 2.9). Those who scored the tutor higher also reported larger gains in critical thinking, self-management, and problem-solving (r = 0.571, p < 0.001).

The qualitative results show that all analyzed transcripts contain evidence of guided questioning (Teaching Presence). By contrast, 85% of responding students apply key concepts in their own words (Cognitive Presence), and the encouragement of students to continue with their learning (Social Presence) appeared in 75% of sessions. Although the survey did not analyze those constructs in detail, the quantitative data, especially a high score for the perceived pedagogical quality of the GPT tutor, acknowledge them implicitly. A linear regression further shows their importance: perceived tutor quality explained 32.6% of the variance in StudCmp (R² = 0.326, β = 0.583, p < 0.001). Therefore, the tutor’s perceived pedagogical quality and the level of student support are both shown in transcripts and reflected by over one-third of students’ self-reported improvement in competences.

Taken together, the two strands of evidence tell a coherent story about why the GPT tutor works. The transcripts show the qualitative themes, describing the student engagement with the AI tool, while the survey results confirm that these processes drive the gains in learning outcomes.

5. Discussion

Generative AI tutors can provide personalized, on-demand help, but their success hinges on sound instructional design and active human supervision. Across the studies already cited, three common themes can be identified: how the AI is configured, how students engage with it, and what the learning environment provides. All three themes are re-confirmed by our empirical results.

When a tutor is carefully prompt-trained on licensed course materials, they can address specific questions. As noted by Cicchetti [14], the same tool can either become a “Digital Aristotle” or a mere cognitive aid. In our case, guided questioning and on-the-spot misconception-fixing appeared in every transcript, and students rated the tutor’s pedagogical quality very highly (M = 4.31, SD = 0.45; d ≈ 2.9). They valued the teacher-like presence, sound prompt engineering, and post-session debriefs built into the system.

Large-scale business-school experiments show that AI tailoring boosts autonomy, competence, and engagement [15,23], and Ahmed’s statistics students reported similar benefits when ChatGPT supported formative tasks [12]. Our data echo those patterns: 90% of learners took the initiative to ask follow-up questions, and 85% explored topics beyond the core material. Their perceived competency score was correspondingly high (M = 4.01, SD = 0.46; d ≈ 2.2), and perceptions of tutor quality and capability gains rose in tandem (r = 0.57).

Instant, personalized feedback is often cited as the engine of motivation and learning in AI-mediated classes [18,22]. It showed up in 95% of our sessions (“Correct!”). Moreover, 80% included new and contextualized examples—the very tactics Li links to surges in self-efficacy and creative thinking [22]. Far from feeling cold or robotic, the tutor frequently offered warm encouragement (“Bravo!”), meeting the social-presence challenge, identified by Razmerita [21]. Even so, a few students did double-check critical points with the lecturers, showing that human oversight remains essential.

In short, when a Custom-GPT tutor is tightly aligned with course goals, scaffolded by thoughtful prompts, and backed by an attentive instructor, it can deliver the kind of responsive, motivating support that modern student-centered pedagogy demands.

Gupta et al. [16] show that quantitative skills do improve with ChatGPT practice, but they warn faculty to provide ethical guidelines and promote autonomous problem-solving to avoid dependence. Rytilahti & Lokkila [20] also demonstrate that a short orientation session is important for the success of AI teaching and learning tools. Otherwise, both the AI tool and students may misinterpret tasks. This finding aligns with several transcripts, in which students were looking for human confirmation when answers became ambiguous. This is an empirical reminder that reflective checkpoints must follow AI feedback.

Our earlier exploratory work confirmed that a business-management chatbot was usable, but did not test formal pedagogical constructs [32]. Our mixed-methods design extends that evidence, as we demonstrate, with a substantial effect size and convergent qualitative themes, why and how a GPT tutor can improve engagement and capability gains, as suggested by Cicchetti’s “Digital Aristotle” model [14].

Our empirical results further support the extant literature and show that a well-prompted, course-aligned GPT tutor, deployed under active instructor facilitation, supplies timely feedback and helps raise student engagement and build their competencies. However, unless ethical guidelines and reflective practice are emphasized, the speed and convenience of AI-based tutoring may lead to excessive dependence on AI learning chatbots among students. Therefore, future research should compare human-led lectures, combined with AI coaching, against AI-based learning chatbots.

Concerning RQ1, students rated the Custom-GPT tutor very positively (M = 4.31, SD = 0.45 on a 5-point scale), a value that differs from the neutral midpoint by more than 20 standard errors, t(50) = 20.67, p < 0.001. The value of Cohen’s d (2.90) indicates a powerful effect. Reliability for the ten-item scale was high (α = 0.84), supporting the internal coherence of this construct.

In addition, as related to RQ2, perceived tutor quality was strongly associated with students’ self-reported gains in critical thinking, self-regulation, and problem-solving. The perceived pedagogical quality of the GPT tutor correlated moderately with the self-reported improvement in competences (r = 0.57, p < 0.001). A simple linear regression showed that the pedagogical quality accounted for 32.6% of the variance in the competence improvement, as shown by the F-test: F(1,49) = 23.72, p < 0.001. The linear regression coefficients show that each one-point increase in perceived tutor quality corresponded to a 0.58-point rise in competencies (β = 0.58, SE = 0.12). These findings indicate that students who judged the AI tutor as more responsive and informative also perceived substantially greater capability development, confirming the pedagogical potential of personalized GPT tutors.

6. Conclusions

The convergence of qualitative themes, such as just-in-time feedback and student guidance, provided by AI, leads to substantial effect sizes on the perceived GPT pedagogical quality and achieved student competencies. Those empirical results show the real-world impact of aligning AI functionalities with modern teaching and community-of-inquiry frameworks [8,9,10].

We demonstrated the potential of an educational AI tutor to maintain the teaching, cognitive, and social presence, as described by the pedagogical literature [8]. Research participants confirmed their perception of the three forms of presence, as demonstrated by the GPT tutor, and perceived a significant improvement in their self-assessed competencies after taking the course led by the described AI chatbot. Although we took precautions to ensure structured guidance [20], the current empirical results still do not guarantee that the long-term usage of unsupervised AI learning platforms does not translate into over-reliance and other adverse effects for students [20].

In addition, our research was based on the voluntary participation of highly and intrinsically motivated students, which might have introduced a self-selection bias. No control group was involved in the research, so our empirical results are to be interpreted as preliminary and exploratory. However, large and significant effects were still observed and can be confirmed by the Socratic guidance that OpenAI has now formalized in ChatGPT’s newly released “Study Mode”. This new mode of ChatGPT’s user interaction is explicitly intended to deepen understanding, rather than providing quick answers and finished ‘write-ups’, which students can (mis)use, applying only minor revisions and editing of AI-generated text. This industry move supports our conclusion on the high potential of AI for improving the quality of higher education.

This study, along with a previous case of a business management teaching and student support chatbot [32], is based on semi-autonomous, short-term AI usage experiences and cannot provide a long-term assessment of the potential consequences of adopting fully autonomous AI instructors. Future studies should, therefore, assess the potential long-term effects of AI-led courses, incorporate an objective evaluation of AI chatbots on different aspects of student performance, and analyze different co-teaching models. Experimental research designs should be used in different scenarios, in which AI is assigned a semi-autonomous tutoring role, accompanied by human-led instruction and evaluation.

Future studies should be conducted in regular classes, involving compulsory academic evaluation and assignments, in order to produce more generalizable empirical results. Our current results and consequential support for AI-based tutors in higher education could be biased, due to the voluntary nature of enrollment into the mini-course, which might have attracted self-enrollment, due to the novelty of AI tutoring, and the blockchain topic. Therefore, future research should be based on random participant assignment between the experimental and control groups, with comparisons of course grades, learning outcomes, and evaluations of the learning process for both groups.