A Proof-of-Concept of an Integrated VR and AI Application to Develop Classroom Management Competencies in Teachers in Training

Abstract

We designed an interactive virtual reality (VR) application to provide a controlled and yet unpredictable environment for the development of classroom management skills. The simulated environment allows teachers in training to interact with virtual students in realistic and meaningful ways. The VR application allows rich verbal interaction by using artificial intelligence (AI). Initial findings suggest it is a successful proof of concept. In this paper, we focus on the technical implementation. Predictions on educational effectiveness and the educational challenges of pre-service teacher education are discussed. Future developments include rigorous testing and incorporating non-verbal communication based on a multi-dimensional interpersonal behavior model.

1. Introduction

One key challenge confronting novice teachers [1] is to master effective classroom management. Many recently graduated teachers lack self-efficacy and feel inexperienced [2]. The current conventional preparation is typically through reading the theory and observation of classroom practices of skilled teachers. While this ensures recognition of potential problem situations and approaches, it does not allow teachers in training to interact with high school students nor does it reveal the learning process of skilled teachers. Typically, the next stage is learning by doing and reflecting on the outcomes.

In the TU Delft MSc Science Education program, teachers in training complete an internship in a secondary school. Positions for teacher-in-training internships are limited and complicated to supervise. The effectiveness of such an internship depends on various factors, including the active contributions of the teachers in training, the opportunities to experiment with new teaching strategies, and the quality of feedback and mentorship [3,4]. This approach is not systematic, does not allow teachers in training to try out different approaches to the same situation, and requires a long time to develop a varied repertoire. Therefore, it is not adequate enough [5,6]. Role-playing exercises with other teachers in training are effective, but they are hindered by limited resources [7] and are less authentic.

Therefore, we have developed an interactive immersive virtual reality tool to enhance observation, role playing, and learning by doing. VR offers several advantages as a practice environment: it is always available [8], offers safe practice [2], enhances creativity and motivation [9], provides sufficiently authentic learning experiences [10], and offers options for automated feedback [11,12]. Practice without any input from an instructor, who provides feedback and suggestions, is not helpful to learn the skills. Such input triggers evaluation of the teacher in training actions, which are crucial for the learning process. Some developers have tackled the challenge by implementing interaction through a ‘wizard’ (instructor) in real time [13,14,15]. Although very realistic, the interactions are limited to the pre-programmed content and rely on the availability of the instructor.

To open up a broader range of interactions, and to remove limitations imposed by needing an instructor’s presence, we developed a VR environment which includes generative artificial intelligence (AI) modules for facilitating and generating authentic interactions through large language models [16], eliminating the need of an expert button operation. This reduces workload for instructors, broadens possibilities, and enables exploration of various classroom management theories, like interpersonal behavior [17]. For our virtual reality environment, we focus on classroom management as it is one of the biggest challenges for a novice teacher. Within the system, a teacher in training can stand in front of a virtual classroom, and engage with virtual students, or avatars, by name. The objective is to manage the class by controlling the virtual students’ mood or restlessness through addressing the virtual students in a timely and appropriate manner.

This paper describes and analyses the design-based development of our VR and AI application [18], in which the authors act as expert teachers. Other stakeholders, including teachers in training and their instructors, will be included in future work.

Our main research question is the following:

RQ1: 

Can we design and implement an effective, authentic, and versatile interactive VR and AI classroom management education environment which supports the teacher in training’s learning process?

To determine that the environment is technically realistic, we reflect upon the following requirements [19,20]:

• Virtual scene authenticity: To ensure a 3D classroom with realistic characteristics, like natural colors, light, and shades.
• Audience (or avatar) appearance: To ensure that virtual students in a class appear realistic (looks, clothes, sound, etc.).
• Audience behavior: To ensure natural postures, gestures, facial expressions, when in interaction with the teacher or peers, or when working alone.

Besides technical aspects, other factors contribute to the ecological validity of a VR and AI application. Therefore, our second research question is the following:

RQ2: 

Can we make an environment in which students can develop a classroom management skills repertoire that suits their pedagogical development needs?

Throughout the paper, we use the following phrasing for different stakeholders:

• Instructor: a professor teaching at a university, educating teachers in training;
• Teacher in training, or trainee: students who take the MSc of Science Education program;
• Skilled teacher: a high school teacher who is effective in teaching knowledge and skills;
• Novice teacher: a starting teacher, who recently graduated from a Master in Education program;
• A student: a high school student in real life;
• Virtual student, or avatar: a virtual high school student in virtual reality.

2. What Is Known from the Literature

2.1. Developing Effective Interpersonal Styles for Novice Teachers

The skills necessary for effective classroom management can be specified through the lens of pedagogical theories, such as the process–outcome approach [21] or discourse theory [22]. The dynamics of the learning process and of the involved social interactions are both present in the interpersonal behavior model [17,23], developed from the initial work of Leary [24]. This model works with two orthogonal dimensions: influence (vertical axis: dominance vs. passive submission) and proximity (horizontal axis: cooperation vs. opposition). Activity on the vertical axis evokes the opposite behavior (dominant behavior leads to submissive reactions and vice versa), while behavior on the proximity axis results in a similar response (cooperative actions result in cooperation; opposition results in opposition).

Typically, novice teachers score rather low on the subdimensions pertaining to take the lead, take initiative, and confront students, and score rather high on cooperation and leaving the initiative to the students [17]. Their interpersonal skills differ considerably from skilled teachers [23], and therefore a focus on the development of these skills is essential.

The repertoire of appropriate interpersonal interactions can be further specified by a reflection on the various roles of a skilled teacher [25], such as a host, expert, instructor, pedagogue, assessor, or coach. Each of these roles is associated with specific verbal and non-verbal behavior and is detailed in an observation rubric, like [26]. This rubric serves as a checklist to evaluate the behavior of the teachers in training. Classroom management is a recurring challenge for most roles [27]. A key piece of advice is to ensure that each student in the class feels recognized and acknowledged. This requires practice in two ways: mastering the various roles applicable to different teaching situations, and adapting to the variety of students’ behaviors. Each class is different, and each school has a different culture. Teachers in training should become prepared for these differences, but this often requires changing to another internship school; the possibilities for this are limited.

Research using video observation [28,29] and mobile eye tracking [30] indicates that skilled teachers continuously scan the entire class, making quick and correct interpretations of the student’s behavior. In contrast, novice teachers often focus on a few students and lack the ability to quickly and correctly interpret. The process of ‘noticing’ consists of identifying relevant signals of potential disorder (where novice teachers struggle), triggering interpretation and priming appropriate reactions [28], leading to an informed intervention. Developing perception, interpretation, and reaction requires much practice. VR offers repeated practice opportunities and allows focus on specific aspects [28]. Achieving expertise requires deliberate situated practice [31], explicitly to refine specific skills. Deliberate practice necessitates a well-defined goal, motivation, constructive just-in-time feedback, and ample options for repetition to continuously improve over time. Through such practice, novice teachers can eventually excel, easily recognize meaningful patterns, work swiftly with minimal errors, conduct deep problem analysis, and engage in significant self-reflection [32].

2.2. The Opportunities and Challenges of VR in Education

Simulations can offer effective learning environments for teachers in training, to replicate diverse and complex scenarios, enable learners to practice slightly above their current level of performance (zone of proximal development [33]), and integrate new skills in their repertoire. Moreover, simulations afford both independent learning and learning with the help of an expert.

VR applications, however, must pay attention to the situated nature of these complex practices. Situated learning is seen as a social process occurring within authentic, ecologically valid contexts where actions yield similar consequences as in reality [34], contrasting to classroom settings where applying knowledge is often detached from real-world scenarios. “This perspective suggests that knowing and learning cannot be abstracted from the environments in which they occur” (page 2 from [35]). Simulations reduce the complexity of the classroom but should keep as many relevant factors as possible intact, for example, by incorporating various sensory inputs [36].

VR is already finding meaningful applications in education settings, emphasizing learning over entertainment. Both non-immersive VR or desktop VR, at your computer screen, or immersive VR, with a headset [10], are readily available. For instance, one can examine a 3D representation of the heart [37], or experience another location [10] or time. The use of VR allows teachers to represent unobservable, dangerous, or expensive phenomena effectively, enhancing the learning experience [9] in primary and secondary education. Moreover, universities like ours [38,39] are developing and utilizing educational VR applications. The VR zone [40] of the TU Delft is actively involved in developing applications for its students, which include physics simulations [41], escape route testing [42], and testing ship designs [43]. VR environments are perceived as a safe environment to gain experiences before engaging in real practice.

Some VR environments for skills development of teachers in training are already available [13,14,15]. In these simulations, instructors control real-looking virtual students in an online environment. What the teacher in training does not see is that the students they interact with are controlled by an online actor, facilitating the interaction. These simulations focus on isolated skill refinement, broken down into manageable parts for the benefit of the learning process [44] rather than using a whole-task approach, which involves approaching the learning domain in its integrity [45]. The VR environment is perceived as a safe environment to gain teaching experiences before engaging in real practice.

2.3. Incorporating AI for Enhanced Authenticity in Interactive VR Experiences

Realistic authentic immersion in an interactive VR environment is vital to making the system more effective as the degree of authenticity significantly improves the learning outcomes [2]. Particularly, realistic immersion encompassing interpersonal, pedagogical, and subject-matter competencies [46] is key to the improved development of skills for classroom management [47].

Higher levels of authenticity also increase the VR’s efficacy and improve the ability of the teacher in training to transfer what has been learned to a new situation [48]. A realistic environment should include what is likely to happen and does not include unlikely scenarios [49]. Classrooms are complex, multi-faceted, highly variable, and unpredictable entities in which responses to situations cannot easily be calculated from a set of fixed rules of the type ‘If this happens, then do that’. Therefore, we included AI to allow a practically infinite range of interactions. This implies that the same set of input parameters from both the avatar and the teacher in training can generate an infinite amount of different responses.

AI involves the exploration and development of intelligent machines and software that have “the ability to collect knowledge and reason about knowledge to solve complex problems” (page 79 from [50]). AI can be used as a large language model to give natural language outputs, thus providing a realistic language experience [16] including proper human language [51]. AI software creates a dynamic experience, which is unpredictable static gameplay in which the same predetermined (order of) actions are experienced in every run of the program [52]. Therefore, AI gives users the impression that the avatars in the virtual environment exhibit intelligent behavior. This dynamic and realistic role-playing experience ensures that automation and thoughtless execution of tasks are avoided, and that what is taught during teacher training can be immediately applied. In this way, teachers in training will be immersed in a context in which they have to act as professionals and use their skills [2].

2.4. Challenges and Requirements for VR and AI

Educational technology aims at supporting the learning process, but the newest technologies such as AI and VR might feel overwhelming [53], even up to the point where it feels like the technology replaces a teacher [54]. For successful embedding in the learning practice, not only the technology itself, but also the subject, content, and instructor should be taken into account. There are plenty of challenges, which should be taken into account during development of any educational technology, and VR and AI specifically:

• High costs. Not only is the physical hardware like VR glasses expensive, but the software needs to be developed and maintained [55,56]. This also limits accessibility and scalability [53].
• Health problems, like addiction [56] and motion sickness in VR [57], which originates from a mismatch between the visual and the movement perception and expectations. Both software and hardware, like too-realistic VR views, and human factors like VR experience contribute to motion sickness, and therefore, limited VR time is recommended.
• Cognitive (over)load [53,58]. There is so much to see and explore that an immersive VR environment can be too much to digest. When there is a discrepancy between reality and what is seen in VR, attention is diverted from the learning process [55].
• Ethical aspects. AI chatbots are not human but might be perceived as such [59], which leads to trusting incorrect and biased information [60]. While humans have an ethical consciousness whether to trust someone, and hold a person accountable for their actions, AI does not have such a consciousness. Also, for the general public, AI is more of a black box than something they can understand and use [60,61].
• Instructor influence. An instructor is essential in conveying knowledge, and therefore remains absolutely necessary to embed technology in education [54]. Due to limited experience with these new technologies [55], instructors might not be able to judge the complexities [53]. They could develop a feeling of alienation or ignorance [61] and turn away from AI and VR. To successfully embed technology in education, instructors will need proper training [59].
• Lack of strategy and evaluation criteria, for VR [53] and AI [55]. For example, how to assess and compare different implementations of VR, and how to detect and deal with fraudulent use of AI. Evaluation might be possible through a very general System Usability Skill [62], to allow for assessing whether the educational technology is right for its purpose and not just edutainment.

These challenges will need to be included in the evaluation of our proof-of-concept application, to optimize and safeguard the usability of the application in our teacher education.

3. Materials and Methods

In crafting a realistic VR application for classroom management, we went through a first cycle of design-based research (DBR) [18] and adhered to global standards [63].

DBR is a multi-cyclic process of repeated design, implementation, analysis, and evaluation. This paper reports on the first cycle, executed by the authors. In future cycles, other stakeholders will be involved.

The overall design process encompasses two principal elements: the technical implementation and the algorithm development. Technical implementation involves the creation of a virtual classroom and enabling the avatars to move and speak. The motions and interactions within the virtual space are governed by prompt design to model the behavior of the virtual students and thus to create meaningful interactions.

3.1. Technological Implementation

The development of the environment took place in the VR zone of our university, which has the required hardware and software (thus reducing the equipment cost). We opted for an immersive VR experience with a standalone Quest 2 headset [64]. The local version of the environment enables teachers to move more freely and use the application outside the development zone. The environment, developed in Unreal Engine [65], incorporates ReadyPlayerMe avatars [66]. Communication with avatars is facilitated through Azure [67], to enable bidirectional transfer, namely, text to speech and speech to text. An OpenAI platform [68] prompt is used to solicit and model the virtual student’s responses.

Movement within the virtual environment is controlled with the controller’s joystick, and recording of what the trainee says begins when he/she presses a designated button, like in a walkie-talkie. Figure 1 and Figure 2 illustrate both the front-end and back-end of the environment.

In the VR environment, several features are designed to support the trainee in practice learning: the name tag of students on the table, the virtual student’s text (displayed on top of the student), and the trainee’s text (displayed at the bottom of the screen).

3.2. The Virtual Student’s Behavior as Algorithm

To model the virtual student’s behavior, we created individual character sheets for each virtual student, detailing their aptitude for conversation and their reaction speed to various external stimuli. For the sake of simplicity, we streamlined the characterization of virtual students to a single mood factor. This factor ranges from calm and cooperative (‘green’) to agitated and anxious (‘red’).

Teachers in training and instructors of the Science and Engineering Education (SEEd) department compiled a thorough inventory of undesirable student behaviors. They proposed employing a linear escalation ladder to address these behaviors. While initially this ladder seemed compatible with our chosen mood factor, we soon recognized that classroom situations are influenced by multiple variables and require a multi-dimensional approach. Consequently, we chose to implement a flow diagram instead, which resulted in identification of variables that influence transitions between states.

We initially modeled a single type of virtual student behavior (grey highlight in Figure 3), with the intention of expanding to additional behaviors once we have established a framework. The behavior selected for our application’s initial focus is disruptive classroom conversation. The considered factors were the relation of the topic of the conversation to the learning goal, the volume (loud or soft), and the trainee’s response to a student. As depicted in Figure 3, virtual students talking off-topic and loudly (the top orange “loud”) can be either redirected to start working (in yellow), or become upset if corrected in a negative manner (in red). The colored smileys serve as indicators of the escalating (one dimensional) behavior, in a multi-dimensional flow chart.

The behavioral modeling is implemented within the code of Unreal Engine using standard conditional statements. Effectiveness is determined by a combination of factors including spoken words, the distance between trainee and virtual student, and whether the trainee’s gaze is towards the virtual student. The mood factor is represented by a number from 1 to 10, with changes in increments up to 3, based on derived effectiveness (depending on the contribution of content, distance, and gaze). The determination of whether the impact on the mood factor is positive, negative, or neutral, is facilitated via OpenAI, version GPT-3.5.

The virtual student’s mood is not only influenced by trainee–student interaction. Much like in a regular classroom, virtual students also observe and are influenced by their peers’ behavior. We simulated this via a ‘mood contagion’ mechanism, similar to a pandemic disease simulation. To simplify the process, the environment initiates one virtual student with an elevated mood. The proximity between virtual students increases their likelihood of getting ‘infected’ and experiencing a similar mood change. A negative mood from one virtual student spreads in the classroom, at a rate of one virtual student per 30 s. The trainee’s challenge lies in identifying the virtual student who is distracted or causes distractions and in intervening to restore a positive mood.

Furthermore, the mood of each virtual student is programmed to gradually deteriorate every 30 s. This mirrors the fact that adolescents are susceptible to distraction and have limited cognitive control over their executive functions [69,70,71], and may therefore benefit from guidance to maintain focus. For the trainee, balancing between providing individual attention to a specific student and monitoring the whole class poses a challenge.

3.3. Front-End Design for OpenAI

OpenAI is used to assess the factors, represented by the white blocks in the flowchart, (Figure 3), considering all inputs to determine how to adjust the state of the virtual student, which currently only involves a mood factor. Utilizing prompt design with OpenAI, we applied a ‘classification’ approach to assess whether a sentence is positive, negative, or neutral. To aid the model’s decision-making process, we supplied example sentences categorized correctly. As a result, OpenAI generates a corresponding response, to be spoken by the virtual student.

To distinguish between volume levels, the sound level is compared to a threshold value, which will be established through measuring teacher input during the VR introduction. Interpretation of the trainee’s behavior by AI is based on classified example statements that are provided (see

Table 1. Prompt design, containing 1. example statements, 2. situational description, 3. example teacher’s text, which are the inputs to AI. In grey, the output (classification of the trainee’s text) from AI is displayed.

1. Example statements	“What on earth do you think you are doing” is negative. “Could you discuss with one another with lower volume” is neutral. “You are doing a good job” is positive.
2. Description of the situation	Bob is concentrated and working on Pythagoras’ theorem. Bob is 12 years old, does not understand the exercise, and asks: “Can you explain me how I can apply Pythagoras’ theorem?”
3. Input trainee + Output classification AI	“Well, isn’t that on page 100?”	Neutral
	“Didn’t you just read that?”	Neutral
	“Surely you can do that yourself?”	Negative
	“That’s a good question, let me explain.”	Positive

). If the generated classification proves inadequate, further future finetuning and prompt designing is required. In Table 1, an example of a situation, input and classification is given.

The classification proved to be adequate, but in an initial run, our virtual student Bob tended to respond with excessive politeness, not reflecting the idiom of a typical 14-year-old student. For instance, in response to a positive teacher remark, Bob replied “Thank you very much; I sincerely appreciate you take your valuable time to help me” and this completely lacks the casual voice expected from a young Dutch adolescent. Therefore, we introduced additional context information to the situation information represented in Table 1, including preferential topics and example phrases of adolescents, also to enable more realistic conversations between two virtual students, as well as more authentic replies to the teacher in training, such as “That would be great”, or an impatient “Why aren’t you helping me?” The number and variety of examples provided contributed to the authenticity of the updated answer.

4. Results

The primary objective of our application is to enhance teacher-in-training education by creating a VR application with the highest possible level of authenticity. We analyzed this by comparing it to three criteria pertaining to the virtual environment itself, the visual appearance of the virtual students, and the interactive behavior of the virtual students. For each criterion, we outline the measures taken to enhance authenticity, and describe the implementations from the point of view of the developers/authors.

4.1. Authentic Virtual Environment

The teachers in training, our future participants, must experience the virtual classroom with a high level of authenticity with regard to all factors that have impact on interpersonal skills development. To accomplish this, we created the environment from an image of a regular Dutch classroom, featuring three rows of two tables adjacent to each other, and three tables positioned behind each other, reflecting a conventional frontal teaching classroom. Note that in a standard classroom, there would be three rows of five tables deep to accommodate 30 students. However, for testing purposes and due to uncertainty about the computational demands of the standalone application, we began with a smaller classroom.

All authors, put in the trainee position, were able to easily identify their location in the classroom in relation to the tables for the virtual students. Even in an empty classroom, it is essential that a teacher is mobile. While physical movement could be achieved by taking real steps, this would require a large empty space. Instead, movement is simulated using a joystick, allowing the trainee to physically turn in the desired direction, and take simulated one-meter steps with the joystick. The easiness of the control, plus limiting the time in VR to 10 min, ensured avoiding motion sickness.

During tests, we found that trainees easily adapt to the movement controls. The trainee’s height was standardized at 1.70 m. The virtual classroom is shown in Figure 4.

4.2. Appearance of the Virtual Students

The virtual student’s appearance is perceived through two sensory modalities: sight (encompassing both looks and movement) and hearing (sounds and speech). We use existing avatars [66], which exhibit distinctly human characteristics while also being clearly animated. The choice for such avatars, besides availability, ensures that trainees feel at ease (as they are not interacting with real humans) while quickly recognizing the setting [72]. The variables for each avatar include gender, height, skin color, clothing, hair style, facial features, and a name. To ensure diversity and distinguishability among avatars, these variables are randomly selected. Although not designed to resemble specific individuals, one author found an avatar strikingly resembled one of their students, underscoring the realistic character of that specific avatar.

In terms of auditory perception, there are only three Dutch voices through Azure, or five if we include Flemish [67]. To introduce variation, we manipulated the pitch (i.e., highness of tones in a voice) and speed. We think that this variation resulted in the avatars’ speech being perceived as sufficiently different.

4.3. Behavior of the Virtual Students

The behavior of our virtual students is influenced by the teacher in training’s language, gaze, and position. All virtual students in our virtual classroom are interactive, with unoccupied chairs reserved for additional non-interactive avatars (avatars with which the trainee cannot interact, and who will not respond to the trainee). Currently, virtual students exhibit no immediate response unless directly looked at or addressed by name. In the non-responsive situation, an animation corresponding to their current state in their flowchart (Figure 3) is triggered in time. Care is taken to ensure that these background movements are asynchronous and occur at slightly different speeds to prevent the ‘uncanny valley effect’ [73], which would occur when every virtual student displays the same behavior simultaneously.

Virtual students (re)direct their gaze towards the trainee when addressed by name, or when the trainee is in close proximity (nearer than one meter). The rotation of the virtual student’s heads can occur in either direction but is constrained to the shortest angular distance, to prevent unnatural movement (e.g., almost 360 degrees).

In the standard scenario, the virtual students are supposed to work independently on a task defined by the trainee. During interactions, the virtual students respond to their own name, as well as to a group of names and the phrase ‘everyone’, although the latter can only be used once every minute, as the standard scenario implies independent work practice. When a trainee addresses multiple virtual students, only one will respond, acting as a representative of the group, to prevent interference of output given by several avatars all at once. Initially, the virtual students lacked memory, resulting in a lack of continuity in interactions, as they were unable to elaborate on previous interactions. Therefore, every line of interaction with the virtual students is added to the context (like Table 1), resulting in realistic responses within a prolonged conversation.

During interaction, the virtual student should only hear what the trainee intends to say, without picking up accidental background noise. The trainee clicks a button and waits for a red light in the application, similar to an ‘on air’ sign. The trainee then speaks, and after a brief pause, the listening function stops. Initially, there was some delay, which posed a challenge because it is natural to begin speaking immediately. However, software updates resulted in faster processing time, making both speaking by the trainee and responding by the avatar more natural and without delay.

5. Discussion

5.1. Advantages of Interactive VR Applications in Teacher Training Education

While conventional observation and learning by doing offer authentic engagement, and role-playing exercises afford immediate expert feedback, the possibilities are limited for systematic trying and improving a wide variety of interpersonal strategies in different teacher roles to a wide variety of students and in a wide variety of classroom cultures. VR offers far more accessibility in space and time (e.g., see [37,42]), but most current VR classroom applications either require simultaneous expert input [13,14,15] or are merely passive observations in an immersive virtual environment [2]. With our environment, we managed to combine the best of both by including interactive AI-controlled interaction.

AI algorithms enable dynamic authentic interactions, which can be tailored by prompt design. The provided contexts lead to realistic verbal responses [74]. AI also leads to natural variation in avatar behaviour and is adaptive to the avatar characteristics, like mood, and sensitive to the teacher’s style of interaction. The adaptive nature of AI therefore leads to authentic learning experiences and enhances the development of a varied repertoire.

A careful balance is made between the level of authenticity, and number of options and details within the VR environment. This is done to avoid cognitive overload and keep the focus on the teacher in training’s task, which is to manage the classroom.

5.2. Educational Implications and the Role of the Teacher

The combination of VR and AI can result in active learning [75], but only if the essential role of the teacher is well employed. Most crucial is embedding the simulation in the whole degree program, to ensure alignment between goals, teaching formats, and assessment [76]. Before engaging with any technology, the instructor needs to explain the purpose of the simulation to the teacher in training, and after (or even during) simulations, constructive feedback must be given [11] and the instructor needs to discuss transferability from the VR environment to the real (internship) classroom. The instructor can help to adapt the environment to the specific needs and learning goals of different teachers in training through extended prompt design with appropriate context features, and ask reflective follow-up questions [77]. Only then can the combination of human control and automation be more than the sum of the human and AI parts [78].

5.3. Future Development of This VR Application

Apart from incremental improvements and testing, we will take on several design challenges for the next version. The current version is limited to verbal interaction. Therefore, the next step is to include non-verbal interaction, which leads to new (and old) challenges, including the necessity [15] to record the trainee’s facial expressions and posture, translate these into a certain mood, and combine this with the verbal input. On the virtual student’s side, the animation of the virtual student’s facial expression and posture must match their spoken responses. A quite different challenge is the inclusion of various learning theories and opinions on classroom order. Also, we plan to develop scenarios for formats that deviate from whole class instruction, such as supervising small project groups. The aim is that our teachers in training can experience and practice all these theories and situations instead of just hearing or reading about them.

We also want to further develop the character of our virtual students to enable them to act as unique individuals. This can be achieved through dedicated character sheets with relevant background information such as hobbies, native tongue, intrinsic motivation for a subject, and academic performance level.

When further developing the interactive VR and AI environment, we envision that the computational demand will be higher, as more information needs to be processed and classified. Therefore, previous decisions might have to be reconsidered, like whether the application is standalone or not, how many interactive avatars should be included, and how to deal with recognizing gesturing, because this conflicts with using a joystick to speak and walk.

6. Conclusions

We conclude that our environment meets the criteria and is a proof of concept for an environment that promises to deliver engaging and lifelike learning experiences for teachers in training, by combining the immersive capabilities of VR and the versatility of AI algorithms. The environment already demonstrates a significant level of authenticity because of the use of large language AI models. Besides the recognizable looks, the virtual classroom and virtual students appear and behave in a sufficiently realistic manner. Within this safe and accessible environment, teachers in training can practice and develop their skills in scenarios they can identify with and yet are unpredictable. This meets their needs for developing confidence and repertoire [2] and simultaneously reduces anxiety.

Obviously, testing with teachers in training is the next step. While the first design cycle with the authors as expert teachers gave excellent opportunities for development and improvements, we need to follow up with (future) rigorous testing with our main stakeholder, which are the teachers in training. Then, instructors, who play the biggest role in effectively embedding the VR and AI environment in the MSc education courses, will also be included.

Future developments point at the inclusion of non-verbal behavior, different theoretical views on classroom management, and different teaching formats, which, we hope, will advance this interactive VR and AI environment to an effective and less instructor-intensive learning environment for teachers in training.