Boosting Engineering Education with Virtual Reality: An Experiment to Enhance Student Knowledge Retention

: This article is about experiments investigating teaching and learning processes and their effects on students. Specifically, the laboratory experiment method aims to determine if using virtual reality in classes leads to better learning outcomes, knowledge retention, satisfaction, engagement, and attractiveness compared to traditional teaching methods. The study found that students who used VR (Experimental Group—EG) had significantly better learning outcomes (with an average of 5.9747) compared to the control group (Control Group—CG), who only had traditional classes (with an average of 4.6229). The study employed a Likert scale from 1 to 7. The difference between EG and CG was 29.2%. Furthermore, the study found that students in the EG had higher knowledge retention, satisfaction, engagement, and attractiveness compared to the CG. All measurements were above 6.4 on the same scale. This study is important because it explores innovative teaching methods and their potential to improve learning outcomes, satisfaction, and efficiency. It also opens up avenues for further research on teaching methodologies for undergraduate students.


Introduction
The search for more efficient teaching-learning approaches and positive impacts on student learning is ongoing and carried out by researchers, teachers, and professionals in the field of education.In response to this discussion and the need for change, a conference and research article addressed MIT's program for this purpose [1].The research article explored the integration of new technologies in engineering education, emphasizing the positive impact of cognitive approaches for the teaching-learning process on the training and performance of engineers.The teaching-learning process plays a crucial role in engineering education, connecting theoretical knowledge with practical application.Cognitive internships organized before professional courses transform students' engineering knowledge from a superficial understanding to a broader awareness [2].New approaches employed to attain the anticipated level of efficacy involve the implementation of Education 4.0 principles, which encompass the utilization of advanced and transformative methodologies targeted at higher education [3][4][5][6][7][8].
The use of VR in engineering education has shown a 60% increase in learning through experiments compared to traditional teaching methods [29].Additionally, 83% of participants reported greater satisfaction with the use of VR [30].However, there needs to be more publications on the application of VR in teaching-learning processes, specifically in the field of production engineering, with limited topics such as supply chain management, process flow, layout organization, and lean manufacturing concepts [31,32].There are several opportunities for future research in areas such as production systems management, production planning and control, material handling, production simulation, production process management, quality control, work organization, risk analysis and prevention, and ergonomics [33], as well as emergent fields such as Industry 5.0 and renewable energy systems management.
The purpose of this study is to assess the effectiveness of VR in enhancing learning outcomes compared to traditional teaching methods.Through an experimental setup involving two distinct groups, the study seeks to determine whether the group exposed to VR exhibits superior learning rates compared to those subjected to traditional teaching.It also aims to measure the extent of knowledge retention across both groups as well as the perceived levels of satisfaction, engagement, and attractiveness among students utilizing VR in contrast to students following conventional teaching approaches.
The study is important because it delves into the area of production sequencing, which has yet to be thoroughly examined in the realm of production engineering processes with VR.Additionally, the utilization of traditional teaching methods in engineering classes, given the new technologies implemented in the manufacturing realm, may lead to lower learning rates and knowledge gaps [33].Previous studies have shown positive impacts on learning [34,35] and on the satisfaction, engagement, and attractiveness perceived by students subjected to VR approaches [36,37].
The rest of the article is structured in terms of theoretical background, materials and methods, results, discussions, and conclusions.

Theoretical Background
Consistently, educators employ technological tools to enhance teaching and learning processes, with the goal of promoting more substantial student learning.Notable examples include gamification [38], PhET simulations [39], Kahoot [40], augmented reality [41,42], the Internet of Things (IoT) [43], flipped classrooms [44,45], and VR [46,47], among others.Teaching strategies have their own characteristics.For instance, industrial gamification applies game elements to enhance learning [48,49], problem-solving, cooperation, motivation, and engagement [38].PhET simulation is a platform that creates interactive simulations in physics, chemistry, mathematics, earth sciences, and biology to facilitate students' learning processes, participation, and engagement [50,51].Kahoot is an educational gaming platform that allows students to play interactive games and quizzes to improve their absorption of subjects and disciplines [40,52].
Along with VR, augmented reality (AR) offers the exciting potential for virtual objects to be seamlessly integrated into the real world, allowing for human interaction and visualization [53].This cutting-edge technology immerses users in a multi-sensory experience by projecting virtual objects directly into the human eye through optical headsets, AR glasses, and other display devices.Virtual objects turn out to be incredibly realistic, with detailed surface characteristics and high scene resolution [54,55].VR and AR need to rely on the Internet of Things (IoT) infrastructures, which refer to a network of physical objects embedded with sensors, software, and other technologies that enable them to communicate and share data with other devices and systems via the Internet.IoT integrated with VR and AR has created new possibilities in the realm of education, fostering improved interactions between people and physical objects in a classroom environment [55,56].
The flipped classroom model involves students being introduced to the subject material before coming to class, allowing them to develop initial understanding and concepts.This preparatory work often takes place in virtual teaching environments through various interactive activities such as games, videos, and exercises [44].VR technology can create fully immersive and interactive environments, offering users a unique and impactful experience that can enhance cognitive performance in various ways, including access, awareness, engagement, creativity, and action [37,57].Although VR is not widely used in engineering education, it has the potential to create a virtual learning environment that can greatly benefit students' cognitive performance [33,46].

VR and Impact on Learning
Previous studies have highlighted VR's positive effects on the teaching and learning experience, including increased learning and problem-solving abilities [35].VR has also been shown to enhance the learning experience in dentistry classes and support proper positioning in physical exercises [34,58].Additionally, it has been found to improve understanding of flat and three-dimensional geometric figures [59] and foster greater creativity in elementary school students [60].Specifically in engineering education, VR has shown positive learning outcomes compared to traditional teaching methods.Such outcomes include an increased understanding of automation [61], a focus on visualizing projects and systems [62], improved learning in software engineering development training compared to traditional methods [63], and greater understanding in classes on mechanical testing [64].Specifically for engineering software development, VR may lead to more than 100% improvement in learning [65].
Conversely, certain studies indicate that VR may have a less significant impact on student learning, particularly in areas related to product development [25].However, in the field of robotics, the findings suggest a greater understanding of certain topics when VR is utilized [26].While VR is commonly employed in educational settings with various learning outcomes, there are instances where the results are not optimistic.Various studies have evaluated the impact of knowledge retention, although to a lesser extent and with varying results.For example, in laparoscopy surgery training, no differences in skills and knowledge were observed between groups using traditional teaching methods versus VR support after one month [66].However, there are conflicting findings, with one study showing significant retention of learning by children [67], while another study found that students using VR achieved 80% retention compared to 25% retention for those using traditional teaching methods [68].

VR and Impact on Student Satisfaction and Engagement
Previous studies have assessed various aspects of incorporating VR into teaching and learning, including user satisfaction, engagement, and attractiveness.For instance, a study found a 76% increase in satisfaction among dentistry students [34], while another reported a 4.28 motivation score (on a Likert scale of 1-5) for science students [37].Additionally, VR was found to enhance engagement in training for industrial systems [36] and boost motivation in practical nursing studies [69].
Regarding engineering education processes, numerous studies report positive outcomes.For example, research has shown increased engagement in testing mechanical materials [64], greater satisfaction in developing prototypes with VR and additive manufacturing [70], and a 92% student understanding that VR enhances learning and makes the process more attractive, leading to increased engagement in classes [71].Additionally, a survey revealed that 92% of students feel more satisfied when using VR in teaching and learning processes, particularly in the context of electrical systems [72] and the simulation of welding processes with robots [73].
Conversely, the research conducted by [74] did not find any significant increase in motivation, satisfaction, or any other attribute among participants as a result of using the virtual environment, particularly in the cultural training program for high school students.In the earthmoving simulation, participants noted numerous flaws in the simulator and found it difficult to operate, ultimately impacting their satisfaction [75].Similarly, difficulties in manipulating the applications were observed during the construction of bridge projects, which hindered the potential benefits of the new application [76].
Based on the search for new disruptive technologies for application in the teachinglearning processes of production engineering, future research opportunities, and the positive impacts related to the learning and satisfaction of participants in the experiments, the following three research questions are presented: • RQ1-Is there more significant learning from students who use VR in teachinglearning than from students who only take traditional classes?• RQ2-Did the group that used VR in the teaching-learning processes perceive satisfaction, engagement, and attractiveness in the activities, compared to classes that only used the traditional teaching method?• RQ3-Does applying VR impact knowledge retention three weeks after using the knowledge test acquired by students?

Materials and Methods
The study utilizes a laboratory experiment.Unlike a field experiment, a laboratory experiment operates on a smaller scale and controls a limited number of variables.In contrast, a field experiment would involve true magnitude and require a longer duration, potentially impacting academic records with potential consequences for students.The presence of real agents and their genuine responses distinguishes the laboratory experiment from an in silico experiment, which simulates agent behavior using software, which can lead to errors if the behaviors are not fully known or modeled.The advantage of a laboratory experiment is the ability to make corrections without real-world implications, but it may overlook certain variables and nuances present in a larger-scale experiment.Following the experiment, a mathematical analysis is required to validate results and test hypotheses.The EG and the CG were chosen for comparative analysis [76].
The study was structured into six stages, which are as follows: 1. Definition of research objectives; 2.
Development of the VR artifact; 3.
Validation of the VR artifact with experts; 4.
Application of the VR artifact in controlled groups; 5.
Analysis and interpretation of results; 6.
Conclusions and proposals for future research.
The introduction describes the first step of the method and presents the cognitive approach to teaching.VR is also discussed, along with some examples of its application and results in learning, satisfaction, attractiveness, and student engagement.The second, third, and fourth steps reference the steps presented in Figure 1.Step 5 analyzed the results and discussed the literature on the topic.Finally, step 6 presented conclusions and proposals for future research.
The steps presented in Figure 1 are described below: In Stage I, the model's assumptions were utilized to simulate production scenarios.Such assumptions involved adjusting various inputs such as processing times, production batches, setup times, MTBF, and MTTR.The production system output indicators, such as WIP levels, production lead time, quantities produced, and production costs, were then evaluated.
In Stage II, the Plant Simulation software was utilized to model dynamic systems, allowing for the simulation of input changes and control of output indicators, as well as the generation of 3D VR images.Non-immersive VR was employed without direct interaction with students.The steps presented in Figure 1 are described below: In Stage I, the model's assumptions were utilized to simulate production scenarios.Such assumptions involved adjusting various inputs such as processing times, production batches, setup times, MTBF, and MTTR.The production system output indicators, such as WIP levels, production lead time, quantities produced, and production costs, were then evaluated.
In Stage II, the Plant Simulation software was utilized to model dynamic systems, allowing for the simulation of input changes and control of output indicators, as well as the generation of 3D VR images.Non-immersive VR was employed without direct interaction with students.
Moving on to Stage III, the researchers presented and validated the VR model with three experts: two engineering professors and a market professional with over a decade of experience in production sequencing.This validation process involved a comprehensive evaluation of the model's input data, output indicators, and the visual and dynamic aspects of the VR environment.
In Stage IV of the study, an experiment was conducted to assess the impact of a specific teaching method on student learning, satisfaction, engagement, and attractiveness.The sample comprised 68 students from the Production Engineering Course, with 34 students in each group-the control group (CG) and the experimental group (EG).The sample was defined in a non-probabilistic way for convenience [77].The total student population in the course is 230.All students were invited to participate, and those who agreed formed the two groups based on availability.The control group received ten hours of traditional teaching.In comparison, the experimental group received six and a half hours of traditional teaching and an additional three and a half hours with VR support.Three weeks after the last session, a study was conducted to evaluate the knowledge retention level of each group [68].
Stages V, VI, and VII-three stages were applied for knowledge factual assessments, and four analyses were carried out.RQ1 is investigated by testing hypotheses H1, H2, and H3.RQ2 is investigated by the qualitative part of the questionnaire presented in Appendix B, which belongs to Stage VII of the method.RQ3 is investigated by testing hypothesis H4.The hypotheses are as follows: • H1-the previous knowledge of the groups is the same on production sequencing.The hypothesis alternative is H11-there is a difference in the previous knowledge between the groups on production sequencing.Moving on to Stage III, the researchers presented and validated the VR model with three experts: two engineering professors and a market professional with over a decade of experience in production sequencing.This validation process involved a comprehensive evaluation of the model's input data, output indicators, and the visual and dynamic aspects of the VR environment.
In Stage IV of the study, an experiment was conducted to assess the impact of a specific teaching method on student learning, satisfaction, engagement, and attractiveness.The sample comprised 68 students from the Production Engineering Course, with 34 students in each group-the control group (CG) and the experimental group (EG).The sample was defined in a non-probabilistic way for convenience [77].The total student population in the course is 230.All students were invited to participate, and those who agreed formed the two groups based on availability.The control group received ten hours of traditional teaching.In comparison, the experimental group received six and a half hours of traditional teaching and an additional three and a half hours with VR support.Three weeks after the last session, a study was conducted to evaluate the knowledge retention level of each group [68].
Stages V, VI, and VII-three stages were applied for knowledge factual assessments, and four analyses were carried out.RQ1 is investigated by testing hypotheses H1, H2, and H3.RQ2 is investigated by the qualitative part of the questionnaire presented in Appendix B, which belongs to Stage VII of the method.RQ3 is investigated by testing hypothesis H4.The hypotheses are as follows: • H1-the previous knowledge of the groups is the same on production sequencing.The hypothesis alternative is H 11 -there is a difference in the previous knowledge between the groups on production sequencing.

•
H2-the knowledge of the groups on production sequencing did not vary after classes.The hypothesis alternative is H 21 -there is a difference between the current and previous knowledge of groups on production sequencing after classes.

•
H3-the learning levels of the groups on production sequencing after classes are the same.The hypothesis alternative is H 31 -there is a difference between the learning levels of the groups on production sequencing after classes.

•
H4-the retention of knowledge by the groups on production sequencing is the same three weeks after classes.The hypothesis alternative is H 41 -there is a difference between the retention of knowledge of the groups on production sequencing after three weeks.
The p-value must be <0.01 in all evaluations to accept the alternative hypothesis.
A preliminary test was conducted to gauge the existing knowledge of both groups.Following the third meeting, a subsequent test was administered to assess the learning progress of each group.Lastly, three weeks after the conclusion of the last class, a test measured the retention of knowledge among the groups.All tests were scored on a scale from 0.00 to 10.00.Table 1 shows the quantitative method utilized to compare the groups, which was the Student's t-test, and the findings.The t-test was selected to determine if the difference in means is statistically significant [77].The Shapiro-Wilk test was used to assess the normality of the data from each group based on their prior knowledge and retention.To accept the alternative hypothesis, a significance level of α = 1% was applied to all tests.SPSS version 22 was used for the analysis.The results of the normality tests can be found in Appendix A.
In the final stage, the experimental group received a questionnaire consisting of 12 questions.The questions were divided into three blocks concerning issues related to satisfaction (questions 1-4), engagement (questions 5-8), and attractiveness (questions 9-12).The Likert scale, ranging from 1 to 7, was used for the questionnaire.The Likert scale is a widely used tool in research to measure people's attitudes and opinions.It provides a systematic method for quantifying subjective perspectives, ensuring a balanced range of choices.It includes a central neutral point surrounded by an equal number of options on each side of the scale [78].The questionnaire and scale are available in Appendix B. Six qualitative questions were utilized to assess students' perceptions of satisfaction, appeal, and involvement in using VR for teaching and learning.Qualitative responses were categorized according to at least one keyword: ease of comprehension, realism, increased class dynamism, attractiveness, engagement, willingness to participate, scenario-based examples, class interest, reduced boredom, enjoyable experience, enhanced class participation, motivation, attentiveness, and positive impact on happiness.
References [29,30,34,[79][80][81] provide empirical evidence on the feasibility and applicability of the questions, ensuring the validity of the questionnaire.Previous empirical studies involving the aspects captured by the questions ensure that the involved issues are likely to be found in such an experiment.The quantitative part of the questionnaire was structured into three blocks that focused on aspects related, respectively, to satisfaction, engagement, and attractiveness of the application.Cronbach's alpha calculation helped to assess the reliability of the responses.The reliability was confirmed as all alphas are greater than 0.60 [76] (0.73, 0.83, and 0.69, respectively).A limitation of the study is the identification of underlying factors, which is forwarded for further applications with an adequate number of respondents.Appendix B shows the questions, blocks, references, and type of response, quantitative or qualitative.
The Research Ethics Committee approved this research from the Federal Institute of Rio Grande do Sul-IFRS (Brazil) under CAAE 52081521.0.0000.8024.

Results
Table 2 provides demographic information on the sample.The average age of the participants 28.9 years 28.7 years Previously familiar with the topic of production sequencing 6 7

Description of Artifact
The VR artifact was created to integrate key elements into the teaching and learning of production sequencing.It was developed using Plant Simulation software to simulate dynamic systems, specifically a production system, through a VR application.The VR artifact utilized six different scenarios to model the production system, with each scenario adjusting input data and controlling output information [79].The data input involved processing time per product, setup time, demand, production batch size, available capacity, production flow, and equipment availability indexes (maintenance and reliability).The system was designed to operate over 14 days, producing eight different products, each with its own specific data.We evaluated the quantities produced, demand fulfillment, cash flow generated, average and maximum production lead time, OEE (overall equipment efficiency), number of setups, and in-process stock levels for each scenario.The initial development of the artifact, including simulations and parameter adjustments, took 40 h.During the validation stage, each evaluator received individual presentations.The evaluators noted several positive aspects, including the impact of scenarios on evaluated indicators, the ability to visualize the production system's operation, the dynamics of inventory in the process, and the ease of adjusting scenario input data.
Additionally, a list was created with six opportunities to modify parameters and the visual aspects of the VR environment.The opportunities included adjusting the production batch size parameter to observe the impact on in-process inventory levels and manufacturing lead time, highlighting the effect on cash flow by increasing work-in-process inventory levels, analyzing the quantity produced exceeding the expected demand for the 14-day simulation, entering a different color for each product in the VR model, incorporating conveyors to enhance material flow visualization in VR, and limiting the number of parts on the conveyor belt awaiting processing to five units to emphasize in-process stock in the buffers.After adjustments, the model was presented to all experts collectively, and all features were reassessed.The experts deemed the model suitable for implementation in achieving the proposed objective.Figure 2 displays the VR images of the production process.
emphasize in-process stock in the buffers.After adjustments, the model was presented to all experts collectively, and all features were reassessed.The experts deemed the model suitable for implementation in achieving the proposed objective.Figure 2 displays the VR images of the production process.

Results of the Previous Knowledge and Retention Tests
The knowledge of the control and experimental groups regarding the evaluated topic was compared before the first class and after correcting the pre-tests, which assess knowledge of production sequencing.After assessing the results, no statistical differences were found between the groups.The results are detailed in Table 3.In Table 3, CG and EG achieved prior knowledge scores of 3.5091 and 3.9361, respectively.Therefore, the first alternative hypothesis (H11) is not supported, suggesting similar levels of previous knowledge in production sequencing.Subsequently, the knowledge acquired by each group after attending the classes was assessed, and the results of the learning t-tests are detailed in Tables 4 and 5.

Results of the Previous Knowledge and Retention Tests
The knowledge of the control and experimental groups regarding the evaluated topic was compared before the first class and after correcting the pre-tests, which assess knowledge of production sequencing.After assessing the results, no statistical differences were found between the groups.The results are detailed in Table 3.In Table 3, CG and EG achieved prior knowledge scores of 3.5091 and 3.9361, respectively.Therefore, the first alternative hypothesis (H 11 ) is not supported, suggesting similar levels of previous knowledge in production sequencing.Subsequently, the knowledge acquired by each group after attending the classes was assessed, and the results of the learning t-tests are detailed in Tables 4 and 5. Tables 4 and 5, which display the results of the t-test, suggest that the second alternative hypothesis, H 21 , is supported based on the level of significance (p-value < 0.01).The results indicate a 51.8% increase in the EG and a 31.7%increase in the CG in the average knowledge acquired from the post-test compared to the pre-test.An additional analysis was carried out to determine whether any group showed greater improvement, and the findings are outlined in Table 6.Table 6 displays the result of the knowledge test administered after classes, with scores of 4.6229 and 5.9747 for the control group (CG) and experimental group (EG), respectively.The t-test yielded a p-value of 0.000012 (less than 0.01).The third alternative hypothesis, H 31 , is supported.The EG group's result was 29.2% higher.EG, who had VR-supported classes, achieved more significant and statistically meaningful learning outcomes than the CG.Table 7 analyzes knowledge retention between the groups conducted three weeks after the initial knowledge tests.The data in Table 7, which evaluate the results for hypothesis 4, show that the experimental group maintained higher results than the control group after three weeks of testing acquired knowledge.The knowledge retention results for the experimental group were, on average, 20.1% higher than those of the control group.The t-test yielded a p-value of 0.0028 (less than 0.01).The fourth alternative hypothesis, H 41 , is supported.The EG's results are statistically higher.Additionally, further analysis was conducted on each group's knowledge and retention test results, and the findings are detailed in Table 8.Upon reviewing the data in Table 8, it is evident that the CG's knowledge retention decreased by 4.8% (from 4.6229 to 4.3988).The EG also exhibited a decrease in knowledge retention and acquisition of 11.6% (from 5.9747 to 5.2841) despite maintaining higher knowledge results.Upon reviewing the data in Table 8, it is evident that the CG's knowledge retention decreased by 4.8% (from 4.6229 to 4.3988).The EG also exhibited a decrease in knowledge retention and acquisition of 11.6% (from 5.9747 to 5.2841) despite maintaining higher knowledge results.

Assessment of Concerns Related to Satisfaction, Engagement, and Attractiveness
Figure 3 presents the EG's perception of the application of VR in teaching-learning processes with VR support.Figure 3 displays the average ratings given by participants in the experimental group for each question in the evaluation.The highest rating was 6.8, while the lowest was 6.2, indicating that, overall, the group that received VR-supported classes strongly believes that VR positively affects student satisfaction, engagement, and attractiveness in teaching.Additionally, Table 9 presents descriptive statistics for the twelve questions, providing information about the experimental group's perception of VR's impact on teaching and learning processes.Figure 3 displays the average ratings given by participants in the experimental group for each question in the evaluation.The highest rating was 6.8, while the lowest was 6.2, indicating that, overall, the group that received VR-supported classes strongly believes that VR positively affects student satisfaction, engagement, and attractiveness in teaching.Additionally, Table 9 presents descriptive statistics for the twelve questions, providing information about the experimental group's perception of VR's impact on teaching and learning processes.The results displayed in Table 9 are consistent with the summarized averages in Figure 3. None of the questionnaire responses fell below a score of 4 on the Likert scale of 1-7, indicating that, overall, the feedback was positive.Upon analyzing the 408 responses (N = 34 × 12 questions), we found only 4 neutral responses (scoring 4), and approximately 93% of respondents selected options on scales 6 and 7. Regarding the blocks (1 to 4 for satisfaction, 5 to 8 for engagement, and 9 to 12 for attractiveness), all satisfaction ratings were at least a 5, with engagement receiving 87% of responses between 6 and 7 and attractiveness garnering 98% of responses between 6 and 7.
The findings reveal that students express high levels of satisfaction, engagement, and interest in using virtual reality (VR) for educational purposes.Figure 4 illustrates that integrating VR into educational activities enhances content comprehension by offering a more immersive learning experience.Participants noted that VR usage enhances class interactivity, engagement, and dynamism.The results displayed in Table 9 are consistent with the summarized averages in Figure 3. None of the questionnaire responses fell below a score of 4 on the Likert scale of 1-7, indicating that, overall, the feedback was positive.Upon analyzing the 408 responses (N = 34 × 12 questions), we found only 4 neutral responses (scoring 4), and approximately 93% of respondents selected options on scales 6 and 7. Regarding the blocks (1 to 4 for satisfaction, 5 to 8 for engagement, and 9 to 12 for attractiveness), all satisfaction ratings were at least a 5, with engagement receiving 87% of responses between 6 and 7 and attractiveness garnering 98% of responses between 6 and 7.
The findings reveal that students express high levels of satisfaction, engagement, and interest in using virtual reality (VR) for educational purposes.Figure 4 illustrates that integrating VR into educational activities enhances content comprehension by offering a more immersive learning experience.Participants noted that VR usage enhances class interactivity, engagement, and dynamism.

Discussion
The initial finding (Tables 4 and 5) showed that regardless of the teaching method (traditional or traditional with VR support), both groups (EG and CG) learned after participating in classes on production sequencing.However, upon evaluating the group results (Table 6), it was determined that the EG achieved a statistically significant learning outcome (5.9747), which was 29.2% higher than the CG (4.6229).The application of VR in teaching-learning processes generated a real opportunity for more significant student learning, corroborating similar previous studies [34,58,59].
A follow-up assessment was conducted on the groups' knowledge retention three weeks after the initial knowledge tests [66], revealing a decrease in the average scores for both groups.The control group (CG) achieved a score of 4.3988, while the experimental group (EG) achieved a score of 5.2841.Statistical analysis yielded a p-value of 0.0028, which indicates significant differences between the groups.The data provide strong evidence that the EG has greater knowledge of production sequencing compared to the CG after three weeks, consistent with previous studies [67,68].
Furthermore, regarding knowledge retention, Table 8 illustrates a 4.8% decrease for the CG (from 4.6229 to 4.3988) and an 11.6% decrease for the EG (from 5.9747 to 5.2841).The EG lost more knowledge than the CG after three weeks.Even if the causes are outside the scope of the present study, it prompts further investigation into potential hypotheses, such as specific environmental factors or lapses in focus during the retention test.Finally, what can be seen from the results of the experiment is that students learn regardless of the teaching method applied.However, integrating VR into teaching and learning fosters a cognitive environment [1,2] conducive to deeper learning and enhanced knowledge retention over a three-week period.Further research and experimentation are required to explore the varying levels of knowledge retention among different groups.
In evaluating the satisfaction, engagement, and attractiveness indexes for EG, all average results exceeded 6.0 on a Likert scale of 1-7.The highest average (6.8) was about the class being more enjoyable with VR, and the lowest average (6.2) was about feeling a greater desire to participate through understanding the content developed in class.Table 9 presents the data from the questionnaires that assessed levels of satisfaction, engagement, and attractiveness.More than 90% of the responses had indices between 6 and 7.These results present positive and relevant impacts on student satisfaction, engagement, and attractiveness rates, as indicated by similar studies [17,34,36,[69][70][71][72][73].
A recent assessment utilized qualitative analysis of participants' essay responses (EG).The findings revealed that VR-based classes lead to a deeper understanding of the subject matter, offering a more realistic, dynamic, and engaging learning experience compared to traditional teaching methods.These results are consistent with similar studies [34][35][36][37]70].The findings indicate that establishing a cognitive teaching and learning environment using VR leads to greater learning and knowledge retention among students compared to traditional teaching methods.Participants find it engaging and satisfying, making it a more attractive option.A critical aspect of these results is the careful construction of the VR environment with expert support and a well-planned experiment.The smooth execution of these steps, along with expert knowledge, is crucial for generating knowledge and creating opportunities to develop solutions to societal needs [80].
Educational settings that prioritize teaching and learning create more effective learning outcomes for students by fostering a cognitive environment, a deeper understanding of the subject matter, and a stronger connection to real-world applications.This approach brings a professional work-like environment into the classroom, promoting dynamic and engaging classes with increased student interaction and participation.Additionally, the use of VR enhances learning and results in higher levels of student engagement, satisfaction, and concentration.

Conclusions
This article aimed to compare the learning levels of two groups: one group received traditional teaching methods, while the other group received VR support.The study also evaluated satisfaction, engagement, and attractiveness with the use of VR in the teaching-learning process.The results showed that both groups learned significantly about production sequencing, the topic studied, regardless of the teaching method.However, the group with VR support showed more significant learning after classes and retained a higher level of knowledge after three weeks.On average, the satisfaction, engagement, and attractiveness indices for the VR-supported group were close to 6.56 points (on a Likert scale of 1-7), indicating that the method generated enthusiasm and was well accepted by the students.The evaluation of qualitative questions highlighted concerns such as ease of understanding the topics covered and the appeal of virtualization, making the classes more dynamic and attractive.
The study has limitations.First, the application of VR focused only on production sequencing, so the results may not apply to some other areas of knowledge.Additionally, the study did not account for the participants' previous experiences, curiosity about the topic, and interest in the activity, which could have affected the retention of information in the VR-supported group.Another limitation is the small number of respondents, which did not allow for the identification of underlying factors in the perceptions of the students.Further studies should focus on knowledge retention over longer periods, such as three or six months after classes.Additionally, developing a standardized method for creating teaching-learning process activities and evaluating their quality and results would be beneficial.Additionally, directing attention toward emerging themes in production and operations management education, such as the shift from Industry 4.0 to human-centric

Question/Reference
Type of Question 13-Highlight some points about your satisfaction (positive or negative) with using the VR artifact in the teaching-learning processes of production sequencing.[34] Qualitative responses 14-Was there any impact (positive or negative) on your motivation to study using VR? [81] 15-Regarding engagement, did you feel more engaged (participative and willing to participate) in class when VR was applied?Please describe.[80] 16-Do you believe that the application of VR as part of the teaching-learning process generates greater attractiveness (student interest) in attending classes?[81] 17-Do you believe that the virtual environment generated by VR provides students with a greater learning opportunity?Why do you think so? [29] 18-Tell us about elements that could be different in this teaching-learning process using VR.[30]

Figure 1 .
Figure 1.Stages of development and application of the VR artifact.

Figure 1 .
Figure 1.Stages of development and application of the VR artifact.

Figure 2 .
Figure 2. Images generated by the artifact in VR.

Figure 2 .
Figure 2. Images generated by the artifact in VR.

Figure 3
Figure3presents the EG's perception of the application of VR in teaching-learning processes with VR support.

Figure 3 .
Figure 3. Satisfaction, engagement, and attractiveness indices of using VR for EG on a scale from 1 to 7.

Figure 3 .
Figure 3. Satisfaction, engagement, and attractiveness indices of using VR for EG on a scale from 1 to 7.

Figure 4 .
Figure 4. Keywords reported in qualitative questions by participants in the experimental group.

Figure 4 .
Figure 4. Keywords reported in qualitative questions by participants in the experimental group.

Table 1 .
Statistical tests applied to CG and EG to evaluate the experiment.
-Shapiro-Wilk to assess data normality -Independent sample t-test to assess whether the means obtained by the groups are similar -Assessment of the proportion of knowledge retained by each group

Table 2 .
Characterization of the groups that participated in the experiment.

Table 3 .
Result of the student t-test in the prior knowledge test.

Table 4 .
The t-test result was used to assess learning in the CG.

Table 3 .
Result of the student t-test in the prior knowledge test.

Table 4 .
The t-test result was used to assess learning in the CG.

Table 5 .
The t-test result is used to evaluate the learning of the EG.

Table 6 .
Learning similarity between groups.

Table 7 .
Knowledge retention levels after three weeks of experiments.

Table 8 .
Comparison between the results of knowledge acquisition and knowledge retention.

Table 9 .
Descriptive statistics of responses on EG's perception of using VR.