A Study on Differences in Educational Method to Periodic Inspection Work of Nuclear Power Plants

Abstract

Construction work and regular inspection work at nuclear power plants involve many special tasks, unlike general on-site work. In addition, the opportunity to transfer knowledge from skilled workers to unskilled workers is limited due to the inability to easily enter the plant and various security and radiation exposure issues. Therefore, in this study, we considered the application of virtual reality (VR) as a method to increase opportunities to learn anytime and anywhere and to transfer knowledge more effectively. In addition, as an interactive learning method to improve comprehension, we devised a system that uses hand tracking and eye tracking to allow participants to experience movements and postures that are closer to the real work in a virtual space. For hand-based work, three actions, “pinch”, “grab”, and “hold”, were reproduced depending on the sizes of the parts and tools, and visual confirmation work was reproduced by the movement of the gaze point of the eyes, faithfully reproducing the special actions of the inspection work. We confirmed that a hybrid learning process that appropriately combines the developed active learning method, using experiential VR, with conventional passive learning methods, using paper and video, can improve the comprehension and retention of special work at nuclear power plants.

1. Introduction

1.1. Background

In this study, we investigated the transfer of technology during regular inspections of nuclear power plants. When we look at the construction of nuclear power plants, the problem is that skilled workers who have experienced many construction projects in the past are unable to transfer their knowledge. Nuclear power plants are a special industry compared with other general plants, so there have not been many construction projects in the past. As a result, the issue is that there are few skilled workers involved in nuclear power plants and few opportunities to transfer their knowledge.

Due to the shortage of skilled workers, it is feared that construction projects will not be able to proceed with the same construction period as before, and in the case of periodic inspections of large-scale plants, the reduction in the number of operating days will cause significant economic losses. Given this environment, there is an urgent need for a system to pass on the knowledge that they have accumulated to the younger generation.

The knowledge of skilled workers is called “tacit knowledge” and exists in the minds of skilled workers on an individual basis. In the 1960s, Michael Polanyi famously referred to “tacit knowledge” with the famous phrase “we can know more than we can.” Physical knowledge is also tacit knowledge. On the other hand, knowledge that can be spoken in sentences and languages is called “explicit knowledge” [1,2]. On-the-job training (OJT) is the main method of education in the field, and tacit knowledge is handed down from skilled workers to younger workers by sharing knowledge through practical work in the field. Therefore, tacit knowledge is only shared by a limited number of members at each site, making it difficult to effectively pass it on in a systematic manner in all sites. It is said that, compared with other general construction industries, nuclear power plants have many one-of-a-kind special equipment and structures, and the need to work with specialized tools, and construction methods makes it difficult to pass on skills [3].

1.2. Knowledge Transfer Process

The knowledge transfer process uses a knowledge management framework. Knowledge acquisition is a series of processes in which human beings encounter various events through various activities, including learning, become aware of the existence of new knowledge, understand it, and incorporate the new knowledge into themselves. The typical frameworks for knowledge acquisition are models called DIKW and SECI.

The DIKW model is “a model or structure widely used in information science and knowledge management”, and it is an acronym for Data, Information, Knowledge, and Wisdom [4]. The DIKW model is represented by a pyramid shape to explain how humans acquire knowledge (Figure 1). First, it interprets “Data” that is represented by meaningless numbers, etc. Next, it becomes “Information” by giving meaning to the data. Then, by combining and systematizing the knowledge gained through experience, learning, and training for multiple pieces of information, it becomes “Knowledge”. Finally, “Wisdom” is the correct recognition of the knowledge and the evaluation of it in light of human beliefs, values, ethics, and morality.

However, the DIKW model is based on the idea of turning objectively observable data (numbers, text, etc.) into wisdom. Of course, a lot of data remain in nuclear power plant construction projects, but what is important is to pass on the tacit knowledge of skilled workers. Therefore, we decided to consider other models of knowledge transfer processes.

Another knowledge transfer process model is the SECI model. The SECI model is a framework that focuses on “systematically sharing individual knowledge to generate higher-order knowledge”, and it is an acronym for Socialization, Externalization, Combination, and Internalization [5,6]. In the SECI model, it is believed that new knowledge can be obtained by combining existing explicit knowledge as a representative model for the effective transfer of knowledge (Figure 2).

The SECI model refers to a series of process models through which tacit knowledge internalized by an individual is first shared with others in the field and is organized as “formal knowledge” by expressing it in a form that can be used by the group to which the individual belongs, then the formal knowledge is consolidated in a form that can be used among multiple groups, and finally, it is acquired by unskilled workers in the field and internalized again.

In “socialization”, it is important to have a mechanism and method for efficiently extracting tacit knowledge possessed by skilled workers in a comprehensive manner without omission as much as possible. In “externalization”, it is important to transform the extracted knowledge into explicit knowledge in the form of documents, educational videos, and other content that can be used by others and that is highly effective in passing on the knowledge. Explicit knowledge is information and knowledge that is known only to individuals and is expressed as standard and objective information on paper or electronic media. In the “combination” stage, it is important to systematize and organize the formalized knowledge as standard knowledge, and to provide a foundation that can be easily accessed and utilized by anyone. “Internalization” requires a system that enables on-site workers to effectively utilize and acquire organized formal knowledge, leading to the rationalization and efficiency improvement of work.

1.3. The Target of This Study in the Knowledge Transfer Process

In this study, we focused on the “externalization” of these processes and examined the application of virtual reality (VR) as a form of formal knowledge for effectively handing down knowledge. Currently, the means of knowledge transfer, other than the OJT, includes classroom learning using paper and video manuals, which have already been formalized, as well as practical learning using mock-up facilities at training centers.

However, when it comes to work on special structures, such as nuclear power plants, there are no general-purpose training centers in various locations, and special facilities in specific locations must be used, resulting in few opportunities for hands-on training. In the case of large-scale special structures, the mock-up facilities themselves are not available, so on-the-job training is required. In addition, the environment in which it is difficult to casually visit the work site itself is another factor that makes it difficult to hand down the knowledge, since each work site has entry restrictions based on security and radiation exposure time. In response to these issues, we decided to investigate the use of VR as a means of passing on knowledge because it is very effective for special work at nuclear power plants, as it enables immersion in a virtual space that simulates the work site and allows hands-on learning without having to choose the time or place.

1.4. Past Research

As a recent research trend of VR in construction, Li et al. surveyed and reviewed 90 papers and articles from 2000 to 2017 [7]. Among them, they stated that many applications have been published that allow users to intuitively learn work procedures in a multi-user environment and have a virtual experience that is like real work, mainly for hazard identification, safety training, and safety instruction. In a study on safety awareness, Eiris et al. compared hazard identification scores using VR and 360-degree panoramic camera hazard identification training content and concluded that VR showed higher scores [8]. Lin et al. conducted an experiment to analyze the effect of spatial knowledge on evacuation behavior during a fire using a virtual subway station model in VR and concluded that there is an interaction between evacuation behavior, evacuation time, evacuation distance, and evacuation speed, depending on the richness of spatial knowledge [9]. Shi et al. measured subjects’ movements in a VR simulation in which they walked on a board placed between two virtual skyscrapers, and reported a case in which subjects’ behavior changed depending on whether the instructions were positive or negative, which may induce risky behavior [10].

In addition, there have been many reports of studies comparing the effectiveness of VR itself with existing methods. Kang et al. compared a learning system using VR to present accident cases with a learning method using paper materials and concluded that the visual representation in VR may improve the acquisition and retention of tacit knowledge [11]. Shi et al. investigated the performance of 2D, 3D, and VR when applied to piping maintenance work education, and concluded that the 3D and VR groups were superior to the 2D group in terms of work time and work quality [12]. Sampaio et al. measured the impact of using VR in university lectures on the ability to understand bridge construction and concluded that it can support regular classes [13]. Paes et al. compared the spatial perceptual performance of virtual models and concluded that immersive VR provided an overall better spatial perceptual performance than non-immersive VR and that the experience and age of the subjects could affect the spatial perceptual performance [14]. Thus, they reported that VR (immersive) is more useful than conventional paper, 2D, and other methods.

In a report on work training, Wang et al. experimented with scaffolding assembly training using VR and concluded that VR could be a more effective approach, as there was a 12% productivity gain compared with the traditional methods [15].

There have been several reports of the application of VR to inspection work at nuclear power plants, the first is incident and accident management training, the second is work process planning and training, the third is radiation visualization and estimation, and the fourth is general physical safety training [16]. Regarding work training, reports have been conducted on fuel exchange training, maintenance activities in high-voltage environments, and training in the central control room [17,18,19]. Some studies involve displaying the VR space on a PC screen, while others use a head-mounted display and handheld VR controllers.

1.5. Research Objectives

As mentioned above, although the basic verification of the effectiveness of VR itself and research on safety awareness have been conducted, there are not many research reports on VR that reproduce more interactive work training at construction sites. In the case of training for special work, such as nuclear power plants, it is difficult to learn through VR experiences that simulate general work due to the complex work procedures for special structures and the special nature of the tools used. In the case of general work, if you have experience at other sites, you can perform similar work at a new site, but in the case of special work, you have no knowledge or experience at all and cannot demonstrate your abilities even when you enter the site. For this reason, it is necessary to conduct skills training in advance at training facilities, but as mentioned in Section 1.3, there are many challenges in education at training facilities.

Therefore, we thought it would be an effective learning method to simulate the experience of training that corresponds to the transmission of knowledge from skilled workers through on-the-job training using VR. In this study, we proposed the following new methodology and aim to verify its effectiveness.

• Development of a VR educational system that enables interactive learning while moving one’s own body as in the field.
• Making VR educational content of the special work, such as periodic inspection, at large-scale nuclear power plants.
• Verification of the differences between VR education and conventional educational methods.
• Suggestion for an educational process to improve the understanding of workers before entering the site.

2. Materials and Methods

2.1. Research Flowchart

Figure 3 shows the flowchart of this study. In the research methodology Section 2.2, we first explain the overview of the periodic inspection of nuclear power plants and the selection of the target work (Section 2.2.1). Next, we describe the detailed functions and features of the development of the experience-based VR education system (Section 2.2.2 and 2.2.3). Then, we describe the development of a support tool for creating content that operates within the VR education system (Section 2.2.4). Finally, we describe the content of the VR education content that allows the participants to experience the selected periodic inspection work (Section 2.2.5).

In Section 2.3, we describe a method for analyzing the difference in comprehension compared with traditional educational methods, such as paper and video in order to measure the impact of the created VR education content on comprehension (Section 2.3.1). We also analyze the comprehension of hybrid learning when each teaching material is combined (Section 2.3.2). These comprehension evaluations are quantitatively measured using written tests. Next, we measure the impact of hybrid learning on work retention (Section 2.3.3). The retention rate is assessed quantitatively by visually observing the trainees as they work with the actual equipment after learning. Finally, an index of the motivation to learn each teaching material is evaluated (Section 2.3.4). This is assessed quantitatively through a questionnaire using the ARCS model.

2.2. Research Methods

2.2.1. Outline of the Periodic Inspection Work and the Work Selected for VR

In the regular inspection of a nuclear power plant, the operation is stopped periodically for a period specified by law, and necessary inspections are carried out depending on the equipment. Depending on the inspection point, the equipment must be disassembled, and parts replaced and reassembled. In this study, we focused on the inspection of the hydraulic control unit (HCU) that drives the control rods in an emergency. The HCU exists in proportion to the number of control rods in a reactor and is a very important equipment that must be inspected repeatedly. HCU periodic inspection work was selected because it requires work in a confined space and is suitable for reproducible learning in VR.

2.2.2. VR System Development Environment

Typical VR development environments include Unity [20] and Unreal Engine [21]. Both development environments have great features for developing VR, and there is no big difference between them. In this research, Unity was adopted because C# was used as the development language.

2.2.3. Development of an Experiential VR System for Work Education

To learn work effectively, it is necessary to have a system that allows users to “experience” the contents not only by viewing them in VR but also by moving their own bodies interactively. Figure 4 shows the learning pyramid, a pyramid-shaped diagram published by the U.S. National Institute for Training and Research, which categorizes the learning methods that are most likely to be retained in the mind (retention rate) [22]. The retention rate of a typical “lecture” is as low as 5%, indicating that passive learning, in which students receive lectures passively, has a low retention rate. In comparison, active learning methods, such as “group discussion” and “hands-on experience”, have a high learning effect, according to the research results. We believe that it is very important to “experience” learning in VR not only by passively viewing content but also by moving one’s own body to improve comprehension and retention rates. Research is also being conducted into how interaction within VR can create a more realistic environment, thereby improving the effectiveness of training and deepening the understanding of the work [23,24,25]. It is known that a higher fidelity of interactivity affects the perception of real situations and improves training effectiveness in VR. Therefore, we examined a device for experiencing work in VR. The experiential VR system we studied is shown in Figure 5.

First, hand-tracking technology was employed to reproduce hand movements, which require the most physical movements during the work. Hand tracking is a technology that reproduces the hand movements of VR subjects in real-time in the virtual space of VR. Until recently, hand controllers were the primary means of interaction in VR, but recent advances allow for natural interaction, and the availability of hand tracking (the use of virtual hands) is crucial for this purpose [26,27,28]. Construction sites involve a great deal of manual work, such as using tools, and carrying, installing, and disassembling items, so faithfully reproducing hand movements is an important element of the interactive training.

There are two methods to realize hand tracking: one is to use an infrared camera attached to the HMD to optically capture and reproduce hand movements in front of the HMD, and the other is to put a glove on the subject and use sensors (acceleration sensor, gyro sensor, piezoelectric sensor, etc., depending on the manufacturer) inside the glove to precisely reproduce hand movements. The optical method has the advantage of being non-contact and inexpensive to implement without a device on the subject’s hand, but it has the disadvantage of not being able to track areas that are not visible in the HMD’s field of view. It is not possible to detect movements that the subject is performing in the unseen range or movements in areas hidden by objects. On the other hand, the glove method has the advantage of complete motion tracking no matter where the hand is positioned but has disadvantages in terms of the complexity and cost of attaching the device to the subject’s hand.

In this study, the glove method was used because the subject’s hands are often hidden behind objects as they work on complex and special structures in periodic power plant inspections. By applying hand tracking, movements such as holding tools and carrying equipment and parts can be reproduced.

Next, “finger pointing and call” and visual checks are two of the most common tasks performed in daily inspections. During routine inspections of critical equipment at nuclear power plants, many checks are made to ensure that there are no foreign objects inside the equipment, that parts are not dirty or scratched, and that replacement parts are installed correctly. If the equipment is assembled with foreign matter or scratches and the subsequent operation is interfered with, it could lead to serious damage. Therefore, it is extremely important to perform visual checks many times before writing them down in checklists and inspection records for each work process. Many studies have been reported on gaze measurement at construction sites, particularly on-site safety management [29,30,31,32,33,34,35,36,37].

Therefore, to realize visual confirmation behavior, eye-tracking technology that reproduces the subject’s eye movement was adopted [38,39,40,41]. Research has also been reported that recreates dangerous areas on a construction site using VR and uses eye tracking to evaluate safety management [42,43]. To perform eye tracking in a VR space, a sensor that measures gaze is installed in the HMD, and the movement of the gazing point is generally detected in a non-contact manner. Two typical methods are the scleral reflection method and the corneal reflection method. The scleral reflection method is a technique that uses the difference in reflectance between the cornea and sclera, irradiating a faint infrared light to the boundary between the cornea and sclera, and capturing the reflected light with a sensor. This method is highly accurate for horizontal measurements but has the disadvantage of not being suitable for vertical measurements. The corneal reflection method generates a reflection point on the cornea, captures its image with a camera, and identifies the reflection point on the cornea and the pupil from the image of the eye captured by the camera. Based on the reflection point and other geometric features, the orientation of the eyeball can be calculated, and the 3D coordinates of the position (focal point) of the eyeball in space can be calculated. Eye tracking can be used not only for visual confirmation tasks but also for the analysis of what the inspection worker is looking at while performing the task.

An example of a subject experiencing an experiential VR system is shown in Figure 6. Eye tracking was used to detect eye movements, and gazing points in the VR space were represented by red spheres to visualize eye movements. In the figure, the user is gazing at the bolt, and the red sphere is displayed. The gaze movement was reproduced by judging the completion of the gaze movement when the red sphere was detected to stay on an object for a certain period of time.

For hand tracking, devices were attached to both hands, and objects simulating both hands were reproduced in the VR space. In the figure, the right hand is holding a tool. In the actual work, movements such as grasping, lifting, carrying, and placing objects of various sizes are required. Therefore, for small parts, the “picking” motion was reproduced with the thumb and index finger. For large objects, virtual contact sensors are placed in the centers of both palms and detect contact between the palms and the object to recreate the action of “holding” the object with both hands. For objects that can be held with one hand, virtual contact sensors were installed at the tips of five fingers, and “grasping” was reproduced by detecting the contact of the five fingers with the object. Thus, by dynamically changing the contact-detection algorithm depending on the size of the object, we have achieved object manipulation that is closer to reality. With conventional controller operation, it is possible to control the grasping and releasing of objects by pressing buttons, but this only reproduces uniform movements, which is a large discrepancy from the movements during real work and does not provide sufficient experience to learn the work. We believe that this algorithm can improve comprehension by reproducing a variety of movements by changing the control method interactively depending on the target object.

To immediately reproduce the sensation of using a tool in real space, the gripping point must be limited to an appropriate position and the direction and angle of the tool must be corrected. Without compensation, the subject may grip at a place where they should not grip, or the point where the tool is applied to the object (the socket in the case of a torque wrench) may not be facing the correct direction, causing stress to the subject.

Therefore, we developed a plug-in tool in Unity to assist the user in defining the tool holding points, angles, and orientations in advance (Figure 7). First, a CG model of the tool is prepared, and a virtual hand object is placed and stored so that the tool is held in the appropriate position, direction, and angle. This predefinition is performed for both the right and left hands. In this way, if a tool is not held properly in the actual VR space, it can be automatically corrected to the predefined state. The same tool can be used for different purposes by providing two different ways of holding it, by defining forward and reverse hands, respectively.

2.2.4. Development of Content Creation Support Tools

As mentioned above, when using VR to reproduce work training for a large-scale plant, it is necessary to reproduce the work procedures for a huge amount of equipment and parts, which requires a great deal of time and effort to create the content. To solve this problem, we developed a tool for creating VR content easily and efficiently without programming as a plug-in for Unity. The screen of the developed content creation support tool is shown in Figure 8. In this system, each action in the VR space is made into commands, which can be combined in any order to freely create VR scenarios. The content creation support tool allows users to freely create VR scenarios by selecting the command they wish to operate from the command selection field, then selecting the object they wish to control with that command from the object group, and finally entering the various attribute information required for that command. The commands to be selected were developed specifically to reproduce the work behavior that is characteristic of the inspection work. The commands are designed to reproduce the user’s actions, such as visual confirmation by eye tracking, object control by hand tracking, installation and removal of parts, replacement of consumable parts, application of lubricant, and lifting of heavy objects by crane or chain block, as well as to support learning by displaying work procedures and presenting knowledge (images, video, audio, and text). This tool enables the efficient and quick creation of VR content for complex special operations in large-scale plants.

2.2.5. Created Content

In this study, content related to the removal of instrumentation units, which is part of the HCU periodic inspection work, was created. The main task of this work is the removal and curing of instrumentation-related parts located at the bottom of the HCU. Approximately 1200 commands were used to create the scenario with the content creation support tool, and the experience time was approximately 40–60 min. Figure 9 is a typical screen example of the created content, which includes the visual confirmation of the nameplate by eye tracking, the loosening of nuts using a double wrench, the removal of O-rings using a bamboo skewer, and the removal of pipes using hand tracking. Figure 10 shows the subject during the content experience; the instrumentation unit is located at the bottom of the HCU, requiring the subject to bend over or lie down, as in the actual work. As shown in the figure, the subject can simulate the actual posture, which makes the learning experience more realistic.

2.3. Method of the Experiment

2.3.1. Experiment to Measure Comprehension Using Learning Materials

To evaluate the effectiveness of the developed interactive VR system, an experiment was conducted to test its effect on comprehension compared with conventional paper-based and video-based learning. Fifteen subjects (

Table 1. Subject list (P: paper, M: movie, V: VR).

Subject No.	P1	P2	P3	P4	P5	M1	M2	M3	M4	M5	V1	V2	V3	V4	V5
Age	61	46	49	52	47	58	53	27	34	42	40	51	36	23	36
Years of field experience	6	0	2	0	0.25	5	0	0	1.75	0.33	0	0	1	0	0
Years of employment	39	7	25	30	22	38	30	3	9	16	12	26	12	1	12

) were divided into three groups (five in each group). The subjects selected were researchers and designers who are not normally involved in field work, so overall, their years of field experience were short. After they studied for approximately one hour with each material, they were asked to take a common test question to quantitatively evaluate their working comprehension. In this evaluation, the target task was the removal of the instrumentation unit of the HCU based on the VR content created.

The paper documents contain detailed descriptions of the names of all the work processes, workflow, detailed movements of each process, necessary tools, and tools to be used, along with photographs of past work situations, and comprehensive information on the checkpoints at various locations and the knowledge of the skilled workers. This material is already being used at the construction sites of nuclear power plants, and even if a worker has no experience in HCU work, they can perform basic work if a worker has some experience in working at construction sites and after studying this material. At nine pages in A4 size, the material can be read in approximately 20–30 min by ordinary workers.

The movie teaching materials use edited footage of past work scenes and 3D CAD to reproduce the HCU itself, parts (parts to be disassembled, replacement parts, and consumables), tools, and worker movements. In addition, detailed work information used in the paper-based teaching materials is displayed on the video, and a narration is added to emphasize the important points, making the contents of the video both visually and aurally educational. The viewing time of the video was edited to approximately 20 min.

As shown in Figure 9 and Figure 10, the VR teaching materials are interactive learning contents that can be experienced by moving the body in the same position as the actual work using eye tracking and hand tracking. Since the content was created with an emphasis on providing an immersive experience, it does not display all the textual information presented in the paper and video materials but allows the user to learn the information necessary for the evaluation of this experiment through the display of text, images, and narration as the content progresses. The duration of the experience varied depending on the behavior of the participants but ranged from 40–60 min. The learning time for all materials was one hour. The paper and video materials could be studied as many times as needed, but the VR materials could only be studied once due to the time required for the experience.

The test consisted of 60 questions, 18 of which were lecture-related questions to acquire basic knowledge and 42 of which were reproduced in VR. Of these, 35 questions reproduced movements using the hands, and seven questions reproduced movements using the eyes. Excerpts from the examination questions are shown in Figure 11, and all the test questions are given in Appendix A.

For example, Q1 asks for basic knowledge of nuclear power plants. It is classified as a “class-related question” because the subject only sees the material displayed in all the teaching materials (paper, video, and VR) and does not reproduce it in relation to their actions in the VR space. Q10 is a question on loosening a bolt on a flange; since the subject holds the wrench by hand, attaches the bolt to the nut, and turns the nut side in the VR space, it was classified as “a question reproduced in VR that reproduces hand actions”. Q6 is a question in which the subject matches the individual number of the HCU to be inspected prior to the work with the nameplate. Since the subject aligns the gazing point visualized using eye tracking with the nameplate, it was categorized as “among the questions reproduced by VR, the question reproduced eye movement”.

Since the subjects’ age, experience in the field, and years of employment varied, their latent basic knowledge differed, so the scores were corrected by weighting the individuals’ abilities (proficiency) in reference to the methods used in previous studies. According to the literature [44,45], the initial achievement level of unskilled workers is assumed to be 60%, and skilled workers are assumed to have 1.7 times the work efficiency of unskilled workers. The subject’s skill evaluation (understanding of the operation of tools and machines used in the work and the workflow) was evaluated at six levels, 5% each, in the range of 100%–130%. The subject’s independence evaluation (breadth of vision and management ability to see the entire work, including intellectual ability, memory ability, and communication ability) was evaluated at six levels, 5% each, in the range of 100%–130%. Multiplying the skill evaluation by the independence evaluation resulted in a work efficiency rating of 169% for the most skilled workers, which is almost equal to the 1.7-fold rating. This method was also applied in the subjects in the experiment of this study, and the evaluation values were objectively calculated and weighted by considering the age, years of employment, years of experience in the field, and other factors for each subject.

For example, if the subject is “able to work independently”, the skill evaluation level is 115% (assuming that the skill evaluation level of the unskilled subject is 100%). In addition, if there is “communication ability” and “on-site management ability” in the independence evaluation, it will be 110%. Multiplying these values by 126.5%, this subject is given approximately 1.27 times more weight than an unskilled subject. Finally, each subject’s test score is subtracted and corrected so that it is equivalent to the unskilled subject, which has the lowest skill and independence ratings.

The 60 test questions can be roughly divided into two categories: (1) questions that can be answered by a person with a high ability who has a lot of basic knowledge (30 questions) and can predict the correct answer without learning the material, and (2) specialized questions specific to the HCU (30 questions) that cannot be answered without learning the material. This correction was made only for the first 30 questions. The final comprehension level was calculated by correcting the test scores so that all the participants were under the same conditions based on the evaluation value of the final test score.

2.3.2. Experiment to Improve Comprehension by Hybrid Learning

An experiment was conducted to test the effect of combining multiple types of learning materials on comprehension. Fifteen subjects (new employees, no field experience) were used in the experiment, and the three types of learning materials used in the previous experiment were used as is. The flow of the experiment is shown in Figure 12.

First, all the subjects studied on paper for one hour and took the first test. Next, all the participants studied the video material for one hour, of which 11 took the second test. The remaining four participants took the second test after studying the VR material for one hour; the questions for the first and second tests were the same, but the order of the questions and the order of the choices were randomly changed. Because this experiment was conducted with new employees who had the same knowledge and experience level, the score correction used in the experiment was not performed.

2.3.3. Learning Retention Rate Measurement Experiment

An experiment was conducted to evaluate the “retention rate” after learning each material. As described in the section on the learning pyramid, it can be predicted that there will be a significant difference in the retention rate between passive and active learning. Therefore, the participants were divided into two groups: passive learning with paper and video, and active learning with paper, video, and VR. After learning, each group was asked to perform actual tasks in the training facility and their work performance was measured.

Four subjects (new employees, no field experience), two each in passive learning and active learning, used the same learning materials as in the previous experiment. As an evaluation method, a “work importance level” was assigned to each work step and weighted, and a “work execution level (whether or not the work was performed correctly)” was measured.

Table 2. Definition of work importance and work implementation.

	Levels	Points	Contents
Work execution level	High	1.0	No mistakes/Work complete
	Middle	0.5	There are any mistakes/Work completed
	Low	0	Work incomplete
Work importance level (weighting)	High	2.0	Cause of an accident
	Middle	1.5	Affects quality
	Low	1.0	Missing work step

shows the definitions of work importance and work execution, and Figure 13 shows the status of the work execution measurement.

The importance of work was defined by skilled workers based on existing work procedures, with “1 point” for omissions that did not affect quality, “1.5 points” for omissions that affected quality, and “2 points” for omissions that were a cause of serious accidents. The work execution level was defined as “0 points” for incomplete work, “0.5 points” for work that was completed despite mistakes, and “1 point” for work that was completed without mistakes. A checklist for measuring the work execution was prepared, and the checker filled in the work execution level while observing the subject at work. Finally, the work retention rate was calculated by multiplying the work importance level and the work execution level.

2.3.4. Motivation to Learn Questionnaire

For each learning method, we conducted a survey using an 18-question 5-point scale questionnaire based on the ARCS model, a framework proposed by John et al. [46] in 1983 that categorizes and organizes four factors to improve and maintain the motivation to learn.

The correspondence between the content of the questionnaire used and each ARCS factor is shown in

Table 3. Correspondence between ARCS factors and questionnaire content.

No.	Questionnaire Contents	Factors
1	Did you enjoy learning?	Attention
2	Does this study stimulate your curiosity?	Attention
3	Is the learning content approachable?	Relevance
4	Are the meaning and purpose of learning clear?	Confidence
5	Do you think it will be useful for your work?	Satisfaction
6	Are you clear about the learning goals you want to reach?	Confidence
7	Do you feel like you were able to do something or understand something while studying?	Satisfaction
8	Would you like to find out more about this work?	Attention
9	Can you do the work in the correct way?	Relevance
10	Would you like to learn the same method for other tasks?	Relevance
11	Do you have a positive attitude towards participating in learning?	Attention
12	Would you like to learn more about HCU’s related work?	Confidence
13	Can I proceed at a pace that suits me while studying?	Relevance
14	Do you think this learning method is fun?	Attention
15	Is this learning method easy to understand?	Relevance
16	Do you think it is important to study before working?	Confidence
17	Do you think you have acquired the knowledge and skills you have learned at this point?	Satisfaction
18	Do you want to further improve your knowledge and skills in HCU work?	Confidence

. It consisted of five questions on “Attention”, five on “Relevance”, five on “Confidence”, and three on “Satisfaction”.

The first factor is “Attention”, which is to arouse the learner’s interest and inquisitiveness, making them think, “This looks interesting”. The second element is “relevance”, which makes the learner feel close to the learning goal and makes the process of reaching the goal seem “rewarding” by making the learner enjoy the process of reaching the goal, rather than just performing it passively. The third factor is “Confidence”, which is to make learner believe that they can complete it by making the goal explicit, giving them opportunities to succeed, and making the material seem successful through their efforts. The fourth factor is “Satisfaction”, which is about giving fair recognition to learners who reach their goals without wasting time on learning outcomes and making them feel “glad they did it”. Questions 1, 2, 8, 11, and 14 were classified as “Attention” factors because they concern the enjoyment of learning, curiosity, interest, and a spirit of inquiry. Questions 3, 9, 10, 13, and 15 were classified as “Relevance” factors because they are related to the goals and objectives for completing the work and are questions aimed at increasing the willingness and motivation to learn. Questions 4, 6, 12, 16, and 18 were classified as “Confidence” factors because they relate to achieving clear learning goals related to HCU work through the learner’s own effort and ingenuity. Questions 5, 7, and 17 were classified as “Satisfaction” factors because they were aimed at feeling the benefits and effects of learning and feeling that they can put it into practice in the workplace.

3. Results

3.1. Result of Experiment to Measure Comprehension Using Learning Materials

The corrected scores of the comprehension test are shown in Figure 14. The horizontal axis shows the groups of each learning material, and the vertical axis shows the average percentage of correct answers for all the questions, lecture-related questions, questions that reproduced hand movements in VR, and questions that reproduced gaze movement in VR, respectively, as bar graphs. The percentage of correct answers for all the questions was 52.7% for paper, 61.6% for video, and 68% for VR. Although VR was used only once during the study time, the percentage of correct answers was higher than for repeatable paper or video, suggesting that it is an effective comprehension material.

However, when focusing on “Lecture-related questions”, the highest percentage of correct answers was obtained on paper. Since video and VR preferentially incorporate visual and auditory information and spatial cognition, they are not suitable for learning to acquire basic knowledge by reading detailed text, while paper-based learning allows for specific knowledge to be established through repetition and relearning, which is thought to be the reason for the high rate of correct answers related to classroom learning.

On the other hand, the percentage of correct answers for “questions that reproduced work with hand and eye movement in VR” was higher for both video and VR. The video, once replayed, allows the user to view work procedures in sequence, based on the visual information and auditory information provided by the narration, allowing the user to learn work steps that reproduce actual work. VR allows the user to immerse themselves in the work site more than video, and interactively experience the work while perceiving the space, which enables them to further master the work operation. In particular, the improvement in the percentage of correct answers for “questions that reproduce work with hand movement” is thought to be largely due to the influence of active learning.

Considering the results of the number of times of study, video, and paper could be studied repeatedly (paper: average 2.4 times, video: average 1.3 times, VR: 1 time), paper materials are the most effective in terms of repeated study. Repeated study allows specific knowledge to be solidified, which is thought to be the reason for the high rate of correct answers to lecture-related questions. However, despite the highest number of study sessions, the correct answer rate for all the questions was low, and it is thought that the learning efficiency is lower than other materials. On the other hand, VR has the disadvantage that it is difficult to study repeatedly or go back and study due to the nature of the content. However, even though VR was studied only once, it was possible to achieve a level of understanding that exceeded the correct answer rate of paper or video, which can be studied repeatedly. It is thought that the VR materials themselves were effective as materials for improving comprehension.

3.2. Result of Experiment to Improve Comprehension Using Hybrid Learning

The results of the measurement of comprehension gains from hybrid learning are shown in Figure 15. The horizontal axis shows the groups of learning materials, and the bar graph on the axis shows the average percentage of correct answers for all the questions, lecture-related questions, questions that reproduced hand movement in VR, and questions that reproduced gaze movement in VR, respectively. The percentage of correct answers for all the questions was 43.8% for the paper-only learning group, 53% for the paper-and-video learning group, and 69.6% for the paper-and-video and VR learning group, showing a significant improvement in the percentage of correct answers in the VR learning group.

Due to the lack of basic knowledge of the subjects’ new employees in this experiment, the percentage of correct answers was significantly low, but the same or a higher percentage of correct answers was obtained by studying the three materials. The breakdown shows that the percentage of correct answers increased as the number of learning materials increased for lecture-related questions, questions that reproduced hand movement, and questions that reproduced eye movement, respectively, with a particularly large increase in the percentage of correct answers for questions that involved work experience in the paper, video, and VR learning group. Compared with the 7~10% improvement with paper and video, the additional use of VR resulted in a 28~30% improvement. In addition to the acquisition of basic knowledge using paper and understanding of the workflow by video, the spatial cognition and interactive work experience using VR are thought to have resulted in a significant improvement in comprehension by integrating the merits of each material.

3.3. Result of Learning Retention Rate Measurement Experiment

The results of the work retention rate evaluation are shown in Figure 16. The horizontal axis is the four subjects, and the vertical axis is the work retention rate. Subjects A and B are in the passive learning group (without VR) and subjects C and D are in the active learning group (with VR). The evaluation results showed that the task retention rate improved by approximately 20% in the VR learning group, confirming that VR learning is effective.

Table 4. Results of the evaluation of the degree of impact on work retention rate.

Work Step	Work Procedures	High Impact
Pre-work	Confirm number of people	Yes
Check HCU unit	Point to check the HCU location number of the main terminal box	Yes
Curing the pressure gauge	Fold the Kim towel into multiple layers
	Cover the surface of the pressure gauge with a Kim towel
	Fix the Kim towel to the pressure gauge with flame-retardant tape
Curing the level switch	Turn around the threaded part and then remove it straight
	Cover the terminal side with flame-retardant tape
	Protect the connector side with a plastic bag (use a sponge)
	Tighten the opening of the plastic bag with flame-retardant tape
	Fix the cable to the pillar of the frame (using wire)	Yes
Curing the pressure switch	Turn around the threaded part and then remove it straight
	Protect the terminal side with flame-retardant tape
	Protect the connector side with a plastic bag (use a sponge)
	Tighten the opening of the plastic bag with flame-retardant tape
	Fix the cable to the pillar of the frame (using wire)	Yes
Label the instrumentation unit	Create a label for PS/PI/main unit using flame-retardant tape
	Paste the label on the PS piping (pasting position)
	Paste the label on the PI piping (pasting position)
	Attach the main body label to the instrumentation block	Yes
Remove PI piping	Remove with priority over PS	Yes
	Set H26 on the body and H36 on the nut
	Turn H36 without moving H26
	After removing it, cure both ends of the pipe
	Protect the terminal between the pressure gauge and the instrumentation block
	Store on designated pallet	Yes
Remove PS piping	Set H26 on the body and H36 on the nut
	Turn H36 without moving H26
	After removing it, cure both ends of the pipe
	Protect the terminal between the pressure switch and the instrumentation block
	Store on designated pallet	Yes
Installing a jack	Make sure the jack’s lifting surface is in the lowest position.	Yes
	Check in advance that the jack’s hydraulic valve is closed
	Place the lifting surface of the jack directly below the instrumentation block
	Carefully raise the lifting surface until it lightly touches the instrumentation block
	Remove the operating handle from the jack
Removing the instrumentation block	Loosen the two diagonal bolts on the ACC side coupling part	Yes
	Remove the two diagonal bolts
	Loosen and remove the four bolts on the double-sided flange on the N2 container
	Remove the last bolt while holding the flange of the ACC side coupling part
	Carefully place the flange on the instrumentation block
	Loosen the hydraulic valve and lower the lifting surface to the lowest position
	Pull the jack toward you while holding the instrumentation block
Curing the instrumentation block	Protect the opening on the ACC side (use it longer to the side)
	Remove the O-ring on the ACC piping side with a bamboo skewer
	Store the O-ring in the specified plastic bag
	Protect the opening on the ACC piping side (use it long enough to reach the side)
	Remove the O-ring on the N2 container side with a bamboo skewer
	Store the O-ring in the specified plastic bag	Yes
	Cover the opening on the N2 container side with flame-retardant tape
	Store the instrumentation block on the designated pallet	Yes
Tidying up	4S implementation

shows the items that had a greater impact on the retention rate in those who experienced VR compared with those who did not experience VR for each check item (items with a difference of 0.5 or more in retention rate were extracted). The work was divided into 12 major work steps, and each item contained detailed work procedures. A characteristic result was that there was a difference in the retention rate between the beginning and the end of each work step. This is not related to the main work step of each item but to the ancillary work steps, such as the pre-work setup, the visual check, and the storage of parts. The main work was steadily performed by each group, and there was no significant difference in the retention rate, but the detailed work, including incidental work, was affected, suggesting the effectiveness of active learning.

3.4. Result of Motivation to Learn Questionnaire

The results of the ARCS questionnaire are shown in Figure 17. For most questions, the ratings improved for paper, video, and VR, in that order. The scores for VR were higher than the scores for the other two materials. This indicates that VR materials are very effective in motivating students to learn; the VR group scored particularly high on the following items: curiosity was stimulated, active learning was possible, learning was fun, and it was important to learn with the task in front of them. The only question where VR scored lower was on the question of whether the pace of the program was appropriate for them. This may be due to the characteristics of the content, which, as mentioned above, is not suitable for learning repeatedly, going back to one’s own pace once one starts learning.

The results of averaging the evaluation scores for each ARCS factor are shown in

Table 5. The results of calculating and averaging the evaluation points of the ARCS questionnaire for each factor.

ARCS Factors	Paper	Movie	VR
Attention	2.6	3.4	4.6
Relevance	2.5	3.0	3.9
Confidence	3.2	3.7	4.3
Satisfaction	2.9	3.1	4.1
Average	2.8	3.3	4.2

. Averaged over all the questions, the paper group scored 2.8 points, the video group 3.3 points, and the VR group 4.3 points, with the VR group obtaining the best results. Looking at the breakdown by factor, the factor “Attention” received a particularly high score. It is thought that curiosity about a new learning method not found in conventional learning methods and the fact that the materials are fun to learn with a game-like sensation improve the motivation to learn.

In the free descriptions, there were comments such as “It was good to experience the actual working posture”, “It helped me prepare myself before entering the field”, and “I did not feel VR sickness because I moved my body”. These results indicated that VR is an effective learning method that not only contributes to improving comprehension of the material itself, which was obtained in the experiments, but also enhances the motivation to learn and improves the opportunities and frequency of education.

3.5. Examination of the Experimental Results

From the results of the above experiments, we found that the VR learning method is effective in terms of comprehension, retention rate, and motivation to learn. On the other hand, the paper-based and video-based learning methods also have different merits, and we believe that the combination of all the learning methods can effectively improve the level of comprehension.

Table 6. Evaluation results that organize the characteristics of each learning material.

Teaching Materials	Paper	Movie	VR	Group Lecture	Training Facilities	On-Site OJT
Basic knowledge acquisition	5	4	3	5	3	2
Understand work procedures	2	4	4	2	5	5
Spatial cognition	1	1	4	1	4	5
Work experience	1	1	4	1	5	5
Cost	5	3	2	4	1	1
Understanding/Retention level	1	3	4	2	5	5
Repeated learning	5	4	3	2	2	1
Learning opportunities/Frequency	5	5	4	2	1	1
Willingness to learn	1	3	5	1	4	3
Average	2.9	3.1	3.7	2.2	3.3	3.1

shows the characteristics of each learning material based on the experimental results. In addition to the paper, video, and VR self-learning methods tested in the experiment, three learning methods—group lectures, training facilities, and OJT—were added as learning with an instructor. A five-point scale was used to evaluate each feature, and the average score was calculated.

As shown in the experimental results, the paper-based learning materials were advantageous in terms of the acquisition of basic knowledge, repetition of learning, and learning opportunities and frequency, but had disadvantages in terms of comprehension, retention rate, and motivation to learn. The video materials were effective in terms of work procedure comprehension and learning opportunities and frequency, and covered the demerits of paper materials, but had some demerits in terms of the acquisition of basic knowledge and the repetition of learning. For the VR materials, while covering the respective disadvantages of paper and video, the scores were generally high for each feature, but there were disadvantages in the acquisition of basic knowledge. There were also disadvantages in terms of the cost of equipment and content production. However, the total average score was high at 3.7.

On the other hand, the results of the estimation for learning with instructors indicated that although group lectures are as suitable for acquiring basic knowledge as paper-based learning, they have disadvantages in terms of cost and repetitive learning compared with paper-based learning; therefore, the evaluation score for group lectures was estimated to be the lowest. Both training facilities and on-the-job training are considered most effective in terms of grasping work procedures, spatial cognition, and work experience because they involve actual work, but they have disadvantages in terms of cost, repetition, learning opportunities, and frequency. In the case of nuclear power plants, there are significant challenges to adopting on-the-job training as a learning method because there are few special training facilities themselves and not many training opportunities, and there are few opportunities from the perspective of security and radiation exposure.

4. Discussion

Based on the above evaluation results, we believe that the combination of these teaching materials can enhance the learning effect. Regarding hybrid learning, the importance of both offline and online learning has been reported [47]. In particular, since COVID-19, many studies have been reported, mainly in the educational field, and the importance of online learning has increased [48,49,50]. However, there are still many challenges with completely online learning, and it is said that it is effective to combine it with face-to-face learning.

Research has also been reported on incorporating VR learning into online learning [51,52,53]. It has been suggested that students can promote effective learning outcomes by being able to study and review at any time as immersive learning materials. However, the research results showed that VR learning has both positive effects and no effect on promoting learning. The results of this study were similar to those of previous studies. It was confirmed that while VR is effective overall, it also has weak areas, and it is very important to supplement the weaknesses of VR learning with traditional paper and video learning. Therefore, we propose the following process for knowledge transfer through hybrid learning.

Specifically, it is considered possible to comprehensively deepen the level of understanding by first acquiring basic knowledge through repetitive learning using paper-based learning materials, then grasping the workflow through learning using video content, and finally experiencing the interaction in a space that simulates an actual work site using VR by moving one’s own body. These studies are repeated as self-study at each site (work site, contractor’s office, etc.) to accumulate basic knowledge and learn how to move in the field. With regard to these self-learning methods, by building a system that allows workers to learn each material online, it will be possible to learn remotely without being in the field. VR teaching materials have the disadvantage of repetitive learning, but we believe that this disadvantage can be overcome if many workers can learn in the VR space at the same time as online. In recent years, technologies such as the Web Graphics Library (WebGL) that can easily handle 3D data online have been developed, and it has become easier to convert VR teaching materials to online.

When sufficient self-study has been completed, the final workflow is studied at the training facility. Since the trainees have acquired sufficient knowledge and movements through self-study using paper, video, and VR, they can move their bodies smoothly in the training facility, enabling effective training in a short period of time. By learning effectively in this way, the trainees can reach a level where they are ready to work immediately when they are put on the job site, and the on-the-job training time can be reduced as much as possible so that the on-site work can be performed efficiently. In this study, we conducted a quantitative evaluation of three types of self-learning materials, but in the future, we would like to quantitatively evaluate the educational effects of training facilities and on-the-job training and study the optimal combination of education.

In addition, we are currently considering incorporating VR training into the learning curriculum of the on-site skills certification system. Before starting work at a nuclear power plant, workers are required to take a training course called skills certification, and if they do not pass the course, they are not allowed to enter the work site. In the skills certification, skilled workers serve as instructors and give group lectures to those who enter the work site, and a confirmation test is conducted at the end of the training to grant certification. The grade changes according to the score of the test, and the worker can have a role as a general worker, team leader, or chief supervisor. Under the current system, skilled supervisors and managers provide training based on past materials, but it is difficult to convey all the knowledge through lectures alone, so additional training through on-the-job training is required.

Therefore, we believe that hybrid training using VR together with educational videos is used for skill certification and that by having students acquire basic knowledge through lectures, understand the workflow through videos, and learn on-site movements through VR experiences, it is possible to comprehensively convey the knowledge of skilled workers in each material, thereby developing human resources capable of performing tasks immediately without the need for OJT. It is also expected to improve the overall quality of the workforce by revamping the grading roles to a higher level.

5. Conclusions

Research on VR as a means of transferring the knowledge of skilled workers in nuclear power plant inspections led to the following conclusions:

• As one of the active learning methods, we developed a VR educational system that enables students to experience interactive work while moving their bodies and compared it with conventional paper and video teaching materials. As a result, VR achieved higher comprehension than the other teaching materials in the comprehension test.
• The results of a comprehension test conducted by learning paper, video, and VR materials in sequence showed that learning all the materials, including VR, in sequence was more effective in improving comprehension compared with learning only paper or paper and video.
• The results of measuring work retention rates at training facilities with and without VR showed that VR learning improved retention rates and affected retention rates in the ancillary tasks more than in the main tasks.
• The results of the evaluation of the characteristics of each material from the experimental results showed that VR is a learning material with no major disadvantages in the inspection work of nuclear power plants and is a learning material with a high overall evaluation value.
• We believe that more effective education can be achieved by appropriately combining various learning methods, such as training facilities and on-the-job training as instructor-led learning, while taking advantage of the advantages of both VR and other learning methods.

The above results confirm that VR education is effective in the inspection work of nuclear power plants, but there are still some issues to be resolved before it can be widely used onsite.

Specifically, considering the setting up and operation of VR equipment on site, a dedicated learning space and personnel with specialized knowledge are required, and an environment that allows on-site workers to actively learn on their own, as is the case with paper and video, is not yet in place. This is one of the reasons why the use of VR as a general learning method has been slow, even though VR itself is an effective learning material. Although the experimental results of this study showed that the willingness to learn is high, the effectiveness is limited unless the facility for learning is in place. In the future, it is necessary to construct facilities and systems that enable on-site workers to easily experience VR anytime and anywhere.

In terms of technology, this study employed two tracking techniques—hand and eye tracking—but the information is insufficient to reproduce the subject’s movements. To experience the work more realistically, it is also important to reproduce the movements of the entire body, and it is necessary to apply a technique such as body tracking. Body tracking is a technique to reproduce the natural movements of the entire body by attaching a tracker to each joint of the subject to detect positional information. Inverse kinematics is a typical method for controlling the skeletal structure of a 3DCG model, such as a character, to make it move. Ultimately, the use of each tracking technology will not only make the work experience more realistic but will also improve the effectiveness of VR education by recording the ideal movements of skilled workers and reproducing them in the VR space as teacher data.

There are also many policy issues involved in applying a VR education system to nuclear power plants. By simulating 3D information using VR, the structure of a nuclear power plant can be visualized, so if the internal structure (passageways, location of nuclear equipment, layout information, etc.) were to be leaked to the outside, it would increase security and terrorism risks. Therefore, due to concerns about information security, the application of digital technology in the nuclear power business has been delayed. To solve these issues, it is necessary to implement strong security measures for communications and cloud environments before applying a VR education system. These are issues that should be addressed through cooperation between many stakeholders, including national agencies and local governments, rather than just one power-generation company.

In the future, we would like to incorporate VR education into the learning curriculum of the on-site skill certification system, with the aim of constructing an effective education system. Specifically, we would like to continue researching educational systems that use the metaverse space. The metaverse is a 3D space that is an extension of the closed VR space, and its unique feature is that many people can connect to it at the same time and have a variety of experiences. It is important to use the metaverse in all tasks in the plant life cycle, such as planning, engineering, procurement, construction, and maintenance, and to create an environment where knowledge can be transferred, and VR education can be experienced as one of the contents within that. Since various information, such as design information and on-site information, can be stored in the metaverse space, it is also possible to create educational content of a higher quality. In the future, we would like to build a system that allows the educational system to grow autonomously by allowing anyone to freely create and register VR educational content in the metaverse.