The Design of Applying Gamification in an Immersive Virtual Reality Virtual Laboratory for Powder-Bed Binder Jetting 3DP Training

Abstract

The integration of Virtual Reality (VR) and gamification techniques can be used to produce a fun virtual laboratory, including virtual spaces and educational content. This study developed a prototype for a virtual laboratory for powder-bed binder jetting three-dimensional printing (3DP) training in universities. The 3DP virtual laboratory is expected to address problems encountered in teaching, training, and practicing with powder-bed binder jetting 3DP. The 3DP Training Virtual Laboratory was developed by using immersive VR technology to simulate two-handed operations. The user evaluation of the first version prototype revealed that the students lacked learning interest and motivation when using the prototype. Thus, gamification technology was integrated into the 3DP virtual laboratory prototype in the mid-implementation development phase. After refining and systematically developing the model to meet the modified requirements, user evaluations on the game elements showed positive feedback. This study concluded that elements of gamification design should be considered at the beginning of the educational or training system development in order to enhance students’ motivation or engagement.

1. Introduction

In order to eliminate problems identified in practicing, training, and operating powder-bed binder jetting three-dimensional (3D) printers, we have developed a prototype for a powder-bed binder jetting 3D printing (3DP) virtual laboratory training platform. Powder-bed binder jetting 3D printing technology has been applied in the development of a wide variety of products in many industries, such as puppet expressions printed for use in stop motion animation, realistic characters printed for important holiday commemorations, complex architecture models for construction communication, and so on. Compared to Fused Deposition Modeling (FDM) printing, 3DP printing has two advantages. It does not require support structures, and target objects can be printed in full color, unlike other 3D printer technologies. Figure 1 illustrates some 3D printing examples.

Several problems were encountered, however, by students in the practical side of their training to operate full-color powder-bed binder jetting 3D printers. The problems are described below.

• Powder-bed binder jetting 3D printing machines have complicated operating procedures, and if they are not followed, the machine and printing parts can be easily damaged. Before starting the machine, the first task is to check the related materials and containers, as drawn in Figure 2a–f. We separate the first task into six parts, including whether the waste powder tank is full and should be recycled or not (Figure 2a), whether the waste ink bottle is full and should be poured or not (Figure 2b), whether the powder feed box has enough powder for the next print or not (Figure 2c), whether the three kinds of colored binder levels are adequate or not (Figure 2e), whether the clear binder level is adequate or not (Figure 2d), and whether the cleaner level is adequate or not (Figure 2f). Each check requires corresponding responses for out-of-tolerance situations.
• After the above examinations have been conducted, multiple parts must be manipulated in order to check and correct them. This includes roller cleaning, wiper cleaning, print head cleaning, white nozzle/color nozzle testing, and head alignment. We summarize that a complete 3DP printing process is separated into six tasks with 24 components. If any of these operations is ignored, or a check not rigorously conducted, then the printing sheet may not only fail to be formed, but it also may damage the machine. Without any 3DP machine feedback mechanism, their operations can easily produce errors if operators lack sufficient experience and skill.
• Dust pollution problems: dust can cause respiratory discomfort in operators, so they must wear industrial masks with effective sealing and filtering. The material used in powder-bed binder jetting 3D printers is a gypsum powder with a large mesh size (the higher the mesh value, the lower the particle size). Many operations, such as adding powder to the powder supply tank and removing the recovered powder tank, the use of the tool sieve powder, and the removal of the prints, inevitably release dust into the air. Ventilation and dust collection equipment do not sufficiently address this problem.
• Crowding problem: generally, space is at a premium, and training facilities are often crowded and too small for the number of trainees, limiting the size of classes. The crowding problem combined with the dust problem affects not only users, but also other machines in the same room. In the space referred to in this study, the powder-bed binder jetting 3D printer was located in a space only large enough for three people.
• The high cost of powder-bed binder jetting 3D printer consumables is prohibitively high for immature practice: using the printer used in this study as an example, at least five kinds of materials, such as the print head for the cyan, magenta, and the yellow binder, the print head for the clear binder, the liquid cleaner, the composite powder, the cyan binder, the yellow binder, the magenta binder, the clear binder and the quick hardening infiltrate, are all required consumables. Each consumable is necessary for every printing part. In the authors’ experience, the improper use or management of consumables can be an additional cause of machine failure or defective work.

The above problems present a barrier to students trying to gain the required experience and skills to safely operate 3DP machines, as mistakes can prove to be extremely costly. How are we able to construct a friendly training environment where students can understand and manipulate the entire process of the powder-bed binder jetting 3D printer without operating the real machine? Henceforth, constructing a virtual laboratory could be a feasible solution [1]. A virtual laboratory can be established in a multiple-type platform company with different type interactions, such as virtual reality [2], web-based [3,4,5], and so on.

For the consideration of many two-handed operations in real manipulations, we decide to apply immersive virtual reality (VR) technology of having two handheld controllers, in order to establish a powder-bed binder jetting 3D printer virtual laboratory. Many research studies [2,6,7,8,9,10,11,12,13,14,15] denoted that VR not only eliminates temporal–spatial restrictions or volume limitations, but also provides two-handed interactions with virtual interactable objects within an immersive virtual environment in order to simulate a high cost, dangerous, micro/macro-level environment, and multiple scenarios beyond space and time. We decided to apply immersive VR technology to simulate the 3DP training environment and the interactions among multiple processes of 3DP machine operations.

Gamification was, initially, feature in the 3DP Training Virtual Reality Virtual Laboratory [16] prototype (in this paper, we call it the 3DP Training VRVL). Some studies [9,10,11,12,13,14,15] pointed out the active influence of VR in training and education, while others proposed that VR can improve students’ degree of participation. Thus, in the virtual laboratory at the very start, we expected that the introduction of VR technology would provide students with sufficient learning motivation. In other words, we had two systematic objectives. The first one was to solve the teaching problem brought about by a realistic environment, and to provide students with a learning environment with a comparatively high degree of participation. The second objective expects VR technology to bring about a good learning motivation for students, because of an immersive user experience. Due to the limitations of time and human resources, we focused on multi-model VR interaction engineering and provided an interactive guiding mechanism design for the 3DP Training Virtual Laboratory, expecting that the VR technology of the virtual reality laboratory would achieve these two objectives. If part of the user experience brought about good results, then we could conduct relevant studies of the teaching model.

The user evaluation in the mid-implementation phase of the prototype showed that, although the interactive simulation and guiding mechanisms obtained positive feedback, we got pessimistic feedback from the students’ motivation to use the prototype. How, then, could we improve the learning motivation to use the prototype? How could we make the prototype more interesting and more persuasive? After advanced deliberation on improving the students’ engagement in the prototype, we therefore decided to integrate gamification into the VR-based prototype of the 3DP Training Virtual Laboratory to address this issue.

Gamification is one kind of motivational science or persuasive technology. Some studies denoted that it could make the learning experience more engaging and enjoyable and could increase motivation. However, how can we determine that our solution is efficient for the prototype before reengineering and restructuring the prototype? Some studies pictured a VR-based development model or gamification development phases that may fit parts of the development progress of our 3DP Training Virtual Laboratory. We therefore constructed an engineering development model to describe the progress of our 3DP Training Virtual Laboratory.

This study, instead of describing any certain educational research method, discusses the consideration of the introduction of gamification during the initial design and development stage when building a VR training or educational system. In this study, the gamification design was only introduced when it was found that students’ degree of participation in the system was low in the middle stage of development.

A comparatively efficient method of system analysis and design information system is to set objectives, correct the system functions and build interfaces by means of iterative incremental development, rapid development and user-participated testing based on the development objectives. The addition or correction of objectives during the development stage may affect the overall system structure or seriously impact its development progress.

This study expects that an education or learning system developed by VR technology can provide students with a high degree of participation in the initial stage. Thus, no gamification design was considered during the initial stage of development. However, because of the feedback of the main users’ experiences during the development, although VR brings about a good 3D interaction experience, it is not sufficient to maintain students’ high degree of participation. Thus, in the middle stage of development, the gamification design was introduced. We present the whole process of system development with a visualized structure and specifically state that, midway through development, we undertook double the manpower and time cost to complete the training system, because of the addition of gamification design. We describe the case development process from the system development process, simple User Experience (UX), and User Interface (UI) evaluation. Although few people have conducted UX midway through the project, the suggestion proposed for the whole process was that, although VR can bring about different immersive user experiences, one must consider that good user experiences cannot guarantee a high degree of participation of system users in continuing to utilize the system. When importing VR technology for teaching or training system development, the integration of gamification design is a comparatively efficient scheme in the system’s structure and function.

We also proposed a correspondence between the training course structure, gamification level and design factors. During the middle stage of VR system development, the development process structure was formed by adding the gamification design or other concepts that affected the development structure. In the future, when the development is finished, tests of user experiences and the teaching model can be conducted, in order to compare and analyze the VR experiment without gamification design and the final design. The remainder of this paper is organized as follows: Section 2 discusses some projects and studies that are related to VR in education, gamification, and the virtual laboratory. Section 3 describes the development model of the 3DP training VRVL. By following the development phases, this study defines related diagrams describing requirements, structures, and associations. Section 4 presents a discussion of this study and other similar studies, while Section 5 offers conclusions and directions for future studies.

2. Related Works

2.1. Immersive Virtual Reality in Education or Training Applications

In terms of VR technologies applied in education and training, most research studies belong to the medical industry [6,7,17], emergency service personnel [8,9], sports [10], industrial training [11], etc. Organizations often embrace this kind of VR training, because not only can it save money, eliminate danger, and improve effort, but it can also help prepare their employees for a job in a safe environment. The authors in [6] examined the impact of virtual reality (VR) surgical simulation on the improvement of psychomotor skills relevant to the performance of laparoscopic cholecystectomy [7]. The objective was to demonstrate that VR training transfers technical skills to the operating room (OR) environment. VR is also used in clinical training and developing the spatial skills of children with mobility impairments [12]. A few applications, including The Narrative-based, Immersive, Constructionist/Collaborative Environments (NICE) project [13] and Virtual Reality Physics Lab [14], have been developed for general education. A 3D, full-scale, highly interactive VR application titled Collaborative VR (CVR) [15] was implemented in a Risk Assessment course at Iowa State University. Students worked in teams on a full-scale, virtual reality model of the International Space Station, where they were required to design a system to solve problems and conduct risk assessment techniques on the system based on the lifecycle of the project. In the present study, our vision is to create multiple VR labs without any limitations for general education. The 3DP Training Virtual Laboratory will be the conceptual model and implementation architecture for further development.

For our VR application design, [2] described a design process for VR application. The design flow is divided into three major processes consisting of 10 steps. The first process is to determine the VR application’s usefulness, which includes two steps: definitions of objectives and VR devices’ specific objective definition. The second process is to develop a technical design of the VR application, which consists of four steps: realism-level design, user VR interaction environment design, multimodal immersive control design, and hardware/software (HW/SW) selection. The third process includes four iterative development and evaluation steps: modeling and programming VR interaction, user evaluations, improved identification of objectives, and subsequent VR modification. The design process is informative for this study on the development of the prototype of 3DP training VRVL, but there are some differences between the general VR application development and the 3DP training VRVL in this study. In the middle of its development, the prototype of the 3DP training VRVL was targeted to integrate gamification into the structure, because of limited user motivation after user evaluation. To integrate gamification or other critical technologies when developing applications is not a refining process, but rather it is a reconstruction or re-thinking process.

2.2. Gamification

The term gamification was coined in the early 2000s [18], and has become a highly popular technology and software trend. It has been used as central idea to incorporate game elements into non-entertainment application domains in order to foster motivation [19]. With the goal of engaging people in a variety of tasks, the use of game elements in non-game contexts is the kernel of gamification. In the field of education [20,21,22,23], there is a growing interest in gamification, as well as its applications and implications, since it provides an alternative to engage and motivate students, promote learning, and facilitate problem-solving during the learning process, as well as to drive desired learning behaviors [24]. In the creation of gamification scenarios, there are three kinds of game components: dynamics, mechanics and elements [25].

• Game dynamics: this refers to the high-level aspects of a game, including constraints, emotions, narrative, progression and relationships, such as rewards, status, achievement, self-expression, competition and altruism, and so on.
• Game mechanics: these are a set of rules that dictate the outcome of interactions within the system, while dynamics are users’ responses to the collections of these mechanics, such as challenges, chance, competition, cooperation, feedback, resource acquisition, rewards, etc. Through these rules, the game designer specifies when and how the state of the game, and therefore that of the players, evolves.
• Game elements: these constitute the basic level of the gamification process, and define the game concepts used in the application, such as achievements, avatars, badges, collections, content unlocking, gifting, leaderboards, levels, points, virtual goods, etc.

Based on a review and study of 26 papers, [21] identified seven different objectives of gamification used in education, including mastering skills, challenging activities, providing guidelines, providing engagement, improving learning, changing behaviors, and supporting socialization. These can be used as the bases of user experience direction.

However, [26] observed that the design challenges of software engineering complexities may be a cause of gamified software failure. Gamification is a design approach employing game elements into different types of systems and services, with the goal of providing game-like experiences [27]; however, the serious context in which gamification is applied generates requirements which may significantly limit the design space, compared with games, and thus adds another level of complexity. Gamification design is thus multifaceted and complex, and how it motivates users and affects behavioral change differs according to different contexts and different target users.

The study described in [26] aimed to advance our holistic understanding of how gamified software should be designed and implemented by developing a method for engineering gamification. The examination of these methods pointed towards seven main phases of engineering gamified software, in which each phase refers to several design principles, as described below.

• Project preparation: the design principles of this phase include clearly identifying project objectives, assessing whether gamification is the right choice to achieve the objectives, that stakeholders and organizations understand and support gamification, that metrics for the evaluation and monitoring of success be defined and used, and that the psychological and behavioral effects of a gamification approach are understood.
• Analysis: users’ needs, motivations and behavior, as well as the characteristics of the context must be understood.
• Ideation: this phase focuses on users’ needs, and users must be involved in this phase.
• Design: the design principles of this phase include expressing test gamification design ideas as early as possible, following an iterative design process, while remembering that designers have profound knowledge of game design and human psychology, that legal and ethical constraints be considered in the design phase, and that users be involved in this phase.
• Implementation: this phase has the same principles as the design phase, in following an iterative design process and involving users—the difference being that the gamification design must be continuously optimized.
• Evaluation: the design principle of this phase is to define and use metrics for the evaluation and monitoring of the project’s success, as well as the psychological and behavioral effects of a gamification approach.
• Monitoring: after the software is released, the psychological and behavioral effects of the gamification approach must still be evaluated and monitored, and the gamification design must be continuously optimized.

From the above studies, the game design elements, the phases of gamification software engineering and the positive and negative considerations of the gamification designs were used as critical references for the modifications described in this paper.

2.3. Virtual Laboratory (VL)

To support an alternative solution in training/learning, the virtual lab could be a feasible solution by integrating virtual devices, algorithms, and engineering designs. A virtual lab is a computer program designed to simulate real laboratory work and experiments [28]. A virtual lab-based education [4,29,30] contains various characteristics, including novel knowledge creation and internalized information construction, providing motivation and guidance for the students and for their automatic registration and evaluation of information, simulating either dangerous and/or expensive experiments, optimizing time consumption on the learning process, improving students’ academic achievements and enhancing their conceptual understanding—features that are especially suitable for children with a visual learning style.

To fulfill the requirements and solve the limitations of education/training/learning in a real environment, virtual laboratories have been applied in various educational domains via multiple engineering infrastructures. The authors in [31] planned to develop a Science, Technology, Engineering, and Mathematics (STEM) based virtual lab as an alternative medium for improving the scientific literacy of junior high school students on the theme of water pollution. The virtual lab was made by using Adobe Flash CS6 to develop interactive experiments with three water pollution indicators. In [31], the virtual lab developed was found to be valid and suitable for the improvements of students’ scientific literacy. It can help students learning about science by giving them access to information, measuring methods, and analysis.

A study in [5] proposed a new concept in virtual and remote laboratories: the augmented remote laboratory (ARL). It was tested in industrial engineering and computer engineering at the School of Engineering, University of Huelva, Huelva, Spain. This ARL is based on Augmented Reality (AR) techniques. The system structure means that this application is locally run on a user’s computer and grants access to the remote lab via Transmission Control Protocol/Internet Protocol (TCP/IP) by means of AR techniques that enable the interactive use of the lab equipment. The results of user evaluation showed that employing ARL improves student outcomes. The authors in [3] constructed a remote virtual laboratory for FDM 3D printer education. The FDM 3D printer is very different from the powder-based binder jetting 3D printer of the present study. In terms of material, the FDM 3D printer uses a plastic material, such as Poly Lactic Acid (PLA), Acrylonitrile Butadiene Styrene (ABS), etc. In terms of the machine operation, the printer head of the FDM 3D printer is capable of heating up the filament to melt the filament-type thermoplastic and to create a support structure from the bottom up; the printer head of the powder-bed binder jetting 3DP is capable of gluing the powder and coloring it to create a 3D printout without a support structure. This research provides another structural technique to construct a virtual laboratory.

3. The Development of 3DP Training VRVL with Gamification

In the mid-development of the prototype, we found an important problem after user evaluations. Even though the virtual environment and interactions were good enough from user evaluations, the students still had little motivation to use it. Some feedback repeatedly pointed to the same issue—that the prototype is neither fun nor interesting. Some feedback talked about why students do not see scores or a progress bar like for other games. All feedback reminded us that, while we were concerned about the prototype simulating the environment and the interactions among various machine operations in VR, we ignored mechanisms that could motivate students.

For these reasons as mentioned above, we tried to integrate gamification in the prototype. However, integration is not simply modification, such as adding videos in different lessons or UI modification, as we processed in the prototype before. It requires reconstruction and a re-thinking of the overall system structure. In the following sections, this study tries to illustrate our development flow for integrating gamification into the prototype of our 3DP training VRVL.

In relation to the gamification development phases [21], VR application development flow [31], and experiences in developing the prototype, we show our development flowchart, which is described in Section 3.1. Moreover, based on the development phases, the engineering design and information are expressed in Section 3.2.

3.1. 3DP VRVL Development Model

The development model of our 3DP VRVL is shown in Figure 3. The figure can first be separated into two parts: the domain-related structure (gray color) and the development flow (skin color). In terms of the domain-related structure, the objectives denoted by the red circle are for constructing the 3DP training VRVL. They contain three major related domains, including 3DP teaching/training, virtual laboratory, and immersive VR, denoted by the green teardrop shape; in the latter, the gamification is integrated into the application, denoted as an orange teardrop shape.

In terms of development flow, this is divided into four major processes: preparation, analysis, ideation and design, and implementation and evaluation. The four major processes are described as follows:

• Preparation: this process comprises several critical elements. The important aims to be addressed are to define the virtual laboratory’s objectives and the virtual laboratory’s contextual structure and simulation level. If development occurs iteratively, then the objectives may need to be modified.
• Analysis: based on the definitions from the preparation process, the major missions of the analysis process are to analyze the VR constrictions and feasibility of the VL requirements, as well as to analyze VR interactions and simulations in the VL context.
• Ideation and design: from the analyzed information, the ideation and design process not only focuses on designing the multimodal interactions and material design of the VRVL, but also focuses on system modeling (such as function, structure, sequence, and so on).
• Implementation and evaluation: the entire development flow is iterative, and the implementation and evaluation processes have their own mission loop. The loop will iteratively execute until the user evaluation is good enough to achieve the objectives and requirements.

3.2. Round 1 of 3DP Training Virtual Reality Based Virtual Laboratory

3.2.1. Preparation

The purpose of this study was to create a 3DP Training Virtual Laboratory in order to eliminate the problems of time, space, environment, possible dangers of machine operation and discomfort caused by dust. The objective of the 3DP Training Virtual Laboratory is to provide students with enough training opportunities to practice 3DP operation without any risks and costs until they pass the evaluation tasks set by the 3DP Training Virtual Laboratory platform.

When simulating the 3DP machine manipulations, immersive VR with two handheld controllers, such as those used in HTC VIVE, Oculus, Sony PlayStation VR, and so on, is suitable to simulate the training scenarios.

3.2.2. Analysis

In the analysis process, we need to analyze the feasibility and simulation levels for various 3DP training scenarios in the consideration of VR limitations. In the 90% training scenarios, like checking/refilling materials, roller cleaning, wiper cleaning, print head cleaning, and white nozzle/color nozzle testing and head alignment, the operations of these scenarios can be fulfilled by VR interactions with two handheld controllers. Only certain scenarios required further examination.

We take two scenarios as examples. The first scenario is to draw the ink into the pipe if the ink pipe is not full before printing, as denoted in Figure 4. In the real training scenario, the trainer needs to pull out the pin from the print head, insert the syringe into the pin, and draw the ink into the pipe. Some difficulties arise in this scenario. How can we simulate the subtle operations of fingers, like connecting the syringe and pin together and then pulling the syringe piston? These operations require coordination of motion and vision. In terms of visual design, a difficult issue is how the ink flowing in the pipe can be properly simulated the real runtime rendering of the game engine, as denoted in Figure 4b. In terms of interaction design, the operations of subtle fingers with three small objects, including syringe, pin, and pipe, are hard to simulate in VR interactions with the handheld controllers, as denoted in Figure 4a. Even if we could simulate this scenario, there would be a big gap between real operations and the virtual interactive simulations.

The second scenario that needs more examination regarding simulation by VR interaction is the changing of the pipe. In a real training scenario, the pipe will be dry and blocked if the 3DP machine is idle for too long. To remedy this, several strenuous operations are necessary: connecting the new pipe and blocked pipe by using a very tiny Teflon tube; pulling and pushing the pipe, following its path; and removing the blocked pipe and connecting the new pipe to the ink binder cartridge. One example is the operation of pulling and pushing the connected pipe along its path. This requires patience and a careful process of pulling and pushing the thin, soft pipe along the narrow path and crossing the operation platform in the up and down directions. It is hard to simulate the interactions between these tiny features and it is difficult to simulate the physical forces acting on pipes, such as friction force, elasticity, resistance force, and pulling force among multiple pipes.

The interaction simulation of the first prototype focused on operations among hard parts and an intractable object, in which simulated events are usually happened in realistic implementation. The scenarios that are not suitably designed for interactive simulation will be re-designed by 3D animations in the VRVL.

3.2.3. Ideation and Design

Our primary concern in the development of the first prototype was to provide adequate training hints throughout the entire training map. Not only did we simulate various interactions for 3DP machine operations, but we also designed multiple assistive training strategies, including multimodal task hint information. The aim of the prototype was to construct a 3DP Virtual Laboratory that provides sufficient guided and interactive mechanisms in order to provide a friendly 3DP training environment efficient enough for students’ self-learning. The primary design functions are described as follows:

• Training map: this provides an outline of the 3DP training. It allows trainers to understand the as-provided direct links to every training task scenario.
• Each training task section has its own workflow chart: the workflow chart presents the operational sequences of certain task scenarios. It provides more detailed information for a training task.
• Each work phase has its own visual hint and textual hint in the virtual space: the hints are designed to guide the next operation.

The design of the 3DP VRVL is shown in Figure 5. Figure 5a describes the major allocations of the main functions. The 3D models of the 3DP machine with interactable objects are shown in Figure 5b.

3.2.4. Implementation and Evaluation

As described in [16], the 3DP Training Virtual Laboratory prototype was constructed by integrating the Unity 3D game engine with Virtual Reality Toolkit (VRTK) and steam VR in a HTC VIVE immersive virtual reality device.

The implementation and evaluation process has its own iterative development phases. In the following section, we present the first version of the prototype of our 3DP training VRVL, which had a two-round iteration in this process.

Round 1 of Implementation and Evaluation Process

In terms of the visual hint design, a text hint, object hint, and workflow hint are provided, as shown in Figure 6a–c. Moreover, after one user evaluation with five participants in the middle stage of the prototype development, we found two pieces of design feedback necessary for the further development of UI interactions. The first point is that most students are usually confused about the VR device interactions if the VR platform does not provide interaction introductions at the beginning of the VRVL, even though some students had VR experience. Various VR applications may provide different interaction methods triggered by the same device buttons. As an example, in the VR environment, players may move themselves by clicking the tap button. Therefore, preparing an introduction of the VR interactions or help at the beginning of the VRVL is a better solution for users to more efficiently interact with the VRVL.

It is necessary to create VRVL interaction tutorial scenarios to teach players how to interact with the prototype. Figure 4 shows the first test scenario. Through the denotations of Figure 7a,b, one can see the differences between the interaction design for right-handed and left-handed controllers.

We invited five students to experience the prototype of the 3DP training VRVL and then observed the situations of how users interact with the prototype. Some students wanted to know the differences between VRVL and real 3DP operations. Some students still felt confused about how to interact with the virtual objects in the 3DP training VRVL, even though every participant had gone through the VRVL interaction tutorial scenario and all of them had VR experiences before. This feedback helped with the development in round 2 of the implementation and evaluation process.

Round 2 of the Implementation and Evaluation Process

From the user feedback, as described above, the prototype entailed a new aim of providing a suitable training video to show the real task operations and then recording videos of the interactive operations in VRVL for each training task scenario, correspondingly. As a result, the video control UI was the most important design parameter here. After the videos were integrated into the prototype, we carried out user evaluation once again. Figure 8 shows the video module in the 3DP VRVL.

The first version prototype focused on simulating the interactions between users and the 3DP printer machine using immersive VR. Teaching guidance is one of the most important parts of this prototype. It is necessary to let the user clearly know what to do next.

To measure the simulation degree of the prototype’s interactions and visual presentation by participant feedback, we administered a questionnaire that asked them to rate their level of agreement about five statements on a five-point scale (one = not at all, five = very). The five items were listed in Figure 9.

Participants in this evaluation were 11 undergraduate students enrolled in 3D-related courses where they were learning about 3D modeling and 3D printing. Evaluation activities took place after the lab session.

As shown in Figure 9, overall, the responses were very positive and above four, except for item 5. While asking participants more about item 5, some students replied that they are not motivated to use the 3DP virtual laboratory, because it is neither fun nor interesting. Some students stated that the true score of the item 5 may be lower than the score they gave. The results were unexpected in relation to what was initially predicted.

Three lecturers teaching 3D printing were invited to give more advice about which mechanisms gave students more motivation and which demands to ignore. We had some discussions and got three points of advice. The first was to apply game-design elements, such as a score and leaderboard to make students feel like they had achieved something or were part of a competition. The second was to ask some questions about the training after students finish a certain task scenario in order to remind them about what they should be learning from the scenario. The way students answer the questions can be fun or entail playable interactions. The third was that the prototype could gather students’ learning information for the teacher and students themselves so that they might efficiently understand their learning process, displayed in a similar manner to game progress scores on a dashboard. In order to increase students’ motivation to use the platform, this study set advanced objectives to integrate gamification into the prototype in the three directions described above.

3.2.5. Discussion

From Figure 3, the general VR development should be gradually refined and completed at this stage. However, from the suggestions of the students and teachers, we thought about the aspect of the research objective more clearly. Aside from hoping that such a system might solve the problems of the practice class, it was expected that the VR training system might positively influence students’ degree of participation. During the feedback about the user experience, the results were not as expected, thus making it difficult for us to continue our research into this teaching method.

From the findings above, gamification design is a feasible approach to think about how to correct the system in order to affect students’ desire for continuous learning, given the feedback of students and lecturers. However, the gamification design affects the original system structure. Thus, we must add a database and a function for information access, as well as thinking further about how to include gamification design in the training system.

3.3. Round 2 of 3DP Training Virtual Reality-Based Virtual Laboratory by Integrating Gamification

3.3.1. Preparation

After the user evaluation described above, the modified objective was to enhance the motivation and engagement of the students. The solution was to integrate gamification in order to make the learning experience more engaging, enjoyable and productive.

We selected the initial game design elements from the users’ advice based on the collection of game design elements [32,33]. The initial game design elements include player status, level, progress, a leaderboard, and badges. Status should present the score of players interacting with each training action and training concept under a certain time pressure. Level is based on users’ preferences for demanding interactive hints, including easy levels (with visual/auditory hints), middle levels (with auditory hints), and hard levels (without hints). Progress will represent the personal learning situation of each task for each player at their corresponding level. The leaderboard represents the top five ranked students with the highest scores. The badges are considered as “symbols of recognition” to offer a sense of achievement after each task.

3.3.2. Analysis

The analysis phase defines the target users and user demands. Figure 10 is a Unified Modeling Language (UML) use case diagram, created with a defined UML, describing the modified user demands for interacting with our 3DP Training Virtual Laboratory based on the three directions as described above.

In terms of targeted user analysis, besides the role of the teacher, 3DP VL trainees are divided into two kinds of users: VR-inexperienced trainee and VR-experienced trainee. Students participating in 3DP training VL are assigned their own user profile in order to save their progress, score, and status in order provide gamification information for game components, such as a leaderboard, points, badges, and so on. VR-inexperienced trainees are provided with an interactive VR interaction guide, with a gamification UI guide designed by the 3DP training VL. The guide is designed to target users who have had few opportunities to interact with immersive VR and thus need to familiarize themselves with the VR controllers and VR headsets. VR-experienced trainees can select training tasks from the 3DP training context agent combined with gamification designs. In terms of teachers, the platform not only provides trainees’ profiles for teachers in order to trace the trainees’ training progress, but also provides a training context modification authority for context modification. The differences between the first version and the new designs of the 3DP training VL are shown in red in Figure 10.

Compared with the first version of the prototype of our 3DP training VRVL, the revised one requires a more conceptual structure design and data management technology support to fulfill the user requirements. How can the game elements be integrated into the prototype? Can the game elements really improve students’ motivation? Answers to these questions will be very useful.

3.3.3. Ideation and Design

In this phase, this study determined the initial defined associations between the training context and the game components, as shown in Figure 11, which could be changed or modified after further evaluation. The left side of Figure 11 describes the structure of the training context in the 3DP Training Virtual Laboratory. The training consists of multiple tasks that may also contain various sub-tasks. Each task consists of multiple operations. Every operation comprises multiple interaction scripts.

In terms of game elements, training-level game elements and task-level game elements are different types of game elements that process various ranges of affordance. Task-level game element states are based on the feedback from the interaction script, and represent whether a user has performed a correct or incorrect action. The duration and count are based on how many operations a user completes in the selected tasks and how long they take to complete these operations. Goals are the task targets. Points and progress are processed from the state information, duration and count. Training-level game elements include the budget and leaderboard, which require a wider range of information. Level is determined by whether users require the assistance of training hints.

The 3DP training process has a number of first-level tasks, including checking materials, checking the machine, preparing print heads, and so on. The materials to be checked in the first-level tasks make up five sub-tasks, including powder checking, binder checking, cleaner checking, ink checking, and wastewater checking. The powder checking sub-task includes multiple interaction scripts, including pulling out the recycled powder tank, checking whether the tank is full or not, sieving powder, adding powder to the powder feed box, and so on. Each interaction script processes the interaction between users and the platform to manage whether each action is right or wrong, in order to give proper feedback and to control the sequence defined by the task workflow.

3.3.4. Implementation and Evaluation

In this process, the first goal is to collect data about whether game elements in the 3DP VRVL could improve user motivation or not, by carrying out a user evaluation. For user evaluation, we presented and introduced the first prototype of the 3DP training VRVL initially. Next, we introduced to the participants the various UI designs within the game elements in the 3DP training VRVL, shown in Figure 12, in which (a), (b), (c), and (d) show the state, progress, badge and leaderboard of affordance, respectively. To measure the improvement level of gamification design for the 3DP training VL, we used a questionnaire with four items, as listed in the first column of

Table 1. Descriptive statistics of the questions on a seven-point Likert scale.

no		Mean	Std. Deviation
1	Do you agree, that the UI of interaction status for each session is suitable to be presented in various symbols?	4.6613	1.31752
2	Do you agree, if you add a progress UI, then it will make students know more about their learning progress and want to complete their studies?	5.1129	1.02597
3	Do you agree, that if a badge UI is added, then it will make students have a sense of accomplishment and further want to complete their studies?	5.0000	1.22809
4	Do you agree, that if a leaderboard UI is added, then it will make students feel competitive and further want to complete your studies?	5.0000	1.35501

. Figure 12a–d and four items from the questionnaire of the user evaluation were published for participants.

Each item was rated by participants on a seven-point Likert scale, with one representing strongly disagree, and seven representing strongly agree. It was conducted with 62 participants, among whom 21% were female and 79% were male, and they were between 19 and 25 years old, as shown in

Table 2. Descriptive statistics of user evaluation.

		Frequency	Percent
Sex	Male	49	79.0
	Female	13	21.0
	Total	62	100.0

. Every student had 3D printing knowledge and FDM 3D printer experience from their academic education. Answers were averaged to form a reliable scale (Cronbach’s alpha = 0.867). As shown in Table 1, overall the responses were positive (mean: 4.94, min: 4.67, max: 5.11). The gamification design can help improve students’ motivation and engagement with the 3DP training VL prototype.

There were some suggestions for the aesthetic setup of the UI design, such as more attention being paid to styling and aesthetics, and so on. Some students also gave more advice about whether the prototype provides other interactive game mechanics, such as avatars and rewards with small interactive games.

The 3DP Training Virtual Laboratory is still in development. The prototype continues to be examined and modified in the iterative design process as well as the implementation and evaluation phases. The gamification will be fine-tuned in the process, according to user experience evaluations.

4. Discussion

The system developed herein is expected to solve the problems of practical operation during teaching and also attract students to continuously use the system until they are familiar with the operation of the powder printer. In order to achieve these two objectives above, at the beginning, VR technology was employed to develop a virtual laboratory. This study gathered students’ positive feedback of the technology in the learning system. Thus, it is expected that the VR immersive experience, as well as the interactive guidance, procedures, and 3D virtual simulation can effectively realize students’ continuous use of such a virtual laboratory in order to become familiar with the operation of the powder printer.

For the research on the virtual laboratory related to 3DP technology, [3] proposed a remote lab, which allows students to share equipment through the network and observe the printing process via the camera; however, the 3DP technology in this study is Fused Deposition Modeling (FDM) printing technology, which is different from powder 3DP in terms of its materials, printing method, and machine implementation, and this remote laboratory is used mainly for software implementation, which involves less calibration or setting up of the machine. Nevertheless, 3DP technology is part of engineering education and needs to be taught.

Regarding the use of VR technology to build and develop a virtual laboratory, VR technology was used to build virtual laboratories in [1,2,6,7,10,11,12,13,14,16,17], and there has been positive learning feedback from chemical education, medical training, STEM teaching and sports training. In the case of experimental simulation, proper simulation and simulation-based applications can usually enable students to easily express their true reactions, thereby increasing the speed of learning. It enables the students to make their own hypotheses and develop their own problem-solving methods on this topic, it simplifies the complex information given to them through technology, and provides them with the opportunity to learn while they are doing. At the same time, the advantages of a VR virtual laboratory in engineering education were also clearly pointed out in [2]. They also represent the training and teaching problems that this study hopes to solve, which are organized as follows: (1) VR can simulate the use of expensive laboratory equipment that is difficult to obtain in real time, (2) using VR can avoid potential damage to real machines, due to its misuse by students on actual courses, (3) virtual reality solves the difficulty of developing practice classes in real laboratory environments under overcrowding conditions, (4) virtual reality improves the prevention of occupational hazards, (5) VR allows students to participate in the interaction of the complete manufacturing process, including problems and solutions that are seldom encountered, (6) VR can shorten the time-consuming and discontinuous real operation process into a practice of continuous key tasks. In the development of the entire VR application system, reference is also made to the VR development process described in [2]. At the same time, [2] also stated that engineering education, real simulation and interactivity are the foci of VR development in teaching. Because these studies put forward the advantages of VR in education, as well as development suggestions, this study introduced VR technology to solve the problems encountered by students in powder 3D printing training and education. Therefore, this study focused much of its efforts on how to use VR to simulate the operation of the powder 3D printer station and interactive guidance. The authors in [16] described how to modularize VR interaction.

To rapidly develop and correct the 3D interaction and visual simulation function of the system in real time, we took a small number of user experiences and feedback for system correction. When the system functions were almost completed, we once again took a small number of user experiences and feedback. We expected that 3D interaction, guidance, and visual simulation can produce an immersive VR experience and offer students more varied learning experiences. However, we did not expect that a simple problem coming from the feedback of students’ continuous learning after their experiences could produce a barely satisfactory result. Originally, after such experiences, if good feedback is achieved, then a study of the teaching model can be successfully conducted. However, the system’s progress stagnated here, and we had no idea of whether to continue with it or not. After comprehensively analyzing the suggestions of three teachers on this system, although the developments were insufficient, we knew that the modification of the entire system structure could be affected. For example, we needed to add database design and data access for the examination of the teaching model in the future. Researchers can still choose whether to add gamification design or not, so as to better improve students’ degree of learning participation in this system. Although few people have such experiences, their experiences can be used by developers to correct a system’s functions.

As can be seen in the original development of the system above, research into using this VR system in teaching can be conducted if the UX testing experience of the first round of system development is positive. However, the evaluation of student participation and motivation in the UX experience was, unexpectedly, not the same as the results of the simulation and interactive experience. Therefore, we must analyze further whether or not VR applications can get positive feedback in teaching.

The technology of Collaborative VR (CVR) was used in the research of [15], where CVR was implemented in a Risk Assessment course at Iowa State University, which allowed many users to work in a virtual reality environment and build systems. It was hoped that this experience would create the shared mental model that they needed, and they were required to effectively assess the risk of the system. However, the results showed limited success in assessing the system risks. As per our analysis of the results of that study, the HTC VIVE for CVR did not have the function of simulating touch or smell, and the transmission of signals through the tactile or olfactory channels may enhance the risk assessment of industrial systems. However, the HTC VIVE had an audio function and could be used to enhance risk assessment through audio sensory information when CVR was not realized.

As this study expected that VR can enhance students’ participation in powder 3DP learning, the interactive tutorial of the entire virtual laboratory still simulates actual experimental teaching. Therefore, students hoped to have interesting game elements added in the wake of our reaction to their feedback, in order to prolong their attention and persistence in the laboratory. Compared to the research of [15], the application in this case did not achieve the desired effect, but the difference is that the analysis of the results of [15] showed some of the signals that needed to be simulated, such as touch or smell, which cannot be communicated to users by VR devices, while an auditory feature was not implemented in the application, so it was difficult to obtain effective results. In addition, because the emphasis was on student participation and the enhancement of learning motivation, it is also necessary to add enhancement factors for real learning interactive simulation, instead of just real simulation. Although VR initially gives students a sense of novelty and curiosity, if there is no enhancement factor in the learning content, it will not increase the students’ learning continuity and motivation.

Compared to the research of [1,2,6,7,10,11,12,13,14,16,17], in relation to medical surgery and sports simulation training, the trainee had a certain target for their training outcome because of the highly professional level and the actual seriousness of the training. At the same time, the aim of other chemistry and physics virtual laboratory research is not to enhance the sustainability of learning and learning motivation, but to evaluate VR applications with the study results and to solve the problem of training limitations in the real environment, which is why we achieved different results by using VR to develop teaching or training virtual laboratories. Of course, in this study, the small number of users in the first round of system development is also a factor that must be considered. The reason for having a small number of users is that it takes longer to test VR, because VR requires higher standard hardware, it is impossible to set up multiple VRs for testing at the same time, and the testing of each VR requires the auxiliary personnel to carry out hardware-assisted loading and preliminary hardware interactive teaching. Under the restrictions of the hardware and auxiliary personnel, the progress of the testing was slow, so not many people participated in this process.

Because it seems that the use of VR technology and 3D simulation to build a virtual laboratory cannot have a more obvious effect on enhancing student participation and learning motivation, the suggestions of students and other teachers, as well as the analysis of the related teaching research literature, were sorted and analyzed before the gamification system was added in the second round of system development. In the overall process of gamification system development, we referred to the development framework of the game software presented in [26], as well as the VR software development process that was put forward in [2], and we proposed a research and development process architecture that is based on the research and development status of this study. Therefore, our development process was based on integrating the concept of [2,26], and it was developed according to the development process of this study.

In terms of the mechanism and design of gamification, [21,23] conducted a large-scale literature review and investigation into the analysis of game elements in engineering education. We used these game elements for the second round of system development. However, to implement these game elements in the virtual laboratory of VR, the course structure of the virtual laboratory must be mapped to a game structure to facilitate the implementation of the game elements; therefore, this study proposed an architectural diagram of the curriculum that corresponds to the game concepts (as shown in Figure 11).

In short, this study carried out two rounds of system development under the restrictions of manpower development, auxiliary manpower, and hardware. It was originally hoped that the introduction of VR technology could enhance the continuity and motivation of students in a virtual laboratory; however, in the system’s user experience and feedback, it was understood that the technology of VR was not enough to enhance these aspects, so we also added the gamification design. Throughout this process, we have referenced and integrated the relevant literature and research cases. Therefore, if a teaching system is developed in the future, with the goal of strengthening student participation, it is recommended that the method of gamification design be considered in the course content design.

5. Conclusions

Although the prototype system of 3DP printing is still in development, our focus is on illustrating the virtual laboratory built by VR technology. It is suggested that some appropriate gamification elements be added at the beginning of the development, instead of only considering the construction of core functions, because the software construction consumes a lot of time and labor. As we only had two members on the development team, we did not only need understand the technology, process, and implementation of 3DP printing, but we also needed to practice using the VR technology. In addition, in the later stages of system development, only the author remained in the team.

This study did not take into consideration the importance of gamification design. It was only until we carried out a user evaluation of the first prototype for the establishment of our virtual laboratory that we realized that the core functions and interactive simulation technology were not enough, and then we thought further about the impact of gamification design on the students’ motivation. From the development framework put forward in this study, as well as the details of system development described in Section 3, we realize that gamification design can be introduced at the beginning of the development of the virtual laboratory. The overall development can be more efficient if we give more consideration to the students’ experience in the teaching content. Although the system has not yet been completed, we hope to share this development experience with other developers.

This study integrated gamification into the immersive 3DP Training Virtual Laboratory prototype developed in [16] in order to include edutainment in the learning content of the software. Our platform for students is proposed to support 3DP training through improved engagement and understanding. The aim of the first prototype was to eliminate the problems of time, space, environment, the possible dangers of machine operation, and discomfort caused by dust. After the user experience evaluation conducted in [16], this study sought to better motivate students to use the platform, in order to increase their experience of operating 3DP. Therefore, in the mid-development stage of the prototype, we decided to integrate gamification. Because the decision impacted the entire system structure, the development model of the prototype is different from the development phases of general VR applications. We also proposed a development model for integrating gamification into the prototype of a VR application.

This study has thus designed various game elements to be integrated into the 3DP training platform, and conducted a further user review survey, which yielded positive feedback and recommendations. Future work will modify the prototype according to user evaluations, and further explore the relationship between learning efficiency and gamification in our 3DP Training Virtual Laboratory.