Digital Twin for Developing and Verifying Semiconductor Packaging License Models ^†

Abstract

The traditional semiconductor packaging training process is time-consuming and carries the risk of damaging precision equipment due to improper operation. Additionally, the retirement of experienced trainers has led to loss of specialized training and testing expertise. To address these challenges, digital twin technology is applied to training packaging engineers. We conducted an empirical study at the packaging production line of the Minghsin University of Science and Technology to address talent training bottlenecks and imbalances between supply and demand. First, an integrated software and hardware system was designed by combining digital twin and mixed reality (MR). The development process of the digital twin system for the wafer-dicing machine includes on-site visits, machine operation instructions, certification content development, expert validity construction, small-scale testing and modifications. We compared the pre- and post-experiment scores of industry experts to evaluate the operation time of five participants and their feedback. Digital twin and MR for simulated training increased proficiency in operation. The digital twin training and certification model developed in this study improved students’ pass rates in certification exams.

1. Introduction

We analyzed the current situation of training for semiconductor packaging certification. The identified issues included the following: (1) limited skilled workforce to preserve and pass down knowledge and skills, (2) leveraging strong educators to teach logical thinking and exceptional key-point communication skills, and (3) time-consuming training and certification. By utilizing digital twin and artificial intelligence (AI) technologies, the experience of technicians can be preserved and transmitted to apprentices to teach valuable know-how.

Digital technology can be used to benefit students and colleagues by using virtual avatars on an integrated platform. In-depth interviews with scholars revealed that tasks such as bonding wire and changing ceramic nozzles are particularly challenging. Currently, candidate performance is evaluated based on scoring standards to save time and improve performance outcomes. Therefore, developing solutions using digital twin technology and enhancing the efficiency and effectiveness of training for certification in the semiconductor packaging industry is required.

2. Related Work

2.1. Digital Twin

The importance and impact of digital twin (DT) technology has become evident in product development, manufacturing, and service sectors. DT technology allows enterprises to create a digital replica that mirrors the physical object or system in a virtual environment. Digital twins can be updated in real-time to reflect the status and behavior of their physical counterparts. DT enhances the flexibility of product design and manufacturing efficiency and allows for more precise maintenance of products [1,2,3]. DT uses real-time simulation and predictive analysis to support every stage of the product lifecycle from design to production, and during product servicing. This technology is beneficial in identifying potential issues early as it reduces the need for physical testing and prototype development, thereby saving costs and accelerating time-to-market [4].

The application of DT in Industry 4.0 is significant. It integrates with technologies such as the Internet of Things (IoT), cloud computing, and AI using an efficient data and analytics network. This enables the manufacturing industry to monitor and optimize production processes in real time. DT enhances operational transparency that allows businesses to make faster, data-driven decisions [1]. Furthermore, DT also aids enterprises in transitioning towards sustainable production methods. For instance, in vertical farming, DT is used to simulate and optimize planting strategies to improve resource efficiency and crop yield [1].

In summary, the importance of DT technology lies in its ability to provide deep insights and predictions for accelerating innovation, improving efficiency, and achieving a higher degree of customer customization. As this technology continues to develop and be applied, its impact across various industries is expected to expand further.

2.2. Mixed Reality (MR)

MR combines virtual reality (VR) and augmented reality (AR), overlaying virtual information to provide an immersive experience. MR is transforming industrial training and operations. In semiconductor manufacturing, MR allows for operators and engineers to receive more intuitive training, familiarizing them with the operation of equipment such as wafer-slicing machines. MR is used to simulate work scenarios during training, enabling learners to master operational skills, enhance safety awareness, and reduce errors in actual operations. MR technology allows trainees to overlay virtual information in real-world environments. For example, operators can see the internal structure of machinery, operational processes, and real-time data through MR devices to more intuitively understand and master complex operational skills. Additionally, MR provides interactive educational content, such as step-by-step guidance and real-time feedback for trainees to practice their knowledge in real environments.

2.3. Training

DT and MR technologies, including the Meta Quest Pro, and semiconductor packaging equipment, such as wafer-slicing machines, show a tightly integrated and complementary relationship in creating an efficient and intelligent ecosystem in the industrial and technological fields. DT is used to create virtual models of physical devices. These models replicate the appearance of the physical devices and simulate their operational behaviors, making DT prominent in semiconductor manufacturing. The complex operations and high precision requirements of equipment such as wafer-slicing machines make DT an ideal tool for optimizing production processes, reducing downtime, and conducting preventative maintenance. By simulating real operational environments, engineers can test and adjust equipment in the virtual world to ensure optimal performance when operating in real-world settings.

The combination of DT technology with MR in training enhances training effectiveness. Virtual models of equipment generated by DT can be interactively manipulated within an MR environment, allowing users to practice in a virtual setting. For example, by using the Meta Quest Pro for simulating operations on a wafer-slicing machine, learners can perform cutting operations on a virtual model, observing and analyzing every detail of the cutting process. Virtual training helps learners better understand how the equipment operates and allows practice in a safe and low-cost environment. In the highly competitive and technology-intensive industry of semiconductor manufacturing, the integration of DT, MR, and advanced training equipment such as wafer-slicing machines is leading the Industry 4.0 revolution, driving the industry towards more intelligent and digital operations.

2.4. MR Training System

Astronaut training is complex and costly due to its specific infrastructure and equipment requirements. Hence, interactive user experiences must be simulated using immersive technologies to model the cockpit environment. Astronaut training is conducted to evaluate the integration of DT with immersive technologies for effective training. Although MR immersive technologies and DT offer advantages and applications, their synergistic integration can create a specialized training platform. Initially, by incorporating spatial elements, complex scenarios can be created to simulate and confirm the consistency of developed DT with actual systems, thus validating their practicality in astronaut training [5]. Using DT and MR for training in semiconductor packaging equipment such as wafer-slicing machines, die bonders, and wire bonders has advantages and disadvantages as follows.

2.4.1. Advantages

• Enhanced Training Efficiency and Safety: MR headsets provide an immersive training environment, allowing trainees to interact with 3D models of the equipment. This hands-on experience helps trainees to understand complex mechanical operations without risking damage to expensive equipment or accidents [6].
• Immediate Feedback: DT is used to simulate real-time scenarios and provide feedback, enhancing learning outcomes through visualized interfaces of operational results.
• Cost Reduction: Training on virtual models can reduce the risk of actual machine downtime, thus preventing production losses. This is particularly important in semiconductor manufacturing, where the costs of equipment downtime are very high [7].
• Improved Maintenance and Troubleshooting: DT is used to predict potential failures and maintenance needs based on real-time data. Training maintenance personnel with these predictive insights can prevent unexpected downtimes [8]. Trainees can learn how to diagnose and troubleshoot in a simulated environment, preparing for real scenarios without the fear of damaging equipment.

2.4.2. Disadvantages

• High Initial Investment: Implementing DT and MR requires substantial initial investments, including costs for software, hardware, and infrastructure. This may be a barrier for smaller companies [8].
• Maintenance and Updates: DT models and MR systems must be updated with data and software versions for with investment and technical expertise.
• Complexity and Data Demands: DT requires extensive data to accurately simulate real-world conditions. Collecting and processing this data is challenging in dynamic and complex manufacturing environments [7].
• Training and Adoption: MR headsets and DT interfaces have steep learning curves, potentially requiring additional training for employees [7].

In summary, although using DT and MR headsets in training for semiconductor packaging equipment offers advantages, efficiency, safety, and cost savings, there are challenges such as high initial costs, complex data management, and the need for specialized training and integration. Enterprises need to consider these factors when adopting these advanced training technologies.

2.5. Factory Valve Inspection

In factory settings, computer vision is used for status tracking, with Meta Quest technology informing users to what degree a valve or switch to be turned. Image recognition is used to transmit information to the MR system in a one-way interaction, mostly utilizing animations or step-by-step instructions. For example, the state of a rotating valve can be determined through image detection to decide the current attributes of pipelines or equipment. A virtual 3D model can be created to the right side of the actual electrical equipment at a 1:1 scale (Figure 1) which is a concept of DT. A comparative model needs to be built including physical devices to provide demonstrations or guidance in videos or on interfaces for one-on-one training or learning processes [9].

Figure 1. Concept of DT: The picture on the left is the digital twin of the wire cutting machine. The picture on the right is a digital twin of a six-axis machining center.

DT is efficient for the operation of certain equipment, whether for repeated training or assessment. For instance, DT allows for tracking and retrospective analysis due to prolonged operation or errors. Multiple machine MR connections allow for operating simultaneously, including inspecting valves, which indicates a chemical gas leak. This could happen due to natural disasters or human damage within the factory, leading to leaks. This training method determines valves requiring pressure reduction or pipelines to be isolated. It involves image recognition combined with MR training techniques. It also incorporates process diagrams or checkpoints to facilitate content input and verification (Figure 2).

Figure 2. The virtual object in the picture is the transparent tube water flow effect test mode.

DT and MR have potential advantages in training for critical equipment in factories. DT technology creates an accurate digital replica of physical devices, enabling operators to perform operational simulations, troubleshoot, and optimize performance within a virtual environment. This provides a safe training setting, reducing the risk of damage to actual equipment from operational errors and records and analyzes operational data, offering training evaluations and recommendations for improvements [9]. DT and MR technologies enhance the quality and efficiency of packaging equipment training by providing a safe, efficient, and interactive environment, effectively addressing many issues inherent in traditional training methods.

3. Method and Implementation

3.1. Live Demonstration and Observation

To effectively master the operational details of the actual machines, on-site video recording and photography are required. The managers of the semiconductor institute-like production line are requested to operate all processes. The recordings are then transcribed into verbatim scripts and audiovisual files, which are finally reviewed by the managers for accuracy. In the operation, each item and test content must be verbally described to take notes and discuss scoring standards in real-time. In this study, a wafer-dicing machine was used for two examinations: the wafer-cutting process and the blade changing process of the dicing machine. Each major topic consists of five sub-questions, with each question scored out of 10 points, making a total of 100 points.

3.2. Development of DT and MR Content

We conducted a detailed analysis and study of the wafer-dicing machine to precise 3D modeling (Figure 3). Unity software was used for the development of the platform. It was used for its robust 3D rendering capabilities and the suite of tools and plugins that support various VR and MR devices. We used the Quest Pro, an advanced MR headset for high-resolution display and accurate spatial tracking capabilities for realizing highly interactive DT simulations.

Figure 3. The picture on the left is a 3D digital simulation of a wafer-slicing machine. The picture on the right is a digital twin close-up of the process of replacing the cutting blade.

In a 3D modeling process, tracking technologies were used to ensure the accurate alignment of the model in physical space, enabling operators to interact with the model in the MR environment. This integration of technology allows for creating a digital twin of the wafer-dicing machine that is precise and interactive to enhance the training and operational understanding for users.

A real wafer-dicing machine was replicated 1:1 using DT. In collaboration with instructors, we collected detailed operations and safety precautions to design an interactive user interface that allows trainees to engage in immersive training using Quest Pro. Using the Unity3D development tool, we integrated the DT-built machine model with interactive operating steps and developed the MR application to realize interactive training scenarios.

In training, the spatial positioning of the MR device allows for moving the virtual training machine for synchronized operation. It also relocates the machine model in classrooms and dormitories, allowing trainees to conduct simulated training outside of a production-line-like environment. This flexibility enhances learning outcomes by providing practical experience in a variety of settings, ensuring that trainees can practice and refine their skills in diverse and controlled environments.

3.3. Validation

Once the system integration was completed, experts reviewed functionalities and steps of the UI interface displayed on screens. The invited industry experts from a leading domestic packaging equipment manufacturer possessed over 10 years of experience. They operated the MR simulation training system and provided feedback for necessary modifications. Managers from the production-line-like equipment site assisted instructors in training the participants. In using the MR headsets for operations, the following issues were identified.

• The decimal point appears too quickly after setting the dimensions of the wafer size.
• Clicking to start the wafer-feeding process requires an additional explanation in task box number five.
• Manual adjustment steps for horizontal alignment need a screen-operable option.
• After confirming F1 for horizontal alignment, there is a missing Display Change action.
• The sixth prompt box changed to “Click to Start”.
• The screen displaying completion of blade changing (BBD value setup) appears too early.

These issues were addressed through iterative improvements to ensure the simulation training system meets the requirements and effectively prepares participants for certification.

3.4. Optimization and Testing of Interactive Interface

Following the instructions by industry instructors and specialists, we solved the issues such as changing the prompt window to manual horizontal alignment requiring checking both sides for horizontal alignment. The START button is used to begin wafer feeding (Figure 4). These iterative corrections involved multiple reviews and confirmations among the development team, industry instructors, and administrators to ensure the accuracy and rationality of the MR simulation system.

Figure 4. Corrections to the machine interface screen: Correction (left), Correction (middle), Correction (right).

3.5. Small-Scale Experiment

The participants were graduates from the Minghsin University of Science and Technology who majored in electrical engineering and were engaged in the experiments of this study. Before the experiment, each participant submitted an informed consent form and were compensated with a gift card. Before the participants began virtual operations, adjustments were made to the height of the machine’s display screen due to variations in participant height. However, these adjustments were made after the training started, and the time taken to adjust the screen height was not counted in the overall time calculation.

The participants used the “think aloud” method or recited the steps aloud to reinforce learning. Before the experiment, an industry instructor asked the participants to operate the wafer-dicing machine to score in the pre-test. Each participant operated the actual machine and was trained using the MR simulation. Their performance was assessed again after the MR simulation training by the instructor.

After the experiment, each participant filled out the System Usability Scale (SUS) questionnaire. The survey was conducted to determine the user experience of the MR wafer-dicing simulation training system. The SUS uses a 5-point scale, where 1 represents “strongly disagree” and 5 represents “strongly agree”. The participants selected multiple scores that best matched their experience.

4. Result and Analysis

The participants were being trained at the semiconductor academy to obtain certification in packaging equipment. Each participant completed five sessions of the MR simulation training system, with their times recorded over a week as shown in Table 1, which presents their proficiency and completion rates with the wafer-dicing machine.

Table 1. Overview of five students’ MR simulation training times across five sessions.

Subjects	T1	T2	T3	T4	T5
S1	10 m 54 s	8 m 16 s	6 m 50 s	5 m 11 s	5 m 25 s
S2	10 m 23 s	8 m 11 s	4 m 37 s	5 m 31 s	4 m 46 s
S3	6 m 49 s	6 m 22 s	4 m 41 s	4 m 34 s	6 m 45 s
S4	6 m 57 s	6 m 26 s	5 m 2 s	5 m 13 s	5 m 13 s
S5	6 m 59 s	6 m 28 s	5 m 0 s	7 m 54 s	5 m 53 s
Average	8 m 24 s	7 m 9 s	5 m 14 s	5 m 41 s	5 m 36 s

The average time was 8 min and 24 s in the first session and decreased to 5 min and 36 s in the fifth session. This time reduction was due to their increasing familiarity with the system. They were initially less skilled but eventually adept at completing all tasks more efficiently. In each session, the participants employed the “think aloud” method or recited the operational steps to reinforce learning, further enhancing their mastery of the process.

Table 1 displays scores before and after five MR simulation training sessions under instructor observations. The scores were assigned by the industry instructor both before and after five sessions of MR simulation training. The “Pre-test Score” represents the initial score given by the instructor based on the student’s performance in their first operation of the wafer-dicing machine simulation system. The “Post-test Score” shows the scores after completing five simulation training sessions, indicating significant overall improvement.

In initial scoring, the instructor pointed out common errors made by the participants, which affected their completion times and final scores and guided the participants’ improvements throughout the training sessions.

Table 2 highlights the improvement in students’ machine operation skills, with average scores rising from 76 before training to 93.6 after MR simulation sessions. This significant increase, confirmed by statistical tests, indicates the effectiveness of MR training in enhancing student proficiency. To assess the improvement of scores, a paired t-test was conducted. The p-value was 0.0136, indicating a statistically significant difference. This result implies that the participants significantly improved their skills after the training sessions. The Wilcoxon signed-rank test result showed a statistic of 0.0 and a p-value of 0.0625, suggesting a substantial improvement in their ability. The correlation coefficient between the average training times and scores was 0.133, suggesting that the reduction in practice time is not correlated with the improvement.

Table 2. Instructor-assigned scores for each student’s real machine operation performance before and after training.

Subjects	Score 1	Score 2
S1	90	95
S2	60	91
S3	80	100
S4	70	86
S5	80	96
Average	76	93.6

The paired t-test and Wilcoxon sign rank test results indicate that the participants’ scores were significantly improved after multiple training sessions. While there is correlation between the average training times and final scores, the relationship was not significant, indicating that reduced time is not necessarily correlated with scores.

Expert 1 said that the training system has an excellent foolproof design to prevent errors in interface operation and is user-friendly. Expert 3 noted that the system allows for practice anytime and anywhere, without location constraints. Expert 4 believed the system helps inexperienced learners understand machine operations. Expert 5 found the system convenient for learning, as it enables practice without a real machine.

In terms of learning effectiveness, the four experts felt that the training helped them understand machine operation and reinforced their knowledge of the human–machine interface operations. They had positive views on its application with excellent learning and revision resources for beginners or those unable to use the machines immediately. The system has an excellent user interface for simplifying use and enhancing understanding of procedures through on-screen explanations. However, deficiencies were observed, particularly in blade changing, which requires actual operation to fully grasp. A lack of tactile feedback due to the absence of physical interaction with machine parts was also observed.

The SUS scores for the MR version of the wafer-cutting machine simulation training system indicated high usability with high scores ranging from 62.5–95. The average score was 83. In the SUS, scores above 68 are regarded as good usability standards. Scores exceeding 83 indicate that the system’s usability ranges from good to excellent, suggesting that most users find the system easy to use and well-suited to their needs.

The results of this study demonstrate a high user satisfaction with the system. The user interface is well-designed, aligning with the users’ operational intuitions. This feedback is crucial for further refinements and expansions of the system’s capabilities, meeting user expectations in practical training scenarios.

5. Conclusions

We developed a DT virtual training system to address insufficient hands-on training time on the actual machines. The process encompassed site visits to the pseudo-production line, observation and recording, simulation system planning and design, system development and debugging, and ultimately, practical verification. The participants underwent multiple training sessions to become familiar with the wafer-cutting machine’s examination. A significant correlation between the pre- and post-test scores was observed; the more MR practice they had, the higher their operational scores. Expensive instrumentation and equipment lack an effective management system. If DT technology is effectively utilized, students can learn and practice anytime and anywhere, becoming proficient and ready for future opportunities.

Using this system, knowledge and skills through DT technology can be improved. Future research with more participants and extended duration is necessary to compare differences and correlation and enhance the impact of this simulation training system.