Deep Learning-Enhanced Electronic Packaging Defect Detection via Fused Thermal Simulation and Infrared Thermography

Abstract

Advancements in semiconductor packaging toward higher integration and interconnect density have increased the risk of structural defects—such as missing solder balls, pad delamination, and bridging—that can disrupt thermal conduction paths, leading to localized overheating and potential chip failure. To address the limitations of traditional non-destructive testing methods in detecting micron-scale defects, this study introduces a multimodal detection approach combining finite-element thermal simulation, infrared thermography, and the YOLO11 deep learning network. A comprehensive 3D finite-element model of a ball grid array (BGA) package was developed to analyze the impact of typical defects on both steady-state and transient thermal distributions, providing a solid physical foundation for modeling defect-induced thermal characteristics. An infrared thermal imaging platform was established to capture real thermal images, which were then compared with simulation results to verify physical consistency. An integrated dataset of simulated and infrared images was constructed to enhance the robustness of the detection model. Leveraging the YOLO11 network’s capabilities in end-to-end training, dataset small-object detection, and rapid inference, the system achieved accurate and rapid localization of defect regions. Experimental results show a mean average precision (mAP) of 99.5% at an intersection over union (IoU) threshold of 0.5 and an inference speed of 556 frames per second on the simulation dataset. Training with the hybrid dataset improved detection accuracy on real images from 41.7% to 91.7%, significantly outperforming models trained on a single data source. Furthermore, the maximum temperature discrepancy between simulation and experimental measurements was less than 5%, validating the reliability of the proposed method. This research offers a high-precision, real-time solution for semiconductor packaging defect detection, with substantial potential for industrial application.

1. Introduction

As semiconductor packaging technology develops towards higher integration and stronger functional density, devices have put forward more stringent requirements for thermal reliability. Defects in the packaging structure, such as missing solder balls, faulty soldering of pad, bridging, etc., usually cause the heat flow conduction path to shift, resulting in abnormal local temperature rise, which may cause chip failure or even thermal damage. How to efficiently and accurately detect internal defects in the package has become one of the core challenges in chip manufacturing and packaging quality control.

In terms of defect detection, traditional detection methods rely on visual inspection or single-mode imaging technology, which makes it difficult to meet the requirements of accurate and rapid identification of micron-level defects. In recent years, infrared imaging technology has become an important means of packaging defect detection due to its non-invasive thermal distribution capture capability. Due to its non-contact, real-time, and non-destructive characteristics, infrared defect detection technology has been applied in aerospace [1,2,3,4], the automotive industry [5], construction [6,7], the photovoltaic industry [8,9], and chip packaging [10,11]. At the same time, researchers have tried to use infrared thermal imaging to detect thermal anomalies inside chips, especially active heating infrared imaging [12,13], which can observe thermal distribution distortion in a non-destructive manner to identify internal defects. Technologies such as infrared lock-in thermal imaging [14] and double-thermal imaging [15] have been continuously developed to improve temperature contrast and image accuracy. However, this technology still has limitations in terms of resolution, penetration depth, and noise immunity.

In addition to infrared detection, finite-element analysis reveals the correlation mechanism between defects and temperature field distortion through multi-physics field modeling. It is widely used to model thermal-mechanical multi-physics field problems and has been successfully applied to chip thermal warpage prediction [16], material thermal conductivity difference analysis, and solder ball defect modeling [17]. The finite-element simulation method can systematically reveal the influence of defects on thermal field distribution and construct simulated samples to make up for the lack of experimental data. However, its challenge is that the modeling accuracy is limited by material parameters (such as thermal conductivity, heat capacity, and temperature-dependent behavior of metals and polymers) and computing costs, and its adaptability to batch scenarios still needs to be improved.

With the support of infrared detection and finite-element simulation, deep learning algorithms provide reliable technical support for high-precision real-time detection. In defect detection, deep learning algorithms such as the YOLO series [18] have been widely used in industrial inspection tasks in recent years. They have the advantages of end-to-end training, strong small target recognition ability, and fast reasoning speed, and they are suitable for high-speed and high-precision defect localization in chip packaging. The proposed detection method is designed for inline full inspection. However, in practical production environments, either full inspection or sampling inspection can be applied depending on specific requirements such as throughput, cost, and quality assurance strategy. Many studies have applied algorithms such as YOLO v4 [19,20], v5 [21], v7 [22], v8 [23], and YOLO X [24] to chip surface or internal defect detection and, combined with improvement strategies, to achieve improved detection accuracy. However, its performance is still limited under the conditions of small samples, cross-modal images, and data distribution offset. Relying solely on infrared or measured image training has the problem of insufficient robustness.

To address the limitations of existing methods, recent research trends have focused on integrating physics-based simulation with deep learning to establish multimodal defect detection frameworks. Finite-element analysis (FEA) provides accurate modeling of heat conduction behavior under defined material properties and boundary conditions, offering a reliable physical foundation for understanding temperature field distributions. Meanwhile, end-to-end deep learning models such as the YOLO series demonstrate superior image recognition capabilities, enabling rapid localization of defect regions in complex images. By combining these two complementary approaches, it is possible to achieve collaborative modeling that bridges physical mechanisms and data-driven recognition, thereby enhancing both the interpretability and robustness of defect detection systems.

This work proposes a multimodal detection framework for semiconductor packaging defects that integrates finite-element thermal simulation, infrared thermal imaging, and the YOLO11 deep learning detection network. The main contributions of this work are summarized as follows:

(1)

A comprehensive finite-element model of a typical BGA package is constructed, incorporating representative defects such as missing solder balls and bridging. Both steady-state and transient thermal responses are analyzed to characterize defect-induced temperature anomalies;

(2)

An experimental infrared thermal imaging system is developed to capture real thermal images of defective packages. These measurements are compared with simulation results to validate the physical consistency of the thermal model;

(3)

A hybrid dataset, combining simulated and experimental thermal images, is constructed to train and evaluate the YOLO11 network. This integration significantly enhances the model’s detection accuracy and generalization capability across simulated and real-world scenarios.

2. Proposed Simulation and Experimental Methods

Here, we present a detailed description of the finite-element thermal simulation process for chip packaging defects, the YOLO11-based target detection architecture, and the infrared thermal image acquisition platform. The overall technical workflow is illustrated in Figure 1. First, temperature distribution images of packaged chips under various operating conditions are generated through finite-element simulations and infrared thermal imaging experiments, resulting in two types of datasets: simulated images and experimental images. These datasets are then merged and expanded through data augmentation techniques, including random rotation, occlusion, noise injection, and flipping, to enhance model robustness and generalization. The augmented dataset is subsequently fed into the YOLO11 detection model, which comprises three primary components: a backbone network for feature extraction, a neck network for multi-scale feature fusion, and a detection head for defect localization and classification. The model outputs include detected defect regions, defect types, and corresponding confidence scores. By combining physics-based simulation with data-driven learning, this method leverages both physical interpretability and high-precision detection capability, providing a comprehensive solution for automated chip packaging defect identification.

2.1. Finite-Element Modeling of Packaging Structure

A three-dimensional finite-element model of a ball grid array (BGA) package was developed, consisting of the package substrate, solder pads, solder balls, and printed circuit board (PCB), as illustrated in Figure 2a.

The package substrate is the core carrier of the chip package, which is mainly used to realize the electrical interconnection between the die and the external circuit, mechanical support, and heat dissipation functions. In this simulation, we use a 5 mm × 5 mm × 0.8 mm rectangular block to construct the outer size of the package substrate. Compared with the size of the package substrate, the size of the die is small and can be ignored in terms of geometric scale.

Solder pads, represented as cylindrical copper structures with a diameter of 0.5 mm and thickness of 0.1 mm, were arranged in a 4 × 4 grid on the bottom surface of the substrate with a 1 mm pitch. These pads serve as electrical and mechanical interfaces for solder ball connections. The solder balls were modeled as bulging spheres with a diameter of 0.56 mm and thickness of 0.4 mm, simulating their deformed shape after the reflow process. The PCB, modeled as a 10 mm × 10 mm × 1.6 mm FR-4 block, supports the package and facilitates system-level interconnection. These geometric parameters are consistent with typical specifications used in BGA package designs. Additionally, the same structural configuration was adopted in the experimental setup to ensure comparability between the simulation results and infrared thermal measurements.

An adaptive meshing strategy was employed to refine the mesh in regions with high thermal gradients (Figure 2b), ensuring accurate capture of localized temperature variations. Material properties, including density, thermal conductivity, and specific heat capacity, were defined based on Comsol’s material library and are summarized in

Table 1. Material properties used in the finite-element thermal simulation model.

Component	Material	Density (kg/m3)	Thermal Conductivity (W/(m·K))	Heat Capacity (J/(kg · K))
Substrate	FR-4	1850	0.3	1200
Pad	Cu	8960	kpad 1	385
Solder ball	SAC305	7350	ksolder ball 2	210
PCB	FR-4	1850	0.3	1200

¹ k_pad = 469.9464 + 0.1761 × T; ² k_{solder ball} = 70.52 − 0.007105 × T [25]; T refers to temperature (K).

. Considering that the packaging substrate is the main source of heat, the boundary conditions are set as follows:

(1)

Heat source conditions: A constant temperature of 100 °C or 200 °C was applied to the entire top surface of the package substrate. This setting aims to emulate a uniform thermal load environment, such as that generated during accelerated aging or worst-case thermal testing, and facilitates clear differentiation of defect-induced temperature variations;

(2)

Heat dissipation conditions: The initial temperature of all external boundaries is room temperature 20 °C, and the convection heat transfer coefficient between the external 20 °C air is 5 W/(m²·K). And the additional contact thermal resistance between the package substrate, pads, solder balls, and printed circuit boards is not considered. Finally, the thermal radiation phenomenon of the packaged chip is ignored.

2.2. Experimental Setup for Infrared Thermal Imaging

An experimental platform was constructed to validate the simulation and detection methods (Figure 3). Defective BGA samples were prepared on FR-4 PCBs using a 4 × 4 solder ball array, with missing and bridged solder balls manually introduced. Thermal images were captured using a FLUKE TiS55 infrared imager (Everett, WA, USA) and a VECTECH V-1515 heater (Xiamen, China). Imaging parameters—including emissivity, ambient temperature, and transmittance—were calibrated using SmartView Classic 4.4. To ensure consistency with the simulation, the experimental setup adopted similar thermal boundary conditions: a uniform top-surface heating at 100 °C or 200 °C and ambient air maintained at ~20 °C to match the convective cooling conditions. This alignment enables reliable comparison between experimental and simulated thermal results.

2.3. Dataset Construction and Deep Learning Framework

In order to improve the generalization ability and robustness of the model in defect detection tasks, this study constructed a dataset that integrates finite-element simulation images and infrared thermal imaging images and uses the YOLO11 deep neural network as the core detection framework. Data construction includes simulation image annotation, format conversion, image augmentation, and other preprocessing steps to ensure that defect features are fully expressed.

The simulated images represent the steady-state temperature distributions on the bottom surface of the package substrate under various defect conditions. These images were extracted from finite-element analysis results and serve as the basis for constructing the simulation dataset.

To enhance the diversity and generalization capability of the dataset, a series of data augmentation techniques were applied to both simulated and experimental images. These include geometric transformations such as rotation and flipping as well as thermal noise injection. Specifically, a probabilistic augmentation strategy was employed, involving random rotation (30% probability, with a maximum angle of ±45°), random flipping (50% probability), and random cropping (40% probability, preserving at least 60% of the original frame). In addition, Gaussian noise injection (20% probability) and random occlusion (30% probability, with 2–20% of the area covered by randomly colored blocks) were introduced to simulate real-world image variability. A typical example of the augmented images is shown in Figure 4.

The original simulation samples are 68 images of missing solder balls, missing solder ball and pad, and bridge and mixed defects, which were expanded to 768 images after augmentation and divided into training, validation, and test sets (8:1:1). The model labels include four categories: missing solder balls, missing solder ball and pad, and bridge, corresponding to category indexes 0 to 2, respectively.

The YOLO11 network structure is divided into three parts: the backbone network, the neck network, and the detection head [26]. The backbone adopts the improved structure of CSPDarknet-Deep and introduces the C2PSA attention mechanism and SPPF module to improve the feature extraction capability. The neck adopts a bidirectional weighted feature fusion module to enhance the multi-scale information integration. Finally, the detection head supports three-scale output and has a good effect on small target defect positioning.

2.4. Model Training and Evaluation Metrics

The YOLO11 model was trained using the PyTorch framework (version 2.7.1) on an NVIDIA RTX 3090 GPU for 200 epochs, with an initial learning rate of 0.01 and a batch size of 128. The performance was evaluated using the following metrics: (1) precision, recall, and mAP (mean average precision) at IoU thresholds of 0.5 (mAP50) and 0.5–0.95 (mAP50-95); (2) FPS (frames per second) to assess real-time detection capability; and (3) params (number of model parameters) and FLOPs (floating-point operations per second) to evaluate model complexity. These metrics provide a comprehensive assessment of the model’s accuracy, speed, and computational efficiency, supporting its application in high-speed industrial defect detection.

3. Results and Discussion

3.1. Thermal Simulation Analysis of Packaging Defects

The impact of different packaging defects on the thermal behavior of BGA structures was systematically analyzed through steady-state and transient simulations. Four scenarios were considered: normal packaging, missing solder balls, missing solder ball and pad, and bridging. The heat dissipation pathways under these conditions are schematically illustrated in Figure 5. Under normal conditions, the heat generated by the package substrate can be evenly transferred through the conduction path of substrate–solder pad–solder ball–PCB. However, in the case of a missing solder ball, the heat cannot be conducted through the above path but only conducts to the pad or conducts through the normal solder balls around the missing ball and finally conducts heat laterally on the PCB. When both the solder ball and its corresponding pad are missing, the heat conduction mode is similar to that of missing solder ball. The heat on the package substrate is conducted around the normal solder balls, and it is also conducted laterally on the PCB. In the case of bridging, where two or more solder balls are unintentionally connected, an additional lateral heat conduction path is introduced. This alters the local temperature distribution by enabling heat to flow not only through the normal vertical path but also across the bridged solder balls, potentially impacting the system’s overall thermal performance.

Under steady-state thermal conditions, with a constant temperature of 200 °C applied to the top surface of the FR-4 package substrate, the resulting temperature distribution on the top surface of the PCB is illustrated in Figure 6. Under normal conditions, as shown in Figure 6a, the temperature field exhibits a symmetrical diffusion pattern, with heat transferred from the highest-temperature region at the center of the chip to the PCB through the solder ball array. The temperatures of the four central solder balls are significantly higher than those of the twelve peripheral solder balls, indicating that the peripheral solder balls dissipate heat more efficiently due to enhanced thermal convection. In contrast, the slower heat dissipation of the central solder balls leads to heat accumulation. Meanwhile, the four corners of the square PCB represent the lowest temperature zones, forming cold regions as both adjacent sides facilitate heat dissipation via convection. In the absence of solder balls, as illustrated in Figure 6b, the original symmetrical temperature distribution is disrupted. A cold spot with a temperature below 170 °C emerges in the upper-right corner of the solder ball array, and the temperature gradient of the adjacent solder balls increases. This suggests that heat is redirected around the missing region toward neighboring solder balls, which is consistent with the thermal resistance network analysis. At this point, the four central solder balls still maintain relatively high temperatures, and the cold zones at the PCB corners remain prominent. When the pad is further removed (Figure 6c), the resulting temperature distribution remains highly similar to the case involving only missing solder balls. This implies that a pad not in direct contact with the PCB has a limited effect on the overall temperature field. Under solder ball bridging conditions (Figure 6d), the temperature field largely reverts to a nearly symmetrical diffusion pattern, with no distinct cold spots observed. However, in the lower-left bridging region, the temperatures of the two connected solder balls tend to equalize due to lateral heat conduction between them. Additionally, the temperature in the bridged area is higher than that observed in the original air gap, thereby confirming that bridging may induce abnormal hot spots.

In order to evaluate whether pad loss affects the temperature distribution, the temperature field of the package substrate is further analyzed. By integrating the heat dissipation path analysis with the substrate simulation results, several conditions are compared. Under normal conditions (Figure 7a), the temperature field exhibits a symmetrical diffusion pattern. When solder balls are missing (Figure 7b), a distinct hot spot with a temperature of 197 °C appears at the defect location in the upper-right corner, and a clear circular boundary corresponding to the pad is observed.

Under the condition where both solder balls and solder pads are missing (Figure 7c), a similar hot spot emerges at the upper-right defect, with a slightly higher temperature of 197.22 °C. However, in this case, no clear circular boundary is observed, suggesting that the absence of the pad reduces thermal coupling in that region. In the solder ball bridging condition (Figure 7d), the temperature at the lower-left bridging site is notably lower than that of the surrounding solder balls, with a recorded temperature of 185.36 °C. This phenomenon corresponds to an abnormal cold spot, indicating that bridging may lead to localized thermal anomalies distinct from those caused by missing solder elements.

Further transient analysis reveals that different types of defects have markedly different effects on the heating rate and temperature saturation time. As illustrated in Figure 8, a comparison between the normal package and the solder ball missing condition shows that the heating rate at the center point of the solder ball during the first 2 s reaches 41.9 °C/s under the missing condition, which is 47% higher than the 28.5 °C/s observed under normal conditions. Moreover, temperature saturation is achieved in just 1.4 s, significantly faster than the 37.5 s required under normal conditions, indicating a pronounced local heat load concentration effect.

In contrast, under the solder ball bridging condition, the heating rate at the center of the bridged region during the first 2 s is 32.3 °C/s, representing a 13% increase compared to the normal case. The corresponding temperature saturation time is 17.0 s—faster than the normal condition but still slower than that observed in the solder ball missing condition. This suggests that although the lateral heat conduction path introduced by bridging accelerates the initial heat diffusion, the added thermal mass of the solder reduces the efficiency of longitudinal heat transfer.

Notably, the heating rate and temperature saturation time in the solder ball missing condition and the combined solder ball and pad missing condition are almost completely overlapping. This is because both defects disrupt the vertical heat conduction path, forcing heat to diffuse laterally through the substrate, which has relatively low thermal conductivity. As a result, the heating rate is significantly reduced, and the thermal responses tend to converge.

In terms of heating rate, the solder ball missing condition exhibits the fastest rate at 41.9 °C/s, followed by the solder ball bridging condition at 32.3 °C/s, while the normal packaging condition is the slowest at 28.5 °C/s. When evaluated based on temperature saturation time, the solder ball missing condition again shows the fastest response, reaching saturation in just 1.4 s, followed by the bridging condition at 17.0 s, with the normal condition being the slowest at 37.5 s.

These differences among the three cases are fundamentally attributed to variations in heat transfer paths. Specifically, solder ball loss results in a rapid temperature rise due to circumferential heat flow around the defect. Solder ball bridging, on the other hand, partially compensates for heat transfer efficiency through an alternative lateral conduction channel. Meanwhile, the effect of solder pad absence is essentially masked by the low thermal conductivity of the substrate, as the missing pad does not contribute to effective thermal coupling.

To complement the simulation analysis and further validate the physical observations, representative experimental results are presented prior to neural network-based detection evaluation. Figure 9 shows real infrared thermal images and corresponding optical images of the defective BGA samples introduced in Section 2.2. These images capture the surface temperature distribution and physical structure under different defect conditions, including bridge and missing solder ball.

Figure 9a presents the temperature distribution of a bridged solder ball. The normal solder ball appears in blue, indicating a relatively low temperature at its center. In contrast, the bridged solder ball exhibits an elongated high-temperature region, with the central area showing lower temperature than the surroundings, reflecting lateral heat conduction between bridged solder balls.

Figure 9c shows the infrared thermal response of a missing solder ball. A red hotspot appears at the missing center, indicating elevated temperature due to disrupted lateral heat conduction. The temperature distribution within individual solder balls is visibly uneven, with a gradual cooling trend from the periphery toward the center. Moreover, the isothermal regions vary among different solder balls, highlighting the influence of defect-induced heat flow distortion.

Figure 9b,d display optical images of the actual solder ball structures corresponding to the thermal images. The normal solder balls appear silver, smooth, and uniformly spherical, with consistent size and surface quality. In contrast, at the location without solder ball, the exposed copper pad is clearly visible.

These experimental observations further support the accuracy of the thermal simulation and provide a real-data foundation for training and evaluating the neural network in the next section.

3.2. Performance Evaluation of Defect Detection

In order to evaluate the effect of training in defect detection, the YOLO11 model was trained on two sets of data: using only simulated images and hybrid data of simulated and infrared images and performing defect detection in simulated images and real infrared images, respectively. Before evaluating the results, the following evaluation indicators need to be briefly explained. Precision and recall are important indicator parameters, which are calculated from TP, FP, and FN, as shown in Equations (1) and (2):

$$\mathrm{Precison}={\displaystyle \frac{TP}{TP+FP}}$$

(1)

$$\mathrm{Recall}={\displaystyle \frac{TP}{TP+FN}}$$

(2)

where TP refers to the model correctly detecting an object, FP refers to the model incorrectly detecting an object (including incorrect category or no object), and FN refers to the model not detecting an actual object. Precision indicates how many of the detected objects are correct. The closer its value is to 1, the more accurate the detection. And recall indicates how many of all the real objects the model has found. The closer its value is to 1, the more it has detected.

Under ideal conditions, both precision and recall need to be high. Therefore, with precision as the vertical axis and recall as the horizontal axis, multiple (precision and recall) points are plotted through different confidence thresholds to form a PR curve. The area enclosed by the curve and the coordinate axis is AP (average precision). mAP is the average of APs of multiple categories. Its calculation is shown in Equations (3) and (4):

$$AP={\displaystyle \underset{0}{\overset{1}{\int }}p(r)dr}$$

(3)

$$mAP={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^{N}A{P}_i}$$

(4)

The PR curve in Equation (3) is $p=p(r)$: N is the number of classification categories in Equation (4), and the subscript $i$ is the $i$th category.

In order to refine the accuracy reference indicators in different fields, two indicators, mAP50 and mAP50-95, are introduced here. mAP50 is $IoU\ge 0.5$ to calculate the mAP of all categories when. At this time, the evaluation accuracy is relatively loose, so mAP50 is suitable for the accuracy evaluation of lightweight models or rapid deployment. mAP50-95 calculates the mAP for every 0.05 interval from IoU from 0.5 to 0.95 and then takes the average. Therefore, mAP50-95 can better reflect the actual detection quality of the model than mAP50 and is suitable for more rigorous accuracy evaluation. The numbers contained in the two indicators refer to the intersection over union (IoU), which is defined by Equation (5):

$$IoU={\displaystyle \frac{\left|A\cap B\right|}{\left|A\cup B\right|}}$$

(5)

where the set $A$ refers to the set of predicted boxes, the set $B$ is the set of real boxes, the numerator refers to the overlapping area of the predicted box and the real box, and the denominator refers to the total coverage area of the two boxes. The ratio ranges from [0, 1].

In addition, in the object detection network, the frame rate (frames per second, FPS) is often used to measure the inference speed of the model. In the actual training process, the calculation of FPS is shown in Equation (6):

$$FPS={\displaystyle \frac{1}{t_{pre}+t_i+t_{post}}}$$

(6)

where $t_{pre}$ refers to the image pre-processing time, $t_i$ refers to the image inference time, and $t_{post}$ refers to the image postprocessing time. The larger the FPS value, the faster the model detection speed.

As for model complexity, commonly used indicators are model parameter quantity (params) and computation quantity (floating-point operations per second, FLOPs), which correspond to spatial complexity and time complexity respectively. Params refers to the total number of parameters that need to be trained in the network model, while FLOPs refers to the number of floating-point operations per second. The smaller the values of the two, the more streamlined the model.

Figure 10 presents representative examples of detection results for different defect types in this test, while

Table 2. Performance comparison of different YOLO-based models on the simulation dataset, including detection accuracy (mAP at IoU thresholds of 0.5 and 0.5–0.95), inference speed (frames per second, FPS), model size (number of parameters, params), and computational cost (floating-point operations, FLOPs).

Model	mAP50	mAP50-95	FPS	Params (M)	FLOPs (G)
YOLOv8 [27]	99.5%	94.5%	667	3.01	8.2
YOLOv9t [28]	99.5%	93.5%	387	2.01	7.9
YOLOv10n [29]	99.3%	92.9%	667	2.71	8.4
YOLO11n [26]	99.5%	93.0%	556	2.59	6.4

summarizes the key performance metrics of various models on the simulated dataset. In the simulated thermal image shown in Figure 10a, three types of defects—missing solder ball, missing solder ball and pad, and bridge—are accurately detected by the model, with corresponding confidence scores of 0.91, 0.91, and 0.94, respectively. In the experimental infrared image shown in Figure 10b, the detected defects include a missing solder ball and a bridge, with confidence scores of 0.83 and 0.85. From the perspective of detection accuracy, YOLOv8 demonstrates the best overall performance, achieving an mAP50-95 of 94.5%, slightly surpassing other models. This indicates greater robustness across varying IoU thresholds. All models exhibit excellent performance in terms of mAP50, with values exceeding 99.3%. Among them, YOLOv8, YOLOv9t, and YOLO11n each reach 99.5%, confirming the reliability of the YOLO series in high-confidence detection tasks. YOLOv10n shows a slightly lower mAP50, which may be attributed to reduced feature representation capability due to its lightweight architecture.

In terms of inference speed, both YOLOv8 and YOLOv10n achieve 667 FPS, significantly outperforming YOLO11n and YOLOv9t. Notably, YOLOv10n maintains the same inference speed as YOLOv8 while reducing the number of parameters by 10%, showcasing the advantages of a lightweight design. YOLOv9t has the lowest FPS among the models. Although it delivers high accuracy, the trade-off in speed makes it more suitable for high-precision but non-real-time applications.

Regarding model complexity, YOLOv9t has the smallest number of parameters, yet its FLOPs are comparable to those of YOLOv10n, suggesting a compact structure but suboptimal computational efficiency. In contrast, YOLO11n exhibits the lowest FLOPs—22% lower than those of YOLOv8—and a parameter count similar to YOLOv10n, reflecting an efficient architectural design for computational performance. Although YOLOv8 does not achieve the lowest parameter count or FLOPs, it provides the best trade-off between accuracy and speed.

For YOLO11n, the mAP50 is on par with YOLOv8 and YOLOv9t, and it surpasses YOLOv10n. Its FLOPs are the lowest among the compared models, ranging from 18% to 24% lower, which makes it well-suited for low-power environments. It also achieves a high frame rate of 556 FPS, demonstrating strong inference speed. The lightweight design also facilitates deployment on edge devices. Overall, YOLO11n presents a viable solution for intelligent industrial defect detection by offering high accuracy, computational efficiency, and strong real-time performance.

In the hybrid dataset, the training effect of the YOLO11 model is shown in Figure 11. The calculations of L_box and L_cls are shown in Equations (7) and (8):

$$L_{box}=1-IoU(b,b_p)$$

(7)

$$L_{cls}={\displaystyle \frac{1}{N_b}}{\displaystyle \sum _{i=1}^{N_b}-\log ({\widehat{y}}_i,t_i)}$$

(8)

where $b$ refers to ground truth box, $b_p$ refers to predicted box, $N_b$ is the number of samples in batch, and ${\widehat{y}}_i,t_i$ is the probability of the real label category predicted by the $i$th sample.

In the figure, L_box represents the bounding box loss, that is, the loss between the predicted box and the marked box. As the training progresses, L_box gradually decreases, with an obvious convergence trend, indicating that the training has been able to locate the defect. L_cls represents the classification loss, which is used to measure the difference between the predicted category and the true category. There are fewer classification categories here, L_cls converges quickly, and the category prediction ability training effect is better. L_dfl represents the distribution focal loss [30], which is used to assist L_box and improve recognition accuracy. It shows a convergence trend similar to the first two curves, indicating that the accuracy of model training has steadily improved.

The final prediction results are shown in

Table 3. Comparison of defect detection performance on real infrared images using models trained with different datasets, showing the number of successfully identified defects and corresponding recognition rates.

Model Dataset	Number of Test Images	Recognized Defects	Unrecognized Defects	Recognition Rate
Simulation Dataset	12	5	7	41.7%
Hybrid Datasets	12	11	1	91.7%

. After training with the hybrid dataset, the target detection effect is greatly improved. Of the same 12 images, 11 images can be successfully identified, with a recognition rate of 91.7%. Compared with the 41.7% recognition rate of the model trained only with the simulation dataset, the recognition rate increased by 120%. At the same time, the images that failed to be successfully identified were checked, and the results showed that a solder ball missing defect was successfully marked, but the confidence was only 0.27, so it was ignored. In summary, in terms of real target detection, real datasets need to be added to further optimize detection capabilities and realize industrial applications.

4. Conclusions

This paper presents an integrated defect detection framework for semiconductor packaging, combining finite-element thermal simulation, infrared thermal imaging, and YOLO11-based deep learning. By modeling typical defects—including missing solder balls, missing pads, and solder ball bridging—the study systematically analyzed their influence on steady-state and transient thermal distributions. The simulation results demonstrated strong consistency with experimental infrared measurements, validating the physical accuracy of the simulation model.

A hybrid dataset composed of simulated and experimentally captured thermal images was constructed to train the YOLO11 network. The proposed model achieved a high detection accuracy of 99.5% mAP50 on the simulation dataset and maintained a real-time inference speed of 556 FPS. More importantly, integrating real thermal images into the training process improved the model’s defect recognition rate on experimental data from 41.7% to 91.7%, effectively addressing domain adaptation challenges. These results confirm that the proposed multimodal framework offers a practical, high-precision, and real-time solution for industrial semiconductor packaging defect detection.

Although the hybrid dataset significantly improves the model performance, it still has the following limitations: First, the number of experimental image samples is limited, and the defect types are unevenly distributed, which may affect the model’s ability to distinguish between multiple types of targets. Second, the resolution of infrared imaging is limited, and the boundaries of some defects are blurred, affecting the accuracy of target positioning. In future work, efforts will focus on enlarging the experimental dataset, integrating advanced imaging technologies, refining the model for edge deployment, and extending the method to support a wider range of packaging structures and defect types. These advancements aim to promote the development of intelligent, automated, and scalable defect detection solutions for semiconductor manufacturing.