A Confidence Calibration Based Ensemble Method for Oriented Electrical Equipment Detection in Thermal Images

Abstract

Detecting oriented electrical equipment plays a fundamental role in enabling intelligent defect diagnosis in power systems. However, existing oriented object detection methods each have their own limitations, making it challenging to achieve robust and accurate detection under varying conditions. This work proposes a model ensemble approach that leverages the complementary strengths of two representative detectors—Oriented R-CNN and S²A-Net—to enhance detection performance. Recognizing that discrepancies in confidence score distributions may negatively impact ensemble results, this work first designs a calibration method to align the confidence levels of predictions from each model. Following calibration, a soft non-maximum suppression (Soft-NMS) strategy is employed to fuse the outputs, effectively refining the final detections by jointly considering spatial overlap and the calibrated confidence scores. The proposed method is evaluated on an infrared image dataset for electric power equipment detection. Experimental results demonstrate that our approach not only improves the performance of each individual model by 1.95 mean Average Precision (mAP) but also outperforms other state-of-the-art methods.

1. Introduction

Equipment inspection plays a crucial role in ensuring the safety, reliability, and efficiency of electrical systems. Through regular inspections, potential hazards, such as physical damage and insulation degradation, can be promptly identified and addressed, thereby reducing the risk of equipment failure, minimizing unexpected downtime, and preventing potential safety incidents. In modern electrical equipment inspection, thermal imaging cameras have been widely used due to their non-contact nature, which significantly enhance inspector safety while maintaining operational continuity. These infrared cameras capture detailed thermal images that reveal precise temperature distribution patterns across electrical components, enabling maintenance teams to conduct comprehensive thermal analysis and identify potential anomalies. For accurate equipment status diagnosis, a fundamental step is to detect electrical equipment in thermal images, which serves as the foundation for subsequent thermal pattern analysis and fault diagnosis.

To date, a variety of methods, including segmentation-based approaches [1,2,3] and object detection-based techniques [4,5,6], have been developed for the detection of electrical equipment in thermal images. While segmentation-based methods excel at identifying equipment regions, they are often susceptible to background interference, which can disrupt the precise localization of component centerlines—a crucial aspect for some subsequent diagnostic tasks. On the other hand, object detection methods typically utilize upright bounding boxes to identify equipment components. However, due to the handheld capture manner commonly employed during routine inspections, equipment may appear in various orientations, as shown in Figure 1. As a result, these upright bounding boxes can incorporate excessive background information, introducing noise and potentially compromising the accuracy of subsequent diagnostics. For these reasons, this work opts to use oriented object detection methods to identify power equipment more accurately.

Although numerous oriented object detection methods, including both one-stage [7,8,9,10] and two-stage approaches [11,12,13,14], have been developed, their performance in detecting electrical equipment within complex environments remains suboptimal. As illustrated in Figure 2, challenges such as variations in component sizes and background interference frequently lead to issues like missed detections and false positives. The localizations of oriented bounding boxes also differ across methods. These limitations significantly hinder the practical effectiveness of these methods in real-world electrical equipment detection applications.

Motivated by the observation that different oriented object detection methods exhibit unique strengths and complementary characteristics, this work proposes an ensemble approach that integrates two representative detectors: a two-stage method, Oriented R-CNN [15], which excels at producing detections with accurate localization, and a one-stage method, S²A-Net [8], which demonstrates strong performance in detecting dense and small objects with higher precision. However, directly ensembling the detection results from these two models leads to degraded performance. This is primarily due to inconsistencies in the confidence scores of the predictions, as illustrated in Figure 3. These inconsistencies may arise from differences in model architectures, training objectives, and output distributions. To address this issue, our method employs a confidence calibration method [16,17] to align the predicted confidence levels with their true likelihoods. Once calibrated, it ensembles the results using a post-processing strategy based on soft non-maximum suppression, which further refines the final detections by considering the overlap and confidence of bounding boxes in a more nuanced manner. The effectiveness of our proposed ensemble framework has been validated on a dataset consisting of infrared images for electric power equipment detection, demonstrating considerable improvements over individual models in terms of both precision and recall.

2. Related Works

2.1. Electrical Equipment Detection

Electrical equipment detection is a fundamental step towards the intelligent diagnosis of power system conditions. Consequently, various deep learning-based techniques have been employed for equipment detection. Existing methods can be broadly categorized into segmentation-based and object-detection-based approaches. For instance, Yan et al. [1] and Meng et al. [2] adopted Mask R-CNN [18] for segmenting power equipment, while Tang et al. [3] utilized DeeplabV3+ to segment equipment structures susceptible to thermal faults. Although segmentation-based methods excel at detecting equipment regions, they are often sensitive to background interference, particularly when pinpointing the centerline of equipment—a key element for certain diagnostic tasks.

In object-detection-based methods, Han et al. [4] utilized MobileNet for feature extraction and devised a region of interest (ROI) selection method to identify equipment. Zhang et al. [5] employed Faster R-CNN for electrical equipment detection, while Qi et al. [6] implemented a lightweight model of the Single Shot Multibox Detector (SSD) [19] for real-time detection of electrical apparatus. Most of these methods detect equipment using axis-aligned bounding boxes, which often encompass excessive background regions, potentially interfering with subsequent anomaly diagnostics. An exception is the work by Gong et al. [20], who developed an oriented YOLO [21] model to detect slanted electrical equipment. Our study similarly focuses oriented equipment detection.

2.2. Oriented Object Detection

In many real-world scenarios, such as remote sensing [12] and power inspection [20], objects of interest often appear in tilted or skewed orientations due to the shooting angle. Thus, object detection using upright bounding boxes may include excessive noisy background, making oriented object detection, which can encompass objects more tightly, a topic of significant interest. So far, various deep learning-based methods have been developed to detect oriented objects. For example, a set of methods, including RRPN [11], RoI Transformer [12], ReDet [13], Gliding Vertex [14] and Oriented R-CNN [15], have constructed their frameworks based on two-stage detection methods like Faster R-CNN, generating either horizontal or oriented region of proposals (RoIs) for oriented bounding box regression. Other methods, like RetinaNet [7], ${\mathrm{S}}^2\mathrm{A}\text{-}\mathrm{Net}$ [8], ${\mathrm{R}}^3\mathrm{Det}$ [9] and RTMDet [10], design their frameworks using one-stage or anchor-free detection methods, directly producing oriented bounding boxes for efficiency. Additionally, some approaches [22] also introduced new losses, such as a Pixels-IoU (PIoU) loss, leveraging both the angle and intersection over Union (IoU), for precise oriented bounding box regression.

Unlike the aforementioned methods that focus on designing new network architectures or loss functions, our work observes that different methods perform variably across different scenarios and may be complementary to each other. This work proposes an ensemble approach of multiple oriented object detection methods to enhance detection performance for electrical equipment detection. Specifically, it ensembles two different models, including Oriented R-CNN [15] and ${\mathrm{S}}^2\mathrm{A}\text{-}\mathrm{Net}$ [8], which were selected based on their complementary strengths in handling different aspects of oriented object detection. The Oriented R-CNN demonstrates superior performance in generating high-quality oriented proposals, while ${\mathrm{S}}^2\mathrm{A}\text{-}\mathrm{Net}$ shows strong performance in detecting densely packed or small objects. Their complementary capabilities enable our work to achieve high oriented object detection performance. Notably, our ensemble method can be applied to any number of other models as well, offering flexibility for various detection scenarios and computational requirements.

2.3. Ensemble Learning

Ensemble learning, a methodology that combines multiple base models to create a more powerful model, has been incorporated into deep learning [23,24] to improve performance and reduce the risk of overfitting. Ensemble deep learning techniques vary in the choice of homogeneous or heterogeneous base models and in the fusion of decisions through voting, meta-learning, or other strategies. In conventional object detection where bounding boxes are upright, Terzi [25] designed four ensemble strategies to combine nine object detectors like YOLOv3, Faster R-CNN, and RetinaNet, boosting the performance of anatomical and pathological object detection. Although various ensemble strategies have been designed, they basically average the confidence scores of the intersecting bounding boxes. Casado-García and Heras [26] combined affirmative, consensus, and unanimous strategies to fuse the detected bounding boxes obtained by different object detection methods.

However, in the context of upright object detection, Oksuz et al. [16,17] observed that directly ensembling the outputs of multiple detection models can result in performance degradation. This issue arises due to inconsistencies in confidence distributions, which are caused by variations in network architectures, the use of different classifiers, or discrepancies in localization heads. To address this, they proposed a confidence calibration method prior to ensembling and demonstrated improved performance by integrating the calibrated results. Although various ensemble methods have been proposed for upright (axis-aligned) object detection, there is still a lack of approaches specifically tailored for oriented object detection. Our work extends the concept of confidence calibration to the oriented object detection setting, aiming to enhance the performance of ensemble models by better integrating predictions from different OOD detectors.

3. The Proposed Method

This work proposes a confidence calibration-based ensemble method to fuse the detection results from two representative oriented object detection models: Oriented R-CNN [15] and S²A-Net [8]. An overview of our method is illustrated in Figure 4. Given an infrared image of electrical equipment, our approach first uses Oriented R-CNN and S²A-Net to detect oriented object instances independently. Next, a confidence calibration module is applied to adjust the predicted confidence scores, aligning them more closely with the actual likelihood of correctness. After calibration, our method ensembles the results using a soft non-maximum suppression (Soft-NMS) strategy, which refines the final detections by considering both the spatial overlap and the calibrated confidence scores of the bounding boxes.

Formally, an image I, along with two sets of oriented bounding boxes, $\{{\widehat{c}}_i^1,{\widehat{ob}}_i^1,{\widehat{p}}_i^1\}$ and $\{{\widehat{c}}_j^2,{\widehat{ob}}_j^2,{\widehat{p}}_j^2\}$, are detected by Oriented R-CNN and S²A-Net, respectively. Here, $\widehat{c}$ is the category label, $\widehat{ob}$ denotes the oriented bounding box, and $\widehat{p}$ is the confidence score. Our method first calibrates their confidence distributions. Then, the calibrated bounding boxes are fused using a soft non-maximum suppression (soft-NMS) strategy to produce the final ensembled results.

3.1. The Base-Oriented Object Detection Models

3.1.1. Oriented R-CNN

Oriented R-CNN [15] is a two-stage framework designed for oriented object detection. Built on the widely used Faster R-CNN architecture with a feature pyramid network (FPN) backbone, Oriented R-CNN introduces an oriented region proposal network (Oriented RPN) to generate high-quality oriented proposals directly, eliminating the need for complex intermediate steps. It also employs an oriented region of interest (RoI) alignment module to extract rotation-invariant features, together with an oriented R-CNN head for proposal classification and regression, as illustrated in Figure 5. By integrating the designed components, Oriented R-CNN achieves a high accuracy and efficiency in oriented object detection, outperforming previous approaches while maintaining a streamlined, end-to-end trainable pipeline.

For training, Oriented R-CNN adopts the cross-entropy loss for classification and Smooth L1 loss for bounding box regression. Each bounding box is parameterized by six values: $(x,y,w,h,\Delta \alpha ,\Delta \beta )$, where $(x,y)$ denote the centroid, $(w,h)$ represent the width and height of the horizontal bounding box, and $(\Delta \alpha ,\Delta \beta )$ correspond to the midpoint offsets.

3.1.2. S²A-Net

The single-shot alignment network (S²A-Net) [8] is an one-stage oriented object detection method built upon the RetinaNet framework. To better handle objects of varying sizes, the standard FPN backbone is replaced with the LKSNet backbone [27], which adaptively selects convolutional kernels that are better suited for detecting large or small objects. S²A-Net also introduces a feature alignment module (FAM), which refines anchor boxes and aligns features through alignment convolution. Additionally, it incorporates an oriented detection module (ODM) that utilizes rotating filters to effectively encode orientation information, as illustrated in Figure 6.

For training, S²A-Net employs the Focal Loss for classification and the Smooth L1 Loss for regression. Each oriented bounding box is parameterized by five values: $(x,y,w,h,\theta )$, where $(x,y)$ denotes the center coordinates, $(w,h)$ represent the width and height, and $\theta$ is the rotation angle.

3.2. Confidence Calibration

Due to differences in network architectures, including varying backbones, one-stage versus two-stage frameworks, distinct bounding box representations, and loss functions, Oriented R-CNN and S²A-Net exhibit significant discrepancies in the confidence scores of their predicted detections, as illustrated in Figure 3. As highlighted by MoCAE [17], such inconsistencies may lead to performance degradation when performing a model ensemble. Therefore, it is essential to calibrate the confidence scores to ensure they accurately reflect the true quality of the predictions.

3.2.1. Calibration Error

The first step toward calibrating an oriented object detector is to define a suitable calibration error that quantifies the misalignment between prediction confidence and actual performance. Our work extends the localization-aware calibration error (LaECE) [16], originally proposed for upright object detection, to the oriented detection setting.

Let an oriented object detector be represented as $f:X\mapsto {\left\{{\widehat{c}}_i,{\widehat{ob}}_i,{\widehat{p}}_i\right\}}^N$, where ${\widehat{c}}_i$, ${\widehat{ob}}_i$, and ${\widehat{p}}_i$ denote the predicted class, oriented bounding box, and confidence score of the i-th prediction, respectively, and N is the number of predictions. Then, the extended calibration error is defined as follows:

$$LaECE={\displaystyle \frac{1}{C}}\sum _{c=1}^C\sum _{j=1}^J{\displaystyle \frac{\left|{\widehat{\mathcal{D}}}_j^c\right|}{\left|{\widehat{\mathcal{D}}}^c\right|}}\left|{\overline{p}}_j^c-{precision}^c\left(j\right)\times {\overline{IoU}}^c\left(j\right)\right|,$$

(1)

where C is the number of classes, J is the number of confidence bins, ${\widehat{\mathcal{D}}}_j^c$ denotes the set of predictions for class c falling into bin j, ${\overline{p}}_j^c$ is the average predicted confidence in bin j for class c, ${precision}^c\left(j\right)$ is the precision of the j-th bin for c-th class, and ${\overline{IoU}}^c\left(j\right)$ is the mean Intersection over Union (IoU) for the predictions in that bin. This formulation jointly accounts for both classification confidence and localization quality, making it well-suited for evaluating calibration in oriented object detection tasks. A lower LaECE indicates that the model’s confidence scores are more reliable and better aligned with its true detection performance.

3.2.2. Model Calibration

An oriented object detection model is calibrated if its classification and localization performance jointly align with the predicted confidence ${\widehat{p}}_i$. Formally, this condition can be expressed as:

$$\underset{\mathrm{Classification}\mathrm{perf}.}{\underbrace{\mathbb{P}\left({\widehat{c}}_i=c_i\mid {\widehat{p}}_i\right)}}\underset{\mathrm{Localisation}\mathrm{perf}.}{\underbrace{{\mathbb{E}}_{{\widehat{ob}}_i\in B_i\left({\widehat{p}}_i\right)}\left[IoU\left({\widehat{ob}}_i,o{b}_{\psi \left(i\right)}\right)\right]}}={\widehat{p}}_i,\forall {\widehat{p}}_i\in [0,1]$$

(2)

where $\mathbb{P}\left({\widehat{c}}_i=c_i\mid {\widehat{p}}_i\right)$ denotes the probability that the predicted class ${\widehat{c}}_i$ matches the ground truth class $c_i$ given the confidence score ${\widehat{p}}_i$, and $IoU\left({\widehat{ob}}_i,o{b}_{\psi \left(i\right)}\right)$ measures the overlap between the predicted oriented bounding box ${\widehat{ob}}_i$ and the ground truth box $o{b}_{\psi \left(i\right)}$. The expectation is taken over all predictions falling within the confidence level ${\widehat{p}}_i$, denoted as the bin $B_i\left({\widehat{p}}_i\right)$.

Thus, this work trains a calibrator ${\zeta }_{\theta }:[0,1]\to [0,1]$ using input–target pairs $\left({\widehat{p}}_i,IoU({\widehat{b}}_i,b_{\psi \left(i\right)})\right)$, in which ${\zeta }_{\theta }$ is a simple linear function with only two learnable parameters. Therefore, linear regression is employed for efficient optimization.

3.3. Model Ensemble

After calibrating each oriented object detector, our method aggregates the predictions with the calibrated confidence scores using a soft non-maximum suppression (NMS) strategy [17]. In soft NMS, confidence scores are updated as follows:

$${\widehat{p}}_i=\left\{\begin{array}{cc}{\widehat{p}}_i\hfill & \mathrm{if}IoU\left({\widehat{b}}_i,{\widehat{b}}_{\alpha }\right)<{IoU}_{NMS}\hfill \\ {\widehat{p}}_i\times \left(1-IoU\left({\widehat{b}}_i,{\widehat{b}}_{\alpha }\right)\right)\hfill & \mathrm{else},\hfill \end{array}\right.$$

(3)

where ${IoU}_{NMS}$ is the predefined IoU threshold for the standard NMS. This soft strategy penalizes overlapping boxes rather than discarding them entirely, which helps preserve potentially useful detections and improves overall ensemble performance.

4. Experiments

4.1. Dataset and Evaluation Metric

Due to the proprietary nature of substation data, there is currently no publicly available thermal image dataset specifically tailored for electrical equipment detection. To overcome this limitation and facilitate the evaluation of our ensemble method, this work curated a comprehensive dataset comprising infrared images captured during routine maintenance inspections at various substations in Shandong Province, China. These images were acquired using handheld thermal cameras. Each image in the dataset has been carefully annotated with oriented bounding boxes and labeled according to the corresponding equipment component categories.

The dataset includes four primary categories of power transformation equipment: Current Transformer (CT), Potential Transformer (PT), Surge Arrester (SA), and Circuit Breaker (CB). It also covers six critical component types: insulator, expander, grading ring, flange, interrupter, and coupler. In total, the dataset consists of 4062 thermal images, with 3046 allocated for training and 508 reserved for testing. These images exhibit considerable diversity in terms of appearance, viewing angles, and background contexts, posing significant challenges for object detection. This variability enhances the robustness of the dataset, enabling the comprehensive evaluation of detection algorithms under realistic and complex operational scenarios.

For evaluation metrics, as is common practice [8,12,14,15], this work adopts Average Precision (AP) and mean Average Precision (mAP), which are standard metrics in object detection and oriented object detection. AP is defined as the area under the precision–recall curve for a specific category, reflecting both the precision and recall of the detector. mAP is the mean of AP values across all categories, providing an overall assessment of detection performance. Our experiments follow the common practice of computing AP at an Intersection over Union (IoU) threshold of 0.5, denoted as AP@0.5, unless otherwise specified.

4.2. Experimental Setup

This work ensembles two oriented object detection methods: Oriented R-CNN [15] and S²A-Net [8]. Each model is independently trained on our training set using its default parameter settings. Both detectors are trained for 12 epochs, with Oriented R-CNN employing ResNet-50-FPN [28,29] and S²A-Net utilizing LSKNet [27] as their respective feature extraction backbones. All training is conducted on a single RTX 2080Ti GPU with a batch size of 2. The experiments are implemented and evaluated using the MMRotate framework [30]. For confidence calibration, the number of bins J is set to 25, and in Soft NMS, the ${IoU}_{NMS}$ threshold is set to 0.3.

4.3. The Results of Confidence Calibration

Figure 7 presents the reliability diagrams, illustrating the calibration performance for each class across confidence bins. The performance is measured by the product of precision and IoU. In a well-calibrated model, the predicted confidence should closely match the actual detection accuracy. This relationship is visualized by the proximity of the reliability curve to the diagonal line (perfect calibration). That is, all histogram bars align along the diagonal, indicating ideal confidence-to-accuracy correspondence. From the figure, it is observed that calibration significantly improves the alignment of predictions, bringing the histograms closer to the diagonal dash line. Specifically, the LaECE for Oriented R-CNN decreases from 35.34% before calibration to 11.86% after calibration, while for S²A-Net, it drops from 13.65% to 4.43%. This demonstrates the effectiveness of the calibration process in reducing prediction uncertainty and enhancing reliability.

4.4. Quantitative Evaluation

Table 1. Quantitative comparison results.

Model	mAP	AP for 6 Classes in EPED Dataset
		Insulator	Expander	Grading-Ring	Flange	Interrupter	Coupler
RoI Transformer	89.35	90.48	90.78	88.94	89.16	98.75	77.98
Gliding Vertex	88.97	89.85	89.56	89.11	88.41	97.39	79.48
YOLOv11OOD	91.76	90.25	90.56	92.21	89.41	98.30	89.82
Oriented RCNN	89.45	90.26	90.38	88.67	89.25	98.77	79.39
S2A-Net	90.95	89.70	89.75	90.35	89.18	98.04	88.70
Ensemble	90.87	90.03	90.35	95.29	89.21	98.96	81.41
Cal-Ensemble	92.85	90.58	90.62	96.50	89.80	99.11	90.48

presents a comparative analysis of the performance of our calibration-based ensemble method, the ensemble method without calibration, and the two individual detectors: Oriented R-CNN and S²A-Net. For broader comparison, this work also includes results from three additional oriented object detection methods: RoI Transformer [12], Gliding Vertex [14], and YOLOv11-OOD [31].

The results in Table 1 indicate that S²A-Net achieves the best overall performance among the four individual methods, particularly excelling in detecting small components such as couplers. However, directly ensembling S²A-Net with Oriented R-CNN without calibration leads to a slight decline in performance. In contrast, our calibration-based ensemble improves the mean Average Precision (mAP) by 1.9% and outperforms all other compared methods. Notably, the performance gains are especially significant for small components such as grading rings and couplers, highlighting the effectiveness of the proposed calibration-enhanced ensemble strategy.

4.5. Qualitative Visualization

Figure 8 presents qualitative comparisons between the individual methods, the ensemble without calibration, and our calibration-based ensemble. As shown in the figure, individual methods may suffer from missed detections, false positives, or inaccurate localization. These issues are significantly mitigated by our calibration-based ensemble, leading to noticeably improved detection performance.

4.6. Complexity Analysis

Note that our proposed method is designed to enhance detection performance by post-processing the outputs of existing oriented object detection (OOD) models. Importantly, our approach does not modify the internal architectures of the base models nor introduce additional trainable parameters during inference. Therefore, the complexity of each individual model remains unchanged. The additional computational overhead introduced by our method is minimal, primarily consisting of confidence calibration (via lightweight linear regression) and bounding box fusion across models. This post-hoc ensembling strategy allows us to improve detection accuracy without significantly increasing computational cost or inference time.

5. Conclusions

This paper presents a confidence calibration-based model ensemble framework to improve oriented object detection for electrical equipment in infrared imagery. By integrating Oriented R-CNN and S²A-Net through a regression-based confidence calibration followed by soft non-maximum suppression, our method effectively leverages the complementary strengths of both detectors. The calibrated fusion not only enhances detection accuracy but also improves robustness under diverse imaging conditions. Experimental results on a specialized dataset for power system equipment confirm that our approach significantly outperforms the individual baseline models. These findings highlight the potential of confidence-aware ensemble strategies in advancing intelligent defect diagnosis within power systems. Future work may explore dynamic weighting schemes or extend the framework to real-time applications in complex environments.