Foreign Object Detection Network for Transmission Lines from Unmanned Aerial Vehicle Images

Abstract

Foreign objects such as balloons and nests often lead to widespread power outages by coming into contact with transmission lines. The manual detection of these is labor-intensive work. Automatic foreign object detection on transmission lines is a crucial task for power safety and is becoming the mainstream method, but the lack of datasets is a restriction. In this paper, we propose an advanced model termed YOLOv8 Network with Bidirectional Feature Pyramid Network (YOLOv8_BiFPN) to detect foreign objects on power transmission lines. Firstly, we add a weighted cross-scale connection structure to the detection head of the YOLOv8 network. The structure is bidirectional. It provides interaction between low-level and high-level features, and allows information to spread across feature maps of different scales. Secondly, in comparison to the traditional concatenation and shortcut operations, our method integrates information between different scale features through weighted settings. Moreover, we created a dataset of Foreign Object detection on Transmission Lines from a Drone-view (FOTL_Drone). It consists of 1495 annotated images with six types of foreign object. To our knowledge, FOTL_Drone stands out as the most comprehensive dataset in the field of foreign object detection on transmission lines, which encompasses a wide array of geographic features and diverse types of foreign object. Experimental results showcase that YOLOv8_BiFPN achieves an average precision of 90.2% and an [email protected] of 0.896 across various categories of foreign objects, surpassing other models.

1. Introduction

With the increasing demand for electricity, there has been a surge in the construction of overhead transmission lines [1,2,3]. This has been an important task for the protection of transmission lines [4,5,6]. Consequently, incidents involving foreign objects like bird nests and kites coming into contact with these lines have been increasing [7,8,9]. Foreign objects on power lines pose a significant risk of causing tripping incidents, which can severely impact the secure and stable operation of the power grid [10,11,12]. The automatic detection of foreign objects in time can provide useful information for managers or maintenance workers to handle the hidden danger targets.

Traditional methods rely on hand-crafted features. Tiand et al. [13] proposed an insulator detection method based on features fusion and Support Vector Machine (SVM) classifier. However, the reliance on the manual design and selection of features restricts its generalization capability and detection accuracy. Song et al. [14] introduced a local contour detection method that does not consider the relationship between power lines and foreign objects. Although it improves detection speed, it lacks consideration of the physical relationship between power lines. The foreign objects may limit its overall performance in practical applications. In summary, hand-crafted feature-based methods are easy to implement and perform well in detecting foreign objects on transmission lines in simple scenarios. However, when it comes to detecting foreign objects in complex scenarios, these methods exhibit limitations.

Currently, AI-based methods are being applied to object detection with unmanned aerial vehicles [15,16,17,18,19,20]. They are capable of automatically learning features, and identifying the positions of foreign objects. AI-based drone inspection has become an effective means to recognize the foreign objects on transmission lines [21,22,23]. Researchers around the globe have proposed several methods for detecting foreign objects [24,25,26]. According to the stages, these methods can be categorized into end-to-end detectors and multi-stage detectors.

End-to-End Detectors. Li et al. [27] proposed a lightweight model, which leverages YOLOv3, MobileNetV2, and depthwise separable convolutions. They considered various foreign objects such as smoke and fire. The method achieves a smaller model size and higher detection speed compared to existing models. However, its detection accuracy needs improvement, especially in detecting small-sized objects. Wu et al. [28] introduced an improved YOLOX technique for foreign object detection on transmission lines. It enhances recognition accuracy by embedding the convolutional block attention module and optimizes using the Generalized Intersection over Union (GIoU) loss. The results show that the enhanced YOLOX has an mAP improvement of 4.24%. However, the lack of detailed classification of foreign objects hinders the implementation of corresponding removal measures. Wang et al. [29] proposed an improved model based on YOLOv8m. They introduced a global attention module to focus on obstructed foreign objects. The Spatial Pyramid Pooling Fast (SPPF) module is replaced by the Spatial Pyramid Pooling and Cross Stage Partial Connection (SPPCSPC) module to augment the model’s multi-scale feature extraction capability. Ji et al. [30] designs an Multi-Fusion Mixed Attention Module-YOLO (MFMAM-YOLO) to efficiently identify foreign objects on transmission lines in complex environments. It provides a valuable tool for maintaining the integrity of transmission lines. Yang et al. [31] developed a foreign object recognition algorithm based on the Denoising Convolutional Neural Network (DnCNN) and YOLOv8. They considered various types of foreign object like bird nests, kites, and balloons. Its mean average precision (mAP) is 82.9%. However, there is a significant room for improvement in detection performance, and the diversity of categories in the dataset needs to be enriched.

Multi-Stage Detectors. Guo et al. [32] initially constructed a dataset for transmission line images and then utilized the Faster Region-based Convolutional Neural Network (Faster R-CNN) to detect foreign objects such as fallen objects, kites, and balloons. Compared to end-to-end detection, it can handle various shapes and sizes of foreign objects, demonstrating superior versatility and adaptability. Chen et al. [33] employed Mask R-CNN for detecting foreign objects on transmission lines. The method showed strong performance in speed, efficiency, and precision for detecting foreign objects. Dong et al. [34] explored the issue of detecting overhead transmission lines from aerial images captured by Unmanned Aerial Vehicles (UAV). The proposed method combines shifted window and balanced feature pyramid with Cascade R-CNN technology to enhance feature representative capabilities. It achieves 7.8%, 11.8%, and 5.5% higher detection accuracy than the baseline for mAP50, relative small and medium mAP, respectively.

Both end-to-end algorithms and multi-stage algorithms obtain good results on the given datasets. Multi-stage detection algorithms demonstrate clear advantages in detection accuracy, while the computational resources and detection time required are also substantial. In contrast, end-to-end detectors are trained with light models, and expected to achieve a trade-off between detection accuracy and speed. Our work belongs to end-to-end detectors.

The dataset for foreign object detection on transmission lines is crucial for model training. They can be categorized into two groups: simulated datasets [35,36] and realistic datasets [7,37,38,39]. Simulated datasets are generated by data augmentation techniques. Ge et al. [35] augmented their database using image enhancement techniques and manually labeled a dataset comprising 3685 images of bird nests on transmission lines. Similarly, Bi et al. [36] created a synthetic dataset focusing on bird nests on transmission towers under different environmental conditions. The dataset comprised 2864 images for training and 716 images for testing. Although these datasets simulate changes in environmental conditions through data augmentation techniques, they primarily cover a singular category of foreign objects on transmission lines. Realistic datasets are collected from real-world scenarios. Ning et al. [37] expanded the variety in their created dataset, encompassing categories such as construction vehicles, flames, and cranes, a total of 1400 images. Zhang et al. [38] established a dataset, consisting of 20,000 images, including categories like wildfires, additional objects impacting power lines, tower cranes, and regular cranes. Liu et al. [7], based on aerial images, focused on bird nests, hanging objects, wildfires, and smoke as the four categories for detection, creating a dataset comprising 2496 images. Yu et al. [39] refined five types of construction vehicle with bird nests from drone-view images. Li et al. created a dataset including four classes: bird nests, balloons, kites, and trash [40].

Existing datasets are not open source and suffer from limitations such as a limited variety of foreign objects, limited scenarios, and low quality. This significantly impedes progress in research within this field.

The motivation of this paper is to propose a deep learning-based approach for automatic foreign object detection. Some detection results are presented in the Figure 1. Considering the fact of lacking datasets, we create a comprehensive dataset to facilitate the model training for foreign object detection on power transmission lines.

The main contributions of this paper are summarized as follows:

• We propose an end-to-end method YOLOv8_BiFPN to detect foreign objects on power transmission lines. It integrates a weighted bidirectional cross-scale connection structure into the detection head of the YOLOv8 network. It outperforms the base-line model significantly.
• This paper creates a dataset of foreign objects on transmission lines from drone perspective named FOTL_Drone. It covers six distinct types of foreign object with 1495 annotated images, which is currently the most comprehensive dataset for the detection of foreign objects on transmission lines.
• Experimental results on the proposed FOTL_Drone dataset demonstrate YOLOv8_BiFPN model’s effectiveness, with an average precision of 90.2% and an average [email protected] of 0.896, outperforming other models.

The paper is structured as follows. Section 2 provides detailed information about the proposed FOTL_Drone dataset. Our method is described in Section 3, while Section 4 presents the experimental results. Finally, Section 5 concludes all the work.

2. The Proposed Dataset

Recently, drone inspection has gradually emerged as the primary means for detecting foreign objects on transmission lines. In this section, we will present a dataset of foreign objects on transmission lines from drone perspective (FOTL_Drone). This dataset comprises a total of 1495 annotated images, capturing six types of foreign object. The images in the dataset are primarily sourced from the following:

• Image search: Google, Bing, Baidu, Sogou, etc.
• Screenshots from video sites: YouTube, Bilibili, etc.
• Some images are captured from drone-view inspections on transmission lines.

To generate class labels, a visual image annotation tool called labelimg is employed. Annotations are saved in XML format following the PASCAL VOC standard. Six categories are included: nest, kite, balloon, fire, person, and monkey. These images are split into training and testing sets in an 8:2 ratio, with the training set comprising 1196 images and the testing set containing 299 images.

Figure 2 presents the analysis of the proposed FOTL_Drone dataset. Figure 2a sheds light on four key aspects: the data volume of the training set, showing balanced sample quantities across categories except for bird nests; the distribution and quantity of bounding box sizes, indicating diverse box dimensions; the position of bounding box center points within the images; and the distribution of target height-to-width ratios. Figure 2b depicts label correlation modeling, where cell color intensity in the matrix indicates the correlation between labels, with darker cells representing stronger associations, while the lighter cells indicating weaker ones. Figure 3 illustrates six classes of foreign object on transmission lines.

Moreover, the FOTL_Drone dataset is different from other datasets for the following reasons:

• Many existing datasets only have a number of limited categories of foreign objects on transmission lines, for example, two or three. In contrast, our FOTL_Drone expands the breadth of potential foreign objects on transmission lines to six classes.
• The majority of existing datasets cover a limited range of scenarios. The images in the proposed FOTL_Drone dataset are selected from the drone perspective, which adds scene diversity. This is attributed to the variable shooting positions of the drone.
• Existing datasets only include the static foreign objects. In contrast, our dataset not only includes static foreign objects such as kites and balloons, but also moving objects such as power workers and monkeys.

3. Methodology

By integrating a weighted bidirectional cross-scale connection structure into the detection head of the YOLOv8 network, we propose a detector fusing YOLOv8 with Bidirectional Feature Pyramid Network (YOLOv8_BiFPN). Its network architecture is depicted in Figure 4.

3.1. The Proposed YOLOv8_BiFPN

In the task of using drones to detect foreign objects on transmission lines, complex and variable environmental conditions are unavoidable, such as weather, lighting, and background interference. These factors pose limitations on traditional target detection algorithms in terms of extracting target features and adapting to different scenarios. The linear structure of power lines and the diversity of foreign objects make detection diffult, as they may appear in various shapes, scales, and orientations.

YOLOv8 achieves multi-scale feature fusion through its Feature Pyramid Network (FPN) architecture, allowing it to extract features at different scales. However, traditional FPN utilizes a unidirectional top-down pathway for feature fusion. This may result in not fully leveraging feature information to effectively capture and distinguish subtle differences between targets and backgrounds. Therefore, we introduce a Bidirectional Feature Pyramid Network (BiFPN) structure into the detection head of YOLOv8 to enhance the performance of target detection. Specifically, the BiFPN module is integrated during the process of connecting three layers of the network to form the detection head, as illustrated in Figure 4. By enhancing multi-scale feature fusion and bidirectional information transmission, the network can better adapt to complex environments and diverse target shapes.

BiFPN is a simple and efficient pyramid network that is first proposed in EfficientDet [41]. It introduces learnable weights to understand the importance of different input features while repeatedly applying both top-down and bottom-up multi-scale feature fusion. As illustrated in Figure 4, we incorporate the BiFPN_Concat structure into the neck of YOLOv8 to achieve bidirectional flow of feature information across different scales. This bidirectional flow can be seen as an effective exchange of information between different scales. The design of BiFPN_Concat aims to enhance the efficiency and effectiveness of feature fusion by reinforcing the bidirectional flow of features, thereby improving the performance of object detection.

BiFPN adopts a fast normalization-weighted fusion, which can be represented by Equation (1). In this equation, the application of Rectified Linear Unit (ReLU) after each $w_i$ ensures $w_i>=0$. The term $ϵ$ is a very small number used to ensure numerical stability. The normalized weight values obtained through this process are similar to softmax, ranging from 0 to 1. Overall, it demonstrates that this rapid fusion strategy can achieve with high accuracy compared to the fusion strategy of softmax, coupled with faster training speed.

$$O=\sum _i{\displaystyle \frac{w_i}{ϵ+{\sum }_j{w}_j}}·I_i$$

(1)

To give a good understanding of the pyramid structure of BiFPN, we have illustrated its detailed architecture in Figure 5. In comparison to FPN, it removes nodes with only one input edge to simplify the bidirectional network. When a node has only one input edge and lacks feature fusion, its contribution to the feature network aimed at integrating different features becomes minimal. As shown in Figure 5, the first node on the right side of $P_5$ is removed. The green lines in the figure represent edges added between the same layer’s original input nodes and output nodes. This is aimed at integrating more features without significantly increasing costs. The labeled “repeated blocks” in the figure represent treating each bidirectional pathway as a feature network layer and repeating the same layer multiple times to achieve higher-level feature fusion.

To illustrate the computational process of weighted feature fusion, we described the two fusion features of the BiFPN shown in the i-th level in the figure using Equation (2) and (3). In these equations, $P_i^{td}$ denotes the intermediate feature at the i-th level in the top-down hierarchy, while $P_i^{out}$ represents the output feature at the i-th level in the bottom-up hierarchy.

$$P_i^{td}=Conv\left({\displaystyle \frac{w_1·P_i^{in}+w_2·Resize\left(P_{i+1}^{in}\right)}{w_1+w_2+ϵ}}\right)$$

(2)

$$\begin{array}{c}\hfill P_i^{out}=Conv\left({\displaystyle \frac{w_1^{\prime }·P_i^{in}+w_2^{\prime }·P_i^{td}+w_3^{\prime }·Resize\left(P_{i-1}^{out}\right)}{w_1^{\prime }+w_2^{\prime }+w_3^{\prime }+ϵ}}\right)\end{array}$$

(3)

3.2. The Loss Functions

The loss function of YOLOv8_BiFPN comprises multiple components, including classification loss and regression loss. Specifically, the regression loss encompasses two distinct loss functions.

Classification Loss. Binary Cross-Entropy (BCE) is employed as the classification loss in YOLOv8_BiFPN. The computation process of BCE can be represented by Equation (4), where y represents a binary label of 0 or 1, and $p\left(y\right)$ denotes the probability of the output belonging to the label y. In simple terms, for the case where the label y is 1, if the predicted value $p\left(y\right)$ approaches 1, the value of loss function will tend towards 0. Conversely, if the predicted value $p\left(y\right)$ approaches 0 in this case, the value of loss function will be large.

$$\begin{array}{c}\hfill \mathrm{Loss}=-{\displaystyle \frac{1}{\mathrm{N}}}\sum _{\mathrm{i}=1}^{\mathrm{N}}{\mathrm{y}}_{\mathrm{i}}·log\left(\mathrm{p}\left({\mathrm{y}}_{\mathrm{i}}\right)\right)+(1-{\mathrm{y}}_{\mathrm{i}})·log(1-\mathrm{p}\left({\mathrm{y}}_{\mathrm{i}}\right))\end{array}$$

(4)

Regression Loss. The regression branch employs Distribution Focal Loss (DFL) and CIoU Loss. The computation process of DFL can be represented by Equation (5). Where, $S_i$ and $S_{i+1}$ represent the probability predictions at the i-th and i + 1-th distribution positions. y represents the actual bounding box position. $y_i$ and $y_{i+1}$ represent the two discrete positions closest to y, which are the actual values at positions i and $i+1$. During actual training, DFL is utilized to compute the loss for both the bounding box distribution probability and the label distribution probability, thereby optimizing each boundary. Subsequently, the bounding box distribution probability is restored to prediction boxes, and the CIoU loss is applied to compute the loss between the predicted boxes and the actual boxes, thus optimizing the overall prediction boxes.

$$\begin{array}{c}\hfill \mathbf{DFL}({\mathcal{S}}_i,{\mathcal{S}}_{i+1})=-\left((y_{i+1}-y)log\left({\mathcal{S}}_i\right)+(y-y_i)log\left({\mathcal{S}}_{i+1}\right)\right)\end{array}$$

(5)

4. Experiments

In this section, we conduct a series of experiments to validate the advantages of the proposed YOLOv8_BiFPN model on the FOTL_Drone dataset. Firstly, Section 4.1 introduces the overall experimental environment and evaluation metrics. Section 4.2 elucidates the rationale behind choosing YOLOv8_BiFPN. Thirdly, our method is compared with other methods on the FOTL_Drone dataset in Section 4.3. Section 4.4 presents some discussions.

4.1. Experimental Settings and Evaluation Metrics

The experimental environment is detailed in

Table 1. Running environment for experiments.

Device	Configuration
Operating system	Windows 11
GPU	NVIDIA GeForce RTX 4060
GPU accelerator	CUDA 11.8
Scripting language	Python 3.8
Framework	Pytorch 2.0
Compilers	Anaconda, Pycharm

. To facilitate comparisons with other models, the wide evaluation metrics are used.

First, Intersection-over-Union (IoU) represents the overlap or intersection between the candidate bounding box and the ground truth bounding box, as expressed in Equation (6).

$$IoU={\displaystyle \frac{area\left(C\right)\cap area\left(G\right)}{area\left(C\right)\cup area\left(G\right)}}$$

(6)

Common metrics Precision, Recall, Average Precision (AP) and mean Average Precision (mAP) are often used for statistical analysis and comparison,

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(7)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(8)

$$A{P}_{class}={\int }_0^{1}P\left(R\right)dR$$

(9)

$$mAP={\displaystyle \frac{1}{n}}\sum _{i=1}^{n}A{P}_i$$

(10)

where $TP$, $FP$, and $FN$ represent the number of true positives, false positives and false negatives. n is the number of classes. $AP$ is introduced to evaluate the accuracy of each class with Recall and Precision. $mAP$ is adopted to assess the overall performance of models. In this paper, the used mAP metrics are [email protected] and [email protected]:95.

4.2. Model Selection

To enhance the performance of the baseline model and investigate the impact of the feature pyramid, a series of experiments have been conducted using BiFPN modules with different embedding forms. We insert the BiFPN module at five distinct positions within the YOLOv8 network. Figure 6 illustrates the detailed Neck structure of improved YOLOv8, denoted as 1-YOLOv8_BiFPN, 2-YOLOv8_BiFPN, 3-YOLOv8_BiFPN, 4-YOLOv8_BiFPN, and 5-YOLOv8_BiFPN, corresponding to the different insertion points. Subsequently, the original YOLOv8 model and the five modified YOLOv8 models were evaluated on the FOTL_Drone dataset.

We have divided the FOTL_Drone dataset into a training set and a test set with an 8:2 ratio, and initialized the training process with pre-trained weights. The batch size is set to 4, and the training is conducted for 80 epochs. All models are trained under the same running environment.

The detection performance of the aforementioned models on the FOTL_Drone dataset is shown in

Table 2. Detailed comparison of original YOLOv8 and five variants on FOTL_Drone dataset.

Model	Class	Precision	Recall	[email protected]	[email protected]:95	Infertime (ms)
YOLOv8	Nest	94.9	87.8	95.2	59.7	80.5
	Kite	93.0	82.9	86.4	59.0	62.2
	Balloon	89.7	81.8	91.5	64.8	69.9
	Fire	81.6	76.5	79.8	43.1	58.0
	Person	83.3	84.8	89.0	49.9	70.3
	Monkey	82.3	78.7	83.6	47.5	65.2
1-YOLOv8 _BiFPN	Nest	92.8	91.3	95.7	58.4	81.5
	Kite	90.5	79.0	87.2	62.0	62.9
	Balloon	94.1	86.4	91.5	65.6	60.3
	Fire	84.1	78.4	85.7	40.5	64.5
	Person	84.0	83.0	84.8	43.7	57.0
	Monkey	82.0	73.5	76.5	40.2	76.8
2-YOLOv8 _BiFPN	Nest	93.6	86.1	94.1	61.0	69.6
	Kite	93.2	85.3	86.7	62.4	63.6
	Balloon	95.7	81.6	94.6	66.7	75.4
	Fire	88.2	78.4	85.1	41.4	58.9
	Person	79.7	81.1	86.0	46.7	56.0
	Monkey	82.2	69.1	79.8	43.0	60.2
3-YOLOv8 _BiFPN	Nest	95.5	92.9	97.0	59.9	65.4
	Kite	90.9	83.3	89.5	63.8	65.7
	Balloon	93.9	84.3	93.6	69.0	82.4
	Fire	90.1	76.5	85.4	44.1	60.7
	Person	85.6	77.4	86.7	47.1	58.6
	Monkey	85.0	71.7	85.6	49.1	61.9
4-YOLOv8 _BiFPN	Nest	97.8	88.2	95.0	55.5	79.3
	Kite	90.3	77.8	85.0	60.2	63.2
	Balloon	92.0	83.6	92.8	66.3	65.6
	Fire	82.3	72.5	82.3	43.0	65.7
	Person	70.6	79.2	85.6	47.1	60.8
	Monkey	82.7	63.2	75.8	42.7	72.3
5-YOLOv8 _BiFPN	Nest	95.8	89.4	92.7	55.5	75.5
	Kite	90.5	79.4	88.4	61.9	63.7
	Balloon	95.9	86.0	95.1	65.9	70.0
	Fire	85.3	72.5	80.6	40.3	60.8
	Person	80.5	90.6	89.7	44.2	61.6
	Monkey	92.6	70.2	85.6	45.6	66.2

.

Table 3. Average comparison of original YOLOv8 and five variants on FOTL_Drone dataset.

Model	Precision	Recall	[email protected]	[email protected]:95	Infertime (ms)
YOLOv8	87.5	82.1	87.6	54.0	67.7
1-YOLOv8_BiFPN	87.9	81.9	86.9	51.7	67.2
2-YOLOv8_BiFPN	88.8	80.3	87.7	53.5	63.9
3-YOLOv8_BiFPN	90.2	81.0	89.6	55.5	65.8
4-YOLOv8_BiFPN	86.0	77.4	86.1	52.5	67.8
5-YOLOv8_BiFPN	90.1	81.4	88.7	52.2	66.3

presents the average performance of the models across different categories. Additionally, it includes the inference time of each model. The models with improvements show minimal differences in inference time compared to YOLOv8. In terms of precision, 4-YOLOv8_BiFPN exhibits a 1.5% decrease compared to YOLOv8, while the other improved models keep an increasing tendency. As far as mAP is concerned, only 3-YOLOv8_BiFPN surpasses YOLOv8 in both metrics [email protected] and [email protected]:95. It is evident that the performance of 3-YOLOv8_BiFPN surpasses that of all other models.

We have analyzed and discussed the results from Table 2 and Table 3. Since the drones capture images from various angles and distances, the size of foreign objects varies significantly. This requires the model to effectively detect objects at multiple scales. The 1-YOLOv8_BiFPN introduces BiFPN early into the lower-level feature fusion and results in information loss or excessive processing of features. The 2-YOLOv8_BiFPN and the 4-YOLOv8_BiFPN integrate BiFPN into small and large object detection head, respectively. Detecting small objects requires high-resolution features. However, BiFPN over-processes low-level features, reducing the detailed information. Large objects are usually detected with high-level features, and BiFPN leads to inappropriate changes in the resolution of feature maps. The 5-YOLOv8_BiFPN integrates BiFPN into all concat operations; however, the complex fusion mechanisms across all levels may not meet the task demands. The 3-YOLOv8_BiFPN integrates BiFPN into a medium object detection head. Medium-level features provide balanced resolution and abstraction, ideal for multi-scale feature fusion. As a result, it achieves superior performance.

To enhance the detection performance of the network comprehensively, we explored various improvements to the YOLOv8 architecture. For instance, we have introduced the Slim Neck [42], Coordinate Attention (CoordAtt) [43], and Revisiting Mobile CNN From ViT Perspective (RepViT) [44], respectively.

Table 4. Detailed comparison of YOLOv8_BiFPN and improved models on FOTL_Drone dataset.

Class	Model	Precision	Recall	[email protected]	[email protected]:95	Infertime (ms)
Nest	+BiFPN	95.5	92.9	97.0	59.9	65.4
	+RepViT	97.4	88.1	94.7	59.4	71.3
	+CoordAtt	89.9	89.4	93.5	57.6	115.9
	+SlimNeck	89.8	93.0	95.4	57.6	66.3
Kite	+BiFPN	90.9	83.3	89.5	63.8	65.7
	+RepViT	90.4	85.4	88.2	60.1	92.2
	+CoordAtt	90.5	81.2	87.4	58.8	175.3
	+SlimNeck	84.7	85.4	86.5	58.4	68.4
Balloon	+BiFPN	93.9	84.3	93.6	69.0	82.4
	+RepViT	92.0	83.2	91.7	65.0	99.2
	+CoordAtt	95.1	90.9	95.3	64.2	158.9
	+SlimNeck	86.4	81.0	90.6	64.2	75.3
Fire	+BiFPN	90.1	76.5	85.4	44.1	60.7
	+RepViT	95.3	79.1	89.9	44.3	72.4
	+CoordAtt	82.6	78.4	79.9	40.8	139.0
	+SlimNeck	81.7	80.4	84.6	43.5	64.6
Person	+BiFPN	85.6	77.4	86.7	47.1	58.6
	+RepViT	74.7	79.2	85.4	47.1	80.0
	+CoordAtt	78.3	88.4	86.5	44.8	124.2
	+SlimNeck	75.9	81.1	83.4	46.7	62.2
Monkey	+BiFPN	85.0	71.7	85.6	49.1	61.9
	+RepViT	87.1	78.9	81.7	40.8	93.1
	+CoordAtt	77.3	81.7	82.7	46.3	225.9
	+SlimNeck	72.5	81.7	82.4	47.1	72.7
Average	+BiFPN	90.2	81.0	89.6	55.5	65.8
	+RepViT	89.5	82.3	88.6	52.8	84.7
	+CoordAtt	85.6	85.0	87.6	52.1	156.5
	+SlimNeck	81.8	83.8	87.2	52.9	68.3

shows the detection results of 3-YOLOv8_BiFPN in comparison to other enhanced models across six categories in the FOTL_Drone dataset. The 3-YOLOv8_BiFPN exhibits leading performance metrics in terms of [email protected] and [email protected]:95 across all categories. Apart from YOLOv8_CoordAtt, the difference in inference time among other models is relatively small. In summary, considering the performance among these models, the 3-YOLOv8_BiFPN is selected as the final improved YOLOv8 network, which is also shown in Figure 4.

Figure 7 illustrates the key metric curves of the proposed YOLOv8_BiFPN model during the training process. The top four sub-images depict curves of common metrics such as precision, recall, and F1 score as functions of confidence threshold. It shows all the changings of six types of foreign object. The bottom six plots show the variations of loss function values and various evaluation metrics with training iteration numbers increasing in the training process. This clearly sheds light on the model’s dynamic changes.

4.3. Comparison on FOTL_Drone Dataset

(1) Quantitative Results: To validate the effectiveness of the YOLOv8_BiFPN model, we compared it with four end-to-end models including Gold_YOLO [45], YOLOv8, YOLOv7 [46], and RetinaNet [47]. The experiments were conducted under the same environment with default parameter settings.

Table 5. Quantitative results of various models on FOTL_Drone dataset.

Model	Parameters (M)	Precision (%)	Recall (%)	[email protected] (%)
YOLOv8	3.0	87.5	82.1	87.6
YOLOv7 [46]	37.2	87.2	81.2	85.3
RetinaNet [47]	37.7	85.8	85.6	87.1
Gold_YOLO [45]	5.6	87.5	82.1	87.5
Ours	3.0	90.2	81.0	89.6

presents the comparative results on FOTL_Drone dataset. Our YOLOv8_BiFPN achieves an [email protected] score of 89.6%, surpassing other comparative models by more than 2%. Taking all metrics into consideration, our model outperforms other end-to-end models in the task of detecting foreign objects on transmission lines.

(2) Visual Results: For visual comparison, YOLOv7, YOLOv8, and our method are presented in Figure 8; specifically, the comparative output images obtained from YOLOv7, YOLOv8, and YOLOv8_BiFPN. The output images are annotated with detailed information about the detected targets, including their categories and positional details, along with the corresponding prediction probability values.

As shown in Figure 8, a visual comparison of the methods using the FOTL_Drone dataset is presented. Different colors of predicted boxes represent different classes, i.e., green for the monkey class, orange for the balloon class, red for the nest class, dark yellowish-green for the person class, and brown-orange for the fire class. The colors of arrows represent detection errors, i.e., the yellow arrows point to false positive boxes, the red arrows point to overlapping boxes, and the green arrows point to undetected objects. Figure 8d shows that the proposed method can reduce the detection errors well.

More detection results are shown in Figure 9. The number of targets and complexity progressively increase from the first row to the last row. Some objects tend to be dark, one potential reason for which is that most of images are captured with the sky as the background. For example, when the view is from the bottom to top, foreign objects look like dark objects, such as the nests and the workers in the second row. The occlusion is also a challenging scene, because the objects are easily occluded by the infrastructure of transmission lines, such as steel structures and lines. As far as the fire images are concerned, they are treated as an important issue for the safety of transmission lines. It can be seen from Figure 9 that our method can detect it accurately, even it is caused by a UAV.

As a consequence, the proposed method performs remarkably well on the FOTL_Drone dataset, especially for moderate and complex scenes. It outperforms other end-to-end models in terms of efficacy and accuracy.

4.4. Discussions

Although the proposed YOLOv8_BiFPN exhibits high mAPs in the detection of foreign objects on transmission lines, we admit that there are some detection failures. Figure 10 illustrates two types of detection error: missed detection and false detection. Missed detection is indicated by the red arrow. In (a), the target is relatively small and unclear, resulting in the missing detection. In (b), a traffic light is detected as a balloon, which is indicated by a yellow arrow. It can be seen from the figure that the traffic light is similar to the red balloon. These failures need to be handled. There is still room for the improvement of the obscured and small target detection.

To analyze model performance for different sizes of objects, we have classified the test set in FOTL_Drone dataset. Based on the proportion of objects in the images, the 299 images were divided into large objects (83 images), medium objects (110 images) and small objects (106 images). These images were tested on YOLOv8_BiFPN, and the detection results are presented in

Table 6. The performance of YOLOv8_BiFPN for different sizes of objects.

Image Size	Precision (%)	Recall (%)	[email protected] (%)	[email protected]:95 (%)
Large	87.1	84.6	90.2	63.1
Medium	90.2	88.9	93.2	58.6
Small	79.9	76.4	85.0	45.9

. It shows that the model performs well in detecting large and medium objects, achieving [email protected] above 90%. However, its performance drops significantly for small objects. Thus, it is a crucial task to improve the model’s performance for small object detection.

In the future, we aim to enhance the performance of the detector in two ways: (1) collect more foreign object images on transmission line from a drone perspective to further enrich the FOTL_Drone dataset; (2) explore the incorporation of contextual information for improving detection accuracy.

5. Conclusions

In this paper, we proposed a rapid and efficient detector for foreign object detection on transmission lines. It is an end-to-end method and aims to provide help to ensure the stability of the power grid. The proposed feature fusion model, termed YOLOv8_BiFPN, integrates a weighted interactive feature pyramid into the YOLOv8. Moreover, we have created the FOTL_Drone dataset, comprising 1495 annotated images of foreign objects. To our knowledge, this is the first comprehensive dataset used for foreign object detection on transmission lines. Experimental results on the FOTL_Drone dataset demonstrate an overall [email protected] of 89.6% for our approach. In the future, we will invest in research on ways to improve the detection accuracy for obscured and small targets on transmission lines. Hopefully our work can contribute to the development of a more intelligent foreign object detection system on power transmission lines in both the academic and engineering domains.