Research on UAV Autonomous Recognition and Approach Method for Linear Target Splicing Sleeves Based on Deep Learning and Active Stereo Vision

Abstract

This study proposes an autonomous recognition and approach method for unmanned aerial vehicles (UAVs) targeting linear splicing sleeves. By integrating deep learning and active stereo vision, this method addresses the navigation challenges faced by UAVs during the identification, localization, and docking of splicing sleeves on overhead power transmission lines. First, a two-stage localization strategy, LC (Local Clustering)-RB (Reparameterization Block)-YOLO (You Only Look Once)v8n (OBB (Oriented Bounding Box)), is developed for linear target splicing sleeves. This strategy ensures rapid, accurate, and reliable recognition and localization while generating precise waypoints for UAV docking with splicing sleeves. Next, virtual reality technology is utilized to expand the splicing sleeve dataset, creating the D_SS dataset tailored to diverse scenarios. This enhancement improves the robustness and generalization capability of the recognition model. Finally, a UAV approach splicing sleeve (UAV-ASS) visual navigation simulation platform is developed using the Robot Operating System (ROS), the PX4 open-source flight control system, and the GAZEBO 3D robotics simulator. This platform simulates the UAV’s final approach to the splicing sleeves. Experimental results demonstrate that, on the D_SS dataset, the RB-YOLOv8n(OBB) model achieves a mean average precision (mAP0.5) of 96.4%, with an image inference speed of 86.41 frames per second. By incorporating the LC-based fine localization method, the five rotational bounding box parameters (x, y, w, h, and angle) of the splicing sleeve achieve a mean relative error (MRE) ranging from 3.39% to 4.21%. Additionally, the correlation coefficients (ρ) with manually annotated positions improve to 0.99, 0.99, 0.98, 0.95, and 0.98, respectively. These improvements significantly enhance the accuracy and stability of splicing sleeve localization. Moreover, the developed UAV-ASS visual navigation simulation platform effectively validates high-risk algorithms for UAV autonomous recognition and docking with splicing sleeves on power transmission lines, reducing testing costs and associated safety risks.

1. Introduction

With growing demands for modern industrial and infrastructure monitoring, unmanned aerial vehicles (UAVs) have become indispensable tools due to their efficiency and flexibility. UAVs play a critical role in the inspection and maintenance of power transmission lines. They have become a primary method for ensuring the safety and stability of transmission lines where high-definition cameras are often employed to assess key components visually [1,2,3]. However, for splicing sleeves and other crimped metal fittings on transmission lines, high-payload UAVs are required to carry portable digital radiography (DR) equipment. This task demands skilled UAV operators to carefully align and dock the equipment along the splicing sleeve’s axis until it is accurately positioned and suspended over the splicing sleeve, after which DR inspection of the sleeve is performed [4,5,6].

Figure 1 illustrates the process of a high-payload quadrotor UAV carrying DR equipment during takeoff, approach, aerial descent, and docking under the operator’s control. The approach, docking, and successful suspension are critical steps for the UAV to participate in the DR inspection successfully. To assist the operator in achieving a precise UAV approach and DR equipment docking at the final stage, three key challenges must be addressed: (1) reliable aerial identification of the splicing sleeve by the UAV; (2) accurate measurement of the splicing sleeve’s position, including the relative distance and spatial orientation between the UAV and the sleeve; (3) development of a robust docking approach strategy that ensures reliable and efficient suspension.

Figure 2 shows the aerial imagery captured by the UAV during the approach for recognition and localization of the splicing sleeve. The splicing sleeve shown in the image presents several challenges, such as a variable scale, high aspect ratio, arbitrary orientation, and occupying only a small portion of the available camera pixels.

Selecting the right sensing devices is crucial to ensure UAVs successfully recognize and approach their targets. Traditional systems based on the Global Positioning System (GPS) and Inertial Navigation System (INS) lack the precision required for high-accuracy aerial docking tasks due to their limited accuracy [7,8]. Although ultrasonic sensors [9] and Light Detection and Ranging (LiDAR) [10,11] perform well in obstacle detection and avoidance, they are expensive, bulky, and susceptible to noise and reflection interference in complex environments, necessitating sophisticated compensation controls. In contrast, UAVs equipped with vision sensors and artificial intelligence technology can use stereo vision systems to capture and process environmental information in real time, allowing for both target recognition and enhanced positioning accuracy [12,13,14]. This technology is gaining increasing attention. Compared with traditional passive stereo vision systems, active stereo vision systems [15,16,17], which utilize structured light or other active light source technologies, offer higher adaptability and accuracy, particularly under complex lighting and environmental conditions.

Despite significant progress in UAV recognition, localization, and visual navigation research, end-stage visual navigation docking still faces numerous challenges. For example, lighting variations and target diversity in complex environments affect the robustness of visual navigation systems, and different target shapes require the application of varying target detection algorithms. During the UAV’s final approach to the splicing sleeve, it is essential to ensure accurate recognition of the sleeve and to guarantee precise localization. In addition, limitations in computational resources and real-time requirements impose higher demands on algorithm efficiency. Therefore, achieving high-precision recognition and localization in the final stage of UAV operation, while improving the real-time performance of the docking process [18,19], remains a key challenge and focus of the current research.

To address these challenges, we propose a UAV autonomous recognition and approach method for linear target splicing sleeves, integrating deep learning and active stereo vision, referred to as UAV-ASS, aimed at solving the key challenges faced by UAVs equipped with DR equipment during the final approach and docking stages with overhead power transmission line splicing sleeves. The main contributions of this study include:

(1)

A two-stage rapid and precise localization strategy based on the LC-RB-YOLOv8n (OBB) framework is proposed to address the issues of inaccurate positioning and unstable distance measurement faced by UAVs during high-altitude search, recognition, and approach tasks involving linear target splicing sleeves. This strategy first utilizes reparameterization of training results to obtain a lightweight and fast splicing sleeve recognition model. Subsequently, a local clustering algorithm is employed to enhance the positioning accuracy of splicing sleeves, and finally, the depth values of the splicing sleeves are extracted using the linear nearest neighbor averaging method.

(2)

To address the high costs and safety risks associated with image acquisition of splicing sleeves on high-altitude power transmission lines, as well as the difficulties in obtaining sufficient and representative data under complex and variable weather conditions and terrain in real scenarios, the construction of realistic splicing sleeve virtual scenes is proposed. This approach expands the real splicing sleeve dataset D_real to meet the requirements of diverse scenarios, thereby enhancing the robustness and generalization ability of the splicing sleeve recognition model.

(3)

To reduce the high-risk nature of visual navigation experiments involving UAV recognition and approach to high-altitude splicing sleeves and to improve algorithm verification and testing efficiency, a UAV-ASS visual simulation platform is proposed. This platform is built using the PX4 open-source UAV flight control system, the ROS, and the physical simulation platform GAZEBO, effectively reducing testing costs and safety risks.

The following is an overview of the key components of this study. Section 2 reviews related research work pertinent to this study. Section 3 systematically presents the principles of the UAV-ASS method based on LC-RB-YOLOv8n(OBB), including the optimization of the rotational object detection model for transmission line splicing sleeves, localization fine-tuning, and waypoint planning. Section 4 provides a comparative experimental analysis to evaluate the effectiveness of the proposed UAV-ASS visual navigation algorithm. Section 5 outlines the conclusions of this study along with suggestions for future work.

2. Related Work

The method of UAV autonomous recognition and approach to linear targets, combining deep learning and active stereo vision, primarily involves the recognition and localization of key targets by UAVs, as well as UAV autonomous docking and landing technology. This section will briefly review the relevant research in these two areas and further elaborate on the uniqueness of this study. The specific contents include:

2.1. UAV Stereo Vision for Key Target Recognition and Localization

At present, UAVs equipped with vision-sensing technology are extensively used for image recognition and localization of power transmission lines and other critical targets. Typically, image processing techniques or deep learning algorithms are employed to extract target features and achieve precise localization. Jia et al. [20] proposed a real-time method for obtaining the distance between a UAV and the corresponding clamp using the YOLOv8n(Det) algorithm in combination with a 3D coordinate detection algorithm based on stereo cameras. This approach provides guidance to ensure the UAV remains in a safe position. Li et al. [21] leveraged RGB-D saliency detection, together with real-time flight data and device parameters, to determine the longitude, latitude, and altitude of insulators. Elsaharti et al. [22] developed macro feature vectors from UAV-captured images in real time and matched them with pre-stored vectors from CAD models, successfully achieving rapid indoor target localization. Daramouskas et al. [23] proposed a UAV-based target detection, tracking, and localization solution using optical cameras with an improved YOLOv4 network for target detection, and combining the positioning information from four UAV cameras. Li et al. [24] applied UAVs for fire object detection. They captured 2D fire images via sensors, computed depth maps with stereo vision, and reduced interference through HSV-Mask filters and a non-zero mean method. GPS and Inertial Measurement Unit (IMU) module data were combined to obtain the latitude, longitude, and altitude coordinates of the fire areas. Li et al. [15] mounted the Intel Realsense D455 depth camera on a UAV and applied deep learning-based object detection algorithms to identify longan fruits, achieving precise localization using RGB-D information. While these studies addressed the problem of 3D position measurements for aerial UAV targets, the measurements of target position, especially the aerial angular orientation, remain largely unexplored.

Table 1. Comparison of Related Studies on UAV Stereo Vision for Target Recognition and Localization.

Study	Target Type	Depth Sensing Method	Algorithm	Key Contribution
Jia et al. [20]	Overhead clamp	Passive stereo vision	YOLOv8n + 3D coordinate detection	Real-time distance measurement between UAV and clamp to ensure safety.
Li et al. [21]	Insulator	RGB-D depth detection	RGB-D saliency detection	Combines real-time flight data to locate insulators’ longitude, latitude, and altitude.
Elsaharti et al. [22]	Indoor target	Passive stereo vision + CAD model matching	Real-time macro feature vector matching	Matches features captured by UAV to pre-stored CAD models for rapid indoor localization.
Daramouskas et al. [23]	General targets	Multi-camera localization	Improved YOLOv4 + multi-camera fusion	Introduces multi-camera-based detection for improved precision and tracking.
Li et al. [24]	Fire objects	Passive stereo vision	HSV-Mask filtering + non-zero mean method	Detects fire areas, computes depth, and combines GPS/IMU data for geographical localization.
Li et al. [15]	Longan fruits	Active stereo vision(D455)	MobileNet + YOLOv4	Improves the speed and accuracy of target detection and location for longan picking by UAVs based on vision.
This study	Overhead splicing sleeves	Active stereo vision	LC-RB-YOLOv8n(OBB)	Addresses the problem of rapid recognition and precise distance measurement for UAV docking with linear targets.

compares the differences in UAV applications of stereo vision for target recognition and localization in the related literature.

2.2. UAV Autonomous Docking and Landing Technology

The challenge of visually guided UAV docking and landing has been a key research focus in this field [25]. Li et al. [26] explored autonomous docking between UAVs and mobile platforms based on Ultra Wideband (UWB) and vision sensors, proposing an integrated estimation and control scheme that is divided into three stages: hovering, approaching, and landing. Yang et al. [27] designed a hybrid system combining UAVs and climbing robots for multi-scale power line inspection. They developed a special feature extraction operator and a density feature recognition algorithm, using stereo vision to measure the depth of the power line landing points, thereby enabling stable autonomous landing of the hybrid UAV inspection robots on power lines. Chen et al. [28] proposed a pan-tilt-based visual servo system, which uses onboard camera status and image data to guide precise UAV landing on a square platform in Global Navigation Satellite System (GNSS)-denied environments, applying different strategies at various landing stages. Zhou et al. [29] applied an improved ant colony algorithm for UAV flight path planning, using deep learning algorithms to identify insulators on overhead transmission lines and locate defects, significantly reducing inspection time and improving efficiency. Although these studies demonstrate the use of visual perception for navigation in different scenarios, current research on UAV visual navigation [25,30] offers relatively little discussion on how to achieve UAV recognition, approach, and suspended docking to linear aerial targets, such as splicing sleeves.

Table 2. Comparison of Related Studies on UAV Autonomous Docking and Landing Technology.

Study	Target Scenario	Sensor Type	Algorithm	Key Contribution
Li et al. [26]	Docking with moving platforms	UWB + vision sensors	Integrated estimation and control	Proposed a three-stage hovering, approaching, and landing control method for UAV docking.
Yang et al. [27]	Powerline inspection	Stereo vision	Feature extraction + depth measurement	Combined UAV and climbing robots for multi-scale powerline inspection, enabling stable landing.
Chen et al. [28]	GNSS-denied environment	Omnidirectional camera + gimbal system	Image-guided landing control	Guided UAV landing on a square platform in GNSS-denied conditions with stage-specific strategies.
Zhou et al. [29]	Overhead line inspection	Camera + deep learning	Improved ant colony algorithm + defect detection	Introduced UAV path planning and visual inspection, significantly improving inspection efficiency.
This study	Overhead splicing sleeves	Active stereo vision	LC-RB-YOLOv8n(OBB) + virtual simulation	Proposes a deep learning and active stereo vision-based docking method, solving angular measurement challenges.

summarizes the above-mentioned studies.

2.3. Uniqueness of This Study

This study presents a unique approach that integrates deep learning and active stereo vision to enable autonomous recognition and approach of linear aerial targets, such as splicing sleeves, by UAVs. Unlike previous studies that primarily focused on target localization and three-dimensional spatial measurements, this study not only emphasizes precise target localization but also places special emphasis on target angular orientation measurement, which is crucial for reliable docking with linear aerial structures. Additionally, this study expands the dataset using virtual reality technology and establishes the UAV-ASS visual simulation platform to validate the proposed method, providing valuable insights for the application of UAVs in the power industry.

3. Methodology

Figure 3 illustrates the schematic diagram of the proposed UAV-ASS method, which consists of four main modules: (1) a two-stage rapid and accurate localization strategy for rotational targets (LC-RB-YOLOv8n(OBB)); (2) coordinate transformation and waypoints planning; (3) dataset construction incorporating virtual reality; and (4) the UAV-ASS visual navigation simulation platform. Initially, the LC-RB-YOLOv8n(OBB) module deployed on the UAV reads aligned RGB and depth images of the splicing sleeve from the stereo depth camera in real-time, outputting the finely adjusted rotational target position of the splicing sleeve (including rotation angle Ɵ_F and position parameters x_OF, y_OF, w_F, h_F) and the distance between the UAV and the splicing sleeve, D_UAV-SS. Next, the pixel coordinates of the splicing sleeve are transformed into UAV body coordinates, and flight waypoints are generated through route planning, enabling the UAV to autonomously approach and dock with the splicing sleeve. This study also constructs a dataset adapted to diverse application scenarios using virtual reality technology. Additionally, a UAV-ASS visual navigation simulation platform integrating the ROS, the open-source UAV autopilot PX4, and the GAZEBO robotics simulation system is independently developed to validate the proposed UAV-ASS method.

3.1. A Two-Stage Rapid and Accurate Localization Strategy for Rotational Targets (LC-RB-YOLOv8n(OBB))

3.1.1. Rapid Localization of Rotational Object Detection Using RB-YOLOv8n(OBB)

The target detection algorithm used for UAV approaches to splicing sleeves must be able to handle multi-scale targets and provide high-precision angle measurements, and at the same time, meet the real-time processing requirements for high-frame-rate image transmission during high-altitude, high-speed UAV operations. This study presents a lightweight, end-to-end rotational object detection model, RB-YOLOv8n(OBB) to address these requirements. The model offers excellent real-time performance. The model is based on the YOLOv8n(OBB) [31] rotational object detection module, which has demonstrated outstanding performance on the remote sensing dataset DOTA [32], and has been further refined and optimized for the unique challenges of this application.

Figure 4 illustrates the network architecture diagram of the RB-YOLOv8n(OBB) model proposed in this study. The network consists of three main components: the Backbone, Neck, and Head. The Backbone includes three modules—CBS, RepBlock, and SPPF -with nine feature output layers. The structures of the CBS and SPPF modules are shown in the Figure 4. The CBS module is used for feature extraction, normalization, and non-linear processing, while the SPPF module is for pooling and concatenation of feature maps at different scales. The RepBlock module replaces the C2f module in the classical YOLOv8n(OBB) backbone, with its core function being to obtain an efficient inference network through reparameterization of the trained network. This facilitates deployment on edge hardware, enabling lightweight, real-time processing. The feature maps from the 4th, 6th, and 9th layers of the Backbone are fed into the Neck.

The Neck consists of three modules: RepBlock-s2, Concat, and Upsample. The RepBlock-s2 module replaces the C2f-s2 module in the classical YOLOv8n(OBB) Neck, following the same enhancement strategy used in the Backbone. The Concat and Upsample modules achieve bottom-up PAN (Path Aggregation Network) and top-down FPN (Feature Pyramid Network) feature fusion, enhancing the multi-scale feature fusion capabilities. Finally, the feature maps from the 15th, 18th, and 21st layers of the Neck are passed to the Head.

The Head adopts a decoupled structure, generating feature maps for the bounding box, rotation angle, and classification loss. The feature map sizes for predicting bounding boxes are 80 × 80 × 64, 40 × 40 × 64, and 20 × 20 × 64; for predicting rotation angles, the feature map sizes are 80 × 80 × 1, 40 × 40 × 1, and 20 × 20 × 1. Since the rotational object detection task focuses solely on splicing sleeves, the classification feature map has only one channel. To calculate the bounding box regression loss $L_{reg}$ in RB-YOLOv8n(OBB), the Gaussian probabilistic ProbIoU is used to better capture the overlap and uncertainty of the bounding boxes.

The following section focuses on introducing the principles and advantages of replacing the C2f module in the YOLOv8n(OBB) with the RepBlock module. Figure 5a illustrates the structure of the original C2f module in the YOLOv8n (OBB) backbone network. This module consists of 2 CBS layers and n Bottleneck modules, connected through Split and Concat operations. The inference network shares the same structure as the training network. To further enhance inference speed and reduce the computational complexity of the model, the RepBlock module was introduced. Through reparameterization, the multi-branch structure of RepConv (Training State) in the training phase is transformed into a single-branch structure RepConv (Inference State), as shown in Figure 5b. Although the total parameter count of the RepBlock module increases during training, resulting in extended training time, it significantly improves inference speed while maintaining detection accuracy during the inference phase.

In the training phase, RepConv consists of three branches: 3 × 3 conv, 1 × 1 conv, and an Identity branch, where the Identity branch performs no operations. Each branch undergoes BatchNorm2d standardization, after which the outputs are summed up and passed through the activation function to produce Y_train. The input feature map is set to be X, the activation function is set to be SiLU, and the normalization function BN. Then we have [33]:

Y_train = SiLU(BN(Conv_3×3(X) + BN(Conv_1×1(X) + BN(Identity(X)))

(1)

The reparameterization process in this study fuses the convolutional layer (Conv) and batch normalization (BN) layer of each branch into an equivalent convolutional layer (Conv_eq), which are then stacked to form the final convolutional layer in the inference phase, as shown in Figure 5. Assuming that after training, the weight W⁽ⁿ⁾ of the Conv layer in a certain branch of RepConv (training state) is known, and the BN layer parameters include the running mean μ⁽ⁿ⁾, running variance σ²⁽ⁿ⁾, scaling factor γ⁽ⁿ⁾, bias β⁽ⁿ⁾, and a small constant ε⁽ⁿ⁾, with the standard deviation given by ${std}^{\left(\mathrm{n}\right)}=\sqrt{{\sigma }^{2\left(\mathrm{n}\right)}+{\epsilon }^{\left(\mathrm{n}\right)}}$, After adjustment by the BN layer, the Conv_eq weight W′⁽ⁿ⁾ and bias b′⁽ⁿ⁾ for this branch can be calculated as follows [33]:

$${W^{\prime }}^{\left(n\right)}=W^{\left(n\right)}\cdot {\displaystyle \frac{{\gamma }^{\left(n\right)}}{st{d}^{\left(n\right)}}}=W^{\left(n\right)}\cdot {\displaystyle \frac{{\gamma }^{\left(n\right)}}{\sqrt{{\sigma }^{2\left(n\right)}+{\epsilon }^{\left(n\right)}}}};\hspace{1em}{b^{\prime }}^{\left(n\right)}={\beta }^{\left(n\right)}-{\displaystyle \frac{{\mu }^{\left(n\right)}{\gamma }^{\left(n\right)}}{st{d}^{\left(n\right)}}}={\beta }^{\left(n\right)}-{\displaystyle \frac{{\mu }^{\left(n\right)}{\gamma }^{\left(n\right)}}{\sqrt{{\sigma }^{2\left(n\right)}+{\epsilon }^{\left(n\right)}}}}$$

(2)

In Equation (2), (n) is used as an index with values of (3), (1), and (0), representing the parameters of the 3 × 3 conv, 1 × 1 conv, and Identity branch, respectively.

The Conv_eq weights and biases for each branch in the RepConv structure during the inference state were determined as follows:

(1)

In the Conv_eq calculation for the 3 × 3 conv branch, since the weight W⁽³⁾ obtained during training has the same dimensions as the weight in the inference phase, the equivalent weight W′⁽³⁾ and bias b′⁽³⁾ for the 3 × 3 conv can be directly calculated by substituting the trained values of W⁽³⁾, μ⁽³⁾, σ²⁽³⁾, γ⁽³⁾, β⁽³⁾ and ε⁽³⁾ into Equation (2).

(2)

In the Conv_eq for the 1 × 1 conv branch, since the weight W⁽¹⁾ obtained during training does not have the same dimensions as the weight in the inference phase, it can be expanded using a padding operation (Pad) to form W⁽¹⁾_3×3, ensuring that W⁽¹⁾_3×3 matches the dimensions of the weight in the inference phase. The expanded W⁽¹⁾_3×3 is denoted as:

$$W_{3\times 3}^{\left(1\right)}=Pad\left(W^{\left(1\right)}\right)=\left[\begin{array}{ccc}0& 0& 0\\ 0& W^{\left(1\right)}& 0\\ 0& 0& 0\end{array}\right]$$

(3)

At this point, the equivalent weight W′⁽¹⁾ and bias b′⁽¹⁾ for the 1 × 1 conv branch can be calculated by substituting W⁽¹⁾_3×3, μ⁽¹⁾, σ²⁽¹⁾, γ⁽¹⁾, β⁽¹⁾ and ε⁽¹⁾ into Equation (2).

(3)

In the Conv_eq calculation for the Identity branch, since this branch only contains the BN layer, a 3 × 3 pseudo convolutional kernel is first created, with the center of the kernel set to 1 and the remaining elements set to zero. The number of convolutional kernels is made equal to the number of input channels. The pseudo weight for this branch, W⁽⁰⁾, is denoted as:

$$W^{\left(0\right)}=\left[\begin{array}{ccc}0& 0& 0\\ 0& 1& 0\\ 0& 0& 0\end{array}\right]$$

(4)

At this point, the equivalent weight W′⁽⁰⁾ and bias b′⁽⁰⁾ for the Identity branch can be calculated by substituting W⁽⁰⁾, μ⁽⁰⁾, σ²⁽⁰⁾, γ⁽⁰⁾, β⁽⁰⁾ and ε⁽⁰⁾ into Equation (2).

Therefore, the expression for the output Y_inference of a single branch in the RepConv during the inference phase is given by:

$$Y_{inference}=\mathrm{SiLU}\left({\mathrm{Conv}}_{3\times 3}\left(X,W_{\mathrm{e}q},b_{eq}\right)\right)=\mathrm{SiLU}\left({\mathrm{Conv}}_{3\times 3}\left(X,{W^{\prime }}^{\left(3\right)}+{W^{\prime }}^{\left(1\right)}+{W^{\prime }}^{\left(0\right)},b{\prime }^{\left(3\right)}+b{\prime }^{\left(1\right)}+b{\prime }^{\left(0\right)}\right)\right)$$

(5)

The reparameterization separates the inference and training processes, with the model’s computations shown in Equations (1) and (5), respectively. Compared to the RepConv structure in the training phase, the RepConv structure during inference is significantly simplified, dramatically reducing computational complexity and facilitating hardware implementation. The accuracy and real-time performance of this structure are further evaluated in subsequent experiments.

The splicing sleeve’s rotated bounding box is generated using RB-YOLOv8n(OBB). Figure 6 illustrates the rotated bounding box output from the rapid localization, represented as B_{C_RGB} = (x_OC, y_OC, w_C, h_C, θ_C), where x_OC and y_OC are the horizontal and vertical coordinates of the center point O of the rotated bounding box, respectively, and w_C, h_C and θ_C represent the width, height, and rotation angle of the bounding box, respectively.

3.1.2. Fine Localization of Rotational Object Detection Using Local Clustering (LC)

When the UAV equipped with a depth camera performs high-altitude search and identification tasks for splicing sleeves, characteristics such as the high aspect ratio, arbitrary angle, and small pixel ratio of the linear target splicing sleeves, along with UAV body vibrations, make B_{C_RGB} = (x_OC, y_OC, w_C, h_C, θ_C) insufficient for accurately and reliably representing the position of the splicing sleeve, as shown in Figure 6.

Figure 7 illustrates the schematic diagrams of the fine localization process using local clustering. First, the B_{C_RGB} = (x_OC, y_OC, w_C, h_C, θ_C) bounding box is mapped to identify the local region R_DE in the depth image that contains the rapidly localized splicing sleeve, thereby reducing the computational area. Next, the depth values D_R of the local region R_DE are extracted, and clustering and fitting are performed on D_R to obtain a more accurately localized bounding box B_{R_Depth} = (x′, y′, w′, h′, θ′), enhancing the precision of the localization. Finally, through coordinate transformation, the value of B_{R_Depth} = (x′, y′, w′, h′, θ′) in the depth image is mapped to B_{F_RGB} = (x_OF, y_OF, w_F, h_F, θ_F) in the RGB image, completing the transition from rapid localization to fine localization of the splicing sleeve.

The boundary of the local region R_DE, which contains the rapidly localized rectangular box of the splicing sleeve, and the clustering and fitting process of the depth values D_R within region R_DE, are determined as follows:

(1)

Determination of the Local Region boundary R_DE for the Rapidly Localized Rectangular Box of the Splicing Sleeve:

The rapid localization result of the splicing sleeve in the RGB image, B_{C_RGB} = (x_OC, y_OC, w_C, h_C, θ_C), obtained by RB-YOLOv8n(OBB), is mapped to the depth image as B_{C_DE} = (x_OC, y_OC, w_C, h_C, θ_C). The boundary of the local region R_DE must fully enclose the splicing sleeve’s rectangular box B_{C_DE}, with an additional margin for fine adjustment of B_{C_DE}, as shown in Figure 7a.

The four corner points of the splicing sleeve’s rectangular box B_{C_DE} are set to be (x_i, y_i), where i ∈ {1, 2, 3, 4}. The minimum horizontal and vertical coordinates of these corner points are defined as x_min = Min(x₁, x₂, x₃, x₄) and y_min = Min(y₁, y₂, y₃, y₄), respectively; the maximum horizontal and vertical coordinates are defined as x_max = Max(x₁, x₂, x₃, x₄) and y_max = Max(y₁, y₂, y₃, y₄), respectively. The smallest region for R_DE is then given by:

$$R_{\mathrm{DE}\_\min }=\left\{\left(x,y\right)\in {ℝ}^2|x_{\min }\le x\le x_{\max },y_{\min }\le y\le y_{\max }\right\}$$

(6)

With the assumption that the margin for fine-tuning the localization is δ in both the horizontal and vertical directions, the expression for the local region R_DE is given by:

$$R_{\mathrm{DE}}=\left\{\left(x,y\right)\in {ℝ}^2|x_{\min }-\delta \le x\le x_{\max }+\delta ,y_{\min }-\delta \le y\le y_{\max }+\delta \right\}$$

(7)

The value of δ is determined based on the principle of sufficiency, with smaller values being preferable. Once the boundary calculation for the local region R_DE is completed, subsequent operations, such as depth value clustering and fitting are executed within the R_DE.

(2)

Clustering and Fitting of Depth Values D_R in the Local Region R_DE

Due to the presence of the splicing sleeve, the depth values in the local region R_DE exhibit different distributions. By calculating the depth values D_R in this region via clustering and fitting, a more accurate representation of the splicing sleeve’s position, B_{R_DE} = (x′, y′, w′, h′, θ′), is obtained. For clustering D_R, the fast and convenient K-Means algorithm could be used, and the fitting of the minimum area rotated bounding box is achieved using the cv2.minAreaRect() function from the OpenCV library, as shown in Figure 7b.

①

Clustering of Depth Values D_R in R_DE

First, the depth values $D_R=\left\{D\left(x_i,y_i\right)|\left(x_i,y_i\right)\in R_{DE}\right\}$ are extracted, and then the K-Means algorithm to cluster D_R is applied. The number of clusters is k, the cluster centers are μ_i, the DR samples are d_n, and the indicator variable is r_ni. The objective is to minimize the cost function J, which is expressed as follows [34]:

$$J_{\min }={\displaystyle \sum _{n=1}^N}{\displaystyle \sum _{i=1}^K}{r}_{ni}{‖d_n-{\mu }_i‖}^2;\hspace{1em}where r_{ni}=\left\{\begin{array}{c}1,\hspace{1em}{d}_n\in cluster i\\ 0,\hspace{1em}{d}_n\notin cluster i\end{array}\right.$$

(8)

To solve this equation, the clustering centers μ_i for D_R are first initialized, and then the samples d_n are assigned to their respective clusters. Next, the clusters are updated based on ${\mu }_i={\displaystyle \sum _{n=1}^N}{r}_{ni}{d}_n/{\displaystyle \sum _{n=1}^N}{r}_{ni}$, and the centers μ_i for each cluster are recalculated. This process continues until the cluster centers no longer show significant changes, at which point the loss function J is considered to have reached its minimum value J_min. After clustering the D_R values, the cluster to which the depth value D (x′ = x_OC − x_min + δ, y′ = y_OC − y_min + δ) belongs is identified as the splicing sleeve category.

②

Depth Value D_R Clustering and Fitting

The point set for the depth clustering category corresponding to the splicing sleeve is defined as $P={\left\{\left(x_i,y_i\right)\right\}}_{i=1}^N$. The cv2.minAreaRect() function from the OpenCV library is used to fit a minimum area of the rotated bounding box for the point set P, resulting in the rotated bounding box B_{R_DE} in the local region R_DE of the depth map, as follows:

B_{R_DE} = cv2.minAreaRect(P) = (x′, y′, w′, h′, θ′)

(9)

The rotated bounding box B_{R_DE} in the local region R_DE of the depth map cannot be used directly. It must first be transformed into the rotated bounding box B_{F_RGB} on the RGB image plane, which represents the final result of fine localization after applying LC to the rotational object (see Figure 7c). The expression is given as:

B_{F_RGB} = (x_OF, y_OF, w_F, h_F, θ_F) = (x′ + x_min − δ, y′ + y_min − δ, w′, h′, θ′)

(10)

After B_{F_RGB} = (x_OF, y_OF, w_F, h_F, θ_F) is obtained, the UAV’s relative distance to the splicing sleeve D_UAV-SS is determined from the depth map. D(x_i, y_i) is set to represent the depth values along the line (y − y_OF) = tan(θ_F) (x − x_OF) in the depth map, within the range of [−σ/2, +σ/2] pixels, where σ = w_F/10. After removing outliers caused by factors such as the smooth surface of the splicing sleeve, texture loss, and lighting variations, assuming there are N valid depth points, and with θ_F being the rotation angle of the bounding box, as shown in Figure 8, the D_UAV-SS is calculated using the linear nearest neighbor averaging method as:

$$D_{\mathrm{UAV}-\mathrm{SS}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{x_i=x_{\mathrm{OF}}-\frac{\sigma }{2}\cos {\theta }_{\mathrm{F}}}^{x_i=x_{\mathrm{OF}}+\frac{\sigma }{2}\cos {\theta }_{\mathrm{F}}}D(x_i,\tan {\theta }_F\times (x_i-x_{\mathrm{OF}})+y_{\mathrm{OF}})}; where D(x_i,\tan {\theta }_F\times (x_i-x_{\mathrm{OF}})+y_{\mathrm{OF}})\ne null$$

(11)

Real-time high-precision recognition and distance measurement of the splicing sleeve by the UAV at high altitudes are achieved following the above steps. Algorithm 1 provides the pseudocode for the above process.

Algorithm 1 Fusion K-Means clustering rotating target fine-tuning algorithm
Input:depth_image, rgb_image Output:rect_new(Fine-tuning rotating target recognition results), depth_value
1.	k_means_trimming_functions(rag_image, depth_image)
2.	result = inference_detector(model, rgb_image); // Rotating target recognition results
3.	key_point = extract_clustering_target_regions(result); // Return the key parameters of the horizontal box area of the rotating target result according to the result.
4.	if key_point! = 0: // Determine whether there are recognition results that satisfy the conditions
5.	depth_target_regions = depth_image(key_point); // Cut horizontal box area on depth_image
6.	num_clusters = 3; // Set the initial clustering value. When shooting from high altitude, determine the clustering value based on the complexity of the background.
7.	cluster_depth_image = K_Means_fuction(num_clusters,depth_target_regions); // Return clustering results
8.	min_depth_class = np.argmin([np.mean(depth_target_regions[cluster_depth_image == i]) for i in range(num_clusters)]); // Finding the Depth Minimum Category
9.	rect = cv2.minAreaRect(points_cv); // Fitting a minimum area rotating frame
10.	center_x, center_y, width, heigth, angle = rect; // Get the center point coordinates, width, height and rotation angle of the rotated box after fine-tuning.
11.	depth_value = depth_target_regions[int(center_y), int(center_x)];                         // Extract the depth value of the center point of the rotating frame
12.	rect_new = rect + key_points; // Adjust the spinning frame coordinates to depth_image coordinates
13.	return (rect_new, depth_value); // Returns the coordinates and depth of the original rotated frame.
14.	else:
15.	return (0,0); // Returns the null value

3.2. UAV-ASS Coordinate Transformation and Waypoint Planning

The calculation of the splicing sleeve’s center point relative to the UAV body coordinates involves four coordinate systems, as shown in Figure 9: the pixel coordinate system O_xy, the image coordinate system O_{p_xy}, the camera coordinate system O_{c_xyz}, and the UAV body coordinate system O_{b_xyz}.

The transformation matrices between the pixel coordinate system O_xy and the image coordinate system O_{p_xy}, the image coordinate system O_{p_xy} and the camera coordinate system O_{c_xyz}, and the camera coordinate system O_{c_xyz} and the UAV body coordinate system O_{b_xyz} are denoted as K₁, K₂, and K₃, respectively. The origin O_p of the image coordinate system O_{p_xy} has coordinates (x_op, y_op) in the pixel coordinate system O_{p_xy}. The pixel sizes along the horizontal axis x_p and vertical axis y_p of the image coordinate system O_{p_xy} are dx_p and dy_p, respectively, and the camera focal length is f, where dx, dy, and f are the intrinsic parameters of the camera. The rotation and translation matrices from the camera coordinate system O_{c_xyz} to the UAV body coordinate system O_{b_xyz} are ^bR_c3×3 and ^bT_c3×1, respectively, and are primarily determined by the camera’s mounting position on the UAV. Thus, K₁, K₂, and K₃ are expressed as:

$$K_1={\left[\begin{array}{ccc}1/\mathrm{d}{x}_p& 0& x_{op}\\ 0& 1/\mathrm{d}{y}_p& y_{op}\\ 0& 0& 1\end{array}\right]}^{-1}; K_2={\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& f& 0\end{array}\right]}^{-1}; K_3={\left[\begin{array}{cc}{}^{b}R_{c3\times 3}& {}^{b}T_{c3\times 1}\\ \overrightarrow{0}& 1\end{array}\right]}^{-1}$$

(12)

According to Equations (10) and (11), (x_OF, y_OF) and D_UAV-SS are obtained, and the relationship of the splicing sleeve’s center point relative to the UAV body coordinates (x_b, y_b, z_b) is expressed as follows:

$${\left[\begin{array}{cccc}{x}_b& y_b& z_b& 1\end{array}\right]}^{\mathrm{T}}=D_{\mathrm{UAV}-\mathrm{S}S}{K}_3{K}_2{K}_1{\left[\begin{array}{ccc}{x}_{OF}& y_{OF}& 1\end{array}\right]}^{\mathrm{T}}$$

(13)

The splicing sleeve is typically in a horizontal position while in service. When the UAV hovers at high altitude and identifies the splicing sleeve, the position of the splicing sleeve relative to the UAV can be simplified as ^bP_ss = (x_b, y_b, z_b, θ_b), where θ_b is the angle between the longitudinal axis of the splicing sleeve and the UAV’s body axis (i.e., the angle θ_F) between the long side of the splicing sleeve and the y-axis of the pixel coordinate system). The criteria for successful docking are that the UAV hovers 1.0 m above the splicing sleeve (this value depends on camera imaging parameters and the length of the equipment suspension line) and that θ_b = 0, as shown in Figure 10. This indicates that the splicing sleeve is centered in the camera’s image and that its longitudinal axis is parallel to the y-axis of the pixel coordinate system. The final position of the splicing sleeve relative to the UAV can be expressed as ^bP_ss|_end = (0, 0, −1.0, 0).

To improve docking success rates, an intermediate waypoint ^bP_ss|_mid= (0, 0, (−1.0 − ∆, 0)) is inserted between the starting point ^bP_ss|_start and the end point ^bP_ss|_end, where ∆ can be adjusted as needed. First, the UAV moves from ^bP_ss|_start to ^bP_ss|_mid, during which both position and orientation must be adjusted, ensuring that Ɵ_b is achieved upon reaching ^bP_ss|_mid. Next, the UAV moves from ^bP_ss|_mid to ^bP_ss|_end, which only involves adjusting the UAV’s altitude. These two steps generate flight waypoints using linear interpolation of position and orientation, thus enhancing flight smoothness. The UAV’s position relative to the world coordinate system is set to be ^wP_b, and its orientation relative to the world coordinate system be ^wq_b. The angle between the initial orientation ^wq_b|_start and the final orientation ^wq_b|_end is θ, with t ∈ [0, 1]. The UAV’s position at any waypoint is calculated by the following equation:

$${}^{\mathrm{w}}\mathrm{P}_{\mathrm{b}}\left(t\right)={}^{\mathrm{w}}\mathrm{P}_{\mathrm{b}|\mathrm{start}}\left(1-t\right)+{}^{\mathrm{w}}\mathrm{P}_{\mathrm{b}|\mathrm{mid}}t ; {}^{\mathrm{w}}\mathrm{q}_{\mathrm{b}}\left(t\right)={\displaystyle \frac{\sin \left(\left(1-t\right)\theta \right)}{\sin \theta }}{}^{\mathrm{w}}\mathrm{q}_{\mathrm{b}|\mathrm{start}}+{\displaystyle \frac{\sin \left(t\theta \right)}{\sin \theta }}{}^{\mathrm{w}}\mathrm{q}_{\mathrm{b}|\mathrm{end}}$$

(14)

4. Experimental and Results Analysis

This experiment aims to validate the feasibility and effectiveness of a UAV autonomous recognition and approach method for linear target splicing sleeves, integrating deep learning and active stereo vision. The main components include experiments on UAV recognition and localization of splicing sleeves on overhead transmission lines, as well as a series of UAV-ASS visual simulation experiments.

4.1. UAV Recognition and Localization Experiments for Splicing Sleeves on Overhead Transmission Lines

The UAV recognition and localization experiments for splicing sleeves on overhead transmission lines include the construction of the experimental dataset D_SS, the RB-YOLOv8n(OBB) rotational object detection rapid localization, as well as the LC rotational object detection fine localization.

4.1.1. Construction of the Experimental Dataset D_SS

In UAV inspection tasks, traditional data collection methods require substantial human and material resources, especially when capturing images of splicing sleeves installed on high-altitude transmission lines, which entails high costs and safety risks. Moreover, the complex and variable weather conditions and terrain make it extremely challenging to acquire sufficient and representative data in real-world scenarios [35]. To address the issue of limited samples in the real splicing sleeve dataset D_real, this study combines the real dataset D_real, a virtual reality scene dataset D_vr, and data augmentation techniques to construct a splicing sleeve dataset capable of meeting diverse scene requirements.

In this study, the large-scale 3D modeling software Blender 3.6 was used to create a virtual transmission line environment (including splicing sleeves) featuring typical scenarios such as farmland, villages, and forests. By varying perspectives, depths of field, low visibility (e.g., fog), and high noise conditions, diverse virtual reality scenes were generated, and the built-in camera tool in the software was employed to capture the scene data (as shown in Figure 11). This virtual scene augmentation method provides controllability and diversity of data samples, allowing for the simulation of various extreme conditions. It not only achieves low-cost and efficient data acquisition but also compensates for the high cost and difficulty associated with collecting complex scene data in real environments. Furthermore, it significantly enhances the robustness and generalization capabilities of the model. Through virtual scene augmentation, a total of 13,500 splicing sleeve image samples were generated in this study.

Moreover, the real splicing sleeve dataset, D_real, consists of 200 aerial images of 10 splicing sleeves from 6 high-voltage 220 kV power transmission lines in 3 regions of southern China. Based on the D_real and D_vr, data augmentation techniques such as color enhancement/weakening, contrast enhancement/weakening, perspective transformation, distortion, elastic deformation, and scaling [36,37,38] were applied to create the dataset D_au, with a total of 7900 images. The purpose of data augmentation is to simulate image distortions such as warping and blurring caused by UAV vibrations during actual aerial photography, enabling the splicing sleeve deep learning model to better adapt to real-world applications. If DA represents the data augmentation function, the augmented dataset D_au is expressed as:

D_au = DA(D_real + D_vr)

(15)

By combining D_real, D_vr, and D_au, a dataset D_SS with 21,600 images was constructed. During model training, D_SS was divided into training, validation, and test sets in a 7:2:1 ratio.

4.1.2. RB-YOLOv8n(OBB) Rotational Object Detection Rapid Localization Experiment

This section describes the rapid localization performance of RB-YOLOv8n(OBB) using the D_SS dataset relative to 12 mainstream rotational object detection algorithms, such as YOLOv8n/s/m/l/x-OBB, STD+HIViT-B [39], LSKNet-S* [40], RTMDet-R-I [41], KLD+R3Det [42], GWD+R3Det [43], S2anet [44], and Oriented_rcnn [45]. The key performance metrics used for comparison include mean average precision (mAP0.5), inference time per image (Spend, ms/img), inference speed (FPS), and model size (ModelSize).

All models were trained in the deep learning framework PyTorch under the Ubuntu 20.04.6 LTS operating system, using hardware that includes an NVIDIA GeForce RTX 3090 GPU, an AMD Ryzen 9 5950X 16-core 3.40 GHz processor, and 64.0 GB of memory. When evaluating the inference speed (FPS) for images of size 848 × 480, the GPU is pre-warmed before calculating the inference time.

Table 3. Comparison of Key Performance Indicators for Different Models on the Splicing Sleeve Dataset.

Rotating Target Detection Model	Precision (%)	Recall (%)	mAP0.5 (%)	Spend (ms/img)	FPS (f/s)	Model Size (MB)
Ours	96.7	95.7	96.4	11.57	86.41	9.4
YOLOv8n-OBB [46]	98.8	88.2	94.5	12.17	82.17	6.6
YOLOv8s-OBB [46]	95.8	87.8	95.4	13.35	74.91	23.3
YOLOv8m-OBB [46]	96.2	92.5	96.2	13.47	74.24	53.3
YOLOv8l-OBB [46]	98.4	93.9	97.2	15.54	64.35	89.5
YOLOv8x-OBB [46]	98.8	94.6	98.7	23.58	42.41	139.5
KLD+R3Det t [42]	95.8	93.5	96.3	50.92	19.64	315.3
GWD+R3Det [43]	95.2	90.1	95.9	37.04	27.00	309.5
S2A-Net [44]	94.3	93	95.3	37.17	26.90	309.6
Oriented_RCNN [45]	98.3	92.6	97.4	38.14	26.22	330.3
STD+HIViT-B [39]	99.0	98.5	99.1	28.41	35.20	290.5
LSKNet-S* [40]	97.2	96.5	98.9	27.70	36.10	237.6
RTMDet-R-I [41]	93.4	95.2	98.4	39.97	25.02	350.1

presents detailed values of precision, recall, mean average precision (mAP0.5%), inference time per image (Spend, ms/img), inference speed (FPS, f/s), and model size (Model Size, MB) for various models. Figure 12 provides a visual comparison of model size, mAP, and FPS through a two-dimensional scatter plot. As shown in Table 3 and Figure 12, our model (RB-YOLOv8n(OBB)) achieves an mAP0.5 of 96.4%, an inference time (Spend) of 11.57 ms, and a processing speed of 86.41 FPS. Compared to the original YOLOv8n-OBB model, although the model size increased by 2.8 MB, mAP0.5 increased by 2% inference time decreased by 0.6 ms, and FPS improved by 4.24 fps. This indicates that in RB-YOLOv8n(OBB), replacing the original cf2 module with the RepBlock module, and using different network structures for training and inference, where multiple branches are reparameterized into a single branch during inference, leads to superior mAP0.5, Spend, and FPS performance.

The results in Table 3 and Figure 12 also show that comparison with other models in the YOLOv8-OBB series (such as YOLOv8s, YOLOv8m, YOLOv8l, and YOLOv8x), RB-YOLOv8n(OBB) demonstrates significant advantages in terms of Spend, FPS, and Model Size. Although the mAP0.5 of YOLOv8m-OBB is 96.2%, which is very close to the 96.4% achieved by RB-YOLOv8n(OBB), the Model Size of YOLOv8m-OBB is 53.3 MB, much larger than the 9.4 MB of RB-YOLOv8n(OBB). Although the mAP0.5 values are comparable, RB-YOLOv8n(OBB) exhibits much better results in terms of Spend, FPS, and Model Size.

Compared to other models such as KLD+R3Det, GWD+R3Det, S2A-Net, Oriented_RCNN, STD+HIViT-B, LSKNet-S*, and RTMDet-R-I, the RB-YOLOv8n(OBB) model has a similar mAP0.5, ranking at a medium level. However, in terms of Spend, FPS, and Model Size, RB-YOLOv8n(OBB) significantly outperforms these models, as shown in Figure 12b.

To verify the practical effectiveness and general applicability of the RB-YOLOv8n(OBB) model, performance tests were conducted in various hazy static, real/simulated static, and UAV aerial dynamic scenarios. Figure 13 shows the recognition results of RB-YOLOv8n(OBB) in various environments, with (a)–(c), (d)–(f), and (g)–(i) corresponding to the hazy static, real/virtual, and UAV aerial dynamic scenarios, respectively. The dynamic recognition results are shown in the Supplementary Video S0. The results demonstrate that RB-YOLOv8n(OBB) has high general applicability and allows for effective recognition of splicing sleeves in various scenarios.

In summary, by replacing the original cf2 module with the RepBlock module and using reparameterization to convert the multi-branch network structure into a single-branch structure, RB-YOLOv8n(OBB) achieved performance improvements: inference speed reached 86.41 (f/s), mean average precision (mAP0.5) was 96.4%, and the model size was only 9.4 MB. These results fully demonstrate the effectiveness of the improvements to YOLOv8n(OBB). RB-YOLOv8n(OBB) strikes an excellent balance between compact model size and outstanding inference speed, making it particularly suitable for real-time applications.

4.1.3. LC Rotational Object Detection Fine Localization Experiment

To quantitatively analyze the differences between the rapid localization method for rotating targets and the precise localization method incorporating local clustering, we collected 500 sets of target images containing RGB and depth images using a depth camera under various angles, distances, depths of field, and scene conditions in both a real laboratory environment and a virtual scene created in Blender 3.6.

The position of the splicing sleeve is represented by five parameters: x, y, w, h, and angle. The splicing sleeves in 500 RGB images were annotated with nominal values (Nom) using the rotational object annotation software roLabelImg (version 3.0). The RB-YOLOv8n(OBB) model could predict the rapid localization of the splicing sleeves in the RGB images, denoted as B_{C_RGB}. Subsequently, the LC rotational object detection fine localization algorithm was applied to build the LC-RB-YOLOv8n(OBB) method, providing the precise position of the splicing sleeves in the RGB images, denoted as B_{F_RGB}. Figure 14 illustrates the positions of Nom, B_{C_RGB}, and B_{F_RGB} for the same target.

In this study, four evaluation metrics were used to measure the differences between Nom, B_{C_RGB}, and B_{F_RGB}: mean absolute error (MAE), mean relative error (MRE), root mean squared error (RMSE), and Spearman’s rank correlation coefficient ρ. The Spearman correlation coefficient ρ is determined by the following formula:

$$\rho =1-{\displaystyle \frac{6{\displaystyle \sum _{i=1}^n{d}_i^2}}{n\left(n^2-1\right)}}$$

(16)

where d_i is the rank difference of each parameter in the data points, and n = 500.

Smaller values of MAE, MRE, and RMSE indicate a smaller difference between the localization results and Nom, reflecting higher localization accuracy. When ρ approaches 1, it suggests a higher consistency between the localization method and the Nom annotation method.

Table 4. Comparison Table of B_{C_RGB}, B_{F_RGB}, and Nom Difference Indicators for 500 Data Sets.

Model		BC_RGB				BF_RGB
	Els	MAE	MRE	RMSE	ρ	MAE	MRE	RMSE	ρ
Parameters
x		40.55	9.05%	51.25	0.95	15.28	3.39%	19.47	0.99
y		40.38	9.34%	51.58	0.95	16.07	3.65%	19.88	0.99
w		23.66	12.99%	28.77	0.85	7.49	4.21%	9.51	0.98
h		2.47	12.7%	3.06	0.66	0.77	3.96%	0.95	0.95
angle		7.94	12.97%	9.86	0.83	2.33	3.83%	2.97	0.98

provides a comparison of difference metrics for 500 sets of B_{C_RGB}, B_{F_RGB}, and Nom, detailing the differences across four evaluation metrics (Els) for the five coordinate parameters x, y, w, h, and angle. Figure 15 visually illustrates the differences in MAE, MRE, RMSE, and ρ between the two methods.

As shown in Table 4 and Figure 15, the localization results of B_{F_RGB} using LC-RB-YOLOv8n(OBB) demonstrate superior performance across four evaluation metrics. For the five coordinate parameters x, y, w, h, and angle, the MAE values for B_{F_RGB} decreased from 40.55, 40.38, 23.66, 2.47, and 7.94 to 15.28, 16.07, 7.49, 0.77, and 2.33, respectively, showing a substantial improvement in localization precision. The MRE values for B_{F_RGB} decreased significantly, from 9.05%, 9.34%, 12.99%, 12.7%, and 12.97% to 3.39%, 3.65%, 4.21%, 3.96%, and 3.83%, respectively, thereby improving localization accuracy. The RMSE values of B_{F_RGB} for x, y, w, h, and angle ranged from 0.95 to 19.88, achieving relative improvements over the RMSE values of B_{C_RGB} by 62.01%, 61.46%, 66.94%, 68.95%, and 69.88%, respectively. This indicates that the introduction of the clustering-based LC rotational object detection refinement algorithm into RB-YOLOv8n(OBB) significantly enhances the stability of the x, y, w, h, and angle parameters. Additionally, Figure 15d shows a radar plot illustrating improvements in the correlation coefficients for B_{F_RGB}, with increases of 0.04, 0.04, 0.13, 0.26, and 0.15 for the parameters x, y, w, h, and angle, reaching values of 0.99, 0.99, 0.98, 0.95, and 0.98, respectively. These results indicate that LC-RB-YOLOv8n(OBB) achieves high consistency with manual annotation in rotating target localization.

Figure 16 shows the positioning results of B_{C_RGB}, B_{F_RGB}, and B_{F_Depth}. Panels (a)–(c), as well as (d)–(f), represent the B_{C_RGB}, B_{F_RGB}, and the B_{F_Depth} results when the distance between the UAV and the splicing sleeve is 4.8 m and 1.2 m, respectively. Panels (g)–(i) display the positioning results of B_{C_RGB}, B_{F_RGB}, and B_{F_Depth} using an Intel D455 depth camera in a laboratory environment. The figures clearly show that the B_{F_RGB} bounding box better fits the splicing sleeve, with significant improvements in the accuracy of angle, width, and height localization.

The experimental results indicate that the LC-RB-YOLOv8n(OBB) method significantly outperforms RB-YOLOv8n(OBB) in terms of localization accuracy and consistency with annotations. The optimization algorithm based on local clustering significantly reduces MAE, MRE, and RMSE values for rotational targets, improving the stability of angle and size parameters. This improvement is crucial for high-precision UAV docking tasks, providing a solid foundation for accurate docking of linear targets.

4.2. UAV-ASS Visual Simulation Experiment Series

The experimental results discussed above verified that the LC-RB-YOLOv8n(OBB) model effectively recognizes and locates splicing sleeves. However, after recognition, relative positional information is still needed to guide the UAV-mounted equipment for DR inspection of the splicing sleeve. The UAV-ASS visual simulation series includes the construction of the UAV-ASS visual simulation platform, studies on the effect of UAV rotation on D_UAV-SS, and studies of the UAV-ASS waypoint planning and docking.

4.2.1. UAV-ASS Visual Simulation Platform

This study developed the UAV-ASS visual simulation system to accelerate the validation of the scientifically founded and effective UAV-based recognition and approach algorithms, which are based on LC-RB-YOLOv8n(OBB). Figure 17 shows the structural diagram of the UAV-ASS visual simulation system built on the Ubuntu 20.04.6 LTS platform. The system consists of several units, including the splicing sleeve visual recognition and positioning module LC-RB-YOLOv8n(OBB), the UAV control system, the communication layer, and the GAZEBO simulation module. In the visual recognition and positioning module, nodes are used to read RGB images and depth maps from the depth camera. The LC-RB-YOLOv8n(OBB) is invoked to identify and locate the splicing sleeve. The results of the recognition and positioning are then transferred through a topic communication mechanism. The UAV control system unit, using the open-source PX4 flight controller, implements functions such as UAV altitude, position, and speed control, while also adding coordinate transformation and waypoint planning. The GAZEBO simulator, with its high-fidelity physical simulation capabilities, integrates the UAV and environmental models. The environmental models were created using Blender 3.6 to generate realistic 3D scenes of high-voltage power lines and splicing sleeves. The communication layer enables data transmission between ROS and the UAV, and the control command transmission according to the MAVLink and MAVROS protocol.

Figure 18 shows the interface of the UAV-ASS visual simulation system. Figure 18a displays the main interface of the UAV-ASS simulation. Figure 18b,c show the resulting RGB images and the depth map of the UAV’s visual recognition and localization of the splicing sleeve, respectively. The output results include x_OF, y_OF, w_F, h_F, θ_F, D_UAV-SS. Please refer to the Supplementary Video S1, for more details.

4.2.2. UAV Fixed-Point Rotation Impact on D_UAV-SS Experiment

In the UAV-ASS visual simulation system, the UAV is set to maintain a fixed height above the splicing sleeve while rotating intermittently. RGB images and depth maps are captured at a rate of 20 frames per second over a given time, and two localization methods are used to obtain B_{C_RGB} (including B_{C_Depth}) and B_{F_RGB} (including B_{F_Depth}). The depth value D_UAV-SS is then calculated according to Equation (11). Figure 19 shows the changes in D_UAV-SS determined with the B_{C_Depth} (Figure 19a, and B_{F_Depth} (Figure 19b methods, with the horizontal axis representing time (s) and the vertical axis representing distance (m).

Figure 19b shows that the D_UAV-SS values obtained using the B_{C_Depth} method fluctuate significantly due to UAV rotation, which causes inaccurate fitting of the splicing sleeve’s bounding box by the B_{C_RGB} and B_{C_Depth} methods. In contrast, Figure 19c shows that the D_UAV-SS values obtained using the B_{F_Depth} method remain stable at 3.4 m, even when the UAV rotates. The B_{F_RGB} (including B_{F_Depth}) method accurately fits the splicing sleeve despite the UAV rotation. This indicates that the B_{F_RGB} method effectively mitigates the effects of UAV rotation, improving the stability of distance measurement. The Supplementary Video S2, provides more details.

This experiment further validated the robustness and stability of the LC-RB-YOLOv8n(OBB) method, particularly its potential applications in dynamic scenarios, providing a solid technical foundation for precise depth measurement of UAVs in complex rotational and dynamic environments.

4.2.3. UAV-ASS Waypoint Planning and Docking Experiment

According to the UAV-ASS approach planning described in Section 2.2, the splicing sleeve’s final position relative to the UAV’s body in the simulation system is set as ^bP_ss|_end = (0, 0, −1, 0), which serves as the criterion for determining successful docking. After the UAV identifies the splicing sleeve during the high-altitude search, the ROS system retrieves the UAV’s world coordinates at the starting and end points through coordinate transformation and generates flight waypoints according to Equation (14). The simulated process of final identification and docking is divided into three stages: hovering and identification, position and altitude adjustments.

Figure 20 shows video screenshots of the UAV’s body coordinate trajectory and the B_{F_RGB}-located splicing sleeve position (including position and depth) during the recognition and docking process. In frames 1 to 7, the UAV is in the position adjustment stage; from frame 8 onwards, it enters the altitude adjustment stage, and by frame 11, the UAV is hovering approximately 1.0 m above the splicing sleeve. The Supplementary Video S3, shows the detailed steps. Figure 21 illustrates the corresponding changes in the UAV’s position adjustment process during the approach and docking, as seen in Figure 20. During the position adjustment stage, the angle θ between the splicing sleeve and the UAV’s body axis (i.e., the UAV’s yaw angle) adjusts from 60° to 0°. At the final position, the UAV’s world coordinates are (14.5 m, −14.3 m, 12.2 m).

To verify the robustness of the approach algorithm, UAV recognition and approach to the splicing sleeve from different initial positions were studied. The UAV’s initial yaw angle was set to 30°, and the end recognition and docking algorithm was executed from four positions on the same plane above the splicing sleeve: (14.5 m, −12.5 m, 15.5 m), (15.5 m, −14.5 m, 15.5 m), (14.5 m, −16.5 m, 15.5 m), and (13.5 m, −14.5 m, 15.5 m). The world coordinates of the splicing sleeve’s center point in GAZEBO were set to (14.5 m, −14.5 m, 11.1 m), with the splicing sleeve’s axis aligned with the X-axis of the world coordinate system. Figure 22 shows the UAV’s four different initial positions, and

Table 5. Errors in Recognition and Approach to Splicing Sleeve from Different Starting Positions in Simulations.

	Location Error	X-Axis Error/m	Y-Axis Error/m	Z-Axis Error/m	Angle Error/°
Starting Position
① (14.5 m, −12.5 m, 15.5 m)		−0.05	+0.06	−0.07	2
② (15.5 m, −14.5 m, 15.5 m)		+0.03	+0.03	−0.06	3
③ (14.5 m, −16.5 m, 15.5 m)		−0.05	+0.06	+0.04	−4
④ (13.5 m, −14.5 m, 15.5 m)		+0.07	+0.05	−0.06	2
MAE		0.050	0.050	0.058	2.750

presents the resulting errors in the recognition and approach experimental data for different starting positions.

The Table 5 shows that the UAV, starting from four different positions, flew to the target with an average absolute error (MAE) of 0.050 m, 0.050 m, 0.058 m in the X-, Y-, and Z-axes, respectively, and an angular error of 2.75°. This is comparable to the X-axis and Y-axis positioning errors of 0.050 m and 0.0650 m described in Ma’s method [47] and Gong’s 5 Points method [8] for UAV autonomous aerial refueling using binocular depth cameras. However, Ma’s method and Gong’s 5 Points method did not mention errors in the Z-axis or angular direction. The angular error of 2.75° is acceptable for mechanical docking in real-world environments when using ⌒- or Λ-shaped clamps [47,48].

The experiment validated the robustness and applicability of the UAV-ASS way-point planning and docking algorithm under various initial conditions. The results demonstrated that the algorithm achieved high-precision localization along the X-, Y-, and Z-axes while providing accurate angular measurements, with angular errors within the acceptable range for mechanical docking. This study further highlights the practical potential of the UAV-ASS algorithm in complex and dynamic environments, offering essential technical support for precise recognition and docking of linear tar-gets in UAV missions.

5. Conclusions and Future Works

To achieve autonomous recognition and approach control of linear target splicing sleeves by unmanned aerial vehicles (UAVs), this study proposes an approach that integrates deep learning and active stereo vision for UAV recognition and approach of linear targets, with experimental validation providing the following insights:

(1)

An algorithm based on the LC-RB-YOLOv8n(OBB) framework was designed specifically for linear target splicing sleeves. This algorithm employs a two-stage localization strategy, achieving rapid and precise localization of splicing sleeves with high reliability. The recognition model within this algorithm attains an average detection accuracy (mAP0.5) of 96.4% and an image inference speed of 86.41 f/s, meeting the real-time and lightweight requirements for high-altitude UAV detection. Additionally, the algorithm integrates depth images of splicing sleeves and performs local clustering analysis of depth values to further enhance the accuracy and reliability of sleeve localization. This algorithm provides a valuable reference for UAV autonomous recognition and docking with linear targets.

(2)

Typical virtual scenarios of splicing sleeves containing linear targets were constructed using the 3D modeling software Blender 3.6 to expand the dataset. This approach addresses the high cost and safety risks associated with capturing images of high-altitude transmission line splicing sleeves. It also improves the robustness and generalization ability of the model, providing a valuable reference for constructing deep learning datasets for aerial key targets in the power industry.

(3)

Utilizing the open-source PX4 UAV flight control platform, the Robot Operating System (ROS), and the Gazebo physics simulation platform, a UAV-ASS visual simulation platform was developed to quickly validate high-risk algorithms for UAV autonomous recognition and approach of overhead transmission line splicing sleeves. This platform provides an efficient simulation validation tool for high-altitude UAV operations in the power industry, effectively reducing testing costs and safety risks.

In summary, these methods and insights provide essential support for the development of UAV autonomous recognition and docking technologies in complex environments. Future work will focus on the following directions: firstly, exploring vibration compensation and error correction methods, such as Kalman filtering or machine learning models; secondly, investigating the stability of the docking process by employing robust control and adaptive feedback mechanisms; thirdly, conducting in-depth research on 6D pose estimation for non-parallel splicing sleeves; and finally, addressing obstacle avoidance and path optimization during the approach phase, as well as studying approaches to splicing sleeves under live-line conditions.