1. Introduction
Overhead electricity transmission and distribution lines are expanding due to increased human population and energy demand. Power lines can create high risks for a wide range of bird species, affecting their habitats in breeding, nesting, and wintering areas [
1]. Bird mortality caused by power lines is considerable in areas rich in avifauna for species with particular habits (e.g., eagle owl) or in situations favourable to the transit of migrants. Bird accidents have been estimated to affect about 350 bird species [
2] and bird mortality caused by power lines affects 1 million birds per year in the Netherlands, 175 million per year in the United States [
3], and around 1 billion per year worldwide [
4]. Apart from bird deaths, collisions with power lines also present a problem for power companies due to potential outages following collisions [
5]. For this reason, a set of devices known as bird diverters has been designed to make power lines visible to birds and keep them away.
Many bird diverters are available on the market. Some common bird diverters are depicted in
Figure 1 including the clip-type diverter (left) and the helical bird diverter (right). These devices can be mainly classified into active (
Figure 1 (left)) and passive (
Figure 1 (right) diverters. An active bird diverter is composed of a part moved by the wind. One group of passive bird diverters are helical objects made of plastic or aluminium elements that must be wrapped around the power cable (see
Figure 1).
Bird diverters are traditionally installed by highly specialized human operators, who reach the lines with aerial platforms, helicopters, or forklifts. These operations can be costly and dangerous because of the high heights (15–50 m). In addition, when the power line is not energized, operators can be in contact with electrified conductors. This requires additional safety protocols and conductive suits or insulated tools, further complicating the task.
In this work, we only consider the manipulation of helical bird diverters among the different kinds of diverters since their installation and removal processes are intrinsically complicated. This process can be split into the following steps.
Slip the larger end of the bird diverter onto the conductor or strand (
Figure 2a).
Rotate the bird diverter until the first helix lies naturally along the power line (
Figure 2b).
Wrap the securing helix around the power line (
Figure 2c).
In this context, to remove the diverter these steps must be executed in the opposite order.
Besides the installation process, bird diverters also need maintenance and periodic substitution. Bird diverter removal also presents difficulties. This paper proposes a novel mechatronic solution to carry out the removal process of helical bird diverters.
The envisioned solution considers an autonomous unmanned aerial vehicle (UAV) transporting a gripper that is designed to remove bird diverters. The UAV is teleoperated to move close to the operation site, where it can autonomously detect the helical bird diverter on the power line. This information is, in turn, used to detect the correct grasping point and start the automatic removal procedure. In this way, the human operator can safely stay on the ground while supervising this autonomous activity and bring the UAV back to land once the removal operation is concluded.
This work’s main contributions are the presentation of the designed prototype (
Figure 3) and the machine learning algorithm to detect the bird diverter’s presence and orientation to plan the gripper motion during the removal process. This paper shows the preliminary results in a laboratory environment considering the gripper mounted on a standard robotic arm, whereas the power-line grid is emulated through a proper mock-up. In future work, a UAV should be used to continue the development process.
The remainder of the paper is organized as follows.
Section 2 presents an overview of similar systems.
Section 3 discusses the proposed prototype. In
Section 4, the overall system architecture is presented and in
Section 5, the system is evaluated in a realistic industrial environment. Finally,
Section 6 presents the conclusions and future directions.
2. Related Works
Bird diverters are intended to make aerial lines visible to birds. To potentially improve effectiveness, they can have different colours that alternate on the three wires of the power line [
6] (see
Figure 1). Recently, the interest in aerial systems for installing such devices has grown. In general, aerial manipulators [
7] are used to perform inspection and maintenance tasks in difficult or dangerous sites, such as oil and gas pipelines [
8,
9,
10], chemical plants, or bridges [
11,
12], rather than the installation of devices, valve turning, and other maintenance operations [
13,
14,
15,
16]. Several platforms can transport massive payloads such as industrial arms or similar [
17,
18]. Related to our domain, the use of UAVs to preserve power grids has been discussed in different works. For example, in [
19], the authors present a system that is able to land on power lines using a LIDAR sensor. In contrast, in [
20], a pneumatic system is presented to deploy devices on power lines. Of course, considering the shape of helical bird diverters, the latter approach is unsuitable for the discussed application.
Another common type of diverter is based on a clip mechanism. The installation of this kind of bird diverter has already been tested in other works. In [
21], an aerial robot is used to install the clip-type device, whereas in [
22], a dual-arm lightweight system is presented to install helical diverters.
For helical bird diverter removal, a proper gripper mechanism must be designed. The literature on the gripper design and grasping mechanism varies from simple claw configurations [
23] to more complex prototypes [
24,
25]. A comprehensive review of these technologies can be found in [
26,
27]. However, two arms are needed due to the movements needed to manipulate the helical diverter (see
Figure 2). In this context, the most advanced technology is represented by the LiCAS A1 [
28] dual-arm manipulator, a lightweight and compliant dual-arm manipulator [
29] characterized by 4 degrees of freedom for each arm. This system has already been exploited to manipulate a helical diverter [
22]. However, with respect to the latter work, our solution is simpler since no complex trajectory must be planned. In addition, in the above-mentioned work, the authors do not present the entire installation process of the diverter but only the early stages, excluding the more complex part of the handling, which, as pointed out by the authors, may require modifications to the clamps.
The main difference between the above-mentioned and proposed approaches is that, in our system, the manipulation step requires only one grasping point placed on the extreme side of the helical diverter. This is made possible thanks to the capacity of the tool to perform continuous rotations around the electric cable to install or remove the diverter. Moreover, we tested the proposed device in a laboratory mock-up with a fixed industrial arm instead of a real aerial manipulator. However, the use of this type of system to perform aerial manipulation and human–robot interaction for the installation of bird diverters has already been investigated in [
30].
Finally, visual feedback is used to detect the helical tool to enable the autonomous execution of the manipulation task. In particular, the YOLO (You Only Look Once) V4 algorithm [
31,
32] was used to process the vision data received from an Intel Realsense depth sensor. YOLO is a widely used algorithm, that is used mainly for object detection. In this work, we properly train the YOLO algorithm to recognize the helical diverter.
3. Bird Diverter Manipulation Device
This section describes the mechanical design of the bird diverter manipulation tool.
Overall, the main idea behind the tool designed in this work is to use a single rotational mechanism to control the diverter and wrap the power cable using an integrated system. In this way, the complexity behind the manipulation of the helical diverter shown in
Figure 2 can be overcome. Any other Degrees of Freedom (DoFs) needed to finalize the manipulation task (i.e., the linear motion needed to flow over the power cable) can be achieved using an aerial system. The designed gripper consists of one DoF.
The mechanical description of the gripper is presented below. The proposed prototype is composed of 41 standard mechanical and electronic components. Eleven customized parts were designed and realized with 3D printing systems, using PET-G material to guarantee the device’s integrity during the manipulation task. The overall size of the prototype is () mm and its weight is about kg.
The device’s mechanics were designed on a worm-toothed wheel to guarantee high torques at low speeds and a guillotine element to hold the diverter in the toothed wheel. All the internal components of the manipulation device are stored in an external chassis composed of three main parts: one upper, one lower, and one rear. The first two pieces contain circular housing to store the toothed wheel. Similarly, a circular opening of 30 degrees allows the electric cable to be grasped by the device, as shown in the left of
Figure 4.
The internal housing of the device chassis was designed to store the motor and the driven shafts. Its assembly is depicted on the right of
Figure 4.
In this context, each shaft comprises a central part, which is 16 mm long, and two extremities, which are 12 mm long. The supports to lock the rotation of the worm screw and hold the pulley are located on these components. The first component that is fit into the shafts is the worm screw. Moreover, the two shafts have two opposite screws to allow two rotations, one right-handed and one left-handed. At their extremities, there are two spacers with diameters of 20 and 18 mm, respectively, followed by ball bearings with a dynamic load rating of 6800 N and a maximum rotation speed of 22,000 rpm. These are the unique elements that interlock with the inside of the container and anchor the shafts and other mechanical elements. The last spacer and the pulley are fit after the ball bearing. The pulley is made up of 25 teeth for belts of a maximum thickness of 15 mm. A cavity also forms the key between the shaft and the pulley. The linear joint connecting the 12 mm motor shaft to the 6 mm electric gear motor is the last element that is keyed on the motor shaft. The two shafts rotate at the same speed thanks to the “pulley-belt” mechanism positioned in the internal housing in the back part of the device. The strap has a circumference of 525 mm and comprises 105 teeth.
Since the distance between the two shafts is smaller than 525 mm, a PET-G belt tensioner with ball bearings was created to favour the correct functioning of the “belt-pulley” mechanism. Two worm screws parallel to the sides of the toothed wheel are used to ensure the complete rotation of the wheel even when the 30-degree opening corresponds with one of the screws. In this case, the rotation movement will be momentarily supported only by the other screw, as shown on the left of
Figure 5.
Furthermore, to facilitate the wheel’s movement, five ball rollers are inserted in each container, with an allowed load of 2 N each, which also perform the task of distancing the wheel from the containers, reducing the friction between the two moulded components in the PET-G.
The toothed wheel, with a diameter of 150 mm, is equipped with an opening of about 30 degrees to facilitate its placement on the cable to make the wheel’s rotation axis coincide with the axis of the cable. The internally hollow toothed wheel is divided into a container and a cover. It is externally characterized by teeth, the number and dimensions of which are calculated with reference, in particular, to the mechanical characteristics of the worm screw, that is, 64 teeth (see the right of
Figure 5). An additional opening was also made to facilitate the entry of the diverter into the wheel. This opening has an egg-shaped maximum size of
mm.
On one side of the wheel, there is a protrusion (
Figure 6), which performs the levering work of the diverter during the initial phase of the removal process. The force impressed on the diverter was estimated to be approximately 58 N, and the gear motor was calibrated accordingly, considering the gear wheel–worm gear ratio and the friction of the various components.
Inside the container is a mechanism called a “guillotine” which, through linear guides fixed with screws, is operated by a linear actuator. The vertical movement of the component locks the diverter into the device, ensuring a rigid grip. The linear actuator pulls the guillotine towards itself while the two arms on the left, by inertia, move inside the compartment, blocking the diverter (see
Figure 7). This mechanism is designed for diverters that do not exceed 12 mm in diameter. A protrusion was added to the wheel, which performs the lever work on the diverter during the rotation of the toothed wheel.
The rear casing is designed to incorporate electrical circuits that control the device. A protrusion is dedicated to the connection between the proposed device and the manipulator (i.e., a Kuka IIWA manipulator, in our case, see
Figure 8 left, but it can be adapted to drones in the future). External to this cover, there is also the housing for the gear motor driver and, on its opposite side, the space needed to fit the microcontroller devoted to the system control. The goal of the microcontroller is to actuate the gripper, considering the input provided by an external program. To this end, the controller is connected to a companion computer via a serial communication bridge and receives the commands to activate/deactivate the stepper and the motor drivers. More information about the control program is provided in
Section 4.
Finally, the gripper needs to be powered by a 12-volt power input to start the onboard microcontroller and actuate the internal mechanism. For this, a standard four-cell LiPo battery (which generates a nominal output voltage of 14.8 V (4 × 3.7 V) can be used to power both the avionics of the UAV and the gripper system. The complete prototype is shown on the right of
Figure 8.
This device was designed for diverters that can be installed on a cable with a maximum diameter of 11 mm. This limitation mainly depends on the size of the toothed wheel. However, this was sufficient to demonstrate the functionalities of the proposed prototype.
4. System Architecture
The overall system architecture is depicted in
Figure 9.
The system’s input is represented by the coloured image and the depth matrix generated by the Intel 435d depth camera. The data generated by the sensor are used by the Image Elaboration module that receives the coloured image matrix from the depth sensor and uses the YOLO library to identify the helical bird diverter. The YOLO algorithm uses neural networks to provide real-time and accurate object detection to identify an object and its position in an image. The YOLO algorithm uses convolutional neural networks (CNNs) to detect objects in real time, and the algorithm requires only a single forward propagation through a neural network to detect objects. This means that the prediction of the whole image is performed in a single execution of the algorithm. The Image Elaboration module additionally receives the depth matrix to convert the pixel points into 3D coordinates. To allow the identification of the grasping point on the diverter, the YOLO algorithm was trained to recognize the whole diverter (to assess its presence in the image) and two distinct parts on it, the LD (Line Diverter) back and LD front parts. The output of the trained system can be seen in the top image of
Figure 10.
In the proposed case study, additional information is extracted from the diverter, exploiting the output of the YOLO library. In particular, this work considers a case in which a diverter that is already placed on the power cable must be removed from the line. In this context, two additional pieces of information are needed: the grasping point of the diverter and its orientation on the cable. For this reason, the output of the YOLO algorithm (i.e., the detected object and the region of the image in which it is contained) is used to threshold the image in order to distinguish the background from the diverter (see bottom-right of
Figure 10). Then, the pixels belonging to the diverter are converted into 3D points and interpolated to obtain a line, depicted in pink in the bottom-left image of
Figure 10. Starting from this line, two pieces of information are extracted. First, the extremity of the tool (at the other end of the giant spirals) represents the point where the gripper must be positioned at the start of the removal task (
). This point is indicated by a yellow dot on the far left of the bottom-left image of
Figure 10. The other point indicates the orientation of the diverter, and it is necessary to calculate the correct orientation to insert the cable in the guillotine mechanism. Overall, these data are represented by the inclination of the interpolation line with respect to the three Euler angles. Finally, the steps between the last two spirals of the LD front part are detected for future usage (yellow and blue dots in the bottom-left image of
Figure 10).
Therefore, the output of the Image Elaboration module is represented by the objects detected in the image and their characteristic points (i.e., the grasping point and the diverter orientation). These data are sent to the Planner and Gripper Control module, which commands the manipulator and the gripper to accomplish the manipulation task. In this context, since the 3D coordinates of the salient points with respect to the depth camera are known, they can be converted into the manipulator base reference frame. With this information, the diverter grasping point on which the device acts is calculated. A combination of manipulator and gripper motions is used to remove the diverter, as described in
Section 5. Moreover, the interface between the gripper and the system is established via a serial interface, which allows for communication between the companion computer and the microcontroller installed inside the gripper. Here, information, such as the guillotine tool’s desired actuation and the gripper’s overall state, is exchanged.
Finally, to manipulate the bird diverter, the manipulator must be actuated. The robotic manipulator employed for the experimental evaluation of the prototype is an LBR Kuka iiwa manipulator. It has seven joints and coupling torque sensors integrated into all the axes. The Kuka iiwa manipulator is modular and programmed using Java. The robot control is implemented using Kuka FRI libraries to directly control the desired position of the robot joints, solving the direct and inverse kinematic problems on the companion computer. Since the considered application requires that the gripper centres the power line cable with its guillotine mechanism, a force controller was implemented. In this way, the estimation of the pose of the diverter and, consequently, the power cable is imprecise. Physical contact with the cable will drive the mechanism into the correct housing. For this reason, the Admittance Control module implements a Cartesian admittance controller [
33]. This controller is implemented considering the forces acting on the robot end effector. The dynamic relationship between the applied forces and the motion of the robot is established using the following admittance control scheme:
where
,
, and
are positive definite diagonal matrices representing the desired virtual inertia, damping, and stiffness, respectively. Following this formula, the output of the controller is the compliant position command
for the end effector of the robot, given a destination
. Here, the input of this module is the Cartesian position
selected by the Planner module and the estimated force acting on the end effector (
). This module is also in charge of solving the forward and inverse kinematic problems. In this context, this module receives the desired position in the operative space to convert them into the desired joint angles for the manipulator. At the same time, the pose of the end effector is calculated, starting from the current joint values of the arm. The core of this module is based on the well-known Kinematics and Dynamics Library (KDL).
5. Experiments
In this section, an autonomous bird diverter removal experiment is conducted to assess the effectiveness of the proposed prototype. Finally, the evaluation of the helical diverter detection system based on the YOLO algorithm is presented.
A sketch of the overall task is shown in
Figure 11. In this context, the trajectory planned and executed during the task is indicated by a blue line. The estimated pose of the diverter is shown in front of the industrial manipulator.
The detailed process was carried out by the actuation of the manipulator and gripper. It is worth noting that to install/remove the diverter correctly, two rotations are needed. In the first part, where the diameters of the spirals are small, a clockwise rotation is needed. In contrast, in the second part, where the spirals are large, the rotation needed to remove the diverter must be counterclockwise (see
Figure 2).
The removal process was split into the following steps, as shown in
Figure 12. After the object was detected on the scene, the cable was approached. The approaching point was selected as an elevated point over the grasping point and detected thanks to the image elaboration algorithm described in the previous section.
In this context, the gripper was positioned vertically on the cable. Then, the opening of the tool was inserted into the cable (
Figure 12a). At this point, the diverter was ready to be removed and the first rotation of the gear wheel of the gripper was performed (
Figure 12b). During the wheel rotation, a linear motion of the manipulator towards the large spirals was planned and executed, continuing the rotation of the gear wheel. Thanks to the admittance controller, if the helical diverter part was not removed yet, the manipulator held its position (
Figure 12c,d). Since the large part of the diverter was removed, the end effector was inclined with respect to the cable (
Figure 12e) and then the guillotine mechanism was closed to grip the diverter. However, at this point in the task, only the large spirals fit into the cable. So, a counterclockwise rotation was actuated on the toothed wheel (
Figure 12f,g). Finally, a vertical motion was executed to complete the task (
Figure 12h).
To move the manipulator, some waypoints were generated starting from the pose of the diverter detected by the depth sensor. Therefore, a trapezoidal trajectory was planned from the waypoints. During the execution of the trajectories, the admittance controller was properly tuned so that the gripper was not forced until the next part of the diverter was unlocked from the cable.
Figure 13 shows the pilot case study for removing the helical bird diverter. During the same experiment, we recorded the Cartesian position and orientation of the end effector (
Figure 13 and
Figure 14) and estimated the forces acting on the robotic gripper (
Figure 15). The cable was placed parallel to the x-axis of the robot’s reference frame in this experiment. Hence, the first motion of the robot was to reach the power line, as shown in the first graph in
Figure 13. Then, the second graph in
Figure 13 (the motion along the y-axis) shows the device’s progress over the cable. Similarly, the graphs in
Figure 14 show a single change in the orientation of the end effector during the final step of extracting the diverter. In this context, the yaw angle of the end effector did not change since it was already aligned with the final orientation of the task.
Finally, the forces involved in the removal process acting on the robot’s end effector are shown in
Figure 15. The force graphs are mainly characterized by peaks in contact with the cable and during some phases of the removal task, in particular, the removal of the widest spirals of the diverter.
Object Recognition Evaluation
To allow for the recognition of the helical diverter, the YOLO version 4 algorithm was adequately trained. Several pictures with different sizes and light conditions were taken in this context. In particular, 120 pictures were considered for the system training and the GoogleColab [
34] hardware accelerator was used. The diverter and its subparts were labelled in each image, namely the diverter, LD front, and LD back.
The trained network was tested on an adequately made test set composed of 400 images, in which
of the images showed the diverter placed on the floor or installed on power lines, whereas the other
of the images showed power line cables without bird diverters. To assess the performance of the recognition system, the accuracy percentage,
, was calculated considering the following formula:
where
is the true positive,
the true negative,
the false positive, and
the false negative. These values were calculated as the percentages of times that the detector correctly identified the diverter in an image. The results are shown in
Table 1, where the accuracy percentage was calculated for all the recognizable elements from the YOLO library.
The results reported in
Table 1 demonstrate that the vision system was able to detect both the whole diverter and its parts, allowing the control system to exploit the information about the diverter orientation (i.e., detect the grasping point of the diverter).