6-DOF Bilateral Teleoperation Hybrid Control System for Power Distribution Live-Line Operation Robot

: In the master-slave heterogeneous teleoperation, the workspace of the slave manipulator is usually much larger than that of the master manipulator. This paper proposes a 6-DOF bilateral hybrid teleoperation control strategy to map the workspace of the manipulators without changing the operation accuracy. The hybrid control includes the admittance and force control based on the feedback of the force sensor at the end of the manipulator. The two control strategies switched autonomously through the positioning of the Sigma.7 handle in the workspace. Compared with the classic bilateral teleoperation control, it overcomes the limitation of pre-matching the workspace of the master and slave. When the tool contacts a rigid environment, the robot can make adaptive compensation through the admittance controller even if the operator has not responded. We conduct extensive experiments to evaluate the changes in displacement and velocity before and after the switching process and under different admittance controller parameters. Finally, teleoperation is applied to live-line operation in distribution networks. The experiment proved that the control strategy is more consistent with human operation habits and can improve assembly success rate and efficiency.


Introduction
The size of the global grid is growing, and existing transmission and distribution lines are aging. Accordingly, the maintenance and overhaul of power grid lines are becoming increasingly prominent. Live-line work is a method widely recognized by the power system to overhaul high-voltage electrical equipment without power failure, thus, ensuring uninterrupted power supply for users and improving social economic benefits. Workers are required to operate in an aerial environment under high-temperature and high-voltage conditions, making their jobs difficult and dangerous [1]. Therefore, it is necessary to develop a power distribution live-line operation robot (PDLOR) to replace workers in harsh environments to ensure personnel safety and improve work efficiency. In the past, many countries have carried out a number of studies on the key technologies of PDLORs [2,3].
Among them, teleoperation technology as a stable and reliable remote control method can adapt to simultaneously completing a variety of live-line work and can be flexibly adjusted in unstructured scenarios [4]. During teleoperation, an operator conducts remote observation and control on the ground [5][6][7]. Short delay, immense fieldangle visual feedback, and high tracking performance of a master-slave manipulator can significantly increase the transparency of a teleoperation system. The latest teleoperation technologies have combined visual feedback, force feedback, and tactile feedback to increase the transparency of teleoperation systems [8][9][10][11].
When the master hand and the remote manipulator have the same topology, joint angles are usually used for mapping control [12]. However, when the size of the master hand is much smaller than that of the slave manipulator, the displacement and speed on the end of the master are amplified for the slave, which makes the fine operation quite difficult. There are many mapping methods for heterogeneous teleoperation, such as joint-joint mapping, point-point mapping in the workspace, rate mapping, and hybrid mapping [13][14][15][16][17]. The main problems are the difference between the master and slave motion tracks and the difference between the master and slave workspaces [18]. Li et al. [19] proposed a master-slave heterogeneous mapping method based on link-attitude constraint. Gong et al. [20] captured the motion data of a human arm with a wearable motioncapturing system and matched it to a remote manipulator. Using an appropriate motionmatching teleoperation strategy can result in more consistency with human operation habits and can reduce the processes of learning and adaptation.
Live-line work tasks [21,22] can be divided into two categories according to the type of contact used when a robot works. One type is flexible contact between a tool and an object, such as the contact between a wire connection tool and a wire during power line connection work, which can be accomplished via visual recognition, positioning, and robot planning [23]. The other type is rigid contact, such as the process of assembling a peg hole between the copper connection terminal of a wire and a lightning arrester during arrester installation, a process that can only be completed under vision /force fusion [22].
During the whole operation process, a remote manipulator needs to complete wire grasping and installation, as well as arrester gripping and other operations. This requires a teleoperation system to control a remote manipulator's movement in a range that is far larger than that of the working space of the master robot; at the same time, it needs to conduct fine-scale shaft-hole assembly operations in contact with the rigid environment. Bilateral teleoperation control [24,25] is a solution for contact tasks. However, most of the prior research is focused on bilateral control under variable delay [26], which can be ignored for the close-distance (about 10 m) teleoperation of a PDLOR. We focus on bilateral control under the condition that the workspace of the master robot is much smaller than that of the remote manipulator in a master-slave heterogeneous teleoperation system. At the same time, when a tool contacts a rigid environment, the slave manipulator can show a certain degree of flexibility.
To map the motion of the master device to the remote manipulator and provide the operator with a natural operating experience, a virtual reality bilateral hybrid teleoperation control system is constructed in this study. The proposed control system consists of three parts: (1) Mapping between the master device and the remote device. In view of the inconsistent configurations of the master device and the remote manipulator, as well as the large difference in the size of the workspaces, a hybrid control strategy is designed to switch automatically according to the current position and rotation on the end of the master device. (2) Haptic rendering of the feedback force from the end-force sensor when a tool interacts with the environment, and haptic rendering of the virtual force generated by superimposing the virtual simulation space. This feedback enables the operator to perceive interactions between the remote robot and the environment while being guided by the virtual force field. (3) Real-time rendering of binocular vision from the robot visual perception system of the head-mounted display (HMD). During teleoperation training, this can be switched to the visualization of a virtual simulation environment for teleoperation skill training. (4) Through the integration of the above parts, a human-centered teleoperation system that conforms to human operating habits is constructed.

Estimation of Contact Forces
A 6-D force sensor is installed between the end of the manipulator and the tool. The value of the force sensor is affected by the weight of the tool and the posture of the manipulator. Thus, the sensor value was not equal to the environmental contact force. We need to balance the force and torque produced by the end tool under gravity [27].
Assuming that the initial value of the 6-D force sensor is  (1) and (2): , tW G can be solved with the following: , When ˆe xt  is equal to 0, combining (1), (2), and (3) yields the following: The variables and constants are separated: where I is a third-order unit matrix with unknowns concentrated in the rightmost matrix of (5).

Teleoperation Control Route
In this section, we introduce in detail the force feedback teleoperation framework combined with virtual reality. The teleoperation system built in this study was divided into two parts: the master control haptic device and the remote PDLOR. The master operating end was located on the ground and included a 7-DOF force feedback handle similar to Sigma.7, a wearable HTC VIVE display, and an industrial computer. The remote end consisted of a PDLOR with two UR10 manipulators installed at 45° inclinations from the main body, as shown in Figure 1. The end of the manipulator was equipped with an ATI 6-D force sensor that was connected with the work tool through a flange and a quickchange interface. The visual perception system was integrated on a post behind the robot and included a monocular camera and a zed Mini binocular stereo camera, which could observe video footage from different perspectives with the movement of a 2-D PTZ. Each end was equipped with an industrial computer that interacted with the sensors and actuators, which were connected to each other. The industrial computers used TCP/IP network protocol to communicate through a wireless bridge [28]. Figure 1. Framework diagram of a force feedback teleoperation system combined with virtual reality. On the left is the master control terminal, which is usually located in the ground cockpit. On the right is the PDLOR. When working, the PDLOR is loaded into the insulated working bucket of a bucket truck and raised to the overhead line for operation.

Admittance Teleoperation Control Mode
To be able to perform operations involving contact with the environment during teleoperation, we added an admittance control in the task space (world cartesian coordinate system) to the position-based bilateral control. This addition enabled the manipulator to exhibit elastic properties, such as a mass-spring-damper system with a small range of adaptive capabilities [29,30].
We defined M as the coordinate system of the master-end haptic device and H as the coordinate system of the human hand in contact with the master device, which was consistent with the pose of the master end.
. Therefore, the closed-loop kinetic equation in S can be obtained as follows:  , , ,  represent the tracking error values of the state parameters for the current feedback on the manipulator's end relative to the set reference value. Their specific expressions in S are as follows:  is used to convert the rotation matrix to a rotation vector.

Force-Based Speed Control Mode
The workspace of the master device was much smaller than that of the slave, and the two configurations were different. One method of matching the two workspaces is to enlarge the workspace of the master device and then match it with the remote robot. Another method is to perform teleoperation under the ratio of the workspace of the robotic arm to that of the master device, known as the master-slave mapping ratio. This ratio is equivalent to amplifying the displacement mapping and velocity mapping coefficients of teleoperation master-slave matching. Hence, it makes fine-scale local operation relatively difficult. To move or rotate in a large range without affecting the fine-scale local operation, we added a force-based speed control strategy, as shown in Figure 2. The specific description is as follows.  (13) and (14): is the rotation matrix of the force sensor in W , and 66 m k   is the mapping scale factor for displacement and rotation. R e is the accumulated deviation of the rotation matrix in the speed control after integrating d s over time and completing the exponential mapping of the rotation matrix using the Euler-Rodrigues equation: Since the rotation is based on the fixed coordinate system W , R e is multiplied to the left to obtain d R .

Force Feedback from Haptic Devices
The haptic device was a force feedback handle similar to Sigma.7, as shown in Figure  3. Joints 1 to 3 were parallel-type mechanisms of the Delta robot and determined the position of the end of the master hand. Joints 4 to 6 were series-type mechanisms, and the three joint axes intersected at one point, which determined the current posture of the haptic device. Therefore, the output position and pose of the haptic device were independent. The last joint was a control switch similar to a gripper, and we defined a virtual switch based on this joint to trigger the start of teleoperation.
where all forces are in the M coordinate system, and is the feedback coefficient of the environmental contact force.
Assuming that the initial point coordinate system of the haptic device is  (16): where , pv kk are the elastic coefficient and damping coefficient of the set potential field, respectively.
H P is the current position, and H V is the linear velocity of the haptic device. set r is the set maximum working radius of the admittance control; if it exceeds this range, it switches to speed control. Symbol  refers to the Euclidean norm. Through the above definition of the potential field, an effect similar to operating in an elastic sphere can be obtained. Touching the boundary area generates a feedback force toward the center of the sphere, indicating that the operator switched modes.
To better explain Equation (16), we simplify the workspace of the master hand to a two-dimensional plane, as shown in Figure 3b. The physical meaning of To sum up, Equation (16) mainly describes that the middle area is primarily affected by damping force, and the edge area is affected by potential field force plus damping force. Through the above definition of the potential field, an effect similar to operating in an elastic sphere can be obtained. Touching the boundary area generates a feedback force toward the center of the sphere, indicating that the operator switched modes. The construction of the virtual potential field in the rotational direction is shown in Figure 3. The torque direction potential field constraints are decomposed and established by the independent variables , xz  in the graph.
x  is the angle between Here,

( ) 3
Pro  is the vector after projecting the space vector to the specified plane, and 3 () Norm  is the normalization of the vector to a unit vector. r s determines the direction of r T , which is determined by (21) and (22)  , the teleoperation system is in the admittance control mode; otherwise, it is in the speed control mode.

Visual Feedback of Real-World and Virtual Simulation
When the HMD was used to observe the operation of the remote robot, the pan-andtilt camera on the robot switched to follow the movement of the helmet. The positioning of the HMD depended on the supporting fixed-position light tower. The PTZ camera on the robot was designed to have 2-Dof. We took only the pitch angle and yaw angle in the helmet attitude as inputs and converted them into corresponding motor angles before sending them to the PTZ for servo-motion. Through the PTZ camera follow-up observation system, the remote on-site environment could be observed in a large range without adding multiple cameras, which enhanced the immersion of the remote operation.
The camera used was a ZED mini binocular camera. The baseline length of the binoculars was 63 mm, which is similar human pupillary distance (55-70 mm). As the server, the remote computer completed the binocular image acquisition of the ZED mini, H.264 encoding compression, and transmission. The ground-end computer received the video as a client, transferred it to a Unity3D engine for rendering, and finally, output it to the HMD for display. The observation perspective included the virtual simulation robot and the real scene image from the perspective of the binocular camera, as shown in Figure 4. As shown in Figure 2, the control route of the robot is to control the virtual robot in the virtual model through haptic devices. Then, another thread of the external control program will read the changes of the virtual robot in real-time and map the motion to the physical robot. Therefore, the real robot follows the virtual robot for servo movement. When the operator only needed to train teleoperation skills, he could disconnect the mapping function, practice teleoperation control with the VR teleoperation system, and become familiar with the operation process of the RDLOR.

Experimental Evaluation
To demonstrate the effectiveness of this hybrid teleoperating system, we describe a series of experiments in this section and present the experimental results. The overall hardware and software configuration during the experiment was as follows.
The haptic device in the Sigma.7 configuration supported input and output of up to 4 kHz. The higher the frequency, the smoother the rendered forces. The remote arm was a UR10 manipulator, and the end tool was a Robotiq two-finger gripper. We used UR-Script to communicate with the manipulator through TCP network protocol. The maximum communication frequency was 125 Hz. The communication frequency of the force sensor setting was also 125 Hz, and a low-pass filter of 73 Hz was used simultaneously to improve the signal stability of the force sensor. Therefore, the frequency of the main control program was set to 125 Hz; if the frequency was too high, it had no obvious meaning.

End-Load Compensation
In this section, the method mentioned in Section 2 was used to calculate the load identification parameters of the sensor at the end of the manipulator and to verify its accuracy. First, we saved ˆs  and B E R in six different poses and solved for 0, , tS p  , and , tW G using the method described in Section 2. The results of the solution are shown in Table 1.  Figure 5. It can be clearly observed that, under the condition that ˆs  varied greatly, ˆe xt  tended toward 0, which was in line with the conclusion. Since the force sensor was affected by vibration and acceleration changes during the movement of the manipulator, ˆe xt  still had a certain range of fluctuation during movement. On this basis, we added a Kalman filter, as shown by the yellow line, to reduce the fluctuation of ˆe xt  and provide a more accurate environmental feedback force for the hybrid force control strategy. The average force deviation is 0.181 N, and the maximum force deviation is 0.513 N. The average torque deviation is 0.006 Nm, and the maximum torque deviation is 0.036 Nm.

Switching of Hybrid Control
To more clearly show the force on the end of the manipulator, as well as the changes in displacement and velocity during the control-switching process, we teleoperated the remote arm to touch the environment. The whole process consisted of transition from the admittance control mode to the speed control mode, finally, switching back to the admittance control mode. The results are shown in Figure 6. The key parameters used in the experiments were as follows. For the admittance control parameters, the displacement direction showed

Influence of Admittance Control Parameters
This section compares the response differences when the external force had the same step-change under different admittance control parameters. In the experiment, we set the environment contact virtual force in the z-axis direction of W on the force sensor, as shown by the yellow dotted line in Figure 7a

Application in PDLOR
The replacement of a lightning arrester in live-line work of a distribution network is an important task. In the installation of a lightning arrester, it is necessary to install wire with a copper connection terminal to the upper terminal on the lightning arrester, as shown in Figure 8. An upper terminal consists of a section of M10 stud with a maximum outer diameter of about 9.96 mm, and the through-hole of a copper terminal is about 10.5 mm. In addition, when a wire is clamped, there is a certain amount of stress between the wire and the gripper, making it very difficult for a robot to perform the assembly. When remote-operated peg-hole assembly is performed with only visual feedback, it can easily cause a robot to be overstressed, causing a protective stop. We completed the process of performing teleoperated perforation under the conditions of weak feedback, bilateral control, and hybrid control.
We compared teleoperated perforation processes under no force feedback, bilateral control, and hybrid control conditions. First, make the subjects familiar with the teleoperation mode, and then the experiment was conducted after training. Each subject carries out 60 assembly tasks in each teleoperation mode, a total of 180 times per person in three modes. A total of five healthy adults (four males and one female) participated in this study. A protective stop of the robot is considered as assembly failure. The results of the experiment are shown in Figure 9. Among them, the success rate of teleoperation assembly under hybrid control is the highest, which is 98%. The success rate of bilateral control was 96%, and the teleoperation without force feedback mode was the lowest, with a success rate of only 63%. In the assembly process, the contact force of powerless feedback is the largest. The hybrid control of contact force in X and Y direction is lower than that in bilateral control, which means that the assembly is smoother. The contact force in Z direction is similar, but the hybrid control is relatively more stable. From the analysis of the average assembly time, the hybrid control time is shorter and the efficiency is higher. Figure 9 shows the data for a typical successful assembly process under hybrid control conditions, mainly including the displacement data of the master-slave manipulator in W in the three directions of x, y, and z and the force data of the z-direction. The whole assembly process included the six stages of (a)−(e), as shown in Figure 10. In stages (a) and (b), since the other end of the wire was fixed, although the terminals were not in contact, there was a certain amount of stress between the gripper and the wire that changed with the movement of the gripper. Stage (c) was the alignment stage for the peghole assembly, and , dz F in W , which was generated by the virtual potential field, fluctuated roughly in the range of 6-7 N, as shown in Figure 10. At the same time, the x and y direction imparted a certain value of d F so that the end could be fine-tuned in the XOY plane until the peg-hole was aligned. In this stage, , sz X fluctuated in a range that was imperceptible to the naked eye, and the contact force , ez F also fluctuated in the range of 3−10 N. The starting time of stage (d) was 1 T , and the ending time was 2 T . In this stage, the copper connection terminal descended along the stud under the action of the desired force. There was relatively obvious displacement in the z-direction, and the main force was the stress of the wire. At 2 T , the copper terminal reached the bottom of the upper terminal of the arrester, and

Conclusions
We proposed a hybrid control method that took into account both large-scale teleoperation and fine-scale local operation with a master-slave teleoperation system when the workspace of the master hand was much smaller than that of the remote arm. We applied this control strategy to a distribution network live-line working robot so that the robot could perform tasks involving physical interaction with the operation target. Remote binocular visual feedback rendered through a VR HMD and haptic feedback from remote force sensors greatly reduced the transparency of the teleoperation system.
Through the parameter identification of the end load, we can accurately obtain the contact force between the tool and the environment. The maximum force and torque deviations are 0.513 N and 0.036 Nm, respectively. We analyze the switching process of the hybrid control strategy and test the influence of the change of admittance parameters on the remote system. The response of the system is consistent with the simulation results in Simulink. Through the phase difference analysis of the displacement curve, the dynamic tracking delay of the teleoperation system is about 240 ms.
The upper lead assembly was completed during the installation of an arrester, which is difficult in live-line operations on distribution networks. The experiment proves that the hybrid teleoperation control can effectively complete the shaft-hole assembly operation under less contact force compared with other methods. The average assembly time is the lowest, about 25 s. However, the method presented some limitations, which are summarized here. First, the calculation of additional force did not consider the influence of acceleration on the force sensor, and the estimation of additional force may cause a certain amount of error for a heavy tool. In addition, there are processes that cannot maintain a constant contact force during the switching process, such as processes in which the expected force at the switching boundary is 0 N, which can be optimized in the future. Finally, the PDLOR is transported to overhead lines by a bucket truck during actual operation, causing the robot itself to vibrate with a low frequency and a small amplitude. We plan to further verify the performance of the teleoperation system outdoors.