6-DOF Bilateral Teleoperation Hybrid Control System for Power Distribution Live-Line Operation Robot

Abstract

In the master-slave heterogeneous teleoperation, the workspace of the slave manipulator is usually much larger than that of the master manipulator. This paper proposes a 6-DOF bilateral hybrid teleoperation control strategy to map the workspace of the manipulators without changing the operation accuracy. The hybrid control includes the admittance and force control based on the feedback of the force sensor at the end of the manipulator. The two control strategies switched autonomously through the positioning of the Sigma.7 handle in the workspace. Compared with the classic bilateral teleoperation control, it overcomes the limitation of pre-matching the workspace of the master and slave. When the tool contacts a rigid environment, the robot can make adaptive compensation through the admittance controller even if the operator has not responded. We conduct extensive experiments to evaluate the changes in displacement and velocity before and after the switching process and under different admittance controller parameters. Finally, teleoperation is applied to live-line operation in distribution networks. The experiment proved that the control strategy is more consistent with human operation habits and can improve assembly success rate and efficiency.

1. Introduction

The size of the global grid is growing, and existing transmission and distribution lines are aging. Accordingly, the maintenance and overhaul of power grid lines are becoming increasingly prominent. Live-line work is a method widely recognized by the power system to overhaul high-voltage electrical equipment without power failure, thus, ensuring uninterrupted power supply for users and improving social economic benefits. Workers are required to operate in an aerial environment under high-temperature and high-voltage conditions, making their jobs difficult and dangerous [1]. Therefore, it is necessary to develop a power distribution live-line operation robot (PDLOR) to replace workers in harsh environments to ensure personnel safety and improve work efficiency. In the past, many countries have carried out a number of studies on the key technologies of PDLORs [2,3].

Among them, teleoperation technology as a stable and reliable remote control method can adapt to simultaneously completing a variety of live-line work and can be flexibly adjusted in unstructured scenarios [4]. During teleoperation, an operator conducts remote observation and control on the ground [5,6,7]. Short delay, immense field-angle visual feedback, and high tracking performance of a master-slave manipulator can significantly increase the transparency of a teleoperation system. The latest teleoperation technologies have combined visual feedback, force feedback, and tactile feedback to increase the transparency of teleoperation systems [8,9,10,11].

When the master hand and the remote manipulator have the same topology, joint angles are usually used for mapping control [12]. However, when the size of the master hand is much smaller than that of the slave manipulator, the displacement and speed on the end of the master are amplified for the slave, which makes the fine operation quite difficult. There are many mapping methods for heterogeneous teleoperation, such as joint–joint mapping, point–point mapping in the workspace, rate mapping, and hybrid mapping [13,14,15,16,17]. The main problems are the difference between the master and slave motion tracks and the difference between the master and slave workspaces [18]. Li et al. [19] proposed a master–slave heterogeneous mapping method based on link-attitude constraint. Gong et al. [20] captured the motion data of a human arm with a wearable motion-capturing system and matched it to a remote manipulator. Using an appropriate motion-matching teleoperation strategy can result in more consistency with human operation habits and can reduce the processes of learning and adaptation.

Live-line work tasks [21,22] can be divided into two categories according to the type of contact used when a robot works. One type is flexible contact between a tool and an object, such as the contact between a wire connection tool and a wire during power line connection work, which can be accomplished via visual recognition, positioning, and robot planning [23]. The other type is rigid contact, such as the process of assembling a peg hole between the copper connection terminal of a wire and a lightning arrester during arrester installation, a process that can only be completed under vision /force fusion [22].

During the whole operation process, a remote manipulator needs to complete wire grasping and installation, as well as arrester gripping and other operations. This requires a teleoperation system to control a remote manipulator’s movement in a range that is far larger than that of the working space of the master robot; at the same time, it needs to conduct fine-scale shaft-hole assembly operations in contact with the rigid environment. Bilateral teleoperation control [24,25] is a solution for contact tasks. However, most of the prior research is focused on bilateral control under variable delay [26], which can be ignored for the close-distance (about 10 m) teleoperation of a PDLOR. We focus on bilateral control under the condition that the workspace of the master robot is much smaller than that of the remote manipulator in a master–slave heterogeneous teleoperation system. At the same time, when a tool contacts a rigid environment, the slave manipulator can show a certain degree of flexibility.

To map the motion of the master device to the remote manipulator and provide the operator with a natural operating experience, a virtual reality bilateral hybrid teleoperation control system is constructed in this study. The proposed control system consists of three parts:

(1)

Mapping between the master device and the remote device. In view of the inconsistent configurations of the master device and the remote manipulator, as well as the large difference in the size of the workspaces, a hybrid control strategy is designed to switch automatically according to the current position and rotation on the end of the master device.

(2)

Haptic rendering of the feedback force from the end-force sensor when a tool interacts with the environment, and haptic rendering of the virtual force generated by superimposing the virtual simulation space. This feedback enables the operator to perceive interactions between the remote robot and the environment while being guided by the virtual force field.

(3)

Real-time rendering of binocular vision from the robot visual perception system of the head-mounted display (HMD). During teleoperation training, this can be switched to the visualization of a virtual simulation environment for teleoperation skill training.

(4)

Through the integration of the above parts, a human-centered teleoperation system that conforms to human operating habits is constructed.

2. Estimation of Contact Forces

A 6-D force sensor is installed between the end of the manipulator and the tool. The value of the force sensor is affected by the weight of the tool and the posture of the manipulator. Thus, the sensor value was not equal to the environmental contact force. We need to balance the force and torque produced by the end tool under gravity [27].

Assuming that the initial value of the 6-D force sensor is ${\widehat{\tau }}_0\in {ℝ}^6$ under the condition of no force, $F_0,M_0\in {ℝ}^3$ are the force and torque components, respectively. Similarly, the actual contact force between the tool and the environment is ${\widehat{\tau }}_{ext}\in {ℝ}^6$. The force and torque are $F_{ext},M_{ext}\in {ℝ}^3$, respectively. When the sensor is subjected to environmental contact force and tool gravity, the reading is ${\widehat{\tau }}_S\in {ℝ}^6$, and $F_S={\left[\begin{array}{ccc}{F}_{S,x}& F_{S,y}& F_{S,z}\end{array}\right]}^{\mathrm{{\rm T}}}$ $M_S={\left[\begin{array}{ccc}{T}_{S,x}& T_{S,y}& T_{S,z}\end{array}\right]}^{\mathrm{{\rm T}}}$ are the force and torque components of ${\widehat{\tau }}_S$, respectively. The relative coordinate of the center of gravity on the tool in the sensor coordinate system ${\mathcal{F}}_S$ is $p_{t,S}={\left[p_{t,x},p_{t,y},p_{t,z}\right]}^{\mathrm{{\rm T}}}$. The gravity of the tool in world coordinates ${\mathcal{F}}_W$ is $G_{t,W}={\left[0,0,g_{t,z}\right]}^{\mathrm{{\rm T}}}$. The relationship between the value of the force sensor, the attitude of the manipulator, and the gravity of the tool when there is ${\widehat{\tau }}_{ext}$ under ${\mathcal{F}}_S$ can then be obtained as shown in (1) and (2):

$${\widehat{\tau }}_{ext}=\left[\begin{array}{c}{F}_{ext}\\ M_{ext}\end{array}\right]=\left[\begin{array}{c}{F}_S-F_0-{(R_S^W)}^{\mathrm{{\rm T}}}{G}_{t,W}\\ M_S-M_0-K_1{(R_S^W)}^T{G}_{t,W}\end{array}\right]$$

(1)

$$K_1=\left[\begin{array}{ccc}0& -p_{t,z}& p_{t,y}\\ p_{t,z}& 0& -p_{t,x}\\ -p_{t,y}& p_{t,x}& 0\end{array}\right]$$

(2)

where $R_S^W=R_B^W{R}_E^B{R}_S^E$, $K_1$ is the antisymmetric matrix with respect to $p_{t,S}$, transforming the force into the corresponding torque. $R_B^W\in SO(3)$ is the rotation matrix of the base of the manipulator relative to ${\mathcal{F}}_W$. $R_E^B\in SO(3)$ is the rotation matrix of the end of the robot relative to the base, which changes with the posture of the manipulator. $R_S^E\in SO(3)$ is the rotation matrix of the sensor relative to the robot end of the manipulator. ${\widehat{\tau }}_0, p_{t,S}$, and $G_{t,W}$ can be solved with the following:

$$k_{t,S}=M_0-K_1{\mathcal{F}}_0$$

(3)

When ${\widehat{\tau }}_{ext}$ is equal to 0, combining (1), (2), and (3) yields the following:

$$M_S=K_1{\mathcal{F}}_S+k_{t,S}$$

(4)

The variables and constants are separated:

$$M_S=\left[\begin{array}{cc}{K}_2& I\end{array}\right]\left[\begin{array}{c}{p}_{t,S}\\ k_{t,S}\end{array}\right]$$

(5)

$$K_2=\left[\begin{array}{ccc}0& F_{S,z}& -F_{S,y}\\ -F_{S,z}& 0& F_{S,x}\\ F_{S,y}& -F_{S,x}& 0\end{array}\right]$$

(6)

where $I$ is a third-order unit matrix with unknowns concentrated in the rightmost matrix of (5). $p_{t,S}$ and $k_{t,S}$ are obtained by measuring $n (n\ge 3)$ groups $M_S,K_2$ under different attitudes and solving the overdetermined linear equation system using the least squares method. Similarly, (7) can be obtained by converting (1), and $G_{t,W},F_0$ can be obtained by solving multiple groups of $F_S,R_S^W$:

$$F_S=\left[\begin{array}{cc}{(R_S^W)}^{\mathrm{{\rm T}}}& I\end{array}\right]\left[\begin{array}{c}{G}_{t,W}\\ F_0\end{array}\right]$$

(7)

Finally, the solved parameters are input into (3) to calculate $M_0$. At this point, the identification of the initial values of the force sensor and load parameters of the robot end tool is complete. Through (1), the current external force ${\widehat{\tau }}_{ext}$ under different manipulator postures can be obtained.

3. Teleoperation System

3.1. Teleoperation Control Route

In this section, we introduce in detail the force feedback teleoperation framework combined with virtual reality. The teleoperation system built in this study was divided into two parts: the master control haptic device and the remote PDLOR. The master operating end was located on the ground and included a 7-DOF force feedback handle similar to Sigma.7, a wearable HTC VIVE display, and an industrial computer. The remote end consisted of a PDLOR with two UR10 manipulators installed at 45° inclinations from the main body, as shown in Figure 1. The end of the manipulator was equipped with an ATI 6-D force sensor that was connected with the work tool through a flange and a quick-change interface. The visual perception system was integrated on a post behind the robot and included a monocular camera and a zed Mini binocular stereo camera, which could observe video footage from different perspectives with the movement of a 2-D PTZ. Each end was equipped with an industrial computer that interacted with the sensors and actuators, which were connected to each other. The industrial computers used TCP/IP network protocol to communicate through a wireless bridge [28].

3.2. Teleoperation Hybrid Control

3.2.1. Admittance Teleoperation Control Mode

To be able to perform operations involving contact with the environment during teleoperation, we added an admittance control in the task space (world cartesian coordinate system) to the position-based bilateral control. This addition enabled the manipulator to exhibit elastic properties, such as a mass–spring–damper system with a small range of adaptive capabilities [29,30].

We defined ${\mathcal{F}}_M$ as the coordinate system of the master-end haptic device and ${\mathcal{F}}_H$ as the coordinate system of the human hand in contact with the master device, which was consistent with the pose of the master end. $p_S^B\in {ℝ}^3$ and $v_S^B\in {ℝ}^3$ are used to denote the position and velocity, respectively, of the robot arm end-sensor’s center $O_S$ in ${\mathcal{F}}_B$. The attitude and angular velocity of ${\mathcal{F}}_S$ relative to ${\mathcal{F}}_B$ are $R_S^B\in SO(3)$ and ${\omega }_S^B\in {ℝ}^3$, respectively. The attitude of ${\mathcal{F}}_B$ relative to ${\mathcal{F}}_W$ is $R_B^W\in SO(3)$. Therefore, the closed-loop kinetic equation in ${\mathcal{F}}_S$ can be obtained as follows:

$$M_A\left[\begin{array}{c}{\dot{v}}_S^S\\ {\dot{\omega }}_S^S\end{array}\right]+D_A\left[\begin{array}{c}{e}_v\\ e_{\omega }\end{array}\right]+K_A\left[\begin{array}{c}{e}_p\\ e_R\end{array}\right]={\widehat{\tau }}_{ext}$$

(8)

where virtual inertia $M_A\in {ℝ}^{6\times 6}$, virtual damping $D_A\in {ℝ}^{6\times 6}$, and virtual stiffness $K_A\in {ℝ}^{6\times 6}$ are the adjustment parameters of the controller. ${\widehat{\tau }}_{ext}\in {ℝ}^6$ is the interaction force between the tool and the environment in ${\mathcal{F}}_S$. $e_p,e_R,e_v,e_{\omega }$ represent the tracking error values of the state parameters for the current feedback on the manipulator’s end relative to the set reference value. Their specific expressions in ${\mathcal{F}}_S$ are as follows:

$$e_p={(R_S^B)}^{\mathrm{{\rm T}}}(p_S^B-p_{S,ref}^B)$$

(9)

$$e_R=Rodrigues({(R_S^B)}^{\mathrm{{\rm T}}}{R}_{S,ref}^B)$$

(10)

$$e_v={(R_S^B)}^{\mathrm{T}}(v_S^B-v_{S,ref}^B)$$

(11)

$$e_{\omega }={(R_S^B)}^{\mathrm{{\rm T}}}({\omega }_S^B-{\omega }_{S,ref}^B)$$

(12)

where $p_{S,ref}^B\in {ℝ}^3$ and $v_{S,ref}^B\in {ℝ}^3$ are the expected position and velocity, respectively, in ${\mathcal{F}}_B$. The attitude $R_{S,ref}^B\in SO(3)$ and angular velocity ${\omega }_{S,ref}^B\in {ℝ}^3$ are also in ${\mathcal{F}}_B$. $Rodrigues(\cdot )\in so(3)$ is used to convert the rotation matrix to a rotation vector.

3.2.2. Force-Based Speed Control Mode

The workspace of the master device was much smaller than that of the slave, and the two configurations were different. One method of matching the two workspaces is to enlarge the workspace of the master device and then match it with the remote robot. Another method is to perform teleoperation under the ratio of the workspace of the robotic arm to that of the master device, known as the master–slave mapping ratio. This ratio is equivalent to amplifying the displacement mapping and velocity mapping coefficients of teleoperation master–slave matching. Hence, it makes fine-scale local operation relatively difficult. To move or rotate in a large range without affecting the fine-scale local operation, we added a force-based speed control strategy, as shown in Figure 2. The specific description is as follows.

The input of the control strategy is the virtual potential field force ${\widehat{\tau }}_d\in {ℝ}^6$ generated by the 7-DOF haptic device, and the environmental contact force ${\widehat{\tau }}_{ext}$ is as calculated in Section 2. The outputs are the moving speed ${\dot{x}}_d\in {ℝ}^3$ of the end of the manipulator and the speed $s_d\in so(3)$ of the rotation vector, which are represented in ${\mathcal{F}}_W$. The relationship between the parameters is shown in (13) and (14):

$$\left[\begin{array}{c}{\dot{x}}_d\\ s_d\end{array}\right]=\left[\begin{array}{cc}{R}_S^W& 0_{3\times 3}\\ 0_{3\times 3}& R_S^W\end{array}\right]k_m\left[{\widehat{\tau }}_d-{\widehat{\tau }}_{ext}\right]$$

(13)

$$R_d=e_R{R}_E^W=Rodrigue{s}^{-1}({\displaystyle \int s_{d}dt)}{R}_E^W$$

(14)

where $R_S^W\in SO(3)$ is the rotation matrix of the force sensor in ${\mathcal{F}}_W$, and $k_m\in {ℝ}^{6\times 6}$ is the mapping scale factor for displacement and rotation. $e_R$ is the accumulated deviation of the rotation matrix in the speed control after integrating $s_d$ over time and completing the exponential mapping of the rotation matrix using the Euler–Rodrigues equation: $Rodrigue{s}^{-1}(\cdot ):so(3)\to SO(3)$. Since the rotation is based on the fixed coordinate system ${\mathcal{F}}_W$, $e_R$ is multiplied to the left to obtain $R_d$.

3.3. Force Feedback from Haptic Devices

The haptic device was a force feedback handle similar to Sigma.7, as shown in Figure 3. Joints 1 to 3 were parallel-type mechanisms of the Delta robot and determined the position of the end of the master hand. Joints 4 to 6 were series-type mechanisms, and the three joint axes intersected at one point, which determined the current posture of the haptic device. Therefore, the output position and pose of the haptic device were independent. The last joint was a control switch similar to a gripper, and we defined a virtual switch based on this joint to trigger the start of teleoperation.

Feedback force ${\widehat{\tau }}_H^M\in {ℝ}^6$ is composed of two parts: one is feedback after scaling the environmental contact force ${\widehat{\tau }}_{ext}^M$ and the other is feedback of the virtual potential field force ${\widehat{\tau }}_v^M\in {ℝ}^6$ on the end of the main device. The relationship between these parts is shown in (15):

$${\widehat{\tau }}_H^M=k_e{\widehat{\tau }}_{ext}^M+{\widehat{\tau }}_v^M$$

(15)

where all forces are in the ${\mathcal{F}}_M$ coordinate system, and $k_e\in {ℝ}^{6\times 6}$ is the feedback coefficient of the environmental contact force.

Assuming that the initial point coordinate system of the haptic device is ${\mathcal{F}}_C=\{O_C,x_C,y_C,z_C\}$, $O_C$ is the center point of the workspace of the haptic device. $x_C,y_C,z_C$ are the unit vectors of the three coordinate axes of ${\mathcal{F}}_C$ in ${\mathcal{F}}_M$, which can be directly obtained from the column vector of the rotation matrix $R_C^M$ of ${\mathcal{F}}_C$ relative to ${\mathcal{F}}_M$. The generation of virtual potential force in the translation direction is relatively simple. For $r_c=‖P_H-O_C‖-r_{set}$, the expression of the potential field force is shown as (16):

$$f_v=\left\{\begin{array}{c}-k_v{V}_H, r_c\le 0\\ -k_p{r}_c(P_H-O_C)/‖P_H-O_C‖-k_v{V}_H, r_c>0\end{array}\right.$$

(16)

where $k_p,k_v$ are the elastic coefficient and damping coefficient of the set potential field, respectively. $P_H$ is the current position, and $V_H$ is the linear velocity of the haptic device. $r_{set}$ is the set maximum working radius of the admittance control; if it exceeds this range, it switches to speed control. Symbol $‖\cdot ‖$ refers to the Euclidean norm. Through the above definition of the potential field, an effect similar to operating in an elastic sphere can be obtained. Touching the boundary area generates a feedback force toward the center of the sphere, indicating that the operator switched modes.

To better explain Equation (16), we simplify the workspace of the master hand to a two-dimensional plane, as shown in Figure 3b. The physical meaning of $-k_v{V}_H$ is damping force. It is opposite the $V_H$ direction. $-r_c(P_H-O_C)/‖P_H-O_C‖\hspace{0.33em}$ is actually $\overrightarrow{P_H{P}_E}$. To sum up, Equation (16) mainly describes that the middle area is primarily affected by damping force, and the edge area is affected by potential field force plus damping force. Through the above definition of the potential field, an effect similar to operating in an elastic sphere can be obtained. Touching the boundary area generates a feedback force toward the center of the sphere, indicating that the operator switched modes.

The construction of the virtual potential field in the rotational direction is shown in Figure 3. The torque direction potential field constraints are decomposed and established by the independent variables ${\theta }_x,{\theta }_z$ in the graph. ${\theta }_x$ is the angle between $-x_H$ and $-x_C$. $\mathrm{Pro}\left(z_C\right)$ is the projection of the z-axis of ${\mathcal{F}}_C$ on the YOZ plane of ${\mathcal{F}}_H$, and ${\theta }_z$ is the angle between $\mathrm{Pro}\left(z_C\right)$ and $z_H$. Assuming that the maximum cone angle that can be freely turned is ${\theta }_{x,set}$, the maximum roll angle is ${\theta }_{z,set}$.

$$\cos \left({\theta }_x\right)=x_H\cdot x_C/\left(\left|x_H\right|\left|x_C\right|\right)$$

(17)

$$\cos \left({\theta }_z\right)=z_H\cdot \mathrm{Pro}\left(z_C\right)/\left(\left|z_H\right|\left|\mathrm{Pro}\left(z_C\right)\right|\right)$$

(18)

$$T_c=\left\{\begin{array}{c}0, {\theta }_x\le {\theta }_{x,set}\\ k_x\left({\theta }_x-{\theta }_{x,set}\right)Norm(x_H\times x_C), {\theta }_x>{\theta }_{x,set}\end{array}\right.$$

(19)

$$T_r=\left\{\begin{array}{c}0, {\theta }_z\le {\theta }_{z,set}\\ k_z{s}_r\left({\theta }_z-{\theta }_{z,set}\right)Norm(x_C), {\theta }_z>{\theta }_{z,set}\end{array}\right.$$

(20)

Here, $\mathrm{Pro}\left(\cdot \right)\in {ℝ}^3$ is the vector after projecting the space vector to the specified plane, and $Norm(\cdot )\in {ℝ}^3$ is the normalization of the vector to a unit vector. $s_r$ determines the direction of $T_r$, which is determined by (21) and (22):

$$s_r=\left\{\begin{array}{c}1, \cos \left({\theta }_s\right)>0\\ -1, \cos \left({\theta }_s\right)\le 0\end{array}\right.$$

(21)

$$\cos \left({\theta }_s\right)=x_C\cdot \left(z_H\times \mathrm{Pro}\left(z_C\right)\right)/\left(\left|x_C\right|\left|z_H\times \mathrm{Pro}\left(z_C\right)\right|\right)$$

(22)

Therefore, the combined virtual potential field torque $T_v$ is as follows:

$$T_v=T_c+T_r$$

(23)

Thus far, the potential field force in the rotating space is established according to the values of ${\theta }_x$, ${\theta }_z$ and ${\theta }_{x,set}$, ${\theta }_{z,set}$. ${\theta }_{x,set}$, ${\theta }_{z,set}$ needs to be a suitable value based on the maximum rotational workspace of the haptic device. In rotation, when ${\theta }_x$ is less than ${\theta }_{x,set}$ and ${\theta }_z$ is less than ${\theta }_{z,set}$, the teleoperation system is in the admittance control mode; otherwise, it is in the speed control mode.

3.4. Visual Feedback of Real-World and Virtual Simulation

When the HMD was used to observe the operation of the remote robot, the pan-and-tilt camera on the robot switched to follow the movement of the helmet. The positioning of the HMD depended on the supporting fixed-position light tower. The PTZ camera on the robot was designed to have 2-Dof. We took only the pitch angle and yaw angle in the helmet attitude as inputs and converted them into corresponding motor angles before sending them to the PTZ for servo-motion. Through the PTZ camera follow-up observation system, the remote on-site environment could be observed in a large range without adding multiple cameras, which enhanced the immersion of the remote operation.

The camera used was a ZED mini binocular camera. The baseline length of the binoculars was 63 mm, which is similar human pupillary distance (55–70 mm). As the server, the remote computer completed the binocular image acquisition of the ZED mini, H.264 encoding compression, and transmission. The ground-end computer received the video as a client, transferred it to a Unity3D engine for rendering, and finally, output it to the HMD for display. The observation perspective included the virtual simulation robot and the real scene image from the perspective of the binocular camera, as shown in Figure 4. As shown in Figure 2, the control route of the robot is to control the virtual robot in the virtual model through haptic devices. Then, another thread of the external control program will read the changes of the virtual robot in real-time and map the motion to the physical robot. Therefore, the real robot follows the virtual robot for servo movement. When the operator only needed to train teleoperation skills, he could disconnect the mapping function, practice teleoperation control with the VR teleoperation system, and become familiar with the operation process of the RDLOR.

4. Experimental Evaluation

To demonstrate the effectiveness of this hybrid teleoperating system, we describe a series of experiments in this section and present the experimental results. The overall hardware and software configuration during the experiment was as follows.

The haptic device in the Sigma.7 configuration supported input and output of up to 4 kHz. The higher the frequency, the smoother the rendered forces. The remote arm was a UR10 manipulator, and the end tool was a Robotiq two-finger gripper. We used URScript to communicate with the manipulator through TCP network protocol. The maximum communication frequency was 125 Hz. The communication frequency of the force sensor setting was also 125 Hz, and a low-pass filter of 73 Hz was used simultaneously to improve the signal stability of the force sensor. Therefore, the frequency of the main control program was set to 125 Hz; if the frequency was too high, it had no obvious meaning.

4.1. End-Load Compensation

In this section, the method mentioned in Section 2 was used to calculate the load identification parameters of the sensor at the end of the manipulator and to verify its accuracy. First, we saved ${\widehat{\tau }}_s$ and $R_E^B$ in six different poses and solved for ${\widehat{\tau }}_0,p_{t,S}$, and $G_{t,W}$ using the method described in Section 2. The results of the solution are shown in

Table 1. Calculated Load identification parameters of the force sensor at the end of the manipulator.

$F_{0,x}$	$F_{0,y}$	$F_{0,z}$	$M_{0,x}$	$M_{0,y}$	$M_{0,z}$
2.7014	−2.8861	6.2768	0.0357	−0.2755	−0.0141
$p_{t,x}$	$p_{t,x}$	$p_{t,z}$	$g_{t,x}$	$g_{t,y}$	$g_{t,z}$
0.0007	0.0005	0.0553	−0.0858	0.0313	−10.9563

.

To verify the accuracy of the data, we recorded ${\widehat{\tau }}_s,R_E^B$ and calculated ${\widehat{\tau }}_{ext}$ while changing the pose of the manipulator. The results are shown in Figure 5. It can be clearly observed that, under the condition that ${\widehat{\tau }}_s$ varied greatly, ${\widehat{\tau }}_{ext}$ tended toward 0, which was in line with the conclusion. Since the force sensor was affected by vibration and acceleration changes during the movement of the manipulator, ${\widehat{\tau }}_{ext}$ still had a certain range of fluctuation during movement. On this basis, we added a Kalman filter, as shown by the yellow line, to reduce the fluctuation of ${\widehat{\tau }}_{ext}$ and provide a more accurate environmental feedback force for the hybrid force control strategy. The average force deviation is 0.181 N, and the maximum force deviation is 0.513 N. The average torque deviation is 0.006 Nm, and the maximum torque deviation is 0.036 Nm.

4.2. Switching of Hybrid Control

To more clearly show the force on the end of the manipulator, as well as the changes in displacement and velocity during the control-switching process, we teleoperated the remote arm to touch the environment. The whole process consisted of transition from the admittance control mode to the speed control mode, finally, switching back to the admittance control mode. The results are shown in Figure 6.

The key parameters used in the experiments were as follows. For the admittance control parameters, the displacement direction showed $M_A=1$, $D_A=100$, and $K_A=300$, and the rotation direction showed $M_A=1$, $D_A=100$, and $K_A=300$. For the parameters of the speed control, the conversion factor of force and linear velocity was $1/300$, and the conversion factor of torque and the rotation vector was $1/10$. For the virtual potential field parameters, $k_p=500$, $k_v=10$, $k_x=2.0$, and $k_z=2.0$. Regarding the mapping coefficient $k_e$ of the environmental contact force in the haptic device, the coefficients of force and torque were 0.5 and 0.2, respectively.

For the admittance control stage without force, see Mode I in Figure 6a−c, where $F_{e,z}$ is essentially kept at 0. However, due to the movement of the manipulator, the sensor value fluctuated when the acceleration changed greatly.

In this phase, the displacement and rotation of the end of the remote manipulator followed the master handle, and the linear and angular velocities remained consistent, as shown in Figure 6b−e. After entering the speed control stage, the displacement and speed of the master hand and the remote arm became inconsistent, as shown in Mode II in a−c. In this mode, $V_{s,z}\propto \left(F_{d,z}-F_{e,z}\right)$, the movement stopped after $F_{d,z}=F_{e,z}$, and the environmental contact force $F_{e,z}$ at the end followed $F_{d,z}$. After moving in the opposite direction and switching back to the admittance control, the displacement difference and velocity of the master and remote ends were consistent again, as shown in the second section of Mode I in b and c. The difference between rotation and translation was that rotation was affected by $T_{e,z}$ in the admittance control stage. As the expected rotation vector $R_{m,rz}$ input by the master hand increased, the $R_{s,rz}$ of the remote arm also increased. However, due to the existence of $T_{e,z}$, there was deviation $e_R$ between the two, as shown in Mode I in d and e. When switching to the speed control, as shown in Mode II, since the desired torque $T_{d,z}$ on the switching boundary was 0 Nm, which was smaller than the current environmental torque $T_{e,z}$, reverse rotation occurred, and the contact torque $T_{e,z}$ decreased. As the desired torque $T_{d,z}$ increased or decreased, a corresponding rotation occurred from the end of the manipulator, and $T_{e,z}$ followed the change of $T_{d,z}$. The output force ${\widehat{\tau }}_m^W\in {ℝ}^6$ of the master haptic device was set to a maximum value. If the result of (15) exceeded the set maximum value, then the output force was output according to the maximum value of ${\widehat{\tau }}_m^W$, such as (d), at the maximum torque.

4.3. Influence of Admittance Control Parameters

This section compares the response differences when the external force had the same step-change under different admittance control parameters. In the experiment, we set the environment contact virtual force in the z-axis direction of ${\mathcal{F}}_W$ on the force sensor, as shown by the yellow dotted line in Figure 7a−c. The force and displacement of the end of the manipulator in the z-direction in ${\mathcal{F}}_W$ for different groups are shown in Figure 7d−f. The experiments were divided into three groups, each of which had only a single changed admittance control parameter. In Figure 7a−c, the desired extra force $F_{n,z}$ was consistent, but the actual force fluctuated due to changes in acceleration caused by the movement of the robotic arm. The greater the end-load and acceleration, the greater the fluctuation in the force sensor value. Figure 7d−f shows the response of the robot arm in the z-direction displacement of ${\mathcal{F}}_W$ under the above force step signal. A comparison of the dotted and solid lines shows that the designed admittance control system was consistent with the simulation results.

The phase difference shows that the execution delay of the robotic arm was about 240 ms. Under the premise of system stability, $K_A$ determined the offset of the displacement after stabilization. $D_A$ and $M_A$ affected the time to reach a steady state, that is, the magnitude of the acceleration. The larger $D_A$ and $M_A$ were, the longer the system took to reach a steady state. Together, $K_A$, $D_A$, and $M_A$ determined the stability of this second-order system. The response curve for displacement depended on whether the system was overdamped, underdamped, or critically damped, as shown in Figure 7f. It is noteworthy that since the controller in the experiment was a discrete control system realized at a frequency of 125 Hz, when the damping $D_A$ was large relative to $K_A$ and $M_A$, it caused instability of the system.

4.4. Application in PDLOR

The replacement of a lightning arrester in live-line work of a distribution network is an important task. In the installation of a lightning arrester, it is necessary to install wire with a copper connection terminal to the upper terminal on the lightning arrester, as shown in Figure 8. An upper terminal consists of a section of M10 stud with a maximum outer diameter of about 9.96 mm, and the through-hole of a copper terminal is about 10.5 mm. In addition, when a wire is clamped, there is a certain amount of stress between the wire and the gripper, making it very difficult for a robot to perform the assembly. When remote-operated peg-hole assembly is performed with only visual feedback, it can easily cause a robot to be overstressed, causing a protective stop.

We completed the process of performing teleoperated perforation under the conditions of weak feedback, bilateral control, and hybrid control.

We compared teleoperated perforation processes under no force feedback, bilateral control, and hybrid control conditions. First, make the subjects familiar with the teleoperation mode, and then the experiment was conducted after training. Each subject carries out 60 assembly tasks in each teleoperation mode, a total of 180 times per person in three modes. A total of five healthy adults (four males and one female) participated in this study. A protective stop of the robot is considered as assembly failure. The results of the experiment are shown in Figure 9. Among them, the success rate of teleoperation assembly under hybrid control is the highest, which is 98%. The success rate of bilateral control was 96%, and the teleoperation without force feedback mode was the lowest, with a success rate of only 63%. In the assembly process, the contact force of powerless feedback is the largest. The hybrid control of contact force in X and Y direction is lower than that in bilateral control, which means that the assembly is smoother. The contact force in Z direction is similar, but the hybrid control is relatively more stable. From the analysis of the average assembly time, the hybrid control time is shorter and the efficiency is higher. Figure 9 shows the data for a typical successful assembly process under hybrid control conditions, mainly including the displacement data of the master–slave manipulator in ${\mathcal{F}}_W$ in the three directions of x, y, and z and the force data of the z-direction.

The whole assembly process included the six stages of (a)−(e), as shown in Figure 10. In stages (a) and (b), since the other end of the wire was fixed, although the terminals were not in contact, there was a certain amount of stress between the gripper and the wire that changed with the movement of the gripper. Stage (c) was the alignment stage for the peg-hole assembly, and $F_{d,z}$ in ${\mathcal{F}}_W$, which was generated by the virtual potential field, fluctuated roughly in the range of 6–7 N, as shown in Figure 10. At the same time, the x and y direction imparted a certain value of $F_d$ so that the end could be fine-tuned in the $XOY$ plane until the peg-hole was aligned. In this stage, $X_{s,z}$ fluctuated in a range that was imperceptible to the naked eye, and the contact force $F_{e,z}$ also fluctuated in the range of 3−10 N. The starting time of stage (d) was $T_1$, and the ending time was $T_2$. In this stage, the copper connection terminal descended along the stud under the action of the desired force. There was relatively obvious displacement in the z-direction, and the main force was the stress of the wire. At $T_2$, the copper terminal reached the bottom of the upper terminal of the arrester, and $F_{e,z}$ increased instantly. During stage (e), $F_{d,z}$ decreased, and both $X_{s,z}$ and $F_{e,z}$ from the end of the manipulator tended to be stable. Stage (d) started the release of the gripper, and $F_{e,z}$ tended toward 0 N.

5. Conclusions

We proposed a hybrid control method that took into account both large-scale teleoperation and fine-scale local operation with a master–slave teleoperation system when the workspace of the master hand was much smaller than that of the remote arm. We applied this control strategy to a distribution network live-line working robot so that the robot could perform tasks involving physical interaction with the operation target. Remote binocular visual feedback rendered through a VR HMD and haptic feedback from remote force sensors greatly reduced the transparency of the teleoperation system.

Through the parameter identification of the end load, we can accurately obtain the contact force between the tool and the environment. The maximum force and torque deviations are 0.513 N and 0.036 Nm, respectively. We analyze the switching process of the hybrid control strategy and test the influence of the change of admittance parameters on the remote system. The response of the system is consistent with the simulation results in Simulink. Through the phase difference analysis of the displacement curve, the dynamic tracking delay of the teleoperation system is about 240 ms.

The upper lead assembly was completed during the installation of an arrester, which is difficult in live-line operations on distribution networks. The experiment proves that the hybrid teleoperation control can effectively complete the shaft-hole assembly operation under less contact force compared with other methods. The average assembly time is the lowest, about 25 s.

However, the method presented some limitations, which are summarized here. First, the calculation of additional force did not consider the influence of acceleration on the force sensor, and the estimation of additional force may cause a certain amount of error for a heavy tool. In addition, there are processes that cannot maintain a constant contact force during the switching process, such as processes in which the expected force at the switching boundary is 0 N, which can be optimized in the future. Finally, the PDLOR is transported to overhead lines by a bucket truck during actual operation, causing the robot itself to vibrate with a low frequency and a small amplitude. We plan to further verify the performance of the teleoperation system outdoors.