A Wearable Upper Limb Exoskeleton for Intuitive Teleoperation of Anthropomorphic Manipulators

Abstract

Teleoperation technology combines the strength and accuracy of robots with the perception and cognition abilities of human experts, allowing the robots to work as an avatar of the operator in dangerous environments. The motion compatibility and intuitiveness of the human–machine interface directly affect the quality of teleoperation. However, many motion capture methods require special working environments or need bulky mechanisms. In this research, we proposed a wearable, lightweight, and passive upper limb exoskeleton, which takes intuitiveness and human-machine compatibility as a major concern. The upper limb pose estimation and teleoperation mapping control methods based on the exoskeleton are also discussed. Experimental results showed that by the help of the upper limb exoskeleton, people can achieve most areas of the normal range of motion. The proposed mapping control methods were verified on a 14-DOF anthropomorphic manipulator and showed good performance in teleoperation tasks.

1. Introduction

Over the last few decades, robots have been replacing humans in many scenarios, relieving people of heavy physical labor and harmful jobs. Although the rapid evolution of artificial intelligence (AI) enables the robot to work in an autonomous fashion, there are still many situations in which robots need to be teleoperated by humans, such as working in unstructured environments or critical tasks in which failures are intolerable. Especially in dangerous scenarios such as chemical plants [1], disaster sites [2], explosives demolition [3], or space exploration [4], teleoperated robots are still the most reasonable and practical solution for working as remote avatars of human operators.

The human-machine interface plays an important role in teleoperation. The intuitiveness of the human-machine interface directly affects the transparency between the operator and the slave robot. Keyboards and joysticks are the most common human-machine interface, which are easy to use to provide simple commands in teleoperation tasks [5,6]. However, the degree of freedom (DOF) available from such devices is usually much smaller than the task space dimensions. So, they can be used for very limited slave devices, while the lack of DOFs also challenges the space perception of the operator. Haptic devices like PHANTOM [7] and Force Dimension SIGMA [8] are also frequently used as master devices in teleoperation, which can provide force feedback while measuring the operator’s motion commands. Although the physical feedback improves the interactivity in teleoperation, the range of motion for such devices is often very limited, which decreases the accuracy and intuitiveness of controlling human-scale slave devices.

The above methods only generate a control command for the end-effectors of the slave robots. However, for complex slave devices like anthropomorphic robots, the human–machine interface is required to provide more information than only the end-effector pose. In contrast, directly capturing the body motion is more convenient and intuitive for controlling the whole body of anthropomorphic robots, which is very friendly to nonprofessional users. Optical motion capture technology was used to control the NAO humanoid robot to mimic the human operator [9]. The RGBD cameras like Microsoft Kinect can extract the skeleton status of the human body and map it into robot-control commands [10,11]. Most optical or camera-based motion capture devices are sensitive to illumination conditions, which limits application to indoor environments. In addition, fixed installation of the camera or tracker is mandatory. The body pose can also be measured by attaching a set of IMU sensors and used as the master side control commands [12,13]. The IMU sensors are lightweight and easy to install, which can overcome the environmental limitations of camera and optical devices. However, the positions of installation on the body do not result in them being accurately identical every time. Therefore, calibration and correction algorithms are needed before use while the inevitable shifting over time problem of the inertial components makes it unsuitable for long time, high accuracy, and high-reliability teleoperations.

Exoskeletons have been widely used in rehabilitation [14,15] and motion assistance [16,17,18]. Recently, research studies on exoskeletons have also been extended to master devices for teleoperation. Many exoskeleton devices [19,20,21,22] have been developed to acquire upper limb postures and perform master-slave teleoperation. Compared with the previously discussed motion measurement methods, the exoskeleton does not require a special environment and installation. The rigid body structures also guarantee measurement accuracy and are calibration free. Force feedback features were applied in some previous studies [19,21], which improve the presence of the operator. Meanwhile, the motors, transmission mechanism, and high-capacity battery make the system too bulky to be worn by humans for a long time.

So, in this research, we proposed to develop a wearable, lightweight, and passive upper limb exoskeleton, which is comfortable to wear and provides an intuitive human-machine interface for the teleoperation control of anthropomorphic manipulators. A prototype of the developed exoskeleton is illustrated in Figure 1. We first analyzed the motion of the upper limbs and established a simplified 7-DOF kinematic model of the upper limb. Then we elaborated on the mechanism design and the data acquisition and transmission methods of the exoskeleton system. To overcome the challenges in making the shoulder mechanism compact and compatible with human motions, we developed a spherical scissor mechanism to mitigate the limitation in range of motion. For the teleoperation of the anthropomorphic manipulators, we devised joint space and task space mapping control strategies based on the proposed exoskeleton device. The pros and cons of the two methods are also discussed. Finally, we conducted a series of experiments to verify the human-machine compatibility of the exoskeleton mechanism and the performance of the mapping control methods.

The major contributions of this work can be summarized as follows.

• We present a complete solution for measuring precisely upper limb posture with a wearable exoskeleton device. Compared with existing works in [19,20,21,22], our exoskeleton can be steadily fixed on the torso by the curved back frame and carrying system, which provides self-alignment capabilities and guarantees measurement accuracy.
• We seek to make a balance between the complexity and human-machine compatibility of the device. A spherical scissor mechanism is proposed for the exoskeleton shoulder to maximize the device’s range of motion without making the system bulky. The overall mass of the device is only 4.8 kg, which is lighter than most existing similar devices.
• We provide both joint space and task space control strategies for performing teleoperation of anthropomorphic manipulators with the exoskeleton device. The flexible control strategies allow the exoskeleton to adapt to different types of slave devices and application requirements.

2. Design and Implementation

2.1. Upper Limb Motions and Modeling

The upper limb is the most dexterous part of the human body which can generate very complex motions. Master devices for teleoperation measure the upper limb status in real time, and the data can be used to recover the upper limb posture. So, a basic understanding of the motions of the upper limb and establishing a simplified model are essential.

The motions of the upper limb have been studied for a long time in human kinesiology fields. It is well acknowledged that the upper limb motions can be described with seven rotational DOFs [23,24], including three DOFs at the glenohumeral (GH) joint, one DOF at the elbow joint, three DOFs at the forearm and the wrist joints. The motion patterns are presented on the left side of Figure 2.

In research [26], ranges of the seven basic motions of normal people have been studied (

Table 1. Range of motion of the upper limb joints on the normal male subject.

Motion	Normal Range (deg)
Shoulder Flexion/Extension	158/53
Shoulder Abduction/Adduction	170/0
Shoulder Medial/Lateral	70/90
Elbow Flexion/Extension	146/0
Forearm pronation/supination	71/84
Wrist Abduction/Adduction	19/33
Wrist Flexion/Extension	73/71

), which are an important reference in determining the mechanical specifications of the exoskeleton. If the exoskeleton seriously reduced the range of motion in some DOFs, the wearer could feel uncomfortable and have difficulty in performing teleoperation.

According to the motion patterns and the positions of the rotational joints, we model the upper limb as a 7-DOF serial structure as depicted on the right side of Figure 2.

2.2. Shoulder Mechanism

Functionally, we use the simplified 3-DOF serial model to describe the main rotational motions (flexion/extension, medial/lateral rotation, and abduction/adduction) of the human GH joint.

The axes of the three rotational joints are perpendicular to each other and converge at the same point, which is located at the inside of the human arm. Aligning the central point of the rotational joints to the rotation center of the GH joint is significant for improving comfortability and accuracy for posture measurement. The shoulder mechanism connects the upper arm link to the base frame on the back. However, the rotational center of the human GH joint is surrounded by bones, muscles, and skin. The body tissues occupy too much working space, which makes it challenging to design the shoulder mechanism of the exoskeleton. Many previous research studies on upper limb exoskeletons made efforts to improve the comfortability of shoulder motions. One solution is to enlarge the working radius of the shoulder mechanism to keep it away from the human body. Another idea is to increase the DOFs to improve the matching degree for human motions. Research studies [22,27] propose using 5–6 DOFs to fit the shoulder motions. However, these methods lead to bulky and complex mechanisms, rising weight, costs, and difficulty in modeling and control.

The scissor mechanism is deployable in space and has been long used for the lifting mechanism. The scissor mechanism with curved linkages is a variant of traditional ones, which can be deployed and folded on the surface of a sphere. So, it is especially suitable for the exoskeleton shoulder mechanism, where compactness and lightweight are desired. The concept of a scissor mechanism with curved linkages has been used on the shoulder part of rehabilitation devices [28,29,30].

In this research, a 3-DOF spherical scissor mechanism was designed for the exoskeleton shoulder. The spherical scissor mechanism reduces space occupation and weight and also makes mobility compatible with the range of motion of normal people as listed in Table 1.

As dedepicted in Figure 3, the spherical scissor mechanism is composed of six curved linkages and seven common joints. The two longer curved linkages have twice the arc length of the shorter curved linkages. Three 17-bit absolute encoders (Netzer, DS-25, Misgav, Israel) are installed on joints J, K, and H to measure the rotational angles. The axes of all the joints pass through the same point at the sphere center, which is referred as the remote center of motion, or RCM. To make the spherical scissor mechanism properly compatible with the shoulder motions, the RCM of the joints should be configured near the rotational center of the GH joint of the human body as much as is possible. Every curved linkage of the mechanism is an arc between the joints at its two ends on the great circle. All the curved linkages have the same radius. The above conditions guarantee the spherical scissor assembly always moves on the surface of the sphere.

In the development of a practical prototype, the mechanical specifications should be determined. The radius of the mechanism working sphere should be compatible with the wearer’s physical measurements. By measuring the armhole size of a set of subjects, the radius of the sphere occupied by bones, muscles, and skin around the shoulder is estimated to be about 60 mm on average. Considering the possible translation of the GH joint center [31] and the thickness of the clothes, we set the radius of the mechanism working sphere as 100 mm.

The curvature angle of each linkage will affect the range of the medial/lateral motions. Ideally, the curvature angle of the fully folded and fully deployed mechanism should be zero and twice the curvature angle of the longer linkage respectively. However, because some of the space is occupied by bearings and encoders in a practical mechanism, the adjacent linkages cannot completely overlap in the above extreme conditions.

As depicted in Figure 4, the pitch angle is defined as the span of a single rhombus of the spherical scissor mechanism. When fully deployed (Figure 4), ${\theta }_S$ is limited by the collision of the bearings installed at joint A and joint B. The point I is defined as the intersection of the arc between AB and the equator of the sphere. When the bearings at joint A and joint B collide, the curvature angle from A to I is approximate to $r_b/r$, where $r_b$ is the radius of the bearings and $r$ is the radius of the working sphere. The curvature angle of the shorter linkage is defined as $\alpha$. In the spherical triangle AIH, the relationship of $\alpha$, ${\theta }_S$, and $r_b$ can be derived from the spherical cosine theorem as follows:

$$\cos \alpha =\cos \frac{r_b}{r}\cos \frac{{\theta }_S}{2}+\sin \frac{r_b}{r}\sin \frac{{\theta }_S}{2}\cos \angle AIH$$

(1)

Because the $\angle AIH$ is a right angle, the second term on the right side of Equation (1) is always zero. So, the total pitch angle of the spherical scissor mechanism under its maximum deployed condition can be derived as follows:

$${\theta }_{2M}=2{\theta }_S=4\arccos \frac{\cos \alpha }{\cos \frac{r_b}{r}}$$

(2)

Similarly, the spherical scissor mechanism will be limited by the collision of the encoders at J, K, and H to achieve the fully folded status, as illustrated in Figure 5.

The radius of the encoders at J, K, and H is defined as $r_e$. The curvature angle from the collision point to the axis of the encoder is approximate to $r_e/r$. So, the total pitch angle of the spherical scissor mechanism under its maximum folded condition can be derived as follows:

$${\theta }_{2L}=4{\theta }_E=4\mathrm{deg}(\frac{r_e}{r})$$

(3)

In the development of a prototype, the diameter of the bearings at joints A and B is 16 mm. The diameter of the encoders at joints J, K, and H is 30 mm after installation. If we desire the range of motion of the medial direction to be greater than 45°, the minimum curvature angle value $\alpha$ can be determined as 34.36° according to Equation (1). So, we use 35° as the curvature angle of the shorter linkage for convenience. The total pitch angle of the maximum folded mechanism is 34.39°. So, as illustrated in Figure 6, the expected range of motion in lateral and medial directions will be 55.61° and 47.61° respectively in the designed prototype.

2.3. Shoulder Pose Estimation

The 3-DOF spherical scissor mechanism is kinematically equivalent to the 3R shoulder part in Figure 2. So, the shoulder pose, which is described by the joint positions of the 3R mechanism ($q_1$, $q_2$, and $q_3$), can be estimated from the decomposition of the spherical motion of the exoskeleton shoulder. The spherical scissor assembly is connected with the upper arm linkage E by joint J and connected with the base frame F by joint K. Considering that all the curved linkages have identical curvature radius and RCM, only the orientation of the upper arm frame should be concerned with the moving of the shoulder, which is illustrated as the rotation of linkage E with respect to the base frame on linkage F in Figure 7. For convenience in describing, coordination systems are established as follows. For the base frame ${\mathsf{\Omega }}_0$, the X axis is parallel with linkage F and the positive direction is pointing to the center of the back wearer. The Z axis is perpendicular to X and points upward. For the ${\mathsf{\Omega }}_3$ frame fixed with linkage E, the Z axis is parallel to the rotational axis of joint J and the positive direction is pointing outward from the sphere. The X axis is parallel to the upper arm linkage E and the positive direction is pointing to the elbow joint.

When the wearer performs shoulder abduction/adduction motion and moves ${\theta }_1$, the frame ${\mathsf{\Omega }}_3$ will rotate ${\theta }_1$ around the Y axis of the fixed frame, as illustrated in Figure 8.

When the wearer performs shoulder medial/lateral motion and moves ${\theta }_2$, the frame ${\mathsf{\Omega }}_3$ will rotate ${\theta }_2$ around the X axis of itself, as illustrated in Figure 9.

When the wearer performs shoulder flexion/extension motion and moves ${\theta }_3$, the frame ${\mathsf{\Omega }}_3$ will rotate ${\theta }_3$ around the Z axis of itself, as illustrated in Figure 10.

The composition rotation matrix of the above motions is derived as below:

$$\begin{array}{l}{R}_E=RotY({\theta }_1)R_{0}RotX({\theta }_2)RotZ({\theta }_3)\\ =\left[\begin{array}{ccc}-\mathrm{c}{\theta }_3\mathrm{s}{\theta }_1-\mathrm{c}{\theta }_1\mathrm{s}{\theta }_2\mathrm{s}{\theta }_3& \mathrm{s}{\theta }_1\mathrm{s}{\theta }_3-\mathrm{c}{\theta }_1\mathrm{c}{\theta }_3\mathrm{s}{\theta }_2& -\mathrm{c}{\theta }_1\mathrm{c}{\theta }_2\\ -\mathrm{c}{\theta }_2\mathrm{s}{\theta }_3& -\mathrm{c}{\theta }_2\mathrm{c}{\theta }_3& \mathrm{s}{\theta }_2\\ \mathrm{s}{\theta }_1\mathrm{s}{\theta }_2\mathrm{s}{\theta }_3-\mathrm{c}{\theta }_1\mathrm{c}{\theta }_3& \mathrm{c}{\theta }_1{\mathrm{s}}_3-\mathrm{c}{\theta }_3\mathrm{s}{\theta }_1\mathrm{s}{\theta }_2& \mathrm{c}{\theta }_2\mathrm{s}{\theta }_1\end{array}\right]\end{array}$$

(4)

where $R_0$ is the initial orientation of ${\mathsf{\Omega }}_3$ with respect to ${\mathsf{\Omega }}_0$, and RotX, RotY, and RotZ is the rotation matrix of the rotations around different axes. $\mathrm{c}{\theta }_i$ and $\mathrm{s}{\theta }_i$ are short for the cosine and sine values of ${\theta }_i$.

The angle value of ${\theta }_1$, ${\theta }_2$, and ${\theta }_3$ can be calculated from the readings of absolute encoders at joints J, K, and H.

As illustrated in Figure 11, the encoder at joint H reads the angle between the longer linkage C and D, which is recorded as ${\phi }_2$. The encoder at joint J reads the angle between the shorter linkage B1 and upper arm linkage E, which is recorded as ${\phi }_3$. The value of ${\phi }_3$ is composed of ${\theta }_3$ and a half of ${\phi }_2$. Similarly, ${\phi }_1$ is the reading of encoder at joint K, which is composed of ${\theta }_1$ and a half of ${\phi }_2$. The relationship between ${\theta }_2$ and ${\phi }_2$ can be derived from the spherical cosine theorem as follows:

$$\cos \frac{{\theta }_2}{2}={\cos }^2\alpha +{\sin }^2\alpha \cos (\pi -{\phi }_2)$$

(5)

So, we can express the 3-DOF motions of the human shoulder with the encoder readings as follows:

$$\begin{array}{l}{\theta }_1={\phi }_1-\frac{{\phi }_2}{2}\\ {\theta }_2=2\arccos ({\cos }^2\alpha +{\sin }^2\alpha \cos (\pi -{\phi }_2))\\ {\theta }_3={\phi }_3-\frac{{\phi }_2}{2}\end{array}$$

(6)

Accordingly, when the joint positions of the 3R shoulder mechanism in Figure 2 are $q_1$, $q_2$, and $q_3$, the orientation of the upper arm can be derived with forward kinematics as follows:

$$\begin{array}{l}{}_3{}^{0}R=\\ \left[\begin{array}{ccc}\mathrm{s}{q}_2& -\mathrm{c}{q}_2\mathrm{s}{q}_3& -\mathrm{c}{q}_2\mathrm{c}{q}_3\\ -\mathrm{c}{q}_2\mathrm{s}{q}_1& -\mathrm{c}{q}_1\mathrm{c}{q}_3-\mathrm{s}{q}_1\mathrm{s}{q}_2\mathrm{s}{q}_3& \mathrm{c}{q}_1\mathrm{s}{q}_3-\mathrm{c}{q}_3\mathrm{s}{q}_1\mathrm{s}{q}_2\\ -\mathrm{c}{q}_1\mathrm{c}{q}_2& \mathrm{c}{q}_3\mathrm{s}{q}_1-\mathrm{c}{q}_1\mathrm{s}{q}_2\mathrm{s}{q}_3& -\mathrm{s}{q}_1\mathrm{s}{q}_3-\mathrm{c}{q}_1\mathrm{c}{q}_3\mathrm{s}{q}_2\end{array}\right]\end{array}$$

(7)

With the function $R_{\mathrm{E}}={}_3{}^{0}R$, $q_1$, $q_2$, and $q_3$ can be solved as follows:

$$\begin{array}{l}{q}_1=\arctan \frac{-\mathrm{c}{\theta }_2\mathrm{s}{\theta }_3}{\mathrm{s}{\theta }_1\mathrm{s}{\theta }_2\mathrm{s}{\theta }_3-\mathrm{c}{\theta }_1\mathrm{c}{\theta }_3}\\ q_2=\arcsin (-\mathrm{c}{\theta }_3\mathrm{s}{\theta }_1-\mathrm{c}{\theta }_1\mathrm{s}{\theta }_2\mathrm{s}{\theta }_3)\\ q_3=\arctan \frac{\mathrm{s}{\theta }_1\mathrm{s}{\theta }_3-\mathrm{c}{\theta }_1\mathrm{c}{\theta }_3\mathrm{s}{\theta }_2}{-\mathrm{c}{\theta }_1\mathrm{c}{\theta }_2}\end{array}$$

(8)

2.4. Position-Orientation Decoupled Wrist Mechanism

Humans have spherical wrist joints as depicted in Figure 2. When performing sophisticated manipulations such as surgery, such a structure can help people control the position and orientation of hands independently. Many anthropomorphic and industrial manipulators, like the KUKA LBR series [32] and Justin [33], are also designed to have the spherical 3-DOF wrist mechanism to achieve human-like dexterous behaviors. In teleoperation, it is necessary to decouple orientation from the position in the obtained end-effector pose, so that the operators’ intent could be intuitively presented on the slave manipulators.

The difficulties in the design of the wrist mechanism are similar to those of the shoulder mechanism. Especially for the forearm pronation/supination mechanism, the space of the rotational axis is occupied by the wearer’s forearm, so the bearings and encoder cannot be installed along the rotational axis. Existing works [34,35] proposed using the curved rigid rails to support the pronation/supination motion from outside of the forearm. The rotations are measured with IMU or encoder at the motor end. However, IMU sensors or indirect measurement of motion limit the precision of the entire system, while the use of curved rails makes the wrist mechanism bulky and hardly wearable.

In this research, we designed a compact 3-DOF wrist mechanism as depicted in Figure 12. To align with the pronation/supination rotational axis of the human arm and achieve compactness in the mechanism for measuring the forearm motion, we proposed using a thin-wall rolling bearing with an internal diameter of up to 90 mm to connect the forearm linkage and the wrist mechanism. A 19-bit absolute encoder with a hollow floating shaft (Netzer, DS-130, Misgav, Israel) is installed back to the bearing to measure the pronation/supination motion directly. The 2-DOF serial mechanism for measuring the flexion/extension and abduction/adduction motions of the wrist is installed on the inner ring of the forearm pronation/supination mechanism. The axes of the two DOFs are designed to converge with the forearm rotation axis at the same point, so that the wrist mechanism can work like the spherical joint. A versatile joystick is installed on the end of the wrist mechanism. The wearer’s hands can pass through the forearm rotation ring and hold the joystick. The joint angle values $[{\theta }_4,{\theta }_5,{\theta }_6,{\theta }_7]$ can be read from the absolute encoders installed on the elbow, the forearm, and the wrist mechanism. The values can be directly mapped to the joint angle values $[q_4,q_5,q_6,q_7]$ in the serial model in Section 2.1.

2.5. Data Acquisition and Transmission

We designed the exoskeleton as a distributed data acquisition system. To obtain an accurate estimation of the wearer’s upper limb pose, the joint angle values should be acquired and synchronously transmitted to the master controller. We developed a low-profile data acquisition module (DAQM) to read encoder data and transmit the values in real time. The DAQMs are installed near the absolute encoders of every joint. The DAQMs access the absolute encoders via the SSI interface and update the joint values at a 10 ms period. A DAQM connected with the absolute encoder is illustrated in Figure 13.

The DAQMs use STM32 as the MCU, which integrates the CAN controller. The DAQMs transmit the read joint data to the master controller via the CAN bus. The data acquisition network of the exoskeleton is composed of 17 nodes, including 14 DAQMs for encoders on the joints, 2 joystick controllers, and a master controller. It is important to plan the communication parameters, so that the data transmission can achieve good real-time performance and synchronicity. The DAQM nodes and the joystick controllers use a unified 4 bytes CAN frame protocol, which includes 3 bytes for saving the encoder data or joystick commands, and 1 byte for transmitting the status and diagnosis information. Encoders with 24-bit resolution or lower will all be supported. According to the data frame structure in CAN2.0A standard [36], the 4 byte frame contains several mandatory segments, including interframe space, SOF, arbitration ID, SRR, IDE, RTR, DLC, data field, CRC, ACK, and EOF. The summation of bit length for such a frame can reach 79. Additionally, some stuffed bits may be inserted to satisfy the CAN data link layer protocol. When transmitting, if the transmitter encounters a run of 5 successive ones or zeros it inserts a bit of the other polarity. The maximum length of the stuffed frame is estimated as below:

$$L_m=\left(⌊\frac{34+8S_m}{5}⌋\right)+48+8S_m$$

(9)

where $S_m$ is the byte length of the payload in the frame. When $S_m$ is 4, the maximum length of the frame will be 92 bits. Ideally, the CAN bus bandwidth used by node i is calculated as:

$$B_i=\frac{F_i\ast L_m}{biterate}$$

(10)

where $F_i$ is the sending frequency of node i, and bitrate is the baud rate of the CAN bus.

The CiA group suggests that an average busload of 50% should not be exceeded [37]. So, the limits should be taken into consideration when deciding the bit rate and frequency. The maximum length between two nodes in the exoskeleton is about 3 m. Therefore, we conservatively select 500 kbps as the CAN bus baud rate. The 16 nodes in the exoskeleton use identical 4 bytes CAN frame format. When 10 ms is used as the data upload period, every single node will occupy 1.84% of the full bandwidth of the CAN bus. The maximum overall busload for all 16 data transmission nodes is estimated at 29.44%.

We use a BeagleBone Black embedded computer, which is powered by an AM3358 cortex-A8 microprocessor, as the master controller. The BeagleBone Black has both a CAN bus and Ethernet interface on the board, which makes it perfect for working as a gateway between the local CAN network and the external Ethernet.

The software stack of the master controller is depicted in Figure 14. The BeagleBone Black runs UBUNTU 18.04LTS OS. We developed the gateway application for transferring data between the CAN interface and the Ethernet interface, which performs data exchange with two independent threads. The user space application obtains access to the CAN controller hardware through an open-source library libsocketcan [38], which provides socket-style API.

2.6. Implementation Details of a Prototype

The shoulder joint, elbow joint, and wrist joint are connected with two parallel carbon fiber tubes. To make the exoskeleton flexible to the variant physical size of the wearer, we designed the adjustable linkage mounting holes (Figure 15) on the connection parts. The length of the upper arm, forearm, and palm can be adjusted individually by taking advantage of this mechanism. The total length of the upper limb exoskeleton is adjustable from 460 mm to 540 mm. Empirically, with such an adjustment range, the exoskeleton device can accommodate wearers from 160 cm to 185 cm.

We built a back frame as the installation base of the two exoskeleton arms. The shape of the back frame was designed to conform to the curve of the back of the human body. A soft nylon fabric carrying system was sewn onto the front side of the back frame. With the help of the back frame and the carrying system, the load of the whole system can be transferred and distributed evenly among the shoulder, back, and waist, so that the wearer will not feel uncomfortable when wearing the system for a long time. Additionally, the carrying system helps with fixing the back frame to the torso tightly and steadily, which guarantees that the RCM of the shoulder mechanism properly aligns with the rotational center of the GH joint.

The power supply and electrical systems, including the battery pack, DC/DC converter, and slave controller board, were installed behind the back frame and are protected by a plastic shield, as illustrated in Figure 16. We used a 24 V, 5700 mAh battery pack (TB48S, DJI, Shenzhen, China) as the power source of the system, which can support the system to run continuously over 36 h. A step-down DC/DC converter (URA2405LD-30WR3, MORNSON, Guangzhou, China) converts the 24 V power from the battery to 5 V for supplying the slave controller and the DAQMs.

The back frame and the joint parts were mainly manufactured with high-strength aluminum alloy A7075P. The upper arm linkage, forearm linkage, and wrist linkage were made of 5 mm carbon fiber tubes, which are lightweight and high stiffness.

The overall mass (including the battery) of the prototype is 4.8 kg. The mass of the moving part on a single arm is 1.1 kg, and 0.76 kg of the mass is distributed on the forearm mechanism. However, part of the load will be borne by the back frame and transferred to the torso when properly wearing the exoskeleton.

3. Motion Mapping to the Anthropomorphic Manipulators

In this section, we use a human scale 14-DOF dual arm anthropomorphic manipulator to present the teleoperation control method with the proposed upper limb exoskeleton.

As depicted in Figure 17, the arm of the 14-DOF manipulator has a similar kinematic structure to the human upper limb model described in Section 2.1. So, directly mapping the joint space positions obtained from the exoskeleton to the slave manipulator is a very straightforward approach in the isomorphic master-slave context. In addition, task space mapping could be a more general method for teleoperation control of the manipulators with different kinematic structures. We describe the two types of mapping strategies in this section.

3.1. Joint Space Mapping

As described in Section 2.3 and Section 2.4, the upper limb pose can be estimated with the joint position data obtained from the exoskeleton master. We can use the resolved 7-dimensional joint position vector $q_{master}$ to recover the wearer’s upper limb pose on the 7-DOF kinematic model in Figure 2. Every single arm of the slave robot is driven by the 7-dimensional control command vector $q_{slave}$. We use the following mapping to convert $q_{master}$ to $q_{slave}$ for implementing the joint space teleoperation control:

$$q_{slave}=M_{trans}{q}_{master}+q_{offset}$$

(11)

where $M_{trans}$ is a transformation matrix which is used to adjust the joint space mapping sequence and scale between the master and the slave. The vector $q_{offset}$ is used to reflect the initial state difference between the master and the slave.

The joint space mapping strategy is simple and intuitive for the operator. The movement on the operator’s specific joint will instantly drive the counterpart of the slave robot. The joint space mapping method is also proper for time-sensitive situations because no inverse kinematic calculation is required. However, small differences in kinematic parameters may exist between the master and slave, which could lead to considerable absolute error in the position control of the end-effector.

3.2. Task Space Mapping

If high precision control of the end-effector is demanded, or the DOF and kinematic structure of the slave manipulator is significantly different from the 7-DOF upper limb model, task space mapping control will be essential in teleoperation.

The task space mapping control method takes a 6-DOF end-effector target pose, which is provided by the operator’s hand pose, and performs inverse kinematics (IK) to convert the task space target into joint space commands. However, in the teleoperation control of the anthropomorphic manipulators, not only the end-effector tracking should be cared about, but also imposing the human posture into the slave manipulator is preferred. Most anthropomorphic manipulators, including the one depicted in Figure 17, have redundant kinematic structures, which means the ability to adjust their posture in the null space of the end-effector. The exoskeleton master device, which can capture the posture of the entire upper limb, enables the posture mimicry features on such redundant manipulators in teleoperation. However, if a kinematic discrepancy exists between the master and slave, it could be impossible to meet simultaneously both the end-effector tracking and the posture mimicry requirements in teleoperation control. To address this dilemma, we proposed a hierarchical scheme to coordinate the target pose-tracking task of the end-effector and the posture control task. To this end, we developed the control scheme (Figure 18) based on the CLIK algorithm [39] and additionally designed a posture mimicry controller.

The hierarchical CLIK scheme generates velocity control commands $\dot{q}$ for the slave joints as follows:

$$\dot{q}=J^{†}(q)({\dot{x}}_d+K_p(x_d-x))+(I-J^{†}(q)J(q)){\dot{q}}_0$$

(12)

where $J^{†}(q)$ is the Moore–Penrose inverse of $J(q)$. The 6-DOF vector $x_d$ provides the desired end-effector target pose, which can be obtained from the master status and forward kinematics. The 6-DOF vector $x$ is the current pose of the slave end-effector. The positive definite matrix $K_p$ guarantees the convergence of $\dot{q}$. The matrix $(I-J^{†}(q)J(q))$ projects any joint space velocity ${\dot{q}}_0$ onto the null space of the slave end-effector [39].

The two items on the right side of Equation (12) act on two different tasks: the first item keeps the slave end-effector tracking the target pose, which is treated as the prior task; the second item only adjusts the slave posture in null space, which is treated as the secondary task.

In this research, the secondary task aims at making the slave manipulator mimic the operator’s posture. We achieve this by performing alignment of the upper arm orientation between the master and the slave. An objective function is established to reflect the orientation error between the master and the slave as follows:

$$H(q)={\delta }_x^2+{\delta }_y^2+{\delta }_z^2$$

(13)

where the vector ${\delta }_{{\Theta }}={({\delta }_x,{\delta }_y,{\delta }_z)}^T$ is the angular difference between two rotation matrices, whose components are available from the rotation differential matrix below:

$$S({\delta }_{{\Theta }})=\left[\begin{array}{ccc}0& -{\delta }_z& {\delta }_y\\ {\delta }_z& 0& -{\delta }_x\\ -{\delta }_y& {\delta }_x& 0\end{array}\right]$$

(14)

For the two rotation matrices ${}_m{}^0\mathit{R}{}_3$ and ${}_s{}^0\mathit{R}{}_3$ representing the upper arm orientation of the master and the slave respectively, their angular differential can be derived by [40]:

$$S({\delta }_{{\Theta }})={}_m{}^0\mathit{R}{}_3{}_s{}^0\mathit{R}{{}_3}^T-I_{3\times 3}$$

(15)

Then we define ${\dot{q}}_0$ as a negative proportion of the gradient of the objective function $H(q)$ as follows:

$$\begin{array}{l}{\dot{q}}_0=-\eta \nabla H(q)\\ \nabla H(q)={\left(\frac{\partial H(q)}{\partial q}\right)}^T\end{array}$$

(16)

where $\eta$ is a positive scalar, and the gradient vector $\nabla H(q)$ can minimize the posture error as soon as possible.

4. Experiments

4.1. Experimental Setup

Figure 19 illustrates the experimental setup for evaluating the proposed exoskeleton and the teleoperation control methods.

The exoskeleton master is connected to the slave robot via Ethernet and sends master control commands as UDP packages every 10 ms. The controller of the slave robot is based on ROS. An ROS node “/exo_master_agent” receives the UDP command packages and maps the master commands to joint space commands for the slave robot and publishes them to the ROS topic “/joint_state” to control the robot. The “/exo_master_agent” also subscribes the topic “/slave_joint_state”, which reflects the real-time state of all the slave joints and sends them to the master side for monitoring. A virtual slave robot can be observed on the master side through RVIZ, which is a 3D visualization tool for ROS.

Both of the models for the master and the slave are described by URDF files in ROS. The states of the interested frames in the URDF files can be tracked and recorded with TF in ROS.

The subject for evaluating the range of motion is male, 178 cm, 74 kg. The DH parameters for the exoskeleton master and the slave robot are listed in

Table 2. DH parameters for the master and the slave.

Joint Index	Master				Slave
i	${\alpha }_{i-1}$	$a_{i-1}$	$d_i$	${\theta }_i$	${\alpha }_{i-1}$	$a_{i-1}$	$d_i$	${\theta }_i$
1	$\pi /2$	0	0.19	$q_1$	$\pi /2$	0	0.235	$q_1$
2	$-\pi /2$	0	0	$q_2$	$-\pi /2$	0	0	$q_2$
3	$\pi /2$	0	0.245	$q_3$	$\pi /2$	0	0.265	$q_3$
4	$-\pi /2$	0	0	$q_4$	$-\pi /2$	0	0	$q_4$
5	$-\pi /2$	0	−0.25	$q_5$	$-\pi /2$	0	−0.272	$q_5$
6	$\pi /2$	0	0	$q_6$	$\pi /2$	0	0	$q_6$
7	$\pi /2$	0	0	$q_7$	$\pi /2$	0	0	$q_7$
EE	0	0	0.04	0	0	0	0.168	0

.

4.2. Range of Motion Evaluation

To assess how much the range of motion will be limited by wearing the proposed exoskeleton, the wearer was instructed to perform the seven motions depicted in Figure 2 and try to achieve the maximum range. We took three photos for each motion to record the positive limit, negative limit, and neutral positions. Then the range of motion was measured on the pictures with ImageJ [41]. We also recorded the data for the same subject without wearing the exoskeleton to evaluate the active coverage of the range of motion. Examples of the measurements for shoulder flexion/extension are illustrated in Figure 20. The measurement results are listed in

Table 3. Measurement results of the range of motion with and without the exoskeleton.

Motions	With Exoskeleton (deg)	Without Exoskeleton (deg)	Coverage (%)
Shoulder Flexion/Extension	168/30	173/30	97/100
Shoulder Abduction/Adduction	85/32	139/36	61/89
Shoulder Medial/Lateral	47/47	62/47	76/100
Elbow Flexion/Extension	127/0	127/0	100/-
Forearm pronation/supination	90/81	90/85	100/95
Wrist Abduction/Adduction	34/55	34/55	100/100
Wrist Flexion/Extension	62/35	62/45	100/78

. It should be noted that this method only provides a preliminary evaluation of the range of motions. So, we use an integer percentage to present coverage of the range of motion.

4.3. Precision and Dynamic Performance Evaluation

We performed teleoperation in a simulation environment to check if the proposed mapping control methods work well. The joint space and task space mapping strategies were tested individually. In each experiment, the operator was instructed to control the virtual manipulator to draw specific trajectories covering most areas of the working space. The 6-DOF human hand pose and the slave end-effector pose were recorded to evaluate the precision and dynamic performance of target tracking. We also recorded the elbow trajectories of both sides to evaluate the posture similarity between master and slave in teleoperation. The following error results of simulated teleoperation experiments with the joint space control strategy and task space control strategy were illustrated in Figure 21 and Figure 22 respectively.

4.4. Task Demonstrations on a Real Anthropomorphic Manipulator

We performed two teleoperation tasks to demonstrate the intuitiveness of the proposed exoskeleton device and the mapping control methods. The experiments are illustrated in Figure 23.

In task 1, the operator controlled the slave robot to grasp a tennis ball and put it into a box. In task 2, the operator controlled the slave robot to transfer a screwdriver from one hand to the other.

The task was executed with both task space control and joint space control 10 times. The success rates are listed in

Table 4. Task success rates under different control strategies.

Control Strategy	Task 1	Task 2
Joint space control	80%	40%
Task space control	60%	90%

.

5. Discussion

From the results listed in Table 3, we found that the most severe cases of limited mobility occurred in the shoulder abduction motion. When wearing the exoskeleton, only 61% of the normal range of motion in this direction could be reached. Because the back frame has a curved outline to fit the shape of the wearer’s torso, the shoulder mechanism will collide with the back frame when the wearer lifts his arms in the abduction direction to about 85°. Fortunately, the worst case will only happen when the wearer lifts his arms in the frontal plane, If the wearer’s arms are slightly (more than 10°) ahead of the frontal plane, the lifting motion will not be limited at all. So, in most practical situations, arm-lifting activities are not limited by the exoskeleton. Similarly, the subject was required to keep the elbow close to his torso when measuring the range of motion in the shoulder medial/lateral direction. Consequently, the coverage result of shoulder medial direction was reported as 76%. However, if the wearer is allowed to move his elbow slightly, the limitation will be eliminated. In 10 of the 14 basic upper limb motions, the exoskeleton covered more than 95% of the normal range of motion.

When the joint space control strategy was applied, the end-effector tracking errors between the master and slave of the 6D Cartesian space are illustrated in (a–f) of Figure 21. We found significant tracking errors in the positional dimensions, while the errors vary with position. This is caused by the difference in kinematic parameters (Table 2) between master and slave. Especially for the Y dimension, the maximum error between the master and slave can reach 0.12 m in the test trajectory. Because the gripper of the slave manipulator is much longer than the human hand along the Y axis, the position of the slave end-effector frame will be 0.15 m ahead of that of the master in the initial state. As a result, the position control will be biased by the absolute mechanical difference when moving along the Y axis. This error will be diminished as the end-effector gets close to the frontal plane. This reason also applies to the other two axes. However, the upper limb posture is defined by the relative position of the shoulder, elbow, and wrist. The prerequisite for applying joint space control strategy is similar kinematic structure and arm length ratio (upper arm: forearm) between the manipulator and the human arm. We suppose that the prerequisites could ensure a similar appearance between the master and slave when applying joint space mapping. In our experimental setup, the arm length ratio is 1:1.02 on the master side and 1:1.03 on the slave side. So, from the 3D trajectories depicted in (g) of Figure 21, we found that, although absolute differences always existed between the master and the slave, both the trajectories of the slave end-effector and elbow could maintain good similarity with the master.

When the task space control strategy was applied, as illustrated in (a–f) of Figure 22, the slave end-effector maintains good tracking performance in all dimensions. The average pose errors between the master and the slave are 7 mm in position and 0.019 rad in orientation for the testing trajectory. In the 3D trajectory depicted in (g) of Figure 22, the end-effector of the master and the slave always kept close to each other. However, the elbow trajectories show an obvious difference, which means the posture of the slave may be a little different from the operator. As described in Section 3.2, in a hierarchical CLIK control framework, the end-effector tracking task is treated as the main task, while the posture mimicry task is treated as the secondary task. As the overall length of the kinematic chains of the master and slave are 0.535 m and 0.705 m respectively, the posture mimicry task must make way for the precision of end-effector tracking.

In the practical task demonstrations with our exoskeleton master, both the above control strategies were tested. For task 1, the ball picking task, the key factor for success is delicate regulation of the position and orientation of the 2-finger robotic hand, so that it can reach a proper gripping point. Because a position–orientation decoupled wrist mechanism is applied in the exoskeleton, the operator will feel more intuitive to perform such an operation with joint space control. In task 2, the key factor for successfully transferring objects from one hand to another is the perception of the relative position and movements of the two hands. So, with task space control, the operator could feel better control of the position of the end-effector.

6. Conclusions

In this paper, we presented the development of an upper limb exoskeleton for intuitive teleoperation of anthropomorphic manipulators. We designed the exoskeleton human-machine interface as a 14-DOF, lightweight, and passive wearable device, which is only 4.8 kg in total. With the help of a curved back frame and carrying system, the exoskeleton can be steadily fixed on the torso, providing a self-alignment feature for improving measurement accuracy. A compact spherical scissor mechanism was proposed as the shoulder mechanism to mitigate the limitation of the range of motion when wearing the exoskeleton.

A comparison evaluation showed that when wearing the proposed exoskeleton, the wearer can reach most areas of the normal range of motion. In 10 of the 14 basic upper limb motions, the exoskeleton covered more than 95% of the normal range of motion. Therefore, as a human-machine interface, the exoskeleton will not make the user feel natural or uncomfortable in teleoperation. Both the joint space and task space control strategies were devised and tested on a 14-DOF dual-arm anthropomorphic manipulator. Demonstration tasks were also performed to show the different features of the two control methods. The joint space control strategy provides a simple and intuitive mapping between the master and slave. It could be particularly effective in situations where individual control of some joints is required. However, in the case where accurate control of the end-effector is desired, or the kinematic structure is significantly different between the master and slave, the task space control strategy will be essential. A simulation experiment showed that, although the 14-DOF slave manipulator was different in some mechanical parameters from the master, good tracking accuracy (7 mm position error and 0.019 rad orientation error on average) could be achieved as well as maintaining human-like posture on the slave side with the task space control strategy. The task space control method makes the proposed exoskeleton a general-purpose master device for the teleoperation of different types of anthropomorphic manipulators.

Our future work will use the exoskeleton as an intuitive human-machine interface to teach robot manipulation skills with imitation learning. The trained autonomous control system could assist the teleoperation in a shared autonomy paradigm.