Next Article in Journal
A Robust Electric Power-Steering-Angle Controller for Autonomous Vehicles with Disturbance Rejection
Next Article in Special Issue
Measurement, Evaluation, and Control of Active Intelligent Gait Training Systems—Analysis of the Current State of the Art
Previous Article in Journal
Formal Verification and Analysis of 5G AKA Protocol Using Mixed Strand Space Model
Previous Article in Special Issue
A Dual-Armed Robotic Puncture System: Design, Implementation and Preliminary Tests
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Human Joint Torque Estimation Based on Mechanomyography for Upper Extremity Exosuit

1
State Key Laboratory of Robotics and System, Harbin Institute of Technology (HIT), Harbin 150001, China
2
Weapon Equipment Research Institute, China South Industries Group Corporation, Beijing 102202, China
*
Author to whom correspondence should be addressed.
Electronics 2022, 11(9), 1335; https://doi.org/10.3390/electronics11091335
Submission received: 25 February 2022 / Revised: 15 April 2022 / Accepted: 20 April 2022 / Published: 22 April 2022
(This article belongs to the Special Issue Physical Diagnosis and Rehabilitation Technologies)

Abstract

:
Human intention recognition belongs to the algorithm basis for exoskeleton robots to generate synergic movements and provide corresponding assistance. In this article, we acquire and analyze the mechanomyography (MMG) to estimate the current joint torque and apply this method to the rehabilitation training research of the upper extremity exosuit. In order to obtain relatively pure biological signals, a MMG processing method based on the Hilbert-Huang Transform (HHT) is proposed to eliminate the mixed noise and motion artifacts. After extracting features and forming the dataset, a random forest regression (RFR) model is designed to build the mapping relationship between MMG and human joint output through offline learning. In addition, an upper extremity exosuit is constructed for multi-joint assistance. Based on the above research, we develop a torque estimation-based control strategy and make it responsible for the intention understanding and motion servo of this customized system. Finally, an actual test verifies the accuracy and reliability of this recognition algorithm, and an efficiency evaluation experiment also proves the feasibility for power assistance.

1. Introduction

As is the case for most of the potential technical equipment for rehabilitation training and movement assistance, the exoskeleton system has always attracted attention in related research fields for the elderly and the disabled [1,2]; however, with some key issues not being resolved, the performance of this wearable robot remains relatively limited [3]. Among them, human intention perception belongs to a pretty critical research point and needs to be studied further. Traditional methods try to achieve this function by monitoring limb kinematics data [4] or human-machine interaction information [5], but problems such as response lag greatly restrict its actual effect. In recent years, recognition methods based on biological signals have emerged and gained wide attention [6], which are expected to realize more effective intention understanding.
Biological signals are often generated before the execution of corresponding actions, which show a certain degree of motion predictability and can make intention recognition more timely and accurate. In addition to eye tracking and galvanic skin response (GSR) [7], which are not suitable for combining with exoskeleton control technology, commonly used types in current research mainly contain electroencephalogram (EEG) [8], surface electromyogram (sEMG) [9], and mechanomyography [10]. The EEG signal originates from the potential of the external electrical field that fluctuates around nerve cells in the brain, and is often applied to classify specific movement patterns [11,12]. Due to its instability and susceptibility to interference, EEG-based methods for data collection, signal processing, and intent identification needs to be further improved and optimized. The sEMG signal represents the total action potentials of different motor units innervated by certain motor neurons, which can be obtained through electrodes placed on the corresponding muscle tissue [13]. It has the advantage of high sensitivity and low delay, but many factors will diminish its effectiveness for data collection, such as skin surface cleanliness, air humidity, and patch electrode position [14]. MMG reflects the mechanical vibration signal generated when the skeletal muscle contracts, and it contains abundant information, such as the number of muscle fibers participating in the exercise, and the amplitude and frequency of vibration [15]. Compared with the above two signals, it has the following advantages [16].
  • The measuring device can be used directly without touching the skin, which simplifies the preparation before data collection;
  • the signal has strong anti-interference ability and will not be affected by environmental variables such as sweat, humidity, or electromagnetics;
  • the equipment cost is pretty low because the data collection task can be completed by using an acceleration sensor that meets the accuracy requirements.
However, there are still some limitations in the practical application of MMG. For example, it is easily contaminated by low-frequency motion artifacts, and sensitive to sudden step noises; therefore, a suitable signal processing algorithm is needed to extract a relatively pure sequence.
At present, classification and regression algorithms have been widely used in many studies, such as the trajectory control of a redundant robot [17] and hand gesture recognition for teleoperated surgical robot systems [18], which can also complete the analysis of human motion intention. As a relatively common and easy-to-use method, motion pattern recognition [19,20] aims at establishing the mapping relationship from sensor information to finite human motion states through classification models. Then, the control subsystem will generate corresponding commands according to current motion pattern and will deliver them to underlying drivers. Joint angle prediction attempts to calculate the limb position at the next moment based on regression models [21,22], which can effectively avoid the response lag of the exosuit. After that, the motion state of the power-assisted system can be dynamically adjusted through position closed-loop control. Joint torque estimation also belongs to a direct and effective method of obtaining intentions [23,24]. The desired torque can be calculated through the Hill type model [25] or obtained from the biological signals using machine learning algorithms [26], and can act as the input parameter of torque closed-loop control to regulate the motor output. The last method will provide a reference for the direct control of external torque that assists joint motion, which is quite practical for upper extremity exosuits that need to achieve an expected power-assisted efficiency.
If we want to use the abovementioned MMG signal to estimate the joint torque, it is obviously quite difficult through conventional mathematical derivation. The machine learning algorithm can train the mapping model very well based on the existing data, and it shows an excellent fitting ability in many research fields, such as breathing pattern detection [27] and human activity identification [28]; therefore, this method should be able to describe the complex and nonlinear relationship between MMG and joint torques.
The inherent characteristics of the soft exosuit based on Bowden cables greatly increases the difficulty for designing control strategies [29,30]. The gravity compensation algorithm is a simple and common method for motion generation, which will output an active torque to offset the joint load imposed by the limb weight [31,32]; however, due to the large error of compensation model and the ignorance of dynamic characteristics, it may cause the system response to, more or less, mismatch the joint movement. The exosuit named CRUX [33] tries to control the target arm to follow the reference trajectory of the healthy one, consequently completing the active rehabilitation training process [34]. This method will limit the subjective initiative of the wearer to a certain extent, and is not suitable for situations where both arms need assistance. Some scholars from Italy have proposed a threshold method based on sEMG [35]. When the signal amplitude of the wearer’s measured muscle exceeds the set value, the wearable system starts to produce a power-assisted effect for motion immediately. The most obvious disadvantage of this method is that it only outputs a constant driving force, but the required joint torque changes dynamically under different motion states. In general, the control logic for the upper extremity exosuit still has some defects, and remains to be further explored.
In this paper, we intend to complete data preprocessing and feature extraction using MMG, then establish the mapping relationship from this signal to the joint torque based on the machine learning algorithm, and finally apply it to the rehabilitation training research of upper extremity exosuit. MMG is collected through an inertial measurement unit (IMU) and synthesized by the linear accelerations along three axes. The HHT will filter this original signal to obtain a relatively pure result. We extract three features from the processed data and combine them with the collected joint torque to form data sets for training and testing. A RFR model is designed as the algorithm framework for joint torque estimation, and its parameters are determined through offline training. According to the above research results, a control algorithm based on joint torque estimation will take charge of the motion control for an upper extremity exosuit. Eventually, some experiments are carried out to test the accuracy of torque estimation and the efficiency of power assistance. The main contributions and highlights of this study are summarized as follows.
  • We have attempted to use the MMG signal as the medium for the exosuit to understand human intentions, and to demonstrate the feasibility and effectiveness of this approach.
  • A series of MMG-related methods for signal acquisition, filtering, and feature extraction have been developed.
  • A regression model reflecting the nonlinear relationship between muscle activation and joint output is constructed.
  • A torque estimation-based control algorithm is designed and applied to the multi-joint motion assistance of upper extremity exosuit, which can significantly amplify the limb strength.
The remaining research content of this article is organized as follows. Section 2 describes the measurement and calculation methods for MMG, its corresponding joint torque, and how to process the original signal to construct data sets. Section 3 introduces the design details of RFR model and uses a large amount of test data to fit the desired quantitative relationship. In Section 4, we have developed a control strategy for the upper extremity exosuit based on torque estimation. Section 5 proves the feasibility of the above methods through some experiments. Section 6 is the conclusion.

2. Data Sets Acquisition

2.1. Raw Information Collection

In order to obtain the MMG signal and corresponding joint torque at the same time, we built a measurement platform for joint information collection. Figure 1a shows how to use this device to get relevant data about elbow static flexion and extension.
During this process, a six-dimension force sensor and a high-performance IMU are responsible for detecting forces along three directions at the end of limb, and gathering MMG at the brachioradialis of forearm, respectively. When subjects hold the force measuring rod and try to perform specific actions, this platform will transmit corresponding data to the host computer through a data line and save them in files.
With readings from a six-dimension force sensor (Fx and Fy) as input, the joint torque ( T elbow ) can be calculated by a certain mathematical relation. The mechanical model of the elbow is shown in Figure 1b, where A , F , and L, respectively, represent the current posture vector of human arm, the resultant force vector at the end, and the length of forearm. Then, we can dynamically obtain the joint torque values during elbow static flexion and extension through the following formulas.
F = F x + F y
T elbow = L ( A A ×   F )
During dynamic flexion/extension without using the platform, it can be seen in Figure 2 that when the measured muscle contracts or relaxes and drives the human joint to rotate, the IMU can perceive some regular acceleration signals in the x, y, and z directions synchronously, and the same is true for the process of static joint output. In order to integrate all the effective information, we calculate the sum of linear acceleration vectors along three axes (ACCx, ACCy, and ACCz), and take its modulus as the original MMG signal (MMG(t)), which can be expressed as the following equation.
MMG ( t ) = ACC x 2 + ACC y 2 + ACC z 2

2.2. MMG Signal Processing

However, the method of using linear acceleration values for MMG characterization inevitably mixes low-frequency artifacts into the collected original signal, such as gravitational acceleration, IMU posture changing artifacts caused by muscle deformation, and motion artifacts introduced by upper limb movements. Moreover, high-frequency white noise may also be superimposed on it. In order to improve the quality of data and lay the foundation for subsequent feature extraction, an effective filtering method must be applied to eliminate the abovementioned interferences. Traditional signal processing methods are mostly based on Fourier analysis, but these ones have limited effects in practical applications of processing MMG due to its non-linear and non-stationary characteristics. With reference to related literatures, we decide to use HHT to analyze the original data, which is more suitable for these kinds of signals.
The HHT consists of two parts, namely empirical mode decomposition (EMD) and the Hilbert transform. EMD can decompose a complex signal into a limited number of intrinsic mode functions (IMFs) and a residual based on the local time scale characteristics of itself. The specific implementation steps are as follows. First, we find all the local maximum points and local minimum points of the original MMG signal and fit their respective envelopes through the cubic spline curve. Then, the mean value of the upper and lower envelopes (U(t) and L(t)) will be subtracted from the original sequence to get the remaining part with the low frequency information removed (x(t)).
x ( t ) = MMG ( t )     1 2 ( U ( t ) + L ( t ) )
If the number of extreme values and zero crossings on the entire data set of x(t) differs by 0 or 1, and the average value of its two envelopes remains zero at any time, then IMFi(t) = x(t), otherwise it is necessary to let MMG(t) = x(t), and repeat the above steps until these two conditions are met. Next, we remove the obtained IMFi(t) from MMG(t) and repeat all the above steps again with the remaining part (ri(t)) to get other IMF components until ri(t) is a constant or monotonic function. As shown in Figure 3a, the original MMG signal is decomposed into 9 IMFs and 1 residual (res(t)) according to the signal frequency, which can be expressed as follows.
MMG ( t ) = i = 1 9 IMF i ( t ) + res ( t )
After completing the above analysis, it is time to select the IMFs dominated by MMG through certain methods and reorganize them to obtain a relatively pure signal. To eliminate the noise-dominated IMFs, we introduce the concept of autocorrelation (RIMF(t1,t2)), which reflects the correlation degree of signal values at different times (t1 and t2). Its normalized expression form, (ρIMF(τ)), can be obtained with the following formula, where τ = t1t2.
R IMF ( t 1 , t 2 ) = E [ IMF ( t 1 ) · IMF ( t 2 ) ]
ρ IMF ( τ ) = R IMF ( τ ) R x ( 0 )
If the normalized autocorrelation curve belongs to an impulse function close to the zero point, it can be ascertained that the corresponding IMF is dominated by noise, because noise has randomness and a weak correlation at every moment. After the calculations in Figure 3b, the first IMF can be classified as such a disturbance.
To exclude the IMFs dominated by motion artifacts, we try to find the discrimination basis from the energy distribution of each order component. After calculating the energy value ( E IMF i ) of each IMF according to the following Formula (8), it is revealed that energy contained in each IMF has increased significantly, starting from the sixth order. By analyzing the data of different participants, we find that removing these IMFs as motion artifacts can achieve a better result.
In the end, we believe that IMF2~IMF5 can effectively characterize the muscle activity, and the filtered signal can be obtained with them being recombined. It can be seen from Figure 4 that there is a pretty clear correspondence between the processed MMG and joint torque.
E IMF i = 1 n j = 1 n [ IMF i ( j ) ] 2

2.3. Feature Extraction

Considering that the change in limb strength is often accompanied by the fluctuation of the muscle fiber’s vibration amplitude, we select the root mean square (RMS) as the time domain characteristic of MMG. It reflects the effective value of data amplitude and can be calculated with the following formula.
RMS MMG = 1 N i = 1 N X i 2
Due to the correlation between muscle activity and its vibration frequency, mean power frequency (MPF) is used to represent the frequency domain characteristic of MMG. It is necessary to perform the Hilbert transform on the filtered MMG to analyze its frequency spectrum, and then integrate it on the time axis to obtain the Hilbert marginal spectrum that characterizes the relationship between signal frequency (fi) and energy (Ei). Then, MPF can be calculated through the following equation.
MPF MMG = i = 1 N f i E i i = 1 N E i
The MMG signal may contain information about the number of muscle fibers involved in power output; therefore, we additionally introduce the concept of sample entropy (SampEn) to describe the characteristic from a non-linear perspective, which can quantify the complexity of the time series.
In order to not lose continuity information in the sequence, we apply the sliding window strategy to extract these characteristic values of the filtered MMG signal. The window length and step length are set as 500 ms and 50 ms, respectively. So far, the data set of elbow static flexion/extension is established with RMS, MPF, and SampEn of the MMG signal as features, and joint torque as the label. We can also use similar methods to obtain relevant information of shoulder static flexion/extension and static adduction/abduction.

3. Off-Line Torque Estimation

3.1. Regression Model Design

There is a relatively complicated non-linear relationship between MMG and human joint torque, which seems difficult to accurately describe using the traditional polynomial regression model. We try to introduce a machine learning method to solve this problem, using a large amount of test data to fit the real mapping law. Considering that the joint torque estimation algorithm is oriented to a wearable power-assisted system, the requirement for its stability and reliability must take precedence over that of other aspects. Since RFR (shown as Figure 5) has these advantages, it can be well qualified for this task.
The RFR belongs to a bagging type algorithm of ensemble learning, which aims at improving overall performance by packaging and combining several weak models, namely decision trees, into a strong one. The entire model consists of multiple classification and regression trees (CARTs) that are not related to each other. All CARTs jointly determine the final output result. The specific implementation steps of the algorithm are described as follows.
  • Randomly extract any number of samples from the training set to form multiple new sub-training sets;
  • use each sub-training set to train a CART separately. During this process, it is necessary to randomly obtain any number of features from all the features, and then select the optimal segmentation point to cut the subtree;
  • repeat step 2 to obtain multiple trained CARTs;
  • calculate the average of all the CARTs’ prediction results and use it as the final estimated value.
Only when more than half of the CARTs make wrong predictions will the output of the RFR model seriously deviate from the true value. Even if an abnormal data point appears, it does not affect the performance of entire algorithm too much, which fully reflects the strong robustness to stop interference signals.

3.2. Off-Line Training and Testing

We recruited three healthy adult men to participate in training data acquisition. Based on the abovementioned platform and methods, the information collection experiment for three motion modes obtains 150,000 sets of sample data in total. In accordance with the idea of cross-validation, one-fifteenth of them are selected as the test set, and the remaining data act as the training set. Finally, on the basis of setting the number of sub-regression trees of the RFR to 10, and the minimum leaf size to 1, the training process has been carried out, and the verification result of the test set is also obtained.
Taking shoulder static adduction/abduction as an example, Figure 6a shows that the minimum mean square error (MSE) decreases and tends to be stable with the increase in iterations, and Figure 6b indicates that the difference between the predicted results and the actual values on the test set is relatively small. In general, the model training effect has reached the desired level.
To measure the predictive performance of trained RFR model, a root mean square error (RMSE) and a coefficient of determination (R2) are introduced as evaluation indexes. The RMSE is a commonly used method to express numerical errors, representing the sample standard deviation of the difference between the predicted value and the actual one. It can be calculated by using the following formula.
RMSE = 1 n t = 1 n ( y ^ t y t ) 2
The R2 reflects how much the regression relationship can account for changes to the dependent variable. A higher value indicates that the regression model can produce better prediction results. The corresponding calculation process is shown below.
R 2 = 1 t = 1 n y ^ t y t ) 2 t = 1 n ( y t y ¯ ) 2

4. Test Platform Construction

4.1. Overview of the Upper Extremity Exosuit

We intend to take advantage of the abovementioned research to design a control logic for an upper extremity exosuit, so that it can perform rehabilitation training functions according to human intentions. As shown in Figure 7, this wearable system aims at providing active assistance for shoulder flexion/extension, shoulder adduction/abduction, and elbow flexion/extension of the left arm.
It contains three sets of cable-driven modules, each of which is responsible for driving the bidirectional motion for one degree of freedom. The sensing network consists of three IMUs, six tension sensors, and three absolute encoders which are integrated in motors, and are in charge of completing multiple tasks, such as MMG signal collection, limb posture perception, human-machine interaction information acquisition, and servo motor state reading. As the main control board, STM32F407IGH6 will serve as the brain of the system to perform core functions such as feature extraction, motion intent identification, and motor servo control. Components communicate with each other through CAN bus for data feedback and instruction delivery.
On the basis of the abovementioned hardware, the exosuit can be driven to assist the human limb coupled with a suitable control algorithm.

4.2. Torque Estimation-Based Control Strategy

As shown in Figure 8, the control logic framework of the upper extremity exosuit consists of two layers, namely, the intent analysis part based on torque estimation, and the motion control part based on torque closed-loop.
When subjects wear this exosuit for collaborative movement, the upper controller will obtain the triaxial accelerations from the target muscle through the accelerometer embedded in the IMU, and synthesize them into an original MMG. After completing the EMD-based filtering operation, it screens out relatively pure signals and extracts the three characteristics including RMS, MPF, and SampEn. The trained RFR model uses these features as an input to estimate the expected joint torque value at the current moment. Finally, the corresponding control commands will be sent to the lower layer.
The lower controller calculates the actual joint torque using the tension sensor readings at the end of the Bowden cable and compares it with the expected joint torque received from the upper layer to obtain their error value. Then, a standard PID algorithm generates motor drive commands based on the torque error, and controls the cable-driven module to provide appropriate assistance at the joint.
To realize this control strategy through a program code, we develop an embedded software based on the μC/OS III operating system. Five sub-tasks, including sensing data reception, signal processing, feature extraction, torque estimation, and motor servo control, are set up in order of priority from high to low. The execution frequency of each is assigned by setting different cycle times. Through the division of the abovementioned sub-task modules, we strengthen the real-time performance of programs under the premise of clarifying the control code logic for the upper extremity exosuit. In addition, it is convenient for subsequent optimization work.

5. Experiment on Exosuit

5.1. Reliability Analysis Experiment for Torque Estimation

The parameters of the RFR model are determined by offline training on a PC. After transplantation to the control system of the exosuit, an evaluation experiment needs to be carried out to examine its actual application effect for different people. We recruited three male volunteers aged 22–27 to complete this experiment. Among them, two subjects (marked as Subject 1 and Subject 2) who have taken part in the training data set collection for torque estimation, are selected to join the experimental group, and one (marked as Subject 3), who did not participate in that process, is assigned to the control group. It is worth noting that volunteers should not have done any high-intensity exercise 24 hours before the tests, to avoid affecting the physiological state of the muscles. During the experiment, they are told to exert an external force that changes approximately in accordance with the sine law on the measurement platform. All subjects knew and agreed with relevant experimental procedures in advance. Research related to this article was approved by the Laboratory Academic Committee of the State Key Laboratory of Robotics and System, Harbin Institute of Technology.
The embedded system, mounted on an upper extremity exosuit, calculates the estimated torque in real time, and sends them to a PC after being processed. The sensor on the measurement platform obtains the force data at the end of the arm, which is converted into the actual torque value in the PC. As it only aims to evaluate the reliability of torque estimation, we have shielded the subtask of the motor servo control in the program, so as to avoid the influence of man-machine coupling.
Figure 9 demonstrates the elbow joint torque estimation results of the upper extremity exosuit on three subjects. In the experimental group, it is obvious that the estimated torque looks very close to the actual value in terms of magnitude and variation trend. Under this condition, the model performance behaves in a relatively stable manner, and the identification result remains rather accurate; however, in the control group, the estimated torque cannot effectively follow the change of the actual value.
We use the RMSE and R2 introduced above to quantitatively describe the identification effect for different subjects. It can be seen from Table 1 that the RMSE of the experimental group is lower than that of the control group, indicating that the error between the actual and estimated value is smaller for the joint torque of Subject 1 and Subject 2. Moreover, the R2 in the experimental group comes up to 100%, which, when closely compared with the control group, means that the trained RFR model can perform better when utilizing the biological signals of Subject 1 and Subject 2.
From the above qualitative and quantitative description, the following two conclusions can be obtained.
  • If we have collected a person’s MMG signal for training the RFR model offline, the reliability of the online torque estimation will remain at a pretty high level when they wear the exosuit that uses the trained model;
  • utilizing a trained model to estimate the joint torque of unknown subjects online may significantly weaken the effectiveness of identification.
The reason may be that muscle activation varies among different people when they output the same joint torque, or different thicknesses of adipose layers more or less influences MMG propagation; therefore, when using the exosuit for power assistance, it is necessary to independently train a matching torque estimation model for the wearer based on his/her biological information.

5.2. Efficiency Evaluation Experiment for Power Assistance

In order to verify the actual power-assisted effect of this method, we selected a healthy subject, and collected his MMG signals at the brachioradialis, deltoid, and ectopectoralis to customize a set of RFR models for him. After transplanting these trained models to the embedded system, and enabling all the subtasks of control program, the subject wears the exosuit to perform elbow static flexion/extension, shoulder static flexion/extension, and shoulder static adduction/abduction on the measurement platform, and tries to complete three evaluation experiments. Other conditions and requirements are basically the same as the above experiment. An emergency stop switch needs to be held by the right hand all the way through the experiment, to ensure that the experiment can be stopped in time if an accident occurs.
Figure 10 shows the performance evaluation experiments for joint movement assistance. We take three torque values, which are estimated by the RFR model, calculated by the tension sensor on the cable, and converted by the six-dimension force sensor on the measurement platform as human-exerted, exosuit-generated, and the total output, respectively.
Figure 11 describes the changing situation of different torques in the typical time period of each motion mode. Obviously, it can be seen that the upper extremity exosuit can produce additional assistance in the three degrees of freedom of the shoulder and elbow joints, although its actual output is smaller than the torque estimated by the physiological signal. This error can be attributed to the loss of power transmission caused by friction between cable and sheath, or the calculation model deviation induced by suit deformation. Moreover, the total output far exceeds the human effort, indicating that this exosuit can significantly enhance joint strength.
We introduce the sum of absolute values (ASUM) to quantitatively reflect the average level of the three torques in different motion modes, and the corresponding results are shown in Figure 12.
ASUM = 1 n i = 0 n | T i |
It is obvious that the sum of torque generated by the exosuit, and that exerted by a human, is not equal to the actual total output. The combined effect of factors such as identification error, transmission error, and calculation error, may lead to this gap between the expected and the actual. We expect to describe the power-assisted efficiency (P) through analyzing the ratio of ASUMexosuit to ASUMtotal.
P = ASUM exosuit ASUM total
The calculation results show that when the upper extremity exosuit independently assists elbow static flexion/extension, shoulder static flexion/extension, and shoulder static adduction/abduction, the corresponding power-assisted efficiencies come up to 30.81%, 29.66%, and 25.78%, respectively. These data mean that when a person is equipped with this wearable robot, the output torque for each joint of the upper limb can be roughly reduced by a quarter to a third.

6. Conclusions and Future Work

In this article, we propose a MMG-based joint torque estimation algorithm which realizes the decoding from biological signals to limb strength, and transplant it into the control system of an exosuit to calculate the motor commands for assisting the multi-joint motion of the upper limb. Two sets of experiments are carried out to test the reliability of torque estimation and the efficiency of power assistance.
The data collection and signal processing methods used in this paper effectively establish the data sets which reflect human body information. The specially designed measurement platform can obtain the MMG of muscles and corresponding joint torque in a relatively accurate and convenient way. The HHT successfully eliminates the interference components in the original MMG signal, and lays a solid foundation for future extraction.
A technical approach to estimate joint torque from the MMG signal is built through training the RFR model offline. The reliability analysis experiment shows that this method can enable the exosuit to accurately identify the wearer’s current joint torque, but the model parameters need to be specially trained for everyone.
A torque estimation-based control strategy is successfully applied to the motion control of the upper extremity exosuit. The efficiency evaluation experiment indicates that the exosuit using this algorithm can significantly enhance the limb strength of wearers.
Based on the actual execution of the research, we believe that the current work has the following limitations. First of all, the nonlinear disturbance caused by transmission friction and motion hysteresis significantly reduces the control performance and power-assisted efficiency of the upper extremity exosuit. In addition, the MMG-based torque estimation algorithm has limitations in its application. The model parameters may be trained separately for each person, and even each muscle.
Therefore, future work and research directions should aim to break through the above limitations. First, an error compensation algorithm for this cable-driven system should be introduced into the control logic, in an attempt to offset interferences caused by nonlinear characteristics. Second, more general intention recognition algorithms need to be studied further, which can meet the usage requirements of every wearer without additional training preparation.

Author Contributions

Methodology, data analysis, writing, Y.S.; conceptualization, project administration, W.D.; experimental verification, data preprocessing, W.L.; resources, supervision, L.H.; schematic analysis, resources, X.W.; structure design, resources, P.L.; formal analysis, software, Y.G. All authors have read and agreed to the published version of the manuscript.

Funding

This work was funded by the pre-research project in the manned spacefight field of China (Project Number 020202).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Acknowledgments

The authors thank all those who have provided suggestions and assistance for this research and paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Majidi Fard Vatan, H.; Nefti-Meziani, S.; Davis, S.; Saffari, Z.; El-Hussieny, H. A review: A Comprehensive Review of Soft and Rigid Wearable Rehabilitation and Assistive Devices with a Focus on the Shoulder Joint. J. Intell. Robot. Syst. 2021, 102, 9. [Google Scholar] [CrossRef]
  2. Sanjuan, J.D.; Castillo, A.D.; Padilla, M.A.; Quintero, M.C.; Gutierrez, E.E.; Sampayo, I.P.; Hernandez, J.R.; Rahman, M.H. Cable driven exoskeleton for upper-limb rehabilitation: A design review. Robot. Auton. Syst. 2020, 126, 103445. [Google Scholar] [CrossRef]
  3. Pérez Vidal, A.F.; Rumbo Morales, J.Y.; Ortiz Torres, G.; Sorcia Vázquez, F.d.J.; Cruz Rojas, A.; Brizuela Mendoza, J.A.; Rodríguez Cerda, J.C. Soft Exoskeletons: Development, Requirements, and Challenges of the Last Decade. Actuators 2021, 10, 166. [Google Scholar] [CrossRef]
  4. Chen, C.; Wu, X.; Liu, D.; Feng, W.; Wang, C. Design and Voluntary Motion Intention Estimation of a Novel Wearable Full-Body Flexible Exoskeleton Robot. Mob. Inf. Syst. 2017, 2017, 8682168. [Google Scholar] [CrossRef] [Green Version]
  5. Serrancoli, G.; Falisse, A.; Dembia, C.; Vantilt, J.; Tanghe, K.; Lefeber, D.; Jonkers, I.; De Schutter, J.; De Groote, F. Subject-Exoskeleton Contact Model Calibration Leads to Accurate Interaction Force Predictions. IEEE Trans. Neural Syst. Rehabil. Eng. 2019, 27, 1597–1605. [Google Scholar] [CrossRef] [PubMed]
  6. Zhang, X.; Li, H.; Lu, Z.; Yin, G. Homology Characteristics of EEG and EMG for Lower Limb Voluntary Movement Intention. Front. Neurorobot. 2021, 15, 642607. [Google Scholar] [CrossRef]
  7. Saravanan, A.; Jangir, N.K.; Chakki, S. A Method to Control Bionic Arm Using Galvanic Skin Response. In Proceedings of the 8th International Conference on Communication Systems and Networks (COMSNETS), Bangalore, India, 5–10 January 2016. [Google Scholar]
  8. Benabid, A.L.; Costecalde, T.; Eliseyev, A.; Charvet, G.; Verney, A.; Karakas, S.; Foerster, M.; Lambert, A.; Morinière, B.; Abroug, N.; et al. An exoskeleton controlled by an epidural wireless brain–machine interface in a tetraplegic patient: A proof-of-concept demonstration. Lancet Neurol. 2019, 18, 1112–1122. [Google Scholar] [CrossRef]
  9. Li, Z.; Huang, Z.; He, W.; Su, C.-Y. Adaptive Impedance Control for an Upper Limb Robotic Exoskeleton Using Biological Signals. IEEE Trans. Ind. Electron. 2017, 64, 1664–1674. [Google Scholar] [CrossRef]
  10. Pan, C.-T.; Chang, C.-C.; Yang, Y.-S.; Yen, C.-K.; Kao, Y.-H.; Shiue, Y.-L. Development of MMG sensors using PVDF piezoelectric electrospinning for lower limb rehabilitation exoskeleton. Sens. Actuators A Phys. 2020, 301, 111708. [Google Scholar] [CrossRef]
  11. Li, Z.; Li, J.; Zhao, S.; Yuan, Y.; Kang, Y.; Chen, C.L.P. Adaptive Neural Control of a Kinematically Redundant Exoskeleton Robot Using Brain-Machine Interfaces. IEEE Trans. Neural Netw. Learn. Syst. 2019, 30, 3558–3571. [Google Scholar] [CrossRef]
  12. Gordleeva, S.Y.; Lukoyanov, M.V.; Mineev, S.A.; Khoruzhko, M.A.; Mironov, V.I.; Kaplan, A.Y.; Kazantsev, V.B. Exoskeleton Control System Based on Motor-Imaginary Brain–Computer Interface. Sovrem. Tehnol. V Med. 2017, 9, 31–36. [Google Scholar] [CrossRef] [Green Version]
  13. Trigili, E.; Grazi, L.; Crea, S.; Accogli, A.; Carpaneto, J.; Micera, S.; Vitiello, N.; Panarese, A. Detection of movement onset using EMG signals for upper-limb exoskeletons in reaching tasks. J. Neuroeng. Rehabil. 2019, 16, 45. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Xu, H.; Xiong, A. Advances and Disturbances in sEMG-Based Intentions and Movements Recognition: A Review. IEEE Sens. J. 2021, 21, 13019–13028. [Google Scholar] [CrossRef]
  15. Esposito, D.; Andreozzi, E.; Fratini, A.; Gargiulo, G.D.; Savino, S.; Niola, V.; Bifulco, P. A Piezoresistive Sensor to Measure Muscle Contraction and Mechanomyography. Sensors 2018, 18, 2553. [Google Scholar] [CrossRef] [Green Version]
  16. Woodward, R.B.; Shefelbine, S.J.; Vaidyanathan, R. Pervasive Monitoring of Motion and Muscle Activation: Inertial and Mechanomyography Fusion. IEEE/ASME Trans. Mechatron. 2017, 22, 2022–2033. [Google Scholar] [CrossRef]
  17. Su, H.; Hu, Y.; Karimi, H.R.; Knoll, A.; Ferrigno, G.; De Momi, E. Improved recurrent neural network-based manipulator control with remote center of motion constraints: Experimental results. Neural Netw. 2020, 131, 291–299. [Google Scholar] [CrossRef] [PubMed]
  18. Qi, W.; Ovur, S.E.; Li, Z.; Marzullo, A.; Song, R. Multi-Sensor Guided Hand Gesture Recognition for a Teleoperated Robot Using a Recurrent Neural Network. IEEE Robot. Autom. Lett. 2021, 6, 6039–6045. [Google Scholar] [CrossRef]
  19. Wu, H.; Huang, Q.; Wang, D.; Gao, L. A CNN-SVM combined model for pattern recognition of knee motion using mechanomyography signals. J. Electromyogr. Kinesiol. 2018, 42, 136–142. [Google Scholar] [CrossRef]
  20. Shiman, F.; Lopez-Larraz, E.; Sarasola-Sanz, A.; Irastorza-Landa, N.; Spuler, M.; Birbaumer, N.; Ramos-Murguialday, A. Classification of different reaching movements from the same limb using EEG. J. Neural. Eng. 2017, 14, 046018. [Google Scholar] [CrossRef]
  21. Xie, H.; Li, G.; Zhao, X.; Li, F. Prediction of Limb Joint Angles Based on Multi-Source Signals by GS-GRNN for Exoskeleton Wearer. Sensors 2020, 20, 1104. [Google Scholar] [CrossRef] [Green Version]
  22. Gautam, A.; Panwar, M.; Biswas, D.; Acharyya, A. MyoNet: A Transfer-Learning-Based LRCN for Lower Limb Movement Recognition and Knee Joint Angle Prediction for Remote Monitoring of Rehabilitation Progress From sEMG. IEEE J. Transl. Eng. Health Med. 2020, 8, 2100310. [Google Scholar] [CrossRef] [PubMed]
  23. Lu, L.; Wu, Q.; Chen, X.; Shao, Z.; Chen, B.; Wu, H. Development of a sEMG-based torque estimation control strategy for a soft elbow exoskeleton. Robot. Auton. Syst. 2019, 111, 88–98. [Google Scholar] [CrossRef]
  24. Gui, K.; Liu, H.; Zhang, D. A Practical and Adaptive Method to Achieve EMG-Based Torque Estimation for a Robotic Exoskeleton. IEEE/ASME Trans. Mechatron. 2019, 24, 483–494. [Google Scholar] [CrossRef]
  25. Ao, D.; Song, R.; Gao, J. Movement Performance of Human-Robot Cooperation Control Based on EMG-Driven Hill-Type and Proportional Models for an Ankle Power-Assist Exoskeleton Robot. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 25, 1125–1134. [Google Scholar] [CrossRef]
  26. Wu, Q.; Chen, B.; Wu, H. Neural-network-enhanced torque estimation control of a soft wearable exoskeleton for elbow assistance. Mechatronics 2019, 63, 102279. [Google Scholar] [CrossRef]
  27. Qi, W.; Aliverti, A. A Multimodal Wearable System for Continuous and Real-time Breathing Pattern Monitoring During Daily Activity. IEEE J. Biomed. Health Inform. 2020, 24, 2199–2207. [Google Scholar] [CrossRef]
  28. Qi, W.; Wang, N.; Su, H.; Aliverti, A. DCNN based human activity recognition framework with depth vision guiding. Neurocomputing 2021, 486, 261–271. [Google Scholar] [CrossRef]
  29. Dinh, B.K.; Xiloyannis, M.; Cappello, L.; Antuvan, C.W.; Yen, S.-C.; Masia, L. Adaptive backlash compensation in upper limb soft wearable exoskeletons. Robot. Auton. Syst. 2017, 92, 173–186. [Google Scholar] [CrossRef]
  30. Dinh, B.K.; Xiloyannis, M.; Antuvan, C.W.; Cappello, L.; Masia, L. Hierarchical Cascade Controller for Assistance Modulation in a Soft Wearable Arm Exoskeleton. IEEE Robot. Autom. Lett. 2017, 2, 1786–1793. [Google Scholar] [CrossRef]
  31. Little, K.; Antuvan, C.W.; Xiloyannis, M.; de Noronha, B.A.P.S.; Kim, Y.G.; Masia, L.; Accoto, D. IMU-based assistance modulation in upper limb soft wearable exosuits. In Proceedings of the 16th IEEE International Conference on Rehabilitation Robotics(ICORR), Toronto, ON, Canada, 24–28 June 2019; pp. 1197–1202. [Google Scholar]
  32. Xiloyannis, M.; Chiaradia, D.; Frisoli, A.; Masia, L. Physiological and kinematic effects of a soft exosuit on arm movements. J. Neuroeng. Rehabil. 2019, 16, 29. [Google Scholar] [CrossRef]
  33. Lessard, S.; Pansodtee, P.; Robbins, A.; Baltaxe-Admony, L.B.; Trombadore, J.M.; Teodorescu, M.; Agogino, A.; Kurniawan, S. CRUX: A compliant robotic upper-extremity eXosuit for lightweight, portable, multi-joint muscular augmentation. In Proceedings of the International Conference on Rehabilitation Robotics (ICORR), London, UK, 17–20 July 2017; pp. 1633–1638. [Google Scholar]
  34. Lessard, S.; Pansodtee, P.; Robbins, A.; Trombadore, J.M.; Kurniawan, S.; Teodorescu, M. A Soft Exosuit for Flexible Upper-Extremity Rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng. 2018, 26, 1604–1617. [Google Scholar] [CrossRef] [PubMed]
  35. Hosseini, M.; Meattini, R.; San-Millan, A.; Palli, G.; Melchiorri, C.; Paik, J. A sEMG-Driven Soft ExoSuit Based on Twisted String Actuators for Elbow Assistive Applications. IEEE Robot. Autom. Lett. 2020, 5, 4094–4101. [Google Scholar] [CrossRef]
Figure 1. Information collection process during elbow static flexion/extension: (a) measurement platform; (b) joint torque calculation model for elbow.
Figure 1. Information collection process during elbow static flexion/extension: (a) measurement platform; (b) joint torque calculation model for elbow.
Electronics 11 01335 g001
Figure 2. The IMU data during elbow dynamic flexion/extension with low strength: (a) three-axis euler angles; (b) three-axis accelerations.
Figure 2. The IMU data during elbow dynamic flexion/extension with low strength: (a) three-axis euler angles; (b) three-axis accelerations.
Electronics 11 01335 g002
Figure 3. MMG signal processing: (a) decomposing result through EMD; (b) autocorrelation function curves of the first four IMFs.
Figure 3. MMG signal processing: (a) decomposing result through EMD; (b) autocorrelation function curves of the first four IMFs.
Electronics 11 01335 g003
Figure 4. Comparison between the filtered MMG of the brachioradialis muscle and the corresponding elbow joint torque: (a) the change curve of joint torque; (b) the change curve of filtered MMG.
Figure 4. Comparison between the filtered MMG of the brachioradialis muscle and the corresponding elbow joint torque: (a) the change curve of joint torque; (b) the change curve of filtered MMG.
Electronics 11 01335 g004
Figure 5. Schematic diagram of RFR algorithm.
Figure 5. Schematic diagram of RFR algorithm.
Electronics 11 01335 g005
Figure 6. Machine learning effect of shoulder static adduction/abduction: (a) iterative training process with training set; (b) verification result of test set.
Figure 6. Machine learning effect of shoulder static adduction/abduction: (a) iterative training process with training set; (b) verification result of test set.
Electronics 11 01335 g006
Figure 7. Pictures of an upper extremity exosuit: (a) the front; (b) the back.
Figure 7. Pictures of an upper extremity exosuit: (a) the front; (b) the back.
Electronics 11 01335 g007
Figure 8. Torque estimation-based control strategy.
Figure 8. Torque estimation-based control strategy.
Electronics 11 01335 g008
Figure 9. Elbow joint torque estimation results: (a) experimental result of Subject 1; (b) experimental result of Subject 2; (c) experimental result of Subject 3.
Figure 9. Elbow joint torque estimation results: (a) experimental result of Subject 1; (b) experimental result of Subject 2; (c) experimental result of Subject 3.
Electronics 11 01335 g009
Figure 10. Actual power-assisted experiments: (a) experiment for elbow static flexion/extension; (b) experiment for shoulder static flexion/extension; (c) experiment for shoulder static adduction/abduction.
Figure 10. Actual power-assisted experiments: (a) experiment for elbow static flexion/extension; (b) experiment for shoulder static flexion/extension; (c) experiment for shoulder static adduction/abduction.
Electronics 11 01335 g010
Figure 11. Variations of different torques in each motion mode: (a) torque curves for elbow static flexion; (b) torque curves for shoulder static flexion; (c) torque curves for shoulder static abduction.
Figure 11. Variations of different torques in each motion mode: (a) torque curves for elbow static flexion; (b) torque curves for shoulder static flexion; (c) torque curves for shoulder static abduction.
Electronics 11 01335 g011
Figure 12. ASUM of different torques under each joint movement.
Figure 12. ASUM of different torques under each joint movement.
Electronics 11 01335 g012
Table 1. Evaluation results of torque estimation on three subjects.
Table 1. Evaluation results of torque estimation on three subjects.
GroupsParticipantsRMSER2
Experimental groupSubject 11.98120.9532
Subject 21.70080.8620
Control groupSubject 33.42610.6824
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Shi, Y.; Dong, W.; Lin, W.; He, L.; Wang, X.; Li, P.; Gao, Y. Human Joint Torque Estimation Based on Mechanomyography for Upper Extremity Exosuit. Electronics 2022, 11, 1335. https://doi.org/10.3390/electronics11091335

AMA Style

Shi Y, Dong W, Lin W, He L, Wang X, Li P, Gao Y. Human Joint Torque Estimation Based on Mechanomyography for Upper Extremity Exosuit. Electronics. 2022; 11(9):1335. https://doi.org/10.3390/electronics11091335

Chicago/Turabian Style

Shi, Yongjun, Wei Dong, Weiqi Lin, Long He, Xinrui Wang, Pengjie Li, and Yongzhuo Gao. 2022. "Human Joint Torque Estimation Based on Mechanomyography for Upper Extremity Exosuit" Electronics 11, no. 9: 1335. https://doi.org/10.3390/electronics11091335

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop