Gait Synergy Analysis and Modeling on Amputees and Stroke Patients for Lower Limb Assistive Devices

The concept of synergy has drawn attention and been applied to lower limb assistive devices such as exoskeletons and prostheses for improving human–machine interaction. A better understanding of the influence of gait kinematics on synergies and a better synergy-modeling method are important for device design and improvement. To this end, gait data from healthy, amputee, and stroke subjects were collected. First, continuous relative phase (CRP) was used to quantify their synergies and explore the influence of kinematics. Second, long short-term memory (LSTM) and principal component analysis (PCA) were adopted to model interlimb synergy and intralimb synergy, respectively. The results indicate that the limited hip and knee range of motions (RoMs) in stroke patients and amputees significantly influence their synergies in different ways. In interlimb synergy modeling, LSTM (RMSE: 0.798° (hip) and 1.963° (knee)) has lower errors than PCA (RMSE: 5.050° (hip) and 10.353° (knee)), which is frequently used in the literature. Further, in intralimb synergy modeling, LSTM (RMSE: 3.894°) enables better synergy modeling than PCA (RMSE: 10.312°). In conclusion, stroke patients and amputees perform different compensatory mechanisms to adapt to new interlimb and intralimb synergies different from healthy people. LSTM has better synergy modeling and shows a promise for generating trajectories in line with the wearer’s motion for lower limb assistive devices.


Introduction
Nowadays, lower limb assistive devices are designed to help subjects with motor impairments, such as amputees and stroke patients, restore locomotion function. Significant achievements have been made for lower limb exoskeletons and active prostheses such as HAL [1], Lokomat [2], powered ankle-foot prostheses [3] and powered transfemoral prostheses [4]. In our group, we have developed an active lower limb exoskeleton [5], knee brace [6], and ankle-foot prosthesis [7] to help individuals who have lost mobility or suffered from pathological gait restore/regain locomotion function. Active assistive devices were designed to help the injured joints or replace the amputated joints to reproduce the required function. To achieve that, these assistive devices need to identify the intention/locomotion mode of the user and then automatically adjust the controllers to provide the desired assistance.
The muscle synergy hypothesis [8][9][10][11][12] and the concept of synergy have drawn lots of concerns and been applied to lower limb assistive devices. In neurology, various human

Gait Data Acquisition
The research subjects of this study are healthy people, stroke survivors, and amputees. To analyze and model their gait synergy, the first step is to obtain sufficient gait data. We collected gait data from healthy, amputee and stroke subjects (listed in Table 1). During the experiments, written consent was obtained from each subject. First, 8 male able-bodied subjects without gait pathology (height: 1.73 ± 0.07 m, weight: 61.6 ± 5.2 kg) were recruited in the healthy group. All subjects wore an IMU-based wearable motion capture system (Perception Neuron, 1.0, Noitom, Beijing, China). The IMU-based system can measure joint angle for gait analysis [33][34][35]. The wearable motion capture system is convenient, with acceptable accuracy as compared to the traditional optical motion capture system [36,37]. Multiple cycles of gait data can be recorded continuously. Seven inertial measurement unit (IMU) nodes were placed on the wearer's lower limbs by tight straps with Velcro. The acceleration, angular velocity, and quaternion of the lower limb parts and angle of different joints could be obtained from the wearable system. Gait experiments were conducted on level ground (a 19 m walkway). Subjects were instructed to walk at their self-selected speeds. Enough rest was given between two trials. There are 20 trials for each subject.
Second, 12 stroke subjects were recruited from a hospital ( Figure 1) based on the criteria: (1) left or right hemiparesis; (2) over 2 months since onset; and (3) able to walk 12 m without help from others. Similarly, all subjects were asked to walk on a 6 m walkway with the same wearable motion capture system (Noitom ® Perception Neuron, 1.0). Considering their leg weakness, they walked 5 trials at their self-selected speeds and enough rest was given between each trial. All of them were informed that they are allowed to leave the Second, 12 stroke subjects were recruited from a hospital (Figure 1) based on the criteria: (1) left or right hemiparesis; (2) over 2 months since onset; and (3) able to walk 12 m without help from others. Similarly, all subjects were asked to walk on a 6 m walkway with the same wearable motion capture system (Noitom ® Perception Neuron, 1.0). Considering their leg weakness, they walked 5 trials at their self-selected speeds and enough rest was given between each trial. All of them were informed that they are allowed to leave the walking test any time they wanted. There were 11 subjects (height: 1.66 ± 0.07 m; weight: 49.9 ± 8.5 kg; onset time: 16.3 ± 21.8 mo) who completed the test, since 1 subject quit after the first trial. Finally, we obtained amputee gait data from 8 unilateral transfemoral amputee subjects (height: 1.75 ± 0.07 m; weight: 70.3 ± 9.1 kg; time since amputation: 10.2 ± 6.3y) from [38]. They all had worn prostheses for over six months. No gait-related diseases were reported by them. Each subject walked on a 10 m path at a self-selected speed.

Synergy Analysis
CRP is adopted to quantify the disturbed interlimb and intralimb synergies of the stroke patients and amputees and then explore the influence of kinematics on the disturbed synergies.

Continuous Relative Phase
CRP is adopted to quantify the disturbed interlimb and intralimb synergies of the stroke patients and amputees and then explore the influence of kinematics on the disturbed synergies. In this study, we use a Hilbert transform to calculate the CRP. The Hilbert transform was suggested by Peters et al. [28] to analyze interlimb and intralimb gait synergies. First, phase angles are calculated based on a measured signal x(t) and its Hilbert transform H(t) = H(x(t)): Here, P.V. means that the integral is based on the Cauchy principal value. Then the phase angle can be calculated by: Finally, we obtained amputee gait data from 8 unilateral transfemoral amputee subjects (height: 1.75 ± 0.07 m; weight: 70.3 ± 9.1 kg; time since amputation: 10.2 ± 6.3y) from [38]. They all had worn prostheses for over six months. No gait-related diseases were reported by them. Each subject walked on a 10 m path at a self-selected speed.

Synergy Analysis
CRP is adopted to quantify the disturbed interlimb and intralimb synergies of the stroke patients and amputees and then explore the influence of kinematics on the disturbed synergies.

Continuous Relative Phase
CRP is adopted to quantify the disturbed interlimb and intralimb synergies of the stroke patients and amputees and then explore the influence of kinematics on the disturbed synergies. In this study, we use a Hilbert transform to calculate the CRP. The Hilbert transform was suggested by Peters et al. [28] to analyze interlimb and intralimb gait synergies. First, phase angles are calculated based on a measured signal x(t) and its Hilbert transform H(t) = H(x(t)): Here, P.V. means that the integral is based on the Cauchy principal value. Then the phase angle can be calculated by: The CRP at time t CRP t between the two signals x 1 (t) and x 2 (t) can be calculated by: A self-written program based on MATLAB (The Mathworks Inc., Natick, MA, USA) was used for synergy analysis on the data from the able-bodied, stroke and amputated subjects. One representative gait cycle's joint angles were randomly selected for each subject's limbs in each trial. Then, the joint angles of one gait cycle were interpolated to 100% for convenience. The data were then curve fitted by the sum of sinusoid functions using the curve-fitting tool in MATLAB. The riding waves and uneven data were eliminated.
The root mean square error (RMSE) and Pearson correlation coefficient (PCC) were used to measure the similarities and correlations of the two CRPs, respectively. The RMSE denotes the deviations or closeness between two curves. The following equation calculates the RMSE between the two CRP values CRP 1 and CRP 2 : PCC measures the matching degree between two groups of values.

Decomposition Index
To better analyze interlimb and intralimb synergies, we also adopt the concept of the decomposition index (DI), which is a scale proposed by Bastian et al. [39] to quantify decomposition movements. A decomposition movement occurs when two adjacent or related joints have different movements (one moves while the other pauses). The DI is the percentage of the time of decomposition movement in a gait cycle: In the above equation, T dec is the duration of decomposition movement in a gait cycle. T Gait refers to the duration of a gait cycle. By definition, when the angular velocity of a joint is less than 5 • /s, it is regarded as paused. However, when the angular velocity is higher than 5 • /s, it is considered to be moving. For the lower limbs, there are three pairs of joints (hip-knee, knee-ankle and hip-ankle) that have DIs. In this study, the DI of the hip-knee joint is considered to help analyze interlimb and intralimb synergies. Then, the relationships between the DI and different kinds of CRPs were analyzed. Note that the decomposition movement is not considered a physical defect but a compensatory movement for the gait. The changes in the DI reflect subjects' motion adjustments. It is thus interesting to explore the decomposition movements and DI in both stroke patients and amputees.

Stepwise Regression
Here, we explore the relationship between gait kinematics and synergies in stroke patients and amputees. Stepwise multiple linear regression was performed to identify the determinants of interlimb and intralimb synergies (CRPs). Stepwise regression is a step-by-step process to select independent variables to build a regression model. In each step, a variable is considered to be added or removed from the variables based on the defined criterion. After each iteration, statistical significance was tested. Before stepwise regression, the collinearities among independent variables were diagnosed. The analysis was performed on SPSS v19 from IBM, with a significance level of 0.1. For example, when analyzing gait data of the affected sides of stroke patients, DI sw , DI sta , percentage, ROM hip , ROM knee , and speed were selected for collinearity diagnostics.

Synergy Modeling
LSTM was adopted to model the interlimb and intralimb synergies of able-bodied subjects, respectively. Then, we also used PCA, which is commonly used in the literature to model healthy subjects' different synergies. At last, comparisons were made between the experimental results of LSTM and PCA on the modeling of interlimb and intralimb synergies.

Long Short-Term Memory Neural Network
LSTM networks are commonly used recurrent neural networks. LSTM is popular in recent years for its ability to regain long-term information. For example, LSTM can be employed to classify human activity [40], forecast PM 2.5 [41], and monitor machine health [42]. The following equations give the algorithm for an LSTM model with forget gates: where the internal state s (t) is influenced by the forget gate f (t) , and where (σ) stands for sigmoid functions. The internal state s (t) is influenced by the forget gate f. W 1 , W 2 and b stand for the corresponding weight matrices and bias parameters, respectively. All the parameters in Equations (7)-(10) are determined during multiple training runs. Interlimb and intralimb synergies, the relationships among the kinematics in the same or different leg, can be regarded as time-series models, thus LSTM is employed to model them.

Interlimb Synergy Modeling
LSTM is employed to model interlimb synergy, the relationship among the kinematics of the two lower limbs. Thus, an LSTM model is built where the angular velocities and angle data of one side's limb are set as the input and the angle data (knee or hip joint) of the other side's limb are the output. Inter-subject simulation is conducted to investigate the universality of the modeled "healthy" interlimb synergy. Specifically, for motion data of any randomly selected subject from the 8 healthy subjects, we estimate one side's knee and hip angles using the contralateral motion data (angles and angular velocities) based on the interlimb synergy modeled from the other 7 subjects. Then, the estimated knee and hip angles of one side are compared with the original data (ground truth). There are 8 turns where each subject's data become the testing data and the LSTM model is trained by the data of the other 7 subjects. The RMSE, Pearson correlation, R 2 , and Mean Absolute Error (MAE) are adopted to quantify the estimation performance.
To have a fair comparison, we also employ PCA-based linear regression to perform simulations following the steps in [19,43] based on the above-mentioned protocols. Note that we can finally obtain the linear equations (i.e., the interlimb synergy) for each subject's data between one side's hip or knee angle and the contralateral motion data. Estimation errors during the 8 turns of the simulations were calculated.

Intralimb Synergy Modeling
LSTM is also employed to model intralimb synergy, the relationship among the kinematics of two joints in the same leg. Previously, Hernandez et al. [44] also used artificial neural networks to model intralimb synergy and then estimated knee angle. However, 5 IMUs were used. In [45], we compared our LSTM-based method with other existing methods that used different types or numbers of sensors and proved to have better esti-mation performance. Again, we also employed PCA-based linear regression and calculate simulation errors to have a fair comparison.
In LSTM modeling, the original output of one IMU on the wearer's thigh (angular velocities and accelerations) is set as the input while the wearer's knee angle data (measured by two IMUs) are the output. Inter-subject simulation is also conducted. We estimate each subject's knee angle based on the intralimb synergy modeled from the other 7 subjects. Then, the estimated knee angle is compared with the measured knee angle to quantify the estimation performance. Again, there are 8 turns in total.

Summary of Gait Data
As afore-mentioned, we obtained gait kinematics from stroke, able-bodied, and amputee subjects. Table 2 lists all gait parameters involved in this research (all values are means). There are interlimb and intralimb CRPs quantifying the interlimb and intralimb synergies of three groups of participants. Here, the RMSEs of the CRP values are all marked as CRP for convenience. Note that there are sound sides and affected sides for both the stroke and amputee groups. Interlimb CRP is marked as CRP (inter) . The CRP in the stance phase (CRP st ) and the swing phase (CRP sw ) are both considered. The percentage stands for the swing phase percentage, the percentage of swing phase of a whole gait cycle. First, from Table 2 and Figure 2, we can find that the RMSEs of the interlimb and intralimb CRPs (synergies) of the stroke group are larger than those of amputee group. The RMSEs of the CRPs in the amputee group are larger than those in the healthy group. Second, the stroke group's DI is larger than the amputee group's, while the amputee group's DI is larger than that of the healthy group. For both the stroke and amputee groups, the DI of the affected or amputated side is larger than the DI of the sound side. Furthermore, both stroke and amputee groups exhibit limited knee and hip range of motions (RoMs). The RoMs of the affected or amputated sides in both stroke and amputee groups are smaller than the sound side.

Synergy Analysis on Stroke and Amputee Subjects
Stepwise multiple linear regression is performed to explore the relationships among gait kinematics and synergies (including interlimb and intralimb synergies) in stroke patients and amputees. Note that only the model with the best regression performance is chosen and shown for each group. Thus, we summarize the factors that have significant influences on interlimb and intralimb synergies (CRPs) of the stroke, healthy, and amputee subjects in Table 3.

Synergy Analysis on Stroke and Amputee Subjects
Stepwise multiple linear regression is performed to explore the relationships among gait kinematics and synergies (including interlimb and intralimb synergies) in stroke patients and amputees. Note that only the model with the best regression performance is chosen and shown for each group. Thus, we summarize the factors that have significant influences on interlimb and intralimb synergies (CRPs) of the stroke, healthy, and amputee subjects in Table 3.   First, DIsta is the DI in the stance phase. DIsta is strongly related to the intralimb synergies of the stroke subject's sound side in the stance phase, those of amputee's amputated side, and those of healthy subjects. Further, it is strongly related to the interlimb synergy of stroke subjects in the stance phase. On the other hand, DIsw is a determinant of the intralimb synergy of stroke patients' affected sides and the interlimb synergy of amputees in the swing phase. Compared with healthy subjects, both limbs of the stroke patients and the amputated limbs of the amputees exhibit higher degrees of decomposition movements. This higher decomposition movement significantly influences their interlimb and intralimb synergies. However, amputee's sound side does not show strong decomposition movement. This may be because amputees and stroke patients perform different compensatory mechanisms to adapt to new interlimb and intralimb synergies, unlike healthy people.
Second, stroke patients and amputees exhibit limited RoMs in hip and knee joints, compared with healthy subjects. ROM knee is a determinant of the stroke patients' interlimb synergy and intralimb synergy in the stance phase. ROM hip has a strong correlation with the stroke patients' intralimb synergy in the swing phase. This may be because the stroke patients' sound-side legs make compensations in the push-off and swing phases [46], so as to restrict the movement of the inverted-pendulum swing. As a result, disturbed synergy was caused. For amputees, the RoMs of the hip and knee have significant impacts on the intralimb synergy of the amputated limb. Thus, we can conclude that limited RoMs in the hip and knee joints in stroke patients and amputees significantly influence their interlimb and intralimb synergies. However, this influence is different in the two groups. For example, stroke patients' interlimb synergy is mainly influenced by ROM knee , while ROM hip influences amputees' interlimb synergy.
Furthermore, walking speed is a determinant of the intralimb synergy of the sound side and the interlimb synergy for stroke groups. Patients with better CRPs may have greater walking speeds in the tests. This result shows that speed is important in rehabilitation training design for stroke groups.
The above findings can provide useful guidance in rehabilitation training design and assistive device improvement for stroke patients and amputees. The orange lines in Figure 3 show the estimated knee and hip angles by PCA-based regression in one subject. In each simulation session (eight in total), the RMSE, Pearson correlation, R 2 , and Mean Absolute Error (MAE) are adopted to quantify the estimation performance (average results are given in Table 3). The RMSEs in each session are plotted in Figure 4a. Note that, in each session, one subject's knee and hip angles are estimated based on the interlimb synergy modeled from the other seven subjects.

Interlimb Synergy Modeling
The Experimental results in Table 4 indicate that the LSTM model has good estimation performance and modeling on intralimb synergy. The mean RMSE for all simulation sessions is 0.796 • and 1.963 • for the hip and knee, respectively. The mean MAEs are 0.632 • (hip) and 1.412 • (knee). The mean Pearson correlations (0.998 and 0.996) and R 2 (0.996 and 0.993) are all very close to 1. In Figure 4a, we can find that the RMSEs of different simulation sessions (when one subject's joint angles are the testing data while the synergy is modeled from the other seven subjects' data) of the hip and knee angles are close. In addition, the other three scales in all simulation sessions are very close. It can be concluded that the LSTM-based intralimb synergy model has good universality on different subjects.

Experimental Results of PCA
The blue lines in Figure 3 show the estimated knee and hip angles by PCA-based regression in one subject. Note that the interlimb synergy is modeled from the other 7 subjects by PCA. The deviations between the PCA-estimated and original knee and hip trajectories are larger than those between the LSTM-estimated and original ones. For the RMSE, the RMSEs of 5.050 • (hip) and 10.353 • (knee) by PCA are larger than those of 0.796 • (hip) and 1.963 • (knee) by the LSTM, respectively (listed in Table 4). Moreover, compared with the results by PCA, the Pearson correlation and R 2 in simulations by LSTM are much closer to one. It can be concluded that LSTM has better extraction and modeling on interlimb synergy over PCA. The blue lines in Figure 3 show the estimated knee and hip angles by PCA-based regression in one subject. Note that the interlimb synergy is modeled from the other 7 subjects by PCA. The deviations between the PCA-estimated and original knee and hip trajectories are larger than those between the LSTM-estimated and original ones. For the RMSE, the RMSEs of 5.050° (hip) and 10.353° (knee) by PCA are larger than those of 0.796° (hip) and 1.963° (knee) by the LSTM, respectively (listed in Table 4). Moreover, compared with the results by PCA, the Pearson correlation and R 2 in simulations by LSTM are much closer to one. It can be concluded that LSTM has better extraction and modeling on interlimb synergy over PCA.

Intralimb Synergy Modeling
LSTM is also adopted to model intralimb synergy. The input of the LSTM model is the thigh kinematics (acceleration and angular velocity) while the output is knee angle. Other LSTM model setting are similar to the interlimb synergy LSTM model.
In the simulation, the RMSE, Pearson correlation, R 2 . and MAE are calculated. The RMSE in each session (when one subject's knee angles are the testing data while the synergy is modeled from the other seven subjects' data) is plotted in Figure 4b. In intralimb synergy modeling, the mean RMSE is 3.894 • and the mean MAE is 2.193 • . The mean Pearson correlation and R 2 are 0.981 and 0.961, respectively. Compared with the results by PCA (RMSE: 10.312 • and MAE: 8.448 • , listed in Table 4), the RMSE and MAE results by LSTM are smaller. Additionally, the Pearson correlation and R 2 in simulations by LSTM are much closer to one. Therefore, the simulation results over the eight subjects' data indicate that LSTM has smaller simulation errors. Additionally, the Pearson correlation and R 2 in simulations by LSTM are much closer to one. Therefore, the simulation results over the eight subjects' data indicate that the intralimb synergy model by LSTM has good universality over different subjects and has better modeling on interlimb synergy over PCA.
In the simulation, the RMSE, Pearson correlation, R 2 . and MAE are calculated. The RMSE in each session (when one subject's knee angles are the testing data while the synergy is modeled from the other seven subjects' data) is plotted in Figure 4b. In intralimb synergy modeling, the mean RMSE is 3.894° and the mean MAE is 2.193°. The mean Pearson correlation and R 2 are 0.981 and 0.961, respectively. Compared with the results by PCA (RMSE: 10.312° and MAE: 8.448°, listed in Table 4), the RMSE and MAE results by LSTM are smaller. Additionally, the Pearson correlation and R 2 in simulations by LSTM are much closer to one. Therefore, the simulation results over the eight subjects' data indicate that LSTM has smaller simulation errors. Additionally, the Pearson correlation and R 2 in simulations by LSTM are much closer to one. Therefore, the simulation results over the eight subjects' data indicate that the intralimb synergy model by LSTM has good universality over different subjects and has better modeling on interlimb synergy over PCA.

Discussions
In the first study, CRP is used to quantify interlimb and intralimb synergies in stroke patients and amputees. Both stroke patients and amputees have one impaired limb. The impaired limb shows limited hip and knee RoMs. They adopt different interlimb and in-  Figure 4. (a) RMSE of simulations of hip and knee angles by interlimb LSTM models; (b) RMSE of simulations of knee angles by intralimb LSTM models. There is one simulation session for each subject's data. For example, when estimating subject A's joint angles, the synergy (interlimb and intralimb) is modeled from the other 7 subjects' data. "Mean" is the average RMSE of different simulation sessions.

Discussions
In the first study, CRP is used to quantify interlimb and intralimb synergies in stroke patients and amputees. Both stroke patients and amputees have one impaired limb. The impaired limb shows limited hip and knee RoMs. They adopt different interlimb and intralimb synergies or gait patterns, different from healthy people, to compensate for the impaired function of one side. This compensation mechanism adapts them to a higher degree of decomposition movement. Additionally, the limited hip and knee RoMs significantly influence their synergy, but the influence is different in stroke patients and amputees. Stroke patients' interlimb synergy is mainly influenced by knee ROM while that of the hip influences amputees' interlimb synergy. This gives us an in-depth view of the motion of stroke patients and amputees. These findings can provide useful guidance in rehabilitation training design and assistive device improvement for stroke patients and amputees.
Second, we tried to find a better method to model synergy. LSTM is adopted to model the interlimb and intralimb synergies of able-bodied subjects. We modeled synergy from able-bodied subjects because we want to use "healthy" synergy in the control of lower limb assistive devices to guide the patients who have disturbed synergy. However, this "healthy" synergy needs to have universality over various subjects. Experiments were conducted for modeling interlimb and intralimb synergy.
In the experiments of interlimb synergy modeling, we also adopt PCA-based regression to model synergy from the same data and then compare the results by LSTM to have a comparison. Experimental results on data from eight subjects indicate that LSTM has better extraction and modeling on interlimb synergy over PCA, and the LSTM model has universality over different subjects. This idea can be applied to the trajectory generation for lower limb exoskeletons that are designed for stroke patients. In future applications, we can first model interlimb synergy from a large group of able-bodied subjects. Then, the trajectory of the rehabilitative exoskeleton (on the paretic side of the wearer) is generated online based on the wearer's motion data for the sound side and the trained synergy (LSTM model) to encourage the wearer's active engagement and provide advisable therapeutic effects. This trajectory is adaptive to different wearers and to one wearer's different gait. This idea is promising for further exploration since engagement is an important factor in stroke rehabilitation.
In the experiments on intralimb synergy modeling, the results indicate that the intralimb synergy model by LSTM has good universality over different subjects. This method can be used to estimate knee angle based on the kinematics of the thigh (on the same side). Additionally, the kinematics of the thigh are measured by only a single IMU. Our results are also compared with those by PCA and prove to enable better synergy modeling over PCA.
The intralimb synergy LSTM model can also be applied in lower limb assistive devices, such as transfemoral prostheses. It can help generate a harmonious knee trajectory in line with the amputee's residual limb's motion to improve human-machine interaction [18,19]. Moreover, the motion intention of the wearer can be deduced by the residual limb.

Conclusions
In this study, we analyzed disturbed synergies in stroke patients and amputees for lower limb assistive devices. CRP and LSTM are used to quantify and model interlimb and intralimb synergies, respectively. The relationship among gait kinematics and synergies in stroke patients and amputees has been revealed. First, they perform different compensatory mechanisms to adapt to new interlimb and intralimb synergies, altered from healthy people. The limited hip and knee RoMs significantly influence their different synergies in different ways. Second, LSTM enables better interlimb and intralimb synergy modeling and shows promise for generating trajectories in line with the wearer's motion and improving human-machine interaction for lower limb assistive devices such as exoskeletons and active prostheses.  Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.

Data Availability Statement:
The data that support the findings of this study are available from the corresponding author, W.H.L., upon reasonable request.