# Motor Function Evaluation of Hemiplegic Upper-Extremities Using Data Fusion from Wearable Inertial and Surface EMG Sensors

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Sensing Devices

#### 2.2. Subjects

#### 2.3. Design of Standard Testing Tasks

#### 2.4. Experimental Protocol

#### 2.5. Data Analysis

#### 2.5.1. Data Preprocessing and Segmentation

_{1}, y

_{1}, z

_{1}for the IMU1 and x

_{2}, y

_{2}, z

_{2}for the IMU2, respectively. The instantaneous summation of vector magnitudes from both gyroscopes was computed at each moment according to Equation (1). Then a magnitude thresholding approach was performed, where the threshold T

_{R}was set to be 3 degrees per second through many pretests. The onset time was determined to be the moment when S(t) rose over the threshold, while the moment when S(t) fell below and its following 2-s signals were kept below the threshold indicated the offset time. If S(t) momentarily fell down below the threshold within 2 s, the data within an entire task performance procedure could not be interrupted. Finally, the determined onset and offset were applied to all data channels (consisting of 10 EMG channels, 6 accelerometer axes and 6 gyroscope axes in total) to select a data segment corresponding to each task repetition. Figure 4 illustrates the data segmentation process in detail. The following feature extraction and motor function evaluation analyses were performed on these selected data segments:

#### 2.5.2. Feature Extraction

- Motion data profile (MDP): The profile of each data segment is a straightforward representation of the task performance. In order to calculate motion data profile, the recorded data were processed according to sensor type. For each channel of surface EMG signals, a moving average processing was first performed to produce an EMG envelope through calculating mean value of rectified EMG signals within a sliding window with a window length of 256 ms and a window increment of 8 ms. Then, all channel EMG envelopes were simultaneously normalized in amplitude by the maximal value among all envelop values in 10 channels. The 6-axis accelerometer data from two IMUs were normalized in magnitude by its maximal absolute value so as to keep the signals within the range between −1 and +1. The similar process was also applied to the 6-axis gyroscope data as well. Subsequently, the normalized data segment consisting of 10 EMG channels, six accelerometer axes and six gyroscope axes was further normalized in time to 256 sample points, to alleviate time duration variation of task performance. Finally, the motion data profile was produced as a 22 × 256 data matrix for each data segment.
- Time duration: The time duration of each data segment was specifically calculated to reflect proficiency of task performance, while such information was not involved in the above MDP due to the normalization process.
- IMU extremum number: Within each data segment, the number of local minima and maxima was computed for each axis of both IMUs and then summed up as a feature as well.
- EMG power distribution: After the root mean square (RMS) of each surface EMG channel was computed, the percentage of one channel EMG RMS to summation of the RMS values from all 10 channels was subsequently obtained, thus producing a 10-element vector indicating EMG power distribution across channels [30].
- IMU power distribution: After the root mean square (RMS) of each axis of accelerometer/ gyroscope was computed, the percentage of a RMS value for one accelerometer/gyroscope axis to summation of the RMS values over all three axes was subsequently obtained, thus producing four (from two accelerometers and two gyroscopes) 3-element vectors indicating movement power distribution across axes.
- Accelerometer/gyroscope intensity ratio: At each moment, a magnitude of the 3-axis vector of an accelerometer/gyroscope was computed. After the RMS value of the magnitude time series was calculated for each accelerometer/gyroscope, the ratio of such RMS value from the accelerometer/gyroscope in IMU1 to the RMS value in IMU2 was subsequently obtained as a feature.
- Mean and maximum value: A mean value and a maximum value of the magnitude time series for each accelerometer or gyroscope was computed, therefore producing eight features from both accelerometer and both gyroscopes.

#### 2.5.3. Motor Function Evaluation

**W**represents the transformed data matrix with feature dimensionality reduced to s, and ${H}^{-1}$ represents the transformation matrix. We set s = 1 in this study, since the produced EI should be a one-dimensional quantitative score. Three commonly used unsupervised algorithms for matrix decomposition are principal component analysis (PCA), multidimensional scaling (MDS), and non-negative matrix factorization (NMF). We examined their applications in this paper, with a brief introduction as follows:

- PCA algorithm: PCA is a very popular technique for dimensionality reduction. Given a set of high-dimensional data, PCA aims to find a linear subspace of lower dimension and such a reduced subspace attempts to maintain most of the variability of the data [31]. In the process of factorization,
**V**was centralized first to eliminate the influence of dimension. The transformation matrix ${H}^{-1}$ would be obtained by obtaining the eigenvalue and eigenvector of the covariance matrix of the centralized matrix. - MDS algorithm: MDS is another classical approach that maps the original high dimensional space to a lower dimensional space with an attempt to preserve pairwise distances [31]. In the process of performing the metric MDS, a squared proximity matrix is set, with elements ${d}_{ij}^{*}$ representing the Euclidean distances between high-dimensional sample i and j (i, j = 1, …, m and i ≠ j,). Sammon’s nonlinear mapping criterion was chosen as the goodness-of-fit criterion. It aims to minimize the loss function Stress [32] given in Equation (3), where ${d}_{ij}$ is the distance between low-dimensional sample i and j. These distances ${d}_{ij}$ is initialized to be random values and then updated via a iterative process using rules reported in [32] so as to minimize the Stress:$$Stres{s}_{D}\left({V}_{1},{V}_{2},\dots ,{V}_{m}\right)=\frac{1}{{{\displaystyle \sum}}_{i<j}{d}_{ij}^{*}}{\displaystyle \sum}_{i<j}\frac{{\left({d}_{ij}^{*}-{d}_{ij}\right)}^{2}}{{d}_{ij}^{*}}$$
- NMF algorithm: This method of matrix decomposition has previously and widely been used for muscle synergy analysis [18]. In this paper, NMF was used for dimensionality reduction just like the above two algorithms. In the process of factorization,
**W**and**H**were initialized to be random values first, and were updated using rules [18] given in Equation (4):$${W}_{\mathit{i}\mathit{j}}\mathbf{\leftarrow}{W}_{\mathit{i}\mathit{j}}\frac{{\mathbf{\left(}\mathit{V}{\mathit{H}}^{\mathit{T}}\mathbf{\right)}}_{\mathit{i}\mathit{j}}}{{\mathbf{\left(}\mathit{W}\mathit{H}{\mathit{H}}^{\mathit{T}}\mathbf{\right)}}_{\mathit{i}\mathit{j}}},{H}_{\mathit{j}\mathit{k}}\mathbf{\leftarrow}{H}_{\mathit{j}\mathit{k}}\frac{{\mathbf{\left(}{\mathit{W}}^{\mathit{T}}\mathit{V}\mathbf{\right)}}_{\mathit{j}\mathit{k}}}{{\mathbf{\left(}{\mathit{W}}^{\mathit{T}}\mathit{W}\mathit{V}\mathbf{\right)}}_{\mathit{j}\mathit{k}}},W\left(\mathbf{i}\right)\leftarrow \frac{\mathit{W}\left(\mathit{i}\right)}{\Vert \mathit{W}\left(\mathit{i}\right)\Vert}\mathbf{\forall}\text{}\mathrm{column}\prime i\prime ,$$

**V**and the vector of observations y, the LASSO solves the L1-penalized regression problem of finding a vector $z={\left({z}_{1},\cdots ,{z}_{m}\right)}^{T}$ to minimize the algebraic expression (5) [34,35]. First, z is initialized to be random values and then updated multiple times in iterations through the least angle regression-elastic net algorithm [36,37]. In this paper, the observation values were the FMUE related-item scores and the values in $\mathrm{z}$ were the desired EIs mapped from the observation values:

## 3. Results

## 4. Discussion

## 5. Conclusions

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Jonsson, A.C.; Lindgren, I.; Hallström, B.; Norrving, B.; Lindgren, N. Determinants of quality of life in stroke survivors and their informal caregivers. Stroke
**2005**, 36, 803–808. [Google Scholar] [CrossRef] [PubMed] - Gladstone, D.J.; Danells, C.J.; Black, S.E. The Fugl-Meyer assessment of motor recovery after stroke: A critical review of its measurement properties. Neurorehabil. Neural Repair
**2002**, 16, 232–240. [Google Scholar] [CrossRef] [PubMed] - Signal, L.; Balan, A.O.; Black, M.J. HumanEva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. Int. J. Comput. Vis.
**2010**, 87, 4–27. [Google Scholar] [CrossRef] - Lee, E.H.; Goh, J.C.; Bose, K. Value of gait analysis in the assessment of surgery in cerebral palsy. Arch. Phys. Med. Rehabil.
**1992**, 73, 642–646. [Google Scholar] [PubMed] - Flanagan, A.; Krzak, J.; Johnson, P.; Urban, M. Evaluation of short-term intensive orthotic garment use in children who have cerebral palsy. Pediatr. Phys. Ther.
**2009**, 21, 201–204. [Google Scholar] [CrossRef] [PubMed] - Kanai, A.; Kiyama, S.; Goto, H.; Tomita, H.; Tanaka, A.; Kunimi, M.; Okada, T.; Nakai, T. Use of the sit-to-stand task to evaluate motor function of older adults using telemetry. BMC Geriatr.
**2016**, 16, 121. [Google Scholar] [CrossRef] [PubMed] - Nixon, M.E.; Howard, A.M.; Chen, Y. Quantitative evaluation of the Microsoft Kinect for use in an upper extremity virtual rehabilitation environment. In Proceedings of the International Conference on Virtual Rehabilitation, Philadelphia, PA, USA, 26–29 August 2013.
- Stančin, S.; Tomažič, S. Time- and computation-efficient calibration of MEMS 3D accelerometers and gyroscopes. Sensors
**2014**, 14, 14885–14925. [Google Scholar] [CrossRef] [PubMed] - Wong, W.; Wong, M.; Lo, K.H. Clinical applications of sensors for human posture and movement analysis: A review. Prosthet. Orthot. Int.
**2007**, 31, 62–75. [Google Scholar] [CrossRef] [PubMed] - Margarito, J.; Helaoui, R.; Bianchi, A.M.; Sartor, F.; Bonomi, A.G. User-independent recognition of sports activities from a single wrist-worn accelerometer: A template-matching-based approach. IEEE Trans. Biomed. Eng.
**2016**, 63, 788–796. [Google Scholar] [CrossRef] [PubMed] - Strømmen, A.M.; Christensen, T.; Jensen, K. Quantitative measurement of physical activity in acute ischemic stroke and transient ischemic attack. Stroke
**2014**, 45, 3649–3655. [Google Scholar] [CrossRef] [PubMed] - Noorkõiv, M.; Rodgers, H.; Price, C.I. Accelerometer measurement of upper extremity movement after stroke: A systematic review of clinical studies. J. Neuro Eng. Rehabilit.
**2014**, 11, 144. [Google Scholar] [CrossRef] [PubMed] - Rand, D.; Eng, J.J. Disparity between functional recovery and daily use of the upper and lower extremities during subacute stroke rehabilitation. Neurorehabil. Neural Repair
**2012**, 26, 76–84. [Google Scholar] [CrossRef] [PubMed] - Patel, S.; Hughes, R.; Hester, H.; Stein, J.; Metin, A.; Dy, J.G.; Bonato, P. A novel approach to monitor rehabilitation outcomes in stroke survivors using wearable technology. Proc. IEEE
**2010**, 98, 450–461. [Google Scholar] [CrossRef] - Gubbi, J.; Rao, A.S. Motor recovery monitoring using acceleration measurements in post-acute stroke patients. Biomed. Eng. Online
**2013**, 12, 33. [Google Scholar] [CrossRef] [PubMed] - Tao, W.; Liu, T.; Zheng, R.; Feng, H. Gait analysis using wearable sensors. Sensors
**2012**, 12, 2255–2283. [Google Scholar] [CrossRef] [PubMed] - Wang, Q.; Chen, X.; Chen, R.; Chen, Y.; Zhang, X. Electromyography-based locomotion pattern recognition and personal positioning towards improves context-awareness application. IEEE Trans. Syst. Man Cybern.
**2013**, 43, 1216–1227. [Google Scholar] [CrossRef] - Tang, L.; Li, F.; Cao, S.; Zhang, X.; Wu, D.; Chen, X. Muscle synergy analysis in children with cerebral palsy. J. Neural Eng.
**2015**, 12, 046017. [Google Scholar] [CrossRef] [PubMed] - Liu, L.; Chen, X.; Lu, Z.; Cao, S.; Wu, D.; Zhang, X. Development of an EMG-ACC-based upper limb rehabilitation training system. IEEE Trans. Neural Syst. Rehabil. Eng.
**2016**, PP, 1. [Google Scholar] [CrossRef] [PubMed] - Zhang, X.; Chen, X.; Li, Y.; Lantz, V.; Wang, K.; Yang, J. A framework for hand gesture recognition based on accelerometer and EMG sensors. IEEE Trans. Syst. Man Cybern.
**2011**, 41, 1064–1076. [Google Scholar] [CrossRef] - Li, Y.; Chen, X.; Zhang, X.; Wang, K.; Wang, Z. A sign-component-based framework for Chinese sign language recognition using accelerometer and sEMG data. IEEE Trans. Biomed. Eng.
**2012**, 59, 2695–2704. [Google Scholar] [PubMed] - Su, R.; Chen, X.; Cao, S.; Zhang, X. Random forest-based recognition of isolated sign language subwords using data from accelerometers and surface electromyographic sensors. Sensors
**2016**, 16, 100. [Google Scholar] [CrossRef] [PubMed] - Lu, Z.; Chen, X.; Li, Q.; Zhang, X.; Zhou, P. A hand gesture recognition framework and wearable gesture-based interaction prototype for mobile devices. IEEE Trans. Hum. Mach. Syst.
**2014**, 44, 293–299. [Google Scholar] [CrossRef] - Kosmidou, V.E.; Hadjileontiadis, L.J. Sign language recognition using intrinsic-mode sample entropy on sEMG and accelerometer data. IEEE Trans. Biomed. Eng.
**2009**, 56, 2879–2890. [Google Scholar] [CrossRef] [PubMed] - Chang, W.; Dai, L.; Shen, S.; Tan, J.; Zhu, Z.; Duan, F. A hierarchical hand motions recognition method based on IMU and sEMG sensors. In Proceedings of the 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO), Zhuhai, China, 6–9 December 2015.
- Roy, S.H.; Cheng, M.S.; Chang, S.; Moore, J.; DeLuca, G.; Nawab, S.H.; De Luca, C. A combined sEMG and accelerometer system for monitoring functional activity in stroke. IEEE Trans. Neural Syst. Rehabil. Eng.
**2009**, 17, 585–594. [Google Scholar] [CrossRef] [PubMed] - Li, H.; Han, S.; Pan, M. Lower-limb motion classification for hemiparetic patients through IMU and EMG signal processing. In Proceedings of the 2016 International Conference on Biomedical Engineering (BME-HUST), Hanoi, Vietnam, 5–6 October 2016.
- Otten, P.; Kim, J.; Son, S. A framework to automate assessment of upper-limb motor function impairment: A feasibility study. Sensors
**2015**, 15, 20097–20114. [Google Scholar] [CrossRef] [PubMed] - Naghdi, S.; Ansari, N.N.; Mansouri, K.; Hasson, S. A neurophysiological and clinical study of Brunnstrom recovery stages in the upper limb following stroke. Brain Inj.
**2010**, 24, 1372–1378. [Google Scholar] [CrossRef] [PubMed] - Dyer, J.O.; Maupas, E.; MeloSde, A.; Bourbonnais, D.; Forget, R. Abnormal coactivation of knee and ankle extensors is related to changes in heteronymous spinal pathways after stroke. J. Neuroeng. Rehabil.
**2011**, 8, 41. [Google Scholar] [CrossRef] [PubMed] - Ghodsi, A. Dimensionality Reduction: A Short Tutorial, 1st ed.; Department of Statistics and Actuarial Science, University of Waterloo: Waterloo, ON, Canada, 2006; pp. 14–15. [Google Scholar]
- Sammon, J.W. A nonlinear mapping for data structure analysis. IEEE Trans. Comp.
**1969**, 18, 401–409. [Google Scholar] [CrossRef] - Liu, J.; Chen, J.; Ye, J. Large-scale sparse logistic regression. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Pairs, France, 28 June–1 July 2009.
- Tibshirani, R. Regression shrinkage and selection via the LASSO: A retrospective. J. R. Stat. Soc. Ser. B (Stat. Methodol.)
**1996**, 58, 267–288. [Google Scholar] [CrossRef] - Kim, S.J.; Koh, K.; Lusting, M.; Boyd, S.; Gorinevsky, D. An interior-point method for large-scale l
_{1}-regularized least squares. IEEE J. Sel. Top. Signal Process.**2007**, 1, 606–617. [Google Scholar] [CrossRef] - Efron, B.; Hastie, T.; Johnstone, I.; Tibshirani, R. Least angle regression. Ann. Stat.
**2004**, 32, 407–451. [Google Scholar] - Zou, H.; Hastie, T. Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B (Stat. Methodol.)
**2005**, 67, 301–320. [Google Scholar] [CrossRef]

**Figure 1.**Schematic of the placement and orientation of the sensors in the experiment. The right upper limb is taken as an example to illustrate here. The red ones stand for EMG sensors, and the blue ones stand for IMUs. IMU’s z-axis is perpendicular outside to the plan.

**Figure 2.**Sketch map of the canonical tasks (taking the right upper limb as an example): (

**1**) wrist flexion; (

**2**) wrist extension; (

**3**) shoulder flexion to 90°, elbow at 0°; (

**4**) shoulder abduction to 90°, elbow at 0° and forearm pronated; (

**5**) flip a piece of paper; (

**6**) fetch and hold a ball put on the table initially and keep shoulder flexion to 90°, elbow at 0° and palm down; (

**7**) fetch and hold a cylindrical roll put on the table initially and keep shoulder flexion to 90°, elbow at 0°and palm towards the body; (

**8**) finger to nose; (

**9**) touch the back of the shoulder; (

**10**) keep shoulder at 0°, elbow at 90° and palm down, then do supination/pronation of the forearm for twice; (

**11**) flex elbow three times as fast as possible.

**Figure 3.**Flow chart of data analysis. PCA: principal component analysis; MDS: multidimensional scaling; NMF: non-negative matrix factorization; LASSO: least absolute shrinkage and selection operator.

**Figure 4.**Illustration of the data segmentation process when representative data channels are used as examples from a healthy subject and a stroke subject performing TASK 8, respectively. For each subject, there are three repetitions of task performance. The black solid line represents the threshold T

_{R}.

**Figure 5.**Plot of FMUE score versus the global EI, from PCC (

**a**–

**c**) and DTW (

**d**–

**f**) with EMG data alone (

**a**,

**d**), inertial data alone (

**b**,

**e**) and combined data (

**c**,

**f**), respectively. R

^{2}reports the coefficient of determination from the linear regression analysis. The ‘×’ and the bar denote the mean and 1.96-times SD of the healthy subjects’ EI. The range between the two dashed lines over the horizontal axis represents the normal range. Circles and triangles represent data from individual healthy and stroke subjects, respectively.

**Figure 6.**Plot of score of FMUE-related items versus the EI for TASK 3 (

**a**) and TASK 7 (

**b**). All symbols and lines appear in the same way as illustrated in Figure 5.

**Figure 7.**Plot of score of FMUE-related items versus the EI for TASK 7 (

**a**–

**d**), and plot of FMUE score versus the EI (

**e**–

**h**), using PCA (

**a**,

**e**), MDS (

**b**,

**f**), NMF (

**c**,

**g**), and LASSO (

**d**,

**h**), respectively. All symbols and lines appear in the same way as illustrated in Figure 5.

No. | Sex | Height (cm) | Weight (kg) | Paretic Side | Age (Years) | Onset (Days) | FMUE Score | Brunnstrom Stage |
---|---|---|---|---|---|---|---|---|

1 | Male | 175 | 71 | Left | 72 | 40 | 50 | 4 |

2 | Female | 159 | 48 | Right | 52 | 33 | 58 | 5 |

3 | Male | 181 | 81 | Right | 50 | 11 | 59 | 5 |

4 | Male | 162 | 65 | Right | 58 | 21 | 40 | 4 |

5 | Female | 173 | 66 | Right | 53 | 366 | 37 | 4 |

6 | Male | 176 | 75 | Right | 30 | 457 | 25 | 4 |

7 | Male | 168 | 68 | Right | 61 | 68 | 48 | 4 |

8 | Female | 162 | 49 | Left | 75 | 48 | 40 | 5 |

9 | Male | 176 | 71 | Left | 46 | 49 | 41 | 3 |

10 | Male | 170 | 68 | Left | 69 | 10 | 10 | 3 |

11 | Female | 165 | 55 | Left | 50 | 36 | 21 | 3 |

12 | Female | 158 | 49 | Left | 50 | 51 | 48 | 5 |

13 | Male | 175 | 72 | Right | 51 | 78 | 24 | 3 |

14 | Female | 155 | 45 | Right | 81 | 81 | 51 | 5 |

15 | Male | 178 | 72 | Left | 44 | 225 | 38 | 3 |

16 | Female | 163 | 66 | Left | 51 | 584 | 35 | 4 |

17 | Male | 169 | 73 | Left | 59 | 40 | 57 | 5 |

18 | Male | 175 | 72 | Left | 43 | 71 | 45 | 4 |

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license ( http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Li, Y.; Zhang, X.; Gong, Y.; Cheng, Y.; Gao, X.; Chen, X.
Motor Function Evaluation of Hemiplegic Upper-Extremities Using Data Fusion from Wearable Inertial and Surface EMG Sensors. *Sensors* **2017**, *17*, 582.
https://doi.org/10.3390/s17030582

**AMA Style**

Li Y, Zhang X, Gong Y, Cheng Y, Gao X, Chen X.
Motor Function Evaluation of Hemiplegic Upper-Extremities Using Data Fusion from Wearable Inertial and Surface EMG Sensors. *Sensors*. 2017; 17(3):582.
https://doi.org/10.3390/s17030582

**Chicago/Turabian Style**

Li, Yanran, Xu Zhang, Yanan Gong, Ying Cheng, Xiaoping Gao, and Xiang Chen.
2017. "Motor Function Evaluation of Hemiplegic Upper-Extremities Using Data Fusion from Wearable Inertial and Surface EMG Sensors" *Sensors* 17, no. 3: 582.
https://doi.org/10.3390/s17030582