# Human Activity Recognition of Individuals with Lower Limb Amputation in Free-Living Conditions: A Pilot Study

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Data Collection and Pre-Processing

#### 2.2. Annotation of Slopes

#### 2.3. Introducing “Label Resolution”

- Level 1: No Terrain Resolution. Labels only contain the condition of the traversed ground (ground is flat, ground is sloped, or is traversing stairs)
- Level 2: Hard/Soft Terrain Resolution. Terrains are resolved into whether the terrain involved is hard/soft. Hard terrains included concrete, stone, and gravel, while soft terrains included grass, sand and red ash (a type of clay pitch).
- Level 3: Full Resolution: All Labels use their full label resolution.

#### 2.4. Experiment 1: Comparison of Supervised Classifiers

#### 2.5. Experiment 2: Label Terrain Resolution

#### 2.6. Experiment 3: Subject Cross-Validation

- Method 1: Healthy Participant Training. For the best performing classifiers, train data on the healthy participants, then use LOSO validation to test the classification accuracy for each of the ILLA participants individually.
- Method 2: Amputee Participant Training. All data from healthy participants are ignored. LOSO validation is again performed with the best classifiers, this time training the data on all-minus-one ILLA participants and testing on the remaining ILLA participant, repeating the process for each ILLA.

## 3. Results

#### 3.1. Participant Information

#### 3.2. Experiment 1: Classifier Optimization

#### 3.3. Experiment 2: Label Terrain Resolution

#### 3.4. Experiment 3: Subject Cross-Validation

## 4. Discussion

#### 4.1. Main Findings and Interpretations

#### 4.2. Comparisons with Other Studies

## 5. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Appendix A. Feature List

#### Appendix A.1. Condensed Feature List

Feature | Explanation |
---|---|

Statistical and Time Domain Features | |

Mean | $\overline{X}\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\frac{{\sum}_{i=1}^{n}{X}_{i}}{n}$ |

Median | $Med\left(X\right)\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\frac{\left(X\left[\frac{n-1}{2}\right]+X\left[\frac{n+1}{2}\right]\right)}{2}$ |

Variance | ${\sigma}^{2}=\frac{{\sum}_{i\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}1}^{n}{({X}_{i}\phantom{\rule{4pt}{0ex}}-\phantom{\rule{4pt}{0ex}}\overline{X})}^{2}}{n-1}$ |

Root Mean Square | $RMS\left(X\right)\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\sqrt{\frac{1}{n}{\left(\overline{X}\right)}^{2}}$ |

Crest Factor | $c\left(X\right)\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\frac{|{X}_{peak}|}{RMS\left(X\right)}$ |

L1 Norm | $L1\left(X\right)\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}{\sum}_{i\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}1}^{n}X$ |

L2 Norm | $L2\left(X\right)\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\sqrt{{\sum}_{i\phantom{\rule{4pt}{0ex}}=1}^{n}{\left|X\right|}^{2}}$ |

Variance of the sample-wise Norm ${}^{a}$ | ${\sigma}_{norm}^{2}\phantom{\rule{4pt}{0ex}}={\sigma}^{2}\left(L2({X}_{i},{Y}_{i},{Z}_{i})\right)\phantom{\rule{4pt}{0ex}};\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}i\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}1,2\phantom{\rule{4pt}{0ex}}\cdots \phantom{\rule{4pt}{0ex}}n$ |

Skewness | $SV\left(X\right)\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\frac{{\sum}_{i=1}^{n}{({X}_{i}\phantom{\rule{4pt}{0ex}}-\phantom{\rule{4pt}{0ex}}\overline{X})}^{3}}{(N-1)\ast {\sigma}^{3}}$ |

Kurtosis | $KV\left(X\right)\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\frac{{\sum}_{i=1}^{n}{({X}_{i}\phantom{\rule{4pt}{0ex}}-\phantom{\rule{4pt}{0ex}}\overline{X})}^{4}}{(N-1)\ast {\sigma}^{4}}$ |

25th Quartile | ${Q}_{25}\left(X\right)\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}X\left[\frac{1}{4(n+1)}\right]$ |

75th Quartile | ${Q}_{75}\left(X\right)\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}X\left[\frac{3}{4(n+1)}\right]$ |

Interquartile Range | $IQR\left(X\right)\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}{Q}_{75}\left(X\right)\phantom{\rule{4pt}{0ex}}-\phantom{\rule{4pt}{0ex}}{Q}_{25}\left(X\right)$ |

Maximum | $Max\left(X\right)$ |

Minimum | $Min\left(X\right)$ |

Range | $Max\left(X\right)\phantom{\rule{4pt}{0ex}}-\phantom{\rule{4pt}{0ex}}Min\left(X\right)$ |

Mean Absolute Deviation | $MAD\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\frac{1}{n}{\sum}_{i\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}1}^{n}\left|{X}_{i}\phantom{\rule{4pt}{0ex}}-\phantom{\rule{4pt}{0ex}}\overline{X}\right|$ |

Signal Magnitude Area | $SMA\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\frac{1}{n}({\sum}_{i\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}1}^{n}\left|{X}_{i}\right|\phantom{\rule{4pt}{0ex}}+\phantom{\rule{4pt}{0ex}}{\sum}_{i\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}1}^{n}\left|{Y}_{i}\right|\phantom{\rule{4pt}{0ex}}+\phantom{\rule{4pt}{0ex}}{\sum}_{i\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}1}^{n}\left|{Z}_{i}\right|\phantom{\rule{4pt}{0ex}})\phantom{\rule{4pt}{0ex}}$ |

Energy | $E\left(X\right)\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}{\sum}_{i\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}1}^{n}{\left|{X}_{i}\right|}^{2}$ |

Power | $P\left(X\right)\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\frac{\sqrt{{\sum}_{i=1}^{n}{\left|{X}_{i}\right|}^{2}}}{n}$ |

Entropy | $H\left(X\right)\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}{\sum}_{i\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}1}^{n}\left({P}_{i}\right){log}_{2}\left({P}_{i}\right)\phantom{\rule{4pt}{0ex}}where\phantom{\rule{4pt}{0ex}}{P}_{i}\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\frac{\frac{{X}_{i}}{Max\left({X}_{i}\right)}}{{\sum}_{i=1}^{n}\left(\frac{{X}_{i}}{Max\left({X}_{i}\right)}\right)}$ |

Integral Features ${}^{b}$ | [23] |

Inter-axis Correlation Coefficients | $r\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\frac{{\sum}_{i=1}^{n}({X}_{i}\phantom{\rule{4pt}{0ex}}-\phantom{\rule{4pt}{0ex}}\overline{X})({Y}_{i}-\overline{Y})}{\sqrt{{\sum}_{i=1}^{n}{({X}_{i}\phantom{\rule{4pt}{0ex}}-\phantom{\rule{4pt}{0ex}}\overline{X})}^{2}{({Y}_{i}-\overline{Y})}^{2}}}$ |

Eigenvalues of Dominant Direction | [71] |

Frequency Features | |

Spectral Energy | E(FFT(X)) |

Spectral Centroid | $SC\phantom{\rule{4pt}{0ex}}=\phantom{\rule{4pt}{0ex}}\frac{{\sum}_{k-1}^{n}\left|X(i,k)\right|\ast f\left(k\right)}{{\sum}_{k-1}^{n}\left|\right|X(i,k)|}$ |

Spectral Entropy | H(FFT(X)) |

Cepstral Coefficients | [24] |

Wavelet Features ^{c} | |

Tamura Coefficients | [72] |

Nyan Coefficients | [73] |

Sekine Coefficients | [74] |

Wang Coefficients | [75] |

Fractal Dimension | [76] |

Preece Coefficients | [77] |

^{a}—to elaborate, the variance of the sample-wise norm is calculated by acquiring the L2 norm of the X, Y and Z axis for each sample of the input signal from 1 to n. ${\sigma}_{\mathit{norm}}^{2}$ is the variance of the calculated norms across the entire segment.

^{b}—The integral features used from Rosati et al. [23] are: “Mean of integral of Superior-Inferior (SI) acceleration”, “Mean of double integral of SI acceleration”, “Mean of integral of Anterio-Posterior (AP) acceleration”, “Root Mean Square of integral of AP acceleration”, “Root Mean Square of double integral of AP acceleration”. Using the axes of the ActivPAL during gait, SI and AP directions are approximated as the X and Z axis respectively.

^{c}—All Wavelet Features are utilised from Table 1 of Preece et al. [77] and is the recommended starting point for understanding the wavelet features used.

## Appendix B. Confusion Matrices

#### Appendix B.1. Experiment 2: SVM and LSTM Confusion Matrices

#### Appendix B.2. Experiment 3: Confusion Matrices

**Figure A7.**Support Vector Machine Confusion Chart for ILLA participant A1 as trained by healthy individuals.

**Figure A8.**Support Vector Machine Confusion Chart for ILLA participant A2 as trained by healthy individuals.

**Figure A9.**Support Vector Machine Confusion Chart for ILLA participant A3 as trained by healthy individuals.

**Figure A10.**Support Vector Machine Confusion Chart for ILLA participant A4 as trained by healthy individuals.

**Figure A11.**Support Vector Machine Confusion Chart for ILLA participant A1 as trained by all other ILLA participants.

**Figure A12.**Support Vector Machine Confusion Chart for ILLA participant A2 as trained by all other ILLA participants.

**Figure A13.**Support Vector Machine Confusion Chart for ILLA participant A3 as trained by all other ILLA participants.

**Figure A14.**Support Vector Machine Confusion Chart for ILLA participant A4 as trained by all other ILLA participants.

**Figure A15.**Long Short-Term Memory Network Confusion Chart for ILLA participant A1 as trained by healthy individuals.

**Figure A16.**Long Short-Term Memory Network Confusion Chart for ILLA participant A2 as trained by healthy individuals.

**Figure A17.**Long Short-Term Memory Network Confusion Chart for ILLA participant A3 as trained by healthy individuals.

**Figure A18.**Long Short-Term Memory Network Confusion Chart for ILLA participant A4 as trained by healthy individuals.

**Figure A19.**Long Short-Term Memory Network Confusion Chart for ILLA participant A1 as trained by all other ILLA participants.

**Figure A20.**Long Short-Term Memory Network Confusion Chart for ILLA participant A2 as trained by all other ILLA participants.

**Figure A21.**Long Short-Term Memory Network Confusion Chart for ILLA participant A3 as trained by all other ILLA participants.

**Figure A22.**Long Short-Term Memory Network Confusion Chart for ILLA participant A4 as trained by all other ILLA participants.

## References

- Bussmann, J.B.; Grootscholten, E.A.; Stam, H.J. Daily physical activity and heart rate response in people with a unilateral transtibial amputation for vascular disease. Arch. Phys. Med. Rehabil.
**2004**, 85, 240–244. [Google Scholar] [CrossRef] - Langford, J.; Dillon, M.P.; Granger, C.L.; Barr, C. Physical activity participation amongst individuals with lower limb amputation. Disabil. Rehabil.
**2019**, 41, 1063–1070. [Google Scholar] [CrossRef] [PubMed] - Bragaru, M.; Van Wilgen, C.P.; Geertzen, J.H.B.; Ruijs, S.G.J.B.; Dijkstra, P.U.; Dekker, R. Barriers and Facilitators of Participation in Sports: A Qualitative Study on Dutch Individuals with Lower Limb Amputation. PLoS ONE
**2013**, 8, e59881. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Deans, S.; Burns, D.; McGarry, A.; Murray, K.; Mutrie, N. Motivations and barriers to prosthesis users participation in physical activity, exercise and sport: A review of the literature. Prosthet. Orthot. Int.
**2012**, 36, 260–269. [Google Scholar] [CrossRef] [Green Version] - Littman, A.J.; Boyko, E.J.; Thompson, M.L.; Haselkorn, J.K.; Sangeorzan, B.J.; Arterburn, D.E. Physical activity barriers and enablers in older Veterans with lower-limb amputation. J. Rehabil. Res. Dev.
**2014**, 51, 895–906. [Google Scholar] [CrossRef] - Bragaru, M.; Dekker, R.; Geertzen, J.H.B.; Dijkstra, P.U. Amputees and Sports. Sport. Med.
**2011**, 41, 721–740. [Google Scholar] [CrossRef] - Wetterhahn, K.; Hanson, C.; Levy, C. Effect of Participation in Physical Activity on Body Image of Amputees. Am. J. Phys. Med. Rehabil. / Assoc. Acad. Physiatr.
**2002**, 81, 194–201. [Google Scholar] [CrossRef] - Chadwell, A.; Diment, L.; Micó-Amigo, M.; Morgado Ramírez, D.Z.; Dickinson, A.; Granat, M.; Kenney, L.; Kheng, S.; Sobuh, M.; Ssekitoleko, R.; et al. Technology for monitoring everyday prosthesis use: A systematic review. J. NeuroEng. Rehabil.
**2020**, 17. [Google Scholar] [CrossRef] - Cruciani, F.; Cleland, I.; Nugent, C.; Mccullagh, P.; Synnes, K.; Hallberg, J. Automatic Annotation for Human Activity Recognition in Free Living Using a Smartphone. Sensors
**2018**, 18, 2203. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Ellis, K.; Kerr, J.; Godbole, S.; Lanckriet, G. Multi-sensor physical activity recognition in free-living. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, Seattle, WA, USA, 14–17 September 2014. [Google Scholar] [CrossRef] [Green Version]
- Fullerton, E.; Heller, B.; Munoz-Organero, M. Recognizing Human Activity in Free-Living Using Multiple Body-Worn Accelerometers. IEEE Sens. J.
**2017**, 17, 5290–5297. [Google Scholar] [CrossRef] [Green Version] - Cruciani, F.; Sun, C.; Zhang, S.; Nugent, C.; Li, C.; Song, S.; Cheng, C.; Cleland, I.; Mccullagh, P. A Public Domain Dataset for Human Activity Recognition in Free-Living Conditions. In Proceedings of the 2019 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), Leicester, UK, 19–23 August 2019. [Google Scholar] [CrossRef]
- Labarrière, F.; Thomas, E.; Calistri, L.; Optasanu, V.; Gueugnon, M.; Ornetti, P.; Laroche, D. Machine Learning Approaches for Activity Recognition and/or Activity Prediction in Locomotion Assistive Devices—A Systematic Review. Sensors
**2020**, 20, 6345. [Google Scholar] [CrossRef] - Zhang, K.; Xiong, C.; Zhang, W.; Liu, H.; Lai, D.; Rong, Y.; Fu, C. Environmental Features Recognition for Lower Limb Prostheses Toward Predictive Walking. IEEE Trans. Neural Syst. Rehabil. Eng.
**2019**, 27, 465–476. [Google Scholar] [CrossRef] - Zhang, K.; Zhang, W.; Xiao, W.; Liu, H.; De Silva, C.W.; Fu, C. Sequential Decision Fusion for Environmental Classification in Assistive Walking. IEEE Trans. Neural Syst. Rehabil. Eng.
**2019**, 27, 1780–1790. [Google Scholar] [CrossRef] - Jung, I. A review of privacy-preserving human and human activity recognition. Int. J. Smart Sens. Intell. Syst.
**2020**, 13, 1–13. [Google Scholar] [CrossRef] - Grant, P.M.; Ryan, C.G.; Tigbe, W.W.; Granat, M.H. The validation of a novel activity monitor in the measurement of posture and motion during everyday activities. Br. J. Sport. Med.
**2006**, 40, 992–997. [Google Scholar] [CrossRef] [Green Version] - Lyden, K.; Keadle, S.K.; Staudenmayer, J.; Freedson, P.S. The activPALTM Accurately Classifies Activity Intensity Categories in Healthy Adults. Med. Sci. Sport. Exerc.
**2017**, 49, 1022–1028. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Buis, A.; Dumbleton, T.; Murray, K.; McHugh, B.; McKay, G.; Sexton, S. Measuring the Daily Stepping Activity of People with Transtibial Amputation Using the ActivPAL™ Activity Monitor. JPO J. Prosth. Orthot.
**2014**, 26, 43–47. [Google Scholar] [CrossRef] [Green Version] - Van Hees, V.T.; Gorzelniak, L.; Dean León, E.C.; Eder, M.; Pias, M.; Taherian, S.; Ekelund, U.; Renström, F.; Franks, P.W.; Horsch, A.; et al. Separating Movement and Gravity Components in an Acceleration Signal and Implications for the Assessment of Human Daily Physical Activity. PLoS ONE
**2013**, 8, e61691. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Frølich, L.; Dowding, I. Removal of muscular artifacts in EEG signals: A comparison of linear decomposition methods. Brain Inform.
**2018**, 5, 13–22. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Dehghani, A.; Sarbishei, O.; Glatard, T.; Shihab, E. A Quantitative Comparison of Overlapping and Non-Overlapping Sliding Windows for Human Activity Recognition Using Inertial Sensors. Sensors
**2019**, 19, 5026. [Google Scholar] [CrossRef] [Green Version] - Rosati, S.; Balestra, G.; Knaflitz, M. Comparison of Different Sets of Features for Human Activity Recognition by Wearable Sensors. Sensors
**2018**, 18, 4189. [Google Scholar] [CrossRef] [PubMed] [Green Version] - San-Segundo, R.; Montero, J.M.; Barra-Chicote, R.; Fernandez, F.; Pardo, J.M. Feature extraction from smartphone inertial signals for human activity segmentation. Signal Process.
**2016**, 120, 359–372. [Google Scholar] [CrossRef] - Abdul Yamin, N.A.A.; Amran, M.; Basaruddin, K.; Salleh, A.; Rusli, W. Ground reaction force response during running on different surface hardness. ARPN J. Eng. Appl. Sci.
**2017**, 12, 2313–2318. [Google Scholar] - Nedorubova, A.; Kadyrova, A.; Khlyupin, A. Human Activity Recognition using Continuous Wavelet Transform and Convolutional Neural Networks. arXiv
**2021**, arXiv:2106.12666. [Google Scholar] - Meg. Elevation on Strava FAQs. 2021. Available online: https://support.strava.com/hc/en-us/articles/115001294564-Elevation-on-Strava-FAQs (accessed on 1 September 2021).
- Bosch. Pressure sensor BMP280. Available online: https://www.bosch-sensortec.com/products/environmental-sensors/pressure-sensors/bmp280/ (accessed on 1 September 2021).
- Barberi, J. GPS vs. Smartphone vs. Cyclocomputer (Round 2). Available online: https://www.singletracks.com/mtb-gear/gps-accuracy-gps-vs-smartphone-vs-cyclocomputer/ (accessed on 1 September 2021).
- Meg. Why is GPS Data Sometimes Inaccurate? Available online: https://support.strava.com/hc/en-us/articles/216917917-Why-is-GPS-data-sometimes-inaccurate- (accessed on 1 September 2021).
- 2010 ADA standards for accessible design. Available online: https://www.ada.gov/2010ADAstandards_index.htm (accessed on 1 September 2021).
- Guo, X.; Yin, Y.; Dong, C.; Yang, G.; Zhou, G. On the Class Imbalance Problem. In Proceedings of the 2008 Fourth International Conference on Natural Computation, NW Washington, DC, USA, 18–20 October 2008. [Google Scholar] [CrossRef]
- Chawla, N.; Bowyer, K.; Hall, L.; Kegelmeyer, W. SMOTE: Synthetic Minority Over-sampling Technique. J. Artif. Intell. Res. (JAIR)
**2002**, 16, 321–357. [Google Scholar] [CrossRef] - Anguita, D.; Ghio, A.; Oneto, L.; Parra, X.; Reyes-Ortiz, J.L. Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine. In Lecture Notes in Computer Science; Springer: Berlin, Germany, 2012; pp. 216–223. [Google Scholar] [CrossRef] [Green Version]
- Kaghyan, S.; Sarukhanyan, H. Activity Recognition using K-nearest neighbor algorithm on smartphone with tri-axial accelerometer. Int. J. Inf. Model. Anal.
**2012**, 1, 146–156. [Google Scholar] - Xu, L.; Yang, W.; Cao, Y.; Li, Q. Human activity recognition based on random forests. In Proceedings of the 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), Guilin, China, 29–31 July 2017. [Google Scholar] [CrossRef]
- Subasi, A.; Dammas, D.H.; Alghamdi, R.D.; Makawi, R.A.; Albiety, E.A.; Brahimi, T.; Sarirete, A. Sensor Based Human Activity Recognition Using Adaboost Ensemble Classifier. Procedia Comput. Sci.
**2018**, 140, 104–111. [Google Scholar] [CrossRef] - Gupta, P.; Dallas, T. Feature Selection and Activity Recognition System Using a Single Triaxial Accelerometer. IEEE Trans. Biomed. Eng.
**2014**, 61, 1780–1786. [Google Scholar] [CrossRef] - Inoue, M.; Inoue, S.; Nishida, T. Deep recurrent neural network for mobile human activity recognition with high throughput. Artif. Life Robot.
**2018**, 23, 173–185. [Google Scholar] [CrossRef] [Green Version] - Bengio, Y. Practical Recommendations for Gradient-Based Training of Deep Architectures; Springer: Berlin, Heidelberg, 2012. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv
**2015**, arXiv:1412.6980. [Google Scholar] - Bergstra, J.; Bengio, Y. Random Search for Hyper-Parameter Optimization. J. Mach. Learn. Res.
**2012**, 13, 281–305. [Google Scholar] - Marcot, B.G.; Hanea, A.M. What is an optimal value of k in k-fold cross-validation in discrete Bayesian network analysis? Comput. Stat.
**2021**, 36, 2009–2031. [Google Scholar] [CrossRef] - Fearn, T. Double Cross-Validation. NIR News
**2010**, 21, 14–15. [Google Scholar] [CrossRef] - Davie-Smith, F.; Hebenton, J.; Scott, H. A Survey of the Lower Limb Amputee Population in Scotland 2015 Full Report. Scottish Physihother. Amputee Res. Group. Available online: https://bacpar.csp.org.uk/publications/sparg-report-2015 (accessed on 1 September 2021).
- Park, S.E.; Ho, Y.J.; Chun, M.H.; Choi, J.; Moon, Y. Measurement and Analysis of Gait Pattern during Stair Walk for Improvement of Robotic Locomotion Rehabilitation System. Appl. Bionics Biomech.
**2019**, 2019, 1495289. [Google Scholar] [CrossRef] [Green Version] - Kimel-Naor, S.; Gottlieb, A.; Plotnik, M. The effect of uphill and downhill walking on gait parameters: A self-paced treadmill study. J. Biomech.
**2017**, 60, 142–149. [Google Scholar] [CrossRef] - Allamy, H.; Khan, R. Methods to Avoid Over-Fitting and Under-Fitting In Supervised Machine Learning (Comparative Study). In Computer Science, Communication & Instrumentation Devices; World Press: Singapore, 2014; pp. 163–172. [Google Scholar] [CrossRef]
- Zhao, Y.; Yang, R.; Chevalier, G.; Gong, M. Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors. Math. Probl. Eng.
**2017**, 2017, 7316954. [Google Scholar] [CrossRef] - Roggen, D.; Calatroni, A.; Rossi, M.; Holleczek, T.; Förster, K.; Tröster, G.; Lukowicz, P.; Bannach, D.; Pirkl, G.; Ferscha, A.; et al. Collecting complex activity datasets in highly rich networked sensor environments. In Proceedings of the 2010 Seventh International Conference on Networked Sensing Systems (INSS), Kassel, Germany, 15–18 June 2010; pp. 233–240. [Google Scholar] [CrossRef] [Green Version]
- Sun, J.; Fu, Y.; Li, S.; He, J.; Xu, C.; Tan, L. Sequential Human Activity Recognition Based on Deep Convolutional Network and Extreme Learning Machine Using Wearable Sensors. J. Sens.
**2018**, 2018, 8580959. [Google Scholar] [CrossRef] - Huang, Z.; Niu, Q.; You, I.; Pau, G. Acceleration Feature Extraction of Human Body Based on Wearable Devices. Energies
**2021**, 14, 924. [Google Scholar] [CrossRef] - Keras. Available online: https://keras.io/ (accessed on 1 September 2021).
- Qazi, N.; Raza, K. Effect of Feature Selection, SMOTE and under Sampling on Class Imbalance Classification. In Proceedings of the 2012 UKSim 14th International Conference on Computer Modelling and Simulation, Cambridge, UK, 28–30 March 2012. [Google Scholar] [CrossRef]
- Flores, A.C.; Icoy, R.I.; Pena, C.F.; Gorro, K.D. An Evaluation of SVM and Naive Bayes with SMOTE on Sentiment Analysis Data Set. In Proceedings of the 2018 International Conference on Engineering, Applied Sciences, and Technology (ICEAST), Phuket, Thailand, , 4–7 July 2018. [Google Scholar] [CrossRef]
- Mai, J.; Xu, D.; Li, H.; Zhang, S.; Tan, J.; Wang, Q. Implementing a SoC-FPGA Based Acceleration System for On-Board SVM Training for Robotic Transtibial Prostheses. In Proceedings of the IEEE International Conference on Real-time Computing and Robotics (RCAR), Kandima, Maldives, 1–5 August 2018. [Google Scholar] [CrossRef]
- Mai, J.; Chen, W.; Zhang, S.; Xu, D.; Wang, Q. Performance analysis of hardware acceleration for locomotion mode recognition in robotic prosthetic control. In Proceedings of the 2018 IEEE International Conference on Cyborg and Bionic Systems (CBS), Shenzhen, China, 25–27 October 2018. [Google Scholar] [CrossRef]
- Shell, C.E.; Klute, G.K.; Neptune, R.R. Identifying classifier input signals to predict a cross-slope during transtibial amputee walking. PLoS ONE
**2018**, 13, e0192950. [Google Scholar] [CrossRef] [Green Version] - Stolyarov, R.; Burnett, G.; Herr, H. Translational Motion Tracking of Leg Joints for Enhanced Prediction of Walking Tasks. IEEE Trans. Biomed. Eng.
**2018**, 65, 763–769. [Google Scholar] [CrossRef] [PubMed] - Su, B.Y.; Wang, J.; Liu, S.Q.; Sheng, M.; Jiang, J.; Xiang, K. A CNN-Based Method for Intent Recognition Using Inertial Measurement Units and Intelligent Lower Limb Prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng.
**2019**, 27, 1032–1042. [Google Scholar] [CrossRef] - Gyllensten, I.C.; Bonomi, A.G. Identifying Types of Physical Activity With a Single Accelerometer: Evaluating Laboratory-trained Algorithms in Daily Life. IEEE Trans. Biomed. Eng.
**2011**, 58, 2656–2663. [Google Scholar] [CrossRef] - Liu, S.; Gao, R.X.; John, D.; Staudenmayer, J.; Freedson, P.S. SVM-based multi-sensor fusion for free-living physical activity assessment. In Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, USA, 30 August–3 September 2011. [Google Scholar] [CrossRef]
- Liu, L.; Wang, S.; Hu, B.; Qiong, Q.; Wen, J.; Rosenblum, D.S. Learning structures of interval-based Bayesian networks in probabilistic generative model for human complex activity recognition. Pattern Recognit.
**2018**, 81, 545–561. [Google Scholar] [CrossRef] - Hu, B.; Dixon, P.C.; Jacobs, J.V.; Dennerlein, J.T.; Schiffman, J.M. Machine learning algorithms based on signals from a single wearable inertial sensor can detect surface- and age-related differences in walking. J. Biomech.
**2018**, 71, 37–42. [Google Scholar] [CrossRef] [PubMed] - Hashmi, M.Z.U.H.; Riaz, Q.; Hussain, M.; Shahzad, M. What Lies Beneath One’s Feet? Terrain Classification Using Inertial Data of Human Walk. Appl. Sci.
**2019**, 9, 3099. [Google Scholar] [CrossRef] [Green Version] - Russell, B.; Mcdaid, A.; Toscano, W.; Hume, P. Moving the Lab into the Mountains: A Pilot Study of Human Activity Recognition in Unstructured Environments. Sensors
**2021**, 21, 654. [Google Scholar] [CrossRef] - O’Brien, C.M.; Duda, J.L.; Kitas, G.D.; Veldhuijzen Van Zanten, J.J.C.S.; Metsios, G.S.; Fenton, S.A.M. Measurement of sedentary time and physical activity in rheumatoid arthritis: An ActiGraph and activPAL™ validation study. Rheumatol. Int.
**2020**, 40, 1509–1518. [Google Scholar] [CrossRef] [PubMed] - Wezenberg, D.; Van Der Woude, L.H.; Faber, W.X.; De Haan, A.; Houdijk, H. Relation Between Aerobic Capacity and Walking Ability in Older Adults With a Lower-Limb Amputation. Arch. Phys. Med. Rehabil.
**2013**, 94, 1714–1720. [Google Scholar] [CrossRef] - Vageskar, E. Activity Recognition for Stroke Patients. Master’s Thesis, Norweigan University of Science and Technology, Trondheim, Norway, 2017. [Google Scholar]
- Jamieson, A. Dataset for: Construction of a Clinical Activity Monitoring Framework Based on Free-living Investigations of Individuals with Lower Limb Amputation; University of Strathclyde: Glasgow, UK, 2021. [Google Scholar] [CrossRef]
- Zhang, M.; Sawchuk, A.A. A Feature Selection-Based Framework for Human Activity Recognition Using Wearable Multimodal Sensors. In Proceedings of the 6th International Conference on Body Area Networks, Brussels, Belgium, 7–8 November 2011; pp. 92–98. [Google Scholar]
- Tamura, T.; Sekine, M.; Ogawa, M.; Togawa, T.; Fukui, Y. Classification of acceleration waveforms during walking by wavelet transform. Methods Inf. Med.
**1997**, 36, 356–359. [Google Scholar] - Nyan, M.N.; Tay, F.E.H.; Seah, K.H.W.; Sitoh, Y.Y. Classification of gait patterns in the time–frequency domain. J. Biomech.
**2006**, 39, 2647–2656. [Google Scholar] [CrossRef] [PubMed] - Sekine, M.; Tamura, T.; Togawa, T.; Fukui, Y. Classification of waist-acceleration signals in a continuous walking record. Med. Eng. Phys.
**2000**, 22, 285–291. [Google Scholar] [CrossRef] - Wang, N.; Ambikairajah, E.; Lovell, N.H.; Celler, B.G. Accelerometry Based Classification of Walking Patterns Using Time-frequency Analysis. In Proceedings of the 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Lyon, France, 23–26 August 2007. [Google Scholar] [CrossRef]
- Sekine, M.; Tamura, T.; Akay, M.; Fujimoto, T.; Togawa, T.; Fukui, Y. Discrimination of walking patterns using wavelet-based fractal analysis. IEEE Trans. Neural Syst. Rehabil. Eng.
**2002**, 10, 188–196. [Google Scholar] [CrossRef] [PubMed] - Preece, S.J.; Goulermas, J.Y.; Kenney, L.P.J.; Howard, D. A Comparison of Feature Extraction Methods for the Classification of Dynamic Activities From Accelerometer Data. IEEE Trans. Biomed. Eng.
**2009**, 56, 871–879. [Google Scholar] [CrossRef] [PubMed]

**Figure 2.**Example of a recording set-up. AP = ActivPAL. The smartphone and ActivPAL would both be worn under clothing.

**Figure 5.**Overview of Long-Short Term Memory (LSTM) Matlab architecture. The input represents a 40 sample segment of a 3 dimensional (triaxial) input of raw accelerometer data.

**Table 1.**List of all collected activities in investigation and their total corresponding sample counts.

Terrain Label | Total No. of Samples |
---|---|

Concrete, Flat | 12,200 |

Concrete, Camber, Parallel | 3923 |

Concrete, Downhill | 2549 |

Concrete, Uphill | 2427 |

Grass, Flat | 1346 |

Concrete, Camber, Perpendicular | 877 |

Upstairs | 691 |

Downstairs | 656 |

Concrete, Camber, Downhill, Parallel | 572 |

Concrete, Camber, Uphill, Parallel | 540 |

Stone, Flat | 496 |

Sand, Flat | 232 |

Stone, Uphill | 176 |

Grass, Downhill | 151 |

Concrete, Camber, Downhill, Perpendicular | 93 |

Concrete, Camber, Uphill, Perpendicular | 93 |

Stone, Downhill | 80 |

Grass, Uphill | 78 |

Gravel, Downhill | 46 |

Gravel, Flat | 32 |

Sand, Uphill | 31 |

Red Ash, Flat | 26 |

Sand, Downhill | 12 |

Gravel, Uphill | 3 |

Classifier | Parameters Tuned |
---|---|

SVM | Kernel Type, Box Size |

KNN | Number of neighbours, Distance Metric, Distance Weight |

Random Forest | Maximum number of learners, maximum number of splits |

AdaBoost | Maximum number of learners, maximum number of splits, learning rate |

Naïve-Bayes | Type of distribution, Type of kernel (for kernel distribution) |

Discriminant Analysis | Type of discriminant analysis (Linear, Quadratic) |

LSTM | Dropout factor, Number of Hidden Units |

Participant | Height (m) | Weight (kg) | Age (Years) | Gender |
---|---|---|---|---|

#1 | 1.80 | 84 | 24 | Male |

#2 | 1.65 | 63 | 51 | Female |

#3 | 1.62 | 65 | 18 | Female |

#4 | 1.97 | 99 | 25 | Male |

#5 | 1.92 | 102 | 25 | Male |

#6 | 1.83 | 89 | 24 | Male |

#7 | 1.84 | 88 | 25 | Male |

#8 | 1.78 | 98 | 25 | Male |

Participant | Height (m) | Weight (kg) | Age (years) | Gender | Type of Amputation | Type of Prosthesis | Origin | Years since Amputation |
---|---|---|---|---|---|---|---|---|

#1 | 1.79 | 95 | 55 | Male | Unilateral transtibial | TSB socket, ESR foot | Traumatic | 3 |

#2 | 1.70 | 86 | 57 | Male | Unilateral transtibial | TSB socket, ESR foot | Traumatic | 32 |

#3 | 1.72 | 110 | 40 | Male | Unilateral transtibial | PTB socket, ESR foot | Traumatic | 33 |

#4 | 1.52 | 61 | 48 | Female | Bilateral transtibial | TSB socket, ESR foot | Vascular | 4 |

**Table 5.**Comparisons of Supervised Classifiers under 5-fold Cross Validation and the best suited hyperparameters for each.

Classifier | 5-Fold Accuracy (%) | Fold Std. (±%) | Best Parameters |
---|---|---|---|

SVM | 77.22 | 0.54 | Kernel: Gaussian | Box Constraint: 193 |

KNN | 75.76 | 2.26 | Num. Neighbours: 12 | Distance Metric: Correlation | Distance Weight: Squared Inverse |

Random Forest | 71.84 | 0.69 | Num. Learners: 188 | Num. Splits: 4392 |

Adaboost | 69.23 | 3.18 | Num. Learners: 211 | Num. Splits: 910 | Learn Rate: 0.07 |

NB | 64.40 | 0.95 | Gaussian Kernel Distribution |

LDA | 73.91 | 0.93 | Quadratic |

LSTM | 78.46 | 2.89 | Dropout Factor: 0.2 | Num. Hidden Units: 190 |

**Table 6.**Comparison of mean classification accuracy in SVM and LSTM classifiers for varying resolutions of activity label, using 5-fold cross-validation.

Label Resolution Level | 1 | 2 | 3 |
---|---|---|---|

SVM accuracy (%) | 77.22 ± 0.54 | 62.60 ± 2.73 | 32.74 ± 2.46 |

LSTM accuracy (%) | 78.46 ± 2.89 | 73.77 ± 1.83 | 41.85 ± 4.19 |

Activity | No. of Test Samples | LSTM F1 Scores | SVM F1 Scores |
---|---|---|---|

Downstairs | 131 | 80.0 | 74.6 |

Upstairs | 138 | 80.1 | 75.7 |

Flat | 3826 | 83.8 | 84.2 |

Uphill | 670 | 68.7 | 69.9 |

Downhill | 701 | 62.8 | 61.7 |

Activity | No. of Test Samples | LSTM F1 Scores | SVM F1 Scores |
---|---|---|---|

Downstairs | 131 | 80.3 | 75.8 |

Upstairs | 138 | 77.4 | 75.1 |

Hard, Flat | 3506 | 80.2 | 82.3 |

Hard, Uphill | 647 | 69.9 | 70.8 |

Hard, Downhill | 668 | 63.1 | 61.5 |

Soft, Flat | 321 | 57.1 | 61.3 |

Soft, Uphill | 22 | 27.1 | 36.2 |

Soft, Downhill | 33 | 27.6 | 36.7 |

Activity | No. of Test Samples | LSTM F1 Scores | SVM F1 Scores |
---|---|---|---|

Concrete, Camber, Downhill, Parallel | 114 | 34.8 | 38.0 |

Concrete, Camber, Downhill, Perpendicular | 19 | 3.7 | 0.0 |

Concrete, Camber, Parallel | 784 | 50.8 | 62.3 |

Concrete, Camber, Perpendicular | 176 | 8.9 | 4.4 |

Concrete, Camber, Uphill, Parallel | 108 | 39.3 | 50.5 |

Concrete, Camber, Uphill, Perpendicular | 18 | 3.2 | 0.0 |

Concrete, Downhill | 510 | 48.1 | 58.3 |

Concrete, Flat | 2440 | 42.4 | 72.5 |

Concrete, Uphill | 485 | 55.5 | 69.5 |

Downstairs | 132 | 71.5 | 74.9 |

Grass, Downhill | 30 | 31.5 | 37.1 |

Grass, Flat | 269 | 51.0 | 62.6 |

Grass, Uphill | 16 | 25.1 | 29.1 |

Gravel, Downhill | 9 | 17.8 | 24.2 |

Gravel, Flat | 6 | 7.4 | 0.0 |

Gravel, Uphill | 1 | 0.0 | 0.0 |

Red Ash, Flat | 5 | 17.4 | 25.0 |

Sand, Downhill | 2 | 0.0 | 0.0 |

Sand, Flat | 47 | 33.3 | 52.0 |

Sand, Uphill | 6 | 18.6 | 27.0 |

Stone, Downhill | 16 | 15.9 | 9.4 |

Stone, Flat | 99 | 18.4 | 19.4 |

Stone, Uphill | 35 | 38.3 | 41.3 |

Upstairs | 139 | 69.4 | 73.4 |

**Table 10.**Mean Leave-One-Subject-Out (LOSO) accuracies of each of the four lower limb-amputated subjects, using models trained only on individuals without gait impairment.

Participant | A1 | A2 | A3 | A4 | Mean | Inter-Subject Std. |
---|---|---|---|---|---|---|

SVM accuracy (%) | 52.41 | 60.92 | 46.44 | 58.41 | 54.55 | ±5.61 |

LSTM accuracy (%) | 24.97 | 48.71 | 14.79 | 25.60 | 28.52 | ±12.42 |

**Table 11.**Mean Leave-One-Subject-Out (LOSO) accuracies of each of the four lower limb-amputated subjects, using models trained only on lower limb amputated individuals.

Participant | A1 | A2 | A3 | A4 | Mean | Inter-Subject Std. |
---|---|---|---|---|---|---|

SVM accuracy (%) | 58.12 | 52.99 | 54.08 | 61.55 | 56.68 | ±3.40 |

LSTM accuracy (%) | 53.97 | 10.26 | 43.22 | 16.96 | 31.10 | ±18.06 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Jamieson, A.; Murray, L.; Stankovic, L.; Stankovic, V.; Buis, A.
Human Activity Recognition of Individuals with Lower Limb Amputation in Free-Living Conditions: A Pilot Study. *Sensors* **2021**, *21*, 8377.
https://doi.org/10.3390/s21248377

**AMA Style**

Jamieson A, Murray L, Stankovic L, Stankovic V, Buis A.
Human Activity Recognition of Individuals with Lower Limb Amputation in Free-Living Conditions: A Pilot Study. *Sensors*. 2021; 21(24):8377.
https://doi.org/10.3390/s21248377

**Chicago/Turabian Style**

Jamieson, Alexander, Laura Murray, Lina Stankovic, Vladimir Stankovic, and Arjan Buis.
2021. "Human Activity Recognition of Individuals with Lower Limb Amputation in Free-Living Conditions: A Pilot Study" *Sensors* 21, no. 24: 8377.
https://doi.org/10.3390/s21248377