A Novel Gait Phase Recognition Method Based on DPF-LSTM-CNN Using Wearable Inertial Sensors
Abstract
:1. Introduction
2. Methods
2.1. Data Collection
2.2. Data Preprocessing
2.3. Structure of LSTM-CNN Neural Network
2.3.1. Structure of LSTM
2.3.2. Structure of CNN
2.3.3. Structure of LSTM-CNN
2.4. Evaluation Method
3. Experiment and Results of Gait Recognition
3.1. The Training of LSTM-CNN
3.2. Experiment and Results Based on DPF-LSTM-CNN
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Chen, B.; Ma, H.; Qin, L.-Y.; Gao, F.; Chan, K.-M.; Law, S.-W.; Qin, L.; Liao, W.-H. Recent developments and challenges of lower extremity exoskeletons. J. Orthop. Transl. 2016, 5, 26–37. [Google Scholar] [CrossRef] [Green Version]
- Wang, L.; Chen, C.; Dong, W.; Du, Z.; Shen, Y.; Zhao, G. Locomotion Stability Analysis of Lower Extremity Augmentation Device. J. Bionic Eng. 2019, 16, 99–114. [Google Scholar] [CrossRef]
- Chen, C.-F.; Du, Z.-J.; He, L.; Shi, Y.-J.; Wang, J.-Q.; Xu, G.-Q.; Zhang, Y.; Wu, D.-M.; Dong, W. Development and Hybrid Control of an Electrically Actuated Lower Limb Exoskeleton for Motion Assistance. IEEE Access 2019, 7, 169107–169122. [Google Scholar] [CrossRef]
- Zeng, D.; Qu, C.; Ma, T.; Qu, S.; Yin, P.; Zhao, N.; Xia, Y. Research on a gait detection system and recognition algorithm for lower limb exoskeleton robot. J. Braz. Soc. Mech. Sci. Eng. 2021, 43, 298. [Google Scholar] [CrossRef]
- Martinez, A.; Lawson, B.; Durrough, C.; Goldfarb, M. A Velocity-Field-Based Controller for Assisting Leg Movement During Walking with a Bilateral Hip and Knee Lower Limb Exoskeleton. IEEE Trans. Robot. 2019, 35, 307–316. [Google Scholar] [CrossRef]
- Zheng, T.; Zhu, Y.; Zhang, Z.; Zhao, S.; Chen, J.; Zhao, J. Parametric Gait Online Generation of a Lower-limb Exoskeleton for Individuals with Paraplegia. J. Bionic Eng. 2018, 15, 941–949. [Google Scholar] [CrossRef]
- Xu, D.; Liu, X.; Wang, Q. Knee Exoskeleton Assistive Torque Control Based on Real-Time Gait Event Detection. IEEE Trans. Med. Robot. Bionics 2019, 1, 158–168. [Google Scholar] [CrossRef]
- Grimmer, M.; Schmidt, K.; Duarte, J.E.; Neuner, L.; Koginov, G.; Riener, R. Stance and Swing Detection Based on the Angular Velocity of Lower Limb Segments During Walking. Front. Neurorobot. 2019, 13, 57. [Google Scholar] [CrossRef] [Green Version]
- Ding, Z.; Yang, C.; Xing, K.; Ma, X.; Yang, K.; Guo, H.; Yi, C.; Jiang, F. The Real Time Gait Phase Detection Based on Long Short-Term Memory. In Proceedings of the IEEE Third International Conference on Data Science in Cyberspace (DSC), Guangzhou, China, 18–21 June 2018; pp. 33–38. [Google Scholar] [CrossRef]
- Rajib, G. Faster R-CNN and recurrent neural network based approach of gait recognition with and without carried objects. Expert Syst. Appl. 2022, 205, 117730. [Google Scholar]
- Bukhari, M.; Durrani, M.Y.; Gillani, S.; Yasmin, S.; Rho, S.; Yeo, S.-S. Exploiting vulnerability of convolutional neural network-based gait recognition system. J. Supercomput. 2022, 78, 18578–18597. [Google Scholar] [CrossRef]
- Mathivanan, B.; Perumal, P. Gait Recognition Analysis for Human Identification Analysis—A Hybrid Deep Learning Process. Wirel. Pers. Commun. 2022, 126, 555–579. [Google Scholar] [CrossRef]
- Coelho, R.M.; Gouveia, J.; Botto, M.A.; Krebs, H.I.; Martins, J. Real-time walking gait terrain classification from foot-mounted Inertial Measurement Unit using Convolutional Long Short-Term Memory neural network. Expert Syst. Appl. 2022, 203, 117306. [Google Scholar] [CrossRef]
- Wang, J.; Wu, D.; Gao, Y.; Wang, X.; Li, X.; Xu, G.; Dong, W. Integral Real time Locomotion Mode Recognition Based on GA CNN for Lower Limb Exoskeleton. J. Bionic Eng. 2022, 19, 1359–1373. [Google Scholar] [CrossRef]
- Zhang, Z.; Wang, Z.; Lei, H.; Gu, W. Gait phase recognition of lower limb exoskeleton system based on the integrated network model. Biomed. Signal Process. Control. 2022, 76, 103693. [Google Scholar] [CrossRef]
- Su, B.; Smith, C.; Gutierrez Farewik, E. Gait Phase Recognition Using Deep Convolutional Neural Network with Inertial Measurement Units. Biosensors 2020, 10, 109. [Google Scholar] [CrossRef]
- Tran, L.; Hoang, T.; Nguyen, T.; Kim, H.; Choi, D. Multi-Model Long Short-Term Memory Network for Gait Recognition Using Window-Based Data Segment. IEEE Access 2021, 9, 23826–23839. [Google Scholar] [CrossRef]
- Wu, X.; Yuan, Y.; Zhang, X.; Wang, C.; Xu, T.; Tao, D. Gait Phase Classification for a Lower Limb Exoskeleton System Based on a Graph Convolutional Network Model. IEEE Trans. Ind. Electron. 2022, 69, 4999–5008. [Google Scholar] [CrossRef]
- Chen, C.-F.; Du, Z.-J.; He, L.; Shi, Y.-J.; Wang, J.-Q.; Dong, W. A Novel Gait Pattern Recognition Method Based on LSTM-CNN for Lower Limb Exoskeleton. J. Bionic Eng. 2021, 18, 1059–1072. [Google Scholar] [CrossRef]
- Kreuzer, D.; Munz, M. Deep Convolutional and LSTM Networks on Multi-Channel Time Series Data for Gait Phase Recognition. Sensors 2021, 21, 789. [Google Scholar] [CrossRef]
- Zeni, J.A., Jr.; Richards, J.G.; Higginson, J.S. Two simple methods for determining gait events during treadmill and overground walking using kinematic data. Gait Posture 2008, 27, 710–714. [Google Scholar] [CrossRef] [Green Version]
- Yu, L.; Zheng, J.; Wang, Y.; Song, Z.; Zhan, E. Adaptive method for real-time gait phase detection based on ground contact forces. Gait Posture 2015, 41, 269–275. [Google Scholar] [CrossRef]
- Zahradka, N.; Verma, K.; Behboodi, A.; Bodt, B.; Wright, H.; Lee, S.C.K. An Evaluation of Three Kinematic Methods for Gait Event Detection Compared to the Kinetic-Based ‘Gold Standard’. Sensors 2020, 20, 5272. [Google Scholar] [CrossRef]
- Zhou, Q.; Shan, J.; Fang, B.; Zhang, S.; Sun, F.; Ding, W.; Wang, C.; Zhang, Q. Personal-specific gait recognition based on latent orthogonal feature space. Cogn. Comput. Syst. 2021, 3, 61–69. [Google Scholar] [CrossRef]
- Lu, Y.; Wang, H.; Qi, Y.; Xi, H. Evaluation of classification performance in human lower limb jump phases of signal correlation information and LSTM models. Biomed. Signal Process. Control 2021, 64, 102279. [Google Scholar] [CrossRef]
- Wang, X.; Dong, D.; Chi, X.; Wang, S.; Miao, Y.; An, M.; Gavrilov, A.I. sEMG-based consecutive estimation of human lower limb movement by using multi-branch neural network. Biomed. Signal Process. Control 2021, 68, 102781. [Google Scholar] [CrossRef]
- Wang, C.; Wu, X.; Ma, Y.; Wu, G.; Luo, Y. A Flexible Lower Extremity Exoskeleton Robot with Deep Locomotion Mode Identification. Complexity 2018, 2018, 5712108. [Google Scholar] [CrossRef]
Groups | Position Combination of IMU |
---|---|
1 | Left and right thighs |
2 | Left and right shanks |
3 | Left and right feet |
4 | Left and right thighs and shanks |
5 | Left and right thighs and feet |
6 | Left and right shanks and feet |
7 | Left and right thighs, shanks and feet |
Groups | Gait Phases | Pre (%) | Rec (%) | F1-Score (%) |
---|---|---|---|---|
1 | SU-RHS | 76.43 | 86.67 | 81.23 |
SW-L | 81.89 | 85.84 | 83.82 | |
SU-LHS | 78.75 | 91.11 | 84.48 | |
SW-R | 95.01 | 80.27 | 87.02 | |
2 | SU-RHS | 91.89 | 84.17 | 87.86 |
SW-L | 86.79 | 86.79 | 86.79 | |
SU-LHS | 85.86 | 87.11 | 86.48 | |
SW-R | 91.78 | 92.91 | 92.34 | |
3 | SU-RHS | 74.43 | 83.35 | 78.64 |
SW-L | 83.06 | 81.22 | 82.13 | |
SU-LHS | 86.50 | 81.95 | 88.63 | |
SW-R | 83.26 | 83.92 | 83.59 | |
4 | SU-RHS | 93.60 | 92.46 | 93.03 |
SW-L | 88.39 | 94.15 | 91.18 | |
SU-LHS | 92.51 | 97.86 | 95.11 | |
SW-R | 90.65 | 90.03 | 90.34 | |
5 | SU-RHS | 96.17 | 92.77 | 94.44 |
SW-L | 94.08 | 88.79 | 91.36 | |
SU-LHS | 84.00 | 94.24 | 88.83 | |
SW-R | 91.63 | 94.72 | 93.15 | |
6 | SU-RHS | 97.28 | 84.13 | 90.23 |
SW-L | 86.38 | 97.22 | 91.48 | |
SU-LHS | 88.33 | 91.22 | 89.75 | |
SW-R | 95.88 | 89.78 | 92.73 | |
7 | SU-RHS | 98.96 | 95.62 | 97.26 |
SW-L | 96.42 | 95.27 | 95.84 | |
SU-LHS | 89.08 | 94.23 | 91.58 | |
SW-R | 98.91 | 96.66 | 97.77 |
Subject | LSTM-CNN-1 | DPF-LSTM-CNN | ||
---|---|---|---|---|
Acc (%) | Macro-F1 (%) | Acc (%) | Macro-F1 (%) | |
1 | 94.84 | 93.67 | 97.63 | 97.04 |
2 | 96.37 | 96.11 | 97.84 | 97.29 |
3 | 94.12 | 95.23 | 95.29 | 96.24 |
4 | 92.66 | 94.74 | 98.63 | 95.86 |
5 | 94.58 | 93.29 | 97.33 | 96.19 |
6 | 93.38 | 93.67 | 95.43 | 96.35 |
7 | 91.88 | 92.79 | 98.11 | 94.83 |
8 | 94.76 | 94.28 | 97.21 | 96.92 |
9 | 93.71 | 95.77 | 96.78 | 97.03 |
10 | 94.43 | 94.36 | 97.02 | 96.74 |
11 | 95.01 | 94.96 | 96.75 | 98.05 |
12 | 91.33 | 93.47 | 95.87 | 96.74 |
13 | 93.71 | 95.61 | 97.23 | 95.39 |
14 | 95.44 | 94.35 | 98.39 | 96.54 |
15 | 96.36 | 95.81 | 96.56 | 97.06 |
16 | 95.27 | 92.57 | 97.76 | 97.88 |
17 | 93.19 | 93.55 | 96.83 | 96.43 |
18 | 92.32 | 95.18 | 97.73 | 95.78 |
19 | 94.79 | 94.83 | 97.39 | 94.35 |
20 | 95.32 | 93.44 | 98.46 | 96.79 |
Average | 94.17 | 94.38 | 97.21 | 96.48 |
Models | DPF-LSTM-CN | DCNN | CNN-LSTM |
---|---|---|---|
Accuracy | 97.21% | 95.37% | 94.57% |
Macro-F1 | 96.48% | 94.86% | 95.38% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, K.; Liu, Y.; Ji, S.; Gao, C.; Zhang, S.; Fu, J. A Novel Gait Phase Recognition Method Based on DPF-LSTM-CNN Using Wearable Inertial Sensors. Sensors 2023, 23, 5905. https://doi.org/10.3390/s23135905
Liu K, Liu Y, Ji S, Gao C, Zhang S, Fu J. A Novel Gait Phase Recognition Method Based on DPF-LSTM-CNN Using Wearable Inertial Sensors. Sensors. 2023; 23(13):5905. https://doi.org/10.3390/s23135905
Chicago/Turabian StyleLiu, Kun, Yong Liu, Shuo Ji, Chi Gao, Shizhong Zhang, and Jun Fu. 2023. "A Novel Gait Phase Recognition Method Based on DPF-LSTM-CNN Using Wearable Inertial Sensors" Sensors 23, no. 13: 5905. https://doi.org/10.3390/s23135905
APA StyleLiu, K., Liu, Y., Ji, S., Gao, C., Zhang, S., & Fu, J. (2023). A Novel Gait Phase Recognition Method Based on DPF-LSTM-CNN Using Wearable Inertial Sensors. Sensors, 23(13), 5905. https://doi.org/10.3390/s23135905