A Lightweight Pose Sensing Scheme for Contactless Abnormal Gait Behavior Measurement
Abstract
:1. Introduction
- We constructed a lightweight OP model with Depthwise Separable Convolution for real-time extraction of abnormal gait features. This significantly reduced the computing workload required for hardware-intensive devices.
- We performed a 3D reconstruction on the 2D lower limb data extracted from subjects and obtained a total of 11 abnormal gait features from that data. Then, we further processed the extracted data to obtain step length features. These steps improved the data structure and diversified feature types.
- We used machine learning algorithms to filter and classify abnormal gait features to the measurement of abnormal gait behavior caused by different diseases.
2. Experimental Method
2.1. Establishment of Experimental Models
2.2. 3D Construction of Lower Limbs
2.3. Extraction of Step Length Features
3. Analysis of Abnormal Gait Behavior
3.1. Analysis of Gait Characteristics for Different Diseases
3.2. Collection of Experimental Data
3.3. Feature Screening
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Yaman, O.; Ertam, F.; Tuncer, T. Automated Parkinson’s Disease Recognition Based on Statistical Pooling Method Using Acoustic Features. Med. Hypotheses 2020, 135, 109483. [Google Scholar] [CrossRef]
- Sakar, C.O.; Kursun, O. Telediagnosis of Parkinson’s Disease Using Measurements of Dysphonia. J. Med. Syst. 2010, 34, 591–599. [Google Scholar] [CrossRef]
- Sijobert, B.; Azevedo Coste, C.; Denys, J.; Geny, C. IMU Based Detection of Freezing of Gait and Festination in Parkinson’s Disease. In Proceedings of the 2014 IEEE 19th International Functional Electrical Stimulation Society Annual Conference (IFESS), Kuala Lumpur, Malaysia, 17–19 September 2014; pp. 1–3. [Google Scholar]
- Zhao, H.; Wang, Z.; Qiu, S.; Shen, Y.; Wang, J. IMU-Based Gait Analysis for Rehabilitation Assessment of Patients with Gait Disorders. In Proceedings of the 2017 4th International Conference on Systems and Informatics, Hangzhou, China, 11–13 November 2017; pp. 622–626. [Google Scholar]
- Wang, L.; Sun, Y.; Li, Q.; Liu, T.; Yi, J. IMU-Based Gait Normalcy Index Calculation for Clinical Evaluation of Impaired Gait. IEEE J. Biomed. Health Inform. 2021, 25, 3–12. [Google Scholar] [CrossRef]
- Li, W.; Lu, W.; Sha, X.; Xing, H.; Lou, J.; Sun, H.; Zhao, Y. Wearable Gait Recognition Systems Based on MEMS Pressure and Inertial Sensors: A Review. IEEE Sens. J. 2022, 22, 1092–1104. [Google Scholar] [CrossRef]
- Guan, Z.; Li, S.; Cheng, Y.; Man, C.; Mao, W. A Video-Based Fall Detection Network by Spatio-Temporal Joint-Point Model on Edge Devices. In Proceedings of the 2021 Design, Automation & Test in Europe, Virtual, 1–5 February 2021; pp. 422–427. [Google Scholar]
- Sato, K.; Nagashima, Y.; Mano, T.; Iwata, A.; Toda, T. Quantifying Normal and Parkinsonian Gait Features from Home Movies: Practical Application of a Deep Learning–Based 2D Pose Estimator. PLoS ONE 2019, 14, e0223549. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Liu, C.; Tan, X.; Wang, P.; Zhang, Y.; Li, Y. Recognition Algorithm of Parkinson’s Disease Based on Weighted Local Discriminant Preservation Projection Embedded Ensemble Algorithm. In Proceedings of the BIBE 2019 the Third International Conference on Biological Information and Biomedical Engineering, Hangzhou, China, 20–22 July 2019; pp. 100–105. [Google Scholar]
- Guo, Y.; Deligianni, F.; Gu, X.; Yang, G.Z. 3-D Canonical Pose Estimation and Abnormal Gait Recognition with a Single RGB-D Camera. IEEE Robot. Autom. Lett. 2019, 4, 3617–3624. [Google Scholar] [CrossRef] [Green Version]
- D’Antonio, E.; Taborri, J.; Mileti, I.; Rossi, S.; Patane, F. Validation of a 3D Markerless System for Gait Analysis Based on OpenPose and Two RGB Webcams. IEEE Sens. J. 2021, 21, 17064–17075. [Google Scholar] [CrossRef]
- Chen, H.L.; Huang, C.C.; Yu, X.G.; Xu, X.; Sun, X.; Wang, G.; Wang, S.J. An Efficient Diagnosis System for Detection of Parkinson’s Disease Using Fuzzy k-Nearest Neighbor Approach. Expert Syst. Appl. 2013, 40, 263–271. [Google Scholar] [CrossRef]
- Hariharan, M.; Polat, K.; Sindhu, R. A New Hybrid Intelligent System for Accurate Detection of Parkinson’s Disease. Comput. Methods Programs Biomed. 2014, 113, 904–913. [Google Scholar] [CrossRef]
- Cao, Z.; Hidalgo, G.; Simon, T.; Wei, S.E.; Sheikh, Y. OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE Trans. Pattern Anal. Mach. Intell. 2021, 43, 172–186. [Google Scholar] [CrossRef] [Green Version]
- Abe, K.; Tabei, K.-I.; Matsuura, K.; Kobayashi, K.; Ohkubo, T. OpenPose-Based Gait Analysis System For Parkinson’s Disease Patients From Arm Swing Data. In Proceedings of the 2021 International Conference on Advanced Mechatronic Systems, Tokyo, Japan, 9–12 December 2021; pp. 61–65. [Google Scholar]
- Yamamoto, M.; Shimatani, K.; Hasegawa, M.; Kurita, Y.; Ishige, Y.; Takemura, H. Accuracy of Temporo-Spatial and Lower Limb Joint Kinematics Parameters Using OpenPose for Various Gait Patterns with Orthosis. IEEE Trans. Neural Syst. Rehabil. Eng. 2021, 29, 2666–2675. [Google Scholar] [CrossRef]
- Mai, W.; Wu, F.; Guo, Z.; Xiang, Y.; Liu, G.; Chen, X. A Fall Detection Alert System Based on Lightweight Openpose and Spatial-Temporal Graph Convolution Network. J. Phys. Conf. Ser. 2021, 2035, 012036. [Google Scholar] [CrossRef]
- Nagai, T.; Takahashi, Y.; Endo, K.; Ikegami, R.; Ueno, R.; Yamamoto, K. Analysis of Spastic Gait in Cervical Myelopathy: Linking Compression Ratio to Spatiotemporal and Pedobarographic Parameters. Gait Posture 2018, 59, 152–156. [Google Scholar] [CrossRef]
- Zeng, H.; Chen, W. An Evaluation Approach of Multi-Person Movement Synchronization Level Using OpenPose. In Proceedings of the 40th Chinese Control Conference, Shanghai, China, 26–28 July 2021; pp. 3900–3905. [Google Scholar]
- Alcock, L.; Galna, B.; Perkins, R.; Lord, S.; Rochester, L. Step Length Determines Minimum Toe Clearance in Older Adults and People with Parkinson’s Disease. J. Biomech. 2018, 71, 30–36. [Google Scholar] [CrossRef]
- Armand, S.; Decoulon, G.; Bonnefoy-Mazure, A. Gait Analysis in Children with Cerebral Palsy. EFORT Open Rev. 2016, 1, 448–460. [Google Scholar] [CrossRef]
- Schmitt, D.; Vap, A.; Queen, R.M. Effect of End-Stage Hip, Knee, and Ankle Osteoarthritis on Walking Mechanics. Gait Posture 2015, 42, 373–379. [Google Scholar] [CrossRef]
- Hoogkamer, W.; Potocanac, Z.; van Calenbergh, F.; Duysens, J. Quick Foot Placement Adjustments during Gait Are Less Accurate in Individuals with Focal Cerebellar Lesions. Gait Posture 2017, 58, 390–393. [Google Scholar] [CrossRef]
- Hussain, I.; Park, S.J. Prediction of Myoelectric Biomarkers in Post-Stroke Gait. Sensors 2021, 21, 5334. [Google Scholar] [CrossRef]
- Hussain, I.; Park, S.J. Quantitative Evaluation of Task-Induced Neurological Outcome after Stroke. Brain Sci. 2021, 11, 900. [Google Scholar] [CrossRef]
- Hussain, I.; Park, S.J. HealthSOS: Real-Time Health Monitoring System for Stroke Prognostics. IEEE Access 2020, 8, 213574–213586. [Google Scholar] [CrossRef]
- Tang, Q.; Xia, G.; Zhang, X.; Long, F. A Customer Churn Prediction Model Based on XGBoost and MLP. In Proceedings of the 2020 International Conference on Computer Engineering and Application, ICCEA 2020, Guangzhou, China, 18–20 March 2020; pp. 608–612. [Google Scholar]
- Rahman, B.; Hendric Spits Warnars, H.L.; Subirosa Sabarguna, B.; Budiharto, W. Heart Disease Classification Model Using K-Nearest Neighbor Algorithm. In Proceedings of the 2021 6th International Conference on Informatics and Computing, ICIC 2021, Hyderabad, India, 16–17 September 2021. [Google Scholar]
- Zhang, S.; Li, X.; Zong, M.; Zhu, X.; Wang, R. Efficient KNN Classification with Different Numbers of Nearest Neighbors. IEEE Trans. Neural Netw. Learn. Syst. 2018, 29, 1774–1785. [Google Scholar] [CrossRef]
- Priyadarshini, R.K.; Bazila, A.B.; Nagamani, T. Gradient Boosted Decision Tree Based Classification for Recognizing Human Behavior. In Proceedings of the 2019 International Conference on Advances in Computing & Communication Engineering (ICACCE-2019), Sathyamangalam, India, 4–6 April 2019. [Google Scholar]
- Zheng, X.; Wang, Z.; Chung, W. Efficient Parameter Selection for SVM: TheCase of Business Intelligence Categorization. In Proceedings of the IEEE International Conference on Intelligence and Security Informatics: Security and Big Data, Beijing, China, 22–24 July 2017; pp. 158–160. [Google Scholar]
- Sheykhmousa, M.; Mahdianpari, M.; Ghanbari, H.; Mohammadimanesh, F.; Ghamisi, P.; Homayouni, S. Support Vector Machine Versus Random Forest for Remote Sensing Image Classification: A Meta-Analysis and Systematic Review. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 13, 6308–6325. [Google Scholar] [CrossRef]
- Özel, E.; Tekin, R.; Kaya, Y. Implementation of Artifact Removal Algorithms in Gait Signals for Diagnosis of Parkinson Disease. Traitement Signal 2021, 38, 587–597. [Google Scholar] [CrossRef]
Gait | Gait Characteristics | Corresponding Types of Diseases |
---|---|---|
Magnetic step (or Freezing gait) | The walking steps are small and the movements are stiff and slow. | This gait may indicate Parkinson’s disease. The patient has symptoms of tremor, stiff limbs, and slow movement [20] |
Mop step | The patient moves their left and right legs at inconsistent paces, and tends to walk by dragging their feet. | This gait may indicate lumbar disc herniation or cervical spondylitis myelopathy. Due to nerve compression, the patient has weak muscle on one leg, and generally drags one foot during walking [18] |
Scissor Step | The patient tends to walk with their toes facing inward and their legs crossed. | This gait may indicate cerebral palsy or spinal cord injury, which can lead to impaired neurological function and affect physical activity [21] |
Intermittent fragmentation | The patient experiences lameness and often feels the need to stop and rest due to pain and numbness in legs. | This gait may indicate osteoarthritis, lumbar spinal stenosis, vasculitis, or diabetes [22] |
Drunk step | The patient cannot walk in a straight line and tend to stagger. | This gait may indicate cerebral hemorrhage, cerebral infarction, brain tumor, or cerebellar lesions. These diseases can cause cerebellar damage or cerebellar dysfunction [23]. |
Number of Features | Score-2D | Score-3D |
---|---|---|
11 | 0.9167 | 0.9306 |
10 | 0.9028 | 0.8889 |
9 | 0.8889 | 0.9167 |
8 | 0.8472 | 0.9306 |
7 | 0.8611 | 0.8611 |
6 | 0.8472 | 0.8750 |
5 | 0.8056 | 0.8333 |
4 | 0.7639 | 0.6944 |
3 | 0.7083 | 0.7083 |
2 | 0.5556 | 0.5833 |
1 | 0.3333 | 0.4028 |
Machine Learning Algorithms | Parameters |
---|---|
GB | = 10, loss function = deviance, subsample = 1.0. |
KN | Weights = distance, n = 4, distance measure = 1. |
MLP | Activation = ReLU, = (50,50), optimizer = Adam, = 800, = 1. |
RF | Number of decision trees = 57. |
SVM | Kernel = ‘linear’, Kernel coefficient = 1. |
Machine Learning Algorithms | 2D—11 Features | 3D—8 Features | 3D—11 Features | |||
---|---|---|---|---|---|---|
Recall | Precision | Recall | Precision | Recall | Precision | |
GB | 0.7661 | 0.7778 | 0.8333 | 0.8611 | 0.8194 | 0.8472 |
KN | 0.7211 | 0.7361 | 0.7500 | 0.7778 | 0.7533 | 0.7638 |
MLP | 0.7557 | 0.7778 | 0.7944 | 0.8055 | 0.8344 | 0.8472 |
RF | 0.8888 | 0.8918 | 0.9167 | 0.9213 | 0.9032 | 0.9048 |
SVM | 0.7881 | 0.7918 | 0.8917 | 0.9027 | 0.8571 | 0.8611 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhao, Y.; Li, J.; Wang, X.; Liu, F.; Shan, P.; Li, L.; Fu, Q. A Lightweight Pose Sensing Scheme for Contactless Abnormal Gait Behavior Measurement. Sensors 2022, 22, 4070. https://doi.org/10.3390/s22114070
Zhao Y, Li J, Wang X, Liu F, Shan P, Li L, Fu Q. A Lightweight Pose Sensing Scheme for Contactless Abnormal Gait Behavior Measurement. Sensors. 2022; 22(11):4070. https://doi.org/10.3390/s22114070
Chicago/Turabian StyleZhao, Yuliang, Jian Li, Xiaoai Wang, Fan Liu, Peng Shan, Lianjiang Li, and Qiang Fu. 2022. "A Lightweight Pose Sensing Scheme for Contactless Abnormal Gait Behavior Measurement" Sensors 22, no. 11: 4070. https://doi.org/10.3390/s22114070
APA StyleZhao, Y., Li, J., Wang, X., Liu, F., Shan, P., Li, L., & Fu, Q. (2022). A Lightweight Pose Sensing Scheme for Contactless Abnormal Gait Behavior Measurement. Sensors, 22(11), 4070. https://doi.org/10.3390/s22114070