A Class-Imbalanced Deep Learning Fall Detection Algorithm Using Wearable Sensors
Abstract
:1. Introduction
2. Related Work
2.1. Fall Detection
2.1.1. Threshold
2.1.2. Machine Learning
2.1.3. Deep Learning
2.2. Imbalance Algorithms
2.2.1. Data-Level Methods
2.2.2. Algorithm-Level Methods
3. Proposed Method
3.1. Residual Learning
3.2. Output Threshold Moving
4. Experimental Design and Results
4.1. Dataset and Labeling
4.2. Data Preprocessing
4.3. Evaluation Method
- True positive (TP): The ADL events have been correctly classified.
- True negative (TN): The fall events have been correctly detected.
- False positive (FP): Fall events that have not been detected.
- False negative (FN): A false alarm situation occurs.
4.4. Experimental Results
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Falls in Older Persons: Risk Factors and Prevention. 2019. Available online: https://www.ncbi.nlm.nih.gov/books/NBK235613/ (accessed on 20 August 2021).
- World Health Organization. Falls. 2021. Available online: https://www.who.int/news-room/fact-sheets/detail/falls (accessed on 20 August 2021).
- Rougier, C.; Meunier, J.; St-Arnaud, A.; Rousseau, J. Robust video surveillance for fall detection based on human shape deformation. IEEE Trans. Circuits Syst. Video Technol. 2011, 21, 611–622. [Google Scholar] [CrossRef]
- Abdo, H.; Amin, K.M.; Hamad, A.M. Fall detection based on RetinaNet and MobileNet convolutional neural networks. In Proceedings of the 2020 15th International Conference on Computer Engineering and Systems (ICCES), Cairo, Egypt, 5–16 December 2020; pp. 1–7. [Google Scholar]
- Chen, Y.; Du, R.; Luo, K.; Xiao, Y. Fall detection system based on real-time pose estimation and SVM. In Proceedings of the 2021 IEEE 2nd International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), Nanchang, China, 26–28 March 2021; pp. 990–993. [Google Scholar]
- Huang, Z.; Liu, Y.; Fang, Y.; Horn, B.K.P. Video-based Fall Detection for Seniors with Human Pose Estimation. In Proceedings of the 2018 4th International Conference on Universal Village (UV), Boston, MA, USA, 21–24 October 2018; pp. 1–4. [Google Scholar]
- Lu, N.; Wu, Y.; Feng, L.; Song, J. Deep learning for fall detection: Three-Dimensional CNN combined with LSTM on video kinematic data. IEEE J. Biomed. Health Inform. 2018, 23, 314–323. [Google Scholar] [CrossRef]
- Jeon, S.-B.; Nho, Y.-H.; Park, S.-J.; Kim, W.-G.; Tcho, I.-W.; Kim, D.; Kwon, D.-S.; Choi, Y.-K. Self-powered fall detection system using pressure sensing triboelectric nanogenerators. Nano Energy 2017, 41, 139–147. [Google Scholar] [CrossRef]
- Chen, W.-H.; Ma, H.-P. A fall detection system based on infrared array sensors with tracking capability for the elderly at home. In Proceedings of the 2015 17th International Conference on E-health Networking, Application & Services (HealthCom), Boston, MA, USA, 14–17 October 2015; pp. 428–434. [Google Scholar]
- Alex, J.S.R.; Abai Kumar, M.; Swathy, D.V. Deep Learning Approaches for Fall Detection Using Acoustic Information. In Advances in Smart Grid Technology. Lecture Notes in Electrical Engineering; Springer: Singapore, 2021; Volume 688. [Google Scholar]
- Ariani, A.; Redmond, S.J.; Chang, D.; Lovell, N.H. Software simulation of unobtrusive falls detection at night-time using passive infrared and pressure mat sensors. In Proceedings of the 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, Buenos Aires, Argentina, 31 August–4 September 2010; pp. 2115–2118. [Google Scholar]
- Dinh, C.; Struck, M. A new real-time fall detection approach using fuzzy logic and a neural network. In Proceedings of the 6th International Workshop on Wearable, Micro, and Nano Technologies for Personalized Health, Oslo, Norway, 24–26 June 2009; pp. 57–60. [Google Scholar]
- Bourke, A.K.; Lyons, G.M. A threshold-based fall-detection algorithm using a bi-axial gyroscope sensor. Med. Eng. Phys. 2008, 30, 84–90. [Google Scholar] [CrossRef] [PubMed]
- Chen, X.; Jiang, S.; Lo, B. Subject-independent slow fall detection with wearable sensors via deep learning. In Proceedings of the 2020 IEEE SENSORS, Rotterdam, The Netherlands, 25–28 October 2020. [Google Scholar]
- Sun, J.; Fu, Y.; Li, S.; He, J.; Xu, C.; Tan, L. Sequential human activity recognition based on deep convolutional network and extreme learning machine using wearable sensors. J. Sens. 2018, 2018, 1–10. [Google Scholar] [CrossRef]
- Desai, K.; Mane, P.; Dsilva, M.; Zare, A.; Shingala, P.; Ambawade, D. A Novel Machine Learning Based Wearable Belt for Fall Detection. In Proceedings of the 2020 IEEE International Conference on Computing, Power and Communication Technologies (GUCON), Greater Noida, India, 2–4 October 2020; pp. 502–505. [Google Scholar]
- Hu, L.; Chen, Y.; Wang, S.; Wang, J.; Shen, J.; Jiang, X.; Shen, Z. Less annotation on personalized activity recognition using context data. In Proceedings of the 2016 Intl IEEE Conferences on Ubiquitous Intelligence & Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People, and Smart World Congress (UIC/ATC/ScalCom/CBDCom/IoP/SmartWorld), Toulouse, France, 18–21 July 2016; pp. 327–332. [Google Scholar]
- Wongyai, C. A Development of a wrist-worn fall detection using binary neural network. In Proceedings of the 2021 9th International Electrical Engineering Congress (iEECON), Pattaya, Thailand, 10–12 March 2021; pp. 472–475. [Google Scholar]
- Casilari, E.; Ramón, J.A.S.; García, J.M.C. Analysis of a Smartphone-Based Architecture with Multiple Mobility Sensors for Fall Detection. PLoS ONE 2016, 11, e0168069. [Google Scholar] [CrossRef] [Green Version]
- Li, Q.; Stankovic, J.A.; Hanson, M.A.; Barth, A.T.; Lach, J.; Zhou, G. Accurate, fast fall detection using gyroscopes and accelerometer-derived posture information. In Proceedings of the 2009 Sixth International Workshop on Wearable and Implantable Body Sensor Networks, Berkeley, CA, USA, 3–5 June 2009; pp. 138–143. [Google Scholar]
- Giuffrida, D.; Benetti, G.; de Martini, D.; Facchinetti, T. Fall detection with supervised machine learning using wearable sensors. In Proceedings of the 2019 IEEE 17th International Conference on Industrial Informatics (INDIN), Helsinki, Finland, 22–25 July 2019; pp. 253–259. [Google Scholar]
- Yu, S.; Chen, H.; Brown, R.A. Hidden markov model-based fall detection with motion sensor orientation calibration: A Case for real-life home monitoring. IEEE J. Biomed. Health Inform. 2017, 22, 1847–1853. [Google Scholar] [CrossRef] [PubMed]
- Martinez-Villaseor, L.; Ponce, H. Design and Analysis for Fall Detection System Simplification. J. Vis. Exp. 2020, 158, e60361. [Google Scholar]
- Wang, G.; Li, Q.; Wang, L.; Zhang, Y.; Liu, Z. CMFALL: A cascade and parallel multi-state fall detection algorithm using waist-mounted tri-axial accelerometer signals. IEEE Trans. Consum. Electron. 2020, 66, 261–270. [Google Scholar] [CrossRef]
- Waheed, M.; Afzal, H.; Mehmood, K. NT-FDS—A Noise Tolerant Fall Detection System Using Deep Learning on Wearable Devices. Sensors 2021, 21, 2006. [Google Scholar] [CrossRef] [PubMed]
- Chawla, N.V.; Bowyer, K.W.; Hall, L.O.; Kegelmeyer, W.P. SMOTE: Synthetic Minority Over-sampling Technique. J. Artif. Intell. Res. 2002, 16, 321–357. [Google Scholar] [CrossRef]
- Lee, H.; Park, M.; Kim, J. Plankton classification on imbalanced large scale database via convolutional neural networks with transfer learning. In Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, 25–28 September 2016; pp. 3713–3717. [Google Scholar]
- Lin, T.Y.; Goyal, P.; Girshick, R.; He, K.; Dollár, P. Focal Loss for Dense Object Detection. In Proceedings of the IEEE international conference on computer vision, Venice, Italy, 22–29 October 2017. [Google Scholar]
- Wang, S.; Liu, W.; Wu, J.; Cao, L.; Meng, Q.; Kennedy, P.J. Training deep neural networks on imbalanced data sets. In Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada, 24–29 July 2016; pp. 4368–4374. [Google Scholar]
- Fuqua, D.; Razzaghi, T. A cost-sensitive convolution neural network learning for control chart pattern recognition. Expert Syst. Appl. 2020, 150, 113275. [Google Scholar] [CrossRef]
- Buda, M.; Maki, A.; Mazurowski, M.A. A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw. 2018, 106, 249–259. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Ioffe, S.; Szegedy, C. Batch Normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning, Lille, France, 7–9 July 2015. [Google Scholar]
- Frank, K.; Vera Nadales, M.J.; Robertson, P.; Pfeifer, T. Bayesian recognition of motion related activities with inertial sensors. In Proceedings of the 12th ACM International Conference on Ubiquitous Computing, New York, NY, USA, 26 September 2010; pp. 445–446. [Google Scholar]
- Vavoulas, G.; Chatzaki, C.; Malliotakis, T.; Pediaditis, M. The Mobiact dataset: Recognition of Activities of Daily living using Smartphones. In Proceedings of the International Conference on Information and Communication Technologies for Ageing Well and e-Health (ICT4AWE), Rome, Italy, 21 April 2016. [Google Scholar]
- Gasparrini, S.; Cippitelli, E.; Spinsante, S.; Gambi, E. A Depth-Based Fall Detection System Using a Kinect® Sensor. Sensors 2014, 14, 2756–2775. [Google Scholar] [CrossRef] [PubMed]
- Medrano, C.; Igual, R.; Plaza, I.; Castro, M. Detecting Falls as Novelties in Acceleration Patterns Acquired with Smartphones. PLoS ONE 2014, 9, e94811. [Google Scholar] [CrossRef] [Green Version]
- Kwolek, B.; Kepski, M. Human fall detection on embedded platform using depth maps and wireless accelerometer. Comput. Methods Programs Biomed. 2014, 117, 489–501. [Google Scholar] [CrossRef]
- Ojetola, O.; Gaura, E.; Brusey, J. Data Set for Fall Events and Daily Activities from Inertial Sensors. In Proceedings of the 6th ACM Multimedia Systems Conference (MMSys’15), Portland, OR, USA; 2015; pp. 243–248. [Google Scholar]
- Vilarinho, T.; Farshchian, B.; Bajer, D.G.; Dahl, O.H.; Egge, I.; Hegdal, S.S.; Lones, A.; Slettevold, J.N.; Weggersen, S.M. A Combined Smartphone and Smartwatch Fall Detection System. In Proceedings of the 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing (CIT/IUCC/DASC/PICOM), Liverpool, UK, 26–28 October 2015; pp. 1443–1448. [Google Scholar]
- Wertner, A.; Czech, P.; Pammer-Schindler, V. An Open Labelled Dataset for Mobile Phone Sensing Based Fall Detection. In Proceedings of the 12th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (MOBIQUITOUS 2015), Coimbra, Portugal, 22–24 July 2015; pp. 277–278. [Google Scholar]
- Sucerquia, A.; López, J.D.; Vargas-Bonilla, J.F. SisFall: A Fall and Movement Dataset. Sensors 2017, 17, 198. [Google Scholar] [CrossRef]
- Micucci, D.; Mobilio, M.; Napoletano, P. UniMiB SHAR: A new dataset for human activity recognition using acceleration data from smartphones. IEEE Sens. Lett. 2016, 2, 1101. [Google Scholar] [CrossRef] [Green Version]
- Martínez-Villaseñor, L.; Ponce, H.; Brieva, J.; Moya-Albor, E.; Núñez-Martínez, J.; Peñafort-Asturiano, C. UP-fall detection dataset: A multimodal approach. Sensors 2019, 19, 1988. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; et al. TensorFlow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation. (OSDI), Savannah, GA, USA, 2–4 November 2016; pp. 265–283. [Google Scholar]
- Hader, G.K.; Ben Ismail, M.M.; Bchir, O. Automatic fall detection using region-based convolutional neural network. Int. J. Inj. Control. Saf. Promot. 2020, 27, 546–557. [Google Scholar] [CrossRef]
- Musci, M.; De Martini, D.; Blago, N.; Facchinetti, T.; Piastra, M. Online Fall Detection using Recurrent Neural Networks. arXiv 2018, arXiv:1804.04976. [Google Scholar]
- Yao, S.; Hu, S.; Zhao, Y.; Zhang, A.; Abdelzaher, T. DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing. In Proceedings of the 26th International Conference on World Wide Web, Perth, Australia, 3 April 2016. [Google Scholar]
- Grzymala-Busse, J.W.; Goodwin, L.K.; Grzymala-Busse, W.J.; Zheng, X. An Approach to Imbalanced Data Sets Based on Changing Rule Strength. In Rough-Neural Computing. Cognitive Technologies. Springer: Berlin/Heidelberg, Germany, 2004. [Google Scholar]
- Kubat, M.; Holte, R.C.; Matwin, S. Machine Learning for the Detection of Oil Spills in Satellite Radar Images. Mach. Learn. 1998, 30, 195–215. [Google Scholar] [CrossRef] [Green Version]
- Radivojac, P.; Chawla, N.V.; Dunker, A.K.; Obradovic, Z. Classification and knowledge discovery in protein databases. J. Biomed. Inform. 2004, 37, 224–239. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Authors | Num of Samples | Num of ADL Samples | Num of Fall Samples | Imbalance Ratio |
---|---|---|---|---|
Bourke et al. [13] | 480 | 240 | 240 | 1.00 |
Casilari et al. [19] | 530 | 371 | 159 | 2.33 |
Li et al. [20] | 350 | 225 | 125 | 1.8 |
Yu et al. [22] | 585 | 385 | 200 | 1.925 |
Martinez-Villaseor et al. [23] | 559 | 304 | 255 | 1.19 |
Wang et al. [24] | 3580 | 1790 | 1790 | 1.00 |
Waheed et al. [25] * | 770 | 395 | 375 | 1.05 |
8250 | 4530 | 3720 | 1.22 |
Dataset | Num of Type of ADLs/Falls | Num of Samples (ADLs/Falls) | Type of Sensors * |
---|---|---|---|
DLR [34] | 15/1 | 1017 (961/56) | A, G, M |
MobiAct [35] | 9/4 | 2526 (1879/647) | A, G, O |
TST Fall detection [36] | 4/4 | 264 (132/132) | A |
tFall [37] | 7/8 | 10,909 (9883/1026) | A |
UR Fall Detection [38] | 5/4 | 70 (40/30) | A |
Cogent Labs [39] | 8/6 | 1968 (1520/448) | A, G |
Gravity Project [40] | 7/12 | 117 (45/72) | A |
Graz [41] | 10/4 | 2460 (2240/220) | A, O |
UMAFall [19] | 8/3 | 531 (322/209) | A, G, M |
SisFall [42] | 19/15 | 4505 (2707/1798) | A, A, G |
UniMiB SHAR [43] | 9/8 | 7013 (5314/1699) | A |
UP-Fall [44] | 6/5 | 559 (304/255) | A, G |
Model | Architecture |
---|---|
5-CNN | Input + 5 × (Convolution + ReLU) + Softmax |
LSTM | Input + 2 × (LSTM + Dropout) + Softmax |
DeepSense | 3 × (Input + Convolution) + Concatenate + 2 × LSTM + Softmax |
ResNet34 | Input + (3 + 4 + 6 + 3) × (Convolution + ReLU) + Softmax |
ResNet10 | Input + (1 + 1 + 1 + 1) × (Convolution + ReLU) + Softmax |
Model | Sensitivity (%) | Specificity (%) | F-Score (%) | AUC (%) |
---|---|---|---|---|
5-CNN | 96.54 | 54.41 | 97.10 | 94.81 |
LSTM | 95.21 | 35.91 | 96.09 | 92.41 |
DeepSense | 97.17 | 64.37 | 96.84 | 89.03 |
ResNet34 | 97.33 | 65.08 | 97.79 | 97.06 |
ResNet10 | 97.91 | 72.89 | 98.25 | 98.27 |
Method | Sensitivity (%) | Specificity (%) | F-Score (%) | AUC (%) |
---|---|---|---|---|
Baseline (ResNet10) | 97.91 | 72.89 | 98.25 | 98.27 |
Class-Weight | 98.03 | 74.49 | 98.31 | 98.31 |
SMOTE | 97.49 | 66.88 | 97.70 | 96.39 |
SMOTE + Threshold | 99.17 | 89.98 | 97.56 | 96.80 |
Focal Loss | 97.44 | 66.49 | 97.90 | 97.39 |
Proposed | 99.33 | 91.86 | 98.44 | 98.35 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhang, J.; Li, J.; Wang, W. A Class-Imbalanced Deep Learning Fall Detection Algorithm Using Wearable Sensors. Sensors 2021, 21, 6511. https://doi.org/10.3390/s21196511
Zhang J, Li J, Wang W. A Class-Imbalanced Deep Learning Fall Detection Algorithm Using Wearable Sensors. Sensors. 2021; 21(19):6511. https://doi.org/10.3390/s21196511
Chicago/Turabian StyleZhang, Jing, Jia Li, and Weibing Wang. 2021. "A Class-Imbalanced Deep Learning Fall Detection Algorithm Using Wearable Sensors" Sensors 21, no. 19: 6511. https://doi.org/10.3390/s21196511
APA StyleZhang, J., Li, J., & Wang, W. (2021). A Class-Imbalanced Deep Learning Fall Detection Algorithm Using Wearable Sensors. Sensors, 21(19), 6511. https://doi.org/10.3390/s21196511