An Enhanced Ensemble Deep Neural Network Approach for Elderly Fall Detection System Based on Wearable Sensors
Abstract
:1. Introduction
- The proposed work introduces a wearable fall detection framework that can accurately detect Non-Fall, Pre-Fall, and Fall events. In the case of a Pre-Fall event, the framework will activate safety measures to prevent severe head injuries before the individual hits the ground. Additionally, the framework will issue remote notifications in the event of a fall to ensure timely medical assistance.
- A novel class-based ensemble architecture was developed for receiving sensory data through a head model, resulting in improved accuracy due to every ensemble memorizing a single class.
- We conducted experiments on different configurations of the proposed architecture’s CNN and RNN components. By combining the use of CNN and RNN, our model can capture both short-term and long-term dependencies in human motion. The CNN model focuses on short-term dependencies, while the ensemble RNN model captures long-term dependencies by analyzing feature maps over a sequence of data. Our result analysis confirms that this architecture provides higher accuracy than other models.
2. Materials and Methods
2.1. Dataset and Material
- Non-fall: the time interval during which the person performs ADLs.
- Pre-fall: the time interval during which a person transitions from a controlled to a dangerous state, which may result in a fall.
- Fall: the time interval during which the person is in a state transition that leads to a fall.
2.2. Proposed FDS Framwork
2.3. Architectural Motivation
2.4. Architecture
2.5. Data Augmentation
- Rotation: Rotation of sensor data refers to the transformation of the data by rotating it around a specified axis or point in three-dimensional space. The rotation can be represented by three angles, known as Euler angles, which specify the amount of rotation around each axis. In this experiment, we uses a rotation angle in the range radian.
- Scaling: Scaling sensor data refers to rescaling the magnitude of the sensor data in a window by multiplying it by a random scalar, where the random scalar is sampled from a normal distribution with a mean of 1 and a standard deviation of 0.1. The choice of a standard deviation of 0.1 for the scaling factor helps to ensure that the augmented data are not too different from the original data, while still introducing some variability
- Jitter: Jittering is a data augmentation technique used to simulate sensor noise. It adds random noise to the sensor data to make it more robust against both additive and multiplicative noise. By adding noise to the data, the model can learn to be more resilient to unexpected variations in the sensor readings. In this case, the standard deviation of the noise added was set to 0.01, which determines the amount of random noise that is added to the sensor data during the jittering process.
Algorithm 1: Online data augmentation algorithm. |
3. Results
3.1. Evaluation Metrics
- Accuracy: Accuracy is one of the most fundamental evaluation metrics. It can be formally defined as the ratio of accurateness over all experiments. It can be defined as:
- Sensitivity: Sensitivity is also known as recall score or true positive rate. It refers to the correctness of the true positive events of each available class. It can be mathematically defined as
- Specificity: Specificity is also known as the true negative rate. It refers to the percentage of all negative samples that the model correctly predicts as negative. It can be represented as
3.2. Model’s Training
3.3. Result Analysis
4. Discussion
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
Abbreviations
FDS | Fall Detection Systems |
DL | Deep Learning |
IMU | Inertial Measurement Units |
FPS | fall prevention systems |
RNN | Recurrent Nueral Network |
LSTM | Long-Short Term Memory |
CNN | convolutional Neural Network |
ADL | Activity of Daily Livings |
GRU | Gated Recurrent Unit |
DAU | Data Analytics Unit |
References
- Silva de Lima, A.L.; Evers, L.J.; Hahn, T.; Bataille, L.; Hamilton, J.L.; Little, M.A.; Okuma, Y.; Bloem, B.R.; Faber, M.J. Freezing of gait and fall detection in Parkinson’s disease using wearable sensors: A systematic review. J. Neurol. 2017, 264, 1642–1654. [Google Scholar] [CrossRef] [PubMed]
- El Halabi, N.; Daou, R.A.Z.; Achkar, R.; Hayek, A.; Börcsök, J. Monitoring system for prediction and detection of epilepsy seizure. In Proceedings of the 2019 Fourth International Conference on Advances in Computational Tools for Engineering Applications (ACTEA), Beirut, Lebanon, 3–5 July 2019; pp. 1–7. [Google Scholar]
- Butt, F.S.; La Blunda, L.; Wagner, M.F.; Schäfer, J.; Medina-Bulo, I.; Gómez-Ullate, D. Fall detection from electrocardiogram (ecg) signals and classification by deep transfer learning. Information 2021, 12, 63. [Google Scholar] [CrossRef]
- Xi, X.; Tang, M.; Miran, S.M.; Luo, Z. Evaluation of feature extraction and recognition for activity monitoring and fall detection based on wearable sEMG sensors. Sensors 2017, 17, 1229. [Google Scholar] [CrossRef] [PubMed]
- Hsieh, C.Y.; Liu, K.C.; Huang, C.N.; Chu, W.C.; Chan, C.T. Novel hierarchical fall detection algorithm using a multiphase fall model. Sensors 2017, 17, 307. [Google Scholar] [CrossRef]
- Choi, A.; Kim, T.H.; Yuhai, O.; Jeong, S.; Kim, K.; Kim, H.; Mun, J.H. Deep learning-based near-fall detection algorithm for fall risk monitoring system using a single inertial measurement unit. IEEE Trans. Neural Syst. Rehabil. Eng. 2022, 30, 2385–2394. [Google Scholar] [CrossRef]
- Choi, S.D.; Guo, L.; Kang, D.; Xiong, S. Exergame technology and interactive interventions for elderly fall prevention: A systematic literature review. Appl. Ergon. 2017, 65, 570–581. [Google Scholar] [CrossRef]
- Qiu, H.; Rehman, R.Z.U.; Yu, X.; Xiong, S. Application of wearable inertial sensors and a new test battery for distinguishing retrospective fallers from non-fallers among community-dwelling older people. Sci. Rep. 2018, 8, 16349. [Google Scholar] [CrossRef]
- Musci, M.; De Martini, D.; Blago, N.; Facchinetti, T.; Piastra, M. Online fall detection using recurrent neural networks on smart wearable devices. IEEE Trans. Emerg. Top. Comput. 2020, 9, 1276–1289. [Google Scholar] [CrossRef]
- Torti, E.; Fontanella, A.; Musci, M.; Blago, N.; Pau, D.; Leporati, F.; Piastra, M. Embedding recurrent neural networks in wearable systems for real-time fall detection. Microprocess. Microsyst. 2019, 71, 102895. [Google Scholar] [CrossRef]
- Yu, X.; Qiu, H.; Xiong, S. A novel hybrid deep neural network to predict pre-impact fall for older people based on wearable inertial sensors. Front. Bioeng. Biotechnol. 2020, 8, 63. [Google Scholar] [CrossRef]
- Anwary, A.R.; Yu, H.; Vassallo, M. Optimal foot location for placing wearable IMU sensors and automatic feature extraction for gait analysis. IEEE Sens. J. 2018, 18, 2555–2567. [Google Scholar] [CrossRef]
- Anwary, A.R.; Yu, H.; Vassallo, M. An automatic gait feature extraction method for identifying gait asymmetry using wearable sensors. Sensors 2018, 18, 676. [Google Scholar] [CrossRef] [PubMed]
- Anwary, A.R.; Yu, H.; Vassallo, M. Gait evaluation using procrustes and euclidean distance matrix analysis. IEEE J. Biomed. Health Inform. 2018, 23, 2021–2029. [Google Scholar] [CrossRef] [PubMed]
- Casilari, E.; Santoyo-Ramón, J.A.; Cano-García, J.M. Analysis of public datasets for wearable fall detection systems. Sensors 2017, 17, 1513. [Google Scholar] [CrossRef] [PubMed]
- Vilarinho, T.; Farshchian, B.; Bajer, D.G.; Dahl, O.H.; Egge, I.; Hegdal, S.S.; Lønes, A.; Slettevold, J.N.; Weggersen, S.M. A combined smartphone and smartwatch fall detection system. In Proceedings of the 2015 IEEE International Conference on Computer and Information Technology, Ubiquitous Computing and Communications, Dependable, Autonomic and Secure Computing, Pervasive Intelligence and Computing, Liverpool, UK, 26–28 October 2015; pp. 1443–1448. [Google Scholar]
- Medrano, C.; Igual, R.; Plaza, I.; Castro, M. Detecting falls as novelties in acceleration patterns acquired with smartphones. PLoS ONE 2014, 9, e94811. [Google Scholar] [CrossRef] [PubMed]
- Vavoulas, G.; Pediaditis, M.; Chatzaki, C.; Spanakis, E.G.; Tsiknakis, M. The mobifall dataset: Fall detection and classification with a smartphone. Int. J. Monit. Surveill. Technol. Res. (IJMSTR) 2014, 2, 44–56. [Google Scholar] [CrossRef]
- Casilari, E.; Santoyo-Ramón, J.A.; Cano-García, J.M. Umafall: A multisensor dataset for the research on automatic fall detection. Procedia Comput. Sci. 2017, 110, 32–39. [Google Scholar] [CrossRef]
- Micucci, D.; Mobilio, M.; Napoletano, P. Unimib shar: A dataset for human activity recognition using acceleration data from smartphones. Appl. Sci. 2017, 7, 1101. [Google Scholar] [CrossRef]
- Sucerquia, A.; López, J.D.; Vargas-Bonilla, J.F. SisFall: A fall and movement dataset. Sensors 2017, 17, 198. [Google Scholar] [CrossRef]
- Hansen, L.K.; Salamon, P. Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 1990, 12, 993–1001. [Google Scholar] [CrossRef]
- Tao, S. Deep neural network ensembles. In Proceedings of the International Conference on Machine Learning, Optimization, and Data Science, Siena, Italy, 10–13 September 2019; pp. 1–12. [Google Scholar]
- Bishop, C.M. Neural Networks for Pattern Recognition; Oxford University Press: Oxford, UK, 1995. [Google Scholar]
- Dietterich, T.G. Ensemble methods in machine learning. In Proceedings of the International Workshop on Multiple Classifier Systems, Cagliari, Italy, 21–23 June 2000; pp. 1–15. [Google Scholar]
- Acar, E. Effect of error metrics on optimum weight factor selection for ensemble of metamodels. Expert Syst. Appl. 2015, 42, 2703–2709. [Google Scholar] [CrossRef]
- Al-Khateeb, T.; Masud, M.M.; Al-Naami, K.M.; Seker, S.E.; Mustafa, A.M.; Khan, L.; Trabelsi, Z.; Aggarwal, C.; Han, J. Recurring and novel class detection using class-based ensemble for evolving data stream. IEEE Trans. Knowl. Data Eng. 2015, 28, 2752–2764. [Google Scholar] [CrossRef]
- Ohi, A.Q.; Mridha, M.F.; Hamid, M.A.; Monowar, M.M.; Kateb, F.A. Fabricnet: A fiber recognition architecture using ensemble convnets. IEEE Access 2021, 9, 13224–13236. [Google Scholar] [CrossRef]
- Maida, A. Cognitive computing and neural networks: Reverse engineering the brain. In Handbook of Statistics; Elsevier: Amsterdam, The Netherlands, 2016; Volume 35, pp. 39–78. [Google Scholar]
- Um, T.T.; Pfister, F.M.; Pichler, D.; Endo, S.; Lang, M.; Hirche, S.; Fietzek, U.; Kulić, D. Data augmentation of wearable sensor data for parkinson’s disease monitoring using convolutional neural networks. In Proceedings of the 19th ACM International Conference on Multimodal Interaction, Glasgow, UK, 13–17 November 2017; pp. 216–220. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Chollet, F. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 1251–1258. [Google Scholar]
- Ramachandran, P.; Zoph, B.; Le, Q.V. Searching for activation functions. arXiv 2017, arXiv:1710.05941. [Google Scholar]
- Chung, J.; Gulcehre, C.; Cho, K.; Bengio, Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv 2014, arXiv:1412.3555. [Google Scholar]
- Schuster, M.; Paliwal, K.K. Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 1997, 45, 2673–2681. [Google Scholar] [CrossRef]
Dataset | Type of Sensors | Sensor Position | Number of Subject (Male/Female) | Age Range | Trials | Number of Activity Types (ADL/Fall) | Number of Samples (ADL/Fall) |
---|---|---|---|---|---|---|---|
Vilarinho et al. [16] | Accelerometer, Gyroscope and Megnetometer. | Wrist, thigh pocket | 3 | 22–32 | - | 22 (7/15) | 117 (45/72) |
T-fall [17] | Accelerometer | Thigh pocket | 10 (7/3) | 20–42 | 3 | - | 10,909 (9883/1026) |
Mobi-fall [18] | Accelerometer, Gyroscope | Thigh pocket | 24 (17/7) | 22–47 | 1/3/6 | 13 (9/4) | 630 (342/288) |
Uma-fall [19] | Accelerometer, Gyroscope and Magnetometer | Waist, wrist, ankle and chest. | 17 (11/6) | 18–55 | 3 | 11 (8/3) | 561 (332/209) |
Umi-shar [20] | Accelerometer | Thigh pocket | 30 (6/24) | 18–60 | - | 17 (9/8) | 11,771 (7579/4192) |
Sis-fall [21] | Accelerometer and Gyroscope | waist | 38: 15 elderly, 23 young adults | 18–75 | 1/5 | 34 (19/15) | 4505 (2207/1798) |
Head Model | Ensemble Model | Sensitivity | ||||
---|---|---|---|---|---|---|
No. of skipped Convolutional Blocks | Width | No. of Recurrent layers | Width | Non-Fall | Pre-Fall | Fall |
2 | (16,16,16) | 1 | (16) | 0.89 | 0.87 | 0.94 |
2 | (16,16,32) | 1 | (32) | 0.88 | 0.85 | 0.91 |
2 | (16,16,16) | 2 | (16,16) | 0.90 | 0.88 | 0.94 |
2 | (16,16,32) | 2 | (32,64) | 0.88 | 0.87 | 0.91 |
2 | (16,16,16) | 3 | (32,32,64) | 0.88 | 0.88 | 0.90 |
2 | (16,32,64) | 3 | (64,64,128) | 0.87 | 0.86 | 0.89 |
2 | (16,16.16) | 4 | (16,16,32,32) | 0.89 | 0.88 | 0.90 |
2 | (16,16,32) | 4 | (32,32,64,64) | 0.89 | 0.82 | 0.88 |
3 | (16,16,16) | 2 | (16,16) | 0.91 | 0.89 | 0.96 |
3 | (16,32,32) | 2 | (32,32) | 0.90 | 0.87 | 0.94 |
3 | (16,32,64) | 3 | (64,64,128) | 0.90 | 0.88 | 0.94 |
3 | (16,16,16) | 3 | (16,16,16) | 0.91 | 0.89 | 0.97 |
3 | (16,16,32) | 4 | (32,32,64,64) | 0.91 | 0.87 | 0.94 |
3 | (16,32,64) | 4 | (64,64,128,128) | 0.90 | 0.88 | 0.93 |
4 | (16,16,16) | 2 | (16,16) | 0.92 | 0.90 | 0.94 |
4 | (16,16,32) | 2 | (32,64) | 0.90 | 0.89 | 0.92 |
4 | (16,32,64) | 3 | (64,64,128) | 0.89 | 0.87 | 0.89 |
4 | (16,16,32) | 3 | (32,32,64) | 0.90 | 0.88 | 0.90 |
4 | (16,16,16) | 4 | (16,16,16,16) | 0.92 | 0.90 | 0.95 |
4 | (16,32,64) | 4 | (64,64,128,128) | 0.89 | 0.88 | 0.91 |
Head Model | Ensemble Model | Sensitivity | |||||
---|---|---|---|---|---|---|---|
Convolutions | Pooling | Activations | Recurrent layers | Dropout Rate | Non-Fall | Pre-Fall | Fall |
Conv | Max | ReLU | LSTM | 0.5 | 0.91 | 0.89 | 0.97 |
Conv | Max | ReLU | GRU | 0.25 | 0.90 | 0.89 | 0.93 |
Conv | Max | ReLU | Bi-directional | 0.8 | 0.91 | 0.89 | 0.94 |
Conv | Max | Swish | LSTM | 0.8 | 0.92 | 0.87 | 0.96 |
Conv | Max | Swish | GRU | 0.5 | 0.90 | 0.88 | 0.94 |
Conv | Max | Swish | Bi-directional | 0.25 | 0.93 | 0.89 | 0.97 |
Conv | Average | ReLU | LSTM | 0.5 | 0.89 | 0.87 | 0.93 |
Conv | Average | ReLU | GRU | 0.25 | 0.88 | 0.84 | 0.89 |
Conv | Average | ReLU | Bi-directional | 0.8 | 0.90 | 0.83 | 0.85 |
Conv | Average | Swish | LSTM | 0.25 | 0.90 | 0.90 | 0.90 |
Conv | Average | Swish | GRU | 0.5 | 0.88 | 0.86 | 0.90 |
Conv | Average | Swish | Bi-directional | 0.8 | 0.90 | 0.89 | 0.91 |
SeparableConv | Max | ReLU | LSTM | 0.8 | 0.91 | 0.89 | 0.96 |
SeparableConv | Max | ReLU | GRU | 0.25 | 0.90 | 0.88 | 0.95 |
SeparableConv | Max | ReLU | Bi-directional | 0.5 | 0.92 | 0.90 | 0.96 |
SeparableConv | Max | Swish | LSTM | 0.5 | 0.96 | 0.94 | 0.98 |
SeparableConv | Max | Swish | GRU | 0.25 | 0.92 | 0.88 | 0.95 |
SeparableConv | Max | Swish | Bi-directional | 0.8 | 0.95 | 0.91 | 0.98 |
SeparableConv | Average | ReLU | LSTM | 0.5 | 0.89 | 0.85 | 0.91 |
SeparableConv | Average | ReLU | GRU | 0.25 | 0.87 | 0.80 | 0.90 |
SeparableConv | Average | ReLU | Bi-directional | 0.5 | 0.89 | 0.85 | 0.91 |
SeparableConv | Average | Swish | LSTM | 0.8 | 0.88 | 0.86 | 0.92 |
SeparableConv | Average | Swish | GRU | 0.25 | 0.87 | 0.83 | 0.87 |
SeparableConv | Average | Swish | Bi-directional | 0.5 | 0.89 | 0.86 | 0.90 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Mohammad, Z.; Anwary, A.R.; Mridha, M.F.; Shovon, M.S.H.; Vassallo, M. An Enhanced Ensemble Deep Neural Network Approach for Elderly Fall Detection System Based on Wearable Sensors. Sensors 2023, 23, 4774. https://doi.org/10.3390/s23104774
Mohammad Z, Anwary AR, Mridha MF, Shovon MSH, Vassallo M. An Enhanced Ensemble Deep Neural Network Approach for Elderly Fall Detection System Based on Wearable Sensors. Sensors. 2023; 23(10):4774. https://doi.org/10.3390/s23104774
Chicago/Turabian StyleMohammad, Zabir, Arif Reza Anwary, Muhammad Firoz Mridha, Md Sakib Hossain Shovon, and Michael Vassallo. 2023. "An Enhanced Ensemble Deep Neural Network Approach for Elderly Fall Detection System Based on Wearable Sensors" Sensors 23, no. 10: 4774. https://doi.org/10.3390/s23104774