A Data Augmentation Approach to Distracted Driving Detection
Abstract
:1. Introduction
2. Materials and Methods
2.1. Driving Operation Area
2.2. Methods for Data Augmentation
2.3. Methods for CNN Classification
2.4. Wide-Angle Dataset
3. Results
3.1. Results for Driving Operation Area Extraction
3.2. Results for CNN Classification
3.3. Tests on Wide-Angle Dataset
4. Discussion
5. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- World Health Organization. Global Status Report on Road Safety 2018: Summary; World Health Organization: Geneva, Switzerland, 2018. [Google Scholar]
- Peden, M. Global collaboration on road traffic injury prevention. Int. J. Inj. Control Saf. Promot. 2005, 12, 85–91. [Google Scholar] [CrossRef]
- Singh, S. Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey; National Highway Traffic Safety Administration: Washington, DC, USA, 2015. [Google Scholar]
- Vasilash, G.S. Distraction and Risk. Automot. Des. Prod. 2018, 130, 6. [Google Scholar]
- Kaber, D.B.; Liang, Y.; Zhang, Y.; Rogers, M.L.; Gangakhedkar, S. Driver performance effects of simultaneous visual and cognitive distraction and adaptation behavior. Transp. Res. Part F-Traffic Psychol. Behav. 2012, 15, 491–501. [Google Scholar] [CrossRef]
- Strickland, D. How Autonomous Vehicles Will Shape the Future of Surface Transportation. 2013. Available online: https://www.govinfo.gov/content/pkg/CHRG-113hhrg85609/pdf/CHRG-113hhrg85609.pdf (accessed on 21 December 2020).
- Liu, D. Driver status monitoring and early warning system based on multi-sensor fusion. In Proceedings of the 2020 International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS), Vientiane, Laos, 11–12 January 2020. [Google Scholar] [CrossRef]
- Yanfei, L.; Yu, Z.; Junsong, L.; Jing, S.; Feng, F.; Jiangsheng, G. Towards Early Status Warning for Driver’s Fatigue Based on Cognitive Behavior Models. In Proceedings of the Digital Human Modeling and Applications in Health, Safety, Ergonomics, and Risk Management: 4th International Conference, DHM 2013, Held as Part of HCI International 2013, Las Vegas, NV, USA, 21–26 July 2013. [Google Scholar] [CrossRef]
- Liu, X.; Zhu, Y.D.; Fujimura, K. Real-time pose classification for driver monitoring. In Proceedings of the IEEE 5th International Conference on Intelligent Transportation Systems, Singapore, 3–6 September 2002; pp. 174–178. [Google Scholar]
- Eren, H.; Celik, U.; Poyraz, M. Stereo vision and statistical based behaviour prediction of driver. In Proceedings of the 2007 IEEE Intelligent Vehicles Symposium, Istanbul, Turkey, 13–15 June 2007; pp. 657–662. [Google Scholar]
- Zhao, C.H.; Zhang, B.L.; He, J.; Lian, J. Recognition of driving postures by contourlet transform and random forests. IET Intell. Transp. Syst. 2012, 6, 161–168. [Google Scholar] [CrossRef]
- Zhao, C.; Gao, Y.; He, J.; Lian, J. Recognition of driving postures by multiwavelet transform and multilayer perceptron classifier. Eng. Appl. Artif. Intell. 2012, 25, 1677–1686. [Google Scholar] [CrossRef]
- Chihang, Z.; Bailing, Z.; Jie, L.; Jie, H.; Tao, L.; Xiaoxiao, Z. Classification of Driving Postures by Support Vector Machines. In Proceedings of the 2011 Sixth International Conference on Image and Graphics, Hefei, China, 12–15 August 2011; pp. 926–930. [Google Scholar] [CrossRef]
- Zhao, C.H.; Zhang, B.L.; Zhang, X.Z.; Zhao, S.Q.; Li, H.X. Recognition of driving postures by combined features and random subspace ensemble of multilayer perceptron classifiers. Neural Comput. Appl. 2013, 22, S175–S184. [Google Scholar] [CrossRef]
- Yan, C.; Coenen, F.; Zhang, B.L. Driving Posture Recognition by Joint Application of Motion History Image and Pyramid histogram of Oriented Gradients. Int. J. Veh. Technol. 2014, 846–847. [Google Scholar] [CrossRef]
- Yan, C.; Zhang, B.; Coenen, F. Driving Posture Recognition by Convolutional Neural Networks. In Proceedings of the 2015 11th International Conference on Natural Computation (Icnc), Zhangjiajie, China, 15–17 August 2015; pp. 680–685. [Google Scholar]
- Yan, S.; Teng, Y.; Smith, J.S.; Zhang, B. Driver Behavior Recognition Based on Deep Convolutional Neural Networks. In Proceedings of the 2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (Icnc-Fskd), Changsha, China, 13–15 August 2016; pp. 636–641. [Google Scholar]
- Liu, Y.Q.; Zhang, T.; Li, Z. 3DCNN-Based Real-Time Driver Fatigue Behavior Detection in Urban Rail Transit. IEEE Access 2019, 7, 144648–144662. [Google Scholar] [CrossRef]
- Jin, C.C.; Zhu, Z.J.; Bai, Y.Q.; Jiang, G.Y.; He, A.Q. A Deep-Learning-Based Scheme for Detecting Driver Cell-Phone Use. IEEE Access 2020, 8, 18580–18589. [Google Scholar] [CrossRef]
- Hu, Y.C.; Lu, M.Q.; Lu, X.B. Feature refinement for image-based driver action recognition via multi-scale attention convolutional neural network. Signal Process. Image Commun. 2020, 81. [Google Scholar] [CrossRef]
- Kaggle. State Farm Distracted Driver Detection. Available online: https://www.kaggle.com/c/state-farm-distracted-driver-detection/data (accessed on 21 December 2020).
- Alotaibi, M.; Alotaibi, B. Distracted driver classification using deep learning. Signal Image Video Process. 2019. [Google Scholar] [CrossRef]
- Lu, M.Q.; Hu, Y.C.; Lu, X.B. Driver action recognition using deformable and dilated faster R-CNN with optimized region proposals. Appl. Intell. 2020, 50, 1100–1111. [Google Scholar] [CrossRef]
- Valeriano, L.C.; Napoletano, P.; Schettini, R. Recognition of driver distractions using deep learning. In Proceedings of the 2018 IEEE 8th International Conference on Consumer Electronics, Berlin, Germany, 2–5 September 2018. [Google Scholar]
- Moslemi, N.; Azmi, R.; Soryani, M. Driver Distraction Recognition using 3D Convolutional Neural Networks. In Proceedings of the 2019 4th International Conference on Pattern Recognition and Image Analysis, Tehran, Iran, 6–7 March 2019; pp. 145–151. [Google Scholar] [CrossRef]
- Eraqi, H.M.; Abouelnaga, Y.; Saad, M.H.; Moustafa, M.N. Driver Distraction Identification with an Ensemble of Convolutional Neural Networks. J. Adv. Transp. 2019. [Google Scholar] [CrossRef]
- Abouelnaga, Y.; Eraqi, H.M.; Moustafa, M.N. Real-time Distracted Driver Posture Classification. arXiv 2017, arXiv:abs/1706.09498. [Google Scholar]
- Baheti, B.; Gajre, S.; Talbar, S.; IEEE. Detection of Distracted Driver using Convolutional Neural Network. In Proceedings of the 2018 IEEE/Cvf Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, Utah, USA, 18–22 June 2018; pp. 1145–1151. [Google Scholar] [CrossRef]
- Petrovska, B.; Zdravevski, E.; Lameski, P.; Corizzo, R.; Štajduhar, I.; Lerga, J. Deep learning for feature extraction in remote sensing: A case-study of aerial scene classification. Sensors 2020, 20, 3906. [Google Scholar] [CrossRef]
- Petrovska, B.; Atanasova-Pacemska, T.; Corizzo, R.; Mignone, P.; Lameski, P.; Zdravevski, E. Aerial scene classification through fine-tuning with adaptive learning rates and label smoothing. Appl. Sci. 2020, 10, 5792. [Google Scholar] [CrossRef]
- Zhao, Z.; Luo, Z.; Li, J.; Chen, C.; Piao, Y. When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on A Multitask Learning Framework. Remote Sens. 2020, 12, 3276. [Google Scholar] [CrossRef]
- Izadpanahkakhk, M.; Razavi, S.M.; Taghipour-Gorjikolaie, M.; Zahiri, S.H.; Uncini, A. Deep region of interest and feature extraction models for palmprint verification using convolutional neural networks transfer learning. Appl. Sci. 2018, 8, 1210. [Google Scholar] [CrossRef] [Green Version]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 1137–1149. [Google Scholar] [CrossRef] [Green Version]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef]
- Szegedy, C.; Ioffe, S.; Vanhoucke, V.; Alemi, A. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. In Proceedings of the Thirty-First Aaai Conference on Artificial Intelligence, San Francisco, CA, USA, 4–9 February 2017; pp. 4278–4284. [Google Scholar]
- Chollet, F. Xception: Deep Learning with Depthwise Separable Convolutions. In Proceedings of the 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 1800–1807. [Google Scholar]
- Pan, S.J.; Yang, Q. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 2009, 22, 1345–1359. [Google Scholar] [CrossRef]
- Selvaraju, R.R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; Batra, D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization. In Proceedings of the 16th IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 618–626. [Google Scholar]
- Redmon, J.; Farhadi, A. Yolov3: An incremental improvement. arXiv 2018, arXiv:1804.02767. [Google Scholar]
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.-Y.; Berg, A.C. Ssd: Single shot multibox detector. In European Conference on Computer Vision; Springer: Cham, Switzerland, 2016; pp. 21–37. [Google Scholar]
Model | fps | mAP (0.75) | mAR (0.75) |
---|---|---|---|
faster R-CNN | 10.50 | 0.6271 | 0.6572 |
YOLOV3 | 37.21 | 0.5390 | 0.5568 |
SSD | 49.52 | 0.5767 | 0.5812 |
Model | Source | Loss | Top-1 Acc. |
---|---|---|---|
AlexNet | AUC | 0.3753 | 0.9314 |
DOA | 0.3402 | 0.9386 | |
InceptionV4 | AUC | 0.3041 | 0.9506 |
DOA | 0.2771 | 0.9572 | |
Xception | AUC | 0.2320 | 0.9531 |
DOA | 0.2156 | 0.9655 |
Model | Top-1 Acc. |
---|---|
AlexNet | 0.9396 |
InceptionV4 | 0.9603 |
Xception | 0.9697 |
Model | Top-1 Acc. |
---|---|
GA weighted ensemble of all 5 [26] | 0.9598 |
VGG [28] | 0.9444 |
VGG with regularization [28] | 0.9631 |
ResNet + HRNN + modified Inception [22] | 0.9236 |
Our method | 0.9697 |
Model | Source | Top-1 Acc. |
---|---|---|
Xception | Wide-angle Dataset | 0.8131 |
DOA of Wide-angle Dataset | 0.8394 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, J.; Wu, Z.; Li, F.; Zhang, J. A Data Augmentation Approach to Distracted Driving Detection. Future Internet 2021, 13, 1. https://doi.org/10.3390/fi13010001
Wang J, Wu Z, Li F, Zhang J. A Data Augmentation Approach to Distracted Driving Detection. Future Internet. 2021; 13(1):1. https://doi.org/10.3390/fi13010001
Chicago/Turabian StyleWang, Jing, ZhongCheng Wu, Fang Li, and Jun Zhang. 2021. "A Data Augmentation Approach to Distracted Driving Detection" Future Internet 13, no. 1: 1. https://doi.org/10.3390/fi13010001