# Research on Driver Status Recognition System of Intelligent Vehicle Terminal Based on Deep Learning

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Research Methods

#### 2.1.1. Convolutional Neural Network

#### 2.1.2. 3D Convolutional Neural Network

#### 2.1.3. YOLOv3

#### 2.1.4. Advantages of YOLOv3

#### 2.2. Experiments

#### 2.2.1. Experimental Environment Construction

#### 2.2.2. Data Set Production

## 3. Results

## 4. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Hu, X.H.; Zheng, M.T.Y. Research Progress and Prospects of Vehicle Driving Behavior Prediction. World Electr. Veh. J.
**2021**, 12, 88. [Google Scholar] [CrossRef] - Zhai, W.; Zhang, X.; Hou, H.; Meng, Q. Olfactory electroencephalogram signal recognitionbased on wavelet energy moment. Shengwu Yixue Gongchengxue Zazhi
**2020**, 3, 399–404. [Google Scholar] - Zhai, X.L.; Tin, C. Automated ECG Classification Using Dual Heartbeat Coupling Based on Convolutional Neural Network. IEEE Access
**2018**, 6, 27465–27472. [Google Scholar] [CrossRef] - Kaur, A. Wheelchair control for disabled patients using EMG/EOG based human machine interface: A review. IEEE Access
**2020**, 6, 1–22. [Google Scholar] - Deo, R.C. Machine Learning in Medicine. Sensors
**2015**, 132, 1920–1930. [Google Scholar] - Godara, M.; Sanchez-Lopez, A.; De Raedt, R. Manipulating avoidance motivation to modulate attention bias for negative information in dysphoria: An eye-tracking study. J. Behav. Ther. Exp. Psychiatry
**2021**, 3, 101613. [Google Scholar] [CrossRef] [PubMed] - Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. Commun. ACM
**2017**, 3, 84–90. [Google Scholar] [CrossRef] - Chiroma, H.; Gital, A.Y.; Rana, N.; Abdulhammid, S.M.; Muhammad, A.N.; Umar, A.Y.; Abubakar, A.I. Nature Inspired Meta-heuristic Algorithms for Deep Learning: Recent Progress and Novel Perspective. Adv. Comput. Vis.
**2019**, 943, 59–70. [Google Scholar] - Karpathy, A.; Toderici, G.; Shetty, S.; Leung, T.; Sukthankar, R.; Li, F.-F. Large-scale Video Classification with Convolutional Neural Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 23–28 June 2014; pp. 1725–1732. [Google Scholar]
- Ren, S.Q.; He, K.M.; Girshick, R.; Sun, J.L. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Neural Inf. Process. Syst.
**2015**, 28, 91–99. [Google Scholar] [CrossRef] [Green Version] - Alcaraz, J.C.; Moghaddamnia, S.; Peissig, J. Efficiency of deep neural networks for joint angle modeling in digital gait assessment. Eurasip J. Adv. Signal Process.
**2021**, 1, 10. [Google Scholar] [CrossRef] - Yu, H.F.; Sun, Y.; Lu, D.; Cai, K.; Yu, X. Discrimination of lung cancer and adjacent normal tissues based on permittivity by optimized probabilistic neural network. Nan Fang Yi Ke Da Xue Xue Bao = J. South. Med. Univ.
**2020**, 59, 370–378. [Google Scholar] - Bin, J.; Mei-xia, Q.; Qing-wei, L.; Shu-hua, C.; Yun-peng, Z. Study on Spectra Decomposition of White Dwarf Main-Sequence Binary Stars Based on Generation Antagonistic Network. Spectrosc. Spectr. Anal.
**2020**, 40, 3298–3302. [Google Scholar] - Alharthi, A.S.; Yunas, S.U.; Ozanyan, K.B. Deep Learning for Monitoring of Human Gait: A Review. IEEE Sens. J.
**2019**, 19, 9575–9591. [Google Scholar] [CrossRef] [Green Version] - Zeng, Y.; Dai, H.H. CT Image Segmentation of Liver Tumor with Deep Convolutional Neural Network. J. Med. Imaging Health Inform.
**2021**, 11, 337–344. [Google Scholar] [CrossRef] - Mu, G.R.; Yang, Y.P.; Gao, Y.Z.; Feng, Q. Multi-scale 3D convolutional neural network-based segmentation of head and neck organs at risk. Nan Fang Yi Ke Da Xue Xue Bao = J. South Med. Univ.
**2020**, 40, 491–498. [Google Scholar] - Li, X.K.; Liu, X.Y.; Li, Y.M.; Cao, H.; Chen, Y.; Lin, Y.; Huang, X. Human activity recognition based on the inertial information and convolutional neural network. Sheng Wu Yi Xue Gong Cheng Xue Za Zhi = J. Biomed. Eng.
**2020**, 37, 596–601. [Google Scholar] - Li, Q.; Chen, Z.; He, J.J.; Hao, S.Y.; Wang, R.; Yang, H.T.; Sun, H.J. Reconstructing the 3D digital core with a fully convolutional neural network. Appl. Geophys.
**2020**, 17, 401–410. [Google Scholar] [CrossRef] - Mousavi, S.A.S.; Zhang, X.; Seigler, T.M.; Hoagg, J.B. Characteristics That Make Linear Time-Invariant Dynamic Systems Difficult for Humans to Control. IEEE Trans. Human-Mach. Syst.
**2021**, 51, 141–151. [Google Scholar] [CrossRef] - Dian-Wei, W.; Peng-Fei, H.; Jiu-Lun, F.; Ying, L.; Zhi-Jie, X.; Jing, W. Multispectral image enhancement based on illuminance-reflection imaging model and morphology operation. Acta Phys. Sin.
**2018**, 67, 21. [Google Scholar] - Wu, Z.; Zeng, J.X. Aircraft target recognition in remote sensing image based on saliency map and multi feature. Chin. J. Image Graph.
**2017**, 22, 532–541. [Google Scholar] - Hong, L.; Wei, W.; Xiao-Min, Y.; Bin-Yu, Y.; Kai, L.; Jeon, G. Multispectral image enhancement based on Retinex by using structure extraction. Chin. J. Image Graph.
**2016**, 65, 16. [Google Scholar] - Ayachi, R.; Said, Y.; Abdelaali, A.B. Pedestrian Detection Based on Light-Weighted Separable Convolution for Advanced Driver Assistance Systems. Neural Process. Lett.
**2020**, 52, 2655–2668. [Google Scholar] [CrossRef] - Jeong, Y.; Lee, B.; Han, J.H.; Oh, J. Ocular Axial Length Prediction Based on Visual Interpretation of Retinal Fundus Images via Deep Neural Network. IEEE J. Sel. Top. Quantum Electron.
**2021**, 27, 7200407. [Google Scholar] [CrossRef] - Hu, S.M.; Liang, D.; Yang, G.Y.; Yang, G.W.; Zhou, W.Y. Jittor: A novel deep learning framework with meta-operators and unified graph execution. Sci. China Inf. Sci.
**2020**, 63, 118–138. [Google Scholar] [CrossRef] - Ketkar, N. Introduction to PyTorch. In Deep Learning with Python; Apress: Berkeley, CA, USA, 2017. [Google Scholar]
- Ou, X.F.; Yan, P.C.; Wang, H.P.; Xu, B.; He, W.; Zhang, G.Y.; Xu, Z. Research on Moving Target Detection Method Based on Deep Frame Difference Convolutional Neural Network. Chin. J. Electron.
**2020**, 48, 2384–2393. [Google Scholar] - Yuan, C.X.; Jia, D.N.; Zhou, S.H. Research and application of convolutional neural network in mining area prediction. J. Eng. Sci.
**2020**, 42, 1597–1604. [Google Scholar] - Li, X.; Jiang, Y.; Li, M.; Yin, S. Lightweight Attention Convolutional Neural Network for Retinal Vessel Image Segmentation. IEEE Trans. Ind. Inform.
**2021**, 17, 1958–1967. [Google Scholar] [CrossRef] - He, Z.Y.; Wang, Y.; Zhang, P.H.; Zuo, K.; Liang, P.F.; Zeng, J.Z.; Zhou, S.T.; Guo, L.; Huang, M.T.; Cui, X. Establishment and test results of an artificial intelligence burn depth recognition model based on convolutional neural network. Zhonghua shao shang za zhi/Chin. J. Burn.
**2020**, 3, 399–404. [Google Scholar] - Firino, C.; Zhang, C.; Patras, P.; Banchs, A.; Widmer, J. A Machine-Learning-Based Framework for Optimizing the Operation of Future Networks. IEEE Commun. Mag.
**2020**, 58, 20–25. [Google Scholar] [CrossRef] - Tang, Y.; Teng, Q.; Zhang, L.; Min, F.; He, J. Layer-Wise Training Convolutional Neural Networks With Smaller Filters for Human Activity Recognition Using Wearable Sensors. IEEE Sens. J.
**2021**, 21, 581–592. [Google Scholar] [CrossRef] - Li, X.B.; Wang, W.Q. Learning discriminative features via weights-biased softmax loss. Pattern Recognit.
**2020**, 107, 107405. [Google Scholar] [CrossRef] - Redmon, J.; Farhadi, A. YOLO9000: Better, Faster, Stronger. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 6517–6525. [Google Scholar]
- Canepa, A.; Ragusa, E.; Zunino, R.; Gastaldo, P. T-RexNet-A Hardware-Aware Neural Network for Real-Time Detection of Small Moving Objects. Sensors
**2021**, 21, 1252. [Google Scholar] [CrossRef] - Cass, S. Nvidia makes it easy to embed AI: The Jetson nano packs a lot of machine-learning power into DIY projects- Hands on. IEEE Spectr.
**2020**, 57, 14–16. [Google Scholar] [CrossRef]

**Figure 2.**Schematic diagram of 2D and 3D convolutional neural network. (

**a**) Applying 2D convolution on an image results in an image; (

**b**) Applying 2D convolution on a video volume(multiple frames as multiple channels) also results in an image; (

**c**) Applying 3D convolution on a video volume results in another volume, preserving temporal information of the input signal.

**Figure 3.**C3D convolutional neural network structure. This architecture consists of 1 hardwired layer, 3 convolution layers, 2 subsampling layers, and 1 full connection layer. Among them, H is hardwired layer, C is convolution layer, S is subsampling layer, $7@60\times 40$ represents 7 continuous frames of $60\times 40$.

**Figure 10.**Forecast results. The recognition predicted results of the smoking are shown in subplot (

**a**–

**c**). Then, we can see the predicted result of phone from subplot (

**d**–

**f**). Finally, subplot (

**g**–

**i**) show the results of the identification for drinking.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Xu, Y.; Peng, W.; Wang, L.
Research on Driver Status Recognition System of Intelligent Vehicle Terminal Based on Deep Learning. *World Electr. Veh. J.* **2021**, *12*, 137.
https://doi.org/10.3390/wevj12030137

**AMA Style**

Xu Y, Peng W, Wang L.
Research on Driver Status Recognition System of Intelligent Vehicle Terminal Based on Deep Learning. *World Electric Vehicle Journal*. 2021; 12(3):137.
https://doi.org/10.3390/wevj12030137

**Chicago/Turabian Style**

Xu, Yiming, Wei Peng, and Li Wang.
2021. "Research on Driver Status Recognition System of Intelligent Vehicle Terminal Based on Deep Learning" *World Electric Vehicle Journal* 12, no. 3: 137.
https://doi.org/10.3390/wevj12030137