Next Article in Journal
Identity-as-a-Service: An Adaptive Security Infrastructure and Privacy-Preserving User Identity for the Cloud Environment
Previous Article in Journal
Word Sense Disambiguation Using Cosine Similarity Collaborates with Word2vec and WordNet
Previous Article in Special Issue
Combining Facial Expressions and Electroencephalography to Enhance Emotion Recognition
Article Menu
Issue 5 (May) cover image

Export Article

Open AccessArticle

Convolutional Two-Stream Network Using Multi-Facial Feature Fusion for Driver Fatigue Detection

School of Software, South China Normal University, Guangzhou 510641, China
*
Author to whom correspondence should be addressed.
Future Internet 2019, 11(5), 115; https://doi.org/10.3390/fi11050115
Received: 23 February 2019 / Revised: 19 April 2019 / Accepted: 29 April 2019 / Published: 14 May 2019
(This article belongs to the Special Issue Special Issue on the Future of Intelligent Human-Computer Interface)
  |  
PDF [3044 KB, uploaded 14 May 2019]
  |     |  

Abstract

Road traffic accidents caused by fatigue driving are common causes of human casualties. In this paper, we present a driver fatigue detection algorithm using two-stream network models with multi-facial features. The algorithm consists of four parts: (1) Positioning mouth and eye with multi-task cascaded convolutional neural networks (MTCNNs). (2) Extracting the static features from a partial facial image. (3) Extracting the dynamic features from a partial facial optical flow. (4) Combining both static and dynamic features using a two-stream neural network to make the classification. The main contribution of this paper is the combination of a two-stream network and multi-facial features for driver fatigue detection. Two-stream networks can combine static and dynamic image information, while partial facial images as network inputs can focus on fatigue-related information, which brings better performance. Moreover, we applied gamma correction to enhance image contrast, which can help our method achieve better results, noted by an increased accuracy of 2% in night environments. Finally, an accuracy of 97.06% was achieved on the National Tsing Hua University Driver Drowsiness Detection (NTHU-DDD) dataset. View Full-Text
Keywords: fatigue detection; multi-task cascaded convolutional networks; optical flow; gamma correction; feature fusion fatigue detection; multi-task cascaded convolutional networks; optical flow; gamma correction; feature fusion
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Liu, W.; Qian, J.; Yao, Z.; Jiao, X.; Pan, J. Convolutional Two-Stream Network Using Multi-Facial Feature Fusion for Driver Fatigue Detection. Future Internet 2019, 11, 115.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Future Internet EISSN 1999-5903 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top