Trends and Future Prospects of the Drowsiness Detection and Estimation Technology
Abstract
:1. Introduction
- Often occurs between midnight and 6 a.m. or in the late afternoon. During both times, the circadian rhythm, the body’s internal clock that controls sleep, is reduced;
- In many cases, a single driver (without a passenger) has run off the road at high speed with no sign of braking;
- Often occurs on local roads and highways.
- Do not interfere with the driver’s safe driving environment;
- Can be equipped in a vehicle and withstand hard operating environments;
- Can detect driver’s drowsiness in real-time;
- Have a wide detection range from shallow to deep sleep;
- Consider all drivers are detectable targets;
- Have low cost and high scalability than other applications.
2. Purpose of Subjective Evaluation for Drowsiness Detection or Estimation Systems
3. Drowsiness Detection and Estimation Based on Biometric Information
4. Drowsiness Detection and Estimation Based on Vehicle Behavior
5. Drowsiness Detection and Estimation Based on Graphic Information of a Driver
6. Combining Multiple Types of Data
7. Summary of Current Technology Trends
8. Arousal Level Detection and Estimation Technology for Autonomous Driving
9. Summary
Funding
Institutional Review Board Statement
Informed Consent Statement
Conflicts of Interest
References
- NHTSA. Drowsy Driving. Available online: https://www.nhtsa.gov/risky-driving/drowsy-driving (accessed on 17 September 2021).
- Czeisler, C.A.; Wickwire, E.M.; Barger, L.K.; Dement, W.C.; Gamble, K.; Hartenbaum, N.; Ohayon, M.M.; Pelayo, R.; Phillips, B.; Strohl, K.; et al. Sleep-Deprived Motor Vehicle Operators Are Unfit to Drive: A Multidisciplinary Expert Consensus Statement on Drowsy Driving. Sleep Health 2016, 2, 94–99. [Google Scholar] [CrossRef]
- Ngxande, M.; Burke, M. Driver Drowsiness Detection Using Behavioral Measures and Machine Learning Techniques: A Review of State-of-Art Techniques. Pattern Recognit. Assoc. S. Afr. Robot. Mechatron. 2017, 6, 2018. [Google Scholar]
- Bajaj, J.S.; Kumar, N.; Kaushal, R.K. Comparative Study to Detect Driver Drowsiness. In Proceedings of the 2021 International Conference on Advance Computing and Innovative Technologies in Engineering, Greater Noida, India, 4–5 March 2021; pp. 678–683. [Google Scholar]
- National Safety Council. Drivers Are Falling Asleep behind the Wheel. Available online: https://www.nsc.org/road/safety-topics/fatigued-driver (accessed on 17 September 2021).
- Jacobé de Naurois, C.J.; Bourdin, C.; Stratulat, A.; Diaz, E.; Vercher, J.L. Detection and Prediction of Driver Drowsiness Using Artificial Neural Network Models. Accid. Anal. Prev. 2019, 126, 95–104. [Google Scholar] [CrossRef] [PubMed]
- Junaedi, S.; Akbar, H. Driver Drowsiness Detection Based on Face Feature and PERCLOS, International Conference on Computation in Science and Engineering. J. Phys. Conf. Ser. 2018, 1090, 012037. [Google Scholar] [CrossRef]
- Nishiyama, J. Research on the Detection of Signs of Sleepiness in Humans by Pupil Fluctuation and Eye Movement. Ph.D. Thesis, Chubu University, Aichi, Japan, 2011. [Google Scholar]
- Sahayadhas, A.; Sundaraj, K.; Murugappan, M. Detecting Driver Drowsiness Based on Sensors: A Review. Sensors 2012, 12, 16937–16953. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Arefnezhad, S.; Samiee, S.; Eichberger, A.; Nahvi, A. Driver Drowsiness Detection Based on Steering Wheel Data Applying Adaptive Neuro-Fuzzy Feature Selection. Sensors 2019, 19, 943. [Google Scholar] [CrossRef] [Green Version]
- Daza, I.G.; Bergasa, L.M.; Bronte, S.; Yebes, J.J.; Almazán, J.; Arroyo, R. Fusion of Optimized Indicators from Advanced Driver Assistance Systems (ADAS) for Driver Drowsiness Detection. Sensors 2014, 14, 1106–1131. [Google Scholar] [CrossRef]
- SAE International. SAE J3016 Levels of Driving Automation. Available online: https://www.sae.org/news/2019/01/sae-updates-j3016-automated-driving-graphic (accessed on 25 August 2021).
- Synopsys. The 6 Levels of Vehicle Autonomy Explained. Available online: https://www.synopsys.com/automotive/autonomous-driving-levels.html (accessed on 7 October 2021).
- Rather, A.A.; Sofi, T.A.; Mukhtar, N. A Survey on Fatigue and Drowsiness Detection Techniques in Driving. In Proceedings of the 2021 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS), Greater Noida, India, 19–20 February 2021. [Google Scholar]
- Saini, V.; Saini, R. Driver drowsiness detection system and techniques: A review. Int. J. Comput. Sci. Inf. Technol. 2014, 5, 4245–4249. [Google Scholar]
- Ramzan, M.; Khan, H.U.; Awan, S.M.; Ismail, A.; Ilyas, M.; Mahmood, A. A survey on state-of-the-art drowsiness detection techniques. IEEE Access 2019, 7, 61904–61919. [Google Scholar] [CrossRef]
- Shahid, A.; Wilkinson, K.; Marcu, S.; Shapiro, C.M. Karolinska Sleepiness Scale (KSS). In STOP, THAT and One Hundred Other Sleep Scales; Springer: New York, NY, USA, 2011; pp. 209–210. [Google Scholar]
- HFC-Drowsiness Scale®. Driver Drowsiness’ Impairs Safety of Road Traffic. Available online: https://human-factors-consult.de/en/competencies/hfc-drowsiness-scale/ (accessed on 17 September 2021).
- Kitajima, H.; Numata, N.; Yamamoto, K.; Goi, Y. Prediction of Automobile Driver Sleepiness (1st Report, Rating of Sleepiness Based on Facial Expression and Examination of Effective Predictor Indexes of Sleepiness). Jpn. Soc. Mech. Eng. J. 1997, 63, 93–100. [Google Scholar] [CrossRef] [Green Version]
- Hachisuka, S.; Ishida, K.; Enya, T.; Kamijo, M. Engineering Psychology and Cognitive Ergonomics. In Proceedings of the 9th International Conference, EPCE 2011, Held as Part of HCI International 2011, Orlando, FL, USA, 9–14 July 2011. [Google Scholar]
- Nakano, Y.; Miyakawa, A.; Sano, S. Detection of Driver Drowsiness Level; Fujitsu; Fujitsu Ltd.: Tokyo, Japan, 2008; Volume 59, pp. 416–420. (In Japanese) [Google Scholar]
- Alhazmi, S. Towards Context-Based Fatigue Detection System in Vehicular Area Network; University of Ottawa: Ottawa, ON, Canada, 2013. [Google Scholar]
- Lee, B.L.; Lee, B.G.; Chung, W.Y. Standalone Wearable Driver Drowsiness Detection System in a Smartwatch. IEEE Sens. J. 2016, 16, 5444–5451. [Google Scholar] [CrossRef]
- Murata, A.; Naitoh, K. Multinomial Logistic Regression Model for Predicting Driver’s Drowsiness Using Only Behavioral Measures. J. Traffic Transp. Eng. 2015, 3, 80–90. [Google Scholar]
- Baccour, M.H.; Driewer, F.; Schäck, T.; Kasneci, E. Camera-Based Driver Drowsiness State Classification Using Logistic Regression Models. In IEEE International Conference on Systems, Man, and Cybernetics (SMC); IEEE Publications: Toronto, ON, Canada, 2020; pp. 1–8. [Google Scholar]
- Takahashi, I.; Yokoyama, K.; Nokura, K. Arousal Level Preservation System While Driving. Jpn. J. Ergon. 2009, 45, 262–263. (In Japanese) [Google Scholar]
- Arakawa, T. Measurement and Analysis of the Driver’s State to Evaluate Overconfidence and Reliance on Autonomous Driving. J. Jpn. Soc. Fuzzy Theor. Intell. Inform. 2020, 32, 98–104. (In Japanese) [Google Scholar] [CrossRef]
- Kaida, K.; Takahashi, M.; Akerstedt, T.; Nakata, A.; Otsuka, Y.; Haratani, T.; Fukasawa, K. Validation of the Karolinska Sleepiness Scale Against Performance and EEG Variables. Clin. Neurophysiol. 2006, 117, 1574–1581. [Google Scholar] [CrossRef]
- Anund, A.; Fors, C.; Hallvig, D.; Åkerstedt, T.; Kecklund, G. Observer Rated Sleepiness and Real Road Driving: An Explorative Study. PLoS ONE 2013, 8, e64782. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Borghini, G.; Astolfi, L.; Vecchiato, G.; Mattia, D.; Babiloni, F. Measuring Neurophysiological Signals in Aircraft Pilots and Car Drivers for the Assessment of Mental Workload, Fatigue and Drowsiness. Neurosci. Biobehav. Rev. 2014, 44, 58–75. [Google Scholar] [CrossRef]
- Dong, Y.; Hu, Z.; Uchimura, K.; Murayama, N. Driver Inattention Monitoring System for Intelligent Vehicles: A Review. IEEE Trans. Intell. Transport. Syst. 2010, 12, 596–614. [Google Scholar] [CrossRef]
- De Gennaro, L.; Ferrara, M.; Curcio, G.; Cristiani, R. Antero-Posterior EEG Changes During the Wakefulness–Sleep Transition. Clin. Neurophysiol. 2001, 112, 1901–1911. [Google Scholar] [CrossRef]
- Chaabene, S.; Bouaziz, B.; Boudaya, A.; Hökelmann, A.; Ammar, A.; Chaari, L. Convolutional Neural Network for Drowsiness Detection Using EEG Signals. Sensors 2021, 21, 1734. [Google Scholar] [CrossRef] [PubMed]
- Stancin, I.; Frid, N.; Cifrek, M.; Jovic, A. EEG signal multichannel frequency-domain ratio indices for drowsiness detection based on multicriteria optimization. Sensors 2021, 21, 6932. [Google Scholar] [CrossRef] [PubMed]
- Peiris, M.T.; Jones, R.D.; Davidson, P.R.; Carroll, G.J.; Signal, T.L.; Parkin, P.J.; van den Berg, M.; Bones, P.J. Identification of Vigilance Lapses Using EEG/EOG by Expert Human Raters Conf. Proc. IEEE Eng. Med. Biol. Soc. 2005, 2005, 5735–5737. [Google Scholar]
- Healey, J.A.; Picard, R.W. Detecting Stress During Real-World Driving Tasks Using Physiological Sensors. IEEE Trans. Intell. Transp. Syst. 2005, 6, 156–166. [Google Scholar] [CrossRef] [Green Version]
- Rebolledo-Mendez, G.; Reyes, A.; Paszkowicz, S.; Domingo, M.C.; Skrypchuk, L. Developing a Body Sensor Network to Detect Emotions During Driving. IEEE Trans. Intell. Transp. Syst. 2014, 15, 1850–1854. [Google Scholar] [CrossRef]
- Satti, A.T.; Kim, J.; Yi, E.; Cho, H.Y.; Cho, S. Microneedle Array Electrode-Based Wearable EMG System for Detection of Driver Drowsiness Through Steering Wheel Grip. Sensors 2021, 21, 5091. [Google Scholar] [CrossRef]
- Nakagawa, T.; Kawachi, T.; Futatsuyama, K.; Nishii, K. Monitoring the Physical Condition of Drivers as They Drive. Denso Tech. Rev. 2016, 21, 103–108. (In Japanese) [Google Scholar]
- Uchenna, E.N.; Leonard, A.T.; Nnamdi, O.A.; Chiemezie, O.E. Drowsy Driver Detection and Alert System Using Pulse Sensor. IJES 2020, 9, 37–46. [Google Scholar]
- Arakawa, T. A Review of Heartbeat Detection Systems for Automotive Applications. Sensors 2021, 21, 6112. [Google Scholar] [CrossRef]
- Kundinger, T.; Sofra, N.; Riener, A. Assessment of the Potential of Wrist-Worn Wearable Sensors for Driver Drowsiness Detection. Sensors 2020, 20, 1029. [Google Scholar] [CrossRef] [Green Version]
- Kundinger, T.; Riener, A. The Potential of Wrist-Worn Wearables for Driver Drowsiness Detection: A Feasibility Analysis. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization, Online, 12–18 July 2020; pp. 117–125. [Google Scholar]
- Oyama, H.; Arakawa, T. Development of Warning System that Estimates Driver’s Arousal Level Based on Unsteady Driving Phenomenon and Evaluation of Driver’s Condition Based on EEG. J. Soc. Automot. Eng. Jpn. 2004, 58, 89–94. [Google Scholar]
- Hino Motors, Ltd. HINO SELEGA. Available online: https://www.hino.co.jp/selega/safety/index.html (accessed on 17 September 2021).
- Mazda. DAA (Driver Attention Alert). Available online: https://www.mazda.com/en/innovation/technology/safety/active_safety/daa/ (accessed on 17 September 2021).
- Honda. Driver Attention Monitor. Available online: https://www.honda-mideast.com/en/technology/Driver-Attention-Monitor (accessed on 17 September 2021).
- Volvo. Available online: https://www.volvocars.com/en-th/support/manuals/v40/2017w17/driver-support/driver-alert-system/driver-alert-control-dac (accessed on 17 September 2021).
- Jaguar. Jaguar Technology Is a Wake-Up Call for Drivers. Available online: https://media.jaguar.com/news/2019/11/jaguar-technology-wake-call-drivers (accessed on 17 September 2021).
- Jeon, Y.; Kim, B.; Baek, Y. Ensemble CNN to Detect Drowsy Driving with in-Vehicle Sensor Data. Sensors 2021, 21, 2372. [Google Scholar] [CrossRef] [PubMed]
- Toyota. Toyota Enhances Pre-Crash Safety System with Driver-Monitoring Function. Available online: https://global.toyota/en/detail/248128 (accessed on 17 September 2021).
- Aisin, A. Develops New Products for Enabling Advanced Driving Assist Technology—Adopted in Toyota “MIRAI” and LEXUS “LS”. Available online: https://www.aisin.com/en/news/2021/005406.html (accessed on 17 September 2021).
- Subaru. Subaru All-Around Safety. Available online: https://www.subaru-global.com/ebrochure/Forester/2020my/ISEN/safety/index.html (accessed on 17 September 2021).
- Nissan. Nissan Unveils New Skyline with ProPILOT 2.0 Driver Assistance System. Available online: https://global.nissannews.com/en/releases/190716-01-e (accessed on 17 September 2021).
- Thanko. Eye Catcher, a Warning System for Drivers Who Fall Asleep or Look Aside. Available online: https://www.thanko.jp/view/item/000000002960 (accessed on 17 September 2021).
- Yupiteru, E.-C.M. Available online: https://www.yupiteru.co.jp/products/okite/ews-cm1/ (accessed on 17 September 2021).
- Babusiak, B.; Hajducik, A.; Medvecky, S.; Lukac, M.; Klarak, J. Design of Smart Steering Wheel for Unobtrusive Health and Drowsiness Monitoring. Sensors 2021, 21, 5285. [Google Scholar] [CrossRef] [PubMed]
- Dinges, D.F.; Mallis, M.M.; Maislin, G.; Powell, J.W. Evaluation of Techniques for Ocular Measurement as an Index of Fatigue and the Basis for Alertness Management. Available online: https://ntlrepository.blob.core.windows.net/lib/21000/21900/21955/PB99150237.pdf (accessed on 26 November 2021).
- Hanowski, R.J. The Drowsy Driver Warning System Field Operational Test: Data Collection Methods. Available online: https://vtechworks.lib.vt.edu/bitstream/handle/10919/55072/810035.pdf (accessed on 9 October 2021).
- Tanaka, Y. Arousal Level and Blink Activity. Shinrigaku Kenkyu 1999, 70, 1–8. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Cardone, D.; Filippini, C.; Mancini, L.; Pomante, A.; Tritto, M.; Nocco, S.; Perpetuini, D.; Merla, A. Driver Drowsiness Evaluation by Means of Thermal Infrared Imaging: Preliminary Results. In Infrared Sensors, Devices and Applications XI; International Society for Optics and Photonics: Bellingham, DC, USA, 2021; Volume 11831, p. 118310. [Google Scholar]
- Celecia, A.; Figueiredo, K.; Vellasco, M.; González, R. A portable fuzzy driver drowsiness estimation system. Sensors 2020, 20, 4093. [Google Scholar] [CrossRef]
- Tashakori, M.; Nahvi, A.; Ebrahimian Hadi Kiashari, S.S. Driver Drowsiness Detection Using Facial Thermal Imaging in a Driving Simulator. Proc. Inst. Mech. Eng. H 2021. [Google Scholar] [CrossRef] [PubMed]
- Chakkravarthy, P. Smart Monitoring of the Status of Driver Using the Dashboard Vehicle Camera. Int. J. New Pract. Manag. Eng. 2020, 9, 1–7. [Google Scholar] [CrossRef]
- Suhaiman, A.A.; May, Z.; Rahman, N.A.A. Development of an Intelligent Drowsiness Detection System for Drivers Using Image Processing Technique. In Proceedings of the 2020 IEEE Student Conference on Research and Development (SCOReD), Batu Pahat, Malaysia, 27–29 September 2020. [Google Scholar]
- Viola, P.; Jones, M.J. Robust Real-Time Face Detection. Int. J. Comput. Vis. 2004, 57, 137–154. [Google Scholar] [CrossRef]
- Manu, B.N. Facial Features Monitoring for Real Time Drowsiness Detection. In Proceedings of the 2016 12th International Conference on Innovations in Information Technology (IIT), Al Ain, Uniter Arab Emirates, 28–30 November 2016; p. 2016. [Google Scholar]
- Selvakumar, K.; Jerome, J.; Rajamani, K.; Shankar, N. Real-Time Vision Based Driver Drowsiness Detection Using Partial Least Squares Analysis. J. Sign. Process. Syst. 2016, 85, 263–274. [Google Scholar] [CrossRef]
- Li, K.; Gong, Y.; Ren, Z. A Fatigue Driving Detection Algorithm Based on Facial Multi-Feature Fusion. IEEE Access 2020, 8, 101244–101259. [Google Scholar] [CrossRef]
- Képešiová, Z.; Cigánek, J.; Kozák, Š. Driver Drowsiness Detection Using Convolutional Neural Networks. In Cybernetics & Informatics (K&I); IEEE Publications: Piscataway Township, NJ, USA, 2020; pp. 1–6. [Google Scholar]
- Dua, M.; Shakshi; Singla, R.; Raj, S.; Jangra, A. Deep CNN Models-Based Ensemble Approach to Driver Drowsiness Detection. Neural Comput. Appl. 2021, 33, 3155–3168. [Google Scholar] [CrossRef]
- Yang, C.; Wang, X.; Mao, S. Unsupervised Drowsy Driving Detection with RFID. IEEE Trans. Veh. Technol. 2020, 69, 8151–8163. [Google Scholar] [CrossRef]
- Fatima, A.; Gowda, A.K.; Bhavitha, Y.; Chandana, C.U.; Tauseef, M. Drowsiness Detection System Using Machine Learning. New Arch. Int. J. Contemp. Archit. 2021, 8, 246–252. [Google Scholar]
- Jabbar, R.; Al-Khalifa, K.; Kharbeche, M.; Alhajyaseen, W.; Jafari, M.; Jiang, S. Real-time Driver Drowsiness Detection for Android Application Using Deep Neural Networks Techniques. In Proceedings of the 9th International Conference on Ambient Systems, Networks and Technologies, Porto, Potugal, 8–11 May 2018. [Google Scholar]
- Ma, Y.; Zhang, S.; Qi, D.; Luo, Z.; Li, R.; Potter, T.; Zhang, Y. Driving Drowsiness Detection with EEG Using a Modified Hierarchical Extreme Learning Machine Algorithm with Particle Swarm Optimization: A Pilot Study. Electronics 2020, 9, 775. [Google Scholar] [CrossRef]
- Ariizumi, R.; Kawaguchi, M.; Arakawa, T.; Ooue, N.; Murayama, M. Drowsiness Estimation using Echo State Network. In Proceedings of the 2019 JSAE Annual Congress (Autumn), Yokohama, Japan, 22–24 May 2019. [Google Scholar]
- Connected Automated driving.eu. Are You Willing to Sleep in Your Self Driving Car? Available online: https://www.connectedautomateddriving.eu/blog/are-you-willing-to-sleep-in-your-self-driving-car/ (accessed on 18 September 2021).
- Taoka, T.; Nagata, M.; Motomura, H. Development of Interior Space That Improves Performance by Power Nap During Autonomous Driving. In Proceedings of the of Autumn Congress 2021, Society of Automotive Engineers of Japan, Online, 13–15 October 2021; pp. 1–5. [Google Scholar]
- Shirakawa, S. Sleep-Wake Rhythm in Human and Light. J. Illum. Eng. Inst. Jpn. 2000, 84, 354–361. (In Japanese) [Google Scholar] [CrossRef] [Green Version]
- Hiramatsu, M.; Kasai, J.; Taguchi, M. A Study of Effects of Odor on Relieving Drowsiness. Trans. Soc. Automot. Eng. Jpn. 1995, 26, 88–93. [Google Scholar]
- Soga, J.; Matsunaga, T.; Kum, K.C.; Soma, Y.; Uchiyama, H.; Fukumoto, I. Basic Research of Awakening in Comfort. In Proceedings of the Annual Meeting of Japan Ergonomics Society, 1F3-2, Funabashi, Chiba, 16–17 December 2017; Volume 2017. [Google Scholar]
Methods | Measurement Information | Measurement Target | Measurement Method | Measurement Index | Advantages | Disadvantages |
---|---|---|---|---|---|---|
Contact method | Biometric information | Heartbeat, pulse wave, aspiration, brain wave, myoelectric, eye movement, etc. | Heart rate monitor, pulse wavemeter, electroencephalograph, electromyograph, nystagmus, etc. | Heart rate, chaos analysis, alpha wave, theta wave, muscle action potential, vestibular oculomotor reflex, etc. | High drowsiness detection performance can be obtained [10]. | Driver behavior adversely affects the reliability of the designed system [10]. |
Non-contact method | Vehicle behavior | Steering pattern, distance between lane and vehicle, speed, distance between vehicles, etc. | Steering angle sensor, white line recognition camera, laser radar, etc. | Steering frequency, meandering rate, steering volume, monotonous steering | Estimates can be obtained in a way that is unobtrusive to the driver [11]. | The accuracy of detection and estimation depends on road conditions and the environment. It is useful only when the driver is holding the steering wheel. |
Driver’s graphic information | Open rate of eyes, blink, pupil, voice, and expression | Camera, microphone | Opening and closing rates of eyes, number of blinks, time of closing eyes, pupil fluctuation, chaos analysis of speech sound, drowsy categorization by expression, etc. | Intuitional index and easy-to-understand, high accuracy. | Different lighting conditions may disrupt the detection performance [10]. |
HFC | Description |
---|---|
1 | Wide awake, vivid attention |
2 | Highly concentrated, focused attention |
3 | Attentive but calm |
4 | No activation, no drowsiness, no pronounced tendency for reactive behavior |
5 | Slightly dozing, ready to respond |
6 | Signs of drowsiness but effortlessly awake |
7 | Obvious drowsiness, but mainly focused on driving tasks |
8 | Battling with drowsiness. Difficulty with driving tasks, but mainly perceptual |
9 | Feeling foggy, listless, inactive for long periods of time, microsleep is occurring or may be occurring |
D-ORS0 (Alert) | B-ORS0 (Alert) |
---|---|
Awareness: driver’s reactions are high and fast Driving: normal | Blink: normal Yawning: no Body position: sitting still Body movements: hardly |
D-ORS1 (First signs of sleepiness) | B-ORS1 (First signs of sleepiness) |
Awareness: driver’s reactions are relatively normal and fast Driving: light steering wheel operation | Blink: sporadic prolonged closure of the eyelids, followed by increased blinking frequency Yawning: occasionally Body position: sometimes change position Body movements: sometimes |
D-ORS2 (Severe sleepiness) | B-ORS2 (Severe sleepiness (microsleep)) |
Awareness: driver reacts slowly Driving: cannot drive steadily and turns the steering wheel too far | Blink: driver’s eyes are half-closed, and his/her gaze vacant Yawning: frequently Body position: frequently change Body movement: frequently |
Level | Phenomenon |
---|---|
1 | Do not look sleepy at all; gaze moves quickly and frequently, blink at a constant rate of about 2 times every 2 s, and body movements are active. |
2 | Slightly sleepy, open lips, slow eye movement. |
3 | Looks somewhat sleepy, blinks slowly and frequently, mouth moves, sits up straight, and puts hands on face. |
4 | Looks quite sleepy and blinks as if conscious. Unnecessary movements of the entire body, such as shaking the head or moving the shoulders up and down. Frequent yawning and deep breathing. Slow blinking or eye movements. |
5 | Looks very sleepy, eyelids closed, head tilted forward or back. |
Methods | Measurement Information | Previous Studies | Method | Accuracy |
---|---|---|---|---|
Contact method | Biometric information | Satti et al. [38] | Electromyogram measurement from electrodes attached to the steering wheel Electrocardiographic measurements from a wearable sensor on the wrist. | NA |
Kundinger et al. [42] | ≧92% | |||
Kundinger et al. [43] | ≧90% | |||
Non-contact method | Vehicle behavior | Subaru [44] Hino [45] Mazda [46] Honda [47] Volvo [48] Jaguar [49] | Detects changes in vehicle behavior and warns from HMI. | NA |
Arefnezhad et al. [10] | Apply ANFIS with steering angle as input. | 98.12% | ||
Jeon et al. [50] | Estimation by ensemble network model using steering and pedal pressure as input. | 94.2% | ||
Graphic information (of driver) | Toyota [51] Subaru [53] Nissan [54] Hino [45] Thanko [55] Yupiteru [56] | Warnings for closed eyes and side glances. | NA | |
Toyota [52] | Stops the car when the driver is not in a good posture or does not respond to warnings. | |||
Cardone et al. [61] | Applied PERCLOS to visible images obtained by a thermal imaging camera and classified “wakefulness”, “fatigue”, and “dozing” by deep learning. Support vector machine, K-nearest neighbor method, and decision tree were used to classify sleepiness based on the temperature patterns of the forehead and cheeks. | Approximately 65% | ||
Tashakori et al. [63] | 84% | |||
Non-contact method | Graphic information (of driver) | Celecia et al. [62] | Fuzzy inference system to estimate sleepiness from eye and mouth information. | 95.5% |
Chakkravarthy [64] | EAR | 75% when blinking, 35% when wearing glasses, and 25% when hair is hanging over the face | ||
Manu [67] | Correlation coefficient template matching. | 94.58% | ||
Li et al. [69] | Detecting fatigue from driver’s eye closure time, few blinks, and few yawns. | 95.10% | ||
Képešiová et al. [70] | Learning grayscale face images with CNN. | 98.02% | ||
Dua et al. [71] | Detects drowsiness by considering four different types of features (hand gestures, facial expressions, behavioral features, and head movements) using four deep learning models: AlexNet, VGG-FaceNet, FlowImageNet, and ResNet. | 85% | ||
Yang et al. [72] | Nodding detection using LSTM autoencoder on RFID tag data. | ≧90% | ||
Jabber et al. [74] | Facial landmarks from images were detected and estimated by a system based on multilayers perception classifiers. | 81% | ||
Ma et al. [75] | Classified the driver’s drowsiness by PSO-H-ELM based on the power spectrum density of EEG data. | 83.12% | ||
Multiple methods | de Naurois et al. [6] | Modeled using the information on eyelid closure, eye and head movements, and driving time. Logistic regression with Eye Closure, head movement, KSS, HFC, etc., as explanatory variables. | MSE of drowsiness level: 0.22 | |
Baccour et al. [27] | Pulse, respiration, and center of gravity information were obtained, and ESN was used for estimation. | 72.7% | ||
Ariizumi et al. [76] | 83.3% |
SAE’s Autonomous Driving Level | Purpose of the Technology |
---|---|
0, 1, 2 and 3 | Detect and estimate the driver’s drowsiness and notify the driver of the result to prevent human error caused by drowsiness. |
4 and 5 | Detect and estimate the driver’s drowsiness and makes the driver sleep so that they can comfortably reach the destination. |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Arakawa, T. Trends and Future Prospects of the Drowsiness Detection and Estimation Technology. Sensors 2021, 21, 7921. https://doi.org/10.3390/s21237921
Arakawa T. Trends and Future Prospects of the Drowsiness Detection and Estimation Technology. Sensors. 2021; 21(23):7921. https://doi.org/10.3390/s21237921
Chicago/Turabian StyleArakawa, Toshiya. 2021. "Trends and Future Prospects of the Drowsiness Detection and Estimation Technology" Sensors 21, no. 23: 7921. https://doi.org/10.3390/s21237921
APA StyleArakawa, T. (2021). Trends and Future Prospects of the Drowsiness Detection and Estimation Technology. Sensors, 21(23), 7921. https://doi.org/10.3390/s21237921