Search Results (9)

Car accidents remain a leading cause of death worldwide, with drowsiness and distraction accounting for roughly 25% of fatal crashes in Ecuador. This study presents a real-time driver monitoring system that uses computer vision and deep learning to detect fatigue, distraction, and emotions from facial expressions. It combines a MobileNetV2-based CNN trained on RAF-DB for emotion recognition and MediaPipe’s 468 facial landmarks to compute the EAR (Eye Aspect Ratio), the MAR (Mouth Aspect Ratio), the gaze, and the head pose. Tests with 27 participants in both real and simulated driving environments showed strong results. There was a 100% accuracy in detecting distraction, 85.19% for yawning, and 88.89% for eye closure. The system also effectively recognized happiness (100%) and anger/disgust (96.3%). However, it struggled with sadness and failed to detect fear, likely due to the subtlety of real-world expressions and limitations in the training dataset. Despite these challenges, the results highlight the importance of integrating emotional awareness into driver monitoring systems, which helps reduce false alarms and improve response accuracy. This work supports the development of lightweight, non-invasive technologies that enhance driving safety through intelligent behavior analysis. Full article

Driver fatigue is a primary factor in traffic accidents and poses a serious threat to road safety. To address this issue, this paper proposes a multi-feature fusion fatigue detection method based on an improved YOLOv8 model. First, the method uses an enhanced YOLOv8 model to achieve high-precision face detection. Then, it crops the detected face regions. Next, the lightweight PFLD (Practical Facial Landmark Detector) model performs keypoint detection on the cropped images, extracting 68 facial feature points and calculating key indicators related to fatigue status. These indicators include the eye aspect ratio (EAR), eyelid closure percentage (PERCLOS), mouth aspect ratio (MAR), and head posture ratio (HPR). To mitigate the impact of individual differences on detection accuracy, the paper introduces a novel sliding window model that combines a dynamic threshold adjustment strategy with an exponential weighted moving average (EWMA) algorithm. Based on this framework, blink frequency (BF), yawn frequency (YF), and nod frequency (NF) are calculated to extract time-series behavioral features related to fatigue. Finally, the driver’s fatigue state is determined using a comprehensive fatigue assessment algorithm. Experimental results on the WIDER FACE and YAWDD datasets demonstrate this method’s significant advantages in improving detection accuracy and computational efficiency. By striking a better balance between real-time performance and accuracy, the proposed method shows promise for real-world driving applications. Full article

We developed a multi-feature drowsiness detection model employing eye aspect ratio (EAR), mouth aspect ratio (MAR), head pose angles (yaw, pitch, and roll), and a Raspberry Pi 4 for real-time applications. The model was trained on the NTHU-DDD dataset and optimized using long short-term memory (LSTM) deep learning algorithms implemented using TensorFlow version 2.14.0. The model enabled robust drowsiness detection at a rate of 10 frames per second (FPS). The system embedded with the model was constructed for live image capture. The camera placement was adjusted for optimal positioning in the system. Various features were determined under diverse conditions (day, night, and with and without glasses). After training, the model showed an accuracy of 95.23%, while the accuracy ranged from 91.81 to 95.82% in validation. In stationary and moving vehicles, the detection accuracy ranged between 51.85 and 85.71%. Single-feature configurations exhibited an accuracy of 51.85 to 72.22%, while in dual features, the accuracy ranged from 66.67 to 75%. An accuracy of 80.95 to 85.71% was attained with the integration of all features. Challenges in the drowsiness included diminished accuracy with MAR alone and delayed prediction during transitions from non-drowsy to drowsy status. These findings underscore the model’s applicability in detecting drowsiness while highlighting the necessity for refinement. Through algorithm optimization, dataset expansion, and the integration of additional features and feedback mechanisms, the model can be improved in terms of performance and reliability. Full article

This paper presents a novel approach to mind-wandering prediction in the context of webcam-based online learning. We implemented a Singular Value Decomposition (SVD)-based 1D temporal eye-signal extraction method, which relies solely on eye landmark detection and eliminates the need for gaze tracking or specialized hardware, then extract suitable features from the signals to train the prediction model. Our thorough experimental framework facilitates the evaluation of our approach alongside baseline models, particularly in the analysis of temporal eye signals and the prediction of attentional states. Notably, our SVD-based signal captures both subtle and major eye movements, including changes in the eye boundary and pupil, surpassing the limited capabilities of eye aspect ratio (EAR)-based signals. Our proposed model exhibits a 2% improvement in the overall Area Under the Receiver Operating Characteristics curve (AUROC) metric and 7% in the F1-score metric for ‘not-focus’ prediction, compared to the combination of EAR-based and computationally intensive gaze-based models used in the baseline study These contributions have potential implications for enhancing the field of attentional state prediction in online learning, offering a practical and effective solution to benefit educational experiences. Full article

Significantly increased eye blink rate and partial blinks have been well documented in patients with dry eye disease (DED), a multifactorial eye disorder with few effective methods for clinical diagnosis. In this study, a point of care mHealth App named “EyeScore” was developed, utilizing blink rate and patterns as early clinical biomarkers for DED. EyeScore utilizes an iPhone for a 1-min in-app recording of eyelid movements. The use of facial landmarks, eye aspect ratio (EAR) and derivatives enabled a comprehensive analysis of video frames for the determination of eye blink rate and partial blink counts. Smartphone videos from ten DED patients and ten non-DED controls were analyzed to optimize EAR-based thresholds, with eye blink and partial blink results in excellent agreement with manual counts. Importantly, a clinically relevant algorithm for the calculation of “eye healthiness score” was created, which took into consideration eye blink rate, partial blink counts as well as other demographic and clinical risk factors for DED. This 10-point score can be conveniently measured anytime with non-invasive manners and successfully led to the identification of three individuals with DED conditions from ten non-DED controls. Thus, EyeScore can be validated as a valuable mHealth App for early DED screening, detection and treatment monitoring. Full article

The rapid expansion of video conferencing and remote works due to the COVID-19 pandemic has resulted in a massive volume of video data to be analyzed in order to understand the audience engagement. However, analyzing this data efficiently, particularly in real-time, poses a scalability challenge as online events can involve hundreds of people and last for hours. Existing solutions, especially open-sourced contributions, usually require dedicated and expensive hardware, and are designed as centralized cloud systems. Additionally, they may also require users to stream their video to remote servers, which raises privacy concerns. This paper introduces scalable and efficient computer vision algorithms for analyzing face orientation and eye blink in real-time on edge devices, including Android, iOS, and Raspberry Pi. An example solution is presented for proctoring online meetings, workplaces, and exams. It analyzes audiences on their own devices, thus addressing scalability and privacy issues, and runs at up to 30 fps on a Raspberry Pi. The proposed face orientation detection algorithm is extremely simple, efficient, and able to estimate the head pose in two degrees of freedom, horizontal and vertical. The proposed Eye Aspect Ratio (EAR) with simple adaptive threshold demonstrated a significant improvement in terms of false positives and overall accuracy compared to the existing constant threshold method. Additionally, the algorithms are implemented and open sourced as a toolkit with modular, cross-platform MediaPipe Calculators and Graphs so that users can easily create custom solutions for a variety of purposes and devices. Full article

Drowsiness is a major risk factor for road safety, contributing to serious injury, death, and economic loss on the road. Driving performance decreases because of increased drowsiness. In several different applications, such as facial movement analysis and driver safety, blink detection is an essential requirement that is used. The extremely rapid blink rate, on the other hand, makes automatic blink detection an extremely challenging task. This research paper presents a technique for identifying eye blinks in a video series recorded by a car dashboard camera in real time. The suggested technique determines the facial landmark positions for each video frame and then extracts the vertical distance between the eyelids from the facial landmark positions. The algorithm that has been proposed estimates the facial landmark positions, extracts a single scalar quantity by making use of Eye Aspect Ratio (EAR), and identifies the eye closeness in each frame. In the end, blinks are recognized by employing the modified EAR threshold value in conjunction with a pattern of EAR values in a relatively short period of time. Experimental evidence indicates that the greater the EAR threshold, the worse the AUC’s accuracy and performance. Further, 0.18 was determined to be the optimum EAR threshold in our research. Full article

Fatigue driving has always received a lot of attention, but few studies have focused on the fact that human fatigue is a cumulative process over time, and there are no models available to reflect this phenomenon. Furthermore, the problem of incorrect detection due to facial expression is still not well addressed. In this article, a model based on BP neural network and time cumulative effect was proposed to solve these problems. Experimental data were used to carry out this work and validate the proposed method. Firstly, the Adaboost algorithm was applied to detect faces, and the Kalman filter algorithm was used to trace the face movement. Then, a cascade regression tree-based method was used to detect the 68 facial landmarks and an improved method combining key points and image processing was adopted to calculate the eye aspect ratio (EAR). After that, a BP neural network model was developed and trained by selecting three characteristics: the longest period of continuous eye closure, number of yawns, and percentage of eye closure time (PERCLOS), and then the detection results without and with facial expressions were discussed and analyzed. Finally, by introducing the Sigmoid function, a fatigue detection model considering the time accumulation effect was established, and the drivers’ fatigue state was identified segment by segment through the recorded video. Compared with the traditional BP neural network model, the detection accuracies of the proposed model without and with facial expressions increased by 3.3% and 8.4%, respectively. The number of incorrect detections in the awake state also decreased obviously. The experimental results show that the proposed model can effectively filter out incorrect detections caused by facial expressions and truly reflect that driver fatigue is a time accumulating process. Full article

The research on driver fatigue detection is of great significance to improve driving safety. This paper proposes a real-time comprehensive driver fatigue detection algorithm based on facial landmarks to improve the detection accuracy, which detects the driver’s fatigue status by using facial video sequences without equipping their bodies with other intelligent devices. A tasks-constrained deep convolutional network is constructed to detect the face region based on 68 key points, which can solve the optimization problem caused by the different convergence speeds of each task. According to the real-time facial video images, the eye feature of the eye aspect ratio (EAR), mouth aspect ratio (MAR) and percentage of eye closure time (PERCLOS) are calculated based on facial landmarks. A comprehensive driver fatigue assessment model is established to assess the fatigue status of drivers through eye/mouth feature selection. After a series of comparative experiments, the results show that this proposed algorithm achieves good performance in both accuracy and speed for driver fatigue detection. Full article