Search Results (95)

With the development of human–computer interaction technology, non-contact interaction based on gaze tracking and facial movements has become a research hotspot. Traditional mouse-and-keyboard methods pose challenges for people with disabilities or limited hand movements, while existing gaze-tracking systems often rely on expensive hardware or lack sufficient accuracy. This paper designs and implements a real-time system using ordinary cameras, achieving natural, efficient interaction via multimodal input combination. The system uses an improved MobileNetV2 backbone to construct GazeTrackNet for gaze estimation. It adopts MediaPipe Face Mesh to detect facial landmarks. Meanwhile, it applies geometric feature analysis, including eye aspect ratio and mouth aspect ratio, to identify actions such as blinking and mouth opening. It adopts a hybrid control strategy that combines gaze jumping and head fine-tuning, using mouth state as the main control switch. Key contributions include a lightweight gaze-tracking algorithm that enables stable and efficient gaze detection on consumer-grade hardware, a multimodal interaction strategy based on facial movement that improves system stability and ease of use, and a complete prototype system that achieves real-time performance on standard laptops. Experimental results show an average gaze average angle error of 3.0°, 97% eye state recognition accuracy, and end-to-end latency below 70 ms. The system can satisfy the requirements of daily desktop interaction under normal indoor lighting, and shows potential for future barrier-free interaction applications after further validation with target users. Existing gaze-tracking methods either suffer from low precision on lightweight devices or bring heavy computational overhead. Common facial recognition approaches also face frequent false trigger interference. Compared with them, our scheme achieves balanced accuracy and real-time performance via an attention-enhanced structure, and the designed dual anti-shake mechanism effectively suppresses misjudgment, delivering a more stable hands-free interaction experience. Full article

This article presents the design, implementation, and experimental validation of an autonomous drone system for search and rescue operations in cluttered GNSS-denied environments. The proposed platform integrates advanced navigation, mapping, and victim-detection capabilities, leveraging a suite of RGB-D cameras and edge-AI computation for real-time perception and decision-making. A key contribution is the integration of an eye-blink-detection pipeline for onboard assessment of the consciousness states of detected victims, enabling the drone to prioritize rescue efforts based on victim alertness. The system employs a modular software architecture with a pipeline that combines a U-Net segmentation network with a MultiScaleLSTM classifier, achieving approximately 97.73% accuracy and a combined inference latency of 6.35 ms on the NVIDIA Jetson Xavier-NX. Experimental results demonstrate the drone’s ability to autonomously explore unknown environments, accurately detect and classify victims, and operate effectively in real-world scenarios. The article also discusses observed challenges, such as computational bottlenecks and false positive detections, and outlines future directions for improving system robustness and autonomy. Full article

Driver drowsiness detection has become an important application of sensor-based monitoring systems aimed at improving road safety. This review focuses on sensing technologies and physiological parameters used for real-time drowsiness detection in drivers. The surveyed approaches are categorized into physiological sensing methods, including electroencephalography (EEG), electrocardiography (ECG), galvanic skin response (GSR), and photoplethysmography (PPG), and mechanical sensing methods, including respiration rate, eye blinking, head movement, yawning, and steering wheel gripping force. Each method is analyzed from a sensor system perspective, considering signal acquisition principles, measurement location, and practical deployment constraints. In addition, the reviewed techniques are evaluated based on real-time capability, level of sensor attachment, cost, restriction of user movement, and suitability for standalone operation. The comparison highlights that mechanical sensing approaches provide non-invasive and cost-effective solutions; however, they are sensitive to environmental noise and behavioral variability. In contrast, physiological sensing methods offer more direct and earlier indicators of fatigue-related changes in biosignals, although they typically require wearable or contact-based sensors and more complex acquisition systems. The review further indicates that multimodal sensor fusion is increasingly being adopted to improve robustness and reliability in real-world driving conditions. Overall, this work provides a structured overview of sensing modalities and highlights key considerations for designing efficient, real-time driver monitoring systems. Full article

Given its substantial contribution to traffic accidents, one of the main goals of intelligent driver-assistance systems has become the detection and mitigation of driver fatigue to enhance driving safety and comfort. Among various approaches, vision-based facial analysis using deep learning has emerged as an effective and non-intrusive method for identifying driver drowsiness, as a key manifestation of fatigue. However, current drowsiness detection models do not account for demographic factors like gender, even though recent research has shown gender behavioral differences such as eye closure duration, blink frequency, yawning patterns, and facial muscle relaxation. In this paper, we present a fine-grained multi-stream transformer architecture that incorporates gender-awareness and shifted-windows attention for spatial feature fusion. Integrating gender embedding, by modulating the region-based features, allows the model to effectively learn gender-conditioned drowsiness features to minimize bias and diluted representations. Using the NTHU-DDD dataset, we evaluated two-stream and three-stream variants for gender-aware and gender-agnostic across three facial region contexts: the face region with a 20% margin, bare face region, and key facial regions (face, eyes, and mouth). A comprehensive ablation study was conducted to identify the most effective model setup. The results demonstrate that incorporating gender embedding improves detection performance, achieving an accuracy of 95.47% on the evaluation set. Moreover, using the proposed three-stream model (SWT-DD-3S) produced better results. Full article

Eye tracking in virtual reality (VR) head-mounted displays poses substantial engineering challenges, particularly under immersive display configurations with large fields of view (FOV), where optical layout, illumination, and image acquisition impose nontrivial system constraints. To address these design constraints, we present an integrated near-eye eye-tracking prototype tailored for immersive VR headsets, combining customized hardware components and a real-time software pipeline. The proposed system integrates optimized near-eye illumination and image acquisition with a pupil detection module and a deep learning-based gaze-vector estimation model, forming a real-time software pipeline for stable end-to-end gaze mapping under fixed calibration conditions. Under identical system settings, calibration procedures, and gaze-point mapping conditions, we evaluate the proposed gaze-vector estimation model through a controlled model-level ablation. The attention-enhanced model achieves an average angular deviation of 1.15°, corresponding to a 61.4% relative reduction compared with a baseline ResNet-152 model without attention. To demonstrate the usability of the system outputs at the application level, we further implement a real-time visualization example that integrates pupil diameter, gaze vectors, and blink events to depict the temporal evolution of eye-movement signals. This work provides a cost-effective and reproducible engineering reference for near-eye eye-movement acquisition and visualization in immersive VR settings and serves as a technical foundation for subsequent interaction design or behavioral analysis studies. Full article

Steady-state visual evoked potential (SSVEP)-based brain–computer interface (BCI) technology offers a promising solution for wheelchair control by translating neural signals into navigation commands. A major challenge—signal noise caused by eye blinks—is addressed in this feasibility study through real-time blink detection and correction. The proposed design utilizes sensors to capture both SSVEP and blink signals, enabling the isolation and compensation of interference, which improves control accuracy by 14.68%. Real-time correction during blinks significantly enhances system reliability and responsiveness. Furthermore, user data and global positioning system (GPS) trajectories are uploaded to the cloud via Wi-Fi 6E for continuous safety monitoring. This approach not only restores mobility for users with physical disabilities but also promotes independence and spatial autonomy. Full article

Driver fatigue is a primary factor in traffic accidents and poses a serious threat to road safety. To address this issue, this paper proposes a multi-feature fusion fatigue detection method based on an improved YOLOv8 model. First, the method uses an enhanced YOLOv8 model to achieve high-precision face detection. Then, it crops the detected face regions. Next, the lightweight PFLD (Practical Facial Landmark Detector) model performs keypoint detection on the cropped images, extracting 68 facial feature points and calculating key indicators related to fatigue status. These indicators include the eye aspect ratio (EAR), eyelid closure percentage (PERCLOS), mouth aspect ratio (MAR), and head posture ratio (HPR). To mitigate the impact of individual differences on detection accuracy, the paper introduces a novel sliding window model that combines a dynamic threshold adjustment strategy with an exponential weighted moving average (EWMA) algorithm. Based on this framework, blink frequency (BF), yawn frequency (YF), and nod frequency (NF) are calculated to extract time-series behavioral features related to fatigue. Finally, the driver’s fatigue state is determined using a comprehensive fatigue assessment algorithm. Experimental results on the WIDER FACE and YAWDD datasets demonstrate this method’s significant advantages in improving detection accuracy and computational efficiency. By striking a better balance between real-time performance and accuracy, the proposed method shows promise for real-world driving applications. Full article

We present the eyeglass-type switch, an eyeglass-mounted eye/blink switch designed for nurse-call operation by people with severe motor impairments, with a particular focus on amyotrophic lateral sclerosis (ALS). The system targets real-world bedside constraints—low illumination at night, supine posture, and network-independent operation—by combining near-infrared (NIR) LED illumination with an NIR eye camera and executing all processing on a small, GPU-free computer. A two-stage convolutional pipeline estimates eight periocular landmarks and the pupil center; eye-closure is detected either by a binary classifier or by an angle criterion derived from landmarks, which also skips pupil estimation during closure. User intent is determined by crossing a caregiver-tunable “off-area” around neutral gaze, implemented as rectangular or sector shapes. Four output modes—single, continuous, long-press, and hold-to-activate—are supported for both oculomotor and eyelid inputs. Safety is addressed via relay-based electrical isolation from the nurse-call circuit and audio feedback for state indication. The prototype runs at 18 fps on commodity hardware. In feature-point evaluation, mean errors were 2.84 pixels for landmarks and 1.33 pixels for the pupil center. In a bedside task with 12 healthy participants, the system achieved $F=0.965$ in single mode and $F=0.983$ in hold-to-activate mode; blink-only input yielded $F=0.993$. Performance was uniformly high for right/left/up and eye-closure cues, with lower recall for downward gaze due to eyelid occlusion, suggesting camera placement or threshold tuning as remedies. The results indicate that the proposed switch provides reliable, low-burden nurse-call control under nighttime conditions and offers a practical input option for emergency alerts and augmentative and alternative communication (AAC) workflows. Full article

Neurological disorders significantly affect ocular surface homeostasis, influencing parameters such as blink rate (BR), tear production, corneal nerve density, and sensitivity. This review summarizes recent findings on ocular surface alterations associated with neurological diseases, including Alzheimer’s disease (AD), Parkinson’s disease (PD), Guillain-Barré syndrome (GBS), trigeminal neuralgia (TN), multiple sclerosis (MS), and Charcot–Marie–Tooth disease (CMT). Notably, ocular manifestations such as reduced BR, decreased tear break-up time (TBUT), impaired tear secretion, and corneal nerve fiber loss are consistently reported. In AD, elevated tear amyloid-beta and tau proteins emerge as promising biomarkers for early disease detection. PD patients frequently experience dry eye symptoms attributed to reduced BR and tear film instability. GBS is linked to lagophthalmos and corneal nerve impairment, potentially leading to severe ocular surface damage. TN demonstrates bilateral ocular surface dysfunction despite unilateral neuropathic symptoms. MS is associated with significant ocular surface alterations, reflecting broader neuroinflammatory and autonomic disturbances. Similarly, CMT patients show reduced corneal sensitivity and tear production, underscoring the systemic nature of neurological impacts. Awareness of these ocular manifestations is essential for improving patient care and guiding future research into ocular biomarkers and targeted therapies. Full article

Wearable electroencephalography (EEG) enables brain monitoring in real-world environments beyond clinical settings; however, the relaxed constraints of the acquisition setup often compromise signal quality. This review examines methods for artifact detection and for the identification of artifact categories (e.g., ocular) and specific sources (e.g., eye blink) in wearable EEG. A systematic search was conducted across six databases using the query: (“electroencephalographic” OR “electroencephalography” OR “EEG”) AND (“Artifact detection” OR “Artifact identification” OR “Artifact removal” OR “Artifact rejection”) AND “wearable”. Following PRISMA guidelines, 58 studies were included. Artifacts in wearable EEG exhibit specific features due to dry electrodes, reduced scalp coverage, and subject mobility, yet only a few studies explicitly address these peculiarities. Most pipelines integrate detection and removal phases but rarely separate their impact on performance metrics, mainly accuracy (71%) when the clean signal is the reference and selectivity (63%), assessed with respect to physiological signal. Wavelet transforms and ICA, often using thresholding as a decision rule, are among the most frequently used techniques for managing ocular and muscular artifacts. ASR-based pipelines are widely applied for ocular, movement, and instrumental artifacts. Deep learning approaches are emerging, especially for muscular and motion artifacts, with promising applications in real-time settings. Auxiliary sensors (e.g., IMUs) are still underutilized despite their potential in enhancing artifact detection under ecological conditions. Only two studies addressed artifact category identification. A mapping of validated pipelines per artifact type and a survey of public datasets are provided to support benchmarking and reproducibility. Full article

Road traffic accidents remain a major global public health concern, where complex urban driving environments significantly elevate drivers’ visual load and accident risks. Unlike existing research that adopts a macro perspective by considering multiple factors such as the driver, vehicle, and road, this study focuses on the driver’s visual load, a key safety factor, and its direct source—the driver’s visual environment. We have developed an interpretable framework combining computer vision and machine learning to quantify how road scene features influence oculomotor behavior and scene-induced visual load, establishing a complete and interpretable link between scene features, eye movement behavior, and visual load. Using the DR(eye)VE dataset, visual attention demand is established through occlusion experiments and confirmed to correlate with eye-tracking metrics. K-means clustering is applied to classify visual load levels based on discriminative oculomotor features, while semantic segmentation extracts quantifiable road scene features such as the Green Visibility Index, Sky Visibility Index and Street Canyon Enclosure. Among multiple machine learning models (Random Forest, Ada-Boost, XGBoost, and SVM), XGBoost demonstrates optimal performance in visual load detection. SHAP analysis reveals critical thresholds: the probability of high visual load increases when pole density exceeds 0.08%, signage surpasses 0.55%, or buildings account for more than 14%; while blink duration/rate decrease when street enclosure exceeds 38% or road congestion goes beyond 25%, indicating elevated visual load. The proposed framework provides actionable insights for urban design and driver assistance systems, advancing traffic safety through data-driven optimization of road environments. Full article

The aim of this study is to analyze the signal generated in the brain for a specific motor task and to identify the region where it occurs. For this purpose, electroencephalography (EEG) signals were divided into delta, theta, alpha, and beta frequency sub-bands, and feature extraction was performed by looking at the time-frequency characteristics of the signals belonging to the obtained sub-bands. The epoch corresponding to motor imagery or action and the signal source in the brain were determined by power spectral density features. This study focused on a hand open–close motor task as an example. A machine learning structure was used for signal recognition and classification. The highest accuracy of 92.9% was obtained with the neural network in relation to signal recognition and action realization. In addition to the classification framework, this study also incorporated advanced preprocessing and energy analysis techniques. Eye blink artifacts were automatically detected and removed using independent component analysis (ICA), enabling more reliable spectral estimation. Furthermore, a detailed channel-based and sub-band energy analysis was performed using fast Fourier transform (FFT) and power spectral density (PSD) estimation. The results revealed that frontal electrodes, particularly Fp1 and AF7, exhibited dominant energy patterns during both real and imagined motor tasks. Delta band activity was found to be most pronounced during rest with T1 and T2, while higher-frequency bands, especially beta, showed increased activity during motor imagery, indicating cognitive and motor planning processes. Although 30 s epochs were initially used, event-based selection was applied within each epoch to mark short task-related intervals, ensuring methodological consistency with the 2–4 s windows commonly emphasized in the literature. After artifact removal, motor activity typically associated with the C3 region was also observed with greater intensity over the frontal electrode sites Fp1, Fp2, AF7, and AF8, demonstrating hemispheric symmetry. The delta band power was found to be higher than that of other frequency bands across T0, T1, and T2 conditions. However, a marked decrease in delta power was observed from T0 to T1 and T2. In contrast, beta band power increased by approximately 20% from T0 to T2, with a similar pattern also evident in gamma band activity. These changes indicate cognitive and motor planning processes. The novelty of this study lies in identifying the electrode that exhibits the strongest signal characteristics for a specific motor activity among 64-channel EEG recordings and subsequently achieving high-performance classification of the corresponding motor activity. Full article

This paper presents a novel EEG-based learning system designed to enhance the efficiency and effectiveness of studying by dynamically adjusting the difficulty level of learning materials based on real-time attention levels. In the training phase, EEG signals corresponding to high and low concentration levels are recorded while participants engage in quizzes to learn and memorize Chinese characters. The attention levels are determined based on performance metrics derived from the quiz results. Following extensive preprocessing, the EEG data undergoes several feature extraction steps: removal of artifacts due to eye blinks and facial movements, segregation of waves based on their frequencies, similarity indexing with respect to delay, binary thresholding, and (PCA). These extracted features are then fed into a k-NN classifier, which accurately distinguishes between high and low attention brain wave patterns, with the labels derived from the quiz performance indicating high or low attention. During the implementation phase, the system continuously monitors the user’s EEG signals while studying. When low attention levels are detected, the system increases the repetition frequency and reduces the difficulty of the flashcards to refocus the user’s attention. Conversely, when high concentration levels are identified, the system escalates the difficulty level of the flashcards to maximize the learning challenge. This adaptive approach ensures a more effective learning experience by maintaining optimal cognitive engagement, resulting in improved learning rates, reduced stress, and increased overall learning efficiency. Our results indicate that this EEG-based adaptive learning system holds significant potential for personalized education, fostering better retention and understanding of Chinese characters. Full article

Lou Gehrig’s disease, also known as ALS, is a progressive neurodegenerative condition that weakens muscles and can lead to paralysis as it progresses. For patients with severe paralysis, eye-tracking devices such as eye mouse enable communication. However, the equipment is expensive, and the calibration process is very difficult and frustrating for patients to use. To alleviate this problem, we propose a simple and efficient method to type texts intuitively with graphical guidance on the screen. Specifically, the method detects patients’ eye blinks in video frames to navigate through three sequential steps, narrowing down the choices from 9 letters, to 3 letters, and finally to a single letter (from a 26-letter alphabet). In this way, a patient is able to rapidly type a letter of the alphabet by blinking a minimum of three times and a maximum of nine times. The proposed method integrates an API of large language model (LLM) to further accelerate text input and correct sentences in terms of typographical errors, spacing, and upper/lower case. Experiments on ten participants demonstrate that the proposed method significantly outperforms three state-of-the-art methods in both typing speed and typing accuracy, without requiring any calibration process. Full article

Fatigue in high-stress work environments poses significant risks to employee performance and safety. This study introduces a non-invasive fatigue detection system utilizing facial parameters processed via a Long Short-Term Memory (LSTM) neural network, coupled with a human–machine interaction interface via a Telegram chatbot. The system analyzes eye blink patterns and facial expression changes captured through a webcam, achieving an accuracy of 92.35% on the UTA-RLDD dataset. An interactive feedback mechanism allows users to verify predictions, enhancing system adaptability. We further propose a multimodal AI framework to integrate physiological and environmental data, laying the groundwork for broader applications. This approach provides an effective solution for early fatigue detection and adaptive collaboration between humans and machines in real-time settings. Full article