Search Results (68)

Car accidents remain a leading cause of death worldwide, with drowsiness and distraction accounting for roughly 25% of fatal crashes in Ecuador. This study presents a real-time driver monitoring system that uses computer vision and deep learning to detect fatigue, distraction, and emotions from facial expressions. It combines a MobileNetV2-based CNN trained on RAF-DB for emotion recognition and MediaPipe’s 468 facial landmarks to compute the EAR (Eye Aspect Ratio), the MAR (Mouth Aspect Ratio), the gaze, and the head pose. Tests with 27 participants in both real and simulated driving environments showed strong results. There was a 100% accuracy in detecting distraction, 85.19% for yawning, and 88.89% for eye closure. The system also effectively recognized happiness (100%) and anger/disgust (96.3%). However, it struggled with sadness and failed to detect fear, likely due to the subtlety of real-world expressions and limitations in the training dataset. Despite these challenges, the results highlight the importance of integrating emotional awareness into driver monitoring systems, which helps reduce false alarms and improve response accuracy. This work supports the development of lightweight, non-invasive technologies that enhance driving safety through intelligent behavior analysis. Full article

Work-related musculoskeletal disorders (MSDs) remain a global occupational health priority, with recognized limitations in current point-in-time assessment methodologies. This research extends prior computer vision ergonomic assessment approaches by implementing biological proprioceptive feedback principles into a continuous, real-time monitoring system. Unlike traditional periodic ergonomic evaluation methods such as “Rapid Upper Limb Assessment” (RULA), our bio-inspired system translates natural proprioceptive mechanisms—which enable continuous postural monitoring through spinal feedback loops operating at 50–150 ms latencies—into automated assessment technology. The system integrates (1) markerless 3D pose estimation via MediaPipe Holistic (33 anatomical landmarks at 30 FPS), (2) depth validation via Orbbec Femto Mega RGB-D camera (640 × 576 resolution, Time-of-Flight sensor), and (3) proprioceptive-inspired alert architecture. Experimental validation with 40 adult participants (age 18–25, n = 26 female, n = 14 male) performing standardized load-lifting tasks (6 kg) demonstrated that 62.5% exhibited critical postural risk (RULA ≥ 5) during dynamic movement versus 7.5% at static rest, with McNemar test $p<0.001$ (Cohen’s $h=1.22$, 95% CI: 0.91–0.97). The system achieved 95% Pearson correlation between risk elevation and alert activation, with response latency of $42.1\pm 8.3$ ms. This work demonstrates technical feasibility for continuous occupational monitoring. However, long-term prospective studies are required to establish whether continuous real-time feedback reduces workplace injury incidence. The biomimetic design framework provides a systematic foundation for translating biological feedback principles into occupational health technology. Full article

Markerless human pose estimation is increasingly used for kinematic assessment, but evidence of its applicability to upper limb movements across different ranges of motion (ROM) remains limited. This study examined the accuracy and reliability of a markerless pose estimation system for shoulder, elbow and wrist flexion–extension analysis under full and partial ROM tasks. Ten healthy participants performed standardized movements which were synchronously recorded, with an optoelectronic motion capture system used as a reference. Joint angles were compared using RMSE, percentage RMSE (%RMSE), accuracy (Acc), intraclass correlation coefficients (ICC), and Pearson correlation of ROM values. The markerless system reproduced the temporal morphology of the movement with high coherence, showing ICC values above 0.91 for the elbow and 0.94 for the shoulder in full ROM trials. Wrist tracking presented the lowest RMSE values and low inter-subject variability. The main critical aspect was a systematic underestimation of maximum flexion, especially at the shoulder, indicating a magnitude bias likely influenced by occlusion and joint geometry rather than by temporal fluctuations. Despite this limitation, the system adapted consistently to different ROM amplitudes, maintaining proportional variations in joint excursion across tasks. Overall, the findings outline the conditions in which markerless pose estimation provides reliable upper limb kinematics and where methodological improvements are still required, particularly in movements involving extreme flexion and occlusion. Full article

This paper presents Aerokinesis, an IoT-based software–hardware system for intuitive gesture-driven control of quadcopter unmanned aerial vehicles (UAVs), developed within the Robot Operating System 2 (ROS2) framework. The proposed system addresses the challenge of providing an accessible human–drone interaction interface for operators in scenarios where traditional remote controllers are impractical or unavailable. The architecture comprises two hierarchical control levels: (1) high-level discrete command control utilizing a fully connected neural network classifier for static gesture recognition, and (2) low-level continuous flight control based on three-dimensional hand keypoint analysis from a depth camera. The gesture classification module achieves an accuracy exceeding 99% using a multi-layer perceptron trained on MediaPipe-extracted hand landmarks. For continuous control, we propose a novel approach that computes Euler angles (roll, pitch, yaw) and throttle from 3D hand pose estimation, enabling intuitive four-degree-of-freedom quadcopter manipulation. A hybrid signal filtering pipeline ensures robust control signal generation while maintaining real-time responsiveness. Comparative user studies demonstrate that gesture-based control reduces task completion time by 52.6% for beginners compared to conventional remote controllers. The results confirm the viability of vision-based gesture interfaces for IoT-enabled UAV applications. Full article

Advancements in artificial intelligence have significantly enhanced communication for individuals with hearing impairments. This study presents a robust cross-lingual Sign Language Recognition (SLR) framework for Turkish, American English, and Arabic sign languages. The system utilizes the lightweight MediaPipe library for efficient hand landmark extraction, ensuring stable and consistent feature representation across diverse linguistic contexts. Datasets were meticulously constructed from nine public-domain sources (four Arabic, three American, and two Turkish). The final training data comprises curated image datasets, with frames for each language carefully selected from varying angles and distances to ensure high diversity. A comprehensive comparative evaluation was conducted across three state-of-the-art deep learning architectures—ConvNeXt (CNN-based), Swin Transformer (ViT-based), and Vision Mamba (SSM-based)—all applied to identical feature sets. The evaluation demonstrates the superior performance of contemporary vision Transformers and state space models in capturing subtle spatial cues across diverse sign languages. Our approach provides a comparative analysis of model generalization capabilities across three distinct sign languages, offering valuable insights for model selection in pose-based SLR systems. Full article

This paper presents an immersive teleoperation framework for service robots that combines real-time 3D human pose estimation with a Virtual Reality (VR) interface to support intuitive, natural robot control. The operator is tracked using MediaPipe for 2D landmark detection and an Intel RealSense D455 RGB-D (Red-Green-Blue plus Depth) camera for depth acquisition, enabling 3D reconstruction of key joints. Joint angles are computed using efficient vector operations and mapped to the kinematic constraints of an anthropomorphic arm on the CHARMIE service robot. A VR-based telepresence interface provides stereoscopic video and head-motion-based view control to improve situational awareness during manipulation tasks. Experiments in real-world object grasping demonstrate reliable arm teleoperation and effective telepresence; however, vision-only estimation remains limited for axial rotations (e.g., elbow and wrist yaw), particularly under occlusions and unfavorable viewpoints. The proposed system provides a practical pathway toward low-cost, sensor-driven, immersive human–robot interaction for service robotics in dynamic environments. Full article

Accurate and accessible motor function quantification is important for monitoring the movement disorders’ progression. Manual muscle testing models and wearable sensors can be costly or reduce degrees of freedom. Artificial intelligence, especially human pose estimation (PE), offers promising alternatives. This work aims to compare the accuracy of a 2D PE tool for the Finger Tapping Test (FTT) with a 3D infrared motion capture system (MoCap). PE tracked three anatomical landmarks (wrist, thumb, index finger), while reflective markers were placed at corresponding locations on both tools to measure wrist-centered angles. Different trials of slow and rapid FTT sessions were statistically analyzed by rank correlation analysis, Friedman, Bland–Altman, and Kruskal–Wallis to assess agreement and repeatability. PE and MoCap measurements showed no significant differences (p > 0.05), with high reliability (ICC 0.87–0.91), low variability (CV 6–8.6%), and negligible effect size. Bland–Altman slopes indicated minor amplitude-dependent bias, while RMSE (2.92–4.48°) and MAPE (6.38–8.22%) errors occurred in slow and rapid conditions. These results demonstrate that 2D PE provides a reliable, accessible, and low-cost alternative for quantifying finger movement. The findings suggest that PE can serve as an assistive method for monitoring motor function. Future studies can be population-level studies with patients with neurological disorders. Full article

Head posture assessment commonly involves measuring cranial angles, with photogrammetry favored for its simplicity over CT scans or goniometers. However, most photo-based measurements remain manual, making them time-consuming and inefficient. Existing automatic measuring approaches often requires specific markers and clean backgrounds, limiting their usability. We present AutoMCA, a robust automatic measurement system for cranial angles using accessible markers and tolerating typical indoor backgrounds. AutoMCA integrates MediaPipe Pose, a machine-learning solution, for head–neck segmentation and applies color thresholding and morphological operations for marker detection. Validation tests demonstrated Pearson correlation coefficients above 0.98 compared to manual Kinovea measurements for both the craniovertebral angle (CVA) and cranial rotation angle (CRA), confirming high accuracy. Further validation on individuals with neck disorders showed similarly strong correlations, supporting clinical applicability. Speed comparison tests revealed that AutoMCA significantly reduces measurement time compared to traditional photogrammetry. Robustness tests confirmed reliable performance across varied backgrounds and marker types. In conclusion, AutoMCA measures head posture efficiency and lowers the requirements for instruments and space, making the assessment more versatile and applicable. Full article

For human–robot gesture-driven control, this paper proposes a deep learning-based approach that employs both static and dynamic gestures to drive and control robots for object-grasping and delivery tasks. The method utilizes two-dimensional Convolutional Neural Networks (2D-CNNs) for static gesture recognition and a hybrid architecture combining three-dimensional Convolutional Neural Networks (3D-CNNs) and Long Short-Term Memory networks (3D-CNN+LSTM) for dynamic gesture recognition. Results on a custom gesture dataset demonstrate validation accuracies of 95.38% for static gestures and 93.18% for dynamic gestures, respectively. Then, in order to control and drive the robot to perform corresponding tasks, hand pose estimation was performed. The MediaPipe machine learning framework was first employed to extract hand feature points. These 2D feature points were then converted into 3D coordinates using a depth camera-based pose estimation method, followed by coordinate system transformation to obtain hand poses relative to the robot’s base coordinate system. Finally, an experimental platform for human–robot gesture-driven interaction was established, deploying both gesture recognition models. Four participants were invited to perform 100 trials each of gesture-driven object-grasping and delivery tasks under three lighting conditions: natural light, low light, and strong light. Experimental results show that the average success rates for completing tasks via static and dynamic gestures are no less than 96.88% and 94.63%, respectively, with task completion times consistently within 20 s. These findings demonstrate that the proposed approach enables robust vision-based robotic control through natural hand gestures, showing great prospects for human–robot collaboration applications. Full article

High-precision analyses of baseball pitching have traditionally relied on optical motion capture systems, which, despite their accuracy, are complex and impractical for widespread use. Classifying sequential pitching phases, essential for biomechanical evaluation, conventionally requires manual expert labeling, a time-consuming and labor-intensive process. Accurate identification of phase boundaries is critical because they correspond to key temporal events related to pitching injuries. This study developed and validated a smartphone-based system for automatically classifying the five key pitching phases—wind-up, stride, arm-cocking, arm acceleration, and follow-through—using pose estimation artificial intelligence and machine learning. Slow-motion videos (240 frames per second, 1080p) of 500 healthy right-handed high school pitchers were recorded from the front using a single smartphone. Skeletal landmarks were extracted using MediaPipe, and 33 kinematic features, including joint angles and limb distances, were computed. Expert-annotated phase labels were used to train classification models. Among the models evaluated, Light Gradient Boosting Machine (LightGBM) achieved a classification accuracy of 99.7% and processed each video in a few seconds demonstrating feasibility for on-site analysis. This system enables high-accuracy phase classification directly from video without motion capture, supporting future tools to detect abnormal pitching mechanics, prevent throwing-related injuries, and broaden access to pitching analysis. Full article

Human Pose Estimation (HPE) models have varied applications and represent a cutting-edge branch of study, whose systems such as MediaPipe (MP), OpenPose (OP), and AlphaPose (ALP) show marked success. One of these areas, however, that is inadequately researched is the impact of image degradation on the accuracy of HPE models. Image degradation refers to images whose visual quality has been purposefully degraded by means of techniques, such as brightness adjustments (which can lead to an increase or a decrease in the intensity levels), geometric rotations, or resolution downscaling. The study of how these types of degradation impact the performance functionality of HPE models is an under-researched domaina that is a virtually unexplored area. In addition, current methods of the efficacy of existing image restoration techniques have not been rigorously evaluated and improving degraded images to a high quality has not been well examined in relation to improving HPE models. In this study, we explicitly clearly demonstrate a decline in the precision of the HPE model when image quality is degraded. Our qualitative and quantitative measurements identify a wide difference in performance in identifying landmarks as images undergo changes in brightness, rotation, or reductions in resolution. Additionally, we have tested a variety of existing image enhancement methods in an attempt to enhance their capability in restoring low-quality images, hence supporting improved functionality of HPE. Interestingly, for rotated images, using Pillow of OpenCV improves landmark recognition precision drastically, nearly restoring it to levels we see in high-quality images. In instances of brightness variation and in low-quality images, however, existing methods of enhancement fail to yield the improvements anticipated, highlighting a large direction of study that warrants further investigation and calls for additional research. In this regard, we proposed a wide-ranging system for classifying different types of image degradation systematically and for selecting appropriate algorithms for image restoration, in an effort to restore image quality. A key finding is that in a related study of current methods, the Tuned RotNet model achieves 92.04% accuracy, significantly outperforming the baseline model and surpassing the official RotNet model in predicting rotation degree of images, where the accuracy of official RotNet and Tuned RotNet classifiers were 61.59% and 92.04%, respectively. Furthermore, in an effort to facilitate future research and make it easier for other studies, we provide a new dataset of reference images and corresponding degenerated images, addressing a notable gap in controlled comparative studies, since currently there is a lack of controlled comparatives. Full article

An essential component of modern musculoskeletal rehabilitation systems is treadmills of various sizes, the control of which may rely either on manual adjustment of treadmill speed, fixed for the entire training session, or on automatic regulation based on analysis of the user’s movements and velocity. The aim of this study was to experimentally compare the control functions of an adaptive treadmill designed for musculoskeletal rehabilitation and to assess the influence of the hardware configuration and tracking systems on user stability and the smoothness of transient processes. Two running platforms (of different lengths, one equipped with handrails and one without), two tracking systems (virtual reality trackers and a computer vision system using the MediaPipe Pose model), and three control functions—linear, nonlinear, and proportional-integral-derivative (PID)—were investigated. A set of metrics with both metrical and physiological interpretability was proposed (including positional stability, duration and amplitude of transient processes in position and velocity, subjective assessment, and others), all integrated into a single quality control criterion. This study presents extensive experimental research comparing various designs of adaptive running platforms and tracking systems, exploring the relationships between the available working area length and user comfort, and determining the optimal parameters for the selected control functions. The optimal control function was identified as the linear law for the tracking system based on virtual reality trackers and the PID function for the computer-vision-based tracking system. The conducted experiments made it possible to formulate recommendations regarding the minimum permissible working area length of treadmill platforms and the selection of tracking systems and control functions for musculoskeletal rehabilitation systems. The obtained results are of practical relevance for developing adaptive rehabilitation simulators and creating control algorithms that ensure smooth and stable treadmill motion, thereby enhancing user comfort, efficiency, and safety during musculoskeletal rehabilitation exercises. Full article

Learning from Demonstration (LfD) enables robots to acquire manipulation skills by observing human actions. However, existing methods often face challenges such as high computational cost, limited generalizability, and a loss of key interaction details. This study presents a compact representation for interaction recognition in LfD that encodes human–object interactions using 2D wrist trajectories and 3D object poses. A lightweight extraction pipeline combines MediaPipe-based wrist tracking with FoundationPose-based 6-DoF object estimation to obtain these trajectories directly from RGB-D video without specialized sensors or heavy preprocessing. Experiments on the GRAB and FPHA datasets show that the representation effectively captures task-relevant interactions, achieving 94.6% accuracy on GRAB and 96.0% on FPHA with well-calibrated probability predictions. Both Bidirectional Long Short-Term Memory (Bi-LSTM) with attention and Transformer architectures deliver consistent performance, confirming robustness and generalizability. The method achieves sub-second inference, a memory footprint under 1 GB, and reliable operation on both GPU and CPU platforms, enabling deployment on edge devices such as NVIDIA Jetson. By bridging pose-based and object-centric paradigms, this approach offers a compact and efficient foundation for scalable robot learning while preserving essential spatiotemporal dynamics. Full article

This paper presents our first results obtained in the direction of using a humanoid robot to perform a robot dance at a level comparable to that of a human dancer. The scope of this first approach is limited to performing an offline analysis of the movements of the arms of the dancer and to replicating these movements with the arms of the robot. To this end, the movements of a dancer performing a static robot dance (without moving the hips or feet) were recorded. These movements were analyzed offline using the MediaPipe BlazePose framework, adapted to the mechanical configuration of the arms of the humanoid robot, and finally reproduced by the robot. Results showed that MediaPipe has some inaccuracies when detecting sudden movements of the dancer’s arms that appeared blurred in the images. In general, the humanoid robot was capable of replicating the movement of the dancer’s arms but was unable to follow the original rhythm of the robotic dance due to acceleration limitations of its actuators. Full article

Background. Clinical assessment of Parkinson’s disease (PD) is limited by high subjectivity and inter-rater variability. Markerless video analysis, namely Computer Vision (CV), offers objective and scalable characterization of motor signs. We systematically reviewed CV technologies suited for PD diagnosis, symptom monitoring, and treatment management. Methods. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we searched PubMed for articles published between 1 January 1984 and 9 May 2025. We used the following search strategy: (“Parkinson Disease” [MeSH Terms] OR “parkinson’s disease” OR “parkinson disease”) AND (“computer vision” OR “video analysis” OR “pose estimation” OR “OpenPose” OR “DeepLabCut” OR “OpenFace” OR “YOLO” OR “MediaPipe” OR “markerless motion capture” OR “skeleton tracking”). Results. Out of 154 identified studies, 45 met eligibility criteria and were synthesized. Gait was assessed in 42% of studies, followed by bradykinesia items (17.7%). OpenPose and custom CV solutions were each used in 36% of studies, followed by MediaPipe (16%), DeepLabCut (9%), YOLO (4%). Across aims, CV pipelines consistently showed diagnostic discrimination and severity tracking aligned with expert ratings. Conclusions. CV non-invasively quantifies PD motor impairment, holding potential for objective diagnosis, longitudinal monitoring, and therapy response. Guidelines for standardized video-recording protocols and software usage are needed for real-world applications. Full article