Search Results (60)

Hand gestures are essential for interaction in augmented and virtual reality (AR/VR), allowing users to intuitively manipulate virtual objects and engage with human–machine interfaces (HMIs). Accurate gesture recognition is critical for effective task execution. However, users often encounter difficulties due to the lack of immediate and clear feedback from head-mounted displays (HMDs). Current tracking technologies cannot always guarantee reliable recognition, leaving users uncertain about whether their gestures have been successfully detected. To address this limitation, haptic feedback can play a key role by confirming gesture recognition and compensating for discrepancies between the visual perception of fingertip contact with virtual objects and the actual system recognition. The goal of this paper is to compare a simple vibrotactile ring with a full glove device and identify their possible improvements for a fundamental gesture like pinching and fine moving of objects using Microsoft HoloLens 2. Where the pinch action is considered an essential fine motor skill, augmented reality integrated with haptic feedback can be useful to notify the user of the recognition of the gestures and compensate for misaligned visual perception between the tracked fingertip with respect to virtual objects to determine better performance in terms of spatial precision. In our experiments, the participants’ median distance error using bare hands over all axes was 10.3 mm (interquartile range [IQR] = 13.1 mm) in a median time of 10.0 s (IQR = 4.0 s). While both haptic devices demonstrated improvement in participants precision with respect to the bare-hands case, participants achieved with the full glove median errors of 2.4 mm (IQR = 5.2) in a median time of 8.0 s (IQR = 6.0 s), and with the haptic rings they achieved even better performance with median errors of 2.0 mm (IQR = 2.0 mm) in an even better median time of only 6.0 s (IQR= 5.0 s). Our outcomes suggest that simple devices like the described haptic rings can be better than glove-like devices, offering better performance in terms of accuracy, execution time, and wearability. The haptic glove probably compromises hand and finger tracking with the Microsoft HoloLens 2. Full article

Hand motion monitoring plays a vital role in medical rehabilitation, sports training, and human–computer interaction. High-sensitivity wearable biosensors are essential for accurate gesture recognition and precise motion analysis. In this work, we propose a high-sensitivity wearable glove based on a push–pull optical fiber sensor, designed to enhance the sensitivity and accuracy of hand motion biosensing. The sensor employs diagonal core reflectors fabricated at the tip of a four-core fiber, which interconnect symmetric fiber channels to form a push–pull sensing mechanism. This mechanism induces opposite wavelength shifts in fiber Bragg gratings positioned symmetrically under bending, effectively decoupling temperature and strain effects while significantly enhancing bending sensitivity. Experimental results demonstrate superior bending-sensing performance, establishing a solid foundation for high-precision gesture recognition. The integrated wearable glove offers a compact, flexible structure and straightforward fabrication process, with promising applications in precision medicine, intelligent human–machine interaction, virtual reality, and continuous health monitoring. Full article

Glove-based human–machine interfaces (HMIs) offer a natural, intuitive way to capture finger motions for gesture recognition, virtual interaction, and robotic control. However, many existing systems suffer from complex fabrication, limited sensitivity, and reliance on external power. Here, we present a flexible, self-powered glove HMI based on a minimalist triboelectric nanogenerator (TENG) sensor composed of a conductive fabric electrode and textured Ecoflex layer. Surface micro-structuring via 3D-printed molds enhances triboelectric performance without added complexity, achieving a peak power density of 75.02 μW/cm² and stable operation over 13,000 cycles. The glove system enables real-time LED brightness control via finger-bending kinematics and supports intelligent recognition applications. A convolutional neural network (CNN) achieves 99.2% accuracy in user identification and 97.0% in object classification. By combining energy autonomy, mechanical simplicity, and machine learning capabilities, this work advances scalable, multi-functional HMIs for applications in assistive robotics, augmented reality (AR)/(virtual reality) VR environments, and secure interactive systems. Full article

This study presents a networked cyber-physical architecture that integrates a Reinforcement Learning-based Digital Twin (DT) to enable zero-delay interaction between physical and digital components in smart agriculture. The proposed system allows real-time remote control of a robotic arm inside a hydroponic greenhouse, using a sensor-equipped Wearable Glove (SWG) for hand motion capture. The DT operates in three coordinated modes: Real2Digital, Digital2Real, and Digital2Digital, supporting bidirectional synchronization and predictive simulation. A core innovation lies in the use of a Reinforcement Learning model to anticipate hand motions, thereby compensating for network latency and enhancing the responsiveness of the virtual–physical interaction. The architecture was experimentally validated through a detailed communication delay analysis, covering sensing, data processing, network transmission, and 3D rendering. While results confirm the system’s effectiveness under typical conditions, performance may vary under unstable network scenarios. This work represents a promising step toward real-time adaptive DTs in complex smart greenhouse environments. Full article

Virtual reality (VR) technology has shown potential as a viable tool for rehabilitation. VR is a well-recognized technology that creates immersive experiences to enhance engagement and encourage more effective participation in activities. In the current study, it has been shown that using a standard VR system setup can effectively increase participant motivation for various rehabilitation applications. However, there is a research gap in terms of participant motivation, relating to the intervention of integrating data gloves into VR to improve visibility in hand tracking for rehabilitation. This study presents and assesses an integrated approach utilizing VR and data glove technology to evaluate upper extremity function in a young, healthy population, comparing this to traditional methods. Participants’ intrinsic motivation was measured using the Intrinsic Motivation Inventory (IMI). The findings indicate that the combined immersive environment outperforms conventional practice in most aspects. Therefore, this research also sheds light on the fact that a data glove is promising technology in rehabilitation applications that can augment positive experiences while having no adverse effects on the VR system. Full article

Virtual reality (VR) is a promising tool in spinal cord injury (SCI) rehabilitation, particularly through virtual adaptations of functional tests like the Box and Block test (BBT). However, a comprehensive dynamic comparison between real and virtual BBT is lacking. This study investigates the kinematic and electromyographic (EMG) differences between healthy individuals and SCI patients performing both real (RBBT) and virtual (VBBT) versions of the BBT. An electromagnetic motion-tracking system, an instrumented glove, and surface EMG electrodes were used to capture hand trajectories, joint angles, and forearm muscle activation. The analysis included cycle-averaged and temporal kinematic and EMG parameters. Our findings reveal that both groups showed increased trajectory length and velocity peaks during the VBBT, with more pronounced increases in SCI patients. Unlike healthy individuals, SCI patients also showed increased finger and thumb flexion during VBBT. Cycle-averaged EMG values were lower in healthy participants during VBBT, likely due to reduced motor demands and lack of real grasping. Conversely, SCI patients exhibited higher muscle activity, suggesting impaired coordination and compensatory overactivation. Healthy individuals showed consistent temporal kinematic synergies and muscle activation, whereas they were altered in SCI patients, especially during reaching. These findings highlight the need for rehabilitation strategies to improve motor control and feedback integration. Full article

This study introduces a novel system for evaluating hand motor function through synergy-based analysis during object manipulation in virtual and mixed-reality environments. Conventional assessments of hand function are often subjective, relying on visual observation by therapists or patient-reported outcomes. To address these limitations, we developed a system that utilizes the leap motion controller (LMC) to capture finger motion data without the constraints of glove-type devices. Spatial synergies were extracted using principal component analysis (PCA) and Varimax rotation, providing insights into finger motor coordination with the sparse decomposition. Additionally, we incorporated the HoloLens 2 to create a mixed-reality object manipulation task that enhances spatial awareness for the user, improving natural interaction with virtual objects. Our results demonstrate that synergy-based analysis allows for the systematic detection of hand movement abnormalities that are not captured through traditional task performance metrics. This system demonstrates promise in advancing rehabilitation by enabling more objective and detailed evaluations of finger motor function, facilitating personalized therapy, and potentially contributing to the early detection of motor impairments in the future. Full article

The exploitation of immersive simulation platforms to improve traditional training techniques in the agricultural industry sector would enable year-round accessibility, flexibility, safety, and consistent high-quality training for agricultural operators. An innovative workflow in virtual simulations for training and educational purposes includes an immersive environment in which the operator can interact with plants through haptic interfaces, following instructions imparted by a non-playing character (NPC) instructor. This study allows simulating the pruning of a complex case study, a hazelnut tree, reproduced in very high detail to offer agricultural operators a more realistic and immersive training environment than those currently existing. The process of creating a multisensorial environment starts with the integrated survey of the plant with a laser scanner and photogrammetry and then generates a controllable parametric model from roots to leaves with the exact positioning of the original branches. The model is finally inserted into a simulation, where haptic gloves with tactile resistance responsive to model collisions are tested. The results of the experimentation demonstrate the correct execution of this innovative design simulation, in which branches and leaves can be cut using a shear, with immediate sensory feedback. The project therefore aims to finalize this product as a realistic training platform for pruning, but not limited to it, paving the way for high-fidelity simulation for many other types of operations and specializations. Full article

This paper presents the theoretical foundation, practical implementation, and empirical evaluation of a glove for interaction with 3-D virtual environments. At the dawn of the “Spatial Computing Era”, where users continuously interact with 3-D Virtual and Augmented Reality environments, the need for a practical and intuitive interaction system that can efficiently engage 3-D elements is becoming pressing. Over the last few decades, there have been attempts to provide such an interaction mechanism using a glove. However, glove systems are currently not in widespread use due to their high cost and, we propose, due to their inability to sustain high levels of performance under certain situations. Performance deterioration has been observed due to the distortion of the local magnetic field caused by ordinary ferromagnetic objects present near the glove’s operating space. There are several areas where reliable hand-tracking gloves could provide a next generation of improved solutions, such as American Sign Language training and automatic translation to text and training and evaluation for activities that require high motor skills in the hands (e.g., playing some musical instruments, training of surgeons, etc.). While the use of a hand-tracking glove toward these goals seems intuitive, some of the currently available glove systems may not meet the accuracy and reliability levels required for those use cases. This paper describes our concept of an interaction glove instrumented with miniature magnetic, angular rate, and gravity (MARG) sensors and aided by a single camera. The camera used is an off-the-shelf red, green, and blue–depth (RGB-D) camera. We describe a proof-of-concept implementation of the system using our custom “GMVDK” orientation estimation algorithm. This paper also describes the glove’s empirical evaluation with human-subject performance tests. The results show that the prototype glove, using the GMVDK algorithm, is able to operate without performance losses, even in magnetically distorted environments. Full article

One of the most widely used strategies for learning support is the use of Information and Communication Technologies (ICTs), due to the variety of applications and benefits they provide in the educational field. This article describes the design and implementation of an immersive application supported by Senso gloves and 3D environments for learning English as a second language in Ecuador. The following steps should be considered for the app design: (1) the creation of a classroom with characteristics similar to a real classroom and different buttons to navigate through the scenarios; (2) the creation of a virtual environment where text, images, examples, and audio are added according to the grammatical topic; (3) the creation of a dynamic environment for assessment in which multiple choice questions are interacted with, followed by automatic grading with direct feedback. The results showed that the interaction between the physical and virtual environment through navigation tests with the glove in different 3D environments achieved a complete activation and navigation rate. Teachers showed a clear interest in using the application in their classes as an additional teaching tool to complement the English language teaching process, given that it can increase motivation and memorization in students, as it is an easy-to-use application, and the 3D environments designed are attractive, which would make classes more dynamic. In addition, the availability of the application at any place and time represents a support for the current academic community as it adapts to the needs of today’s world. Full article

Wearable motion capture gloves enable the precise analysis of hand and finger movements for a variety of uses, including robotic surgery, rehabilitation, and most commonly, virtual augmentation. However, many motion capture gloves restrict natural hand movement with a closed-palm design, including fabric over the palm and fingers. In order to alleviate slippage, improve comfort, reduce sizing issues, and eliminate movement restrictions, this paper presents a new low-cost data glove with an innovative open-palm and finger-free design. The new design improves usability and overall functionality by addressing the limitations of traditional closed-palm designs. It is especially beneficial in capturing movements in fields such as physical therapy and robotic surgery. The new glove incorporates resistive flex sensors (RFSs) at each finger and an inertial measurement unit (IMU) at the wrist joint to measure wrist flexion, extension, ulnar and radial deviation, and rotation. Initially the sensors were tested individually for drift, synchronisation delays, and linearity. The results show a drift of 6.60°/h in the IMU and no drift in the RFSs. There was a 0.06 s delay in the data captured by the IMU compared to the RFSs. The glove’s performance was tested with a collaborate robot testing setup. In static conditions, it was found that the IMU had a worst case error across three trials of 7.01° and a mean absolute error (MAE) averaged over three trials of 4.85°, while RFSs had a worst case error of 3.77° and a MAE of 1.25° averaged over all five RFSs used. There was no clear correlation between measurement error and speed. Overall, the new glove design proved to accurately measure joint angles. Full article

Immersive human–machine interaction relies on comprehensive sensing and feedback systems, which enable transmission of multiple pieces of information. However, the integration of increasing numbers of feedback actuators and sensors causes a severe issue in terms of system complexity. In this work, we propose a pressure-sensing and feedback glove that enables multi-dimensional pressure sensing and feedback with a minimalist design of the functional units. The proposed glove consists of modular strain and pressure sensors based on films of liquid metal microchannels and coin vibrators. Strain sensors located at the finger joints can simultaneously project the bending motion of the individual joint into the virtual space or robotic hand. For subsequent tactile interactions, the design of two symmetrically distributed pressure sensors and vibrators at the fingertips possesses capabilities for multi-directional pressure sensing and feedback by evaluating the relationship of the signal variations between two sensors and tuning the feedback intensities of two vibrators. Consequently, both dynamic and static multi-dimensional pressure communication can be realized, and the vibrational actuation can be monitored by a liquid-metal-based sensor via a triboelectric sensing mechanism. A demonstration of object interaction indicates that the proposed glove can effectively detect dynamic force in varied directions at the fingertip while offering the reconstruction of a similar perception via the haptic feedback function. This device introduces an approach that adopts a minimalist design to achieve a multi-functional system, and it can benefit commercial applications in a more cost-effective way. Full article

Brain–computer interfaces (BCIs) are pivotal in translating neural activities into control commands for external assistive devices. Non-invasive techniques like electroencephalography (EEG) offer a balance of sensitivity and spatial-temporal resolution for capturing brain signals associated with motor activities. This work introduces MOVING, a Multi-Modal dataset of EEG signals and Virtual Glove Hand Tracking. This dataset comprises neural EEG signals and kinematic data associated with three hand movements—open/close, finger tapping, and wrist rotation—along with a rest period. The dataset, obtained from 11 subjects using a 32-channel dry wireless EEG system, also includes synchronized kinematic data captured by a Virtual Glove (VG) system equipped with two orthogonal Leap Motion Controllers. The use of these two devices allows for fast assembly (∼1 min), although introducing more noise than the gold standard devices for data acquisition. The study investigates which frequency bands in EEG signals are the most informative for motor task classification and the impact of baseline reduction on gesture recognition. Deep learning techniques, particularly EEGnetV4, are applied to analyze and classify movements based on the EEG data. This dataset aims to facilitate advances in BCI research and in the development of assistive devices for people with impaired hand mobility. This study contributes to the repository of EEG datasets, which is continuously increasing with data from other subjects, which is hoped to serve as benchmarks for new BCI approaches and applications. Full article

Virtual reality (VR) offers immersive visual and auditory experiences, transporting users to alternate realities. However, existing VR systems lack realistic haptic feedback mechanisms, resulting in unsatisfactory immersive experiences. In this study, we developed and tested a haptic glove that simulates realistic tactile sensations, enhancing user interaction with virtual artifacts. Our research investigates the impact of finger-specific vibrotactile feedback (FSVF) on user experience in virtual museum environments. Using a mixed-methods approach, 30 participants engaged in object-manipulation tasks in three settings: no haptic feedback, standard controller feedback, and vibrotactile glove feedback. The findings demonstrate that the vibrotactile glove approach considerably improves user accuracy, efficiency, immersion, and satisfaction compared with other traditional interaction methods. Participants completed tasks more accurately and quickly with the glove, reporting high levels of engagement and immersion. The results highlight the potential of advanced haptic feedback in transforming virtual reality technology, particularly for educational and cultural applications. Further, they provide valuable insights for designing and applying future haptic technology in immersive environments. Full article

In virtual reality (VR), a factor that can maximize user immersion is the development of an intuitive and sensory interaction method. Physical devices such as controllers or data gloves of existing VR devices are used to control the movement intentions of the user, but their shortfall is that grip strength and detailed muscle strength cannot be reflected. Therefore, this study intended to establish a more sensory VR environment compared to existing methods by reflecting the grip strength of the flexor digitorum profundus of the user of the VR content. In this experiment, the muscle activity of the flexor digitorum profundus was obtained from six subjects based on surface electromyography, and four objects with differing intensity were created within a VR program in which the objects were made to be destroyed depending on muscle activity. As a result, satisfaction was improved because the users could sensitively interact with the objects inside the VR environment, and the intended motion control of the user was reflected in the VR content. Full article