Search Results (18)

With the rapid development of smart manufacturing, the optimization of real-time monitoring in operating procedures has become a crucial issue in modern industry. Traditional disassembly and assembly (D/A) work, relying on human experience and visual inspection, lacks immediacy and a quantitative basis, further affecting operating quality and efficiency. This study aims to develop a thin-film force sensor and an inertial measurement unit (IMU)-integrated wearable device for monitoring and analyzing operators’ behavioral characteristics during D/A tasks. First, by having operators wear self-made smart gloves and 17 IMU sensors, the work tables with three different heights are equipped with a mechatronics machine for the D/A experiment. Common D/A motions are designed into the experiment. Several subjects are invited to execute the standardized operating procedure, with upper limbs used to collect data on operators’ hand gestures and movements. Then, the measured data are applied to verify the performance measure functional best path of machine D/A. The results reveal that the system could effectively identify various D/A motions as well as observe operators’ force difference and motion mode, which, through the theory of performance indicator optimization and the verification of data analysis, could provide a reference for the best path planning, D/A sequence, and work table height design in the machine D/A process. The optimal workbench height for a standing operator is 5 to 10 cm above their elbow height. Performing assembly and disassembly tasks at this optimal height can help the operator save between 14.3933% and 35.2579% of physical effort. Such outcomes could aid in D/A behavior monitoring in industry, worker training, and operational optimization, as well as expand the application to instant feedback design for automation and smartization in a smart factory. Full article

This study presents a novel investigation into immersive teleoperation systems using collaborative, device-agnostic interfaces for advancing smart haptics in industrial assistive applications. The research focuses on evaluating the quality of experience (QoE) of users interacting with a teleoperation system comprising a local robotic arm, a robot gripper, and heterogeneous remote tracking and haptic feedback devices. By employing a modular device-agnostic framework, the system supports flexible configurations, including one-user-one-equipment (1U-1E), one-user-multiple-equipment (1U-ME), and multiple-users-multiple-equipment (MU-ME) scenarios. The experimental set-up involves participants manipulating predefined objects and placing them into designated baskets by following specified 3D trajectories. Performance is measured using objective QoE metrics, including temporal efficiency (time required to complete the task) and spatial accuracy (trajectory similarity to the predefined path). In addition, subjective QoE metrics are assessed through detailed surveys, capturing user perceptions of presence, engagement, control, sensory integration, and cognitive load. To ensure flexibility and scalability, the system integrates various haptic configurations, including (1) a Touch kinaesthetic device for precision tracking and grounded haptic feedback, (2) a DualSense tactile joystick as both a tracker and mobile haptic device, (3) a bHaptics DK2 vibrotactile glove with a camera tracker, and (4) a SenseGlove Nova force-feedback glove with VIVE trackers. The modular approach enables comparative analysis of how different device configurations influence user performance and experience. The results indicate that the objective QoE metrics varied significantly across device configurations, with the Touch and SenseGlove Nova set-ups providing the highest trajectory similarity and temporal efficiency. Subjective assessments revealed a strong correlation between presence and sensory integration, with users reporting higher engagement and control in scenarios utilizing force feedback mechanisms. Cognitive load varied across the set-ups, with more complex configurations (e.g., 1U-ME) requiring longer adaptation periods. This study contributes to the field by demonstrating the feasibility of a device-agnostic teleoperation framework for immersive industrial applications. It underscores the critical interplay between objective task performance and subjective user experience, providing actionable insights into the design of next-generation teleoperation systems. Full article

Dexterous robotic grasping with multifingered hands remains a critical challenge in non-visual environments, where diverse object geometries and material properties demand adaptive force modulation and tactile-aware manipulation. To address this, we propose the Reinforcement Learning-Based Multimodal Perception (RLMP) framework, which integrates human-like grasping intuition through operator-worn gloves with tactile-guided reinforcement learning. The framework’s key innovation lies in its Tactile-Driven DCNN architecture—a lightweight convolutional network achieving 98.5% object recognition accuracy using spatiotemporal pressure patterns—coupled with an RL policy refinement mechanism that dynamically correlates finger kinematics with real-time tactile feedback. Experimental results demonstrate reliable grasping performance across deformable and rigid objects while maintaining force precision critical for fragile targets. By bridging human teleoperation with autonomous tactile adaptation, RLMP eliminates dependency on visual input and predefined object models, establishing a new paradigm for robotic dexterity in occlusion-rich scenarios. Full article

Immersive human–machine interaction relies on comprehensive sensing and feedback systems, which enable transmission of multiple pieces of information. However, the integration of increasing numbers of feedback actuators and sensors causes a severe issue in terms of system complexity. In this work, we propose a pressure-sensing and feedback glove that enables multi-dimensional pressure sensing and feedback with a minimalist design of the functional units. The proposed glove consists of modular strain and pressure sensors based on films of liquid metal microchannels and coin vibrators. Strain sensors located at the finger joints can simultaneously project the bending motion of the individual joint into the virtual space or robotic hand. For subsequent tactile interactions, the design of two symmetrically distributed pressure sensors and vibrators at the fingertips possesses capabilities for multi-directional pressure sensing and feedback by evaluating the relationship of the signal variations between two sensors and tuning the feedback intensities of two vibrators. Consequently, both dynamic and static multi-dimensional pressure communication can be realized, and the vibrational actuation can be monitored by a liquid-metal-based sensor via a triboelectric sensing mechanism. A demonstration of object interaction indicates that the proposed glove can effectively detect dynamic force in varied directions at the fingertip while offering the reconstruction of a similar perception via the haptic feedback function. This device introduces an approach that adopts a minimalist design to achieve a multi-functional system, and it can benefit commercial applications in a more cost-effective way. Full article

Manipulation involves both fine tactile feedback, with dynamic transients perceived by fingerpad mechanoreceptors, and kinaesthetic force feedback, involving the whole hand musculoskeletal structure. In teleoperation experiments, these fundamental aspects are usually divided between different setups at the operator side: those making use of lightweight gloves and optical tracking systems, oriented toward tactile-only feedback, and those implementing exoskeletons or grounded manipulators as haptic devices delivering kinaesthetic force feedback. At the level of hand interfaces, exoskeletons providing kinaesthetic force feedback undergo a trade-off between maximum rendered forces and bandpass of the embedded actuators, making these systems unable to properly render tactile feedback. To overcome these limitations, here, we investigate a full upper limb exoskeleton, covering all the upper limb body segments from shoulder to finger phalanxes, coupled with linear voice coil actuators at the fingertips. These are developed to render wide-bandwidth tactile feedback together with the kinaesthetic force feedback provided by the hand exoskeleton. We investigate the system in a pick-and-place teleoperation task, under two different feedback conditions (visual-only and visuo-haptic). The performance based on measured interaction forces and the number of correct trials are evaluated and compared. The study demonstrates the overall feasibility and effectiveness of a complex full upper limb exoskeleton (seven limb-actuated DoFs plus five hand DoFs) capable of combined kinaesthetic and tactile haptic feedback. Quantitative results show significant performance improvements when haptic feedback is provided, in particular for the mean and peak exerted forces, and for the correct rate of the pick-and-place task. Full article

Recently, VR-based training applications have become popular and promising, as they can simulate real-world situations in a safe, repeatable, and cost-effective way. For immersive simulations, various input devices have been designed and proposed to increase the effectiveness of training. In this study, we developed a novel device that generates 3D hand motion data and provides haptic force feedback for VR interactions. The proposed device can track 3D hand positions using a combination of the global position estimation of ultrasonic sensors and the hand pose estimation of inertial sensors in real time. For haptic feedback, shape–memory alloy (SMA) actuators were designed to provide kinesthetic forces and an efficient power control without an overheat problem. Our device improves upon the shortcomings of existing commercial devices in tracking and haptic capabilities such that it can track global 3D positions and estimate hand poses in a VR space without using an external suit or tracker. For better flexibility in handling and feeling physical objects compared to exoskeleton-based devices, we introduced an SMA-based actuator to control haptic forces. Overall, our device was designed and implemented as a lighter and less bulky glove which provides comparable accuracy and performance in generating 3D hand motion data for a VR training application (i.e., the use of a fire extinguisher), as demonstrated in the experimental results. Full article

Human–robot interaction is of the utmost importance as it enables seamless collaboration and communication between humans and robots, leading to enhanced productivity and efficiency. It involves gathering data from humans, transmitting the data to a robot for execution, and providing feedback to the human. To perform complex tasks, such as robotic grasping and manipulation, which require both human intelligence and robotic capabilities, effective interaction modes are required. To address this issue, we use a wearable glove to collect relevant data from a human demonstrator for improved human–robot interaction. Accelerometer, pressure, and flexi sensors were embedded in the wearable glove to measure motion and force information for handling objects of different sizes, materials, and conditions. A machine learning algorithm is proposed to recognize grasp orientation and position, based on the multi-sensor fusion method. Full article

In this study, we developed a new haptic–mixed reality intravenous (HMR-IV) needle insertion simulation system, providing a bimanual haptic interface integrated into a mixed reality system with programmable variabilities considering real clinical environments. The system was designed for nursing students or healthcare professionals to practice IV needle insertion into a virtual arm with unlimited attempts under various changing insertion conditions (e.g., skin: color, texture, stiffness, friction; vein: size, shape, location depth, stiffness, friction). To achieve accurate hand–eye coordination under dynamic mixed reality scenarios, two different haptic devices (Dexmo and Geomagic Touch) and a standalone mixed reality system (HoloLens 2) were integrated and synchronized through multistep calibration for different coordinate systems (real world, virtual world, mixed reality world, haptic interface world, HoloLens camera). In addition, force-profile-based haptic rendering proposed in this study was able to successfully mimic the real tactile feeling of IV needle insertion. Further, a global hand-tracking method, combining two depth sensors (HoloLens and Leap Motion), was developed to accurately track a haptic glove and simulate grasping a virtual hand with force feedback. We conducted an evaluation study with 20 participants (9 experts and 11 novices) to measure the usability of the HMR-IV simulation system with user performance under various insertion conditions. The quantitative results from our own metric and qualitative results from the NASA Task Load Index demonstrate the usability of our system. Full article

Despite the benefits of learning an instrument, many students drop out early because it can be frustrating for the student, expensive for the caregiver, and loud for the household. Virtual Reality (VR) and Extended Reality (XR) offer the potential to address these challenges by simulating multiple instruments in an engaging and motivating environment through headphones. To assess the potential for commercial VR to augment musical experiences, we used standard VR implementation processes to design four virtual trumpet interfaces: camera-tracking with tracked register selection (two ways), camera-tracking with voice activation, and a controller plus a force-feedback haptic glove. To evaluate these implementations, we created a virtual music classroom that produces audio, notes, and finger pattern guides loaded from a selected Musical Instrument Digital Interface (MIDI) file. We analytically compared these implementations against physical trumpets (both acoustic and MIDI), considering features of ease of use, familiarity, playability, noise, and versatility. The physical trumpets produced the most reliable and familiar experience, and some XR benefits were considered. The camera-based methods were easy to use but lacked tactile feedback. The haptic glove provided improved tracking accuracy and haptic feedback over camera-based methods. Each method was also considered as a proof-of-concept for other instruments, real or imaginary. Full article

The wearable force feedback glove provides a promising solution for enhancing immersion during teleoperation. In this study, a lightweight five-finger exoskeleton force feedback glove (EFFG) was designed, enabling driving force detection and flexible force feedback. This wireless prototype weighs only 278 g. The glove features a bionic structure and optimized linkage length to ensure operator safety while providing extensive coverage of the finger working space. Moreover, a detailed illustration of the kinematic and dynamic analyses, as well as the circuit structure, was presented. With this prototype as the basis, an isomorphic teleoperation system is designed to achieve force feedback during teleoperation. Concurrently, a driving force-based impedance controller was proposed to enable smooth and precise force feedback. Finally, the performance of the EFFG prototype was evaluated in both unconstrained and constrained environments, demonstrating that the proposed glove is lightweight, capable of detecting driving force, and provides flexible force feedback. Full article

Many diseases, such as stroke, arthritis, and spinal cord injury, can cause severe hand impairment. Treatment options for these patients are limited by expensive hand rehabilitation devices and dull treatment procedures. In this study, we present an inexpensive soft robotic glove for hand rehabilitation in virtual reality (VR). Fifteen inertial measurement units are placed on the glove for finger motion tracking, and a motor—tendon actuation system is mounted onto the arm and exerts forces on fingertips via finger-anchoring points, providing force feedback to fingers so that the users can feel the force of a virtual object. A static threshold correction and complementary filter are used to calculate the finger attitude angles, hence computing the postures of five fingers simultaneously. Both static and dynamic tests are performed to validate the accuracy of the finger-motion-tracking algorithm. A field-oriented-control-based angular closed-loop torque control algorithm is adopted to control the force applied to the fingers. It is found that each motor can provide a maximum force of 3.14 N within the tested current limit. Finally, we present an application of the haptic glove in a Unity-based VR interface to provide the operator with haptic feedback while squeezing a soft virtual ball. Full article

During open surgery, a surgeon relies not only on the detailed view of the organ being operated upon and on being able to feel the fine details of this organ but also heavily relies on the combination of these two senses. In laparoscopic surgery, haptic feedback provides surgeons information on interaction forces between instrument and tissue. There have been many studies to mimic the haptic feedback in laparoscopic-related telerobotics studies to date. However, cutaneous feedback is mostly restricted or limited in haptic feedback-based minimally invasive studies. We argue that fine-grained information is needed in laparoscopic surgeries to study the details of the instrument’s end and can convey via cutaneous feedback. We propose an exoskeleton haptic hand wearable which consists of five 4 × 4 miniaturized fingertip actuators, 80 in total, to convey cutaneous feedback. The wearable is described as modular, lightweight, Bluetooth, and WiFi-enabled, and has a maximum power consumption of 830 mW. Software is developed to demonstrate rapid tactile actuation of edges; this allows the user to feel the contours in cutaneous feedback. Moreover, to demonstrate the idea as an object displayed on a flat monitor, initial tests were carried out in 2D. In the second phase, the wearable exoskeleton glove is then further developed to feel 3D virtual objects by using a virtual reality (VR) headset demonstrated by a VR environment. Two-dimensional and 3D objects were tested by our novel untethered haptic hand wearable. Our results show that untethered humans understand actuation in cutaneous feedback just in a single tapping with 92.22% accuracy. Our wearable has an average latency of 46.5 ms, which is much less than the 600 ms tolerable delay acceptable by a surgeon in teleoperation. Therefore, we suggest our untethered hand wearable to enhance multimodal perception in minimally invasive surgeries to naturally feel the immediate environments of the instruments. Full article

Haptic technology allows us to experience tactile and force sensations without the need to expose ourselves to specific environments. It also allows a more immersive experience with virtual reality devices. This paper presents the development of a soft haptic glove for kinesthetic perception. It is lightweight and soft to allow for a more natural hand movement. This prototype actuates two fingers with two shape memory alloy (SMA) springs. Finite element (FE) simulations of the spring have been carried out to set the dimensions of the actuators. Flexible stretch sensors provide feedback to the system to calculate the tension of the cables attached to the fingers. The control can generate several recognizable levels of force for any hand position since the objects to be picked up can vary in weight and dimension. The glove can generate three levels of force (100, 200 and 300 g) to evaluate more easily the proper functioning. We realized tests on 15 volunteers simulating forces in various order after a quick training. We also asked volunteers about the experience for comfort, global experience and simplicity). Results were satisfactory in both aspects: the glove fulfilled its function, and the users were comfortable with it. Full article

Force feedback gloves allow users to touch and manipulate virtual objects intuitively. Compared with gloves providing active feedback force, gloves with passive feedback force are promising in terms of safety and low weight, but simulating the variable stiffness of virtual objects is more challenging. Addressing this difficulty, we propose a five-fingered glove with passive force feedback employing a variable ratio lever mechanism. The stiffness of the proposed glove is tuned by changing the structural stiffness of this mechanism rather than by applying torque control at each joint of the finger. The switch between free and constrained space is realized in real time by locking/unlocking the revolute joints of the glove using a servo motor. Furthermore, a predictive control mode is proposed to reduce the response time of the control system, and the actual response time is less than the limit of the delay (45 ms) that humans can perceive between visual and haptic stimuli. Experimental results show that the linear stiffness at the fingertip ranges from 0.89 to 619.89 N/m, and the maximum backdrive force of the proposed glove is less than 0.147 N. Full article

Sensory feedback from wearables can be effective to learn better movement through enhanced information and engagement. Facilitating greater user cognition during movement practice is critical to accelerate gains in motor function during rehabilitation following brain or spinal cord trauma. This preliminary study presents an approach using an instrumented glove to leverage sense of agency, or perception of control, to provide training feedback for functional grasp. Seventeen able-bodied subjects underwent training and testing with a custom-built sensor glove prototype from our laboratory. The glove utilizes onboard force and flex sensors to provide inputs to an artificial neural network that predicts achievement of “secure” grasp. Onboard visual and audio feedback was provided during training with progressively shorter time delay to induce greater agency by intentional binding, or perceived compression in time between an action (grasp) and sensory consequence (feedback). After training, subjects demonstrated a significant reduction (p < 0.05) in movement pathlength and completion time for a functional task involving grasp-move-place of a small object. Future work will include a model-based algorithm to compute secure grasp, virtual reality immersion, and testing with clinical populations. Full article