Search Results (378)

Visual body-tracking techniques have seen widespread adoption in applications such as motion analysis, human–machine interaction, tele-robotics and extended reality (XR). These systems typically provide 2D landmark coordinates corresponding to key limb positions. However, to construct a meaningful 3D kinematic model for body joint reconstruction, a mapping must be established between these visual landmarks and the underlying joint parameters of individual body parts. This paper presents a method for constructing a 3D kinematic model of the human hand using calibrated 2D landmark-tracking data augmented with depth information. The proposed approach builds a hierarchical model in which the palm serves as the root coordinate frame, and finger landmarks are used to compute both forward and inverse kinematic solutions. Through step-by-step examples, we demonstrate how measured hand landmark coordinates are used to define the palm reference frame and solve for joint angles for each finger. These solutions are then used in a visualization framework to qualitatively assess the accuracy of the reconstructed hand motion. As a future work, the proposed model offers a foundation for model-based hand kinematic estimation and has utility in scenarios involving occlusion or missing data. In such cases, the hierarchical structure and kinematic solutions can be used as generative priors in an optimization framework to estimate unobserved landmark positions and joint configurations. The novelty of this work lies in its model-based approach using real sensor data, without relying on wearable devices or synthetic assumptions. Although current validation is qualitative, the framework provides a foundation for future robust estimation under occlusion or sensor noise. It may also serve as a generative prior for optimization-based methods and be quantitatively compared with joint measurements from wearable motion-capture systems. Full article

This paper delves into the potential of an optimized differential system within an underactuated tendon-driven soft robotic gripper, a crucial component that enhances the grasping abilities by allowing fingers to secure objects adapting to different shapes and geometries. The original version of the differential system exhibited a certain degree of deformability, which introduced some functional advantages. In particular, its flexibility allowed for more delicate grasping operations by acting as a force reducer and enabling a more gradual application of contact forces, an essential feature when handling fragile objects. Nonetheless, while these benefits are noteworthy, a rigid differential remains more effective for achieving firm and secure grasps. The primary goal of this study is to analyze the differential’s performance through FEM simulations and deformation experiments, assessing its structural behavior under various conditions. Additionally, the research explores an innovative differential geometry aimed at striking the ideal balance, ensuring a robust grasp while retaining a controlled degree of deformability. By refining the differential’s design, this study seeks to enhance the efficiency of underactuated soft robotic grippers, ultimately enhancing their capabilities in handling diverse objects ensuring a compliant and secure grasp with optimized efficiency. Full article

This study presents a novel drop-off strategy for automated fabric handling in intelligent apparel manufacturing, addressing the critical challenge of drift-free placement of lightweight, flexible textiles. A pneumatically driven retractable plate is introduced as an auxiliary device, along with machine vision technology, to eliminate drop-off deviations inherent in traditional soft grippers. By synchronizing the retraction motion of the plate with soft gripper release, the fabric is transferred onto the target surface without free-fall drift, achieving sub-0.5 mm alignment accuracy across 15 fabric types. Machine vision-based inspection validates drop-off quality in real time. This work offers a low-cost, drift-free drop-off solution for pre-sewing automation. Full article

Robotic assembly of electrical connectors enables the automation of high-efficiency production of electronic products. A rigid gripper is adopted as the end-effector by the majority of existing works with a force–torque sensor installed at the wrist, which suffers from very limited perception capability of the manipulated objects. Moreover, the grasping and movement actions, as well as the inconsistency between the robot base and the end-effector frame, tend to result in angular misalignment, usually leading to assembly failure. Bio-inspired by the human finger, we designed a tactile finger in this paper with three characteristics: (1) Compliance: A soft ‘skin’ layer provides passive compliance for plenty of manipulation actions, thus increasing the tolerance for alignment errors. (2) Tactile Perception: Two types of sensing elements are embedded into the soft skin to tactilely sense the involved contact status. (3) Enhanced manipulation force: A rigid fingernail is designed to enhance the manipulation force and enable potential delicate operations. Moreover, a tactile-based alignment algorithm is proposed to search for the optimal orientation angle about the z axis. In the application of U-disk insertion, the three characteristics are validated and a success rate of 100% is achieved, whose generalization capability is also validated through the assembly of three types of electrical connectors. Full article

Ants use their mandibles—effectively a two-finger gripper—for a wide range of grasping activities. Here, we investigate whether mimicking the internal hairs found on ant mandibles can improve performance of a two-finger parallel plate robot gripper. With bin-picking applications in mind, the gripper fingers are long and slim, with interchangeable soft gripping pads that can be hairy or hairless. A total of 2400 video-documented experiments have been conducted, comparing hairless to hairy pads with different hair patterns. Simply by adding hairs, the grasp success rate was increased by at least $29\%$, and the number of objects that remain securely gripped during manipulation more than doubled. This result not only advances the state of the art in grasping technology, but also provides novel insight into the mechanical role of mandible hairs in ant biology. Full article

Soft continuum robots are constructed from soft and compliant materials and can provide high flexibility and adaptability to various applications. They have theoretically infinite degrees of freedom (DOFs) and can generate highly nonlinear behaviors, which leads to challenges in accurately modeling and controlling their deformation, compliance, and behaviors. Inspired by animals, embodied intelligence utilizes physical bodies as an intelligent resource for information processing and task completion and offloads the computational cost of central control, which provides a unique approach to understanding and modeling soft robotics. In this study, we propose a theoretical framework to explain and guide distributed sensing enabled embodied intelligence for soft finger manipulation from a physics-based perspective. Specifically, we aim to provide a theoretical foundation to guide future sensor design and placement by addressing two key questions: (1) whether and why the state of a specific material point such as the tip trajectory of a soft finger can be predicted using distributed sensing, and, (2) how many sensors are sufficient for accurate prediction. These questions are critical for the design of soft and compliant robotic systems with embedded sensing for embodied intelligence. In addition to theoretical analysis, the study presents a feasible approach for real-time trajectory prediction through optimized sensor placement, with results validated through both simulation and experiment. The results showed that the tip trajectory of a soft finger can be predicted with a finite number of sensors with proper placement. While the proposed method is demonstrated in the context of soft finger manipulation, the framework is theoretically generalizable to other compliant soft robotic systems. Full article

Assistive technologies, particularly multi-fingered robotic hands (MFRHs), are critical for enhancing the quality of life for individuals with upper-limb disabilities. However, achieving precise and stable control of such systems remains a significant challenge. This study proposes an Improved Grey Wolf Optimization (IGWO)-tuned Linear Quadratic Regulator (LQR) to enhance the control performance of an MFRH. The MFRH was modeled using Denavit–Hartenberg kinematics and Euler–Lagrange dynamics, with micro-DC motors selected based on computed torque requirements. The LQR controller, optimized via IGWO to systematically determine weighting matrices, was benchmarked against PID and PID-PSO controllers under diverse input scenarios. For step input, the IGWO-LQR achieved a settling time of 0.018 s with zero overshoot for Joint 1, outperforming PID (settling time: 0.0721 s; overshoot: 6.58%) and PID-PSO (settling time: 0.042 s; overshoot: 2.1%). Similar improvements were observed across all joints, with Joint 3 recording an IAE of 0.001334 for IGWO-LQR versus 0.004695 for PID. Evaluations under square-wave, sine, and sigmoid inputs further validated the controller’s robustness, with IGWO-LQR consistently delivering minimal tracking errors and rapid stabilization. These results demonstrate that the IGWO-LQR framework significantly enhances precision and dynamic response. Full article

Wearable sensors have rapidly advanced, enabling applications such as human activity monitoring, electronic skin, and biomimetic robotics. To meet the growing demands of these applications, multifunctional sensing has become essential for wearable devices. However, most existing studies predominantly focus on enhancing single-function sensing capabilities. This study introduces a multifunctional sensor that combines high stretchability for strain and pressure detection with ultraviolet (UV) sensing capability. To achieve simultaneous detection of strain, pressure, and UV light, a multi-sensing approach was employed: a capacitive method for strain and pressure detections and a resistive method utilizing a pn-heterojunction diode for UV detection. In the capacitive method, polyaniline (PANI) served as parallel-plate electrodes, while silicon-based elastomer acted as the dielectric layer. This configuration enabled up to 100% elongation and enhanced operational stability through encapsulation. The sensor demonstrated a strong linear relationship between capacitance value changes reasonably based on the area of PANI, and showed a good linearity with an R-squared value of 0.9918. It also detected pressure across a wide range, from low (0.4 kPa) to high (9.4 kPa). Furthermore, for wearable applications, the sensor reliably captured capacitance variations during finger bending at different angles. For UV detection, a pn-heterojunction diode composed of p-type silicon and n-type zinc oxide nanorods exhibited a rapid response time of 6.1 s and an on/off ratio of 13.8 at −10 V. Durability under 100% tensile strain was confirmed through Von Mises stress calculations using finite element modeling. Overall, this multifunctional sensor offers significant potential for a variety of applications, including human motion detection, wearable technology, and robotics. Full article

Multi-fingered dexterous hands provide superior dexterity in complex manipulation tasks due to their high degrees of freedom (DOFs) and biomimetic structures. Inspired by the anatomical structure of human tendons and muscles, numerous robotic hands powered by pneumatic artificial muscles (PAMs) have been created to replicate the compliant and adaptable features of biological hands. Nonetheless, PAMs have inherent nonlinear and hysteresis behaviors that create considerable challenges to achieving real-time control accuracy and stability in dexterous hands. In order to address these challenges, this paper proposes a hierarchical control architecture that employs a fuzzy PID strategy to optimize the nonlinear control of pneumatic artificial muscles (PAMs). The FPGA-based hardware integrates a multi-channel digital-to-analog converter (DAC) and a multiplexed acquisition module, facilitating the independent actuation of 20 PAMs and the real-time monitoring of 20 joints. The software implements a fuzzy PID algorithm that dynamically adjusts PID parameters based on both the error and the error rate, thereby effectively managing the nonlinear behaviors of the hand. Experimental results demonstrate that the designed control system achieves high precision in controlling the angle of a single finger joint, with errors maintained within ±1°. In scenarios involving multi-finger cooperative grasping and biomimetic motion demonstrations, the system exhibits excellent synchronization and real-time performance. These results validate the efficacy of the fuzzy PID control strategy and confirm that the proposed system fulfills the precision and stability requirements for complex operational tasks, providing robust support for the application of PAM-driven multi-fingered dexterous hands. Full article

The widespread use of WiFi Internet-of-Things (IoT) devices has rendered them valuable tools for detecting information about the physical environment. Recent studies have demonstrated that WiFi Channel State Information (CSI) can detect physical events like movement, occupancy increases, and gestures. This paper empirically investigates the conditions under which WiFi sensing technology remains effective for keystroke detection. To achieve this timely goal of assessing whether it can raise any privacy concerns, experiments are conducted using commodity hardware to predict the accuracy of WiFi CSI in detecting keys pressed on a keyboard. Our novel results show that, in an ideal setting with a robotic arm, the position of a specific key can be predicted with $99\%$ accuracy using a simple machine learning classifier. Furthermore, human finger localisation over a key and actual key-press recognition is also successfully achieved, with $94\%$ and $89\%$ reduced accuracy values, respectively. Moreover, our detailed investigation reveals that to ensure high accuracy, the gap distance between each test object must be substantial, while the size of the test group should be limited. Finally, we show WiFi sensing technology has limitations in small-scale gesture recognition for generic settings where proper device positioning is crucial. Specifically, detecting keyed words achieves an overall accuracy of $94\%$ for the forefinger and $87\%$ for multiple fingers when only the right hand is used. Accuracy drops to $56\%$ when using both hands. We conclude WiFi sensing is effective in controlled indoor environments, but it has limitations due to the device location and the limited granularity of sensing objects. Full article

This study presents the design, fabrication, and experimental validation of a two-finger robotic gripper featuring a 135° V-shaped fingertip profile tailored for lightweight waste collection in laboratory-scale environmental robotics. The gripper was developed with a strong emphasis on cost-effectiveness and manufacturability, utilizing a desktop 3D printer and off-the-shelf servomotors. A four-bar linkage mechanism enables parallel jaw motion and ensures stable surface contact during grasping, achieving a maximum opening range of 71.5 mm to accommodate common cylindrical objects. To validate structural integrity, finite element analysis (FEA) was conducted under a 0.6 kg load, yielding a safety factor of 3.5 and a peak von Mises stress of 12.75 MPa—well below the material yield limit of PLA. Experimental testing demonstrated grasp success rates of up to 80 percent for typical waste items, including bottles, disposable cups, and plastic bags. While the gripper performs reliably with rigid and semi-rigid objects, further improvements are needed for handling highly deformable materials such as thin films or soft bags. The proposed design offers significant advantages in terms of rapid prototyping (a print time of approximately 10 h), modularity, and low manufacturing cost (with an estimated in-house material cost of USD 20 to 40). It provides a practical and accessible solution for small-scale robotic waste-collection tasks and serves as a foundation for future developments in affordable, application-specific grippers. Full article

This paper presents the design and evaluation of a novel soft pneumatic actuator featuring two independent bending chambers, enabling independent joint actuation and localization for rehabilitation purposes. The actuator’s dual-chamber configuration provides flexibility for applications requiring customized bending profiles. To measure the bending angle of the finger joints in real time, a camera-based system is employed, utilizing a deep learning detection model to localize the joints and estimate their bending angles. This approach provides a non-intrusive, sensor-free alternative to hardware-based measurement methods, reducing complexity and wiring typically associated with wearable devices. Experimental results demonstrate the effectiveness of the proposed actuator in achieving bending angles of 105 degrees for the metacarpophalangeal (MCP) joint and 95 degrees for the proximal interphalangeal (PIP) joint, as well as a gripping force of 9.3 N. The vision system also captures bending angles with a precision of 98%, indicating potential applications in fields such as rehabilitation and human–robot interaction. Full article

Glove-based human–machine interfaces (HMIs) offer a natural, intuitive way to capture finger motions for gesture recognition, virtual interaction, and robotic control. However, many existing systems suffer from complex fabrication, limited sensitivity, and reliance on external power. Here, we present a flexible, self-powered glove HMI based on a minimalist triboelectric nanogenerator (TENG) sensor composed of a conductive fabric electrode and textured Ecoflex layer. Surface micro-structuring via 3D-printed molds enhances triboelectric performance without added complexity, achieving a peak power density of 75.02 μW/cm² and stable operation over 13,000 cycles. The glove system enables real-time LED brightness control via finger-bending kinematics and supports intelligent recognition applications. A convolutional neural network (CNN) achieves 99.2% accuracy in user identification and 97.0% in object classification. By combining energy autonomy, mechanical simplicity, and machine learning capabilities, this work advances scalable, multi-functional HMIs for applications in assistive robotics, augmented reality (AR)/(virtual reality) VR environments, and secure interactive systems. Full article

It is great challenge for visually impaired (VI) people to shop in narrow and crowded farmers’ markets. However, there is no research related to guiding them in farmers’ markets worldwide. This paper proposes the Radio-Frequency–Visual Tag Positioning and Automatic Detection (RFTPAD) algorithm to quickly build a high-precision navigation map. It combines the advantages of visual beacons and radio-frequency signal beacons to accurately calculate the guide robot’s coordinates to correct its positioning error and simultaneously perform the task of mapping and detecting information. Furthermore, this paper proposes the A*-Fixed-Route Navigation (A*-FRN) algorithm, which controls the robot to navigate along fixed routes and prevents it from making frequent detours in crowded aisles. Finally, this study equips the guide robot with a flexible robotic arm and proposes the Intelligent-Robotic-Arm-Guided Shopping (IRAGS) algorithm to guide VI people to quickly select fresh products or guide merchants to pack and weigh products. Multiple experiments conducted in a 1600 m² market demonstrate that compared with the classic mapping method, the accuracy of RFTPAD is improved by 23.9%. What is more, compared with the general navigation method, the driving trajectory length of A*-FRN is 23.3% less. Furthermore, the efficiency of guiding VI people to select products by a robotic arm is 100% higher than that through a finger to search and touch. Full article

Gestures play an integral role in human communication. Our research aims to develop a gesture understanding system that allows for better interpretation of human instructions in household robotics settings. We conducted an experiment with 34 participants who used pointing gestures to teach concepts to an assistant. Gesture data were analyzed using manual annotations (MAXQDA) and the computational methods of pose estimation and k-means clustering. The study revealed that participants tend to maintain consistent pointing styles, with one-handed pointing and index finger gestures being the most common. Gaze and pointing often co-occur, as do leaning forward and pointing. Using our gesture categorization algorithm, we analyzed gesture information values. As the experiment progressed, the information value of gestures remained stable, although the trends varied between participants and were associated with factors such as age and gender. These findings underscore the need for gesture recognition systems to balance generalization with personalization for more effective human–robot interaction. Full article