Search Results (632)

Despite rapid advances in robot learning, the coordination of closed-loop manipulation in cluttered environments remains a challenging and relatively underexplored problem. We present a novel two-level hierarchical architecture for a depth vision-equipped robotic arm that integrates pushing, grasping, and high-level decision making. Central to our approach is a prioritised action–selection mechanism that facilitates efficient early-stage learning via behaviour cloning (BC), while enabling scalable exploration through reinforcement learning (RL). A high-level decision neural network (DNN) selects between grasping and pushing actions, and two low-level action neural networks (ANNs) execute the selected primitive. The DNN is trained with RL, while the ANNs follow a hybrid learning scheme combining BC and RL. Notably, we introduce an automated demonstration generator based on oriented bounding boxes, eliminating the need for manual data collection and enabling precise, reproducible BC training signals. We evaluate our method on a challenging manipulation task involving five closely packed cubic objects. Our system achieves a completion rate (CR) of 100%, an average grasping success (AGS) of 93.1% per completion, and only 7.8 average decisions taken for completion (DTC). Comparative analysis against three baselines—a grasping-only policy, a fixed grasp-then-push sequence, and a cloned demonstration policy—highlights the necessity of dynamic decision making and the efficiency of our hierarchical design. In particular, the baselines yield lower AGS (86.6%) and higher DTC (10.6 and 11.4) scores, underscoring the advantages of content-aware, closed-loop control. These results demonstrate that our architecture supports robust, adaptive manipulation and scalable learning, offering a promising direction for autonomous skill coordination in complex environments. Full article

Ants use their mandibles—effectively a two-finger gripper—for a wide range of grasping activities. Here, we investigate whether mimicking the internal hairs found on ant mandibles can improve performance of a two-finger parallel plate robot gripper. With bin-picking applications in mind, the gripper fingers are long and slim, with interchangeable soft gripping pads that can be hairy or hairless. A total of 2400 video-documented experiments have been conducted, comparing hairless to hairy pads with different hair patterns. Simply by adding hairs, the grasp success rate was increased by at least $29\%$, and the number of objects that remain securely gripped during manipulation more than doubled. This result not only advances the state of the art in grasping technology, but also provides novel insight into the mechanical role of mandible hairs in ant biology. Full article

Sensing transparent objects has many applications in human daily life, including robot navigation and grasping. However, this task presents significant challenges due to the unpredictable nature of scenes that extend beyond/behind transparent objects, particularly the lack of fixed visual patterns and strong background interference. This paper aims to solve the transparent object segmentation problem by leveraging the intrinsic global modeling capabilities of transformer architectures. We design a Query Parsing Module (QPM) that innovatively formulates segmentation as a dictionary lookup problem, differing fundamentally from conventional pixel-wise mechanisms, e.g., via attention-based prototype matching, and a set of learnable class prototypes as query inputs. Based on QPM, we propose a high-performance transformer-based end-to-end segmentation model, Transparent Object Segmentation through Query (TOSQ). TOSQ’s encoder is based on the Segformer’s backbone, and its decoder consists of a series of QPM modules, which progressively refine segmentation masks by the proposed QPMs. TOSQ achieves state-of-the-art performance on the Trans10K-V2 dataset (76.63% mIoU, 95.34% Acc), with particularly significant gains in challenging categories like windows (+23.59%) and glass doors (+11.22%), demonstrating its superior capability in transparent object segmentation. Full article

Grasping objects with a high degree of anthropomorphism is a critical component in the field of highly anthropomorphic robotic grasping. However, the accuracy of contact maps and the irrationality of the grasping gesture become challenges for grasp generation. In this paper, we propose a reasonably improved generation scheme, called Diffangle-Grasp, consisting of two parts: contact map generation based on a conditional variational autoencoder (CVAE), sharing the potential space with the diffusion model, and optimized grasping generation, conforming to the physical laws and the natural pose. The experimental findings demonstrate that the proposed method effectively reduces the loss in contact map reconstruction by 9.59% in comparison with the base model. Additionally, it enhances the naturalness by 2.15%, elevates the success rate of grasping by 3.27%, reduces the penetration volume by 11.06%, and maintains the grasping simulation displacement. The comprehensive comparison and qualitative analysis with mainstream schemes also corroborate the rationality of the improvement. In this paper, we provide a comprehensive account of our contributions to enhancing the accuracy of contact maps and the naturalness of grasping gestures. We also offer a detailed technical feasibility analysis for robotic human grasping. Full article

Accurate segmentation of densely stacked and weakly textured objects remains a core challenge in robotic depalletizing for industrial applications. To address this, we propose MaskNet, an instance segmentation network tailored for RGB-D input, designed to enhance recognition performance under occlusion and low-texture conditions. Built upon a Vision Transformer backbone, MaskNet adopts a dual-branch architecture for RGB and depth modalities and integrates multi-modal features using an attention-based fusion module. Further, spatial and channel attention mechanisms are employed to refine feature representation and improve instance-level discrimination. The segmentation outputs are used in conjunction with regional depth to optimize the grasping sequence. Experimental evaluations on camshaft depalletizing tasks demonstrate that MaskNet achieves a precision of 0.980, a recall of 0.971, and an F1-score of 0.975, outperforming a YOLO11-based baseline. In an actual scenario, with a self-designed flexible magnetic gripper, the system maintains a maximum grasping error of 9.85 mm and a 98% task success rate across multiple camshaft types. These results validate the effectiveness of MaskNet in enabling fine-grained perception for robotic manipulation in cluttered, real-world scenarios. Full article

The commercialization of selective broccoli harvesters, a critical response to agricultural labor shortages, is hampered by end effectors with large operational envelopes and poor adaptability to complex field conditions. To address these limitations, this study developed and evaluated a novel end-effector with an integrated transverse cutting mechanism and a foldable grasping cavity. Unlike conventional fixed cylindrical cavities, our design utilizes actuated grasping arms and a mechanical linkage system to significantly reduce the operational footprint and enhance maneuverability. Key design parameters were optimized based on broccoli morphological data and experimental measurements of the maximum stem cutting force. Furthermore, dynamic simulations were employed to validate the operational trajectory and ensure interference-free motion. Field tests demonstrated an operational success rate of 93.33% and a cutting success rate of 92.86%. The end effector successfully operated in dense planting environments, effectively avoiding interference with adjacent broccoli heads. This research provides a robust and promising solution that advances the automation of broccoli harvesting, paving the way for the commercial adoption of robotic harvesting technologies. Full article

A mobile manipulation robot combines the navigation capability of unmanned ground vehicles and manipulation advantage of robotic arms. However, the development of a mobile manipulation robot is challenging due to the integration requirement of numerous heterogeneous subsystems. In this paper, we propose a multifunctional mobile manipulation robot by integrating perception, mapping, navigation, object detection, and grasping functions into a seamless workflow to conduct search-and-fetch tasks. To realize navigation and collision avoidance in complex environments, a new hierarchical motion planning strategy is proposed by fusing global and local planners. Control Lyapunov Function (CLF) and Control Barrier Function (CBF) are employed to realize path tracking and to guarantee safety during navigation. The convolutional neural network and the gripper’s kinematic constraints are adopted to construct a learning-optimization hybrid grasping algorithm to generate precise grasping poses. The efficiency of the developed mobile manipulation robot is demonstrated by performing indoor fetching experiments, showcasing its promising capabilities in real-world applications. Full article

This paper presents a Self-Learning Robotic System (SLRS) for healthcare assistance using Deep Imitation Learning (DIL). The proposed SLRS solution can observe and replicate human demonstrations, thereby acquiring complex skills without the need for explicit task-specific programming. It incorporates modular components for perception (i.e., advanced computer vision methodologies), actuation (i.e., dynamic interaction with patients and healthcare professionals in real time), and learning. The innovative approach of implementing a hybrid model approach (i.e., deep imitation learning and pose estimation algorithms) facilitates autonomous learning and adaptive task execution. The environmental awareness and responsiveness were also enhanced using both a Convolutional Neural Network (CNN)-based object detection mechanism using YOLOv8 (i.e., with 94.3% accuracy and 18.7 ms latency) and pose estimation algorithms, alongside a MediaPipe and Long Short-Term Memory (LSTM) framework for human action recognition. The developed solution was tested and validated in healthcare, with the aim to overcome some of the current challenges, such as workforce shortages, ageing populations, and the rising prevalence of chronic diseases. The CAD simulation, validation, and verification tested functions (i.e., assistive functions, interactive scenarios, and object manipulation) of the system demonstrated the robot’s adaptability and operational efficiency, achieving an 87.3% task completion success rate and over 85% grasp success rate. This approach highlights the potential use of an SLRS for healthcare assistance. Further work will be undertaken in hospitals, care homes, and rehabilitation centre environments to generate complete holistic datasets to confirm the system’s reliability and efficiency. Full article

The ocean encompasses the majority of the Earth’s surface and harbors substantial energy resources. Nevertheless, the intricate and asymmetrically distributed underwater environment renders existing target detection performance inadequate. This paper presents an enhanced YOLOv8s approach for underwater robot object detection to address issues of subpar image quality and low recognition accuracy. The precise measures are enumerated as follows: initially, to address the issue of model parameters, we optimized the ninth convolutional layer by substituting certain conventional convolutions with adaptive deformable convolution DCN v4. This modification aims to more effectively capture the deformation and intricate features of underwater targets, while simultaneously decreasing the parameter count and enhancing the model’s ability to manage the deformation challenges presented by underwater images. Furthermore, the Triplet Attention module is implemented to augment the model’s capacity for detecting multi-scale targets. The integration of low-level superficial features with high-level semantic features enhances the feature expression capability. The original CIoU loss function was ultimately substituted with Shape IoU, enhancing the model’s performance. In the underwater robot grasping experiment, the system shows particular robustness in handling radial symmetry in marine organisms and reflection symmetry in artificial structures. The enhanced algorithm attained a mean Average Precision (mAP) of 87.6%, surpassing the original YOLOv8s model by 3.4%, resulting in a marked enhancement of the object detection model’s performance and fulfilling the real-time detection criteria for underwater robots. Full article

Strawberry harvesting represents a labor-intensive agricultural operation where existing end-effector pose control algorithms frequently exhibit insufficient precision in fruit grasping, often resulting in unintended damage to target fruits. Concurrently, deep learning-based pose control algorithms suffer from inherent training instability, slow convergence rates, and inefficient learning processes in complex environments characterized by high-density fruit clusters and occluded picking scenarios. To address these challenges, this paper proposes an enhanced reinforcement learning framework DRFW-TQC that integrates Dynamic L2 Regularization for adaptive model stabilization and a Group-Wise Feature Weighting Network for discriminative feature representation. The methodology further incorporates a picking posture traction mechanism to optimize end-effector orientation control. The experimental results demonstrate the superior performance of DRFW-TQC compared to the baseline. The proposed approach achieves a 16.0% higher picking success rate and a 20.3% reduction in angular error with four target strawberries. Most notably, the framework’s transfer strategy effectively addresses the efficiency challenge in complex environments, maintaining an 89.1% success rate in eight-strawberry while reducing the timeout count by 60.2% compared to non-adaptive methods. These results confirm that DRFW-TQC successfully resolves the tripartite challenge of operational precision, training stability, and environmental adaptability in robotic fruit harvesting systems. Full article

This research study investigates the impact of a virtual dashboard on the quality of task execution in robotic teleoperation. More specifically, this study investigates how a virtual dashboard improves user awareness and grasp precision in a teleoperated pick-and-place task by providing users with critical information in real-time. An experiment was conducted with 30 participants in a robotic teleoperated task to measure their task performance in two different experimental conditions: a control group used conventional interfaces, and an experimental group utilized the virtual dashboard with additional information. Research findings indicate that integrating a virtual dashboard improves grasping accuracy, reduces user fatigue, and speeds up task completion, thereby improving task effectiveness and the quality of the experience. Full article

Multi-fingered dexterous hands provide superior dexterity in complex manipulation tasks due to their high degrees of freedom (DOFs) and biomimetic structures. Inspired by the anatomical structure of human tendons and muscles, numerous robotic hands powered by pneumatic artificial muscles (PAMs) have been created to replicate the compliant and adaptable features of biological hands. Nonetheless, PAMs have inherent nonlinear and hysteresis behaviors that create considerable challenges to achieving real-time control accuracy and stability in dexterous hands. In order to address these challenges, this paper proposes a hierarchical control architecture that employs a fuzzy PID strategy to optimize the nonlinear control of pneumatic artificial muscles (PAMs). The FPGA-based hardware integrates a multi-channel digital-to-analog converter (DAC) and a multiplexed acquisition module, facilitating the independent actuation of 20 PAMs and the real-time monitoring of 20 joints. The software implements a fuzzy PID algorithm that dynamically adjusts PID parameters based on both the error and the error rate, thereby effectively managing the nonlinear behaviors of the hand. Experimental results demonstrate that the designed control system achieves high precision in controlling the angle of a single finger joint, with errors maintained within ±1°. In scenarios involving multi-finger cooperative grasping and biomimetic motion demonstrations, the system exhibits excellent synchronization and real-time performance. These results validate the efficacy of the fuzzy PID control strategy and confirm that the proposed system fulfills the precision and stability requirements for complex operational tasks, providing robust support for the application of PAM-driven multi-fingered dexterous hands. Full article

Aiming to solve the problem that most existing apple-picking robots operate with a single arm and that the overall efficiency of the machine needs to be further improved, a prototype of a dual-arm picking robot was built, and its picking operation planning method was studied. Firstly, based on the configuration and motion mode of the AUBO-i5 robotic arm, the overlapping dual-arm layout of the workspace was determined. Then, a prototype of a dual-arm apple-picking robot was built, and, based on the designed dual-arm spatial layout, a dual-arm picking operation zoning planning method was proposed. The experimental results showed that in the four simulation experiments, the highest value of the maximum parallel operation proportion of the dual arms was 83%, and the lowest value was 50.6%. The highest value of the maximum operation length of the single arm was 7323 mm, and the lowest value was 5654 mm. The total length of the dual-arm operation path was 12,705 mm, and the lowest value was 8770 mm. Furthermore, a fruit-picking sequence planning method based on dual robotic arm operation was proposed. Fruit traversal simulation verification experiments were conducted. The results showed that there was no conflict between the left and right arms during the motion of the dual robotic arms. Finally, the proposed dual-arm robot operation zoning and picking sequence planning method was validated in the apple experimental station. The results showed that the proportion of dual-arm parallel operations was the lowest at 50.7% and the highest at 72.4%. The total length of the dual-arm operation path was the highest at 8604 mm and the lowest at 6511 mm. Full article

This study presents the design, fabrication, and experimental validation of a two-finger robotic gripper featuring a 135° V-shaped fingertip profile tailored for lightweight waste collection in laboratory-scale environmental robotics. The gripper was developed with a strong emphasis on cost-effectiveness and manufacturability, utilizing a desktop 3D printer and off-the-shelf servomotors. A four-bar linkage mechanism enables parallel jaw motion and ensures stable surface contact during grasping, achieving a maximum opening range of 71.5 mm to accommodate common cylindrical objects. To validate structural integrity, finite element analysis (FEA) was conducted under a 0.6 kg load, yielding a safety factor of 3.5 and a peak von Mises stress of 12.75 MPa—well below the material yield limit of PLA. Experimental testing demonstrated grasp success rates of up to 80 percent for typical waste items, including bottles, disposable cups, and plastic bags. While the gripper performs reliably with rigid and semi-rigid objects, further improvements are needed for handling highly deformable materials such as thin films or soft bags. The proposed design offers significant advantages in terms of rapid prototyping (a print time of approximately 10 h), modularity, and low manufacturing cost (with an estimated in-house material cost of USD 20 to 40). It provides a practical and accessible solution for small-scale robotic waste-collection tasks and serves as a foundation for future developments in affordable, application-specific grippers. Full article

Real-time computational capability for simultaneous grasping force and displacement determination constitutes a critical enabler for stable and reliable grasping performance in dexterous robotic grasping. To accelerate the computational efficiency of LCP in the dexterous grasping problem, as well as to ensure the stability and reproducibility of the algorithm’s output, the NSNMGK method, which incorporates sequential projection iterations across all greedy-selected active constraint rows within each NSNGRK framework iteration cycle, is developed. In each NSNMGK iteration, sequential projection operations are systematically applied to all active constraint rows, satisfying the greedy criterion. This processing strategy ensures the full utilization of qualifying constraints within the greedy subset through a same generalized Jacobian evaluation per iteration cycle. The methodology effectively mitigates inherent limitations of conventional randomized row selection, including unpredictable iteration counts and computational overhead from repeated Jacobian updates, while maintaining deterministic convergence behavior. The method’s convergence theory is rigorously established, with benchmark analyses demonstrating marked improvements in computational efficiency over the NSNGRK framework. Experimental validation in dexterous robotic grasping scenarios further confirms enhanced convergence rates through reduced iteration counts and shortened computational durations relative to existing approaches. Full article