Search Results (14)

This survey provides a comprehensive review of the integration of large language models (LLMs) into autonomous robotic systems, organized around four key pillars: locomotion, navigation, manipulation, and voice-based interaction. We examine how LLMs enhance robotic autonomy by translating high-level natural language commands into low-level control signals, supporting semantic planning and enabling adaptive execution. Systems like SayTap improve gait stability through LLM-generated contact patterns, while TrustNavGPT achieves a 5.7% word error rate (WER) under noisy voice-guided conditions by modeling user uncertainty. Frameworks such as MapGPT, LLM-Planner, and 3D-LOTUS++ integrate multi-modal data—including vision, speech, and proprioception—for robust planning and real-time recovery. We also highlight the use of physics-informed neural networks (PINNs) to model object deformation and support precision in contact-rich manipulation tasks. To bridge the gap between simulation and real-world deployment, we synthesize best practices from benchmark datasets (e.g., RH20T, Open X-Embodiment) and training pipelines designed for one-shot imitation learning and cross-embodiment generalization. Additionally, we analyze deployment trade-offs across cloud, edge, and hybrid architectures, emphasizing latency, scalability, and privacy. The survey concludes with a multi-dimensional taxonomy and cross-domain synthesis, offering design insights and future directions for building intelligent, human-aligned robotic systems powered by LLMs. Full article

Droplet spreading is a fascinating phenomenon. Especially when the droplet spreads, reacts, and dissolves on and into metal substrates. This reactive wetting mainly occurs at high temperatures, with a vast number of applications in industry and material science. It is common to monitor and study the process using a side-view projection of the droplet, focusing on the dynamics and shape of its contact line. However, when the spreading is monitored top-view, rich and non-trivial spatio-temporal patterns are revealed during different stages of the process. These patterns call for a different type of study of the perimeter of the entire droplet. Statistical physics is the natural candidate to perform such tasks, using tools developed for the study of kinetic roughening of advancing interfaces. In this review, we demonstrate the use of these tools, the growth, roughness, and persistence exponents, to study the spreading of mercury droplets on metal-on-glass at room temperature, which by itself is a unique experimental system at this range of temperatures. The universality of the results is discussed in comparison with similar patterns of reactive wetting at high temperatures. Full article

Robot learning from human demonstration pioneers an effective mapping paradigm for endowing robots with human-like operational capabilities. This paper proposes a bio-signal-guided robot adaptive stiffness learning framework grounded in the conclusion that muscle activation of the human arm is positively correlated with the endpoint stiffness. First, we propose a human-teleoperated demonstration platform enabling real-time modulation of robot end-effector stiffness by human tutors during operational tasks. Second, we develop a dual-stage probabilistic modeling architecture employing the Gaussian mixture model and Gaussian mixture regression to model the temporal–motion correlation and the motion–sEMG relationship, successively. Third, a real-world experiment was conducted to validate the effectiveness of the proposed skill transfer framework, demonstrating that the robot achieves online adaptation of Cartesian impedance characteristics in contact-rich tasks. This paper provides a simple and intuitive way to plan the Cartesian impedance parameters, transcending the classical method that requires complex human arm endpoint stiffness identification before human demonstration or compensation for the difference in human–robot operational effects after human demonstration. Full article

Manipulators performing pick-and-place tasks with objects of complex shapes must consider not only how to grasp the objects but also how to maneuver them out of a bin. In this paper, we explore the industrial challenge of picking hook-shaped components, whose interlocking nature often leads to failed attempts at safely retrieving a single component at a time. Rather than explicitly modeling contact-rich interactions within optimization-based motion planners, we tackle this challenge by leveraging recent advances in sampling-based optimization and parallelizable physics simulators to predict the impact of motion on the separating subgoal, aimed at resolving interlocking. The proposed framework generates candidate trajectories initialized from a user-provided demonstration, which are then simulated and evaluated in a physics simulator to optimize robot trajectories in joint space while considering the entire planning horizon. We validate our approach through real-world experiments on a manipulator, demonstrating improved success rates in terms of separating interlocked objects compared to the industrial baseline. Full article

This research explores the SL-Block system within an architecture framework by embracing building modularity, combinatorial design, topological interlocking, machine learning, and tactile sensor-based robotic assembly. The SL-Block, composed of S and L-shaped tetracubes, possesses a unique self-interlocking feature that allows for reversible joining and the creation of various 2D or 3D structures. In architecture modularity, the high degree of reconfigurability and adaptability of the SL-Block system introduces a new element of interest. Unlike modularization strategies that emphasize large-scale volumetric modules or standardized building components, using small-scale generic building blocks provides greater flexibility in maximizing design variations and reusability. Furthermore, the serial repetition and limited connectivity of building elements reduce the efforts required for bespoke manufacturing and automated assembly. In this article, we present our digital design and robotic assembly strategies for developing dry-jointed modular construction with SL-Blocks. Drawing on combinatorics and graph theory, we propose computational design methods that can automatically generate hierarchical SL-Block assemblies from given shapes. To address the physical complexities of contact-rich assembly tasks, we develop robotics using two distinct methods: pre-programmed assembly and sensor-based reinforcement learning. Through a series of demonstrators, we showcase the ability of SL-Blocks not only to reconfigure conventional building tectonics but also to create new building configurations. Full article

Tactile sensing has become indispensable for contact-rich dynamic robotic manipulation tasks. It provides robots with a better understanding of the physical environment, which is a vital supplement to robotic vision perception. Compared with other existing tactile sensors, vision-based tactile sensors (VBTSs) stand out for augmenting the tactile perception capabilities of robotic systems, owing to superior spatial resolution and cost-effectiveness. Despite their advantages, VBTS production faces challenges due to the lack of standardised manufacturing techniques and heavy reliance on manual labour. This limitation impedes scalability and widespread adoption. This paper introduces a rapid monolithic manufacturing technique and evaluates its performance quantitatively. We further develop and assess C-Sight, a novel VBTS sensor manufactured using this technique, focusing on its tactile reconstruction capabilities. Experimental results demonstrate that the monolithic manufacturing technique enhances VBTS production efficiency significantly. Also, the fabricated C-Sight sensor exhibits its reliable tactile perception and reconstruction capabilities, proofing the validity and feasibility of the monolithic manufacturing method. Full article

Robotic contact-rich insertion tasks present a significant challenge for motion planning due to the complex force interaction between robots and objects. Although many learning-based methods have shown success in contact tasks, most methods need sampling or exploring to gather sufficient experimental data. However, it is both time-consuming and expensive to conduct real-world experiments repeatedly. On the other hand, while the virtual world enables low cost and fast computations by simulators, there still exists a huge sim-to-real gap due to the inaccurate point contact model. Although finite element analysis might generate accurate results for contact tasks, it is computationally expensive. As such, this study proposes a motion planning framework with bilevel optimization to leverage relatively accurate force information with fast computation time. This framework consists of Dynamic Movement Primitives (DMPs) used to parameterize motion trajectories, Black-Box Optimization (BBO), a derivative-free approach, integrated to improve contact-rich insertion policy with hydroelastic contact model, and simulated variability to account for visual uncertainty in the real world. The accuracy of the simulated model is then validated by comparing our contact results with a benchmark Peg-in-Hole task. Using these integrated DMPs and BBO with hydroelastic contact model, the motion trajectory generated in planning is capable of guiding the robot towards successful insertion with iterative refinement. Full article

The introduction of collaborative robots in industrial environments reinforces the need to provide these robots with better cognition to accomplish their tasks while fostering worker safety without entering into safety shutdowns that reduce workflow and production times. This paper presents a novel strategy that combines the execution of contact-rich tasks, namely disassembly, with real-time collision avoidance through machine learning for safe human-robot interaction. Specifically, a goal-conditioned reinforcement learning approach is proposed, in which the removal direction of a peg, of varying friction, tolerance, and orientation, is subject to the location of a human collaborator with respect to a 7-degree-of-freedom manipulator at each time step. For this purpose, the suitability of three state-of-the-art actor-critic algorithms is evaluated, and results from simulation and real-world experiments are presented. In reality, the policy’s deployment is achieved through a new scalable multi-control framework that allows a direct transfer of the control policy to the robot and reduces response times. The results show the effectiveness, generalization, and transferability of the proposed approach with two collaborative robots against static and dynamic obstacles, leveraging the set of available solutions in non-monotonic tasks to avoid a potential collision with the human worker. Full article

Although interest in using wearable sensors to characterize movement disorders is growing, there is a lack of methodology for developing clinically interpretable biomarkers. Such digital biomarkers would provide a more objective diagnosis, capturing finer degrees of motor deficits, while retaining the information of traditional clinical tests. We aim at digitizing traditional tests of cognitive and memory performance to derive motor biometrics of pen-strokes and voice, thereby complementing clinical tests with objective criteria, while enhancing the overall characterization of Parkinson’s disease (PD). 35 participants including patients with PD, healthy young and age-matched controls performed a series of drawing and memory tasks, while their pen movement and voice were digitized. We examined the moment-to-moment variability of time series reflecting the pen speed and voice amplitude. The stochastic signatures of the fluctuations in pen drawing speed and voice amplitude of patients with PD show a higher signal-to-noise ratio compared to those of neurotypical controls. It appears that contact motions of the pen strokes on a tablet evoke sensory feedback for more immediate and predictable control in PD, while voice amplitude loses its neurotypical richness. We offer new standardized data types and analytics to discover the hidden motor aspects within the cognitive and memory clinical assays. Full article

Autonomous planning robotic contact-rich manipulation has long been a challenging problem. Automatic peeling of glass substrates of LCD flat panel displays is a typical contact-rich manipulation task, which requires extremely high safe handling through the manipulation process. To this end of peeling glass substrates automatically, the system model is established from data and is used for the online planning of the robot motion in this paper. A simulation environment is designed to pretrain the process model with deep learning-based neural network structure to avoid expensive and time-consuming collection of real-time data. Then, an online learning algorithm is introduced to tune the pretrained model according to the real-time data from the peeling process experiments to cover the uncertainties of the real process. Finally, an Online Learning Model Predictive Path Integral (OL-MPPI) algorithm is proposed for the optimal trajectory planning of the robot. The performance of our algorithm was validated through glass substrate peeling tasks of experiments. Full article

Remotely-sensed data are a source of rich information and are valuable for precision agricultural tasks such as soil quality, plant disease analysis, crop stress assessment, and allowing for better management. It is necessary to validate the accuracy of land surface temperature (LST) that is acquired from an unmanned aerial vehicle (UAV) and satellite-based remote sensing and verify these data by a comparison with in situ LST. Comprehensive studies at the field scale are still needed to understand the suitability of UAV imagery and resolution, for which ground measurement is used as a reference. In this study, we examined the accuracy of surface temperature data that were obtained from a thermal infrared (TIR) sensor placed on a UAV. Accordingly, we evaluated the LST from the Landsat 8 satellite for the same specific periods. We used contact thermometers to measure LSTs in situ for comparison and evaluation. Between 18 August and 2 September 2020, UAV imagery and in situ measurements were carried out. The effectiveness of high-resolution UAVs imagery and of Landsat 8 imagery was evaluated by considering a regression and correlation coefficient analysis. The data from the satellite photography was compared to the UAV imagery using statistical metrics after it had been pre-processed. Ground control points (GCPs) were collected to create a rigorous geo-referenced dataset of UAV imagery that could be compared to the geo-referenced satellite and aerial imagery. The UAV TIR LST showed higher accuracy (R² 0.89, 0.90, root-mean-square error (RMSE) 1.07, 0.70 °C) than the Landsat LST accuracy (R² 0.70, 0.73, (RMSE) 0.78 °C). The relationship between LST and the available soil water content (SWC) was also observed. The results suggested that the UAV-SMC correlation was negative (−0.85) for the image of DOY 230, while this value remains approximately constant (−0.86) for the DOY 245. Our results showed that satellite imagery that was coherent and correlated with UAV images could be useful to assess the general conditions of the field while the UAV favors localized circumscribed areas that the lowest resolution of satellites missed. Accordingly, our results could help with urban area and environmental planning decisions that take into account the thermal environment. Full article

The reduction in CO₂ emissions is a major task for the coming decades. Accelerated weathering of limestone (AWL) can be used to capture CO₂ from effluent gas streams and store it as bicarbonate in marine environments. We give an overview of the fundamental aspects of AWL, including associated CO₂ emissions during the operation of AWL, characteristics of the accumulating bicarbonate-rich product water, and factors influencing the outgassing of CO₂ from the ocean back into the atmosphere. Based on these aspects, we identify locations where AWL could be carried out favorably. The energy demand for AWL reduces the theoretical CO₂ sequestration potential, for example, by only 5% in the case of a 100 km transport of limestone on roads. AWL-derived product water is characterized by high alkalinity but low pH values and, once in contact with the atmosphere, passive outgassing of CO₂ from AWL-derived water occurs. This process is mainly driven by the difference between the fCO₂ in the atmosphere and the oceanic surface layer, as well as the sea surface temperature at the discharge site. Promising sites for AWL may be in Florida or around the Mediterranean Sea, where outgassing could be prevented by injections into deep water layers. Full article

Industrial robot manipulators are playing a significant role in modern manufacturing industries. Though peg-in-hole assembly is a common industrial task that has been extensively researched, safely solving complex, high-precision assembly in an unstructured environment remains an open problem. Reinforcement-learning (RL) methods have proven to be successful in autonomously solving manipulation tasks. However, RL is still not widely adopted in real robotic systems because working with real hardware entails additional challenges, especially when using position-controlled manipulators. The main contribution of this work is a learning-based method to solve peg-in-hole tasks with hole-position uncertainty. We propose the use of an off-policy, model-free reinforcement-learning method, and we bootstraped the training speed by using several transfer-learning techniques (sim2real) and domain randomization. Our proposed learning framework for position-controlled robots was extensively evaluated in contact-rich insertion tasks in a variety of environments. Full article

Advances in Robotics are leading to a new generation of assistant robots working in ordinary, domestic settings. This evolution raises new challenges in the tasks to be accomplished by the robots. This is the case for object manipulation where the detect-approach-grasp loop requires a robust recovery stage, especially when the held object slides. Several proprioceptive sensors have been developed in the last decades, such as tactile sensors or contact switches, that can be used for that purpose; nevertheless, their implementation may considerably restrict the gripper’s flexibility and functionality, increasing their cost and complexity. Alternatively, vision can be used since it is an undoubtedly rich source of information, and in particular, depth vision sensors. We present an approach based on depth cameras to robustly evaluate the manipulation success, continuously reporting about any object loss and, consequently, allowing it to robustly recover from this situation. For that, a Lab-colour segmentation allows the robot to identify potential robot manipulators in the image. Then, the depth information is used to detect any edge resulting from two-object contact. The combination of those techniques allows the robot to accurately detect the presence or absence of contact points between the robot manipulator and a held object. An experimental evaluation in realistic indoor environments supports our approach. Full article