Search Results (68)

Human–machine collaboration in assistive aerial teleoperation is frequently compromised by trust imbalances, which arise from the vehicle’s complex dynamics and the operator’s constrained perceptual feedback. We introduce a novel framework that enhances collaboration by dynamically integrating a model of human trust into the unmanned aerial vehicle’s trajectory planning. We first propose a Machine-Performance-Dependent trust model, specifically tailored for aerial teleoperation, that quantifies trust based on real-time safety and visibility metrics. This model then informs a trust-aware trajectory planning algorithm, which generates smooth and adaptive trajectories that continuously align with the operator’s trust level and intent inferred from control inputs. Extensive simulations conducted in diverse forest environments validate our approach. The results demonstrate that our method achieves task efficiency comparable to that of a trust-unaware baseline while significantly reducing operator workload and improving trajectory smoothness, achieving reductions of up to 23.2% and 43.2%, respectively, in challenging dense environments. By embedding trust dynamics directly into the trajectory optimization loop, this work pioneers a more intuitive, efficient, and resilient paradigm for assistive aerial teleoperation. Full article

This work presents the experimental validation of a reset control mode for a Quasi-Spherical Parallel Manipulator (qSPM), designed as a master device for bilaterally teleoperated telesurgical systems. The reset functionality enables autonomous repositioning of the master device to its central configuration via a joint-space path planning algorithm, executed entirely within the local control loop. Given the non-convex nature of the joint space, the algorithm computes feasible trajectories using a simplified optimization scheme that ensures compliance with mechanical and kinematic constraints. The algorithm was implemented within an ROS Noetic framework and tested across multiple scenarios, including both simulated and physical configurations. The experimental results confirm the device’s ability to reset to the central position in approximately 5 s, maintaining an average residual error below $0.25°$. Computational evaluations demonstrate that each path is generated in less than 10 milliseconds, supporting real-time execution. Additional trials show successful motion toward arbitrary points within the joint space, suggesting the potential for future integration of user-driven repositioning features. These findings highlight the responsiveness, reliability, and experimental feasibility of the proposed control mode, marking a key step toward improving usability in telesurgical environments. Full article

To increase the safety and efficiency of underground mining processes, it is important to advance automation. An important part of that is to achieve autonomous material loading using load–haul–dump (LHD) machines. To be able to autonomously load material from a muck pile, it is crucial to first detect and characterize it in terms of spatial configuration and geometry. Currently, the technologies available on the market that do not require an operator at the stope are only applicable in specific mine layouts or use 2D camera images of the surroundings that can be observed from a control room for teleoperation. However, due to missing depth information, estimating distances is difficult. This work presents a novel approach to muck pile detection developed as part of the EU-funded Next Generation Carbon Neutral Pilots for Smart Intelligent Mining Systems (NEXGEN SIMS) project. It uses a stereo camera mounted on an LHD to gather three-dimensional data of the surroundings. By applying a topological algorithm, a muck pile can be located and its overall shape determined. This system can detect and segment muck piles while driving towards them at full speed. The detected position and shape of the muck pile can then be used to determine an optimal attack point for the machine. This sensor solution was then integrated into a complete system for autonomous loading with an LHD. In two different underground mines, it was tested and demonstrated that the machines were able to reliably load material without human intervention. Full article

This paper proposes a novel virtual-fixtures-based shared control concept for eye surgery systems focusing on cataract procedures, one of the most common ophthalmic surgeries. Current research on haptic force feedback aims to enhance manipulation capabilities by integrating teleoperated medical robots. Our proposed concept utilizes teleoperated medical robots to improve the training of young surgeons by providing haptic feedback during cataract operations based on geometrical virtual fixtures. The core novelty of our concept is the active guidance to the incision point generated directly from the geometrical representation of the virtual fixtures, and, therefore, it is computationally efficient. Furthermore, novel virtual fixtures are introduced for the posterior corneal surface of the eye during the cataract operation. The concept is tested in a human-in-the-loop pilot study, where non-medical engineering students participated. The results indicate that the proposed shared control system is helpful for the test subjects. Therefore, the inclusion of the proposed concept can be beneficial for the training of non-experienced surgeons. Full article

Disassembly is a crucial process in industrial operations, especially in tasks requiring high precision and strict safety standards when handling components with collaborative robots. However, traditional methods often rely on rigid and sequential task planning, which makes it difficult to adapt to unforeseen changes or dynamic environments. This rigidity not only limits flexibility but also leads to prolonged execution times, as operators must follow predefined steps that do not allow for real-time adjustments. Although techniques like teleoperation have attempted to address these limitations, they often hinder direct human–robot collaboration within the same workspace, reducing effectiveness in dynamic environments. In response to these challenges, this research introduces an advanced human–robot interaction (HRI) system leveraging a mixed-reality (MR) interface embedded in a head-mounted device (HMD). The system enables operators to issue real-time control commands using multimodal inputs, including voice, gestures, and gaze tracking. These inputs are synchronized and processed via the Robot Operating System (ROS2), enabling dynamic and flexible task execution. Additionally, the integration of deep learning algorithms ensures precise detection and validation of disassembly components, enhancing accuracy. Experimental evaluations demonstrate significant improvements, including reduced task completion times, enhanced operator experience, and compliance with strict adherence to safety standards. This scalable solution offers broad applicability for general-purpose disassembly tasks, making it well-suited for complex industrial scenarios. Full article

Cyber-physical systems such as Supervisory Control and Data Acquisition (SCADA) have been applied in industrial automation and infrastructure management for decades. They are hybrid tools for administration, monitoring, and continuous control of real physical systems through their computational representation. SCADA systems have evolved along with computing technology, from their beginnings with low-performance computers, monochrome monitors and communication networks with a range of a few hundred meters, to high-performance systems with advanced 3D graphics and wired and wireless computer networks. This article presents a methodology for the design of a SCADA system with a 3D Visualization for Drinking Water Supply, and its implementation in the Lerma Basin System of Mexico City as a case study. The monitoring of water consumption from the wells is presented, as well as the pressure levels throughout the system. The 3D visualization is generated from the GIS information and the communication is carried out using a hybrid radio frequency transmission system, satellite, and telephone network. The pumps that extract water from each well are teleoperated and monitored in real time. The developed system can be scaled to generate a simulator of water behavior of the Lerma Basin System and perform contingency planning. Full article

Teleoperation systems experience degraded control stability and safety due to dynamic communication delays. This study proposes an Interpolation Predictor for Teleoperation under Dynamic Communication Delay (IPT-DCD), a predictor that reconstructs asynchronously received control commands via interpolation and predicts future commands using an encoder–decoder LSTM architecture. To restore the temporal consistency of delayed signals, a signal preprocessing technique called the Backward Shifting and Interpolation (BSI) was applied, enabling the transformation of received data into an undelayed and uniformly sampled format. As a result, the proposed model was capable of generating real-time steering command outputs through a many-to-many time series structure. Furthermore, to evaluate its effectiveness, IPT-DCD was experimentally compared with a baseline model, a Predictor for Teleoperation under Dynamic Communication Delay (PT-DCD). The results reveal that IPT-DCD exhibits significantly greater robustness to large communication delay outliers than the baseline, highlighting its effectiveness in dynamic and unstable teleoperation environments. Full article

This study presents a novel investigation into immersive teleoperation systems using collaborative, device-agnostic interfaces for advancing smart haptics in industrial assistive applications. The research focuses on evaluating the quality of experience (QoE) of users interacting with a teleoperation system comprising a local robotic arm, a robot gripper, and heterogeneous remote tracking and haptic feedback devices. By employing a modular device-agnostic framework, the system supports flexible configurations, including one-user-one-equipment (1U-1E), one-user-multiple-equipment (1U-ME), and multiple-users-multiple-equipment (MU-ME) scenarios. The experimental set-up involves participants manipulating predefined objects and placing them into designated baskets by following specified 3D trajectories. Performance is measured using objective QoE metrics, including temporal efficiency (time required to complete the task) and spatial accuracy (trajectory similarity to the predefined path). In addition, subjective QoE metrics are assessed through detailed surveys, capturing user perceptions of presence, engagement, control, sensory integration, and cognitive load. To ensure flexibility and scalability, the system integrates various haptic configurations, including (1) a Touch kinaesthetic device for precision tracking and grounded haptic feedback, (2) a DualSense tactile joystick as both a tracker and mobile haptic device, (3) a bHaptics DK2 vibrotactile glove with a camera tracker, and (4) a SenseGlove Nova force-feedback glove with VIVE trackers. The modular approach enables comparative analysis of how different device configurations influence user performance and experience. The results indicate that the objective QoE metrics varied significantly across device configurations, with the Touch and SenseGlove Nova set-ups providing the highest trajectory similarity and temporal efficiency. Subjective assessments revealed a strong correlation between presence and sensory integration, with users reporting higher engagement and control in scenarios utilizing force feedback mechanisms. Cognitive load varied across the set-ups, with more complex configurations (e.g., 1U-ME) requiring longer adaptation periods. This study contributes to the field by demonstrating the feasibility of a device-agnostic teleoperation framework for immersive industrial applications. It underscores the critical interplay between objective task performance and subjective user experience, providing actionable insights into the design of next-generation teleoperation systems. Full article

Mobile edge computing was conceived to address the increasing computing demand generated by users at the communication network edge. It is expected to play a significant role in next-generation (5G, 6G, and beyond) communication systems as new applications such as augmented/extended reality, teleoperations, telemedicine, and gaming become prolific. As the networks become denser, more and more edge servers are expected to be deployed, and the question of task offloading becomes more complicated. In this study, we present a framework for task offloading in the presence of multiple edge servers that employ round-robin task scheduling. Most studies in the literature attempt to optimize the offloading process under the assumption that each user generates just a single task, or they generate one task every time slot in a discrete-time system where all the tasks are handled within a slot. Furthermore, first-come-first-served queueing models are typically used in studies where queueing is considered at all. The work presented is novel in that we assume continuous and stochastic task arrivals generated by multiple users and round-robin task scheduling at the edge servers. This setting is considerably more realistic with respect to the existing works, and we demonstrate through extensive simulations that round-robin task scheduling significantly reduces task delay. We also present a comparison of a number of server selection mechanisms. Full article

The teleoperation of robot arms can prevent users from working in hazardous environments, but current teleoperation uses a 2D display and controls the end effector of robot arms, which introduces the problem of a limited view and complex operations. In this study, a teleoperation method for robot arms is proposed, which can control the robot arm by interacting with the digital twins of objects. Based on the objects in the workspace, this method generates a virtual scene containing digital twins. Users can observe the virtual scene from any direction and move the digital twins of the objects at will to control the robot arm. This study compared the proposed method and the traditional method, which uses a 2D display and a game controller, through a pick-and-place task. The proposed method achieved 45% lower scores in NASA-TLX and 31% higher scores in SUS than traditional teleoperation methods. The results indicate that the proposed method can reduce the workload and improve the usability of teleoperation. Full article

Force-projecting bilateral control is an effective method for enhancing the positioning rigidity and stability of teleoperation systems equipped with compliant pneumatically driven followers. However, friction in the pneumatic actuation mechanism has caused a deterioration in force reproducibility between the leader and follower. To solve this problem, this study proposes a practical method of nonlinear friction compensation in force-projecting bilateral control to improve the force reproducibility. The proposed method generates two friction compensation forces: one based on the target admittance velocity from the leader and the other based on the actual velocity of the follower. These forces are seamlessly switched according to the dynamic state of the system to compensate for the follower’s driving force. This enables improved force reproducibility in any motion states of the system while maintaining the advantage of force-projecting bilateral control, which eliminates the need for external force measurement on the follower side. Experiments were conducted using a 1-DOF bilateral control device consisting of an electric linear motor and a pneumatic cylinder, including free motion and contact operations with two types of environments, demonstrating the effectiveness of the proposed method. Full article

Robots are essential for carrying out tasks, for example, in a nuclear industry, where direct human involvement is limited. However, present-day nuclear robots are not versatile due to limited autonomy and higher costs. This research presents a merely teleoperated Dexter^TM nuclear robot’s transformation into an autonomous manipulator for nuclear sort and segregation tasks. The Dexter^TM system comprises a arm client manipulator designed to operate in extreme radiation environments and a similar single/dual-arm local manipulator. In this paper, initially, a kinematic model and convex optimization-based dynamic model identification of a single-arm Dexter^TM manipulator is presented. This model is used for autonomous Dexter^TM control through Robot Operating System (ROS). A new integration framework incorporating vision, AI-based grasp generation and an intelligent radiological surveying method for enhancing the performance of autonomous Dexter^TM is presented. The efficacy of the framework is demonstrated on a mock-up nuclear waste test-bed using similar waste materials found in the nuclear industry. The experiments performed show potency, generality and applicability of the proposed framework in overcoming the entry barriers for autonomous systems in regulated domains like the nuclear industry. Full article

Teleoperation technology enables remote control of machines, but often requires complex manoeuvres that pose significant challenges for operators. To mitigate these challenges, assistive systems have been developed to support teleoperation. This study presents a teleoperation guidance system that provides assistive force feedback to help operators align more accurately with desired trajectories. Two key issues remain: (1) the lack of a flexible, real-time approach to defining desired trajectories and calculating assistive forces, and (2) uncertainty about the effects of forward motion assistance within the assistive forces. To address these issues, we propose a novel approach that captures the posture trajectory of the local control interface, statistically generates a reference trajectory, and incorporates forward motion as an adjustable parameter. In Experiment 1, which involved simulating an object transfer task, the proposed method significantly reduced the operator’s workload compared to conventional techniques, especially in dynamic target scenarios. Experiment 2, which involved more complex paths, showed that assistive forces with forward assistance significantly improved manoeuvring performance. Full article

Improving the autonomy of underwater interventions by remotely operated vehicles (ROVs) can help mitigate the impact of communication delays on operational efficiency. Currently, underwater interventions for ROVs usually rely on real-time teleoperation or preprogramming by operators, which is not only time-consuming and increases the cognitive burden on operators but also requires extensive specialized programming. Instead, this paper uses the intuitive learning from demonstrations (LfD) approach that uses operator demonstrations as inputs and models the trajectory characteristics of the task through the dynamic movement primitive (DMP) approach for task reproduction as well as the generalization of knowledge to new environments. Unlike existing applications of DMP-based robot trajectory learning methods, we propose the underwater DMP (UDMP) method to address the problem that the complexity and stochasticity of underwater operational environments (e.g., current perturbations and floating operations) diminish the representativeness of the demonstrated trajectories. First, the Gaussian mixture model (GMM) and Gaussian mixture regression (GMR) are used for feature extraction of multiple demonstration trajectories to obtain typical trajectories as inputs to the DMP method. The UDMP method is more suitable for the LfD of underwater interventions than the method that directly learns the nonlinear terms of the DMP. In addition, we improve the commonly used homomorphic-based teleoperation mode to heteromorphic mode, which allows the operator to focus more on the end-operation task. Finally, the effectiveness of the developed method is verified by simulation experiments. Full article

Service robots are rapidly transitioning from concept to reality, making significant strides in development. Similarly, the field of prosthetics is evolving at an impressive pace, with both areas now being highly relevant in the industry. Advancements in these fields are continually pushing the boundaries of what is possible, leading to the increasing creation of individual arm and hand prosthetics, either as standalone units or combined packages. This trend is driven by the rise of advanced collaborative robots that seamlessly integrate with human counterparts in real-world applications. This paper presents an open-source, 3D-printed robotic arm that has been assembled and programmed using two distinct approaches. The first approach involves controlling the hand via teleoperation, utilizing a camera and machine learning-based hand pose estimation. This method details the programming techniques and processes required to capture data from the camera and convert it into hardware signals. The second approach employs kinematic control using the Denavit-Hartenbergmethod to define motion and determine the position of the end effector in 3D space. Additionally, this work discusses the assembly and modifications made to the arm and hand to create a cost-effective and practical solution. Typically, implementing teleoperation requires numerous sensors and cameras to ensure smooth and successful operation. This paper explores methods enabled by artificial intelligence (AI) that reduce the need for extensive sensor arrays and equipment. It investigates how AI-generated data can be translated into tangible hardware applications across various fields. The advancements in computer vision, combined with AI capable of accurately predicting poses, have the potential to revolutionize the way we control and interact with the world around us. Full article