Digital Twin-Based Human–Robot Collaborative Systems

A special issue of Robotics (ISSN 2218-6581). This special issue belongs to the section "Industrial Robots and Automation".

Deadline for manuscript submissions: closed (31 May 2024) | Viewed by 19708

Special Issue Editor


E-Mail Website
Guest Editor
HUMAIN-Lab, Department of Computer Science, Democritus University of Thrace, Kavala, Greece
Interests: mobile robotics; social robotics; computational intelligence; augmented reality applications and educational software
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

In recent years, an increasing number of industries have benefited from the use of digital twins, aiming to improve product design, data collection, testing, diagnostics and monitoring. In the field of robotics, a digital twin can be considered a virtual representation of a robot and its processes, within the context of its intended operating environment. A key challenge in the area of human-robot collaborations is to determine appropriate interaction models in order to better understand the requirements for effective and safe human-robot collaboration in shared spaces, such as industry, agriculture and education. Following a digital twin paradigm allows the formation of more reliable virtual platforms for applying the proposed HRI models and studying the complex issues arising from human-robot interactions.

In this Special Issue, original research articles and comprehensive reviews are welcome. Contributions may include novel approaches to the application of digital twins in all cycles of robotic development, in various application areas such as manufacturing, agriculture, education, etc. Research areas may include (but are not limited to) the following:

  • System architectures for digital twins;
  • Human-robot interaction modeling;
  • Digital twins for telepresence and teleoperation tasks;
  • Augmented and virtual reality for digital twins;
  • Industrial applications and implementation.

Dr. Chris Lytridis
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Robotics is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • digital twins
  • human-robot interaction
  • modeling
  • computational models
  • social intelligence
  • cooperation
  • virtual models

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Published Papers (9 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

19 pages, 27719 KiB  
Article
Assistive Control through a Hapto-Visual Digital Twin for a Master Device Used for Didactic Telesurgery
by Daniel Pacheco Quiñones, Daniela Maffiodo and Med Amine Laribi
Robotics 2024, 13(9), 138; https://doi.org/10.3390/robotics13090138 - 11 Sep 2024
Viewed by 585
Abstract
This article explores the integration of a hapto-visual digital twin on a master device used for bilateral teleoperation. The device, known as a quasi-spherical parallel manipulator, is currently employed for the remote center of motion control in teleoperated mini-invasive surgery. After providing detailed [...] Read more.
This article explores the integration of a hapto-visual digital twin on a master device used for bilateral teleoperation. The device, known as a quasi-spherical parallel manipulator, is currently employed for the remote center of motion control in teleoperated mini-invasive surgery. After providing detailed insights into the device’s kinematics, including its geometric configuration, Jacobian, and reachable workspace, the paper illustrates the overall control system, encompassing both hardware and software components. The article describes how a digital twin, which implements a haptic assistive control and a visually enhanced representation of the device, was integrated into the system. The digital twin was then tested with the device: in the experiments, one “student” end-user must follow a predefined “teacher” trajectory. Preliminary results demonstrate how the overall system can pose a good starting point for didactic telesurgery operation. The control action, yet to be optimized and tested on different subjects, indeed seems to grant satisfying performance and accuracy. Full article
(This article belongs to the Special Issue Digital Twin-Based Human–Robot Collaborative Systems)
Show Figures

Figure 1

22 pages, 4999 KiB  
Article
A Framework for Enhanced Human–Robot Collaboration during Disassembly Using Digital Twin and Virtual Reality
by Timon Hoebert, Stephan Seibel, Manuel Amersdorfer, Markus Vincze, Wilfried Lepuschitz and Munir Merdan
Robotics 2024, 13(7), 104; https://doi.org/10.3390/robotics13070104 - 12 Jul 2024
Cited by 1 | Viewed by 1423
Abstract
This paper presents a framework that integrates digital twin and virtual reality (VR) technologies to improve the efficiency and safety of human–robot collaborative systems in the disassembly domain. With the increasing complexity of the handling of end-of-life electronic products and as the related [...] Read more.
This paper presents a framework that integrates digital twin and virtual reality (VR) technologies to improve the efficiency and safety of human–robot collaborative systems in the disassembly domain. With the increasing complexity of the handling of end-of-life electronic products and as the related disassembly tasks are characterized by variabilities such as rust, deformation, and diverse part geometries, traditional industrial robots face significant challenges in this domain. These challenges require adaptable and flexible automation solutions that can work safely alongside human workers. We developed an architecture to address these challenges and support system configuration, training, and operational monitoring. Our framework incorporates a digital twin to provide a real-time virtual representation of the physical disassembly process, allowing for immediate feedback and dynamic adjustment of operations. In addition, VR is used to simulate and optimize the workspace layout, improve human–robot interaction, and facilitate safe and effective training scenarios without the need for physical prototypes. A unique case study is presented, where the collaborative system is specifically applied to the disassembly of antenna amplifiers, illustrating the potential of our comprehensive approach to facilitate engineering processes and enhance collaborative safety. Full article
(This article belongs to the Special Issue Digital Twin-Based Human–Robot Collaborative Systems)
Show Figures

Figure 1

21 pages, 2978 KiB  
Article
A Digital Twin Infrastructure for NGC of ROV during Inspection
by David Scaradozzi, Flavia Gioiello, Nicolò Ciuccoli and Pierre Drap
Robotics 2024, 13(7), 96; https://doi.org/10.3390/robotics13070096 - 26 Jun 2024
Viewed by 2172
Abstract
Remotely operated vehicles (ROVs) provide practical solutions for a wide range of activities in a particularly challenging domain, despite their dependence on support ships and operators. Recent advancements in AI, machine learning, predictive analytics, control theories, and sensor technologies offer opportunities to make [...] Read more.
Remotely operated vehicles (ROVs) provide practical solutions for a wide range of activities in a particularly challenging domain, despite their dependence on support ships and operators. Recent advancements in AI, machine learning, predictive analytics, control theories, and sensor technologies offer opportunities to make ROVs (semi) autonomous in their operations and to remotely test and monitor their dynamics. This study moves towards that goal by formulating a complete navigation, guidance, and control (NGC) system for a six DoF BlueROV2, offering a solution to the current challenges in the field of marine robotics, particularly in the areas of power supply, communication, stability, operational autonomy, localization, and trajectory planning. The vehicle can operate (semi) autonomously, relying on a sensor acoustic USBL localization system, tethered communication with the surface vessel for power, and a line of sight (LOS) guidance system. This strategy transforms the path control problem into a heading control problem, aligning the vehicle’s movement with a dynamically calculated reference point along the desired path. The control system uses PID controllers implemented in the navigator flight controller board. Additionally, an infrastructure has been developed that synchronizes and communicates between the real ROV and its digital twin within the Unity environment. The digital twin acts as a visual representation of the ROV’s movements and considers hydrodynamic behaviors. This approach combines the physical properties of the ROV with the advanced simulation and analysis capabilities of its digital counterpart. All findings were validated at the Point Rouge port located in Marseille and at the port of Ancona. The NGC implemented has proven positive vehicle stability and trajectory tracking in time despite external interferences. Additionally, the digital part has proven to be a reliable infrastructure for a future bidirectional communication system. Full article
(This article belongs to the Special Issue Digital Twin-Based Human–Robot Collaborative Systems)
Show Figures

Figure 1

19 pages, 11423 KiB  
Article
Designing Digital Twins of Robots Using Simscape Multibody
by Giovanni Boschetti and Teresa Sinico
Robotics 2024, 13(4), 62; https://doi.org/10.3390/robotics13040062 - 14 Apr 2024
Cited by 4 | Viewed by 2609
Abstract
Digital twins of industrial and collaborative robots are widely used to evaluate and predict the behavior of manipulators under different control strategies. However, these digital twins often employ simplified mathematical models that do not fully describe their dynamics. In this paper, we present [...] Read more.
Digital twins of industrial and collaborative robots are widely used to evaluate and predict the behavior of manipulators under different control strategies. However, these digital twins often employ simplified mathematical models that do not fully describe their dynamics. In this paper, we present the design of a high-fidelity digital twin of a six degrees-of-freedom articulated robot using Simscape Multibody, a Matlab toolbox that allows the design of robotic manipulators in a rather intuitive and user-friendly manner. This robot digital twin includes joint friction, transmission gears, and electric actuators dynamics. After assessing the dynamic accuracy of the Simscape model, we used it to test a computed torque control scheme, proving that this model can be reliably used in simulations with different aims, such as validating control schemes, evaluating collaborative functions or minimizing power consumption. Full article
(This article belongs to the Special Issue Digital Twin-Based Human–Robot Collaborative Systems)
Show Figures

Figure 1

15 pages, 3678 KiB  
Article
The Impact of Changing Collaborative Workplace Parameters on Assembly Operation Efficiency
by Klemen Kovič, Aljaž Javernik, Robert Ojsteršek and Iztok Palčič
Robotics 2024, 13(3), 36; https://doi.org/10.3390/robotics13030036 - 23 Feb 2024
Cited by 1 | Viewed by 1666
Abstract
Human–robot collaborative systems bring several benefits in using human and robot capabilities simultaneously. One of the critical questions is the impact of these systems on production process efficiency. The search for high-level efficiency is severely dependent on collaborative robot characteristics and motion parameters, [...] Read more.
Human–robot collaborative systems bring several benefits in using human and robot capabilities simultaneously. One of the critical questions is the impact of these systems on production process efficiency. The search for high-level efficiency is severely dependent on collaborative robot characteristics and motion parameters, and the ability of humans to adjust to changing circumstances. Therefore, our research analyzes the effect of the changing collaborative robot motion parameters, acoustic parameters and visual factors in a specific assembly operation, where efficiency is measured through operation times. To conduct our study, we designed a digital twin-based model and a laboratory environment experiment in the form of a collaborative workplace. The results show that changing the motion, acoustic and visual parameters of the collaborative workplace impact the assembly process efficiency significantly. Full article
(This article belongs to the Special Issue Digital Twin-Based Human–Robot Collaborative Systems)
Show Figures

Figure 1

32 pages, 17080 KiB  
Article
User Study to Validate the Performance of an Offline Robot Programming Method That Enables Robot-Independent Kinesthetic Instruction through the Use of Augmented Reality and Motion Capturing
by Fabian Müller, Michael Koch and Alexander Hasse
Robotics 2024, 13(3), 35; https://doi.org/10.3390/robotics13030035 - 23 Feb 2024
Viewed by 1729
Abstract
The paper presents a novel offline programming (OLP) method based on programming by demonstration (PbD), which has been validated through user study. PbD is a programming method that involves physical interaction with robots, and kinesthetic teaching (KT) is a commonly used online programming [...] Read more.
The paper presents a novel offline programming (OLP) method based on programming by demonstration (PbD), which has been validated through user study. PbD is a programming method that involves physical interaction with robots, and kinesthetic teaching (KT) is a commonly used online programming method in industry. However, online programming methods consume significant robot resources, limiting the speed advantages of PbD and emphasizing the need for an offline approach. The method presented here, based on KT, uses a virtual representation instead of a physical robot, allowing independent programming regardless of the working environment. It employs haptic input devices to teach a simulated robot in augmented reality and uses automatic path planning. A benchmarking test was conducted to standardize equipment, procedures, and evaluation techniques to compare different PbD approaches. The results indicate a 47% decrease in programming time when compared to traditional KT methods in established industrial systems. Although the accuracy is not yet at the level of industrial systems, users have shown rapid improvement, confirming the learnability of the system. User feedback on the perceived workload and the ease of use was positive. In conclusion, this method has potential for industrial use due to its learnability, reduction in robot downtime, and applicability across different robot sizes and types. Full article
(This article belongs to the Special Issue Digital Twin-Based Human–Robot Collaborative Systems)
Show Figures

Figure 1

16 pages, 8410 KiB  
Article
A Path to Industry 5.0 Digital Twins for Human–Robot Collaboration by Bridging NEP+ and ROS
by Enrique Coronado, Toshio Ueshiba and Ixchel G. Ramirez-Alpizar
Robotics 2024, 13(2), 28; https://doi.org/10.3390/robotics13020028 - 1 Feb 2024
Cited by 6 | Viewed by 2892
Abstract
The integration of heterogeneous hardware and software components to construct human-centered systems for Industry 5.0, particularly human digital twins, presents considerable complexity. Our research addresses this challenge by pioneering a novel approach that harmonizes the techno-centered focus of the Robot Operating System (ROS) [...] Read more.
The integration of heterogeneous hardware and software components to construct human-centered systems for Industry 5.0, particularly human digital twins, presents considerable complexity. Our research addresses this challenge by pioneering a novel approach that harmonizes the techno-centered focus of the Robot Operating System (ROS) with the cross-platform advantages inherent in NEP+ (a human-centered development framework intended to assist users and developers with diverse backgrounds and resources in constructing interactive human–machine systems). We introduce the nep2ros ROS package, aiming to bridge these frameworks and foster a more interconnected and adaptable approach. This initiative can be used to facilitate diverse development scenarios beyond conventional robotics, underpinning a transformative shift in Industry 5.0 applications. Our assessment of NEP+ capabilities includes an evaluation of communication performance utilizing serialization formats like JavaScript Object Notation (JSON) and MessagePack. Additionally, we present a comparative analysis between the nep2ros package and existing solutions, illustrating its efficacy in linking the simulation environment (Unity) and ROS. Moreover, our research demonstrates NEP+’s applicability through an immersive human-in-the-loop collaborative assembly. These findings offer promising prospects for innovative integration possibilities across a broad spectrum of applications, transcending specific platforms or disciplines. Full article
(This article belongs to the Special Issue Digital Twin-Based Human–Robot Collaborative Systems)
Show Figures

Figure 1

21 pages, 57501 KiB  
Article
A 3D World Interpreter System for Safe Autonomous Crane Operation
by Frank Bart ter Haar, Frank Ruis and Bastian Thomas van Manen
Robotics 2024, 13(2), 23; https://doi.org/10.3390/robotics13020023 - 26 Jan 2024
Viewed by 2117
Abstract
In an effort to improve short-sea shipping in Europe, we present a 3D world interpreter (3DWI) system as part of a robotic container-handling system. The 3DWI is an advanced sensor suite combined with AI-based software and the communication infrastructure to connect to both [...] Read more.
In an effort to improve short-sea shipping in Europe, we present a 3D world interpreter (3DWI) system as part of a robotic container-handling system. The 3DWI is an advanced sensor suite combined with AI-based software and the communication infrastructure to connect to both the crane control and the shore control center. On input of LiDAR data and stereo captures, the 3DWI builds a world model of the operating environment and detects containers. The 3DWI and crane control are the core of an autonomously operating crane that monitors the environment and may trigger an emergency stop while alerting the remote operator of the danger. During container handling, the 3DWI scans for human activity and continuously updates a 3D-Twin model for the operator, enabling situational awareness. The presented methodology includes the sensor suite design, creation of the world model and the 3D-Twin, innovations in AI-detection software, and interaction with the crane and operator. Supporting experiments quantify the performance of the 3DWI, its AI detectors, and safety measures; the detectors reach the top of VisDrone’s leaderboard and the pilot tests show the safe autonomous operation of the crane. Full article
(This article belongs to the Special Issue Digital Twin-Based Human–Robot Collaborative Systems)
Show Figures

Figure 1

26 pages, 8960 KiB  
Article
Virtual Reality Teleoperation System for Mobile Robot Manipulation
by Bryan R. Galarza, Paulina Ayala, Santiago Manzano and Marcelo V. Garcia
Robotics 2023, 12(6), 163; https://doi.org/10.3390/robotics12060163 - 29 Nov 2023
Cited by 5 | Viewed by 3191
Abstract
Over the past few years, the industry has experienced significant growth, leading to what is now known as Industry 4.0. This advancement has been characterized by the automation of robots. Industries have embraced mobile robots to enhance efficiency in specific manufacturing tasks, aiming [...] Read more.
Over the past few years, the industry has experienced significant growth, leading to what is now known as Industry 4.0. This advancement has been characterized by the automation of robots. Industries have embraced mobile robots to enhance efficiency in specific manufacturing tasks, aiming for optimal results and reducing human errors. Moreover, robots can perform tasks in areas inaccessible to humans, such as hard-to-reach zones or hazardous environments. However, the challenge lies in the lack of knowledge about the operation and proper use of the robot. This work presents the development of a teleoperation system using HTC Vive Pro 2 virtual reality goggles. This allows individuals to immerse themselves in a fully virtual environment to become familiar with the operation and control of the KUKA youBot robot. The virtual reality experience is created in Unity, and through this, robot movements are executed, followed by a connection to ROS (Robot Operating System). To prevent potential damage to the real robot, a simulation is conducted in Gazebo, facilitating the understanding of the robot’s operation. Full article
(This article belongs to the Special Issue Digital Twin-Based Human–Robot Collaborative Systems)
Show Figures

Figure 1

Back to TopTop