An up-and-coming concept that seeks to transform how students learn about and study complex systems, as well as how industrial workers are trained, metaverse technology is characterized in this context by its use in virtual simulation and analysis. In this work, a virtual environment is created that duplicates real-world situations and enables immersive and interactive learning in the educational metaverse. For this purpose, we built a digital twin of the Nanyang Technological University (NTU) campus as a foundation, called NTUniverse. It is designed as an educational metaverse in which various academic and analytical applications are digitized as 3D content embedded within this virtual campus. The approach to digitally twinning educational systems and embedding them within virtual campuses enables remote and collaborative learning as well as professional technical skills training. It also makes feasible the analysis of abstract concepts, complicated structures, dynamic processes, and sensitive industrial procedures virtually, which is otherwise challenging if not impossible to perform in the real world. The work offers important insights into the behaviors and interactions of systems in the metaverse by evaluating design choices and user interests. NTUniverse is an attempt to explore a novel approach that addresses remote education and training challenges. Three efforts with NTUniverse will be discussed in this work, including (1) digitalization of the NTU campus; (2) campus train modelling and simulation; and (3) science, technology, engineering and mathematics education. Full article

This study explores the utilization of Neural Radiance Fields (NeRFs), with a specific focus on the Instant NeRFs technique. The objective is to represent three-dimensional (3D) models within the context of the industrial metaverse, aiming to achieve a high-fidelity reconstruction of objects in virtual environments. NeRFs, renowned for their innovative approach, enable comprehensive model reconstructions by integrating diverse viewpoints and lighting conditions. The study employs tools such as Unity, Photon Pun2, and Oculus Interaction SDK to develop an immersive metaverse. Within this virtual industrial environment, users encounter numerous interactive six-dimensional (6D) models, fostering active engagement and enriching the overall experience. While initial implementations showcase promising results, they also introduce computational complexities. Nevertheless, this integration forms the basis for immersive comprehension and collaborative interactions within the industrial metaverse. The evolving potential of NeRF technology promises even more exciting prospects in the future. Full article

As “Industry 4.0” progresses, construction machinery is evolving toward large-scale, automation, and integration, resulting in the equipment becoming increasingly sophisticated, and the designs more difficult. Labor costs, transportation, and time will be huge challenges for construction machinery, and mixed reality technology is one of several possible ways to solve this challenge. The research presented in this paper develops a holographic visual verification platform for a digital prototype of construction machinery based on virtual terminal equipment, through investigating the synchronous remote collaboration of multiple terminal devices in a mixed reality scenario. These included semi-physical virtual-real fusion assembly, multi-person real-time voice communication, dynamic loading of MR model based on a cloud server, virtual imitation control, interface design, and human-computer interaction. The effectiveness of this paper’s method is demonstrated through remote collaborative design cases. These included a double drum roller, loader, and milling planer welding production line, as well as tractor modeling review and virtual simulation manipulation of an aerial work platform. The experimental results show that this visual verification platform is a feasible, low-cost and scalable solution, which brings a qualitative breakthrough to the design, research and development, production and other stages in the field of construction machinery. Full article

In this study, a method, in which parametric design and robotic fabrication are combined into one unified framework, and integrated within a mixed reality environment, where designers can interact with design and fabrication alternatives, and manage this process in collaboration with other designers, is proposed. To achieve this goal, the digital twin of both design and robotic fabrication steps was created within a mixed-reality environment. The proposed method was tested on a design product, which was defined with the shape-grammar method using parametric-modeling tools. In this framework, designers can interact with both design and robotic-fabrication parameters, and subsequent steps are generated instantly. Robotic fabrication can continue uninterrupted with human–robot collaboration. This study contributes to improving design and fabrication possibilities such as mass-customization, and shortens the process from design to production. The user experience and augmented spatial feedback provided by mixed reality are richer than the interaction with the computer screen. Since the whole process from parametric design to robotic fabrication can be controlled by parameters with hand gestures, the perception of reality is richer. The digital twin of parametric design and robotic fabrication is superimposed as holographic content by adding it on top of real-world images. Designers can interact with both design and fabrication processes both physically and virtually and can collaborate with other designers. Full article

Technical documentation creation is a collaborative process involving several departments in R&D. Even though virtual reality (VR) has been demonstrated to facilitate industrial collaboration and advance the product development lifecycle in earlier studies, it has not been utilized for technical documentation review and risk assessment processes in industrial companies. This article presents a case study where the benefits of VR to maintenance documentation reviews and risk assessments were studied. The virtual reality environment was tested by nine domain experts from an industrial company in a user study that replicated their actual real-life industrial collaboration tasks. Both qualitative and quantitative data were collected during the study. Our findings show that collaborative VR has the potential to enhance the documentation review and risk assessment processes. Overall, the concept of using virtual reality for documentation review and risk assessment processes was rated positively by participants, and even though further development is needed for the review tools, VR was viewed as a concept that facilitates collaboration, enhances the current review practices, and increases spatial understanding. The benefits of VR are evident, especially for geographically scattered teams that rarely meet face-to-face or do not have access to the actual physical equipment. In cases where traditional means of communication are not enough, process improvements are needed for documentation review and risk assessment processes, and our proposed solution is VR. Full article

Virtual Reality (VR) is a critical emerging technology in industrial contexts, as it facilitates collaboration and supports the product development lifecycle. However, its broad adoption is constrained by complex and high-cost integration. The use of VR among devices with various immersion and control levels may solve this obstacle, and increase the scalability of VR technologies. This article presents a case study on applying asymmetry between the COVE-VR platform and Microsoft Teams to enable distributed collaboration of multinational departments and enhance the maintenance method and documentation creation processes. Overall, five remote collaborative sessions were held with 20 experts from four countries. Our findings suggest that asymmetry between head-mounted display and Teams users enhances the quality of communication among geographically dispersed teams and their spatial understanding, which positively influences knowledge transfer and efficiency of industrial processes. Based on qualitative evaluation of the asymmetric VR setup, we further suggest a list of guidelines on how to enhance the collaboration efficiency for low-cost distributed asymmetric VR from three perspectives: organization, collaboration and technology. Full article

There is an increasingly urgent need for humans to interactively control robotic systems to perform increasingly precise remote operations, concomitant with the rapid development of space exploration, deep-sea discovery, nuclear rehabilitation and management, and robotic-assisted medical devices. The potential high value of medical telerobotic applications was also evident during the recent coronavirus pandemic and will grow in future. Robotic teleoperation satisfies the demands of the scenarios in which human access carries measurable risk, but human intelligence is required. An effective teleoperation system not only enables intuitive human-robot interaction (HRI) but ensures the robot can also be operated in a way that allows the operator to experience the “feel” of the robot working on the remote side, gaining a “sense of presence”. Extended reality (XR) technology integrates real-world information with computer-generated graphics and has the potential to enhance the effectiveness and performance of HRI by providing depth perception and enabling judgment and decision making while operating the robot in a dynamic environment. This review examines novel approaches to the development and evaluation of an XR-enhanced telerobotic platform for intuitive remote teleoperation applications in dangerous and difficult working conditions. It presents a strong review of XR-enhanced telerobotics for remote robotic applications; a particular focus of the review includes the use of integrated 2D/3D mixed reality with haptic interfaces to perform intuitive remote operations to remove humans from dangerous conditions. This review also covers primary studies proposing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) solutions where humans can better control or interact with real robotic platforms using these devices and systems to extend the user’s reality and provide a more intuitive interface. The objective of this article is to present recent, relevant, common, and accessible frameworks implemented in research articles published on XR-enhanced telerobotics for industrial applications. Finally, we present and classify the application context of the reviewed articles in two groups: mixed reality–enhanced robotic telemanipulation and mixed reality–enhanced robotic tele-welding. The review thus addresses all elements in the state of the art for these systems and ends with recommended research areas and targets. The application range of these systems and the resulting recommendations is readily extensible to other application areas, such as remote robotic surgery in telemedicine, where surgeons are scarce and need is high, and other potentially high-risk/high-need scenarios. Full article

With increasing computing power and data transmission performance of information technologies, the application scenarios for Extended Reality (XR) technologies in industries are growing. Despite the ongoing scientific investigation of industrial XR applications for over 25 years, these technologies are still considered emerging. Within this paper, we present an industry- and business-process agnostic approach for classifying the deployment purposes of XR technologies in value creation. We identified two major research streams regarding the role of XR technologies in value creation: (1) the research initiatives focusing on business-process-specific use case analysis and (2) industry-oriented research reviews. This results in limited identification of suitable application scenarios for new use cases and restricted transferability of the existing use case to future deployments. First, we provide a qualitative analysis of the current research streams. Then, in the second step, by abstracting the XR technology from the existing business processes and the industry-specific context, the generic purposes for XR technologies in value creation are identified and defined. The summary of these deployment purposes results in a taxonomy that enables the identification and transfer of potential use cases of XR technologies in value creation. Full article