Search Results (4)

The recent advances in Industry 4.0 have promoted manufacturing industries towards the use of augmented reality (AR), virtual reality (VR), and mixed reality (MR) for visualization and training applications. AR assistance is extremely helpful in assembly task visualization during the stages of product assembly and in disassembly plan visualization during the repair and maintenance of a product/system. Generating such assembly and disassembly task animations consume a lot of time and demands skilled user intervention. In assembly or disassembly processes, each operation must be validated for geometric feasibility regarding its practical implementation in the real-time product. In this manuscript, a novel method for automated assembly task simulation with improved geometric feasibility testing is proposed and verified. The proposed framework considers the assembly sequence plan as input in the form of textual instructions and generates a virtual assembly task plan for the product; furthermore, these instructions are used to ensure there are no collisions using a combination of multiple linear directions. Once the textual instructions achieve geometric feasibility for the entire assembly operation, the visual animations of the assembly operations are successively produced in a game engine and are integrated with the AR platform in order to visualize them in the physical environment. The framework is implemented on various products and validated for its correctness and completeness. Full article

The engine is the heart of the aircraft and deeply affects performance, flight safety, and airline management, which makes it a critical component requiring maintenance. With the advancement of technology, augmented reality (AR), virtual reality (VR), and mixed reality (MR) technologies have been introduced one after another. To increase maintenance skill ability, it is important to train for aviation maintenance. Therefore, the engine maintenance mixed reality (EMMR) system was developed through mixed reality technology in this study. There are three different scenarios have been developed in the EMMR system for making engine maintenance easy. In the first scenario, an exploded diagram of the disassembled engine with the main components has been made and an animation of the fuel flow has been demonstrated to introduce the basic structure of the engine. In the second scenario, interactive buttons and animations have been used to simulate the removal of the engine from the wing of an airplane so that the users can understand the process of disassembling the engine. In the third scenario, hybrid reality has been implemented to interact with the virtual objects, so that users could simulate the disassembly and replacement of the engine fan blades and get familiar with the whole process. Students have been introduced to and have been familiarized with the Microsoft HoloLens 2 and a survey has been conducted before and after using the developed technology to understand if the currently developed EMMR could help students to understand the process of each engine maintenance in the general classroom and make the learning more efficient at the same time. Full article

Metaverses embedded in our lives create virtual experiences inside of the physical world. Moving towards metaverses in aircraft maintenance, mixed reality (MR) creates enormous opportunities for the interaction with virtual airplanes (digital twin) that deliver a near-real experience, keeping physical distancing during pandemics. 3D twins of modern machines exported to MR can be easily manipulated, shared, and updated, which creates colossal benefits for aviation colleges who still exploit retired models for practicing. Therefore, we propose mixed reality education and training of aircraft maintenance for Boeing 737 in smart glasses, enhanced with a deep learning speech interaction module for trainee engineers to control virtual assets and workflow using speech commands, enabling them to operate with both hands. With the use of the convolutional neural network (CNN) architecture for audio features and learning and classification parts for commands and language identification, the speech module handles intermixed requests in English and Korean languages, giving corresponding feedback. Evaluation with test data showed high accuracy of prediction, having on average 95.7% and 99.6% on the F1-Score metric for command and language prediction, respectively. The proposed speech interaction module in the aircraft maintenance metaverse further improved education and training, giving intuitive and efficient control over the operation, enhancing interaction with virtual objects in mixed reality. Full article

Most electrical substations are remotely monitored and controlled by using Supervisory Control and Data Acquisition (SCADA) applications. Current SCADA systems have been significantly enhanced by utilizing standardized communication protocols and the most prominent is the IEC 61850 international standard. These enhancements enable improvements in different domains of SCADA systems such as communication engineering, data management and visualization of automation process data in SCADA applications. Process data visualization is usually achieved through Human Machine Interface (HMI) screens in substation control centres. However, this visualization method sometimes makes supervision, control and maintenance procedures executed by engineers slow and error-prone because it separates equipment from its automation data. Augmented reality (AR) and mixed reality (MR) visualization techniques have matured enough to provide new possibilities of displaying relevant data wherever needed. This paper presents a novel methodology for visualizing process related SCADA data to enhance and facilitate human-centric activities in substations such as regular equipment maintenance. The proposed solution utilizes AR visualization techniques together with standards-based communication protocols used in substations. The developed proof-of-concept AR application that enables displaying SCADA data on the corresponding substation equipment with the help of AR markers demonstrates originality and benefits of the proposed visualization method. Additionally, the application enables displaying widgets and 3D models of substation equipment to make the visualization more user-friendly and intuitive. The visualized SCADA data needs to be refreshed considering soft real-time data delivery restrictions. Therefore, the proposed solution is thoroughly tested to demonstrate the applicability of proposed methodology in real substations. Full article