Prognostics and Health Management Based on Next-Generation Technologies: A Literature Review

Abstract

With the rapid development of science and technology, the integration and complexity of aerospace vehicles, weaponry, and large-scale chemical equipment are becoming higher and higher. PHM plays an important role in realizing reductions in equipment loss due to failures in many fields. In order to systematically sort through the research history of PHM and deeply analyze the development status of AR and DT technologies in the field of PHM, to clarify the current technical challenges and future development directions and to provide valuable references and insights for researchers, engineers, and decision-makers in the related fields, this paper summarizes the development of PHM in the engineering field. This paper summarizes the development of PHM in the field of engineering, from the initial PHM used in aerospace to the current PHM systems supported by various advanced technologies; analyzes the advantages and shortcomings of the digital twin and augmented reality technologies used for PHM; and organizes and summarizes the future directions and future research focuses of PHM research based on the existing technologies (mainly digital twins and augmented reality). After systematic research and study, we found that the integration of augmented reality and digital twin technologies will provide superb simulation capabilities and immersive operation and bring new challenges and opportunities. Therefore, it is also imperative to address the challenges and limitations that hinder the seamless integration of the new technologies.

1. Introduction

In spaceflight, each space mission requires countries to invest a large amount of their workforce, material, and financial resources. At the same time, the internal structure of spacecraft is complicated. In order to ensure the safety of the lives of astronauts and the successful completion of space missions, scientists have carefully designed the missions and the vehicles. However, vehicles in mission implementations are often in an extremely harsh environment, which massively tests the reliability of their devices once an accident occurs, which can lead to highly significant losses and even jeopardize the safety of personnel. Equipment failures in the chemical and military industries also lead to production stoppages, environmental pollution, and even personal injuries. Therefore, to detect potential failures in advance and take preventive measures, continuous monitoring and prediction of the health status of equipment are essential to ensure production safety and improve productivity. Failures and damages are usually defined as material, geometry, or equipment operation changes. Many methods for identifying failures and damages rely only on periodic inspections or non-destructive testing, so much of our experience of failure can only be acquired after an accident occurs with testing or analysis, which undoubtedly brings substantial economic losses. Prognostics and health management (PHM) provides early fault diagnosis and repair ideas.

PHM is a comprehensive technology that integrates the physical knowledge, information, and data on structures, systems, and components during operation and maintenance in detail [1]. Through this integrated information, PHM enables several vital functions: detecting equipment and process anomalies, diagnosing degraded states and failures, predicting the progression of degradation to failure, and estimating remaining useful life. These functions shift the maintenance strategy from traditional reactive maintenance, which reacts to equipment failures and breakdowns, to a more proactive and preventive maintenance strategy, including scheduled overhaul, proactive prevention, and predictive and integrated planning and management. Nowadays, in the context of Industry 4.0, various technologies, including digital twins, AR, VR, etc., have emerged, and they are also heavily used in various aspects of PHM, dramatically improving the efficiency of fault diagnosis. This paper summarizes the development of PHM in engineering, from the initial PHM used in aviation to the current PHM systems supported by various advanced technologies. Secondly, we analyze the digital twin (DT) and augmented reality (AR) technology for PHM and innovative point-of-service applications. Finally, we analyze the current PHM processes and present our expectations for the future direction of PHM based on the existing technologies. The specific research framework of this paper is shown in Figure 1.

2. Materials and Methods

2.1. The Research Question

The main questions of this research paper are as follows: What are the current challenges encountered in research in the field of PHM? What are the prospects and current challenges in the application of digital twins and augmented reality in PHM?

2.2. Search Strategy

In the broad field of academic research, data collection is not only an indispensable foundation for literature analysis but also a prerequisite for gaining insight into academic development and cutting-edge dynamics. Therefore, it is imperative to obtain a wide range of literature. In this paper, the English literature was obtained from Web of Science and Engineering Village, and the Chinese literature was obtained from China Knowledge Network (CNKI), which is currently regarded the largest academic journal database in China. It is the largest academic journal database in China. Through this selection, the representativeness and authority of our literature sources are guaranteed. The literature review in this paper was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) scheme. The English and Chinese databases were searched separately. When searching for papers, the keywords “Digital Twins”, “Augmented Reality”, and “Prognostics and health management” were used in different combinations with the help of the Boolean operators “AND”, “OR”, and “NOT”. For example, the advanced search formula in WOS is as follows: TS = (PHM) OR TS = (PHM * AR) OR TS = (PHM * DT) OR TS = (DT * AR).

In order to study the progress and development of research studies in PHM, including augmented reality and digital twin technology, in an in-depth and accurate manner, the initially retrieved literature needs to be further screened with the help of pre-determined eligibility criteria so that valuable literature can be included in the study. The eligibility criteria for inclusion in the study are as follows: a publication date from 1 January 2000 to 1 June 2024; type of literature includes journal papers, conference papers, and dissertations; and field of study is engineering, computing, or related engineering and technology fields. Figure 2 provides an overview of the literature screening and selection process. We removed some papers using Endnote X9’s own “Find Duplicates” function. We categorized the retrieved literature by topic in Endnote, PHM, PHM + DT, PHM + AR, and AR + DT, for detailed analysis of each aspect below.

2.3. Analysis of the Data

To comprehensively analyze the research and development of this topic, we used 3032 documents retrieved from three databases. We analyzed them in detail in terms of the number of articles per year and the frequency of keywords. The trend of the number of annual articles from the literature retrieved from the three databases was counted separately, as shown in Figure 3.

An analysis of the number of articles in each database reveals that the growth trend of the number of articles in the three databases is roughly the same. From the figure, we can find that the number of related research topics has gradually increased since 2000. The first surge was generated in 2006, which may have been due to the explosive rise of deep learning at that time, and a variety of advanced deep learning models emerged, providing more and better avenues for PHM research. Then, as we entered the era of Industry 4.0, the rapid development of technologies, including digital twins and augmented reality, once again provided more possibilities for PHM research. As a result, a second surge in the number of publications of related research arose in near 2015. In addition, the Chinese literature from CNKI is relatively sparse and shows a very obvious rise after 2020, exceeding the number of related articles from WOS at this stage.

In order to obtain a full picture of the breadth and depth of the literature, a VOSviewer analysis was subsequently performed, which was displayed as a colorful network of labels using the “co-occurrence” function in VOSviewer. The larger the node circles in the graph, the higher the frequency of the corresponding keywords, indicating that the keywords are the main research topics. The closer the node circles are, the higher the frequency of their co-occurrence in the same literature. The color of each node indicates time, and the gradient color bar in the lower-right corner marks the time course from blue to yellow, indicating the change in the keyword frequency over time. The co-occurrence time zone map effectively illustrates annual research hotspots and their trends over time.

We analyze the co-occurrence of keywords in the retrieved 3032 documents, as in Figure 4, which can more intuitively and clearly describe the research hotspots in the field.

From the map, it can be found that the blue keywords with the highest frequency are health, systems engineering, fault detection, condition-based maintenance, decision-making, and condition monitoring. Therefore, in the early stage, the research on PHM by scholars all over the world focused on the condition-based maintenance of equipment systems and the traditional methods of fault detection. The most frequent yellow keywords are deep learning, forecasting, learning systems, learning algorithms, long short-term memory, and battery management systems. This means that the emergence of various deep learning models has greatly promoted PHM research. More and more scholars have begun to optimize the deep learning models and continuously improve their accuracy and generalization ability to achieve full life cycle monitoring and management of device systems. In addition, we also find that many people are currently applying PHM combined with deep learning to the monitoring of batteries.

3. Research History of PHM

With the rapid development of science and technology, complexity, integration, and intelligence of aerospace vehicles, weapons, and large-scale chemical equipment have become more and more complex; in reducing the loss of equipment due to malfunctions, PHM in the aerospace, manufacturing, and chemical industries and many other areas plays a vital role. PHM mainly utilizes sensing signals collected by sensors from equipment systems under different operating conditions to monitor the health status of the equipment and to mine information about faults or potential faults that have already occurred during operation to achieve life prediction and fault diagnosis. Traditional failure prediction and maintenance are mainly based on past failures or the strategy of investigation and maintenance according to the time of carrying out a reliability analysis, but there are two apparent shortcomings: (1) the high cost of investigation and maintenance; (2) investigation and maintenance of the risk factors are considerable. Therefore, PHM performs well in industrial applications because of its efficient, advanced monitoring means and forward-accurate prediction. The whole process of fault prediction and repair is shown in Figure 5.

3.1. Traditional PHM Research

Initially, PHM was proposed by the U.S. military, relying on intelligent information technology, which they used in the operation and maintenance of aero-engines. GUANGFAN ZHANG et al. proposed sensor selection and localization with a new methodology in PHM systems, which dramatically improved the detection capability of the sensors and minimized the number of required sensors [3]. Liu applied data-based fault prediction algorithms to the study of UAV (Unmanned Aerial Vehicle) PHM systems, specifically by selecting particle filtering algorithms to approximate fit the implied fault mapping mechanism to the system data and to perform fault prediction [4]. SONIA VOHNOUT et al. analyzed the drawbacks of improving the detection by increasing the number of sensor locations, targeting military aircraft, and integrating MEMS sensors with standard commercial microcontrollers to propose an innovative design for PHM data loggers with the advantages of low cost, low power consumption, and lightweight design [5]. DEHUANG CHEN et al. addressed the problem that maintenance technicians could not perform maintenance operations until after the equipment had failed and applied the PHM technology to designing an aircraft maintenance decision-making system based on real-time condition monitoring and successfully realized fault prediction, reporting, maintenance actions, and full-cycle health management [6]. J.R. M et al. proposed that the maturity of PHM focuses on two key performance indicators (KPIs): the NFF (the ratio of undiscovered faults, i.e., the probability of unverified fault detection) and the probability of detection (POD) (which can be accomplished by calculating the global anomaly thresholds that intrude during the simulation of each of the different types of degradation) [7]. MAOGONG JIANG et al. focused on analyzing the relationship between the failure analysis (FA) technique and the PHM system and proved that the FA technique can provide a large amount of information for the PHM system [8]. LU YANG et al. proposed a new idea for designing an aviation PHM framework based on a PHM big data center. They demonstrated the key technologies, scientific issues, and the application system for their engineering solutions in detail [9]. All of these provide a reference for the research and development of aviation PHM systems.

Many scientists’ research and application of PHM systems before 2015 mainly focused on optimizing the sensor performance and improving system analysis and prediction strategies. With the rise of big data and the continuous enhancement of computing power, including the continuous optimization and improvement of various deep learning algorithms such as neural networks (NNs), support vector machines (SVMs), random forests, etc., machine learning (ML), which is the core part of artificial intelligence (AI), has been gradually used in the study of PHM and has demonstrated significant advantages.

3.2. PHM Supported by New-Generation Technology

IKRAM REMADNA et al. provide an overview of the application of deep learning to PHM and show the differences between the existing approaches [10]. SOHEYB AYAD et al. propose a predictive health manager for PHM based on the Internet of Things (loT) and cloud computing technologies to supervise and control geographically dispersed and important machines [11]. WANG WEI et al. proposed fuzzy classification and a neural network based on an on-orbit payload health status monitoring method to address the problem that on-orbit payloads of space vehicles are demanding to maintain promptly and cannot be inspected manually [12]. Tao Fei et al. proposed the emerging technology of a digital twin (DT) that realizes physical–virtual fusion in order to improve the accuracy and efficiency of PHM for complex devices [13]. Yun Y et al. used a data-driven approach with a multilayer perceptual neural network algorithm for the prediction of the remaining service life of the PHM system for a flight engine and found that the method had good accuracy [14]. Wu C et al. proposed a multi-feature deep convolutional migration learning network based on multiple features, which was used on a PHM system to obtain higher-accuracy prediction by extracting multiple features from the raw vibration signals of the bearings [15]. Lee Dongkyu et al. proposed a virtual data generation method for developing a fault analysis tool for fault PHM for RF modules [16]. Yu Y C et al. proposed a deep unsupervised learning-based supervised learning meta-learning method (Autoencoder) for the detection of anomalous vibrations to improve the accuracy of PHM [17]. A summary of the classic papers among these is shown in

Table 1. Summary of classic papers.

References	Field of Research
	Research on Algorithms	Optimization of Sensors	Research on PHM Realization Strategies	PHM Combined with New Technologies	Studies Based on Statistical Data	Related Device Design
Guangfan Zhang et al. [3]	\	√	\	\	√	√
Liu [4]	√	\	\	√	\	\
Sonia Vohnout et al. [5]	\	√	√	\	\	√
Dehuang Chen et al. [6]	\	\	√	\	\	√
J.R. M et al. [7]	√	\	√	\	\	\
Maogong Jiang et al. [8]	√	\	√	\	\	\
Lu Yang et al. [9]	\	\	√	\	\	\
Soheyb Ayad et al. [11]	√	\	√	√	\	\
Wang Wei et al. [12]	√	\	\	√	\	\
Tao Fei et al. [13]	\	\	√	√	\	\
Wu C et al. [15]	√	\	\	\	√	\
Lee Dongkyu et al. [16]	\	\	√	\	\	√
Yu Y C et al. [17]	√	\	\	\	√	\

\ indicates not considered. √ indicates discussed. After literature research, we found that most scholars and organizations have achieved more accurate predictions by continuously improving the sensing accuracy of sensors, and with the development of deep learning algorithms, many scholars have used them in intelligent O&M, which greatly improves the traditional failure prediction and maintenance processes in terms of the insufficiency of failure data, inaccuracy of prediction, complexity of the program, and other difficulties. In addition, some emerging technologies, including digital twins, AR, VR, UAVs (Unmanned Aerial Vehicles), etc., are slowly coming into everyone’s view, so some scholars consider combining them with traditional PHM to shift to a new era of intelligent factories and have achieved some results.

.

4. Application of Digital Twins to PHM

4.1. Research Status

Since its first appearance in 2002, the concept of DTs has evolved. Given the complexity of the concept, a wide variety of definitions can be found in the literature. Rosen et al. state that the concept of a digital twin is two identical physical and virtual spaces that are mirrored to analyze what is happening at various stages of an object’s life cycle [18]. Boschert and Rosen elaborate that a DT encompasses all useful physical and functional information. Both authors agree that it is not only data but also algorithms that describe behavior and determine operations in production [19]. In recent years, Tao Fei et al. added some features to digital twins to define DTs, i.e., the self-evolution of DTs, meaning something living that constantly changes, improves, and evolves while maintaining a comparison between the physical and virtual spaces, and interaction and fusion are the two critical aspects of DTs, so, based on the described features, a systematic “thinking approach” can help to detect DTs.

From the literature we have retrieved, it is clear that DTs have become increasingly popular in fault diagnosis and intelligent maintenance in recent years, and this phenomenon should be attributed to the growing maturity of various aspects of technology, such as the Internet of Things, big data, and the efficiency of machine learning algorithms.

This section focuses on the relationship between digital twins and PHM and the current status of research in this area carried out by many scholars and organizations.

The first to apply digital twins to device PHM was Tao Fei et al. [20]. Li QW et al. proposed a PHM system based on advanced digital twin (DT) technology. They evaluated the safety status of the FAST cable network and predicted the fatigue life of the components in the network through finite element analysis of the DT model [21]. Gu Jia proposed a digital-twin-based PHM system architecture for high-speed rolling stock by constructing a digital twin model of high-speed rail physical entities, fusing real-time data on physical and virtual devices and related twin data and providing theoretical guidance for the engineering of the PHM system [22]. Based on the working principle of a rutting machine, Yang Jia combines its common failure manifestations with digital twin technology and proposes a PHM framework for a rutting machine based on a digital twin [23]. Wenting Han et al. reviewed the research progress on digital-twin-driven fault diagnosis for rocket control systems by addressing the problems in fault diagnosis and health management in the aerospace field and classifying NASA’s digital twin goals [24]. Similarly, Zhu et al. investigated on-board health monitoring and predictive maintenance for civil aircraft engines and utilized digital twin technology to achieve aircraft safety monitoring and fault warning for civil aircraft [25]. A definition of digital twins’ maturity is shown in Figure 6.

4.2. Prospects for Digital-Twin-Driven PHM

In industrial production, equipment failure and maintenance are common problems that not only cause wasted production line downtime but also increase maintenance costs, thus affecting the productivity and profitability of an organization. Digital twin technology can predict potential equipment failures and suggest repairs in advance through real-time monitoring and analysis of digital models of actual physical systems. This predictive maintenance helps to reduce equipment downtime and repair costs, improve equipment reliability and stability, and thus increase productivity and product quality. Digital twins have continued to gain popularity in the aerospace, automotive manufacturing, construction, and energy industries.

With the development of the industrial Internet, the prospects for applying digital twin technology will become even broader. The industrial Internet provides a broader space for developing digital twin technology. Production data can be collected in real time through sensors and intelligent devices, providing more reliable data support for establishing and optimizing digital twin models. At the same time, the continuous progress of artificial intelligence and extensive data analysis technology also provides more possibilities for the application of digital twin technology, which can realize the intelligent management and optimization of production processes.

However, the application of digital twin technology also faces some challenges. First, establishing digital twin models requires a lot of real-time data support. The data’s quality and accuracy are crucial for the model’s establishment and predictive maintenance. Second, the application of digital twin technology requires enterprises to have specific technical and talent reserves, including the ability to understand and apply digital twin technology and an in-depth understanding of the production process.

The application of digital twin technology in the industrial field is promising and can provide enterprises with more reliable and efficient production management and maintenance solutions. With the continuous development of the industrial Internet and artificial intelligence technology, the application of digital twin technology will also usher in more innovations and breakthroughs, bringing more opportunities and challenges to industrial production. In the future, digital twin technology is expected to become an essential support for industrial intelligence and digital transformation, promote changes in industrial production modes and management, and realize intelligence and efficiency within industrial production and system fault diagnosis and maintenance.

5. Application of AR to PHM

Augmented reality technology effectively applies computer, imaging, and information technology. By adjusting the angle and position of the camera, an image is electronically processed so that the virtual object is combined with the actual situation to create a maintenance scene characterized by two-way interaction. In terms of the current practical applications of augmented reality technology, the primary way to integrate information is to create a real world and a virtual world combined with a scene so that some of the original information that is difficult for the human body to perceive is processed by the computer so it can be intuitively demonstrated, thus giving people a feeling beyond reality. The application of augmented reality technology mainly includes three-dimensional modeling, multimedia, multi-sensors, scene fusion, real-time video display, and other means, thus significantly reducing maintenance difficulty.

Xu Yida researches the application of augmented reality technology to civil aircraft fault diagnosis and maintenance, first analyzing the technical principles and main characteristics of augmented reality technology and then elaborating on the practical application of augmented reality technology to civil aircraft fault diagnosis and maintenance according to the three aspects of maintenance object identification technology, maintenance object positioning methods, and maintenance environment construction technology [26]. Zhao Bo et al. summarized the application of virtual assembly and virtual maintenance technology to aerospace. They analyzed the subsequent development trends in China’s aerospace field: combining it with augmented reality technology, relying on domestic platforms, and developing in the direction of miniaturization and lightweight design [27]. Zheng Yao et al. overviewed the three major supporting technologies of augmented reality, looked ahead to the potential applications of augmented reality to manned space engineering, analyzed the technical difficulties involved, and put forward corresponding solutions [28].

The current landscape of AR technology in fault diagnosis and maintenance encompasses a broad spectrum of research and development efforts, both domestically and internationally. In human–computer interaction, AR technology offers a transformative approach to how humans interact with machines and systems. Through the seamless integration of digital overlays, AR facilitates intuitive and immersive interfaces, enabling technicians to access real-time diagnostic information and maintenance instructions hands-free. This paradigm shift in human–computer interaction can elevate the efficiency and safety of maintenance procedures, thereby mitigating the operational risks and enhancing overall productivity. In conclusion, the convergence of augmented reality (AR) technology with fault diagnosis and maintenance practices heralds a new era of innovation and efficiency. By harnessing the immersive and interactive capabilities of AR, industries stand to benefit from enhanced diagnostic precision, streamlined repair strategies, and proactive maintenance interventions. As we navigate the evolving landscape of AR applications, it is imperative to critically examine the intricate interplay between AR technology, human–computer interaction, diagnostic and repair strategies, and the integration with digital twin systems, ushering in a paradigm shift in fault diagnosis and maintenance methodologies.

5.1. AR Human–Computer Interaction

In the forthcoming discussion, we aim to encapsulate the global advancements in AR technology, focusing on its application to human–computer interaction and diagnostic and repair strategies. Augmented reality (AR) technology has garnered widespread attention in recent years due to its ability to overlay digital information onto the physical world, creating an immersive and interactive environment. This technology is promising to revolutionize fault diagnosis and maintenance processes across various industries.

In visual tracking, Peter Kim et al. proposed VisMerge, which combines a thermal imaging head-mounted display (HMD) and algorithms that can temporally and spectrally merge video streams from different optical bands into the same field of view [29]. MOHAMED ABDELNABY et al. used an IntelRealSense (RS) camera to overcome or enhance the problem of developing augmented reality scenarios that can be used to integrate natural human activities with virtual worlds or real-time computer graphics [30]. Malta A et al. presented a deep learning neural network-based task assistant model using YOLOv5s that could successfully inspect parts in real-time video streams with high accuracy, thus helping train professionals to learn to handle new devices using augmented reality [31]. KUN QIAN describes the development of a WAS system with maintenance and assembly instructions, discusses the problem of fast markerless augmented reality (AR) instruction generation based on a monocular camera, and implements a learning-based module for fast inspection of equipment panels, facilitating the deployment of the system [32]. Claudio Cusano proposed a module for the visual recognition of aircraft mechanical parts, which was included in the design of the Alenia Maki M346 in an intelligent maintenance prototype system [33]. Raphael Grasset et al. combined visual saliency algorithms with edge analysis to identify potentially significant image regions and geometric constraints for placing labels. Dandachi et al. proposed an improved image augmented reality approach by acting on two axes in the augmented reality process. First, a machine learning step is incorporated into the detection part. Second, the augmented image is aligned using a statistical appearance model and a dense image descriptor covariance matrix [34]. Chen Yuxiang et al. used RGB-D cameras to optimize the computationally time-consuming part of the traditional visual SLAM method in order to solve the problems of poor real-time 3D scene modeling and the low robustness of loopback detection due to limited hardware resources and the insufficient computational power of the head-mounted device in the application of equipment maintenance using augmented reality [35]. Based on pupil center vision tracking technology using corneal reflection, Xu Xingmin constructed two sets of vision tracking hardware frameworks, namely a stationary-head pedestal type and a helmet type with free motion of the head, and developed a software system to support them. In terms of gesture recognition, You Chao uses a new gesture recognition method to recognize the whole hand of a participant by using gesture image attribute features to preprocess the input gesture to calculate the Fourier description subset of the gesture boundary point sequence and obtain the gesture feature vector by calculating the Euclidean distance and the similarity between the input gesture and a sample gesture in the database for recognition of four types of gestures in the system according to the recognition results. Real-time drawing of auxiliary pictures and realization of virtual overlay functionality within systems have been achieved using augmented reality registration techniques. Nadia Zenati-Henda et al. proposed a real-time collaborative system for remote assistance, in which the remote assistants use gesture-based mobile augmented reality. This method can be used to help remote workers perform manual tasks [36].

It is worth mentioning that Liping Wang et al. proposed a PHM system that combines deep learning and AR. In this technological framework, they skillfully used deep learning models to achieve highly accurate predictions of the remaining useful life (RUL) of equipment. They used AR as an aid to superimpose the prediction results and maintenance guidance information onto the real world. Through a series of rigorous validations, this study not only demonstrated the feasibility and effectiveness of AR in PHM but also proved its significant advantages over the traditional methods [37].

Delving into using augmented reality (AR) technology in fault diagnosis and maintenance is indispensable. This innovation could enhance the precision and expediency of fault diagnosis while yielding substantial reductions in maintenance expenses.

5.2. Integration of AR and DTs

After researching the literature on AR and DT integration, drawing on the 3R technology designed and combining with the technical framework of DTs in the field of manufacturing, we have compiled a technical framework for the AR and DT integration used in industrial PHM, as shown in Figure 7.

The application framework for AR-DT technology consists of three parts: the physical layer, the virtual layer, and the application layer. The physical and application layers interact through the virtual layer, and the virtual space is the link between the physical space and the AR space. The physical layer constructs digital twins for devices through sensor monitoring and other data collection means. The virtual layer includes twin data and models for real-time interaction, design, manufacturing, and other necessary data. The virtual 3D model is projected into the physical space through virtual reality peripherals (movable displays, data gloves, position trackers and helmets, etc.) in the application layer. The operator can perform fault disputes and repairs on the digital twin of the target device through the AR device.

After organizing and summarizing more than 200 literature works retrieved from China Knowledge Network and the Web of Science database, we found that the research on AR technology mainly focuses on vision and gesture in human–computer interaction to provide a technical basis for AR. After organizing them, we found that more and more scholars are considering the application of digital twins to the process of PHM and intelligent maintenance and propose using AR technology in digital twin processes to enhance the connection between digital twins and the real world. For example, Lu Shanyu et al. propose an augmented reality-based multi-view interaction method for digital twin processing systems in response to the problems of heterogeneous and diverse contextual data and the need for more efficient visualization of the human–computer interface in digital twin processing systems [38]. Ding Zhikun et al. propose a digital twin-based AR multi-person collaborative assembly architecture, which decomposes the assembly sequence according to the complexity of collaborative assembly [39]. Sanglub Akekathed et al. proposed a framework that combines augmented reality and digital twin technology (AR-DT) to facilitate the development of digital capabilities [40]. Leeb Kyuhyup et al. explored a construction machine’s operation and job tracking by applying CPSs based on AR and DT [41]. Kuo Wei Ting et al. proposed a digital twin architecture that integrates the latest Mixed Reality (MR) devices and realizes real-time informatics integration of Mixed Reality and artificial intelligence, validated through simulation experiments of warehouse management scenarios [42]. Ke Shiqian et al. designed an interactive augmentation framework that combines virtual reality, augmented reality, and digital twins [43]. Sahoo Santosh Kumar et al. used an augmented reality-based digital twin system to monitor the machining process [44]. Begout Pierre et al. developed a tool that allows operators to match digital twins with their physical twins (PTs) in an augmented reality environment and update their positions when reconfiguring the system [45]. Liu Xinyu et al. developed a device-driven, large-scale, spatially unfoldable human–robot collaborative assembly system using digital twins and wearable augmented reality devices [46].

The realization of accurate human–computer interaction in AR requires multimodal data coupling, fast bidirectional data feedback, and accurate modeling, which requires the support of hardware with powerful algorithms. We investigated several studies that can be used to demonstrate the integration of AR into other industrial applications, such as healthcare, aerospace maintenance, and construction. On the other hand, by integrating the use of DT-AR, the recognition of object types and sizes can be further increased and diversified. Moreover, generative modeling or semi-supervised learning can address data imbalance and a lack of data. In the future, it will be possible to perform statistical tests to study the computational speed and accuracy of the algorithms in terms of recognition.

6. Challenges and Potential

6.1. Major Challenges

PHM has become an academic research priority due to its sophistication in prediction and diagnosis, which is likely to improve industrial military equipment’s safety and reliability significantly.

• As PHM systems become progressively more complex, the data required become syntactically accurate and significant, which requires algorithmic models that can be afforded to be trained in computing and computational hardware. Although deep learning and transfer learning algorithms are improving, they still need help being made operational and in collecting fault parameters from large and complex equipment, and more importantly, the data are often abnormal, scarce, incomplete, and untagged.
• In addition, more advanced theories and devices that can be applied to improving PHM systems are needed.
• One of the problems with PHM is the need for greater interpretability of the algorithmic data, which reduces the trust in their use, especially for safety-critical applications. This has led to the need to find ways to improve transparency and interpretability in order to more clearly understand what and how models predict and ultimately build trust in their use.
• Digital twins (DTs) enable full life cycle inspection and maintenance of installations through advanced simulation, optimization, prediction, and decision-making capabilities. The problem is that the complexity of standardizing DTs has yet to be eliminated due to the limitations of overlaying multiple technologies with multiple data. The implementation of DTs involves support from multiple disciplines, multiple industries, and multiple software. In order to realize full-cycle PHM for digital-twin-to-device pairing, a large number of sensor detections are still required, which will not only consume a large amount of money but also be less efficient. Although there many scholars have optimized and applied machine learning to data detection and transmission for digital twins, there are still some shortcomings that affect the real-time accuracy of PHM.
• Augmented reality (AR) can solve complex equipment situations with few maintenance personnel and technical difficulties and in harsh environments. However, SLAM technology applied in AR, the mainstream visual tracking technology nowadays, has not been able to build an accurate virtual model by extracting features when the image features are insufficient. Through the literature research, we found that the technology and equipment in human–computer interaction have made significant progress. However, there are still problems, such as the interaction dimensions needing to be higher, the realism not being high, and low user acceptance. In terms of equipment, there are even problems such as the significant weight, large size, and cumbersome calibration of AR-supporting equipment, which hinder the popularization of AR.
• The combination of AR and DT opens up a challenging path for PHM, which has the disadvantage of a single technology logic in the currently existing AR-DT-supported PHM.

6.2. Potential and Directions of Development

• After summarizing, we find that PHM needs to be improved, especially in terms of prediction, which needs to be further solved through continuous improvement and exploration of the algorithms, which need to be deployed in practical research with reasonable and adequate arrangements.
• With the emergence of some new technologies and new methods which provide more selective ways for PHM to be realized, including drones, the IoT, digital twins, AR, VR, MR, mobile assistive devices, etc., the research on PHM should be focused on its combination with new technologies and improvement in specific situations, which will surely obtain great results.
• PHM based on DTs should focus more on data cleaning and dimensionality reduction and solve the problem of vast computation volumes to achieve real-time prediction and maintenance accuracy.
• With the development of AR technology, digital twins integrated with AR technology help to enhance the interaction between digital twins and the real world, and more and more scholars have been engaged in this research. Optimizing their technical and logical relationships further and exploring more efficient and reasonable PHM techniques and methods based on AR-DT is necessary for combining AR and digital twins. In addition, the lightweight design of digital twins is an issue that deserves profound consideration, related to whether the operator can accurately and quickly perform whole life cycle fault monitoring and diagnosis for equipment using AR technology.

6.3. Practical and Theoretical Implications

The integration and application of augmented reality (AR) and digital twin (DT) technology have injected new vitality into the field of PHM, enriching the connotations of theoretical research and having far-reaching impacts on practical applications.

Firstly, in terms of theory, it can expand the theoretical framework for PHM and promote cross-fertilization between PHM and other disciplines, which is more likely to generate new research directions. Moreover, the effective application of new technologies to PHM will indirectly promote technological progress in related fields, such as more accurate algorithms, real-time and more accurate DT models, and more realistic and interactive AR experiences. In addition, research in related fields will inevitably lead to close cooperation between academia and industry, thus promoting the exchange of research experiences to solve technical problems.

Practically speaking, the application of new technologies to PHM can greatly improve the efficiency and accuracy of fault prediction and diagnosis, reduce the complexity and error rates of maintenance operations, improve the skill levels of operators, and reduce training costs and time. By realizing intelligent prediction and maintenance management for equipment, the level and competitiveness of intelligent manufacturing can be further enhanced. The application of AR and DT to PHM helps reduce energy consumption and minimize waste emissions, which is important for promoting sustainable development in the industrial sector.

7. Conclusions

The study of PHM has evolved considerably over the last two decades, representing a confluence of interdisciplinary research efforts in engineering, computer science, and advanced technologies. The convergence of these fields has broadened the scope of PHM and presented new challenges and opportunities in the quest for more robust and efficient systems. Integrating deep learning techniques holds promise for enhancing the predictive capabilities of PHM, facilitating more accurate and timely prognostic assessments. Similarly, advancements in sensing technology offer the potential to enable real-time monitoring and analysis, thereby bolstering the overall effectiveness of PHM applications. The integration of augmented reality and digital twin technologies stands to revolutionize the landscape of PHM, offering immersive visualization and enhanced simulation capabilities. This trend underscores the growing recognition of the transformative potential of digital twins and augmented reality in redefining the paradigms of PHM. While acknowledging the strides made in PHM research, addressing the challenges and limitations that impede the seamless integration of new technologies is imperative.

In conclusion, the evolution of PHM, in tandem with advancements in deep learning, sensing technology, augmented reality, and digital twin methodologies, has ushered in a new era of possibilities. As we navigate the frontiers of PHM research and development, it is incumbent upon the scholarly community to forge ahead in addressing the challenges and shortcomings while charting a course toward realizing more sophisticated and efficacious PHM systems.