A Review on Fault Detection and Process Diagnostics in Industrial Processes

Abstract

The main roles of fault detection and diagnosis (FDD) for industrial processes are to make an effective indicator which can identify faulty status of a process and then to take a proper action against a future failure or unfavorable accidents. In order to enhance many process performances (e.g., quality and throughput), FDD has attracted great attention from various industrial sectors. Many traditional FDD techniques have been developed for checking the existence of a trend or pattern in the process or whether a certain process variable behaves normally or not. However, they might fail to produce several hidden characteristics of the process or fail to discover the faults in processes due to underlying process dynamics. In this paper, we present current research and developments of FDD approaches for process monitoring as well as a broad literature review of many useful FDD approaches.

1. Introduction

With the advent of industry 4.0, current industrial processes are transforming into smart ones. In particular, many modernized industrial processes are equipped with several well-elaborated sensors to gather process-related data for discovering faults existing or arising in processes as well as monitoring the process status. For this change of industrial environments with the full-automation of equipment and process (i.e., operation), more cautious supervision that includes process control and suitable corrective actions is required to guarantee the process efficiency [1,2,3].

It is an important task to maintain a desirable performance in industrial processes which commonly hold several kinds of faults. Among various process supervision techniques, the fault detection and diagnosis (FDD) is a significantly critical control method for accomplishing this task because most industries hope to improve their process performance through a higher level of FDD capability. The basic functions of FDD can be summed up into two parts, namely (1) monitoring the behavior of process (variables) and (2) revealing the fault presence, its characteristics, and root causes of faults. Thus, in order to maintain high process yield and throughput in industrial processes, it is necessary to adopt fast, accurate, and effective detection and diagnosis tools for process or equipment faults that may degrade the performance of the entire system [4,5,6].

Due to many substantial benefits to be achieved from reduction of process- or product-related costs and improvement of quality and productivity, the FDD for many diverse processes has attracted much attention from both various industrial sectors and academia over several decades [7]. In particular, FDD has played an important role in various industrial engineering processes, for example, semiconductor manufacturing, chemical engineering, and software engineering. So, for the economical and optimal operation of processes, there is an increasing demand for the effective detection and diagnosis of suspicious faults to prevent deterioration of the process that may cause deterioration of product yield or process throughput ultimately. Generally, as a fundamental method for supervision of the processes, the FDD task is performed based on several process and equipment data measured by instruments (i.e., sensors). Also, to date, many applications and researches on the detection of a faulty process (or variable) have considered a variety of competent analytical approaches and those on diagnosis, isolation, and identification of causes of a process anomaly after the detection have adopted various mathematical or probabilistic models [5,8].

There have been numerous theoretical and experimental researches on FDD methods for diverse industrial processes. Methods used for FDD can be classified into data-driven, model-based, and knowledge-based approaches [9,10,11]. Particularly, data-driven and model-based approaches require very little modeling efforts and a prior knowledge on process (or variables) of interest and have made a huge impact on FDD for industrial processes thanks to their relative simplicity and effectiveness for process FDD [12]. The on-line real-time FDD is also another important issue in current process monitoring field, especially in some industries involving dangerous processes, for example, a large-scale chemical processes. Through the FDD tool, it is possible to increase process efficiency and safety by detecting abnormal symptoms in the early stages of appearance of process faults. However, it is very hard to detect faults in control loops of chemical processes due to the feedback masking effects. In order to resolve this problem, a bank of FDD models which can distinguish faulty status from normal status is commonly considered [13].

The rest of the paper is organized as follows: The terminologies used in process monitoring field, concept of process monitoring and FDD, and industrial applications of FDD are given in Section 1. In Section 2, data-driven FDD methods and their applications are given. The model-based FDD approaches and their applications are provided in Section 3. The knowledge-based FDD approaches and their applications are given in Section 4. Different types of FDD techniques, including hybrid approaches and fault prognosis, are illustrated in Section 5. Finally, conclusions and future works are provided in Section 6.

1.1. Fault, Failure, and Malfuctioning

It is of importance to clearly define terminologies used in process monitoring field and also to classify those in terms of their characteristics. A fault can be defined as an unacceptable departure of a process variable from a normal state or behavior. Specifically, unallowable deviation indicates the difference between a threshold value and fault value, and this fault may cause a process malfunction or process failure [5,14]. Faults may be already existing in the process or appear at an unknown time and the speed of appearance of faults can be different [15]. Depending on the occurrence time, faults are usually classified as an abrupt fault (referred as stepwise fault), incipient fault (referred as drifting fault), or intermittent fault. Moreover, depending on the way that faults represent, faults are classified as additive or multiplicative faults [14,16].

A failure is defined as an eternal system interruption to execute a demanded function. So, as a termination indicator of a function of a process unit, the failure is attributed to several faults. Depending on predictability, the failure types can be classified into three categories, namely random (unpredictable) failure, deterministic failure, and systematic or casual failure. Finally, a malfunction is defined as a sporadic interruption of a process or system execution and is usually also a result of faults [5].

1.2. Process Monitoring

For effective process monitoring, the use of FDD tools has received considerable attention from both diverse research and application domains and thus many useful FDD methods have been developed [16,17,18,19]. A generic process monitoring procedure has a loop form that is composed of four steps, i.e., fault detection, isolation, identification, and recovery of the process, as shown in Figure 1 [16].

As one of the famous approaches for process monitoring, many statistical process monitoring (SPM) techniques (also referred to statistical process control (SPC) have been used to improve process efficiencies in various industries. In traditional univariate SPM approaches for single process variable monitoring, it is assumed that all the measured process variables follow the normal (i.e., Gaussian) distribution and are independent and identical. However, the conventional SPC methods have a lot of limitations since such an assumption is often fallacious for the data collected from many real-world industrial processes due to their underlying natures such as multivariate, non-linearity, non-Gaussianity, and non-stationarity [20,21]. For example, it is inadequate to apply a univariate control chart to a multivariate process (or system) if the variables in that process involve a non-linearity characteristic and cross-correlation.

To overcome the limitations of the conventional univariate SPC approaches for the monitoring of dynamic industrial processes, several superior multivariate statistical process monitoring (MSPM) techniques including useful statistics (e.g., Hotelling’s T² statistics) have been developed and implemented for many industrial processes [20,22,23]. Also, for monitoring non-measurable process variables (e.g., status, parameters, etc.), useful estimation methods and process-based models have been extensively studied and widely applied [15].

Particularly, with the development of sensor technology which enables to routinely collect data on numerous process variables, various useful process monitoring and fault diagnosis (PM-FD) approaches based on multivariate statistical methods have gained huge attention. MacGregor and Kourti [12] provided an overview of two famous multivariate-based projection methods, that is, PCA (principal component analysis) and PLS (partial least squares) methods, for process monitoring and discussed their uses for continuous processes and batch multivariate processes. In particular, in this study, they described the effectiveness of the combination of multi-block PLS and contribution plots. They applied PCA and PLS to a continuous process and a batch process in mineral processing plants.

Ison and Spanos [24] extended the multivariate SPC (MSPC) system for fault detection to account for long-term trends on semiconductor wafer lots. In this study, to characterize the long-term variability in optical emission data obtained from a semiconductor plasma etcher, they used the log transformation method to linearize the collected data. Also, through the filtration of the known machine aging effect, they integrated optical emission data with collected sensor signals and presented a robust fault detection system. Finally, they provided the framework of a fault diagnostic system involving various data types and taking into account uncertainty.

To diagnose on-board incipient faults in dynamic systems, Basseville [9] provided a general statistical approach to the design of a new fault detection and isolation (FDI) scheme. In this study, they described the key components of a mathematical statistics theory called local approach which allows performing the on-board incipient FDI tasks [9]. In the current industrial situation, it is of importance to timely detect incipient faults for ensuring optimal conditions of the process as well as preventing any serious deterioration of system efficiency. Even though many MSPM approaches have been adopted for fault detection in various industrial processes, the conventional fault detection methods based MSPM may not deal with incipient faults. Ji et al. [25] first addressed six different fault detection indices used in classical PCA and PLS methods and then, based on those, introduced a new incipient fault detection methods based on a comprehensive fault detection index by incorporating two representative smoothing tools, i.e., MA (moving average) and EWMA (exponentially weighted moving average) methods [25,26].

To enhance the performance of MCCs (multivariate control charts) for process monitoring, Peres and Fogliatto [27] presented the cutting-edge of variable selection methods integrated with MSPC approaches. In this research, they identified and categorized thirty MSPC approaches into 10 groups in terms of their purposes and process monitoring steps. Specifically, they divided the whole procedure into two steps, i.e., filter step (preprocessing and postprocessing) and wrapper step. Among various MSPM approaches, PCA-based methods have been widely used for FDD in numerous industrial processes through extracting critical characteristics and monitoring anomalies. As a dimensionality reduction technique, PCA is capable of detecting a fault presence easily by utilizing a variance structure of data [28,29,30,31,32].

Also, as the variants of PCA, recursive PCA (RPCA) [33], dynamic PCA (DPCA) [20,34], and kernel PCA (KPCA) [35] have been used for the purpose of monitoring diverse industrial processes such as adaptive, dynamic, and nonlinear process [23]. Li et al. [33] proposed two RPCA algorithms to adapt for normal process changes in semiconductor manufacturing processes. For this, they first updated the correlation matrix recursively. In this study, they adopted rank-one modification technique for sample-wise recursion and Lanczos tridiagonalization technique for block-wise recursion, respectively [33]. Kruger et al. [29] considered an insensitivity problem of PCA-based process monitoring approach to incipient process faults which usually change variable covariance structure and underlying DPCA decomposition. For this, they incorporated a local approach into MSPC scheme. Also, they isolated faults by producing a 3D fault diagnosis chart which can reveal any alteration due to fault presence. Through this, they could identify dominant process variables which may contribute to anomalies.

Alcala and Qin [19] analyzed and generalized five existing fault diagnosis approaches. In this study, they showed that the five approaches can be integrated into three general ones and then proposed the expectations of the contributions and the relative contributions of the methods. Through the diagnosability analysis for process faults, they presented that some of the diagnosis approaches may not warrant right diagnosis for naïve sensor faults with small magnitudes. They applied Monte Carlo simulation for comparing the efficiencies of the presented fault diagnosis methods. In order to decide adaptive chart parameters as well as the model size, Khediri et al. [22] presented a real-time fast block AKPCA-based variable window monitoring system which allows for estimating non-linearity in processes to trace normal drifts. They applied the proposed model to monitor two benchmark processes and showed the robustness and improved detectability on process drifts.

It is known that the continuous annealing process line (CAPL) involves many complicated characteristics of the process (e.g., cross-correlations among process variables, multiple operation zones, and interconnection of a lot of subsystems). Many practitioners in CAPL are interested in occurrence of typical process faults in which effects are frequently confined in a particular operation zone [36]. So, process monitoring including fault diagnosis of CAPL has been a primary concern because operational stability in the CAPL affects continuous processing of materials as well as quality of final products [37]. For instance, to handle process nonlinearity, Zhang and Ma [38] presented novel multiscale scheme-based approaches, i.e., MSKPCA and MSKPLS. In this study, they applied general kernel PCA and kernel PLS to the multiscale data for capturing correlations between process variables which occurring at different scales and extracted the non-linearity scores of process variables for fault diagnosis. They applied the developed approaches to a CAPL and a furnace process to show the fault diagnosis performance.

1.3. Fault Detection and Diagnosis

The process supervision aims to show the current process status (or behavior), identify undesired symptoms, and take suitable actions for avoiding unwanted losses. As mentioned above, certain faults may be attributed to several kinds of root causes and the fault may cause an interruption of the process if proper actions are not taken. One of the important reasons for supervising processes is to prevent the interruptions and the main purpose of the process supervision is to properly manage the presence of process faults [3]. Thus, the FDD is a very important task in diverse industrial processes and remains an active research area to assure efficient and safe operations as well as the productivity of the process [4,39,40]. The general implementation procedure of a conventional FDD method (or system) can be divided into four sequential steps as shown in Figure 2.

According to generally accepted terminology, fault detection is a task to determine or indicate faults in a process or system and then the faulty process or variable should be isolated because isolating faults gives additional useful information about the faults, e.g., root cause of a fault [5,41]. Fault diagnosis is a comprehensive task to determine fault type, fault size, location of fault, time of fault detected, and behavior of faults through an appropriate assessment of the faults. Generally, the fault diagnosis includes fault detection, isolation, identification, classification, and evaluation, but sometimes the combination of fault isolation and identification is called a fault diagnosis step [5,16]. In addition, fault detection and classification (FDC) is a specialized term commonly used in semiconductor manufacturing industry [6,8]. Several different types of FDC models used in semiconductor manufacturing processes enable the detection of faults in wafers and equipment by extracting important features from numerous measurements recorded by many in-situ sensors and also to classify the faulty wafers or defects (or defect patterns) through the analyzed information. However, due to the preprocessing of the raw data collected by sensors and noises existing in sensor data, the FDD performance often deteriorates [42].

Currently, many useful FDD approaches are applied to diverse industrial processes. So, there has been a wealth of research in the FDD area, e.g., multivariate methods, analytical methods, artificial intelligence, etc. In particular, for dealing with various FDI problems, many useful methods such as hierarchical strategies have been developed and applied [39,43,44].

For isolation of faulty variables, mainly two different kinds of data-driven techniques have been developed, that is, supervised method and unsupervised method. The supervised method requires the already known fault information to determine the fault subspace or a region of abnormal operation for each faulty status while the unsupervised method is to isolate faults without using a priori information, e.g., the contribution plot of faulty variables from measured data. However, we note that the supervised method may not applicable if the priori information on faulty events is not known [41].

Generally, the FDI procedure is composed of two phases, i.e., residual generation phase and residual evaluation phase. To detect a process parameter change which causes certain structural damages, Döhler et al. [45] provided a local approach and two simple statistical tests, i.e., sensitivity and a minmax test-based residual evaluation method. For illustrating the performance of the developed approach, they presented two simulation examples of vibration-based structural damage diagnosis.

Fault diagnosis for many industrial processes also has been a very crucial task under the several specific situations of the processes, e.g., numerous process variables, contaminated or cross-correlated data measured, and complex relations among the symptoms and faults [46]. So, many useful approaches for fault diagnosis, such as classification techniques or fuzzy-based approaches, have been developed to capture the fault characteristics [5,47].

1.4. Industrial Applications of Fault Detection and Diagnosis Methods

To improve many required performances in industrial processes, a variety of advanced FDD techniques as a basic process supervision tool have been developed for many industrial engineering processes, e.g., semiconductor, automobile, microelectronics, chemical, etc. [15,35,47,48]. Iserman [15] briefly summarized basic fault detection methods including parameter estimation methods for continuous-time process, for example, an electrical driven centrifugal pump with a water circulation system. Isermann [47] also introduced basic problems and methods of process supervision, including FDD in some technical processes. In this study, they presented a knowledge-based procedure and several different types of fault detection methods, which can extract certain key features from data and utilize process and signal models. To compare the process behaviors (i.e., normal and abnormal behaviors), they generated analytic symptoms and heuristic symptoms commonly caused by human operators as another source of information. They also introduced classification methods, approximate reasoning, and fuzzy approaches based on if-then-rules [47].

In chemical industry, FDD is a very useful process monitoring tool for ensuring the safety as well as efficiency of processes. So, for safe and efficient operations, chemical industry requires superior FDD methods that can detect and diagnose process faults more efficiently [49,50,51,52]. As one of the popular FDD tools, PCA has been widely used to detect the presence of abnormalities in chemical processes [20]. Further, FDD in chemical processes is known as a key element of the abnormal event management (referred to as AEM) system that encompasses the three key functions of timely fault detection, fault diagnosis, and corrective action for the faults in a process [39].

However, it is known that, rather than applications of FDD, many studies are placing more emphasis on mainly resolving several practical issues in real chemical processes including how to handle data characteristics e.g., non-linearity, non-stationarity, autocorrelation, cross-correlation, non-Gaussian distribution, etc. [1]. Also, most practical chemical process data commonly involve multiscale characteristics. That is to say, the real data measured in chemical processes contains many time-varying and frequency-varying features and noise and some important features are frequently masked since the measured data may be frequently contaminated with unknown errors [40].

For FDD tasks in many chemical processes, PCA-based approaches have been widely utilized. Among them, to enhance fault diagnosability, Chiang et al. [28] provided an information criterion which enables to decide the order of dimension reduction for Fisher’s discriminant analysis (FDA) and discriminant PLS (DPLS). That is to say, by utilizing the properties of FDA and DPLS methods, they provided an optimal order of dimension reduction of fault diagnosis. They showed that, for diagnosing faults, FDA and DPLS are better dimension reduction tools than PCA by applying those to a simulated Tennessee Eastman example.

Zhou et al. [53] discussed fault diagnosis methods for two special chemical processes. In this research, they first analyzed prediction error of the neural network (NN) models for sensor or component fault diagnosis and then applied an RBF neural classification approach for fault isolation. To evaluate the performance of the NN models, they applied them to two exampled chemical processes. Wang et al. [54] introduced a subspace method for designing a robust FDII (fault detection, isolation, and identification) system. In this study, they proposed four different algorithms to generate residuals for FDII. They showed that each of the proposed algorithms are robust to inputs while sensitive to the sensor and actuator faults through the simulation analysis of vehicle lateral dynamic system. Wang et al. [55] presented a new joint-individual process monitoring scheme based on MCCA (multiset canonical correlation analysis) for chemical processes. In this study, they used MCCA to draw common joint features in the whole processes and projected the measured data in an individual operation unit into the subspaces of joint and individual features and derived statistics to simultaneously examine the variations in the subspaces of joint and individual features. Ming and Zhao [1] reviewed the advancement and challenges of FDD approaches used in chemical processes. In this study, they discussed applications of big data analytics and artificial intelligence (AI) approaches which can handle typical problems in practical chemical processes.

The semiconductor industries require a higher level of process and quality management strategies as well as automation of the processes to improve overall production efficiency. The most emerging issue is to develop and apply effective fault detection and classification (FDC) system for semiconductor production processes [8]. Practically, it is very important to diminish or eliminate any unusual variation presented in the entire semiconductor production processes through a quick and exact detection and diagnosis of faults (i.e., abnormal trends or behaviors) for maintaining consistent and safe production [56]. Thus, FDC in semiconductor manufacturing has been recognized as one of the key components of the APC (advanced process control) framework to enhance overall production efficiency (i.e., yield and machine utilization) as well as reducing variations in processes [21,56,57,58,59,60,61,62,63,64,65,66,67,68]. In the semiconductor industry, several FDC approaches have been developed and adopted to detect faults in the processes and identify the corresponding root causes. However, unfortunately, most conventional FDD approaches may not be adequate enough to describe faults presented in both processes and semiconductor wafers due to the significantly high complexity in semiconductor wafer processing and also the distinctive natures of data produced from semiconductor processes such as non-linearity and multimodality [60,61]. For example, the etching process in semiconductor manufacturing is a highly complicated nonlinear process, which considerably impacts wafer quality. Thus, over the past three decades, various effective FDD techniques for semiconductor production processes have been studied [57,62,63,68]. In particular, to overcome the limitations of conventional PCA-based FDD methods, He and Wang [59] developed a fault detection method based on k-NN, namely FD-kNN, for semiconductor manufacturing processes that commonly involves nonlinearity and multimodal trajectories properties. Fan et al. [69] proposed RF (random forests)-based techniques to identify key status variables, processing times, and steps in semiconductor manufacturing processes. In this study, they integrated k-NNs and naïve Bayes classifiers into the ensemble models and utilized a t-SNE (t-distributed stochastic neighbor embedding) method to visualize the key FDC parameters.

In wind farms known as one of the popular renewable energy systems, process condition monitoring is very important task. So, proper supervision and control of wind turbines is necessary for reduction of operation and maintenance costs. Laouti et al. [11] considered an FDD problem in wind turbines. They employed an SVM (support vector machine)-based FDI approach for diagnosing faults in a horizontal-axis wind turbine with variable speeds which consists of three blades and a full converter. They considered a RBF as Kernel in SVM and then investigated faults in actuators, sensors, and process. Dong and Verhaegen [70] investigated data-driven design methods for FDI on a wind turbine problem which contains non-linearity and unknown wind disturbances. In this study, they proposed a bank of robust fault detection filters and implemented it to a simulation model to show their performances. Kusiak and Li [71] presented a three-level fault prediction approach for wind turbines, that is, identification of faults, categorization of faults based on the severity, and prediction of a specific fault by using information provided by SCADA systems and fault files.

In photovoltaic power generation systems, many faults in accordance with changeable outdoor climate conditions often occur within the photovoltaic arrays [72]. However, due to an inherent non-linearity characteristic in photovoltaic applications, fault detection in photovoltaic arrays is known to be very difficult. To detect and classify hidden faults in photovoltaic arrays, Zhao et al. [73] proposed a semi-supervised learning model based on the graph theory (GBSSL) with utilizing a few normalized labeled data for training. Pei and Hao [72] presented a data-driven method for detecting common PV array faults. To develop the fault detection method, they first identified some important fault characteristics and then defined indicators of the voltage and current of the photovoltaic systems. Through the defined indicators which contain characteristic information of faults, they provided the thresholds fault detection and compared the indicators with the corresponding threshold values of fault detection. By using simulation analysis, they showed the detectability of the proposed method.

In rotating machines, the mechanical FDD task in ball bearings is highly critical for reliable operations since the machine breakdown is mainly caused by faults in ball bearings. Kankar et al. [74] provided a new fault detection procedure for ball bearings in rotation machines. In this study, they suggested fault detection approaches based on two machine learning techniques, namely ANN (artificial neural network) and SVM. They showed the performance of the proposed procedures through the comparative experimental study. Kusiak and Verman [75] applied a data mining approach for detecting faults in bearings of the wind turbine generator. In this study, they attempted to optimize five neural network models in order to identify a relationship between several input parameters and temperature in generator bearing and developed fault prediction model by using historical wind turbine data. For validation of the neural network models, they tested them on two different turbine cases.

For diagnosing faulty block and determining faulty variable in continuous annealing process line (CAPL), Liu et al. [36] presented a hierarchical fault diagnosis approach. They, in this study, proposed a new block contribution and variable contribution approaches based on a reconstruction technique for diagnosis of faulty block and for determination of faulty variables, respectively. They applied the RBBC–RBVC methods to two process fault cases, namely strip-break and roll-slippage, in a real industrial CAPL. Hu et al. [76] proposed an adaptive FDII strategy for resolving false alarm problem in a real process. In this study, they utilized the GLR (generalized likelihood ratio) test and singular value decomposition method for fault isolation and identification, and PCA approach for fault diagnosis, respectively. To show the effectiveness of the presented strategy, they applied it to the imperial smelting process (ISP) which is known as one of the typical metallurgical processes.

2. Data-Driven Fault Detection and Diagnosis Methods

Data-driven FDD approaches have received a lot of attention from diverse industries and been widely applied in complex industrial process monitoring [77,78]. The successful use of data-driven FDD approaches depends on the analytical models used as well as the quality of historical data [11]. The systematic categorization of various data-driven FDD methods according to the system characteristics is given in

Table 1. Systematic categorization of data-driven FDD methods.

Dynamic System	Nonlinear System	Non-Gaussian System	Time-Varying/Multimode System	Non-Stationary System
HMM 1, DPCA, DPLS, DNN 2, Filter methods	KPCA, KICA 3, PF, ANNs, KPLS, LKPCA 4	ICA, GMM 5, KPLS, Bayesian estimation	Recursive PCA, Recursive PLS, Isolation estimation	MM, HSMM 6, HMM, MCS 7, SVM

¹ Hidden Markov model, ² Dynamic neural network, ³ Kernel independent component analysis, ⁴ Local KPCA, ⁵ Gaussian mixture model, ⁶ Hidden semi-Markov model, ⁷ Monte Carlo simulation.

.

Among various data-driven FDD approaches, due to the simplicity and efficiency, PCA-based and PLS-based FDD methods are recognized as powerful tools for detecting and diagnosing process faults [30]. Yin et al. [77] reviewed several data-driven methodologies, including design frameworks useful for monitoring and FDD in industrial processes. For this, they first introduced to PCA-based and PLS-based process monitoring schemes and then reviewed necessary modifications for successful implementation of these schemes. Further, to resolve the uncertainty problems, they proposed an integrated adaptive residual generation technique into the subspace-based process monitoring framework. Wang et al. [79] surveyed fault tolerant control techniques as well as data-driven FDD methods. In particular, they discussed the advances as well as general developments of data-driven FDD and knowledge-based FDD approaches widely used in many dynamic industrial processes. They provided a successful application example and presented future research directions, including many challenging issues in FDD.

Yin et al. [80] reviewed several fundamental data-driven PM-FD (process monitoring and fault diagnosis) methods (i.e., PCA, PLS, ICA, FDA, and SAP). In this study, they addressed the developments and several characteristics of these data-driven methods, such as the original ideas, computational complexities, design, and algorithms used. Qin [81] reviewed the advanced developments of several data-driven PM-FD approaches and their applications. In this study, they discussed the use of a latent variable modeling approach and its extensions for fault detection. Also, they described contribution plot approaches for fault diagnosis and identification and the contribution methods based on a reconstruction technique for fault identification, respectively. Furthermore, they discussed fault detectability and identifiability in fault diagnosis.

2.1. Principal Component Analysis (PCA)-Based FDD Methods

Among various statistical FDD approaches, PCA-based approaches have been successfully implemented to highly complex industrial processes for detecting unknown faults during the operations [26,34]. With the general PCA, several different types of PCA techniques, for example, dynamic PCA (DPCA), and also data-driven reconstruction-based contribution approaches have been frequently adopted for FDD in many industrial processes [82]. To capture hidden disturbances in chemical processes, Ku et al. [20] extended general PCA models by incorporating two properties for SPM (statistical process monitoring) (i.e., static and dynamic). They used a method of ‘time lag shift’ for describing dynamic behaviors in general PCA, namely dynamic PCA (DPCA).

Choi et al. [35] proposed a nonlinear FDI scheme based on KPCA technique for chemical processes. In this research, as a single processing monitoring index, they developed a combined fault detection statistic (Hotelling’s T² and Q) and also the confidence limits for fault detection. They also developed a reconstruction error-based fault identification index. Through the application of the proposed approach to several nonlinear processes, they showed the process monitoring performances of it. Sun et al. [83] considered a sensitivity problem between detection capability and false alarm rate in incipient fault detection. To enhance the detectability on incipient faults and minimize the false alarm rate simultaneously, they proposed a novel PCA-based FDD approach and developed a new data preprocessing method. To validate the fault detection performance of the proposed method, they used a boiler leak detection problem.

In order to describe several proper features of semiconductor production processes, He and Wang [58] suggested a novel fault detection approach based on a pattern recognition technique called PC-kNN. In particular, they showed that the proposed PC-kNN method outperforms FD-kNN in terms of memory storage and computational complexity. The contribution plot technique which does not require a priori knowledge has been used for many fault isolation problems. However, due to the smearing effect, this approach may provide misleading conclusions on the faulty variables. In order to resolve this problem, Liu [41] presented a contribution plot approach based on the maximization of RCI (reduction of combined index) by using missing data approach.

Deng et al. [84] discussed the local structure of data for PM-FD of nonlinear processes and provided a simple fault identification method called local KPCA (LKPCA). Specifically, in this study, they first formulated an optimization problem by incorporating the structure analysis into the KPCA and then transformed the formulated problem as a generalized eigenvalue decomposition problem by using a common kernel trick. Also, they provided two monitoring statistics (Hotelling’s T² and Q) based on the proposed method and then used a kernel density estimation method to compute confidence limits. Finally, they built contribution plots for two monitoring statistics considered in this research by using a sensitivity analysis. To show the fault detection performance of LKPCA, they applied it to the Tennessee Eastman process.

Harrou et al. [52] developed a new PCA-based method for improving fault detectability when there is no process model. In this paper, based on the general PCA technique, they utilized the GLR test for fault detection and tested it on a simulated CSTR (continuously stirred tank reactor) data. Lau et al. [46] presented a novel framework for diagnosing faults arising in dynamic industrial processes. The developed on-line framework involving multiscale PCA (MSPCA) for extracting key process features and ANFIS (adaptive neuro-fuzzy inference system) to derive fault-signal correlation through the process data. Specifically, they partitioned the features extracted from the use of MSPCA into two spaces (i.e., score and residual spaces) and attempted to classify different kinds of process faults by using multiple ANFIS. Liu et al. [37] developed a novel fault diagnosis technique for CAPL. For this, they used a multi-level PCA (MLPCA) technique which is an extended version of a multi-block consensus PCA (CPCA), and then, based on the proposed MLPCA algorithm, they presented a decentralized fault diagnosis method. They applied the proposed approaches to real CARL to demonstrate the effectiveness of them.

To resolve the autocorrelation issue in the conventional Hotelling’s T² and Q statistics, Rato and Reis [85] developed a new multivariate approach. In this research, they proposed a bank of multivariate statistics which are based on DPCA and DR (decorrelated residuals), namely DPCA-DR. Harmouche et al. [26] considered the problem of estimating and detecting unpredictable slight disturbances caused by incipient faults. In this study, for this, they adopted KLD (Kullback–Leibler divergence) theory which allows to detect incipient faults often causing small distortions in process. Specifically, they derived amplitude estimates of incipient faults based on the divergence and then evaluated the divergence performance based on detection error probabilities. In particular, as a criterion to compare the fault levels and noise, they considered the fault-to-noise ratio (FNR). Huang and Yan [86] presented a new method for dynamic process monitoring, namely (DPCA–DICA–BI). In this research, they separated Gaussian-distributed data from non-Gaussian-distributed data by using the Jarque-Bera test and applied DPCA and DICA approaches to both Gaussian and non-Gaussian blocks, respectively, in order to monitor dynamic subspaces.

For handling complex and multidimensional faults with large magnitudes, Mnassri et al. [87] presented two new data-driven fault diagnosis methods. In this work, they presented a generalized reconstruction-based contribution (RBC) approach and also RBC ratio (RBCR) an alternative approach. Mansouri et al. [88] proposed a nonlinear fault detection approach, which consists of a GLR test and kernel PCA (KPCA). In this study, they used KPCA for modeling and a GLR test for fault detection. To verify the efficiency of the suggested approach, they tested it on two simulated examples including synthetic data and CSTR data. Bakdi et al. [89] presented a new data-driven FDD scheme for cement manufacturing plants. In this paper, they modeled the manufacturing process by using PCA and evaluated the common multivariate statistics (i.e., Hotelling’s T² and Q) by using adaptive thresholds. To update adaptive threshold values, they employed a popular multivariate control chart, that is, modified EWMA chart. They demonstrated the effectiveness of the proposed FDD scheme through several faulty cases including a faulty case of a cement plant process. To resolve the non-linearity issue of PCA for fault diagnosis, Kallas et al. [32] presented a KPCA-based fault diagnosis approach. In this study, they discussed on the initial points and presented characteristics of three initialization techniques.

There are many difficulties in fault detection for many real industrial processes such as high-dimensionality, non-linearity, non-stationarity, imbalanced ratio of faulty and normal data, unsynchronized batch trajectory, etc. [2,90]. To handle such dynamics in processes, several useful methods have bene developed, e.g., dynamic multivariate analysis (MVA) method, recursive MVA, moving window MVA, multimodal MVA, etc. [30].

Among them, to identify certain underlying relationships among process variables and also efficiently monitor the process involving such complexity, Jiang and Yan [91] presented a new process modeling and monitoring scheme. In order to defect faults in chemical processes having correlated variables, in this study, they developed a parallel PCA–KPCA approach involving two algorithms (i.e., randomized algorithm and genetic algorithm). They provided several case studies including the CSTR process to verify the performance of the presented scheme.

For efficient FDD, the feature extraction has been considered as an important task [90]. In order to describe the process dynamics and handle class imbalance problem, Du [2] proposed a feature extraction approach based on bispectral features and PCA and one-class classifier. They applied the proposed method to a benchmark example of a real semiconductor etching process and evaluated fault detectability of the proposed approach with considering several classification performance measures.

Various PCA-based methods widely used for many FDD problems do not often work well on the fault diagnosis task. To improve fault diagnosability, several PLS-based approaches have been adopted [28]. Choi and Lee [92] presented a novel FDI scheme using a multiblock PLS (MBPLS) approach. In this study, they first introduced four different kinds of monitoring statistics. Then, they provided new block definitions and derived contributions of variables to Hotelling’s T² and Q statistics to distinguish faults. They also provided the relative contribution to both process variables and blocks to the corresponding faults. To verify the efficiency of the presented approach, they tested it to a wastewater treatment process (WWTP) data, which consists of three operation blocks, in a still mill plant. For FDD in chemical processes involving non-linearity, Botre et al. [51] proposed a GLR test based on a PLS technique. In particular, they used a kernel PLS (KPLS) approach for building nonlinear PLS models and evaluated the KPLS models using a GLR test statistic.

It is known that the wavelet-based multiscale data representation technique enables the efficient separation of deterministic features and probabilistic features existing in a system. So, this technique has been frequently used to extract key features from a process or system. In order to deal with autocorrelation and noise issues in chemical processes, Botre et al. [40] presented a fault detection approach utilizing a GLR test based on multiscale PLS (MSPLS) method. For efficient process monitoring and quality prediction, Jia and Zhang [93] proposed new process modeling and monitoring methods. In this study, they utilized a dynamic kernel PLS (D-KPLS) approach to build a process model, considering a dynamic relationship between input parameters and output process variables, and then proposed a quality prediction method as well as the corresponding process monitoring based on the (D-KPLS) algorithm for fault diagnosis.

2.2. Independent Component Analysis (ICA)-Based FDD Methods

Through PCA-based FDD approaches, it is possible to extract information of the fault locations as well as to reduce dimensions. However, the conventional PCA or KPCA might not identify the complex linear or nonlinear relations among process variables since PCA generally assumes Gaussian distribution [31,35,91]. Compared to PCA-based methods, it is known that ICA-based FDD methods are very useful to handle nonlinear process behaviors. So, to detect some complicated faults in the Tennessee Eastman benchmark process that involves the nonlinearity characteristic, Yang et al. [94] presented a hybrid fault detection approach, i.e., a combination of the canonical variate (CV) analysis method for deriving canonical variates and ICA for extracting independent components (ICs), namely CV-ICA method.

Fan and Wang [95] proposed a new FDD method for industrial processes with non-linearity and non-Gaussianity characteristics called kernel dynamic ICA (KDICA). The proposed approach consists of the ICA and kernel method for extracting dynamic characteristics, and the auto-regressive (AR) model for considering dynamics. Further, for fault diagnosis, they developed nonlinear contribution plots. Tong et al. [96] considered a non-Gaussian process monitoring problem and, for this, proposed a modified ICA (MICA), referred to as double-layer ensemble monitoring method (called DEMICA). In this study, through the DEMICA approach, they incorporated all possible MICA models as an ensemble and used the double-layer Bayesian inference technique for monitoring indices.

2.3. Canonical Variate Analysis (CVA)-Based FDD Methods

Conventional PCA- and ICA-based FDD approaches assume a single distribution of process data and the samples are statistically independent of the past data. However, in process monitoring, these assumptions often cause a lot of false alarms and also a missing detection problem [86]. Compared to PCA-based approaches, it is known that the canonical variate analysis (CVA) approach is helpful in identifying the process dynamics [17]. In particular, as a multivariate data-driven technique, the CVA approach provides moderately good performance when there exist serial correlations among process variables [97].

For the efficient process monitoring, Stubbs et al. [98] developed a simplified CVA-based state space model. Also, in this study, they proposed an efficient estimation method of state space matrices and a precise selection procedure of the dimension of state vectors of a model. Russell et al. [34] discussed the performances of DPCA- and CVA-based fault detection methods by applying score-based, state-based, and residual space-based techniques to the Tennessee Eastman benchmark process example. They compared the sensitivity, promptness, and robustness of all the fault detection methods considered.

To identify process variables associated with faults, Jiang et al. [99] presented a contribution method based on a CVA technique. In this research, they developed two different sorts of contributions for both state and residual spaces to separate faulty variables for these two spaces. Jiang et al. [100] presented a new fault diagnosis method called CVA-FDA. In this study, they considered CVA for dimensionality reduction and FDA for fault classification. By using simulation analysis, they proved that the CVA-FDA has better and consistent fault diagnosis performance that the general FDA method.

Jiang et al. [101] also developed another new CVA approach to investigation of correlation structures in process data. In this paper, they used the feature representation approach of causal dependency (CD) to detect any alteration in correlation structure of the process data. Ruiz-Cárcel et al. [97] provided a case study for the assessment of different FDD methods. Particularly, in this study, they considered a CVA-based FDD approach and evaluated its capabilities by applying it to a complex process with different operating conditions. Samuel and Cao [17] proposed a new KCVA-based monitoring approach to nonlinear industrial processes.

2.4. Signal-Based FDD Methods

For the purpose of decomposing the sensor signals in a process individually and describing process dynamics, some useful wavelet-based approaches have been utilized. In order to obtain a hierarchical strategy, Zhang and Hoo [44] considered a fault detection and isolation problem for a biochemical wastewater treatment plant and provided a new FDI (fault detection and isolation) approach based on a bond graph network method. In this study, they considered PCA for reducing dimension, a discrete wavelet transform (DWT) method for representing the dynamics of the process, Mahalanobis distance measure for calculating the confidence intervals, and a Bayesian network for explaining the conditional dependence existing between faulty areas and fault signatures, respectively.

Hadad et al. [102] presented a new FDC system for nuclear power plant. For this, they utilized an ANN-based wavelet transform method and resilient back-propagation algorithm. Konar and Chattopadhyay [103] presented a fault detection algorithm based on SVM and continuous wavelet transform (CWT) method for monitoring of condition in three-phase induction motor. They compared the performance of the presented method to those of DWT (discrete wavelet transform)-based methods. Villez et al. [104] considered a fault detection problem in a nonlinear system and proposed a new fault diagnosis method based on the Kalman filter (KF). The proposed method is an extended version of KF-based FDI for detecting three more critical faults existing in processes.

3. Model-Based Fault Detection and Diagnosis Methods

Due to the limitation of the classical FDD approaches, many useful model-based (especially, process model-based or mathematical model-based) approaches have been considered [50]. Generally, in order to apply model-based FDD approaches to a process fault detection problem, it is necessary for users to understand the process model in the system since these approaches use the relations among many process variables in order to obtain information on changes according to certain faults [5,11,14]. Generally, the model-based FDD approaches are largely classified into three subcategories, as shown in

Table 2. Classification of model-based FDD methods.

Quantitative Model-Based Method	Qualitative Model-Based Method	Process History-Based Method 1
Observer-based Parity Equation-based Kalman filter-based	Causal model Hierarchical model	Expert systems Trend analysis NN, PCA, PLS

¹ Process history-based method also can be divided into quantitative model-based and qualitative model-based method depending on the data types.

.

Frank [105] reviewed several model-based FDI approaches to automatic processes. Particularly, in order to acquire robustness to modeling errors, this study addressed key principles and techniques of the model-based residual generation methods to by adopting and state estimation and parameter identification approaches. Also, this study discussed the observer schemes for three different fault detection types.

Hwang et al. [4] discussed model-based FDIR (fault detection, isolation, and reconfiguration) methods. They classified FDIR methods into three categories based on three techniques, which are techniques of residual generation, statistical decision, and reconfiguration control. They also discussed various model-based residual generation methods for several different situations, statistical residual testing methods for the occurrence of the abrupt faults, and finally implementation techniques of a reconfigurable control strategy.

Schubert et al. [106] presented a new combined FDI scheme, which consists of traditional model-based approach and MSPC techniques. In this research, they utilized subspace model identification (SMI) technique in order to build proper model by tracking the operations in a process and applied the proposed scheme to generate suitable statistics based on multivariate residual. To show the performance of the proposed scheme, they provided three industrial applications involving three faults types, namely sensor faults, process faults, and actuator faults.

3.1. Observer-Based FDD Methods

As a representative work on observer-based FDD methods, Li et al. [107] developed new fault detection approaches based on diagnostic observer for piecewise affine processes. Particularly, in this study, they optimized fault detectability through the application of the FD approach based on a weighted diagnostic observer. Also, to enhance the efficiency of the FD approaches, they integrated the mode observer into the proposed FD system.

Wu et al. [108] considered the fault diagnosis problem in nonlinear systems involving multiple incipient sensor faults. They proposed a novel FDD method based on ToMFIR (total measurable fault information residual) and on SMOs (sliding-mode observers). Specifically, they used a state and output transformation technique to divide the original systems; a subsystem with sensor faults and the other subsystem with actuator faults.

To enhance fault diagnosis performance in rotating machinery by improving the robustness of the conventional FLO (feedback linearization observer) method, Piltan and Kim [109] proposed a new observer-based FDD technique based on a variable structure FLO. Bernardi and Adam [110] proposed two types of observers based on a LPV (linear parameter varying) technique to detect and diagnose both actuator faults and sensor faults in nonlinear chemical processes.

3.2. Parity Equation-Based FDD Methods

Ding et al. [18] reviewed several different types of PM-FD methods. Based on the review, they proposed modified and integrated methods of the PM-FD techniques. Odendaal and Jones [111] considered the actuator fault detection and isolation (FDI) issue. For this, they provided a new FDI approach based on an optimized parity space. Through parity space, they spanned all the parity relations between the outputs and inputs in a system. Then, through the transformation of the spanned parity relations, they optimized a transformation matrix. Also, they adopted a CUSUM procedure to detect the change time of the residual variance and a pseudo-inverse actuator estimation technique to capture the actuator deflections. To show the performance of the proposed approach, they applied it to both real and simulated cases of aircraft control surface actuators.

Zhong et al. [112] considered a fault detection problem in LDTV systems. For this, they proposed a novel FD approach based on the parity space, specially, in order to lessen computational load in FD performance. For this, they formulated the FD problem as an optimization problem and then applied a Krein space system to find out the optimal FD recursively.

3.3. Supervised Learning-Based FDD Methods

In many PM-FD problems, SVM has been frequently used to find certain special patterns and to classify the extracted patterns according to the types of fault. Widodo and Yang [113] presented a survey on the application of SVM approaches for the monitoring of machine status and diagnosing faults and addressed the future developments of SVM, which are towards expertise- or problem-oriented ways.

Park et al. [65] presented a new fault extraction and detection approach based on cubic spline regression and SVM. In this study, they applied a cubic spline regression method to find step changing points and adopted a SVM approach to build a classifier. Particularly, they regarded the coefficient parameters multiplied to each basis of the corresponding cubic spline regression as the process features and then utilized the extracted process features as inputs for constructing a classifier. Banerjee and Das [114] considered a fault diagnosis problem of the dynamic motor condition. In this paper, they transformed motor fault diagnosis problem with multiple sensors into an evidence fusion problem with considering information extracted from sensor data and then utilized SVM and STFT (short term Fourier transform) approach for fault signal classification.

Bordoloi and Tiwari [115] considered a multi-fault classification problem of machine gears and proposed an SVM approach based on frequency domain data. In order to find optimal parameters in the SVM model, they adopted three heuristic methods, that is, the grid-search, GA, and ABCA (artificial bee colony algorithm) methods. They evaluated the performances of the adopted methods by applying those to a multiclass problem. Xiao et al. [116] considered the problem of determining optimal Gaussian kernel parameters of the one-class SVM (OCSVM) which impact on the classification performances. The determination procedure of Gaussian kernel parameters consists of two methods; (1) DFN method which utilizes the distance information of each sample and (2) DTL method which utilizes detectability of tightness of decision boundaries. They proved the efficiency of the proposed methods by applying those to both UCI and the Tennessee Eastman process datasets.

Jing and Hou [117] considered a multi-class classification problem. In this study, they discussed two popular data-driven methods, namely PCA and SVM, for fault classification and also provided several characteristics, including pros and cons of these methods. Wang et al. [118] proposed a new monitoring method for the processes with multimodality. In this research, to detect transitions of modes as well as faults arising in the multimode process, they adopted hidden Markov model (HMM) approach. In particular, they used a probability ratio (PR) strategy based on HMM to differentiate stable modes from transitional modes and also adopted the Viterbi algorithm to classify data into several different modes. Also, they provided a combined process monitoring index (i.e., a combination of Mahalanobis distance and observation probability). They showed the computational efficiency of the proposed scheme through two numerical experiments.

3.4. Unsupervised Learning-Based FDD Methods

The kNN-based fault detection methods can effectively capture unique characteristics in many real industrial processes, for example, multimode batch trajectories and non-linearity [57]. To account for the correlations among numerous process variables in semiconductor manufacturing, novel methods have been developed, for example, multivariate control charts and statistics. However, the traditional multivariate-based techniques may not work well when the variable are non-Gaussian distributions. Verdier and Ferreira [21] considered a fault detection issue in semiconductor manufacturing and proposed a distance-based false alarm detection method. In this study, they proposed a k-NND detection (called k-NND) based on adaptive Mahalanobis distance.

In many industrial processes, it is very common the number of abnormal (or faulty) instances may be smaller than that of normal (stable) ones and also it is moderately difficult task to label each fault according to the fault types. Zhong et al. [119] presented a novel semi-supervised form of the FDA model, referred to as the SFDA model, which considered both labeled and unlabeled data for fault classification. Kwak et al. [67] considered a fault detection problem when the class distributions are not equal, i.e., imbalanced class distribution. In this study, they used an incremental clustering approach to construct on-line fault detection method, namely IC-FDM, with considering class imbalance. To show the performance of IC-FDM, they tested the proposed method on the simulated data gathered from the real semiconductor plasma etching process. Lee and Kim [63] provided a comparative study on fault detection (FD) modeling algorithms with considering the class imbalance issue in sample numbers. For this, they considered 117 combinations of feature extraction methods, feature selection methods, classification methods, and six different process datasets, and then evaluated all the combinations in terms of algorithm performance and adaptability to semiconductor manufacturing processes.

Due to the distance distortion problem of PCA for dimension reduction, a k-NN method may increase the false alarms and fail to detect possible faults. So, in order to resolve this problem, Zhou et al. [57] suggested a novel hybrid fault detection method. In this study, they used a random projection technique and k-NN rule, referred to as the RPkNN method, for handling multimodality and non-linearity problems in semiconductor manufacturing processes. To improve fault detectability, Zhang et al. [120] presented a novel distance-based fault detection approach to semiconductor production processes with multimodality. For this, they utilized the weighted distance of kNNs and also provided new statistics to remove the impact of variance structure and decrease the autocorrelation of statistics.

3.5. AI-Based FDD Methods

For the detection and diagnosis of process faults, many intelligent system-based methods have been developed [113]. In particular, ANN-based approaches have been successfully implemented for detecting incipient machine faults [121]. Jiang et al. [122,123] discussed a hybrid process monitoring scheme which consists of many useful analysis techniques. Jiang et al. [122] presented a new data-driven hybrid fault diagnosis framework for a power transmission-line network. In this study, they provided the basic idea of the proposed framework and employed several fault diagnosis algorithms to build the framework. Jiang et al. [123] showed the applicability of the framework proposed by [122] on a hardware platform. In this study, they tested the robustness of framework and evolvability by implementing the proposed framework to FPGA evaluation board.

Samy et al. [124] considered the multiple sensor fault diagnosis problem in unmanned air vehicle (UAV). For this, they presented a model-based sensor fault diagnosis approach called as SFDIA under the situation of simultaneous and consecutive occurrences of multiple sensor faults. They employed EMRAN-RBF NN-based method for modeling. Tayarani-Bathaie and Khorasani [125] discussed design and development of a FDI scheme for the aircraft gas turbine engines. In this research, they proposed fault detection approaches based on two different types of neural network models, i.e., the dynamic NN model and time delay NN model, and for fault isolation, they used a multilayer perception network model.

Lee et al. [68] presented a convolutional neural network (CNN) model-based FDC method, referred to as FDC-CNN. Particularly, they captured structural characteristics of data by modifying the convolutional layer of conventional CNN model. To show the effectiveness, they tested it on semiconductor CVD process data. Schlechtingen and Santos [126] presented a comparative study on three model-based fault detection approaches for wind turbines, i.e., a regression based model and two ANN-based models. In this study, using a time-series data stored in SCADA system, they developed three model-based normal behavior fault detection models, and then investigated incipient fault detectabilities of the developed models and prediction errors of the second fault. Also, they applied the full signal reconstruction (FSRC) approach and autoregressive approach to detect faults in gearbox bearing and anomalies in state temperature.

For the enhancement of fault detection performance in a semiconductor metal etching process, Ren and Lv [66] provided a new fault detection approach based on the sparse representation technique (called FD-SR). Specially, they used the representation errors between faulty and normal samples as a fault detection classifier. To identify fault occurrence moment is also a very important task. Chang et al. [8] presented a novel FDC approach for semiconductor manufacturing. For the diagnosis of faulty processes, in this study, they modeled the sequential flow features of data patterns, that is, spatiotemporal patterns in times and orders through the whole process. To improve fault detectability in semiconductor manufacturing processes by resolving non-linearity and multimodality problems, Li and Zhang [61] developed a new kNN method based on diffusion maps called DM-kNN. In this research, they applied the diffusion map (DM) technique to reduce dimensionality while keeping the non-linearity of the data and kNN method to detect potential faults using the low feature space extracted from the DM.

4. Knowledge-Based Fault Detection and Diagnosis Methods

Usual multivariate statistics-based FDD methods utilize data related to faults (or anomalies) of a process (or system). However, because of the process complexity and its underlying characteristics, it is often difficult or sometimes impossible to efficiently perform FDD task without aid of knowledge or understanding of the process. So, it would be necessary to consider the knowledge or rules as well as the process data when we develop or apply FDD methods to various industrial processes. Thus, some useful knowledge-based FDD approaches have been developed for several complicated industrial processes [79]. The cause–effect analysis approach based on fault model, the expert systems based on human reasoning, neural network (NN) approaches based on the relationship between the faults and process variables, and a combination of NN and fuzzy logic are known as the representative knowledge-based FDD methods [79,127,128,129].

Leung and Romagnoli [130] proposed a fault diagnosis system for the chemical processes. They, in this study, integrated MSPM scheme into knowledge-based FD system. Particularly, in order to diagnose fault, they considered PCA principles such as Q statistics, PC score contributions, and PC score deviation contributions. Razavi-Far et al. [131] considered a fault diagnosis problem of nuclear power plants. For this, they presented two types of FDI scheme based on neuro-fuzzy network approaches for U-tube steam generators in nuclear power plants. In this paper, they used Takagi-Sugeno fuzzy models for generation of residuals and isolation of faults, and a Mamdani-type neuro-fuzzy network model for FDI.

To enhance interpretability of fault detection results, Gonzalez et al. [31] incorporated process knowledge into a Bayesian network approach for dimension reduction. For the improvement of the efficiency of the developed method for the processes with non-Gaussian variables and non-linearity characteristic, they combined the dimension reduction method with multivariate KDE method and then tested the proposed approach on industrial plant data to compare the performances of PCA and ICA. Verbert et al. [132] considered a fault diagnosis problem in HVAC systems. In this study, they proposed a Bayesian network approach based on multiple models, considering interdependencies of system components and the case of multiple operating modes. So, they developed a different fault diagnosis model based on the Bayesian network and various system conditions, such as interdependency and conservation laws. Don and Khan [133] proposed a new hybrid FDD approach based on process knowledge and process data. They used the HMM (hidden Markov model) for detecting faults and a Bayesian network (BN) for precisely diagnosing the root causes of faults. In particular, they adopted process knowledge and results obtained from HMM as inputs of BN, and used the log-likelihood method to derive conditional probabilities of BN.

5. New and Hybrid FDD Methods, and Fault Prognosis

5.1. New and Hybrid FDD Methods

Chen and Blue [64] proposed a generalized moving variance-based health indicator which is independent of process recipe. In this study, they used a moving variance-covariance method for the estimation of variabilities during a process run, and a EWMA control chart for modeling of the patterns and for monitoring and detecting any changes in the tool health. The demonstrated the effectiveness of the method using two real semiconductor manufacturing processes, i.e., PECVD and PVD. Ko et al. [60] proposed a new fault detection approach for semiconductor manufacturing processes. In this study, they used the sensor signal data produced in semiconductor processes, which involves non-linearity and multimodality characteristics, and then built shared univariate and multivariate models by utilizing the structural features of the collected data, which are the geometric information of sensor signals. In particular, they considered the leaf nodes in the extract tree model as individual recipes and intermediate nodes in the tree as fault-detection criteria. They showed the performance of the proposed approach by using semiconductor etch process data.

In order to handle two unique characteristics in semiconductor manufacturing processes, Yu [56] proposed a new Gaussian mixture model (GMM) based on principal components called PCGMM. Furthermore, they introduced two assessment indices for process states and adopted Bayesian inference strategy to calculate process failure probabilities. To show the performance of the approach proposed, they tested it on two simulated process data and a semiconductor process data. For handling within-mode non-linearity in multimode non-Gaussian processes in the FDD of a chemical process, Yu [134] proposed an inferential monitoring scheme based on nonlinear kernel GMM (namely NKGMM). To show the performance of the developed scheme, they considered a wastewater treatment process with multimode.

Mallick and Imtiaz [135] presented a hybrid FDD method for processes in a dissolution tank. In this research, they adopted conventional PCA approach and Bayesian belief network (BBN) model. The extracted information (i.e., PCA contribution of individual variable) from PCA is used as an input for BBN. Rodrigues et al. [136] proposed a new FDD scheme including fault estimation technique for diagnosing sensor faults in polytopic LPV systems. In this study, they used a polytopic LPV filter approach and fault magnitude estimation method for detecting sensor faults and showed the effectiveness of the synthetic method by applying it to a transport process.

Rusinov et al. [13] applied the chemometrics for efficient fault diagnosis of chemical processes. In this study, they suggested two Kalman filter models (i.e., EKF and UKF models) to distinguish faulty operations and normal operations. Yu [137] presented a novel FID (fault isolation and diagnosis) scheme. In this study, they proposed a Gaussian mixture contribution approach based on the Bayesian inference method (namely BIGMC) for identifying multiple Gaussian modes and integrating variable contributions.

Hao et al. [138] developed a new data-driven FD method. In this study, they focused on the multiplicative degradation problem in fault diagnosis performance and used KPI-based PM-FD schemes to identify process variables with faults and root causes of the corresponding faulty process variables from the performance degradation viewpoint. They validated the performance and applicability of the approach through many examples. Ichalal et al. [139] considered a fault diagnosis problem for nonlinear systems illustrated by the Takagi–Sugeno model. In this study, they synthesized residual generators and the disturbance influence to maximize fault effects but minimize perturbation effects at the same time. Sobhani-Tehrani et al. [140] presented a new hybrid FD approach for nonlinear systems. For this, they adopted a mathematical model and useful computational intelligence techniques. Specifically, they designed an individual neural parameter estimator (NPE) using a single-parameter fault model, and also provided two NPE structures and update rules of decision logics and weights of the respective FDI. Furthermore, they introduced a fault tolerant observer to estimate unmeasured system states.

In order to diagnose four kinds of faults (i.e., unknown, known, dependent multiple, independent multiple faults), Chiang et al. [90] presented a fault diagnosis framework. For this, they considered two approaches (i.e., a modified distance-based approach and modified causal dependency-based approach) and then integrated the features extracted from the two approaches with features based on the propagation path. De Lázaro et al. [141] compared five assessment criteria combined with two data preprocessing techniques, i.e., KFDA and KPCA, to change a Gaussian kernel which is used in fault diagnosis. In this study, they used an ANN model for fault classification and validated the performance of the KFDA and KPCA by testing it on Tennessee Eastman benchmark process data.

Yu et al. [142] adopted a NLGBN (nonlinear Gaussian belief network) model for the fault diagnosis of industrial processes. For this, they proposed a three-layer NLGBN model for feature extraction from noisy data and utilized sigmoidal functions to identify nonlinear relationships among process variables and layer latent variables. Zhang and Ge [43] provided two decision fusion systems based on the Dempster-Shafer evidence theory. Specifically, they first adopted a resampling technique to improve the diversity performance and then used the ‘ALL fusion system’ and ‘SELECTIVE fusion system’ for better decision making in fault identification and classification.

Chen et al. [143] considered the multiplicative fault detection problem in chemical processes. For this, they proposed a fault detection method based on the CCA (canonical correlation analysis) to detect multiplicative incipient faults and then incorporated the fault detection method with a moving window technique. Lee et al. [144] considered class imbalance problem in a fault detection task. In this research, they considered 19 fault detection models and then compared the performances of all the models by using two semiconductor process data. Li et al. [82] provided a new framework for identifying the root cause locations of process faults. For this, they employed Granger causality analysis and also, to identify the causal relationship among candidates, they proposed a causality index based on DTW (dynamic time warping) technique. Marseglia and Raimondo [145] considered the input signal design issue for FDI tasks. In this research, they proposed a multi-parametric programming approach and graph theory to reduce the complexity and improve the computational efficiency in FDI tasks.

To handle many complexity, noise, and dimensionality issues in FDD, Nor et al. [49] developed a novel method for constructing the FDD framework. In this paper, they considered a wavelet analysis approach, KFDA, and SVM classifiers, and then properly combined these approaches to enhance fault diagnosability in chemical processes. To show the performances of the combined methods, they applied them to Tennessee Eastman benchmark process. For detecting incipient faults in a process, Shang et al. [23] suggested a novel data-driven fault detection approach based on recursive transformed component analysis. In this study, they used this method to reduce computational complexity by recursively transforming process data into orthogonal transformed components by using rank-one modification technique. Also, they proposed several statistics to identify process features, e.g., higher-order statistics, detection index, and eigenvalue of Wishart matrices.

Zhang et al. [146] presented a new KPI-based PM-FD framework to improve the monitoring efficiency of large-size industrial processes. They provided a static approach and dynamic approach for two different cases: (1) a case wherein a certain statistical relationship exists between process variables and KPIs and (2) a case wherein a dynamic relationship exists between process variables and KPIs. Zhao and Gao [147] considered problems of data distribution change and sensitivity issue of MSPC techniques to incipient changes of process variables. So, in this research, they presented a new FDD approach based on the feature of data (i.e., data covariance structure) to enhance incipient fault isolation performance. For this, they first considered the distribution dissimilarity decomposition technique to extract dissimilarities existing between faulty and normal conditions of processes and then formulated an optimization problem using a sparse regression technique to isolate faulty process variables related to change of data distribution structure.

Zhao [148] considered a fault detection problem for correlated process data. For this, they proposed a novel fault detection approach, referred to as dynamic graph embedding (DGE), to keep important characteristics of process variables and data such as structural information in process variables and serial (or temporal) correlation among process data. Then, they provided new similarity matrices based on a finite Markov chain to extract significant features of the process. To show the performance of the algorithm proposed, they applied it to the Tennessee Eastman benchmark process. Zhang and Li [62] considered a non-linearity problem in semiconductor etching process for fault detection. For this, they presented a new fault detection approach based on multiway principal polynomial analysis. To show the performance of the presented approach, they tested it on a numerical example and semiconductor etching process data.

5.2. Fault Prognosis

The fault prognosis task for industrial processes is necessary to estimate or predict the process operation time or location before a certain fault or an abnormal change occurs and a suitable action, e.g., a preventive maintenance task, is taken. So, owing to the increasing dynamics and complexity in many real industrial systems, the development of effective fault prognosis methods is receiving great attention from diverse process industries [149].

Orchard et al. [150] introduced a fault diagnosis and failure prognosis framework for nonlinear and non-Gaussian systems. The suggested framework consists of two modules: (1) a FDI module and failure and (2) a failure prognosis module. Specifically, to detect abnormal conditions (i.e., faulty conditions), the FDI module based on a hybrid state-space model and the particle-filtering (PF) approach has been utilized. And then, to estimate the evolution time and compute the state pdf (probability density probability) of the RUL (remaining useful life) of faulty parts (subsytems) in the system, the failure prognosis module based on a resampling algorithm and Epanechnikov kernels has been utilized.

Li et al. [151] presented a multivariate-based fault prognosis technique for industrial processes. In this study, they considered a multivariate time series prediction method with the conventional SPC technique for detecting hidden faults in continuous processes. In particular, they used a reconstruction method to estimate fault magnitudes and then a vector AR model to predict faults with considering a denoising technique based on wavelet. Also, they proposed a new integrated fault index.

Martínez-Rego et al. [152] proposed an automatic fault diagnosis and prognosis system for the predictive maintenance of mechanical components. In this research, they applied an on-line SBLLM (sensitivity-based linear learning model) approach to predict the capabilities of the mechanical components using component vibration data, and then compared the performance of the developed system to that of the OSELM (on-line sequential extreme learning machine) algorithm.

Lu et al. [153] considered an information delay problem in fault prognosis of chemical processes. In this study, they proposed an information synchronization technique based on the time delayed mutual information to decide the directionality of information and estimate time delay among process variables. With the derived SDG (signed diagraph) from the estimated directionality of information and time delay, a PCA-based fault prognosis method has been applied to detecting abnormalities in the early stages of the process.

Ekanayake et al. [154] provides a summary of a graphical model-based fault diagnosis and prognosis techniques, in particular, automata-based methods, Petri Net-based methods, and bond graph-based methods. Pilario et al. [155] proposed a new methodology for incipient fault diagnosis and prognosis. For enhancing the detection sensitivity, in this research, they presented a novel statistical index based on CVA (canonical variate dissimilarity) analysis method. Especially, they utilized contribution maps for fault diagnosis and a combination of CVA state-space prediction method and Kalman filter approach for fault prognosis, respectively]. Vogl et al. [156] reviewed various characteristics of PHM (prognostics and health management) technologies for manufacturing systems including needs, methodologies, best practices, and future challenges of PHM.

Zhong et al. [149] reviewed different types of data-driven fault prognosis approaches for various industrial processes. In this overview, they first introduced critical issues as well as unique characteristics of several data-driven fault prognosis approaches and also addressed the links between fault prognosis methods and fault diagnosis methods. Furthermore, they summarized the current development, future trends, and challenges of researches on data-driven fault prognoses as well as their pros and cons.

6. Conclusions and Future Research

FDD approaches for effective process monitoring have gained considerable attention from both various process-oriented industries as well as academia [18], and thus many useful process monitoring systems including FDD techniques have been exploited and implemented for several industrial processes [19].

However, there are still lots of difficulties in the implementation of the FDD methods for real industrial processes due to the unique characteristics (e.g., multivariate, correlation, non-linearity, non-stationarity, multimodality, class imbalance, etc.). So, to bridge the moderately large gulf between the theoretical approaches and the implementations, it is necessary to consider novel hybrid approaches as well as to design more elaborate FDD models using various intelligent techniques [77,79,122].

Fault prognosis is also a very important future research area. If the location and moment of faults arising in the processes can be estimated or predicted properly, then a suitable action may be taken for preventing further degradation and serious damage of the products, and avoiding certain safety problems in advance [79]. Finally, in the era of big data, the real-time and comprehensive FDD strategies using all the information should be developed. In conclusion, this review would be helpful for both practitioners and researchers in this field to comprehend the key characteristics, applications, and future challenges of the process monitoring and FDD approaches [149].