Quo Vadis Machine Learning-Based Systems Condition Prognosis?—A Perspective

Abstract

Data-driven prognostics and health management (PHM) is key to increasing the productivity of industrial processes through accurate maintenance planning. The increasing complexity of the systems themselves, in addition to cyber-physical connectivity, has brought too many challenges for the discipline. As a result, data complexity challenges have been pushed back to include more decentralized learning challenges. In this context, this perspective paper describes these challenges and provides future directions based on a relevant state-of-the-art review.

1. Mains Challenges

Process health prognosis is essential to reducing downtime during operating conditions, specifically time-to-repair, and increasing the productivity of operating systems through accurate planning of condition-based maintenance tasks, whether the repair process is scheduled under working or non-working conditions. Remaining useful life (RUL), which is the expected time to complete system failure, is the primary focus in the study of system deterioration, aging and damage propagation [1]. Determining RUL requires run-to-failure samples labeled with the real RUL time, which is often difficult to obtain. This being the case, the state of health (SOH) will be assessed instead via health index (HI) and estimating the health stage (HS) [2]. Data-driven methods, especially machine learning, are becoming dominant in the field due to the increasing complexity issues of physical modeling [3]. As a result, increasing system complexity besides advanced cyber-physical connectivity means that machine learning will also be facing challenges related to modeling complexity, decentralized learning, privacy and security as illustrated by Figure 1 [4].

1.1. Modeling Complexity

RUL modeling with machine learning faces many challenges in providing necessary monitoring systems of real-world conditions in terms of generalizing the prediction model to unseen samples for the same system or new similar systems to the studied one. This poor generalization is the result of several facts related to data availability, data complexity, lack of precision, uncertainty of predictions especially for time long-term forecasts, data dynamism and online decision-making.

Data availability: Due to the lack of labeled datasets with real RUL timing, many available works are moving towards accelerated life testing [5,6]. Accelerated degradation experiments provide real-world-like conditions, but lack some real run-to-failure patterns. This is because these experiments are subject to even harsher environments than real ones and recorded samples may suffer from a higher level of non-stationarity. Lack of patterns is the main reason for the poor generalization of training models over unseen samples driven by systems operating in real conditions and not accelerated testing.

Data complexity: It is mentioned that recorded samples from accelerated life experiments resemble incomplete data patterns containing samples with a higher level of complexity due to harsh non-stationary conditions. In addition, even if this data is recorded based on real degradation experiments with real run-to-failure time, continuous change in working conditions due to internal and external constraints (environmental or system-related conditions) drives data with a higher level of dynamicity, massive and rapid change, and produces a very complex feature space that is difficult to manage even with deep representations [1].

Lack of precision: It is undeniable that many works devoted their efforts in estimate model approximate accuracy to well-known metrics such as root mean squared errors (RMSE) and similar metrics. However, it is worth mentioning that prognosis models are not merely a matter of approximation. Indeed, in general, for long-term predictions, the distances between the predicted samples and the desired responses become more distant as the predictions become longer. In this context, it seems that prognosis is a matter of early and late prediction distribution more than of approximation. In this context, precision analysis is considered mandatory to assess the accuracy of the time-to-predict failures.

Precision analysis requires projecting predicted samples into a specific probability distribution function (PDF) that helps determine the amount of early and late predictions as well as their dispersion from the reference value that is assumed to be solutions optimal [7]. For example, Figure 2 shows three different cases of predicted RUL/HI, where the predictions are early, late and good, respectively. It should be mentioned that the data used for illustration, in this case, are related to publicly available linear and cyclic degradation trends of Li-ion batteries [8]. Such a classification could be driven by many approaches, including human-centric approaches and expert backgrounds and assessments of maintenance resource consumption and other potential effects of failures (e.g., damage to reputation and life, financial loss, etc.). Therefore, the decision of whether predictions are good or bad depends on the nature of the system and the predictions. For example, if the system is safety-critical and a failure is completely prohibited, accuracy should be maximized as much as possible. However, deciding what the exact threshold is for this sub-classification depends on the optimal threshold selection standards defined for a specific system [9]. For Figure 2a, prediction errors are scattered far from the center in both early and late cases. Similarly, Figure 2b gives more precision while late predictions dominate. Conversely, Figure 2c shows acceptable prediction results exhibiting less scatter and more concentration towards the preferred reference value.

The distinctive feature of prognostic predictions is that early and late predictions are two different issues and not just a distance from the exact prediction. Indeed, early predictions consume maintenance resources, while late predictions are too harmful and can lead to catastrophic situations and loss of life. Therefore, it is very important to consider penalizing their distributions differently in the PDF function to minimize late predictions as much as possible.

In this case, the main challenge facing machine learning models is to provide a larger concentration of predictions toward the center of the PDF while also balancing (i.e., providing a sort of symmetry) early and late predictions in terms of dispersion to keep the maintenance decision as accurate as possible.

Uncertainty: Uncertainty in RUL/HI prediction models is the result of many factors, including hyperparameters, model structure, approximations, algorithmic and experimental conditions (conditions when aging experiments are made). Uncertainty quantification is necessary to reduce the number of uncertainties in prediction as well as maintenance related to decision-making. Two main categories, namely Bayesian techniques and ensemble learning, are widely investigated [10,11]. The challenge is that existing approaches suffer from some problems. Many of them are computationally prohibitive and can be difficult to calibrate, which can lead to high sampling complexity and may also require major changes in model architecture and training. Figure 3 is introduced to showcase an example of uncertainty quantification of RUL predictions with a 99% confidence interval (CI). In this particular example, predicted samples that fall outside the CI are considered uncertain. The measure of uncertainty in this case is the ratio of samples outside the CI over those inside the CI.

Data dynamism and online decision-making: RUL prediction models have difficulty addressing real online adaptive learning, such as reinforcement learning, in the context of data availability and the difficulty of obtaining such experience in real-world scenarios due to possible mishaps. In addition, simulation is difficult to approach due to the complexity of physical modeling. Thus, most of the studies on this topic are online models based on offline data already collected. These simulations do not address the reality of condition monitoring but they remain theoretically possible [12].

1.2. Decentralized Training

Recent cyberphysical connectivity and decentralized architectures of industrial processes make it difficult to achieve global generalization of machine learning models due to too many challenges such as statistical heterogeneity, systems heterogeneity and communication efficiency in smart infrastructures [13].

Statistical heterogeneity refers to data distribution. Generally speaking, data come from different devices with different working conditions. This means that the data may be non-independent and identically distributed (Non-IDD). In this context, it is therefore very challenging to have a model that can handle this type of collaborative training without experiencing a performance drop. As a result, differences in devices and connectivity methods lead also to differences in data characteristics. This makes it difficult to account for these variations in each training run. Regarding communication efficiency, the challenges remain communication overhead, especially for mobile devices with limited resources (e.g., battery-powered devices). Synchronization between these devices must also be considered when simulating machine learning models due to the nature of communications in smart infrastructure networks.

1.3. Security and Privacy

In smart infrastructures, decentralized learning and data sharing do not satisfy data privacy conflicts of interest. The main challenges in this case are thus to ensure decentralized training under less data sharing. In addition, connectivity makes the immunity of the entire smart infrastructure prone to cyberthreats, which leads to many consequences such as breach of confidentiality, integrity and availability of data.

2. What Do We Have to Work Towards?

Advances in PHM should not be limited to the performance of the prognosis model. Indeed, for recent smart infrastructure technologies, connectivity, privacy and security must be considered. Therefore, prognosis models should be improved in the context of modeling complexity, involving federated learning and secured learning process and information sharing. In other words, what we should design is: “an accurate, precise, secure, and online adaptive decentralized federated learning system” (see last statement from Section 1.1 of [1]).

2.1. Reducing Complexity

In the context of reducing modeling complexity while keeping generalization capability, data generation (e.g., generative models), reducing model architecture, precision analysis, uncertainty quantification and adaptive learning are very important.

Data generation: Generative models, such as autoencoders, including denoising autoencoders [14] and more specifically generative adversarial networks (GANs) [15], are very popular in this field. Denoising autoencoders allow producing robust meaningful representation by training learning models to produce accurate representation under the presence of data corruption. Unlike autoencoders, which are completely unsupervised networks, GANs are used to generate data from different random noises based on two parts, namely the generator, which generates new samples, and the discriminator, which classifies samples as real or false. Generative models are very important in augmenting data by generating new samples that are statistically similar to the training data. These samples are synthetic instances of data very similar to the real ones. In the context of PHM, generative models will help in filling the gap of poor generalization related to the absence of degradation patterns.

In addition, transfer learning also will help generate meaningful representations from different source domains and working conditions to fill in the gap of lack of samples [12].

Improving model architecture: Model architecture increases complexity in terms of computational cost; therefore, more effort should be devoted to developing less complex architectures while keeping accuracy the same as deep networks. In this context, least squares variants and the Kalman filter can be considered when training deep learning models [16,17,18].

Precision analysis: To help the prediction errors to surround the desired PDF value, more effort should be devoted to improving learning models in a prognosis context and not only to accurate approximation. Therefore, the following solutions can be considered: (i) Defining the appropriate loss function to be minimized during the training process, such as the same PDF function of the precision; (ii) Learning from labels autoencoding has also proven its capability to reshape the predicted responses and can contribute to a better fit of the desired results [19].

Uncertainty quantifications: Uncertainty quantification is essential in reducing prediction models’ uncertainty. Accordingly, more efforts should be focused on analyzing the uncertainty of predictions under different algorithmic architectures and data complexity. In this context, the following research areas can be further explored (see [10], § 7.1.2. “Future directions based on applications”): (i) Use of meta-reinforcement learning models for better decision-making with a better certainty; (ii) Approximate Bayesian inference in sequential decision-making applications should be used as an internal procedure of larger methods; (iii) Density Filtering Techniques (ADF); (iii) Ensemble-based sampling; and (iv) Quantification of uncertainties for multi-agent systems.

Adaptive online learning: Reinforcement learning features give the most interesting insights in modern online learning, which allows agents to learn in an interactive environment by trying and correcting their mistakes; it makes predictions based on the feedback of its actions. Additional efforts should be made in a PHM context, as there are only a few contributions of RUL model reconstruction, [20,21,22].

2.2. Federated Learning

Federated learning is the available solution for data privacy in decentralized learning. Its core idea is to train a generalized and global learning model without data sharing [23]. However, works on federated learning are scarce in the context of PHM. Only a few papers are available on this topic [24,25]. Federated learning faces decentralized learning challenges besides privacy concerns related to data sharing. In this context, more emphasis should be placed on considering federated learning for PHM.

2.3. Security

Machine learning-based prognosis models, which are supposed to be federated learning ones, are subject to external threats. In this context, there is almost a complete lack of studies related to PHM while it is the most important area in the field of condition monitoring, especially cyberphysical connectivity and internet of things technologies. As such, more investigation efforts should be devoted to this topic in the future.

3. Conclusions

Most machine learning-based systems condition prognosis studies available in the literature deal with model performance while ignoring intelligent infrastructures’ most important factors, namely decentralized learning, privacy and security of the learning models. In this context, this paper briefly provided readers with the most important challenges faced by data-driven PHM and specifically suggested future guidance to address these challenges. As a result, challenges range from data availability and complexity and drift to statistical heterogeneity, system heterogeneity, communications efficiency, privacy and security of cyber-physical connectivity.