Sensor Virtualization for Anomaly Detection of Turbo-Machinery Sensors—An Industrial Application ^†

Abstract

We apply a Granger causality and auto-correlation analysis to train a recurrent neural network (RNN) that acts as a virtual sensor model. These models can be used to check the status of several hundreds of sensors during turbo-machinery units’ operation. Checking the health of each sensor is a time-consuming activity. Training a supervised algorithm is not feasible because we do not know all the failure modes that the sensors can undergo. We use a semi-supervised approach and train an RNN (LSTM) on non-anomalous data to build a virtual sensor using other sensors as regressors. We use the Granger causality test to identify the set of input sensors for a given target sensor. Moreover, we look at the auto-correlation function (ACF) to understand the temporal dependency in data. We then compare the predicted signal vs. the real one to raise (in case) an anomaly in real time. Results report 96% precision and 100% recall.

1. Introduction

Turbo-machinery units are equipped with hundreds of sensors to monitor their health during functioning [1,2]. Some of these sensors measure primary physical quantities, which can affect the overall health of the machine. Thus, detecting the improper behavior of sensors or mechanical equipment is a critical task in energy [3,4] and the mechanical industry or, generally speaking, in every IOT-related industry [5]. Detecting unexpected behavior is also a challenging task [2,6]; indeed, in many real-world problems, samples from the unexpected classes are of insufficient sizes to be effectively modeled using supervised algorithms [7]. Anomaly detection identifies novelty cases by training only on samples considered normal and then identifying the unusual cases [8,9,10].

1.1. Problem Statement

In this domain, monitoring some sensors is important because they can trigger alerts; in that case, a machine shutdown and manual inspections are required, with an associated cost. Sometimes the triggers are false since they are caused by a sensor failure, not by a machine issue. Hence, early detection is required to avoid undesired shutdowns. Indeed, if a sensor is about to break, service operations can exclude this sensor from the control strategy.

We want to detect possible faults (anomalies) in the sensors installed on our turbo machines (Figure 1) to prevent unnecessary inspection/shutdown efforts by site engineering while making sure that correct triggers, instead, are not ignored.

The challenge consists in dealing with these aspects:

• Early detection is required: only a prompt action allows to avoid the high potential costs of unnecessary shutdowns.
• Up to few thousand sensors need to be checked daily.
• Recall is key: anomalies detected by the tool will be checked by operators and vice versa, where if no alert is given, the anomaly may remain undetected.
• Precision should be kept under control: too many false positives would increase the set of signals to be checked and may invalidate the benefits.

1.2. Related Works

Many other authors have tried to solve similar problems with different techniques: Malhotra et al. [11] apply recurrent neural networks (RNNs) for anomaly detection on aircraft. Park et al. [12] and Pereira [13] uses variational recurrent autoencoder and clustering to detect anomalous time series in healthcare. Geiger et al. [14] applies generative adversarial networks (GANs) and LSTM to identify the temporal correlations of time-series distributions (see also [15,16]). Zheng et al. [17] apply long short-term memory for residual useful life estimation. In a similar research, Strazzera et al. [1] confirm that LSTM outperforms the not recurrent neural network also in the domain adaptation. Zhang et al. [18] extend reinforcement learning (RL) and the Markov decision process [19] to build a general framework for fault prediction and residual useful life estimation. Several other authors (Yang [20], Pawełczyk and Sepe [21]) use machine-learning-based prediction models for gas turbine operating parameters estimation (see also [22] for a small review). They find that machine learning techniques are applicable to any of the gas turbine parameters when reference physics-based models and large sensor measurements datasets are available to validate the accuracy of the data-driven algorithms developed. Escobedo [23] uses the Bayesian technique and feature extraction to scale up to a broad large mechanical equipment fleet.

2. The Dataset

Our data are output from all sensors installed on a turbo machine [1,2] and are acquired at a frequency of one sample per second. Different kinds of sensors like temperature, pressure, speed sensors have been acquired and comprise our database. Among these sensors, the ones which are critical for machine control are considered “output sensors” in our work. In fact, those are the sensors whose health needs to be monitored to be sure that an eventual alarm triggered by them is actually due to a machine failure, not to a probe failure. The remaining sensors can be used as input features for building virtual sensor models (digital twins) of the first set of sensors. In this work, we focus on one target sensor only to explain the process more easily.

The dataset was collected during 14 months of machine operation (1 s sampling interval). It was split into training (10 months data), validation (1 month data) and test (3 months data) sets. The training data have no reported anomaly, while the validation and test sets have some anomalies reported.

3. The Model

3.1. Selection of Input Sensors

There are more than 200 sensors that can be used to build a virtual sensor for each output sensor. We used the Granger Causality [24,25] test to determine the subset of input sensors that have a causal effect on the target. For the target sensor shown here, we identified around 15 input sensors to be used to reconstruct the same.

3.2. Selection of Lookback Window

We used the auto-correlation to find the temporal relations in both input sensors and target sensor to obtain the best “window size” to train the LSTM model.

Figure 2a shows the auto-correlation function (ACF) graph for one of the sensors: we can see high correlation values among all the lags, which represent the presence of strong trend (non-stationary series). Hence, we need to make this series stationary by differentiating to view if there is any seasonality present in the data.

Figure 2b shows the auto-correlation function (ACF) graph after second-order differentiation: there is no significant seasonality and a small degree of trend. Thus, we can conclude the absence of seasonality but the presence of strong trend (strong correlation among first few lag values). Moreover, we know from subject matter experts that in turbo-machinery applications the thermocouple thermal inertia is less than 5 s [26]. Hence, we chose a sliding window of five samples for this temperature-measuring sensor selected as output. Indeed, after five samples, ACF shows highly sparse values.

3.3. Model Training

We used a deep learning model with two long short-term memory (LSTM) layers of 32 nodes each, with tanh activation, followed by four fully connected layers with ReLU activation. We used Adam optimizer to train the model and a callback on the validation set to stop the training. We used a semi-supervised [2] approach and trained the model on non-anomalous data only to build a virtual sensor acting like a digital twin of the sensor itself [11,27]. In Figure 3, we can see that the model is able to correctly reproduce the actual signal.

3.4. Inference Logic

Once the model has been trained, we use it to reconstruct the signal in a time region when sensor anomalies may have occurred. To distinguish between anomalous and non-anomalous samples, we identified a criterion based on the level of agreement of the actual sensor with respect to the virtual one.

Given the actual signal $y_i$, with $i=0,\cdots ,T$, where T is the signal length, and the related virtual signal $\widehat{y_i}$, with $i=0,\cdots ,T$, the discrepancy $\Delta y_i=abs(\widehat{y_i}-y_i)$, $i=0,\cdots ,T$ can be calculated. We declare $y_i$ to be anomalous if its related $\Delta y_i$ is higher than expected. This expected value was derived by looking at the values of $\Delta y_i$ of non-anomalous samples in the validation set. Furthermore, given that the validation set contains both anomalous and non-anomalous samples, we leveraged the different $\Delta y_i$ distribution between non-anomalous and anomalous samples to determine the threshold value. Figure 4 shows the distribution of the discrepancy $\Delta y_i$, $i=0,\cdots ,T_v$, where $T_v$ is the validation set length in the case of anomalous (orange) and non-anomalous (blue) points.

In this example, we can see two non-overlapping distributions: here, we decided to use a threshold of 9 to best discriminate between anomalous and non-anomalous samples.

Another possibility is to leverage the ROC curve to identify the optimal value for the threshold that optimally balances the true positive and false positive rate.

Figure 5 shows the model performance at test time. We can see a good agreement between the actual and the virtual signal in the region where no sensor anomalies occurred (rightmost part of the plot) and, instead, a discrepancy between them in a region where sensor anomalies are present (leftmost part of the plot).

4. Results

Table 1. Model performance on training, validation and test sets.

	ME	MAPE	P90
Training set	0.12	0.61	5.06
Validation set (non-anomalous samples only)	1.45	1.61	5.97
Test set (non-anomalous samples only)	1.89	0.65	6.52

shows the reconstruction performance of the model on training, validation and test sets. For validation and test sets, the metrics are evaluated only using subsets where no anomalies occurred. Please note that the error $\Delta$ is defined here as the deviation between the actual signal y and the reconstructed signal $\widehat{y}$: $\Delta =y-\widehat{y}$, then ME is the mean error, MAPE is the mean absolute percentage error, and P90 is the 90th percentile of the absolute value of the error $\Delta$.

For what concerns the anomaly detection performance, when applying the model to the test set, we are able to detect anomalous signals with 96% precision and 100% recall as summarized in

Table 2. Anomaly detection performance on the test set.

	Precision	Recall
Full test set	96%	100%

.

5. Conclusions

In this work, we presented a real industrial application of sensor anomaly detection in the domain of energy and turbomachinery. We applied a semi-supervised deep learning technique, which can be used to perform anomaly detection in an industrial context. In particular, we applied anomaly detection to turbo-machinery units by training a virtual sensor model for a given sensor. We first selected input features through Granger causality and leveraged auto-correlation and subject matter expertise to identify the best window size for the recurrent neural network chosen (LSTM).

This method can be scaled and extended to almost all the sensors installed on the unit, for a complete sensor anomaly detection system.

Furthermore, once the model has been trained for a single sensor, we can later retrain the model using data collected over time, with a continual learning approach [28] so that the algorithm is able to also take into account data-shift phenomena.

Our next plans focus on the deployment of the inference algorithm on edge devices, i.e., on the MarkVIe system. For this purpose, some model distillation may be required (for a review, see [29]). In particular, we need to detect potential sensor faults as early as possible so that we can exclude the sensor from the control system, thus avoiding undesired shutdowns.