Aero-Engine Fault Detection with an LSTM Auto-Encoder Combined with a Self-Attention Mechanism

Abstract

The safe operation of aero-engines is crucial for ensuring flight safety, and effective fault detection methods are fundamental to achieving this objective. In this paper, we propose a novel approach that integrates an auto-encoder with long short-term memory (LSTM) networks and a self-attention mechanism for the anomaly detection of aero-engine time-series data. The dataset utilized in this study was simulated from real data and injected with fault information. A fault detection model is developed utilizing normal data samples for training and faulty data samples for testing. The LSTM auto-encoder processes the time-series data through an encoder–decoder architecture, extracting latent representations and reconstructing the original inputs. Furthermore, the self-attention mechanism captures long-range dependencies and significant features within the sequences, thereby enhancing the detection accuracy of the model. Comparative analyses with the traditional LSTM auto-encoder, as well as one-class support vector machines (OC-SVM) and isolation forests (IF), reveal that the experimental results substantiate the feasibility and effectiveness of the proposed method, highlighting its potential value in engineering applications.

1. Introduction

In recent years, the civil aviation industry has developed rapidly in China. With the ongoing expansion of domestic airlines and the rapid growth of the fleet, the safety and reliability of civil aircraft have garnered significant attention from both the government and the public [1,2,3]. The safe operation and dependable performance of civil aircraft are largely contingent upon the stability and reliability of the aero-engine. As the core component of the aircraft, the aero-engine is particularly susceptible to various failures due to its complex structure and harsh operational environment, especially as operating time increases [4,5]. This highlights the significance of fault diagnosis as a crucial area of research for guaranteeing aviation safety [6,7,8]. When an aviation engine failure occurs, it can not only disrupt flight operations, but also pose a serious threat to the safety of the passengers and crew onboard. Consequently, establishing an effective fault detection system to detect potential faults in aero-engines in a timely manner is crucial for facilitating prompt repairs and ensuring flight safety [9,10,11].

Fault diagnosis methods for aero-engines can be broadly categorized into three main types: physical model-based fault diagnosis methods, expert system-based fault diagnosis methods, and data-driven fault diagnosis methods. Model-based approaches involve constructing a mathematical model of the engine based on its operational mechanisms and conducting fault detection through sophisticated physical models employing various filtering algorithms [12,13,14,15]. However, the inherent complexity of aero-engines poses significant challenges in directly modeling physical faults. Expert system-based diagnostic methods leverage heuristic expertise and the knowledge of domain specialists to establish causal relationships between system failure modes and abnormal monitoring states [16,17,18]. Nonetheless, these approaches encounter difficulties in acquiring a priori knowledge and performing dynamic reasoning, which limits their accuracy. Conversely, data-driven diagnostic approaches diagnose failure modes by extracting failure features from monitoring data and establishing mapping relationships between these features and the corresponding failure modes [19,20,21,22]. These methods depend on monitoring data collected during system operation and do not necessitate physical models or extensive expertise. With the ongoing accumulation of aircraft maintenance data and advancements in deep learning techniques, the integration of data-driven and deep-learning-based fault diagnosis methods has gradually found application in aviation engine fault diagnosis tasks [23,24,25,26].

Current data-driven fault diagnosis methods for aero-engines primarily rely on time-series data collected from various engine components. Feature extraction and fault diagnosis are accomplished through the integration of deep learning and machine learning techniques. However, the collected aero-engine data are highly imbalanced, with normal data significantly outnumbering fault data. Additionally, labeling must be conducted manually by experts, complicating the acquisition of accurate and representative labeled samples [27]. In contrast, unsupervised fault detection methods do not require labeled data to identify anomalies or faults, making them more promising for development in the context of aero-engines. Liu et al. [28] employed a hyperplane-oriented over-sampling technique to address the issue of sparse fault samples in aero-engine training data. Kang et al. [29] proposed a fault detection model based on distillation learning to solve the problem of limited model generalization due to a lack of high-quality fault data samples for aero-engine rolling bearings. Zhong et al. [30] employed a deep learning approach based on stacked auto-encoder networks (SAE) to diagnose and classify various types and levels of bearing faults. Furthermore, Shi et al. [31] developed an innovative self-supervised domain adversarial generalization framework designed to address the fault diagnosis challenges in aviation high-speed bearings under complex operating conditions. Additionally, Li et al. [32] examined the use of LSTM neural networks along with Dempster–Shafer (D-S) evidence theory to develop a novel fault diagnosis method based on sensor information fusion. Guo et al. [33] presented a signal anomaly detection algorithm leveraging an attention mechanism, which first utilizes the attention mechanism to capture the long-range characteristics of spacecraft telemetry data and subsequently employs a stacked auto-encoder to reduce data dimensions and reconstruct the input signal, yielding an error reconstruction sequence. Anomaly detection of spacecraft telemetry signals is then performed by identifying anomaly indicators within the error reconstruction sequence using a window threshold method. Lastly, Zhao et al. [34] introduced a new model that integrates a variational auto-encoder (VAE) with a regression layer, referred to as a VAR. The VAE comprises a bidirectional LSTM network enhanced with an attentional mechanism. The VAR model was validated through simulations using the C-MAPSS dataset, demonstrating superior performance compared to existing methods.

While the above methods perform well in feature extraction, they often require a large amount of labelled data, limiting their ability to generalize, and have high computational complexity. Meanwhile, aero-engine operation process data are characterized by strong dynamic properties, multi-modal imbalance, and complex coupling. In the field of fault detection in aero-engines, the combination of LSTM and a self-attention mechanism provides a solution to the above problem. The self-attention mechanism further enhances the model’s ability to recognise complex coupling relationships between features at different points in time by calculating correlations between different parts of the sequence, which is particularly important when dealing with multi-modal unbalanced data. This combination allows the model to not only understand the temporal dynamics of the data, but also to flexibly focus on the features that are most critical for fault detection, thus improving the accuracy and reliability of fault detection in complex and variable aero-engine operational data. In addition to this, self-attention mechanisms can be easily integrated with other model structures to improve the performance of these models, providing a promising solution to multi-modal dynamic system failure detection problems.

In summary, an unsupervised fault detection method using an LSTM auto-encoder combined with a self-attention mechanism is proposed in this paper, which is specifically applied to the fault detection of piston engines. The dataset is recorded by the ECU to record the flight status of the aircraft within two hours, resulting in a variety of working modes in the normal data, complex system coupling, and multiple linear relationships between multiple variables in the dynamic system. The attention mechanism can dynamically adjust its focus according to the different input data, so that the model can be more adapted to the dynamic system. It is compared with the LSTM auto-encoder, isolated forest (IF), and one-class support vector machine ( OC-SVM ). The results show that the proposed method is effective in detecting faults.

The structure of this paper is as follows: Section 2 provides an overview of related work, encompassing the foundational concepts of AE, LSTM, and self-attention mechanisms, along with the presentation of the SLAE fault detection method. Section 3 details the experiments conducted using aero-engine data and offers an analysis of the fault data. Finally, Section 4 presents the conclusions drawn from this study.

2. Self-Attention-Based LSTM Auto-Encoder Fault Detection Modeling

2.1. Auto-Encoder

The auto-encoder (AE) is an unsupervised learning model designed specifically for processing unlabeled data [35]. Given the significant imbalance in the collected aero-engine data, characterized by a predominance of normal data and a scarcity of fault data, as well as the requirement for manual labeling by experts, employing an auto-encoder structure for the fault detection task is especially appropriate.

Auto-encoders typically comprise two components: an encoder and a decoder, as illustrated in Figure 1. The encoder is tasked with mapping the input data to a low-dimensional latent space representation, while the decoder transforms the latent representation back to the input space, aiming to reconstruct the original data as accurately as possible.

The encoding and decoding processes are outlined as follows:

$$h=g_{{\theta }_1}\left(x\right)=\sigma \left(W_{1}x+b_1\right)$$

(1)

$$\widehat{x}=g_{{\theta }_2}\left(h\right)=\sigma \left(W_{2}h+b_2\right)$$

(2)

Here, W represents the weights, b denotes the biases, x represents the input data, h is the output of the encoder, $\widehat{x}$ is the output of the decoder, and an activation function is applied, with the sigmoid function being used in this context. The corresponding formula is presented below:

$$\sigma \left(\mathrm{x}\right)={\displaystyle \frac{1}{1+{\mathrm{e}}^{-\mathrm{x}}}}$$

(3)

The encoder receives the input X and maps it to the latent space representation h, while the decoder remaps h back to the input space to produce the reconstructed signal. The loss function is typically defined as the mean squared error, as shown below, where N represents the number of training samples. The objective of network training is to minimize this mean squared error.

$$min{\displaystyle \frac{1}{N}}\sum _{i=1}^N{\Vert x-\tilde{x}\Vert }_2^2$$

(4)

The trained auto-encoder aims to learn the true probability distribution of the input data as accurately as possible. Its reconstruction error serves as an indicator of the likelihood of the input data and is straightforward to compute.

2.2. LSTM Network

LSTM networks are a specialized type of recurrent neural network (RNN) designed for processing and predicting time-series data. They address the challenges of learning long sequences, as well as mitigating issues related to gradient vanishing and gradient explosion [36].

LSTM is used in this paper, so the principles are briefly explained below. LSTM networks have a unique cellular structure, which manages the flow of information via an intricate gating mechanism. Unlike a traditional RNN, LSTM introduces input gates, forget gates, output gates, and a cell state. This gating mechanism can be viewed as a fully connected layer. LSTM facilitates information storage and updates through this gating system, specifically utilizing the Sigmoid function in conjunction with dot product operations. While RNNs only propagate hidden states, LSTM incorporates cell states as well. Figure 2 [37] illustrates the LSTM cell structure, where $i_t$, $f_t$, and $o_t$ represent the input gate, forget gate, and output gate, respectively. $\mathrm{Tanh}$ is the hyperbolic tangent activation function that maps data to the range [−1, 1]. $\sigma$ is the sigmoid activation function that maps the data to the range [0, 1]. $W_i,W_f,W_c,W_o$ represent the weights associated with the input gate, forget gate, cell state, and output gate, respectively. $b_f,b_i,b_c,b_o$ are the bias terms. $x_t$ represents the input at the present time. $h_{t-1}$ and $h_t$ represent the output of the hidden layer at time $t-1$ and t, respectively. ${\tilde{C}}_t$ represents the temporary state of the input gate. $C_{t-1}$ and $C_t$ represent the state at time $t-1$ and t, respectively.

Forgetting Gate: This gate regulates whether past long-term memory information should be discarded.

$$f_t=\sigma (W_f·[h_{t-1},x_t]+b_f)$$

(5)

Input Gate: This gate determines the amount of information that is retrieved from the long-term memory unit for the output.

$${\tilde{C}}_t=tanh(W_c·[h_{t-1},x_t]+b_c)$$

(6)

$$i_t=\sigma (W_i·[h_{t-1},x_t]+b_i)$$

(7)

Output Gate: This gate regulates the amount of information that is extracted from the long-term memory unit for the output.

$$o_t=\sigma (W_o·[h_{t-1},x_t]+b_o)$$

(8)

$$h_t=o_t\ast tanh\left(C_t\right)$$

(9)

Long-Term Memory Unit: Primarily utilized for storing and processing past information, as well as sifting and filtering data.

Short-Term Memory Unit: Designed to hold the most recent output and feed it back into the network.

$$C_t=f_t\ast C_{t-1}+i_t\ast {\tilde{C}}_t$$

(10)

The LSTM model inputs sequential data into an input gate, which writes the data to the long-term memory unit as required. The forget gate regulates which information should be discarded, while the output gate retrieves the necessary information from the long-term memory unit for output.

2.3. Self-Attention Mechanisms

Self-attention is a variant of the attention mechanism. Since its introduction in the 1990s, the Attention mechanism has evolved significantly, culminating in 2014 when Mnih et al. [38] first applied it to the visual domain. In the context of time-series data, the significance of the information carried by the time series signals varies across different windows [39]. The self-attention mechanism serves to extract the important information associated with each time window. Figure 3 [40] illustrates the structure of the self-attention mechanism in detail.

Firstly, the n sequences of length m output from the encoder are passed through a linear layer to obtain the query matrix: $Q=[Q^1,Q^2,\cdots ,Q^m]$, key matrix: $K=[K^1,K^2,\cdots ,K^m]$, value matrix: $V=[V^1,V^2,\cdots ,V^m]$. The calculation process is shown below:

$$\left\{\begin{array}{c}{Q}^i=W^q{X}^i\\ K^i=W^k{X}^i\\ V^i=W^{\nu }{X}^i\end{array}\right.$$

(11)

After that, the dot product between the query vector $Q^i$ and the keyword vector $K^i$ for each time window is calculated, and to prevent the dot product from being too large, the dot product is divided by $\sqrt{d_k}$, where $d_k$ is the dimension of the keyword vector, and the result is normalized to a probability distribution using $SoftMax$. The attention value is multiplied by the value vector $V_i$, and the sum of the weights is calculated to be the attention score. The calculation process is shown below:

$$Attention(Q,K,V)=SoftMax\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(12)

2.4. Fault Detection Model

Building upon the analysis of the AE, LSTM, and self-attention mechanisms, we propose a self-attention-based LSTM auto-encoder fault detection method (SLAE) tailored to aero-engine fault detection. This method utilizes performance parameters from the time series, replacing the traditional AE neural network with an LSTM network. Additionally, a self-attention mechanism is integrated into the output of the encoding region to enhance the model’s ability to learn and extract features from the data. The workflow of SLAE is illustrated in Figure 4.

First, the collected aero-engine data are divided into a training set and a testing set. The input data undergo pre-processing, primarily for standardization. Pre-processing is also a crucial factor influencing the outcomes of model training. The calculation process is detailed below, where $\mu$ represents the mean value of the original data and $\sigma$ denotes the standard deviation of the original data. The fault detection flow chart is shown in Figure 5.

$$x_i^{\prime }={\displaystyle \frac{x_i-\mu }{\sigma }}$$

(13)

The SLAE model utilizes preprocessed data as the input, with the encoder layer comprises a two-layer LSTM network. This LSTM captures dependencies within the input time series and generates an implicit representation. The output from the encoder passes through the self-attention layer, which efficiently extracts important features from the sequence by weighting different time points. The decoder then reconstructs the data into a time series format, allowing for the calculation of the reconstruction error.

$$reconstruction\_error=\Vert x-\widehat{x}{\Vert }_2^2$$

(14)

The network employs the Adam optimizer to minimize the loss function, which is defined using mean squared error (MSE). x and $\widehat{x}$ represent the original data and the reconstructed data, respectively. The square of the L2 normal of the original data and the reconstructed data is calculated as the fault indicator. For the training data, fault indications are calculated for all samples, with a confidence level set at 0.95. The value at the 95th percentile of the reconstruction error serves as the control line. When calculating the reconstruction error for the testing data, any value exceeding this control line is deemed abnormal; otherwise, it is classified as normal.

The SLAE anomaly detection model is an unsupervised deep learning network designed for detecting anomalies. In the case of data bias in aero-engine fault diagnostic data, the SLAE model utilizes a large number of normal operation samples to perform high-precision fault detection on a limited number of anomaly samples.

3. Aero-Engine Fault Detection

3.1. Introduction to the Dataset

This dataset is generated from the simulation of a type of civil aircraft with a piston aircraft engine primarily used for flight training, and injected with fault information. The main components of the engine used in this aircraft include the air intake system, the exhaust system, the cooling system, the fuel system, the engine control system, and the propeller system. The main components of the intake system include the air filter, the intake manifold, and the pilot tube to feel the intake pressure; the main components of the exhaust system include the exhaust manifold, the turbocharger, the wastegate control solenoid valve, the wastegate opening control assembly, and the exhaust pipe; the cooling system mainly comprises the thermostat, the coolant radiator, the mid-cooling radiator, the heat exchanger, and the sliding oil radiator; the main components of the fuel system include the fuel filter, the low-pressure pump, the high-pressure pump, the common rail, and the fuel control system; and the fuel system mainly includes the fuel filter, the fuel low-pressure pump, the fuel high-pressure pump, the fuel common rail, the fuel injector nozzle, and the fuel common rail pressure control valve. The engine control system adopts digital full-authority electronic control technology to control the engine’s work through the engine control unit. The main components of the propeller system include the propeller, the propeller hub, and the gear box.

The parameters that have a significant impact on the normal operation of this type of piston aero-engine include the crankshaft speed, power target value, intake pressure actual value, fuel pressure actual value, atmospheric pressure, DC generator voltage, power actual value, intake pressure target value, fuel pressure target value, and wastegate opening, and these ten parameters are used to make up the data file, as shown in

Table 1. Aero-engine data composition.

Serial Number	Parameterisation	Abridge
1	Crankshaft Revolution Speed	CREV
2	Power Target Value	TP
3	Manifold Intake Air Pressure	MIAP
4	Measured Fuel Pressure	MFP
5	Atmospheric Pressure	AP
6	Direct Current Power Supply Voltage	DCPSV
7	Measured Power	MP
8	Target Intake Air Pressure	TIAP
9	Target Fuel Pressure	TFP
10	Exhaust Valve Opening Position	EVOP

:

In this type of engine, due to the use of digital full-authority electronic control technology, each key component is installed with sensors to collect engine operating parameters, which are recorded and saved by the ECU. During engine operation, the ECU collects and saves data once per second, and each data file contains about 12,000 pieces of data.The data file usually contains three complete flight data records, and the size of the data file is automatically determined by the ECU.

3.2. Fault Data Analysis

This civil aircraft aero-engine dataset contains 24 data files and 13 kinds of faults, among which the aero-engine of the No.1 aircraft is selected for fault diagnosis, containing three pieces of fault data, with the experimental samples containing 11,507, 11,458, and 11,421 entries and having 10 dimensions, containing 10 aero-engine parameters, respectively. The fault data are visualized and fault analysis is performed as follows:

Failure occurs due to sharp fluctuations in wastegate opening (EVOP) and the resulting sharp fluctuations in actual intake pressure. The EVOP ranges from 60–85 percent when flying at altitudes below a certain height and with the throttle stick in the full-throttle position. At a high altitude, an EVOP equaling 99 percent is normal. From Figure 6, it can be seen that at the sampling time from 6800 to 7500, the aero-engine wastegate opening fluctuates, resulting in a sharp fluctuation of the MIAP causing a malfunction. Under normal circumstances, there is a relatively fixed interpolation between the MIAP and the TIAP, instead of the fluctuation.

The wastegate opening EVOP represents the opening of the solenoid valve, not the physical opening. The physical opening of the wastegate is controlled by the actuator cartridge, whose action is controlled by the air diaphragm cartridge. The solenoid valve controls the actuator cartridge, and thus the physical opening of the wastegate, by adjusting the air pressure to the air diaphragm cartridge.

From the results of the analysis in Figure 7, it is clear that the MIAP fluctuates dramatically during engine operation, especially at the sampling moments between 5500 and 6000. This fluctuation is abnormal, as it indicates a serious disturbance of the pressure stability within the intake system. At the same time, we observed an abnormally large difference between the TIAP and the MIAP, which was well outside the range allowed for normal engine operation. This increased difference usually means that the actual amount of air entering the engine does not match the target amount of air preset by the ECU and that there is an air leak in the aero-engine intake manifold.

From the results of the analysis in Figure 8, between the sampling moments 4000 and 4500, the value of the MIAP fluctuates dramatically. Such fluctuations are not normal and may be due to a problem in some part of the intake system. Further inspection revealed that there was an air leak in the intake manifold, which was the direct cause of the MIAP fluctuation. An air leak in the intake manifold causes a reduction in air flow, which affects the combustion efficiency and power output of the engine.

Subsequently, between sampling moments 6900 and 7400, the EVOP fluctuated up and down even though the TIAP remained stable. This fluctuation indicates a problem with the wastegate opening control. The wastegate is an important component of the turbocharging system, and its opening directly affects the efficiency of the turbo and the air intake of the engine.

3.3. Experimental Indicators

To evaluate the performance of the model, the false positive rate (FPR) and false negative rate (FNR) are primarily selected as the experimental metrics. These two metrics provide a comprehensive reflection of the model’s classification errors for both positive and negative class samples, thereby aiding in the assessment of the model’s practicality and stability across various scenarios.

$$\mathrm{FPR}={\displaystyle \frac{\mathrm{FP}}{\mathrm{FP}+\mathrm{TN}}}$$

(15)

$$\mathrm{FNR}={\displaystyle \frac{\mathrm{FN}}{\mathrm{FN}+\mathrm{TP}}}$$

(16)

In the above equation, TP is the number of abnormal sample points that were correctly detected, TN is the number of correctly identified normal samples, FP is the number of normal sample points that were misjudged as abnormal, and FN is the number of abnormal sample points that were misjudged as normal.

A comparative analysis of the leakage rates across different methods was conducted while controlling the false alarm rate, aiming to verify the validity of the proposed model.

3.4. Analysis of Experimental Results

The trained SLAE model was compared with three methods, LSTM-AE, IF, and OC-SVM, and a 95 percent confidence level was taken as the control limit. The following are the detection effects of different methods for the three kinds of faults from Figure 9, Figure 10 and Figure 11.

It can be seen from

Table 2. Training time comparison.

Time Step	AE-LSTM (s)	SLAE (s)
10	300	443
20	408	686
30	512	911
40	608	1178
50	705	1502
60	793	1768
70	873	2082
80	980	2372
90	1079	2833
100	1175	3262

that after the attention mechanism is introduced, the computational complexity is increased, and the training time is also increased. As the time step increases, the training time of the SLAE model is much longer than that of the AE-LSTM. When the time step is 100, the training time of SLAE is close to three times that of the AE-LSTM.

Analyzing the experimental results from

Table 3. Evaluation indicators.

Fault Number	SLAE		AE-LSTM		OC-SVM		IF
	FPR	FNR	FPR	FNR	FPR	FNR	FPR	FNR
1	0.0732	0.5974	0.2463	0.8033	0.0003	1	0.0725	0.9781
2	0.3244	0.5691	0.3462	0.6289	0.0062	1	0.1320	0.6600
3	0.3188	0.5339	0.3749	0.5818	0.0067	1	0.1336	0.6073

, the LSTM auto-encoder combined with the self-attention fault detection method proposed in this paper is compared to the traditional LSTM auto-encoder method, as well as the OC-SVM and IF. When the control limit is a 95 percent confidence level, the SLAE model significantly outperforms the other three methods in detecting Fault 1. Additionally, with the introduction of the attention mechanism in the SLAE model, the false alarm rate decreased compared to the traditional LSTM model.

4. Conclusions

To address the fault detection problem in aero-engines, this paper proposes a fault detection method that combines an LSTM auto-encoder with a self-attention mechanism. This approach effectively captures complex patterns and long-term dependencies in time-series data. The LSTM auto-encoder excels in handling time-series data, while the self-attention mechanism captures global features. Faults are detected based on the reconstruction error between the input data and the model output, and comparative experiments are conducted to validate the feasibility and advantages of the proposed method.

The detection method, while showing superior capabilities compared to traditional auto-encoders in detecting engine faults, is not without its limitations. This unsatisfactory detection could be attributed to factors such as strong correlations between variables and the complexity introduced by multiple operating modes in the normal data, compounded by unbalanced samples between these modes—a challenge that warrants further investigation in future studies. Moreover, this method’s reliance on normal data from each specific engine to train the model presents a practical constraint, as it necessitates the detection of abnormal data associated with these particular engines. The individual training for each engine component is not only time-consuming and labor-intensive but also highlights the need for a more efficient approach. In response to this challenge, further work could use transfer learning for engine fault detection, aiming to improve the efficiency of fault detection and reduce the resources required for model training.