1. Introduction
Launch vehicles can cause several types of vibrations due to intense thrust pulses and aerodynamic mutations during launch and flight. The vibration signal covers low, medium and high frequencies, which seriously affects the equipment structure and electrical system of multiple systems in the rocket, while the low-frequency vibration causes astronauts to have difficulty in physical movement and chaotic decision making, which can easily lead to the failure of rocket launch missions and casualties [
1]. Therefore, suppressing the occurrence of vibration from the material and structure and reducing the harm caused by vibration from the perspective of information science have become important research directions in the field of aerospace engineering.
The launch vehicle engine is a typical fuzzy environment [
2], and relying solely on AI and computational science cannot explore its internal mechanism. For industry and military equipment, PHM (Prognosis and Health Management) studies for a system are much more difficult than the fault detection of a single component. Leng et al. [
3], GE et al. [
4] and Subramanian et al. [
5] proposed self-organizing and self-evolving fuzzy neural networks for different fuzzy systems. However, this approach is very susceptible to anomalous data that can lead to erroneous results. For launch vehicle fuzzy systems, each launch may be a different rocket model, and the same rocket model may also have different types of components. It is difficult to obtain practical verification for the results of the above methods. Rocket engine testing can summarize a large number of failure modes and human experience, which can fit accurately with the actual situation. It also reduces the impact of anomalous data on results.
Researchers have developed various methods for rocket engine PHM, and the current ones can be categorized into three types as follows [
6]:
- (1)
Model-based methods;
- (2)
Expert-experience-based methods;
- (3)
Data-driven methods.
Researchers have conducted extensive research on the identification and evaluation of rocket engine faults. The basic principle of the mathematical model for engine PHM is to treat the output of the engine mathematical model as a standard state and determine the deviation degree of the actual working state of the engine from the standard state through various indicators; if the deviation degree is too large, the engine working conditions in this state are regarded as abnormal. Model-based methods can be divided into two main types: analytical models and system identification models. Based on an analytical model, Davidson et al. [
7] designed a linear engine model for the Advanced Health Management System (AHMS), using the threshold to achieve online engine anomaly detection. Breedveld [
8] compares the effects of a continuous state estimation and discrete state estimation for SSME fault diagnoses and shows that the discrete state estimation performs best when measurement noise is negligible, and the continuous state estimation is used when measurement noise is present. Based on a system identification model, Srivastava et al. [
9] established the correspondence between the spectral intensity of SSME plume characteristics and engine performance parameters through neural network training, and based on this relationship, the condition monitoring of the SSME was carried out. In the 1990s, the United Technologies Research Center (UTRC) used the ARMA algorithm to monitor the steady-state process of the SSME in real time when designing the SSME’s HMS (health management system) [
10]. Model-based approaches focus on how well they fit fuzzy systems, and one model cannot be adapted to different engines. So, model-based approaches lack versatility.
The conventional method is based on expert systems for the problem of the vibration failure of launch vehicle engines, but with the development of technology and artificial intelligence, big data mining has been proven to be an important and effective method [
11]. Engle et al. [
12] developed the expert-system-based inference mechanism PLES (Prelaunch Expert System) for the preparation stage of the SSME during ignition, which facilitated an analysis of the overall health of the system by ground personnel. Kurien et al. [
13] experimented with health management techniques on the Deep Space One project from 1998 to 2001. Perry et al. [
14] developed an expert system for turbopump fault diagnosis. The system contains both shallow knowledge and deep knowledge, which refers to the experience and processes summarized by domain experts in analyzing and processing test data, and deep knowledge refers to any analytical model that can characterize the operating characteristics of engine turbopumps. However, relying heavily on the expert system leads to subjective experience domination [
15]. At the same time, there are three main problems with the use of manual analyses:
- (1)
Personal cognitive limitations, that is, the knowledge that each person has is limited;
- (2)
Diagnoses are inefficient and require a lot of time and human resources;
- (3)
The manual system can only be used for offline diagnoses and cannot meet the needs of online fault diagnoses [
6].
Data-driven methods can be further subdivided into two types: statistical analysis and pattern recognition. Aiswarya et al. [
16] manually extracted the time and frequency domain features of an LRE (liquid rocket engine)’s running signal, used support-vector machines to classify these features and achieved 100% classification accuracy. Y. Wu et al. [
17] proposed a fault detection method with particle swarm optimization and a least-squares support-vector machine to improve the performance of an LRE. B. Wang et al. [
18] proposed a deep separable CNN to predict the remaining useful life of aeroengines with the monitoring data from different sensors. H. Miao et al. [
19] proposed dual-task deep LSTM networks to unify the task of degradation evaluation and the remaining useful life prediction of aeroengines. L. Xu et al. [
15] used the quantum genetic algorithm (QGA) to optimize the BP neural network, trained the BP neural network, which generated two outputs, and performed a fault diagnosis on the liquid rocket engine. A PHM approach based on digital twins is a deep combination of traditional PHM technology and digital twin technology. Mapping physical devices and virtual devices leverages the technology of data simulation and model simulation, which can realize the early prediction and accurate location of faults [
20]. Among data-driven approaches, deep learning methods exhibit excellent adaptability and accuracy. A launch vehicle has a limited number of launches with few failure samples and a relative lack of historical experience. The limited fault data are not enough to support the training of a neural network, so an accurate fault model cannot be obtained, and it is difficult to achieve fault location. Basing a model on real data is susceptible to data scarcity, and basing it on simulation data is also susceptible to simulation model inaccuracy.
Our paper is based on the real-time data of a launch vehicle and realizes the multi-layer weight generation of the overall system according to expert scoring. At the same time, a deep neural network is used to predict the future trend of the vibration signal, and the deviation between the actual data, the prediction data and the ideal state is calculated to obtain the actual and predicted health evaluation matrix. The evaluation set and the weight set are used as inputs, and the health value of the top-level target is obtained through a comprehensive fuzzy assessment. It has important guiding significance for the risk prevention of the ascending section of the rocket engine. Introducing expert experience to data-driven methods can solve the problems of the lack of data and low model accuracy. Using a modular approach, the health assessment of different objects can be achieved by replacing the prediction data and weight relationship. While improving the accuracy of the evaluation, the use of automated artificial intelligence technology reduces economic expenses.
2. Launch Vehicle Engine Health Evaluation System Architecture
2.1. Abstract the Evaluation Hierarchy from the Data Structure
The vibration data used in this paper are derived from the measured data of the Long March series of carrier rocket engines, as shown in
Figure 1.
The data include six types of vibration signals, including POGO vibration, low-frequency vibration, high-frequency vibration, fluctuation, crush and noise. POGO vibration is the pressure generated by multiple engines of the rocket during flight and the periodic vibration generated by the external structure of the rocket, which can easily lead to pressure loss of control in the internal pipeline and instrument damage [
21]. When the rocket is flying at a high Mach speed, due to the strong collision between the rocket structure boundary and the external gas environment, it will produce a strong pulsating pressure, which will easily cause further equipment vibrations [
22]. In the process of firing multiple engines of the rocket and the sharp increase in the acceleration of the rocket, there will be low- and high-frequency signals and shock signals in the thrust system, and there is a widespread noise when a large number of pieces of mechanical equipment are working. Each kind of signal is divided into multiple groups of signal sensor batches, the data of the same batch are collected in the same acquisition sequence and the test conditions are consistent. The dataset in this paper is sampled from the original data, totaling 525 KB, in .CSV format.
Most traditional aerospace equipment evaluation systems are layered from the perspective of hardware, and the evaluation objects are divided according to the level of components. However, it is difficult to inspect all parts in fuzzy systems; thus, engineers often focus on evaluating some parts and completely ignore others. Such evaluation methods do not provide a comprehensive health assessment of internal systems, and there is a possibility of missing alarms. The method researched in this paper divides the vibration signal of the engine system as a whole into three layers from the perspective of data. For fuzzy systems, the precise location of fault sources is difficult to achieve, so the study of overall health is very important.
The system researched in this paper is divided into three layers: the target layer, the sub-target layer and the factor layer, which is convenient for the unified quantification of the health value of multiple influencing factors between the same level, and it is also easy to import weights. According to historical experience, the impact of the six types of vibrations is different for the health state of the launch vehicle engine, and the degree of influence of the data collected by different types of sensors of the same type is also slightly different. The effect size relationship between the factors can be considered to simulate the data and obtain the health value of the multi-level and multi-factor system more accurately.
2.2. Evaluate the Process Architecture
In order to cope with the comprehensive evaluation of multi-data categories and multi-hierarchical systems, this paper proposes a comprehensive evaluation architecture for the vibration signals of launch vehicle engines as shown in
Figure 2.
The main structure of the overall process includes five main modules: spatial division of health states, data prediction, evaluation set generation, weight analysis and fuzzy comprehensive evaluation. Modular structure makes sense for the subsequent addition of evaluation levels and factors. The data preprocessing part completes the outlier cleaning and data format correction of the original data, which is the pre-step of vibration signal trend prediction. Based on machine learning methods and deep neural networks, we extract the continuous local dependencies in the timing signal by virtue of powerful feature-learning ability and complete the prediction of long sequences. As an evaluation index for health assessment, the evaluation set is combined with subjective experience and objective data analysis. The evaluation set of 15 sequences, including six types of vibration signals, was obtained by threshold statistical methods. Weight analysis was based on analytic hierarchy process (AHP). The system of analysis and decision making was decomposed into target layer, sub-layers and evaluation criteria. The weight set generation is based on the weight analysis module, which generates the weight matrix required in the fuzzy comprehensive evaluation. Launch vehicle engine systems are typically fuzzy environments, in which details inside the system are difficult to fit mathematically. Combining the historical experience of the experts and accurate prediction results makes the assessment process more comprehensive.
The health evaluation system for vibration signals of launch vehicle engines constructed in this article has the characteristics of objectivity, multi-layer and multiple factors, and strong technology transfer ability. Based on artificial experience, combined with the vibration signal depth prediction results, the historical experience of rocket testing is considered and the mathematical features that are difficult to summarize are excavated through new technologies. At the same time, the multi-factor and multi-level characteristics also make the evaluation process more comprehensive.
3. Multi-Layer Weight Generation
For the different types and sources of launch vehicle engine vibration signals, it can be difficult to find the specific component where the fault occurred, and the impact of different vibration types on the normal flight of the launch vehicle is different. Data reliability and effectiveness of different sensors cannot be equal. Converting the degree of impact into percentage weights under a uniform measure is necessary for assessing the vehicle’s state of health.
3.1. Analytic Hierarchy Method
For the health value assessment of the launch vehicle engine system, we used a quantitative analysis of multiple conditions and factors. The electrical environment inside the launch vehicle engine is a typical giant system and a fuzzy environment. Therefore, it is impossible to try to grasp the weight of each factor in the system relying on human experience under the same weight standard. Based on historical data and a summary of past failure modes, experts can obtain the weight relationship between two factors. The impact of factor A on the entire system is quantified relative to factor B, but when there are many factors involved, experts cannot directly obtain the multi-level and the multi-factor influence weight relationship.
Considering the hierarchical complexity of system equipment, a multi-layer, multi-factor weight matrix is generated based on analytic hierarchy (AHP) method. The ana-lytic hierarchy method was first proposed by American scientist T.L. Saaty in the 1970s [
23] and has been applied to network system theory and multi-objective comprehensive evaluations. It mainly solves the decision-making problems of interrelated and mutually restricting complex systems and is widely used in aerospace engineering, electrical engineering, operations research and other fields.
The process using AHP to construct a rocket engine vibration signal model and obtain the weights of each type of factor can be roughly divided into the following steps:
- (1)
Establish a hierarchical model according to the type and batch of launch vehicle engine vibration signals;
- (2)
Based on historical statistics and expert experience, construct a judgment matrix according to the weight relationship between factors in the same level, also known as reciprocal matrices:
- (3)
Calculate the maximum eigen root and eigenvector of the reciprocal matrix according to the criteria: to facilitate subsequent calculation;
- (4)
Normalize the feature vectors and calculate the weight values of all elements contained in each layer separately. The weight calculation formula is as follows:
- (5)
Carry out a consistency test on the matrix. The credibility of the manually judged matrix is tested; if there is a contradiction between the two weights of the elements, the consistency test is not passed, and the credibility of the result of introducing the artificially weighted positive and negative matrix is increased, with the consistency index as follows:
The consistency ratio is as follows:
When
< 0.1, it is considered to have passed the test, and
is the established system-level order;
- (6)
Generate subjective weight vectors.
3.2. Weight Generation
The corresponding table of consistency indicators is shown in
Table 1.
The system weight allocation architecture in this paper is shown in
Figure 3. Overall system is summarized as follows:
,
represent POGO vibration, low-frequency vibration, high-frequency vibration, fluctuation, shock and noise, respectively. We can see in the figure that the overall system is divided into three levels, so
is 3. The corresponding immediate mean agreement metric
= 0.52. The overall hierarchy of the vibration signal of the ascent launch vehicle engine is divided into three stages, in which the sub-target layer contains a weight vector:
. The factor layer consists of the following six weight vectors:
,
,
,
,
and
.
The multi-level reciprocal matrix provided by the China Academy of Launch Vehicle Technology is shown as
Table 2. Due to space limitations, this paper only shows the reciprocal matrix of the sub-target layer and a set of factor layers.
The reciprocal matrix, such as low-frequency vibration, is given in the same way, as shown in
Table 3. According to the AHP method, the sub-target layer weight is
and the maximum eigenvalue is 6.003449, so
, and the consistency test is passed. The weight vectors of each factor under the six types of sub-goals are:
,
,
,
,
and
.
After testing, the results of six groups of consistency tests passed. The weight experiment results show that POGO vibration, fluctuation and low-frequency vibration have the greatest influence on the health state of the ascent section of the launch vehicle engine in the sub-target layer. At the same time, according to the expert experience, the data after the sensor batch can better reflect the authenticity and real time of the state, so the weight is also higher, and the results are in line with the experts’ expectations.
4. Vibration Signal Trend Prediction
At present, most of the research on the health state of spacecraft such as launch vehicles is concentrated on their real-time state. Data pre-processing, data measurement, anomaly detection and other steps are used to complete real-time status assessments. Real-time status assessments have a large delay and are of limited reference value to ground-monitoring personnel. The rapid development of artificial intelligence technology has greatly improved the efficiency and accuracy of data mining. Compared with the real-time status assessment as mentioned above, the evaluation of the status of a launch vehicle engine based on prediction data is of more guiding significance for risk prevention and early warning.
4.1. Predict Generic Steps
The general forecasting process is divided into the following three steps:
- (1)
Data preprocessing: complete data cleaning of the original vibration data of the launch vehicle;
- (2)
Model training: use algorithm models to train data;
- (3)
Model prediction and error calculation: predict the data waveform, save the data and calculate the error value of the prediction, including RMSE, R2, and MAPE.
The
RMSE formula is as follows:
The
MAPE formula is as follows:
the
R2 formula is as follows:
represents each acquisition value in the test data, represents each collection point of the prediction data, represents the total number of collection points, and represents the average of real data.
R2 is the degree of model fitting, MAPE is the mean absolute percentage error and RMSE is the root mean square error, reflecting the prediction accuracy of the algorithm from multiple angles.
4.2. Predictive Models
We chose five types of algorithms, including LSTM (long short-term memory) [
24], GRU (gated recurrent unit) [
25], LightGBM (light gradient-boosting machine) [
26], WMA (weighted moving average method) [
27] and ARIMA (autoregressive integrated moving average) [
28] algorithms for experimentation. These five types of algorithms have been widely used in industrial PHM. Among them, LSTM and GRU algorithms are typical neural network algorithms, and LightGBM, WMA and ARIMA algorithms are classic machine learning prediction algorithms. In this paper, a variety of algorithms are used to verify the accuracy and versatility of the method on the vibration data of the launch vehicle.
Since the launch vehicle is tested under a variety of vibration conditions, the collected data are dynamically changing, and the correlation information between sampling points is the key to data trend prediction. Compared with the RNN method, the LSTM method improves the prediction ability of long-term data and solves the problem of gradient explosion and gradient disappearance in the data-training process of RNN method. The GRU method is a very effective variant of the LSTM network, which is simpler than the structure of the LSTM network. LightGBM, WMA and ARIMA are typical machine learning algorithms, which have been widely used in the industry and have been proven to have good effect.
The prediction algorithm relies on TensorFlow and Keras frameworks, as well as basic machine learning libraries to be completed. We used a Intel Core I7-11700k CPU and NVIDIA GeForce Rtx3070 GPU. The software platform was Visual Studio Code 1.77.3. The version of the algorithm dependency library is shown in
Table 4.
Table 5 shows the network structure and parameter settings of LSTM algorithm and GRU algorithm.
Table 6 shows the network parameters of WMA and ARIMA algorithms.
Table 7 shows the algorithm parameter settings for the LightGBM algorithm.
4.3. Predictive Results
Six types of signals are predicted based on five methods, and the error of the prediction results is shown in
Table 8. The prediction effect is shown in
Figure 4.
According to
Table 8 and
Figure 4, the GRU algorithm has the best prediction effect for longer sequences among the five types of algorithms experimented in this paper. The LightGBM algorithm has the most balanced prediction effect for long and short sequences. The prediction effects of weighted moving average algorithm and LSTM on long and short series are also relatively balanced. Due to the short sequence length, the ARIMA algorithm has the worst fit and the lowest prediction accuracy in this experiment.
5. Calculate Health State Space Based on Boxplot Method
The evaluation set generation is mainly based on the predicted data waveform, not the prediction error. The common method is based on the root mean square error (
RMSE) of the prediction result, combined with the membership function to the final result of the prediction
RMSE fuzzing and defusing. As shown in
Figure 5, it is the commonly used trapezoidal membership degree function.
We define the abscissa threshold corresponding to multiple types of evaluation indicators according to expert experience: , , , , , , , . However, this conventional approach is suitable for situations in which the output is difficult to analyze and the evaluation value is difficult to determine. It is a method of approaching the final evaluation indicator as closely as possible when dealing with ambiguity. The actual data used in this article is no longer ideal. The data structure that should be used as a criterion for evaluating the membership function is still vague. At the same time, the threshold of the membership function is more difficult to grasp, and there is no unified evaluation benchmark with the prediction data.
5.1. Boxplot Principle and Design Steps
The boxplot method was invented by the famous American statistician John Tukey and describes the distribution standard of data through five numbers: upper whisker, first quantile (Q1), median (Median), third quantile (Q3) and lower whisker. This method can clearly show the overall distribution of data and outlier information. The interquartile range (IQR: Q3-Q1) is the key computational datum, defined as the difference between the third quartile and the first quartile, and the overall data can be divided autonomously by setting the quarterback spacing and IQR coefficient. Through the boxplot method, we can easily analyze the distribution of vibration signal data. The upper and lower limits of data under normal working conditions are obtained and used as the basis for the division of engine state space.
The healthy state space calculation process is as follows:
- (1)
Read data;
- (2)
Perform data sorting, calculating the first quantile (Q1), median (Median) and third quantile (Q3);
- (3)
Calculate the IQR of the quartile difference;
- (4)
Calculate the upper whisker: (Q3 + 1.5 × IQR) and lower whisker: (Q1 − 1.5 × IQR);
- (5)
Set the coefficient and divide the healthy state interval.
The normal working condition data without setting vibration conditions were experimented according to the boxplot calculation steps, and the quartiles of multiple batches of six types of vibration signals were measured. As the same time, we calculate the upper and lower limit values to obtain the preliminary health range of the evaluation, using this range as a reference limit.
The boxplot analysis of six types of vibration signals under normal working conditions is carried out as shown in
Figure 6:
The upper and lower limits of the data obtained by the analysis are shown in
Table 9.
5.2. State Space Division Based on Boxplots
According to the expert experience, the set of healthy comments is {Excellent, Good, Normal, Bad, Deterioration}, corresponding to each color block, as shown in
Figure 7. The green line represents the predicted data of the vibration signal, and the black line represents the actual vibration data.
The boundary waveform amplitudes of the signal before the vibration condition occurs are the upper whisker and lower whisker, the total number of sampling points of the test data is , the upper and lower thresholds of the reference waveform are set to , , respectively, and the number of points in each threshold interval is .
The state space threshold set by expert experience is as follows:
The status range is set to:
, where
stands for excellent status,
stands for good condition, and so on. Each health state space corresponds to a score.
Each individual evaluation
in the evaluation set matrix is counted by the state interval of each sampling point of the prediction data, and the percentage is scored according to the interval probability:
We followed the steps above to complete the evaluation set generation experiment. Due to space limitations, only the evaluation set of LSTM and GRU algorithms is displayed.
Table 10 is the evaluation set of LSTM, and
Table 11 is the evaluation set of GRU.
Table 12 is the evaluation set of real test data.
6. Fuzzy Comprehensive Assessment
6.1. Principles and Steps of Fuzzy Comprehensive Assessment
The fuzzy comprehensive evaluation method has the advantages of clear results and strong systematization, which is suitable for a more comprehensive and in-depth evaluation of multi-level and multi-factor objects in a fuzzy environment. A large number of expert systems used in spacecraft health management use historical experience to build a knowledge base and make system evaluations more intelligent through extrapolation, mapping and other methods. However, in fact, the unstable working conditions of the launch vehicle engine have a strong relationship with the launch environment and flight trajectory. At the same time, the frequent iteration of launch vehicle engine technology and technology has not led to the rapid replenishment of the expert system knowledge base, so the evaluation method based on expert systems is not universal, accurate or comprehensive.
The weight set and evaluation set calculated from the above chapters are two parallel inputs for the fuzzy comprehensive evaluation method. The evaluation steps of the fuzzy integrated assessment method are as follows:
- (1)
Establish the overall structure of the research objectives and create a multi-level structure to facilitate subsequent bottom-up evaluation;
- (2)
Create a set of healthy comments:
based on the actual object. In
Section 5.2, we have set
;
- (3)
Obtain the weight matrix for the factors in each level and the evaluation matrix of the underlying signal;
- (4)
Perform fuzzy evaluation operations to obtain the fuzzy evaluation vector of the target layer as follows:
- (5)
According to the needs of the project, assign the scores that can be obtained for each type of evaluation and generate a score vector ;
- (6)
Calculate statistical health score, which is the health of the system.
6.2. Scoring Rocket Engine Health Based on Fuzzy Comprehensive Assessment
From above experiment, we can obtain the evaluation matrix: and the weight matrix: . represents the single evaluation vector of the underlying factor layer, represents the sub-target layer weight vector and represents the weight vector for each type of vibration signal.
The score set is set to . Therefore, the health evaluation vector of the target layer launch vehicle engine system under the multi-accumulation vibration signal is shown below:
,
,
,
,
,
.
The health score calculation formula is:
, the real data health score is 95.6946, the prediction evaluation result is shown in
Table 13 and the minimum error is 0.0193%. The status is in line with expert experience.
7. Conclusions
This paper proposes a predictive evaluation method for a launch vehicle engine. This method has the advantages of being objective as well as having a high amount of precision and strong versatility. Compared with the membership function method commonly used in the traditional method, the method based on the boxplot principle is adopted to obtain a higher evaluation accuracy. Combined with the AHP method, the weight factors of various vibration signals are obtained, and the evaluation process is not overly dependent on expert experience. Finally, the fuzzy comprehensive evaluation method is used to obtain the health value of the launch vehicle engine from the bottom up, which is suitable for complex fuzzy systems. The experiment with actual data verified that the method has a high level of accuracy.
The method proposed in the paper has important guiding significance for the healthy evaluation and management of launch vehicles and can effectively reduce the state monitoring pressure of ground survey personnel. At the same time, the preprocessing method of data sampling makes the algorithm lightweight, and, combined with a high-precision data prediction effect, it is suitable for synchronous operations during the launch period. We obtained a health score through the five steps of the spatial division of health states, data prediction, evaluation set generation, a weight analysis and fuzzy comprehensive evaluation, which is a convenient method for ground-monitoring personnel to accurately grasp the flight status of a rocket after launch and can be used in the formulation of flight strategies.