A Review of Anomaly Detection in Spacecraft Telemetry Data

Abstract

Telemetry data play a pivotal role in ensuring the success of spacecraft missions and safeguarding the integrity of spacecraft systems. Therefore, the timely detection and subsequent notification of any abnormal events related to the functionality of spacecraft subsystems are crucial to ensure their safe operation. In recent years, several anomaly detection methods have been developed to monitor spacecraft telemetry data and detect anomalies. This manuscript provides a comprehensive literature review of the existing anomaly detection methods for spacecraft telemetry data. It exposes the challenges faced by such systems, highlights the strengths and limitations of each anomaly detection method, and assesses and compares the performance of these approaches in detecting anomalies. Initial results show that GCN and TCN models have achieved promising precision up to 94%. The paper concludes with a series of recommendations and the potential research directions.

1. Introduction

Spacecraft [1,2,3,4] are complex machines designed to carry out a variety of missions in outer space environments. Key types of spacecraft range from satellites to space probes, space stations, and landers, while applications include scientific research, exploration, communication, navigation, and military. The internal systems of these space machines are equipped with thousands of detailed telemetry channels to monitor parameters such as power, temperature, radiation, and instrumentations. Monitoring and diagnosing the status of telemetry data is one of the most import concerns for spacecraft safety. In fact, failure to detect or notify with regard to any abnormal behavior can result in irreparable loss and/or failure of the spacecraft during launch or while in space. Therefore, it is vital to efficiently and continually detect anomalies present in spacecraft telemetry to guarantee its reliable and safe operation during space missions.

Spacecraft telemetry data are acquired from various subsystems and components at several temporal resolutions. These data are then transmitted to the ground telemetry station where the original variable information of each channel is restored. Telemetry data are then analyzed and interpreted by domain experts. Figure 1 summarizes the typical end-to-end communication between spacecraft, ground station, and domain experts. The complex nature of spacecraft telemetry data poses several challenges [5,6] in developing appropriate anomaly detection methods:

• High data volume: Spacecraft generate large time series telemetry data volumes with high dimensionality and complex relations between telemetry parameters, up to several terabytes of data per day. This high data volume poses significant challenges and difficulties for anomaly detection.
• Complex multivariate data: Spacecraft telemetry data often include noisy and heterogeneous outputs produced by various system components. Some of them are discrete such as the switching status, while the others are continuous, e.g., temperature and radiation. These differences can reduce the performance of anomaly detection methods.
• Lack of ground truth: Anomaly detection methods are hindered by the scarcity of labeled anomalies, as such ground truth is needed both for supervised learning approaches and also for performance evaluation. The absence of labeled anomalies for the validation can be solved by using semi-supervised or unsupervised methods.
• Need for domain expertise: Expert knowledge of spacecraft subsystems and components is often required when analyzing telemetry data, in particular to label anomalies or analyze the performance of the anomaly detection methods.
• Real-time processing capabilities: Given that it is impossible for domain experts to manually monitor thousands of telemetry channels to identify anomalies, automatic techniques which can detect such anomalies in real time are needed. In certain cases, it is desirable to deploy anomaly detection techniques on onboard computers to ensure that issues are detected as quickly as possible.

In recent decades, there has been a growing interest in exploiting anomaly detection methods to monitor spacecraft telemetry data. Many prominent methods and strategies have been proposed for the purpose of identifying anomalies [8,9,10,11]. Such methods allow for mitigating action to be taken before incidents spread. This review paper aims to document the state of anomaly detection in spacecraft telemetry data by exposing and discussing the existing anomaly detection techniques in the literature as well as their performance. Five main criteria [12] were used to evaluate information from the reviewed literature, which are the following:

• Accuracy: Information was extracted from reliable and well-known sources (peer-reviewed journals, books, official websites, etc.) based on proven facts.
• Authority: The authors of the selected papers are affiliated with a reputable university or organization in the subject field.
• Objectivity: The presented results are based on well-defined methodologies and accurate experimental studies.
• Currency: This survey presents an overview of the recent literature in the field of anomaly detection of spacecraft telemetry systems.
• Coverage: Our work provides in-depth coverage of the various methods and schemes used to handle the spacecraft anomaly detection issue.

The main contributions of this survey include the following:

• We review the existing literature and present a detailed taxonomy of the different approaches and strategies addressing the anomaly detection problem in spacecraft telemetry data.
• We introduce the different types of anomalies in spacecraft telemetry data and describe benchmark datasets and evaluation metrics explored in this field. This allows us to perform a fair comparison among anomaly detection methods.
• We compare and assess the performance of the different methodologies for the problem of anomaly detection in spacecraft telemetry data. The results provide researchers and practitioners an overall and comprehensive vision.

This review is organized as follows. In Section 2, we introduce the anomaly detection process and the current types of anomalies in spacecraft telemetry data. Then, we review existing anomaly detection methods for spacecraft telemetry data in Section 3. Datasets and the performance metrics of the reviewed methods are provided in Section 4. Section 5 presents a series of finds and potential research axes. The final section concludes the survey.

2. Anomaly Detection in Spacecraft Telemetry Data

2.1. Anomaly Detection

Anomaly detection [13,14], known also as outlier detection, refers to the procedure of identifying unexpected values or observations that differ significantly from the norm. A broad range of applications of anomaly detection in different domains exists, including industrial control systems [15], Internet of Things [16], medicine [17], and aerospace [18]. With regard to aerospace, one of the applications of anomaly detection is to identify novel or unusual events in spacecraft, enabling specialists to quickly identify abnormal events and take the appropriate corrective action.

2.2. Types of Anomalies

With regard to time series telemetry data, anomalies can be divided into three main categories [5,10,11,19,20] which are summarized below:

• Point anomalies: A point anomaly is an individual data instance which stands out from the expected pattern, range, or norm. For this type of anomaly, temporal information is irrelevant. Therefore, it is considered the easiest to detect.
• Collective anomalies: A collective anomaly, also called group anomaly, corresponds to a sequence of values which differ significantly from the rest of the data.
• Contextual anomalies: An individual data instance or a sequence of values considered as anomalous in a specific context. This means that observing the same value or sequence of values through different contexts will not always give us an indication of anomalous behavior.

Collective and contextual anomalies involve the temporal dimension, and consequently, they require more complex methodologies such as machine and deep learning algorithms.

3. State of the Art Anomaly Detection for Spacecraft Telemetry Data

In the last decade, many anomaly detection methods have been proposed to monitor spacecraft telemetry channels. An extensive literature search reveals that anomaly detection for spacecraft telemetry can be grouped into two main families: thresholding methods and artificial intelligence methods. This section is dedicated to describing the relevant anomaly detection methodologies for each category.

3.1. Thresholding Methods

Thresholding methods, also called Out-Of-Limits (OOL) alarms or statistical models, are the most commonly used techniques to detect anomalies in spacecraft [20,21,22,23]. These approaches aim to check whether the spacecraft telemetry data measurements are within a predefined threshold. Any telemetry parameter that exceeds the said threshold is considered anomalous, and an alarm is activated. Statistical measurements such as the mean, standard deviation, maximum, and minimum are used to compute thresholds.

Thresholding methods have exhibited certain benefits such as their simplicity and being easy to understand and interpret. However, they suffer from some shortcomings. In fact, setting the threshold value depends heavily on domain knowledge and expert experience, which make these methods less precise and very time-consuming. In addition, the presence of complex noise in spacecraft telemetry data could lead to high false alarm rates. In general, thresholding methods ignore complex interactions between telemetry parameters.

To overcome the limitations of thresholding techniques, some improvements have been introduced. For example, Heras and Donati [9] have applied thresholding models in many ESA (European Space Agency) missions by integrating a new automatic telemetry monitoring prototype. The novel detection monitoring approach has been managed to analyze the behavior of 2000 parameters during the XMM (X-ray Multi-Mirror Mission) Newton orbit mission. In [24], another anomaly detection method, based on parametric causality and Double-Criteria Drift Streaming Peaks Over Threshold (DCDSPOT), was proposed to solve the problem of high rates of false negatives. The performance of the DCDSPOT method was assessed using four anonymous telemetry parameters generated by a military communications satellite. Compared to other baseline methods, the proposed method obtained the highest recall (91%) and precision (85%).

3.2. Artificial Intelligence Methods

Over the past few decades, researchers have progressively automated the anomaly detection process using Artificial Intelligence (AI) methods [20,22,25,26,27,28,29,30,31,32,33,34,35,36]. Several AI techniques, designed to learn and extract features from telemetry data, have been used widely to detect abnormalities in spacecraft. These methods mainly include machine and deep learning models.

3.2.1. Machine Learning Methods

Machine Learning (ML) has significant potential to enhance anomaly detection in spacecraft telemetry. ML approaches are used to build a model that can differentiate between normal and abnormal instances in telemetry data. In recent years, several ML approaches have been employed for anomaly detection in spacecraft telemetry systems. Each of these approaches possesses advantages and limitations. The choice of the selected approach depends on the type of anomaly and the availability of ground truth. ML-driven approaches for anomaly detection include Support Vector Machine (SVM) models, nearest neighbors approaches, clustering approaches, dimension reduction methods, and neural networks.

Support Vector Machine (SVM): SVM is a supervised ML method used widely in the literature to identify abnormal behaviors of spacecraft. For instance, SVM [37] was used to detect anomalies in the control subsystem of an anonymized in-orbit satellite. It examined the behavior of 12 parameters (e.g., gyroscope, earth sensor, controller, etc.), and it obtained an accuracy of 97.4%. The use of SVM was also investigated for the NASA SMAP/MSL dataset [38], obtaining a recall of 85.7% and a precision of 84.1%. Another work which utilized the SVM with ESA OPS-SAT telemetry data has been shown in [26]. In this work, SVM achieved 86.62% and 66.37% with respect to precision and recall, respectively. Different unsupervised variants of the SVM model have been used for anomaly detection on spacecraft telemetry such as One-Class SVM (OCSVM) [36,39,40], Least Squares SVM (LS-SVM) [28], and Integrated Least Squares SVM (ILS-SVM) [41].

Nearest neighbors: Anomaly detection of spacecraft telemetry has also been tackled with nearest neighbors approaches. Nearest neighbors [42] anomaly detection methods assume that normal data points arise in dense, clustered neighborhoods, while anomalies appear far from their nearest neighbors. These methods require a distance or similarity measure defined between two data instances. K-Nearest Neighbors (KNN) [10,26,36,40] and improved KNN [43] are two methods which have been used to conduct anomaly detection on spacecraft telemetry data. The experimental study performed on the OPS-SAT telemetry dataset in [26] revealed that the KNN approach provides decent anomaly detection, reaching 61.54% in terms of recall, which is slightly lower than the results obtained by the SVM model. Improved KNN, which was tested on the ADAPT-Lite EPS (Electrical Power System) dataset, has obtained a higher accuracy of anomaly detection compared to the neural network method. However, the proposed method suffered from poor performance and is computationally heavy when faced with large and extremely uneven distribution sample datasets.

Clustering approaches: Detecting unexpected patterns in the spacecraft telemetry data has also been attempted using clustering approaches [39,44,45,46]. For example, Gao et al. [44] developed a normal behavior clustering anomaly detection approach. They performed this method to detect anomalies of six parameters related to the power subsystem of an actual in-orbit satellite. Jin et al. [45] adopted an extended dominant sets clustering algorithm to distinguish between normal and abnormal samples of synthetic and real telemetry datasets generated by the Tianping-2B satellite. The proposed clustering algorithm managed to increase the anomaly detection score by 3–10%. The works presented in [39,46] have proposed different clustering strategies to detect anomalies. The first one uses probabilistic clustering and dimensionality reduction. It was implemented to catch faults in the telemetry data of the Small Demonstration Satellite 4 (SDS-4) of the Japan Aerospace Exploration Agency (JAXA). The second paper proposes a density peak clustering-based two-stage anomaly detection approach to check whether the system of the analogue solar sensor on the Zheda Pixing 1A (ZDPS-1A) satellite is normal or not.

Dimensionality Reduction (DR): DR approaches have emerged as valuable tools for identifying anomalies. The key idea of DR methods [47] is representing the principal system behavior of the spacecraft telemetry data by a much smaller number of latent variables. Recovering the original dimensions of the reduced data system and compare them with the original one has proven to be useful in detecting anomalies. Principal Component Analysis (PCA) [36,48,49], Kernel PCA (KPCA) [48], Mixture Probabilistic PCA (MPPCA) [50], and the Laplacian Eigenmaps Latent Variable Model (LELVM), among others, have been used to identify spacecraft anomalies. These methods have been assessed using different real spacecraft telemetry datasets; some of them are anonymized like in [49], while the others are provided by the EPS data of two satellites owned by JAXA [47,48] or the Korea Multi-Purpose Satellite 2 (KOMPSAT-2) [50]. Considering the results obtained in [47], MPPCA was considered as the best performing among the reference methods (PCA, KPCA, LELVM, etc.) both in the detection accuracy and computational time.

Neural Networks (NN): There are some works which utilized neural networks to detect anomalies in spacecraft telemetry data. For example, an NN-based method was assessed in [37], and it gave an accuracy of 92%. Bernard et al. [51] proposed an improved NN method called Envelop Learning and Monitoring using Error Relaxation (ELMER). This method uses the NN to periodically set new threshold bounds, providing faster detection with fewer false alarms. Ramachandran et al. [52] proposed an Anomaly Detection via Topological-feature Map (ADTM)-based approach. The ADTM method combines a two-layer Artificial Neural Network (ANN) and the Extra Tee Classifier (ETC) approach. The performance of ADTM was validated on telemetry data collected from a Lab-Stationed CubeSat (LabSat), and it proved its performance at detecting both known and unknown anomalies. Extreme Learning Machines (ELMs) are another NN model, introduced by Baireddy et al. in [27], to detect any abnormalities in the behavior of the channels. The current approach is able to learn from the time series data in real time. It lessens the amount of training and data required to obtain a predictor model for each channel. Experiments conducted on the SMAP/MSL dataset have proved that this method is able to achieve comparable performance to the state-of-the-art spacecraft anomaly detection methods with minimal training time and data.

In addition to one-dimensional data, ML methods can support multi-dimensional data. Indeed, these methods perform very well when the size of data is small. However, the above models have suffered considerable challenges due to the increasing volumes of complex telemetry data, which limits the detection performance, results in high false detection rates, and does not allow for the efficient interpretation of anomaly detection of telemetry data. Recently, there has been a shift in interest toward deep learning (DL) methods instead of ML approaches due to their capacity to handle the large volume of data and the complex interactions among multi-variate spacecraft telemetry data.

3.2.2. Deep Learning Methods

With the remarkable growth of DL, a significant interest has also been noted in exploring these advanced algorithms for anomaly detection in space operations. Convolutional Neural Networks (CNNs) [8,9,10,30], Recurrent Neural Networks (RNNs) [20,22,36], and Transformers are different classes of DL models, which have been used recently to detect anomalies in spacecraft telemetry data [31,32,53].

Convolutional Neural Networks (CNNs): CNNs consist of consecutive convolutional, pooling, and fully connected layers that adaptively learn the spatial hierarchies of features from the input sequence data. These architectures have achieved leading performance on various spacecraft anomaly detection tasks. For example, Yuan et al. [54] explored a specific CNN method called Temporal Convolution Network (TCN) to detect anomalies in multi-dimensional spacecraft telemetry data. The performance of TCN was evaluated using a set of real telemetry datasets of solid state power amplifiers which were generated from two different satellites. The telemetry variables of the first satellite include voltages and currents, while the variables provided by the second satellite involve temperatures and currents. The TCN model performed satisfactory detection performance, obtaining an average accuracy of 96% for short time step and 96.45% for long time steps, in addition to a smaller inference time in both cases. Another anomaly detection framework for spacecraft telemetry data based on TCN was proposed in [7]. This novel method utilizes dynamic graph attention to model the complex correlation among variables and time series. It outperformed state-of-the-art models in terms of $F_1$ score, precision, and recall on SMAP/MSL spacecraft datasets.

The Graph Convolutional Network (GCN) is another attempt, proposed in [55], to detect anomalies in spacecraft telemetry data. The results obtained on SMAP/MSL spacecraft datasets support the capability of the GCN in detecting anomalies in telemetry data. Inspired by the success of the GCN in spacecraft anomaly detection, Song et al. [30] developed another GCN variant. The novel method extracts the correlation information between the telemetry variables and provides attention for subsequent telemetry data anomaly detection. It was tested on SMAP/MSL datasets as well, and it obtained a 94.86% precision result and 99.59% recall result, which superseded the results obtain in [55].

The work in [8] discusses other state of-the-art CNN arcitectures such as Residual Networks (ResNet) and the Fully Convolutional Network (FCN). The latter can effectively process time series data, capturing valuable historical information for future prediction. Experimental results, conducted on the typical SMAP and MSL datasets, have proved that the ResNet model presents high performance, taking into account accuracy and computational efficiency, while the FCN shows a marked proficiency in capturing spatial patterns.

Recurrent Neural Networks (RNNs): RNNs present another class of neural networks, which are used to process sequential data. Unlike CNN models, RNNs have connections that loop back, allowing them to maintain the memory of previous inputs in the sequence. Long Short Term Memory (LSTM) [20,22,36] and Gated Recurrent Unit (GRU) [8,56] are the most known RNN architectures. Both of them use gates to learn the relationship between past and current data values. In addition to spatial components, RNNs (LSTMs and GNUs) show a noted proficiency in capturing temporal anomalies in spacecraft telemetry data.

LSTMs were designed to fix the vanishing gradient problem inherent in the original RNN architecture, making them able to learn from long-range sequences. There exist many LSTM models which have been trained specifically to process spacecraft telemetry data. For example, the study originally proposed by Hundman et al. in [20] showed the utility of LSTMs in detecting abnormal behavior in real time series data. This method, which resulted in a precision and recall of 87.5% and 80.0%, respectively, on the NASA SMAP and MSL datasets, was improved upon in [57] by integrating Causality Features (CFs). In contrast to the classic LSTM, the CF-LSTM model treats the correlation between the parameters, which helps to enhance the prediction precision and is sensitive to anomalies. Using 20 parameters generated by a public dataset from a real system, CF-LSTM showed an improved detection precision, recall, and $F_1$ score compared to LSTM and other anomaly detection models. Transfer learning [22], adapted to construct a general pretrained model to any specific telemetry channel, was shown to be effective in reducing the training time on the NASA SMAP/MSL dataset when it was integrated with the LSTM model. The work in [58] shows that a combination between the LSTM model and an attention multi-scale anomaly detection strategy was effective at detecting anomalies on NASA SMAP/MSL benchmark spacecraft data and the hydrogen clock data of the Beidou Navigation Satellite. The multi-scale anomaly detection strategy integrates the detection of global and local features at different time scales which provides higher recall rate and fewer false alarms. In [59], two methodologies that integrated the Bayesian neural networks and the Variational Auto-Encoder (VAE) within the LSTM (MCD-BiLSTM, MCD-BiLSTM+VAE) have been proposed for detecting anomalies. The experimental results on an imbalanced spacecraft telemetry dataset show that the proposed models can obtain higher scores than traditional neural networks.

GRU is another RNN model designed to process temporal features within sequential data. This model, which is built to capture dependencies for sequences of varied lengths, has a simpler structure compared to LSTM. This advantage offers the capacity to model data sequences with reduced computational requirements. The GRU model [7] was trained on the SMAP/MSL datasets to handle the anomaly detection problem. It showed a sufficient performance/training time tradeoff with respect to the conventional methods. GRU method leveraged the benefits of the Extreme Value Theory (EVT) in [56] to process multiple channels of telemetry data and maximize the detection performance. Yu et al. [60] evaluated an anomaly detection approach based on a Variational Auto-Encoder with a Gated Recurrent Unit (GRU-VAE) for anomaly detection and diagnosis of spacecraft system. The GNU-VAE showed lower anomaly detection results on the SMAP/MSL datasets than the other baseline methods.

Transformers: Exploring the self-attention mechanism, transformer-based anomaly detectors have achieved great improvements in time series tasks [8,29,31,32,53]. This success is attributed to their great power of discovering reliable long-range temporal dependencies. The experimental results in [29,32] demonstrated that the different transformer-based architectures present promising results in computational efficiency. For example, they could save about 80% of the training time compared to LSTM. They have the advantage that they can focus on both local and global contexts, especially in scenarios where anomalies are subtle and span over longer durations. In [31], Tuli et al. have proposed a transformer based encoder–decoder model called TranAD able to identify data trends with limited data. The novel approach, tested on eight different spacecraft datasets, managed to improve the $F_1$ score by 11% with a limited training dataset and identify up to 75% of the detected anomalies, which was higher than the reference methods. In [53], a transformer-based regression model was used to analyze one-dimensional time series data of a specific spacecraft parameter. Experimental results demonstrate that this approach surpasses some ML methods like Random Forest and SVM.

3.2.3. Hybrid Models

Hybrid approaches are increasingly common. These approaches integrate different algorithms or methodologies to improve detection accuracy and reliability by exploiting the strengths of different strategies. To benefit from different anomaly detection models, many works have studied hybrid models [8,60,61,62,63]. For example, Tariq et al. [62] combine the LSTM with the MPPCA model for spacecraft anomaly detection. The developed method tends to overcome the limitation of the single channel of the LSTM model and improve the detection performance. It was able to attain a precision of more than 90%. The LSTM model was also combined with the transformer in [8,63]. Compared to the original models (transformer and LSTM), the hybrid transformer–LSTM structure achieved higher precision. It could also effectively extract both global and local features from satellite time series data. Another study [61] considered a hybrid CNN–transformer model, joining the spatial learning efficiency of CNNs with the sequence learning of transformers to extract global and local spatial information. Experiments have demonstrated that the designed model can learn the spatial–temporal correlations from multi-variate time series data and achieve better anomaly detection performance than the literature approaches. Some good results based on CNN and LSTM within a spatial–temporal Generative Adversarial Network (GAN) were obtained in [60].

Experimental studies have shown that the hybrid method presents high aptitudes to detect anomalies in real datasets while conserving a lower false alarm rate. The current state-of-the-art methods with their corresponding benefits, limits, and experimental details can be seen in

Table 1. Summary of anomaly detection methods for spacecraft telemetry systems.

Method(s)	Dataset(s)	Anomaly Type(s)	Metric(s)	Benefit(s) and Limit(s)
Thresholding [9,24]	- ESA XMM-Newton - Military communication satellite	Sequential anomaly	- Precision - Recall - $F_1$ score - Graphic presentations	+ Simple to use + Easy to understand - Not sufficient for the complex spacecraft systems with a several numbers of sensors - Requires engineering effort - Requires amount of tuning
SVM [26,37,38], OCSVM [39], LS-SVM [28], ILS-SVM [41]	- NASA SMAP/MSL - ESA OPS-SAT	- Point - Contextual - Collective	- Accuracy - Precision - Recall - $F_1$ score - Time	+ Capture complex non-linear relationships between features. + Manage with dataset with limited or no labeled anomalies. - Issues with large datasets and high-dimensional spaces.
Nearest neighbors: KNN [10,26], Improved KNN [43]	- ESA OPS-SAT - ADAPT-Lite EPS	—	- Accuracy - Precision - Recall - $F_1$ score - Mirco-$F_1$ score	+ High accuracy - Poor effect when the sample set has uneven distribution. - Long calculation time for large sample dataset.
Clustering approaches [39,44,45,46]	- Tianping-2B - JAXA: SDS-4	—	- AUC - False positive rate - Accuracy	+ Little a priori knowledge + Low construction cost - Unable to detect anomaly in local areas
PCA [48,49], KPCA [48], MPPCA [50], LELVM [47]	- JAXA: EPS - KOMPSAT-2	—	- Reconstruction error - Time	+ Process high-dimensional datasets + Monitor multi-variable statistics - Parameter tuning issues
ELMER [37], ADTM [52], ELMs [27]	- LabSat - NASA SMAP/MSL	—	- Minimum Quantization Error (MQE) - Mean Square Error (RMSE) - Precision - Recall	+ Reduce false positives + Reduce computation time + Detect known and unknown anomalies
CNNs: TCNs [7,54], GCNs [30,55], ResNet, and FCNs [7]	- NASA SMAP/MSL	- Point - Contextual - Collective	- Precision - Recall - $F_1$ score - TPR - FPR - Accuracy - Time	+ High performance in capturing spatial patterns + Able to model high dimensionality and complex correlation among variables + Low false detection rate + Computational efficiency
RNNs: LSTMs [20,22,57,58], GNUs [8,56,60]	- NASA SMAP/MSL - Power System of a Rocket (PSR)	- Point - Contextual - Collective	- Precision - Recall - $F_1$ score	+ High detection performance + Process multiple channels + Perform very well with short sequences + Capture temporal anomalies + Low false positive rates
Transformers [29,31,32,53]	- NASA SMAP/MSL - Numenta Anomaly Benchmark (NAB) - HexagonML (UCR) - MIT-BIH Supraventricular Arrhythmia Database (MBA) - Secure Water Treatment (SWaT) - Water Distribution (WADI) - Server Machine Dataset (SMD) - Multi-Source Distributed System (MSDS)	- Point - Contextual - Collective	- Precision - Recall - $F_1$ score - ROC - AUC	+ Reduced consumption time compared to LSTMs and GNUs + Process length sequences better than CNNs and RNNs + Suitable for intercorrelated anomalies + Early anomaly detection - Require significant computational resources

The symbol “+” is used to refer to the benefits or strengths of the method(s).

. Figure 2 presents the taxonomy of anomaly detection approaches used in spacecraft systems.

Table 2. Performance metrics of different AD methods on NASA SMAP/MSL datasets.

Approach	Precision (%)	Recall (%)
LSTM [20]	87.5	80.0
LSTM + transfer learning [22]	79.4	83.5
Attention TCN [7]	94.74	94.17
ResNet [8]	80.76	58.87
FCN [8]	69.31	58.09
ELMs [27]	76.4	77.9
SVM [38]	84.1	85.7
Bi-Transformer [32]	85.4.0	72.4
TranAD [31]	85.40	99.99
GCN [30]	94.86	99.59
GNU [56]	89.2	89.6
GRU + VAE [60]	81.41	94.46
LSTM + GAN [60]	90.28	92.45
LSTM + multi-scale strategy [55]	87.37	86.54

presents the performance metrics (precision and recall) of anomaly detection for ML/DL approaches on the SMAP/MSL datasets. We highlight the best results in bold. From this table, we can see that the GCN [30] has the highest precision, while TranAD [31] outperforms the reference methods in terms of recall.

3.2.4. Computational Efficiency

Although performance should always be a key priority, computational efficiency has significant interest for real-time spacecraft applications.

Table 3. Computational time of AD methods on NASA SMAP/MSL datasets. Values in italics present the run time for each sample, while the rest correspond to the average training/inference time for each channel/parameter.

Approach	Computational Environment	Training Time (s)	Inference Time (s)
LSTM + transfer learning [22]	—	71.45	—
ResNet [8]	Virtual machine equipped with 8 CPU cores: Intel	135	—
FCN [8]	Xeon Platinum 8260 CPU @ 2.40 GHz and 16 GB of RAM	555	—
Bi-Transformer [32]	—	109.75	—
TranAD [31]	—	0.66	—
GRU + VAE [60]	Huawei G5500 Linux server with NVIDIA Tesla V100	—	0.013345
LSTM + GAN [60]	GPU	—	0.01085

reports the average training/inference time in seconds per channel/sample for some proposed models on the SMAP/MSL datasets. For example, TranAD [31] provides the best performance in training time consumption. This is because TranAD uses positional encoding to push the whole sequence as an input in place of using local windows.

Transfer learning used within the LSTM method [22] saves more training time compared to FCN and ResNet [8] architectures. From our analysis, the computational time required depends highly on the deployment setting and computational resources. All these variables make it difficult to fairly benchmark computational efficiency, although it is clear that the TranAD method has an edge, as also evidenced by the results we obtain for TranAD on a different dataset.

4. Datasets and Performance Evaluation Indices

In this section, we discuss most of the spacecraft telemetry datasets to which anomaly detection approaches in the literature have been applied. The commonly used performance metrics are also highlighted.

4.1. Datasets

The performance of the algorithms and methods introduced in the previous section was demonstrated on several different datasets. Several anomaly detection methods was tested on synthetic [45] or simulated spacecraft telemetry datasets [37,48,54], while over 80% of anomaly detection application research in spacecraft was done using real public or anonymized datasets [7,8,9,20,22,24,26,27,28,29,30,31,32,36,44,47].

In simulated datasets, the anomalous data are produced by injecting fault into the ordinary data. For example, in [54], point and contextual anomalies were injected into the real data to evaluate the efficiency of TCN method. In [37], Gao et al. inserted fault data into the normal data of a control subsystem of an actual in-orbit satellite, using the Matlab/Simulink environment, to detect and identify anomaly type in telemetry data.

Numerous real datasets were designed and used to evaluate the state-of-the-art anomaly detection methods. These datasets, collected from different spacecraft telemetry systems, mainly include the following:

• ESA datasets: ESA benchmark datasets such as Sentinel-1 [36] and OPS-SAT [26,40] were tested with several numbers of ML and DL algorithms for anomaly detection. The ESA Sentinel-1 dataset contains data from 10 channels, while OPS-SAT includes data from 9 channels. In [9], another ESA database, collected from the XMM-Newton orbit, was explored to detect anomalous behavior in space operations using OOL alarms. The XMM-Newton dataset contains around 2000 housekeeping telemetry parameters.
• NASA SMAP/MSL datasets: The Soil Moisture Active Passive (SMAP) satellite and Mars Science Laboratory (MSL) rover data are the most commonly used telemetry datasets [3,7,8,20,29,30,31,32,64] in spacecraft anomaly detection due to dataset availability. The SMAP and MSL are very different missions. The SMAP collects information about the moisture level within the first five centimeters of soil to obtain a better understanding of the carbon and water cycle, while the MSL was designed to study the potential habitability in the Red Planet in the past and present environments. The two data missions were published by the NASA Jet Propulsion Laboratory. They contain 81 channels of telemetry data. The SMAP/MSL data consist of 496,444 telemetry values associated with 105 anomaly sequences. Anomaly measurements present only 16.12% of the total samples. They contain about 75387 anomalous samples. Proportionally, 59% of anomalous data correspond to point anomalies and 41% to contextual abnormalities.
• JAXA datasets: Some approaches in the literature were designed for different JAXA missions. For example, the authors in [21,47,50] have studied the efficiency of DR methods on the EPS of different JAXA satellites such as the Mission Demonstration Satellite 1 (MDS-1) and Data Relay Transponder Satellite (DRTS). In [39], a clustering approach was implemented on a telemetry data acquired by Small Demonstration Satellite 4 (SDS-4) using onboard software by the JAXA satellite.
• Other telemetry datasets: Other spacecraft telemetry datasets such as KOMPSAT-2 [62], the Power System of a Rocket (PSR) [60], and Tianping-2B [45] have been used to study telemetry anomaly detection approaches. For example, KOMPSAT-2 was used to assess the performance of combining the LSTM and MPPCA algorithms in detecting anomalous events. A more advanced LSTM variant was applied to Tianping-2B, where the authors developed an anomaly detection method based on GANs. The work proposed in [60] presents the performance of clustering method to a power system of a rocket satellite.
• Anonymized spacecraft in-orbit datasets: For confidentiality or security purposes, many approaches chose to do not publish the real parameters names, timestamps, source or any other details about the spacecraft datasets used [24,28,44,45].

Indications of telemetry anomalies for many spacecraft, including the SMAP and MSL, can be found within “Incident/Surprise/Anomaly” (ISA) reports. ISA reports provide valuable information about the incidents and anomalies which appear in specific telemetry channels.

We summarize the characteristics of various spacecraft telemetry datasets in

Table 4. List of spacecraft telemetry datasets gleaned from research articles presenting spacecraft anomaly detection techniques. The datasets in italics are publicly available.

Source	Type and Format	Nbr of Values	Nbr of Channels	Nbr of Parameters	Nbr of Anomalies	Approach(es)
Synthetic datasets
Unknown		5635	—	—	224	Clustering [45]
Simulated datasets
Solid state power amplifier		—	—	4 parameters: + Temperatures + Currents1 + Currents2 + Voltages	—	TCN, LSTM [54]
Control subsystem data of an actual in-orbit satellite		25,000	—	12	—	SVM, NN [37]
Real datasets
ESA Sentinel-1	Multi-variate time series	146,887	10	—	7	PCA, KNN, OCSVM, LSTM [36]
ESA OPS-SAT [65]	Univariate time series	303,493	9	- 3 parameters of magnetometer - 6 parameters of photo diode	445	SVM, KNN [26]; KNN, PCA, OCSVM [40]
ESA XMM-Newton	Multi-variate time series	—	—	2000	—	OOL [9]
NASA SMAP/MSL [66]	Multi-variate time series	496,444	81	80	105	LSTM [20]; LSTM + transfer learning [22]; Attention TCN [7]; ResNet, FCN [8]; KNN [10]; ELMs [27]; SVM [38]; Transformers [29,31,32]; GCN [30]; GRU [56]; GRU+VAE, LSTM+GAN [60]; GNN-DTAN [55] LSTM+multi-scale strategy [58]
JAXA SDS-4	Multi-variate time series	—	—	445 parameters of: - Attitude Control Subsys (ACS) - Electrical Power Subsys (EPS) - Thermal Control Subsys (TCS) - Command and Data Handling Subsys Transmitter - Receiver Subsys (TRX)	—	Clustering, OCSVM [39]
JAXA EPS: MDS-1	Multi-variate time series	—	—	—	—	MPPCA [50]
JAXA EPS: DRTS	Multi-variate time series	—	—	—	—	MPCA [50]
JAXA EPS	Multi-variate time series	3900	—	120	—	PCA, KPCA, MPPCA, LELVM [47]
China Academy of Launch Vehicle Technology: PSR	Multi-variate time series	27,928	1	19	—	GRU + VAE, LSTM + GAN [60]
China Aerospace Science and Technology Corporation (CASC): Tianping-2B	Multi-variate time series	21,010	—	Magnetometer parameters	—	Clustering [45]
ADAPT-Lite EPS	Multi-variate time series	—	—	—	—	Improved KNN [43]
LabSat	Multi-variate time series	—	—	—	—	NN [52]
Korea Aerospace Research Institute: KOMPSAT-2	Multi-variate time series	22,028,183	—	—	15	LSTM, LSTM + MPPCA, OCSVM [62]
Military communication satellite	Multi-variate time series	67,968	—	20	—	Thresholding [24]
Anonymized in-orbit spacecraft	Multi-variate time series	—	—	—	—	LS-SVM [28]
Anonymized power subsystem of in-orbit satellite	Univariate time series	—	6	—	—	KNN [44]
Anonymized attitude control system of in-orbit satellite	Multi-variate time series	5000	—	14	—	PCA [49]
Real telemetry system of in-orbit satellite [67]	Multi-variate time series	—	20	—	—	CF-LSTM [57]

. Several articles do not list certain dataset parameters such as the number of telemetry values, the number of channels, and the number of anomalies.

4.2. Evaluation Indices

The implementation of different methods provides the opportunity to compare the obtained results. In order to quantify the anomaly detection performance, the vast majority of papers, especially those which adopt ML and DL models [7,20,21,22,24,25,26,27,28,29,30,31,32,36,37,38,39,41,42,43,44,45,46,47,48,49,50,52,53,54,55,56,60,61,62,63,68], make use of confusion matrices and visualizations. A confusion matrix displays the number of true or false positives (TPs/FPs) and the respective true or false negatives (TNs/FNs). A TP is registered if the detected anomalous sequence overlaps with a ground truth anomalous sequence. If multiple detected anomalies overlap with one labeled sequence, only one TP is recorded. A FN is recorded if no predicted sequences overlap with the ground truth anomalous sequence. On the other hand, a FP is registered if a detected anomaly sequence does not overlap with a true anomaly sequence. The number of correctly detected anomalies is determined by the sum of TPs or TNs, while the set of incorrect anomalies is obtained from the total number of FPs or FNs.

Accuracy, precision, recall, $F_1$ score, true positive ratio (TPR), false positive ratio (FPR), false alarm rate, receiver operating characteristic (ROC), area under curve (AUC) and prediction error (RMSE, MQE…), among others, were employed in the literature as metrics to evaluate the performance of the anomaly detection models.

Some papers, such as in [6], consider that the accuracy metric can be a misleading measure when the data are highly imbalanced. In this case, it is recommended to conduct model evaluation in terms of precision, recall, $F_{0.5}$ and $F_1$ scores. A high precision means that the model has a low false positive rate, while a high recall means that a low number of true anomalies are undetected.

$F_1$ and $F_{0.5}$ are evaluation metrics that assess the overall effectiveness of an anomaly detection model by integrating both recall and precision. The $F_1$ score equally weights precision and recall, providing a balanced measure of a model’s performance. It is especially useful when both precision and recall are equally important in the anomaly detection problem. The $F_{0.5}$ score places more emphasis on precision than recall, making it more suitable for situations where reducing false positives is a priority. $F_1$ and $F_{0.5}$ are defined as follows:

$$F_{1}score={\displaystyle \frac{2\times Precision\times Recall}{Precision+Recall}}$$

(1)

$$F_{0.5}score={\displaystyle \frac{1.25\times Precision\times Recall}{0.25\times Precision+Recall}}$$

(2)

The TPR determines the rate of anomalies correctly identified by a model, while the FPR measures the proportion of normal instances incorrectly reported as anomalies (false alarms).

The thresholding methods use simple performance metrics such as the false positive rate and accuracy. In addition to anomaly detection performance metrics, some studies [9,22,32,47] have accorded importance to the computation time.

5. Discussion

An overview of anomaly detection models in spacecraft telemetry data was presented in this paper. In the existing literature, anomaly detection models for time series data can be categorized based on their main concepts and architectures into two main groups: thresholding methods and AI methods.

Anomaly detection based on thresholding methods relies on simple processing and inspection of data by domain experts. However, with the evolution of spacecraft telemetry systems, the volume of data has increased, making such classical thresholding methods impractical. On the other hand, automatic anomaly detection algorithms, including ML methods, are becoming increasingly prevalent. Over the last decades, several ML models have been suggested for this purpose. However, the efficiency of these methods often degrades when applied to high data volumes or dimensions [7,56].

Since they have the capability to handle complex data and extract features from large-scale datasets, DL models have resulted in significant improvements over traditional ML models, making them more appropriate for anomaly detection applications. DL-based models for spacecraft anomaly detection can be broadly classified into three categories. The first category comprises models that utilize CNNs. CNNs are a family of neural networks used to process multi-variate time series data in spacecraft. Processing both spatial and temporal features in the data, CNNs are able to detect the different kind of anomalous patterns and unexpected events in spacecraft systems with high precision. The second group of methods involves RNN models. The RNNs are designed to process sequential data, allowing for the modeling of temporal features within the sequences. The third category encompasses transformers. Transformers have proven to be the best models to process long-term sequences. They have succeeded in effectively decreasing the requirement of computation resources. CNNs and RNNs models perform very well and even better than transformers for short-length series. To leverage the advantages of different thresholding, ML, and DL approaches, some hybrid models have been suggested.

So far, this study has exposed the state-of-the-art models as well as emerging trends in anomaly detection application in spacecraft telemetry data. To further boost this area of knowledge, it is necessary to identify and discuss the key challenges in this field. The main challenges affecting the anomaly detection of spacecraft telemetry data are presented below:

• Detect complex types of anomalies: Spacecraft data missions represent uneven degrees of difficulty when it comes to anomaly detection. It is typically hard to predict complex types of anomalies such as sequential and contextual anomalies with limited training data and a wide variety of patterns. This can explain the lower performance for some missions like the MSL [20,22,36] and OPS-SAT [26,40].
• Small number of public datasets available: For several reasons (in particular confidential ones), most of the studies chose to do not publish the explored dataset or any significant details about them. This prevents being able to compare and assess the different anomaly identification methodologies’ performances in a fair way and limits our ability to identify the most optimal method.
• Limited availability of labeled anomaly data: The use of semi-supervised or unsupervised methods could be a good solution to solve the problem of lack of ground truth in anomaly detection datasets. However, they still give a lower performance compared to supervised algorithms [40].
• Real time processing: One major challenge in detecting anomalies in spacecraft telemetry data is monitoring channels in near real time to avoid or mitigate a system failure. Early anomaly detection is valuable, but reliable solutions for real-time anomaly detection do not yet exist in the literature.
• On-board processing limitations: Ideally, anomalies should be detected and processed onboard the spacecraft, but this means the model needs to run on the spacecraft computer system, and this brings issues with regard to computational complexity and possible issues with retraining.
• Downlinking issue: Mission details, including telemetry data, are periodically transmitted by spacecraft to ground operators on Earth for analysis. Downlinking is the process of transmitting data from the spacecraft to Earth [69]. The link budget, or the amount of data that can be downlinked, is a constraint on every space mission. Even though the link budget is usually developed for every satellite mission during the planning stage, there are several factors that can affect the link and cause signal attenuation [70]. Therefore, it is not possible to downlink all data, and they may be lost, which consequently affects the anomaly detection performance if such anomaly detection is not done onboard.

A promising avenue for future research would be the application of deep generative models such as GANs to generate realistic anomalies from normal data, in particular to augment ground-truth datasets to include difficult and unexpected anomalies. This could offer a better solution for the limited labeled anomaly issue in datasets. There are a few other works which use self-supervised learning [71], contrastive learning [72], and transfer learning [22], which were recently used to detect anomaly telemetry data. These methods can capture the complex correlations between the various dimensions and generate accurate anomaly detection results. Exploring such algorithms may be an interesting direction for future research.

Since it would make it possible to react to failures and hazards faster, real-time anomaly detection is of extremely high importance to the community. An efficient suggestion for real-time anomaly detection is the exploration of real-world streaming algorithms such as Hierarchical Temporal Memory (HTM) [68]. Another potential research direction for spacecraft anomaly detection is onboard anomaly detection, where anomalies are identified in spacecraft systems. Given the limited resources available in spacecraft devices, studies involving onboard anomaly detection are limited in the existing literature [73,74,75]. Due to their parallelization capacity and limited parameter requirements, DL models, especially CNNs [76], have demonstrated computational efficiency. This benefit makes CNNs a promising direction in anomaly detection for spacecraft onboard applications.

Despite their prominent performance in the spacecraft anomaly detection field, ML/DL models face challenges in explainability [10]. Providing a clear explainability analysis is required to understand why a specific data instance has been identified as anomaly, how ML/DL algorithms make decisions, and what factors contributed to that detection [77]. In this context, explainable ML/DL is a very valuable tool to enhance spacecraft safety and avoid system failures, and it presents an important future research direction.

Training ML/DL models on real spacecraft telemetry data is often a time-consuming and resource-intensive task. Transfer learning, which takes advantage of the knowledge learned from another satellite model, may help to bypass this problem [22]. This method overcomes the need to retrain a model for every new setup. It makes the detection of anomalies with little or even no labeled data or limited computational resources feasible [78,79].

Due to the large volume and complexity of the data generated by spacecraft, big data processing engines such as Apache Spark [80,81] and Apache Flink [82] are key frameworks for evaluating and real-time monitoring spacecraft data. Spacecraft datasets can be efficiently stored, processed, and analyzed thanks to these engines, which would present an important research direction in the future, especially for real-time processing applications.

Finally, digital twins of spacecraft are useful tools for telemetry anomaly detection [83]. Digital twins allow for the generation of simulated data and thereby make it possible to test anomaly detection techniques on the ground before applying them to the spacecraft once it is launched. Profiting from such digital twins [84] in detecting anomalous events for spacecraft telemetry constitutes an interesting research trend.

6. Conclusions

In this work, we have reviewed the state-of-the-art methods in spacecraft anomaly detection and highlighted the benefits and drawbacks of each method. We also compared their performance on benchmark datasets. In summary, the choice of spacecraft anomaly detection method depends highly on the nature of the spacecraft telemetry data (spatial, temporal, complex, small/large size, sequence length, etc.), the type of anomalies, technical constraints, and computational resources.

We can argue that AI models have shown superior performance over traditional thresholding methods.

Nevertheless, a larger number of publicly available labeled anomaly datasets is crucial to ensure the possibility of benchmarking models and the uptake of such techniques. In addition, real-time and onboard implementation of such complex models is an ongoing area of research.