Convolutional Autoencoders for Data Compression and Anomaly Detection in Small Satellite Technologies

Abstract

Small satellite technologies have enhanced the potential and feasibility of geodesic missions through the simplification of design and decreased costs allowing for more frequent launches. On-satellite data acquisition systems can benefit from the implementation of machine learning (ML) for better performance and greater efficiency on tasks such as image processing or feature extraction. This work presents convolutional autoencoders for implementation on the payload of small satellites, designed to achieve the dual functionality of data compression for more efficient off-satellite transmission and at-source anomaly detection to inform satellite data-taking. This capability is demonstrated for the use case of disaster monitoring using aerial image datasets of the African continent, offering avenues for both the implementation of novel ML-based approaches in small satellite applications and the expansion of space technology and artificial intelligence in Africa.

1. Introduction

Small satellites are artificial satellites characterized by weights less than 500 kg and placed in orbit around planets and natural satellites to collect information or for communication [1]. They are used to make Earth observations (EOs) across academia and industry, with relevance in a variety of fields such as basic research and technology demonstration [2], environmental monitoring [3], and disaster management [4]. In recent years, the development of low-Earth-orbit satellite constellations and the democratization of space have boosted the launch of small satellites. This trend is expected to increase in the future [5,6].

Due to their remote location and inaccessibility, small satellites face significant constraints in terms of on-board storage, processing power, and downlink bandwidth. This has led to advances in handling large EO data, such as the development of new technologies to store and downlink data to the ground, as well as advanced on-board satellite processing operations [7,8]. Artificial intelligence (AI) systems have proven to be excellent in extracting information from a high volume of data, such as images and videos, using computer vision and other machine learning techniques [9]. The edge deployment of neural algorithms (i.e., at the source of the data) reduces the need to transmit large volumes of raw data, allowing for more efficient bandwidth usage and faster downstream processing. It also enables the prompt extraction of actionable insights from data streams, supporting faster and more autonomous system responses.

Accordingly, there is a growing interest in exploring AI methods on board to improve small satellite (SmallSat) technologies [10,11]. $\mathsf{\Phi }$-Sat-1 demonstrated the significant role of AI in data analysis and decision-making processes using object detection, which could contribute to environmental monitoring [12]. Additionally, lossy image compression for SmallSats is a critical element in designing an off-satellite data transmission system, and, thus, it has been broadly explored in the literature [13,14,15,16].

A less explored avenue of image processing for SmallSats is the use of anomaly detection, which can allow for the recognition of anomalous images that could be useful for data analysis and satellite operations. Anomaly detection is defined as the recognition of unusual elements in a dataset based on a learned description of a data-driven background-only model, commonly provided by machine learning tools. For example, anomaly detection can help in the early detection and containment of wildfires, flooding, and other natural disasters [17]. Anomaly detection can also be used for the rapid detection of instrumental issues, namely, pixel failures in the camera or other disturbances [18]. Such methods have been studied for satellite telemetry [19,20,21]. However, the area of anomaly detection in SmallSat image analysis remains relatively unexplored.

This work studies the dual use of autoencoders for data compression and anomaly detection via on-satellite deployment for SmallSat technologies. An autoencoder (AE) is an example of an ML architecture that can offer both functionalities. Autoencoders are trained to reconstruct inputs after dimensional reduction through a bottleneck latent space, therein providing a lower-dimensional basis for the input that can be used to generate authentic reproductions of the original input after sampling from the compressed representation. The use of AEs for data compression has been explored in recent years in the context of SmallSat missions [22]. The specific model used here is a convolutional AE (CAE); convolutional layers aid the AE in learning from two-dimensional images. These results expand on those in the previous literature by providing a proof of concept that the ML models commonly used for intelligent image compression are also able to provide anomaly detection capabilities without the need for further optimization or additional processing resources.

Small satellites are cost-effective and can be developed in relatively short timescales compared to other scientific instruments. In this way, they offer a testbed for advanced technology by providing new opportunities for payload designs [23], enabling emerging nations such as Africa to develop and launch space programs. These technologies can benefit Africa in terms of socioeconomic and environmental development, for example, through their application in agricultural surveillance, real-time disaster monitoring, and sustainable development. Therefore, the use case of anomaly detection considered here is the identification of Saharan dust storms and camera defects in aerial satellite images of Dakar, Senegal, taken during the Copernicus Sentinel-2 mission (Figure 1).

The ability to compress images to lower-dimensional representations on a satellite could allow for reduced data transmission to Earth, followed by decompression, enabling more power-efficient data downlinks in the highly constrained satellite computing system [25]. Further, real-time dust storm recognition at the satellite level could allow for dynamic changes in satellite image-taking (i.e., the disruption of standard image-taking patterns to focus on anomalous regions), with significant impacts of improved weather and climate monitoring in the region [26].

2. Datasets and Image Processing

The Sentinel-2 mission is part of the European Union’s Copernicus program consisting of a wide-swath, high-resolution, multi-spectral imaging payload [27]. The mission consists of twin satellites phased at ${180}^{\circ }$, with a high revisit frequency of 5 days at the equator. The payload is a multi-spectral instrument (MSI) that samples 13 spectral bands at different spatial resolutions of 10 m, 20 m, and 60 m. The orbital width is 290 km.

The Sentinel-2 datasets are open-source and easily accessible through the Copernicus Data Space Ecosystem platform. This platform provides a large amount of georeferenced imagery to researchers, developers, and policymakers worldwide [24]. The dataset includes Level-1C products (orthorectified top-of-atmosphere reflectances) and Level-2A products (bottom-of-atmosphere reflectances). The Sentinel-2 MSI acquires measurements with a radiometric resolution of 12 bits. The same measurements are converted to reflectance and stored as 16-bit integers. The choice of Sentinel-2 datasets for this study is not only based on their free, open-access policy and large availability of satellite images but also because the data acquired by the MSI on board the satellite is similar to those acquired by the multi-spectral camera used in SmallSats.

The dataset used in this study consists of 70 Sentinel-2 aerial images of the region of Dakar, Senegal, each covering an area of approximately 12,115 ${\mathrm{k}\mathrm{m}}^2$. Dakar is chosen as the region of interest in Africa because it shows a balance between ocean and land for realistic image feature presence, non-negligible rates of image anomalies such as Saharan dust storm events, and favorable weather conditions for satellite observations (e.g., relatively low cloud coverage). A set of images taken from 2020 to 2024 with a 60 m resolution and <20% cloud cover at the L2A processing level is chosen to develop the CAE model, comprising 70 images in total. Of these, 63 are used for the training set, and the remaining 7 compose the validation and test sets. The anomalous dataset consists of two known dust storm events on 1–5 June 2022 [28]. These two anomalous Sentinel-2 acquisitions are not included in the 70 images in the non-anomalous training/testing dataset and are reserved for a dedicated test set used to examine the anomaly detection capabilities of the ML approach.

Before the training dataset was passed to the network, it was preprocessed as follows: (1) bands were selected using free, open-source QGIS (Quantum Geographic Information System) v3.40 software; (2) the dataset was split into training, validation, and test sets; (3) patches (tiles) were extracted with overlap; (4) the patches were saved with their respective metadata; and (5) normalization and datatype conversion were performed. The details of each step are provided below.

Band selection was performed using QGIS. Bands 2 (blue), 3 (green), and 4 (red) were selected to isolate them in RGB from the rest, and a composite image was created for visualization, as shown in Figure 2. Ultimately, only Band 4 (red) was used to train the CAE to allow it to properly learn both water and land features, which are predominantly present in the blue and green bands, respectively.

Images were split into training (90%) and validation/test (10%) sets based on their date of acquisition. Each image with a 60 m resolution had a size of 1830 × 1830 pixels. Single-band images were then divided into patches of size $256\times 256$ pixels, with an overlap of 135 pixels between adjacent patches, achieving 99.89% image coverage. This was achieved by sliding a window across the image with a stride equal to the patch size minus the overlap, ensuring that neighboring patches shared common regions. The patch size was chosen because it provides a balance between power consumption and processing time and accuracy [29]. The overlap was selected to balance comprehensive feature capture with GPU memory constraints, critical for small satellite applications, based on a systematic evaluation balancing image coverage and computational efficiency. At each position, a patch was extracted and passed to a normalization routine to ensure that the pixel values fell within the range of [0, 1]. Patch extraction was performed after dataset splitting, with identical processing applied to all images in order to prevent data leakage and ensure consistent patch views across sets.

In summary, before preprocessing, the training set has 63 images, the validation has 4 images, the test set has 3 images, and the anomaly test set has 2 images. After preprocessing, the training dataset consists of 12,348 patches, the validation dataset consists of 784 patches, the test dataset consists of 588 patches, and the anomaly test dataset consists of 98 patches, all in Band 4 representation. As all images cover the same aerial area over Dakar, patch extraction is applied after the split into training, test, and validation sets, and the patch extraction process is exactly the same for all images. It is ensured that the same image views are passed to the model in both training and evaluation.

3. Convolutional Autoencoder Model

The architecture of the CAE model is based on the convolutional autoencoder (CAE) proposed in Ref. [29]. It consists of an encoder with convolutional layers that compresses the image to a lower-dimensional latent space and a decoder with deconvolutional layers that reconstructs the image from the latent space to the original input dimensionality. The compression ratio (CR) of the model is shown in Equation (1) and defined as the ratio of the input image size in terms of pixels and the bottleneck size in terms of units:

$$\mathrm{CR}={\displaystyle \frac{\mathrm{patch}\mathrm{height}\times \mathrm{patch}\mathrm{width}\times \mathrm{image}\mathrm{depth}}{\mathrm{latent}\mathrm{space}\mathrm{dimension}}}$$

(1)

The encoder accepts images of input $256\times 256\times 1$ and contains the following: (1) 10 convolutional layers with an increasing number of filters (16, 32, 64, and 128); (2) the leaky ReLU activation function and $\alpha =0.01$ for the better handling of low-intensity features, with each layer having a kernel size of $3\times 3$ and an alternating stride of 1 and 2 for the downsampling convolutional layer; (3) flattening of the output of the last convolutional layer to convert the 3D tensor into a 1D vector; and (4) a bottleneck layer outputting a 1D vector of 8192-dimensional latent space. Therefore, the CAE used here has a latent space with a CR of 8.

The decoder, which is a symmetrical representation of the encoder, performs the reverse process by returning the reconstructed image from the latent space as the final output. It is composed of the following: (1) a latent vector (8192 units) that is reshaped into a 3D tensor of size $8\times 8\times 128$; (2) 10 transposed convolutional layers with leaky ReLU activation using $\alpha =0.01$ to upsample spatial dimensions, returning a feature map of size $256\times 256\times 16$; and (3) a final layer with a linear activation that generates the output with the original size of $256\times 256\times 1$. The loss function employed to train the model uses the structural similarity index measure (SSIM) [30,31], which is designed to align with the human visual perception of images, including image qualities like luminance, contrast, and structural similarities. For this reason, the SSIM is chosen over the mean squared error loss, which focuses only on pixel-wise differences and may produce blurry reconstructions and, as a result, can compromise an image’s structural fidelity. The SSIM loss is thus defined in Equation (2), where $y_{true}$ and $y_{pred}$ are the true and reconstructed images, respectively. $y_{truei}$ and $y_{predi}$ represent the image contents in the ith local window, and M represents the number of local windows in the patch.

$$\mathrm{SSIM}\mathrm{Loss}(y_{true},y_{pred})=1-{\displaystyle \frac{1}{M}}\sum _{i=1}^M\mathrm{SSIM}(y_{truei},y_{predi}).$$

(2)

Figure 3 provides a diagram of the CAE used in this study. It is trained over the preprocessed input image set described in Section 2 for 250 epochs with a batch size of 8, using the Adam optimizer [32] with a learning rate of ${10}^{-4}$. As this work seeks to emulate the state-of-the-art CAE described in Ref. [29] and demonstrate the added anomaly detection capability, further optimization of hyperparameters is not performed in this scope.

4. Results

After training, the CAE is able to perform an intelligent compression of the images taken by the small satellite to a dimensionality with a compression ratio of 8. The CAE output SSIM score for each event can be used to describe the fidelity of the reconstructed output, with high SSIM scores indicating higher-quality reconstructions. Furthermore, events with an exceptionally poor reconstruction quality (or lower SSIM scores) are more likely to be anomalous with respect to the training set. In this way, the SSIM can also be used as a classifier for anomalous images. The performances of the CAE latent space and SSIM score in both compressing data and identifying anomalies are presented below.

4.1. Image Reconstruction and Compression

With a compression ratio in the latent space of 8, the proposed transmission of image data off-satellite using the CAE latent space encoding results in a reduction in the off-satellite data rate of 12.5%. Such data rate reductions can allow for lower-power data transmission systems, which are especially valuable in the power-limited environment of satellites.

The images reconstructed by the CAE in the test set have a mean SSIM score of 0.75, indicating high-fidelity decoding by the model and validating the potential of transmitting images to Earth in a compressed format without the loss of key information. Figure 4 shows some example reconstructions of the satellite image patches processed by the CAE decoder. Comparatively, the original result with this model structure in Ref. [29] provides a mean test set SSIM score of 0.977. However, this is obtained by optimizing a model strictly for reconstruction capabilities, whereas high-quality reconstruction and anomaly detection are inherently competing tasks, thus limiting the optimal performance achievable for either one. The further results below provide a benchmark for the AD capability, obtained from a state-of-the-art reconstruction model, with potential for further targeted optimizations for the single-model deployment of both functionalities (as discussed in Section 4.3).

4.2. Anomaly Detection

In order to test the capability of the CAE to provide model-independent recognition of anomalous satellite images, a variety of both real and synthetic anomalous images are used in the test set. The first anomalies used are the real satellite images collected during known periods of Sahara dust storms, resulting in occluded images with higher-than-normal mean pixel values. Two other anomalies are created synthetically to mimic the impact of camera defects, namely, a “dead pixel” test set, where 5% of the pixels in each patch have values set to 0, and a “hot pixel” test set, where 5% of the pixels per patch are set to 1. Figure 5 provides examples of the three anomaly image classes compared to a normal background image, showing both the true image and the output image reconstructed by the CAE.

Separate test sets are constructed comprising pure samples of each of these synthetic anomalies in order to assess the ability of the CAE SSIM score to isolate them from the background. Figure 6 shows the resulting SSIM score distribution for the training, test, and three anomalous image sets, as well as the receiver operating characteristic (ROC) considering the SSIM as a classifier. The anomaly test sets all share a lower mean SSIM score than the normal data, as represented by the training and test sets, with the hot pixel anomaly set appearing to have the lowest reconstruction quality and therefore the highest likelihood of anomalous origin. Each of the anomaly test sets is also identifiable via thresholding on the SSIM score, with ROC areas under the curves (AUCs) of 0.76 for the dust storm images, 0.81 for the dead pixel set, and 0.99 for the hot pixel set.

4.3. Discussion

Designing a data acquisition system for SmallSat that includes a CAE on the satellite could enable the recognition of anomalous images in real time and at the source of the data. This functionality could enable the rapid monitoring of procedural changes, such as the disruption of a standard satellite scanning process to remain on a particular aerial region or to take more detailed images. In the context of the use case of Sahara dust storm identification, this capability can enhance real-time anomaly monitoring, leading to substantial potential impacts on disaster response and research.

As discussed above, this work extends the CAE developed for small satellite image compression in Ref. [29] and demonstrates that a single model can achieve both compression for off-satellite transmission and at-source anomaly detection, capable of being used to identify natural disasters or camera defects. Similar studies with deep ML models for image compression have been performed in small satellite environments [13,14,15,16], but none of them have explored the anomaly detection potential of these models.

This work aims to start from an existing CAE demonstrated to be close to the state of the art in previous works. However, further optimization of the CAE model could achieve gains in the image reconstruction quality, anomaly detection, or both. A study is conducted with this CAE to scan the model latent space size, considering alternate CR values of 4, 16, and 32. Compared to a CR of 8, which yields a mean SSIM score on the test set of 0.75, the higher CR values considered degrade image reconstruction quality significantly, with mean SSIM test set values of 0.66 and 0.51 for 16 and 32, respectively. This indicates that significantly smaller latent spaces prevent the model from saving enough salient information to perform high-quality decompression, limiting the utility of the model for off-satellite data compression. The model with a lower CR of 4 has $\mathcal{O}$(1)% level improvements in both the average image SSIM score and anomaly AUC. However, a larger latent space corresponds to reduced data compression and a larger volume to be transmitted. Further optimization of the CR would require the consideration of the satellite payload constraints and data acquisition needs.

While the single-CAE approach is promising in its performance, as demonstrated here, and appealing in its simplicity for data acquisition designs, it suffers from some limitations: the lossy nature of CAE-based compression means that the final dataset will have some imperfections with respect to the truth images. Careful estimation of this effect could be applied in downstream analysis as a systematic uncertainty, but this may not be acceptable for certain satellite applications. Additionally, the inherent variability of Earth observation data could result in performance fluctuations or drift from a model trained on a fixed dataset, requiring careful monitoring of the algorithm and re-training or updating over time as needed. In light of these examples and other similar challenges, the adaptation of this concept for a specific data-taking scenario is key for deployment.

This work provides a small-scale and simple proof of concept that modern AI-based image compression algorithms in use for SmallSat readout can also be used for anomaly detection in acquired images. However, considerable future work could enable better performance on both tasks and bring this capability closer to deployment in real SmallSat payload compute hardware. This model is shown to recognize the common classes of anomalies that may appear in SmallSat images, namely, one natural disaster class and two camera errors. The next step will be to explore the extrapolation of this model to recognize (a) anomalies of other kinds (e.g., flooding and volcanoes) and (b) different geographical regions beyond Dakar. Additional studies of the choice of architecture could improve the overall image reconstruction quality, for example, through the use of adversaries such as generative adversarial networks to fine-tune the CAE latent space and enhance output image fidelity. Finally, studies of the possible hardware deployment of this model with more realistic spectral instrumentation are needed, with potential exploration for deployment on traditional CPUs, as well as more efficient platforms such as field-programmable gate arrays (FPGAs).

5. Conclusions

The application of machine learning for small satellite data acquisition is presented for the use case of disaster monitoring in Africa. Open-source datasets containing images taken over Dakar, Senegal, on the Sentinel-2 mission are used to develop a convolutional autoencoder with pixel-based input modeling. The resulting CAE loss upon test set evaluation is able to provide good reconstruction quality of the images, indicating the utility of on-board ML models to perform data compression at the edge to transmit lower data rates to Earth. Additionally, the autoencoder output can identify anomalous events, such as those arising from Sahara dust storm events over Dakar or synthetic pixel issues. Together, these capabilities motivate the use of ML-based methods in the design of on-satellite data acquisition systems for small satellites. The relatively low cost and accessibility of SmallSat technologies make these applications especially relevant for enhancing science and space programs in Africa while providing early career training opportunities in critical technologies.