Editorial to Special Issue “Remote Sensing Image Denoising, Restoration and Reconstruction”

1. Overview of the Issue: Remote Sensing Image Denoising, Restoration and Reconstruction

The motivations behind this Special Issue, announced in 18 August 2020, were the following. Despite great advances in technology, during image acquisition processes, remote sensing images are corrupted by different degradations, such as noise, geometric distortions, changes in illumination, blur of various types (motion, atmospheric turbulence, out-of-focus), etc. To eliminate these degradations, one has to apply data pre- and/or post-processing. Image restoration or reconstruction (IR) is an inverse problem that aims at estimating original images from the observed distorted ones. IR can be applied on a sensor data at the pre-processing stage, to improve image quality and to support further stages of data analysis, object detection and classification, or at the post-processing stage, to reduce distortions caused by lossy coding of images, such as blocking or ringing artifacts.

Thus, there is a wide diversity of questions that one could consider within the scope of this Special Issue, including novel model-based, machine learning methods, hybrid methods of image restoration, image denoising, deblurring (both blind and non-blind), image super-resolution, and many others.

The declared topics of interest included, but were not limited to, the following:

• Image denoising;
• Image deblurring (blind and non-blind);
• Image super-resolution;
• Image dehazing and de-raining;
• Image compression artifacts reduction;
• The effect of image restoration on clustering, classification and target detection;
• Sparse representation and low-rank approximation for image restoration in remote sensing;
• Deep learning models for image restoration, with emphasis on robustness to adversarial attacks and data variation;
• Multimodal image restoration and joint restoration and fusion of multi-sensor data.

In total, we have received 29 submissions to this Special Issue, from which 16 were rejected. The accepted publications can be conditionally divided into the following 3 groups dealing with: (a) data pre-processing and removal of specific types of noise; (b) image super-resolution and restoration; and (c) applications of the processed images.

The first group contains five papers. One of them, “Hyperspectral Image Denoising via Adversarial Learning” by J. Zhang, Z. Cai, F. Chen, and D. Zeng, considers a novel method of hyperspectral (HS) image denoising [1]. The peculiarity of the paper is that the authors consider deep neural network-based models, with special attention being focused upon generative adversarial networks, and taking into account the fact that HS images possess abundant spectral information. An end-to-end HS image denoising model via adversarial learning is proposed. Joint loss functions are employed, including reconstruction loss, structural loss, and adversarial loss. Keeping in mind the lack of available training data for the HS image denoising, an additional benchmark dataset denoted as the Shandong Feicheng Denoising (SFD) dataset is collected. Five types of mixed noise are evaluated. Different amounts of training data are studied.

The paper by Kong et al. [2] deals with the mixed noise removal in hyperspectral images (HSIs). The problem of preservation of fine details in images is considered. To solve this problem, the authors propose a novel total variation (TV) technique called spatial domain spectral residual total variation (SSRTV). The first stage of SSRTV is to calculate the difference between the pixel values of adjacent bands. After this, a two-dimensional TV for the residual image is obtained. The experimental results are presented and show that the SSRTV regularization term changes the structures of noises in original HSIs. In turn, this leads to more efficient removal of mixed noises. The validity of the proposed method is confirmed by experimental results using both simulated and real data. The proposed method is compared to its counterparts. The authors demonstrates the benefits of the proposed technique compared to TV-regularized low-rank matrix/tensor decomposition methods. Both quantitative metrics and visual inspection are used in analysis.

The paper by Wu et al. [3] concerns image destriping. Stripe noise appears in images quite often and it has negative impact on the image quality and the results of their classification. Because of this, destriping (if stripes are present) is a strongly desired operation for the image processing chain. Destriping efficiency depends on the adequacy of a stripe model. The authors propose a novel model based on TV regularization, global low rank, and directional sparsity constraints. Although TV regularization provides detail preservation, the global low rank and directional sparsity are exploited for stripe noise constraining. The optimal solution is produced using the designed alternating minimization scheme. Simulation and real life experimental results are presented. Considerable robustness of the proposed model is demonstrated. A comparison of the proposed solution to several existing destriping models is performed using objective criteria and real-life examples (MODIS images).

The problem of image destriping is also addressed in the paper by Li et al. [4]. The authors consider the reasons stripe noise originates in multispectral images and the outcomes it results in. They advocate applying convolutional neural network (CNN)-based models to solve the destriping task. In detail, the authors propose a multi-scaled column-spatial correction network (CSCNet) where the local structural characteristic of the noise and the image global contextual information are jointly exploited at multiple feature scales. Two modules, namely, the column-based correction module and spatial-based correction module have been designed. In addition, a feature fusion module has been created. It enables the obtaining of discriminative features from the different modules and scales. The authors compared the proposed model to the conventional and deep learning counterparts. Both simulated and real images are used for the purpose. It is shown that the designed CSCNet performs very well according to several criteria.

Qin et al. [5] consider satellite hyperspectral remote sensing images for which the presence of various types of noise can severely decrease the application value. Supervised deep learning has been earlier employed for such image denoising, but the corresponding methods commonly use many clean/noisy training pairs that are difficult to obtain from real life hyperspectral imagery. As an alternative, the authors propose a self-supervised learning-based algorithm called 3S-HSID. Its advantage is that the algorithm is able to simultaneously carry out robust denoising of a single satellite HS image in all bands. Bernoulli sampling of data is exploited. The training model is used for predicting different Bernoulli sampling results. A dropout strategy is employed to prevent overfitting. Simulation results are presented, demonstrating high efficiency of denoising using 3S-HSID compared to several existing counterparts. One advantage is a good preservation of spectral characteristics that are important for HS image classification. The results of denoising for several types of real satellite HS data are presented as well.

Within the second group, there are four papers. The paper by Wang et.al. [6] analyses the problem of robust tensor completion to recover an unknown tensor from incomplete and noisy observations. They use tensor SVD and ADMM for efficient computation of an estimator. Convergence analysis of the proposed method is carried out. Statistical performance of the method is described, namely, upper and low bounds of the estimation error are derived. Extensive experiments have been performed to verify both efficiency and effectiveness of the proposed method. Hyperspectral and multispectral image databases, urban area imagery dataset, point cloud data, aerial video, thermal imaging, and SAR data have been used in experiments.

The paper by L. Zhang et al. [7] proposes a perceptually unpaired super-resolution (SR) method based on the introduced multistage aggregation network. A problem of single image super-resolution is in reconstruction of high-resolution (HR) image from a given low resolution (LR) one. Recently, deep-learning-based methods have been successfully used in SR, outperforming previous model-based methods due to a better capability to model LR-to-HR non-linear mapping.

However, since the previous SR methods are based on assumption that a degradation process to obtain LR image from HR is known, they fail to work properly in case of remote sensing images, when the downsampling operation is unknown. This paper introduces a multi-stage aggregation network used to gradually optimize the model with the degraded self-exemplars and unpaired references. Specifically, the first stage aims to achieve better pixel-wise PSNR, whereas the subsequent stages are adapted to get more realistic texture and details reconstruction.

The paper by Q. Zhang et al. [8] considers a problem of Airborne Radar image super-resolution based on total-variation approach. They propose Gohberg–Semencul (GS) representation based fast TV (GSFTV) method for this problem, which helps to reduce the computational complexity from O(N³) to O(N²) operations of additions/multiplications. Authors have carried out simulation on artificial and real data. The results show that the proposed method can also preserve the target contour. A hardware platform based on FPGA is built to verify the high efficiency of the proposed method.

The paper by Duan et al. [9] considers the problem of thick cloud and cloud shadows removal in remote sensing images. For this purpose, they have proposed the temporal smoothness and sparsity regularized tensor optimization (TSSTO) method, utilizing sparsity norm and unidirectional TV regularizers. They carried out a series of experiments to show the potential of the proposed method.

The third group includes four papers. The first one entitled “Reconstruction of Vegetation Index Time Series Based on Self-Weighting Function Fitting from Curve Features” by W. Zhu et al. [10] considers quite traditional application of remote sensing dealing with vegetation index (VI) estimation from multi-temporal data. Two factors, namely clouds and poor atmospheric conditions, are paid specific attention to, since they are able to lead to biased and noisy estimates for particular time instances. Keeping this in mind, in order to provide more accurate analysis of the vegetation dynamics, the authors propose to apply two procedures of temporal data processing. The first one presumes determining a fitting weight for each VI point employing the curve features of the VI time series, whilst the second one is based on implementing the weighted function fitting to reconstruct the VI time series. It has been shown that the second approach significantly outperforms the corresponding unweighted function fitting with RMSE reduction by 26.82–52.44%. It also outperforms the Savitzky–Golay filtering, thus providing an appropriate accuracy and high robustness in regional applications.

J. Cui et al. have prepared the paper “MODIS Land Surface Temperature Product Reconstruction Based on the SSA-BiLSTM Model” [11] that concerns a land surface temperature (LST) estimation used in studies of substance and energy exchanges between the land surface and the atmosphere, climate changes, and so on. One factor that might negatively affect a successful solution of the considered task are clouds and cloud shadows. The authors have proposed an LST reconstruction method that combines data decomposition with data prediction to obtain spatially and temporally continuous LST data. The proposed method combines the following two processes The first is rough LST reconstruction that exploits the trend features of the data that have been extracted using the Singular Spectrum Analysis model. The second presumes a refined LST reconstruction that employs the short-term features of the data learned by the model called Bidirectional Long Short-Term Memory. A comparative analysis of the propose and known methods is performed for RS data extracted and measured data using several quantitative criteria. The advantages of the designed approach are clearly demonstrated.

The paper “Mid-Infrared Compressive Hyperspectral Imaging” by S. Yang et al. [12] concerns other practical aspects, namely, how to address high dimensional challenges in hyperspectral imaging. The authors pay the main attention to the mid-infrared spectral range for which less studies have been performed. They describe a novel mid-infrared compressive HSI system with the aim to extend the application domain of mid-infrared digital micromirror device. This device is combined with an off-the-shelf infrared spectroradiometer. The goal of the paper consists of capturing the spatial modulated and compressed measurements at different spectral channels. Special side information is used to enhance the reconstruction quality of the infrared hyperspectral images. Experimental data proving the proposed approach efficiency are provided. Moreover, the authors state that they have developed the first mid-infrared compressive hyperspectral imaging approach able to provide a less expensive alternative to the conventional mid-infrared hyperspectral imaging systems.

The paper by K. Yang et al. [13] can be related to both the second and third groups of the papers in this Special Issue. The authors study NDVI and surface reflectance time series produced by Sentinel-2 imagery. In this application, the data are degraded due to the cloud contamination. Then, it becomes extremely desirable to reconstruct the Sentinel-2 NDVI and surface reflectance time series. The authors proposed a new reconstruction method that relies on the penalized least-square regression, which is based on discrete cosine transform. First, cloud-contaminated NDVI in NDVI time series are identified. Second, the NDVI and surface reflectance time series are reconstructed, at this stage the adjusted weights are used as constraints. Some improvements to the basic DCT-PLS method are proposed. First, the DCT formulas have been adapted to irregular interval time series. Second, the method has been implemented in the Google Earth Engine (GEE) platform that makes it convenient for customers. The method performance has been analyzed for several bands of multispectral data for Zhangjiakou and Hangzhou study areas. The method advantages are demonstrated—NDVI and surface reflectance have been reconstructed with low RMSE and high coefficient of determination. It is important that the GEE code is freely available online that makes it useful in landcover classification.

2. Conclusions

From the 13 papers published in this Special Issue, we observe that the remote sensing community is still actively involved in analysis of factors that degrade RS data quality and in the development of new advanced techniques to improve images in order to meet the end-user requirements. Total variation techniques are still widely used, further modified and often employed in combination with low-rank and/or sparsity regularization terms within more complex models. Deep learning approaches, typically based on CNNs, enjoy special attention and demonstrate potentials for boosting various applications. Self-supervised learning mechanisms alleviate the need for many clean/noisy training pairs and widen thereby practical applicability of these models. Similarly, adversarial learning principles demonstrate the potential for handling mixed and complex noise types. The tendency to analyze the efficiency of image pre-processing methods from a data classification viewpoint is observed. Based on these advances, RS data quality has improved, and this facilitates practical use of remote sensing. The editors hope that the research community of remote sensing will enjoy reading papers from this Special Issue.