SARFT-GAN: Semantic-Aware ARConv Fused Top-k Generative Adversarial Network for Remote Sensing Image Denoising

Highlights

What are the main findings?

• We propose ARConv Fused Top-k Attention: it fuses geometry-adaptive ARConv with sparse Top-k attention to couple fine-grained local modeling and long-range aggregation.
• We propose SARFT-GAN: it embeds ARConv Fused Top-k Attention into the generator and introduces a Semantic-Aware Discriminator to exploit semantic priors.

What is the implication of the main finding?

• The method improves perceptual realism and semantic consistency of denoised imagery, benefiting downstream tasks.

Abstract

Optical remote sensing images play a pivotal role in numerous applications, notably feature recognition and scene semantic segmentation. Nevertheless, their efficacy is frequently compromised by various noise types, which detrimentally impact practical usage. We have meticulously crafted a novel attention module amalgamating Adaptive Rectangular Convolution (ARConv) with Top-k Sparse Attention. This design dynamically modifies feature receptive fields, effectively mitigating superfluous interference and enhancing multi-scale feature extraction. Concurrently, we introduce a Semantic-Aware Discriminator, leveraging visual-language prior knowledge derived from the Contrastive Language–Image Pretraining (CLIP) model, steering the generator towards a more realistic texture reconstruction. This research introduces an innovative image denoising model termed the Semantic-Aware ARConv Fused Top-k Generative Adversarial Network (SARFT-GAN). Addressing shortcomings in traditional convolution operations, attention mechanisms, and discriminator design, our approach facilitates a synergistic optimization between noise suppression and feature preservation. Extensive experiments on RRSSRD, SECOND, a private Jilin-1 set, and real-world NWPU-RESISC45 images demonstrate consistent gains. Across three noise levels and four scenarios, SARFT-GAN attains state-of-the-art perceptual quality—achieving the best FID in all 12 settings and strong LPIPS—while remaining competitive on PSNR/SSIM.

1. Introduction

Remote sensing is a non-contact Earth observation technology that captures surface electromagnetic information via onboard sensors, providing essential data for environmental understanding and decision making [1]. With rapid advances in aerospace information technology, remote sensing imagery has been widely adopted in land resource surveys [2], ecological assessment [3], military reconnaissance [4], and disaster monitoring [5]. However, optical images are often degraded by random noise induced by sensor imperfections, atmospheric scattering, and compression. Such noise blurs edges, suppresses textures, and degrades the performance of downstream algorithms (e.g., recognition and classification) [6], thereby reducing the practical value of remote sensing data. While hardware progress can mitigate systematic sources of noise, software denoising remains vital for handling random noise under complex imaging conditions [7].

In optical remote sensing, sensor noise is commonly modeled as Poisson–Gaussian [8]: $Y=\alpha P+\mathcal{N}(0,{\sigma }^2),$ where $P\sim \mathrm{Poisson}\left(y_p\right)$ captures signal-dependent shot noise and $\mathcal{N}(0,{\sigma }^2)$ denotes additive readout noise. Because the variance of P scales with the intensity $y_p$ (i.e., $\mathrm{Var}\left(P\right)=y_p$), practitioners often apply a Variance-Stabilizing Transform (VST) [9] to approximate homoscedasticity and reduce the problem to Gaussian denoising; following this line, we train with synthetic Gaussian noise. Complementary directions include GAN-based noise estimation that learns parameterized noise from real images [10,11] and multi-observation self-supervision (e.g., Neighbor2Neighbor [12]) that models noise without paired data. Our network can be used as a denoising backbone once such data are prepared by these pipelines.

Denoising is inherently ill-posed: a single degraded observation may correspond to many plausible clean images [13]. The core challenge is to suppress noise while preserving fine structures under signal-dependent corruption. Deep learning has made substantial progress; CNN encoder–decoder architectures help separate noise from semantics, attention mechanisms reweight informative regions, and GANs can improve perceptual realism. Yet notable limitations persist in the remote sensing setting: (i) Rigid sampling in standard convolutions. Fixed square kernels (e.g., $3\times 3$) adapt poorly to irregular boundaries and multi-scale objects, hampering detail capture across diverse targets (vehicles vs. buildings). (ii) Indiscriminate global coupling in attention. Vanilla Transformer attention [14] assigns weights across all positions, which can propagate weak or spurious correlations in high-noise regimes and lead to texture distortion [15]. (iii) Semantics-agnostic discriminators. Common GAN discriminators [16,17] emphasize distribution matching but are insensitive to class- or region-specific semantics, risking physically implausible textures in fine-grained areas (e.g., vegetation, facades).

To address these issues, we propose SARFT-GAN, a generative–discriminative architecture tailored for optical remote sensing denoising. Our design couples geometry-adaptive sampling with sparse, noise-robust feature aggregation and injects semantic priors into the adversarial learning process. Extensive evaluations on four datasets demonstrate consistent improvements over strong baselines in both fidelity and perceptual quality.

Our contributions are threefold:

• We develop an ARConv Fused Top-k Attention module that combines geometry-adaptive sampling with sparsified correlation, overcoming the rigidity of fixed kernels and the noise sensitivity of dense attention.
• We introduce a Semantic-Aware Discriminator that leverages priors from pre-trained vision–language models to guide the generator toward physically plausible textures and fine-grained semantic consistency.
• We conduct comprehensive experiments across 3 noise levels × 4 land-cover scenarios (12 settings) and a real-image set, achieving SOTA LPIPS/FID and competitive PSNR/SSIM.

2. Related Work

The extensive topic of image denoising has been addressed by the academic community through a plethora of proposed algorithms, derived from continuous theoretical exploration and methodological innovation. A myriad of these denoising algorithms can be categorized into the following two primary divisions: traditional model-based methods and data-driven deep learning architectures.

2.1. Traditional Image-Denoising Methods

Research on traditional image denoising techniques is well-established and can be primarily divided into two categories: filter-based methods [18,19] and statistical learning-based methods [20,21,22]. These approaches distinguish noise from the signal through mathematical modeling and utilization of image prior knowledge, warranting a detailed examination of their fundamental principles.

2.1.1. Filter-Based Methods

Filtering methods mitigate noise through image smoothing and are categorized into spatial domain filtering and transform domain filtering.

Spatial domain methods employ either local or global pixel correlations to achieve denoising. Initial techniques, such as median and bilinear filtering, were dependent on local neighborhood information, but these often resulted in texture blurring. A significant advancement was the introduction of the Non-Local Means (NLM) algorithm by Buades et al. [18], which performs denoising by averaging globally analogous patches. This method notably enhances the preservation of intricate textures, even though it is computationally inefficient.

Transform domain methods, such as those employed for noise removal in images, operate by mapping the image to either the frequency or wavelet domain. A notable example of this approach is Block Matching and 3D Filtering (BM3D) [19], which integrates both spatial and transform domain strategies. This method functions by grouping similar patches, applying collaborative filtering, and subsequently reconstructing the image via an inverse transform. While BM3D is particularly effective at removing Gaussian noise, it has a tendency to generate artifacts in images that do not contain repetitive structures. Additionally, it requires manual parameter tuning.

2.1.2. Statistical Learning-Based Methods

These methodologies utilize data to ascertain noise distributions and image priors, thereby facilitating the construction of optimization models.

Sparse coding frameworks, exemplified by the K-SVD algorithm [20], utilize dictionary learning to represent image patches as sparse linear combinations and subsequently reconstruct clean images, leveraging sparsity constraints. Notably, this method retains textures effectively but is associated with high computational complexity.

Low-rank approximation techniques, notably Weighted Nuclear Norm Minimization (WNNM) [21], conceptualize image patches as low-rank matrices, achieving denoising via weighted singular value decomposition. While WNNM surpasses traditional methods in detail recovery, it may induce over-smoothing in intricate textures.

Bayesian approaches, including BayesShrink [22], employ adaptive thresholding based on Bayesian estimation to balance the trade-off between denoising and detail preservation. However, these methods are contingent upon accurate assumptions regarding noise distribution and demonstrate reduced robustness in the presence of non-Gaussian noise.

Despite significant achievements, traditional methods face the following bottlenecks. Parameter Sensitivity: Most algorithms require manual tuning of hyperparameters (e.g., patch size, noise variance), limiting their generalization capability. High Computational Cost: Methods like BM3D and K-SVD involve iterative optimization, making their time complexity unsuitable for real-time applications.

2.2. Deep Learning-Based Methods

Deep learning surpasses the limitations of traditional methods’ reliance on prior assumptions by autonomously discerning the mapping relationship between noise and image features via data-driven techniques. This has led to the emergence of two primary technical pathways: methods based on Convolutional Neural Networks (CNN) and those based on Generative Adversarial Networks (GAN).

2.2.1. CNN-Based

Convolutional Neural Network (CNN)-based methodologies model the denoising process directly via end-to-end training, thereby providing efficient inference and robust generalization capabilities. The following discusses some fundamental architectures along with their core principles. The Residual Learning Paradigm, as introduced by Zhang et al. with DnCNN [23], was the first to employ residual learning in the context of image denoising. This network facilitates the reconstruction of clean images by predicting noise residuals, which streamlines the learning objective and bolsters training stability. Multi-scale Feature Fusion: The FFDNet [24] method strategically downsamples noisy images into multi-channel sub-images, which are then used as network inputs. This approach allows for the joint processing of noise features across various scales, thereby enhancing adaptability to non-uniform noise. Attention Mechanism: The RIDNet [25] model employs a Channel Attention module, allowing for the dynamic adjustment of feature channel weights and thereby enhancing the restoration of high-frequency textures. U-Net Structure: CBDNet [26] utilizes an encoder-decoder framework with integrated skip connections. This facilitates the amalgamation of both shallow details and profound semantic features, thereby showcasing exemplary performance in the realm of real-world noise elimination.

CNN-based methods tend to produce overly smooth regions in images, compromising detail and texture information.

2.2.2. GAN-Based

Generative Adversarial Networks (GANs) improve the visual quality of denoising results by employing adversarial training between generators and discriminators.Traditional GAN Architecture: Approaches such as Dehaze-AGGAN [27] incorporate a generator that learns denoising mappings and a discriminator that differentiates between generated images and real clean images. The adversarial loss ensures distribution consistency in the generated results.Conditional GAN: ID-AGAN [28] incorporates noise-level estimation as a conditional input for blind denoising. Notably, its generator utilizes an asymmetric structure, thereby augmenting the decoupling of noise features.Wasserstein GAN: WGAN-GP [29] enhances training stability through the implementation of a gradient penalty, effectively mitigating issues of mode collapse. It has demonstrated exceptional performance in the field of medical image denoising.The Multi-scale Generator: CMFHGAN, as delineated in [30], proposes a generator operating on multiple scales, transitioning from coarse to fine. This unique design executes global denoising, subsequently followed by local texture enhancement, ensuring the effective preservation of details.Self-Attention Mechanism: The Hya-gan model [31] incorporates a hybrid attention module, combining spatial and channel attention, within its generator. This feature enhances the model’s ability to capture long-range dependencies.

Although these GAN-based studies have enhanced the capabilities of generators to produce more realistic textures, limited research has been conducted on optimizing the discriminator, a core component of GANs. The discriminator’s role is critical as it determines whether the distribution learned by the denoise network aligns with the quality standards of real-world images.

3. Method

3.1. Generator

3.1.1. Overview

As illustrated in Figure 1, the generator of SARFT-GAN comprises three components: the shallow feature extraction module, deep feature extraction module, and image reconstruction module.

(1)

Shallow Feature Extraction

A 3 × 3 convolutional layer ${\mathcal{H}}_{\mathrm{SF}}(·)$ extracts shallow features ${\mathbf{F}}_0\in {\mathbb{R}}^{H\times W\times C}$ from the noisy input ${\mathbf{I}}_{\mathrm{noisy}}\in {\mathbb{R}}^{H\times W\times C_{\mathrm{in}}}$:

$${\mathbf{F}}_0={\mathcal{H}}_{\mathrm{SF}}\left({\mathbf{I}}_{\mathrm{noisy}}\right)$$

(1)

This module establishes a mapping from image space to high-dimensional feature space while preserving low-frequency structural information.

(2)

Deep Feature Extraction

Constructs deep features ${\mathbf{F}}_{\mathrm{DF}}\in {\mathbb{R}}^{H\times W\times C}$ through K Residual Adaptive Transformer Blocks (RATB):

$${\mathbf{F}}_i={\mathcal{H}}_{\mathrm{RATB}}^i\left({\mathbf{F}}_{i-1}\right),i=1,2,\dots ,K$$

(2)

$${\mathbf{F}}_{\mathrm{DF}}={\mathcal{H}}_{\mathrm{CONV}}\left({\mathbf{F}}_K\right)$$

(3)

Each RATB contains L Adaptive Attention Layers (AAL) and a 3 × 3 convolutional layer.

(3)

Image Reconstruction Module

Suppresses noise via residual connection:

$${\mathbf{I}}_{\mathrm{clean}}={\mathcal{H}}_{\mathrm{REC}}({\mathbf{F}}_0\oplus {\mathbf{F}}_{\mathrm{DF}})+{\mathbf{I}}_{\mathrm{noisy}}$$

(4)

where ⊕ denotes feature concatenation, and ${\mathcal{H}}_{\mathrm{REC}}$ consists of a 3 × 3 convolutional layer.

3.1.2. ARConv Fused Top-k Attention

In neural architecture design, the global self-attention mechanism from traditional Transformers serves as a foundational computational module. During implementation, a parameterization mechanism partitions the input tensor along the feature dimension into h parallel subspaces. Each subspace independently executes the attention operation, and the resulting h output features (dimension $L\times (d/h)$) are concatenated along the channel axis. A fully connected layer then integrates cross-subspace features. However, this native attention paradigm requires global correlation computation over $L^2$ query-key pairs, incurring significant redundancy. Moreover, inherent limitations of standard convolutions constrain modeling of remote sensing images containing objects with diverse scales.

As shown in Figure 2, we first encode channel context via 1 × 1 convolution and 3 × 3 depthwise convolution, while introducing an ARConv module on the $\mathbf{V}$ branch to construct enhanced feature representation ${\mathbf{V}}_{\mathrm{enhanced}}$:

$${\mathbf{V}}_{\mathrm{enhanced}}=\mathrm{ARConv}\left(\mathbf{V}\right)$$

(5)

This module employs an adaptive rectangular convolution structure. The cross-channel attention computation retains the original flow: the transposed attention matrix $\mathbf{M}$ is computed by reshaping $\mathbf{Q}$ and $\mathbf{K}$, followed by a dynamic top-k selection strategy preserving the top-k significant responses:

$$\mathrm{SparseAtt}(\mathbf{Q},\mathbf{K},{\mathbf{V}}_{\mathrm{enhanced}})=\mathrm{softmax}\left({\mathcal{T}}_k(\mathbf{Q}{\mathbf{K}}^T/\sqrt{d_k})\right){\mathbf{V}}_{\mathrm{enhanced}}$$

(6)

where the ${\mathcal{T}}_k(·)$ operator performs channel-wise adaptive filtering, retaining only the top-k maximum activations per row with normalization.

Here, the value matrix $\mathbf{V}$ is replaced by the enhanced version ${\mathbf{V}}_{\mathrm{enhanced}}$ with autoregressive context modeling. This design enables the model to capture long-range channel dependencies while maintaining computational efficiency.

Finally, multi-head attention fuses subspace features: outputs from multiple sparse attention heads are concatenated along the channel dimension, followed by a learnable linear projection matrix to generate the final representation.

3.1.3. ARConv

ARConv adaptively captures multi-scale features by dynamically learning geometric parameters and sampling strategies for convolution kernels [32]. Its structure is shown in Figure 3 and its core innovation lies in a three-stage mechanism:

(1)

Dynamic Kernel Parameter Learning

ARConv employs a dual-branch structure to predict height feature maps $\mathbf{h}\in {\mathbb{R}}^{H\times W\times 1}$ and width feature maps $\mathbf{w}\in {\mathbb{R}}^{H\times W\times 1}$. Specifically, given input feature map $\mathbf{X}\in {\mathbb{R}}^{H\times W\times C_{\mathrm{in}}}$: A shared feature extractor processes the input Dedicated Height Learner and Width Learner branches generate spatial parameters Sigmoid activation normalizes outputs, scaled by learnable modulation factors:

$$\mathbf{h}=a_h·\sigma \left(f_{{\theta }_1}\left(\mathbf{X}\right)\right)+b_h$$

(7)

$$\mathbf{w}=a_w·\sigma \left(f_{{\theta }_2}\left(\mathbf{X}\right)\right)+b_w$$

(8)

where $a_h,b_h,a_w,b_w$ are learnable coefficients constraining kernel dimensions to preset physical ranges (e.g., $h\in [b_h,a_h+b_h]$). This enables each spatial position to dynamically adjust its receptive field based on local geometric properties.

(2)

Dynamic Sparse Sampling Mechanism

Using the learned height/width parameters, ARConv determines effective sampling points via adaptive selection:

$$k_h=\varphi \left(⌊{\displaystyle \frac{\overline{h}}{n}}⌋\right),k_w=\varphi \left(⌊{\displaystyle \frac{\overline{w}}{m}}⌋\right)$$

(9)

where $\varphi \left(x\right)=x-\left[x\mathrm{is}\mathrm{even}\right]$ ensures an odd number of samples, and $n,m$ are scaling coefficients. Sampling coordinates are generated through grid interpolation:

$${\mathbf{r}}_{ij}=\left({\displaystyle \frac{(2i-k_h-1)h_0}{2k_h}},{\displaystyle \frac{(2j-k_w-1)w_0}{2k_w}}\right)$$

(10)

This produces a non-uniform sampling grid $\mathbf{R}\in {\mathbb{R}}^{k_h\times k_w\times 2}$, aligning kernel shape with target scales.

(3)

Affine Transformation for Spatial Adaptability

To enhance feature representation flexibility, ARConv incorporates the following learnable affine transformation:

$$\mathbf{Y}=\mathrm{Conv}(\mathbf{S};\mathbf{SK})\odot \mathbf{M}+\mathbf{B}$$

(11)

where: $\mathbf{S}\in {\mathbb{R}}^{\left(k_{h}H\right)\times \left(k_{w}W\right)\times C_{\mathrm{in}}}$ is the expanded sampled feature map $\mathbf{M},\mathbf{B}\in {\mathbb{R}}^{H\times W\times C_{\mathrm{out}}}$ are modulation matrix and bias terms predicted by subnetworks. It is worth noting that, rather than instantiating a full set of convolution weights at every spatial location, we use a shared kernel bank to produce a base response and apply a lightweight channel-wise affine modulation (scale and shift per channel) to adapt locally. This replaces a large 4-D weight tensor per location with just two channel vectors, drastically reducing the degrees of freedom and the risk of overfitting.

To provide an intuitive understanding of ARConv’s underlying mechanism, we visualize its geometry-adaptive behavior. As we can see in Figure 4, the convolution kernels adaptively modulate sampling pattern according to local geometry and scale by predicting position-wise height/width maps that deform a shared rectangular grid.

3.2. Semantic-Aware Discriminator

As shown in Figure 5, the Semantic-Aware Discriminator consists of a Semantic Feature Extractor and a Semantic-aware Fusion Block (SeFB) [33].

The Semantic Feature Extractor extracts multi-level semantic features from noise-free reference images $I_{gt}$ to provide semantic priors for the discriminator. We employ the ResNet-50 branch of the pre-trained vision-language model CLIP (CLIP-RN50), which was trained on large-scale image-text pairs and exhibits strong semantic representation capabilities. The RN50 architecture comprises four layers where feature resolution decreases through downsampling while semantic abstraction increases with layer depth. Experiments determined the third convolutional layer features as the optimal semantic source. This is mathematically expressed as follows:

$$S_h={\varphi }_{\mathrm{CLIP}\text{-}\mathrm{RN}50}^{\left(3\right)}\left(I_{gt}\right)$$

(12)

where ${\varphi }^{\left(3\right)}$ denotes the feature extraction function of CLIP-RN50’s third layer, and $S_h$ represents the semantic features.

The Semantic-aware Fusion Block dynamically integrates semantic features $S_h$ with image features $f_I$ (from either generated or real images) through semantic-conditioned feature modulation. The semantic features $S_h$ are first processed by a feature encoder composed of Group Normalization (GN), Layer Normalization (LN), and Self-Attention (SA) to generate a semantically enhanced query vector as follows:

$$Q=\mathrm{LN}\left(\mathrm{SA}\left(\mathrm{GN}\left(S_h\right)\right)\right)$$

(13)

This procedure captures global contextual dependencies within semantic features via the self-attention mechanism. Concurrently, denoised image features $f_{\mathrm{denoise}}$ and noise-free image features $f_{\mathrm{gt}}$ are projected through convolutional layers to generate corresponding key-value pairs $\{K_{\mathrm{denoise}},V_{\mathrm{denoise}}\}$ and $\{K_{\mathrm{gt}},V_{\mathrm{gt}}\}$.

Adaptive fusion of semantic and image features is achieved via the following cross-attention mechanism:

$$f_{(\ast )}^{\mathrm{att}}=\mathrm{Softmax}\left({\displaystyle \frac{Q·K_{(\ast )}^{\top }}{\sqrt{d_k}}}\right)V_{(\ast )}$$

(14)

where $d_k$ is a scaling factor and $(\ast )$ denotes feature processing for different input branches ($denoise$ or $gt$). It is worth noting that cross-attention explicitly aligns semantic priors with visual features via the query–key–value mechanism: semantic embeddings serve as queries while visual tokens act as keys/values, so the attention weights prioritize regions consistent with the semantics and suppress misleading correlations. For example, although water and vegetation may look similar at low resolution, a “water” query assigns higher weights to water surfaces and lower weights to vegetation, thereby mitigating semantic misalignment and achieving more consistent cross-modal alignment. To preserve original detail information, the attention features $f_{(\ast )}^{\mathrm{att}}$ are further concatenated channel-wise with convolution-enhanced features $\mathrm{Conv}\left(f_{(\ast )}\right)$, followed by nonlinear mapping, as follows:

$$f_{(\ast )}^s=\mathrm{Conv}\left(\mathrm{GELU}\left(\mathrm{LN}(f_{(\ast )}^{\mathrm{att}}\oplus \mathrm{Conv}\left(f_{(\ast )}\right))\right)\right)$$

(15)

Here ⊕ indicates channel-wise concatenation, and the GELU activation function introduces smooth nonlinearity to enhance model expressiveness.

4. Experiment

4.1. Settings

4.1.1. Datasets

To enhance model generalization, our data sources comprise two components: public datasets and a custom-built private dataset.

Public Data Section: We utilize original images from the RRSSRD [34] and SECOND [35] datasets as our noise-free images. The RRSSRD is a remote sensing super-resolution dataset established by Dong et al. [34], and includes data from Worldview-2 and Gaofen-2, representing Xiamen City and Jinan City in China. The spatial resolution is 0.6 m, and each image comprises three channels: red (r), green (g) and blue (b). The original image size is 480 × 480 pixels. The SECOND dataset is a change detection dataset covering urban areas such as Hangzhou, Chengdu, and Shanghai in China. We use remote sensing images from this dataset, which are obtained from multi-platform, multi-sensor acquisition, thus ensuring data diversity and practical application value. The original image size is 512 × 512 pixels, and the spatial resolution is at a sub-meter level. It includes categories like non-vegetated surfaces, trees, low vegetation, water, buildings, and playgrounds, which cover typical change scenarios of natural and human activities. We crop the images along the center into 480 × 480 pixel sizes.

Private Data Section: We utilize images captured by the Jilin-1 satellite as our data source. These images have a spatial resolution of 0.5m and are comprised of RGB three channels. The images were taken from Hengyang City, which is situated in central-southern Hunan Province, China, with geographical coordinates ranging from 110°32′16″E–113°16′32″E and 26°07′05″N–27°28′24″N. The city features a complex and diverse terrain with land covers including urban areas, water bodies, farmland, and vegetation. Initially, we crop a full remote sensing image into numerous 480 × 480 pixel size images, discarding the remaining parts. Subsequently, we eliminate images containing large areas of zero-pixel values, considering these as invalid images.

By integrating both public and private data, the original images were corrupted using Additive White Gaussian Noise (AWGN). This process resulted in a total of 12,321 noise-clean image pairs, each measuring 480 × 480 pixels in size.

4.1.2. Implementation Details

In our research, the RATB quantity, AAL quantity, window size, and channel quantity are typically set to 6, 6, 8, and 180, respectively. During the training process, we employ the Adam optimizer [36] for optimization, with a batch size of 4. All denoising methods discussed in this paper are trained from scratch using an auxiliary training set. All experiments are conducted using Pytorch 2.5.1 and Python 3.10 on a server equipped with a 15-core, model Xeon(R) Platinum 8474C CPU and an NVIDIA GEFORCE RTX 4090 GPU (NVIDIA, Santa Clara, CA, USA) with 24G video memory.

4.1.3. Evaluation Metrics

Addressing the dual challenges of detail restoration and visual fidelity in the denoising task, this paper proposes a multi-dimensional evaluation system. This system encompasses pixel-level accuracy, structural consistency, and perceptual authenticity. The specific definitions of indicators and their calculation methods are detailed below.

In the image reconstruction task, the Mean Squared Error (MSE) evaluates the reconstruction fidelity by quantifying the mean of the squared differences between the corresponding pixels of the generated image $X_{denoised}$ and the real high-resolution image $X_{GT}$. Its mathematical expression is expressed as follows:

$$\mathrm{MSE}={\displaystyle \frac{1}{HW}}\sum _{i=1}^H\sum _{j=1}^W{(X_{GT}(i,j)-X_{denoised}(i,j))}^2$$

(16)

where H and W are the height and width of the image respectively. $X_{GT}(i,j)$ and $X_{denoised}(i,j)$ represent the pixel values of the ground truth and the denoised image at the position $(i,j)$ respectively.

The Peak Signal-to-Noise Ratio (PSNR) is constructed based on the Mean Squared Error (MSE). It measures the global fidelity of the reconstructed image by quantifying the ratio of the maximum signal intensity to the noise energy between the denoised image $X_{denoised}$ and the real high-resolution image $X_{GT}$. The calculation formula is expressed as follows:

$$\mathrm{PSNR}=10·{log}_{10}\left({\displaystyle \frac{{\mathrm{MAX}}_I^2}{\mathrm{MSE}}}\right)\mathrm{dB}$$

(17)

where ${\mathrm{MAX}}_I$ represents the maximum range of image pixel values (for example, 255 for 8-bit images), and MSE is the Mean Squared Error.

The Structural Similarity Index (SSIM) evaluates the perceptual consistency between the denoised image $X_{denoised}$ and the real image $X_{GT}$ from three dimensions: luminance, contrast, and structural similarity by simulating the sensitivity of the human visual system to local structural information. The calculation formula is expressed as follows:

$$\mathrm{SSIM}(X_{GT},X_{denoised})={\displaystyle \frac{(2{\mu }_{X_{GT}}{\mu }_{X_{denoised}}+C_1)(2{\sigma }_{X_{GT}{X}_{denoised}}+C_2)}{({\mu }_{X_{GT}}^2+{\mu }_{X_{denoised}}^2+C_1)({\sigma }_{X_{GT}}^2+{\sigma }_{X_{denoised}}^2+C_2)}}$$

(18)

where ${\mu }_{X_{GT}}$, ${\mu }_{X_{denoised}}$ are the local means of the two images respectively, representing luminance information; ${\sigma }_{X_{GT}}^2$, ${\sigma }_{X_{denoised}}^2$ are the local variances, reflecting the contrast; ${\sigma }_{X_{GT}{X}_{denoised}}$ is the covariance, describing the structural correlation; $C_1$, $C_2$ are constant terms to avoid the denominator being zero, usually $C_1={\left(0.01L\right)}^2$, $C_2={\left(0.03L\right)}^2$, and L is the pixel dynamic range.

The Learned Perceptual Image Patch Similarity (LPIPS) measures the perceptual distance between the denoised image $X_{denoised}$ and the real image $X_{GT}$ in the feature space by extracting high-order semantic features through a pre-trained deep neural network. The calculation process can be formalized as:

$$\mathrm{LPIPS}(X_{GT},X_{denoised})=\sum _l{w}_l·{∥{\varphi }_l\left(X_{GT}\right)-{\varphi }_l\left(X_{denoised}\right)∥}_2^2$$

(19)

where ${\varphi }_l(·)$ represents the feature map of the l-th layer of the pre-trained network (such as VGG or AlexNet); $w_l$ is the learnable weight coefficient of each layer, calibrated through human visual preference data; ${\parallel ·\parallel }_2$ is the L2 norm, quantifying the Euclidean distance of feature differences.

The Fréchet Inception Distance (FID) evaluates the overall realism and diversity of the generative model by calculating the distribution difference between the generated image set ${\mathcal{X}}_{denoised}=\{X_{denoised}^{\left(1\right)},X_{denoised}^{\left(2\right)},\dots ,X_{denoised}^{\left(N\right)}\}$ and the real image set ${\mathcal{X}}_{GT}=\{X_{GT}^{\left(1\right)},X_{GT}^{\left(2\right)},\dots ,X_{GT}^{\left(N\right)}\}$ in the deep feature space. Its mathematical expression is expressed as follows:

$$\mathrm{FID}={∥{\mu }_{{\mathcal{X}}_{GT}}-{\mu }_{{\mathcal{X}}_{denoised}}∥}_2^2+\mathrm{Tr}\left({\Sigma }_{{\mathcal{X}}_{GT}}+{\Sigma }_{{\mathcal{X}}_{denoised}}-2{\left({\Sigma }_{{\mathcal{X}}_{GT}}{\Sigma }_{{\mathcal{X}}_{denoised}}\right)}^{1/2}\right)$$

(20)

where ${\mu }_{{\mathcal{X}}_{GT}}$, ${\mu }_{{\mathcal{X}}_{denoised}}\in {\mathbb{R}}^d$ are the feature mean vectors of the two sets ($d=2048$ corresponding to the ’pool3’ layer of Inception-v3); ${\Sigma }_{{\mathcal{X}}_{GT}}$, ${\Sigma }_{{\mathcal{X}}_{denoised}}\in {\mathbb{R}}^{d\times d}$ are the feature covariance matrices; $\mathrm{Tr}(·)$ is the matrix trace operation, which quantifies the degree of match of the distribution structure.

In remote sensing image denoising, PSNR and SSIM are widely used to assess pixel-level fidelity. However, they primarily capture average per-pixel differences and can be insensitive to perceptual quality—particularly in high-noise or structurally complex scenes. By contrast, perceptual metrics such as LPIPS and FID are more sensitive to structural realism and fine textures, which are crucial for human interpretation and downstream applications (e.g., classification, target detection). Methods that score highly on PSNR/SSIM may still yield overly smooth results lacking fine detail. To mitigate this limitation, we report LPIPS and FID alongside PSNR/SSIM to evaluate realism and structural consistency in feature space and provide a more balanced assessment.In this work, our goal is to strike a balance between pixel-level fidelity and perceptual quality. We believe that striking a balance between pixel-level fidelity and perceptual quality is particularly important for practical remote sensing applications.

4.2. Comparisons with State-of-the-Art Algorithms

We compare the proposed SARFT-GAN method with several previous methods, including one local similarity-based method (BM3D [19]), two residual learning and convolutional neural network-based methods (DnCNN [23], BUIFD [37]), three Transformer-based methods (IDTransformer [38], SwinIR [39], CFAT [40]), one deep unfolding network-based method (DUMRN [41]), and one dual-branch method (DRANet [42]). Implementation codes were downloaded from the official websites of each method’s authors or designated code hosting platforms to ensure code accuracy and completeness. In our experiments, all methods used default parameter settings and were retrained using our remote sensing dataset. Evaluations were conducted separately for four different scenarios: building, farmland, vegetation, and water.

In evaluating the performance of denoising, experiments were conducted at three distinct noise intensity levels. During the image corruption phase, three specific noise intensities were introduced: $\sigma$ = 15, $\sigma$ = 25, and $\sigma$ = 50. Here, $\sigma$ denotes the standard deviation of Gaussian noise.

4.2.1. Quantitative Comparison

The quality of denoised images is quantitatively evaluated using four metrics: PSNR, SSIM, LPIPS, and FID.

Table 1. Quantitative results of different methods in four scenarios with additive white Gaussian noise ($\sigma =15$). ↑ indicates that higher values are better and ↓ indicates that lower values are better. The best results are marked in red, while the second-best results are marked in blue.

Scenario	Metric	BM3D	DnCNN	BUIFD	DUMRN	DRANet	IDTransformer	SwinIR	CFAT	Our Model
Building	PSNR ↑	35.54	36.75	33.94	36.71	37.32	36.80	36.55	36.04	36.82
	SSIM ↑	0.9359	0.9490	0.9488	0.9481	0.9553	0.9494	0.9528	0.9459	0.9563
	LPIPS ↓	0.2046	0.1450	0.1822	0.1490	0.1241	0.1427	0.1334	0.1496	0.1220
	FID ↓	86.24	27.50	39.31	30.40	28.19	32.05	28.21	30.34	21.32
Farmland	PSNR ↑	37.65	38.19	34.06	38.25	39.20	38.55	38.95	37.51	38.45
	SSIM ↑	0.9225	0.9344	0.9334	0.9295	0.9454	0.9358	0.9363	0.9294	0.9385
	LPIPS ↓	0.2668	0.2183	0.2486	0.2269	0.1789	0.2106	0.1989	0.2230	0.1770
	FID ↓	128.87	68.16	77.82	72.62	58.73	68.44	65.62	71.05	51.85
Vegetation	PSNR ↑	36.23	37.65	34.63	37.56	37.32	37.66	37.86	36.63	37.99
	SSIM ↑	0.9156	0.9405	0.9385	0.9380	0.9391	0.9395	0.9429	0.9362	0.9446
	LPIPS ↓	0.2507	0.1847	0.2079	0.1902	0.1610	0.1826	0.1780	0.1824	0.1537
	FID ↓	93.93	41.91	50.32	42.40	40.72	44.51	41.78	40.03	38.12
Water	PSNR ↑	42.33	42.61	38.96	42.06	43.60	42.77	43.30	39.69	42.34
	SSIM ↑	0.9633	0.9644	0.9598	0.9622	0.9705	0.9642	0.9636	0.9617	0.9659
	LPIPS ↓	0.2836	0.2943	0.3582	0.2818	0.2487	0.3217	0.3113	0.2892	0.1924
	FID ↓	161.24	82.77	106.62	80.40	111.28	107.47	117.70	89.52	59.24

,

Table 2. Quantitative results of different methods in four scenarios with additive white Gaussian noise ($\sigma =25$). ↑ indicates that higher values are better and ↓ indicates that lower values are better. The best results are marked in red, while the second-best results are marked in blue.

Scenario	Metric	BM3D	DnCNN	BUIFD	DUMRN	DRANet	IDTransformer	SwinIR	CFAT	Our Model
Building	PSNR ↑	32.73	33.39	34.02	34.01	34.33	34.14	33.91	34.21	34.64
	SSIM ↑	0.8914	0.9076	0.9084	0.9140	0.9262	0.9162	0.9122	0.9195	0.9170
	LPIPS ↓	0.2842	0.2273	0.2096	0.2233	0.1871	0.2055	0.2294	0.2068	0.1993
	FID ↓	140.74	60.93	52.59	65.89	64.70	58.71	76.26	67.17	48.02
Farmland	PSNR ↑	35.33	35.77	36.07	35.97	36.68	36.28	35.92	36.44	36.98
	SSIM ↑	0.8795	0.8877	0.8978	0.8900	0.9175	0.8999	0.8915	0.9064	0.8990
	LPIPS ↓	0.3456	0.3130	0.2892	0.3106	0.2695	0.2767	0.3131	0.2777	0.2441
	FID ↓	164.58	118.73	107.23	123.45	106.76	116.13	120.53	121.16	101.74
Vegetation	PSNR ↑	33.35	34.74	34.62	34.46	34.92	35.06	34.80	34.91	35.25
	SSIM ↑	0.8487	0.8900	0.8887	0.8939	0.9039	0.8993	0.8912	0.8972	0.8946
	LPIPS ↓	0.3400	0.2665	0.2583	0.2712	0.2377	0.2444	0.2760	0.2621	0.2352
	FID ↓	145.51	73.97	79.26	75.85	77.86	72.57	93.02	82.92	71.19
Water	PSNR ↑	40.41	40.19	39.91	40.67	41.74	41.24	40.12	40.81	41.89
	SSIM ↑	0.9540	0.9508	0.9533	0.9552	0.9604	0.9578	0.9521	0.9581	0.9597
	LPIPS ↓	0.3489	0.3932	0.3756	0.3828	0.2955	0.3489	0.3673	0.3877	0.2353
	FID ↓	196.00	128.62	169.29	141.43	172.15	173.04	145.18	180.15	102.88

and

Table 3. Quantitative results of different methods in four scenarios with additive white Gaussian noise ($\sigma =50$). ↑ indicates that higher values are better and ↓ indicates that lower values are better. The best results are marked in red, while the second-best results are marked in blue.

Scenario	Metric	BM3D	DnCNN	BUIFD	DUMRN	DRANet	IDTransformer	SwinIR	CFAT	Our Model
Building	PSNR ↑	28.79	29.40	30.01	30.60	30.78	30.94	30.69	30.42	31.23
	SSIM ↑	0.7951	0.8144	0.8421	0.8453	0.8578	0.8543	0.8455	0.8401	0.8648
	LPIPS ↓	0.4144	0.3633	0.3139	0.3336	0.2747	0.3112	0.3268	0.3347	0.3072
	FID ↓	210.23	127.03	138.30	138.30	113.62	117.68	145.56	113.21	97.37
Farmland	PSNR ↑	32.37	32.93	32.87	33.39	34.08	33.87	33.48	33.09	34.27
	SSIM ↑	0.8153	0.8243	0.8343	0.8348	0.8645	0.8464	0.8337	0.8325	0.8632
	LPIPS ↓	0.4643	0.4268	0.3964	0.4035	0.3376	0.3897	0.4099	0.4039	0.3820
	FID ↓	251.98	188.32	186.64	189.70	174.61	179.79	210.20	184.45	167.58
Vegetation	PSNR ↑	29.49	31.40	29.63	31.64	31.98	31.81	31.55	31.39	31.95
	SSIM ↑	0.7335	0.7923	0.7667	0.8010	0.8073	0.8092	0.7950	0.8005	0.8193
	LPIPS ↓	0.4601	0.3999	0.4176	0.3895	0.3544	0.3534	0.3943	0.3660	0.3478
	FID ↓	223.78	158.42	183.06	166.36	142.29	132.30	170.74	127.64	126.88
Water	PSNR ↑	34.20	36.98	33.01	38.25	39.36	39.13	38.46	37.75	39.03
	SSIM ↑	0.9368	0.9253	0.9370	0.9282	0.9487	0.9403	0.9416	0.9386	0.9408
	LPIPS ↓	0.4359	0.4752	0.3896	0.4448	0.3315	0.3607	0.4032	0.4057	0.3044
	FID ↓	259.41	220.70	222.97	256.40	200.96	188.22	223.35	218.11	162.06

present a summary of the quantitative evaluation results across various methods and scenarios; with red indicating the highest performing method and blue signifying the second-best performance. It is evident from these tables that our method consistently achieves superior results on the FID image quality metric, irrespective of the noise level or scenario (a total of 12 conditions: 3 noise levels × 4 scenarios). In terms of LPIPS, our method outperforms in the majority of cases, with only three instances of secondary performance.While our method does not exhibit optimal performance for traditional pixel-level metrics (PSNR and SSIM) in every context, it consistently ranks among the top two. Notably, at a noise level of $\sigma$ = 15, our approach achieves superior performance across all metrics in the vegetation scenario. The FID metric, in particular, is 1.91 lower than that of the next best method. This underscores our method’s marked proficiency in restoring intricate details of vegetation texture and scene semantics under minimal noise conditions. As noise levels escalate, our method maintains a robust performance edge, affirming its outstanding resistance to noise.Our results are competitive on PSNR/SSIM and strongly competitive on FID and LPIPS, indicating that the proposed SARFT-GAN better preserves realistic details and semantic consistency.

4.2.2. Qualitative Comparison

Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10 illustrate the denoising effects of various methodologies under diverse scenarios at a noise level of $\sigma$ = 25. All methods demonstrate adequate denoising capabilities when confronted with moderate noise levels. However, our method clearly surpasses others in terms of noise suppression, as evidenced by the intricate details of road lines in Figure 6 and roof features in Figure 7. While DnCNN and DUMRN exhibit some level of noise suppression, they fall short in terms of restored image quality, leaving noticeable smearing traces. BUIFD, IDTransformer, and SwinIR do not excessively smear, but they fall short in restoring texture details. On the other hand, DRANet and CFAT restores texture details effectively without any smearing, but the resultant image lacks a degree of realism and appears somewhat smoothed. Our proposed method masterfully balances noise suppression and texture detail restoration, yielding a more realistic and effective outcome than its counterparts. Figure 8 and Figure 9 depict the denoising effects at $\sigma$ = 25 for farmland, vegetation, and water scenarios respectively. Consistent with conclusions drawn from the building scenario analysis, our method again proves superior in terms of realism and overall denoising results across different landscape details. This consistently showcases our method’s outstanding performance and robust generalization capability.

Figure 11 and Figure 12 display performance at a noise level of $\sigma$ = 50. High-intensity noise presents substantial challenges. Notably, DnCNN, BUIFD, DUMRN, IDTransformer, SwinIR, and CFAT show significant distortion marked by pronounced smearing traces. In contrast, our method upholds detailed restoration with minimal distortion, even under such high-noise conditions. This underscores its robust strength.

4.3. Real-World Noisy Remote Sensing Images

To further validate the effectiveness of the proposed method in restoring real-world noisy remote sensing images, we evaluate it on two real images from the NWPU-RESISC45 dataset [43]. Because these real images do not have matched clean references, PSNR and SSIM cannot be used in this subsection. Instead, we adopt two no-reference metrics: Natural Image Quality Evaluator (NIQE) [44] and Perceptual Index (PI) [45]. For both metrics, lower is better: NIQE assesses how closely an image conforms to natural scene statistics, and PI reflects overall perceptual quality. Although no-reference IQA metrics are not as precise as full-reference measures such as PSNR and SSIM, they still provide an objective assessment of image quality. As shown in Figure 13, our method exhibits clear advantages. It achieves the best NIQE and PI scores on all test images, indicating superior denoising performance. In terms of visual quality, SwinIR, IDTransformer and Our Model yield similarly pleasing results, which also explains their close NIQE and PI scores. Overall, these experiments demonstrate the effectiveness and reliability of our approach in practical applications.

4.4. Model Complexity and Runtime Comparison

We evaluated efficiency on an NVIDIA GeForce RTX 4090 (NVIDIA, Santa Clara, CA, USA).

Table 4. Computational cost and efficiency comparison.

Metric	DnCNN	SwinIR	DRANet	IDTransformer	CFAT	Our Model
FLOPs [G]	129.08	2644.22	2082.41	486.15	4978.76	2867.39
Params [M]	0.56	11.46	1.62	18.53	21.49	11.56
Average Inference Time [ms]	1.5	1243.9	188	236.2	8123.2	25.8

compares our model with five recent methods (DnCNN, SwinIR, DRANet, IDTransformer, and CFAT) in terms of FLOPs, parameter count, and average per-image inference time; FLOPs and runtime were measured with 480×480 inputs. As shown in Table 4, the FLOPs and parameter count are comparable to those of transformer-based methods and relatively high; this follows from the added capacity of attention and enhanced convolutions, yet is acceptable. In contrast, our model is the second fastest in wall-clock inference time (behind only DnCNN), indicating solid practical efficiency. Overall, the method achieves a satisfactory efficiency.

4.5. Ablation Study

We conduct ablation studies to highlight the importance of each component in our model. Quantitative results are shown in

Table 5. Ablation analysis of SARFT-GAN with different components. ↑ indicates that higher values are better and ↓ indicates that lower values are better.

Method	PSNR ↑	SSIM ↑	LPIPS ↓	FID ↓
Model-1	37.22	0.9423	0.1722	50.64
Model-2	37.93	0.9483	0.1692	47.58
Model-3	37.95	0.9496	0.1843	52.35
Model-4 (Our)	38.91	0.9513	0.1613	42.63

. Results are averaged across test sets. All results were obtained under a noise level of $\sigma$ = 15. Experimental groups include: (a) Model-1: Without ARConv attention (replaced with standard convolution) (b) Model-2: Without Top-k selection attention (c) Model-3: Without Semantic-Aware Discriminator (replaced with PatchGAN) (d) Model-4: Our proposed method (with both ARConv and Top-k selection attention) The table shows that combining Top-k and ARConv achieves optimal performance, while omitting either component degrades results. Model-1 performs slightly worse than Model-2, indicating that removing ARConv has a greater negative impact: removing ARConv harms content-adaptive feature formation and geometry-aligned aggregation, which has a larger impact on pixel-fidelity metrics than removing Top-k sparsification, because Top-k cannot compensate for the loss of ARConv’s geometry-aware receptive fields and enhanced value features. Model-3 outperforms Model-1 and Model-2 on PSNR and SSIM but underperforms them on LPIPS and FID, indicating that the Semantic-Aware Discriminator plays a larger role in improving perceptual quality in our method.

5. Discussion

It should be noted that our evaluation is mainly conducted on datasets which are primarily urban datasets. Although the proposed modules are not tied to specific scene categories, further validation on rural or mixed-land-cover datasets is necessary to fully assess generalization. We leave such evaluations—including agricultural and forested areas—for future work.

In addition to the above, our paper has other limitations. First, our method depends on a large quantity of remote sensing images for training, which may not be easy to obtain in many applications; future work can consider using unsupervised domain adaptation or few-shot learning techniques. Second, the Transformer contains many parameters and may place higher demands on hardware resources. Lightweight design also deserves further exploration.

6. Conclusions

This study introduces SARFT-GAN, an innovative generative adversarial network specifically designed for denoising optical remote sensing images. It achieves a synergistic optimization of noise suppression and detail preservation by overcoming the limitations inherent in traditional convolution operations, attention mechanisms, and discriminator design. The dynamic sampling mechanism, which is based on ARConv, allows for a flexible adjustment of the receptive field in accordance with target geometry. When combined with the Top-k sparse attention strategy, it effectively suppresses redundant interference from irrelevant features in global attention, thereby addressing issues of blurring in high-frequency texture regions. The semantically-aware discriminator further guides the generation process using semantic priors derived from pre-trained vision-language models, ensuring that texture reconstruction in complex scenes (e.g., vegetation, buildings) aligns with physical laws. Ablation studies have been conducted to validate the effectiveness of each SARFT-GAN component, with particular emphasis on the significant contribution of ARConv to performance. In conclusion, the proposed SARFT-GAN model demonstrates promising application potential in practical image denoising.