AWM-GAN: SAR-to-Optical Image Translation with Adaptive Weight Maps

Highlights

What are the main findings?

• The proposed AWM-GAN effectively combines registration correction and adaptive weight mapping, ensuring geometric alignment and transformation consistency between SAR and optical domains.
• The adaptive weight map integrates attribution and uncertainty information to emphasize reliable regions and reduce the influence of uncertain areas during training, thereby enhancing structural preservation and spectral realism.

What is the implication of the main finding?

• The experimental results show that AWM-GAN consistently outperforms existing comparison models across all evaluation metrics, demonstrating superior performance in both structural accuracy and color restoration quality.
• By achieving explainability in cross-modal image translation, AWM-GAN introduces the potential for integrating explainable artificial intelligence (XAI) into the field of remote sensing image generation.

Abstract

Synthetic Aperture Radar (SAR) imagery enables high-resolution observation regardless of weather conditions. However, it is difficult to interpret intuitively due to issues such as speckle noise. In contrast, optical imagery provides realistic visual information but is vulnerable to weather and illumination changes. Leveraging this complementarity, SAR-to-Optical image translation has attracted considerable attention. Nevertheless, existing paired learning approaches are limited by the high cost of large-scale data collection and residual misregistration errors, while unpaired learning approaches are prone to global color shifts and structural distortions. To address these limitations, this study proposes a SAR-to-Optical translation framework that introduces a weight map throughout the training process. The proposed weight map combines Attribution maps and Uncertainty maps to amplify losses in important regions while guiding conservative learning in uncertain areas. Moreover, the weight map is incorporated into the registration stage to refine pixel-wise displacement estimation, preserve boundary and structural consistency, and enhance overall training stability. The experimental results demonstrate that the proposed method outperforms existing approaches on the SAR2Opt and SEN1-2 datasets in all metrics, including PSNR and SSIM.

1. Introduction

Remote sensing imagery serves as a cornerstone for understanding the Earth system and addressing global social and environmental challenges. Its applications span from climate monitoring and land surface dynamics [1,2] to agricultural productivity enhancement [3,4], disaster risk management [5,6], maritime surveillance and port logistics [7,8], as well as urban expansion and infrastructure planning [9,10].

Among the various sensing technologies, Synthetic Aperture Radar (SAR) and optical imagery have emerged as two of the most representative and complementary modalities. SAR employs an active microwave sensing that transmits radar pulses and receives backscattered signals. As a result, SAR has the advantages of enabling observations independent of solar illumination both during day and night, allowing for stable data acquisition regardless of weather conditions such as clouds, fog, and precipitation. In addition, SAR is highly sensitive to the shape and structural characteristics of terrain and man-made objects. Nevertheless, these strengths are counterbalanced by inherent drawbacks, detailed below. Speckle noise, a byproduct of coherent interference, introduces grainy textures that degrade visual quality and hinder subsequent processing [11,12]. Geometric distortions arising from side-looking geometry [13,14] lead to layover and foreshortening effects, complicating alignment with ground truth. Furthermore, the absence of intuitive spectral content renders SAR interpretation highly nontrivial for non-experts.

In contrast, optical imagery captures solar reflectance across visible and near-infrared bands, thereby producing photorealistic representations closely aligned with human visual perception. Optical data convey rich chromatic and textural cues that are essential for tasks such as vegetation monitoring, land cover classification, and damage assessment. Its intuitive interpretability explains the broad adoption of optical datasets in both research and practice. However, optical sensing is inherently constrained by atmospheric and illumination conditions. For example, environmental factors including clouds, fog, and haze, along with varying solar geometry, significantly impact acquisition quality [15,16,17]. Consequently, optical imagery can be unreliable for continuous monitoring in regions with persistent cloud cover or in time-sensitive emergency response scenarios.

Taken together, SAR and optical imagery exhibit highly complementary characteristics. SAR excels at structural reliability under challenging conditions, whereas optical imagery provides spectral fidelity and human interpretability. Their integration or mutual translation has thus become a vibrant research direction in the remote sensing community [18,19,20]. In particular, SAR-to-Optical (S2O) image translation seeks to render SAR images into optical-like forms, alleviating the steep learning curve required for SAR interpretation. This not only enhances usability for non-expert end users but also expands the applicability of SAR data to domains where intuitive interpretation is critical, such as disaster recovery, environmental surveillance, and urban planning. However, to fully realize this potential, it is essential to develop translation models capable of effectively handling the complex distributional differences between SAR and optical data. Consequently, image-to-image (I2I) translation techniques have been applied to the S2O problem, and the approaches can be broadly categorized into paired-data-based supervised methods and unpaired-data-based unsupervised methods.

Early I2I translation research was largely dominated by paired learning, which leverages pixel-wise correspondences between SAR and optical images acquired under near-identical conditions. Representative examples include Pix2Pix [21] and BicycleGAN [22], both of which demonstrated the feasibility of generating visually coherent results. However, paired approaches face three fundamental limitations. First, there is the scarcity and high cost of acquiring well aligned SAR-Optical pairs at scale. Second, residual misalignments often remain in practice due to sensor differences and acquisition timing. Third, these approaches show limited generalizability when applied beyond the training domain. Diffusion-based generative models offer higher-quality synthesis but impose prohibitive computational burdens, constraining their scalability for operational remote sensing.

These challenges motivated the development of unpaired learning strategies. The seminal CycleGAN [23] introduced cycle consistency to learn bidirectional mappings without paired data. Building on this foundation, several extensions have emerged. CUT [24] enhances local structural consistency through contrastive patch learning. NICE-GAN [25] improves efficiency by reusing discriminator layers as encoder components. ASGIT [26] introduces spatial attention for focused translation. StegoGAN [27] leverages embedding-based representations to improve adaptability. More recently, domain-tailored designs such as FG-GAN [28] and the two-stage method of Qing et al. [20] have demonstrated the potential of adapting these ideas specifically to S2O translation.

Despite this progress, unpaired approaches still fall short of delivering reliable S2O translations. Four issues are particularly salient. First, global radiometric deviations arise due to irreconcilable sensing differences, resulting in biased tones and inconsistent brightness that undermine spectral fidelity. Second, fine-grained textures such as vegetation patterns, building surfaces, or road networks are frequently lost, as the models prioritize global structure over local detail. Third, man-made boundaries often suffer from blurring and deformation, exacerbating geometric inconsistencies rather than correcting SAR distortions. Finally, domain shift remains unresolved: models trained on specific datasets generalize poorly when confronted with unseen SAR distributions, severely limiting robustness in real-world deployments.

To address these challenges, this study introduces a novel S2O image translation framework that integrates the strengths of both paired and unpaired paradigms. The framework is built on Cycle-Consistent Adversarial Networks [23], augmented with two critical components. The first is a fine-grained registration module, explicitly designed to correct residual geometric misalignments between SAR and optical data, thereby complementing the weaknesses of paired learning. The second is an adaptive loss re-weighting mechanism, constructed from attribution and uncertainty maps. The attribution map highlights structurally important regions such as edges and boundaries, while the uncertainty map identifies areas of low confidence or potential distortion. Their combination yields a spatial weight map that adaptively modulates cycle and registration losses, strengthening reliable regions while attenuating unreliable ones. Through this synergy, the framework simultaneously enhances optical fidelity, structural consistency, and domain robustness. The main contributions of this study are summarized as follows:

• We propose a registration enhanced CycleGAN framework that explicitly corrects residual geometric errors in SAR-Optical pairs, improving boundary preservation and geometric reliability.
• We design an attribution and uncertainty guided weight map for adaptive loss re-weighting, which reflects spatially varying importance and confidence, thereby boosting both performance and interpretability.
• We conduct extensive experiments on the SAR2Opt [29] and SEN1-2 [30] datasets, demonstrating consistent improvements across all major quantitative metrics, validating the robustness and generalizability of the proposed method.

2. Related Work

2.1. Image-to-Image Translation and SAR-to-Optical

I2I translation aims to learn the distributional discrepancy between different domains and translates input images into corresponding target-domain representations using generative models. Such techniques have been widely applied in diverse areas, including medical imaging, artistic style transfer, autonomous driving perception, and remote sensing, offering the potential to leverage heterogeneous data from sensors with distinct characteristics.

2.1.1. Paired I2I Approaches

Paired data-based approaches rely on aligned image pairs acquired under identical spatiotemporal conditions to learn direct mappings between distinct domains. Leveraging the strengths of supervised learning, this paradigm produces high-quality translations. Figure 1 illustrates representative examples of such pair-based learning based translation.

A representative work, Pix2Pix [21], established one-to-one mappings between inputs and outputs, where pixel-level reconstruction losses enhanced color fidelity and preserved fine-grained details. This approach laid the foundation for subsequent supervised I2I research. Building upon this, BicycleGAN [22] introduced a multimodal latent space to generate diverse outputs from a single input, thereby enhancing flexibility. While such models deliver stable and realistic results when sufficient paired data are available, constructing large-scale high-quality paired datasets in satellite and aerial remote sensing remains challenging. Sensor heterogeneity, acquisition conditions, and viewpoint discrepancies introduce inevitable residual misalignments, which degrade translation performance. Consequently, despite their high potential, paired approaches are constrained by prohibitive data acquisition costs and limited applicability.

2.1.2. Unpaired I2I Approaches

To address the scarcity of paired datasets, unpaired I2I approaches have been proposed. These methods enable learning translations even in the absence of aligned data, which can be particularly beneficial in domains like remote sensing, where data construction is difficult. Figure 2 illustrates the core framework of CycleGAN, highlighting the bidirectional translation enabled by cycle-consistency. CycleGAN [23] introduced the cycle-consistency loss, enforcing input images to be recoverable back to their original domain, thereby enabling bidirectional learning without pairs. Although this framework is groundbreaking, it still suffers from global radiometric shifts, texture degradation, and structural distortions. Several extensions have sought to overcome these limitations.

Additionally, Figure 3 presents examples of various unpaired I2I methods. CUT [24] applied contrastive learning to intermediate generator patches, enhancing structural consistency. NICE-GAN [25] improved efficiency and performance by reusing early discriminator layers as the generator encoder. ASGIT [26] employed spatial attention mechanisms to focus on semantically critical regions. StegoGAN [27] constrained latent embeddings to enforce cross-domain alignment. Compared to paired approaches, these methods offer higher flexibility and achieve moderate improvements in translation quality. Nonetheless, fundamental issues remain, including structural distortions, global color discrepancies, and generalization degradation due to domain mismatch.

2.1.3. S2O-Specific I2I Approaches

S2O image translation entails challenges beyond general I2I tasks. Owing to microwave backscattering properties, SAR imagery inevitably exhibits speckle noise, geometric distortions, and radar shadows, making direct alignment with optical imagery difficult. To address these characteristics, specialized S2O image translation frameworks have been proposed. FG-GAN [28], for instance, aimed to preserve structural information from SAR while reconstructing the spectral fidelity of optical images, incorporating domain adaptation to mitigate cross-domain representation gaps. Similarly, Qing et al. [20] proposed a two-stage framework integrating a registration module into a CycleGAN backbone. This approach simultaneously corrected residual geometric misalignments within paired data and leveraged the strengths of unpaired learning to alleviate dataset scarcity. Nonetheless, these specialized approaches still suffer from limited capability in capturing fine-grained details and tend to exhibit overfitting toward vegetation regions, leading to biased representations. Figure 4 illustrates representative examples of these specialized S2O translation frameworks.

2.2. Attribution and Uncertainty in Vision Translation

I2I translation has demonstrated remarkable success in cross-domain mapping tasks and has emerged as a core technique in computer vision. Nevertheless, persistent concerns remain regarding the reliability and interpretability of the generated outputs. Users often lack transparency regarding why specific results were produced and to what extent they can be trusted. This issue is particularly critical in application domains such as remote sensing and medical imaging, where decision-making requires clear and reliable justification.

To address this limitation, explainable artificial intelligence (XAI) methods have been introduced. XAI aims to convert the complex internal processes of deep models into human-understandable explanations, primarily realized through attribution-based techniques. Attribution methods quantify or visualize the contribution of each input feature to the output, thereby revealing which parts of the input the model relied on and to what degree. Representative techniques include Layer-wise Relevance Propagation (LRP) [31], Grad-CAM [32], and Integrated Gradients [33]. LRP redistributes the prediction score backward through the network to assign a relevance value to each input pixel, thereby indicating which evidence most strongly supports the model’s decision. Grad-CAM visualizes class-discriminative regions by exploiting the final convolutional feature maps, offering heatmaps of network attention. Integrated Gradients accumulate gradients along the path from a baseline to the input, providing a more stable and linear attribution measure. Collectively, these attribution-based methods enhance interpretability by offering explanatory evidence for I2I translation outputs.

In parallel, uncertainty estimation constitutes an equally critical dimension. Uncertainty quantifies the reliability of model predictions and is typically divided into aleatoric and epistemic uncertainty [34]. Aleatoric uncertainty arises from inherent data variability, such as sensor noise, resolution limits, or speckle artifacts in SAR imagery, and is often modeled through variance estimation or pixel-wise probability maps. Epistemic uncertainty, by contrast, reflects model ignorance due to insufficient training data or architectural limitations, highlighting what the model does not know. Estimation techniques include Monte Carlo Dropout [35], which repeatedly applies dropout during inference to obtain output variance, and Bayesian neural networks, which treat weights as probabilistic variables to provide posterior distributions and confidence intervals.

Recent works have increasingly integrated these concepts into I2I translation. For instance, Uncertainty-Guided Progressive GAN [36] employed aleatoric uncertainty as an attention map to iteratively refine uncertain regions. MultiResunc [37] jointly estimated aleatoric and epistemic uncertainty to improve MR-to-CT translation. Karthik et al. [38] incorporated aleatoric-based cycle-consistency and gradient-consistency losses into CycleGAN to enhance boundary preservation and interpretability. Uncertainty-Aware Regularization [39] combined generalized Gaussian aleatoric modeling with total variation regularization to suppress noise-sensitive uncertainty.

In the context of S2O image translation, attribution and uncertainty are particularly relevant. SAR imagery often suffers from information loss and structural deformation due to speckle noise, geometric distortions, and radar shadows. Attribution maps can serve to verify whether models attend to semantically or structurally important regions, such as terrain boundaries, water bodies, or building contours. Meanwhile, uncertainty maps explicitly reveal low-confidence areas, offering practical cues for post-processing and refinement. Ultimately, attribution and uncertainty are likely to serve as key mechanisms not only for enhancing perceptual quality but also for ensuring the reliability and interpretability of S2O image translation outcomes.

3. Data Description

Research on S2O image translation has been actively conducted along with the construction of various paired datasets. However, most existing remote sensing datasets are biased toward optical imagery, while high-quality SAR datasets, especially those organized as SAR-Optical pairs, remain limited. For instance, Sentinel-1/2-based datasets have the advantage of global coverage, but their relatively low spatial resolution makes them less suitable for local-scale analysis. On the other hand, high-resolution SAR datasets collected by satellites such as TerraSAR-X or Capella Space can capture fine-grained urban structures, but their coverage is narrow and the number of available pairs is limited. Therefore, S2O research requires selecting datasets according to the task, with low-resolution datasets enabling large-scale training and high-resolution datasets allowing for precise object-level analysis. In this study, we used two representative paired SAR-Optical datasets, SEN1-2 and SAR2Opt. The former is a large-scale low-resolution dataset covering the global domain, while the latter is a high-resolution dataset that preserves fine urban structures. Below, we describe the characteristics and differences of each dataset in detail.

3.1. SEN1-2

SEN1-2 [30] is a large-scale paired dataset released in 2018, consisting of a total of 282,384 co-registered image patches of size $256\times 256$. Figure 5 presents representative examples of the SEN1-2 dataset.

The dataset covers global regions across all four seasons. SAR images were collected from the Sentinel-1 satellite of the European Space Agency, while the corresponding optical images were obtained from the Sentinel-2 satellite. Sentinel-1 employs a C-band SAR sensor with an approximate spatial resolution of 5 m, and all images were acquired in Interferometric Wide Swath mode with VV polarization. SEN1-2 has become widely used in large-scale applications such as vegetation monitoring, disaster response, and land cover classification due to its openness and large volume. However, its relatively coarse resolution of around 5 m limits its applicability in high-precision tasks that require detailed urban structure analysis.

3.2. SAR2Opt

SAR2Opt [29] is a high-resolution SAR-Optical paired dataset released in 2022, comprising 2076 image pairs of size $600\times 600$. The dataset covers approximately 70 km² across multiple urban regions worldwide. The SAR images were acquired using the X-band sensor of the TerraSAR-X satellite in high-resolution spotlight mode, achieving 1 m spatial resolution. The corresponding optical images were collected from Google Earth Engine. Unlike SEN1-2, SAR2Opt can accurately represent object-level structures within cities, such as individual buildings and ships near harbors, making it particularly useful for high-resolution image translation and object-based analysis. However, its limited dataset size remains a drawback. Figure 6 shows example pairs of SAR–optical images from the SAR2Opt dataset.

4. Preliminaries: Cycle-Consistent Adversarial Networks

Cycle-consistent adversarial networks comprise two generators and two discriminators. The generator $G_{X\to Y}$ learns a mapping from domain X to domain Y, while $G_{Y\to X}$ learns the reverse mapping. The discriminators $D_X$ and $D_Y$ are trained to distinguish real samples from generated ones in their respective domains. This dual-structure enables learning cross-domain correspondences without requiring paired data. The training objective is defined by three complementary loss functions.

The first component is the adversarial loss, which enforces distributional alignment between generated and real samples. By adopting the hinge formulation, the discriminator is optimized as follows:

$${\mathcal{L}}_D={\mathbb{E}}_{y\sim p_Y}\left[max(0,1-D(y\left)\right)\right]+{\mathbb{E}}_{x\sim p_X}\left[max(0,1+D(G\left(x\right)\left)\right)\right],$$

(1)

where $p_X$ and $p_Y$ denote the data distributions of domains X and Y. Here, $x\sim p_X$ and $y\sim p_Y$ represent real samples from each domain. The generator is optimized to maximize discriminator responses:

$${\mathcal{L}}_G=-{\mathbb{E}}_{x\sim p_X}\left[D\left(G\right(x\left)\right)\right],$$

(2)

which drives the generated distribution toward the target domain and ensures perceptual realism. The second component is the cycle-consistency loss, which regularizes the mapping to be approximately invertible. Formally,

$${\mathcal{L}}_{cyc}={\mathbb{E}}_{x\sim p_X}\left[{\parallel G_{Y\to X}\left(G_{X\to Y}\left(x\right)\right)-x\parallel }_1\right]+{\mathbb{E}}_{y\sim p_Y}\left[{\parallel G_{X\to Y}\left(G_{Y\to X}\left(y\right)\right)-y\parallel }_1\right],$$

(3)

where ${\parallel ·\parallel }_1$ is the L1 distance. The first term constrains the round-trip mapping $X\to Y\to X$, and the second enforces $Y\to X\to Y$. This bidirectional constraint reduces mode collapse and preserves semantic consistency in unpaired translation. The third component is the identity loss, which prevents degenerate mappings that alter samples unnecessarily. It is defined as follows:

$${\mathcal{L}}_{id}={\mathbb{E}}_{y\sim p_Y}\left[{\parallel G_{X\to Y}\left(y\right)-y\parallel }_1\right]+{\mathbb{E}}_{x\sim p_X}\left[{\parallel G_{Y\to X}\left(x\right)-x\parallel }_1\right].$$

(4)

this regularization discourages color distortions and structural shifts, thereby preserving domain-specific attributes and stabilizing training.

The overall learning objective is a weighted combination of adversarial, cycle-consistency, and identity losses. This formulation underpins the CycleGAN framework [23], which we adopt as the backbone for our study. On top of this foundation, we design additional modules to address domain-specific challenges in SAR-to-Optical image translation.

5. Methods

Our framework addresses two key challenges in cycle-consistency based SAR-to-Optical (S2O) translation: residual geometric misalignments between paired images and the spatial heterogeneity of confidence across different regions. To tackle these issues, we integrate a registration module that estimates pixel-wise displacement fields and employ attribution- and uncertainty-guided weight maps to adaptively reweight spatial losses. The overall architecture is shown in Figure 7, with details of each component described below.

Specifically, the generator adopts an encoder–decoder–based residual learning structure that captures the domain differences between SAR and optical images while preserving the essential scene structure. The discriminator employs a patch-based architecture that evaluates local-region consistency, thereby ensuring fine-grained texture quality and structural realism. In addition, to mitigate the inherent geometric discrepancies between SAR–Optical image pairs, the model incorporates a multi-resolution encoder–decoder registration network. This module estimates pixel-wise dense displacement fields between the generated image and the target-domain image, and uses them to warp the generated output to match the target geometry.

Furthermore, the model utilizes attribution-based and uncertainty-based information produced during the generation process to construct a weight map, which is then used to spatially re-weight the loss function. By assigning higher weights to structurally meaningful or highly reliable regions and reducing the influence of noisy or uncertain areas, the proposed framework balances structural preservation and spectral fidelity while preventing overfitting to noise. Through the combination of this weight-map–guided learning mechanism and the registration module, the proposed method simultaneously improves geometric alignment and visual expressiveness in S2O translation. The following subsections provide a detailed explanation of the weight-map generation process, the displacement-field estimation process, and the way these weights are finally applied to the overall loss.

5.1. Attribution-Uncertainty Guided Weight Map Generation

The framework relies on generating weight maps that guide the optimization process. At the end of each generator forward pass, a single-channel map W is constructed and refined with attribution and uncertainty information. Separate maps are maintained for each translation direction ($X\to Y$ and $Y\to X$) to reflect domain asymmetry. Figure 8 illustrates example weight maps for each translation direction.

Let the last convolutional feature map of the generator be $F\in {\mathbb{R}}^{B\times C\times h\times w}$, where B is the batch size, C the number of channels, and $h\times w$ the spatial resolution. The average gradient of channel c with respect to a scalar generation loss s is computed as follows:

$${\overline{w}}_c={\displaystyle \frac{1}{hw}}\sum _{i,j}{\displaystyle \frac{\partial s}{\partial F_c(i,j)}}.$$

(5)

here, ${\overline{w}}_c$ measures the sensitivity of channel c to s across spatial positions $(i,j)$. Using these weights, a channel-weighted sum is calculated, followed by ReLU activation and normalization to $[0,1]$, yielding the attribution map:

$$A=\mathrm{norm}\left(\mathrm{ReLU}\left(\sum _c{\overline{w}}_c{F}_c\right)\right).$$

(6)

The uncertainty map represents the model’s confidence. The generator outputs pixel-wise log variances, from which normalized standard deviations are obtained:

$$U=\mathrm{norm}\left(exp\left(0.5·log{\sigma }^2\right)\right).$$

(7)

regions with higher uncertainty values correspond to lower confidence predictions. The final weight map combines both attribution and uncertainty. Attribution is down-weighted in uncertain regions, using $\beta$ as a decay factor and $ϵ$ as a small constant:

$$\tilde{W}=\mathrm{norm}\left({\displaystyle \frac{A}{\beta U+ϵ}}\right).$$

(8)

A curriculum schedule stabilizes training by gradually increasing a ramp-up coefficient $\alpha$. The final map is:

$$W=(1-\alpha )+\alpha \tilde{W}.$$

(9)

Thus, W remains within $[1-\alpha ,1]$, preventing complete suppression of gradients. The map is broadcast along the channel dimension and applied consistently across all spectral bands.

5.2. Displacement-Field Flow Estimator

Geometric misalignments often occur between SAR and Optical images due to differences in sensing mechanisms. To mitigate this, we employ a registration module that predicts dense pixel-wise displacement fields in the form:

$$\varphi =(\mathrm{\Delta }y,\mathrm{\Delta }x)\in {\mathbb{R}}^{2\times H\times W},$$

(10)

where $\mathrm{\Delta }x$ and $\mathrm{\Delta }y$ represent horizontal and vertical displacements. The final convolution layer is initialized to zero to start from identity mapping, avoiding abrupt distortions. Given $\varphi$, the source image $I_a$ is warped with bilinear interpolation:

$${\tilde{I}}_a=B(I_a,G_{id}+S\left(\varphi \right)),$$

(11)

where B is the bilinear sampling operator, $G_{id}$ the normalized identity grid, and $S\left(\varphi \right)$ a scaling function. This process is fully differentiable, enabling end-to-end training. The registration module is optimized with two objectives. The photometric loss ${\mathcal{L}}_{\mathrm{photo}}^{\mathrm{reg}}$ enforces similarity between the warped source ${\tilde{I}}_a$ and target $I_b$:

$${\mathcal{L}}_{\mathrm{photo}}^{\mathrm{reg}}={\parallel {\tilde{I}}_a-I_b\parallel }_1.$$

(12)

here, $I_a$ and $I_b$ form a same-modality pair. In the A→B direction the photometric loss is computed between the generated Optical image and the real Optical image, and in the B→A direction it is computed between the generated SAR image and the real SAR image. The smoothness loss ${\mathcal{L}}_{\mathrm{smooth}}^{\mathrm{reg}}$ encourages spatial regularity in $\varphi$:

$${\mathcal{L}}_{\mathrm{smooth}}^{\mathrm{reg}}=\mathbb{E}\left[{\parallel {\nabla }_x\varphi \parallel }^2\right]+\mathbb{E}\left[{\parallel {\nabla }_y\varphi \parallel }^2\right],$$

(13)

where ${\nabla }_x$ and ${\nabla }_y$ are horizontal and vertical spatial gradients. This regularization suppresses discontinuities, yielding coherent and physically plausible flows.

The registration module in this work is jointly trained with the generators in an end-to-end manner, and all loss terms are integrated into a single total loss optimized through a single backward pass. In addition, a weight map is employed to spatially reweight the major loss components by reflecting the local reliability of each region.

5.3. Weight-Map–Guided Global Loss Reweighting

The weight map (W) reweights major losses, emphasizing reliable regions while attenuating uncertain ones. This stabilizes optimization and improves the fidelity of learned displacement fields. To ensure training stability, W is detached from the computation graph and used solely as a multiplicative weighting factor; so, no gradient flows into the map itself.

For cycle consistency loss ${\mathcal{L}}_{\mathrm{cyc}}$, given a reconstruction rec and reference real with loss weighting coefficient ${\lambda }_{\mathrm{cyc}}$:

$${\mathcal{L}}_{\mathrm{cyc}}={\lambda }_{\mathrm{cyc}}{\displaystyle \frac{{\sum }_{i,j}{W}_{i,j}|\mathrm{rec}(i,j)-\mathrm{real}(i,j)|}{{\sum }_{i,j}{W}_{i,j}+ϵ}}.$$

(14)

For photometric consistency loss ${\mathcal{L}}_{\mathrm{photo}}^{\mathrm{global}}$ between warped predictions and the reference with loss weighting coefficient ${\lambda }_{\mathrm{photo}}$:

$${\mathcal{L}}_{\mathrm{photo}}^{\mathrm{global}}={\lambda }_{\mathrm{photo}}{\displaystyle \frac{{\sum }_{i,j}{W}_{i,j}|\mathcal{T}(\mathrm{fake},\varphi )(i,j)-\mathrm{real}(i,j)|}{{\sum }_{i,j}{W}_{i,j}+ϵ}},$$

(15)

where $\mathcal{T}(\mathrm{fake},\varphi )$ denotes the warped output. Finally, the smoothness loss ${\mathcal{L}}_{\mathrm{smooth}}^{\mathrm{global}}$ is weighted in a boundary-aware manner, applied separately to vertical and horizontal neighbors with loss weighting coefficient ${\lambda }_{\mathrm{smooth}}$:

$${\mathcal{L}}_{\mathrm{smooth}}^{\mathrm{global}}={\lambda }_{\mathrm{smooth}}\left({\displaystyle \frac{{\sum }_{i,j}{W}_{i,j}^v|\varphi (i,j+1)-\varphi (i,j)|}{{\sum }_{i,j}{W}_{i,j}^v+ϵ}}+{\displaystyle \frac{{\sum }_{i,j}{W}_{i,j}^h|\varphi (i+1,j)-\varphi (i,j)|}{{\sum }_{i,j}{W}_{i,j}^h+ϵ}}\right).$$

(16)

here, $W^v$ and $W^h$ denote weights for vertical and horizontal neighbors. This formulation enhances boundary preservation while maintaining global smoothness.

In summary, W systematically modulates spatial losses, amplifying errors in semantically meaningful and reliable regions while reducing the influence of noisy or uncertain areas. This design directs the network to focus on structural fidelity and improves stability in cross-domain translation.

6. Experiments

We compared our model with several representative baselines, including CycleGAN [23], CUT [24], ASGIT [26], Qing et al. [20], and StegoGAN [27]. All models were trained under identical training conditions for 100 epochs to ensure a fair comparison.

6.1. Implementation Details

6.1.1. Dataset Configuration

For the SAR2Opt [29] dataset, we followed the default structure provided in the official GitHub (https://github.com/MarsZhaoYT/SAR2Opt-Heterogeneous-Dataset, accessed on 15 October 2025) repository, which offers an approximate 7:3 split between the training set and the inference set. To ensure consistency, we applied the same 7:3 ratio when partitioning the SEN1-2 [30] dataset, since this dataset does not provide an official split. Additionally, both datasets were preprocessed to take 256 × 256 inputs and to produce outputs of the same 256 × 256 resolution. For fair comparison, all baseline models were also trained and evaluated under identical 256 × 256 resolution conditions.

6.1.2. Training Settings

In our method, the adaptive weighting mechanism is determined by three key hyperparameters, each serving a specific purpose. First, the curriculum ramp-up coefficient $\alpha$ is scheduled to increase gradually from 0.15 to 0.70, enabling the model to shift its focus from global structural alignment in the early stage to uncertainty-guided fine-grained refinement in later epochs. Second, the uncertainty attenuation factor $\beta$ is fixed at 1.2 to prevent excessive amplification of attribution responses in regions with high uncertainty values. Finally, the numerical stabilization term $\epsilon$ is set to ${10}^{-8}$ to avoid division-by-zero issues and to ensure stable computation during the construction of the adaptive weight map.

6.1.3. Evaluation Metrics

To comprehensively evaluate translation and reconstruction quality, we adopt six quantitative metrics covering pixel fidelity, structural similarity, perceptual similarity, spectral consistency, radiometric accuracy, and perceptual color difference:

• Peak Signal-to-Noise Ratio (PSNR): PSNR measures the fidelity of the reconstructed image with respect to the reference image, based on the ratio between signal power and noise power. Higher values indicate closer resemblance to the ground truth. Since PSNR is based on mean squared error (MSE), it emphasizes overall intensity differences but may not fully reflect perceptual quality.
• Structural Similarity Index Measure (SSIM): SSIM evaluates structural similarity by incorporating luminance, contrast, and structural information. Unlike PSNR, SSIM captures perceptual structural fidelity, making it more consistent with human visual perception. Values closer to 1 indicate higher structural similarity.
• Learned Perceptual Image Patch Similarity (LPIPS): LPIPS quantifies perceptual similarity by comparing deep features from pretrained neural networks. It measures the distance between local image patches in feature space, correlating well with human perception. Lower values imply closer perceptual similarity.
• Spectral Angle Mapper (SAM): Widely used in remote sensing, SAM measures the spectral similarity between reconstructed and reference images by computing the angle between spectral vectors. Smaller values indicate better spectral preservation, which is crucial in applications such as land cover and vegetation analysis.
• Relative Global Dimensional Synthesis Error (ERGAS): ERGAS quantifies the overall radiometric distortion between reconstructed and reference images. It is calculated from the normalized root mean square error (RMSE) across spectral bands. Lower values represent higher radiometric consistency.
• CIEDE2000 Color Difference ($\mathrm{\Delta }{E}_{2000}$): Following the International Commission on Illumination (CIE) standard, $\mathrm{\Delta }{E}_{2000}$ measures the perceptual difference in color based on hue, chroma, and lightness. It provides a reliable indication of how close the reconstructed image colors are to the ground truth optical image. Lower values denote better color consistency.

In summary, this evaluation protocol enables a rigorous and multidimensional assessment of the proposed method, considering not only pixel-level accuracy but also structural realism, spectral fidelity, and perceptual quality.

6.2. Qualitative Results

Figure 9 compares the translation results of different models on agricultural regions from the SEN1-2 [29] dataset.

CUT [24] fails in all test cases, producing severely degraded outputs. CycleGAN [23] does not correctly capture colors in (b) and (f), and the remaining examples exhibit substantial loss of fine details. StegoGAN [27] also fails to render realistic colors in (b) and (c), while the other cases suffer from degraded details. ASGIT [26] appears to preserve coarse structural layouts, but struggles to recover colors and local textures. The method of Qing et al. [20] reproduces the overall structure and color distribution to some extent, yet still fails to retain fine details. In contrast, the proposed method accurately restores both structure and spectral information while carefully preserving local details. Representative improvements can be observed in the brown area below the village in (b) and the white patch in the center of (f).

Figure 10 presents visual examples of the results obtained from different models on the SAR2Opt [29] dataset. As shown in (a), (b), and (c), when the images are dominated by vegetation (green areas), all models generate relatively stable results. However, in cases such as (d), (e), and (f), where the scene is either non-vegetated or contains colors other than green within the vegetation regions, existing models fail to properly reproduce those colors. In particular, in (d), although the ground-truth optical image exhibits desert-like textures, all comparison models generate results with greenish tones, whereas our model successfully captures the distinctive desert color. Similarly, (e) and (f) are scenes that are largely covered by vegetation but contain partial blue regions, which the comparison models fail to replicate. In contrast, our method clearly reproduces these blue regions, as can be visually observed.

6.3. Quantitative Results

6.3.1. Accuracy-Based Metrics

Table 1. Quantitative comparison of S2O image translation results on SEN1-2 [30] dataset.

Method	PSNR $(\uparrow )$	SSIM $(\uparrow )$	LPIPS $(\downarrow )$	SAM $(\downarrow )$	ERGAS $(\downarrow )$	$\mathbf{\Delta }{\mathit{E}}_{2000}(\downarrow )$
CycleGAN [23]	9.88	0.09	0.54	13.15	125.82	29.43
CUT [24]	9.39	0.05	0.61	12.42	131.67	31.27
ASGIT [26]	10.62	0.12	0.54	12.16	113.69	27.39
Qing et al. [20]	17.26	0.35	0.36	8.25	45.84	13.38
StegoGAN [27]	9.07	0.05	0.57	12.28	142.76	32.39
Ours	19.02	0.45	0.29	6.78	39.00	10.96

presents the quantitative comparison of S2O translation methods on the SEN1-2 dataset [30]. The conventional unpaired approaches show generally low performance across most metrics. Specifically, CycleGAN achieved a PSNR of 9.88, SSIM of 0.09, LPIPS of 0.54, SAM of 13.15, ERGAS of 125.82, and $\mathrm{\Delta }{E}_{2000}$ of 29.43, indicating poor fidelity and high spectral distortion. CUT exhibited similar limitations with PSNR 9.39, SSIM 0.05, LPIPS 0.61, SAM 12.42, ERGAS 131.67, and $\mathrm{\Delta }{E}_{2000}$ 31.27. StegoGAN also remained at a low level, recording PSNR 9.07, SSIM 0.05, LPIPS 0.57, SAM 12.28, ERGAS 142.76, and $\mathrm{\Delta }{E}_{2000}$ 32.39. Compared with these, ASGIT yielded slightly better values, with PSNR 10.62, SSIM 0.12, LPIPS 0.54, SAM 12.16, ERGAS 113.69, and $\mathrm{\Delta }{E}_{2000}$ 27.39, but the overall accuracy still remained insufficient. A more recent method, Qing et al. [20], demonstrated a clear performance jump, achieving PSNR 17.26, SSIM 0.35, LPIPS 0.36, SAM 8.25, ERGAS 45.84, and $\mathrm{\Delta }{E}_{2000}$ 13.38. This indicates significant improvements in both fidelity and spectral preservation compared to earlier unpaired approaches. Nevertheless, the proposed method surpassed all competing models across every metric. It attained the highest PSNR of 19.02, the best SSIM of 0.45, and the lowest LPIPS of 0.29, confirming superior fidelity, structural consistency, and perceptual quality. In terms of spectral accuracy, it recorded the lowest SAM of 6.78 and the best ERGAS of 39.00, highlighting excellent preservation of radiometric information. Finally, with a $\mathrm{\Delta }{E}_{2000}$ of 10.96, our model achieved the most accurate color reproduction. These results clearly demonstrate that the proposed approach consistently outperforms all baseline methods, providing the most balanced and reliable performance across fidelity, structure, spectrum, and color dimensions.

Table 2. Quantitative comparison of S2O image translation results on SAR2Opt [29] dataset.

Method	PSNR $(\uparrow )$	SSIM $(\uparrow )$	LPIPS $(\downarrow )$	SAM $(\downarrow )$	ERGAS $(\downarrow )$	$\mathbf{\Delta }{\mathit{E}}_{2000}(\downarrow )$
CycleGAN [23]	13.06	0.19	0.75	7.04	60.99	20.82
CUT [24]	13.05	0.17	0.74	6.50	60.06	19.81
ASGIT [26]	12.91	0.18	0.74	6.20	62.91	20.10
Qing et al. [20]	15.73	0.27	0.71	5.18	45.82	14.81
StegoGAN [27]	12.68	0.18	0.74	6.53	64.42	20.63
Ours	16.13	0.28	0.72	4.81	43.49	13.92

summarizes the quantitative results on the SAR2Opt dataset using the same evaluation metrics. The early unpaired baselines show clear limitations. CycleGAN produced PSNR 13.06, SSIM 0.19, LPIPS 0.75, SAM 7.04, ERGAS 60.99, and $\mathrm{\Delta }{E}_{2000}$ 20.82, revealing low fidelity and noticeable spectral distortion. CUT reported very similar outcomes with PSNR 13.05, SSIM 0.17, LPIPS 0.74, SAM 6.50, ERGAS 60.06, and $\mathrm{\Delta }{E}_{2000}$ 19.81. ASGIT reached PSNR 12.91, SSIM 0.18, LPIPS 0.74, SAM 6.20, ERGAS 62.91, and $\mathrm{\Delta }{E}_{2000}$ 20.10, showing only marginal variation. Likewise, StegoGAN remained weak with PSNR 12.68, SSIM 0.18, LPIPS 0.74, SAM 6.53, ERGAS 64.42, and $\mathrm{\Delta }{E}_{2000}$ 20.63. In contrast, the more advanced method of Qing et al. [20] delivered a substantial leap forward, obtaining PSNR 15.73, SSIM 0.27, LPIPS 0.71, SAM 5.18, ERGAS 45.82, and $\mathrm{\Delta }{E}_{2000}$ 14.81. These results marked a notable step up in both reconstruction quality and spectral consistency relative to the earlier baselines. Building on this, the proposed model achieved the best results across all indicators: PSNR 16.13, SSIM 0.28, LPIPS 0.72, SAM 4.81, ERGAS 43.49, and $\mathrm{\Delta }{E}_{2000}$ 13.92. The higher PSNR and SSIM values show improved fidelity and structural coherence, while the reduced LPIPS indicates closer perceptual similarity. Importantly, the lowest SAM and ERGAS highlight superior spectral preservation, and the smallest $\mathrm{\Delta }{E}_{2000}$ confirms the most faithful color reproduction. Overall, these findings demonstrate that the proposed method consistently surpasses existing approaches on the SAR2Opt dataset, offering a well-rounded improvement in fidelity, structure, perception, spectrum, and color accuracy.

6.3.2. Computational Cost Analysis

To evaluate the computational efficiency of the proposed method, we conducted a quantitative comparison with existing methods in terms of training time, inference time, and the number of trainable parameters. For a fair comparison, 4000 image pairs were randomly sampled from the SEN1-2 dataset and split into training and inference sets at a 7:3 ratio, and all models were tested under identical hardware and software conditions.

As summarized in

Table 3. Computation Cost on the SEN1-2 [30] dataset.

Method	Training Time (s)	Inference Time (s)	Trainable Parameters (M)
CycleGAN [23]	22,421.8	186.65	28.29
CUT [24]	23,108.8	80.89	14.70
ASGIT [26]	15,930.5	42.17	28.29
Qing et al. [20]	20,429.8	74.40	32.40
StegoGAN [27]	31,983.7	179.28	30.06
Ours	27,848.3	32.27	32.39

, the proposed method exhibits a slight increase in the number of parameters compared to CycleGAN and ASGIT due to the addition of the registration module and adaptive weighting components. However, the overall model size remains comparable to that of [20] and StegoGAN, indicating that the additional modules do not introduce excessive structural complexity. In terms of inference time, the proposed method achieves the fastest performance among all compared approaches. Lastly, regarding training time, the proposed method incurs somewhat higher computational cost than ASGIT and [20] but maintains a similar level to CycleGAN and CUT. This is attributed to the additional training of the registration module and the continuous computation of attribution and uncertainty maps throughout the training process; nonetheless, the overall training burden remains within a practical range.

In summary, despite the inclusion of additional modules, Ours maintains competitive computational efficiency while delivering higher translation quality. These results demonstrate that the proposed framework offers both effectiveness and practicality for SAR-Optical translation tasks.

6.4. Ablation Study

In this section, we conduct ablation experiments to quantitatively analyze the contribution of each component of the proposed AWM-GAN to the final performance. All experiments were performed under the same conditions for 20 epochs, and the comparison includes the following settings: removing the Registration module, removing the weight map (W), using only the Attribution map (A), using only the Uncertainty map (U), and the full proposed method with all components enabled. The quantitative results obtained for each dataset are summarized in

Table 4. Quantitative evaluation of different module combinations on the SEN1-2 [30] and SAR2Opt [29] datasets.

	SEN1-2		SAR2Opt
	PSNR $(\uparrow )$	SSIM $(\uparrow )$	PSNR $(\uparrow )$	SSIM $(\uparrow )$
w/o W Map	12.62	0.20	14.16	0.19
w/o Registration	12.98	0.23	15.24	0.24
w/ A Map	14.19	0.26	15.41	0.25
w/ U Map	14.41	0.25	16.01	0.25
Ours	14.87	0.27	16.10	0.26

.

First, removing the weight map leads to the largest degradation in both PSNR and SSIM across the two datasets. This indicates that, without reflecting region-wise reliability and importance, the model tends to learn uniformly across the entire image, resulting in reduced structural preservation and weakened detail representation. Next, removing the Registration module also results in noticeable performance degradation. Due to intrinsic sensor differences between SAR and optical images, residual geometric discrepancies inevitably exist, and the registration module mitigates such distortions by correcting these misalignments, thereby enabling stable domain translation. Using only the Attribution map achieves improved performance compared to the W setting, as it assigns higher weights to structurally important regions and encourages focused learning on meaningful areas. However, since it does not suppress highly uncertain regions, its stability remains limited. In contrast, using only the Uncertainty map yields larger PSNR improvements than using only A. This demonstrates that attenuating low-reliability regions, which frequently arise due to speckle characteristics in SAR images, plays a crucial role in enhancing translation stability. The full proposed method, which incorporates all modules, achieves the best performance across all metrics.

Overall, the results show that the proposed method with all components enabled consistently delivers the highest performance. This confirms that each module not only provides meaningful independent contributions but also works synergistically to produce optimal SAR–Optical translation quality.

7. Discussion

This study proposed a novel framework that integrates a registration module with adaptive weight maps to address the limitations of existing approaches in SAR-to-Optical (S2O) image translation. Both qualitative and quantitative results demonstrated that the proposed method consistently outperforms previous approaches across all evaluation metrics. On the SEN1-2 and SAR2Opt datasets, the framework achieved improvements not only in conventional image quality metrics such as PSNR and SSIM, but also in remote sensing-specific metrics including SAM, ERGAS, and $\mathrm{\Delta }{E}_{2000}$. This indicates that the proposed framework ensures not only visual similarity but also spectral and structural reliability.

First, the introduction of the registration module effectively minimized distortions of man-made structures such as building boundaries and road networks. Second, the weight map generated by combining attribution and uncertainty maps enhanced the loss in high-confidence regions while alleviating learning in highly uncertain areas. This design secured both training stability and interpretability, extending the explainability of deep learning-based image translation beyond mere performance improvements. In the context of remote sensing, where interpretability and reliability of results are crucial, this contribution significantly enhances the practical applicability of the method. Third, the proposed framework successfully addressed a key limitation of existing methods, which often perform well on vegetation-dominated scenes but degrade in non-vegetation or complex color environments. The experimental results showed that the proposed approach more accurately restored chromatic information in challenging cases such as desert terrain and regions containing blue spectral components, demonstrating the effectiveness of weight-based loss rebalancing in capturing local characteristics and uncertainty.

Nevertheless, several limitations remain. First, while validation on the representative public datasets SEN1-2 and SAR2Opt is meaningful, the method has not yet been tested under diverse sensor characteristics or extreme weather conditions encountered in real-world satellite operations. Second, although performance improvements were confirmed on high-resolution datasets, the limited data size constrains the assessment of generalization ability. Third, as the framework still relies on a GAN-based structure, further comparative studies are required to assess its computational complexity and long-term scalability against emerging diffusion models.

Future work will focus on enhancing robustness through the collection and validation of more diverse datasets, as well as exploring hybrid frameworks that combine GANs with diffusion models. In addition, to address the current limitation of relying on two datasets, future work will further examine the model’s generalization ability by exploring geographically diverse regions or alternative SAR modalities when suitable paired datasets become available. Moreover, given that the proposed framework is built upon a bidirectional image-to-image translation architecture, it can be naturally extended toward Optical-to-SAR generation. In particular, we plan to investigate the reconstruction of SAR imagery from historical optical observations to improve applicability in scenarios where SAR acquisition is limited or unavailable.