Shift-Invariant Unsupervised Pansharpening Based on Diffusion Model

Highlights

What are the main findings?

• A shift-invariant unsupervised diffusion-based pansharpening framework is proposed.
• A translation error estimation module and loss are introduced to handle spatial offsets.

What are the implication of the main findings?

• The work establishes a new framework for training unsupervised diffusion models in pansharpening.
• The study offers new insights into achieving shift-invariant unsupervised fusion.

Abstract

Pansharpening is a crucial topic in remote sensing, and numerous deep learning-based methods have recently been proposed to explore the potential of deep neural networks (DNNs). However, existing approaches are often sensitive to spatial translation errors between high-resolution panchromatic (HRPan) and low-resolution multispectral (LRMS) images, leading to noticeable artifacts in the fused results. To address this issue, we propose an unsupervised pansharpening method that is robust to translation misalignment between HRPan and LRMS inputs. The proposed framework integrates a shift-invariant module to estimate subpixel spatial offsets and a diffusion-based generative model to progressively enhance spatial and spectral details. Moreover, a multi-scale detail injection module is designed to guide the diffusion process with fine-grained structural information. In addition, a carefully formulated loss function is established to preserve the fidelity of fusion results and facilitate the estimation of translation errors. Experiments conducted on the GaoFen-2, GaoFen-1, and WorldView-2 datasets demonstrate that the proposed method achieves superior fusion quality compared with state-of-the-art approaches and effectively suppresses artifacts caused by translation errors.

1. Introduction

Remote sensing applications, ranging from fine-scale land-cover mapping to urban monitoring, often require imagery that simultaneously exhibits rich spatial textures and accurate spectral information [1,2,3]. However, most existing satellites typically acquire high-resolution panchromatic (HRPan) and low-resolution multispectral (LRMS) signals separately, leading to a fundamental resolution imbalance between spatial and spectral modalities. Pansharpening has emerged as an effective strategy to bridge this gap by reconstructing high-resolution multispectral (HRMS) images from the two complementary sources. Despite the long-standing interest in this task, the rapid evolution of imaging sensors, data volume, and real-world deployment demands has highlighted practical challenges that remain insufficiently addressed by conventional fusion paradigms.

Existing pansharpening approaches can be broadly categorized into two classes: traditional methods and deep learning-based (DL-based) methods. Traditional methods are typically divided into component substitution-based (CS-based) methods [4,5,6], multiresolution analysis-based (MRA-based) methods [7,8,9], and variational optimization-based (VO-based) methods [10,11,12]. CS-based methods can effectively preserve spatial details but often suffer from spectral distortion [13]. In contrast, MRA-based methods generally achieve better spectral fidelity, while their ability to retain fine spatial details is limited [14]. Although VO-based methods can overcome these limitations, they are typically complex and computationally expensive [7,14].

DL-based methods have achieved remarkable progress by leveraging the strong capacity of neural networks, which can be divided into two classes: supervised and unsupervised. Supervised methods typically train models on reduced-resolution images, yet their generalization to full-resolution scenarios remains inconsistent due to scale discrepancies and distribution shifts [15]. Differently, unsupervised methods train models on full-resolution datasets, to address the scale generalization difficulties in supervised methods [16,17,18].

A large number of DL-based unsupervised pansharpening methods have been proposed in recent years. For instance, Lin et al. [19] utilized convolutional neural networks (CNNs) to estimate the blur kernel of LRMS, effectively preserving the spectral fidelity. Zhou et al. [18] developed a cycle-consistent generative adversarial network (UCGAN) to bridge the gap between reduced and full-resolution images. Recently, diffusion models (DMs) have attracted extensive attention owing to their strong generative ability across various vision tasks. Compared with GANs, DMs not only produce more accurate results but also provide a more stable training process without the risk of mode collapse [20]. For example, Rui et al. [21] proposed an unsupervised hyperspectral pansharpening method based on a pretrained DM. However, existing unsupervised pansharpening methods based on DMs mostly depend on pretrained DMs [22,23], while the direct training of native DMs for unsupervised pansharpening from scratch remains underexplored and lacks theoretical justification.

In addition, most methods, whether supervised or unsupervised, assume perfect alignment between HRPan and LRMS. However, in practical satellite imaging, HRPan and LRMS are rarely perfectly aligned due to differences in sensor geometry, acquisition time, or preprocessing, inevitably leading to shift (translation) errors between the two images [24]. Even subpixel misalignment causes spatial details from HRPan to be injected into incorrect LRMS locations, producing ghosting edges, duplicated contours, and spectral-spatial inconsistencies in the fused images, as shown in Figure 1. Therefore, a shift-invariant pansharpening method that is robust to translation offsets between LRMS and HRPan is essential for reliable fusion, as it enables the model to compensate for unknown spatial misalignments and effectively prevent the propagation of fusion artifacts.

Several studies have recognized this issue and proposed corresponding solutions [16,25,26,27,28]. For example, Lee et al. [25] introduced a feature alignment module to generate an adjusted LRMS that better matches the HRPan input. However, this adjusted LRMS is only used as input to the fusion network, while the spectral loss computation still relies on multiple spatially shifted LRMS versions. Similarly, Ciotola et al. [16] estimated the offset between HRPan and LRMS by measuring their correlation across several spatial shifts before training. During loss computation, the LRMS is then shifted according to the estimated offset. Although these strategies improve fusion results compared with those ignoring translation errors, they still suffer from several limitations: (1) redundant computation, as multiple LRMS shifts are required; (2) the need to predefine a limited range of possible displacements; and (3) the offset-adjusted LRMS is not directly involved in loss optimization, limiting the effectiveness of these correction strategies.

To address the above limitations, first, we present a framework for training DMs in the context of unsupervised pansharpening and provide a mathematical proof to support their effectiveness. To better guide the diffusion process, we design a Multiscale Detail Injection module (MSDI) that progressively injects fine-grained spatial and spectral information during generation. Second, to achieve robustness against translation errors, we develop a Translation Error Estimation Module (TEM) together with a dedicated translation estimation loss, which adaptively estimates subpixel offsets between HRPan and LRMS. This design effectively enhances the spatial consistency and suppresses misalignment-induced fusion artifacts. Based on these designs, we propose Shift-Invariant Unsupervised Pansharpening based on Diffusion Model (SIUPan). The main contributions of our work are summarized as follows:

• We propose a training framework for diffusion models in unsupervised pansharpening that directly trains a native DM in an unsupervised setting. We design an MSDI to guide the diffusion process by injecting hierarchical spatial and spectral details, ensuring a balanced improvement in both spatial and spectral fidelity.
• We develop a TEM to adaptively estimate subpixel translation offsets between HRPan and LRMS, enabling shift-invariant fusion without redundant computations or predefined displacement ranges.
• A new translation error estimation loss function is developed to guide the TEM estimate the translation error. A tailored loss function is constructed to jointly maintain spectral fidelity and spatial sharpness.

In the rest of this article, we describe the proposed method in Section 2. In Section 3, we conduct extensive experiments to evaluate the performance of our method. Finally, a brief conclusion is presented in Section 4.

2. Methodology

2.1. Overall Network Architecture

As shown in Figure 2, SIUPan consists of two primary modules: (1) a TEM for estimating the displacements and (2) a diffusion-based fusion module for image fusion. Let ${\mathbf{I}}_{\mathrm{pan}}\in {\mathbb{R}}^{W\times H\times 1}$, ${\mathbf{I}}_{\mathrm{lrms}}\in {\mathbb{R}}^{w\times h\times C}$, and ${\mathbf{I}}_{\mathrm{hrms}}\in {\mathbb{R}}^{W\times H\times C}$ represent the corresponding HRPan, LRMS, and HRMS, respectively, where $W\left(w\right)$ and $H\left(h\right)$ are the width and height of HRPan (LRMS); C is the number of bands of LRMS. LRMS is upsampled to obtain the upsampled LRMS (MS), denoted as ${\mathbf{I}}_{\mathrm{ms}}\in {\mathbb{R}}^{W\times H\times C}$. An 11th-order polynomial interpolator [29,30] is used to upsample LRMS. In SIUPan, ${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{ms}}$ are firstly fed into the TEM to obtain the displacements $(\Delta m^{\theta },\Delta n^{\theta })$. Next, we resample ${\mathbf{I}}_{\mathrm{ms}}$ to obtain the shifted image ${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$. In the fusion module, we first define an initial estimate of HRMS, denoted as ${Î}_{\mathrm{hrms}}={\mathbf{I}}_{\mathrm{pan}}^C$. Here, ${\mathbf{I}}_{\mathrm{pan}}^C$ represents ${\mathbf{I}}_{\mathrm{pan}}$ stacked C times along band axis. The input residual image ${Î}_{\mathrm{res}}={Î}_{\mathrm{hrms}}-{\mathbf{I}}_{\mathrm{Rms}}^{\theta }$ is then added to Gaussian noise ${ϵ}_t$ and fed into the diffusion model. The diffusion model with MSDI generates the residual image ${\mathbf{I}}_{\mathrm{res}}^{\theta }$, which is added to ${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$ to obtain the fused image ${\mathbf{I}}_{\mathrm{hrms}}^{\theta }={\mathbf{I}}_{\mathrm{Rms}}^{\theta }+{\mathbf{I}}_{\mathrm{res}}^{\theta }$. Some notations used in the following discussion are listed in

Table 1. The frequently used notations.

Notation	Size	Description
${\mathbf{I}}_{\mathrm{lrms}}$	$w\times h\times C$	Multi-spectral image
${\mathbf{I}}_{\mathrm{pan}}$	$W\times H\times 1$	Panchromatic image
${\mathbf{I}}_{\mathrm{ms}}$	$W\times H\times C$	Upsampled multi-spectral image
${\mathbf{I}}_{\mathrm{Rms}}$	$W\times H\times C$	Perfectly registered ${\mathbf{I}}_{\mathrm{ms}}$
${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$	$W\times H\times C$	Translation corrected ${\mathbf{I}}_{\mathrm{ms}}$
${\mathbf{I}}_{\mathrm{hrms}}$	$W\times H\times C$	Fused multi-spectral image we desire
${Î}_{\mathrm{hrms}}$	$W\times H\times C$	Estimate of ${\mathbf{I}}_{\mathrm{hrms}}$
${Î}_{\mathrm{hrms}}^{\theta }$	$W\times H\times C$	The output image of SIUPan
${\mathbf{I}}_{\mathrm{res}}$	$W\times H\times C$	${\mathbf{I}}_{\mathrm{hrms}}-{\mathbf{I}}_{\mathrm{Rms}}$
${Î}_{\mathrm{res}}$	$W\times H\times C$	${Î}_{\mathrm{hrms}}-{\mathbf{I}}_{\mathrm{Rms}}^{\theta }$
${Î}_{\mathrm{res}}^{\theta }$	$W\times H\times C$	${Î}_{\mathrm{hrms}}^{\theta }-{\mathbf{I}}_{\mathrm{Rms}}^{\theta }$

.

2.2. Diffusion Model

Similar to most diffusion models [20,31,32,33,34], we utilize a U-Net as the backbone of our denoising network. Here, the SR3 [32] architecture is employed. We modify both the forward and reverse processes of diffusion to adapt it for unsupervised pansharpening. Additionally, MSDI is designed to inject spatial details and spectral information into U-Net for better fusion results.

2.2.1. Unsupervised Diffusion Model

In Denoising Diffusion Probabilistic Models (DDPM) [35], the forward process is defined on samples drawn from the true data distribution. However, in unsupervised pansharpening, the true data

$$x_0={\mathbf{I}}_{\mathrm{res}}={\mathbf{I}}_{\mathrm{hrms}}-{\mathbf{I}}_{\mathrm{Rms}}$$

(1)

is unavailable. To address this challenge, we adopt a proxy estimate

$${x̂}_0={Î}_{\mathrm{res}}={Î}_{\mathrm{hrms}}-{\mathbf{I}}_{\mathrm{Rms}}^{\theta },$$

(2)

where ${Î}_{\mathrm{hrms}}$ is an initial HRMS estimate, which is set as the stacked HRPan along the band axis in our work. The relationship between $x_0$ and ${x̂}_0$ can be described as

$$x_0={x̂}_0+z,$$

(3)

where z is the difference between $x_0$ and ${x̂}_0$. We assume that z follows a Gaussian distribution

$$p\left(z\right)=\mathcal{N}(z;0,{\delta }^2\mathbf{I}),$$

(4)

where $\mathcal{N}$ means the Gaussian distribution, and ${\delta }^2$ is the variance. Thus, ${x̂}_0$ is a Gaussian-perturbed version of $x_0$. This assumption allows us to define a valid diffusion forward process without access to the true data distribution.

The forward process can be expressed as

$$p\left({x̂}_t\right|{x̂}_{t-1})=\mathcal{N}({x̂}_t;\sqrt{1-{\beta }_t}{x̂}_{t-1},{\beta }_t\mathbf{I}),$$

(5)

where $t\in [1:T]$, T is the number of timesteps, and ${\beta }_t$ is a pre-defined variance. In forward process, we can sample ${x̂}_t$ from ${x̂}_0$ in a closed form:

$$p\left({x̂}_t\right|{x̂}_0)=\mathcal{N}({x̂}_t;\sqrt{{\overline{\alpha }}_t}{x̂}_0,(1-{\overline{\alpha }}_t)\mathbf{I}),$$

(6)

where ${\overline{\alpha }}_t={\prod }_{i=0}^t{\alpha }_i$ and ${\alpha }_i=1-{\beta }_i$. Substituting (3) into (6), we obtain:

$${x̂}_t=\sqrt{{\overline{\alpha }}_t}{x}_0+{\eta }_t,{\eta }_t\sim \mathcal{N}(0,(1-{\overline{\alpha }}_t+{\delta }^2{\overline{\alpha }}_t)\mathbf{I}),$$

(7)

indicating that using ${x̂}_0$ simply modifies the variance of the diffusion kernel but preserves the trajectory structure. The pseudocode of the forward process is given in Algorithm 1.

Algorithm 1 The Forward Process.

Input: the number of timestep, T, the estimate of $x_0$, ${x̂}_0$

Output: the noised image, ${x̂}_T$

1: for $t=1:T$ do

2:       $\mathrm{select}{\beta }_t$

3:       $\mathrm{sample}{x̂}_t$ according to (6)

4: end for

In the reverse process, our network predicts $x_0$ directly rather than noise. Thus, we can sample ${x̂}_t$ by the conditional distribution $p\left({x̂}_t\right|x_0)$, which also follows a Gaussian distribution

$$p\left({x̂}_t\right|x_0)=\mathcal{N}({x̂}_t;\sqrt{{\overline{\alpha }}_t}{x}_0,(1-{\overline{\alpha }}_t+{\delta }^2{\overline{\alpha }}_t)\mathbf{I}).$$

(8)

Subsequently, we iteratively update $x_0$ using the sampled ${x̂}_t$ until the sampling process is finished. The pseudocode of the reverse process is given in Algorithm 2. The forward and reverse processes of diffusion model in our work are illustrated in Figure 3.

Algorithm 2 The Reverse Process.

Input: the number of timestep, T, Gaussian noise image, ${x̂}_T$, the network, $f^{\theta }$

Output: sampled image, $x_0$

1: for $t=T:1$ do

2:       $x_0=f^{\theta }({x̂}_t,t)$

3:       $\mathrm{sample}{x̂}_{t-1}$ according to (8)

4: end for

5: $t=t-1$

6: $x_0=f^{\theta }({x̂}_0,t)$

Since our goal is to obtain $x_0$, the training objective is to maximize the log likelihood of $p\left(x_0\right)$, i.e., $lnp\left(x_0\right)$,

$$maxlnp\left(x_0\right)=maxln\int p(x_0,{x̂}_{0:T})d{x̂}_{0:T}.$$

(9)

However, we cannot maximize it directly because of the latent variables (${x̂}_{0:T}$) in it. Therefore, we use its Evidence Lower Bound (ELBO), which can be written as

$$\begin{array}{cc}\hfill lnp\left(x_0\right)\propto & -\parallel x_0-x_0^{\theta }{\parallel }_2^2,\hfill \end{array}$$

(10)

where $x_0^{\theta }$ is the output of our network. The detailed proof of (10) is provided in Appendix A. Let $\Delta e=\parallel x_0-x_0^{\theta }{\parallel }_2^2$; then, the objective function in (9) can be converted to

$$min\Delta e=\parallel x_0-x_0^{\theta }{\parallel }_2^2.$$

(11)

Clearly, there is no ${\mathbf{I}}_{\mathrm{res}}$ (${\mathbf{I}}_{\mathrm{hrms}}$) available in unsupervised pansharpening. Fortunately, we have two transformations of ${\mathbf{I}}_{\mathrm{hrms}}$ (${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$) that can be used for training,

$$\begin{array}{c}\hfill {\mathbf{I}}_{\mathrm{pan}}={\mathcal{T}}_{\mathrm{pan}}\left({\mathbf{I}}_{\mathrm{hrms}}\right),{\mathbf{I}}_{\mathrm{Rms}}^{\theta }={\mathcal{T}}_{\mathrm{Rms}}\left({\mathbf{I}}_{\mathrm{hrms}}\right),\end{array}$$

(12)

where ${\mathcal{T}}_{\mathrm{pan}}$ (${\mathcal{T}}_{\mathrm{Rms}}$) represent the transformation function. Moreover, we modify the objective function (11) into

$$min\Delta e_{\mathrm{pan}}+\Delta e_{\mathrm{Rms}},$$

(13)

where $\Delta e_{\mathrm{pan}}={\mathrm{f}}_{\mathrm{pan}}({\mathcal{T}}_{\mathrm{pan}}({\mathbf{I}}_{\mathrm{Rms}}^{\theta }+{\mathbf{I}}_{\mathrm{res}}^{\theta })-{\mathbf{I}}_{\mathrm{pan}})$, and $\Delta e_{\mathrm{Rms}}={\mathrm{f}}_{\mathrm{Rms}}({\mathcal{T}}_{\mathrm{Rms}}({\mathbf{I}}_{\mathrm{Rms}}^{\theta }+{\mathbf{I}}_{\mathrm{res}}^{\theta })-{\mathbf{I}}_{\mathrm{Rms}}^{\theta })$. ${\mathrm{f}}_{\mathrm{Rms}}$ and ${\mathrm{f}}_{\mathrm{Rms}}$ are the spatial and spectral loss functions, respectively, which will be given in the following.

In fact, we can employ the sample solvers that are compatible with DDPM, as the structure of our DM is consistent with that of DDPM. The Denoising Diffusion Implicit Model sample solver [36] is used in our work.

2.2.2. MSDI

Many studies have shown that incorporating conditional information in DMs can enhance generation quality [20,31,37,38]. In pansharpening, we naturally have two prior conditions, HRPan and LRMS, which are leveraged to provide conditional information. HRPan contains rich spatial details but lacks spectral information, while LRMS provides abundant spectral information but has fewer distinct details. Therefore, to balance the spatial details and spectral information, we design the MSDI.

As shown in Figure 4, the MSDI is a multi-layer-structure module, where each layer consists of two cross-attention blocks and two downsampling blocks. Cross-attention blocks are employed to enhance the spectral information of HRPan features and improve the spatial details of LRMS features. Downsampling blocks are used to obtain feature maps at multiple scales.

For the i-th layer of the MSDI (except the first layer), the inputs, ${\mathbf{F}}_{\mathrm{pan}}^{i-1}$ and ${\mathbf{F}}_{\mathrm{ms}}^{i-1}$, are first downsampled to reduce the size of the feature maps, respectively. Convolution layers are followed to control the depth of the feature map,

$$\begin{array}{cc}\hfill {\mathbf{F}}_{\mathrm{pan}}^{i^{\prime }}& =\mathrm{Conv}\left(\mathrm{Downsample}\left({\mathbf{F}}_{\mathrm{pan}}^{i-1}\right)\right),\hfill \\ \hfill {\mathbf{F}}_{\mathrm{ms}}^{i^{\prime }}& =\mathrm{Conv}\left(\mathrm{Downsample}\left({\mathbf{F}}_{\mathrm{ms}}^{i-1}\right)\right).\hfill \end{array}$$

Sequentially, ${\mathbf{F}}_{\mathrm{ms}}^{i^{″}}$ and ${\mathbf{F}}_{\mathrm{pan}}^{i^{″}}$ are obtained after BatchNormalization and LeakyReLU. Two cross-attentions are employed to enhance the two feature maps, which are formulated as

$$\begin{array}{cc}\hfill {\mathbf{F}}_{\mathrm{pan}}^{i^{\prime \prime \prime }}& =\mathrm{E}({\mathbf{F}}_{\mathrm{pan}}^{i^{″}},{\mathbf{F}}_{\mathrm{ms}}^{i^{″}}),\hfill \\ \hfill {\mathbf{F}}_{\mathrm{ms}}^{i^{\prime \prime \prime }}& =\mathrm{E}({\mathbf{F}}_{\mathrm{ms}}^{i^{″}},{\mathbf{F}}_{\mathrm{pan}}^{i^{″}}),\hfill \end{array}$$

where $\mathrm{E}(\mathrm{input},\mathrm{context})$ is the linear complexity cross-attention [39]. The output of the i-th layer is

$$\begin{array}{cc}\hfill {\mathbf{F}}_{\mathrm{pan}}^i& ={\mathbf{F}}_{\mathrm{pan}}^{i^{″}}+{\mathbf{F}}_{\mathrm{pan}}^{i^{\prime \prime \prime }},\hfill \\ \hfill {\mathbf{F}}_{\mathrm{ms}}^i& ={\mathbf{F}}_{\mathrm{ms}}^{i^{″}}+{\mathbf{F}}_{\mathrm{ms}}^{i^{\prime \prime \prime }}.\hfill \end{array}$$

As shown in Figure 5, ${\mathbf{F}}_{\mathrm{ms}}^{1^{\prime }}$, the input feature, is blurred, and ${\mathbf{F}}_{\mathrm{pan}}^{1^{\prime }}$ has low contrast. After cross-attention blocks, the sharpness in ${\mathbf{F}}_{\mathrm{ms}}^1$ is enhanced, and the contrast in ${\mathbf{F}}_{\mathrm{pan}}^1$ is improved. Finally, ${\mathbf{F}}_{\mathrm{ms}}^i$ and ${\mathbf{F}}_{\mathrm{pan}}^i$ are injected into the encoder to guide image generation.

2.3. Translation Error Estimation Module

The translation errors between ${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{ms}}$, can be described as

$${\left[m_{\mathrm{pan}}{n}_{\mathrm{pan}}\right]}^{\mathrm{T}}={\mathcal{T}}_{\mathrm{R}}\left({\left[m_{\mathrm{ms}}{n}_{\mathrm{ms}}\right]}^{\mathrm{T}}\right)=\mathbf{A}{\left[m_{\mathrm{ms}}{n}_{\mathrm{ms}}1\right]}^{\mathrm{T}},$$

(14)

where $\left[m_{\mathrm{pan}}{n}_{\mathrm{pan}}\right]$ and $\left[m_{\mathrm{ms}}{n}_{\mathrm{ms}}\right]$ are the coordinates in ${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{ms}}$, respectively. $\mathbf{A}$ is an affine transformation matrix, which can be expressed as

$$\mathbf{A}=\left[\begin{array}{ccc}1& 0& \Delta m\\ 0& 1& \Delta n\end{array}\right].$$

(15)

However, $\Delta m$ and $\Delta n$ are unknown. Thus, the TEM is designed to estimate them.

As shown in Figure 6, the TEM is comprised of two structurally identical Feature Extractors and one Feature Integration Block. In the Feature Extractor, the inputs, ${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{ms}}$, are mapped into feature domain via a convolutional layer. Considering our goal is to shift ${\mathbf{I}}_{\mathrm{ms}}$ to match ${\mathbf{I}}_{\mathrm{pan}}$, and they are heterogenous images (with significant style differences), we concatenate ${\mathbf{I}}_{\mathrm{ms}}$ with ${\mathbf{I}}_{\mathrm{pan}}$ at the input of MS Feature Extractor, which provides ${\mathbf{I}}_{\mathrm{ms}}$ with style information from ${\mathbf{I}}_{\mathrm{pan}}$. The Feature Integration Block is composed of a MidBlock, three DownBlocks, a convolutional layer, an activation function, and a global average pooling. The MidBlock, containing two convolutional layers and two activation functions, integrates the two extracted features. Inspired by [40], three cascaded DownBlocks are employed to extract the difference between ${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{ms}}$ in the multiscale feature domain. Each DownBlock applies average pooling with a $3\times 3$ kernel and stride 2 to downsample the feature maps. After a global average pooling, an estimated displacement $[\Delta m^{\theta }\Delta n^{\theta }]$ is obtained.

Our objective is to obtain ${\mathbf{I}}_{\mathrm{Rms}}$, which has the same coordinates as ${\mathbf{I}}_{\mathrm{pan}}$. Then, according to (14), we can get the estimated coordinates of ${\mathbf{I}}_{\mathrm{Rms}}$ as

$${\left[m_{\mathrm{Rms}}^{\theta }{n}_{\mathrm{Rms}}^{\theta }\right]}^{\mathrm{T}}=\left[\begin{array}{ccc}1& 0& \Delta m^{\theta }\\ 0& 1& \Delta n^{\theta }\end{array}\right]{\left[m_{\mathrm{ms}}{n}_{\mathrm{ms}}1\right]}^{\mathrm{T}}.$$

(16)

Then, the estimated ${\mathbf{I}}_{\mathrm{Rms}}$ (noted as ${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$) can be resampled by

$${\mathbf{I}}_{\mathrm{Rms}}^{\theta }(m_{\mathrm{Rms}}^{\theta },n_{\mathrm{Rms}}^{\theta },c)=\sum _i^H\sum _j^W{\mathbf{I}}_{\mathrm{ms}}(i,j,c)k(m_{\mathrm{ms}}-i)k(n_{\mathrm{ms}}-j),$$

(17)

where $k(·)$ is the generic sampling kernel, and c is the band number. Here, the bilinear sampling kernel is used. Moreover, a translation error estimation loss is designed to achieve more accurate estimation of the spatial offsets. Since the true displacements $[\Delta m,\Delta n]$ are unknown, it is infeasible to directly minimize the differences between $[\Delta m,\Delta n]$ and $[\Delta m^{\theta },\Delta n^{\theta }]$. Instead, we construct the loss function based on ${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$, as detailed in Section 2.4.1.

2.4. Loss Function

In SIUPan, the loss consists of two terms: the translation error estimation loss and the fusion loss, which is formulated as

$$L=w_1{L}_{\mathrm{R}}+L_{\mathrm{F}},$$

(18)

where $L_{\mathrm{R}}$ is the translation error estimation loss, $L_{\mathrm{F}}$ is the fusion loss, and $w_1$ is the trade-off parameter.

2.4.1. Translation Error Estimation Loss

The objective of TEM is to make $[\Delta m^{\theta }$ $\Delta n^{\theta }]$ as close as possible to $[\Delta m$ $\Delta n]$. However, since the GT offsets are unavailable, we redefine the objective as evaluating whether ${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$ is properly aligned with ${\mathbf{I}}_{\mathrm{pan}}$. According to [41,42,43,44], if two images are perfectly matched, their phase difference is zero in frequency domain. Thus, we formulate the translation error estimation loss function as the phase difference between ${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$

$$L_{\mathrm{R}}={\mathrm{f}}_{\mathrm{R}}({\mathbf{I}}_{\mathrm{pan}},{\mathbf{I}}_{\mathrm{Rms}}^{\theta }),$$

(19)

where ${\mathrm{f}}_{\mathrm{R}}$ is the translation error estimation loss function. Assuming that two images $\mathbf{X}\in {\mathbb{R}}^{H\times W}$ and $\mathbf{Y}\in {\mathbb{R}}^{H\times W}$ have displacements $[\Delta m^{\prime }$ $\Delta n^{\prime }]$, their phase difference in frequency domain can be estimated using normalized cross-power spectrum (NCPS), noted as $\mathbf{Q}(u,v)$

$$\mathbf{Q}(u,v)={\displaystyle \frac{\tilde{\mathbf{X}}(u,v)\tilde{\mathbf{Y}}(u,v)}{|\tilde{\mathbf{X}}(u,v)\tilde{\mathbf{Y}}(u,v)|}},$$

(20)

where $\tilde{\mathbf{X}}(u,v)$ and $\tilde{\mathbf{Y}}(u,v)$ are Fourier transforms of $\mathbf{X}(m,n)$ and $\mathbf{Y}(m,n)$, respectively. However, NCPS is sensitive to noise [43]. To improve its robustness, Zhu et al. [43] proposed an autocorrelated version, termed ANCPS, which can be formulated as

$$\begin{array}{cc}\hfill \mathbf{R}(\mu ,\nu )& =\mathbf{Q}(u,v){\mathbf{Q}}^{*}(u-\mu ,v-\nu )\hfill \\ \hfill & =e^{j2\pi (u\Delta m^{\prime }/H+v\Delta n^{\prime }/W)},\hfill \end{array}$$

(21)

where $\mathbf{R}(\mu ,\nu )$ is ANCPS, and ${\mathbf{Q}}^{*}$ is the complex conjugate of $\mathbf{Q}$. From (21), we can find the displacements are embedded in the phase of ANCPS. Thus, we can use the ANCPS of ${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$ to estimate their displacements. For simplification, we also denote the displacements and ANCPS of ${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$ as $[\Delta m^{\prime }\Delta n^{\prime }]$ and $\mathbf{R}(\mu ,\nu )$, respectively. To ensure proper alignment in both directions, we must compute $\Delta m^{\prime }$ and $\Delta n^{\prime }$ separately. Thus, we have

$$\begin{array}{cc}\hfill p_x(\mu ,\nu )& =\mathbf{R}(\mu ,\nu ){\mathbf{R}}^{*}(\mu -1,\nu )=e^{j2\pi \Delta m^{\prime }/H},\hfill \\ \hfill p_y(\mu ,\nu )& =\mathbf{R}(\mu ,\nu ){\mathbf{R}}^{*}(\mu ,\nu -1)=e^{j2\pi \Delta n^{\prime }/W},\hfill \end{array}$$

(22)

where ${\mathbf{R}}^{*}$ is the complex conjugate of $\mathbf{R}$. From (22), we can obtain $\Delta m^{\prime }$ and $\Delta n^{\prime }$ by $p_x$ and $p_y$, respectively. To enhance the robustness, we use r pixels to calculate their average phase. Thus, the translation error estimation loss is formulated as

$$L_{\mathrm{R}}={\displaystyle \frac{1}{r}}\sum _i^r|\mathrm{angle}\left(p_x({\mu }_i,{\nu }_i)\right)|+|\mathrm{angle}\left(p_y({\mu }_i,{\nu }_i)\right)|,$$

(23)

where angle means the extracting phase.

2.4.2. Fusion Loss

SIUPan incorporates two categories of fusion loss functions: the spatial loss function (i.e., ${\mathrm{f}}_{\mathrm{pan}}$ in Section 2.2) and the spectral loss function (i.e., ${\mathrm{f}}_{\mathrm{Rms}}$ in Section 2.2). Specifically, the spatial loss consists of one term ($L_{\mathrm{spa}}$), while the spectral loss includes two terms ($L_{\mathrm{spe}1}$ and $L_{\mathrm{spe}2}$). Therefore, the fusion loss is expressed as

$$L_{\mathrm{F}}=w_2{L}_{\mathrm{spa}}+w_3{L}_{\mathrm{spe}1}+w_4{L}_{\mathrm{spe}2},$$

(24)

where $w_2$, $w_3$, and $w_4$ are trade-off parameters.

$L_{\mathrm{spa}}$ is designed to preserve the spatial details. According to refs. [28,45], the correlation coefficient between ${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{hrms}}^{\theta }$ is used to measure the spatial distortion. The correlation coefficient between two images $\mathbf{X}$ and $\mathbf{Y}$ is defined as

$$\mathrm{CORR}(\mathbf{X},\mathbf{Y})={\displaystyle \frac{{\sigma }_{\mathbf{X}\mathbf{Y}}}{{\sigma }_{\mathbf{X}}{\sigma }_{\mathbf{Y}}}},$$

(25)

where ${\sigma }_{\mathbf{X}}$, ${\sigma }_{\mathbf{Y}}$, and ${\sigma }_{\mathbf{X}\mathbf{Y}}$ are the variances and covariance of $\mathbf{X}$ and $\mathbf{Y}$. It is obviously unreasonable to assume that each band of ${\mathbf{I}}_{\mathrm{hrms}}^{\theta }$ would have a high correlation with ${\mathbf{I}}_{\mathrm{pan}}$. Therefore, the correlation coefficient between degraded ${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$ is taken as a threshold mask. Moreover, a $s\times s$ window is used to calculate the local correlation coefficient. The spatial loss is formulated as

$$L_{\mathrm{spa}}={\displaystyle \frac{1}{CS}}\sum _c^C\sum _s^S\left(\left(1-{\rho }^s\left(c\right)\right)u({\rho }_{\mathrm{m}}^s-{\rho }^s\left(c\right))\right),$$

(26)

where ${\rho }^s\left(c\right)=\mathrm{CORR}({\mathbf{I}}_{\mathrm{pan}}^s,{\mathbf{I}}_{\mathrm{hrms}}^{\theta s}\left(c\right))$, and ${\rho }_{\mathrm{m}}^s\left(c\right)=\mathrm{CORR}(\mathrm{G}\left({\mathbf{I}}_{\mathrm{pan}}^s\right),{\mathbf{I}}_{\mathrm{Rms}}^{\theta s}\left(c\right))$. C is the number of bands, and S is the number of windows. $\mathrm{G}(·)$ represents the Gaussian degrading operation, and $u(·)$ is the unit step function.

We design two spectral loss terms $L_{\mathrm{spe}1}$ and $L_{\mathrm{spe}2}$ to preserve the spectral information. Since there is a lack of GT to evaluate the spectral distortion, following Wald’s protocol [46], we degrade ${\mathbf{I}}_{\mathrm{hrms}}^{\theta }$ first and then compute the spectral losses between the degraded ${\mathbf{I}}_{\mathrm{hrms}}^{\theta }$ and ${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$.

Due to the fluctuations in the spectrum, a small change in low-value bands can lead to significant spectral distortions, whereas the situation is reversed in high-value bands. To address this issue, we use the relative difference, which is defined as

$$L_{\mathrm{spe}1}={\displaystyle \frac{1}{C}}\sum _{c=1}^C{\displaystyle \frac{|\mathrm{D}\left(\mathrm{G}\left({\mathbf{I}}_{\mathrm{hrms}}^{\theta }\left(c\right)\right)\right)-\mathrm{D}\left({\mathbf{I}}_{\mathrm{Rms}}^{\theta }\left(c\right)\right)|}{\mathrm{MEAN}\left(\mathrm{D}\left({\mathbf{I}}_{\mathrm{Rms}}^{\theta }\left(c\right)\right)\right)}},$$

(27)

where $\mathrm{D}(·)$ means downsampling.

$L_{\mathrm{spe}2}$ is designed to finetune the shape of spectral curve, ensuring the pixelwise and local spectral similarity, which contains two components. The first one is the Spectral Angle Mapper (SAM) [47,48], which calculates the angle between two spectral vectors, $\mathbf{x}$ and $\mathbf{y}$,

$$\mathrm{SAM}(\mathbf{x},\mathbf{y})=arccos\left({\displaystyle \frac{\mathbf{x}·\mathbf{y}}{\parallel \mathbf{x}\parallel ·\parallel \mathbf{y}\parallel }}\right).$$

(28)

The second one is the structural Similarity Index Measure (SSIM) [49], which is an effective measure of similarity between two images, which is defined as

$$\mathrm{SSIM}(\mathbf{X},\mathbf{Y})={\displaystyle \frac{(2{\mu }_{\mathbf{X}}{\mu }_{\mathbf{Y}}+C_1)(2{\sigma }_{\mathbf{X}\mathbf{Y}}+C_2)}{({\mu }_{\mathbf{X}}^2+{\mu }_{\mathbf{Y}}^2+C_1)({\sigma }_{\mathbf{X}}^2+{\sigma }_{\mathbf{Y}}^2+C_2)}},$$

(29)

where ${\mu }_{\mathbf{X}}$ and ${\mu }_{\mathbf{Y}}$ are the averages of $\mathbf{X}$ and $\mathbf{Y}$, respectively. $C_1$ and $C_2$ are constants to avoid instability. SSIM not only accelerates the convergence but also ensures the local smoothness of the fused results.

Therefore, $L_{\mathrm{spe}2}$ is defined as

$$L_{\mathrm{spe}2}=\mathrm{SAM}(\mathrm{D}\left({\mathbf{I}}_{\mathrm{Rms}}^{\theta }\right),\mathrm{D}\left(\mathrm{G}\left({\mathbf{I}}_{\mathrm{hrms}}^{\theta }\right)\right))+1-\mathrm{SSIM}({\mathbf{I}}_{\mathrm{Rms}}^{\theta },\mathrm{G}\left({\mathbf{I}}_{\mathrm{hrms}}^{\theta }\right)).$$

(30)

3. Experiments

3.1. Data and Training Details

We conduct experiments on three satellite datasets with varying degrees of translation errors. GF2 and part of the WV2 images are sourced from [50]. Although our method is unsupervised, additional experiments are performed on reduced-resolution datasets, degraded according to Wald’s protocol [46] for evaluation purposes. Training samples are generated by cropping 16 × 16 patches from LRMS and 64 × 64 patches from HRPan. The details of the datasets are listed in

Table 2. Details of the datasets. We build full resolution and reduced resolution datasets on GF2, WV2, and GF1 images.

Sensor	Image Details			Resolution		Image Crop
	Bands	Bit	Displacement			Train	Test
GF2	4	10	[1.8 1.9]	Reduced	PAN:3.2m	64 × 64 × 1	512 × 512 × 1
					MS:12.8m	16 × 16 × 4	128 × 128 × 4
					Number	16,000	30
				Full	PAN:0.8m	64 × 64 × 1	512 × 512 × 1
					MS:3.2m	16 × 16 × 4	128 × 128 × 4
					Number	16,000	30
GF1	4	10	[0.6 0.8]	Reduced	PAN:8.0m	64 × 64 × 1	512 × 512 × 1
					MS:32.0m	16 × 16 × 4	128 × 128 × 4
					Number	15,616	27
				Full	PAN:2.0m	64 × 64 × 1	512 × 512 × 1
					MS:8.0m	16 × 16 × 4	128 × 128 × 4
					Number	15,616	27
WV2	4	11	[0.4 0.3]	Reduced	PAN:2.0m	64 × 64 × 1	512 × 512 × 1
					MS:8.0m	16 × 16 × 4	128 × 128 × 4
					Number	16,000	30
				Full	PAN:0.5m	64 × 64 × 1	512 × 512 × 1
					MS:2.0m	16 × 16 × 4	128 × 128 × 4
					Number	16,000	30

. It should be noted that the displacements listed in the table represent preliminary estimates obtained using the method described in [43].

The proposed method is implemented in Pytorch and is trained on a single NVIDIA GeForce RTX 2080Ti GPU. We use the Adam optimizer, the initial learning rate is set to 0.0002, and the decay rate is 0.97. The iteration is 50,000 in our experiment, and the batch size is 64. The trade-off parameters in the loss function are set as $w_1=100$, $w_2=4.5$, $w_3=4$, and $w_4=1$ for the WV2 dataset. The experiments for all methods are performed on the same datasets.

3.2. Experiment Design

The proposed method is compared with eleven pansharpening methods, including the following:

• Traditional methods: adaptive Gram-Schmidt (GSA) [6], spatial fidelity with nonlocal regression (SFNLR) [51], and the nonconvex framelet sparse reconstruction method (NC-FSRM) [52]; these methods are conducted without training.
• Supervised methods: diffusion model with disentangled modulations for sharpening multispectral and hyperspectral images (DDIF) [20], pansharpening GAN (PSGAN) [50], spatial–spectral dual back-project network (S2DBPN) [53] and dual-conditionally guided registration and fusion diffusion network (RFDifNet) [54]; these methods are trained on reduced resolution datasets and tested on both reduced and full resolution datasets. Among them, RFDifNet addresses the misalignment between HRPan and LRMS by explicitly estimating a deformation field to correct the spatial discrepancies between them.
• Unsupervised methods: zero-shot semi-supervised method for pansharpening (ZSPan) [55], pansharpening via deep image prior (PSDip) [56], $\lambda$-CNN-based pansharpening ($\lambda$-PNN) [16], unsupervised cycle-consistent generative adversarial network (UCGAN) [18]; these methods are trained and tested on reduced and full resolution datasets, respectively. Among them, $\lambda$-PNN handles the translation misalignment between HRPan and LRMS by applying multiple shifted versions of the inputs to approximate the spatial offsets between the two images.

To evaluate the performance of different methods, seven widely used metrics are used for quantitative evaluation. Three of these are no-reference metrics used for full resolution assessment, including the spectral distortion index ($D_{\lambda }$), spatial distortion index ($D_{\mathrm{s}}$), and quality with no-reference (QNR) [4,57]. Four of them are reference metrics used for reduced resolution assessment, including the erreur relative globale adimensionnelle de synthèse (ERGAS) [46], SSIM, SAM, and spatial correlation coefficient (SCC) [58]. Since translation errors exist between HRPan and LRMS, it is essential to account for their impact when computing evaluation metrics. Therefore, we compute the metrics using both the original (unaligned) and translation-corrected LRMS–HRPan pairs as reference images to provide a more comprehensive assessment.

3.3. Comparison on the GF2 Dataset

3.3.1. Visual Results

We test the performance of our method and the baseline methods on the GF2 dataset. Figure 7 and Figure 8 display the fusion results at full and reduced resolutions. As shown in Figure 7, noticeable displacements in the GF2 dataset adversely affect all methods that lack translation match, particularly traditional and unsupervised ones, leading to duplicated edges in the fusion results. While $\lambda$-PNN mitigates some artifacts through iterative shift-based alignment, it fails to fully eliminate them. RFDifNet, which estimates a deformation field to register HRPan and LRMS, also struggles with the large misalignment and introduces spatial distortions. In contrast, our method effectively suppresses artifacts by employing TEM, which accurately compensates for the spatial translation between HRPan and MS inputs. Notably, the impact of translation errors becomes less significant on the reduced-resolution dataset (Figure 8), as displacements diminish during image downsampling. Nevertheless, among traditional and unsupervised approaches, our method remains highly competitive, preserving the fine structural details of buildings with higher fidelity.

3.3.2. Quantitative Results

We provide quantitative comparisons on the GF2 dataset.

Table 3. Quantitative results on the GF2 dataset based on translation-corrected LRMS and HRPan. The best result in each group is in bold font.

Type	Model	Full-Resolution Dataset			Reduced-Resolution Dataset
		${\mathit{D}}_{\mathit{\lambda }}$	${\mathit{D}}_{\mathbf{s}}$	QNR	ERGAS	SSIM	SCC	SAM
Traditional	GSA	0.0897	0.0884	0.8348	1.5926	0.9206	0.9666	0.0288
	SFNLR	0.0577	0.1961	0.7577	2.6620	0.8631	0.9064	0.0393
	NC-FSRM	0.0629	0.2051	0.7454	2.4518	0.8838	0.9171	0.0372
Supervised	DDIF	0.0631	0.1194	0.8258	1.2568	0.9569	0.9770	0.0184
	S2DBPN	0.0573	0.1174	0.8327	1.2840	0.9550	0.9765	0.0205
	PSGAN	0.0598	0.1306	0.8183	1.3197	0.9552	0.9756	0.0187
	RFDifNet	0.0477	0.2580	0.7066	1.9688	0.8760	0.9392	0.0264
Unsupervised	ZSPan	0.1309	0.0992	0.7826	2.8301	0.8516	0.9095	0.0500
	PSDip	0.0737	0.2525	0.6923	3.7595	0.7951	0.8641	0.0442
	λ-PNN	0.0687	0.1081	0.8296	2.7382	0.8546	0.9029	0.0369
	UCGAN	0.0600	0.1267	0.8213	2.5537	0.8323	0.9073	0.0344
	SIUPan	0.0557	0.0521	0.8954	1.4554	0.9238	0.9673	0.0238

and

Table 4. Quantitative results on the GF2 dataset based on original LRMS and HRPan. The best result in each group is in bold font.

Type	Model	Full-Resolution Dataset			Reduced-Resolution Dataset
		${\mathit{D}}_{\mathit{\lambda }}$	${\mathit{D}}_{\mathbf{s}}$	QNR	ERGAS	SSIM	SCC	SAM
Traditional	GSA	0.0872	0.0641	0.8589	1.9471	0.8954	0.9504	0.0324
	SFNLR	0.0532	0.2210	0.7376	2.3560	0.8748	0.9274	0.0364
	NC-FSRM	0.0590	0.2300	0.7249	2.1972	0.8885	0.9347	0.0348
Supervised	DDIF	0.0585	0.1293	0.8204	0.6769	0.9817	0.9928	0.0140
	S2DBPN	0.0530	0.1298	0.8245	0.8675	0.9726	0.9892	0.0180
	PSGAN	0.0556	0.1442	0.8089	0.7950	0.9780	0.9914	0.0152
	RFDifNet	0.0435	0.2840	0.6848	2.0089	0.8889	0.9416	0.0293
Unsupervised	ZSPan	0.1269	0.1058	0.7801	2.7796	0.8525	0.9132	0.0496
	PSDip	0.0697	0.2779	0.6714	3.3020	0.8325	0.9026	0.0409
	$\lambda$-PNN	0.0658	0.1247	0.8163	2.4448	0.8636	0.9211	0.0342
	UCGAN	0.0566	0.1493	0.8027	2.7057	0.8077	0.9006	0.0369
	SIUPan	0.0520	0.0642	0.8868	1.7824	0.9015	0.9541	0.0282

show the quantitative results based on aligned and original image pairs, respectively. We emphasize that Table 3 more accurately reflects the fusion quality, as it minimizes the influence of translation misalignments. As shown in Table 3, most methods yield satisfactory results in terms of $D_{\lambda }$, with our method outperforming all other unsupervised methods. Benefiting from MSDI, our approach achieves a better balance between spectral preservation and spatial fidelity. Consequently, ours attains the best performance not only on $D_{\mathrm{s}}$ but also on QNR. For reference metrics, supervised methods naturally perform better due to the availability of GT, while our approach remains highly competitive among unsupervised and traditional methods.

When comparing Table 3 and Table 4, we observe that some methods (e.g., DDIF, ZSPan) achieve better performance when evaluated on the original unregistered pairs because they are trained and inferred on unregistered data, causing their outputs to preserve the same geometric misalignment. In contrast, our method performs worse under this setting because it explicitly aligns HRPan and LRMS, producing fused results that are more consistent with the registered references. This also indicates that evaluation metrics become unreliable when misalignment exists between HRPan and LRMS.

3.4. Comparison on the GF1 Dataset

3.4.1. Visual Results

We further evaluate the proposed method on the GF1 dataset. Figure 9 and Figure 10 present the fusion results at full and reduced resolutions, respectively. Although the GF1 dataset contains only sub-pixel translation errors (typically <1 pixel), these small misalignments still induce noticeable artifacts in the fusion results of methods that do not explicitly handle translation errors (Figure 9). Our method, along with RFDifNet and $\lambda$-PNN, effectively compensates for these errors and better preserves spectral integrity at land-cover boundaries. In contrast, supervised methods exhibit limited capability in preserving fine spatial details at full resolution, leading to noticeable boundary blurring (Figure 9). However, our method achieves an effective balance between spatial and spectral fidelity across both resolutions, outperforming other methods in preserving fine structures.

3.4.2. Quantitative Results

Table 5. Quantitative results on the GF1 dataset based on translation-corrected LRMS and HRPan. The best result in each group is in bold font.

Type	Model	Full-Resolution Dataset			Reduced-Resolution Dataset
		${\mathit{D}}_{\mathit{\lambda }}$	${\mathit{D}}_{\mathbf{s}}$	QNR	ERGAS	SSIM	SCC	SAM
Traditional	GSA	0.0675	0.0672	0.8717	1.3197	0.9208	0.9627	0.0291
	SFNLR	0.0425	0.0668	0.8936	1.7499	0.9028	0.9401	0.0344
	NC-FSRM	0.0445	0.0698	0.8891	1.6488	0.9125	0.9435	0.0331
Supervised	DDIF	0.0442	0.0880	0.8720	0.7454	0.9729	0.9871	0.0160
	S2DBPN	0.0421	0.0629	0.8979	0.8622	0.9642	0.9831	0.0190
	PSGAN	0.0393	0.0841	0.8800	0.8508	0.9667	0.9843	0.0184
	RFDifNet	0.0416	0.0631	0.8980	1.0630	0.9547	0.9760	0.0216
Unsupervised	ZSPan	0.0551	0.0723	0.8777	1.6870	0.8811	0.9295	0.0394
	PSDip	0.0498	0.0837	0.8707	2.0766	0.8707	0.9148	0.0397
	$\lambda$-PNN	0.0616	0.0669	0.8766	1.4342	0.9162	0.9542	0.0283
	UCGAN	0.0402	0.0687	0.8945	1.8216	0.8817	0.9357	0.0395
	SIUPan	0.0419	0.0520	0.9086	1.3738	0.9125	0.9609	0.0283

and

Table 6. Quantitative results on the GF1 dataset based on original LRMS and HRPan. The best result in each group is in bold font.

Type	Model	Full-Resolution Dataset			Reduced-Resolution Dataset
		${\mathit{D}}_{\mathit{\lambda }}$	${\mathit{D}}_{\mathbf{s}}$	QNR	ERGAS	SSIM	SCC	SAM
Traditional	GSA	0.0667	0.0692	0.8706	1.4269	0.9133	0.9558	0.0312
	SFNLR	0.0423	0.0661	0.8945	1.9427	0.8891	0.9267	0.0376
	NC-FSRM	0.0442	0.0691	0.8900	1.8574	0.8967	0.9295	0.0364
Supervised	DDIF	0.0440	0.0873	0.8728	0.6684	0.9743	0.9882	0.0154
	S2DBPN	0.0418	0.0624	0.8985	0.8545	0.9615	0.9824	0.0197
	PSGAN	0.0392	0.0832	0.8811	0.7770	0.9680	0.9852	0.0180
	RFDifNet	0.0413	0.0619	0.8994	1.0244	0.9540	0.9769	0.0219
Unsupervised	ZSPan	0.0761	0.0549	0.8736	1.7844	0.8737	0.9221	0.0411
	PSDip	0.0495	0.0824	0.8721	2.3010	0.8511	0.8970	0.0431
	$\lambda$-PNN	0.0613	0.0681	0.8757	1.6296	0.9048	0.9435	0.0318
	UCGAN	0.0401	0.0681	0.8952	1.9599	0.8668	0.9274	0.0418
	SIUPan	0.0415	0.0526	0.9083	1.4971	0.9063	0.9542	0.0312

present the quantitative results based on the translation-corrected and original images, respectively. As shown in Table 5, supervised methods perform strongly on $D_{\lambda }$, with PSGAN achieving the best overall results. This is because the GF1 dataset is dominated by large homogeneous land-cover regions (e.g., farmland), where the spectral difference between full- and reduced-resolution images is relatively small [59]. Among unsupervised methods, UCGAN achieves the best $D_{\lambda }$, while our method ranks second. However, UCGAN introduces obvious spectral distortions along linear structures such as roads (Figure 9m). Our method achieves the best $D_{\mathrm{s}}$ and QNR, demonstrating its strong spatial reconstruction capability and practical potential.

When comparing Table 5 and Table 6, we observe that the no-reference metrics change only slightly, whereas the reference-based metrics exhibit more noticeable variations. For the no-reference metrics, the differences between the results computed using translation-corrected and original pairs are much smaller on GF1 than on GF2. This further supports our explanation that such improvements on unregistered data mainly stem from training and generating results on unregistered inputs. By comparing the behaviors of both the no-reference and reference metrics across the two tables, we can also find that the no-reference metrics are less sensitive to sub-pixel translation errors.

3.5. Comparison on the WV2 Dataset

3.5.1. Visual Results

Figure 11 and Figure 12 illustrate the fusion results on the WV2 dataset at full and reduced resolutions, respectively. As the displacements between HRPan and MS are minimal (<0.5 pixels in average), no noticeable artifacts appear in the fusion results across all methods. Among unsupervised methods, UCGAN preserves spectral fidelity well but exhibits evident spatial distortion, due to the difficulty in balancing spectral and spatial fidelity. On the reduced resolution dataset, supervised methods still show a significant advantage. Compared with unsupervised and traditional methods, our method achieves better results at both full and reduced resolution.

3.5.2. Quantitative Results

Table 7. Quantitative results on the WV2 dataset based on translation-corrected LRMS and HRPan. The best result in each group is in bold font.

Type	Model	Full-Resolution Dataset			Reduced-Resolution Dataset
		${\mathit{D}}_{\mathit{\lambda }}$	${\mathit{D}}_{\mathbf{s}}$	QNR	ERGAS	SSIM	SCC	SAM
Traditional	GSA	0.0492	0.0603	0.8946	1.5177	0.9532	0.9723	0.0375
	SFNLR	0.0525	0.0553	0.8955	2.1754	0.9306	0.9469	0.0401
	NC-FSRM	0.0514	0.0531	0.8987	2.5857	0.9075	0.9253	0.0460
Supervised	DDIF	0.0472	0.0601	0.8967	1.0864	0.9772	0.9863	0.0212
	S2DBPN	0.0485	0.0540	0.9029	1.1842	0.9729	0.9831	0.0254
	PSGAN	0.0497	0.0591	0.8965	1.0987	0.9762	0.9858	0.0224
	RFDifNet	0.0440	0.0582	0.9005	1.1228	0.9749	0.9840	0.0227
Unsupervised	ZSPan	0.0886	0.0808	0.8400	2.8627	0.9274	0.9295	0.0448
	PSDip	0.0515	0.0782	0.8748	2.7615	0.9004	0.9149	0.0482
	$\lambda$-PNN	0.0618	0.0601	0.8833	2.1212	0.9413	0.9531	0.0391
	UCGAN	0.0509	0.0945	0.8610	2.8051	0.8678	0.9109	0.0471
	SIUPan	0.0472	0.0511	0.9050	1.6833	0.9533	0.9684	0.0358

and

Table 8. Quantitative results on the WV2 dataset based on original LRMS and HRPan. The best result in each group is in bold font.

Type	Model	Full-Resolution Dataset			Reduced-Resolution Dataset
		${\mathit{D}}_{\mathit{\lambda }}$	${\mathit{D}}_{\mathbf{s}}$	QNR	ERGAS	SSIM	SCC	SAM
Traditional	GSA	0.0487	0.0688	0.8870	1.8513	0.9362	0.9606	0.0423
	SFNLR	0.0510	0.0522	0.9000	2.5727	0.9123	0.9277	0.0454
	NC-FSRM	0.0498	0.0515	0.9017	2.5857	0.9075	0.9253	0.0460
Supervised	DDIF	0.0471	0.0590	0.8983	0.9485	0.9802	0.9883	0.0221
	S2DBPN	0.0486	0.0562	0.9015	1.1909	0.9709	0.9827	0.0273
	PSGAN	0.0497	0.0588	0.8976	1.0521	0.9766	0.9859	0.0239
	RFDifNet	0.0439	0.0562	0.9028	1.0787	0.9750	0.9842	0.0240
Unsupervised	ZSPan	0.0872	0.0786	0.8430	3.1150	0.9094	0.9137	0.0487
	PSDip	0.0504	0.0737	0.8801	3.2033	0.8725	0.8869	0.0539
	$\lambda$-PNN	0.0624	0.0618	0.8820	2.5124	0.9240	0.9346	0.0447
	UCGAN	0.0489	0.0867	0.8699	3.1174	0.8437	0.8931	0.0516
	SIUPan	0.0473	0.0526	0.9040	2.0620	0.9366	0.9535	0.0412

present the quantitative results based on the translation-corrected and original images, respectively. As shown in Table 7, on the full-resolution dataset, our method achieves the best performance in both $D_{\mathrm{s}}$ and QNR and ranks second in $D_{\lambda }$. Although RFDifNet attains the best $D_{\lambda }$, its $D_{\mathrm{s}}$ is considerably worse, indicating spatial distortions. On the reduced-resolution dataset, supervised methods maintain superiority. Among unsupervised methods, our method achieves the best performance across all metrics, effectively preserving both spatial and spectral information, which also demonstrates the powerful generation capability of diffusion models.

Comparing Table 7 and Table 8, we observe that differences still exist even under small displacements (<0.5 pixel), indicating that even slight misalignment can influence the evaluation of the fusion quality.

3.6. Ablation

3.6.1. TEM

In the proposed method, TEM is an essential module to match HRPan and MS; thus, we conduct ablation experiments on the GF2 and WV2 datasets. To demonstrate the capability of TEM, we perform GSA using ${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$, as well as ${\mathbf{I}}_{\mathrm{pan}}$ and ${\mathbf{I}}_{\mathrm{ms}}$, respectively, and the quantitative results are tabulated in

Table 9. Quantitative assessment of translation correction on GF2 and WV2 datasets. GSA is used to fuse HRPan and MS.

Dataset	MS	GSA Fusion Results
		${\mathit{D}}_{\mathit{\lambda }}$	${\mathit{D}}_{\mathbf{s}}$	QNR
GF2	${\mathbf{I}}_{\mathrm{ms}}$	0.0897	0.0884	0.8348
	${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$	0.0800	0.0748	0.8553
WV2	${\mathbf{I}}_{\mathrm{ms}}$	0.0492	0.0603	0.8946
	${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$	0.0485	0.0595	0.8960

. For the GF2 dataset, the quality of the GSA fusion results improves significantly after translation correction. In contrast, for the WV2 dataset, only slight improvements are observed. These results indicate that TEM successfully maps HRPan and MS into a shared feature space, effectively reducing translation errors.

To verify the validity of TEM in SIUPan, we remove it from the proposed network and then conduct comparative experiments. As TEM is excluded, ${\mathbf{I}}_{\mathrm{Rms}}^{\theta }$ is unavailable, and thus $L_{\mathrm{R}}$ cannot be computed, requiring its omission. Figure 13 shows the fused images before and after removing TEM, while

Table 10. Quantitative assessment of TEM ablation study on GF2 and WV2 datasets.

Dataset	TEM	Full-Resolution Datasets			Reduced-Resolution Dataset
		${\mathit{D}}_{\mathit{\lambda }}$	${\mathit{D}}_{\mathbf{s}}$	QNR	ERGAS	SSIM	SCC	SAM
GF2	w/o	0.0607	0.1041	0.8409	2.3472	0.8749	0.9298	0.0337
	w/	0.0557	0.0521	0.8954	1.4554	0.9238	0.9673	0.0238
WV2	w/o	0.0481	0.0524	0.9027	1.8962	0.9482	0.9625	0.0386
	w/	0.0472	0.0511	0.9050	1.6833	0.9533	0.9684	0.0358

provides the corresponding quantitative evaluation.

As shown in Figure 13a,b, for the GF2 dataset, removing the TEM leads to pronounced spatial distortions and noticeable artifacts, severely degrading the fusion quality. For the WV2 dataset, where translation errors are minimal, no apparent visual differences are observed in Figure 13c,d; however, the quantitative metrics still show a consistent decline (Table 10). These findings demonstrate that even sub-pixel translation discrepancies can negatively affect the pansharpening performance, highlighting the necessity of the TEM for achieving robust and high-fidelity fusion in SIUPan.

3.6.2. ${X̂}_0$

In SIUPan, we replace $x_0$ with ${x̂}_0$. We assume that the difference between $x_0$ and ${x̂}_0$ follows a zero-mean Gaussian distribution. This assumption is relatively mild, implying that our method should be insensitive to the choice of ${x̂}_0$. To validate this, we conduct experiments using four different estimates of ${x̂}_0$: (1) ${\mathbf{I}}_{\mathrm{ones}}$, (2) ${\mathbf{I}}_{\mathrm{pan}}^C$, (3) ${\mathbf{I}}_{\mathrm{GSA}}$, and (4) GT. Here, ${\mathbf{I}}_{\mathrm{ones}}$ denotes an all-ones matrix, while ${\mathbf{I}}_{\mathrm{GSA}}$ represents the fusion result obtained by the GSA method. Additionally, since GT is only available on reduced resolution datasets, we conducted experiments for the fourth case only at reduced resolution. The results are shown in

Table 11. Quantitative assessment of ${x̂}_0$ ablation study on the WV2 dataset.

${x̂}_0$	Full-Resolution Dataset			Reduced-Resolution Dataset
	${\mathit{D}}_{\mathit{\lambda }}$	${\mathit{D}}_{\mathbf{s}}$	QNR	ERGAS	SSIM	SCC	SAM
${\mathbf{I}}_{\mathrm{ones}}$	0.0491	0.0545	0.9006	1.7288	0.9516	0.9669	0.0364
${\mathbf{I}}_{\mathrm{pan}}^C$	0.0472	0.0511	0.9050	1.6833	0.9533	0.9684	0.0358
${\mathbf{I}}_{\mathrm{GSA}}$	0.0475	0.0519	0.9041	1.6856	0.9531	0.9687	0.0356
GT	-	-	-	1.6889	0.9530	0.9682	0.0357

.

As tabulated in Table 11, ${\mathbf{I}}_{\mathrm{ones}}$ results in a relatively poor performance, because ${\mathbf{I}}_{\mathrm{ones}}-{\mathbf{I}}_{\mathrm{hrms}}$ deviates from the zero-mean assumption in (4). In contrast, the other three estimates produce comparable results, demonstrating that the proposed method is largely insensitive to the choice of ${x̂}_0$. Therefore, considering both the accuracy and efficiency, we use ${\mathbf{I}}_{\mathrm{pan}}^C$ as ${x̂}_0$ in our method.

3.6.3. MSDI

MSDI is designed to inject spatial and spectral information into the encoder when generating HRMS in our work, enabling it to achieve strong performance even on small-scale datasets. To evaluate the effectiveness of the MSDI, we conduct ablation experiments on the WV2 dataset. The MSDI employs cross-attention at multiscale to fully exploit the information contained in HRPan and MS, balancing the spatial and spectral details in the fusion results. As shown in

Table 12. Quantitative assessment of MSDI ablation study on the WV2 dataset.

MSDI	Full-Resolution Dataset			Reduced-Resolution Dataset
	${\mathit{D}}_{\mathit{\lambda }}$	${\mathit{D}}_{\mathbf{s}}$	QNR	ERGAS	SSIM	SCC	SAM
w/o	0.0470	0.1154	0.8440	2.7202	0.8667	0.9152	0.0396
w/	0.0472	0.0511	0.9050	1.6833	0.9533	0.9684	0.0358

, removing the MSDI leads to a noticeable performance drop on the WV2 dataset. In particular, $D_{\mathrm{s}}$ degrades substantially, indicating that without the MSDI, the fused images suffer from spatial distortions. These results verify that the MSDI plays a crucial role in enhancing the spatial consistency and overall fusion quality of SIUPan.

3.6.4. Loss Terms

There are four loss terms in the proposed method, three of which are fusion losses and one of which is registration loss. To validate the necessity of each component, we conduct ablation experiments, and the quantitative results are summarized in

Table 13. Quantitative assessment of different loss functions on the WV2 dataset.

Loss Term				Full-Resolution Dataset			Reduced-Resolution Dataset
${\mathit{L}}_{\mathbf{R}}$	${\mathit{L}}_{\mathbf{spa}}$	${\mathit{L}}_{\mathbf{spe}\mathbf{1}}$	${\mathit{L}}_{\mathbf{spe}\mathbf{2}}$	${\mathit{D}}_{\mathit{\lambda }}$	${\mathit{D}}_{\mathbf{s}}$	QNR	ERGAS	SSIM	SCC	SAM
×	√	√	√	0.0459	0.0536	0.9044	1.7838	0.9455	0.9654	0.0355
√	×	√	√	0.1367	0.6200	0.3307	51.8171	0.0429	0.1461	0.0845
√	√	×	√	0.0542	0.0629	0.8871	1.5635	0.9548	0.9701	0.0362
√	√	√	×	0.0508	0.0546	0.8984	1.7283	0.9520	0.9670	0.0360
√	√	√	√	0.0472	0.0511	0.9050	1.6833	0.9533	0.9684	0.0358

. As shown in Table 13, removing $L_{\mathrm{spa}}$ leads to severely degraded fusion results, as it is the only loss directly constraining the spatial fidelity. The absence of $L_{\mathrm{spe}1}$ causes the model to fail in achieving the best results at full resolution. The loss term $L_{\mathrm{spe}2}$ further refines the spectral curve and helps balance the fusion quality across multiple scales; its removal results in moderate degradation in all metrics. With $L_{\mathrm{R}}$, the TEM provides more accurate translation error estimation between HRPan and MS, leading to improved performance on $D_s$. According to the experiment results, each term of the loss function ensures that the model can achieve good results at both full and reduced resolutions.

4. Conclusions

In this article, we propose a shift-invariant unsupervised pansharpening network. In the proposed method, a translation error estimation module, i.e., TEM, is employed to estimate the displacements between HRPan and MS. Moreover, a multiscale detail injection module is designed to balance the spatial and spectral fidelity. Extensive experiments on the GF2, GF1, and WV2 datasets demonstrate that the proposed method outperforms baseline pansharpening approaches. The ablation studies further confirm that each designed module and loss function plays an essential role in the overall performance of the model.

Compared to supervised methods, unsupervised methods can exploit the original information of HRPan and MS. Nevertheless, because GT is absent, its spectral and spatial losses are computed independently, highlighting the importance of accurately handling translation errors between HRPan and MS. Furthermore, our work primarily focuses on translation errors between HRPan and MS, while potential misalignments among individual MS bands also deserve further study.