DSEPGAN: A Dual-Stream Enhanced Pyramid Based on Generative Adversarial Network for Spatiotemporal Image Fusion

Highlights

What are the main findings?

• A Dual-Stream Enhanced Pyramid based on GAN (DSEPGAN) is proposed for spatiotemporal fusion of remote sensing images.
• The model integrates reversible detail preservation and large-kernel feature reconstruction to enhance fine spatial details.

What are the implications of the main finding?

• DSEPGAN significantly improves detail and edge restoration in regions with pronounced phenological and land-cover changes, ensuring high-fidelity spatiotemporal reconstruction.
• The dual-stream reversible pyramid design provides a new framework for multi-modal image fusion and change analysis.

Abstract

Many deep learning-based spatiotemporal fusion (STF) methods have been proven to achieve high accuracy and robustness. Due to the variable shapes and sizes of objects in remote sensing images, pyramid networks are generally introduced to extract multi-scale features. However, the down-sampling operation in the pyramid structure may lead to the loss of image detail information, affecting the model’s ability to reconstruct fine-grained targets. To address this issue, we propose a novel Dual-Stream Enhanced Pyramid based on Generative Adversarial Network (DSEPGAN) for the spatiotemporal fusion of remote sensing images. The network adopts a dual-stream architecture to separately process coarse and fine images, tailoring feature extraction to their respective characteristics: coarse images provide temporal dynamics, while fine images contain rich spatial details. A reversible feature transformation is embedded in the pyramid feature extraction stage to preserve high-frequency information, and a fusion module employing large-kernel and depthwise separable convolutions captures long-range dependencies across inputs. To further enhance realism and detail fidelity, adversarial training encourages the network to generate sharper and more visually convincing fusion results. The proposed DSEPGAN is compared with widely used and state-of-the-art STF models in three publicly available datasets. The results illustrate that DSEPGAN achieves superior performance across various evaluation metrics, highlighting its notable advantages for predicting seasonal variations in highly heterogeneous regions and abrupt changes in land use.

1. Introduction

Spatiotemporal fusion (STF) [1,2] aims to integrate multi-source remote sensing images with complementary temporal and spatial resolutions to generate high-spatiotemporal-resolution products for diverse applications, including surface urban heat islands [3], evapotranspiration estimation [4], grassland biomass estimation [5], crop phenology analysis [6], and other fields. STF typically relies on coarse images that provide rich temporal dynamics and fine images that contain abundant spatial details, making effective modeling of their heterogeneous information a central challenge.

Traditional STF methods, including unmixing-based [7,8], weight function-based [9,10,11], Bayesian [12,13], and machine learning-based methods [14,15], often depend on assumptions such as linear spectral mixing or linear temporal change, which are difficult to satisfy in complex scenes. Deep learning (DL)-based methods [16,17,18] alleviate these issues by learning nonlinear relationships directly from data. The Convolutional Neural Network (CNN) is the most commonly used DL architecture. Early CNN-based models (e.g., STFDCNN [19]) incorporate hand-crafted modulation strategies, while recent end-to-end frameworks (e.g., EDCSTFN [20]) directly extract and fuse spatiotemporal features. However, fixed-size convolutional kernels of typically 3 × 3 may not adapt well to the irregular shapes and varying sizes of features in remote sensing images, potentially leading to insufficient local feature extraction, information loss, distortion, and blurring.

Therefore, the multiscale mechanism in DL has been introduced into STF. Recent work introduces multi-branch kernels [21], dilated convolutions [22], and pyramid-based designs [23,24]. Pyramid structures, named for their pyramid-like shape, in which the bottom is larger in scale and the top is smaller, can capture information at different resolutions. Multiple downsampling operations using stride-2 convolutions are applied, progressively reducing the size of the feature maps. Although the pyramid framework is effective for capturing multi-scale features, it often leads to the degradation of high-frequency spatial details. This issue is primarily attributable to two compounded factors: the progressive downsampling operations, which introduce an irreversible information bottleneck, and the resulting deep-layer features, which lack the fidelity required for precise texture reconstruction due to high compression. This structural fidelity loss, when combined with the inherent smoothing effect of the Mean Squared Error (MSE) loss function, ultimately results in severe degradation of texture details and a lack of realism in the fused imagery. Generative Adversarial Network (GAN)-based STF models (e.g., GAN-STFM [25], MLFF-GAN [23]) leverage adversarial learning to enhance perceptual sharpness. However, these approaches typically apply the uniform feature extractor to fine and coarse inputs, neglecting their different information characteristics—coarse images emphasize temporal dynamics, while fine images provide spatial structure. This uniform treatment can hinder feature modeling and limit fusion performance. Thus, there is a clear need for an advanced pyramid architecture that can both mitigate the high-frequency detail loss from downsampling and apply tailored, asymmetric extraction strategies to the distinct coarse and fine inputs. This motivates our work on a dual-stream, detail-preserving pyramid structure.

Effectively fusing the multiscale information extracted by the pyramid architecture constitutes another key challenge. Most existing models resort to simple concatenation or element-wise addition followed by standard 1 × 1 or 3 × 3 convolutions, which have limited receptive fields. Reconstructing fine pixel-level details, however, often requires contextual information from a substantially wider spatial area, particularly in highly heterogeneous regions characterized by complex and fragmented landscapes. Transformer-based STF models [26,27], such as STF-Trans [28], address this limitation but at a substantially higher computational cost, thereby limiting their scalability to large remote sensing scenes. Recent advances in large-kernel CNNs [29,30] provide a more efficient alternative, substantially expanding the receptive field while preserving computational practicality.

Building upon the aforementioned analysis, we propose a Dual-Stream Enhanced Pyramid based on Generative Adversarial Network (DSEPGAN) for the Spatiotemporal Fusion of remote sensing images. The primary methodological innovations and contributions of this work are summarized as follows:

• A Dual-Stream Decoupling Framework: We introduce the Pyramid Time Change Extractor (PTCE) and the Pyramid Space Detail Extractor (PSDE), creating a specialized architecture that explicitly decouples the extraction of temporal dynamics from coarse imagery and high-frequency spatial structure from fine imagery. This asymmetric approach ensures dedicated modeling tailored to the heterogeneous nature of multi-sensor inputs.
• Structurally Lossless Spatial Detail Preservation: To fundamentally address the irreversible information bottleneck caused by downsampling, the PSDE is customized with a novel detail-preserving strategy that integrates Affine Coupling Layers and Patch Merging operations. This combination maximizes the fidelity of extracted features, which is crucial for reconstructing sharp texture and edge details in STF products.
• Hierarchical Long-Range Feature Aggregation (HLRFA): We designed the HLRFA module, incorporating advanced feature fusion techniques such as Large Kernel Fusion Blocks. This mechanism efficiently combines segregated temporal and spatial features across multiple scales, substantially expanding the receptive field to capture necessary long-range contextual information without incurring the heavy computational burden associated with Transformer-based models.
• Superior Fusion Performance: Through comprehensive experiments on three benchmark datasets, DSEPGAN demonstrates significant improvements over existing state-of-the-art models, achieving superior performance in both spatial fidelity (sharpness and texture preservation) and spectral consistency (radiometric accuracy), validated by both objective metrics and visual quality assessment.

The remainder of the article is structured as follows: Section 2 reviews the related work. Section 3 introduces the proposed DSEPGAN method in detail. Section 4 evaluates the performance of the proposed approach in predicting both seasonal variations and abrupt changes. Section 5 discusses model efficiency, ablation experiments, and parameter analysis. Finally, Section 6 concludes the study.

2. Related Work

2.1. Multiscale Mechanisms in STF

Multiscale mechanisms used in DL-based STF models can be divided into three categories: (1) Parallel usage of convolutional kernels of different sizes to flexibly perceive multiscale information from feature maps. For example, DMNet [21] simultaneously employs convolutional kernels of sizes 3, 5, and 7. However, increasing the size of the kernels also leads to an increase in parameters and computational load. (2) Using dilated convolutions with different dilation rates can effectively increase the network’s receptive field without introducing additional parameters, such as MANet [22], PDCNN [31], and STFMCNN [32]. (3) Using a pyramid structure, which progressively reduces spatial resolution to expand the receptive field and significantly reduces computational cost. For example, the encoders of MLFF-GAN [23], PSTAF-GAN [33], and DCDGAN-STF [34] employ a cascade of residual blocks and downsampling layers. However, this reliance on standard downsampling operations introduces an irreversible information bottleneck, severely compromising the fidelity of high-frequency spatial details in the generated features. This deficiency motivates the subtle combination of a reversible feature transformation strategy and the pyramid structure.

2.2. Long-Range Dependency Modeling in STF

Long-range spatial dependency modeling is essential in STF. Transformer architectures have been introduced into STF to model Long-range spatial dependency. For instance, SwinSTFM [35] achieves robust spatiotemporal feature modeling by integrating a window-based self-attention mechanism [36] with linear unmixing theory. STF-Trans [28] adopts a two-stage strategy where CNNs handle initial (shallow) feature extraction, while a Vision Transformer module is subsequently employed for deeper modeling. CTSTFM [37] focuses on efficient feature processing by incorporating spatial and channel attention modules alongside cross-attention mechanisms to enhance both extraction and fusion stages. However, their high computational cost stems primarily from the quadratic scaling of self-attention. Furthermore, while focusing on the global context, these models can sometimes neglect the preservation of fine local details. Consequently, a novel solution is urgently required to effectively model long-range dependencies while maintaining computational equilibrium.

3. Proposed Methods

3.1. Pyramid Generator

The core difficulty in STF lies in addressing the asymmetry in input data (a high-resolution image is available only at the reference time) and ensuring maximal preservation of spatial details. To tackle these challenges, we designed the DSEPGAN generator based on three core principles: Decoupling, Preservation, and Efficiency.

As shown in Figure 1, first, a Dual-Stream Encoder is employed to decouple feature extraction, ensuring that time change information and spatial detail are handled specifically. The Pyramid Time Change Extractor (PTCE) is dedicated to capturing the time change from coarse images ($C_0 \mathrm{and} C_1$), while the Pyramid Space Detail Extractor (PSDE) is optimized for extracting and preserving the high-frequency spatial information from the fine image ($F_0$). Second, Invertible Neural Networks (INNs) are introduced into the PSDE to enable theoretically lossless feature transformation, which is crucial for maximizing the fidelity of limited high-resolution spatial details. Third, feature aggregation is performed by the Hierarchical Long-Range Feature Aggregation (HLRFA) module, which efficiently models necessary contextual dependencies across multiple scales using large kernels, avoiding the high computational cost of global attention.

The overall fusion process involves four stages: (1) Decoupled Extraction: PTCE processes $C_0$ and $C_1$ to extract four multi-scale features $C_0^{\left(i\right)}$ and $C_1^{\left(i\right)}$ ($i=1,2,3,4$). PSDE processes $F_0$ to extract four detailed features $D^{\left(i\right)}$ ($i=1,2,3,4$). (2) Initial Fusion: The deepest features $C_0^{\left(4\right)}, C_1^{\left(4\right)},$ and $D^{\left(4\right)}$ first enter the Large Kernel Fusion Block for initial fusion. (3) Hierarchical Aggregation: These fused features, along with the temporal change features $C_0^{\left(i\right)}$, $C_1^{\left(i\right)}$ and spatial detail features $D^{\left(i\right)}$ from remaining stages, sequentially undergo HLRFA fusion from stage 3 to stage 1. (4) Reconstruction: After the final aggregation stage, the features are recovered to the original input size and passed through three convolutional layers to reconstruct the final high-resolution fusion result ${\widehat{F}}_1$.

3.1.1. Pyramid Time Change Extractor

Considering that it is difficult to represent more comprehensive information of remote sensing images with single-scale features, a Pyramid Time Change Extractor is designed. Crucially, the PTCE employs a Siamese network structure with shared weights. This architectural choice ensures that the consistency and discrepancies in features between coarse images are maximized for comparison, allowing us to effectively capture the temporal change information that occurred between the two time points.

As shown in Figure 2, the extractor consists of four convolutional blocks stacked together, and each convolutional block consists of a convolutional layer and a LeakyRelu activation function, where (3,3) denotes a convolution kernel of size 3, and Stride denotes the step size of the convolution operation. To extract features at various scales, the Stride is set to 2 starting from the second convolutional block, thereby creating a pyramidal structure for the features. Each layer of features must be output to the subsequent HLRFA modules. By adopting this structure, firstly, features of different scales and depths relevant to the temporal context can be extracted, which is more important for remote sensing images with more complex scenes. Secondly, reducing feature size also reduces computational cost, enabling efficient extraction of multi-scale temporal features.

3.1.2. Pyramid Space Detail Extractor

In STF, most high-frequency information—such as edges and textures—comes from the fine image at the reference time. Since these details are essential for improving the fidelity of the predicted fine image, accurately preserving them is crucial for STF. The invertible mapping design in Invertible Neural Networks (INNs) [38,39] enables theoretically lossless transformations of features at each layer, allowing the network to preserve information in feature representations better. They have been applied in image super-resolution [40], image denoising [41], image hiding [42], and multimodal image fusion [38,39].

The preservation capability is fundamentally critical for STF because the problem is ill-posed and information-deficient, relying solely on the single high-resolution input ($F_0$). Standard non-invertible convolutions, while effective for feature extraction, inherently lead to information degradation and detail blurring. Mathematically, the invertibility ensures that the Jacobian determinant of the transformation matrix is maintained close to unity ($det\left(J\right)\approx 1$). This property guarantees the preservation of volume in the latent feature space, meaning the limited, high-frequency spatial information (such as edges and textures) is transferred across the deep network layers without being compressed or destroyed. This maximal fidelity is key to improving the structural quality of the final fused image.

The INN is invertible, meaning the network’s input and output can generate each other without any loss of information, making it particularly suitable for tasks requiring the preservation of image integrity. The invertible layers of INN perform specific transformation operations. The affine coupling layer [43,44] employed in this study first divides the input feature tensor ${\mathbf{Z}}_l$ into two parts along the channel dimension, $Z_l^{\left(1:c\right)}$ and $Z_l^{\left(c+1:C\right)}$ (Here, $C$ is the total number of channels, and $c$ is the split point). An additive transformation is applied to $Z_l^{\left(c+1:C\right)}$ and an affine transformation is applied to $Z_l^{\left(1:c\right)}$. Finally, the two transformed feature parts are concatenated along the channel dimension to form the output tensor $Z_{l+1}$. The forward propagation of this process is formulated as follows:

$$Z_{l+1}^{\left(c+1:C\right)}=Z_l^{\left(c+1:C\right)}+\varphi \left(Z_l^{\left(1:c\right)}\right)$$

(1)

$$Z_{l+1}^{\left(1:c\right)}=Z_l^{\left(1:c\right)}\odot \exp \left(\rho \left(Z_{l+1}^{\left(c+1:C\right)}\right)\right)+\eta \left(Z_{l+1}^{\left(c+1:C\right)}\right)$$

(2)

$$Z_{l+1}=\mathrm{cat}\left(Z_{l+1}^{\left(1:c\right)},Z_{l+1}^{\left(c+1:C\right)}\right)$$

(3)

where $\odot$ is the Hadamard product. $\exp \left(·\right)$ is the exponential function. $\rho \left(·\right),$ $\eta \left(·\right)$ and $\varphi \left(·\right)$ are the arbitrary mapping functions which are not necessarily invertible, so it is possible to implement them via a neural network. $\mathrm{cat}\left(\cdot \right)$ denotes the concatenation function along the feature dimension. The inverse propagation of this transformation, which reconstructs the original input $Z_l$, can be expressed as follows:

$$Z_l^{\left(1:c\right)}=\left(Z_{l+1}^{\left(1:c\right)}-\eta \left(Z_{l+1}^{\left(c+1:C\right)}\right)\right)\odot \exp \left(-\rho (Z_{l+1}^{\left(c+1:C\right)})\right)$$

(4)

$$Z_l^{\left(c+1:C\right)}=Z_{l+1}^{\left(c+1:C\right)}-\varphi \left(Z_l^{\left(1:c\right)}\right)$$

(5)

The PSDE architecture is expected to preserve information from fine images as much as possible. The input and output of invertible layers can be mutually generated, preventing information loss. Therefore, invertible layers have been introduced into PSDE.

The PSDE, illustrated in Figure 3, adopts a pyramid architecture similar to the PTCE. To preserve the extracted fine-grained information, a Detail Preservation Module composed of N affine coupling layers is incorporated. Considering the balance between performance and computational cost, the Bottleneck Residual Block (BRB) from MobileNetV2 [45] is employed to implement the mapping functions $\rho \left(·\right)$, $\eta \left(·\right)$, $\varphi \left(·\right)$. The depthwise separable convolutions in BRB significantly reduce computational load and the number of parameters. The overall process of the PSDE is as follows: shallow features are first extracted using a convolutional block, increasing the channel dimension of the input image to $C$. These shallow features then enter four Detail Preservation Modules. To construct the pyramid structure and progressively reduce spatial resolution, a Patch Merging (PM) operation adapted from SwinTransformer [36] is added before the Detail Preservation Module, halving the width and height of the feature map.

3.1.3. Hierarchical Long-Range Feature Aggregation

The design of the HLRFA module is motivated by the critical need for efficient long-range dependency modeling and multiscale feature integration in high-resolution STF. Recovering complex spatial structures requires a vast receptive field. However, global attention mechanisms, while effective at capturing long-range context, incur quadratic computational complexity ($O(N^2)$), making them computationally prohibitive for high-resolution input images.

To address this efficiency bottleneck, we design HLRFA, composed of three sequential components: Feature Upsampling Unit (FUU), Large-Kernel Fusion Block (LKFB), and Cross-Scale Fusion Unit (CSFU), as shown in Figure 4. These components collaboratively enable the aggregation of features across multiple hierarchical levels, capture long-range temporal and spatial dependencies, and maintain a scalable, linear computational complexity ($O\left(N\right)$).

The previous-stage output $F^{\left(i-1\right)}$ is first upsampled to match the spatial resolution of the current stage, ensuring that multiscale features are properly aligned for subsequent fusion. This operation, referred to as the FUU, employs pixel shuffle followed by convolution to expand both spatial and channel dimensions, which can be formally expressed as follows:

$$F_{\mathrm{up}}^{\left(i-1\right)}=\mathrm{PixShuffle}\left(\mathrm{Conv}\left(F^{\left(i-1\right)}\right)\right)$$

(6)

where $\mathrm{Conv}\left(\cdot \right)$ denotes a convolution operation, and $\mathrm{PixShuffle}\left(\cdot \right)$ represents the pixel shuffle function used to rearrange and expand spatial dimensions.

To capture long-range dependencies within the current stage, $D^{\left(i\right)}$, $C_0^{\left(i\right)}$ and $C_1^{\left(i\right)}$ are first concatenated along the feature dimension and reduced via 1 × 1 convolution. In the LKFB, large-kernel convolutions are employed to aggregate contextual information across an extended receptive field, inspired by the self-attention mechanism in Transformer architectures, which enables direct modeling of long-range spatial and temporal dependencies. To further enhance fine-grained information, the output of the large-kernel convolutions is combined with the original detail feature $D^{\left(i\right)}$, providing residual guidance that preserves high-frequency structures while integrating long-range context. Depthwise and pointwise convolutions are then applied to maintain computational efficiency, and a 1 × 1 convolution-based Feed-Forward Network (ConvFFN) further refines the fused representation, yielding features that simultaneously capture global context and local details.

$$F_{\mathrm{same}}^{\left(i\right)}=D^{\left(i\right)}+\mathrm{LKCNN}\left(\mathrm{cat}(D^{\left(i\right)},C_1^{\left(i\right)}-C_0^{\left(i\right)})\right)$$

(7)

The upsampled features from the previous stage, $F_{\mathrm{up}}^{\left(i-1\right)}$ are integrated with the refined current-stage features $F_{\mathrm{same}}^{\left(i\right)}$ and the latest temporal features $C_0^{\left(i\right)}$ within the CSFU. This module not only aggregates multiscale features from consecutive stages but also explicitly accounts for substantial land-cover changes between temporal observations. In regions where significant changes occur, the newly emerged information is primarily captured by $C_1^{\left(i\right)}$, ensuring that the fusion process incorporates these novel details. The combined features are then processed through two successive convolutional layers to produce the output of the current stage:

$$F^{\left(i\right)}=\mathrm{Conv}\left(\mathrm{Conv}\left(\mathrm{cat}\left(F_{\mathrm{up}}^{\left(i-1\right)},F_{\mathrm{same}}^{\left(i\right)},C_1^{\left(i\right)}\right)\right)\right)$$

(8)

This hierarchical design enables progressive aggregation of multiscale features, preserves fine temporal and spatial details, and ensures effective communication between consecutive fusion stages, which is crucial for high-fidelity STF in heterogeneous remote sensing scenes.

3.2. PatchGAN-Based Discriminator

The discriminator architecture adopts PatchGAN [46,47]. Unlike a traditional discriminator that outputs one pixel, signifying whether the generated image is real or fake, PatchGAN produces a patch map of size $N\times N$ ($N$ is much smaller than the input image size), indicating whether the $N\times N$ overlapping image patches from the image are real or fake. This patch-level discriminator design requires fewer parameters than a full-image discriminator and enables handling images of arbitrary size in a fully convolutional manner.

As illustrated in Figure 5, the discriminative network is a straightforward CNN classification network comprising convolutional layers, Batch Normalization (BN), and a LeakyReLU activation function. To ensure the stability of model training, $C_1$ is introduced into the discriminator as a conditional label along with either the generated image or the real image. Specifically, during the training phase of the discriminator, when presented with inputs of real images $F_1$ and $C_1$, it is expected to produce a matrix with values of 1. Conversely, when the inputs are generated images ${\widehat{F}}_1$ and $C_1$, the discriminator is expected to yield a matrix with values of 0.

3.3. Compound Loss Function

The compound loss function consists of four components: the $l_1$ loss, the spectral loss $l_{\mathrm{spect}}$ and the structural loss $l_{\mathrm{struct}}$ and the GAN loss $l_{\mathrm{GAN}}\left(G\right)$. The total loss function of the generated network is as follows:

$$l\left(G\right)=\alpha l_{\mathrm{GAN}}\left(G\right)+l_1+l_{\mathrm{spect}}+l_{\mathrm{struct}}$$

(9)

where $\alpha$ = 0.01. The $l_1$ loss is used to calculate the difference between the values of the fused image and the true image at the same pixel position. To ensure the spectral and structural fidelity of the generated remote sensing images, the cosine similarity and multi-scale structural similarity (MS-SSIM) between the true image and the fused image are computed, respectively:

$$l_1=\frac{1}{K}\sum _{k=1}^K{\parallel T-P\parallel }_1$$

(10)

$$l_{\mathrm{spect}}=I-{\displaystyle \frac{T\cdot P}{{\parallel T\parallel }_2{\parallel P\parallel }_2}}$$

(11)

$$l_{struct}=\mathbf{I}-[l_M(\mathbf{T},\mathbf{P}){]}^{{\upsilon }_M}·\prod _{j=1}^M [c_j(\mathbf{T},\mathbf{P}){]}^{{\rho }_j}[s_j(\mathbf{T},\mathbf{P}){]}^{{\mu }_j}$$

(12)

where $T$ and $P$ denote the ground truth and fused images, respectively. $\Vert ·{\Vert }_p$ represents the $p$-norm, K is the number of pixels in the image, $I$ is the tensor of 1, $M$ is the number of scales, and $l_j$, $c_j$, $s_j$ are the luminance, contrast, and structural comparisons at the $j$-th scale, respectively. ${\upsilon }_M$, ${\rho }_j$, ${\mu }_j$ are the corresponding weighting coefficients.

The GAN loss is the Least Squares GAN loss, specifically:

$$l_{\mathrm{GAN}}\left(G\right)={\displaystyle \frac{1}{2}}{\mathbb{E}}_{F_0,C_0,C_1}[{(D\left(G\left(F_0,C_0,C_1\right),C_1\right)-1)}^2]$$

(13)

$$l_{\mathrm{GAN}}\left(D\right)={\displaystyle \frac{1}{2}}{\mathbb{E}}_{F_1,C_1}[{(D\left(F_1,C_1\right)-1)}^2]+{\displaystyle \frac{1}{2}}{\mathbb{E}}_{F_0,C_0,C_1}[{(D\left(G\left(F_0,C_0,C_1\right),C_1\right))}^2]$$

(14)

4. Experiments and Results

4.1. Study Areas and Datasets

In the experiments, three public datasets from different locations are used in this study: the Coleambally Irrigation Area (CIA) [2], the Lower Gwydir Catchment (LGC) [2], and the Tianjin [48]. The CIA study site is located in southern New South Wales, Australia. The dominant vegetation in the area is rice with modern irrigation systems. The CIA dataset contains 17 cloud-free MODIS-Landsat data pairs from 2001 to 2002. Each image has a spatial size of 1720 × 2040 with six bands. Land cover was essentially unchanged during the period of the CIA dataset collection, but phenology varied considerably and can thus be used to verify the performance of DSEPGAN in predicting phenological change. The LGC study site is located in northern New South Wales and contains 14 pairs of 6 × 2720 × 3200 cloud-free data pairs from 2004 to 2005. A major flood occurred in mid-December 2004, causing inundation of about 44% of the area. The LGC can therefore be recognized as a site where abrupt change scenarios exist and is well-suited to test the performance of DSEPGAN in land cover change prediction. The Tianjin study site is located in the northern part of China and encompasses 27 pairs of MODIS-Landsat data from 2013 to 2019, with a data size of 6 × 2100 × 1970. This dataset can be utilized to evaluate the accuracy of the STF methods in predicting phenological changes in urban areas.

ALL datasets are divided into training and testing parts. For the CIA dataset, images other than the MODIS-Landsat image pairs on 25 November 2001, 12 January 2002, and 22 February 2002 are used as training data. In the test phase, the image pair from 25 November 2001 and the coarse image from 12 January 2002 are used to predict the fine image from 12 January 2002. For the LGC dataset, the image pairs of other dates, excluding 26 November 2004, 12 December 2004, and 28 December 2004, are used as the training data, and in the testing phase, the image pair on 26 November 2004 and the coarse image on 12 December 2004 are used to predict the fine image on 12 December 2004. For the Tianjin dataset, excluding the images from 16 April 2015, 18 May 2015, and 4 May 2016, all other images are designated as training data. During the testing phase, images from 16 April 2015 are used, along with the coarse image from 18 May 2015, to predict the fine image from 18 May 2015. It is worth noting that due to striping noise present in the last two bands of some Tianjin data, only the first four bands of the Tianjin dataset are used. Figure 6 presents three sets of test data, showcasing noticeable differences among them. Specifically, the CIA data exhibit a relatively similar land cover type at two different times, yet significant phenological disparities are evident with noticeable color changes. The LGC data display considerable differences in imagery due to sudden floods. As for the Tianjin dataset, it not only captures phenological changes in urban areas but also reflects color variations stemming from data acquired by different sensors.

4.2. Experiment Design and Evaluation

Three traditional STF methods (STARFM [9], FSDAF [8], and Fit-FC [11]) and five DL-based methods (EDCSTFN [20], GAN-STFM [25], MLFF-GAN [23], STF-Trans [28], and CTSTFM [37]) are included for comparison. These methods cover representative architectures such as GAN-based models, pyramid structures, and CNN-Transformer hybrids. To clearly highlight what distinguishes DSEPGAN from existing approaches,

Table 1. DL-based methods comparison table.

Model	Network Architecture	Multi-Scale Mechanism	Long-Range Modeling	Loss Functions
EDCSTFN	CNN	None	None	MSE loss, feature loss, and structure loss
GAN-STFM	CNN + GAN	None	None	GAN loss, feature loss, spectrum loss, and structure loss
MLFF-GAN	CNN + GAN with a pyramid encoder	Pyramid downsampling	None	GAN loss, L1 loss, spectrum loss, and structure loss
STF-Trans	CNN + Transformer	None	Transformer attention	L1 loss, high-frequency loss, and Total Variation
CTSTFM	CNN + Transformer	multikernel CNN	Transformer attention	L1 loss
DSEPGAN	CNN + INN + GAN with a Dual-Stream pyramid encoder	Pyramid downsampling + detail-preserving mechanism	Large-kernel convolution	GAN loss, L1 loss, spectrum loss, and structure loss

provides a structured comparison of DL-based methods in terms of network architecture, multiscale feature modeling, long-range dependency modeling, and loss function design.

For a fair comparison, similar experimental settings are used for the traditional and DL-based methods. For the three traditional algorithms, the size of the moving window for searching similar pixels is uniformly set to 41 × 41, and the number of similar pixels is 20. Meanwhile, for the five DL-based methods, the same training dataset generation method and data enhancement method are used. When generating the training set, for each prediction date, a pair of images from another date is randomly selected as reference images. The input images are 256 × 256 in size and are cropped from the original images with a step of 200. A total of 1260 sets of samples are obtained from the CIA, 2464 sets of samples are obtained from the LGC, and 2530 sets of samples are obtained from Tianjin. The parameter settings of EDCSTFN, GAN-STFM, and MLFF-GAN follow the original design. Due to the significant difference in size between STF-Trans and other DL-based models (even the smallest model, STF-Trans-Small, contains approximately 20 million parameters), STF-Trans-Small is selected as the comparison model, with the dimension set to 128. CTSTFM uses an embedding dimension of 64. For DSEPGAN, the initial learning rate is 2 × 10⁻⁴. All DL-based methods are trained on 1 NVIDIA RTX 3090 GPU using the data enhancement strategy of image random flip or rotation, and the training strategy of starting from scratch; no other tricks are used. For reproducibility purposes, the implementation of the proposed DSEPGAN is publicly available at https://github.com/ZhouDDCUG/DSEPGAN (accessed on 14 December 2025).

Six different model evaluation metrics are used: Root Mean Square Error (RMSE), Structure Similarity (SSIM), Universal Image Quality Index (UIQI), Correlation Coefficient (CC), Spectral Angle Mapper (SAM), and Erreur Relative Global Adimensionnelle de Synthène (ERGAS). The ideal values of RMSE, SSIM, UIQI, and CC are 0, 1, 1, and 1. Smaller values of RMSE, SAM, and ERGAS and larger values of SSIM, UIQI, and CC indicate the lower uncertainty of the fusion result. Additionally, for image visualization, due to the significant presence of vegetation in the image scenes, the NIR-Red-Green channels are selected as the RGB channels. Scatterplots are also utilized to compare the performance of various methods, depicting the correspondence between pixel values of predicted and actual images across different ranges. Data for generating scatterplots are derived from image band averages with scatter points rendered in different colors based on scatter density. The average absolute difference (AAD) maps are used to observe the error distribution across the entire image.

4.3. Experimental Results for CIA

The ground truth for the CIA data, along with the fusion results obtained by the nine methods and their corresponding AAD maps, is shown in Figure 7. It is worth pointing out that the lower-right corner of the CIA dataset is partially missing, so corresponding deletions have been made in the results for all methods. The image results obtained by all eight methods are greenish, illustrating that all methods correctly predicted the phenological changes to some extent, but there are obvious differences in the details.

Specifically, the results of STARFM, FSDAF, and Fit-FC all suffer from unpleasant coloring-mixing problems. Where the ground truth is white patches, the results obtained by STARFM and FSDAF show varying degrees of yellow, and the results obtained by Fit-FC are light red; where the ground truth is red patches, the results obtained by STARFM and FSDAF appear black, and the boundaries between patches are blurred, which is particularly serious in the STARFM method. The fusion results obtained by EDCSTFN do not have the above serious spectral distortion problem, but they are affected by the spatial blurring checkerboard effect, which makes the feature boundaries seriously distorted. In terms of appearance and texture, GAN-STFM, MLFF-GAN, STF-Trans, CTSTFM, and DSEPGAN outperform the other methods. From the AAD maps, it can be observed that the errors in all results are mostly concentrated in the detail-rich farmland areas, with the latter five methods outperforming the former four methods. The zoomed-in subregion in the black-boxed portion in Figure 7a is shown in Figure 8. Firstly, it is obvious that DSEPGAN predicts the farmland boundary accurately and clearly, which is significantly better than GAN-STFM, MLFF-GAN, STF-Trans, and CTSTFM. Secondly, the color of the circular object in the yellow box is also more similar to the ground truth. It can be concluded that the visual evaluation has a more accurate boundary prediction and detail preservation ability than other DL-based methods by considering multi-scale features, detail preservation, and long-distance feature aggregation.

Table 2. Quantitative assessment results for the CIA dataset with different methods.

	Band	STARFM	FSDAF	Fit-FC	EDCSTFN	GAN-STFM	MLFF-GAN	STF-Trans	CTSTFM	DSEPGAN
RMSE	1	0.0166	0.0161	0.0157	0.0162	0.0126	0.0122	0.0117	0.0133	0.0115
	2	0.0250	0.0234	0.0235	0.0255	0.0180	0.0180	0.0172	0.0173	0.0176
	3	0.0399	0.0364	0.0376	0.0436	0.0293	0.0286	0.0275	0.0273	0.0291
	4	0.0496	0.0509	0.0472	0.0445	0.0401	0.0395	0.0374	0.0371	0.0368
	5	0.0460	0.0459	0.0468	0.0477	0.0396	0.0374	0.0378	0.0375	0.0353
	6	0.0379	0.0384	0.0385	0.0398	0.0344	0.0334	0.0326	0.0316	0.0311
	Avg	0.0358	0.0352	0.0349	0.0362	0.0290	0.0282	0.0274	0.0273	0.0269
SSIM	1	0.8928	0.9007	0.8939	0.8850	0.9250	0.9286	0.9299	0.9271	0.9343
	2	0.8464	0.8594	0.8536	0.8259	0.8966	0.8956	0.8988	0.9026	0.9067
	3	0.7739	0.7910	0.7873	0.7165	0.8413	0.8463	0.8462	0.8543	0.8559
	4	0.6811	0.6740	0.6785	0.7147	0.7608	0.7687	0.7674	0.7835	0.7901
	5	0.7429	0.7498	0.7462	0.7377	0.8006	0.8056	0.804	0.8080	0.8175
	6	0.7748	0.7800	0.7723	0.7749	0.8178	0.8193	0.8231	0.8265	0.8300
	Avg	0.7853	0.7925	0.7886	0.7758	0.8404	0.8440	0.8449	0.8503	0.8557
UIQI	1	0.8140	0.8293	0.8190	0.8152	0.8979	0.9113	0.9131	0.9107	0.9175
	2	0.8152	0.8387	0.8275	0.8023	0.9107	0.9145	0.9209	0.9225	0.9200
	3	0.8165	0.8498	0.8400	0.7815	0.9132	0.9178	0.9254	0.9264	0.9177
	4	0.8275	0.8262	0.8370	0.8806	0.8976	0.9043	0.9124	0.9178	0.9173
	5	0.9222	0.9247	0.9224	0.9174	0.9444	0.9491	0.9534	0.9528	0.9544
	6	0.9206	0.9215	0.9193	0.9174	0.9368	0.9408	0.9465	0.9475	0.9468
	Avg	0.8527	0.8650	0.8609	0.8524	0.9168	0.9230	0.9286	0.9296	0.9290
CC	1	0.8320	0.8370	0.8401	0.8331	0.9022	0.9116	0.9167	0.9148	0.9195
	2	0.8369	0.8501	0.8473	0.8234	0.9153	0.9160	0.9232	0.9247	0.9278
	3	0.8454	0.8654	0.8551	0.8013	0.9165	0.9198	0.9266	0.9275	0.9284
	4	0.8344	0.8281	0.8459	0.8813	0.8994	0.9049	0.9137	0.9187	0.9185
	5	0.9222	0.9249	0.9238	0.9180	0.9451	0.9497	0.954	0.9532	0.9553
	6	0.9210	0.9217	0.9198	0.9177	0.9376	0.9412	0.9472	0.9478	0.9477
	Avg	0.8653	0.8712	0.8720	0.8625	0.9194	0.9239	0.9302	0.9311	0.9328
ERGAS	ALL	1.3146	1.2666	1.2612	1.3488	1.0298	1.0047	0.9697	0.9931	0.9675
SAM	ALL	11.1256	10.9104	10.8326	11.4756	8.8898	8.6679	8.2846	8.2494	8.1507

Bold text highlights the best-performing metrics.

shows the quantitative assessment metrics for the nine methods on CIA data, with the optimal values shown in bold. Compared with all eight comparison methods, DSEPGAN has excellent performance on the CIA data, achieving the best results in five of the six overall metrics (RMSE, SSIM, CC, ERGAS, and SAM) and obtaining the best experimental results. CTSTFM proves to be the strongest competitor, achieving the highest average UIQI value and securing the second-best performance in RMSE and CC. However, DSEPGAN still significantly surpasses CTSTFM in crucial structural (SSIM: 0.8557 vs. 0.8503) and spectral accuracy metrics (e.g., SAM: 8.1507 vs. 8.2494), solidifying its position as the optimal performer. From the table, it can also be found that the experimental results of the DL-based method EDCSTFN are poorer compared with the traditional methods (STARFM, FSDAF, Fit-FC), which may be because EDCSTFN, with a small number of parameters, cannot satisfy the demand of spatiotemporal fusion when the training data are more. The other five DL-based methods outperform traditional methods. GAN-STFM, MLFF-GAN, and DSEPGAN all use the GAN-based STF framework, indicating that the generative adversarial approach is well-suited for STF tasks. Meanwhile, the strong results achieved by CTSTFM and STF-Trans highlight the critical importance of effective long-range dependency modeling in current SOTA fusion architectures. Figure 9 shows the scatter plot of predicted image values and ground truth values. The narrower the scatter region, the higher the pixel similarity. Intuitively, it can be seen that the images generated by DSEPGAN are more similar to the ground truth and have the highest coefficient of determination. Overall, the qualitative and quantitative results show that the proposed DSEPGAN is advantageous in capturing the characteristics of phenological changes.

4.4. Experimental Results for LGC

In the STF task, abrupt land cover changes are more challenging to capture than phenological changes. The ground truth of the LGC, along with the fusion results and AAD maps, is presented in Figure 10. As can be seen, while the eight comparison methods successfully predict the approximate coverage area of the sudden flooding portion, the hybrid model, CTSTFM, struggles with accurately reconstructing the flood’s contour. Figure 11 presents a zoomed-in subregion of the black-boxed portion in Figure 10a. Firstly, the prediction results from the farmland in the upper right corner indicate that DSEPGAN produces clearer boundaries and more uniform colors compared to other methods. Secondly, upon further zooming into the waters within the yellow box, it is evident that results from STARFM, FSDAF, Fit-FC, and EDCSTFN exhibit fuzzy edges, with the waters blending with surrounding features. GAN-STFM and MLFF-GAN fail to predict the shape of the waters accurately, with incorrect or fuzzy boundaries in the lower part. The result of STF-Trans is overall relatively blurry. CTSTFM’s predicted water body exhibits a notably dark tone, and its edges are significantly blurred. This suggests that DSEPGAN outperforms other methods in recovering the target image in scenes with complex changes.

A quantitative comparison of the fusion results for the LGC data is listed in

Table 3. Quantitative assessment results for the LGC dataset with different methods.

	Band	STARFM	FSDAF	Fit-FC	EDCSTFN	GAN-STFM	MLFF-GAN	STF-Trans	CTSTFM	DSEPGAN
RMSE	1	0.0143	0.0149	0.0140	0.0151	0.0146	0.0161	0.0147	0.0167	0.0148
	2	0.0200	0.0207	0.0201	0.0200	0.0207	0.0223	0.0197	0.0234	0.0212
	3	0.0251	0.0258	0.0251	0.0257	0.0264	0.0269	0.0249	0.0320	0.0258
	4	0.0376	0.0397	0.0385	0.0394	0.041	0.0400	0.0357	0.0532	0.0366
	5	0.0568	0.0621	0.0565	0.0590	0.054	0.0533	0.0583	0.0660	0.0514
	6	0.0455	0.0515	0.0446	0.0407	0.0399	0.0404	0.0432	0.0476	0.0374
	Avg	0.0332	0.0358	0.0331	0.0333	0.0328	0.0332	0.0328	0.0398	0.0312
SSIM	1	0.9132	0.9125	0.9233	0.9228	0.9185	0.9059	0.9167	0.9166	0.9171
	2	0.8730	0.8709	0.8800	0.8897	0.8801	0.8709	0.8857	0.8843	0.8832
	3	0.8350	0.8331	0.8438	0.8455	0.8413	0.8356	0.8480	0.8391	0.8492
	4	0.7292	0.7294	0.7405	0.7083	0.7037	0.7239	0.7291	0.6910	0.7481
	5	0.5697	0.5220	0.5532	0.5513	0.5696	0.5948	0.5663	0.5766	0.6168
	6	0.6408	0.5754	0.6274	0.6267	0.6438	0.6533	0.6237	0.6473	0.6739
	Avg	0.7601	0.7405	0.7614	0.7574	0.7595	0.7641	0.7616	0.7591	0.7814
UIQI	1	0.7152	0.7062	0.7124	0.6132	0.6844	0.6839	0.7086	0.5813	0.7361
	2	0.7019	0.6890	0.6943	0.6560	0.6807	0.7125	0.7207	0.5637	0.7305
	3	0.7072	0.6965	0.7007	0.6573	0.696	0.7179	0.7327	0.5366	0.7349
	4	0.7857	0.7794	0.7827	0.7431	0.766	0.7826	0.8112	0.6991	0.8191
	5	0.7531	0.7198	0.7313	0.6645	0.7659	0.7940	0.7571	0.7436	0.8171
	6	0.7205	0.6531	0.7021	0.7316	0.7798	0.7918	0.7670	0.7713	0.8183
	Avg	0.7306	0.7073	0.7206	0.6776	0.7288	0.7471	0.7495	0.6493	0.7760
CC	1	0.7158	0.7076	0.7160	0.6582	0.6878	0.6870	0.7151	0.6073	0.7396
	2	0.7071	0.6916	0.7007	0.6958	0.6888	0.7169	0.7250	0.5902	0.7335
	3	0.7130	0.7004	0.7086	0.6834	0.6994	0.7192	0.7352	0.5469	0.7353
	4	0.8075	0.8011	0.8098	0.7832	0.769	0.7871	0.8265	0.7243	0.8245
	5	0.7909	0.7666	0.7832	0.7695	0.7864	0.7993	0.8111	0.7750	0.8228
	6	0.7873	0.7473	0.7786	0.7871	0.7883	0.7977	0.8070	0.7949	0.8208
	Avg	0.7536	0.7358	0.7495	0.7295	0.7366	0.7512	0.7700	0.6731	0.7794
ERGAS	ALL	2.0655	2.2230	2.0322	2.0245	1.9837	2.0300	2.0232	2.3816	1.9096
SAM	ALL	16.2826	17.0293	16.2303	16.8170	16.7738	16.5577	15.9806	18.0406	15.7259

Bold text highlights the best-performing metrics.

. It is inferred that FSDAF is not necessarily better than STARFM with the same parameter settings. Meanwhile, unlike the CIA data, the gap between the DL-based methods and the traditional methods narrows in the first four bands of LGC data but achieves excellent performance in the last two bands. CTSTFM shows lower robustness in LGC. DSEPGAN demonstrates superior performance, achieving the best results in all six overall assessment metrics. Additionally, DSEPGAN’s SSIM (0.7814) metric is significantly better than that of all other methods (e.g., 0.0173 higher than the next best, MLFF-GAN’s 0.7641), indicating its superior ability to restore image structure in complex, changing scenes. The scatterplot of the averaged bands presented in Figure 12 also indicates that the predicted values of DSEPGAN are closer to the actual values than the other methods, with the highest coefficient of determination. Overall, the proposed method achieves the best fusion results closest to the ground truth.

4.5. Experimental Results for Tianjin

The Tianjin dataset, originating from urban areas, exhibits richer detail features compared to the CIA and LGC. Figure 13 illustrates the ground truth of the Tianjin data, along with the fused images obtained by nine different methods and their corresponding AAD maps. Overall, all results show minimal influence from sensor disparities, essentially predicting urban phenological changes. However, there are some differences in detail among them. The image generated by STARFM displays erroneous yellow and green patches in the upper portion, while Fit-FC shows a yellow area resembling cloud or haze pollution on the left side. Due to pixel-level fusion, the images produced by STARFM, FSDAF, and Fit-FC demonstrate satisfactory detail effects. Color deviations in EDCSTFN are mainly observed in the water regions. The results of STF-Trans and CTSTFM exhibit significant issues with detail loss and blurriness, failing to reconstruct the complex urban textures accurately. From the AAD maps, it can be observed that EDCSTFN exhibits significant errors in the upper part, while DSEPGAN shows relatively lower overall errors.

Figure 14 depicts an enlarged sub-area within the black box of Figure 13a. It can be observed that the upper portion of the results from STARFM, FSDAF, and Fit-FC appear pinkish compared to the ground truth, while EDCSTFN displays a more pronounced color deviation. The MLFF-GAN fused image has a lighter shade of red in the lower portion. The results of STF-Trans exhibit issues with detail loss. Crucially, the result generated by CTSTFM in this magnified sub-area is noticeably blurry and exhibits severe green color distortion compared to the ground truth. Zooming into the magnified section within the yellow box, DSEPGAN appears more similar to the ground truth.

Table 4. Quantitative assessment results for the Tianjin dataset with different methods.

	Band	STARFM	FSDAF	Fit-FC	EDCSTFN	GAN-STFM	MLFF-GAN	STF-Trans	CTSTFM	DSEPGAN
RMSE	1	0.0359	0.0071	0.0083	0.0119	0.0064	0.0060	0.0073	0.0080	0.0055
	2	0.0175	0.0139	0.0082	0.0161	0.0079	0.0076	0.0093	0.0090	0.0068
	3	0.0314	0.0074	0.0081	0.0148	0.0071	0.0080	0.0086	0.0093	0.0064
	4	0.0215	0.0145	0.0155	0.0163	0.0159	0.0160	0.0175	0.0203	0.0149
	Avg	0.0266	0.0107	0.0100	0.0148	0.0093	0.0094	0.0107	0.0117	0.0084
SSIM	1	0.7382	0.9422	0.9319	0.8439	0.9605	0.9592	0.9469	0.9294	0.9631
	2	0.8550	0.8850	0.9465	0.8424	0.9530	0.9487	0.9365	0.9381	0.9608
	3	0.7943	0.9485	0.9436	0.8435	0.9531	0.9454	0.9332	0.9198	0.9625
	4	0.8437	0.8927	0.8820	0.8708	0.8807	0.8853	0.8427	0.8188	0.8922
	Avg	0.8078	0.9171	0.9260	0.8502	0.9368	0.9347	0.9148	0.9015	0.9447
UIQI	1	0.0870	0.8394	0.6238	0.7275	0.7756	0.8418	0.6868	0.6393	0.8706
	2	0.5866	0.7412	0.7293	0.6288	0.6974	0.7838	0.6234	0.6035	0.8286
	3	0.1253	0.8300	0.7397	0.6529	0.7791	0.8037	0.6806	0.6136	0.8600
	4	0.6177	0.8038	0.7628	0.6778	0.7563	0.7673	0.6588	0.5704	0.7986
	Avg	0.3541	0.8036	0.7139	0.6718	0.7521	0.7992	0.6624	0.6067	0.8395
CC	1	0.1660	0.8581	0.7842	0.8467	0.8510	0.8466	0.7513	0.7245	0.8775
	2	0.6835	0.8131	0.7542	0.7782	0.8205	0.7893	0.7313	0.6960	0.8378
	3	0.1841	0.8331	0.7889	0.7786	0.8373	0.8114	0.7308	0.6747	0.8626
	4	0.6210	0.8054	0.7654	0.7129	0.7613	0.7681	0.675	0.5975	0.7989
	Avg	0.4136	0.8274	0.7732	0.7791	0.8175	0.8038	0.7221	0.6732	0.8442
ERGAS	ALL	7.9286	2.4543	2.1782	3.5716	1.8916	1.8925	2.1988	2.3437	1.3594
SAM	ALL	34.8451	14.6957	16.5414	18.0913	15.0764	15.2694	17.8568	19.2186	13.8698

Bold text highlights the best-performing metrics.

presents a quantitative comparison of the fusion results for the Tianjin dataset. Figure 15 illustrates the average band scatter plots for the fusion results of all methods. It can be observed that significant deviations occur in the first and third bands of the results obtained from STARFM, as depicted in Figure 15a, where the scatter points exhibit a noticeable dispersion trend, indicating an uneven distribution with a relatively low coefficient of determination. Interestingly, the traditional methods FSDAF and Fit-FC exhibit strong quantitative indicators in this scenario, outperforming several DL-based methods, in Avg RMSE and Avg SSIM. FSDAF (Figure 15b) shows points densely distributed along the diagonal line. FSDAF and Fit-FC, on the other hand, exhibit promising indicators, especially in the near-infrared band for FSDAF, outperforming all other methods. The scatter points in Figure 15b are densely distributed along the diagonal line. This suggests that conventional methods demonstrate comparable performance to DL-based methods in certain scenarios. GAN-STFM and MLFF-GAN show superior RMSE and SSIM metrics in the first three bands. STF-Trans performs worse on the Tianjin dataset compared to its performance on the CIA and LGC datasets. CTSTFM, similar to its performance on the LGC dataset, fails on this urban scene as well. However, DSEPGAN is the only method that delivers top-tier performance consistently across all bands and all evaluation criteria, demonstrating its adaptability in urban areas.

5. Discussion

5.1. Model Efficiency

The model size, computational complexity, training speed, and inference efficiency are tested to show the model’s efficiency. Model size is typically represented by the number of parameters in the model, which includes all learnable weights and biases. Computational complexity is denoted using floating-point operations (FLOPs) when the input size is (8, 6, 256, 256). FLOPs do not perfectly reflect the model’s training time; therefore, the average training iteration time, referred to as Batch-Time, is also included. Test-Time represents the average inference time required to generate one predicted image.

As listed in

Table 5. Comparison of model efficiency and computational cost.

MODEL		Parameters (M)	FLOPs	Batch-Time (s)	Test-Time (s)
STARFM	\	\	\	\	541.13
FSDAF	\	\	\	\	1262.19
Fit-FC	\	\	\	\	6544.50
EDCSTFN	\	0.28	1.49 × 1011	0.16	8.55
GAN-STFM	Generator	0.58	3.02 × 1011	0.77	11.24
	Discriminator	3.67	8.24 × 107
MLFF-GAN	Generator	5.93	1.09 × 1011	0.35	17.30
	Discriminator	2.78	3.02 × 1010
STF-Trans	\	6.01	4.41 × 1011	0.28	7.06
CTSTFM	\	6.30	3.06 × 1012	1.36	15.74
DSEPGAN	Generator	5.40	2.33 × 1011	0.68	15.10
	Discriminator	2.78	3.02 × 1010

, the inference time for traditional models is significantly higher than that of DL methods, as they rely on computationally expensive, pixel-wise operations like extensive neighborhood searching and iterative optimization. We provide the parameters and FLOPs for both the generators and discriminators of the GAN-based STF methods. EDCSTFN has the fewest parameters and the shortest training time. Despite GAN-STFM having a relatively smaller generator, its computational workload is higher due to the network performing feature extraction at full resolution during forward propagation. This results in the longest training time among the compared models. STF-Trans has the highest computational load, requiring more resources during inference, but its Transformer-based design allows for competitive training speeds due to easier parallelized computation. Crucially, when comparing inference speed, STF-Trans is the fastest deep learning model (7.06 s). DSEPGAN shares the same discriminator as MLFF-GAN, with over 500,000 fewer generator parameters than MLFF-GAN. However, DSEPGAN’s use of INN and large convolutional kernels increases its computational load, resulting in longer training times compared to MLFF-GAN. DSEPGAN achieves a test time of 15.10 s, demonstrating that while it is not the fastest deep learning model, its efficiency is orders of magnitude better than traditional methods, successfully balancing superior accuracy with manageable computational overhead.

5.2. Ablation Study

Different models are designed to investigate the effects of the PTCE, PSDE, and HLRFA modules in the proposed method.

(a) DSEPGA-Diff: The input to the PTCE module is replaced from two coarse images to the difference in the coarse images $\mathsf{\Delta }C=C_1-C_0$. This model is used to analyze the impact of temporal change features obtained in different ways on DSEPGAN.

(b) DSPGAN: The PTCE module is directly replaced with the PSDE module to verify the role of invertible layers in information retention.

(c) DSEPGAN-Conv: The Large Kernel CNN Fusion module in the HLRFA is replaced with a 3 × 3 convolution operator to validate the role of long-range feature modeling in DSEPGAN.

(d) DSEPGAN-Trans: The LKConv Block in the HLRFA is replaced with multi-head self-attention from the Vision Transformer [26]. The input to the self-attention module is a sequence of tokens, each generated from an 8 × 8 image patch.

(e) DSEPGAN w/o ConvFFN: The ConvFFN module in the HLRFA is directly removed to demonstrate its necessity. The ConvFFN follows the large kernel convolution to prevent information loss and enhance the feature representation capability of the model.

The results are shown in

Table 6. Quantitative results and model efficiency of different methods.

Model	CIA			LGC			Parameters	FLOPs (G)	Batch-Time (s)
	RMSE	SSIM	ERGAS	RMSE	SSIM	ERGAS
DSEPGAN-Diff	0.0302	0.8435	1.0669	0.0313	0.7775	1.9279	5,179,218	201.5	0.6403
DSPGAN	0.0275	0.8519	0.9730	0.0315	0.7754	1.9169	4,629,138	189.5	0.4657
DSEPGAN-Conv	0.0289	0.8519	1.0145	0.0328	0.7768	2.0348	4,536,786	189.7	0.4782
DSEPGAN-Trans	0.0278	0.8475	0.9992	0.0334	0.7722	2.0900	10,848,306	186.7	0.5779
DSEPGAN w/o ConvFFN	0.0281	0.8504	1.0010	0.0323	0.7806	1.9838	4,712,322	201.3	0.6146
DSEPGAN	0.0269	0.8557	0.9675	0.0312	0.7814	1.9096	5,396,946	232.8	0.6825

Bold text highlights the best-performing metrics.

, with the best results highlighted in bold. RMSE, SSIM, and ERGAS are used to measure the model’s performance. The ablation results across both the CIA and LGC datasets consistently validate the necessity and effectiveness of the proposed modules. First, the reduced performance of DSEPGAN-Diff on both CIA and LGC suggests that directly inputting the difference between the coarse images may lead to the loss of essential spectral information from the predicted date’s coarse image. Second, the results of DSPGAN (w/o INN) clearly demonstrate the positive contribution of the INN. Third, DSEPGAN-Trans, despite its complexity (10.8 M parameters), did not surpass the full DSEPGAN model. Patch-based tokenization smooths local textures, unlike the Large-Kernel Convolution, which expands the receptive field densely at the pixel level. LKConv provides a more balanced and stable solution for STF by efficiently capturing long-range dependencies while preserving detailed textures. The removal of the ConvFFN module (DSEPGAN w/o ConvFFN) confirms its necessity for stabilizing local feature representation after long-range aggregation.

We observe that introducing the INN and ConvFFN modules increases the number of parameters and computational complexity, leading to longer training times. In particular, the addition of INN has a greater impact on training time due to its inherent high computational complexity and storage requirements. However, all these increases in computational cost remain within acceptable limits.

5.3. Parameter Analysis

5.3.1. Kernel Size

The impact of varying depthwise convolution kernel sizes (i.e., K in Figure 4) on the experimental results is demonstrated in

Table 7. Impact of different kernel sizes on model performance.

Kernel Size	RMSE	SSIM	ERGAS	Parameters	FLOPs (G)	Batch-Time (s)
(3,3,3,3)	0.0272	0.8528	0.9687	5,312,466	225.5	0.6306
(7,7,7,7)	0.0279	0.8542	0.9971	5,333,586	227.3	0.6463
(9,9,9,9)	0.0270	0.8546	0.9844	5,350,482	228.8	0.6528
(13,13,13,13)	0.0269	0.8557	0.9675	5,396,946	232.8	0.6762
(3,7,9,13)	0.0271	0.8525	0.9726	5,334,738	230.7	0.6663
(3,7,15,31)	0.0272	0.8543	0.9615	5,386,578	252.4	0.8043

Bold indicates the best, and underline indicates the second-best.

. The optimal indicators are highlighted in bold, while the sub-optimal indicators are underscored. It can be seen that the model results show a trend of becoming better when the convolutional kernel size increases, indicating that long-range feature aggregation has an optimizing effect on the model. Additionally, the effect of using convolutional kernels of varying sizes at different fusion stages is examined. The experiments show that the fusion results are no better than when K = 13. Moreover, it’s observed that Parameters, FLOPs, and Batch-Time undergo minimal changes with the gradual increment of the convolution kernel size, attributed to the lightweight nature facilitated by depthwise Convolution. Considering the model accuracy and computational cost, the size of the convolution kernel for each fusion stage is set to 13.

5.3.2. Number of Invertible Basic Units

The impact of varying the number of invertible basic units (i.e., N in Figure 3) within each stage of the PSDE on model performance and computational cost is demonstrated in

Table 8. Impact of the number of invertible basic units on model performance.

N	RMSE	SSIM	ERGAS	Parameters	FLOPs (G)	Batch-Time (s)
1	0.0285	0.8491	1.0037	4,685,394	198.9	0.4361
2	0.0278	0.8508	0.9879	5,041,170	215.9	0.5220
3	0.0269	0.8557	0.9675	5,396,946	232.8	0.6762
4	0.0269	0.8557	0.9642	5,752,722	249.7	0.7773

Bold indicates the best, and underline indicates the second-best.

. It shows that increasing the number of invertible modules results in a decrease in the mean error, an increase in structural similarity, and an overall improvement in accuracy. However, this enhancement comes at the cost of increased model parameters. Additionally, it can be observed that when N is increased to 4, the improvement in model performance is not as significant as before. Therefore, in this study, all other experiments are conducted with N set to 3.

5.3.3. Number of Stages

The impact of different numbers of stages on model performance and computational costs is illustrated in

Table 9. Impact of the number of stages on model performance.

Stage	RMSE	SSIM	ERGAS	Parameters	FLOPs (G)	Batch-Time (s)
3	0.0286	0.8475	1.0370	1,329,714	160.7	0.5989
4	0.0269	0.8557	0.9675	5,396,946	232.8	0.6762
5	0.0256	0.8632	0.9268	21,512,466	304.4	0.7321

Bold text highlights the best-performing metrics.

. As the number of stages increases, the model’s ability to preserve image structural information improves, leading to continuous advancements in reducing prediction errors and enhancing image quality. However, as the model becomes more complex and deeper, it also escalates the requirements for parameters and computational resources. To match the size of other comparative models, the feature extraction and fusion in DSEPGAN are configured with four stages, indicating that the entire model operates across four distinct scales.

6. Conclusions

In this study, a Dual-Stream Enhanced Pyramid based on Generative Adversarial Network (DSEPGAN) is proposed for spatiotemporal fusion of remote sensing images. The method introduces several technically distinctive designs aimed at improving detail preservation and temporal–spatial consistency in fused products. The framework employs a dual-stream pyramid to decouple temporal change features from spatial detail features and integrates invertible layers to minimize information loss and retain high-frequency details from fine-resolution imagery. A progressive fusion mechanism based on large-kernel convolution enhances long-range context modeling while maintaining computational efficiency through depthwise and pointwise operations. Moreover, adversarial learning improves the realism and statistical fidelity of the fused images by encouraging the generator to match radiometric and structural characteristics of real observations. Extensive experiments on three benchmark datasets demonstrate that DSEPGAN outperforms state-of-the-art STF methods, achieving superior spatial detail preservation, temporal consistency, and robustness. The framework highlights the theoretical contribution of integrating dual-stream pyramids, invertible layers, and long-range context modeling for unified capture of fine-grained details and global features.

Beyond theoretical contributions, DSEPGAN’s superior fidelity and robustness are vital for complex applications requiring high-precision spatiotemporal data, particularly in heterogeneous landscapes and high-frequency urban change monitoring. This advantage stems directly from our core architectural choices: The invertible detail-preserving mechanism ensures maximal fidelity of high-frequency features (e.g., sharp boundaries and fine structures), which is critical for accurately reconstructing urban infrastructure (road networks and building footprints) and achieving high precision in land cover classification at edges. Concurrently, efficient large-kernel long-range modeling enhances global contextual consistency, helping to resolve spectral mixing over large, fragmented patches—a key requirement for applications like precision agriculture management (crop yield forecasting) and rapid disaster assessment (evaluating damage extent) in complex areas.

Despite its overall effectiveness, the DSEPGAN framework presents inherent trade-offs. Specifically, the invertible modules introduce higher computational overhead and large-kernel convolutions, while efficient, they present an inherent limitation in capturing truly global dependencies. Addressing these challenges, future work will focus on developing lighter, more streamlined invertible structures and exploring highly efficient global context modeling to further enhance fusion performance.